2024-04-25T01:18:41.726 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T01:18:41.726 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --description fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}} --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672321 --verbose -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672321/orig.config.yaml 2024-04-25T01:18:41.778 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T01:18:41.873 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672321 branch: squid description: fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672321' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: fs: - name: a - name: b session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - overall HEALTH_ - \(MON_DOWN\) - SLOW_OPS - slow request sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: cephfs_name: a client.1: cephfs_name: b install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: client.0: cephfs_name: a client.1: cephfs_name: b selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFFMkfUTOwGo5U4OtPFRy5CmKpJlXkNvIN4DuqzWX2DTanmTMsV7XjHV3GA256wj/AqKAnDaxOfo7wHizR+uR9o= smithi187.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJOxhg14KF+E3wrNmPdK1+nBia5IPCfa4VbGMyMZoqkUDLdsa4S4fxH9fcKIbP9pK6r0Ce0Oo4tcbAuPD1hN2Yk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - mon_thrash: check_mds_failover: true revive_delay: 20 thrash_delay: 10 - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T01:18:41.873 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-25T01:18:41.874 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-25T01:18:41.874 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T01:18:41.876 INFO:teuthology.task.internal:Checking packages... 2024-04-25T01:18:41.900 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-25T01:18:41.900 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T01:18:41.900 INFO:teuthology.packaging:ref: None 2024-04-25T01:18:41.900 INFO:teuthology.packaging:tag: None 2024-04-25T01:18:41.900 INFO:teuthology.packaging:branch: squid 2024-04-25T01:18:41.900 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:18:41.901 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-25T01:18:42.100 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-25T01:18:42.102 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T01:18:42.110 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T01:18:42.110 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T01:18:42.143 INFO:teuthology.task.internal:Saving configuration 2024-04-25T01:18:42.162 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T01:18:42.167 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T01:18:42.193 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672321', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 01:13:57.183580', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFFMkfUTOwGo5U4OtPFRy5CmKpJlXkNvIN4DuqzWX2DTanmTMsV7XjHV3GA256wj/AqKAnDaxOfo7wHizR+uR9o='} 2024-04-25T01:18:42.214 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672321', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 01:13:57.185009', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJOxhg14KF+E3wrNmPdK1+nBia5IPCfa4VbGMyMZoqkUDLdsa4S4fxH9fcKIbP9pK6r0Ce0Oo4tcbAuPD1hN2Yk='} 2024-04-25T01:18:42.214 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T01:18:42.220 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-04-25T01:18:42.220 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2024-04-25T01:18:42.220 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T01:18:42.316 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7943aa4040>, signals=[15]) 2024-04-25T01:18:42.317 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T01:18:42.322 INFO:teuthology.task.internal:Opening connections... 2024-04-25T01:18:42.322 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T01:18:42.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T01:18:42.403 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T01:18:42.404 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T01:18:42.480 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T01:18:42.487 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-04-25T01:18:42.504 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-04-25T01:18:42.504 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-04-25T01:18:42.562 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-04-25T01:18:42.562 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-04-25T01:18:42.562 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-04-25T01:18:42.562 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-04-25T01:18:42.562 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-04-25T01:18:42.562 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-04-25T01:18:42.562 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T01:18:42.562 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-04-25T01:18:42.562 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-04-25T01:18:42.562 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T01:18:42.562 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-04-25T01:18:42.562 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T01:18:42.563 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T01:18:42.563 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T01:18:42.563 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-04-25T01:18:42.582 DEBUG:teuthology.orchestra.run.smithi187:> uname -m 2024-04-25T01:18:42.600 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2024-04-25T01:18:42.600 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2024-04-25T01:18:42.657 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="CentOS Stream" 2024-04-25T01:18:42.657 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="9" 2024-04-25T01:18:42.657 INFO:teuthology.orchestra.run.smithi187.stdout:ID="centos" 2024-04-25T01:18:42.657 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="rhel fedora" 2024-04-25T01:18:42.657 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="9" 2024-04-25T01:18:42.657 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el9" 2024-04-25T01:18:42.657 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T01:18:42.658 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31" 2024-04-25T01:18:42.658 INFO:teuthology.orchestra.run.smithi187.stdout:LOGO="fedora-logo-icon" 2024-04-25T01:18:42.658 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T01:18:42.658 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://centos.org/" 2024-04-25T01:18:42.658 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T01:18:42.658 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T01:18:42.658 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T01:18:42.658 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2024-04-25T01:18:42.682 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T01:18:42.691 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T01:18:42.726 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T01:18:42.726 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2024-04-25T01:18:42.729 DEBUG:teuthology.orchestra.run.smithi187:> test '!' -e /home/ubuntu/cephtest 2024-04-25T01:18:42.744 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T01:18:42.750 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T01:18:42.750 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2024-04-25T01:18:42.786 DEBUG:teuthology.orchestra.run.smithi187:> test -z $(ls -A /var/lib/ceph) 2024-04-25T01:18:42.898 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T01:18:42.950 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T01:18:42.965 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-25T01:18:42.965 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-25T01:18:42.965 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T01:18:42.965 DEBUG:teuthology.orchestra.run.smithi187:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T01:18:42.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:18:42.982 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-04-25T01:18:42.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:18:42.983 DEBUG:teuthology.orchestra.run.smithi187:> uname -r 2024-04-25T01:18:43.039 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-437.el9.x86_64 2024-04-25T01:18:43.039 INFO:teuthology.task.kernel:Running kernel on smithi112: 5.14.0-437.el9.x86_64 2024-04-25T01:18:43.039 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum install -y kernel 2024-04-25T01:18:43.040 INFO:teuthology.orchestra.run.smithi187.stdout:5.14.0-437.el9.x86_64 2024-04-25T01:18:43.041 INFO:teuthology.task.kernel:Running kernel on smithi187: 5.14.0-437.el9.x86_64 2024-04-25T01:18:43.041 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum install -y kernel 2024-04-25T01:18:46.862 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - BaseOS 3.6 MB/s | 8.0 MB 00:02 2024-04-25T01:18:48.460 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 2.1 MB/s | 8.0 MB 00:03 2024-04-25T01:18:50.379 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - AppStream 7.7 MB/s | 19 MB 00:02 2024-04-25T01:18:51.535 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 9.4 MB/s | 19 MB 00:02 2024-04-25T01:18:54.979 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - CRB 6.2 MB/s | 6.2 MB 00:00 2024-04-25T01:18:55.969 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 7.6 MB/s | 6.2 MB 00:00 2024-04-25T01:18:57.597 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-04-25T01:18:57.703 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-04-25T01:18:59.381 INFO:teuthology.orchestra.run.smithi187.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-25T01:19:04.457 INFO:teuthology.orchestra.run.smithi187.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-04-25T01:19:06.304 INFO:teuthology.orchestra.run.smithi187.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T01:19:06.304 INFO:teuthology.orchestra.run.smithi187.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T01:19:06.382 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2024-04-25T01:19:06.383 INFO:teuthology.orchestra.run.smithi187.stdout:Nothing to do. 2024-04-25T01:19:06.383 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2024-04-25T01:19:06.491 DEBUG:teuthology.orchestra.run.smithi187:> echo no | sudo yum reinstall kernel || true 2024-04-25T01:19:07.036 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 01:19:04 AM UTC. 2024-04-25T01:19:07.156 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2024-04-25T01:19:07.165 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2024-04-25T01:19:07.165 INFO:teuthology.orchestra.run.smithi187.stdout: Package Architecture Version Repository Size 2024-04-25T01:19:07.166 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2024-04-25T01:19:07.166 INFO:teuthology.orchestra.run.smithi187.stdout:Reinstalling: 2024-04-25T01:19:07.166 INFO:teuthology.orchestra.run.smithi187.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T01:19:07.166 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:19:07.166 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2024-04-25T01:19:07.166 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2024-04-25T01:19:07.166 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:19:07.166 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 255 k 2024-04-25T01:19:07.166 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 0 2024-04-25T01:19:07.166 INFO:teuthology.orchestra.run.smithi187.stderr:Operation aborted. 2024-04-25T01:19:07.227 INFO:teuthology.orchestra.run.smithi187.stdout:Is this ok [y/N]: 2024-04-25T01:19:07.228 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum reinstall -y kernel || true 2024-04-25T01:19:07.720 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 01:19:04 AM UTC. 2024-04-25T01:19:07.836 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2024-04-25T01:19:07.837 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2024-04-25T01:19:07.837 INFO:teuthology.orchestra.run.smithi187.stdout: Package Architecture Version Repository Size 2024-04-25T01:19:07.837 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2024-04-25T01:19:07.837 INFO:teuthology.orchestra.run.smithi187.stdout:Reinstalling: 2024-04-25T01:19:07.837 INFO:teuthology.orchestra.run.smithi187.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T01:19:07.837 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:19:07.837 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2024-04-25T01:19:07.837 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2024-04-25T01:19:07.837 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:19:07.837 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 255 k 2024-04-25T01:19:07.837 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 0 2024-04-25T01:19:07.837 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2024-04-25T01:19:08.208 INFO:teuthology.orchestra.run.smithi187.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.2 MB/s | 255 kB 00:00 2024-04-25T01:19:08.208 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:19:08.208 INFO:teuthology.orchestra.run.smithi187.stdout:Total 687 kB/s | 255 kB 00:00 2024-04-25T01:19:08.224 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2024-04-25T01:19:08.417 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2024-04-25T01:19:08.417 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2024-04-25T01:19:08.422 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2024-04-25T01:19:08.422 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2024-04-25T01:19:08.569 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2024-04-25T01:19:08.642 INFO:teuthology.orchestra.run.smithi187.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T01:19:08.642 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T01:19:09.289 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T01:19:09.289 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T01:19:09.938 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T01:19:09.938 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:19:09.938 INFO:teuthology.orchestra.run.smithi187.stdout:Reinstalled: 2024-04-25T01:19:09.938 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T01:19:09.938 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:19:09.939 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2024-04-25T01:19:10.131 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T01:19:10.177 INFO:teuthology.orchestra.run.smithi187.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T01:19:10.178 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T01:19:10.178 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T01:19:10.178 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T01:19:10.178 DEBUG:teuthology.orchestra.run.smithi187:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T01:19:10.210 INFO:teuthology.orchestra.run.smithi187.stdout:ttyS1 2024-04-25T01:19:10.231 DEBUG:teuthology.parallel:result is None 2024-04-25T01:19:20.438 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 948 kB/s | 21 MB 00:22 2024-04-25T01:19:25.436 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-25T01:19:27.283 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T01:19:27.284 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T01:19:27.378 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-25T01:19:27.379 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2024-04-25T01:19:27.379 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-04-25T01:19:27.479 DEBUG:teuthology.orchestra.run.smithi112:> echo no | sudo yum reinstall kernel || true 2024-04-25T01:19:27.977 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 01:19:25 AM UTC. 2024-04-25T01:19:28.100 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-25T01:19:28.112 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-25T01:19:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2024-04-25T01:19:28.112 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-25T01:19:28.112 INFO:teuthology.orchestra.run.smithi112.stdout:Reinstalling: 2024-04-25T01:19:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T01:19:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:19:28.113 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-04-25T01:19:28.113 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-25T01:19:28.113 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:19:28.113 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 255 k 2024-04-25T01:19:28.113 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 0 2024-04-25T01:19:28.113 INFO:teuthology.orchestra.run.smithi112.stderr:Operation aborted. 2024-04-25T01:19:28.167 INFO:teuthology.orchestra.run.smithi112.stdout:Is this ok [y/N]: 2024-04-25T01:19:28.167 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum reinstall -y kernel || true 2024-04-25T01:19:28.660 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 01:19:25 AM UTC. 2024-04-25T01:19:28.776 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-25T01:19:28.777 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-25T01:19:28.777 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2024-04-25T01:19:28.777 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-25T01:19:28.777 INFO:teuthology.orchestra.run.smithi112.stdout:Reinstalling: 2024-04-25T01:19:28.777 INFO:teuthology.orchestra.run.smithi112.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T01:19:28.777 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:19:28.777 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-04-25T01:19:28.777 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-25T01:19:28.777 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:19:28.777 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 255 k 2024-04-25T01:19:28.778 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 0 2024-04-25T01:19:28.778 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-04-25T01:19:29.155 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.4 MB/s | 255 kB 00:00 2024-04-25T01:19:29.155 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:19:29.155 INFO:teuthology.orchestra.run.smithi112.stdout:Total 675 kB/s | 255 kB 00:00 2024-04-25T01:19:29.177 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-04-25T01:19:29.367 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-04-25T01:19:29.367 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-04-25T01:19:29.373 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-04-25T01:19:29.373 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-04-25T01:19:29.519 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-04-25T01:19:29.597 INFO:teuthology.orchestra.run.smithi112.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T01:19:29.597 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T01:19:30.150 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T01:19:30.150 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T01:19:30.846 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T01:19:30.846 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:19:30.846 INFO:teuthology.orchestra.run.smithi112.stdout:Reinstalled: 2024-04-25T01:19:30.846 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T01:19:30.846 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:19:30.846 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-04-25T01:19:31.027 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T01:19:31.074 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T01:19:31.074 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T01:19:31.074 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T01:19:31.075 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T01:19:31.075 DEBUG:teuthology.orchestra.run.smithi112:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T01:19:31.104 INFO:teuthology.orchestra.run.smithi112.stdout:ttyS1 2024-04-25T01:19:31.128 DEBUG:teuthology.parallel:result is None 2024-04-25T01:19:31.128 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T01:19:31.135 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T01:19:31.135 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T01:19:31.171 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T01:19:31.189 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T01:19:31.196 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T01:19:31.224 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T01:19:31.225 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T01:19:31.227 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T01:19:31.262 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T01:19:31.269 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T01:19:31.269 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T01:19:31.290 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T01:19:31.323 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T01:19:31.331 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T01:19:31.332 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T01:19:31.341 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T01:19:31.343 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T01:19:31.351 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T01:19:31.351 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T01:19:31.376 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T01:19:31.411 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T01:19:31.420 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T01:19:31.420 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T01:19:31.440 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T01:19:31.467 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T01:19:31.536 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T01:19:31.579 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T01:19:31.661 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T01:19:31.698 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:19:31.698 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T01:19:31.761 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T01:19:31.807 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T01:19:31.856 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T01:19:31.942 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T01:19:31.984 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:19:31.984 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T01:19:32.051 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2024-04-25T01:19:32.054 DEBUG:teuthology.orchestra.run.smithi187:> sudo service rsyslog restart 2024-04-25T01:19:32.085 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T01:19:32.134 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T01:19:32.607 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T01:19:32.615 INFO:teuthology.task.internal:Starting timer... 2024-04-25T01:19:32.615 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T01:19:32.647 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T01:19:32.679 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-25T01:19:32.680 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2024-04-25T01:19:32.727 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2024-04-25T01:19:32.729 DEBUG:teuthology.orchestra.run.smithi187:> sudo service auditd rotate 2024-04-25T01:19:32.812 INFO:teuthology.orchestra.run.smithi187.stdout:Rotating logs: 2024-04-25T01:19:32.814 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T01:19:32.814 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2024-04-25T01:19:32.837 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2024-04-25T01:19:32.837 DEBUG:teuthology.orchestra.run.smithi187:> /usr/sbin/getenforce 2024-04-25T01:19:32.868 INFO:teuthology.orchestra.run.smithi187.stdout:Permissive 2024-04-25T01:19:32.868 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi112.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi187.front.sepia.ceph.com': 'permissive'} 2024-04-25T01:19:32.868 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T01:19:32.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:19:32.902 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T01:19:32.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:19:32.934 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T01:19:32.934 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2024-04-25T01:19:32.968 DEBUG:teuthology.orchestra.run.smithi187:> sudo /usr/sbin/setenforce permissive 2024-04-25T01:19:32.998 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T01:19:33.007 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T01:19:33.013 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T01:19:33.153 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T01:19:33.169 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T01:19:33.170 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi112.front.sepia.ceph.com,smithi187.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T01:22:58.695 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi112.front.sepia.ceph.com'), Remote(name='ubuntu@smithi187.front.sepia.ceph.com')] 2024-04-25T01:22:58.696 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T01:22:58.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T01:22:58.775 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-04-25T01:22:58.861 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T01:22:58.862 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-25T01:22:58.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T01:22:58.939 DEBUG:teuthology.orchestra.run.smithi187:> true 2024-04-25T01:22:59.017 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-25T01:22:59.017 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T01:22:59.030 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T01:22:59.031 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T01:22:59.031 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:22:59.033 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T01:22:59.033 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:22:59.061 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T01:22:59.074 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T01:22:59.084 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T01:22:59.097 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2024-04-25T01:22:59.098 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T01:22:59.108 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2024-04-25T01:22:59.120 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: ntpd: command not found 2024-04-25T01:22:59.121 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T01:22:59.130 INFO:teuthology.orchestra.run.smithi187.stdout:506 Cannot talk to daemon 2024-04-25T01:22:59.133 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T01:22:59.143 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T01:22:59.156 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T01:22:59.186 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:22:59.189 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:22:59.189 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-04-25T01:22:59.189 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:22:59.190 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:22:59.190 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:22:59.190 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:22:59.216 INFO:teuthology.orchestra.run.smithi187.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:22:59.219 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:22:59.220 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2024-04-25T01:22:59.220 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:22:59.220 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:22:59.220 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:22:59.220 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:22:59.221 INFO:teuthology.run_tasks:Running task install... 2024-04-25T01:22:59.230 DEBUG:teuthology.task.install:project ceph 2024-04-25T01:22:59.230 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-25T01:22:59.230 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-25T01:22:59.230 INFO:teuthology.task.install:Using flavor: default 2024-04-25T01:22:59.237 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T01:22:59.237 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T01:22:59.237 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T01:22:59.238 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:22:59.240 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T01:22:59.240 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:22:59.390 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T01:22:59.390 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T01:22:59.519 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T01:22:59.519 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T01:22:59.619 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T01:22:59.619 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:22:59.619 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T01:22:59.644 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T01:22:59.644 DEBUG:teuthology.orchestra.run.smithi187:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T01:22:59.709 DEBUG:teuthology.orchestra.run.smithi187:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T01:22:59.782 DEBUG:teuthology.orchestra.run.smithi187:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T01:22:59.809 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T01:22:59.810 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:22:59.810 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T01:22:59.837 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T01:22:59.837 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T01:22:59.845 INFO:teuthology.orchestra.run.smithi187.stdout:check_obsoletes = 1 2024-04-25T01:22:59.847 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum clean all 2024-04-25T01:22:59.903 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T01:22:59.978 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T01:23:00.042 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2024-04-25T01:23:00.044 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2024-04-25T01:23:00.070 INFO:teuthology.orchestra.run.smithi187.stdout:48 files removed 2024-04-25T01:23:00.097 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T01:23:00.268 INFO:teuthology.orchestra.run.smithi112.stdout:48 files removed 2024-04-25T01:23:00.294 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T01:23:01.047 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 168 kB/s | 83 kB 00:00 2024-04-25T01:23:01.189 INFO:teuthology.orchestra.run.smithi187.stdout:ceph packages for x86_64 100 kB/s | 83 kB 00:00 2024-04-25T01:23:01.506 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 28 kB/s | 11 kB 00:00 2024-04-25T01:23:01.831 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-04-25T01:23:01.856 INFO:teuthology.orchestra.run.smithi187.stdout:ceph noarch packages 18 kB/s | 11 kB 00:00 2024-04-25T01:23:02.223 INFO:teuthology.orchestra.run.smithi187.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-04-25T01:23:02.415 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-25T01:23:03.840 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.0 MB 00:01 2024-04-25T01:23:07.141 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - AppStream 8.4 MB/s | 19 MB 00:02 2024-04-25T01:23:07.441 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 4.8 MB/s | 19 MB 00:03 2024-04-25T01:23:11.608 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.2 MB 00:00 2024-04-25T01:23:12.009 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.2 MB 00:01 2024-04-25T01:23:12.993 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 16 kB 00:00 2024-04-25T01:23:13.684 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-25T01:23:14.960 INFO:teuthology.orchestra.run.smithi187.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-25T01:23:16.670 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 5.9 MB/s | 21 MB 00:03 2024-04-25T01:23:20.052 INFO:teuthology.orchestra.run.smithi187.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-25T01:23:21.786 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-25T01:23:22.170 INFO:teuthology.orchestra.run.smithi187.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T01:23:22.170 INFO:teuthology.orchestra.run.smithi187.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T01:23:22.182 INFO:teuthology.orchestra.run.smithi187.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T01:23:22.182 INFO:teuthology.orchestra.run.smithi187.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T01:23:22.183 INFO:teuthology.orchestra.run.smithi187.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T01:23:22.184 INFO:teuthology.orchestra.run.smithi187.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T01:23:22.186 INFO:teuthology.orchestra.run.smithi187.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T01:23:22.188 INFO:teuthology.orchestra.run.smithi187.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T01:23:22.189 INFO:teuthology.orchestra.run.smithi187.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T01:23:22.250 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2024-04-25T01:23:22.259 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2024-04-25T01:23:22.259 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2024-04-25T01:23:22.259 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2024-04-25T01:23:22.259 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2024-04-25T01:23:22.259 INFO:teuthology.orchestra.run.smithi187.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T01:23:22.259 INFO:teuthology.orchestra.run.smithi187.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T01:23:22.260 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T01:23:22.260 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T01:23:22.260 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T01:23:22.260 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T01:23:22.260 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T01:23:22.260 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T01:23:22.260 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T01:23:22.260 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T01:23:22.260 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T01:23:22.260 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T01:23:22.260 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T01:23:22.260 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T01:23:22.260 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T01:23:22.260 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T01:23:22.260 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T01:23:22.261 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T01:23:22.261 INFO:teuthology.orchestra.run.smithi187.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T01:23:22.261 INFO:teuthology.orchestra.run.smithi187.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T01:23:22.261 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T01:23:22.261 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T01:23:22.261 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T01:23:22.261 INFO:teuthology.orchestra.run.smithi187.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T01:23:22.261 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T01:23:22.261 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T01:23:22.261 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T01:23:22.261 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T01:23:22.261 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T01:23:22.261 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T01:23:22.261 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T01:23:22.261 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T01:23:22.262 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T01:23:22.262 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T01:23:22.262 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T01:23:22.262 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T01:23:22.262 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T01:23:22.262 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrading: 2024-04-25T01:23:22.262 INFO:teuthology.orchestra.run.smithi187.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T01:23:22.262 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T01:23:22.262 INFO:teuthology.orchestra.run.smithi187.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T01:23:22.262 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T01:23:22.262 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2024-04-25T01:23:22.262 INFO:teuthology.orchestra.run.smithi187.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T01:23:22.262 INFO:teuthology.orchestra.run.smithi187.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T01:23:22.262 INFO:teuthology.orchestra.run.smithi187.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T01:23:22.263 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T01:23:22.263 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T01:23:22.263 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T01:23:22.263 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T01:23:22.263 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T01:23:22.263 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T01:23:22.263 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T01:23:22.263 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T01:23:22.263 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T01:23:22.263 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T01:23:22.263 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T01:23:22.263 INFO:teuthology.orchestra.run.smithi187.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T01:23:22.263 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T01:23:22.263 INFO:teuthology.orchestra.run.smithi187.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T01:23:22.263 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T01:23:22.264 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T01:23:22.264 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T01:23:22.264 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T01:23:22.264 INFO:teuthology.orchestra.run.smithi187.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T01:23:22.264 INFO:teuthology.orchestra.run.smithi187.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T01:23:22.264 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T01:23:22.264 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T01:23:22.264 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T01:23:22.264 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T01:23:22.264 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T01:23:22.264 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T01:23:22.264 INFO:teuthology.orchestra.run.smithi187.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T01:23:22.264 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T01:23:22.264 INFO:teuthology.orchestra.run.smithi187.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T01:23:22.265 INFO:teuthology.orchestra.run.smithi187.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T01:23:22.265 INFO:teuthology.orchestra.run.smithi187.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T01:23:22.265 INFO:teuthology.orchestra.run.smithi187.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T01:23:22.265 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T01:23:22.265 INFO:teuthology.orchestra.run.smithi187.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T01:23:22.265 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T01:23:22.265 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T01:23:22.265 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T01:23:22.265 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T01:23:22.265 INFO:teuthology.orchestra.run.smithi187.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T01:23:22.265 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T01:23:22.265 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T01:23:22.265 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T01:23:22.265 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T01:23:22.265 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T01:23:22.266 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T01:23:22.266 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T01:23:22.266 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T01:23:22.266 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T01:23:22.266 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T01:23:22.266 INFO:teuthology.orchestra.run.smithi187.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T01:23:22.266 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T01:23:22.266 INFO:teuthology.orchestra.run.smithi187.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T01:23:22.266 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T01:23:22.266 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T01:23:22.266 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T01:23:22.266 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T01:23:22.266 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T01:23:22.266 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T01:23:22.267 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T01:23:22.267 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T01:23:22.267 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T01:23:22.267 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T01:23:22.267 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T01:23:22.267 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T01:23:22.267 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T01:23:22.267 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T01:23:22.267 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T01:23:22.267 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T01:23:22.267 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T01:23:22.267 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T01:23:22.267 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T01:23:22.268 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T01:23:22.268 INFO:teuthology.orchestra.run.smithi187.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T01:23:22.268 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T01:23:22.268 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T01:23:22.268 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T01:23:22.268 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T01:23:22.268 INFO:teuthology.orchestra.run.smithi187.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T01:23:22.268 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T01:23:22.268 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T01:23:22.268 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T01:23:22.268 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T01:23:22.268 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T01:23:22.268 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T01:23:22.268 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T01:23:22.268 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T01:23:22.268 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T01:23:22.269 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T01:23:22.269 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T01:23:22.269 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T01:23:22.269 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T01:23:22.269 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T01:23:22.269 INFO:teuthology.orchestra.run.smithi187.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T01:23:22.269 INFO:teuthology.orchestra.run.smithi187.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T01:23:22.269 INFO:teuthology.orchestra.run.smithi187.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T01:23:22.269 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T01:23:22.269 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T01:23:22.269 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T01:23:22.269 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T01:23:22.269 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T01:23:22.269 INFO:teuthology.orchestra.run.smithi187.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T01:23:22.270 INFO:teuthology.orchestra.run.smithi187.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T01:23:22.270 INFO:teuthology.orchestra.run.smithi187.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T01:23:22.270 INFO:teuthology.orchestra.run.smithi187.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T01:23:22.270 INFO:teuthology.orchestra.run.smithi187.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T01:23:22.270 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T01:23:22.270 INFO:teuthology.orchestra.run.smithi187.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T01:23:22.270 INFO:teuthology.orchestra.run.smithi187.stdout:Installing weak dependencies: 2024-04-25T01:23:22.270 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T01:23:22.270 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:23:22.270 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2024-04-25T01:23:22.270 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2024-04-25T01:23:22.270 INFO:teuthology.orchestra.run.smithi187.stdout:Install 148 Packages 2024-04-25T01:23:22.270 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrade 4 Packages 2024-04-25T01:23:22.271 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:23:22.271 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 220 M 2024-04-25T01:23:22.271 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2024-04-25T01:23:23.544 INFO:teuthology.orchestra.run.smithi187.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-04-25T01:23:23.945 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T01:23:23.946 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T01:23:23.956 INFO:teuthology.orchestra.run.smithi112.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T01:23:23.957 INFO:teuthology.orchestra.run.smithi112.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T01:23:23.958 INFO:teuthology.orchestra.run.smithi112.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T01:23:23.959 INFO:teuthology.orchestra.run.smithi112.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T01:23:23.961 INFO:teuthology.orchestra.run.smithi112.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T01:23:23.963 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T01:23:23.964 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T01:23:24.025 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-25T01:23:24.034 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-25T01:23:24.034 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-04-25T01:23:24.034 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-25T01:23:24.034 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-04-25T01:23:24.034 INFO:teuthology.orchestra.run.smithi112.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T01:23:24.034 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T01:23:24.034 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T01:23:24.034 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T01:23:24.034 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T01:23:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T01:23:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T01:23:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T01:23:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T01:23:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T01:23:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T01:23:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T01:23:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T01:23:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T01:23:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T01:23:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T01:23:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T01:23:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T01:23:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T01:23:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T01:23:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T01:23:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T01:23:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T01:23:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T01:23:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T01:23:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T01:23:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T01:23:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T01:23:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T01:23:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T01:23:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T01:23:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T01:23:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T01:23:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T01:23:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T01:23:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T01:23:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T01:23:24.037 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2024-04-25T01:23:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T01:23:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T01:23:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T01:23:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T01:23:24.037 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-04-25T01:23:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T01:23:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T01:23:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T01:23:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T01:23:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T01:23:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T01:23:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T01:23:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T01:23:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T01:23:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T01:23:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T01:23:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T01:23:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T01:23:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T01:23:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T01:23:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T01:23:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T01:23:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T01:23:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T01:23:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T01:23:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T01:23:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T01:23:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T01:23:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T01:23:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T01:23:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T01:23:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T01:23:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T01:23:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T01:23:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T01:23:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T01:23:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T01:23:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T01:23:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T01:23:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T01:23:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T01:23:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T01:23:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T01:23:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T01:23:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T01:23:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T01:23:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T01:23:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T01:23:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T01:23:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T01:23:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T01:23:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T01:23:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T01:23:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T01:23:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T01:23:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T01:23:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T01:23:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T01:23:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T01:23:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T01:23:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T01:23:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T01:23:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T01:23:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T01:23:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T01:23:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T01:23:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T01:23:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T01:23:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T01:23:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T01:23:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T01:23:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T01:23:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T01:23:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T01:23:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T01:23:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T01:23:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T01:23:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T01:23:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T01:23:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T01:23:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T01:23:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T01:23:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T01:23:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T01:23:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T01:23:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T01:23:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T01:23:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T01:23:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T01:23:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T01:23:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T01:23:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T01:23:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T01:23:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T01:23:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T01:23:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T01:23:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T01:23:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T01:23:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T01:23:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T01:23:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T01:23:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T01:23:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T01:23:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T01:23:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T01:23:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T01:23:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T01:23:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T01:23:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T01:23:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T01:23:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T01:23:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T01:23:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T01:23:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T01:23:24.045 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T01:23:24.045 INFO:teuthology.orchestra.run.smithi112.stdout:Installing weak dependencies: 2024-04-25T01:23:24.045 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T01:23:24.045 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:23:24.045 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-04-25T01:23:24.045 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-25T01:23:24.045 INFO:teuthology.orchestra.run.smithi112.stdout:Install 148 Packages 2024-04-25T01:23:24.045 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 4 Packages 2024-04-25T01:23:24.045 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:23:24.045 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 220 M 2024-04-25T01:23:24.046 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-04-25T01:23:24.620 INFO:teuthology.orchestra.run.smithi187.stdout:(2/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 14 MB/s | 21 MB 00:01 2024-04-25T01:23:24.653 INFO:teuthology.orchestra.run.smithi187.stdout:(3/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 803 kB/s | 890 kB 00:01 2024-04-25T01:23:24.678 INFO:teuthology.orchestra.run.smithi187.stdout:(4/152): ceph-immutable-object-cache-19.0.0-244 2.4 MB/s | 145 kB 00:00 2024-04-25T01:23:24.787 INFO:teuthology.orchestra.run.smithi187.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 14 MB/s | 1.5 MB 00:00 2024-04-25T01:23:24.820 INFO:teuthology.orchestra.run.smithi112.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 32 kB/s | 6.6 kB 00:00 2024-04-25T01:23:25.154 INFO:teuthology.orchestra.run.smithi187.stdout:(6/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 13 MB/s | 4.7 MB 00:00 2024-04-25T01:23:25.612 INFO:teuthology.orchestra.run.smithi112.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.1 MB/s | 890 kB 00:00 2024-04-25T01:23:25.796 INFO:teuthology.orchestra.run.smithi187.stdout:(7/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-04-25T01:23:25.837 INFO:teuthology.orchestra.run.smithi112.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 644 kB/s | 145 kB 00:00 2024-04-25T01:23:25.913 INFO:teuthology.orchestra.run.smithi187.stdout:(8/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 1.9 MB/s | 2.4 MB 00:01 2024-04-25T01:23:26.030 INFO:teuthology.orchestra.run.smithi187.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 216 kB/s | 25 kB 00:00 2024-04-25T01:23:26.746 INFO:teuthology.orchestra.run.smithi112.stdout:(4/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 9.8 MB/s | 21 MB 00:02 2024-04-25T01:23:26.763 INFO:teuthology.orchestra.run.smithi187.stdout:(10/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 10 MB/s | 17 MB 00:01 2024-04-25T01:23:26.830 INFO:teuthology.orchestra.run.smithi187.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 3.3 MB/s | 222 kB 00:00 2024-04-25T01:23:26.872 INFO:teuthology.orchestra.run.smithi187.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 650 kB/s | 27 kB 00:00 2024-04-25T01:23:26.879 INFO:teuthology.orchestra.run.smithi112.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 11 MB/s | 1.5 MB 00:00 2024-04-25T01:23:26.914 INFO:teuthology.orchestra.run.smithi187.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 763 kB/s | 32 kB 00:00 2024-04-25T01:23:27.039 INFO:teuthology.orchestra.run.smithi187.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 5.8 MB/s | 745 kB 00:00 2024-04-25T01:23:27.118 INFO:teuthology.orchestra.run.smithi187.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 2.1 MB/s | 163 kB 00:00 2024-04-25T01:23:27.190 INFO:teuthology.orchestra.run.smithi187.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.7 MB/s | 127 kB 00:00 2024-04-25T01:23:27.230 INFO:teuthology.orchestra.run.smithi112.stdout:(6/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 13 MB/s | 4.7 MB 00:00 2024-04-25T01:23:27.257 INFO:teuthology.orchestra.run.smithi187.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 7.4 MB/s | 501 kB 00:00 2024-04-25T01:23:27.380 INFO:teuthology.orchestra.run.smithi112.stdout:(7/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 1.5 MB/s | 2.4 MB 00:01 2024-04-25T01:23:27.799 INFO:teuthology.orchestra.run.smithi187.stdout:(18/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 9.8 MB/s | 5.3 MB 00:00 2024-04-25T01:23:27.874 INFO:teuthology.orchestra.run.smithi187.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 603 kB/s | 45 kB 00:00 2024-04-25T01:23:27.941 INFO:teuthology.orchestra.run.smithi187.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 2.0 MB/s | 133 kB 00:00 2024-04-25T01:23:28.016 INFO:teuthology.orchestra.run.smithi187.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 2.1 MB/s | 162 kB 00:00 2024-04-25T01:23:28.100 INFO:teuthology.orchestra.run.smithi187.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 3.8 MB/s | 321 kB 00:00 2024-04-25T01:23:28.191 INFO:teuthology.orchestra.run.smithi187.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 3.2 MB/s | 299 kB 00:00 2024-04-25T01:23:28.258 INFO:teuthology.orchestra.run.smithi187.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.5 MB/s | 99 kB 00:00 2024-04-25T01:23:28.334 INFO:teuthology.orchestra.run.smithi187.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-25T01:23:28.514 INFO:teuthology.orchestra.run.smithi112.stdout:(8/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x86 13 MB/s | 17 MB 00:01 2024-04-25T01:23:28.547 INFO:teuthology.orchestra.run.smithi112.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 765 kB/s | 25 kB 00:00 2024-04-25T01:23:28.576 INFO:teuthology.orchestra.run.smithi187.stdout:(26/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 13 MB/s | 3.1 MB 00:00 2024-04-25T01:23:28.651 INFO:teuthology.orchestra.run.smithi187.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.2 MB/s | 171 kB 00:00 2024-04-25T01:23:28.726 INFO:teuthology.orchestra.run.smithi187.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 425 kB/s | 32 kB 00:00 2024-04-25T01:23:28.793 INFO:teuthology.orchestra.run.smithi187.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 2.1 MB/s | 141 kB 00:00 2024-04-25T01:23:28.952 INFO:teuthology.orchestra.run.smithi187.stdout:(30/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 3.2 MB/s | 10 MB 00:03 2024-04-25T01:23:29.211 INFO:teuthology.orchestra.run.smithi187.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 8.4 MB/s | 3.5 MB 00:00 2024-04-25T01:23:29.294 INFO:teuthology.orchestra.run.smithi187.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.9 MB/s | 249 kB 00:00 2024-04-25T01:23:29.370 INFO:teuthology.orchestra.run.smithi187.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 657 kB/s | 49 kB 00:00 2024-04-25T01:23:29.437 INFO:teuthology.orchestra.run.smithi187.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 230 kB/s | 15 kB 00:00 2024-04-25T01:23:29.520 INFO:teuthology.orchestra.run.smithi187.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 3.1 MB/s | 264 kB 00:00 2024-04-25T01:23:29.671 INFO:teuthology.orchestra.run.smithi187.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 5.0 MB/s | 762 kB 00:00 2024-04-25T01:23:29.746 INFO:teuthology.orchestra.run.smithi187.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 243 kB/s | 18 kB 00:00 2024-04-25T01:23:29.930 INFO:teuthology.orchestra.run.smithi187.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 438 kB/s | 80 kB 00:00 2024-04-25T01:23:30.146 INFO:teuthology.orchestra.run.smithi187.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-04-25T01:23:30.205 INFO:teuthology.orchestra.run.smithi187.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-04-25T01:23:30.297 INFO:teuthology.orchestra.run.smithi187.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-25T01:23:30.489 INFO:teuthology.orchestra.run.smithi187.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 6.4 MB/s | 1.2 MB 00:00 2024-04-25T01:23:30.547 INFO:teuthology.orchestra.run.smithi187.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T01:23:30.606 INFO:teuthology.orchestra.run.smithi187.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T01:23:30.707 INFO:teuthology.orchestra.run.smithi187.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-04-25T01:23:30.765 INFO:teuthology.orchestra.run.smithi187.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-25T01:23:31.091 INFO:teuthology.orchestra.run.smithi187.stdout:(47/152): ceph-mgr-diskprediction-local-19.0.0- 3.5 MB/s | 7.4 MB 00:02 2024-04-25T01:23:31.615 INFO:teuthology.orchestra.run.smithi187.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 810 kB/s | 689 kB 00:00 2024-04-25T01:23:31.658 INFO:teuthology.orchestra.run.smithi187.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 1.6 MB/s | 943 kB 00:00 2024-04-25T01:23:31.764 INFO:teuthology.orchestra.run.smithi112.stdout:(10/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 15 MB/s | 48 MB 00:03 2024-04-25T01:23:31.783 INFO:teuthology.orchestra.run.smithi187.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 215 kB/s | 21 kB 00:00 2024-04-25T01:23:31.808 INFO:teuthology.orchestra.run.smithi187.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 557 kB/s | 107 kB 00:00 2024-04-25T01:23:31.872 INFO:teuthology.orchestra.run.smithi112.stdout:(11/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 2.3 MB/s | 10 MB 00:04 2024-04-25T01:23:31.897 INFO:teuthology.orchestra.run.smithi112.stdout:(12/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 1.6 MB/s | 222 kB 00:00 2024-04-25T01:23:31.914 INFO:teuthology.orchestra.run.smithi112.stdout:(13/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 652 kB/s | 27 kB 00:00 2024-04-25T01:23:31.931 INFO:teuthology.orchestra.run.smithi112.stdout:(14/152): libcephfs-devel-19.0.0-2445.gd2c45d97 943 kB/s | 32 kB 00:00 2024-04-25T01:23:31.942 INFO:teuthology.orchestra.run.smithi187.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 302 kB/s | 33 kB 00:00 2024-04-25T01:23:31.982 INFO:teuthology.orchestra.run.smithi112.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 3.2 MB/s | 163 kB 00:00 2024-04-25T01:23:32.000 INFO:teuthology.orchestra.run.smithi187.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 1.4 MB/s | 313 kB 00:00 2024-04-25T01:23:32.049 INFO:teuthology.orchestra.run.smithi112.stdout:(16/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x 737 kB/s | 5.4 MB 00:07 2024-04-25T01:23:32.065 INFO:teuthology.orchestra.run.smithi112.stdout:(17/152): librados-devel-19.0.0-2445.gd2c45d97. 1.5 MB/s | 127 kB 00:00 2024-04-25T01:23:32.084 INFO:teuthology.orchestra.run.smithi187.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 221 kB/s | 18 kB 00:00 2024-04-25T01:23:32.145 INFO:teuthology.orchestra.run.smithi112.stdout:(18/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 3.2 MB/s | 745 kB 00:00 2024-04-25T01:23:32.192 INFO:teuthology.orchestra.run.smithi187.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-04-25T01:23:32.216 INFO:teuthology.orchestra.run.smithi112.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 636 kB/s | 45 kB 00:00 2024-04-25T01:23:32.217 INFO:teuthology.orchestra.run.smithi187.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 83 kB/s | 11 kB 00:00 2024-04-25T01:23:32.292 INFO:teuthology.orchestra.run.smithi112.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.7 MB/s | 133 kB 00:00 2024-04-25T01:23:32.325 INFO:teuthology.orchestra.run.smithi112.stdout:(21/152): libradosstriper1-19.0.0-2445.gd2c45d9 1.8 MB/s | 501 kB 00:00 2024-04-25T01:23:32.334 INFO:teuthology.orchestra.run.smithi187.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-04-25T01:23:32.360 INFO:teuthology.orchestra.run.smithi187.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 384 kB/s | 45 kB 00:00 2024-04-25T01:23:32.392 INFO:teuthology.orchestra.run.smithi112.stdout:(22/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 16 MB/s | 5.3 MB 00:00 2024-04-25T01:23:32.417 INFO:teuthology.orchestra.run.smithi112.stdout:(23/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.3 MB/s | 162 kB 00:00 2024-04-25T01:23:32.443 INFO:teuthology.orchestra.run.smithi112.stdout:(24/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 5.9 MB/s | 299 kB 00:00 2024-04-25T01:23:32.468 INFO:teuthology.orchestra.run.smithi112.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.9 MB/s | 99 kB 00:00 2024-04-25T01:23:32.493 INFO:teuthology.orchestra.run.smithi112.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-25T01:23:32.502 INFO:teuthology.orchestra.run.smithi187.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-04-25T01:23:32.518 INFO:teuthology.orchestra.run.smithi112.stdout:(27/152): python3-rados-19.0.0-2445.gd2c45d97.e 1.6 MB/s | 321 kB 00:00 2024-04-25T01:23:32.535 INFO:teuthology.orchestra.run.smithi187.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-04-25T01:23:32.544 INFO:teuthology.orchestra.run.smithi112.stdout:(28/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 3.3 MB/s | 171 kB 00:00 2024-04-25T01:23:32.577 INFO:teuthology.orchestra.run.smithi112.stdout:(29/152): ceph-grafana-dashboards-19.0.0-2445.g 542 kB/s | 32 kB 00:00 2024-04-25T01:23:32.603 INFO:teuthology.orchestra.run.smithi112.stdout:(30/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 2.3 MB/s | 141 kB 00:00 2024-04-25T01:23:32.636 INFO:teuthology.orchestra.run.smithi187.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 191 kB/s | 25 kB 00:00 2024-04-25T01:23:32.661 INFO:teuthology.orchestra.run.smithi187.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 471 kB/s | 47 kB 00:00 2024-04-25T01:23:32.786 INFO:teuthology.orchestra.run.smithi187.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-04-25T01:23:32.845 INFO:teuthology.orchestra.run.smithi187.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-04-25T01:23:32.903 INFO:teuthology.orchestra.run.smithi187.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 302 kB/s | 35 kB 00:00 2024-04-25T01:23:33.070 INFO:teuthology.orchestra.run.smithi112.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 16 MB/s | 7.4 MB 00:00 2024-04-25T01:23:33.112 INFO:teuthology.orchestra.run.smithi187.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-04-25T01:23:33.120 INFO:teuthology.orchestra.run.smithi112.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 4.8 MB/s | 249 kB 00:00 2024-04-25T01:23:33.162 INFO:teuthology.orchestra.run.smithi112.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 1.2 MB/s | 49 kB 00:00 2024-04-25T01:23:33.205 INFO:teuthology.orchestra.run.smithi187.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-04-25T01:23:33.212 INFO:teuthology.orchestra.run.smithi112.stdout:(34/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 4.2 MB/s | 3.1 MB 00:00 2024-04-25T01:23:33.229 INFO:teuthology.orchestra.run.smithi112.stdout:(35/152): ceph-prometheus-alerts-19.0.0-2445.gd 229 kB/s | 15 kB 00:00 2024-04-25T01:23:33.305 INFO:teuthology.orchestra.run.smithi112.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 9.9 MB/s | 762 kB 00:00 2024-04-25T01:23:33.330 INFO:teuthology.orchestra.run.smithi112.stdout:(37/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 2.2 MB/s | 264 kB 00:00 2024-04-25T01:23:33.338 INFO:teuthology.orchestra.run.smithi187.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 7.1 MB/s | 1.6 MB 00:00 2024-04-25T01:23:33.347 INFO:teuthology.orchestra.run.smithi112.stdout:(38/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 437 kB/s | 18 kB 00:00 2024-04-25T01:23:33.405 INFO:teuthology.orchestra.run.smithi187.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 4.5 MB/s | 911 kB 00:00 2024-04-25T01:23:33.480 INFO:teuthology.orchestra.run.smithi187.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 940 kB/s | 133 kB 00:00 2024-04-25T01:23:33.581 INFO:teuthology.orchestra.run.smithi112.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 321 kB/s | 80 kB 00:00 2024-04-25T01:23:33.647 INFO:teuthology.orchestra.run.smithi187.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-04-25T01:23:33.714 INFO:teuthology.orchestra.run.smithi112.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 803 kB 00:00 2024-04-25T01:23:33.772 INFO:teuthology.orchestra.run.smithi187.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-04-25T01:23:33.831 INFO:teuthology.orchestra.run.smithi187.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 477 kB/s | 87 kB 00:00 2024-04-25T01:23:33.831 INFO:teuthology.orchestra.run.smithi112.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 780 kB/s | 195 kB 00:00 2024-04-25T01:23:33.964 INFO:teuthology.orchestra.run.smithi187.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.8 MB/s | 244 kB 00:00 2024-04-25T01:23:34.006 INFO:teuthology.orchestra.run.smithi112.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 868 kB/s | 253 kB 00:00 2024-04-25T01:23:34.131 INFO:teuthology.orchestra.run.smithi187.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-04-25T01:23:34.156 INFO:teuthology.orchestra.run.smithi187.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-04-25T01:23:34.165 INFO:teuthology.orchestra.run.smithi112.stdout:(43/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 2.2 MB/s | 3.5 MB 00:01 2024-04-25T01:23:34.207 INFO:teuthology.orchestra.run.smithi112.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 3.3 MB/s | 1.2 MB 00:00 2024-04-25T01:23:34.257 INFO:teuthology.orchestra.run.smithi112.stdout:(45/152): python3-ply-3.11-14.el9.noarch.rpm 425 kB/s | 106 kB 00:00 2024-04-25T01:23:34.265 INFO:teuthology.orchestra.run.smithi187.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 192 kB/s | 26 kB 00:00 2024-04-25T01:23:34.290 INFO:teuthology.orchestra.run.smithi187.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 175 kB/s | 19 kB 00:00 2024-04-25T01:23:34.407 INFO:teuthology.orchestra.run.smithi187.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-04-25T01:23:34.415 INFO:teuthology.orchestra.run.smithi112.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 540 kB/s | 135 kB 00:00 2024-04-25T01:23:34.433 INFO:teuthology.orchestra.run.smithi187.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 294 kB/s | 35 kB 00:00 2024-04-25T01:23:34.500 INFO:teuthology.orchestra.run.smithi112.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 427 kB/s | 125 kB 00:00 2024-04-25T01:23:34.542 INFO:teuthology.orchestra.run.smithi112.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 755 kB/s | 215 kB 00:00 2024-04-25T01:23:34.574 INFO:teuthology.orchestra.run.smithi187.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-04-25T01:23:34.750 INFO:teuthology.orchestra.run.smithi187.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-04-25T01:23:34.808 INFO:teuthology.orchestra.run.smithi187.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 952 kB/s | 222 kB 00:00 2024-04-25T01:23:34.875 INFO:teuthology.orchestra.run.smithi187.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 618 kB/s | 77 kB 00:00 2024-04-25T01:23:34.900 INFO:teuthology.orchestra.run.smithi187.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 465 kB/s | 42 kB 00:00 2024-04-25T01:23:34.975 INFO:teuthology.orchestra.run.smithi112.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 247 kB/s | 107 kB 00:00 2024-04-25T01:23:35.026 INFO:teuthology.orchestra.run.smithi187.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 159 kB 00:00 2024-04-25T01:23:35.051 INFO:teuthology.orchestra.run.smithi187.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 1.5 MB/s | 269 kB 00:00 2024-04-25T01:23:35.084 INFO:teuthology.orchestra.run.smithi112.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 198 kB/s | 21 kB 00:00 2024-04-25T01:23:35.176 INFO:teuthology.orchestra.run.smithi187.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-04-25T01:23:35.235 INFO:teuthology.orchestra.run.smithi187.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 318 kB/s | 51 kB 00:00 2024-04-25T01:23:35.259 INFO:teuthology.orchestra.run.smithi112.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 817 kB/s | 689 kB 00:00 2024-04-25T01:23:35.302 INFO:teuthology.orchestra.run.smithi187.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 428 kB/s | 54 kB 00:00 2024-04-25T01:23:35.334 INFO:teuthology.orchestra.run.smithi112.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-04-25T01:23:35.368 INFO:teuthology.orchestra.run.smithi112.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 302 kB/s | 33 kB 00:00 2024-04-25T01:23:35.394 INFO:teuthology.orchestra.run.smithi187.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 458 kB/s | 42 kB 00:00 2024-04-25T01:23:35.443 INFO:teuthology.orchestra.run.smithi112.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 873 kB/s | 313 kB 00:00 2024-04-25T01:23:35.477 INFO:teuthology.orchestra.run.smithi112.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-04-25T01:23:35.494 INFO:teuthology.orchestra.run.smithi187.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 443 kB/s | 44 kB 00:00 2024-04-25T01:23:35.544 INFO:teuthology.orchestra.run.smithi112.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-04-25T01:23:35.586 INFO:teuthology.orchestra.run.smithi112.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-25T01:23:35.611 INFO:teuthology.orchestra.run.smithi187.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.9 MB/s | 221 kB 00:00 2024-04-25T01:23:35.644 INFO:teuthology.orchestra.run.smithi112.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-04-25T01:23:35.711 INFO:teuthology.orchestra.run.smithi187.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 360 kB/s | 36 kB 00:00 2024-04-25T01:23:35.919 INFO:teuthology.orchestra.run.smithi112.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 2.1 MB/s | 582 kB 00:00 2024-04-25T01:23:36.020 INFO:teuthology.orchestra.run.smithi187.stdout:(95/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-04-25T01:23:36.028 INFO:teuthology.orchestra.run.smithi112.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 235 kB/s | 25 kB 00:00 2024-04-25T01:23:36.046 INFO:teuthology.orchestra.run.smithi187.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 142 kB/s | 47 kB 00:00 2024-04-25T01:23:36.137 INFO:teuthology.orchestra.run.smithi112.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 438 kB/s | 47 kB 00:00 2024-04-25T01:23:36.162 INFO:teuthology.orchestra.run.smithi112.stdout:(62/152): librdkafka-1.6.1-102.el9.x86_64.rpm 1.1 MB/s | 662 kB 00:00 2024-04-25T01:23:36.171 INFO:teuthology.orchestra.run.smithi187.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.0 MB/s | 305 kB 00:00 2024-04-25T01:23:36.196 INFO:teuthology.orchestra.run.smithi187.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 461 kB/s | 58 kB 00:00 2024-04-25T01:23:36.271 INFO:teuthology.orchestra.run.smithi112.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-25T01:23:36.330 INFO:teuthology.orchestra.run.smithi187.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 591 kB/s | 94 kB 00:00 2024-04-25T01:23:36.355 INFO:teuthology.orchestra.run.smithi187.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 473 kB/s | 64 kB 00:00 2024-04-25T01:23:36.379 INFO:teuthology.orchestra.run.smithi112.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 325 kB/s | 35 kB 00:00 2024-04-25T01:23:36.429 INFO:teuthology.orchestra.run.smithi112.stdout:(65/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.7 MB/s | 3.0 MB 00:01 2024-04-25T01:23:36.455 INFO:teuthology.orchestra.run.smithi112.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 920 kB/s | 292 kB 00:00 2024-04-25T01:23:36.455 INFO:teuthology.orchestra.run.smithi187.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 358 kB/s | 45 kB 00:00 2024-04-25T01:23:36.480 INFO:teuthology.orchestra.run.smithi187.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 221 kB/s | 22 kB 00:00 2024-04-25T01:23:36.589 INFO:teuthology.orchestra.run.smithi187.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 6.3 MB/s | 863 kB 00:00 2024-04-25T01:23:36.989 INFO:teuthology.orchestra.run.smithi187.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-25T01:23:37.015 INFO:teuthology.orchestra.run.smithi187.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 726 kB/s | 309 kB 00:00 2024-04-25T01:23:37.530 INFO:teuthology.orchestra.run.smithi112.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-04-25T01:23:37.648 INFO:teuthology.orchestra.run.smithi187.stdout:(106/152): ceph-test-19.0.0-2445.gd2c45d97.el9. 4.2 MB/s | 48 MB 00:11 2024-04-25T01:23:37.673 INFO:teuthology.orchestra.run.smithi187.stdout:(107/152): grpc-data-1.46.7-10.el9.noarch.rpm 28 kB/s | 19 kB 00:00 2024-04-25T01:23:37.860 INFO:teuthology.orchestra.run.smithi112.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.1 MB/s | 4.4 MB 00:01 2024-04-25T01:23:37.972 INFO:teuthology.orchestra.run.smithi112.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-04-25T01:23:37.991 INFO:teuthology.orchestra.run.smithi187.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 76 kB/s | 26 kB 00:00 2024-04-25T01:23:38.016 INFO:teuthology.orchestra.run.smithi187.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 142 kB/s | 49 kB 00:00 2024-04-25T01:23:38.173 INFO:teuthology.orchestra.run.smithi112.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 1.4 MB/s | 911 kB 00:00 2024-04-25T01:23:38.408 INFO:teuthology.orchestra.run.smithi187.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 162 kB/s | 67 kB 00:00 2024-04-25T01:23:38.591 INFO:teuthology.orchestra.run.smithi187.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 263 kB/s | 151 kB 00:00 2024-04-25T01:23:38.631 INFO:teuthology.orchestra.run.smithi112.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 2.1 MB/s | 4.7 MB 00:02 2024-04-25T01:23:38.731 INFO:teuthology.orchestra.run.smithi112.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 1.8 MB/s | 1.0 MB 00:00 2024-04-25T01:23:38.840 INFO:teuthology.orchestra.run.smithi112.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 419 kB/s | 87 kB 00:00 2024-04-25T01:23:39.190 INFO:teuthology.orchestra.run.smithi112.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 698 kB/s | 244 kB 00:00 2024-04-25T01:23:39.532 INFO:teuthology.orchestra.run.smithi112.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 726 kB/s | 248 kB 00:00 2024-04-25T01:23:39.632 INFO:teuthology.orchestra.run.smithi112.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-04-25T01:23:39.758 INFO:teuthology.orchestra.run.smithi112.stdout:(77/152): postgresql-server-13.14-1.el9.x86_64. 3.2 MB/s | 5.8 MB 00:01 2024-04-25T01:23:39.783 INFO:teuthology.orchestra.run.smithi112.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 127 kB/s | 19 kB 00:00 2024-04-25T01:23:39.867 INFO:teuthology.orchestra.run.smithi112.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-04-25T01:23:39.892 INFO:teuthology.orchestra.run.smithi112.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-04-25T01:23:40.867 INFO:teuthology.orchestra.run.smithi112.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 257 kB/s | 250 kB 00:00 2024-04-25T01:23:41.049 INFO:teuthology.orchestra.run.smithi187.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 223 kB/s | 548 kB 00:02 2024-04-25T01:23:41.434 INFO:teuthology.orchestra.run.smithi112.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.3 MB/s | 5.1 MB 00:01 2024-04-25T01:23:41.449 INFO:teuthology.orchestra.run.smithi187.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 276 kB/s | 838 kB 00:03 2024-04-25T01:23:41.567 INFO:teuthology.orchestra.run.smithi112.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 579 kB/s | 77 kB 00:00 2024-04-25T01:23:41.668 INFO:teuthology.orchestra.run.smithi112.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 423 kB/s | 42 kB 00:00 2024-04-25T01:23:41.725 INFO:teuthology.orchestra.run.smithi187.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 64 kB/s | 43 kB 00:00 2024-04-25T01:23:41.776 INFO:teuthology.orchestra.run.smithi112.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-04-25T01:23:41.877 INFO:teuthology.orchestra.run.smithi112.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 159 kB 00:00 2024-04-25T01:23:41.900 INFO:teuthology.orchestra.run.smithi187.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 71 kB/s | 32 kB 00:00 2024-04-25T01:23:41.919 INFO:teuthology.orchestra.run.smithi112.stdout:(87/152): python3-oauthlib-3.1.1-5.el9.noarch.r 211 kB/s | 222 kB 00:01 2024-04-25T01:23:41.985 INFO:teuthology.orchestra.run.smithi112.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-04-25T01:23:42.011 INFO:teuthology.orchestra.run.smithi112.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 555 kB/s | 51 kB 00:00 2024-04-25T01:23:42.078 INFO:teuthology.orchestra.run.smithi112.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 584 kB/s | 54 kB 00:00 2024-04-25T01:23:42.170 INFO:teuthology.orchestra.run.smithi112.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 454 kB/s | 42 kB 00:00 2024-04-25T01:23:42.234 INFO:teuthology.orchestra.run.smithi187.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 29 kB/s | 15 kB 00:00 2024-04-25T01:23:42.262 INFO:teuthology.orchestra.run.smithi112.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 482 kB/s | 44 kB 00:00 2024-04-25T01:23:42.454 INFO:teuthology.orchestra.run.smithi112.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 221 kB 00:00 2024-04-25T01:23:42.562 INFO:teuthology.orchestra.run.smithi112.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 331 kB/s | 36 kB 00:00 2024-04-25T01:23:42.663 INFO:teuthology.orchestra.run.smithi112.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 474 kB/s | 47 kB 00:00 2024-04-25T01:23:42.738 INFO:teuthology.orchestra.run.smithi112.stdout:(96/152): python3-babel-2.9.1-2.el9.noarch.rpm 1.5 MB/s | 6.0 MB 00:04 2024-04-25T01:23:42.838 INFO:teuthology.orchestra.run.smithi112.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 579 kB/s | 58 kB 00:00 2024-04-25T01:23:42.947 INFO:teuthology.orchestra.run.smithi112.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 867 kB/s | 94 kB 00:00 2024-04-25T01:23:42.972 INFO:teuthology.orchestra.run.smithi112.stdout:(99/152): socat-1.7.4.1-5.el9.x86_64.rpm 986 kB/s | 305 kB 00:00 2024-04-25T01:23:43.047 INFO:teuthology.orchestra.run.smithi112.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 635 kB/s | 64 kB 00:00 2024-04-25T01:23:43.081 INFO:teuthology.orchestra.run.smithi112.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 413 kB/s | 45 kB 00:00 2024-04-25T01:23:43.273 INFO:teuthology.orchestra.run.smithi112.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 99 kB/s | 22 kB 00:00 2024-04-25T01:23:43.573 INFO:teuthology.orchestra.run.smithi112.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-04-25T01:23:43.865 INFO:teuthology.orchestra.run.smithi112.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 926 kB/s | 548 kB 00:00 2024-04-25T01:23:44.076 INFO:teuthology.orchestra.run.smithi112.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 615 kB/s | 309 kB 00:00 2024-04-25T01:23:44.126 INFO:teuthology.orchestra.run.smithi112.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 75 kB/s | 19 kB 00:00 2024-04-25T01:23:44.184 INFO:teuthology.orchestra.run.smithi187.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 75 kB/s | 172 kB 00:02 2024-04-25T01:23:44.393 INFO:teuthology.orchestra.run.smithi112.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 97 kB/s | 26 kB 00:00 2024-04-25T01:23:44.735 INFO:teuthology.orchestra.run.smithi112.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 142 kB/s | 49 kB 00:00 2024-04-25T01:23:45.027 INFO:teuthology.orchestra.run.smithi112.stdout:(109/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.4 MB/s | 16 MB 00:03 2024-04-25T01:23:45.094 INFO:teuthology.orchestra.run.smithi112.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 4.3 MB/s | 4.4 MB 00:01 2024-04-25T01:23:45.119 INFO:teuthology.orchestra.run.smithi112.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 175 kB/s | 67 kB 00:00 2024-04-25T01:23:45.461 INFO:teuthology.orchestra.run.smithi112.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 348 kB/s | 151 kB 00:00 2024-04-25T01:23:45.720 INFO:teuthology.orchestra.run.smithi112.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 913 kB/s | 548 kB 00:00 2024-04-25T01:23:45.770 INFO:teuthology.orchestra.run.smithi112.stdout:(114/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-04-25T01:23:45.812 INFO:teuthology.orchestra.run.smithi112.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 124 kB/s | 43 kB 00:00 2024-04-25T01:23:46.004 INFO:teuthology.orchestra.run.smithi112.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 113 kB/s | 32 kB 00:00 2024-04-25T01:23:46.037 INFO:teuthology.orchestra.run.smithi112.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 56 kB/s | 15 kB 00:00 2024-04-25T01:23:46.242 INFO:teuthology.orchestra.run.smithi187.stdout:(118/152): libarrow-9.0.0-10.el9.x86_64.rpm 490 kB/s | 4.4 MB 00:09 2024-04-25T01:23:46.246 INFO:teuthology.orchestra.run.smithi112.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 396 kB/s | 172 kB 00:00 2024-04-25T01:23:46.375 INFO:teuthology.orchestra.run.smithi187.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 87 kB/s | 358 kB 00:04 2024-04-25T01:23:46.513 INFO:teuthology.orchestra.run.smithi112.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-04-25T01:23:46.538 INFO:teuthology.orchestra.run.smithi112.stdout:(120/152): python3-cherrypy-18.6.1-2.el9.noarch 671 kB/s | 358 kB 00:00 2024-04-25T01:23:46.571 INFO:teuthology.orchestra.run.smithi112.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 568 kB/s | 303 kB 00:00 2024-04-25T01:23:46.617 INFO:teuthology.orchestra.run.smithi187.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 125 kB/s | 303 kB 00:02 2024-04-25T01:23:46.684 INFO:teuthology.orchestra.run.smithi187.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 80 kB/s | 35 kB 00:00 2024-04-25T01:23:47.014 INFO:teuthology.orchestra.run.smithi112.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 411 kB/s | 206 kB 00:00 2024-04-25T01:23:47.026 INFO:teuthology.orchestra.run.smithi187.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 316 kB/s | 206 kB 00:00 2024-04-25T01:23:47.072 INFO:teuthology.orchestra.run.smithi112.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 289 kB/s | 144 kB 00:00 2024-04-25T01:23:47.189 INFO:teuthology.orchestra.run.smithi112.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 61 kB/s | 11 kB 00:00 2024-04-25T01:23:47.314 INFO:teuthology.orchestra.run.smithi112.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.6 MB/s | 2.0 MB 00:00 2024-04-25T01:23:47.339 INFO:teuthology.orchestra.run.smithi112.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 66 kB/s | 18 kB 00:00 2024-04-25T01:23:47.531 INFO:teuthology.orchestra.run.smithi112.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 68 kB/s | 23 kB 00:00 2024-04-25T01:23:47.582 INFO:teuthology.orchestra.run.smithi112.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 73 kB/s | 19 kB 00:00 2024-04-25T01:23:47.607 INFO:teuthology.orchestra.run.smithi112.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 74 kB/s | 20 kB 00:00 2024-04-25T01:23:47.932 INFO:teuthology.orchestra.run.smithi112.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 132 kB/s | 46 kB 00:00 2024-04-25T01:23:47.943 INFO:teuthology.orchestra.run.smithi187.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 12 kB/s | 11 kB 00:00 2024-04-25T01:23:47.957 INFO:teuthology.orchestra.run.smithi112.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 225 kB/s | 79 kB 00:00 2024-04-25T01:23:48.087 INFO:teuthology.orchestra.run.smithi187.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 103 kB/s | 144 kB 00:01 2024-04-25T01:23:48.218 INFO:teuthology.orchestra.run.smithi112.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 1.5 MB/s | 1.0 MB 00:00 2024-04-25T01:23:48.275 INFO:teuthology.orchestra.run.smithi112.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 168 kB/s | 58 kB 00:00 2024-04-25T01:23:48.454 INFO:teuthology.orchestra.run.smithi187.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 35 kB/s | 18 kB 00:00 2024-04-25T01:23:48.475 INFO:teuthology.orchestra.run.smithi112.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 527 kB/s | 272 kB 00:00 2024-04-25T01:23:48.509 INFO:teuthology.orchestra.run.smithi112.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 56 kB/s | 16 kB 00:00 2024-04-25T01:23:48.637 INFO:teuthology.orchestra.run.smithi187.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 42 kB/s | 23 kB 00:00 2024-04-25T01:23:48.709 INFO:teuthology.orchestra.run.smithi112.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 207 kB/s | 90 kB 00:00 2024-04-25T01:23:48.743 INFO:teuthology.orchestra.run.smithi112.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 106 kB/s | 28 kB 00:00 2024-04-25T01:23:48.776 INFO:teuthology.orchestra.run.smithi112.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 115 kB/s | 31 kB 00:00 2024-04-25T01:23:48.963 INFO:teuthology.orchestra.run.smithi187.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 38 kB/s | 19 kB 00:00 2024-04-25T01:23:49.043 INFO:teuthology.orchestra.run.smithi112.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 134 kB/s | 36 kB 00:00 2024-04-25T01:23:49.085 INFO:teuthology.orchestra.run.smithi112.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 173 kB/s | 59 kB 00:00 2024-04-25T01:23:49.096 INFO:teuthology.orchestra.run.smithi187.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 43 kB/s | 20 kB 00:00 2024-04-25T01:23:49.152 INFO:teuthology.orchestra.run.smithi112.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 426 kB/s | 188 kB 00:00 2024-04-25T01:23:49.419 INFO:teuthology.orchestra.run.smithi112.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 136 kB/s | 51 kB 00:00 2024-04-25T01:23:49.480 INFO:teuthology.orchestra.run.smithi187.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 729 kB/s | 2.0 MB 00:02 2024-04-25T01:23:49.538 INFO:teuthology.orchestra.run.smithi187.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 105 kB/s | 46 kB 00:00 2024-04-25T01:23:49.594 INFO:teuthology.orchestra.run.smithi112.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 203 kB/s | 90 kB 00:00 2024-04-25T01:23:49.619 INFO:teuthology.orchestra.run.smithi112.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 430 kB/s | 230 kB 00:00 2024-04-25T01:23:49.862 INFO:teuthology.orchestra.run.smithi112.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 75 kB/s | 20 kB 00:00 2024-04-25T01:23:49.947 INFO:teuthology.orchestra.run.smithi187.stdout:(131/152): python3-natsort-7.1.1-5.el9.noarch.r 141 kB/s | 58 kB 00:00 2024-04-25T01:23:49.989 INFO:teuthology.orchestra.run.smithi187.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 155 kB/s | 79 kB 00:00 2024-04-25T01:23:50.003 INFO:teuthology.orchestra.run.smithi112.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 731 kB/s | 427 kB 00:00 2024-04-25T01:23:50.129 INFO:teuthology.orchestra.run.smithi112.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 377 kB/s | 192 kB 00:00 2024-04-25T01:23:50.247 INFO:teuthology.orchestra.run.smithi187.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 814 kB/s | 1.0 MB 00:01 2024-04-25T01:23:50.464 INFO:teuthology.orchestra.run.smithi187.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 527 kB/s | 272 kB 00:00 2024-04-25T01:23:50.489 INFO:teuthology.orchestra.run.smithi187.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 33 kB/s | 16 kB 00:00 2024-04-25T01:23:50.554 INFO:teuthology.orchestra.run.smithi112.stdout:(148/152): librados2-19.0.0-2445.gd2c45d97.el9. 6.2 MB/s | 3.4 MB 00:00 2024-04-25T01:23:50.573 INFO:teuthology.orchestra.run.smithi187.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 276 kB/s | 90 kB 00:00 2024-04-25T01:23:50.596 INFO:teuthology.orchestra.run.smithi112.stdout:(149/152): thrift-0.15.0-2.el9.x86_64.rpm 2.2 MB/s | 1.6 MB 00:00 2024-04-25T01:23:50.606 INFO:teuthology.orchestra.run.smithi187.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 198 kB/s | 28 kB 00:00 2024-04-25T01:23:50.632 INFO:teuthology.orchestra.run.smithi187.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 216 kB/s | 31 kB 00:00 2024-04-25T01:23:50.824 INFO:teuthology.orchestra.run.smithi187.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 753 kB/s | 188 kB 00:00 2024-04-25T01:23:50.882 INFO:teuthology.orchestra.run.smithi187.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 215 kB/s | 59 kB 00:00 2024-04-25T01:23:50.899 INFO:teuthology.orchestra.run.smithi187.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 134 kB/s | 36 kB 00:00 2024-04-25T01:23:50.958 INFO:teuthology.orchestra.run.smithi187.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 382 kB/s | 51 kB 00:00 2024-04-25T01:23:51.038 INFO:teuthology.orchestra.run.smithi112.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 2.8 MB/s | 1.3 MB 00:00 2024-04-25T01:23:51.238 INFO:teuthology.orchestra.run.smithi112.stdout:(151/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 2.8 MB/s | 3.1 MB 00:01 2024-04-25T01:23:51.333 INFO:teuthology.orchestra.run.smithi187.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 510 kB/s | 230 kB 00:00 2024-04-25T01:23:51.358 INFO:teuthology.orchestra.run.smithi187.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 195 kB/s | 90 kB 00:00 2024-04-25T01:23:51.934 INFO:teuthology.orchestra.run.smithi187.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 438 kB/s | 427 kB 00:00 2024-04-25T01:23:51.967 INFO:teuthology.orchestra.run.smithi187.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 31 kB/s | 20 kB 00:00 2024-04-25T01:23:52.046 INFO:teuthology.orchestra.run.smithi112.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.7 MB/s | 2.4 MB 00:01 2024-04-25T01:23:52.050 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:23:52.050 INFO:teuthology.orchestra.run.smithi112.stdout:Total 7.9 MB/s | 220 MB 00:28 2024-04-25T01:23:52.275 INFO:teuthology.orchestra.run.smithi187.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 209 kB/s | 192 kB 00:00 2024-04-25T01:23:52.651 INFO:teuthology.orchestra.run.smithi187.stdout:(148/152): librados2-19.0.0-2445.gd2c45d97.el9. 5.0 MB/s | 3.4 MB 00:00 2024-04-25T01:23:52.976 INFO:teuthology.orchestra.run.smithi187.stdout:(149/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 4.5 MB/s | 3.1 MB 00:00 2024-04-25T01:23:53.010 INFO:teuthology.orchestra.run.smithi187.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 3.8 MB/s | 1.3 MB 00:00 2024-04-25T01:23:53.158 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-04-25T01:23:53.301 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-04-25T01:23:53.301 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-04-25T01:23:53.360 INFO:teuthology.orchestra.run.smithi187.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.2 MB/s | 2.4 MB 00:00 2024-04-25T01:23:55.049 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-04-25T01:23:55.049 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-04-25T01:23:55.401 INFO:teuthology.orchestra.run.smithi187.stdout:(152/152): thrift-0.15.0-2.el9.x86_64.rpm 467 kB/s | 1.6 MB 00:03 2024-04-25T01:23:55.407 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:23:55.407 INFO:teuthology.orchestra.run.smithi187.stdout:Total 6.6 MB/s | 220 MB 00:33 2024-04-25T01:23:56.449 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2024-04-25T01:23:56.591 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2024-04-25T01:23:56.591 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2024-04-25T01:23:58.422 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2024-04-25T01:23:58.422 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2024-04-25T01:24:00.967 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-04-25T01:24:01.052 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T01:24:01.088 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T01:24:01.151 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T01:24:01.184 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T01:24:01.210 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T01:24:01.253 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T01:24:01.574 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T01:24:01.686 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:24:01.798 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:24:01.828 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:24:01.902 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:24:01.949 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T01:24:02.099 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T01:24:02.155 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T01:24:02.183 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:24:02.285 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:24:02.309 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T01:24:02.371 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T01:24:02.393 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T01:24:02.411 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T01:24:02.601 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T01:24:02.634 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T01:24:02.741 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T01:24:02.844 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T01:24:03.067 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T01:24:03.101 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T01:24:03.165 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T01:24:03.200 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T01:24:03.244 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T01:24:03.276 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T01:24:03.304 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T01:24:03.373 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T01:24:03.435 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T01:24:03.465 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T01:24:03.517 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T01:24:03.593 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T01:24:03.628 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T01:24:03.671 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T01:24:03.700 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T01:24:03.769 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T01:24:03.815 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T01:24:03.898 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T01:24:04.030 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T01:24:04.084 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T01:24:04.317 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2024-04-25T01:24:04.397 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T01:24:04.435 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T01:24:04.488 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T01:24:04.522 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T01:24:04.548 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T01:24:04.593 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T01:24:04.593 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T01:24:04.680 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T01:24:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T01:24:04.743 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T01:24:04.782 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T01:24:04.823 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T01:24:04.968 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T01:24:04.996 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T01:24:05.024 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:24:05.043 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T01:24:05.097 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T01:24:05.129 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:24:05.158 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:24:05.169 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T01:24:05.202 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T01:24:05.238 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T01:24:05.240 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:24:05.271 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T01:24:05.286 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T01:24:05.321 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T01:24:05.364 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T01:24:05.406 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T01:24:05.417 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T01:24:05.476 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T01:24:05.504 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:24:05.541 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T01:24:05.587 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T01:24:05.606 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:24:05.617 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T01:24:05.630 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T01:24:05.659 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T01:24:05.692 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T01:24:05.714 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T01:24:05.737 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T01:24:05.751 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T01:24:05.784 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T01:24:05.824 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T01:24:05.849 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T01:24:05.874 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T01:24:05.939 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T01:24:05.948 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T01:24:05.972 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T01:24:06.077 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T01:24:06.092 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T01:24:06.124 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T01:24:06.143 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T01:24:06.172 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T01:24:06.177 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T01:24:06.213 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T01:24:06.216 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T01:24:06.247 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T01:24:06.248 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T01:24:06.279 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T01:24:06.311 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T01:24:06.313 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T01:24:06.346 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T01:24:06.346 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T01:24:06.388 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T01:24:06.389 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T01:24:06.422 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T01:24:06.423 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T01:24:06.450 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T01:24:06.509 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T01:24:06.509 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T01:24:06.552 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T01:24:06.564 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T01:24:06.584 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T01:24:06.595 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T01:24:06.629 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T01:24:06.631 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T01:24:06.705 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T01:24:06.729 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T01:24:06.741 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T01:24:06.784 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T01:24:06.785 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T01:24:06.813 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T01:24:06.883 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T01:24:06.936 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T01:24:07.010 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T01:24:07.142 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T01:24:07.196 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T01:24:07.495 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T01:24:07.541 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T01:24:07.575 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T01:24:07.672 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T01:24:07.704 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T01:24:07.732 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T01:24:07.791 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T01:24:07.820 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T01:24:07.821 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T01:24:07.851 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T01:24:07.856 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T01:24:07.894 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T01:24:07.936 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T01:24:08.029 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T01:24:08.058 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T01:24:08.089 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T01:24:08.135 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T01:24:08.138 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T01:24:08.193 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T01:24:08.266 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T01:24:08.297 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T01:24:08.333 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T01:24:08.380 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T01:24:08.416 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T01:24:08.468 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T01:24:08.511 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T01:24:08.652 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T01:24:08.699 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T01:24:08.729 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T01:24:08.765 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T01:24:08.770 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T01:24:08.866 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T01:24:08.881 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T01:24:08.896 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T01:24:08.927 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T01:24:08.953 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T01:24:08.977 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T01:24:09.053 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T01:24:09.178 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T01:24:09.210 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T01:24:09.238 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T01:24:09.272 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T01:24:09.311 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T01:24:09.343 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T01:24:09.373 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T01:24:09.408 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T01:24:09.441 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T01:24:09.482 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T01:24:09.517 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T01:24:09.623 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T01:24:09.672 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T01:24:09.703 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T01:24:09.742 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T01:24:09.839 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T01:24:09.887 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T01:24:10.086 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T01:24:10.171 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T01:24:10.199 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T01:24:10.232 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T01:24:10.542 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T01:24:10.587 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T01:24:10.595 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T01:24:10.644 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T01:24:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T01:24:10.677 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T01:24:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T01:24:10.727 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T01:24:10.775 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T01:24:10.835 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T01:24:10.921 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T01:24:10.954 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T01:24:11.130 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T01:24:11.160 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T01:24:11.197 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T01:24:11.231 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T01:24:11.237 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T01:24:11.294 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T01:24:11.324 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T01:24:11.879 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T01:24:11.991 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T01:24:13.190 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T01:24:13.264 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T01:24:13.284 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T01:24:13.317 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T01:24:13.619 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T01:24:13.642 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:24:13.654 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T01:24:13.672 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:24:13.744 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T01:24:13.776 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T01:24:13.812 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T01:24:14.226 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:24:14.276 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T01:24:14.280 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T01:24:14.307 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T01:24:14.369 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T01:24:14.369 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T01:24:14.369 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T01:24:14.369 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:14.378 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T01:24:14.400 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T01:24:14.402 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T01:24:16.614 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:24:16.655 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:24:17.192 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:24:17.242 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T01:24:17.334 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T01:24:17.334 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T01:24:17.334 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T01:24:17.334 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:17.368 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T01:24:25.886 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T01:24:25.886 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2024-04-25T01:24:25.886 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2024-04-25T01:24:25.886 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2024-04-25T01:24:25.887 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2024-04-25T01:24:25.887 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2024-04-25T01:24:25.887 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /root 2024-04-25T01:24:25.887 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2024-04-25T01:24:25.887 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:26.047 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T01:24:26.063 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T01:24:26.063 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:26.063 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T01:24:26.063 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:27.507 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T01:24:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T01:24:27.532 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:27.532 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T01:24:27.532 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:24:27.532 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:24:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:27.757 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T01:24:27.779 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T01:24:27.779 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:27.779 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T01:24:27.779 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:24:27.779 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:24:27.779 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:28.216 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T01:24:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T01:24:28.239 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:28.239 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T01:24:28.239 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:24:28.239 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:24:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:28.338 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T01:24:28.386 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T01:24:28.442 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T01:24:28.579 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T01:24:28.579 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /sys 2024-04-25T01:24:28.579 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /proc 2024-04-25T01:24:28.579 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /mnt 2024-04-25T01:24:28.580 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /var/tmp 2024-04-25T01:24:28.580 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /home 2024-04-25T01:24:28.580 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /root 2024-04-25T01:24:28.580 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /tmp 2024-04-25T01:24:28.580 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:28.627 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T01:24:28.675 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T01:24:28.746 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T01:24:28.752 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T01:24:28.761 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T01:24:28.761 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:28.761 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T01:24:28.761 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:28.841 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T01:24:28.981 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T01:24:29.017 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T01:24:29.049 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T01:24:29.077 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T01:24:29.483 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T01:24:29.508 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T01:24:29.539 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T01:24:29.602 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T01:24:29.648 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T01:24:30.274 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T01:24:30.299 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T01:24:30.299 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:30.299 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T01:24:30.300 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:24:30.300 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:24:30.300 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:30.525 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T01:24:30.546 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T01:24:30.546 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:30.546 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T01:24:30.546 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:24:30.547 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:24:30.547 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:30.684 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T01:24:30.733 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T01:24:30.984 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T01:24:31.006 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T01:24:31.006 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:31.006 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T01:24:31.006 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:24:31.006 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:24:31.006 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:31.106 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T01:24:31.161 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T01:24:31.217 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T01:24:31.402 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T01:24:31.450 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T01:24:31.519 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T01:24:31.608 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T01:24:31.703 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T01:24:31.742 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T01:24:31.747 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T01:24:31.784 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T01:24:31.816 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T01:24:31.844 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T01:24:31.844 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T01:24:32.005 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T01:24:32.027 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T01:24:32.049 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T01:24:32.050 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:32.050 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T01:24:32.050 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:24:32.050 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:24:32.050 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:32.095 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T01:24:32.105 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T01:24:32.203 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T01:24:32.250 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T01:24:32.259 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T01:24:32.276 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T01:24:32.307 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T01:24:32.368 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T01:24:32.399 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T01:24:33.170 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T01:24:33.214 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T01:24:33.236 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T01:24:33.236 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:33.237 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T01:24:33.237 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:24:33.237 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:24:33.237 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:33.281 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T01:24:33.303 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T01:24:33.303 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:33.303 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T01:24:33.304 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:33.349 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T01:24:33.371 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T01:24:33.371 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:33.371 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T01:24:33.371 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:24:33.371 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:24:33.371 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:33.424 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T01:24:33.458 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T01:24:33.674 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T01:24:33.696 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T01:24:33.696 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:33.697 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T01:24:33.697 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:24:33.697 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:24:33.697 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:34.415 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T01:24:34.442 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T01:24:34.544 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T01:24:34.706 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T01:24:34.736 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T01:24:34.758 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T01:24:34.758 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:34.758 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T01:24:34.758 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:24:34.759 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:24:34.759 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:34.803 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T01:24:34.814 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T01:24:34.911 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T01:24:34.960 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T01:24:35.851 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T01:24:35.897 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T01:24:35.919 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T01:24:35.919 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:35.919 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T01:24:35.919 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:24:35.919 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:24:35.920 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:35.964 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T01:24:35.986 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T01:24:35.986 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:35.986 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T01:24:35.986 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:36.032 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T01:24:36.053 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T01:24:36.054 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:36.054 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T01:24:36.054 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:24:36.054 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:24:36.054 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:36.357 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T01:24:36.379 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T01:24:36.379 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:36.379 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T01:24:36.379 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:24:36.379 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:24:36.380 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:38.211 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T01:24:38.252 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T01:24:38.301 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T01:24:38.334 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T01:24:38.361 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T01:24:38.545 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T01:24:38.599 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T01:24:38.629 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T01:24:39.206 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:24:39.252 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:24:39.272 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:24:40.102 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T01:24:40.190 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T01:24:40.231 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T01:24:40.313 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T01:24:40.344 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T01:24:40.345 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T01:24:40.428 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T01:24:40.470 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T01:24:40.470 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T01:24:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T01:24:40.635 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:24:40.869 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T01:24:40.911 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T01:24:40.951 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T01:24:40.983 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T01:24:41.011 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T01:24:41.182 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T01:24:41.223 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T01:24:41.253 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T01:24:41.830 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:24:41.867 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:24:41.887 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:24:42.703 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T01:24:42.789 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T01:24:42.830 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T01:24:42.910 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T01:24:42.942 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T01:24:42.943 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T01:24:43.036 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T01:24:43.077 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T01:24:43.077 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T01:24:43.154 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T01:24:43.224 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:24:46.670 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:24:46.671 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T01:24:46.671 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T01:24:46.671 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T01:24:46.671 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T01:24:46.671 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T01:24:46.671 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T01:24:46.671 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T01:24:46.671 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:24:46.671 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:24:46.671 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T01:24:46.671 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T01:24:46.671 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T01:24:46.671 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:24:46.672 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T01:24:46.672 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T01:24:46.672 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T01:24:46.672 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T01:24:46.672 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T01:24:46.672 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T01:24:46.672 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T01:24:46.672 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T01:24:46.672 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T01:24:46.672 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T01:24:46.672 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T01:24:46.672 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T01:24:46.672 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T01:24:46.672 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T01:24:46.673 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T01:24:46.673 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T01:24:46.684 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T01:24:46.684 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T01:24:46.684 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T01:24:46.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T01:24:46.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T01:24:46.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T01:24:46.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T01:24:46.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T01:24:46.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T01:24:46.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T01:24:46.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T01:24:46.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T01:24:46.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T01:24:46.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T01:24:46.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T01:24:46.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T01:24:46.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T01:24:46.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T01:24:46.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T01:24:46.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T01:24:46.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T01:24:46.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T01:24:46.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T01:24:46.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T01:24:46.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T01:24:46.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T01:24:46.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T01:24:46.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T01:24:46.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T01:24:46.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T01:24:46.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T01:24:46.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T01:24:46.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T01:24:46.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T01:24:46.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T01:24:46.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T01:24:46.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T01:24:46.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T01:24:46.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T01:24:46.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T01:24:46.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T01:24:46.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T01:24:46.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T01:24:46.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T01:24:46.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T01:24:46.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T01:24:46.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T01:24:46.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T01:24:46.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T01:24:46.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T01:24:46.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T01:24:46.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T01:24:46.689 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T01:24:46.689 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T01:24:46.689 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T01:24:46.690 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T01:24:46.690 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T01:24:46.690 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T01:24:46.690 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T01:24:46.690 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T01:24:46.690 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T01:24:46.690 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T01:24:46.690 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T01:24:46.690 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T01:24:46.690 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T01:24:46.690 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T01:24:46.690 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T01:24:46.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T01:24:46.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T01:24:46.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T01:24:46.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T01:24:46.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T01:24:46.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T01:24:46.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T01:24:46.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T01:24:46.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T01:24:46.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T01:24:46.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T01:24:46.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T01:24:46.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T01:24:46.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T01:24:46.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T01:24:46.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T01:24:46.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T01:24:46.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T01:24:46.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T01:24:46.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T01:24:46.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T01:24:46.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T01:24:46.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T01:24:46.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T01:24:46.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T01:24:46.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T01:24:46.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T01:24:46.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T01:24:46.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T01:24:46.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T01:24:46.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T01:24:46.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T01:24:46.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T01:24:46.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T01:24:46.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T01:24:46.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T01:24:46.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T01:24:46.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T01:24:46.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T01:24:46.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T01:24:46.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T01:24:46.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T01:24:46.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T01:24:46.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T01:24:46.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T01:24:46.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T01:24:46.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T01:24:46.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T01:24:46.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T01:24:46.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T01:24:46.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T01:24:46.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T01:24:46.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T01:24:46.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T01:24:46.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T01:24:46.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T01:24:46.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T01:24:46.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T01:24:46.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T01:24:48.776 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:24:48.776 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:48.776 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2024-04-25T01:24:48.776 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.776 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.776 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T01:24:48.776 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T01:24:48.776 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-04-25T01:24:48.776 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T01:24:48.776 INFO:teuthology.orchestra.run.smithi112.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T01:24:48.776 INFO:teuthology.orchestra.run.smithi112.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T01:24:48.776 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T01:24:48.777 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.777 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.777 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.777 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.777 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:24:48.777 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.777 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.777 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.777 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:24:48.777 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:24:48.778 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:24:48.778 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:24:48.778 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:24:48.778 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.778 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.778 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:24:48.778 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.778 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.778 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.778 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:24:48.778 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:24:48.779 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.779 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.779 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:24:48.779 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T01:24:48.779 INFO:teuthology.orchestra.run.smithi112.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T01:24:48.779 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T01:24:48.779 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T01:24:48.779 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T01:24:48.779 INFO:teuthology.orchestra.run.smithi112.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T01:24:48.779 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T01:24:48.779 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T01:24:48.779 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T01:24:48.779 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T01:24:48.779 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T01:24:48.780 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.780 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.780 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.780 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T01:24:48.780 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T01:24:48.780 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T01:24:48.780 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T01:24:48.780 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T01:24:48.780 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.780 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.780 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T01:24:48.780 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.780 INFO:teuthology.orchestra.run.smithi112.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T01:24:48.780 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T01:24:48.781 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T01:24:48.781 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T01:24:48.781 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T01:24:48.781 INFO:teuthology.orchestra.run.smithi112.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T01:24:48.781 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T01:24:48.781 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T01:24:48.781 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T01:24:48.781 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T01:24:48.781 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T01:24:48.781 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T01:24:48.781 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T01:24:48.782 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T01:24:48.782 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T01:24:48.782 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T01:24:48.782 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T01:24:48.782 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T01:24:48.782 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T01:24:48.782 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T01:24:48.782 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T01:24:48.782 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T01:24:48.782 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T01:24:48.782 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.782 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.782 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.782 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T01:24:48.783 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T01:24:48.783 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T01:24:48.783 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T01:24:48.783 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T01:24:48.783 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T01:24:48.783 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T01:24:48.783 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T01:24:48.783 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T01:24:48.783 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T01:24:48.783 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T01:24:48.783 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T01:24:48.783 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T01:24:48.783 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T01:24:48.783 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T01:24:48.783 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T01:24:48.784 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T01:24:48.784 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T01:24:48.784 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T01:24:48.784 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T01:24:48.784 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T01:24:48.784 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T01:24:48.784 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T01:24:48.784 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T01:24:48.784 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T01:24:48.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T01:24:48.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T01:24:48.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T01:24:48.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T01:24:48.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T01:24:48.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T01:24:48.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T01:24:48.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T01:24:48.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T01:24:48.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T01:24:48.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T01:24:48.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T01:24:48.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T01:24:48.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T01:24:48.786 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T01:24:48.786 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.786 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.786 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T01:24:48.786 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T01:24:48.786 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T01:24:48.786 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.786 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T01:24:48.786 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T01:24:48.786 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T01:24:48.786 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T01:24:48.786 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T01:24:48.786 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T01:24:48.786 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T01:24:48.787 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T01:24:48.787 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T01:24:48.787 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T01:24:48.787 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T01:24:48.787 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T01:24:48.787 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T01:24:48.787 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T01:24:48.787 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T01:24:48.787 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.787 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.787 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:48.787 INFO:teuthology.orchestra.run.smithi112.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T01:24:48.787 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T01:24:48.787 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T01:24:48.787 INFO:teuthology.orchestra.run.smithi112.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T01:24:48.788 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T01:24:48.788 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T01:24:48.788 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T01:24:48.788 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:48.788 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-04-25T01:24:48.791 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:24:48.791 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T01:24:48.791 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T01:24:48.791 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T01:24:48.792 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T01:24:48.792 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T01:24:48.792 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T01:24:48.792 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T01:24:48.792 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:24:48.792 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:24:48.792 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T01:24:48.792 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T01:24:48.792 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T01:24:48.792 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:24:48.792 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T01:24:48.792 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T01:24:48.792 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T01:24:48.792 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T01:24:48.793 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T01:24:48.793 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T01:24:48.793 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T01:24:48.793 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T01:24:48.793 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T01:24:48.793 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T01:24:48.793 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T01:24:48.793 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T01:24:48.793 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T01:24:48.793 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T01:24:48.793 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T01:24:48.793 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T01:24:48.793 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T01:24:48.806 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T01:24:48.806 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T01:24:48.806 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T01:24:48.806 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T01:24:48.806 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T01:24:48.807 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T01:24:48.807 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T01:24:48.807 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T01:24:48.807 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T01:24:48.807 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T01:24:48.807 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T01:24:48.807 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T01:24:48.807 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T01:24:48.807 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T01:24:48.807 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T01:24:48.807 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T01:24:48.807 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T01:24:48.807 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T01:24:48.807 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T01:24:48.808 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T01:24:48.808 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T01:24:48.808 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T01:24:48.808 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T01:24:48.808 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T01:24:48.808 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T01:24:48.808 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T01:24:48.808 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T01:24:48.808 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T01:24:48.808 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T01:24:48.808 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T01:24:48.808 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T01:24:48.808 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T01:24:48.808 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T01:24:48.808 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T01:24:48.808 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T01:24:48.809 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T01:24:48.809 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T01:24:48.809 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T01:24:48.809 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T01:24:48.809 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T01:24:48.809 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T01:24:48.809 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T01:24:48.809 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T01:24:48.809 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T01:24:48.809 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T01:24:48.809 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T01:24:48.809 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T01:24:48.809 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T01:24:48.809 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T01:24:48.809 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T01:24:48.810 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T01:24:48.810 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T01:24:48.810 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T01:24:48.810 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T01:24:48.810 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T01:24:48.810 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T01:24:48.810 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T01:24:48.810 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T01:24:48.810 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T01:24:48.810 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T01:24:48.810 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T01:24:48.810 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T01:24:48.811 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T01:24:48.811 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T01:24:48.811 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T01:24:48.811 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T01:24:48.811 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T01:24:48.811 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T01:24:48.811 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T01:24:48.811 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T01:24:48.811 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T01:24:48.811 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T01:24:48.811 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T01:24:48.811 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T01:24:48.811 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T01:24:48.811 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T01:24:48.812 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T01:24:48.812 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T01:24:48.812 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T01:24:48.812 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T01:24:48.812 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T01:24:48.812 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T01:24:48.812 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T01:24:48.812 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T01:24:48.812 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T01:24:48.812 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T01:24:48.812 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T01:24:48.812 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T01:24:48.812 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T01:24:48.812 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T01:24:48.813 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T01:24:48.813 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T01:24:48.813 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T01:24:48.813 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T01:24:48.813 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T01:24:48.813 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T01:24:48.813 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T01:24:48.813 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T01:24:48.813 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T01:24:48.813 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T01:24:48.813 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T01:24:48.813 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T01:24:48.813 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T01:24:48.813 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T01:24:48.813 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T01:24:48.814 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T01:24:48.814 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T01:24:48.814 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T01:24:48.814 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T01:24:48.814 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T01:24:48.814 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T01:24:48.814 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T01:24:48.814 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T01:24:48.814 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T01:24:48.814 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T01:24:48.814 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T01:24:48.814 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T01:24:48.814 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T01:24:48.814 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T01:24:48.815 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T01:24:48.815 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T01:24:48.815 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T01:24:48.815 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T01:24:48.815 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T01:24:48.815 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T01:24:49.172 DEBUG:teuthology.parallel:result is None 2024-04-25T01:24:51.019 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:24:51.019 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:51.019 INFO:teuthology.orchestra.run.smithi187.stdout:Upgraded: 2024-04-25T01:24:51.019 INFO:teuthology.orchestra.run.smithi187.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.020 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.020 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T01:24:51.020 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T01:24:51.020 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2024-04-25T01:24:51.020 INFO:teuthology.orchestra.run.smithi187.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T01:24:51.020 INFO:teuthology.orchestra.run.smithi187.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T01:24:51.020 INFO:teuthology.orchestra.run.smithi187.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T01:24:51.020 INFO:teuthology.orchestra.run.smithi187.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T01:24:51.020 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.020 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.020 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.021 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.021 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:24:51.021 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.021 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.021 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.021 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:24:51.021 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:24:51.021 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:24:51.021 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:24:51.021 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:24:51.021 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.021 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.021 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:24:51.021 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.021 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.021 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.022 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:24:51.022 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:24:51.022 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.022 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.022 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:24:51.022 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T01:24:51.022 INFO:teuthology.orchestra.run.smithi187.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T01:24:51.022 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T01:24:51.022 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T01:24:51.022 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T01:24:51.022 INFO:teuthology.orchestra.run.smithi187.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T01:24:51.022 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T01:24:51.022 INFO:teuthology.orchestra.run.smithi187.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T01:24:51.022 INFO:teuthology.orchestra.run.smithi187.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T01:24:51.023 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T01:24:51.023 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T01:24:51.023 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.023 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.023 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.023 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T01:24:51.023 INFO:teuthology.orchestra.run.smithi187.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T01:24:51.023 INFO:teuthology.orchestra.run.smithi187.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T01:24:51.023 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T01:24:51.023 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T01:24:51.023 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.023 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.023 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T01:24:51.023 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.023 INFO:teuthology.orchestra.run.smithi187.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T01:24:51.024 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T01:24:51.024 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T01:24:51.024 INFO:teuthology.orchestra.run.smithi187.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T01:24:51.024 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T01:24:51.024 INFO:teuthology.orchestra.run.smithi187.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T01:24:51.024 INFO:teuthology.orchestra.run.smithi187.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T01:24:51.024 INFO:teuthology.orchestra.run.smithi187.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T01:24:51.024 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T01:24:51.024 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T01:24:51.024 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T01:24:51.024 INFO:teuthology.orchestra.run.smithi187.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T01:24:51.024 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T01:24:51.024 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T01:24:51.024 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T01:24:51.025 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T01:24:51.025 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T01:24:51.025 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T01:24:51.025 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T01:24:51.025 INFO:teuthology.orchestra.run.smithi187.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T01:24:51.025 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T01:24:51.025 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T01:24:51.025 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T01:24:51.025 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.025 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.025 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.025 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T01:24:51.025 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T01:24:51.025 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T01:24:51.025 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T01:24:51.025 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T01:24:51.026 INFO:teuthology.orchestra.run.smithi187.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T01:24:51.026 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T01:24:51.026 INFO:teuthology.orchestra.run.smithi187.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T01:24:51.026 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T01:24:51.026 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T01:24:51.026 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T01:24:51.026 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T01:24:51.026 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T01:24:51.026 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T01:24:51.026 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T01:24:51.026 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T01:24:51.026 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T01:24:51.026 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T01:24:51.026 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T01:24:51.027 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T01:24:51.027 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T01:24:51.027 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T01:24:51.027 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T01:24:51.027 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T01:24:51.027 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T01:24:51.027 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T01:24:51.027 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T01:24:51.027 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T01:24:51.027 INFO:teuthology.orchestra.run.smithi187.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T01:24:51.027 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T01:24:51.027 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T01:24:51.027 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T01:24:51.027 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T01:24:51.027 INFO:teuthology.orchestra.run.smithi187.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T01:24:51.028 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T01:24:51.028 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T01:24:51.028 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T01:24:51.028 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T01:24:51.028 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T01:24:51.028 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T01:24:51.028 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.028 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.028 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T01:24:51.028 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T01:24:51.028 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T01:24:51.028 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.028 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T01:24:51.028 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T01:24:51.029 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T01:24:51.029 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T01:24:51.029 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T01:24:51.029 INFO:teuthology.orchestra.run.smithi187.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T01:24:51.029 INFO:teuthology.orchestra.run.smithi187.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T01:24:51.029 INFO:teuthology.orchestra.run.smithi187.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T01:24:51.029 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T01:24:51.029 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T01:24:51.029 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T01:24:51.029 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T01:24:51.029 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T01:24:51.029 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T01:24:51.029 INFO:teuthology.orchestra.run.smithi187.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T01:24:51.029 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.030 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.030 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:24:51.030 INFO:teuthology.orchestra.run.smithi187.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T01:24:51.030 INFO:teuthology.orchestra.run.smithi187.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T01:24:51.030 INFO:teuthology.orchestra.run.smithi187.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T01:24:51.030 INFO:teuthology.orchestra.run.smithi187.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T01:24:51.030 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T01:24:51.030 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T01:24:51.030 INFO:teuthology.orchestra.run.smithi187.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T01:24:51.030 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:51.031 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2024-04-25T01:24:51.432 DEBUG:teuthology.parallel:result is None 2024-04-25T01:24:51.432 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:24:51.582 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T01:24:51.607 INFO:teuthology.orchestra.run.smithi112.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T01:24:51.607 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T01:24:51.608 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T01:24:51.609 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:24:51.825 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T01:24:51.846 INFO:teuthology.orchestra.run.smithi187.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T01:24:51.846 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T01:24:51.846 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T01:24:51.848 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T01:24:51.848 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:24:51.848 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T01:24:51.879 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:24:51.879 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T01:24:51.912 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T01:24:51.912 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:24:51.912 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T01:24:51.945 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T01:24:52.010 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:24:52.010 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T01:24:52.033 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T01:24:52.097 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T01:24:52.097 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:24:52.097 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T01:24:52.122 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T01:24:52.186 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:24:52.186 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T01:24:52.210 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T01:24:52.273 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T01:24:52.273 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:24:52.273 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T01:24:52.298 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T01:24:52.362 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:24:52.362 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T01:24:52.385 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T01:24:52.449 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T01:24:52.532 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T01:24:52.533 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /var/log/ceph 2024-04-25T01:24:52.535 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 777 /var/log/ceph 2024-04-25T01:24:52.558 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T01:24:52.559 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T01:24:52.600 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T01:24:52.624 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T01:24:52.624 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T01:24:52.664 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T01:24:52.695 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T01:24:52.695 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T01:24:52.695 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T01:24:52.695 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:24:52.695 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T01:24:52.743 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T01:24:52.844 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T01:24:52.889 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:24:52.889 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T01:24:52.903 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T01:24:53.008 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T01:24:53.056 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T01:24:53.056 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', '\\(MON_DOWN\\)', 'SLOW_OPS', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T01:24:53.056 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672321', 'branch': 'squid', 'description': 'fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7672321', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'fs': [{'name': 'a'}, {'name': 'b'}], 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', '\\(MON_DOWN\\)', 'SLOW_OPS', 'slow request'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'ceph-fuse': {'client.0': {'cephfs_name': 'a'}, 'client.1': {'cephfs_name': 'b'}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'kclient': {'client.0': {'cephfs_name': 'a'}, 'client.1': {'cephfs_name': 'b'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'seed': 8528, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'sleep_before_teardown': 0, 'subset': '122/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi112.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFFMkfUTOwGo5U4OtPFRy5CmKpJlXkNvIN4DuqzWX2DTanmTMsV7XjHV3GA256wj/AqKAnDaxOfo7wHizR+uR9o=', 'smithi187.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJOxhg14KF+E3wrNmPdK1+nBia5IPCfa4VbGMyMZoqkUDLdsa4S4fxH9fcKIbP9pK6r0Ce0Oo4tcbAuPD1hN2Yk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'fs': [{'name': 'a'}, {'name': 'b'}], 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', '\\(MON_DOWN\\)', 'SLOW_OPS', 'slow request'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'mon_thrash': {'check_mds_failover': True, 'revive_delay': 20, 'thrash_delay': 10}}, {'workunit': {'clients': {'all': ['suites/ffsb.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-24_21:24:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T01:24:53.057 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T01:24:53.059 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T01:24:53.107 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T01:24:53.117 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T01:24:53.171 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:24:53.171 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T01:24:53.193 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T01:24:53.194 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2024-04-25T01:24:53.247 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T01:24:53.247 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:24:53.247 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T01:24:53.247 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:24:53.247 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:24:53.247 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 01:24:41.039086411 +0000 2024-04-25T01:24:53.247 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 01:22:39.973657501 +0000 2024-04-25T01:24:53.247 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 01:22:39.973657501 +0000 2024-04-25T01:24:53.247 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 01:22:39.973657501 +0000 2024-04-25T01:24:53.248 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T01:24:53.309 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T01:24:53.309 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T01:24:53.310 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000248144 s, 2.1 MB/s 2024-04-25T01:24:53.311 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T01:24:53.366 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2024-04-25T01:24:53.420 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T01:24:53.420 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:24:53.420 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T01:24:53.420 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:24:53.420 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:24:53.421 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 01:24:41.039086411 +0000 2024-04-25T01:24:53.421 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 01:22:40.322648261 +0000 2024-04-25T01:24:53.421 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 01:22:40.322648261 +0000 2024-04-25T01:24:53.421 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 01:22:40.322648261 +0000 2024-04-25T01:24:53.421 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T01:24:53.483 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T01:24:53.483 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T01:24:53.483 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000179265 s, 2.9 MB/s 2024-04-25T01:24:53.484 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T01:24:53.540 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2024-04-25T01:24:53.595 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T01:24:53.595 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:24:53.596 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-25T01:24:53.596 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:24:53.596 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:24:53.596 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 01:24:41.040086384 +0000 2024-04-25T01:24:53.596 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 01:22:40.664639206 +0000 2024-04-25T01:24:53.596 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 01:22:40.664639206 +0000 2024-04-25T01:24:53.596 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 01:22:40.664639206 +0000 2024-04-25T01:24:53.596 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T01:24:53.657 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T01:24:53.658 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T01:24:53.658 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000251629 s, 2.0 MB/s 2024-04-25T01:24:53.659 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T01:24:53.715 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2024-04-25T01:24:53.769 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T01:24:53.769 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:24:53.769 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T01:24:53.769 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:24:53.769 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:24:53.769 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 01:24:41.040086384 +0000 2024-04-25T01:24:53.769 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 01:22:40.997630390 +0000 2024-04-25T01:24:53.769 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 01:22:40.997630390 +0000 2024-04-25T01:24:53.769 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 01:22:40.997630390 +0000 2024-04-25T01:24:53.769 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T01:24:53.830 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T01:24:53.830 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T01:24:53.830 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000221852 s, 2.3 MB/s 2024-04-25T01:24:53.832 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T01:24:53.890 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T01:24:53.890 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:24:53.890 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T01:24:53.906 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T01:24:53.906 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_1 2024-04-25T01:24:53.962 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T01:24:53.963 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:24:53.963 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T01:24:53.963 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:24:53.963 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:24:53.963 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-25 01:24:43.592927040 +0000 2024-04-25T01:24:53.963 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-25 01:22:39.973679812 +0000 2024-04-25T01:24:53.963 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-25 01:22:39.973679812 +0000 2024-04-25T01:24:53.963 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-04-25 01:22:39.973679812 +0000 2024-04-25T01:24:53.963 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T01:24:54.026 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-25T01:24:54.026 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-25T01:24:54.026 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000253082 s, 2.0 MB/s 2024-04-25T01:24:54.027 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T01:24:54.084 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_2 2024-04-25T01:24:54.139 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T01:24:54.140 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:24:54.140 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T01:24:54.140 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:24:54.140 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:24:54.140 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-25 01:24:43.593927025 +0000 2024-04-25T01:24:54.140 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-25 01:22:40.484671829 +0000 2024-04-25T01:24:54.140 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-25 01:22:40.484671829 +0000 2024-04-25T01:24:54.140 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-04-25 01:22:40.484671829 +0000 2024-04-25T01:24:54.140 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T01:24:54.202 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-25T01:24:54.202 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-25T01:24:54.202 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000208028 s, 2.5 MB/s 2024-04-25T01:24:54.203 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T01:24:54.259 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_3 2024-04-25T01:24:54.315 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T01:24:54.315 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:24:54.315 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T01:24:54.315 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:24:54.315 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:24:54.315 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-25 01:24:43.593927025 +0000 2024-04-25T01:24:54.315 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-25 01:22:40.836666329 +0000 2024-04-25T01:24:54.315 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-25 01:22:40.836666329 +0000 2024-04-25T01:24:54.315 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-04-25 01:22:40.836666329 +0000 2024-04-25T01:24:54.316 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T01:24:54.376 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-25T01:24:54.377 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-25T01:24:54.377 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000213341 s, 2.4 MB/s 2024-04-25T01:24:54.378 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T01:24:54.433 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_4 2024-04-25T01:24:54.488 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T01:24:54.488 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:24:54.488 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T01:24:54.488 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:24:54.489 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:24:54.489 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-25 01:24:43.593927025 +0000 2024-04-25T01:24:54.489 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-25 01:22:41.185660877 +0000 2024-04-25T01:24:54.489 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-25 01:22:41.185660877 +0000 2024-04-25T01:24:54.489 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-04-25 01:22:41.185660877 +0000 2024-04-25T01:24:54.489 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T01:24:54.551 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-25T01:24:54.551 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-25T01:24:54.551 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000230496 s, 2.2 MB/s 2024-04-25T01:24:54.552 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T01:24:54.608 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T01:24:54.608 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi112.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi187.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T01:24:54.608 INFO:tasks.ceph:Generating config... 2024-04-25T01:24:54.609 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T01:24:54.609 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-04-25T01:24:54.610 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T01:24:54.610 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T01:24:54.610 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T01:24:54.610 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T01:24:54.610 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T01:24:54.610 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T01:24:54.610 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T01:24:54.610 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-04-25T01:24:54.610 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-04-25T01:24:54.610 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-04-25T01:24:54.611 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-04-25T01:24:54.611 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T01:24:54.611 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T01:24:54.611 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T01:24:54.611 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T01:24:54.611 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T01:24:54.611 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T01:24:54.611 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T01:24:54.611 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T01:24:54.611 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T01:24:54.611 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T01:24:54.612 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T01:24:54.612 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T01:24:54.612 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T01:24:54.612 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T01:24:54.612 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T01:24:54.612 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-25T01:24:54.612 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-25T01:24:54.612 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-25T01:24:54.612 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T01:24:54.612 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T01:24:54.613 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-25T01:24:54.613 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-25T01:24:54.613 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T01:24:54.613 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T01:24:54.613 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-25T01:24:54.613 INFO:tasks.ceph:[osd] filestore flush min = 0 2024-04-25T01:24:54.613 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T01:24:54.613 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T01:24:54.613 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T01:24:54.613 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-25T01:24:54.613 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T01:24:54.614 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-25T01:24:54.614 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T01:24:54.614 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T01:24:54.614 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T01:24:54.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T01:24:54.660 INFO:teuthology.orchestra.run.smithi112.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T01:24:54.663 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T01:24:54.706 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T01:24:54.769 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.112'), ('mon.b', '172.21.15.187'), ('mon.c', '[v2:172.21.15.187:3301,v1:172.21.15.187:6790]')] 2024-04-25T01:24:54.770 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.112,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T01:24:54.771 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:24:54.771 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T01:24:54.824 DEBUG:teuthology.orchestra.run.smithi112:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.112 --add b 172.21.15.187 --addv c '[v2:172.21.15.187:3301,v1:172.21.15.187:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:24:54.910 INFO:teuthology.orchestra.run.smithi112.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:24:54.911 INFO:teuthology.orchestra.run.smithi112.stdout:monmaptool: generated fsid a88d8313-513a-4f1d-8e1c-e81e1847e3c2 2024-04-25T01:24:54.911 INFO:teuthology.orchestra.run.smithi112.stdout:setting min_mon_release = quincy 2024-04-25T01:24:54.911 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 0 2024-04-25T01:24:54.911 INFO:teuthology.orchestra.run.smithi112.stdout:fsid a88d8313-513a-4f1d-8e1c-e81e1847e3c2 2024-04-25T01:24:54.911 INFO:teuthology.orchestra.run.smithi112.stdout:last_changed 2024-04-25T01:24:54.928910+0000 2024-04-25T01:24:54.911 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-04-25T01:24:54.928910+0000 2024-04-25T01:24:54.911 INFO:teuthology.orchestra.run.smithi112.stdout:min_mon_release 17 (quincy) 2024-04-25T01:24:54.911 INFO:teuthology.orchestra.run.smithi112.stdout:election_strategy: 1 2024-04-25T01:24:54.911 INFO:teuthology.orchestra.run.smithi112.stdout:0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-25T01:24:54.911 INFO:teuthology.orchestra.run.smithi112.stdout:1: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.b 2024-04-25T01:24:54.911 INFO:teuthology.orchestra.run.smithi112.stdout:2: [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] mon.c 2024-04-25T01:24:54.911 INFO:teuthology.orchestra.run.smithi112.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T01:24:54.912 DEBUG:teuthology.orchestra.run.smithi112:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T01:24:54.926 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID a88d8313-513a-4f1d-8e1c-e81e1847e3c2... 2024-04-25T01:24:54.927 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T01:24:54.969 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T01:24:55.006 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2024-04-25T01:24:55.006 INFO:teuthology.orchestra.run.smithi112.stdout: chdir = "" 2024-04-25T01:24:55.006 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T01:24:55.006 INFO:teuthology.orchestra.run.smithi112.stdout: auth supported = cephx 2024-04-25T01:24:55.006 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: filestore xattr use omap = true 2024-04-25T01:24:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: mon clock drift allowed = 1.000 2024-04-25T01:24:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: osd crush chooseleaf type = 0 2024-04-25T01:24:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: auth debug = true 2024-04-25T01:24:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on old message = true 2024-04-25T01:24:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on bug = true 2024-04-25T01:24:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T01:24:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: mon pg warn max object skew = 0 2024-04-25T01:24:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T01:24:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T01:24:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default size = 2 2024-04-25T01:24:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow primary affinity = true 2024-04-25T01:24:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow pg remap = true 2024-04-25T01:24:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on legacy crush tunables = false 2024-04-25T01:24:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on crush straw calc version zero = false 2024-04-25T01:24:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on no sortbitwise = false 2024-04-25T01:24:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on osd down out interval zero = false 2024-04-25T01:24:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on too few osds = false 2024-04-25T01:24:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T01:24:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T01:24:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: mon_allow_pool_size_one = true 2024-04-25T01:24:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T01:24:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: osd default data pool replay window = 5 2024-04-25T01:24:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: mon allow pool delete = true 2024-04-25T01:24:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: mon cluster log file level = debug 2024-04-25T01:24:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: debug asserts on shutdown = true 2024-04-25T01:24:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: mon health detail to clog = false 2024-04-25T01:24:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: mon host = "172.21.15.112,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2024-04-25T01:24:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = a88d8313-513a-4f1d-8e1c-e81e1847e3c2 2024-04-25T01:24:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.010 INFO:teuthology.orchestra.run.smithi112.stdout:[osd] 2024-04-25T01:24:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: osd journal size = 100 2024-04-25T01:24:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub load threshold = 5.0 2024-04-25T01:24:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub max interval = 600 2024-04-25T01:24:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: osd mclock profile = high_recovery_ops 2024-04-25T01:24:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: osd recover clone overlap = true 2024-04-25T01:24:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: osd recovery max chunk = 1048576 2024-04-25T01:24:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug shutdown = true 2024-04-25T01:24:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug op order = true 2024-04-25T01:24:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug verify stray on activate = true 2024-04-25T01:24:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug trim objects = true 2024-04-25T01:24:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: osd open classes on start = true 2024-04-25T01:24:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug pg log writeout = true 2024-04-25T01:24:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: osd deep scrub update digest min age = 30 2024-04-25T01:24:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: osd map max advance = 10 2024-04-25T01:24:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: journal zero on create = true 2024-04-25T01:24:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: filestore ondisk finisher threads = 3 2024-04-25T01:24:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: filestore apply finisher threads = 3 2024-04-25T01:24:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: bdev debug aio = true 2024-04-25T01:24:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug misdirected ops = true 2024-04-25T01:24:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: bdev async discard = True 2024-04-25T01:24:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: bdev enable discard = True 2024-04-25T01:24:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore allocator = bitmap 2024-04-25T01:24:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore block size = 96636764160 2024-04-25T01:24:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore fsck on mount = True 2024-04-25T01:24:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluefs = 1/20 2024-04-25T01:24:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluestore = 1/20 2024-04-25T01:24:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T01:24:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: debug osd = 20 2024-04-25T01:24:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: debug rocksdb = 4/10 2024-04-25T01:24:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: filestore flush min = 0 2024-04-25T01:24:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T01:24:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd full ratio = 0.9 2024-04-25T01:24:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd nearfull ratio = 0.8 2024-04-25T01:24:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: osd blocked scrub grace period = 3600 2024-04-25T01:24:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: osd failsafe full ratio = 0.95 2024-04-25T01:24:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: osd max markdown count = 1000 2024-04-25T01:24:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: osd objectstore = bluestore 2024-04-25T01:24:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2024-04-25T01:24:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.014 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr] 2024-04-25T01:24:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T01:24:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: debug mgr = 20 2024-04-25T01:24:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-04-25T01:24:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-04-25T01:24:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-04-25T01:24:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-04-25T01:24:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: mgr/telemetry/nag = false 2024-04-25T01:24:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: debug client = 20 2024-04-25T01:24:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.015 INFO:teuthology.orchestra.run.smithi112.stdout:[mon] 2024-04-25T01:24:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T01:24:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-04-25T01:24:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: debug paxos = 20 2024-04-25T01:24:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-04-25T01:24:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: mon data avail warn = 5 2024-04-25T01:24:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: mon mgr mkfs grace = 240 2024-04-25T01:24:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-04-25T01:24:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd reporter subtree level = osd 2024-04-25T01:24:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd prime pg temp = true 2024-04-25T01:24:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-04-25T01:24:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T01:24:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T01:24:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T01:24:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T01:24:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T01:24:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T01:24:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: # 1m isn't quite enough 2024-04-25T01:24:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: mon_down_mkfs_grace = 2m 2024-04-25T01:24:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_filestore_osds = false 2024-04-25T01:24:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: mon down mkfs grace = 300 2024-04-25T01:24:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: mon op complaint time = 120 2024-04-25T01:24:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:24:55.017 INFO:teuthology.orchestra.run.smithi112.stdout:[client] 2024-04-25T01:24:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: rgw cache enabled = true 2024-04-25T01:24:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable ops log = true 2024-04-25T01:24:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable usage log = true 2024-04-25T01:24:55.018 INFO:teuthology.orchestra.run.smithi112.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T01:24:55.018 INFO:teuthology.orchestra.run.smithi112.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T01:24:55.018 INFO:teuthology.orchestra.run.smithi112.stdout: client mount timeout = 600 2024-04-25T01:24:55.018 INFO:teuthology.orchestra.run.smithi112.stdout: client_shutdown_timeout = 120 2024-04-25T01:24:55.018 INFO:teuthology.orchestra.run.smithi112.stdout: debug client = 20 2024-04-25T01:24:55.018 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T01:24:55.018 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2024-04-25T01:24:55.018 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2024-04-25T01:24:55.018 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.a] 2024-04-25T01:24:55.018 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.b] 2024-04-25T01:24:55.018 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.c] 2024-04-25T01:24:55.018 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2024-04-25T01:24:55.019 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2024-04-25T01:24:55.019 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2024-04-25T01:24:55.019 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2024-04-25T01:24:55.019 INFO:teuthology.orchestra.run.smithi112.stdout:[mds] 2024-04-25T01:24:55.019 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds = 20 2024-04-25T01:24:55.019 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds balancer = 20 2024-04-25T01:24:55.019 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T01:24:55.019 INFO:teuthology.orchestra.run.smithi112.stdout: mds bal fragment size max = 10000 2024-04-25T01:24:55.019 INFO:teuthology.orchestra.run.smithi112.stdout: mds bal merge size = 5 2024-04-25T01:24:55.019 INFO:teuthology.orchestra.run.smithi112.stdout: mds bal split bits = 3 2024-04-25T01:24:55.020 INFO:teuthology.orchestra.run.smithi112.stdout: mds bal split size = 100 2024-04-25T01:24:55.020 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug frag = True 2024-04-25T01:24:55.020 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug scatterstat = True 2024-04-25T01:24:55.020 INFO:teuthology.orchestra.run.smithi112.stdout: mds op complaint time = 180 2024-04-25T01:24:55.020 INFO:teuthology.orchestra.run.smithi112.stdout: mds verify scatter = True 2024-04-25T01:24:55.020 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2024-04-25T01:24:55.020 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2024-04-25T01:24:55.020 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2024-04-25T01:24:55.021 INFO:teuthology.orchestra.run.smithi187.stdout:[global] 2024-04-25T01:24:55.022 INFO:teuthology.orchestra.run.smithi187.stdout: chdir = "" 2024-04-25T01:24:55.022 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T01:24:55.022 INFO:teuthology.orchestra.run.smithi187.stdout: auth supported = cephx 2024-04-25T01:24:55.022 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.022 INFO:teuthology.orchestra.run.smithi187.stdout: filestore xattr use omap = true 2024-04-25T01:24:55.022 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.022 INFO:teuthology.orchestra.run.smithi187.stdout: mon clock drift allowed = 1.000 2024-04-25T01:24:55.022 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.022 INFO:teuthology.orchestra.run.smithi187.stdout: osd crush chooseleaf type = 0 2024-04-25T01:24:55.022 INFO:teuthology.orchestra.run.smithi187.stdout: auth debug = true 2024-04-25T01:24:55.023 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.023 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on old message = true 2024-04-25T01:24:55.023 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on bug = true 2024-04-25T01:24:55.023 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.023 INFO:teuthology.orchestra.run.smithi187.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T01:24:55.023 INFO:teuthology.orchestra.run.smithi187.stdout: mon pg warn max object skew = 0 2024-04-25T01:24:55.023 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.023 INFO:teuthology.orchestra.run.smithi187.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T01:24:55.023 INFO:teuthology.orchestra.run.smithi187.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T01:24:55.023 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.023 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default size = 2 2024-04-25T01:24:55.024 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.024 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow primary affinity = true 2024-04-25T01:24:55.024 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow pg remap = true 2024-04-25T01:24:55.024 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on legacy crush tunables = false 2024-04-25T01:24:55.024 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on crush straw calc version zero = false 2024-04-25T01:24:55.024 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on no sortbitwise = false 2024-04-25T01:24:55.024 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on osd down out interval zero = false 2024-04-25T01:24:55.024 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on too few osds = false 2024-04-25T01:24:55.024 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T01:24:55.024 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T01:24:55.024 INFO:teuthology.orchestra.run.smithi187.stdout: mon_allow_pool_size_one = true 2024-04-25T01:24:55.024 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.025 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T01:24:55.025 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.025 INFO:teuthology.orchestra.run.smithi187.stdout: osd default data pool replay window = 5 2024-04-25T01:24:55.025 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.025 INFO:teuthology.orchestra.run.smithi187.stdout: mon allow pool delete = true 2024-04-25T01:24:55.025 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.025 INFO:teuthology.orchestra.run.smithi187.stdout: mon cluster log file level = debug 2024-04-25T01:24:55.025 INFO:teuthology.orchestra.run.smithi187.stdout: debug asserts on shutdown = true 2024-04-25T01:24:55.025 INFO:teuthology.orchestra.run.smithi187.stdout: mon health detail to clog = false 2024-04-25T01:24:55.025 INFO:teuthology.orchestra.run.smithi187.stdout: mon host = "172.21.15.112,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2024-04-25T01:24:55.025 INFO:teuthology.orchestra.run.smithi187.stdout: fsid = a88d8313-513a-4f1d-8e1c-e81e1847e3c2 2024-04-25T01:24:55.025 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.026 INFO:teuthology.orchestra.run.smithi187.stdout:[osd] 2024-04-25T01:24:55.026 INFO:teuthology.orchestra.run.smithi187.stdout: osd journal size = 100 2024-04-25T01:24:55.026 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.026 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub load threshold = 5.0 2024-04-25T01:24:55.026 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub max interval = 600 2024-04-25T01:24:55.026 INFO:teuthology.orchestra.run.smithi187.stdout: osd mclock profile = high_recovery_ops 2024-04-25T01:24:55.026 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.026 INFO:teuthology.orchestra.run.smithi187.stdout: osd recover clone overlap = true 2024-04-25T01:24:55.026 INFO:teuthology.orchestra.run.smithi187.stdout: osd recovery max chunk = 1048576 2024-04-25T01:24:55.026 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.026 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug shutdown = true 2024-04-25T01:24:55.027 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug op order = true 2024-04-25T01:24:55.027 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug verify stray on activate = true 2024-04-25T01:24:55.027 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug trim objects = true 2024-04-25T01:24:55.027 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.027 INFO:teuthology.orchestra.run.smithi187.stdout: osd open classes on start = true 2024-04-25T01:24:55.027 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug pg log writeout = true 2024-04-25T01:24:55.027 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.027 INFO:teuthology.orchestra.run.smithi187.stdout: osd deep scrub update digest min age = 30 2024-04-25T01:24:55.027 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.027 INFO:teuthology.orchestra.run.smithi187.stdout: osd map max advance = 10 2024-04-25T01:24:55.027 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.027 INFO:teuthology.orchestra.run.smithi187.stdout: journal zero on create = true 2024-04-25T01:24:55.028 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.028 INFO:teuthology.orchestra.run.smithi187.stdout: filestore ondisk finisher threads = 3 2024-04-25T01:24:55.028 INFO:teuthology.orchestra.run.smithi187.stdout: filestore apply finisher threads = 3 2024-04-25T01:24:55.028 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.028 INFO:teuthology.orchestra.run.smithi187.stdout: bdev debug aio = true 2024-04-25T01:24:55.028 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug misdirected ops = true 2024-04-25T01:24:55.028 INFO:teuthology.orchestra.run.smithi187.stdout: bdev async discard = True 2024-04-25T01:24:55.028 INFO:teuthology.orchestra.run.smithi187.stdout: bdev enable discard = True 2024-04-25T01:24:55.028 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore allocator = bitmap 2024-04-25T01:24:55.028 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore block size = 96636764160 2024-04-25T01:24:55.028 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore fsck on mount = True 2024-04-25T01:24:55.028 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluefs = 1/20 2024-04-25T01:24:55.028 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluestore = 1/20 2024-04-25T01:24:55.028 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-04-25T01:24:55.029 INFO:teuthology.orchestra.run.smithi187.stdout: debug osd = 20 2024-04-25T01:24:55.029 INFO:teuthology.orchestra.run.smithi187.stdout: debug rocksdb = 4/10 2024-04-25T01:24:55.029 INFO:teuthology.orchestra.run.smithi187.stdout: filestore flush min = 0 2024-04-25T01:24:55.029 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T01:24:55.029 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd full ratio = 0.9 2024-04-25T01:24:55.029 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd nearfull ratio = 0.8 2024-04-25T01:24:55.029 INFO:teuthology.orchestra.run.smithi187.stdout: osd blocked scrub grace period = 3600 2024-04-25T01:24:55.029 INFO:teuthology.orchestra.run.smithi187.stdout: osd failsafe full ratio = 0.95 2024-04-25T01:24:55.029 INFO:teuthology.orchestra.run.smithi187.stdout: osd max markdown count = 1000 2024-04-25T01:24:55.029 INFO:teuthology.orchestra.run.smithi187.stdout: osd objectstore = bluestore 2024-04-25T01:24:55.029 INFO:teuthology.orchestra.run.smithi187.stdout: osd op complaint time = 180 2024-04-25T01:24:55.029 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.029 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr] 2024-04-25T01:24:55.030 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-04-25T01:24:55.030 INFO:teuthology.orchestra.run.smithi187.stdout: debug mgr = 20 2024-04-25T01:24:55.030 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-04-25T01:24:55.030 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-04-25T01:24:55.030 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-04-25T01:24:55.030 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-04-25T01:24:55.030 INFO:teuthology.orchestra.run.smithi187.stdout: mgr/telemetry/nag = false 2024-04-25T01:24:55.030 INFO:teuthology.orchestra.run.smithi187.stdout: debug client = 20 2024-04-25T01:24:55.030 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.030 INFO:teuthology.orchestra.run.smithi187.stdout:[mon] 2024-04-25T01:24:55.030 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-04-25T01:24:55.030 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-04-25T01:24:55.030 INFO:teuthology.orchestra.run.smithi187.stdout: debug paxos = 20 2024-04-25T01:24:55.031 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-04-25T01:24:55.031 INFO:teuthology.orchestra.run.smithi187.stdout: mon data avail warn = 5 2024-04-25T01:24:55.031 INFO:teuthology.orchestra.run.smithi187.stdout: mon mgr mkfs grace = 240 2024-04-25T01:24:55.031 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-04-25T01:24:55.031 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd reporter subtree level = osd 2024-04-25T01:24:55.031 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd prime pg temp = true 2024-04-25T01:24:55.031 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-04-25T01:24:55.031 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.031 INFO:teuthology.orchestra.run.smithi187.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T01:24:55.031 INFO:teuthology.orchestra.run.smithi187.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T01:24:55.031 INFO:teuthology.orchestra.run.smithi187.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T01:24:55.031 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.031 INFO:teuthology.orchestra.run.smithi187.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T01:24:55.031 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T01:24:55.032 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T01:24:55.032 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.032 INFO:teuthology.orchestra.run.smithi187.stdout: # 1m isn't quite enough 2024-04-25T01:24:55.032 INFO:teuthology.orchestra.run.smithi187.stdout: mon_down_mkfs_grace = 2m 2024-04-25T01:24:55.032 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.032 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_filestore_osds = false 2024-04-25T01:24:55.032 INFO:teuthology.orchestra.run.smithi187.stdout: mon down mkfs grace = 300 2024-04-25T01:24:55.032 INFO:teuthology.orchestra.run.smithi187.stdout: mon op complaint time = 120 2024-04-25T01:24:55.032 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T01:24:55.032 INFO:teuthology.orchestra.run.smithi187.stdout:[client] 2024-04-25T01:24:55.032 INFO:teuthology.orchestra.run.smithi187.stdout: rgw cache enabled = true 2024-04-25T01:24:55.032 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable ops log = true 2024-04-25T01:24:55.032 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable usage log = true 2024-04-25T01:24:55.033 INFO:teuthology.orchestra.run.smithi187.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T01:24:55.033 INFO:teuthology.orchestra.run.smithi187.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T01:24:55.033 INFO:teuthology.orchestra.run.smithi187.stdout: client mount timeout = 600 2024-04-25T01:24:55.033 INFO:teuthology.orchestra.run.smithi187.stdout: client_shutdown_timeout = 120 2024-04-25T01:24:55.033 INFO:teuthology.orchestra.run.smithi187.stdout: debug client = 20 2024-04-25T01:24:55.033 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-04-25T01:24:55.033 INFO:teuthology.orchestra.run.smithi187.stdout: rados mon op timeout = 900 2024-04-25T01:24:55.033 INFO:teuthology.orchestra.run.smithi187.stdout: rados osd op timeout = 900 2024-04-25T01:24:55.033 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.a] 2024-04-25T01:24:55.033 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.b] 2024-04-25T01:24:55.033 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.c] 2024-04-25T01:24:55.033 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.a] 2024-04-25T01:24:55.033 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.c] 2024-04-25T01:24:55.034 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.b] 2024-04-25T01:24:55.034 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.d] 2024-04-25T01:24:55.034 INFO:teuthology.orchestra.run.smithi187.stdout:[mds] 2024-04-25T01:24:55.034 INFO:teuthology.orchestra.run.smithi187.stdout: debug mds = 20 2024-04-25T01:24:55.034 INFO:teuthology.orchestra.run.smithi187.stdout: debug mds balancer = 20 2024-04-25T01:24:55.034 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-04-25T01:24:55.034 INFO:teuthology.orchestra.run.smithi187.stdout: mds bal fragment size max = 10000 2024-04-25T01:24:55.034 INFO:teuthology.orchestra.run.smithi187.stdout: mds bal merge size = 5 2024-04-25T01:24:55.034 INFO:teuthology.orchestra.run.smithi187.stdout: mds bal split bits = 3 2024-04-25T01:24:55.034 INFO:teuthology.orchestra.run.smithi187.stdout: mds bal split size = 100 2024-04-25T01:24:55.034 INFO:teuthology.orchestra.run.smithi187.stdout: mds debug frag = True 2024-04-25T01:24:55.034 INFO:teuthology.orchestra.run.smithi187.stdout: mds debug scatterstat = True 2024-04-25T01:24:55.034 INFO:teuthology.orchestra.run.smithi187.stdout: mds op complaint time = 180 2024-04-25T01:24:55.035 INFO:teuthology.orchestra.run.smithi187.stdout: mds verify scatter = True 2024-04-25T01:24:55.035 INFO:teuthology.orchestra.run.smithi187.stdout: osd op complaint time = 180 2024-04-25T01:24:55.035 INFO:teuthology.orchestra.run.smithi187.stdout: rados mon op timeout = 900 2024-04-25T01:24:55.035 INFO:teuthology.orchestra.run.smithi187.stdout: rados osd op timeout = 900 2024-04-25T01:24:55.035 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T01:24:55.035 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T01:24:55.114 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T01:24:55.114 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:24:55.114 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T01:24:55.129 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:24:55.129 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T01:24:55.184 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T01:24:55.184 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:24:55.185 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T01:24:55.185 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T01:24:55.256 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:24:55.256 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:24:55.310 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T01:24:55.310 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:24:55.310 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T01:24:55.310 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T01:24:55.343 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:24:55.343 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:24:55.397 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T01:24:55.398 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T01:24:55.398 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T01:24:55.446 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T01:24:55.448 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T01:24:55.507 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T01:24:55.509 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T01:24:55.509 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T01:24:55.558 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T01:24:55.560 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T01:24:55.583 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T01:24:55.671 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T01:24:55.674 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T01:24:55.696 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T01:24:55.744 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T01:24:55.747 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T01:24:55.770 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T01:24:55.857 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T01:24:55.860 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T01:24:55.882 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T01:24:55.883 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T01:24:55.921 INFO:teuthology.orchestra.run.smithi112.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T01:24:55.933 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T01:24:55.971 INFO:teuthology.orchestra.run.smithi187.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T01:24:55.983 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T01:24:55.983 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi112.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi187.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T01:24:55.983 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T01:24:56.006 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T01:24:56.006 INFO:tasks.ceph:role: osd.0 2024-04-25T01:24:56.006 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T01:24:56.006 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T01:24:56.077 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:24:56.077 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:24:56.077 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:24:56.077 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T01:24:56.077 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:24:56.077 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:24:56.077 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:24:56.077 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T01:24:56.077 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:24:56.077 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:24:56.601 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2024-04-25T01:24:56.603 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2024-04-25T01:24:56.603 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T01:24:56.708 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T01:24:56.738 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T01:24:56.802 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T01:24:56.802 INFO:tasks.ceph:role: osd.1 2024-04-25T01:24:56.802 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T01:24:56.803 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T01:24:56.873 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:24:56.873 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:24:56.873 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:24:56.873 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T01:24:56.873 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:24:56.873 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:24:56.874 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:24:56.874 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T01:24:56.874 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:24:56.874 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:24:57.337 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2024-04-25T01:24:57.339 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2024-04-25T01:24:57.339 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T01:24:57.431 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T01:24:57.458 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T01:24:57.522 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T01:24:57.522 INFO:tasks.ceph:role: osd.2 2024-04-25T01:24:57.522 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T01:24:57.522 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T01:24:57.593 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:24:57.594 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:24:57.594 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:24:57.594 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T01:24:57.594 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:24:57.594 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:24:57.594 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:24:57.594 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T01:24:57.594 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:24:57.594 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:24:58.077 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2024-04-25T01:24:58.079 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2024-04-25T01:24:58.080 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T01:24:58.203 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T01:24:58.231 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T01:24:58.294 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T01:24:58.295 INFO:tasks.ceph:role: osd.3 2024-04-25T01:24:58.295 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T01:24:58.295 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T01:24:58.366 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:24:58.366 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:24:58.366 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:24:58.366 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T01:24:58.366 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:24:58.366 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:24:58.366 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:24:58.366 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T01:24:58.366 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:24:58.367 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:24:58.885 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2024-04-25T01:24:58.886 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2024-04-25T01:24:58.887 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T01:24:59.020 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T01:24:59.051 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:24:59.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:24:59.165+0000 7f446094e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T01:24:59.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:24:59.166+0000 7f446094e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T01:24:59.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:24:59.166+0000 7f446094e640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T01:24:59.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:24:59.166+0000 7f446094e640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T01:24:59.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:24:59.166+0000 7f446094e640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T01:24:59.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:24:59.166+0000 7f446094e640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T01:25:02.010 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T01:25:02.038 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:25:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:02.148+0000 7f9f54413640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T01:25:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:02.148+0000 7f9f54413640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T01:25:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:02.148+0000 7f9f54413640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T01:25:02.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:02.148+0000 7f9f54413640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T01:25:02.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:02.148+0000 7f9f54413640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T01:25:02.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:02.149+0000 7f9f54413640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T01:25:04.991 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T01:25:05.018 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:25:05.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:05.126+0000 7f18a6425640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T01:25:05.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:05.126+0000 7f18a6425640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T01:25:05.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:05.126+0000 7f18a6425640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T01:25:05.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:05.126+0000 7f18a6425640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T01:25:05.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:05.126+0000 7f18a6425640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T01:25:05.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:05.126+0000 7f18a6425640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T01:25:07.955 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T01:25:07.983 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:25:08.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:08.090+0000 7f4ac400f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T01:25:08.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:08.090+0000 7f4ac400f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T01:25:08.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:08.090+0000 7f4ac400f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T01:25:08.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:08.090+0000 7f4ac400f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T01:25:08.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:08.090+0000 7f4ac400f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T01:25:08.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:08.091+0000 7f4ac400f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T01:25:10.919 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T01:25:10.946 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T01:25:10.975 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T01:25:10.976 INFO:tasks.ceph:role: osd.4 2024-04-25T01:25:10.976 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T01:25:10.976 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T01:25:11.050 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:25:11.051 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:25:11.051 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:25:11.051 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T01:25:11.051 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:25:11.051 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:25:11.051 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:25:11.051 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T01:25:11.051 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:25:11.051 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:25:11.548 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-04-25T01:25:11.550 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-04-25T01:25:11.550 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T01:25:11.674 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T01:25:11.705 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T01:25:11.769 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T01:25:11.769 INFO:tasks.ceph:role: osd.5 2024-04-25T01:25:11.769 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T01:25:11.769 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T01:25:11.840 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:25:11.841 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:25:11.841 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:25:11.841 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T01:25:11.841 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:25:11.841 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:25:11.841 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:25:11.841 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T01:25:11.841 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:25:11.841 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:25:12.299 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-04-25T01:25:12.301 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-04-25T01:25:12.301 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T01:25:12.390 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T01:25:12.417 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T01:25:12.481 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T01:25:12.481 INFO:tasks.ceph:role: osd.6 2024-04-25T01:25:12.481 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T01:25:12.481 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T01:25:12.551 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:25:12.551 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:25:12.551 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:25:12.551 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T01:25:12.551 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:25:12.551 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:25:12.551 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:25:12.551 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T01:25:12.551 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:25:12.551 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:25:13.050 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-04-25T01:25:13.052 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-04-25T01:25:13.052 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T01:25:13.207 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T01:25:13.235 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T01:25:13.299 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T01:25:13.299 INFO:tasks.ceph:role: osd.7 2024-04-25T01:25:13.299 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T01:25:13.299 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T01:25:13.369 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:25:13.369 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:25:13.369 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:25:13.369 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T01:25:13.370 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:25:13.370 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:25:13.370 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:25:13.370 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T01:25:13.370 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:25:13.370 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:25:13.870 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-04-25T01:25:13.872 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-04-25T01:25:13.873 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T01:25:14.006 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T01:25:14.035 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:25:14.127 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:14.147+0000 7f220fae1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T01:25:14.127 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:14.147+0000 7f220fae1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T01:25:14.127 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:14.147+0000 7f220fae1640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T01:25:14.127 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:14.147+0000 7f220fae1640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T01:25:14.127 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:14.147+0000 7f220fae1640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T01:25:14.128 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:14.148+0000 7f220fae1640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T01:25:16.982 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T01:25:17.010 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:25:17.101 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:17.121+0000 7f1b1789d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T01:25:17.101 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:17.121+0000 7f1b1789d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T01:25:17.101 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:17.121+0000 7f1b1789d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T01:25:17.101 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:17.121+0000 7f1b1789d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T01:25:17.101 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:17.121+0000 7f1b1789d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T01:25:17.101 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:17.122+0000 7f1b1789d640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T01:25:19.968 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T01:25:19.996 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:25:20.085 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:20.104+0000 7f65e120e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T01:25:20.085 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:20.105+0000 7f65e120e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T01:25:20.085 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:20.105+0000 7f65e120e640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T01:25:20.085 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:20.105+0000 7f65e120e640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T01:25:20.085 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:20.105+0000 7f65e120e640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T01:25:20.085 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:20.105+0000 7f65e120e640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T01:25:22.934 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T01:25:22.960 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:25:23.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:23.066+0000 7fe0a7bc0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T01:25:23.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:23.066+0000 7fe0a7bc0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T01:25:23.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:23.066+0000 7fe0a7bc0640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T01:25:23.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:23.066+0000 7fe0a7bc0640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T01:25:23.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:23.066+0000 7fe0a7bc0640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T01:25:23.047 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:23.067+0000 7fe0a7bc0640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T01:25:23.058 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:25:23.061 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:25:25.900 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T01:25:25.927 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T01:25:25.927 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:25:25.927 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T01:25:25.956 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:25:25.956 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T01:25:26.024 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:25:26.024 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T01:25:26.092 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:25:26.092 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T01:25:26.158 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:25:26.159 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T01:25:26.225 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:25:26.225 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T01:25:26.290 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:25:26.290 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T01:25:26.354 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:25:26.355 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T01:25:26.383 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:25:26.383 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T01:25:26.451 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:25:26.451 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T01:25:26.518 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:25:26.518 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T01:25:26.585 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:25:26.585 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T01:25:26.650 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:25:26.650 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T01:25:26.714 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:25:26.714 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T01:25:26.778 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:25:26.779 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T01:25:26.794 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:25:26.794 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T01:25:26.833 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T01:25:26.834 DEBUG:teuthology.orchestra.run.smithi112:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T01:25:26.837 DEBUG:teuthology.orchestra.run.smithi187:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T01:25:26.877 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr.y] 2024-04-25T01:25:26.877 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDnsClmh9rVGxAAa1LenWUXxSwKk/VIDzA0Hw== 2024-04-25T01:25:26.877 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2024-04-25T01:25:26.877 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDnsClmXHyHIhAAVT/HSghkvW3+Kf4yvDxliw== 2024-04-25T01:25:26.877 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2024-04-25T01:25:26.877 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDnsClmnbVIKRAAO4HxVcrfEg98lUqA9OQTGA== 2024-04-25T01:25:26.877 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.0] 2024-04-25T01:25:26.877 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDrsClmYcvxCRAATTJI+duXSDNe9w6SwTuRIw== 2024-04-25T01:25:26.877 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.1] 2024-04-25T01:25:26.878 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDusClmpjXmCBAAEqyRHL09nfwkg/MkxwGAZA== 2024-04-25T01:25:26.878 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.2] 2024-04-25T01:25:26.878 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDxsClmhuaRBxAAngzxFoHNNpAzEOnhat6YEQ== 2024-04-25T01:25:26.878 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.3] 2024-04-25T01:25:26.878 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQD0sClm0VBxBRAAtKkG+gT/jKuV7G36PnYxHg== 2024-04-25T01:25:26.878 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr.x] 2024-04-25T01:25:26.878 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDnsClm6qx6HxAAtAZBa/g8YO3FVRG2ZRWUPg== 2024-04-25T01:25:26.878 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2024-04-25T01:25:26.878 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDnsClmXxSiLRAAIpVBrLleXMNdEIivtKhHRA== 2024-04-25T01:25:26.878 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2024-04-25T01:25:26.878 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDnsClmbfZfNBAARjQEUKGogu4lmETOfAvAGw== 2024-04-25T01:25:26.878 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.4] 2024-04-25T01:25:26.879 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQD6sClmW3PUCBAAfeUrlypwwiuNfqfja06ctw== 2024-04-25T01:25:26.879 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.5] 2024-04-25T01:25:26.879 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQD9sClmsk9FBxAALy6ys98DehgZouBAg9k/uw== 2024-04-25T01:25:26.879 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.6] 2024-04-25T01:25:26.879 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQAAsSlmM7ZLBhAAYCeOoMewz2Qhke9ovp4T8Q== 2024-04-25T01:25:26.879 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.7] 2024-04-25T01:25:26.879 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQADsSlm21wBBBAAAwmxsG000h3zg12FYoLnIw== 2024-04-25T01:25:26.879 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-04-25T01:25:26.879 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDnsClmdd4rOBAAehIwgzoBvUF4wFUfMOJ6jQ== 2024-04-25T01:25:26.879 INFO:teuthology.orchestra.run.smithi112.stdout:[client.1] 2024-04-25T01:25:26.879 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDnsClmMbcrOxAArJSJvyMDDmSa5Vq0AD7/0Q== 2024-04-25T01:25:26.895 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr.y] 2024-04-25T01:25:26.895 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDnsClmh9rVGxAAa1LenWUXxSwKk/VIDzA0Hw== 2024-04-25T01:25:26.895 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.a] 2024-04-25T01:25:26.895 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDnsClmXHyHIhAAVT/HSghkvW3+Kf4yvDxliw== 2024-04-25T01:25:26.896 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.c] 2024-04-25T01:25:26.896 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDnsClmnbVIKRAAO4HxVcrfEg98lUqA9OQTGA== 2024-04-25T01:25:26.896 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.0] 2024-04-25T01:25:26.896 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDrsClmYcvxCRAATTJI+duXSDNe9w6SwTuRIw== 2024-04-25T01:25:26.896 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.1] 2024-04-25T01:25:26.896 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDusClmpjXmCBAAEqyRHL09nfwkg/MkxwGAZA== 2024-04-25T01:25:26.896 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.2] 2024-04-25T01:25:26.896 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDxsClmhuaRBxAAngzxFoHNNpAzEOnhat6YEQ== 2024-04-25T01:25:26.896 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.3] 2024-04-25T01:25:26.896 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQD0sClm0VBxBRAAtKkG+gT/jKuV7G36PnYxHg== 2024-04-25T01:25:26.896 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr.x] 2024-04-25T01:25:26.896 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDnsClm6qx6HxAAtAZBa/g8YO3FVRG2ZRWUPg== 2024-04-25T01:25:26.896 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.b] 2024-04-25T01:25:26.897 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDnsClmXxSiLRAAIpVBrLleXMNdEIivtKhHRA== 2024-04-25T01:25:26.897 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.d] 2024-04-25T01:25:26.897 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDnsClmbfZfNBAARjQEUKGogu4lmETOfAvAGw== 2024-04-25T01:25:26.897 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.4] 2024-04-25T01:25:26.897 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQD6sClmW3PUCBAAfeUrlypwwiuNfqfja06ctw== 2024-04-25T01:25:26.897 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.5] 2024-04-25T01:25:26.897 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQD9sClmsk9FBxAALy6ys98DehgZouBAg9k/uw== 2024-04-25T01:25:26.897 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.6] 2024-04-25T01:25:26.897 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQAAsSlmM7ZLBhAAYCeOoMewz2Qhke9ovp4T8Q== 2024-04-25T01:25:26.897 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.7] 2024-04-25T01:25:26.897 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQADsSlm21wBBBAAAwmxsG000h3zg12FYoLnIw== 2024-04-25T01:25:26.897 INFO:teuthology.orchestra.run.smithi187.stdout:[client.0] 2024-04-25T01:25:26.897 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDnsClmdd4rOBAAehIwgzoBvUF4wFUfMOJ6jQ== 2024-04-25T01:25:26.897 INFO:teuthology.orchestra.run.smithi187.stdout:[client.1] 2024-04-25T01:25:26.898 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDnsClmMbcrOxAArJSJvyMDDmSa5Vq0AD7/0Q== 2024-04-25T01:25:26.898 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T01:25:26.923 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T01:25:26.987 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:25:26.989 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:25:27.040 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:25:27.043 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:25:27.093 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:25:27.096 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:25:27.144 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:25:27.146 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:25:27.195 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:25:27.197 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:25:27.245 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:25:27.247 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:25:27.295 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T01:25:27.298 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T01:25:27.346 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:25:27.348 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:25:27.399 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:25:27.401 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:25:27.452 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:25:27.454 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:25:27.502 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:25:27.505 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:25:27.553 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:25:27.555 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:25:27.604 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:25:27.607 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:25:27.655 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T01:25:27.658 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T01:25:27.709 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T01:25:27.712 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T01:25:27.763 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T01:25:27.763 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T01:25:27.786 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T01:25:27.879 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T01:25:27.902 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T01:25:27.925 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T01:25:28.017 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T01:25:28.040 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T01:25:28.104 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T01:25:28.196 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T01:25:28.220 DEBUG:teuthology.orchestra.run.smithi112:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:25:28.222 DEBUG:teuthology.orchestra.run.smithi187:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:25:28.274 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T01:25:28.274 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T01:25:28.274 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T01:25:28.276 INFO:tasks.ceph.mon.a:Started 2024-04-25T01:25:28.276 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T01:25:28.277 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T01:25:28.316 INFO:tasks.ceph.mon.b:Started 2024-04-25T01:25:28.316 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T01:25:28.316 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T01:25:28.318 INFO:tasks.ceph.mon.c:Started 2024-04-25T01:25:28.318 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T01:25:28.318 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T01:25:28.318 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T01:25:28.320 INFO:tasks.ceph.mgr.y:Started 2024-04-25T01:25:28.320 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T01:25:28.321 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T01:25:28.323 INFO:tasks.ceph.mgr.x:Started 2024-04-25T01:25:28.323 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T01:25:28.323 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T01:25:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.439+0000 7f42f8c10640 1 Processor -- start 2024-04-25T01:25:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.439+0000 7f42f8c10640 1 -- start start 2024-04-25T01:25:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.440+0000 7f42f8c10640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42f40c7ab0 0x7f42f40c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.440+0000 7f42f8c10640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f42f40c5a60 0x7f42f406d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.440+0000 7f42f8c10640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42f4156850 con 0x7f42f41563c0 2024-04-25T01:25:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.440+0000 7f42f8c10640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f42f40c5ea0 con 0x7f42f40c5a60 2024-04-25T01:25:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.440+0000 7f42f8c10640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f42f40c6160 con 0x7f42f40c7ab0 2024-04-25T01:25:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.440+0000 7f42f2d9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f42f41563c0 0x7f42f40c91a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:41454/0 (socket says 172.21.15.112:41454) 2024-04-25T01:25:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.440+0000 7f42f2d9d640 1 -- 172.21.15.112:0/3820660840 learned_addr learned my addr 172.21.15.112:0/3820660840 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:25:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.440+0000 7f42f259c640 1 --2- 172.21.15.112:0/3820660840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42f40c7ab0 0x7f42f40c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.440+0000 7f42f1d9b640 1 --2- 172.21.15.112:0/3820660840 >> v2:172.21.15.187:3300/0 conn(0x7f42f40c5a60 0x7f42f406d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.440+0000 7f42f259c640 1 -- 172.21.15.112:0/3820660840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42f40c7ab0 msgr2=0x7f42f40c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:25:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.440+0000 7f42f259c640 1 -- 172.21.15.112:0/3820660840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42f40c7ab0 msgr2=0x7f42f40c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:25:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.440+0000 7f42f259c640 1 --2- 172.21.15.112:0/3820660840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42f40c7ab0 0x7f42f40c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:25:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.440+0000 7f42f1d9b640 1 -- 172.21.15.112:0/3820660840 >> v2:172.21.15.187:3300/0 conn(0x7f42f40c5a60 msgr2=0x7f42f406d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:25:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.440+0000 7f42f1d9b640 1 -- 172.21.15.112:0/3820660840 >> v2:172.21.15.187:3300/0 conn(0x7f42f40c5a60 msgr2=0x7f42f406d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:25:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.440+0000 7f42f1d9b640 1 --2- 172.21.15.112:0/3820660840 >> v2:172.21.15.187:3300/0 conn(0x7f42f40c5a60 0x7f42f406d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:25:28.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.440+0000 7f42f259c640 1 --2- 172.21.15.112:0/3820660840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42f40c7ab0 0x7f42f40c7e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:25:28.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.441+0000 7f42f1d9b640 1 --2- 172.21.15.112:0/3820660840 >> v2:172.21.15.187:3300/0 conn(0x7f42f40c5a60 0x7f42f406d680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:25:28.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.598+0000 7f42f159a640 1 -- 172.21.15.112:0/3820660840 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 140288076 0 0) 0x7f42f4156850 con 0x7f42f41563c0 2024-04-25T01:25:28.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.598+0000 7f42f159a640 1 -- 172.21.15.112:0/3820660840 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42dc003520 con 0x7f42f41563c0 2024-04-25T01:25:28.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.598+0000 7f42f159a640 1 -- 172.21.15.112:0/3820660840 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3882954677 0 0) 0x7f42dc003520 con 0x7f42f41563c0 2024-04-25T01:25:28.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.598+0000 7f42f159a640 1 -- 172.21.15.112:0/3820660840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42f40c7ab0 msgr2=0x7f42f40c7e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:25:28.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.598+0000 7f42f159a640 1 --2- 172.21.15.112:0/3820660840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42f40c7ab0 0x7f42f40c7e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:28.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.598+0000 7f42f159a640 1 -- 172.21.15.112:0/3820660840 >> v2:172.21.15.187:3300/0 conn(0x7f42f40c5a60 msgr2=0x7f42f406d680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:25:28.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.598+0000 7f42f159a640 1 --2- 172.21.15.112:0/3820660840 >> v2:172.21.15.187:3300/0 conn(0x7f42f40c5a60 0x7f42f406d680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:28.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.598+0000 7f42f159a640 1 -- 172.21.15.112:0/3820660840 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f406cba0 con 0x7f42f41563c0 2024-04-25T01:25:28.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.598+0000 7f42f159a640 1 -- 172.21.15.112:0/3820660840 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f42ec0019f0 con 0x7f42f41563c0 2024-04-25T01:25:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.598+0000 7f42f159a640 1 -- 172.21.15.112:0/3820660840 >> v1:172.21.15.112:6789/0 conn(0x7f42f41563c0 legacy=0x7f42f40c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.599+0000 7f42f159a640 1 --2- 172.21.15.112:0/3820660840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42f40c7ab0 0x7f42dc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.599+0000 7f42f159a640 1 --2- 172.21.15.112:0/3820660840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42dc003fe0 0x7f42dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.599+0000 7f42f159a640 1 --2- 172.21.15.112:0/3820660840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42dc004d40 0x7f42dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.599+0000 7f42f159a640 1 -- 172.21.15.112:0/3820660840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f42f40c5ea0 con 0x7f42dc004d40 2024-04-25T01:25:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.599+0000 7f42f159a640 1 -- 172.21.15.112:0/3820660840 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f42f40c6160 con 0x7f42dc003fe0 2024-04-25T01:25:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.599+0000 7f42f159a640 1 -- 172.21.15.112:0/3820660840 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f42dc00bac0 con 0x7f42f40c7ab0 2024-04-25T01:25:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.599+0000 7f42f2d9d640 1 --2- 172.21.15.112:0/3820660840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42dc004d40 0x7f42dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.599+0000 7f42f1d9b640 1 --2- 172.21.15.112:0/3820660840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42dc003fe0 0x7f42dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.599+0000 7f42f259c640 1 --2- 172.21.15.112:0/3820660840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42f40c7ab0 0x7f42dc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.599+0000 7f42f2d9d640 1 -- 172.21.15.112:0/3820660840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42f40c7ab0 msgr2=0x7f42dc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.599+0000 7f42f2d9d640 1 --2- 172.21.15.112:0/3820660840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42f40c7ab0 0x7f42dc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.599+0000 7f42f2d9d640 1 -- 172.21.15.112:0/3820660840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42dc003fe0 msgr2=0x7f42dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.599+0000 7f42f2d9d640 1 --2- 172.21.15.112:0/3820660840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42dc003fe0 0x7f42dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.599+0000 7f42f2d9d640 1 -- 172.21.15.112:0/3820660840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42dc003520 con 0x7f42dc004d40 2024-04-25T01:25:28.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.600+0000 7f42f2d9d640 1 --2- 172.21.15.112:0/3820660840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42dc004d40 0x7f42dc0051c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0013d0 tx=0x7f42dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f402ab7be689d246 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:28.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.600+0000 7f42f159a640 1 -- 172.21.15.112:0/3820660840 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec00a4e0 con 0x7f42dc004d40 2024-04-25T01:25:28.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.600+0000 7f42f159a640 1 -- 172.21.15.112:0/3820660840 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42ec02f070 con 0x7f42dc004d40 2024-04-25T01:25:28.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.600+0000 7f42f159a640 1 -- 172.21.15.112:0/3820660840 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec02bce0 con 0x7f42dc004d40 2024-04-25T01:25:28.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.601+0000 7f42f8c10640 1 -- 172.21.15.112:0/3820660840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42dc004d40 msgr2=0x7f42dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:28.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.601+0000 7f42f8c10640 1 --2- 172.21.15.112:0/3820660840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42dc004d40 0x7f42dc0051c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0013d0 tx=0x7f42dc005830 comp rx=0 tx=0).stop 2024-04-25T01:25:28.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.601+0000 7f42f2d9d640 1 -- 172.21.15.112:0/3820660840 reap_dead start 2024-04-25T01:25:28.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.601+0000 7f42f8c10640 1 -- 172.21.15.112:0/3820660840 shutdown_connections 2024-04-25T01:25:28.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.601+0000 7f42f8c10640 1 -- 172.21.15.112:0/3820660840 >> 172.21.15.112:0/3820660840 conn(0x7f42f40cb0e0 msgr2=0x7f42f40cb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:28.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.601+0000 7f42f8c10640 1 -- 172.21.15.112:0/3820660840 shutdown_connections 2024-04-25T01:25:28.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.601+0000 7f42f8c10640 1 -- 172.21.15.112:0/3820660840 wait complete. 2024-04-25T01:25:28.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.603+0000 7f42f8c10640 1 Processor -- start 2024-04-25T01:25:28.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.603+0000 7f42f8c10640 1 -- start start 2024-04-25T01:25:28.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.604+0000 7f42f8c10640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42dc004d40 0x7f42f4154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:28.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.604+0000 7f42f8c10640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42f4156cd0 0x7f42f41ef170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:28.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.604+0000 7f42f8c10640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42f41ef6b0 0x7f42f40c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:28.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.604+0000 7f42f8c10640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f42f41532a0 con 0x7f42f4156cd0 2024-04-25T01:25:28.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.604+0000 7f42f8c10640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f42f406d160 con 0x7f42f41ef6b0 2024-04-25T01:25:28.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.604+0000 7f42f8c10640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f42f40c6440 con 0x7f42dc004d40 2024-04-25T01:25:28.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.604+0000 7f42f1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42f4156cd0 0x7f42f41ef170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:28.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.604+0000 7f42f1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42f4156cd0 0x7f42f41ef170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37472/0 (socket says 172.21.15.112:37472) 2024-04-25T01:25:28.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.604+0000 7f42f1d9b640 1 -- 172.21.15.112:0/3735183441 learned_addr learned my addr 172.21.15.112:0/3735183441 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:28.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.604+0000 7f42f259c640 1 --2- 172.21.15.112:0/3735183441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42dc004d40 0x7f42f4154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:28.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.604+0000 7f42f2d9d640 1 --2- 172.21.15.112:0/3735183441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42f41ef6b0 0x7f42f40c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:28.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.604+0000 7f42f1d9b640 1 -- 172.21.15.112:0/3735183441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42dc004d40 msgr2=0x7f42f4154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:28.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.604+0000 7f42f1d9b640 1 --2- 172.21.15.112:0/3735183441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42dc004d40 0x7f42f4154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:28.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.604+0000 7f42f1d9b640 1 -- 172.21.15.112:0/3735183441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42f41ef6b0 msgr2=0x7f42f40c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:28.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.604+0000 7f42f1d9b640 1 --2- 172.21.15.112:0/3735183441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42f41ef6b0 0x7f42f40c21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:28.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.604+0000 7f42f1d9b640 1 -- 172.21.15.112:0/3735183441 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f40c41e0 con 0x7f42f4156cd0 2024-04-25T01:25:28.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.604+0000 7f42f1d9b640 1 --2- 172.21.15.112:0/3735183441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42f4156cd0 0x7f42f41ef170 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f42e40108d0 tx=0x7f42e4010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:28.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.605+0000 7f42e2ffd640 1 -- 172.21.15.112:0/3735183441 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e401a020 con 0x7f42f4156cd0 2024-04-25T01:25:28.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.605+0000 7f42e2ffd640 1 -- 172.21.15.112:0/3735183441 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e400e9c0 con 0x7f42f4156cd0 2024-04-25T01:25:28.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.605+0000 7f42e2ffd640 1 -- 172.21.15.112:0/3735183441 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e400ed90 con 0x7f42f4156cd0 2024-04-25T01:25:28.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.605+0000 7f42f8c10640 1 -- 172.21.15.112:0/3735183441 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f40c1590 con 0x7f42f4156cd0 2024-04-25T01:25:28.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.605+0000 7f42f8c10640 1 -- 172.21.15.112:0/3735183441 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f4155210 con 0x7f42f4156cd0 2024-04-25T01:25:28.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.605+0000 7f42e2ffd640 1 -- 172.21.15.112:0/3735183441 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f42e401d020 con 0x7f42f4156cd0 2024-04-25T01:25:28.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.605+0000 7f42e2ffd640 1 -- 172.21.15.112:0/3735183441 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f42e40149a0 con 0x7f42f4156cd0 2024-04-25T01:25:28.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.606+0000 7f42f8c10640 1 -- 172.21.15.112:0/3735183441 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f4068b90 con 0x7f42f4156cd0 2024-04-25T01:25:28.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.608+0000 7f42e2ffd640 1 -- 172.21.15.112:0/3735183441 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f42f4068b90 con 0x7f42f4156cd0 2024-04-25T01:25:28.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.672+0000 7f42f8c10640 1 -- 172.21.15.112:0/3735183441 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f42f41eead0 con 0x7f42f4156cd0 2024-04-25T01:25:28.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.674+0000 7f42e2ffd640 1 -- 172.21.15.112:0/3735183441 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f42e4024020 con 0x7f42f4156cd0 2024-04-25T01:25:28.654 INFO:teuthology.orchestra.run.smithi112.stderr:adjusted tunables profile to default 2024-04-25T01:25:28.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.675+0000 7f42f8c10640 1 -- 172.21.15.112:0/3735183441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42f4156cd0 msgr2=0x7f42f41ef170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:28.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.675+0000 7f42f8c10640 1 --2- 172.21.15.112:0/3735183441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42f4156cd0 0x7f42f41ef170 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f42e40108d0 tx=0x7f42e4010d90 comp rx=0 tx=0).stop 2024-04-25T01:25:28.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.675+0000 7f42f8c10640 1 -- 172.21.15.112:0/3735183441 shutdown_connections 2024-04-25T01:25:28.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.675+0000 7f42f8c10640 1 --2- 172.21.15.112:0/3735183441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42f41ef6b0 0x7f42f40c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:28.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.675+0000 7f42f8c10640 1 --2- 172.21.15.112:0/3735183441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42f4156cd0 0x7f42f41ef170 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:28.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.675+0000 7f42f8c10640 1 --2- 172.21.15.112:0/3735183441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42dc004d40 0x7f42f4154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:28.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.675+0000 7f42f8c10640 1 -- 172.21.15.112:0/3735183441 >> 172.21.15.112:0/3735183441 conn(0x7f42f40cb0e0 msgr2=0x7f42f414dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:28.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.675+0000 7f42f8c10640 1 -- 172.21.15.112:0/3735183441 shutdown_connections 2024-04-25T01:25:28.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:28.675+0000 7f42f8c10640 1 -- 172.21.15.112:0/3735183441 wait complete. 2024-04-25T01:25:28.666 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T01:25:28.666 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T01:25:28.666 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:25:28.666 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T01:25:28.689 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:25:28.690 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T01:25:28.692 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:28.712+0000 7fb40300e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T01:25:28.700 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:28.721+0000 7f6756c0d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T01:25:28.714 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:25:28.714 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T01:25:28.776 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:25:28.776 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T01:25:28.840 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:25:28.840 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T01:25:28.842 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:28.862+0000 7f6756c0d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:25:28.844 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:28.865+0000 7fb40300e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:25:28.863 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:25:28.863 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T01:25:28.898 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:28.918+0000 7f6756c0d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T01:25:28.899 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:28.919+0000 7fb40300e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T01:25:28.921 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:25:28.921 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T01:25:28.985 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:25:28.985 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T01:25:29.048 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 687b49ac-8bbb-4826-99c3-007b21a72cff 0 2024-04-25T01:25:29.135 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:29.156+0000 7fb40300e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T01:25:29.146 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:29.166+0000 7f6756c0d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T01:25:29.181 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:29.201+0000 7fb40300e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T01:25:29.186 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.206+0000 7f81e1c31640 1 Processor -- start 2024-04-25T01:25:29.186 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.206+0000 7f81e1c31640 1 -- start start 2024-04-25T01:25:29.187 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.207+0000 7f81e1c31640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f81dc0add20 0x7f81dc0c4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.187 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.207+0000 7f81e1c31640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81dc155da0 0x7f81dc1561e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.187 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.207+0000 7f81e1c31640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81dc06dc20 con 0x7f81dc14fa40 2024-04-25T01:25:29.187 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.207+0000 7f81e1c31640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f81dc06d9d0 con 0x7f81dc0add20 2024-04-25T01:25:29.187 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.207+0000 7f81e1c31640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f81dc06e210 con 0x7f81dc155da0 2024-04-25T01:25:29.187 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.207+0000 7f81d2ffd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f81dc0add20 0x7f81dc0c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.188 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.207+0000 7f81dbfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81dc155da0 0x7f81dc1561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.188 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.207+0000 7f81db7fe640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f81dc14fa40 0x7f81dc06e830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:52354/0 (socket says 172.21.15.187:52354) 2024-04-25T01:25:29.188 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.207+0000 7f81db7fe640 1 -- 172.21.15.187:0/1788943914 learned_addr learned my addr 172.21.15.187:0/1788943914 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-04-25T01:25:29.188 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.207+0000 7f81daffd640 1 -- 172.21.15.187:0/1788943914 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 436477949 0 0) 0x7f81dc06dc20 con 0x7f81dc14fa40 2024-04-25T01:25:29.188 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.207+0000 7f81dbfff640 1 -- 172.21.15.187:0/1788943914 >> v2:172.21.15.187:3300/0 conn(0x7f81dc0add20 msgr2=0x7f81dc0c4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:29.188 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.207+0000 7f81dbfff640 1 --2- 172.21.15.187:0/1788943914 >> v2:172.21.15.187:3300/0 conn(0x7f81dc0add20 0x7f81dc0c4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.188 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.207+0000 7f81dbfff640 1 -- 172.21.15.187:0/1788943914 >> v1:172.21.15.187:6789/0 conn(0x7f81dc14fa40 legacy=0x7f81dc06e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.188 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.207+0000 7f81dbfff640 1 -- 172.21.15.187:0/1788943914 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc06e4f0 con 0x7f81dc155da0 2024-04-25T01:25:29.188 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.207+0000 7f81d2ffd640 1 --2- 172.21.15.187:0/1788943914 >> v2:172.21.15.187:3300/0 conn(0x7f81dc0add20 0x7f81dc0c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:25:29.190 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.208+0000 7f81dbfff640 1 --2- 172.21.15.187:0/1788943914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81dc155da0 0x7f81dc1561e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009d90 tx=0x7f81d402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b03455ddb73814f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:29.190 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.208+0000 7f81daffd640 1 -- 172.21.15.187:0/1788943914 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4007740 con 0x7f81dc155da0 2024-04-25T01:25:29.190 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.208+0000 7f81daffd640 1 -- 172.21.15.187:0/1788943914 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d40078e0 con 0x7f81dc155da0 2024-04-25T01:25:29.190 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.208+0000 7f81daffd640 1 -- 172.21.15.187:0/1788943914 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4007cb0 con 0x7f81dc155da0 2024-04-25T01:25:29.190 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.209+0000 7f81e1c31640 1 -- 172.21.15.187:0/1788943914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81dc155da0 msgr2=0x7f81dc1561e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.190 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.209+0000 7f81e1c31640 1 --2- 172.21.15.187:0/1788943914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81dc155da0 0x7f81dc1561e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009d90 tx=0x7f81d402a660 comp rx=0 tx=0).stop 2024-04-25T01:25:29.190 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.209+0000 7f81e1c31640 1 -- 172.21.15.187:0/1788943914 shutdown_connections 2024-04-25T01:25:29.190 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.209+0000 7f81e1c31640 1 --2- 172.21.15.187:0/1788943914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81dc155da0 0x7f81dc1561e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.190 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.209+0000 7f81e1c31640 1 --2- 172.21.15.187:0/1788943914 >> v2:172.21.15.187:3300/0 conn(0x7f81dc0add20 0x7f81dc0c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.190 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.209+0000 7f81e1c31640 1 -- 172.21.15.187:0/1788943914 >> 172.21.15.187:0/1788943914 conn(0x7f81dc0c49b0 msgr2=0x7f81dc0c4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:29.190 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.209+0000 7f81e1c31640 1 -- 172.21.15.187:0/1788943914 shutdown_connections 2024-04-25T01:25:29.190 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.209+0000 7f81e1c31640 1 -- 172.21.15.187:0/1788943914 wait complete. 2024-04-25T01:25:29.191 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.210+0000 7f81e1c31640 1 Processor -- start 2024-04-25T01:25:29.191 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.210+0000 7f81e1c31640 1 -- start start 2024-04-25T01:25:29.193 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.211+0000 7f81e1c31640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81dc0add20 0x7f81dc0c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.193 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.211+0000 7f81e1c31640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81dc14fa40 0x7f81dc14a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.193 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.211+0000 7f81e1c31640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc155da0 0x7f81dc13f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.193 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.211+0000 7f81e1c31640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc0c28b0 con 0x7f81dc155da0 2024-04-25T01:25:29.193 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.211+0000 7f81e1c31640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc156720 con 0x7f81dc0add20 2024-04-25T01:25:29.193 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.211+0000 7f81e1c31640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f81dc06ef40 con 0x7f81dc14fa40 2024-04-25T01:25:29.193 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.211+0000 7f81db7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81dc0add20 0x7f81dc0c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.193 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.211+0000 7f81db7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81dc0add20 0x7f81dc0c55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:57044/0 (socket says 172.21.15.187:57044) 2024-04-25T01:25:29.194 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.211+0000 7f81db7fe640 1 -- 172.21.15.187:0/510191782 learned_addr learned my addr 172.21.15.187:0/510191782 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T01:25:29.194 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.211+0000 7f81d2ffd640 1 --2- 172.21.15.187:0/510191782 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81dc14fa40 0x7f81dc14a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.194 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.211+0000 7f81dbfff640 1 --2- 172.21.15.187:0/510191782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc155da0 0x7f81dc13f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.194 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.211+0000 7f81d2ffd640 1 -- 172.21.15.187:0/510191782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81dc0add20 msgr2=0x7f81dc0c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.194 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.211+0000 7f81d2ffd640 1 --2- 172.21.15.187:0/510191782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81dc0add20 0x7f81dc0c55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.194 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.211+0000 7f81d2ffd640 1 -- 172.21.15.187:0/510191782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc155da0 msgr2=0x7f81dc13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.194 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.211+0000 7f81d2ffd640 1 --2- 172.21.15.187:0/510191782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc155da0 0x7f81dc13f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.194 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.211+0000 7f81d2ffd640 1 -- 172.21.15.187:0/510191782 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc0c7c20 con 0x7f81dc14fa40 2024-04-25T01:25:29.194 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.211+0000 7f81db7fe640 1 --2- 172.21.15.187:0/510191782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81dc0add20 0x7f81dc0c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:25:29.194 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.211+0000 7f81dbfff640 1 --2- 172.21.15.187:0/510191782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc155da0 0x7f81dc13f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:25:29.194 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.211+0000 7f81d2ffd640 1 --2- 172.21.15.187:0/510191782 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81dc14fa40 0x7f81dc14a390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f81c400e8a0 tx=0x7f81c400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:29.194 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.212+0000 7f81d8ff9640 1 -- 172.21.15.187:0/510191782 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4017070 con 0x7f81dc14fa40 2024-04-25T01:25:29.194 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.212+0000 7f81d8ff9640 1 -- 172.21.15.187:0/510191782 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c400c940 con 0x7f81dc14fa40 2024-04-25T01:25:29.194 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.212+0000 7f81d8ff9640 1 -- 172.21.15.187:0/510191782 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c400cd30 con 0x7f81dc14fa40 2024-04-25T01:25:29.195 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.212+0000 7f81e1c31640 1 -- 172.21.15.187:0/510191782 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81dc148730 con 0x7f81dc14fa40 2024-04-25T01:25:29.195 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.212+0000 7f81e1c31640 1 -- 172.21.15.187:0/510191782 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f81dc14a8d0 con 0x7f81dc14fa40 2024-04-25T01:25:29.195 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.212+0000 7f81d8ff9640 1 -- 172.21.15.187:0/510191782 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f81c401a070 con 0x7f81dc14fa40 2024-04-25T01:25:29.195 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.212+0000 7f81d8ff9640 1 -- 172.21.15.187:0/510191782 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f81c401c790 con 0x7f81dc14fa40 2024-04-25T01:25:29.195 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.213+0000 7f81e1c31640 1 -- 172.21.15.187:0/510191782 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81dc04f500 con 0x7f81dc14fa40 2024-04-25T01:25:29.195 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.215+0000 7f81d8ff9640 1 -- 172.21.15.187:0/510191782 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f81c40092e0 con 0x7f81dc14fa40 2024-04-25T01:25:29.195 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:29.215+0000 7f6756c0d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T01:25:29.266 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.286+0000 7f81e1c31640 1 -- 172.21.15.187:0/510191782 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "687b49ac-8bbb-4826-99c3-007b21a72cff", "id": 0} v 0) -- 0x7f81dc0c3de0 con 0x7f81dc14fa40 2024-04-25T01:25:29.272 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.292+0000 7f81d8ff9640 1 -- 172.21.15.187:0/510191782 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "687b49ac-8bbb-4826-99c3-007b21a72cff", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f81c4005990 con 0x7f81dc14fa40 2024-04-25T01:25:29.272 INFO:teuthology.orchestra.run.smithi187.stdout:0 2024-04-25T01:25:29.273 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.293+0000 7f81e1c31640 1 -- 172.21.15.187:0/510191782 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81dc14fa40 msgr2=0x7f81dc14a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.273 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.293+0000 7f81e1c31640 1 --2- 172.21.15.187:0/510191782 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81dc14fa40 0x7f81dc14a390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f81c400e8a0 tx=0x7f81c400ed60 comp rx=0 tx=0).stop 2024-04-25T01:25:29.273 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.293+0000 7f81e1c31640 1 -- 172.21.15.187:0/510191782 shutdown_connections 2024-04-25T01:25:29.274 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.293+0000 7f81e1c31640 1 --2- 172.21.15.187:0/510191782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc155da0 0x7f81dc13f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.274 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.293+0000 7f81e1c31640 1 --2- 172.21.15.187:0/510191782 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81dc14fa40 0x7f81dc14a390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.274 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.293+0000 7f81e1c31640 1 --2- 172.21.15.187:0/510191782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81dc0add20 0x7f81dc0c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.274 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.293+0000 7f81e1c31640 1 -- 172.21.15.187:0/510191782 >> 172.21.15.187:0/510191782 conn(0x7f81dc0c49b0 msgr2=0x7f81dc1deeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:29.274 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.293+0000 7f81e1c31640 1 -- 172.21.15.187:0/510191782 shutdown_connections 2024-04-25T01:25:29.274 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.293+0000 7f81e1c31640 1 -- 172.21.15.187:0/510191782 wait complete. 2024-04-25T01:25:29.284 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 8fc51f65-937b-4a6e-83ac-9c4e6113c738 1 2024-04-25T01:25:29.379 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.399+0000 7f7fcfcba640 1 Processor -- start 2024-04-25T01:25:29.380 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.400+0000 7f7fcfcba640 1 -- start start 2024-04-25T01:25:29.381 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.400+0000 7f7fcfcba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fc80add20 0x7f7fc80c4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.381 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.400+0000 7f7fcfcba640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7fc8155da0 0x7f7fc8156210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.382 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.400+0000 7f7fcfcba640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fc806dc20 con 0x7f7fc814fa40 2024-04-25T01:25:29.382 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.400+0000 7f7fcfcba640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f7fc806d9d0 con 0x7f7fc8155da0 2024-04-25T01:25:29.382 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.400+0000 7f7fcfcba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7fc806e210 con 0x7f7fc80add20 2024-04-25T01:25:29.382 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.401+0000 7f7fcd255640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fc80add20 0x7f7fc80c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.382 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.401+0000 7f7fcda56640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f7fc814fa40 0x7f7fc806e830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:52368/0 (socket says 172.21.15.187:52368) 2024-04-25T01:25:29.382 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.401+0000 7f7fcda56640 1 -- 172.21.15.187:0/3287014313 learned_addr learned my addr 172.21.15.187:0/3287014313 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-04-25T01:25:29.382 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.401+0000 7f7fcd255640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fc80add20 0x7f7fc80c4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:53220/0 (socket says 172.21.15.187:53220) 2024-04-25T01:25:29.382 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.401+0000 7f7fce257640 1 --2- 172.21.15.187:0/3287014313 >> v2:172.21.15.112:3300/0 conn(0x7f7fc8155da0 0x7f7fc8156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.382 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.401+0000 7f7fcd255640 1 -- 172.21.15.187:0/3287014313 >> v2:172.21.15.112:3300/0 conn(0x7f7fc8155da0 msgr2=0x7f7fc8156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:29.382 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.401+0000 7f7fcd255640 1 --2- 172.21.15.187:0/3287014313 >> v2:172.21.15.112:3300/0 conn(0x7f7fc8155da0 0x7f7fc8156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.382 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.401+0000 7f7fcd255640 1 -- 172.21.15.187:0/3287014313 >> v1:172.21.15.187:6789/0 conn(0x7f7fc814fa40 legacy=0x7f7fc806e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.382 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.401+0000 7f7fcd255640 1 -- 172.21.15.187:0/3287014313 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc806e4f0 con 0x7f7fc80add20 2024-04-25T01:25:29.383 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.401+0000 7f7fcca54640 1 -- 172.21.15.187:0/3287014313 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 151311474 0 0) 0x7f7fc806dc20 con 0x7f7fc814fa40 2024-04-25T01:25:29.383 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.401+0000 7f7fce257640 1 --2- 172.21.15.187:0/3287014313 >> v2:172.21.15.112:3300/0 conn(0x7f7fc8155da0 0x7f7fc8156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:25:29.383 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.402+0000 7f7fcd255640 1 --2- 172.21.15.187:0/3287014313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fc80add20 0x7f7fc80c4370 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7fb80097a0 tx=0x7f7fb802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a80684f3e80ed72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:29.383 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.402+0000 7f7fcca54640 1 -- 172.21.15.187:0/3287014313 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8037070 con 0x7f7fc80add20 2024-04-25T01:25:29.383 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.402+0000 7f7fcca54640 1 -- 172.21.15.187:0/3287014313 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb8032440 con 0x7f7fc80add20 2024-04-25T01:25:29.383 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.402+0000 7f7fcca54640 1 -- 172.21.15.187:0/3287014313 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8032810 con 0x7f7fc80add20 2024-04-25T01:25:29.386 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.402+0000 7f7fcfcba640 1 -- 172.21.15.187:0/3287014313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fc80add20 msgr2=0x7f7fc80c4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.386 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.402+0000 7f7fcfcba640 1 --2- 172.21.15.187:0/3287014313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fc80add20 0x7f7fc80c4370 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7fb80097a0 tx=0x7f7fb802a180 comp rx=0 tx=0).stop 2024-04-25T01:25:29.386 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.402+0000 7f7fcfcba640 1 -- 172.21.15.187:0/3287014313 shutdown_connections 2024-04-25T01:25:29.386 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.402+0000 7f7fcfcba640 1 --2- 172.21.15.187:0/3287014313 >> v2:172.21.15.112:3300/0 conn(0x7f7fc8155da0 0x7f7fc8156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.402+0000 7f7fcfcba640 1 --2- 172.21.15.187:0/3287014313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fc80add20 0x7f7fc80c4370 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.402+0000 7f7fcfcba640 1 -- 172.21.15.187:0/3287014313 >> 172.21.15.187:0/3287014313 conn(0x7f7fc80c49b0 msgr2=0x7f7fc80c4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:29.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.402+0000 7f7fcfcba640 1 -- 172.21.15.187:0/3287014313 shutdown_connections 2024-04-25T01:25:29.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.402+0000 7f7fcfcba640 1 -- 172.21.15.187:0/3287014313 wait complete. 2024-04-25T01:25:29.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.403+0000 7f7fcfcba640 1 Processor -- start 2024-04-25T01:25:29.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.404+0000 7f7fcfcba640 1 -- start start 2024-04-25T01:25:29.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.404+0000 7f7fcfcba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fc80add20 0x7f7fc8154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.404+0000 7f7fcfcba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fc814fa40 0x7f7fc8142500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.404+0000 7f7fcfcba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fc8155da0 0x7f7fc8142a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.404+0000 7f7fcfcba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc80c28b0 con 0x7f7fc8155da0 2024-04-25T01:25:29.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.404+0000 7f7fcfcba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc8155850 con 0x7f7fc814fa40 2024-04-25T01:25:29.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.404+0000 7f7fcfcba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7fc806ef40 con 0x7f7fc80add20 2024-04-25T01:25:29.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.404+0000 7f7fcda56640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fc80add20 0x7f7fc8154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.404+0000 7f7fcd255640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fc814fa40 0x7f7fc8142500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.404+0000 7f7fcda56640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fc80add20 0x7f7fc8154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:53222/0 (socket says 172.21.15.187:53222) 2024-04-25T01:25:29.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.404+0000 7f7fcda56640 1 -- 172.21.15.187:0/1122790688 learned_addr learned my addr 172.21.15.187:0/1122790688 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T01:25:29.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.404+0000 7f7fce257640 1 --2- 172.21.15.187:0/1122790688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fc8155da0 0x7f7fc8142a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.405+0000 7f7fcda56640 1 -- 172.21.15.187:0/1122790688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fc814fa40 msgr2=0x7f7fc8142500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.405+0000 7f7fcda56640 1 --2- 172.21.15.187:0/1122790688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fc814fa40 0x7f7fc8142500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.405+0000 7f7fcda56640 1 -- 172.21.15.187:0/1122790688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fc8155da0 msgr2=0x7f7fc8142a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.405+0000 7f7fcda56640 1 --2- 172.21.15.187:0/1122790688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fc8155da0 0x7f7fc8142a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.405+0000 7f7fcda56640 1 -- 172.21.15.187:0/1122790688 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc80c6970 con 0x7f7fc80add20 2024-04-25T01:25:29.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.405+0000 7f7fcd255640 1 --2- 172.21.15.187:0/1122790688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fc814fa40 0x7f7fc8142500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:25:29.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.405+0000 7f7fcda56640 1 --2- 172.21.15.187:0/1122790688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fc80add20 0x7f7fc8154160 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400c4d0 tx=0x7f7fc400c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:29.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.405+0000 7f7fbe7fc640 1 -- 172.21.15.187:0/1122790688 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4018070 con 0x7f7fc80add20 2024-04-25T01:25:29.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.405+0000 7f7fbe7fc640 1 -- 172.21.15.187:0/1122790688 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc4007500 con 0x7f7fc80add20 2024-04-25T01:25:29.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.405+0000 7f7fcfcba640 1 -- 172.21.15.187:0/1122790688 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc8148dd0 con 0x7f7fc80add20 2024-04-25T01:25:29.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.405+0000 7f7fbe7fc640 1 -- 172.21.15.187:0/1122790688 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4004a20 con 0x7f7fc80add20 2024-04-25T01:25:29.389 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.405+0000 7f7fcfcba640 1 -- 172.21.15.187:0/1122790688 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc814ada0 con 0x7f7fc80add20 2024-04-25T01:25:29.389 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.405+0000 7f7fbe7fc640 1 -- 172.21.15.187:0/1122790688 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7fc401b070 con 0x7f7fc80add20 2024-04-25T01:25:29.389 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.405+0000 7f7fbe7fc640 1 -- 172.21.15.187:0/1122790688 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f7fc40127b0 con 0x7f7fc80add20 2024-04-25T01:25:29.389 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.406+0000 7f7fcfcba640 1 -- 172.21.15.187:0/1122790688 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc814e470 con 0x7f7fc80add20 2024-04-25T01:25:29.389 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.409+0000 7f7fbe7fc640 1 -- 172.21.15.187:0/1122790688 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7fc814e470 con 0x7f7fc80add20 2024-04-25T01:25:29.461 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.481+0000 7f7fcfcba640 1 -- 172.21.15.187:0/1122790688 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8fc51f65-937b-4a6e-83ac-9c4e6113c738", "id": 1} v 0) -- 0x7f7fc804f500 con 0x7f7fc80add20 2024-04-25T01:25:29.469 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.489+0000 7f7fbe7fc640 1 -- 172.21.15.187:0/1122790688 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8fc51f65-937b-4a6e-83ac-9c4e6113c738", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f7fc80c6970 con 0x7f7fc80add20 2024-04-25T01:25:29.469 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-04-25T01:25:29.470 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.490+0000 7f7fcfcba640 1 -- 172.21.15.187:0/1122790688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fc80add20 msgr2=0x7f7fc8154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.470 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.490+0000 7f7fcfcba640 1 --2- 172.21.15.187:0/1122790688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fc80add20 0x7f7fc8154160 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400c4d0 tx=0x7f7fc400c990 comp rx=0 tx=0).stop 2024-04-25T01:25:29.470 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.490+0000 7f7fcfcba640 1 -- 172.21.15.187:0/1122790688 shutdown_connections 2024-04-25T01:25:29.470 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.490+0000 7f7fcfcba640 1 --2- 172.21.15.187:0/1122790688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fc8155da0 0x7f7fc8142a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.471 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.490+0000 7f7fcfcba640 1 --2- 172.21.15.187:0/1122790688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fc814fa40 0x7f7fc8142500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.471 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.490+0000 7f7fcfcba640 1 --2- 172.21.15.187:0/1122790688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fc80add20 0x7f7fc8154160 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.471 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.490+0000 7f7fcfcba640 1 -- 172.21.15.187:0/1122790688 >> 172.21.15.187:0/1122790688 conn(0x7f7fc80c49b0 msgr2=0x7f7fc80c4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:29.471 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.490+0000 7f7fcfcba640 1 -- 172.21.15.187:0/1122790688 shutdown_connections 2024-04-25T01:25:29.471 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.490+0000 7f7fcfcba640 1 -- 172.21.15.187:0/1122790688 wait complete. 2024-04-25T01:25:29.483 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 8ad36ec9-5911-4be0-b2df-f57d9b3a1092 2 2024-04-25T01:25:29.582 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.602+0000 7f1a0f838640 1 Processor -- start 2024-04-25T01:25:29.582 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.603+0000 7f1a0f838640 1 -- start start 2024-04-25T01:25:29.584 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.603+0000 7f1a0f838640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1a08153110 0x7f1a0806e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.584 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.603+0000 7f1a0f838640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1a0806eb70 0x7f1a08159250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.584 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.603+0000 7f1a0f838640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a0806b5d0 con 0x7f1a0806c3c0 2024-04-25T01:25:29.584 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.603+0000 7f1a0f838640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f1a080c36f0 con 0x7f1a0806eb70 2024-04-25T01:25:29.584 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.603+0000 7f1a0f838640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1a08159ea0 con 0x7f1a08153110 2024-04-25T01:25:29.584 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.603+0000 7f1a0d5d4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1a08153110 0x7f1a0806e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.584 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.603+0000 7f1a0d5d4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1a08153110 0x7f1a0806e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:57062/0 (socket says 172.21.15.187:57062) 2024-04-25T01:25:29.585 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.603+0000 7f1a0d5d4640 1 -- 172.21.15.187:0/4233576821 learned_addr learned my addr 172.21.15.187:0/4233576821 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T01:25:29.585 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.603+0000 7f1a0ddd5640 1 --1- 172.21.15.187:0/4233576821 >> v1:172.21.15.187:6789/0 conn(0x7f1a0806c3c0 0x7f1a08159790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:52378/0 (socket says 172.21.15.187:52378) 2024-04-25T01:25:29.585 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.604+0000 7f1a0cdd3640 1 --2- 172.21.15.187:0/4233576821 >> v2:172.21.15.112:3300/0 conn(0x7f1a0806eb70 0x7f1a08159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.585 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.604+0000 7f1a0d5d4640 1 -- 172.21.15.187:0/4233576821 >> v2:172.21.15.112:3300/0 conn(0x7f1a0806eb70 msgr2=0x7f1a08159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:29.585 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.604+0000 7f1a0d5d4640 1 --2- 172.21.15.187:0/4233576821 >> v2:172.21.15.112:3300/0 conn(0x7f1a0806eb70 0x7f1a08159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.585 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.604+0000 7f1a0d5d4640 1 -- 172.21.15.187:0/4233576821 >> v1:172.21.15.187:6789/0 conn(0x7f1a0806c3c0 legacy=0x7f1a08159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.585 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.604+0000 7f19fffff640 1 -- 172.21.15.187:0/4233576821 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1591277880 0 0) 0x7f1a0806b5d0 con 0x7f1a0806c3c0 2024-04-25T01:25:29.585 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.604+0000 7f1a0d5d4640 1 -- 172.21.15.187:0/4233576821 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a080c2220 con 0x7f1a08153110 2024-04-25T01:25:29.585 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.604+0000 7f1a0cdd3640 1 --2- 172.21.15.187:0/4233576821 >> v2:172.21.15.112:3300/0 conn(0x7f1a0806eb70 0x7f1a08159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:25:29.585 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.604+0000 7f1a0d5d4640 1 --2- 172.21.15.187:0/4233576821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a08153110 0x7f1a0806e630 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1a0806f720 tx=0x7f1a0402a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=407892e8560b8ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:29.585 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.605+0000 7f19fffff640 1 -- 172.21.15.187:0/4233576821 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04037070 con 0x7f1a08153110 2024-04-25T01:25:29.585 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.605+0000 7f19fffff640 1 -- 172.21.15.187:0/4233576821 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a04032440 con 0x7f1a08153110 2024-04-25T01:25:29.585 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.605+0000 7f19fffff640 1 -- 172.21.15.187:0/4233576821 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04032810 con 0x7f1a08153110 2024-04-25T01:25:29.588 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.605+0000 7f1a0f838640 1 -- 172.21.15.187:0/4233576821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a08153110 msgr2=0x7f1a0806e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.588 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.605+0000 7f1a0f838640 1 --2- 172.21.15.187:0/4233576821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a08153110 0x7f1a0806e630 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1a0806f720 tx=0x7f1a0402a1e0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.605+0000 7f1a0f838640 1 -- 172.21.15.187:0/4233576821 shutdown_connections 2024-04-25T01:25:29.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.605+0000 7f1a0f838640 1 --2- 172.21.15.187:0/4233576821 >> v2:172.21.15.112:3300/0 conn(0x7f1a0806eb70 0x7f1a08159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.605+0000 7f1a0f838640 1 --2- 172.21.15.187:0/4233576821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a08153110 0x7f1a0806e630 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.605+0000 7f1a0f838640 1 -- 172.21.15.187:0/4233576821 >> 172.21.15.187:0/4233576821 conn(0x7f1a080c7a70 msgr2=0x7f1a080c7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:29.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.605+0000 7f1a0f838640 1 -- 172.21.15.187:0/4233576821 shutdown_connections 2024-04-25T01:25:29.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.605+0000 7f1a0f838640 1 -- 172.21.15.187:0/4233576821 wait complete. 2024-04-25T01:25:29.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.606+0000 7f1a0f838640 1 Processor -- start 2024-04-25T01:25:29.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.606+0000 7f1a0f838640 1 -- start start 2024-04-25T01:25:29.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.607+0000 7f1a0f838640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a0806c3c0 0x7f1a08148420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.607+0000 7f1a0f838640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a0806eb70 0x7f1a081459b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.607+0000 7f1a0f838640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a08153110 0x7f1a08145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.607+0000 7f1a0f838640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a081534f0 con 0x7f1a0806c3c0 2024-04-25T01:25:29.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.607+0000 7f1a0f838640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1a08153670 con 0x7f1a0806eb70 2024-04-25T01:25:29.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.607+0000 7f1a0f838640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1a0806bef0 con 0x7f1a08153110 2024-04-25T01:25:29.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.607+0000 7f1a0cdd3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a0806eb70 0x7f1a081459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.607+0000 7f1a0cdd3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a0806eb70 0x7f1a081459b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:57068/0 (socket says 172.21.15.187:57068) 2024-04-25T01:25:29.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.607+0000 7f1a0cdd3640 1 -- 172.21.15.187:0/3965775726 learned_addr learned my addr 172.21.15.187:0/3965775726 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T01:25:29.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.607+0000 7f1a0ddd5640 1 --2- 172.21.15.187:0/3965775726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a08153110 0x7f1a08145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.607+0000 7f1a0d5d4640 1 --2- 172.21.15.187:0/3965775726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a0806c3c0 0x7f1a08148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.608+0000 7f1a0cdd3640 1 -- 172.21.15.187:0/3965775726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a08153110 msgr2=0x7f1a08145ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.608+0000 7f1a0cdd3640 1 --2- 172.21.15.187:0/3965775726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a08153110 0x7f1a08145ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.608+0000 7f1a0cdd3640 1 -- 172.21.15.187:0/3965775726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a0806c3c0 msgr2=0x7f1a08148420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.608+0000 7f1a0cdd3640 1 --2- 172.21.15.187:0/3965775726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a0806c3c0 0x7f1a08148420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.608+0000 7f1a0cdd3640 1 -- 172.21.15.187:0/3965775726 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a080aa7d0 con 0x7f1a0806eb70 2024-04-25T01:25:29.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.608+0000 7f1a0cdd3640 1 --2- 172.21.15.187:0/3965775726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a0806eb70 0x7f1a081459b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f19f800e730 tx=0x7f19f800ebf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:29.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.608+0000 7f19fdffb640 1 -- 172.21.15.187:0/3965775726 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f800f040 con 0x7f1a0806eb70 2024-04-25T01:25:29.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.608+0000 7f19fdffb640 1 -- 172.21.15.187:0/3965775726 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19f8004230 con 0x7f1a0806eb70 2024-04-25T01:25:29.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.608+0000 7f1a0f838640 1 -- 172.21.15.187:0/3965775726 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a080aaab0 con 0x7f1a0806eb70 2024-04-25T01:25:29.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.608+0000 7f19fdffb640 1 -- 172.21.15.187:0/3965775726 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f800ee40 con 0x7f1a0806eb70 2024-04-25T01:25:29.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.608+0000 7f19fdffb640 1 -- 172.21.15.187:0/3965775726 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f19f800ca60 con 0x7f1a0806eb70 2024-04-25T01:25:29.591 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.608+0000 7f1a0f838640 1 -- 172.21.15.187:0/3965775726 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a080c6000 con 0x7f1a0806eb70 2024-04-25T01:25:29.591 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.608+0000 7f19fdffb640 1 -- 172.21.15.187:0/3965775726 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f19f8003a30 con 0x7f1a0806eb70 2024-04-25T01:25:29.591 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.609+0000 7f1a0f838640 1 -- 172.21.15.187:0/3965775726 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a0804f500 con 0x7f1a0806eb70 2024-04-25T01:25:29.591 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.611+0000 7f19fdffb640 1 -- 172.21.15.187:0/3965775726 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f19f8017020 con 0x7f1a0806eb70 2024-04-25T01:25:29.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.684+0000 7f1a0f838640 1 -- 172.21.15.187:0/3965775726 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8ad36ec9-5911-4be0-b2df-f57d9b3a1092", "id": 2} v 0) -- 0x7f1a080c7ff0 con 0x7f1a0806eb70 2024-04-25T01:25:29.669 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.690+0000 7f19fdffb640 1 -- 172.21.15.187:0/3965775726 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8ad36ec9-5911-4be0-b2df-f57d9b3a1092", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f19f8003c30 con 0x7f1a0806eb70 2024-04-25T01:25:29.670 INFO:teuthology.orchestra.run.smithi187.stdout:2 2024-04-25T01:25:29.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.691+0000 7f1a0f838640 1 -- 172.21.15.187:0/3965775726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a0806eb70 msgr2=0x7f1a081459b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.691+0000 7f1a0f838640 1 --2- 172.21.15.187:0/3965775726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a0806eb70 0x7f1a081459b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f19f800e730 tx=0x7f19f800ebf0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.691+0000 7f1a0f838640 1 -- 172.21.15.187:0/3965775726 shutdown_connections 2024-04-25T01:25:29.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.691+0000 7f1a0f838640 1 --2- 172.21.15.187:0/3965775726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a08153110 0x7f1a08145ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.691+0000 7f1a0f838640 1 --2- 172.21.15.187:0/3965775726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a0806eb70 0x7f1a081459b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.691+0000 7f1a0f838640 1 --2- 172.21.15.187:0/3965775726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a0806c3c0 0x7f1a08148420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.691+0000 7f1a0f838640 1 -- 172.21.15.187:0/3965775726 >> 172.21.15.187:0/3965775726 conn(0x7f1a080c7a70 msgr2=0x7f1a080c4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:29.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.691+0000 7f1a0f838640 1 -- 172.21.15.187:0/3965775726 shutdown_connections 2024-04-25T01:25:29.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.691+0000 7f1a0f838640 1 -- 172.21.15.187:0/3965775726 wait complete. 2024-04-25T01:25:29.681 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 62da8b34-63d3-4f21-94f6-126f6b310912 3 2024-04-25T01:25:29.779 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.799+0000 7fede76e1640 1 Processor -- start 2024-04-25T01:25:29.779 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.800+0000 7fede76e1640 1 -- start start 2024-04-25T01:25:29.780 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.800+0000 7fede76e1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fede0153310 0x7fede0156650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.781 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.800+0000 7fede76e1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede00a85d0 0x7fede00c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.781 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.800+0000 7fede76e1640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fede0155b90 con 0x7fede006df70 2024-04-25T01:25:29.781 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.800+0000 7fede76e1640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fede00c7590 con 0x7fede0153310 2024-04-25T01:25:29.781 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.800+0000 7fede76e1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fede00c8530 con 0x7fede00a85d0 2024-04-25T01:25:29.781 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.800+0000 7fede547d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fede0153310 0x7fede0156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.781 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.800+0000 7fede5c7e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede00a85d0 0x7fede00c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.781 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.800+0000 7fede547d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fede0153310 0x7fede0156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:57072/0 (socket says 172.21.15.187:57072) 2024-04-25T01:25:29.781 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.800+0000 7fede547d640 1 -- 172.21.15.187:0/1985170403 learned_addr learned my addr 172.21.15.187:0/1985170403 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T01:25:29.781 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.800+0000 7fede5c7e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede00a85d0 0x7fede00c59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:53228/0 (socket says 172.21.15.187:53228) 2024-04-25T01:25:29.781 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.801+0000 7fede547d640 1 -- 172.21.15.187:0/1985170403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede00a85d0 msgr2=0x7fede00c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:29.781 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.801+0000 7fede547d640 1 --2- 172.21.15.187:0/1985170403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede00a85d0 0x7fede00c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.781 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.801+0000 7fede547d640 1 -- 172.21.15.187:0/1985170403 >> v1:172.21.15.187:6789/0 conn(0x7fede006df70 legacy=0x7fede00c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.781 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.801+0000 7fedd7fff640 1 -- 172.21.15.187:0/1985170403 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 533246798 0 0) 0x7fede0155b90 con 0x7fede006df70 2024-04-25T01:25:29.782 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.801+0000 7fede547d640 1 -- 172.21.15.187:0/1985170403 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede00c8810 con 0x7fede0153310 2024-04-25T01:25:29.783 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.801+0000 7fede547d640 1 --2- 172.21.15.187:0/1985170403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fede0153310 0x7fede0156650 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7feddc009870 tx=0x7feddc02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1179229109b4847d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:29.783 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.801+0000 7fedd7fff640 1 -- 172.21.15.187:0/1985170403 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc037070 con 0x7fede0153310 2024-04-25T01:25:29.783 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.802+0000 7fedd7fff640 1 -- 172.21.15.187:0/1985170403 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feddc032440 con 0x7fede0153310 2024-04-25T01:25:29.783 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.802+0000 7fedd7fff640 1 -- 172.21.15.187:0/1985170403 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc032810 con 0x7fede0153310 2024-04-25T01:25:29.783 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.802+0000 7fede76e1640 1 -- 172.21.15.187:0/1985170403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fede0153310 msgr2=0x7fede0156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.784 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.802+0000 7fede76e1640 1 --2- 172.21.15.187:0/1985170403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fede0153310 0x7fede0156650 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7feddc009870 tx=0x7feddc02a130 comp rx=0 tx=0).stop 2024-04-25T01:25:29.784 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.802+0000 7fede76e1640 1 -- 172.21.15.187:0/1985170403 shutdown_connections 2024-04-25T01:25:29.784 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.802+0000 7fede76e1640 1 --2- 172.21.15.187:0/1985170403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede00a85d0 0x7fede00c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.784 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.802+0000 7fede76e1640 1 --2- 172.21.15.187:0/1985170403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fede0153310 0x7fede0156650 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.784 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.802+0000 7fede76e1640 1 -- 172.21.15.187:0/1985170403 >> 172.21.15.187:0/1985170403 conn(0x7fede00c8f80 msgr2=0x7fede00c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:29.784 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.802+0000 7fede76e1640 1 -- 172.21.15.187:0/1985170403 shutdown_connections 2024-04-25T01:25:29.784 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.802+0000 7fede76e1640 1 -- 172.21.15.187:0/1985170403 wait complete. 2024-04-25T01:25:29.784 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.803+0000 7fede76e1640 1 Processor -- start 2024-04-25T01:25:29.784 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.803+0000 7fede76e1640 1 -- start start 2024-04-25T01:25:29.786 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.804+0000 7fede76e1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede006df70 0x7fede0151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.786 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.804+0000 7fede76e1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fede00a85d0 0x7fede014fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.786 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.804+0000 7fede76e1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fede0153310 0x7fede01500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.787 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.804+0000 7fede76e1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fede0156130 con 0x7fede00a85d0 2024-04-25T01:25:29.787 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.804+0000 7fede76e1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fede0155e50 con 0x7fede0153310 2024-04-25T01:25:29.787 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.804+0000 7fede76e1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fede00c5ef0 con 0x7fede006df70 2024-04-25T01:25:29.787 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.804+0000 7fede5c7e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fede0153310 0x7fede01500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.787 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.804+0000 7fede547d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede006df70 0x7fede0151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.787 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.804+0000 7fede5c7e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fede0153310 0x7fede01500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:57082/0 (socket says 172.21.15.187:57082) 2024-04-25T01:25:29.787 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.804+0000 7fede5c7e640 1 -- 172.21.15.187:0/462253672 learned_addr learned my addr 172.21.15.187:0/462253672 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T01:25:29.787 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.804+0000 7fede4c7c640 1 --2- 172.21.15.187:0/462253672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fede00a85d0 0x7fede014fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.787 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.804+0000 7fede5c7e640 1 -- 172.21.15.187:0/462253672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede006df70 msgr2=0x7fede0151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.787 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.804+0000 7fede5c7e640 1 --2- 172.21.15.187:0/462253672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede006df70 0x7fede0151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.787 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.804+0000 7fede5c7e640 1 -- 172.21.15.187:0/462253672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fede00a85d0 msgr2=0x7fede014fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.787 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.804+0000 7fede5c7e640 1 --2- 172.21.15.187:0/462253672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fede00a85d0 0x7fede014fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.787 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.804+0000 7fede5c7e640 1 -- 172.21.15.187:0/462253672 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede0071250 con 0x7fede0153310 2024-04-25T01:25:29.787 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.804+0000 7fede4c7c640 1 --2- 172.21.15.187:0/462253672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fede00a85d0 0x7fede014fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:25:29.788 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.805+0000 7fede5c7e640 1 --2- 172.21.15.187:0/462253672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fede0153310 0x7fede01500d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fedd800efc0 tx=0x7fedd800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:29.788 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.805+0000 7fedd5ffb640 1 -- 172.21.15.187:0/462253672 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8018020 con 0x7fede0153310 2024-04-25T01:25:29.788 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.805+0000 7fedd5ffb640 1 -- 172.21.15.187:0/462253672 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd8014480 con 0x7fede0153310 2024-04-25T01:25:29.788 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.805+0000 7fede76e1640 1 -- 172.21.15.187:0/462253672 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede006fac0 con 0x7fede0153310 2024-04-25T01:25:29.788 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.805+0000 7fedd5ffb640 1 -- 172.21.15.187:0/462253672 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8014870 con 0x7fede0153310 2024-04-25T01:25:29.788 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.805+0000 7fede76e1640 1 -- 172.21.15.187:0/462253672 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede0070080 con 0x7fede0153310 2024-04-25T01:25:29.788 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.805+0000 7fedd5ffb640 1 -- 172.21.15.187:0/462253672 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fedd8005d10 con 0x7fede0153310 2024-04-25T01:25:29.788 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.805+0000 7fedd5ffb640 1 -- 172.21.15.187:0/462253672 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fedd8013870 con 0x7fede0153310 2024-04-25T01:25:29.788 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.806+0000 7fede76e1640 1 -- 172.21.15.187:0/462253672 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede00702c0 con 0x7fede0153310 2024-04-25T01:25:29.789 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.810+0000 7fedd5ffb640 1 -- 172.21.15.187:0/462253672 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fede00702c0 con 0x7fede0153310 2024-04-25T01:25:29.862 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.882+0000 7fede76e1640 1 -- 172.21.15.187:0/462253672 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "62da8b34-63d3-4f21-94f6-126f6b310912", "id": 3} v 0) -- 0x7fede0071480 con 0x7fede0153310 2024-04-25T01:25:29.868 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.888+0000 7fedd5ffb640 1 -- 172.21.15.187:0/462253672 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "62da8b34-63d3-4f21-94f6-126f6b310912", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fedd8022070 con 0x7fede0153310 2024-04-25T01:25:29.868 INFO:teuthology.orchestra.run.smithi187.stdout:3 2024-04-25T01:25:29.869 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.889+0000 7fede76e1640 1 -- 172.21.15.187:0/462253672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fede0153310 msgr2=0x7fede01500d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.869 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.889+0000 7fede76e1640 1 --2- 172.21.15.187:0/462253672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fede0153310 0x7fede01500d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fedd800efc0 tx=0x7fedd800c490 comp rx=0 tx=0).stop 2024-04-25T01:25:29.869 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.889+0000 7fede76e1640 1 -- 172.21.15.187:0/462253672 shutdown_connections 2024-04-25T01:25:29.869 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.889+0000 7fede76e1640 1 --2- 172.21.15.187:0/462253672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fede0153310 0x7fede01500d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.869 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.889+0000 7fede76e1640 1 --2- 172.21.15.187:0/462253672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fede00a85d0 0x7fede014fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.869 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.889+0000 7fede76e1640 1 --2- 172.21.15.187:0/462253672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede006df70 0x7fede0151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.869 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.889+0000 7fede76e1640 1 -- 172.21.15.187:0/462253672 >> 172.21.15.187:0/462253672 conn(0x7fede00c8f80 msgr2=0x7fede0155520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:29.870 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.889+0000 7fede76e1640 1 -- 172.21.15.187:0/462253672 shutdown_connections 2024-04-25T01:25:29.870 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.889+0000 7fede76e1640 1 -- 172.21.15.187:0/462253672 wait complete. 2024-04-25T01:25:29.874 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:29.895+0000 7fb40300e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T01:25:29.879 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 3372f51c-f7ca-4a3b-81ed-cc71b4b8006b 4 2024-04-25T01:25:29.936 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:29.956+0000 7f6756c0d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T01:25:29.962 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:29.982+0000 7fb40300e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T01:25:29.976 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.997+0000 7fc6965ed640 1 Processor -- start 2024-04-25T01:25:29.976 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.997+0000 7fc6965ed640 1 -- start start 2024-04-25T01:25:29.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.997+0000 7fc6965ed640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc6900a85d0 0x7fc690156650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.997+0000 7fc6965ed640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc690155b90 con 0x7fc690153310 2024-04-25T01:25:29.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.997+0000 7fc6965ed640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6900c8530 con 0x7fc69006df70 2024-04-25T01:25:29.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.997+0000 7fc6965ed640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc6900c7590 con 0x7fc6900a85d0 2024-04-25T01:25:29.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.998+0000 7fc694b8a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc6900a85d0 0x7fc690156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.998+0000 7fc694b8a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc6900a85d0 0x7fc690156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:57098/0 (socket says 172.21.15.187:57098) 2024-04-25T01:25:29.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.998+0000 7fc694b8a640 1 -- 172.21.15.187:0/2157594114 learned_addr learned my addr 172.21.15.187:0/2157594114 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T01:25:29.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.998+0000 7fc68f7fe640 1 --1- 172.21.15.187:0/2157594114 >> v1:172.21.15.187:6789/0 conn(0x7fc69006df70 0x7fc6900c59b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:52384/0 (socket says 172.21.15.187:52384) 2024-04-25T01:25:29.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.998+0000 7fc694b8a640 1 -- 172.21.15.187:0/2157594114 >> v1:172.21.15.187:6789/0 conn(0x7fc69006df70 legacy=0x7fc6900c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.998+0000 7fc694b8a640 1 -- 172.21.15.187:0/2157594114 >> v1:172.21.15.112:6789/0 conn(0x7fc690153310 legacy=0x7fc6900c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.979 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.998+0000 7fc694b8a640 1 -- 172.21.15.187:0/2157594114 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6900c8810 con 0x7fc6900a85d0 2024-04-25T01:25:29.980 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.998+0000 7fc694b8a640 1 --2- 172.21.15.187:0/2157594114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6900a85d0 0x7fc690156650 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc688009e50 tx=0x7fc68802a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e20dcb2a672664 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:29.980 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.999+0000 7fc68effd640 1 -- 172.21.15.187:0/2157594114 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc688036070 con 0x7fc6900a85d0 2024-04-25T01:25:29.980 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.999+0000 7fc68effd640 1 -- 172.21.15.187:0/2157594114 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc688031600 con 0x7fc6900a85d0 2024-04-25T01:25:29.980 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.999+0000 7fc68effd640 1 -- 172.21.15.187:0/2157594114 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6880319d0 con 0x7fc6900a85d0 2024-04-25T01:25:29.980 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.999+0000 7fc6965ed640 1 -- 172.21.15.187:0/2157594114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6900a85d0 msgr2=0x7fc690156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.980 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.999+0000 7fc6965ed640 1 --2- 172.21.15.187:0/2157594114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6900a85d0 0x7fc690156650 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc688009e50 tx=0x7fc68802a6a0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.980 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.999+0000 7fc6965ed640 1 -- 172.21.15.187:0/2157594114 shutdown_connections 2024-04-25T01:25:29.980 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.999+0000 7fc6965ed640 1 --2- 172.21.15.187:0/2157594114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6900a85d0 0x7fc690156650 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.981 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.999+0000 7fc6965ed640 1 -- 172.21.15.187:0/2157594114 >> 172.21.15.187:0/2157594114 conn(0x7fc6900c8f80 msgr2=0x7fc6900c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:29.981 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.999+0000 7fc6965ed640 1 -- 172.21.15.187:0/2157594114 shutdown_connections 2024-04-25T01:25:29.981 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:29.999+0000 7fc6965ed640 1 -- 172.21.15.187:0/2157594114 wait complete. 2024-04-25T01:25:29.981 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.000+0000 7fc6965ed640 1 Processor -- start 2024-04-25T01:25:29.981 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.001+0000 7fc6965ed640 1 -- start start 2024-04-25T01:25:29.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.001+0000 7fc6965ed640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc69006df70 0x7fc6900c2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.001+0000 7fc6965ed640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6900a85d0 0x7fc6900c1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.001+0000 7fc6965ed640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc690153310 0x7fc69014c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:29.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.001+0000 7fc6965ed640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc690156130 con 0x7fc69006df70 2024-04-25T01:25:29.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.001+0000 7fc6965ed640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc690155e50 con 0x7fc690153310 2024-04-25T01:25:29.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.001+0000 7fc6965ed640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc6900c60c0 con 0x7fc6900a85d0 2024-04-25T01:25:29.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.001+0000 7fc68f7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6900a85d0 0x7fc6900c1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.984 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.001+0000 7fc694b8a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc690153310 0x7fc69014c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.984 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.001+0000 7fc694b8a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc690153310 0x7fc69014c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:57106/0 (socket says 172.21.15.187:57106) 2024-04-25T01:25:29.984 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.001+0000 7fc694b8a640 1 -- 172.21.15.187:0/207452818 learned_addr learned my addr 172.21.15.187:0/207452818 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T01:25:29.984 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.001+0000 7fc68f7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6900a85d0 0x7fc6900c1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:53248/0 (socket says 172.21.15.187:53248) 2024-04-25T01:25:29.984 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.001+0000 7fc68ffff640 1 --2- 172.21.15.187:0/207452818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc69006df70 0x7fc6900c2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:29.984 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.002+0000 7fc694b8a640 1 -- 172.21.15.187:0/207452818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6900a85d0 msgr2=0x7fc6900c1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.984 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.002+0000 7fc694b8a640 1 --2- 172.21.15.187:0/207452818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6900a85d0 0x7fc6900c1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.984 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.002+0000 7fc694b8a640 1 -- 172.21.15.187:0/207452818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc69006df70 msgr2=0x7fc6900c2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:29.984 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.002+0000 7fc694b8a640 1 --2- 172.21.15.187:0/207452818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc69006df70 0x7fc6900c2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:29.984 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.002+0000 7fc694b8a640 1 -- 172.21.15.187:0/207452818 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6900c0a20 con 0x7fc690153310 2024-04-25T01:25:29.984 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.002+0000 7fc68f7fe640 1 --2- 172.21.15.187:0/207452818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6900a85d0 0x7fc6900c1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:25:29.984 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.002+0000 7fc694b8a640 1 --2- 172.21.15.187:0/207452818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc690153310 0x7fc69014c810 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc688034040 tx=0x7fc688004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:29.984 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.002+0000 7fc68cff9640 1 -- 172.21.15.187:0/207452818 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68803f020 con 0x7fc690153310 2024-04-25T01:25:29.985 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.002+0000 7fc68cff9640 1 -- 172.21.15.187:0/207452818 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc688037040 con 0x7fc690153310 2024-04-25T01:25:29.985 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.002+0000 7fc6965ed640 1 -- 172.21.15.187:0/207452818 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc690159c80 con 0x7fc690153310 2024-04-25T01:25:29.985 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.002+0000 7fc68cff9640 1 -- 172.21.15.187:0/207452818 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc688036070 con 0x7fc690153310 2024-04-25T01:25:29.985 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.002+0000 7fc6965ed640 1 -- 172.21.15.187:0/207452818 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6900c0340 con 0x7fc690153310 2024-04-25T01:25:29.985 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.002+0000 7fc68cff9640 1 -- 172.21.15.187:0/207452818 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc688042020 con 0x7fc690153310 2024-04-25T01:25:29.985 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.002+0000 7fc68cff9640 1 -- 172.21.15.187:0/207452818 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fc688044d10 con 0x7fc690153310 2024-04-25T01:25:29.985 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.003+0000 7fc6965ed640 1 -- 172.21.15.187:0/207452818 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc69004f500 con 0x7fc690153310 2024-04-25T01:25:29.986 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.007+0000 7fc68cff9640 1 -- 172.21.15.187:0/207452818 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc688049030 con 0x7fc690153310 2024-04-25T01:25:30.005 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:30.025+0000 7fb40300e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T01:25:30.029 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:30.049+0000 7f6756c0d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T01:25:30.058 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.079+0000 7fc6965ed640 1 -- 172.21.15.187:0/207452818 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3372f51c-f7ca-4a3b-81ed-cc71b4b8006b", "id": 4} v 0) -- 0x7fc690071480 con 0x7fc690153310 2024-04-25T01:25:30.064 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.084+0000 7fc68cff9640 1 -- 172.21.15.187:0/207452818 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3372f51c-f7ca-4a3b-81ed-cc71b4b8006b", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fc688007bd0 con 0x7fc690153310 2024-04-25T01:25:30.064 INFO:teuthology.orchestra.run.smithi187.stdout:4 2024-04-25T01:25:30.065 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.085+0000 7fc6965ed640 1 -- 172.21.15.187:0/207452818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc690153310 msgr2=0x7fc69014c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.065 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.085+0000 7fc6965ed640 1 --2- 172.21.15.187:0/207452818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc690153310 0x7fc69014c810 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc688034040 tx=0x7fc688004a40 comp rx=0 tx=0).stop 2024-04-25T01:25:30.065 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.085+0000 7fc6965ed640 1 -- 172.21.15.187:0/207452818 shutdown_connections 2024-04-25T01:25:30.066 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.085+0000 7fc6965ed640 1 --2- 172.21.15.187:0/207452818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc690153310 0x7fc69014c810 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.066 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.085+0000 7fc6965ed640 1 --2- 172.21.15.187:0/207452818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6900a85d0 0x7fc6900c1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.066 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.085+0000 7fc6965ed640 1 --2- 172.21.15.187:0/207452818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc69006df70 0x7fc6900c2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.066 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.085+0000 7fc6965ed640 1 -- 172.21.15.187:0/207452818 >> 172.21.15.187:0/207452818 conn(0x7fc6900c8f80 msgr2=0x7fc6900c3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:30.066 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.086+0000 7fc6965ed640 1 -- 172.21.15.187:0/207452818 shutdown_connections 2024-04-25T01:25:30.066 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.086+0000 7fc6965ed640 1 -- 172.21.15.187:0/207452818 wait complete. 2024-04-25T01:25:30.074 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:30.095+0000 7f6756c0d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T01:25:30.076 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new c7728c82-5f37-484c-ab71-f486b9bf8bc2 5 2024-04-25T01:25:30.171 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:30.191+0000 7fb40300e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T01:25:30.173 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.193+0000 7f888d6ee640 1 Processor -- start 2024-04-25T01:25:30.173 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.193+0000 7f888d6ee640 1 -- start start 2024-04-25T01:25:30.174 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.194+0000 7f888d6ee640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8888151260 0x7f88880c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:30.174 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.194+0000 7f888d6ee640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88880c8740 0x7f88880c7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:30.174 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.194+0000 7f888d6ee640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8888153c10 con 0x7f88880c5ef0 2024-04-25T01:25:30.174 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.194+0000 7f888d6ee640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f8888153ef0 con 0x7f8888151260 2024-04-25T01:25:30.174 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.194+0000 7f888d6ee640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f88881541d0 con 0x7f88880c8740 2024-04-25T01:25:30.174 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.194+0000 7f88877fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88880c8740 0x7f88880c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:30.174 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.194+0000 7f88877fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88880c8740 0x7f88880c7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:53258/0 (socket says 172.21.15.187:53258) 2024-04-25T01:25:30.175 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.194+0000 7f88877fe640 1 -- 172.21.15.187:0/2840949190 learned_addr learned my addr 172.21.15.187:0/2840949190 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T01:25:30.175 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.194+0000 7f8886ffd640 1 --2- 172.21.15.187:0/2840949190 >> v2:172.21.15.112:3300/0 conn(0x7f8888151260 0x7f88880c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:30.175 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.194+0000 7f88877fe640 1 -- 172.21.15.187:0/2840949190 >> v2:172.21.15.112:3300/0 conn(0x7f8888151260 msgr2=0x7f88880c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:30.175 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.194+0000 7f88877fe640 1 --2- 172.21.15.187:0/2840949190 >> v2:172.21.15.112:3300/0 conn(0x7f8888151260 0x7f88880c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.175 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.194+0000 7f88877fe640 1 -- 172.21.15.187:0/2840949190 >> v1:172.21.15.112:6789/0 conn(0x7f88880c5ef0 legacy=0x7f88880c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.175 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.194+0000 7f88877fe640 1 -- 172.21.15.187:0/2840949190 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88881544b0 con 0x7f88880c8740 2024-04-25T01:25:30.176 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.195+0000 7f88877fe640 1 --2- 172.21.15.187:0/2840949190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88880c8740 0x7f88880c7590 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8880009e50 tx=0x7f888002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4f0d266694ac074 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:30.176 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.195+0000 7f8885ffb640 1 -- 172.21.15.187:0/2840949190 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880036070 con 0x7f88880c8740 2024-04-25T01:25:30.176 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.195+0000 7f8885ffb640 1 -- 172.21.15.187:0/2840949190 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8880031600 con 0x7f88880c8740 2024-04-25T01:25:30.177 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.196+0000 7f8885ffb640 1 -- 172.21.15.187:0/2840949190 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88800319d0 con 0x7f88880c8740 2024-04-25T01:25:30.177 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.196+0000 7f888d6ee640 1 -- 172.21.15.187:0/2840949190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88880c8740 msgr2=0x7f88880c7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.177 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.196+0000 7f888d6ee640 1 --2- 172.21.15.187:0/2840949190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88880c8740 0x7f88880c7590 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8880009e50 tx=0x7f888002a710 comp rx=0 tx=0).stop 2024-04-25T01:25:30.177 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.196+0000 7f888d6ee640 1 -- 172.21.15.187:0/2840949190 shutdown_connections 2024-04-25T01:25:30.177 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.196+0000 7f888d6ee640 1 --2- 172.21.15.187:0/2840949190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88880c8740 0x7f88880c7590 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.177 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.196+0000 7f888d6ee640 1 --2- 172.21.15.187:0/2840949190 >> v2:172.21.15.112:3300/0 conn(0x7f8888151260 0x7f88880c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.177 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.196+0000 7f888d6ee640 1 -- 172.21.15.187:0/2840949190 >> 172.21.15.187:0/2840949190 conn(0x7f88880c8f80 msgr2=0x7f88880c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:30.177 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.196+0000 7f888d6ee640 1 -- 172.21.15.187:0/2840949190 shutdown_connections 2024-04-25T01:25:30.177 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.196+0000 7f888d6ee640 1 -- 172.21.15.187:0/2840949190 wait complete. 2024-04-25T01:25:30.180 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.197+0000 7f888d6ee640 1 Processor -- start 2024-04-25T01:25:30.180 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.197+0000 7f888d6ee640 1 -- start start 2024-04-25T01:25:30.180 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.198+0000 7f888d6ee640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88880c5ef0 0x7f88880c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:30.180 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.198+0000 7f888d6ee640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88880c8740 0x7f88881eecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:30.180 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.198+0000 7f888d6ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8888151260 0x7f88881ef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:30.180 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.198+0000 7f888d6ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8888157ab0 con 0x7f8888151260 2024-04-25T01:25:30.180 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.198+0000 7f888d6ee640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8888157930 con 0x7f88880c5ef0 2024-04-25T01:25:30.180 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.198+0000 7f888d6ee640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8888157c30 con 0x7f88880c8740 2024-04-25T01:25:30.180 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.198+0000 7f88867fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88880c8740 0x7f88881eecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:30.180 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.198+0000 7f8886ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88880c5ef0 0x7f88880c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:30.180 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.198+0000 7f8886ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88880c5ef0 0x7f88880c6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:57112/0 (socket says 172.21.15.187:57112) 2024-04-25T01:25:30.181 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.198+0000 7f88867fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88880c8740 0x7f88881eecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:53274/0 (socket says 172.21.15.187:53274) 2024-04-25T01:25:30.181 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.198+0000 7f8886ffd640 1 -- 172.21.15.187:0/1536715333 learned_addr learned my addr 172.21.15.187:0/1536715333 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T01:25:30.181 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.198+0000 7f88877fe640 1 --2- 172.21.15.187:0/1536715333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8888151260 0x7f88881ef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:30.181 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.198+0000 7f8886ffd640 1 -- 172.21.15.187:0/1536715333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88880c8740 msgr2=0x7f88881eecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.181 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.199+0000 7f8886ffd640 1 --2- 172.21.15.187:0/1536715333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88880c8740 0x7f88881eecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.181 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.199+0000 7f8886ffd640 1 -- 172.21.15.187:0/1536715333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8888151260 msgr2=0x7f88881ef1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.181 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.199+0000 7f8886ffd640 1 --2- 172.21.15.187:0/1536715333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8888151260 0x7f88881ef1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.181 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.199+0000 7f8886ffd640 1 -- 172.21.15.187:0/1536715333 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888806ce70 con 0x7f88880c5ef0 2024-04-25T01:25:30.181 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.199+0000 7f88867fc640 1 --2- 172.21.15.187:0/1536715333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88880c8740 0x7f88881eecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:25:30.181 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.199+0000 7f8886ffd640 1 --2- 172.21.15.187:0/1536715333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88880c5ef0 0x7f88880c6fd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f887c00e9b0 tx=0x7f887c00ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:30.181 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.199+0000 7f886f7fe640 1 -- 172.21.15.187:0/1536715333 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c017070 con 0x7f88880c5ef0 2024-04-25T01:25:30.181 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.199+0000 7f886f7fe640 1 -- 172.21.15.187:0/1536715333 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f887c00c9a0 con 0x7f88880c5ef0 2024-04-25T01:25:30.181 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.199+0000 7f886f7fe640 1 -- 172.21.15.187:0/1536715333 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c00cd90 con 0x7f88880c5ef0 2024-04-25T01:25:30.181 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.199+0000 7f888d6ee640 1 -- 172.21.15.187:0/1536715333 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f888806d150 con 0x7f88880c5ef0 2024-04-25T01:25:30.181 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.199+0000 7f888d6ee640 1 -- 172.21.15.187:0/1536715333 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88880c7920 con 0x7f88880c5ef0 2024-04-25T01:25:30.182 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.199+0000 7f886f7fe640 1 -- 172.21.15.187:0/1536715333 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f887c01a070 con 0x7f88880c5ef0 2024-04-25T01:25:30.182 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.199+0000 7f886f7fe640 1 -- 172.21.15.187:0/1536715333 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f887c01ca70 con 0x7f88880c5ef0 2024-04-25T01:25:30.182 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.200+0000 7f888d6ee640 1 -- 172.21.15.187:0/1536715333 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f888804f500 con 0x7f88880c5ef0 2024-04-25T01:25:30.182 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.202+0000 7f886f7fe640 1 -- 172.21.15.187:0/1536715333 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f887c01c4f0 con 0x7f88880c5ef0 2024-04-25T01:25:30.226 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:30.246+0000 7fb40300e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T01:25:30.249 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:30.270+0000 7f6756c0d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T01:25:30.254 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.274+0000 7f888d6ee640 1 -- 172.21.15.187:0/1536715333 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c7728c82-5f37-484c-ab71-f486b9bf8bc2", "id": 5} v 0) -- 0x7f88880c8000 con 0x7f88880c5ef0 2024-04-25T01:25:30.259 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.280+0000 7f886f7fe640 1 -- 172.21.15.187:0/1536715333 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c7728c82-5f37-484c-ab71-f486b9bf8bc2", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f887c01c7a0 con 0x7f88880c5ef0 2024-04-25T01:25:30.260 INFO:teuthology.orchestra.run.smithi187.stdout:5 2024-04-25T01:25:30.261 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.281+0000 7f888d6ee640 1 -- 172.21.15.187:0/1536715333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88880c5ef0 msgr2=0x7f88880c6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.261 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.281+0000 7f888d6ee640 1 --2- 172.21.15.187:0/1536715333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88880c5ef0 0x7f88880c6fd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f887c00e9b0 tx=0x7f887c00ee70 comp rx=0 tx=0).stop 2024-04-25T01:25:30.261 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.281+0000 7f888d6ee640 1 -- 172.21.15.187:0/1536715333 shutdown_connections 2024-04-25T01:25:30.261 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.281+0000 7f888d6ee640 1 --2- 172.21.15.187:0/1536715333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8888151260 0x7f88881ef1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.261 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.281+0000 7f888d6ee640 1 --2- 172.21.15.187:0/1536715333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88880c8740 0x7f88881eecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.261 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.281+0000 7f888d6ee640 1 --2- 172.21.15.187:0/1536715333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88880c5ef0 0x7f88880c6fd0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.261 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.281+0000 7f888d6ee640 1 -- 172.21.15.187:0/1536715333 >> 172.21.15.187:0/1536715333 conn(0x7f88880c8f80 msgr2=0x7f88880c3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:30.261 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.281+0000 7f888d6ee640 1 -- 172.21.15.187:0/1536715333 shutdown_connections 2024-04-25T01:25:30.261 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.281+0000 7f888d6ee640 1 -- 172.21.15.187:0/1536715333 wait complete. 2024-04-25T01:25:30.271 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 70148781-f925-44e3-a6f3-68b5f58b2b22 6 2024-04-25T01:25:30.279 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:30.299+0000 7fb40300e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T01:25:30.316 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:30.336+0000 7f6756c0d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T01:25:30.367 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.387+0000 7f99509ee640 1 Processor -- start 2024-04-25T01:25:30.367 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.387+0000 7f99509ee640 1 -- start start 2024-04-25T01:25:30.368 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.388+0000 7f99509ee640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f994c06df70 0x7f994c156650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:30.368 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.388+0000 7f99509ee640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f994c0a85d0 0x7f994c0c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:30.368 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.388+0000 7f99509ee640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f994c155b90 con 0x7f994c153310 2024-04-25T01:25:30.368 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.388+0000 7f99509ee640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f994c0c7590 con 0x7f994c06df70 2024-04-25T01:25:30.368 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.388+0000 7f99509ee640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f994c0c8530 con 0x7f994c0a85d0 2024-04-25T01:25:30.368 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.388+0000 7f9949d9b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f994c06df70 0x7f994c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:30.368 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.388+0000 7f994ad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f994c0a85d0 0x7f994c0c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:30.369 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.388+0000 7f9949d9b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f994c06df70 0x7f994c156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:57124/0 (socket says 172.21.15.187:57124) 2024-04-25T01:25:30.369 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.388+0000 7f9949d9b640 1 -- 172.21.15.187:0/3273377174 learned_addr learned my addr 172.21.15.187:0/3273377174 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T01:25:30.369 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.388+0000 7f994ad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f994c0a85d0 0x7f994c0c59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:53286/0 (socket says 172.21.15.187:53286) 2024-04-25T01:25:30.369 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.388+0000 7f994ad9d640 1 -- 172.21.15.187:0/3273377174 >> v2:172.21.15.187:3300/0 conn(0x7f994c06df70 msgr2=0x7f994c156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:30.369 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.388+0000 7f994ad9d640 1 --2- 172.21.15.187:0/3273377174 >> v2:172.21.15.187:3300/0 conn(0x7f994c06df70 0x7f994c156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.369 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.388+0000 7f994ad9d640 1 -- 172.21.15.187:0/3273377174 >> v1:172.21.15.112:6789/0 conn(0x7f994c153310 legacy=0x7f994c0c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.369 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.388+0000 7f994ad9d640 1 -- 172.21.15.187:0/3273377174 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c0c8810 con 0x7f994c0a85d0 2024-04-25T01:25:30.369 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.388+0000 7f9949d9b640 1 --2- 172.21.15.187:0/3273377174 >> v2:172.21.15.187:3300/0 conn(0x7f994c06df70 0x7f994c156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:25:30.370 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.389+0000 7f994ad9d640 1 --2- 172.21.15.187:0/3273377174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f994c0a85d0 0x7f994c0c59b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9944009d90 tx=0x7f994402a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e39491074f2c3c99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:30.370 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.389+0000 7f994959a640 1 -- 172.21.15.187:0/3273377174 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944007740 con 0x7f994c0a85d0 2024-04-25T01:25:30.370 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.389+0000 7f994959a640 1 -- 172.21.15.187:0/3273377174 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99440078e0 con 0x7f994c0a85d0 2024-04-25T01:25:30.371 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.389+0000 7f994959a640 1 -- 172.21.15.187:0/3273377174 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944007cb0 con 0x7f994c0a85d0 2024-04-25T01:25:30.371 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.390+0000 7f99509ee640 1 -- 172.21.15.187:0/3273377174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f994c0a85d0 msgr2=0x7f994c0c59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.371 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.390+0000 7f99509ee640 1 --2- 172.21.15.187:0/3273377174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f994c0a85d0 0x7f994c0c59b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9944009d90 tx=0x7f994402a530 comp rx=0 tx=0).stop 2024-04-25T01:25:30.371 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.390+0000 7f99509ee640 1 -- 172.21.15.187:0/3273377174 shutdown_connections 2024-04-25T01:25:30.371 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.390+0000 7f99509ee640 1 --2- 172.21.15.187:0/3273377174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f994c0a85d0 0x7f994c0c59b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.371 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.390+0000 7f99509ee640 1 --2- 172.21.15.187:0/3273377174 >> v2:172.21.15.187:3300/0 conn(0x7f994c06df70 0x7f994c156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.371 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.390+0000 7f99509ee640 1 -- 172.21.15.187:0/3273377174 >> 172.21.15.187:0/3273377174 conn(0x7f994c0c8f80 msgr2=0x7f994c0c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:30.371 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.390+0000 7f99509ee640 1 -- 172.21.15.187:0/3273377174 shutdown_connections 2024-04-25T01:25:30.371 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.390+0000 7f99509ee640 1 -- 172.21.15.187:0/3273377174 wait complete. 2024-04-25T01:25:30.371 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:30.390+0000 7f6756c0d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T01:25:30.374 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.391+0000 7f99509ee640 1 Processor -- start 2024-04-25T01:25:30.374 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.391+0000 7f99509ee640 1 -- start start 2024-04-25T01:25:30.374 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f99509ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f994c06df70 0x7f994c14fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:30.374 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f99509ee640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f994c0a85d0 0x7f994c157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:30.374 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f99509ee640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f994c153310 0x7f994c1589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:30.374 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f99509ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f994c156130 con 0x7f994c06df70 2024-04-25T01:25:30.374 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f99509ee640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f994c155e50 con 0x7f994c0a85d0 2024-04-25T01:25:30.374 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f99509ee640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f994c0c5ef0 con 0x7f994c153310 2024-04-25T01:25:30.374 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f994ad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f994c153310 0x7f994c1589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:30.374 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f9949d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f994c0a85d0 0x7f994c157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:30.375 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f994ad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f994c153310 0x7f994c1589d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:53296/0 (socket says 172.21.15.187:53296) 2024-04-25T01:25:30.375 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f994ad9d640 1 -- 172.21.15.187:0/437476898 learned_addr learned my addr 172.21.15.187:0/437476898 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T01:25:30.375 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f994a59c640 1 --2- 172.21.15.187:0/437476898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f994c06df70 0x7f994c14fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:30.375 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f9949d9b640 1 -- 172.21.15.187:0/437476898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f994c153310 msgr2=0x7f994c1589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.375 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f9949d9b640 1 --2- 172.21.15.187:0/437476898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f994c153310 0x7f994c1589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.375 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f9949d9b640 1 -- 172.21.15.187:0/437476898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f994c06df70 msgr2=0x7f994c14fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.375 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f9949d9b640 1 --2- 172.21.15.187:0/437476898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f994c06df70 0x7f994c14fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.375 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f9949d9b640 1 -- 172.21.15.187:0/437476898 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c1efea0 con 0x7f994c0a85d0 2024-04-25T01:25:30.375 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f994ad9d640 1 --2- 172.21.15.187:0/437476898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f994c153310 0x7f994c1589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:25:30.375 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f9949d9b640 1 --2- 172.21.15.187:0/437476898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f994c0a85d0 0x7f994c157660 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f994000e930 tx=0x7f994000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:30.375 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f993affd640 1 -- 172.21.15.187:0/437476898 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994000ca00 con 0x7f994c0a85d0 2024-04-25T01:25:30.375 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f993affd640 1 -- 172.21.15.187:0/437476898 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f994000cba0 con 0x7f994c0a85d0 2024-04-25T01:25:30.375 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f993affd640 1 -- 172.21.15.187:0/437476898 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940004030 con 0x7f994c0a85d0 2024-04-25T01:25:30.375 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.392+0000 7f99509ee640 1 -- 172.21.15.187:0/437476898 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994c1f0180 con 0x7f994c0a85d0 2024-04-25T01:25:30.375 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.393+0000 7f99509ee640 1 -- 172.21.15.187:0/437476898 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994c1502f0 con 0x7f994c0a85d0 2024-04-25T01:25:30.376 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.393+0000 7f993affd640 1 -- 172.21.15.187:0/437476898 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9940003610 con 0x7f994c0a85d0 2024-04-25T01:25:30.376 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.393+0000 7f993affd640 1 -- 172.21.15.187:0/437476898 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f9940013d80 con 0x7f994c0a85d0 2024-04-25T01:25:30.376 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.394+0000 7f99509ee640 1 -- 172.21.15.187:0/437476898 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f994c0689b0 con 0x7f994c0a85d0 2024-04-25T01:25:30.376 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.396+0000 7f993affd640 1 -- 172.21.15.187:0/437476898 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f994c0689b0 con 0x7f994c0a85d0 2024-04-25T01:25:30.390 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:30.410+0000 7fb40300e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:25:30.447 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.467+0000 7f99509ee640 1 -- 172.21.15.187:0/437476898 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "70148781-f925-44e3-a6f3-68b5f58b2b22", "id": 6} v 0) -- 0x7f994c04f500 con 0x7f994c0a85d0 2024-04-25T01:25:30.453 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.473+0000 7f993affd640 1 -- 172.21.15.187:0/437476898 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "70148781-f925-44e3-a6f3-68b5f58b2b22", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f9940017020 con 0x7f994c0a85d0 2024-04-25T01:25:30.453 INFO:teuthology.orchestra.run.smithi187.stdout:6 2024-04-25T01:25:30.454 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.474+0000 7f99509ee640 1 -- 172.21.15.187:0/437476898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f994c0a85d0 msgr2=0x7f994c157660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.454 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.474+0000 7f99509ee640 1 --2- 172.21.15.187:0/437476898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f994c0a85d0 0x7f994c157660 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f994000e930 tx=0x7f994000edf0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.454 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.474+0000 7f99509ee640 1 -- 172.21.15.187:0/437476898 shutdown_connections 2024-04-25T01:25:30.454 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.474+0000 7f99509ee640 1 --2- 172.21.15.187:0/437476898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f994c153310 0x7f994c1589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.454 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.474+0000 7f99509ee640 1 --2- 172.21.15.187:0/437476898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f994c0a85d0 0x7f994c157660 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.454 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.474+0000 7f99509ee640 1 --2- 172.21.15.187:0/437476898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f994c06df70 0x7f994c14fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.455 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.474+0000 7f99509ee640 1 -- 172.21.15.187:0/437476898 >> 172.21.15.187:0/437476898 conn(0x7f994c0c8f80 msgr2=0x7f994c156550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:30.455 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.474+0000 7f99509ee640 1 -- 172.21.15.187:0/437476898 shutdown_connections 2024-04-25T01:25:30.455 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.474+0000 7f99509ee640 1 -- 172.21.15.187:0/437476898 wait complete. 2024-04-25T01:25:30.464 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new e2f643a5-c4c5-439d-8b97-02a9d5343d94 7 2024-04-25T01:25:30.489 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:30.509+0000 7f6756c0d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:25:30.561 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.581+0000 7fd53bfc8640 1 Processor -- start 2024-04-25T01:25:30.561 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.581+0000 7fd53bfc8640 1 -- start start 2024-04-25T01:25:30.563 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.582+0000 7fd53bfc8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd534151220 0x7fd5340aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:30.563 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.582+0000 7fd53bfc8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd5340c38c0 0x7fd53406eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:30.563 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.582+0000 7fd53bfc8640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd534153bb0 con 0x7fd534154670 2024-04-25T01:25:30.564 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.583+0000 7fd53bfc8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd5340c3e80 con 0x7fd5340c38c0 2024-04-25T01:25:30.564 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.583+0000 7fd53bfc8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd534153e70 con 0x7fd534151220 2024-04-25T01:25:30.564 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.583+0000 7fd539d64640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd534151220 0x7fd5340aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:30.564 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.583+0000 7fd53a565640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd5340c38c0 0x7fd53406eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:30.564 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.583+0000 7fd539d64640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd534151220 0x7fd5340aea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:53312/0 (socket says 172.21.15.187:53312) 2024-04-25T01:25:30.564 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.583+0000 7fd53a565640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd5340c38c0 0x7fd53406eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:57148/0 (socket says 172.21.15.187:57148) 2024-04-25T01:25:30.564 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.583+0000 7fd539d64640 1 -- 172.21.15.187:0/720586931 learned_addr learned my addr 172.21.15.187:0/720586931 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T01:25:30.564 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.583+0000 7fd539d64640 1 -- 172.21.15.187:0/720586931 >> v2:172.21.15.187:3300/0 conn(0x7fd5340c38c0 msgr2=0x7fd53406eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:30.564 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.583+0000 7fd539d64640 1 --2- 172.21.15.187:0/720586931 >> v2:172.21.15.187:3300/0 conn(0x7fd5340c38c0 0x7fd53406eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.564 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.583+0000 7fd539d64640 1 -- 172.21.15.187:0/720586931 >> v1:172.21.15.112:6789/0 conn(0x7fd534154670 legacy=0x7fd5340c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.564 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.583+0000 7fd539d64640 1 -- 172.21.15.187:0/720586931 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd534154150 con 0x7fd534151220 2024-04-25T01:25:30.564 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.583+0000 7fd53a565640 1 --2- 172.21.15.187:0/720586931 >> v2:172.21.15.187:3300/0 conn(0x7fd5340c38c0 0x7fd53406eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:25:30.564 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.584+0000 7fd539d64640 1 --2- 172.21.15.187:0/720586931 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd534151220 0x7fd5340aea50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd524009870 tx=0x7fd52402a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12bec08507a80a53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:30.565 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.584+0000 7fd538d62640 1 -- 172.21.15.187:0/720586931 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524037070 con 0x7fd534151220 2024-04-25T01:25:30.565 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.584+0000 7fd538d62640 1 -- 172.21.15.187:0/720586931 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd524032440 con 0x7fd534151220 2024-04-25T01:25:30.565 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.584+0000 7fd538d62640 1 -- 172.21.15.187:0/720586931 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524032810 con 0x7fd534151220 2024-04-25T01:25:30.567 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.584+0000 7fd53bfc8640 1 -- 172.21.15.187:0/720586931 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd534151220 msgr2=0x7fd5340aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.568 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.584+0000 7fd53bfc8640 1 --2- 172.21.15.187:0/720586931 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd534151220 0x7fd5340aea50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd524009870 tx=0x7fd52402a130 comp rx=0 tx=0).stop 2024-04-25T01:25:30.568 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.584+0000 7fd53bfc8640 1 -- 172.21.15.187:0/720586931 shutdown_connections 2024-04-25T01:25:30.568 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.584+0000 7fd53bfc8640 1 --2- 172.21.15.187:0/720586931 >> v2:172.21.15.187:3300/0 conn(0x7fd5340c38c0 0x7fd53406eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.568 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.584+0000 7fd53bfc8640 1 --2- 172.21.15.187:0/720586931 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd534151220 0x7fd5340aea50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.568 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.584+0000 7fd53bfc8640 1 -- 172.21.15.187:0/720586931 >> 172.21.15.187:0/720586931 conn(0x7fd5340c8f40 msgr2=0x7fd5340c9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:30.568 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.585+0000 7fd53bfc8640 1 -- 172.21.15.187:0/720586931 shutdown_connections 2024-04-25T01:25:30.568 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.585+0000 7fd53bfc8640 1 -- 172.21.15.187:0/720586931 wait complete. 2024-04-25T01:25:30.568 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.586+0000 7fd53bfc8640 1 Processor -- start 2024-04-25T01:25:30.568 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.586+0000 7fd53bfc8640 1 -- start start 2024-04-25T01:25:30.568 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.586+0000 7fd53bfc8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5340c38c0 0x7fd534156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:30.568 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.586+0000 7fd53bfc8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd534151220 0x7fd53414c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:30.568 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.586+0000 7fd53bfc8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd534154670 0x7fd5341df5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:30.568 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.586+0000 7fd53bfc8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd53406f530 con 0x7fd534151220 2024-04-25T01:25:30.568 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.586+0000 7fd53bfc8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd5340c4d20 con 0x7fd534154670 2024-04-25T01:25:30.569 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.586+0000 7fd53bfc8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd53406f6b0 con 0x7fd5340c38c0 2024-04-25T01:25:30.569 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.586+0000 7fd539d64640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5340c38c0 0x7fd534156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:30.569 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.586+0000 7fd53a565640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd534154670 0x7fd5341df5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:30.569 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.587+0000 7fd53a565640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd534154670 0x7fd5341df5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:57158/0 (socket says 172.21.15.187:57158) 2024-04-25T01:25:30.569 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.587+0000 7fd53a565640 1 -- 172.21.15.187:0/1186860186 learned_addr learned my addr 172.21.15.187:0/1186860186 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T01:25:30.569 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.587+0000 7fd539563640 1 --2- 172.21.15.187:0/1186860186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd534151220 0x7fd53414c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:30.569 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.587+0000 7fd539d64640 1 -- 172.21.15.187:0/1186860186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd534154670 msgr2=0x7fd5341df5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.569 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.587+0000 7fd539d64640 1 --2- 172.21.15.187:0/1186860186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd534154670 0x7fd5341df5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.569 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.587+0000 7fd539d64640 1 -- 172.21.15.187:0/1186860186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd534151220 msgr2=0x7fd53414c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.569 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.587+0000 7fd539d64640 1 --2- 172.21.15.187:0/1186860186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd534151220 0x7fd53414c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.569 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.587+0000 7fd539d64640 1 -- 172.21.15.187:0/1186860186 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5341ed410 con 0x7fd5340c38c0 2024-04-25T01:25:30.569 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.587+0000 7fd539d64640 1 --2- 172.21.15.187:0/1186860186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5340c38c0 0x7fd534156b80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd524031000 tx=0x7fd5240046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:30.569 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.587+0000 7fd52a7fc640 1 -- 172.21.15.187:0/1186860186 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524041020 con 0x7fd5340c38c0 2024-04-25T01:25:30.569 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.587+0000 7fd52a7fc640 1 -- 172.21.15.187:0/1186860186 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd524030660 con 0x7fd5340c38c0 2024-04-25T01:25:30.569 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.587+0000 7fd52a7fc640 1 -- 172.21.15.187:0/1186860186 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524037040 con 0x7fd5340c38c0 2024-04-25T01:25:30.569 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.587+0000 7fd53bfc8640 1 -- 172.21.15.187:0/1186860186 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5341576d0 con 0x7fd5340c38c0 2024-04-25T01:25:30.570 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.587+0000 7fd53bfc8640 1 -- 172.21.15.187:0/1186860186 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5340ada50 con 0x7fd5340c38c0 2024-04-25T01:25:30.570 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.587+0000 7fd52a7fc640 1 -- 172.21.15.187:0/1186860186 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd524044020 con 0x7fd5340c38c0 2024-04-25T01:25:30.570 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.587+0000 7fd52a7fc640 1 -- 172.21.15.187:0/1186860186 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fd524030de0 con 0x7fd5340c38c0 2024-04-25T01:25:30.570 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.588+0000 7fd53bfc8640 1 -- 172.21.15.187:0/1186860186 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd53404f500 con 0x7fd5340c38c0 2024-04-25T01:25:30.570 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.591+0000 7fd52a7fc640 1 -- 172.21.15.187:0/1186860186 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd52404b030 con 0x7fd5340c38c0 2024-04-25T01:25:30.643 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.663+0000 7fd53bfc8640 1 -- 172.21.15.187:0/1186860186 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e2f643a5-c4c5-439d-8b97-02a9d5343d94", "id": 7} v 0) -- 0x7fd5340c5730 con 0x7fd5340c38c0 2024-04-25T01:25:30.651 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.671+0000 7fd52a7fc640 1 -- 172.21.15.187:0/1186860186 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e2f643a5-c4c5-439d-8b97-02a9d5343d94", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fd52403db50 con 0x7fd5340c38c0 2024-04-25T01:25:30.651 INFO:teuthology.orchestra.run.smithi187.stdout:7 2024-04-25T01:25:30.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.672+0000 7fd53bfc8640 1 -- 172.21.15.187:0/1186860186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5340c38c0 msgr2=0x7fd534156b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.672+0000 7fd53bfc8640 1 --2- 172.21.15.187:0/1186860186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5340c38c0 0x7fd534156b80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd524031000 tx=0x7fd5240046c0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.672+0000 7fd53bfc8640 1 -- 172.21.15.187:0/1186860186 shutdown_connections 2024-04-25T01:25:30.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.672+0000 7fd53bfc8640 1 --2- 172.21.15.187:0/1186860186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd534154670 0x7fd5341df5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.672+0000 7fd53bfc8640 1 --2- 172.21.15.187:0/1186860186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd534151220 0x7fd53414c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.653 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.672+0000 7fd53bfc8640 1 --2- 172.21.15.187:0/1186860186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5340c38c0 0x7fd534156b80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.653 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.672+0000 7fd53bfc8640 1 -- 172.21.15.187:0/1186860186 >> 172.21.15.187:0/1186860186 conn(0x7fd5340c8f40 msgr2=0x7fd53414f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:30.653 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.672+0000 7fd53bfc8640 1 -- 172.21.15.187:0/1186860186 shutdown_connections 2024-04-25T01:25:30.653 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:25:30.672+0000 7fd53bfc8640 1 -- 172.21.15.187:0/1186860186 wait complete. 2024-04-25T01:25:30.662 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T01:25:30.662 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T01:25:30.664 INFO:tasks.ceph.osd.0:Started 2024-04-25T01:25:30.665 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T01:25:30.665 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T01:25:30.668 INFO:tasks.ceph.osd.1:Started 2024-04-25T01:25:30.668 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T01:25:30.669 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T01:25:30.671 INFO:tasks.ceph.osd.2:Started 2024-04-25T01:25:30.671 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T01:25:30.671 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T01:25:30.673 INFO:tasks.ceph.osd.3:Started 2024-04-25T01:25:30.673 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T01:25:30.673 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T01:25:30.705 INFO:tasks.ceph.osd.4:Started 2024-04-25T01:25:30.705 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T01:25:30.705 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T01:25:30.707 INFO:tasks.ceph.osd.5:Started 2024-04-25T01:25:30.707 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T01:25:30.707 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T01:25:30.709 INFO:tasks.ceph.osd.6:Started 2024-04-25T01:25:30.709 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T01:25:30.709 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T01:25:30.712 INFO:tasks.ceph.osd.7:Started 2024-04-25T01:25:30.712 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:25:30.797 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:30.817+0000 7fb40300e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T01:25:30.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.843+0000 7f2e24770640 1 Processor -- start 2024-04-25T01:25:30.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.843+0000 7f2e24770640 1 -- start start 2024-04-25T01:25:30.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.844+0000 7f2e24770640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2e1c1596a0 0x7f2e1c152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:30.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.844+0000 7f2e24770640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2e1c0c72b0 0x7f2e1c160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:30.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.844+0000 7f2e24770640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e1c159b90 con 0x7f2e1c1592c0 2024-04-25T01:25:30.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.844+0000 7f2e24770640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f2e1c0c6e20 con 0x7f2e1c1596a0 2024-04-25T01:25:30.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.844+0000 7f2e24770640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2e1c06c970 con 0x7f2e1c0c72b0 2024-04-25T01:25:30.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.844+0000 7f2e21d0b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2e1c1596a0 0x7f2e1c152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:30.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.844+0000 7f2e2250c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2e1c1592c0 0x7f2e1c0c9070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:41478/0 (socket says 172.21.15.112:41478) 2024-04-25T01:25:30.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.844+0000 7f2e2250c640 1 -- 172.21.15.112:0/4265174311 learned_addr learned my addr 172.21.15.112:0/4265174311 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:25:30.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.844+0000 7f2e22d0d640 1 --2- 172.21.15.112:0/4265174311 >> v2:172.21.15.187:3300/0 conn(0x7f2e1c0c72b0 0x7f2e1c160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:30.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.845+0000 7f2e21d0b640 1 -- 172.21.15.112:0/4265174311 >> v2:172.21.15.187:3300/0 conn(0x7f2e1c0c72b0 msgr2=0x7f2e1c160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:30.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.845+0000 7f2e21d0b640 1 --2- 172.21.15.112:0/4265174311 >> v2:172.21.15.187:3300/0 conn(0x7f2e1c0c72b0 0x7f2e1c160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.845+0000 7f2e21d0b640 1 -- 172.21.15.112:0/4265174311 >> v1:172.21.15.112:6789/0 conn(0x7f2e1c1592c0 legacy=0x7f2e1c0c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.845+0000 7f2e21d0b640 1 -- 172.21.15.112:0/4265174311 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1c06cc50 con 0x7f2e1c1596a0 2024-04-25T01:25:30.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.845+0000 7f2e21d0b640 1 --2- 172.21.15.112:0/4265174311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e1c1596a0 0x7f2e1c152d00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0c21d0 tx=0x7f2e1802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14179512e01a26c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:30.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.845+0000 7f2e2150a640 1 -- 172.21.15.112:0/4265174311 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e18037070 con 0x7f2e1c1596a0 2024-04-25T01:25:30.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.846+0000 7f2e2150a640 1 -- 172.21.15.112:0/4265174311 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e1802add0 con 0x7f2e1c1596a0 2024-04-25T01:25:30.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.846+0000 7f2e2150a640 1 -- 172.21.15.112:0/4265174311 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e18032660 con 0x7f2e1c1596a0 2024-04-25T01:25:30.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.846+0000 7f2e24770640 1 -- 172.21.15.112:0/4265174311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e1c1596a0 msgr2=0x7f2e1c152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.846+0000 7f2e24770640 1 --2- 172.21.15.112:0/4265174311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e1c1596a0 0x7f2e1c152d00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0c21d0 tx=0x7f2e1802a080 comp rx=0 tx=0).stop 2024-04-25T01:25:30.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.846+0000 7f2e24770640 1 -- 172.21.15.112:0/4265174311 shutdown_connections 2024-04-25T01:25:30.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.846+0000 7f2e24770640 1 --2- 172.21.15.112:0/4265174311 >> v2:172.21.15.187:3300/0 conn(0x7f2e1c0c72b0 0x7f2e1c160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.846+0000 7f2e24770640 1 --2- 172.21.15.112:0/4265174311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e1c1596a0 0x7f2e1c152d00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.846+0000 7f2e24770640 1 -- 172.21.15.112:0/4265174311 >> 172.21.15.112:0/4265174311 conn(0x7f2e1c0cb170 msgr2=0x7f2e1c0cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:30.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.846+0000 7f2e24770640 1 -- 172.21.15.112:0/4265174311 shutdown_connections 2024-04-25T01:25:30.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.846+0000 7f2e24770640 1 -- 172.21.15.112:0/4265174311 wait complete. 2024-04-25T01:25:30.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.847+0000 7f2e24770640 1 Processor -- start 2024-04-25T01:25:30.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.847+0000 7f2e24770640 1 -- start start 2024-04-25T01:25:30.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.848+0000 7f2e24770640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e1c0c72b0 0x7f2e1c1ecb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:30.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.848+0000 7f2e24770640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e1c1592c0 0x7f2e1c0c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:30.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.848+0000 7f2e24770640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e1c1596a0 0x7f2e1c0c3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:30.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.848+0000 7f2e24770640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c06c510 con 0x7f2e1c1592c0 2024-04-25T01:25:30.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.848+0000 7f2e24770640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c06c390 con 0x7f2e1c1596a0 2024-04-25T01:25:30.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.848+0000 7f2e24770640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2e1c06db20 con 0x7f2e1c0c72b0 2024-04-25T01:25:30.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.848+0000 7f2e21d0b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e1c1592c0 0x7f2e1c0c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:30.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.848+0000 7f2e21d0b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e1c1592c0 0x7f2e1c0c4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37518/0 (socket says 172.21.15.112:37518) 2024-04-25T01:25:30.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.848+0000 7f2e21d0b640 1 -- 172.21.15.112:0/3084412032 learned_addr learned my addr 172.21.15.112:0/3084412032 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:30.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.848+0000 7f2e22d0d640 1 --2- 172.21.15.112:0/3084412032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e1c1596a0 0x7f2e1c0c3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:30.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.848+0000 7f2e2250c640 1 --2- 172.21.15.112:0/3084412032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e1c0c72b0 0x7f2e1c1ecb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:30.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.848+0000 7f2e21d0b640 1 -- 172.21.15.112:0/3084412032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e1c0c72b0 msgr2=0x7f2e1c1ecb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.848+0000 7f2e21d0b640 1 --2- 172.21.15.112:0/3084412032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e1c0c72b0 0x7f2e1c1ecb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.848+0000 7f2e21d0b640 1 -- 172.21.15.112:0/3084412032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e1c1596a0 msgr2=0x7f2e1c0c3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.848+0000 7f2e21d0b640 1 --2- 172.21.15.112:0/3084412032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e1c1596a0 0x7f2e1c0c3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.848+0000 7f2e21d0b640 1 -- 172.21.15.112:0/3084412032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1c0c23e0 con 0x7f2e1c1592c0 2024-04-25T01:25:30.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.848+0000 7f2e21d0b640 1 --2- 172.21.15.112:0/3084412032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e1c1592c0 0x7f2e1c0c4b60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2e18000c00 tx=0x7f2e180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:30.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.849+0000 7f2e12ffd640 1 -- 172.21.15.112:0/3084412032 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e18037040 con 0x7f2e1c1592c0 2024-04-25T01:25:30.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.849+0000 7f2e12ffd640 1 -- 172.21.15.112:0/3084412032 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e18042070 con 0x7f2e1c1592c0 2024-04-25T01:25:30.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.849+0000 7f2e12ffd640 1 -- 172.21.15.112:0/3084412032 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1803d660 con 0x7f2e1c1592c0 2024-04-25T01:25:30.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.849+0000 7f2e24770640 1 -- 172.21.15.112:0/3084412032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e1c0c26c0 con 0x7f2e1c1592c0 2024-04-25T01:25:30.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.849+0000 7f2e24770640 1 -- 172.21.15.112:0/3084412032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e1c0c2c80 con 0x7f2e1c1592c0 2024-04-25T01:25:30.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.849+0000 7f2e12ffd640 1 -- 172.21.15.112:0/3084412032 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2e1803d800 con 0x7f2e1c1592c0 2024-04-25T01:25:30.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.849+0000 7f2e12ffd640 1 -- 172.21.15.112:0/3084412032 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f2e1803b3f0 con 0x7f2e1c1592c0 2024-04-25T01:25:30.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.850+0000 7f2e24770640 1 -- 172.21.15.112:0/3084412032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e1c0679a0 con 0x7f2e1c1592c0 2024-04-25T01:25:30.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.852+0000 7f2e12ffd640 1 -- 172.21.15.112:0/3084412032 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2e1803dc80 con 0x7f2e1c1592c0 2024-04-25T01:25:30.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.913+0000 7f2e24770640 1 -- 172.21.15.112:0/3084412032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e1c0682a0 con 0x7f2e1c1592c0 2024-04-25T01:25:30.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.913+0000 7f2e12ffd640 1 -- 172.21.15.112:0/3084412032 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f2e1803b690 con 0x7f2e1c1592c0 2024-04-25T01:25:30.893 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:25:30.893 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":10,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:25:30.665595+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:25:30.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.914+0000 7f2e24770640 1 -- 172.21.15.112:0/3084412032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e1c1592c0 msgr2=0x7f2e1c0c4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:30.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.914+0000 7f2e24770640 1 --2- 172.21.15.112:0/3084412032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e1c1592c0 0x7f2e1c0c4b60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2e18000c00 tx=0x7f2e180046c0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.914+0000 7f2e24770640 1 -- 172.21.15.112:0/3084412032 shutdown_connections 2024-04-25T01:25:30.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.914+0000 7f2e24770640 1 --2- 172.21.15.112:0/3084412032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e1c1596a0 0x7f2e1c0c3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.914+0000 7f2e24770640 1 --2- 172.21.15.112:0/3084412032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e1c1592c0 0x7f2e1c0c4b60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.914+0000 7f2e24770640 1 --2- 172.21.15.112:0/3084412032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e1c0c72b0 0x7f2e1c1ecb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:30.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.914+0000 7f2e24770640 1 -- 172.21.15.112:0/3084412032 >> 172.21.15.112:0/3084412032 conn(0x7f2e1c0cb170 msgr2=0x7f2e1c0ca510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:30.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.914+0000 7f2e24770640 1 -- 172.21.15.112:0/3084412032 shutdown_connections 2024-04-25T01:25:30.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:30.914+0000 7f2e24770640 1 -- 172.21.15.112:0/3084412032 wait complete. 2024-04-25T01:25:30.905 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T01:25:30.905 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T01:25:30.915 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:30.935+0000 7f6756c0d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T01:25:31.064 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:31.085+0000 7fb40300e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T01:25:31.116 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:31.136+0000 7fb40300e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T01:25:31.174 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:31.195+0000 7f6756c0d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T01:25:31.226 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:31.247+0000 7f6756c0d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T01:25:31.230 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:31.250+0000 7fb40300e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T01:25:31.272 INFO:tasks.ceph.osd.1.smithi112.stderr:2024-04-25T01:25:31.293+0000 7f0bdca9d640 -1 Falling back to public interface 2024-04-25T01:25:31.277 INFO:tasks.ceph.osd.3.smithi112.stderr:2024-04-25T01:25:31.297+0000 7f81a280f640 -1 Falling back to public interface 2024-04-25T01:25:31.285 INFO:tasks.ceph.osd.0.smithi112.stderr:2024-04-25T01:25:31.305+0000 7f90d2219640 -1 Falling back to public interface 2024-04-25T01:25:31.293 INFO:tasks.ceph.osd.2.smithi112.stderr:2024-04-25T01:25:31.313+0000 7f3a16b57640 -1 Falling back to public interface 2024-04-25T01:25:31.307 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-04-25T01:25:31.327+0000 7fbe9e225640 -1 Falling back to public interface 2024-04-25T01:25:31.311 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-04-25T01:25:31.331+0000 7f46befa1640 -1 Falling back to public interface 2024-04-25T01:25:31.315 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-04-25T01:25:31.335+0000 7f1432610640 -1 Falling back to public interface 2024-04-25T01:25:31.320 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-04-25T01:25:31.340+0000 7f438a1be640 -1 Falling back to public interface 2024-04-25T01:25:31.348 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:31.368+0000 7f6756c0d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T01:25:31.512 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:31.533+0000 7fb40300e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T01:25:31.562 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:31.582+0000 7fb40300e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T01:25:31.629 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:31.649+0000 7f6756c0d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T01:25:31.676 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:31.696+0000 7f6756c0d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T01:25:31.710 DEBUG:teuthology.orchestra.run.smithi112:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T01:25:31.717 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:31.737+0000 7fb40300e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T01:25:31.799 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.819+0000 7fa060790640 1 Processor -- start 2024-04-25T01:25:31.799 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.819+0000 7fa060790640 1 -- start start 2024-04-25T01:25:31.800 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.820+0000 7fa060790640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa0580ae840 0x7fa0580c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:31.800 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.820+0000 7fa060790640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa0580aef10 0x7fa058156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:31.800 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.820+0000 7fa060790640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0580c60c0 0x7fa0580c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:31.801 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.820+0000 7fa060790640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa0580c52a0 con 0x7fa0580aef10 2024-04-25T01:25:31.801 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.820+0000 7fa060790640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa0580c5580 con 0x7fa0580ae840 2024-04-25T01:25:31.801 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.820+0000 7fa060790640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa0580c5860 con 0x7fa0580c60c0 2024-04-25T01:25:31.801 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.820+0000 7fa05dd2b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa0580aef10 0x7fa058156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:31.801 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.820+0000 7fa05dd2b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa0580aef10 0x7fa058156cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37534/0 (socket says 172.21.15.112:37534) 2024-04-25T01:25:31.801 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.820+0000 7fa05dd2b640 1 -- 172.21.15.112:0/3074595802 learned_addr learned my addr 172.21.15.112:0/3074595802 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:31.801 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.820+0000 7fa05ed2d640 1 --2- 172.21.15.112:0/3074595802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0580c60c0 0x7fa0580c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:31.801 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.820+0000 7fa05e52c640 1 --2- 172.21.15.112:0/3074595802 >> v2:172.21.15.187:3300/0 conn(0x7fa0580ae840 0x7fa0580c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:31.801 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.820+0000 7fa05dd2b640 1 -- 172.21.15.112:0/3074595802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0580c60c0 msgr2=0x7fa0580c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:31.801 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.820+0000 7fa05dd2b640 1 --2- 172.21.15.112:0/3074595802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0580c60c0 0x7fa0580c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:31.801 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.820+0000 7fa05dd2b640 1 -- 172.21.15.112:0/3074595802 >> v2:172.21.15.187:3300/0 conn(0x7fa0580ae840 msgr2=0x7fa0580c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:31.801 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.820+0000 7fa05dd2b640 1 --2- 172.21.15.112:0/3074595802 >> v2:172.21.15.187:3300/0 conn(0x7fa0580ae840 0x7fa0580c5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:31.801 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.820+0000 7fa05dd2b640 1 -- 172.21.15.112:0/3074595802 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa058071120 con 0x7fa0580aef10 2024-04-25T01:25:31.802 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.820+0000 7fa05e52c640 1 --2- 172.21.15.112:0/3074595802 >> v2:172.21.15.187:3300/0 conn(0x7fa0580ae840 0x7fa0580c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:25:31.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.821+0000 7fa05dd2b640 1 --2- 172.21.15.112:0/3074595802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0580aef10 0x7fa058156cb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa0580c6710 tx=0x7fa04802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a7cc0a7debbc12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:31.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.821+0000 7fa05d52a640 1 -- 172.21.15.112:0/3074595802 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048037070 con 0x7fa0580aef10 2024-04-25T01:25:31.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.821+0000 7fa05d52a640 1 -- 172.21.15.112:0/3074595802 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa048032440 con 0x7fa0580aef10 2024-04-25T01:25:31.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.821+0000 7fa05d52a640 1 -- 172.21.15.112:0/3074595802 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048032810 con 0x7fa0580aef10 2024-04-25T01:25:31.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.822+0000 7fa060790640 1 -- 172.21.15.112:0/3074595802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0580aef10 msgr2=0x7fa058156cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:31.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.822+0000 7fa060790640 1 --2- 172.21.15.112:0/3074595802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0580aef10 0x7fa058156cb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa0580c6710 tx=0x7fa04802a140 comp rx=0 tx=0).stop 2024-04-25T01:25:31.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.822+0000 7fa060790640 1 -- 172.21.15.112:0/3074595802 shutdown_connections 2024-04-25T01:25:31.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.822+0000 7fa060790640 1 --2- 172.21.15.112:0/3074595802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0580c60c0 0x7fa0580c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:31.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.822+0000 7fa060790640 1 --2- 172.21.15.112:0/3074595802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0580aef10 0x7fa058156cb0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:31.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.822+0000 7fa060790640 1 --2- 172.21.15.112:0/3074595802 >> v2:172.21.15.187:3300/0 conn(0x7fa0580ae840 0x7fa0580c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:31.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.822+0000 7fa060790640 1 -- 172.21.15.112:0/3074595802 >> 172.21.15.112:0/3074595802 conn(0x7fa0580c91d0 msgr2=0x7fa0580c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:31.806 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.822+0000 7fa060790640 1 -- 172.21.15.112:0/3074595802 shutdown_connections 2024-04-25T01:25:31.806 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.822+0000 7fa060790640 1 -- 172.21.15.112:0/3074595802 wait complete. 2024-04-25T01:25:31.806 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.823+0000 7fa060790640 1 Processor -- start 2024-04-25T01:25:31.806 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.823+0000 7fa060790640 1 -- start start 2024-04-25T01:25:31.806 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.824+0000 7fa060790640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0580ae840 0x7fa0581ec290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:31.806 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.824+0000 7fa060790640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0580aef10 0x7fa05814f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:31.806 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.824+0000 7fa060790640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0580c60c0 0x7fa0581e9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:31.806 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.824+0000 7fa060790640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa05806f530 con 0x7fa0580ae840 2024-04-25T01:25:31.806 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.824+0000 7fa060790640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa0580c4ba0 con 0x7fa0580aef10 2024-04-25T01:25:31.806 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.824+0000 7fa060790640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa05806f6b0 con 0x7fa0580c60c0 2024-04-25T01:25:31.806 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.824+0000 7fa05e52c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0580ae840 0x7fa0581ec290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:31.806 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.824+0000 7fa05e52c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0580ae840 0x7fa0581ec290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37540/0 (socket says 172.21.15.112:37540) 2024-04-25T01:25:31.806 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.824+0000 7fa05e52c640 1 -- 172.21.15.112:0/3221271495 learned_addr learned my addr 172.21.15.112:0/3221271495 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:31.807 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.824+0000 7fa05dd2b640 1 --2- 172.21.15.112:0/3221271495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0580aef10 0x7fa05814f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:31.807 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.824+0000 7fa05ed2d640 1 --2- 172.21.15.112:0/3221271495 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0580c60c0 0x7fa0581e9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:31.807 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.824+0000 7fa05e52c640 1 -- 172.21.15.112:0/3221271495 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0580c60c0 msgr2=0x7fa0581e9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:31.807 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.824+0000 7fa05e52c640 1 --2- 172.21.15.112:0/3221271495 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0580c60c0 0x7fa0581e9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:31.807 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.824+0000 7fa05e52c640 1 -- 172.21.15.112:0/3221271495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0580aef10 msgr2=0x7fa05814f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:31.807 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.824+0000 7fa05e52c640 1 --2- 172.21.15.112:0/3221271495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0580aef10 0x7fa05814f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:31.807 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.824+0000 7fa05e52c640 1 -- 172.21.15.112:0/3221271495 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0581e9360 con 0x7fa0580ae840 2024-04-25T01:25:31.807 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.824+0000 7fa05e52c640 1 --2- 172.21.15.112:0/3221271495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0580ae840 0x7fa0581ec290 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa05400d7d0 tx=0x7fa05400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:31.807 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.824+0000 7fa04effd640 1 -- 172.21.15.112:0/3221271495 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054004030 con 0x7fa0580ae840 2024-04-25T01:25:31.807 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.825+0000 7fa04effd640 1 -- 172.21.15.112:0/3221271495 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0540041d0 con 0x7fa0580ae840 2024-04-25T01:25:31.807 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.825+0000 7fa04effd640 1 -- 172.21.15.112:0/3221271495 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05400b6c0 con 0x7fa0580ae840 2024-04-25T01:25:31.807 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.825+0000 7fa060790640 1 -- 172.21.15.112:0/3221271495 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa058145330 con 0x7fa0580ae840 2024-04-25T01:25:31.807 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.825+0000 7fa060790640 1 -- 172.21.15.112:0/3221271495 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa058145050 con 0x7fa0580ae840 2024-04-25T01:25:31.807 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.825+0000 7fa04effd640 1 -- 172.21.15.112:0/3221271495 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa05400b860 con 0x7fa0580ae840 2024-04-25T01:25:31.807 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.825+0000 7fa04effd640 1 -- 172.21.15.112:0/3221271495 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fa054003d90 con 0x7fa0580ae840 2024-04-25T01:25:31.808 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.825+0000 7fa060790640 1 -- 172.21.15.112:0/3221271495 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa058075800 con 0x7fa0580ae840 2024-04-25T01:25:31.808 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.827+0000 7fa04effd640 1 -- 172.21.15.112:0/3221271495 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa05400bc90 con 0x7fa0580ae840 2024-04-25T01:25:31.832 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:31.853+0000 7f6756c0d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T01:25:31.860 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:31.880+0000 7fb40300e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T01:25:31.868 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.889+0000 7fa060790640 1 -- 172.21.15.112:0/3221271495 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0580c4770 con 0x7fa0580ae840 2024-04-25T01:25:31.869 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.889+0000 7fa04effd640 1 -- 172.21.15.112:0/3221271495 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fa054014b50 con 0x7fa0580ae840 2024-04-25T01:25:31.869 INFO:teuthology.misc.health.smithi112.stdout: 2024-04-25T01:25:31.869 INFO:teuthology.misc.health.smithi112.stdout:{"epoch":10,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:25:30.665595+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:25:31.870 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.890+0000 7fa060790640 1 -- 172.21.15.112:0/3221271495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0580ae840 msgr2=0x7fa0581ec290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:31.870 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.890+0000 7fa060790640 1 --2- 172.21.15.112:0/3221271495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0580ae840 0x7fa0581ec290 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa05400d7d0 tx=0x7fa05400dc90 comp rx=0 tx=0).stop 2024-04-25T01:25:31.870 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.890+0000 7fa060790640 1 -- 172.21.15.112:0/3221271495 shutdown_connections 2024-04-25T01:25:31.870 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.890+0000 7fa060790640 1 --2- 172.21.15.112:0/3221271495 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0580c60c0 0x7fa0581e9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:31.870 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.890+0000 7fa060790640 1 --2- 172.21.15.112:0/3221271495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0580aef10 0x7fa05814f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:31.870 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.890+0000 7fa060790640 1 --2- 172.21.15.112:0/3221271495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0580ae840 0x7fa0581ec290 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:31.870 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.890+0000 7fa060790640 1 -- 172.21.15.112:0/3221271495 >> 172.21.15.112:0/3221271495 conn(0x7fa0580c91d0 msgr2=0x7fa0580c38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:31.870 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.890+0000 7fa060790640 1 -- 172.21.15.112:0/3221271495 shutdown_connections 2024-04-25T01:25:31.871 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:31.890+0000 7fa060790640 1 -- 172.21.15.112:0/3221271495 wait complete. 2024-04-25T01:25:31.879 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T01:25:31.912 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:31.932+0000 7fb40300e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T01:25:31.970 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:31.990+0000 7f6756c0d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T01:25:31.982 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:32.002+0000 7fb40300e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T01:25:32.019 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:32.039+0000 7f6756c0d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T01:25:32.026 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:32.047+0000 7fb40300e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T01:25:32.087 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:32.108+0000 7f6756c0d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T01:25:32.132 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:32.153+0000 7f6756c0d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T01:25:32.501 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:32.521+0000 7fb40300e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T01:25:32.604 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:32.625+0000 7f6756c0d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T01:25:32.615 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:32.636+0000 7fb40300e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T01:25:32.725 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T01:25:32.746+0000 7f6756c0d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T01:25:33.630 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:33.650+0000 7fb3a37c1640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T01:25:33.632 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T01:25:33.653+0000 7fb3a37c1640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T01:25:34.752 INFO:tasks.ceph.osd.0.smithi112.stderr:2024-04-25T01:25:34.773+0000 7f90d2219640 -1 osd.0 0 log_to_monitors true 2024-04-25T01:25:34.777 INFO:tasks.ceph.osd.3.smithi112.stderr:2024-04-25T01:25:34.798+0000 7f81a280f640 -1 osd.3 0 log_to_monitors true 2024-04-25T01:25:34.778 INFO:tasks.ceph.osd.1.smithi112.stderr:2024-04-25T01:25:34.799+0000 7f0bdca9d640 -1 osd.1 0 log_to_monitors true 2024-04-25T01:25:34.787 INFO:tasks.ceph.osd.2.smithi112.stderr:2024-04-25T01:25:34.808+0000 7f3a16b57640 -1 osd.2 0 log_to_monitors true 2024-04-25T01:25:34.798 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-04-25T01:25:34.819+0000 7fbe9e225640 -1 osd.4 0 log_to_monitors true 2024-04-25T01:25:34.867 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-04-25T01:25:34.888+0000 7f46befa1640 -1 osd.6 0 log_to_monitors true 2024-04-25T01:25:34.873 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-04-25T01:25:34.893+0000 7f438a1be640 -1 osd.5 0 log_to_monitors true 2024-04-25T01:25:34.878 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-04-25T01:25:34.898+0000 7f1432610640 -1 osd.7 0 log_to_monitors true 2024-04-25T01:25:38.663 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-04-25T01:25:38.683+0000 7f46ba714640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T01:25:38.676 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-04-25T01:25:38.697+0000 7f46b2d46640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:25:38.686 DEBUG:teuthology.orchestra.run.smithi112:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T01:25:38.702 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-04-25T01:25:38.723+0000 7fbe99998640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T01:25:38.706 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-04-25T01:25:38.727+0000 7f142d67a640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T01:25:38.715 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-04-25T01:25:38.736+0000 7fbe91fca640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:25:38.720 INFO:tasks.ceph.osd.3.smithi112.stderr:2024-04-25T01:25:38.741+0000 7f819d9dc640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T01:25:38.723 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-04-25T01:25:38.744+0000 7f1425cac640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:25:38.724 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-04-25T01:25:38.745+0000 7f4385931640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T01:25:38.732 INFO:tasks.ceph.osd.3.smithi112.stderr:2024-04-25T01:25:38.753+0000 7f819600e640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:25:38.736 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-04-25T01:25:38.757+0000 7f437df63640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:25:38.738 INFO:tasks.ceph.osd.0.smithi112.stderr:2024-04-25T01:25:38.758+0000 7f90ceb41640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T01:25:38.739 INFO:tasks.ceph.osd.1.smithi112.stderr:2024-04-25T01:25:38.759+0000 7f0bd8210640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T01:25:38.746 INFO:tasks.ceph.osd.0.smithi112.stderr:2024-04-25T01:25:38.767+0000 7f90c593b640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:25:38.748 INFO:tasks.ceph.osd.2.smithi112.stderr:2024-04-25T01:25:38.769+0000 7f3a122ca640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T01:25:38.750 INFO:tasks.ceph.osd.1.smithi112.stderr:2024-04-25T01:25:38.771+0000 7f0bd0842640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:25:38.765 INFO:tasks.ceph.osd.2.smithi112.stderr:2024-04-25T01:25:38.786+0000 7f3a0a8fc640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:25:38.797 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.817+0000 7f74a406b640 1 Processor -- start 2024-04-25T01:25:38.797 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.817+0000 7f74a406b640 1 -- start start 2024-04-25T01:25:38.799 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.818+0000 7f74a406b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f749c0b09f0 0x7f749c06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:38.799 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.818+0000 7f74a406b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f749c0c90c0 0x7f749c0c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:38.799 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.818+0000 7f74a406b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f749c0c5d60 con 0x7f749c0b0610 2024-04-25T01:25:38.799 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.818+0000 7f74a406b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f749c155c60 con 0x7f749c0c90c0 2024-04-25T01:25:38.799 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.818+0000 7f74a406b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f749c0c6040 con 0x7f749c0b09f0 2024-04-25T01:25:38.799 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.818+0000 7f74a1e07640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f749c0b0610 0x7f749c1560f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54786/0 (socket says 172.21.15.112:54786) 2024-04-25T01:25:38.799 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.818+0000 7f74a1e07640 1 -- 172.21.15.112:0/2724087362 learned_addr learned my addr 172.21.15.112:0/2724087362 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:25:38.799 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.818+0000 7f74a2608640 1 --2- 172.21.15.112:0/2724087362 >> v2:172.21.15.112:3300/0 conn(0x7f749c0c90c0 0x7f749c0c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:38.799 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.818+0000 7f74a1606640 1 --2- 172.21.15.112:0/2724087362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f749c0b09f0 0x7f749c06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:38.799 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.818+0000 7f74a2608640 1 -- 172.21.15.112:0/2724087362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f749c0b09f0 msgr2=0x7f749c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:38.800 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.818+0000 7f74a2608640 1 --2- 172.21.15.112:0/2724087362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f749c0b09f0 0x7f749c06c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:38.800 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.818+0000 7f74a2608640 1 -- 172.21.15.112:0/2724087362 >> v1:172.21.15.112:6789/0 conn(0x7f749c0b0610 legacy=0x7f749c1560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:38.800 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.819+0000 7f74a2608640 1 -- 172.21.15.112:0/2724087362 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749c0c6320 con 0x7f749c0c90c0 2024-04-25T01:25:38.800 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.818+0000 7f74a0e05640 1 -- 172.21.15.112:0/2724087362 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 136936613 0 0) 0x7f749c0c5d60 con 0x7f749c0b0610 2024-04-25T01:25:38.800 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.819+0000 7f74a1606640 1 --2- 172.21.15.112:0/2724087362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f749c0b09f0 0x7f749c06c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:25:38.800 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.819+0000 7f74a2608640 1 --2- 172.21.15.112:0/2724087362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749c0c90c0 0x7f749c0c6d40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7494009d90 tx=0x7f749402a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46310d596403b552 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:38.800 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.819+0000 7f74a0e05640 1 -- 172.21.15.112:0/2724087362 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7494007740 con 0x7f749c0c90c0 2024-04-25T01:25:38.800 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.819+0000 7f74a0e05640 1 -- 172.21.15.112:0/2724087362 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74940078e0 con 0x7f749c0c90c0 2024-04-25T01:25:38.800 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.819+0000 7f74a0e05640 1 -- 172.21.15.112:0/2724087362 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7494007cb0 con 0x7f749c0c90c0 2024-04-25T01:25:38.803 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.820+0000 7f74a406b640 1 -- 172.21.15.112:0/2724087362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749c0c90c0 msgr2=0x7f749c0c6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:38.803 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.820+0000 7f74a406b640 1 --2- 172.21.15.112:0/2724087362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749c0c90c0 0x7f749c0c6d40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7494009d90 tx=0x7f749402a530 comp rx=0 tx=0).stop 2024-04-25T01:25:38.803 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.820+0000 7f74a406b640 1 -- 172.21.15.112:0/2724087362 shutdown_connections 2024-04-25T01:25:38.803 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.820+0000 7f74a406b640 1 --2- 172.21.15.112:0/2724087362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749c0c90c0 0x7f749c0c6d40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:38.803 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.820+0000 7f74a406b640 1 --2- 172.21.15.112:0/2724087362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f749c0b09f0 0x7f749c06c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:38.803 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.820+0000 7f74a406b640 1 -- 172.21.15.112:0/2724087362 >> 172.21.15.112:0/2724087362 conn(0x7f749c0cb7f0 msgr2=0x7f749c0c7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:38.804 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.820+0000 7f74a406b640 1 -- 172.21.15.112:0/2724087362 shutdown_connections 2024-04-25T01:25:38.804 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.820+0000 7f74a406b640 1 -- 172.21.15.112:0/2724087362 wait complete. 2024-04-25T01:25:38.804 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.821+0000 7f74a406b640 1 Processor -- start 2024-04-25T01:25:38.804 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.821+0000 7f74a406b640 1 -- start start 2024-04-25T01:25:38.804 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74a406b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749c0b0610 0x7f749c0c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:38.804 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74a406b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f749c0b09f0 0x7f749c14e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:38.804 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74a406b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f749c0c90c0 0x7f749c14eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:38.804 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74a406b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f749c156980 con 0x7f749c0b0610 2024-04-25T01:25:38.804 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74a406b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f749c156800 con 0x7f749c0c90c0 2024-04-25T01:25:38.804 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74a406b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f749c0c7280 con 0x7f749c0b09f0 2024-04-25T01:25:38.804 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74a1e07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749c0b0610 0x7f749c0c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:38.804 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74a1e07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749c0b0610 0x7f749c0c2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39252/0 (socket says 172.21.15.112:39252) 2024-04-25T01:25:38.804 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74a1e07640 1 -- 172.21.15.112:0/682462110 learned_addr learned my addr 172.21.15.112:0/682462110 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:38.804 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74a1606640 1 --2- 172.21.15.112:0/682462110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f749c0b09f0 0x7f749c14e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:38.804 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74a2608640 1 --2- 172.21.15.112:0/682462110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f749c0c90c0 0x7f749c14eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:38.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74a1e07640 1 -- 172.21.15.112:0/682462110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f749c0b09f0 msgr2=0x7f749c14e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:38.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74a1e07640 1 --2- 172.21.15.112:0/682462110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f749c0b09f0 0x7f749c14e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:38.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74a1e07640 1 -- 172.21.15.112:0/682462110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f749c0c90c0 msgr2=0x7f749c14eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:38.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74a1e07640 1 --2- 172.21.15.112:0/682462110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f749c0c90c0 0x7f749c14eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:38.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74a1e07640 1 -- 172.21.15.112:0/682462110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749c1565c0 con 0x7f749c0b0610 2024-04-25T01:25:38.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74a1e07640 1 --2- 172.21.15.112:0/682462110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749c0b0610 0x7f749c0c2af0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f748c00c430 tx=0x7f748c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:38.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74927fc640 1 -- 172.21.15.112:0/682462110 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c0074e0 con 0x7f749c0b0610 2024-04-25T01:25:38.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74927fc640 1 -- 172.21.15.112:0/682462110 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f748c0047e0 con 0x7f749c0b0610 2024-04-25T01:25:38.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74927fc640 1 -- 172.21.15.112:0/682462110 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c004bd0 con 0x7f749c0b0610 2024-04-25T01:25:38.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74a406b640 1 -- 172.21.15.112:0/682462110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f749c1ee5d0 con 0x7f749c0b0610 2024-04-25T01:25:38.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.822+0000 7f74a406b640 1 -- 172.21.15.112:0/682462110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f749c1eeb90 con 0x7f749c0b0610 2024-04-25T01:25:38.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.823+0000 7f74a406b640 1 -- 172.21.15.112:0/682462110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f749c075800 con 0x7f749c0b0610 2024-04-25T01:25:38.805 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.824+0000 7f74927fc640 1 -- 172.21.15.112:0/682462110 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f748c011070 con 0x7f749c0b0610 2024-04-25T01:25:38.806 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.824+0000 7f74927fc640 1 --2- 172.21.15.112:0/682462110 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f748006bda0 0x7f748006e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:38.806 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.824+0000 7f74927fc640 1 -- 172.21.15.112:0/682462110 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f748c089a90 con 0x7f749c0b0610 2024-04-25T01:25:38.806 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.824+0000 7f74a1606640 1 --2- 172.21.15.112:0/682462110 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f748006bda0 0x7f748006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:38.806 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.824+0000 7f74a1606640 1 --2- 172.21.15.112:0/682462110 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f748006bda0 0x7f748006e260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f749c155980 tx=0x7f7498008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:38.808 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:38.829+0000 7f74927fc640 1 -- 172.21.15.112:0/682462110 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f748c058c60 con 0x7f749c0b0610 2024-04-25T01:25:38.982 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:39.002+0000 7f74a406b640 1 -- 172.21.15.112:0/682462110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f749c0c5b40 con 0x7f749c0b0610 2024-04-25T01:25:38.983 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:39.003+0000 7f74927fc640 1 -- 172.21.15.112:0/682462110 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f748c05cf60 con 0x7f749c0b0610 2024-04-25T01:25:38.983 INFO:teuthology.misc.health.smithi112.stdout: 2024-04-25T01:25:38.983 INFO:teuthology.misc.health.smithi112.stdout:{"epoch":12,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:25:36.659312+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:25:38.985 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:39.005+0000 7f74a406b640 1 -- 172.21.15.112:0/682462110 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f748006bda0 msgr2=0x7f748006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:38.985 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:39.005+0000 7f74a406b640 1 --2- 172.21.15.112:0/682462110 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f748006bda0 0x7f748006e260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f749c155980 tx=0x7f7498008040 comp rx=0 tx=0).stop 2024-04-25T01:25:38.985 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:39.005+0000 7f74a406b640 1 -- 172.21.15.112:0/682462110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749c0b0610 msgr2=0x7f749c0c2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:38.985 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:39.005+0000 7f74a406b640 1 --2- 172.21.15.112:0/682462110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749c0b0610 0x7f749c0c2af0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f748c00c430 tx=0x7f748c00c8f0 comp rx=0 tx=0).stop 2024-04-25T01:25:38.985 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:39.005+0000 7f74a406b640 1 -- 172.21.15.112:0/682462110 shutdown_connections 2024-04-25T01:25:38.985 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:39.005+0000 7f74a406b640 1 --2- 172.21.15.112:0/682462110 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f748006bda0 0x7f748006e260 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:38.985 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:39.005+0000 7f74a406b640 1 --2- 172.21.15.112:0/682462110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f749c0c90c0 0x7f749c14eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:38.985 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:39.005+0000 7f74a406b640 1 --2- 172.21.15.112:0/682462110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f749c0b09f0 0x7f749c14e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:38.985 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:39.005+0000 7f74a406b640 1 --2- 172.21.15.112:0/682462110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749c0b0610 0x7f749c0c2af0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:38.985 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:39.005+0000 7f74a406b640 1 -- 172.21.15.112:0/682462110 >> 172.21.15.112:0/682462110 conn(0x7f749c0cb7f0 msgr2=0x7f749c0c7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:38.985 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:39.005+0000 7f74a406b640 1 -- 172.21.15.112:0/682462110 shutdown_connections 2024-04-25T01:25:38.986 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:39.005+0000 7f74a406b640 1 -- 172.21.15.112:0/682462110 wait complete. 2024-04-25T01:25:38.995 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T01:25:45.801 DEBUG:teuthology.orchestra.run.smithi112:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T01:25:45.901 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.922+0000 7f3db51a9640 1 Processor -- start 2024-04-25T01:25:45.902 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.922+0000 7f3db51a9640 1 -- start start 2024-04-25T01:25:45.903 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.922+0000 7f3db51a9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3db00c3d80 0x7f3db01525b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:45.903 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.922+0000 7f3db51a9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3db01491a0 0x7f3db0070d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:45.903 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.923+0000 7f3db51a9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3db00c1cd0 con 0x7f3db0073030 2024-04-25T01:25:45.904 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.923+0000 7f3db51a9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f3db00c1a80 con 0x7f3db01491a0 2024-04-25T01:25:45.904 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.923+0000 7f3db51a9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3db00c1fb0 con 0x7f3db00c3d80 2024-04-25T01:25:45.904 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.923+0000 7f3daf59e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3db01491a0 0x7f3db0070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:45.904 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.923+0000 7f3dae59c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3db0073030 0x7f3db0148a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54796/0 (socket says 172.21.15.112:54796) 2024-04-25T01:25:45.904 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.923+0000 7f3dae59c640 1 -- 172.21.15.112:0/2896891071 learned_addr learned my addr 172.21.15.112:0/2896891071 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:25:45.904 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.923+0000 7f3daed9d640 1 --2- 172.21.15.112:0/2896891071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3db00c3d80 0x7f3db01525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:45.904 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.923+0000 7f3daf59e640 1 -- 172.21.15.112:0/2896891071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3db00c3d80 msgr2=0x7f3db01525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:45.904 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.923+0000 7f3daf59e640 1 --2- 172.21.15.112:0/2896891071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3db00c3d80 0x7f3db01525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:45.904 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.923+0000 7f3daf59e640 1 -- 172.21.15.112:0/2896891071 >> v1:172.21.15.112:6789/0 conn(0x7f3db0073030 legacy=0x7f3db0148a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:45.904 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.923+0000 7f3dadd9b640 1 -- 172.21.15.112:0/2896891071 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3557574861 0 0) 0x7f3db00c1cd0 con 0x7f3db0073030 2024-04-25T01:25:45.904 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.923+0000 7f3daf59e640 1 -- 172.21.15.112:0/2896891071 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db00c2290 con 0x7f3db01491a0 2024-04-25T01:25:45.904 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.924+0000 7f3daf59e640 1 --2- 172.21.15.112:0/2896891071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db01491a0 0x7f3db0070d20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009d90 tx=0x7f3da802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0315aecafcf1660 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:45.904 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.924+0000 7f3dadd9b640 1 -- 172.21.15.112:0/2896891071 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8031600 con 0x7f3db01491a0 2024-04-25T01:25:45.905 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.924+0000 7f3dadd9b640 1 -- 172.21.15.112:0/2896891071 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3da80317a0 con 0x7f3db01491a0 2024-04-25T01:25:45.905 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.924+0000 7f3dadd9b640 1 -- 172.21.15.112:0/2896891071 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8031b70 con 0x7f3db01491a0 2024-04-25T01:25:45.907 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.924+0000 7f3db51a9640 1 -- 172.21.15.112:0/2896891071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db01491a0 msgr2=0x7f3db0070d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:45.907 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.924+0000 7f3db51a9640 1 --2- 172.21.15.112:0/2896891071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db01491a0 0x7f3db0070d20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009d90 tx=0x7f3da802a5d0 comp rx=0 tx=0).stop 2024-04-25T01:25:45.908 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.924+0000 7f3db51a9640 1 -- 172.21.15.112:0/2896891071 shutdown_connections 2024-04-25T01:25:45.908 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.924+0000 7f3db51a9640 1 --2- 172.21.15.112:0/2896891071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db01491a0 0x7f3db0070d20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:45.908 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.924+0000 7f3db51a9640 1 --2- 172.21.15.112:0/2896891071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3db00c3d80 0x7f3db01525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:45.908 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.924+0000 7f3db51a9640 1 -- 172.21.15.112:0/2896891071 >> 172.21.15.112:0/2896891071 conn(0x7f3db006d460 msgr2=0x7f3db006b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:45.908 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.924+0000 7f3db51a9640 1 -- 172.21.15.112:0/2896891071 shutdown_connections 2024-04-25T01:25:45.908 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.924+0000 7f3db51a9640 1 -- 172.21.15.112:0/2896891071 wait complete. 2024-04-25T01:25:45.908 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.926+0000 7f3db51a9640 1 Processor -- start 2024-04-25T01:25:45.908 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.926+0000 7f3db51a9640 1 -- start start 2024-04-25T01:25:45.908 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3db51a9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3db00c3d80 0x7f3db013b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:45.908 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3db51a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db01491a0 0x7f3db014a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:45.908 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3db51a9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3db00c76e0 0x7f3db01ec320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:45.908 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3db51a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3db00c37d0 con 0x7f3db01491a0 2024-04-25T01:25:45.908 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3db51a9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3db00c2d70 con 0x7f3db00c3d80 2024-04-25T01:25:45.908 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3db51a9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3db00c4160 con 0x7f3db00c76e0 2024-04-25T01:25:45.908 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3dae59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db01491a0 0x7f3db014a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:45.909 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3dae59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db01491a0 0x7f3db014a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39284/0 (socket says 172.21.15.112:39284) 2024-04-25T01:25:45.909 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3dae59c640 1 -- 172.21.15.112:0/1669953344 learned_addr learned my addr 172.21.15.112:0/1669953344 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:45.909 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3daed9d640 1 --2- 172.21.15.112:0/1669953344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3db00c3d80 0x7f3db013b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:45.909 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3daf59e640 1 --2- 172.21.15.112:0/1669953344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3db00c76e0 0x7f3db01ec320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:45.909 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3dae59c640 1 -- 172.21.15.112:0/1669953344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3db00c76e0 msgr2=0x7f3db01ec320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:45.909 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3dae59c640 1 --2- 172.21.15.112:0/1669953344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3db00c76e0 0x7f3db01ec320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:45.909 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3dae59c640 1 -- 172.21.15.112:0/1669953344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3db00c3d80 msgr2=0x7f3db013b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:45.909 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3dae59c640 1 --2- 172.21.15.112:0/1669953344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3db00c3d80 0x7f3db013b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:45.909 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3dae59c640 1 -- 172.21.15.112:0/1669953344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db013bcd0 con 0x7f3db01491a0 2024-04-25T01:25:45.909 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3dae59c640 1 --2- 172.21.15.112:0/1669953344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db01491a0 0x7f3db014a7b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3da400c410 tx=0x7f3da400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:45.909 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3d977fe640 1 -- 172.21.15.112:0/1669953344 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da40074e0 con 0x7f3db01491a0 2024-04-25T01:25:45.909 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3d977fe640 1 -- 172.21.15.112:0/1669953344 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3da40047e0 con 0x7f3db01491a0 2024-04-25T01:25:45.909 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3db51a9640 1 -- 172.21.15.112:0/1669953344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db0149650 con 0x7f3db01491a0 2024-04-25T01:25:45.910 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3d977fe640 1 -- 172.21.15.112:0/1669953344 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4004bd0 con 0x7f3db01491a0 2024-04-25T01:25:45.910 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.927+0000 7f3db51a9640 1 -- 172.21.15.112:0/1669953344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db0148310 con 0x7f3db01491a0 2024-04-25T01:25:45.910 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.928+0000 7f3db51a9640 1 -- 172.21.15.112:0/1669953344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3db0075800 con 0x7f3db01491a0 2024-04-25T01:25:45.911 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.929+0000 7f3d977fe640 1 -- 172.21.15.112:0/1669953344 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3da4011070 con 0x7f3db01491a0 2024-04-25T01:25:45.911 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.929+0000 7f3d977fe640 1 --2- 172.21.15.112:0/1669953344 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3d9c06be30 0x7f3d9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:45.911 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.929+0000 7f3daed9d640 1 --2- 172.21.15.112:0/1669953344 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3d9c06be30 0x7f3d9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:45.911 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.929+0000 7f3d977fe640 1 -- 172.21.15.112:0/1669953344 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f3da408a6a0 con 0x7f3db01491a0 2024-04-25T01:25:45.911 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.930+0000 7f3daed9d640 1 --2- 172.21.15.112:0/1669953344 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3d9c06be30 0x7f3d9c06e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3da0006fd0 tx=0x7f3da0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:45.913 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:45.934+0000 7f3d977fe640 1 -- 172.21.15.112:0/1669953344 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3da400e280 con 0x7f3db01491a0 2024-04-25T01:25:46.088 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:46.108+0000 7f3db51a9640 1 -- 172.21.15.112:0/1669953344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3db00748e0 con 0x7f3db01491a0 2024-04-25T01:25:46.089 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:46.109+0000 7f3d977fe640 1 -- 172.21.15.112:0/1669953344 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12908 (secure 0 0 0) 0x7f3da4059430 con 0x7f3db01491a0 2024-04-25T01:25:46.089 INFO:teuthology.misc.health.smithi112.stdout: 2024-04-25T01:25:46.089 INFO:teuthology.misc.health.smithi112.stdout:{"epoch":16,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:25:42.691350+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:25:46.091 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:46.111+0000 7f3db51a9640 1 -- 172.21.15.112:0/1669953344 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3d9c06be30 msgr2=0x7f3d9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:46.091 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:46.111+0000 7f3db51a9640 1 --2- 172.21.15.112:0/1669953344 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3d9c06be30 0x7f3d9c06e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3da0006fd0 tx=0x7f3da0008040 comp rx=0 tx=0).stop 2024-04-25T01:25:46.091 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:46.111+0000 7f3db51a9640 1 -- 172.21.15.112:0/1669953344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db01491a0 msgr2=0x7f3db014a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:46.091 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:46.111+0000 7f3db51a9640 1 --2- 172.21.15.112:0/1669953344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db01491a0 0x7f3db014a7b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3da400c410 tx=0x7f3da400c8d0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.092 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:46.111+0000 7f3db51a9640 1 -- 172.21.15.112:0/1669953344 shutdown_connections 2024-04-25T01:25:46.092 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:46.111+0000 7f3db51a9640 1 --2- 172.21.15.112:0/1669953344 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3d9c06be30 0x7f3d9c06e2f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.092 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:46.111+0000 7f3db51a9640 1 --2- 172.21.15.112:0/1669953344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3db00c76e0 0x7f3db01ec320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.092 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:46.111+0000 7f3db51a9640 1 --2- 172.21.15.112:0/1669953344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db01491a0 0x7f3db014a7b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.092 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:46.111+0000 7f3db51a9640 1 --2- 172.21.15.112:0/1669953344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3db00c3d80 0x7f3db013b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.092 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:46.111+0000 7f3db51a9640 1 -- 172.21.15.112:0/1669953344 >> 172.21.15.112:0/1669953344 conn(0x7f3db006d460 msgr2=0x7f3db0071100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:46.092 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:46.111+0000 7f3db51a9640 1 -- 172.21.15.112:0/1669953344 shutdown_connections 2024-04-25T01:25:46.092 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T01:25:46.111+0000 7f3db51a9640 1 -- 172.21.15.112:0/1669953344 wait complete. 2024-04-25T01:25:46.101 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T01:25:46.101 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T01:25:46.101 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T01:25:46.101 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T01:25:46.103 INFO:tasks.ceph.mds.a:Started 2024-04-25T01:25:46.103 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T01:25:46.103 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T01:25:46.105 INFO:tasks.ceph.mds.c:Started 2024-04-25T01:25:46.105 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T01:25:46.105 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T01:25:46.108 INFO:tasks.ceph.mds.b:Started 2024-04-25T01:25:46.108 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T01:25:46.108 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T01:25:46.110 INFO:tasks.ceph.mds.d:Started 2024-04-25T01:25:46.110 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T01:25:46.111 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:25:46.176 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2024-04-25T01:25:46.176 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2024-04-25T01:25:46.191 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-04-25T01:25:46.191 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-04-25T01:25:46.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.231+0000 7f9e589c1640 1 Processor -- start 2024-04-25T01:25:46.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.231+0000 7f9e589c1640 1 -- start start 2024-04-25T01:25:46.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.232+0000 7f9e589c1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9e541596a0 0x7f9e54152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:46.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.232+0000 7f9e589c1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9e540c72b0 0x7f9e54160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:46.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.232+0000 7f9e589c1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e54159b90 con 0x7f9e541592c0 2024-04-25T01:25:46.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.232+0000 7f9e589c1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f9e540c6e20 con 0x7f9e540c72b0 2024-04-25T01:25:46.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.232+0000 7f9e589c1640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9e5406c970 con 0x7f9e541596a0 2024-04-25T01:25:46.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.232+0000 7f9e52d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9e540c72b0 0x7f9e54160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:46.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.232+0000 7f9e5259c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9e541592c0 0x7f9e540c9070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54826/0 (socket says 172.21.15.112:54826) 2024-04-25T01:25:46.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.232+0000 7f9e5259c640 1 -- 172.21.15.112:0/2474929419 learned_addr learned my addr 172.21.15.112:0/2474929419 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:25:46.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.232+0000 7f9e52d9d640 1 -- 172.21.15.112:0/2474929419 >> v2:172.21.15.187:3300/0 conn(0x7f9e541596a0 msgr2=0x7f9e54152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:46.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.232+0000 7f9e52d9d640 1 --2- 172.21.15.112:0/2474929419 >> v2:172.21.15.187:3300/0 conn(0x7f9e541596a0 0x7f9e54152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.232+0000 7f9e52d9d640 1 -- 172.21.15.112:0/2474929419 >> v1:172.21.15.112:6789/0 conn(0x7f9e541592c0 legacy=0x7f9e540c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:46.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.232+0000 7f9e5159a640 1 -- 172.21.15.112:0/2474929419 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4259851639 0 0) 0x7f9e54159b90 con 0x7f9e541592c0 2024-04-25T01:25:46.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.232+0000 7f9e52d9d640 1 -- 172.21.15.112:0/2474929419 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5406cc50 con 0x7f9e540c72b0 2024-04-25T01:25:46.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.233+0000 7f9e52d9d640 1 --2- 172.21.15.112:0/2474929419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e540c72b0 0x7f9e54160380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9e540bea40 tx=0x7f9e4c02a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31adcee64d1e7863 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:46.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.233+0000 7f9e5159a640 1 -- 172.21.15.112:0/2474929419 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c007740 con 0x7f9e540c72b0 2024-04-25T01:25:46.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.233+0000 7f9e5159a640 1 -- 172.21.15.112:0/2474929419 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e4c0078e0 con 0x7f9e540c72b0 2024-04-25T01:25:46.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.233+0000 7f9e5159a640 1 -- 172.21.15.112:0/2474929419 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c007cb0 con 0x7f9e540c72b0 2024-04-25T01:25:46.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.234+0000 7f9e589c1640 1 -- 172.21.15.112:0/2474929419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e540c72b0 msgr2=0x7f9e54160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:46.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.234+0000 7f9e589c1640 1 --2- 172.21.15.112:0/2474929419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e540c72b0 0x7f9e54160380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9e540bea40 tx=0x7f9e4c02a560 comp rx=0 tx=0).stop 2024-04-25T01:25:46.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.234+0000 7f9e589c1640 1 -- 172.21.15.112:0/2474929419 shutdown_connections 2024-04-25T01:25:46.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.234+0000 7f9e589c1640 1 --2- 172.21.15.112:0/2474929419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e540c72b0 0x7f9e54160380 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.234+0000 7f9e589c1640 1 --2- 172.21.15.112:0/2474929419 >> v2:172.21.15.187:3300/0 conn(0x7f9e541596a0 0x7f9e54152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.234+0000 7f9e589c1640 1 -- 172.21.15.112:0/2474929419 >> 172.21.15.112:0/2474929419 conn(0x7f9e540cb170 msgr2=0x7f9e540cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:46.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.234+0000 7f9e589c1640 1 -- 172.21.15.112:0/2474929419 shutdown_connections 2024-04-25T01:25:46.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.234+0000 7f9e589c1640 1 -- 172.21.15.112:0/2474929419 wait complete. 2024-04-25T01:25:46.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.235+0000 7f9e589c1640 1 Processor -- start 2024-04-25T01:25:46.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.235+0000 7f9e589c1640 1 -- start start 2024-04-25T01:25:46.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.236+0000 7f9e589c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e540c72b0 0x7f9e540c33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:46.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.236+0000 7f9e589c1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e541592c0 0x7f9e541eb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:46.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.236+0000 7f9e589c1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e541596a0 0x7f9e5414c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:46.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.236+0000 7f9e5259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e540c72b0 0x7f9e540c33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:46.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.236+0000 7f9e589c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5406c510 con 0x7f9e540c72b0 2024-04-25T01:25:46.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.236+0000 7f9e5259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e540c72b0 0x7f9e540c33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39316/0 (socket says 172.21.15.112:39316) 2024-04-25T01:25:46.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.236+0000 7f9e5259c640 1 -- 172.21.15.112:0/741920849 learned_addr learned my addr 172.21.15.112:0/741920849 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:46.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.236+0000 7f9e589c1640 1 -- 172.21.15.112:0/741920849 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5406c390 con 0x7f9e541596a0 2024-04-25T01:25:46.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.236+0000 7f9e589c1640 1 -- 172.21.15.112:0/741920849 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9e5406db20 con 0x7f9e541592c0 2024-04-25T01:25:46.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.237+0000 7f9e51d9b640 1 --2- 172.21.15.112:0/741920849 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e541592c0 0x7f9e541eb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:46.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.237+0000 7f9e52d9d640 1 --2- 172.21.15.112:0/741920849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e541596a0 0x7f9e5414c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:46.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.237+0000 7f9e5259c640 1 -- 172.21.15.112:0/741920849 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e541592c0 msgr2=0x7f9e541eb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:46.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.237+0000 7f9e5259c640 1 --2- 172.21.15.112:0/741920849 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e541592c0 0x7f9e541eb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.237+0000 7f9e5259c640 1 -- 172.21.15.112:0/741920849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e541596a0 msgr2=0x7f9e5414c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:46.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.237+0000 7f9e5259c640 1 --2- 172.21.15.112:0/741920849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e541596a0 0x7f9e5414c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.237+0000 7f9e5259c640 1 -- 172.21.15.112:0/741920849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e540c41b0 con 0x7f9e540c72b0 2024-04-25T01:25:46.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.237+0000 7f9e5259c640 1 --2- 172.21.15.112:0/741920849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e540c72b0 0x7f9e540c33e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400c4d0 tx=0x7f9e4400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:46.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.237+0000 7f9e42ffd640 1 -- 172.21.15.112:0/741920849 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44018070 con 0x7f9e540c72b0 2024-04-25T01:25:46.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.237+0000 7f9e42ffd640 1 -- 172.21.15.112:0/741920849 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e44007500 con 0x7f9e540c72b0 2024-04-25T01:25:46.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.237+0000 7f9e42ffd640 1 -- 172.21.15.112:0/741920849 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44004a20 con 0x7f9e540c72b0 2024-04-25T01:25:46.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.237+0000 7f9e589c1640 1 -- 172.21.15.112:0/741920849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e540bf8c0 con 0x7f9e540c72b0 2024-04-25T01:25:46.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.237+0000 7f9e589c1640 1 -- 172.21.15.112:0/741920849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e5414c590 con 0x7f9e540c72b0 2024-04-25T01:25:46.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.238+0000 7f9e589c1640 1 -- 172.21.15.112:0/741920849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e540679a0 con 0x7f9e540c72b0 2024-04-25T01:25:46.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.239+0000 7f9e42ffd640 1 -- 172.21.15.112:0/741920849 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9e44004bc0 con 0x7f9e540c72b0 2024-04-25T01:25:46.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.239+0000 7f9e42ffd640 1 --2- 172.21.15.112:0/741920849 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9e3406be30 0x7f9e3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:46.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.239+0000 7f9e42ffd640 1 -- 172.21.15.112:0/741920849 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f9e44089c60 con 0x7f9e540c72b0 2024-04-25T01:25:46.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.239+0000 7f9e51d9b640 1 --2- 172.21.15.112:0/741920849 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9e3406be30 0x7f9e3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:46.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.239+0000 7f9e51d9b640 1 --2- 172.21.15.112:0/741920849 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9e3406be30 0x7f9e3406e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9e54155c90 tx=0x7f9e48009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:46.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.243+0000 7f9e42ffd640 1 -- 172.21.15.112:0/741920849 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9e440583a0 con 0x7f9e540c72b0 2024-04-25T01:25:46.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.421+0000 7f9e589c1640 1 -- 172.21.15.112:0/741920849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e540bf6b0 con 0x7f9e540c72b0 2024-04-25T01:25:46.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.422+0000 7f9e42ffd640 1 -- 172.21.15.112:0/741920849 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12908 (secure 0 0 0) 0x7f9e4405c6a0 con 0x7f9e540c72b0 2024-04-25T01:25:46.402 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:25:46.402 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":16,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:25:42.691350+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:25:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.424+0000 7f9e589c1640 1 -- 172.21.15.112:0/741920849 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9e3406be30 msgr2=0x7f9e3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.424+0000 7f9e589c1640 1 --2- 172.21.15.112:0/741920849 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9e3406be30 0x7f9e3406e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9e54155c90 tx=0x7f9e48009060 comp rx=0 tx=0).stop 2024-04-25T01:25:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.424+0000 7f9e589c1640 1 -- 172.21.15.112:0/741920849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e540c72b0 msgr2=0x7f9e540c33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.424+0000 7f9e589c1640 1 --2- 172.21.15.112:0/741920849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e540c72b0 0x7f9e540c33e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400c4d0 tx=0x7f9e4400c990 comp rx=0 tx=0).stop 2024-04-25T01:25:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.424+0000 7f9e589c1640 1 -- 172.21.15.112:0/741920849 shutdown_connections 2024-04-25T01:25:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.424+0000 7f9e589c1640 1 --2- 172.21.15.112:0/741920849 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9e3406be30 0x7f9e3406e2f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.424+0000 7f9e589c1640 1 --2- 172.21.15.112:0/741920849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e541596a0 0x7f9e5414c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.424+0000 7f9e589c1640 1 --2- 172.21.15.112:0/741920849 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e541592c0 0x7f9e541eb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.424+0000 7f9e589c1640 1 --2- 172.21.15.112:0/741920849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e540c72b0 0x7f9e540c33e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.424+0000 7f9e589c1640 1 -- 172.21.15.112:0/741920849 >> 172.21.15.112:0/741920849 conn(0x7f9e540cb170 msgr2=0x7f9e54152ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:46.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.424+0000 7f9e589c1640 1 -- 172.21.15.112:0/741920849 shutdown_connections 2024-04-25T01:25:46.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.424+0000 7f9e589c1640 1 -- 172.21.15.112:0/741920849 wait complete. 2024-04-25T01:25:46.415 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:25:40.679119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:25:46.415 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:25:46.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.536+0000 7f788d8c3640 1 Processor -- start 2024-04-25T01:25:46.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.536+0000 7f788d8c3640 1 -- start start 2024-04-25T01:25:46.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.536+0000 7f788d8c3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78880ca370 0x7f788806c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:46.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.536+0000 7f788d8c3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f78880ca8a0 0x7f78880c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:46.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.536+0000 7f788d8c3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78881534a0 con 0x7f78880c9250 2024-04-25T01:25:46.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.536+0000 7f788d8c3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7888156990 con 0x7f78880ca8a0 2024-04-25T01:25:46.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.536+0000 7f788d8c3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7888156500 con 0x7f78880ca370 2024-04-25T01:25:46.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.537+0000 7f78867fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78880ca370 0x7f788806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:46.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.537+0000 7f78877fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f78880ca8a0 0x7f78880c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:46.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.537+0000 7f78867fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78880ca370 0x7f788806c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:43642/0 (socket says 172.21.15.112:43642) 2024-04-25T01:25:46.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.537+0000 7f78867fc640 1 -- 172.21.15.112:0/912522834 learned_addr learned my addr 172.21.15.112:0/912522834 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:46.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.537+0000 7f7886ffd640 1 --1- 172.21.15.112:0/912522834 >> v1:172.21.15.187:6789/0 conn(0x7f78880c9250 0x7f788806bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:44076/0 (socket says 172.21.15.112:44076) 2024-04-25T01:25:46.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.537+0000 7f78867fc640 1 -- 172.21.15.112:0/912522834 >> v2:172.21.15.187:3300/0 conn(0x7f78880ca8a0 msgr2=0x7f78880c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:46.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.537+0000 7f78867fc640 1 --2- 172.21.15.112:0/912522834 >> v2:172.21.15.187:3300/0 conn(0x7f78880ca8a0 0x7f78880c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.537+0000 7f78867fc640 1 -- 172.21.15.112:0/912522834 >> v1:172.21.15.187:6789/0 conn(0x7f78880c9250 legacy=0x7f788806bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:46.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.537+0000 7f78867fc640 1 -- 172.21.15.112:0/912522834 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78880cae30 con 0x7f78880ca370 2024-04-25T01:25:46.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.537+0000 7f78877fe640 1 --2- 172.21.15.112:0/912522834 >> v2:172.21.15.187:3300/0 conn(0x7f78880ca8a0 0x7f78880c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:25:46.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.538+0000 7f78867fc640 1 --2- 172.21.15.112:0/912522834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78880ca370 0x7f788806c420 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f787c009870 tx=0x7f787c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7ac806c3d971c66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:46.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.538+0000 7f7885ffb640 1 -- 172.21.15.112:0/912522834 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c037070 con 0x7f78880ca370 2024-04-25T01:25:46.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.538+0000 7f7885ffb640 1 -- 172.21.15.112:0/912522834 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f787c032440 con 0x7f78880ca370 2024-04-25T01:25:46.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.538+0000 7f7885ffb640 1 -- 172.21.15.112:0/912522834 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c032810 con 0x7f78880ca370 2024-04-25T01:25:46.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.538+0000 7f788d8c3640 1 -- 172.21.15.112:0/912522834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78880ca370 msgr2=0x7f788806c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:46.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.538+0000 7f788d8c3640 1 --2- 172.21.15.112:0/912522834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78880ca370 0x7f788806c420 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f787c009870 tx=0x7f787c02a1c0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.538+0000 7f788d8c3640 1 -- 172.21.15.112:0/912522834 shutdown_connections 2024-04-25T01:25:46.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.538+0000 7f788d8c3640 1 --2- 172.21.15.112:0/912522834 >> v2:172.21.15.187:3300/0 conn(0x7f78880ca8a0 0x7f78880c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.538+0000 7f788d8c3640 1 --2- 172.21.15.112:0/912522834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78880ca370 0x7f788806c420 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.538+0000 7f788d8c3640 1 -- 172.21.15.112:0/912522834 >> 172.21.15.112:0/912522834 conn(0x7f78880cb310 msgr2=0x7f78880cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:46.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.538+0000 7f788d8c3640 1 -- 172.21.15.112:0/912522834 shutdown_connections 2024-04-25T01:25:46.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.538+0000 7f788d8c3640 1 -- 172.21.15.112:0/912522834 wait complete. 2024-04-25T01:25:46.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.539+0000 7f788d8c3640 1 Processor -- start 2024-04-25T01:25:46.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.540+0000 7f788d8c3640 1 -- start start 2024-04-25T01:25:46.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.540+0000 7f788d8c3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78880c9250 0x7f78881576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:46.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.540+0000 7f788d8c3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78880ca370 0x7f7888157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:46.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.540+0000 7f788d8c3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78880ca8a0 0x7f78880c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:46.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.540+0000 7f788d8c3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7888155eb0 con 0x7f78880ca8a0 2024-04-25T01:25:46.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.540+0000 7f788d8c3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7888155bd0 con 0x7f78880ca370 2024-04-25T01:25:46.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.540+0000 7f788d8c3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7888156190 con 0x7f78880c9250 2024-04-25T01:25:46.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.540+0000 7f78877fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78880ca8a0 0x7f78880c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:46.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.540+0000 7f78877fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78880ca8a0 0x7f78880c3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39320/0 (socket says 172.21.15.112:39320) 2024-04-25T01:25:46.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.540+0000 7f78877fe640 1 -- 172.21.15.112:0/320189945 learned_addr learned my addr 172.21.15.112:0/320189945 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:46.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.541+0000 7f7886ffd640 1 --2- 172.21.15.112:0/320189945 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78880c9250 0x7f78881576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:46.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.541+0000 7f78867fc640 1 --2- 172.21.15.112:0/320189945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78880ca370 0x7f7888157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:46.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.541+0000 7f78877fe640 1 -- 172.21.15.112:0/320189945 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78880c9250 msgr2=0x7f78881576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:46.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.541+0000 7f78877fe640 1 --2- 172.21.15.112:0/320189945 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78880c9250 0x7f78881576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.541+0000 7f78877fe640 1 -- 172.21.15.112:0/320189945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78880ca370 msgr2=0x7f7888157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:46.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.541+0000 7f78877fe640 1 --2- 172.21.15.112:0/320189945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78880ca370 0x7f7888157c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.541+0000 7f78877fe640 1 -- 172.21.15.112:0/320189945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78880c22a0 con 0x7f78880ca8a0 2024-04-25T01:25:46.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.541+0000 7f78867fc640 1 --2- 172.21.15.112:0/320189945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78880ca370 0x7f7888157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:25:46.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.541+0000 7f78877fe640 1 --2- 172.21.15.112:0/320189945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78880ca8a0 0x7f78880c3ed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f788000e880 tx=0x7f788000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:46.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.541+0000 7f786ffff640 1 -- 172.21.15.112:0/320189945 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880017070 con 0x7f78880ca8a0 2024-04-25T01:25:46.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.541+0000 7f786ffff640 1 -- 172.21.15.112:0/320189945 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7880004020 con 0x7f78880ca8a0 2024-04-25T01:25:46.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.541+0000 7f788d8c3640 1 -- 172.21.15.112:0/320189945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78881ec140 con 0x7f78880ca8a0 2024-04-25T01:25:46.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.541+0000 7f786ffff640 1 -- 172.21.15.112:0/320189945 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78800043f0 con 0x7f78880ca8a0 2024-04-25T01:25:46.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.541+0000 7f788d8c3640 1 -- 172.21.15.112:0/320189945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78881ec400 con 0x7f78880ca8a0 2024-04-25T01:25:46.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.542+0000 7f788d8c3640 1 -- 172.21.15.112:0/320189945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f788804f500 con 0x7f78880ca8a0 2024-04-25T01:25:46.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.542+0000 7f786ffff640 1 -- 172.21.15.112:0/320189945 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f788000c940 con 0x7f78880ca8a0 2024-04-25T01:25:46.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.543+0000 7f786ffff640 1 --2- 172.21.15.112:0/320189945 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f787406be30 0x7f787406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:46.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.543+0000 7f786ffff640 1 -- 172.21.15.112:0/320189945 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f788008a850 con 0x7f78880ca8a0 2024-04-25T01:25:46.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.543+0000 7f7886ffd640 1 --2- 172.21.15.112:0/320189945 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f787406be30 0x7f787406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:46.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.543+0000 7f7886ffd640 1 --2- 172.21.15.112:0/320189945 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f787406be30 0x7f787406e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f78780012e0 tx=0x7f7878005b70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:46.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.547+0000 7f786ffff640 1 -- 172.21.15.112:0/320189945 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f78800595e0 con 0x7f78880ca8a0 2024-04-25T01:25:46.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.719+0000 7f788d8c3640 1 -- 172.21.15.112:0/320189945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f788806d970 con 0x7f78880ca8a0 2024-04-25T01:25:46.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.719+0000 7f786ffff640 1 -- 172.21.15.112:0/320189945 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f7880058ee0 con 0x7f78880ca8a0 2024-04-25T01:25:46.699 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T01:25:46.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.721+0000 7f788d8c3640 1 -- 172.21.15.112:0/320189945 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f787406be30 msgr2=0x7f787406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:46.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.721+0000 7f788d8c3640 1 --2- 172.21.15.112:0/320189945 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f787406be30 0x7f787406e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f78780012e0 tx=0x7f7878005b70 comp rx=0 tx=0).stop 2024-04-25T01:25:46.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.721+0000 7f788d8c3640 1 -- 172.21.15.112:0/320189945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78880ca8a0 msgr2=0x7f78880c3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:46.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.721+0000 7f788d8c3640 1 --2- 172.21.15.112:0/320189945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78880ca8a0 0x7f78880c3ed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f788000e880 tx=0x7f788000ed40 comp rx=0 tx=0).stop 2024-04-25T01:25:46.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.721+0000 7f788d8c3640 1 -- 172.21.15.112:0/320189945 shutdown_connections 2024-04-25T01:25:46.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.721+0000 7f788d8c3640 1 --2- 172.21.15.112:0/320189945 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f787406be30 0x7f787406e2f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.721+0000 7f788d8c3640 1 --2- 172.21.15.112:0/320189945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78880ca8a0 0x7f78880c3ed0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.721+0000 7f788d8c3640 1 --2- 172.21.15.112:0/320189945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78880ca370 0x7f7888157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.721+0000 7f788d8c3640 1 --2- 172.21.15.112:0/320189945 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78880c9250 0x7f78881576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.721+0000 7f788d8c3640 1 -- 172.21.15.112:0/320189945 >> 172.21.15.112:0/320189945 conn(0x7f78880cb310 msgr2=0x7f78881548a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:46.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.721+0000 7f788d8c3640 1 -- 172.21.15.112:0/320189945 shutdown_connections 2024-04-25T01:25:46.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.721+0000 7f788d8c3640 1 -- 172.21.15.112:0/320189945 wait complete. 2024-04-25T01:25:46.711 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:25:46.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.827+0000 7f99b68a5640 1 Processor -- start 2024-04-25T01:25:46.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.827+0000 7f99b68a5640 1 -- start start 2024-04-25T01:25:46.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.828+0000 7f99b68a5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f99b0152f80 0x7f99b0158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:46.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.828+0000 7f99b68a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99b0071e00 0x7f99b00c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:46.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.828+0000 7f99b68a5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99b0155a10 con 0x7f99b0072660 2024-04-25T01:25:46.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.828+0000 7f99b68a5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f99b0072d10 con 0x7f99b0152f80 2024-04-25T01:25:46.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.828+0000 7f99b68a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f99b0153360 con 0x7f99b0071e00 2024-04-25T01:25:46.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.828+0000 7f99affff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f99b0152f80 0x7f99b0158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.828+0000 7f99af7fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f99b0072660 0x7f99b01561d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54834/0 (socket says 172.21.15.112:54834) 2024-04-25T01:25:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.828+0000 7f99af7fe640 1 -- 172.21.15.112:0/3691886407 learned_addr learned my addr 172.21.15.112:0/3691886407 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:25:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.828+0000 7f99b4e42640 1 --2- 172.21.15.112:0/3691886407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99b0071e00 0x7f99b00c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.828+0000 7f99affff640 1 -- 172.21.15.112:0/3691886407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99b0071e00 msgr2=0x7f99b00c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.828+0000 7f99affff640 1 --2- 172.21.15.112:0/3691886407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99b0071e00 0x7f99b00c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.828+0000 7f99affff640 1 -- 172.21.15.112:0/3691886407 >> v1:172.21.15.112:6789/0 conn(0x7f99b0072660 legacy=0x7f99b01561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.828+0000 7f99aeffd640 1 -- 172.21.15.112:0/3691886407 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 947186047 0 0) 0x7f99b0155a10 con 0x7f99b0072660 2024-04-25T01:25:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.828+0000 7f99affff640 1 -- 172.21.15.112:0/3691886407 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b00aa060 con 0x7f99b0152f80 2024-04-25T01:25:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.828+0000 7f99b4e42640 1 --2- 172.21.15.112:0/3691886407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99b0071e00 0x7f99b00c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:25:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.829+0000 7f99affff640 1 --2- 172.21.15.112:0/3691886407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99b0152f80 0x7f99b0158ef0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f99a00097a0 tx=0x7f99a002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada498feebe17604 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.829+0000 7f99aeffd640 1 -- 172.21.15.112:0/3691886407 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0037070 con 0x7f99b0152f80 2024-04-25T01:25:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.829+0000 7f99aeffd640 1 -- 172.21.15.112:0/3691886407 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99a002adb0 con 0x7f99b0152f80 2024-04-25T01:25:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.829+0000 7f99aeffd640 1 -- 172.21.15.112:0/3691886407 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0032660 con 0x7f99b0152f80 2024-04-25T01:25:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.830+0000 7f99b68a5640 1 -- 172.21.15.112:0/3691886407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99b0152f80 msgr2=0x7f99b0158ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.830+0000 7f99b68a5640 1 --2- 172.21.15.112:0/3691886407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99b0152f80 0x7f99b0158ef0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f99a00097a0 tx=0x7f99a002a060 comp rx=0 tx=0).stop 2024-04-25T01:25:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.830+0000 7f99b68a5640 1 -- 172.21.15.112:0/3691886407 shutdown_connections 2024-04-25T01:25:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.830+0000 7f99b68a5640 1 --2- 172.21.15.112:0/3691886407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99b0071e00 0x7f99b00c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.830+0000 7f99b68a5640 1 --2- 172.21.15.112:0/3691886407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99b0152f80 0x7f99b0158ef0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.830+0000 7f99b68a5640 1 -- 172.21.15.112:0/3691886407 >> 172.21.15.112:0/3691886407 conn(0x7f99b006c6c0 msgr2=0x7f99b006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.830+0000 7f99b68a5640 1 -- 172.21.15.112:0/3691886407 shutdown_connections 2024-04-25T01:25:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.830+0000 7f99b68a5640 1 -- 172.21.15.112:0/3691886407 wait complete. 2024-04-25T01:25:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.831+0000 7f99b68a5640 1 Processor -- start 2024-04-25T01:25:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.831+0000 7f99b68a5640 1 -- start start 2024-04-25T01:25:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.832+0000 7f99b68a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99b0071e00 0x7f99b006f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.832+0000 7f99b68a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99b0072660 0x7f99b014aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.832+0000 7f99b68a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99b0152f80 0x7f99b014b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.832+0000 7f99b68a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f99b0155cb0 con 0x7f99b0072660 2024-04-25T01:25:46.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.832+0000 7f99b68a5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f99b01568e0 con 0x7f99b0152f80 2024-04-25T01:25:46.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.832+0000 7f99b68a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f99b00aa620 con 0x7f99b0071e00 2024-04-25T01:25:46.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.832+0000 7f99af7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99b0072660 0x7f99b014aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:46.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.832+0000 7f99af7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99b0072660 0x7f99b014aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39346/0 (socket says 172.21.15.112:39346) 2024-04-25T01:25:46.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.832+0000 7f99af7fe640 1 -- 172.21.15.112:0/4091860110 learned_addr learned my addr 172.21.15.112:0/4091860110 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:46.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.832+0000 7f99affff640 1 --2- 172.21.15.112:0/4091860110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99b0071e00 0x7f99b006f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:46.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.832+0000 7f99b4e42640 1 --2- 172.21.15.112:0/4091860110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99b0152f80 0x7f99b014b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:46.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.832+0000 7f99af7fe640 1 -- 172.21.15.112:0/4091860110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99b0071e00 msgr2=0x7f99b006f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:46.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.832+0000 7f99af7fe640 1 --2- 172.21.15.112:0/4091860110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99b0071e00 0x7f99b006f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.832+0000 7f99af7fe640 1 -- 172.21.15.112:0/4091860110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99b0152f80 msgr2=0x7f99b014b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:46.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.832+0000 7f99af7fe640 1 --2- 172.21.15.112:0/4091860110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99b0152f80 0x7f99b014b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:46.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.832+0000 7f99af7fe640 1 -- 172.21.15.112:0/4091860110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b014b930 con 0x7f99b0072660 2024-04-25T01:25:46.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.832+0000 7f99af7fe640 1 --2- 172.21.15.112:0/4091860110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99b0072660 0x7f99b014aeb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f99a400c410 tx=0x7f99a400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:46.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.832+0000 7f99acff9640 1 -- 172.21.15.112:0/4091860110 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a40074e0 con 0x7f99b0072660 2024-04-25T01:25:46.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.832+0000 7f99acff9640 1 -- 172.21.15.112:0/4091860110 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99a40047e0 con 0x7f99b0072660 2024-04-25T01:25:46.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.832+0000 7f99b68a5640 1 -- 172.21.15.112:0/4091860110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b0157de0 con 0x7f99b0072660 2024-04-25T01:25:46.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.833+0000 7f99acff9640 1 -- 172.21.15.112:0/4091860110 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a4004bd0 con 0x7f99b0072660 2024-04-25T01:25:46.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.833+0000 7f99b68a5640 1 -- 172.21.15.112:0/4091860110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b01583a0 con 0x7f99b0072660 2024-04-25T01:25:46.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.833+0000 7f99b68a5640 1 -- 172.21.15.112:0/4091860110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b00689b0 con 0x7f99b0072660 2024-04-25T01:25:46.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.834+0000 7f99acff9640 1 -- 172.21.15.112:0/4091860110 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f99a4011070 con 0x7f99b0072660 2024-04-25T01:25:46.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.834+0000 7f99acff9640 1 --2- 172.21.15.112:0/4091860110 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f999c06be60 0x7f999c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:46.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.834+0000 7f99acff9640 1 -- 172.21.15.112:0/4091860110 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f99a408a5c0 con 0x7f99b0072660 2024-04-25T01:25:46.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.834+0000 7f99affff640 1 --2- 172.21.15.112:0/4091860110 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f999c06be60 0x7f999c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:46.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.835+0000 7f99affff640 1 --2- 172.21.15.112:0/4091860110 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f999c06be60 0x7f999c06e320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f99a0034000 tx=0x7f99a002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:46.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:46.839+0000 7f99acff9640 1 -- 172.21.15.112:0/4091860110 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f99a4059350 con 0x7f99b0072660 2024-04-25T01:25:47.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:47.046+0000 7f99b68a5640 1 -- 172.21.15.112:0/4091860110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99b00c2590 con 0x7f99b0072660 2024-04-25T01:25:47.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:47.046+0000 7f99acff9640 1 -- 172.21.15.112:0/4091860110 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f99a4058c50 con 0x7f99b0072660 2024-04-25T01:25:47.026 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:25:47.026 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/278686477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":278686477},{"type":"v1","addr":"172.21.15.112:6835","nonce":278686477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T01:25:47.027 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 2 2024-04-25T01:25:47.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:47.048+0000 7f99b68a5640 1 -- 172.21.15.112:0/4091860110 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f999c06be60 msgr2=0x7f999c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:47.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:47.048+0000 7f99b68a5640 1 --2- 172.21.15.112:0/4091860110 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f999c06be60 0x7f999c06e320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f99a0034000 tx=0x7f99a002e000 comp rx=0 tx=0).stop 2024-04-25T01:25:47.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:47.048+0000 7f99b68a5640 1 -- 172.21.15.112:0/4091860110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99b0072660 msgr2=0x7f99b014aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:47.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:47.048+0000 7f99b68a5640 1 --2- 172.21.15.112:0/4091860110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99b0072660 0x7f99b014aeb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f99a400c410 tx=0x7f99a400c8d0 comp rx=0 tx=0).stop 2024-04-25T01:25:47.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:47.048+0000 7f99b68a5640 1 -- 172.21.15.112:0/4091860110 shutdown_connections 2024-04-25T01:25:47.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:47.048+0000 7f99b68a5640 1 --2- 172.21.15.112:0/4091860110 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f999c06be60 0x7f999c06e320 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:47.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:47.048+0000 7f99b68a5640 1 --2- 172.21.15.112:0/4091860110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99b0152f80 0x7f99b014b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:47.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:47.048+0000 7f99b68a5640 1 --2- 172.21.15.112:0/4091860110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99b0072660 0x7f99b014aeb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:47.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:47.048+0000 7f99b68a5640 1 --2- 172.21.15.112:0/4091860110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99b0071e00 0x7f99b006f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:47.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:47.048+0000 7f99b68a5640 1 -- 172.21.15.112:0/4091860110 >> 172.21.15.112:0/4091860110 conn(0x7f99b006c6c0 msgr2=0x7f99b01502a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:47.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:47.048+0000 7f99b68a5640 1 -- 172.21.15.112:0/4091860110 shutdown_connections 2024-04-25T01:25:47.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:47.048+0000 7f99b68a5640 1 -- 172.21.15.112:0/4091860110 wait complete. 2024-04-25T01:25:49.040 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:25:49.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.168+0000 7ff076f9e640 1 Processor -- start 2024-04-25T01:25:49.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.168+0000 7ff076f9e640 1 -- start start 2024-04-25T01:25:49.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.169+0000 7ff076f9e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0700cb5c0 0x7ff070156d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:49.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.169+0000 7ff076f9e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff07014f460 0x7ff07014f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:49.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.169+0000 7ff076f9e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff070157250 con 0x7ff0701562a0 2024-04-25T01:25:49.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.169+0000 7ff076f9e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff070159300 con 0x7ff07014f460 2024-04-25T01:25:49.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.169+0000 7ff076f9e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff0700c6330 con 0x7ff0700cb5c0 2024-04-25T01:25:49.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.169+0000 7ff07679d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff07014f460 0x7ff07014f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:49.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.169+0000 7ff075f9c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0700cb5c0 0x7ff070156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:49.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.169+0000 7ff075f9c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0700cb5c0 0x7ff070156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:60310/0 (socket says 172.21.15.112:60310) 2024-04-25T01:25:49.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.169+0000 7ff07679d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff07014f460 0x7ff07014f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:52610/0 (socket says 172.21.15.112:52610) 2024-04-25T01:25:49.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.169+0000 7ff075f9c640 1 -- 172.21.15.112:0/4121064223 learned_addr learned my addr 172.21.15.112:0/4121064223 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:49.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.169+0000 7ff07679d640 1 -- 172.21.15.112:0/4121064223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0700cb5c0 msgr2=0x7ff070156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:49.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.169+0000 7ff07679d640 1 --2- 172.21.15.112:0/4121064223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0700cb5c0 0x7ff070156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.169+0000 7ff07679d640 1 -- 172.21.15.112:0/4121064223 >> v1:172.21.15.187:6789/0 conn(0x7ff0701562a0 legacy=0x7ff070159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:49.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.169+0000 7ff07679d640 1 -- 172.21.15.112:0/4121064223 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0700c5d70 con 0x7ff07014f460 2024-04-25T01:25:49.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.169+0000 7ff074f9a640 1 -- 172.21.15.112:0/4121064223 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 657087385 0 0) 0x7ff070157250 con 0x7ff0701562a0 2024-04-25T01:25:49.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.169+0000 7ff075f9c640 1 --2- 172.21.15.112:0/4121064223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0700cb5c0 0x7ff070156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:25:49.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.170+0000 7ff07679d640 1 --2- 172.21.15.112:0/4121064223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff07014f460 0x7ff07014f8b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff068009d90 tx=0x7ff06802a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8cee8484cb00fcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:49.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.170+0000 7ff074f9a640 1 -- 172.21.15.112:0/4121064223 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068007740 con 0x7ff07014f460 2024-04-25T01:25:49.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.170+0000 7ff074f9a640 1 -- 172.21.15.112:0/4121064223 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0680078e0 con 0x7ff07014f460 2024-04-25T01:25:49.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.170+0000 7ff074f9a640 1 -- 172.21.15.112:0/4121064223 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068007cb0 con 0x7ff07014f460 2024-04-25T01:25:49.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.171+0000 7ff076f9e640 1 -- 172.21.15.112:0/4121064223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff07014f460 msgr2=0x7ff07014f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:49.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.171+0000 7ff076f9e640 1 --2- 172.21.15.112:0/4121064223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff07014f460 0x7ff07014f8b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff068009d90 tx=0x7ff06802a530 comp rx=0 tx=0).stop 2024-04-25T01:25:49.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.171+0000 7ff076f9e640 1 -- 172.21.15.112:0/4121064223 shutdown_connections 2024-04-25T01:25:49.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.171+0000 7ff076f9e640 1 --2- 172.21.15.112:0/4121064223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff07014f460 0x7ff07014f8b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.171+0000 7ff076f9e640 1 --2- 172.21.15.112:0/4121064223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0700cb5c0 0x7ff070156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.171+0000 7ff076f9e640 1 -- 172.21.15.112:0/4121064223 >> 172.21.15.112:0/4121064223 conn(0x7ff0700745b0 msgr2=0x7ff0700749c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:49.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.171+0000 7ff076f9e640 1 -- 172.21.15.112:0/4121064223 shutdown_connections 2024-04-25T01:25:49.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.171+0000 7ff076f9e640 1 -- 172.21.15.112:0/4121064223 wait complete. 2024-04-25T01:25:49.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.172+0000 7ff076f9e640 1 Processor -- start 2024-04-25T01:25:49.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.172+0000 7ff076f9e640 1 -- start start 2024-04-25T01:25:49.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.173+0000 7ff076f9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0700cb5c0 0x7ff0700c2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:49.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.173+0000 7ff076f9e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff07014f460 0x7ff0701e84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:49.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.173+0000 7ff076f9e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0701562a0 0x7ff0700c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:49.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.173+0000 7ff076f9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff070156720 con 0x7ff0700cb5c0 2024-04-25T01:25:49.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.173+0000 7ff076f9e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff070158f90 con 0x7ff0701562a0 2024-04-25T01:25:49.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.173+0000 7ff076f9e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff0700be8c0 con 0x7ff07014f460 2024-04-25T01:25:49.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.173+0000 7ff075f9c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0700cb5c0 0x7ff0700c2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:49.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.173+0000 7ff075f9c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0700cb5c0 0x7ff0700c2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41058/0 (socket says 172.21.15.112:41058) 2024-04-25T01:25:49.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.173+0000 7ff075f9c640 1 -- 172.21.15.112:0/1968499712 learned_addr learned my addr 172.21.15.112:0/1968499712 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:49.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.173+0000 7ff07679d640 1 --2- 172.21.15.112:0/1968499712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0701562a0 0x7ff0700c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:49.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.173+0000 7ff07579b640 1 --2- 172.21.15.112:0/1968499712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff07014f460 0x7ff0701e84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:49.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.174+0000 7ff075f9c640 1 -- 172.21.15.112:0/1968499712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff07014f460 msgr2=0x7ff0701e84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:49.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.174+0000 7ff075f9c640 1 --2- 172.21.15.112:0/1968499712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff07014f460 0x7ff0701e84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.174+0000 7ff075f9c640 1 -- 172.21.15.112:0/1968499712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0701562a0 msgr2=0x7ff0700c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:49.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.174+0000 7ff075f9c640 1 --2- 172.21.15.112:0/1968499712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0701562a0 0x7ff0700c4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.174+0000 7ff075f9c640 1 -- 172.21.15.112:0/1968499712 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0701eede0 con 0x7ff0700cb5c0 2024-04-25T01:25:49.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.174+0000 7ff075f9c640 1 --2- 172.21.15.112:0/1968499712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0700cb5c0 0x7ff0700c2b30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00e9b0 tx=0x7ff06c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:49.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.174+0000 7ff0667fc640 1 -- 172.21.15.112:0/1968499712 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c017070 con 0x7ff0700cb5c0 2024-04-25T01:25:49.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.174+0000 7ff0667fc640 1 -- 172.21.15.112:0/1968499712 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff06c00c9a0 con 0x7ff0700cb5c0 2024-04-25T01:25:49.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.174+0000 7ff0667fc640 1 -- 172.21.15.112:0/1968499712 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c00cd50 con 0x7ff0700cb5c0 2024-04-25T01:25:49.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.174+0000 7ff076f9e640 1 -- 172.21.15.112:0/1968499712 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0701e89f0 con 0x7ff0700cb5c0 2024-04-25T01:25:49.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.174+0000 7ff076f9e640 1 -- 172.21.15.112:0/1968499712 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0700ad450 con 0x7ff0700cb5c0 2024-04-25T01:25:49.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.175+0000 7ff076f9e640 1 -- 172.21.15.112:0/1968499712 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0701ef290 con 0x7ff0700cb5c0 2024-04-25T01:25:49.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.175+0000 7ff0667fc640 1 -- 172.21.15.112:0/1968499712 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff06c01a070 con 0x7ff0700cb5c0 2024-04-25T01:25:49.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.176+0000 7ff0667fc640 1 --2- 172.21.15.112:0/1968499712 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff05406be30 0x7ff05406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:49.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.176+0000 7ff07579b640 1 --2- 172.21.15.112:0/1968499712 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff05406be30 0x7ff05406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:49.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.176+0000 7ff0667fc640 1 -- 172.21.15.112:0/1968499712 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff06c08a9c0 con 0x7ff0700cb5c0 2024-04-25T01:25:49.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.176+0000 7ff07579b640 1 --2- 172.21.15.112:0/1968499712 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff05406be30 0x7ff05406e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0700c20d0 tx=0x7ff060009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:49.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.180+0000 7ff0667fc640 1 -- 172.21.15.112:0/1968499712 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff06c08ce20 con 0x7ff0700cb5c0 2024-04-25T01:25:49.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.389+0000 7ff076f9e640 1 -- 172.21.15.112:0/1968499712 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff07004f500 con 0x7ff0700cb5c0 2024-04-25T01:25:49.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.389+0000 7ff0667fc640 1 -- 172.21.15.112:0/1968499712 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3873 (secure 0 0 0) 0x7ff06c0596c0 con 0x7ff0700cb5c0 2024-04-25T01:25:49.369 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:25:49.369 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/4047219598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4047219598},{"type":"v1","addr":"172.21.15.187:6834","nonce":4047219598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2204640907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2204640907},{"type":"v1","addr":"172.21.15.187:6835","nonce":2204640907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/278686477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":278686477},{"type":"v1","addr":"172.21.15.112:6835","nonce":278686477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2115984531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2115984531},{"type":"v1","addr":"172.21.15.112:6837","nonce":2115984531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T01:25:49.369 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 3 2024-04-25T01:25:49.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.391+0000 7ff076f9e640 1 -- 172.21.15.112:0/1968499712 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff05406be30 msgr2=0x7ff05406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:49.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.391+0000 7ff076f9e640 1 --2- 172.21.15.112:0/1968499712 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff05406be30 0x7ff05406e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0700c20d0 tx=0x7ff060009040 comp rx=0 tx=0).stop 2024-04-25T01:25:49.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.391+0000 7ff076f9e640 1 -- 172.21.15.112:0/1968499712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0700cb5c0 msgr2=0x7ff0700c2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:49.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.391+0000 7ff076f9e640 1 --2- 172.21.15.112:0/1968499712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0700cb5c0 0x7ff0700c2b30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00e9b0 tx=0x7ff06c00ee70 comp rx=0 tx=0).stop 2024-04-25T01:25:49.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.391+0000 7ff076f9e640 1 -- 172.21.15.112:0/1968499712 shutdown_connections 2024-04-25T01:25:49.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.391+0000 7ff076f9e640 1 --2- 172.21.15.112:0/1968499712 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff05406be30 0x7ff05406e2f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.391+0000 7ff076f9e640 1 --2- 172.21.15.112:0/1968499712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0701562a0 0x7ff0700c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.391+0000 7ff076f9e640 1 --2- 172.21.15.112:0/1968499712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff07014f460 0x7ff0701e84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.391+0000 7ff076f9e640 1 --2- 172.21.15.112:0/1968499712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0700cb5c0 0x7ff0700c2b30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.391+0000 7ff076f9e640 1 -- 172.21.15.112:0/1968499712 >> 172.21.15.112:0/1968499712 conn(0x7ff0700745b0 msgr2=0x7ff070157ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:49.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.391+0000 7ff076f9e640 1 -- 172.21.15.112:0/1968499712 shutdown_connections 2024-04-25T01:25:49.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.391+0000 7ff076f9e640 1 -- 172.21.15.112:0/1968499712 wait complete. 2024-04-25T01:25:49.382 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:25:49.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.498+0000 7fceb0c2c640 1 Processor -- start 2024-04-25T01:25:49.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.498+0000 7fceb0c2c640 1 -- start start 2024-04-25T01:25:49.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.499+0000 7fceb0c2c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fceac0713c0 0x7fceac0c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:49.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.499+0000 7fceb0c2c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceac159760 con 0x7fceac0c34b0 2024-04-25T01:25:49.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.499+0000 7fceb0c2c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceac1599e0 con 0x7fceac0c9230 2024-04-25T01:25:49.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.499+0000 7fceb0c2c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fceac159cc0 con 0x7fceac0713c0 2024-04-25T01:25:49.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.499+0000 7fceaad9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fceac0c34b0 0x7fceac160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50662/0 (socket says 172.21.15.112:50662) 2024-04-25T01:25:49.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.499+0000 7fceaad9d640 1 -- 172.21.15.112:0/3424014757 learned_addr learned my addr 172.21.15.112:0/3424014757 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:25:49.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.499+0000 7fcea9d9b640 1 --2- 172.21.15.112:0/3424014757 >> v2:172.21.15.187:3300/0 conn(0x7fceac0713c0 0x7fceac0c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:49.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.499+0000 7fcea959a640 1 -- 172.21.15.112:0/3424014757 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3226762701 0 0) 0x7fcea4001110 con 0x7fceac0c34b0 2024-04-25T01:25:49.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.500+0000 7fcea959a640 1 -- 172.21.15.112:0/3424014757 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce94003520 con 0x7fceac0c34b0 2024-04-25T01:25:49.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.500+0000 7fcea959a640 1 -- 172.21.15.112:0/3424014757 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 111516930 0 0) 0x7fce94003520 con 0x7fceac0c34b0 2024-04-25T01:25:49.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.500+0000 7fcea9d9b640 1 -- 172.21.15.112:0/3424014757 >> v1:172.21.15.187:6789/0 conn(0x7fceac0c9230 legacy=0x7fceac0c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:49.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.500+0000 7fcea9d9b640 1 -- 172.21.15.112:0/3424014757 >> v1:172.21.15.112:6789/0 conn(0x7fceac0c34b0 legacy=0x7fceac160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:49.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.500+0000 7fcea9d9b640 1 -- 172.21.15.112:0/3424014757 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceac070230 con 0x7fceac0713c0 2024-04-25T01:25:49.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.500+0000 7fcea9d9b640 1 --2- 172.21.15.112:0/3424014757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceac0713c0 0x7fceac0c77b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcea00097a0 tx=0x7fcea002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69fde1680dbd1507 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:49.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.501+0000 7fcea959a640 1 -- 172.21.15.112:0/3424014757 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0037070 con 0x7fceac0713c0 2024-04-25T01:25:49.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.501+0000 7fcea959a640 1 -- 172.21.15.112:0/3424014757 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea002abc0 con 0x7fceac0713c0 2024-04-25T01:25:49.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.501+0000 7fcea959a640 1 -- 172.21.15.112:0/3424014757 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4001840 con 0x7fceac0713c0 2024-04-25T01:25:49.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.501+0000 7fceb0c2c640 1 -- 172.21.15.112:0/3424014757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceac0713c0 msgr2=0x7fceac0c77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:49.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.501+0000 7fceb0c2c640 1 --2- 172.21.15.112:0/3424014757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceac0713c0 0x7fceac0c77b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcea00097a0 tx=0x7fcea002a130 comp rx=0 tx=0).stop 2024-04-25T01:25:49.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.501+0000 7fceb0c2c640 1 -- 172.21.15.112:0/3424014757 shutdown_connections 2024-04-25T01:25:49.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.501+0000 7fceb0c2c640 1 --2- 172.21.15.112:0/3424014757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceac0713c0 0x7fceac0c77b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.501+0000 7fceb0c2c640 1 -- 172.21.15.112:0/3424014757 >> 172.21.15.112:0/3424014757 conn(0x7fceac0745b0 msgr2=0x7fceac074990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:49.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.501+0000 7fceb0c2c640 1 -- 172.21.15.112:0/3424014757 shutdown_connections 2024-04-25T01:25:49.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.501+0000 7fceb0c2c640 1 -- 172.21.15.112:0/3424014757 wait complete. 2024-04-25T01:25:49.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.502+0000 7fceb0c2c640 1 Processor -- start 2024-04-25T01:25:49.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.503+0000 7fceb0c2c640 1 -- start start 2024-04-25T01:25:49.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.503+0000 7fceb0c2c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceac0c34b0 0x7fceac0aa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:49.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.503+0000 7fceb0c2c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceac0c9230 0x7fceac1550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:49.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.503+0000 7fceb0c2c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fceac155600 0x7fceac154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:49.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.503+0000 7fceb0c2c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fceac0c9790 con 0x7fceac0c9230 2024-04-25T01:25:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.503+0000 7fceb0c2c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fceac0c9610 con 0x7fceac0c34b0 2024-04-25T01:25:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.503+0000 7fceb0c2c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fceac0c9910 con 0x7fceac155600 2024-04-25T01:25:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.503+0000 7fcea9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceac0c9230 0x7fceac1550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.503+0000 7fcea9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceac0c9230 0x7fceac1550c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41074/0 (socket says 172.21.15.112:41074) 2024-04-25T01:25:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.503+0000 7fcea9d9b640 1 -- 172.21.15.112:0/707496162 learned_addr learned my addr 172.21.15.112:0/707496162 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.503+0000 7fcea9d9b640 1 -- 172.21.15.112:0/707496162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fceac155600 msgr2=0x7fceac154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.503+0000 7fcea9d9b640 1 --2- 172.21.15.112:0/707496162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fceac155600 0x7fceac154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.503+0000 7fcea9d9b640 1 -- 172.21.15.112:0/707496162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceac0c34b0 msgr2=0x7fceac0aa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.503+0000 7fceaa59c640 1 --2- 172.21.15.112:0/707496162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceac0c34b0 0x7fceac0aa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.504+0000 7fcea9d9b640 1 --2- 172.21.15.112:0/707496162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceac0c34b0 0x7fceac0aa5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.504+0000 7fcea9d9b640 1 -- 172.21.15.112:0/707496162 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceac1ebb70 con 0x7fceac0c9230 2024-04-25T01:25:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.504+0000 7fceaa59c640 1 --2- 172.21.15.112:0/707496162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceac0c34b0 0x7fceac0aa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:25:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.504+0000 7fcea9d9b640 1 --2- 172.21.15.112:0/707496162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceac0c9230 0x7fceac1550c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcea0035040 tx=0x7fcea00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.504+0000 7fce9affd640 1 -- 172.21.15.112:0/707496162 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0037050 con 0x7fceac0c9230 2024-04-25T01:25:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.504+0000 7fce9affd640 1 -- 172.21.15.112:0/707496162 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea0043070 con 0x7fceac0c9230 2024-04-25T01:25:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.504+0000 7fceb0c2c640 1 -- 172.21.15.112:0/707496162 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceac1ebe50 con 0x7fceac0c9230 2024-04-25T01:25:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.504+0000 7fce9affd640 1 -- 172.21.15.112:0/707496162 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0006e10 con 0x7fceac0c9230 2024-04-25T01:25:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.504+0000 7fceb0c2c640 1 -- 172.21.15.112:0/707496162 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceac1ec410 con 0x7fceac0c9230 2024-04-25T01:25:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.505+0000 7fceb0c2c640 1 -- 172.21.15.112:0/707496162 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceac0682a0 con 0x7fceac0c9230 2024-04-25T01:25:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.505+0000 7fce9affd640 1 -- 172.21.15.112:0/707496162 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcea003ca30 con 0x7fceac0c9230 2024-04-25T01:25:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.505+0000 7fce9affd640 1 --2- 172.21.15.112:0/707496162 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fce8c06be60 0x7fce8c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.506+0000 7fce9affd640 1 -- 172.21.15.112:0/707496162 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fcea00ab5a0 con 0x7fceac0c9230 2024-04-25T01:25:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.506+0000 7fceaa59c640 1 --2- 172.21.15.112:0/707496162 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fce8c06be60 0x7fce8c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.506+0000 7fceaa59c640 1 --2- 172.21.15.112:0/707496162 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fce8c06be60 0x7fce8c06e320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fceac14f470 tx=0x7fce9c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:49.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.510+0000 7fce9affd640 1 -- 172.21.15.112:0/707496162 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcea007a450 con 0x7fceac0c9230 2024-04-25T01:25:49.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.684+0000 7fceb0c2c640 1 -- 172.21.15.112:0/707496162 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fceac0c3290 con 0x7fceac0c9230 2024-04-25T01:25:49.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.685+0000 7fce9affd640 1 -- 172.21.15.112:0/707496162 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12908 (secure 0 0 0) 0x7fcea0079d50 con 0x7fceac0c9230 2024-04-25T01:25:49.666 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:25:49.666 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":16,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:25:42.691350+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:25:49.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.687+0000 7fceb0c2c640 1 -- 172.21.15.112:0/707496162 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fce8c06be60 msgr2=0x7fce8c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:49.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.687+0000 7fceb0c2c640 1 --2- 172.21.15.112:0/707496162 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fce8c06be60 0x7fce8c06e320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fceac14f470 tx=0x7fce9c009040 comp rx=0 tx=0).stop 2024-04-25T01:25:49.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.687+0000 7fceb0c2c640 1 -- 172.21.15.112:0/707496162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceac0c9230 msgr2=0x7fceac1550c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:49.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.687+0000 7fceb0c2c640 1 --2- 172.21.15.112:0/707496162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceac0c9230 0x7fceac1550c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcea0035040 tx=0x7fcea00046c0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.687+0000 7fceb0c2c640 1 -- 172.21.15.112:0/707496162 shutdown_connections 2024-04-25T01:25:49.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.687+0000 7fceb0c2c640 1 --2- 172.21.15.112:0/707496162 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fce8c06be60 0x7fce8c06e320 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.687+0000 7fceb0c2c640 1 --2- 172.21.15.112:0/707496162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fceac155600 0x7fceac154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.687+0000 7fceb0c2c640 1 --2- 172.21.15.112:0/707496162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceac0c9230 0x7fceac1550c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.687+0000 7fceb0c2c640 1 --2- 172.21.15.112:0/707496162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceac0c34b0 0x7fceac0aa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.687+0000 7fceb0c2c640 1 -- 172.21.15.112:0/707496162 >> 172.21.15.112:0/707496162 conn(0x7fceac0745b0 msgr2=0x7fceac0c4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:49.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.687+0000 7fceb0c2c640 1 -- 172.21.15.112:0/707496162 shutdown_connections 2024-04-25T01:25:49.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.687+0000 7fceb0c2c640 1 -- 172.21.15.112:0/707496162 wait complete. 2024-04-25T01:25:49.677 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:25:40.679119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:25:49.678 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:25:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.797+0000 7f6824b7e640 1 Processor -- start 2024-04-25T01:25:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.797+0000 7f6824b7e640 1 -- start start 2024-04-25T01:25:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.798+0000 7f6824b7e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68200ca370 0x7f682006c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.798+0000 7f6824b7e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f68200ca8a0 0x7f68200c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.798+0000 7f6824b7e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68201534a0 con 0x7f68200c9250 2024-04-25T01:25:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.798+0000 7f6824b7e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6820156990 con 0x7f68200ca8a0 2024-04-25T01:25:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.798+0000 7f6824b7e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6820156500 con 0x7f68200ca370 2024-04-25T01:25:49.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.798+0000 7f681e59c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f68200c9250 0x7f682006bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50668/0 (socket says 172.21.15.112:50668) 2024-04-25T01:25:49.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.798+0000 7f681e59c640 1 -- 172.21.15.112:0/1963590506 learned_addr learned my addr 172.21.15.112:0/1963590506 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:25:49.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.798+0000 7f681dd9b640 1 --2- 172.21.15.112:0/1963590506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68200ca370 0x7f682006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:49.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.798+0000 7f681ed9d640 1 --2- 172.21.15.112:0/1963590506 >> v2:172.21.15.187:3300/0 conn(0x7f68200ca8a0 0x7f68200c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:49.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.798+0000 7f681d59a640 1 -- 172.21.15.112:0/1963590506 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3347358315 0 0) 0x7f68201534a0 con 0x7f68200c9250 2024-04-25T01:25:49.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.798+0000 7f681d59a640 1 -- 172.21.15.112:0/1963590506 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6808003520 con 0x7f68200c9250 2024-04-25T01:25:49.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.799+0000 7f681d59a640 1 -- 172.21.15.112:0/1963590506 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2579643829 0 0) 0x7f6808003520 con 0x7f68200c9250 2024-04-25T01:25:49.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.799+0000 7f681ed9d640 1 -- 172.21.15.112:0/1963590506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68200ca370 msgr2=0x7f682006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:49.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.799+0000 7f681ed9d640 1 --2- 172.21.15.112:0/1963590506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68200ca370 0x7f682006c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.799+0000 7f681ed9d640 1 -- 172.21.15.112:0/1963590506 >> v1:172.21.15.112:6789/0 conn(0x7f68200c9250 legacy=0x7f682006bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:49.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.799+0000 7f681ed9d640 1 -- 172.21.15.112:0/1963590506 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68200cae30 con 0x7f68200ca8a0 2024-04-25T01:25:49.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.799+0000 7f681dd9b640 1 --2- 172.21.15.112:0/1963590506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68200ca370 0x7f682006c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:25:49.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.800+0000 7f681ed9d640 1 --2- 172.21.15.112:0/1963590506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68200ca8a0 0x7f68200c9630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f68180097a0 tx=0x7f681802a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18d3575152c64d2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:49.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.800+0000 7f681d59a640 1 -- 172.21.15.112:0/1963590506 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818037070 con 0x7f68200ca8a0 2024-04-25T01:25:49.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.800+0000 7f681d59a640 1 -- 172.21.15.112:0/1963590506 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f681802ab10 con 0x7f68200ca8a0 2024-04-25T01:25:49.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.800+0000 7f681d59a640 1 -- 172.21.15.112:0/1963590506 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68100019f0 con 0x7f68200ca8a0 2024-04-25T01:25:49.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.800+0000 7f6824b7e640 1 -- 172.21.15.112:0/1963590506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68200ca8a0 msgr2=0x7f68200c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:49.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.800+0000 7f6824b7e640 1 --2- 172.21.15.112:0/1963590506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68200ca8a0 0x7f68200c9630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f68180097a0 tx=0x7f681802a080 comp rx=0 tx=0).stop 2024-04-25T01:25:49.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.800+0000 7f6824b7e640 1 -- 172.21.15.112:0/1963590506 shutdown_connections 2024-04-25T01:25:49.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.800+0000 7f6824b7e640 1 --2- 172.21.15.112:0/1963590506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68200ca8a0 0x7f68200c9630 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.800+0000 7f6824b7e640 1 --2- 172.21.15.112:0/1963590506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68200ca370 0x7f682006c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.800+0000 7f6824b7e640 1 -- 172.21.15.112:0/1963590506 >> 172.21.15.112:0/1963590506 conn(0x7f68200cb310 msgr2=0x7f68200cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:49.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.800+0000 7f6824b7e640 1 -- 172.21.15.112:0/1963590506 shutdown_connections 2024-04-25T01:25:49.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.800+0000 7f6824b7e640 1 -- 172.21.15.112:0/1963590506 wait complete. 2024-04-25T01:25:49.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.802+0000 7f6824b7e640 1 Processor -- start 2024-04-25T01:25:49.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.802+0000 7f6824b7e640 1 -- start start 2024-04-25T01:25:49.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.802+0000 7f6824b7e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68200c9250 0x7f68200bea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:49.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.802+0000 7f6824b7e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68200ca370 0x7f682014ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:49.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.802+0000 7f6824b7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68200ca8a0 0x7f682014f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:49.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.802+0000 7f6824b7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6820155eb0 con 0x7f68200ca8a0 2024-04-25T01:25:49.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.802+0000 7f6824b7e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6820155bd0 con 0x7f68200ca370 2024-04-25T01:25:49.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.802+0000 7f6824b7e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6820156190 con 0x7f68200c9250 2024-04-25T01:25:49.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.802+0000 7f681ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68200ca8a0 0x7f682014f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:49.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.802+0000 7f681ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68200ca8a0 0x7f682014f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41090/0 (socket says 172.21.15.112:41090) 2024-04-25T01:25:49.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.802+0000 7f681ed9d640 1 -- 172.21.15.112:0/1312452869 learned_addr learned my addr 172.21.15.112:0/1312452869 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:49.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.803+0000 7f681dd9b640 1 --2- 172.21.15.112:0/1312452869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68200ca370 0x7f682014ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:49.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.803+0000 7f681e59c640 1 --2- 172.21.15.112:0/1312452869 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68200c9250 0x7f68200bea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.803+0000 7f681ed9d640 1 -- 172.21.15.112:0/1312452869 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68200c9250 msgr2=0x7f68200bea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.803+0000 7f681ed9d640 1 --2- 172.21.15.112:0/1312452869 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68200c9250 0x7f68200bea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.803+0000 7f681ed9d640 1 -- 172.21.15.112:0/1312452869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68200ca370 msgr2=0x7f682014ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.803+0000 7f681ed9d640 1 --2- 172.21.15.112:0/1312452869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68200ca370 0x7f682014ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.803+0000 7f681ed9d640 1 -- 172.21.15.112:0/1312452869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682014fa60 con 0x7f68200ca8a0 2024-04-25T01:25:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.803+0000 7f681ed9d640 1 --2- 172.21.15.112:0/1312452869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68200ca8a0 0x7f682014f470 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f68180097a0 tx=0x7f68180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.803+0000 7f680effd640 1 -- 172.21.15.112:0/1312452869 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818037050 con 0x7f68200ca8a0 2024-04-25T01:25:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.803+0000 7f680effd640 1 -- 172.21.15.112:0/1312452869 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6818043070 con 0x7f68200ca8a0 2024-04-25T01:25:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.803+0000 7f680effd640 1 -- 172.21.15.112:0/1312452869 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818006e10 con 0x7f68200ca8a0 2024-04-25T01:25:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.803+0000 7f6824b7e640 1 -- 172.21.15.112:0/1312452869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682014fd20 con 0x7f68200ca8a0 2024-04-25T01:25:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.803+0000 7f6824b7e640 1 -- 172.21.15.112:0/1312452869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68201eb8c0 con 0x7f68200ca8a0 2024-04-25T01:25:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.804+0000 7f6824b7e640 1 -- 172.21.15.112:0/1312452869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f682004f500 con 0x7f68200ca8a0 2024-04-25T01:25:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.804+0000 7f680effd640 1 -- 172.21.15.112:0/1312452869 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f681803ca50 con 0x7f68200ca8a0 2024-04-25T01:25:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.805+0000 7f680effd640 1 --2- 172.21.15.112:0/1312452869 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f680006be60 0x7f680006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.805+0000 7f680effd640 1 -- 172.21.15.112:0/1312452869 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f68180ab470 con 0x7f68200ca8a0 2024-04-25T01:25:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.805+0000 7f681e59c640 1 --2- 172.21.15.112:0/1312452869 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f680006be60 0x7f680006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.805+0000 7f681e59c640 1 --2- 172.21.15.112:0/1312452869 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f680006be60 0x7f680006e320 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f68100012e0 tx=0x7f6810009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:49.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.809+0000 7f680effd640 1 -- 172.21.15.112:0/1312452869 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6818079cd0 con 0x7f68200ca8a0 2024-04-25T01:25:49.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.981+0000 7f6824b7e640 1 -- 172.21.15.112:0/1312452869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6820154530 con 0x7f68200ca8a0 2024-04-25T01:25:49.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.981+0000 7f680effd640 1 -- 172.21.15.112:0/1312452869 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f681807dfd0 con 0x7f68200ca8a0 2024-04-25T01:25:49.961 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T01:25:49.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.983+0000 7f6824b7e640 1 -- 172.21.15.112:0/1312452869 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f680006be60 msgr2=0x7f680006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:49.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.983+0000 7f6824b7e640 1 --2- 172.21.15.112:0/1312452869 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f680006be60 0x7f680006e320 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f68100012e0 tx=0x7f6810009040 comp rx=0 tx=0).stop 2024-04-25T01:25:49.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.983+0000 7f6824b7e640 1 -- 172.21.15.112:0/1312452869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68200ca8a0 msgr2=0x7f682014f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:49.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.983+0000 7f6824b7e640 1 --2- 172.21.15.112:0/1312452869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68200ca8a0 0x7f682014f470 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f68180097a0 tx=0x7f68180046c0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.983+0000 7f6824b7e640 1 -- 172.21.15.112:0/1312452869 shutdown_connections 2024-04-25T01:25:49.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.983+0000 7f6824b7e640 1 --2- 172.21.15.112:0/1312452869 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f680006be60 0x7f680006e320 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.983+0000 7f6824b7e640 1 --2- 172.21.15.112:0/1312452869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68200ca8a0 0x7f682014f470 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.983+0000 7f6824b7e640 1 --2- 172.21.15.112:0/1312452869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68200ca370 0x7f682014ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.983+0000 7f6824b7e640 1 --2- 172.21.15.112:0/1312452869 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68200c9250 0x7f68200bea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:49.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.983+0000 7f6824b7e640 1 -- 172.21.15.112:0/1312452869 >> 172.21.15.112:0/1312452869 conn(0x7f68200cb310 msgr2=0x7f68201538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:49.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.984+0000 7f6824b7e640 1 -- 172.21.15.112:0/1312452869 shutdown_connections 2024-04-25T01:25:49.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:49.984+0000 7f6824b7e640 1 -- 172.21.15.112:0/1312452869 wait complete. 2024-04-25T01:25:49.974 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T01:25:50.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.091+0000 7f5a5dca3640 1 Processor -- start 2024-04-25T01:25:50.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.091+0000 7f5a5dca3640 1 -- start start 2024-04-25T01:25:50.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.092+0000 7f5a5dca3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a58072d80 0x7f5a581536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.092+0000 7f5a5dca3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5a58071db0 0x7f5a58072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.092+0000 7f5a5dca3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a5806e0f0 con 0x7f5a580c1a80 2024-04-25T01:25:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.092+0000 7f5a5dca3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5a580c1f00 con 0x7f5a58071db0 2024-04-25T01:25:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.092+0000 7f5a5dca3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5a580c2180 con 0x7f5a58072d80 2024-04-25T01:25:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.092+0000 7f5a577fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a58072d80 0x7f5a581536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.092+0000 7f5a57fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5a58071db0 0x7f5a58072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.092+0000 7f5a56ffd640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f5a580c1a80 0x7f5a5814d340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:44534/0 (socket says 172.21.15.112:44534) 2024-04-25T01:25:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.092+0000 7f5a577fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a58072d80 0x7f5a581536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:60352/0 (socket says 172.21.15.112:60352) 2024-04-25T01:25:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.092+0000 7f5a56ffd640 1 -- 172.21.15.112:0/1135769774 learned_addr learned my addr 172.21.15.112:0/1135769774 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:25:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.092+0000 7f5a57fff640 1 --2- 172.21.15.112:0/1135769774 >> v2:172.21.15.187:3300/0 conn(0x7f5a58071db0 0x7f5a58072240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:52670/0 (socket says 172.21.15.112:52670) 2024-04-25T01:25:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.093+0000 7f5a57fff640 1 -- 172.21.15.112:0/1135769774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a58072d80 msgr2=0x7f5a581536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.093+0000 7f5a57fff640 1 --2- 172.21.15.112:0/1135769774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a58072d80 0x7f5a581536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.093+0000 7f5a57fff640 1 -- 172.21.15.112:0/1135769774 >> v1:172.21.15.187:6789/0 conn(0x7f5a580c1a80 legacy=0x7f5a5814d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.093+0000 7f5a57fff640 1 -- 172.21.15.112:0/1135769774 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a580c2460 con 0x7f5a58071db0 2024-04-25T01:25:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.093+0000 7f5a567fc640 1 -- 172.21.15.112:0/1135769774 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 668157902 0 0) 0x7f5a4c001110 con 0x7f5a580c1a80 2024-04-25T01:25:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.093+0000 7f5a57fff640 1 --2- 172.21.15.112:0/1135769774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a58071db0 0x7f5a58072240 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5a50009d90 tx=0x7f5a5002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b212c14592553270 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.094+0000 7f5a567fc640 1 -- 172.21.15.112:0/1135769774 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a50007740 con 0x7f5a58071db0 2024-04-25T01:25:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.094+0000 7f5a567fc640 1 -- 172.21.15.112:0/1135769774 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a500078e0 con 0x7f5a58071db0 2024-04-25T01:25:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.094+0000 7f5a567fc640 1 -- 172.21.15.112:0/1135769774 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a50007cb0 con 0x7f5a58071db0 2024-04-25T01:25:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.094+0000 7f5a5dca3640 1 -- 172.21.15.112:0/1135769774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a58071db0 msgr2=0x7f5a58072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.094+0000 7f5a5dca3640 1 --2- 172.21.15.112:0/1135769774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a58071db0 0x7f5a58072240 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5a50009d90 tx=0x7f5a5002a660 comp rx=0 tx=0).stop 2024-04-25T01:25:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.094+0000 7f5a5dca3640 1 -- 172.21.15.112:0/1135769774 shutdown_connections 2024-04-25T01:25:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.094+0000 7f5a5dca3640 1 --2- 172.21.15.112:0/1135769774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a58071db0 0x7f5a58072240 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.094+0000 7f5a5dca3640 1 --2- 172.21.15.112:0/1135769774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a58072d80 0x7f5a581536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.094+0000 7f5a5dca3640 1 -- 172.21.15.112:0/1135769774 >> 172.21.15.112:0/1135769774 conn(0x7f5a5806b160 msgr2=0x7f5a5806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:50.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.094+0000 7f5a5dca3640 1 -- 172.21.15.112:0/1135769774 shutdown_connections 2024-04-25T01:25:50.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.094+0000 7f5a5dca3640 1 -- 172.21.15.112:0/1135769774 wait complete. 2024-04-25T01:25:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.095+0000 7f5a5dca3640 1 Processor -- start 2024-04-25T01:25:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.095+0000 7f5a5dca3640 1 -- start start 2024-04-25T01:25:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.096+0000 7f5a5dca3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a58071db0 0x7f5a580c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.096+0000 7f5a5dca3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a58072d80 0x7f5a58156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.096+0000 7f5a5dca3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a580c1a80 0x7f5a580c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.096+0000 7f5a5dca3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a58071890 con 0x7f5a58071db0 2024-04-25T01:25:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.096+0000 7f5a5dca3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5a580c2a20 con 0x7f5a580c1a80 2024-04-25T01:25:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.096+0000 7f5a5dca3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5a5814da50 con 0x7f5a58072d80 2024-04-25T01:25:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.096+0000 7f5a577fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a58071db0 0x7f5a580c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.096+0000 7f5a577fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a58071db0 0x7f5a580c7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41100/0 (socket says 172.21.15.112:41100) 2024-04-25T01:25:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.096+0000 7f5a577fe640 1 -- 172.21.15.112:0/477175542 learned_addr learned my addr 172.21.15.112:0/477175542 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.096+0000 7f5a57fff640 1 --2- 172.21.15.112:0/477175542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a580c1a80 0x7f5a580c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.096+0000 7f5a56ffd640 1 --2- 172.21.15.112:0/477175542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a58072d80 0x7f5a58156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.096+0000 7f5a577fe640 1 -- 172.21.15.112:0/477175542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a58072d80 msgr2=0x7f5a58156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.097+0000 7f5a577fe640 1 --2- 172.21.15.112:0/477175542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a58072d80 0x7f5a58156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.097+0000 7f5a577fe640 1 -- 172.21.15.112:0/477175542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a580c1a80 msgr2=0x7f5a580c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.097+0000 7f5a577fe640 1 --2- 172.21.15.112:0/477175542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a580c1a80 0x7f5a580c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:50.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.097+0000 7f5a577fe640 1 -- 172.21.15.112:0/477175542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a581ed1c0 con 0x7f5a58071db0 2024-04-25T01:25:50.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.097+0000 7f5a56ffd640 1 --2- 172.21.15.112:0/477175542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a58072d80 0x7f5a58156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:25:50.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.097+0000 7f5a577fe640 1 --2- 172.21.15.112:0/477175542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a58071db0 0x7f5a580c7ac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5a4800e9b0 tx=0x7f5a4800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:50.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.097+0000 7f5a3ffff640 1 -- 172.21.15.112:0/477175542 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a48017070 con 0x7f5a58071db0 2024-04-25T01:25:50.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.097+0000 7f5a3ffff640 1 -- 172.21.15.112:0/477175542 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a4800c9a0 con 0x7f5a58071db0 2024-04-25T01:25:50.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.097+0000 7f5a5dca3640 1 -- 172.21.15.112:0/477175542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a581577e0 con 0x7f5a58071db0 2024-04-25T01:25:50.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.097+0000 7f5a3ffff640 1 -- 172.21.15.112:0/477175542 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4800cd90 con 0x7f5a58071db0 2024-04-25T01:25:50.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.097+0000 7f5a5dca3640 1 -- 172.21.15.112:0/477175542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a581ecee0 con 0x7f5a58071db0 2024-04-25T01:25:50.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.098+0000 7f5a5dca3640 1 -- 172.21.15.112:0/477175542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a581eccd0 con 0x7f5a58071db0 2024-04-25T01:25:50.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.098+0000 7f5a3ffff640 1 -- 172.21.15.112:0/477175542 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5a4801a070 con 0x7f5a58071db0 2024-04-25T01:25:50.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.098+0000 7f5a3ffff640 1 --2- 172.21.15.112:0/477175542 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5a4406be30 0x7f5a4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:50.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.099+0000 7f5a3ffff640 1 -- 172.21.15.112:0/477175542 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f5a4808aae0 con 0x7f5a58071db0 2024-04-25T01:25:50.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.099+0000 7f5a56ffd640 1 --2- 172.21.15.112:0/477175542 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5a4406be30 0x7f5a4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:50.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.099+0000 7f5a56ffd640 1 --2- 172.21.15.112:0/477175542 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5a4406be30 0x7f5a4406e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c007fd0 tx=0x7f5a4c007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:50.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.103+0000 7f5a3ffff640 1 -- 172.21.15.112:0/477175542 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5a480591a0 con 0x7f5a58071db0 2024-04-25T01:25:50.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.277+0000 7f5a5dca3640 1 -- 172.21.15.112:0/477175542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5a580682a0 con 0x7f5a58071db0 2024-04-25T01:25:50.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.277+0000 7f5a3ffff640 1 -- 172.21.15.112:0/477175542 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f5a4805d4a0 con 0x7f5a58071db0 2024-04-25T01:25:50.257 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:25:50.257 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2024-04-25T01:25:50.257 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:25:50.257 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2024-04-25T01:25:50.257 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2024-04-25T01:25:50.257 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:25:50.257 INFO:teuthology.orchestra.run.smithi112.stdout:] 2024-04-25T01:25:50.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.279+0000 7f5a5dca3640 1 -- 172.21.15.112:0/477175542 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5a4406be30 msgr2=0x7f5a4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:50.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.279+0000 7f5a5dca3640 1 --2- 172.21.15.112:0/477175542 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5a4406be30 0x7f5a4406e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c007fd0 tx=0x7f5a4c007dc0 comp rx=0 tx=0).stop 2024-04-25T01:25:50.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.279+0000 7f5a5dca3640 1 -- 172.21.15.112:0/477175542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a58071db0 msgr2=0x7f5a580c7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:50.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.279+0000 7f5a5dca3640 1 --2- 172.21.15.112:0/477175542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a58071db0 0x7f5a580c7ac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5a4800e9b0 tx=0x7f5a4800ee70 comp rx=0 tx=0).stop 2024-04-25T01:25:50.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.279+0000 7f5a5dca3640 1 -- 172.21.15.112:0/477175542 shutdown_connections 2024-04-25T01:25:50.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.279+0000 7f5a5dca3640 1 --2- 172.21.15.112:0/477175542 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5a4406be30 0x7f5a4406e2f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:50.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.279+0000 7f5a5dca3640 1 --2- 172.21.15.112:0/477175542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a580c1a80 0x7f5a580c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:50.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.279+0000 7f5a5dca3640 1 --2- 172.21.15.112:0/477175542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a58072d80 0x7f5a58156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:50.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.279+0000 7f5a5dca3640 1 --2- 172.21.15.112:0/477175542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a58071db0 0x7f5a580c7ac0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:50.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.279+0000 7f5a5dca3640 1 -- 172.21.15.112:0/477175542 >> 172.21.15.112:0/477175542 conn(0x7f5a5806b160 msgr2=0x7f5a5806b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:50.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.279+0000 7f5a5dca3640 1 -- 172.21.15.112:0/477175542 shutdown_connections 2024-04-25T01:25:50.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.279+0000 7f5a5dca3640 1 -- 172.21.15.112:0/477175542 wait complete. 2024-04-25T01:25:50.269 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'a' 2024-04-25T01:25:50.269 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create a_metadata --pg_num_min 64 2024-04-25T01:25:50.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.389+0000 7fbc72fb7640 1 Processor -- start 2024-04-25T01:25:50.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.390+0000 7fbc72fb7640 1 -- start start 2024-04-25T01:25:50.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.390+0000 7fbc72fb7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbc6c1566c0 0x7fbc6c0c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:50.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.390+0000 7fbc72fb7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc6c0c6270 con 0x7fbc6c14fbd0 2024-04-25T01:25:50.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.390+0000 7fbc72fb7640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc6c06c410 con 0x7fbc6c06cef0 2024-04-25T01:25:50.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.390+0000 7fbc72fb7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbc6c06c1c0 con 0x7fbc6c1566c0 2024-04-25T01:25:50.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.390+0000 7fbc70d53640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fbc6c14fbd0 0x7fbc6c0c5a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50676/0 (socket says 172.21.15.112:50676) 2024-04-25T01:25:50.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.390+0000 7fbc70d53640 1 -- 172.21.15.112:0/2787326578 learned_addr learned my addr 172.21.15.112:0/2787326578 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:25:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6b7fe640 1 -- 172.21.15.112:0/2787326578 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 91169773 0 0) 0x7fbc6c0c6270 con 0x7fbc6c14fbd0 2024-04-25T01:25:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6b7fe640 1 -- 172.21.15.112:0/2787326578 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc5c003520 con 0x7fbc6c14fbd0 2024-04-25T01:25:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc71554640 1 --2- 172.21.15.112:0/2787326578 >> v2:172.21.15.187:3300/0 conn(0x7fbc6c1566c0 0x7fbc6c0c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6b7fe640 1 -- 172.21.15.112:0/2787326578 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 532035179 0 0) 0x7fbc5c003520 con 0x7fbc6c14fbd0 2024-04-25T01:25:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6b7fe640 1 -- 172.21.15.112:0/2787326578 >> v2:172.21.15.187:3300/0 conn(0x7fbc6c1566c0 msgr2=0x7fbc6c0c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6b7fe640 1 --2- 172.21.15.112:0/2787326578 >> v2:172.21.15.187:3300/0 conn(0x7fbc6c1566c0 0x7fbc6c0c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6b7fe640 1 -- 172.21.15.112:0/2787326578 >> v1:172.21.15.187:6789/0 conn(0x7fbc6c06cef0 legacy=0x7fbc6c0c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6b7fe640 1 -- 172.21.15.112:0/2787326578 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6c06c6f0 con 0x7fbc6c14fbd0 2024-04-25T01:25:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6b7fe640 1 -- 172.21.15.112:0/2787326578 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fbc60001670 con 0x7fbc6c14fbd0 2024-04-25T01:25:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6b7fe640 1 -- 172.21.15.112:0/2787326578 >> v1:172.21.15.112:6789/0 conn(0x7fbc6c14fbd0 legacy=0x7fbc6c0c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6b7fe640 1 --2- 172.21.15.112:0/2787326578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc5c0039b0 0x7fbc5c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6b7fe640 1 --2- 172.21.15.112:0/2787326578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc5c004800 0x7fbc5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6b7fe640 1 --2- 172.21.15.112:0/2787326578 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c0052e0 0x7fbc5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6b7fe640 1 -- 172.21.15.112:0/2787326578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c06c1c0 con 0x7fbc5c004800 2024-04-25T01:25:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6b7fe640 1 -- 172.21.15.112:0/2787326578 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c00c1b0 con 0x7fbc5c0039b0 2024-04-25T01:25:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6b7fe640 1 -- 172.21.15.112:0/2787326578 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbc5c00c3b0 con 0x7fbc5c0052e0 2024-04-25T01:25:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6bfff640 1 --2- 172.21.15.112:0/2787326578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc5c004800 0x7fbc5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc70d53640 1 --2- 172.21.15.112:0/2787326578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc5c0039b0 0x7fbc5c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc71554640 1 --2- 172.21.15.112:0/2787326578 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c0052e0 0x7fbc5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6bfff640 1 -- 172.21.15.112:0/2787326578 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c0052e0 msgr2=0x7fbc5c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6bfff640 1 --2- 172.21.15.112:0/2787326578 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c0052e0 0x7fbc5c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6bfff640 1 -- 172.21.15.112:0/2787326578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc5c0039b0 msgr2=0x7fbc5c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6bfff640 1 --2- 172.21.15.112:0/2787326578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc5c0039b0 0x7fbc5c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc71554640 1 -- 172.21.15.112:0/2787326578 reap_dead start 2024-04-25T01:25:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.391+0000 7fbc6bfff640 1 -- 172.21.15.112:0/2787326578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5c003520 con 0x7fbc5c004800 2024-04-25T01:25:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.392+0000 7fbc6bfff640 1 --2- 172.21.15.112:0/2787326578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc5c004800 0x7fbc5c004c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c06d3a0 tx=0x7fbc5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2571dcb396da5018 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.392+0000 7fbc6b7fe640 1 -- 172.21.15.112:0/2787326578 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58001370 con 0x7fbc5c004800 2024-04-25T01:25:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.392+0000 7fbc6b7fe640 1 -- 172.21.15.112:0/2787326578 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc580052d0 con 0x7fbc5c004800 2024-04-25T01:25:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.392+0000 7fbc6b7fe640 1 -- 172.21.15.112:0/2787326578 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5802ab50 con 0x7fbc5c004800 2024-04-25T01:25:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.393+0000 7fbc72fb7640 1 -- 172.21.15.112:0/2787326578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc5c004800 msgr2=0x7fbc5c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.393+0000 7fbc72fb7640 1 --2- 172.21.15.112:0/2787326578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc5c004800 0x7fbc5c004c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c06d3a0 tx=0x7fbc5c005830 comp rx=0 tx=0).stop 2024-04-25T01:25:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.393+0000 7fbc72fb7640 1 -- 172.21.15.112:0/2787326578 shutdown_connections 2024-04-25T01:25:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.393+0000 7fbc72fb7640 1 --2- 172.21.15.112:0/2787326578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc5c004800 0x7fbc5c004c80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.393+0000 7fbc72fb7640 1 -- 172.21.15.112:0/2787326578 >> 172.21.15.112:0/2787326578 conn(0x7fbc6c0ad050 msgr2=0x7fbc6c0ad460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.393+0000 7fbc72fb7640 1 -- 172.21.15.112:0/2787326578 shutdown_connections 2024-04-25T01:25:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.393+0000 7fbc72fb7640 1 -- 172.21.15.112:0/2787326578 wait complete. 2024-04-25T01:25:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.394+0000 7fbc72fb7640 1 Processor -- start 2024-04-25T01:25:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.394+0000 7fbc72fb7640 1 -- start start 2024-04-25T01:25:50.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.394+0000 7fbc72fb7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c004800 0x7fbc6c149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:50.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.394+0000 7fbc72fb7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc6c149420 0x7fbc6c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:50.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.394+0000 7fbc72fb7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc6c1488f0 0x7fbc6c148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:50.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.395+0000 7fbc72fb7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c0c5bc0 con 0x7fbc6c1488f0 2024-04-25T01:25:50.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.395+0000 7fbc72fb7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c0c5a40 con 0x7fbc6c149420 2024-04-25T01:25:50.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.395+0000 7fbc72fb7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbc6c0c5d40 con 0x7fbc5c004800 2024-04-25T01:25:50.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.395+0000 7fbc71554640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc6c1488f0 0x7fbc6c148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:50.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.395+0000 7fbc71554640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc6c1488f0 0x7fbc6c148d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41114/0 (socket says 172.21.15.112:41114) 2024-04-25T01:25:50.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.395+0000 7fbc71554640 1 -- 172.21.15.112:0/3266878341 learned_addr learned my addr 172.21.15.112:0/3266878341 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:50.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.395+0000 7fbc70d53640 1 --2- 172.21.15.112:0/3266878341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c004800 0x7fbc6c149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:50.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.395+0000 7fbc6bfff640 1 --2- 172.21.15.112:0/3266878341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc6c149420 0x7fbc6c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.395+0000 7fbc71554640 1 -- 172.21.15.112:0/3266878341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c004800 msgr2=0x7fbc6c149900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.395+0000 7fbc71554640 1 --2- 172.21.15.112:0/3266878341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c004800 0x7fbc6c149900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.395+0000 7fbc71554640 1 -- 172.21.15.112:0/3266878341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc6c149420 msgr2=0x7fbc6c0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.395+0000 7fbc71554640 1 --2- 172.21.15.112:0/3266878341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc6c149420 0x7fbc6c0af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.395+0000 7fbc71554640 1 -- 172.21.15.112:0/3266878341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6c1eed90 con 0x7fbc6c1488f0 2024-04-25T01:25:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.395+0000 7fbc71554640 1 --2- 172.21.15.112:0/3266878341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc6c1488f0 0x7fbc6c148d70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbc64011bf0 tx=0x7fbc6400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.395+0000 7fbc697fa640 1 -- 172.21.15.112:0/3266878341 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6400fdb0 con 0x7fbc6c1488f0 2024-04-25T01:25:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.395+0000 7fbc697fa640 1 -- 172.21.15.112:0/3266878341 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc6401d070 con 0x7fbc6c1488f0 2024-04-25T01:25:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.395+0000 7fbc72fb7640 1 -- 172.21.15.112:0/3266878341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc6c1ef070 con 0x7fbc6c1488f0 2024-04-25T01:25:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.395+0000 7fbc697fa640 1 -- 172.21.15.112:0/3266878341 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc64015670 con 0x7fbc6c1488f0 2024-04-25T01:25:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.395+0000 7fbc72fb7640 1 -- 172.21.15.112:0/3266878341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc6c1ef630 con 0x7fbc6c1488f0 2024-04-25T01:25:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.396+0000 7fbc72fb7640 1 -- 172.21.15.112:0/3266878341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc6c0682a0 con 0x7fbc6c1488f0 2024-04-25T01:25:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.396+0000 7fbc697fa640 1 -- 172.21.15.112:0/3266878341 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbc64015810 con 0x7fbc6c1488f0 2024-04-25T01:25:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.397+0000 7fbc697fa640 1 --2- 172.21.15.112:0/3266878341 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbc5006be60 0x7fbc5006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.397+0000 7fbc697fa640 1 -- 172.21.15.112:0/3266878341 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fbc6408b2c0 con 0x7fbc6c1488f0 2024-04-25T01:25:50.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.397+0000 7fbc70d53640 1 --2- 172.21.15.112:0/3266878341 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbc5006be60 0x7fbc5006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:50.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.397+0000 7fbc70d53640 1 --2- 172.21.15.112:0/3266878341 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbc5006be60 0x7fbc5006e320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbc60007c10 tx=0x7fbc600078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:50.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.401+0000 7fbc697fa640 1 -- 172.21.15.112:0/3266878341 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbc64010040 con 0x7fbc6c1488f0 2024-04-25T01:25:50.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:50.574+0000 7fbc72fb7640 1 -- 172.21.15.112:0/3266878341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "a_metadata", "pg_num_min": 64} v 0) -- 0x7fbc6c1eb820 con 0x7fbc6c1488f0 2024-04-25T01:25:51.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.333+0000 7fbc697fa640 1 -- 172.21.15.112:0/3266878341 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "a_metadata", "pg_num_min": 64}]=0 pool 'a_metadata' created v17) ==== 128+0+0 (secure 0 0 0) 0x7fbc64059960 con 0x7fbc6c1488f0 2024-04-25T01:25:51.313 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'a_metadata' created 2024-04-25T01:25:51.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.337+0000 7fbc72fb7640 1 -- 172.21.15.112:0/3266878341 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbc5006be60 msgr2=0x7fbc5006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:51.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.337+0000 7fbc72fb7640 1 --2- 172.21.15.112:0/3266878341 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbc5006be60 0x7fbc5006e320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbc60007c10 tx=0x7fbc600078f0 comp rx=0 tx=0).stop 2024-04-25T01:25:51.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.337+0000 7fbc72fb7640 1 -- 172.21.15.112:0/3266878341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc6c1488f0 msgr2=0x7fbc6c148d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:51.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.337+0000 7fbc72fb7640 1 --2- 172.21.15.112:0/3266878341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc6c1488f0 0x7fbc6c148d70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbc64011bf0 tx=0x7fbc6400f580 comp rx=0 tx=0).stop 2024-04-25T01:25:51.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.337+0000 7fbc72fb7640 1 -- 172.21.15.112:0/3266878341 shutdown_connections 2024-04-25T01:25:51.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.337+0000 7fbc72fb7640 1 --2- 172.21.15.112:0/3266878341 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbc5006be60 0x7fbc5006e320 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:51.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.337+0000 7fbc72fb7640 1 --2- 172.21.15.112:0/3266878341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc6c1488f0 0x7fbc6c148d70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:51.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.337+0000 7fbc72fb7640 1 --2- 172.21.15.112:0/3266878341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc6c149420 0x7fbc6c0af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:51.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.337+0000 7fbc72fb7640 1 --2- 172.21.15.112:0/3266878341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c004800 0x7fbc6c149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:51.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.337+0000 7fbc72fb7640 1 -- 172.21.15.112:0/3266878341 >> 172.21.15.112:0/3266878341 conn(0x7fbc6c0ad050 msgr2=0x7fbc6c0ad430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:51.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.338+0000 7fbc72fb7640 1 -- 172.21.15.112:0/3266878341 shutdown_connections 2024-04-25T01:25:51.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.338+0000 7fbc72fb7640 1 -- 172.21.15.112:0/3266878341 wait complete. 2024-04-25T01:25:51.332 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create a_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T01:25:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.460+0000 7f6ed26da640 1 Processor -- start 2024-04-25T01:25:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.460+0000 7f6ed26da640 1 -- start start 2024-04-25T01:25:51.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.461+0000 7f6ed26da640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6ecc150c80 0x7f6ecc070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:51.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.461+0000 7f6ed26da640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6ecc157600 0x7f6ecc0c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:51.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.461+0000 7f6ed26da640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ecc072b60 con 0x7f6ecc0726d0 2024-04-25T01:25:51.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.461+0000 7f6ed26da640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f6ecc072e40 con 0x7f6ecc157600 2024-04-25T01:25:51.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.461+0000 7f6ed26da640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6ecc073120 con 0x7f6ecc150c80 2024-04-25T01:25:51.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.461+0000 7f6ed0c77640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6ecc157600 0x7f6ecc0c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:51.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.461+0000 7f6ecb7fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6ecc0726d0 0x7f6ecc0ac920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50678/0 (socket says 172.21.15.112:50678) 2024-04-25T01:25:51.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.461+0000 7f6ecb7fe640 1 -- 172.21.15.112:0/1224252254 learned_addr learned my addr 172.21.15.112:0/1224252254 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:25:51.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.461+0000 7f6ed0c77640 1 -- 172.21.15.112:0/1224252254 >> v2:172.21.15.187:3300/0 conn(0x7f6ecc150c80 msgr2=0x7f6ecc070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:51.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.461+0000 7f6ed0c77640 1 --2- 172.21.15.112:0/1224252254 >> v2:172.21.15.187:3300/0 conn(0x7f6ecc150c80 0x7f6ecc070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:51.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.461+0000 7f6ed0c77640 1 -- 172.21.15.112:0/1224252254 >> v1:172.21.15.112:6789/0 conn(0x7f6ecc0726d0 legacy=0x7f6ecc0ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:51.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.461+0000 7f6ed0c77640 1 -- 172.21.15.112:0/1224252254 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc0ac120 con 0x7f6ecc157600 2024-04-25T01:25:51.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.461+0000 7f6ecaffd640 1 -- 172.21.15.112:0/1224252254 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3401740943 0 0) 0x7f6ecc072b60 con 0x7f6ecc0726d0 2024-04-25T01:25:51.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.462+0000 7f6ed0c77640 1 --2- 172.21.15.112:0/1224252254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ecc157600 0x7f6ecc0c2920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0c3970 tx=0x7f6ec402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465dfe3bf045801f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:51.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.462+0000 7f6ecaffd640 1 -- 172.21.15.112:0/1224252254 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4007740 con 0x7f6ecc157600 2024-04-25T01:25:51.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.462+0000 7f6ecaffd640 1 -- 172.21.15.112:0/1224252254 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec40078e0 con 0x7f6ecc157600 2024-04-25T01:25:51.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.462+0000 7f6ecaffd640 1 -- 172.21.15.112:0/1224252254 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4007cb0 con 0x7f6ecc157600 2024-04-25T01:25:51.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.463+0000 7f6ed26da640 1 -- 172.21.15.112:0/1224252254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ecc157600 msgr2=0x7f6ecc0c2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:51.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.463+0000 7f6ed26da640 1 --2- 172.21.15.112:0/1224252254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ecc157600 0x7f6ecc0c2920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0c3970 tx=0x7f6ec402a5f0 comp rx=0 tx=0).stop 2024-04-25T01:25:51.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.463+0000 7f6ed26da640 1 -- 172.21.15.112:0/1224252254 shutdown_connections 2024-04-25T01:25:51.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.463+0000 7f6ed26da640 1 --2- 172.21.15.112:0/1224252254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ecc157600 0x7f6ecc0c2920 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:51.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.463+0000 7f6ed26da640 1 --2- 172.21.15.112:0/1224252254 >> v2:172.21.15.187:3300/0 conn(0x7f6ecc150c80 0x7f6ecc070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:51.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.463+0000 7f6ed26da640 1 -- 172.21.15.112:0/1224252254 >> 172.21.15.112:0/1224252254 conn(0x7f6ecc0c3a10 msgr2=0x7f6ecc0c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:51.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.463+0000 7f6ed26da640 1 -- 172.21.15.112:0/1224252254 shutdown_connections 2024-04-25T01:25:51.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.463+0000 7f6ed26da640 1 -- 172.21.15.112:0/1224252254 wait complete. 2024-04-25T01:25:51.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.464+0000 7f6ed26da640 1 Processor -- start 2024-04-25T01:25:51.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.464+0000 7f6ed26da640 1 -- start start 2024-04-25T01:25:51.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.465+0000 7f6ed26da640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ecc150c80 0x7f6ecc14a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:51.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.465+0000 7f6ed26da640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ecc157600 0x7f6ecc148970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:51.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.465+0000 7f6ed26da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ecc1541a0 0x7f6ecc148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:51.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.465+0000 7f6ed26da640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc0ab8f0 con 0x7f6ecc1541a0 2024-04-25T01:25:51.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.465+0000 7f6ed26da640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc157a50 con 0x7f6ecc150c80 2024-04-25T01:25:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.465+0000 7f6ed26da640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6ecc151060 con 0x7f6ecc157600 2024-04-25T01:25:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.465+0000 7f6ed0c77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ecc1541a0 0x7f6ecc148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.465+0000 7f6ed0c77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ecc1541a0 0x7f6ecc148eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41130/0 (socket says 172.21.15.112:41130) 2024-04-25T01:25:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.465+0000 7f6ed0c77640 1 -- 172.21.15.112:0/1116706783 learned_addr learned my addr 172.21.15.112:0/1116706783 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.465+0000 7f6ecb7fe640 1 --2- 172.21.15.112:0/1116706783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ecc157600 0x7f6ecc148970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.465+0000 7f6ecbfff640 1 --2- 172.21.15.112:0/1116706783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ecc150c80 0x7f6ecc14a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.465+0000 7f6ed0c77640 1 -- 172.21.15.112:0/1116706783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ecc157600 msgr2=0x7f6ecc148970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.465+0000 7f6ed0c77640 1 --2- 172.21.15.112:0/1116706783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ecc157600 0x7f6ecc148970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.465+0000 7f6ed0c77640 1 -- 172.21.15.112:0/1116706783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ecc150c80 msgr2=0x7f6ecc14a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.465+0000 7f6ed0c77640 1 --2- 172.21.15.112:0/1116706783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ecc150c80 0x7f6ecc14a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.465+0000 7f6ed0c77640 1 -- 172.21.15.112:0/1116706783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc14b130 con 0x7f6ecc1541a0 2024-04-25T01:25:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.465+0000 7f6ed0c77640 1 --2- 172.21.15.112:0/1116706783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ecc1541a0 0x7f6ecc148eb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009e50 tx=0x7f6ec4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.465+0000 7f6ec8ff9640 1 -- 172.21.15.112:0/1116706783 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4040070 con 0x7f6ecc1541a0 2024-04-25T01:25:51.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.465+0000 7f6ec8ff9640 1 -- 172.21.15.112:0/1116706783 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec4031cc0 con 0x7f6ecc1541a0 2024-04-25T01:25:51.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.466+0000 7f6ec8ff9640 1 -- 172.21.15.112:0/1116706783 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4045680 con 0x7f6ecc1541a0 2024-04-25T01:25:51.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.466+0000 7f6ed26da640 1 -- 172.21.15.112:0/1116706783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ecc149790 con 0x7f6ecc1541a0 2024-04-25T01:25:51.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.466+0000 7f6ed26da640 1 -- 172.21.15.112:0/1116706783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ecc155990 con 0x7f6ecc1541a0 2024-04-25T01:25:51.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.467+0000 7f6ed26da640 1 -- 172.21.15.112:0/1116706783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ecc04f500 con 0x7f6ecc1541a0 2024-04-25T01:25:51.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.467+0000 7f6ec8ff9640 1 -- 172.21.15.112:0/1116706783 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ec4039040 con 0x7f6ecc1541a0 2024-04-25T01:25:51.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.467+0000 7f6ec8ff9640 1 --2- 172.21.15.112:0/1116706783 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6eb806be30 0x7f6eb806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:51.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.467+0000 7f6ec8ff9640 1 -- 172.21.15.112:0/1116706783 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5222+0+0 (secure 0 0 0) 0x7f6ec40ac480 con 0x7f6ecc1541a0 2024-04-25T01:25:51.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.467+0000 7f6ecbfff640 1 --2- 172.21.15.112:0/1116706783 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6eb806be30 0x7f6eb806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:51.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.468+0000 7f6ecbfff640 1 --2- 172.21.15.112:0/1116706783 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6eb806be30 0x7f6eb806e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009990 tx=0x7f6ebc009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:51.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.472+0000 7f6ec8ff9640 1 -- 172.21.15.112:0/1116706783 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6ec407b0a0 con 0x7f6ecc1541a0 2024-04-25T01:25:51.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:51.646+0000 7f6ed26da640 1 -- 172.21.15.112:0/1116706783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "a_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6ecc0c3550 con 0x7f6ecc1541a0 2024-04-25T01:25:52.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.335+0000 7f6ec8ff9640 1 -- 172.21.15.112:0/1116706783 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "a_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'a_data' created v18) ==== 160+0+0 (secure 0 0 0) 0x7f6ec407a9a0 con 0x7f6ecc1541a0 2024-04-25T01:25:52.315 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'a_data' created 2024-04-25T01:25:52.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.339+0000 7f6ed26da640 1 -- 172.21.15.112:0/1116706783 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6eb806be30 msgr2=0x7f6eb806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:52.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.339+0000 7f6ed26da640 1 --2- 172.21.15.112:0/1116706783 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6eb806be30 0x7f6eb806e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009990 tx=0x7f6ebc009110 comp rx=0 tx=0).stop 2024-04-25T01:25:52.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.339+0000 7f6ed26da640 1 -- 172.21.15.112:0/1116706783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ecc1541a0 msgr2=0x7f6ecc148eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:52.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.339+0000 7f6ed26da640 1 --2- 172.21.15.112:0/1116706783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ecc1541a0 0x7f6ecc148eb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009e50 tx=0x7f6ec4004a40 comp rx=0 tx=0).stop 2024-04-25T01:25:52.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.345+0000 7f6ed26da640 1 -- 172.21.15.112:0/1116706783 shutdown_connections 2024-04-25T01:25:52.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.345+0000 7f6ed26da640 1 --2- 172.21.15.112:0/1116706783 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6eb806be30 0x7f6eb806e2f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:52.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.345+0000 7f6ed26da640 1 --2- 172.21.15.112:0/1116706783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ecc1541a0 0x7f6ecc148eb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:52.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.345+0000 7f6ed26da640 1 --2- 172.21.15.112:0/1116706783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ecc157600 0x7f6ecc148970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:52.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.345+0000 7f6ed26da640 1 --2- 172.21.15.112:0/1116706783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ecc150c80 0x7f6ecc14a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:52.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.345+0000 7f6ed26da640 1 -- 172.21.15.112:0/1116706783 >> 172.21.15.112:0/1116706783 conn(0x7f6ecc0c3a10 msgr2=0x7f6ecc070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:52.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.348+0000 7f6ed26da640 1 -- 172.21.15.112:0/1116706783 shutdown_connections 2024-04-25T01:25:52.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.349+0000 7f6ed26da640 1 -- 172.21.15.112:0/1116706783 wait complete. 2024-04-25T01:25:52.343 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new a a_metadata a_data 2024-04-25T01:25:52.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.461+0000 7f62b1f42640 1 Processor -- start 2024-04-25T01:25:52.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.461+0000 7f62b1f42640 1 -- start start 2024-04-25T01:25:52.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.462+0000 7f62b1f42640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f62ac14fbe0 0x7f62ac0c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:52.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.462+0000 7f62b1f42640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f62ac0c2e60 0x7f62ac0726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:52.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.462+0000 7f62b1f42640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62ac06f290 con 0x7f62ac06d460 2024-04-25T01:25:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.462+0000 7f62b1f42640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f62ac14ffc0 con 0x7f62ac0c2e60 2024-04-25T01:25:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.462+0000 7f62b1f42640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f62ac1533a0 con 0x7f62ac14fbe0 2024-04-25T01:25:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.462+0000 7f62abfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f62ac0c2e60 0x7f62ac0726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.462+0000 7f62abfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f62ac0c2e60 0x7f62ac0726c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41140/0 (socket says 172.21.15.112:41140) 2024-04-25T01:25:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.462+0000 7f62aaffd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f62ac06d460 0x7f62ac071fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50680/0 (socket says 172.21.15.112:50680) 2024-04-25T01:25:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.462+0000 7f62abfff640 1 -- 172.21.15.112:0/4058032078 learned_addr learned my addr 172.21.15.112:0/4058032078 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.462+0000 7f62ab7fe640 1 --2- 172.21.15.112:0/4058032078 >> v2:172.21.15.187:3300/0 conn(0x7f62ac14fbe0 0x7f62ac0c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.462+0000 7f62abfff640 1 -- 172.21.15.112:0/4058032078 >> v2:172.21.15.187:3300/0 conn(0x7f62ac14fbe0 msgr2=0x7f62ac0c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.462+0000 7f62abfff640 1 --2- 172.21.15.112:0/4058032078 >> v2:172.21.15.187:3300/0 conn(0x7f62ac14fbe0 0x7f62ac0c2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.462+0000 7f62abfff640 1 -- 172.21.15.112:0/4058032078 >> v1:172.21.15.112:6789/0 conn(0x7f62ac06d460 legacy=0x7f62ac071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.462+0000 7f62aa7fc640 1 -- 172.21.15.112:0/4058032078 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3832093677 0 0) 0x7f62ac06f290 con 0x7f62ac06d460 2024-04-25T01:25:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.462+0000 7f62abfff640 1 -- 172.21.15.112:0/4058032078 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ac06efb0 con 0x7f62ac0c2e60 2024-04-25T01:25:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.462+0000 7f62ab7fe640 1 --2- 172.21.15.112:0/4058032078 >> v2:172.21.15.187:3300/0 conn(0x7f62ac14fbe0 0x7f62ac0c2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:25:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.463+0000 7f62abfff640 1 --2- 172.21.15.112:0/4058032078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62ac0c2e60 0x7f62ac0726c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f62ac068020 tx=0x7f62a402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ffc89e7f9eb36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.463+0000 7f62aa7fc640 1 -- 172.21.15.112:0/4058032078 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4036070 con 0x7f62ac0c2e60 2024-04-25T01:25:52.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.463+0000 7f62aa7fc640 1 -- 172.21.15.112:0/4058032078 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62a4031600 con 0x7f62ac0c2e60 2024-04-25T01:25:52.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.463+0000 7f62aa7fc640 1 -- 172.21.15.112:0/4058032078 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a40319d0 con 0x7f62ac0c2e60 2024-04-25T01:25:52.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.464+0000 7f62b1f42640 1 -- 172.21.15.112:0/4058032078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62ac0c2e60 msgr2=0x7f62ac0726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:52.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.464+0000 7f62b1f42640 1 --2- 172.21.15.112:0/4058032078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62ac0c2e60 0x7f62ac0726c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f62ac068020 tx=0x7f62a402a6a0 comp rx=0 tx=0).stop 2024-04-25T01:25:52.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.464+0000 7f62b1f42640 1 -- 172.21.15.112:0/4058032078 shutdown_connections 2024-04-25T01:25:52.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.464+0000 7f62b1f42640 1 --2- 172.21.15.112:0/4058032078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62ac0c2e60 0x7f62ac0726c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:52.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.464+0000 7f62b1f42640 1 --2- 172.21.15.112:0/4058032078 >> v2:172.21.15.187:3300/0 conn(0x7f62ac14fbe0 0x7f62ac0c2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:52.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.464+0000 7f62b1f42640 1 -- 172.21.15.112:0/4058032078 >> 172.21.15.112:0/4058032078 conn(0x7f62ac0c3a10 msgr2=0x7f62ac0c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:52.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.464+0000 7f62b1f42640 1 -- 172.21.15.112:0/4058032078 shutdown_connections 2024-04-25T01:25:52.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.464+0000 7f62b1f42640 1 -- 172.21.15.112:0/4058032078 wait complete. 2024-04-25T01:25:52.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.465+0000 7f62b1f42640 1 Processor -- start 2024-04-25T01:25:52.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.465+0000 7f62b1f42640 1 -- start start 2024-04-25T01:25:52.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.465+0000 7f62b1f42640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62ac06d460 0x7f62ac0ac730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:52.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.465+0000 7f62b1f42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62ac0c2e60 0x7f62ac1454f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:52.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.465+0000 7f62b1f42640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62ac14fbe0 0x7f62ac145a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:52.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.465+0000 7f62b1f42640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac072c00 con 0x7f62ac0c2e60 2024-04-25T01:25:52.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.465+0000 7f62b1f42640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac153030 con 0x7f62ac06d460 2024-04-25T01:25:52.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.465+0000 7f62b1f42640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f62ac0c3700 con 0x7f62ac14fbe0 2024-04-25T01:25:52.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.466+0000 7f62aaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62ac0c2e60 0x7f62ac1454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:52.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.466+0000 7f62aaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62ac0c2e60 0x7f62ac1454f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41142/0 (socket says 172.21.15.112:41142) 2024-04-25T01:25:52.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.466+0000 7f62aaffd640 1 -- 172.21.15.112:0/3619122108 learned_addr learned my addr 172.21.15.112:0/3619122108 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:52.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.466+0000 7f62abfff640 1 --2- 172.21.15.112:0/3619122108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62ac14fbe0 0x7f62ac145a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:52.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.466+0000 7f62ab7fe640 1 --2- 172.21.15.112:0/3619122108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62ac06d460 0x7f62ac0ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:52.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.466+0000 7f62aaffd640 1 -- 172.21.15.112:0/3619122108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62ac14fbe0 msgr2=0x7f62ac145a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:52.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.466+0000 7f62aaffd640 1 --2- 172.21.15.112:0/3619122108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62ac14fbe0 0x7f62ac145a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:52.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.466+0000 7f62aaffd640 1 -- 172.21.15.112:0/3619122108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62ac06d460 msgr2=0x7f62ac0ac730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:52.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.466+0000 7f62aaffd640 1 --2- 172.21.15.112:0/3619122108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62ac06d460 0x7f62ac0ac730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:52.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.466+0000 7f62aaffd640 1 -- 172.21.15.112:0/3619122108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ac149f20 con 0x7f62ac0c2e60 2024-04-25T01:25:52.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.466+0000 7f62aaffd640 1 --2- 172.21.15.112:0/3619122108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62ac0c2e60 0x7f62ac1454f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f62a000c4d0 tx=0x7f62a000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:52.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.466+0000 7f6293fff640 1 -- 172.21.15.112:0/3619122108 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a0018070 con 0x7f62ac0c2e60 2024-04-25T01:25:52.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.466+0000 7f6293fff640 1 -- 172.21.15.112:0/3619122108 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62a0007500 con 0x7f62ac0c2e60 2024-04-25T01:25:52.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.466+0000 7f62b1f42640 1 -- 172.21.15.112:0/3619122108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ac145f70 con 0x7f62ac0c2e60 2024-04-25T01:25:52.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.466+0000 7f6293fff640 1 -- 172.21.15.112:0/3619122108 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a0004a20 con 0x7f62ac0c2e60 2024-04-25T01:25:52.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.466+0000 7f62b1f42640 1 -- 172.21.15.112:0/3619122108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ac1dc820 con 0x7f62ac0c2e60 2024-04-25T01:25:52.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.467+0000 7f62b1f42640 1 -- 172.21.15.112:0/3619122108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62ac04f500 con 0x7f62ac0c2e60 2024-04-25T01:25:52.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.468+0000 7f6293fff640 1 -- 172.21.15.112:0/3619122108 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f62a001b070 con 0x7f62ac0c2e60 2024-04-25T01:25:52.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.468+0000 7f6293fff640 1 --2- 172.21.15.112:0/3619122108 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f629806be30 0x7f629806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:52.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.468+0000 7f6293fff640 1 -- 172.21.15.112:0/3619122108 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5599+0+0 (secure 0 0 0) 0x7f62a008a3e0 con 0x7f62ac0c2e60 2024-04-25T01:25:52.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.468+0000 7f62ab7fe640 1 --2- 172.21.15.112:0/3619122108 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f629806be30 0x7f629806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:52.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.469+0000 7f62ab7fe640 1 --2- 172.21.15.112:0/3619122108 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f629806be30 0x7f629806e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f629c00b840 tx=0x7f629c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:52.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.472+0000 7f6293fff640 1 -- 172.21.15.112:0/3619122108 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f62a0058f70 con 0x7f62ac0c2e60 2024-04-25T01:25:52.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:52.685+0000 7f62b1f42640 1 -- 172.21.15.112:0/3619122108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "a", "metadata": "a_metadata", "data": "a_data"} v 0) -- 0x7f62ac0c1af0 con 0x7f62ac0c2e60 2024-04-25T01:25:53.116 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:25:53.119 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:25:53.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.355+0000 7f6293fff640 1 -- 172.21.15.112:0/3619122108 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "a", "metadata": "a_metadata", "data": "a_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 157+0+0 (secure 0 0 0) 0x7f62a0058870 con 0x7f62ac0c2e60 2024-04-25T01:25:53.336 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T01:25:53.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.362+0000 7f62b1f42640 1 -- 172.21.15.112:0/3619122108 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f629806be30 msgr2=0x7f629806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:53.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.362+0000 7f62b1f42640 1 --2- 172.21.15.112:0/3619122108 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f629806be30 0x7f629806e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f629c00b840 tx=0x7f629c008040 comp rx=0 tx=0).stop 2024-04-25T01:25:53.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.362+0000 7f62b1f42640 1 -- 172.21.15.112:0/3619122108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62ac0c2e60 msgr2=0x7f62ac1454f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:53.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.362+0000 7f62b1f42640 1 --2- 172.21.15.112:0/3619122108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62ac0c2e60 0x7f62ac1454f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f62a000c4d0 tx=0x7f62a000c990 comp rx=0 tx=0).stop 2024-04-25T01:25:53.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.363+0000 7f62b1f42640 1 -- 172.21.15.112:0/3619122108 shutdown_connections 2024-04-25T01:25:53.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.363+0000 7f62b1f42640 1 --2- 172.21.15.112:0/3619122108 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f629806be30 0x7f629806e2f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.363+0000 7f62b1f42640 1 --2- 172.21.15.112:0/3619122108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62ac14fbe0 0x7f62ac145a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.363+0000 7f62b1f42640 1 --2- 172.21.15.112:0/3619122108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62ac0c2e60 0x7f62ac1454f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.363+0000 7f62b1f42640 1 --2- 172.21.15.112:0/3619122108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62ac06d460 0x7f62ac0ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.363+0000 7f62b1f42640 1 -- 172.21.15.112:0/3619122108 >> 172.21.15.112:0/3619122108 conn(0x7f62ac0c3a10 msgr2=0x7f62ac06d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:53.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.371+0000 7f62b1f42640 1 -- 172.21.15.112:0/3619122108 shutdown_connections 2024-04-25T01:25:53.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.371+0000 7f62b1f42640 1 -- 172.21.15.112:0/3619122108 wait complete. 2024-04-25T01:25:53.364 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:25:53.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.481+0000 7f5c7d91b640 1 Processor -- start 2024-04-25T01:25:53.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.481+0000 7f5c7d91b640 1 -- start start 2024-04-25T01:25:53.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.482+0000 7f5c7d91b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5c781596a0 0x7f5c78152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:53.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.482+0000 7f5c7d91b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c780c72b0 0x7f5c78160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:53.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.482+0000 7f5c7d91b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c78159b90 con 0x7f5c781592c0 2024-04-25T01:25:53.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.482+0000 7f5c7d91b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f5c780c6e20 con 0x7f5c780c72b0 2024-04-25T01:25:53.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.482+0000 7f5c7d91b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5c7806c970 con 0x7f5c781596a0 2024-04-25T01:25:53.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.482+0000 7f5c777fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c780c72b0 0x7f5c78160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:53.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.482+0000 7f5c777fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c780c72b0 0x7f5c78160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41150/0 (socket says 172.21.15.112:41150) 2024-04-25T01:25:53.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.482+0000 7f5c777fe640 1 -- 172.21.15.112:0/1596004592 learned_addr learned my addr 172.21.15.112:0/1596004592 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:53.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.482+0000 7f5c767fc640 1 --2- 172.21.15.112:0/1596004592 >> v2:172.21.15.187:3300/0 conn(0x7f5c781596a0 0x7f5c78152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:53.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.482+0000 7f5c777fe640 1 -- 172.21.15.112:0/1596004592 >> v2:172.21.15.187:3300/0 conn(0x7f5c781596a0 msgr2=0x7f5c78152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:53.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.482+0000 7f5c777fe640 1 --2- 172.21.15.112:0/1596004592 >> v2:172.21.15.187:3300/0 conn(0x7f5c781596a0 0x7f5c78152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.482+0000 7f5c777fe640 1 -- 172.21.15.112:0/1596004592 >> v1:172.21.15.187:6789/0 conn(0x7f5c781592c0 legacy=0x7f5c780c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:53.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.482+0000 7f5c777fe640 1 -- 172.21.15.112:0/1596004592 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7806cc50 con 0x7f5c780c72b0 2024-04-25T01:25:53.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.483+0000 7f5c777fe640 1 --2- 172.21.15.112:0/1596004592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c780c72b0 0x7f5c78160380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c780bea40 tx=0x7f5c7002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c20ce6e2b1f3a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:53.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.483+0000 7f5c75ffb640 1 -- 172.21.15.112:0/1596004592 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70037070 con 0x7f5c780c72b0 2024-04-25T01:25:53.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.483+0000 7f5c75ffb640 1 -- 172.21.15.112:0/1596004592 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c70032440 con 0x7f5c780c72b0 2024-04-25T01:25:53.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.483+0000 7f5c75ffb640 1 -- 172.21.15.112:0/1596004592 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70032810 con 0x7f5c780c72b0 2024-04-25T01:25:53.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.484+0000 7f5c7d91b640 1 -- 172.21.15.112:0/1596004592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c780c72b0 msgr2=0x7f5c78160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:53.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.484+0000 7f5c7d91b640 1 --2- 172.21.15.112:0/1596004592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c780c72b0 0x7f5c78160380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c780bea40 tx=0x7f5c7002a150 comp rx=0 tx=0).stop 2024-04-25T01:25:53.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.484+0000 7f5c7d91b640 1 -- 172.21.15.112:0/1596004592 shutdown_connections 2024-04-25T01:25:53.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.484+0000 7f5c7d91b640 1 --2- 172.21.15.112:0/1596004592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c780c72b0 0x7f5c78160380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.484+0000 7f5c7d91b640 1 --2- 172.21.15.112:0/1596004592 >> v2:172.21.15.187:3300/0 conn(0x7f5c781596a0 0x7f5c78152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.484+0000 7f5c7d91b640 1 -- 172.21.15.112:0/1596004592 >> 172.21.15.112:0/1596004592 conn(0x7f5c780cb170 msgr2=0x7f5c780cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:53.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.484+0000 7f5c7d91b640 1 -- 172.21.15.112:0/1596004592 shutdown_connections 2024-04-25T01:25:53.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.484+0000 7f5c7d91b640 1 -- 172.21.15.112:0/1596004592 wait complete. 2024-04-25T01:25:53.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.485+0000 7f5c7d91b640 1 Processor -- start 2024-04-25T01:25:53.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.485+0000 7f5c7d91b640 1 -- start start 2024-04-25T01:25:53.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c7d91b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c780c72b0 0x7f5c780c1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:53.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c7d91b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c781592c0 0x7f5c781e2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:53.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c7d91b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c781596a0 0x7f5c7814c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:53.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c7d91b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7806c510 con 0x7f5c781592c0 2024-04-25T01:25:53.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c7d91b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7806c390 con 0x7f5c780c72b0 2024-04-25T01:25:53.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c7d91b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5c7806db20 con 0x7f5c781596a0 2024-04-25T01:25:53.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c767fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c781592c0 0x7f5c781e2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:53.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c767fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c781592c0 0x7f5c781e2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41166/0 (socket says 172.21.15.112:41166) 2024-04-25T01:25:53.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c767fc640 1 -- 172.21.15.112:0/2975269260 learned_addr learned my addr 172.21.15.112:0/2975269260 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:53.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c76ffd640 1 --2- 172.21.15.112:0/2975269260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c780c72b0 0x7f5c780c1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:53.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c777fe640 1 --2- 172.21.15.112:0/2975269260 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c781596a0 0x7f5c7814c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:53.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c767fc640 1 -- 172.21.15.112:0/2975269260 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c781596a0 msgr2=0x7f5c7814c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:53.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c767fc640 1 --2- 172.21.15.112:0/2975269260 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c781596a0 0x7f5c7814c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c767fc640 1 -- 172.21.15.112:0/2975269260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c780c72b0 msgr2=0x7f5c780c1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:53.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c767fc640 1 --2- 172.21.15.112:0/2975269260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c780c72b0 0x7f5c780c1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c767fc640 1 -- 172.21.15.112:0/2975269260 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c78153610 con 0x7f5c781592c0 2024-04-25T01:25:53.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c767fc640 1 --2- 172.21.15.112:0/2975269260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c781592c0 0x7f5c781e2980 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00d5c0 tx=0x7f5c6c00da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:53.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c5ffff640 1 -- 172.21.15.112:0/2975269260 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c0125f0 con 0x7f5c781592c0 2024-04-25T01:25:53.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c5ffff640 1 -- 172.21.15.112:0/2975269260 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c6c019070 con 0x7f5c781592c0 2024-04-25T01:25:53.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c5ffff640 1 -- 172.21.15.112:0/2975269260 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c0129d0 con 0x7f5c781592c0 2024-04-25T01:25:53.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c7d91b640 1 -- 172.21.15.112:0/2975269260 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c781538f0 con 0x7f5c781592c0 2024-04-25T01:25:53.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.486+0000 7f5c7d91b640 1 -- 172.21.15.112:0/2975269260 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c781e2ec0 con 0x7f5c781592c0 2024-04-25T01:25:53.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.487+0000 7f5c7d91b640 1 -- 172.21.15.112:0/2975269260 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c780679a0 con 0x7f5c781592c0 2024-04-25T01:25:53.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.488+0000 7f5c5ffff640 1 -- 172.21.15.112:0/2975269260 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c6c012b70 con 0x7f5c781592c0 2024-04-25T01:25:53.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.488+0000 7f5c5ffff640 1 --2- 172.21.15.112:0/2975269260 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c6406be60 0x7f5c6406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:53.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.488+0000 7f5c76ffd640 1 --2- 172.21.15.112:0/2975269260 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c6406be60 0x7f5c6406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:53.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.488+0000 7f5c5ffff640 1 -- 172.21.15.112:0/2975269260 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5689+0+0 (secure 0 0 0) 0x7f5c6c08b010 con 0x7f5c781592c0 2024-04-25T01:25:53.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.489+0000 7f5c76ffd640 1 --2- 172.21.15.112:0/2975269260 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c6406be60 0x7f5c6406e320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5c781e3210 tx=0x7f5c68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:53.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.493+0000 7f5c5ffff640 1 -- 172.21.15.112:0/2975269260 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5c6c059410 con 0x7f5c781592c0 2024-04-25T01:25:53.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.666+0000 7f5c7d91b640 1 -- 172.21.15.112:0/2975269260 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c780682a0 con 0x7f5c781592c0 2024-04-25T01:25:53.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.667+0000 7f5c5ffff640 1 -- 172.21.15.112:0/2975269260 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16622 (secure 0 0 0) 0x7f5c6c05d710 con 0x7f5c781592c0 2024-04-25T01:25:53.647 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:25:53.647 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":19,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:25:52.686961+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:25:53.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.669+0000 7f5c7d91b640 1 -- 172.21.15.112:0/2975269260 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c6406be60 msgr2=0x7f5c6406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:53.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.669+0000 7f5c7d91b640 1 --2- 172.21.15.112:0/2975269260 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c6406be60 0x7f5c6406e320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5c781e3210 tx=0x7f5c68009040 comp rx=0 tx=0).stop 2024-04-25T01:25:53.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.669+0000 7f5c7d91b640 1 -- 172.21.15.112:0/2975269260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c781592c0 msgr2=0x7f5c781e2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:53.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.669+0000 7f5c7d91b640 1 --2- 172.21.15.112:0/2975269260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c781592c0 0x7f5c781e2980 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00d5c0 tx=0x7f5c6c00da80 comp rx=0 tx=0).stop 2024-04-25T01:25:53.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.669+0000 7f5c7d91b640 1 -- 172.21.15.112:0/2975269260 shutdown_connections 2024-04-25T01:25:53.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.669+0000 7f5c7d91b640 1 --2- 172.21.15.112:0/2975269260 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c6406be60 0x7f5c6406e320 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.669+0000 7f5c7d91b640 1 --2- 172.21.15.112:0/2975269260 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c781596a0 0x7f5c7814c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.669+0000 7f5c7d91b640 1 --2- 172.21.15.112:0/2975269260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c781592c0 0x7f5c781e2980 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.669+0000 7f5c7d91b640 1 --2- 172.21.15.112:0/2975269260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c780c72b0 0x7f5c780c1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.669+0000 7f5c7d91b640 1 -- 172.21.15.112:0/2975269260 >> 172.21.15.112:0/2975269260 conn(0x7f5c780cb170 msgr2=0x7f5c780ca5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:53.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.669+0000 7f5c7d91b640 1 -- 172.21.15.112:0/2975269260 shutdown_connections 2024-04-25T01:25:53.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.669+0000 7f5c7d91b640 1 -- 172.21.15.112:0/2975269260 wait complete. 2024-04-25T01:25:53.660 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:25:53.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.777+0000 7f255836a640 1 Processor -- start 2024-04-25T01:25:53.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.777+0000 7f255836a640 1 -- start start 2024-04-25T01:25:53.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.777+0000 7f255836a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25501536c0 0x7f25500c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:53.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.777+0000 7f255836a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f255006e0f0 con 0x7f2550072d80 2024-04-25T01:25:53.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f255836a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25500c2180 con 0x7f2550071db0 2024-04-25T01:25:53.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f255836a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2550072950 con 0x7f25501536c0 2024-04-25T01:25:53.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2556106640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2550072d80 0x7f255014d340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50682/0 (socket says 172.21.15.112:50682) 2024-04-25T01:25:53.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2556106640 1 -- 172.21.15.112:0/623220899 learned_addr learned my addr 172.21.15.112:0/623220899 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:25:53.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2555905640 1 --2- 172.21.15.112:0/623220899 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25501536c0 0x7f25500c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:53.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2555104640 1 -- 172.21.15.112:0/623220899 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 520603639 0 0) 0x7f254c001490 con 0x7f2550072d80 2024-04-25T01:25:53.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2555104640 1 -- 172.21.15.112:0/623220899 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f253c003520 con 0x7f2550072d80 2024-04-25T01:25:53.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2555104640 1 -- 172.21.15.112:0/623220899 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2905670786 0 0) 0x7f253c003520 con 0x7f2550072d80 2024-04-25T01:25:53.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2555104640 1 -- 172.21.15.112:0/623220899 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25501536c0 msgr2=0x7f25500c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:53.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2555104640 1 --2- 172.21.15.112:0/623220899 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25501536c0 0x7f25500c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2555104640 1 -- 172.21.15.112:0/623220899 >> v1:172.21.15.187:6789/0 conn(0x7f2550071db0 legacy=0x7f2550072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:53.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2555104640 1 -- 172.21.15.112:0/623220899 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25500c2460 con 0x7f2550072d80 2024-04-25T01:25:53.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2555104640 1 -- 172.21.15.112:0/623220899 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f254c001bc0 con 0x7f2550072d80 2024-04-25T01:25:53.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2555104640 1 -- 172.21.15.112:0/623220899 >> v1:172.21.15.112:6789/0 conn(0x7f2550072d80 legacy=0x7f255014d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:53.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2555104640 1 --2- 172.21.15.112:0/623220899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f253c0039e0 0x7f253c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:53.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2555104640 1 --2- 172.21.15.112:0/623220899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f253c004800 0x7f253c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:53.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2555104640 1 --2- 172.21.15.112:0/623220899 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25501536c0 0x7f253c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:53.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2555104640 1 -- 172.21.15.112:0/623220899 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2550072950 con 0x7f253c004800 2024-04-25T01:25:53.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2555104640 1 -- 172.21.15.112:0/623220899 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f253c00baa0 con 0x7f253c0039e0 2024-04-25T01:25:53.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2555104640 1 -- 172.21.15.112:0/623220899 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f253c00bca0 con 0x7f25501536c0 2024-04-25T01:25:53.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2555905640 1 --2- 172.21.15.112:0/623220899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f253c004800 0x7f253c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:53.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2556106640 1 --2- 172.21.15.112:0/623220899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f253c0039e0 0x7f253c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:53.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.778+0000 7f2556907640 1 --2- 172.21.15.112:0/623220899 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25501536c0 0x7f253c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:53.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.779+0000 7f2555905640 1 -- 172.21.15.112:0/623220899 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25501536c0 msgr2=0x7f253c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:53.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.779+0000 7f2555905640 1 --2- 172.21.15.112:0/623220899 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25501536c0 0x7f253c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.779+0000 7f2555905640 1 -- 172.21.15.112:0/623220899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f253c0039e0 msgr2=0x7f253c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:53.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.779+0000 7f2555905640 1 --2- 172.21.15.112:0/623220899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f253c0039e0 0x7f253c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.779+0000 7f2555905640 1 -- 172.21.15.112:0/623220899 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253c003520 con 0x7f253c004800 2024-04-25T01:25:53.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.779+0000 7f2556907640 1 --2- 172.21.15.112:0/623220899 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25501536c0 0x7f253c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:25:53.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.779+0000 7f2555905640 1 --2- 172.21.15.112:0/623220899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f253c004800 0x7f253c004c80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2540009a90 tx=0x7f253c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7051e68a60a8c8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:53.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.779+0000 7f2555104640 1 -- 172.21.15.112:0/623220899 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540036070 con 0x7f253c004800 2024-04-25T01:25:53.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.779+0000 7f2555104640 1 -- 172.21.15.112:0/623220899 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2540031400 con 0x7f253c004800 2024-04-25T01:25:53.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.779+0000 7f2555104640 1 -- 172.21.15.112:0/623220899 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540009690 con 0x7f253c004800 2024-04-25T01:25:53.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.780+0000 7f255836a640 1 -- 172.21.15.112:0/623220899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f253c004800 msgr2=0x7f253c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:53.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.780+0000 7f255836a640 1 --2- 172.21.15.112:0/623220899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f253c004800 0x7f253c004c80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2540009a90 tx=0x7f253c005830 comp rx=0 tx=0).stop 2024-04-25T01:25:53.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.780+0000 7f2556907640 1 -- 172.21.15.112:0/623220899 reap_dead start 2024-04-25T01:25:53.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.780+0000 7f255836a640 1 -- 172.21.15.112:0/623220899 shutdown_connections 2024-04-25T01:25:53.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.780+0000 7f255836a640 1 -- 172.21.15.112:0/623220899 >> 172.21.15.112:0/623220899 conn(0x7f255006b160 msgr2=0x7f255006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:53.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.780+0000 7f255836a640 1 -- 172.21.15.112:0/623220899 shutdown_connections 2024-04-25T01:25:53.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.780+0000 7f255836a640 1 -- 172.21.15.112:0/623220899 wait complete. 2024-04-25T01:25:53.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.781+0000 7f255836a640 1 Processor -- start 2024-04-25T01:25:53.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.781+0000 7f255836a640 1 -- start start 2024-04-25T01:25:53.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f255836a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f253c004800 0x7f25500c04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:53.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f255836a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25500c09f0 0x7f25501ee930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:53.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f255836a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25501eee70 0x7f2550146460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:53.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f255836a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2550071890 con 0x7f25500c09f0 2024-04-25T01:25:53.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f255836a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f25500c2a20 con 0x7f253c004800 2024-04-25T01:25:53.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f255836a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f255014da50 con 0x7f25501eee70 2024-04-25T01:25:53.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f2555905640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25500c09f0 0x7f25501ee930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:53.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f2555905640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25500c09f0 0x7f25501ee930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41180/0 (socket says 172.21.15.112:41180) 2024-04-25T01:25:53.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f2555905640 1 -- 172.21.15.112:0/127682561 learned_addr learned my addr 172.21.15.112:0/127682561 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:53.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f2556106640 1 --2- 172.21.15.112:0/127682561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f253c004800 0x7f25500c04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:53.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f2556907640 1 --2- 172.21.15.112:0/127682561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25501eee70 0x7f2550146460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:53.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f2555905640 1 -- 172.21.15.112:0/127682561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25501eee70 msgr2=0x7f2550146460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:53.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f2555905640 1 --2- 172.21.15.112:0/127682561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25501eee70 0x7f2550146460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f2555905640 1 -- 172.21.15.112:0/127682561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f253c004800 msgr2=0x7f25500c04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:53.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f2555905640 1 --2- 172.21.15.112:0/127682561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f253c004800 0x7f25500c04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f2555905640 1 -- 172.21.15.112:0/127682561 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2550147290 con 0x7f25500c09f0 2024-04-25T01:25:53.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f2556907640 1 --2- 172.21.15.112:0/127682561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25501eee70 0x7f2550146460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:25:53.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f2555905640 1 --2- 172.21.15.112:0/127682561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25500c09f0 0x7f25501ee930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f254002ab50 tx=0x7f2540004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:53.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f2546ffd640 1 -- 172.21.15.112:0/127682561 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540041020 con 0x7f25500c09f0 2024-04-25T01:25:53.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f2546ffd640 1 -- 172.21.15.112:0/127682561 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f254003c470 con 0x7f25500c09f0 2024-04-25T01:25:53.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f2546ffd640 1 -- 172.21.15.112:0/127682561 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540036040 con 0x7f25500c09f0 2024-04-25T01:25:53.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f255836a640 1 -- 172.21.15.112:0/127682561 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25501ef2f0 con 0x7f25500c09f0 2024-04-25T01:25:53.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.782+0000 7f255836a640 1 -- 172.21.15.112:0/127682561 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25501eb6d0 con 0x7f25500c09f0 2024-04-25T01:25:53.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.783+0000 7f255836a640 1 -- 172.21.15.112:0/127682561 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25500682a0 con 0x7f25500c09f0 2024-04-25T01:25:53.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.784+0000 7f2546ffd640 1 -- 172.21.15.112:0/127682561 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2540044020 con 0x7f25500c09f0 2024-04-25T01:25:53.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.784+0000 7f2546ffd640 1 --2- 172.21.15.112:0/127682561 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f253406be30 0x7f253406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:53.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.784+0000 7f2556106640 1 --2- 172.21.15.112:0/127682561 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f253406be30 0x7f253406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:53.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.784+0000 7f2546ffd640 1 -- 172.21.15.112:0/127682561 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5689+0+0 (secure 0 0 0) 0x7f25400ab1d0 con 0x7f25500c09f0 2024-04-25T01:25:53.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.785+0000 7f2556106640 1 --2- 172.21.15.112:0/127682561 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f253406be30 0x7f253406e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f25500c1140 tx=0x7f254c006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:53.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.788+0000 7f2546ffd640 1 -- 172.21.15.112:0/127682561 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f254004b070 con 0x7f25500c09f0 2024-04-25T01:25:53.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.962+0000 7f255836a640 1 -- 172.21.15.112:0/127682561 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2550152710 con 0x7f25500c09f0 2024-04-25T01:25:53.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.963+0000 7f2546ffd640 1 -- 172.21.15.112:0/127682561 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16622 (secure 0 0 0) 0x7f2540079c20 con 0x7f25500c09f0 2024-04-25T01:25:53.943 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:25:53.943 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":19,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:25:52.686961+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:25:53.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.965+0000 7f255836a640 1 -- 172.21.15.112:0/127682561 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f253406be30 msgr2=0x7f253406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:53.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.965+0000 7f255836a640 1 --2- 172.21.15.112:0/127682561 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f253406be30 0x7f253406e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f25500c1140 tx=0x7f254c006040 comp rx=0 tx=0).stop 2024-04-25T01:25:53.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.965+0000 7f255836a640 1 -- 172.21.15.112:0/127682561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25500c09f0 msgr2=0x7f25501ee930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:53.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.965+0000 7f255836a640 1 --2- 172.21.15.112:0/127682561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25500c09f0 0x7f25501ee930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f254002ab50 tx=0x7f2540004730 comp rx=0 tx=0).stop 2024-04-25T01:25:53.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.965+0000 7f255836a640 1 -- 172.21.15.112:0/127682561 shutdown_connections 2024-04-25T01:25:53.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.965+0000 7f255836a640 1 --2- 172.21.15.112:0/127682561 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f253406be30 0x7f253406e2f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.965+0000 7f255836a640 1 --2- 172.21.15.112:0/127682561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25501eee70 0x7f2550146460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.965+0000 7f255836a640 1 --2- 172.21.15.112:0/127682561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25500c09f0 0x7f25501ee930 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.965+0000 7f255836a640 1 --2- 172.21.15.112:0/127682561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f253c004800 0x7f25500c04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:53.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.965+0000 7f255836a640 1 -- 172.21.15.112:0/127682561 >> 172.21.15.112:0/127682561 conn(0x7f255006b160 msgr2=0x7f25500c1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:53.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.965+0000 7f255836a640 1 -- 172.21.15.112:0/127682561 shutdown_connections 2024-04-25T01:25:53.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:53.965+0000 7f255836a640 1 -- 172.21.15.112:0/127682561 wait complete. 2024-04-25T01:25:53.957 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set a standby_count_wanted 0 2024-04-25T01:25:54.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.077+0000 7ff69a9ce640 1 Processor -- start 2024-04-25T01:25:54.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.078+0000 7ff69a9ce640 1 -- start start 2024-04-25T01:25:54.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.078+0000 7ff69a9ce640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff6940c9250 0x7ff6940c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:54.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.078+0000 7ff69a9ce640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6940ca8a0 0x7ff69406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:54.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.078+0000 7ff69a9ce640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6941534a0 con 0x7ff6940ca370 2024-04-25T01:25:54.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.078+0000 7ff69a9ce640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7ff694156990 con 0x7ff6940c9250 2024-04-25T01:25:54.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.078+0000 7ff69a9ce640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff694156500 con 0x7ff6940ca8a0 2024-04-25T01:25:54.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.078+0000 7ff693fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff6940c9250 0x7ff6940c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:54.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.078+0000 7ff693fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff6940c9250 0x7ff6940c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41186/0 (socket says 172.21.15.112:41186) 2024-04-25T01:25:54.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.078+0000 7ff693fff640 1 -- 172.21.15.112:0/2734694609 learned_addr learned my addr 172.21.15.112:0/2734694609 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:54.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.079+0000 7ff698f6b640 1 --2- 172.21.15.112:0/2734694609 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6940ca8a0 0x7ff69406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:54.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.079+0000 7ff693fff640 1 -- 172.21.15.112:0/2734694609 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6940ca8a0 msgr2=0x7ff69406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:54.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.079+0000 7ff693fff640 1 --2- 172.21.15.112:0/2734694609 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6940ca8a0 0x7ff69406c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:54.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.079+0000 7ff693fff640 1 -- 172.21.15.112:0/2734694609 >> v1:172.21.15.187:6789/0 conn(0x7ff6940ca370 legacy=0x7ff69406bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:54.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.079+0000 7ff693fff640 1 -- 172.21.15.112:0/2734694609 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6940cae30 con 0x7ff6940c9250 2024-04-25T01:25:54.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.079+0000 7ff698f6b640 1 --2- 172.21.15.112:0/2734694609 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6940ca8a0 0x7ff69406c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:25:54.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.079+0000 7ff693fff640 1 --2- 172.21.15.112:0/2734694609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6940c9250 0x7ff6940c9630 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff684009870 tx=0x7ff68402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b82cbe2e224767d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:54.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.079+0000 7ff692ffd640 1 -- 172.21.15.112:0/2734694609 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684037070 con 0x7ff6940c9250 2024-04-25T01:25:54.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.079+0000 7ff692ffd640 1 -- 172.21.15.112:0/2734694609 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff684032440 con 0x7ff6940c9250 2024-04-25T01:25:54.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.079+0000 7ff692ffd640 1 -- 172.21.15.112:0/2734694609 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684032810 con 0x7ff6940c9250 2024-04-25T01:25:54.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.080+0000 7ff69a9ce640 1 -- 172.21.15.112:0/2734694609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6940c9250 msgr2=0x7ff6940c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:54.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.080+0000 7ff69a9ce640 1 --2- 172.21.15.112:0/2734694609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6940c9250 0x7ff6940c9630 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff684009870 tx=0x7ff68402a180 comp rx=0 tx=0).stop 2024-04-25T01:25:54.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.080+0000 7ff69a9ce640 1 -- 172.21.15.112:0/2734694609 shutdown_connections 2024-04-25T01:25:54.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.080+0000 7ff69a9ce640 1 --2- 172.21.15.112:0/2734694609 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6940ca8a0 0x7ff69406c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:54.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.080+0000 7ff69a9ce640 1 --2- 172.21.15.112:0/2734694609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6940c9250 0x7ff6940c9630 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:54.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.080+0000 7ff69a9ce640 1 -- 172.21.15.112:0/2734694609 >> 172.21.15.112:0/2734694609 conn(0x7ff6940cb310 msgr2=0x7ff6940cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:54.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.080+0000 7ff69a9ce640 1 -- 172.21.15.112:0/2734694609 shutdown_connections 2024-04-25T01:25:54.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.080+0000 7ff69a9ce640 1 -- 172.21.15.112:0/2734694609 wait complete. 2024-04-25T01:25:54.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.081+0000 7ff69a9ce640 1 Processor -- start 2024-04-25T01:25:54.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.081+0000 7ff69a9ce640 1 -- start start 2024-04-25T01:25:54.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.082+0000 7ff69a9ce640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6940c9250 0x7ff6941e0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:54.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.082+0000 7ff69a9ce640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6940ca370 0x7ff6941e14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:54.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.082+0000 7ff69a9ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6940ca8a0 0x7ff69414cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:54.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.082+0000 7ff69a9ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff694155eb0 con 0x7ff6940ca8a0 2024-04-25T01:25:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.082+0000 7ff69a9ce640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff694155bd0 con 0x7ff6940ca370 2024-04-25T01:25:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.082+0000 7ff69a9ce640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff694156190 con 0x7ff6940c9250 2024-04-25T01:25:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.082+0000 7ff698f6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6940ca8a0 0x7ff69414cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.082+0000 7ff698f6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6940ca8a0 0x7ff69414cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41196/0 (socket says 172.21.15.112:41196) 2024-04-25T01:25:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.082+0000 7ff698f6b640 1 -- 172.21.15.112:0/1871852935 learned_addr learned my addr 172.21.15.112:0/1871852935 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.082+0000 7ff693fff640 1 --2- 172.21.15.112:0/1871852935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6940c9250 0x7ff6941e0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.082+0000 7ff6937fe640 1 --2- 172.21.15.112:0/1871852935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6940ca370 0x7ff6941e14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.082+0000 7ff698f6b640 1 -- 172.21.15.112:0/1871852935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6940c9250 msgr2=0x7ff6941e0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.082+0000 7ff698f6b640 1 --2- 172.21.15.112:0/1871852935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6940c9250 0x7ff6941e0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.082+0000 7ff698f6b640 1 -- 172.21.15.112:0/1871852935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6940ca370 msgr2=0x7ff6941e14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.082+0000 7ff698f6b640 1 --2- 172.21.15.112:0/1871852935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6940ca370 0x7ff6941e14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.082+0000 7ff698f6b640 1 -- 172.21.15.112:0/1871852935 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6941490c0 con 0x7ff6940ca8a0 2024-04-25T01:25:54.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.082+0000 7ff698f6b640 1 --2- 172.21.15.112:0/1871852935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6940ca8a0 0x7ff69414cc40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00cdb0 tx=0x7ff68c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:54.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.083+0000 7ff690ff9640 1 -- 172.21.15.112:0/1871852935 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c003b90 con 0x7ff6940ca8a0 2024-04-25T01:25:54.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.083+0000 7ff690ff9640 1 -- 172.21.15.112:0/1871852935 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff68c003d50 con 0x7ff6940ca8a0 2024-04-25T01:25:54.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.083+0000 7ff69a9ce640 1 -- 172.21.15.112:0/1871852935 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69414c660 con 0x7ff6940ca8a0 2024-04-25T01:25:54.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.083+0000 7ff690ff9640 1 -- 172.21.15.112:0/1871852935 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c004260 con 0x7ff6940ca8a0 2024-04-25T01:25:54.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.083+0000 7ff69a9ce640 1 -- 172.21.15.112:0/1871852935 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6941ec550 con 0x7ff6940ca8a0 2024-04-25T01:25:54.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.084+0000 7ff69a9ce640 1 -- 172.21.15.112:0/1871852935 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6940c75d0 con 0x7ff6940ca8a0 2024-04-25T01:25:54.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.084+0000 7ff690ff9640 1 -- 172.21.15.112:0/1871852935 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff68c004400 con 0x7ff6940ca8a0 2024-04-25T01:25:54.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.084+0000 7ff690ff9640 1 --2- 172.21.15.112:0/1871852935 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff68006be30 0x7ff68006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:54.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.085+0000 7ff690ff9640 1 -- 172.21.15.112:0/1871852935 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5689+0+0 (secure 0 0 0) 0x7ff68c08b710 con 0x7ff6940ca8a0 2024-04-25T01:25:54.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.085+0000 7ff693fff640 1 --2- 172.21.15.112:0/1871852935 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff68006be30 0x7ff68006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:54.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.085+0000 7ff693fff640 1 --2- 172.21.15.112:0/1871852935 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff68006be30 0x7ff68006e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff684034000 tx=0x7ff68402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:54.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.089+0000 7ff690ff9640 1 -- 172.21.15.112:0/1871852935 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff6940c75d0 con 0x7ff6940ca8a0 2024-04-25T01:25:54.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.297+0000 7ff69a9ce640 1 -- 172.21.15.112:0/1871852935 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "a", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff69404f500 con 0x7ff6940ca8a0 2024-04-25T01:25:54.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.355+0000 7ff690ff9640 1 -- 172.21.15.112:0/1871852935 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "a", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 113+0+0 (secure 0 0 0) 0x7ff68c014030 con 0x7ff6940ca8a0 2024-04-25T01:25:54.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.357+0000 7ff69a9ce640 1 -- 172.21.15.112:0/1871852935 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff68006be30 msgr2=0x7ff68006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:54.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.357+0000 7ff69a9ce640 1 --2- 172.21.15.112:0/1871852935 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff68006be30 0x7ff68006e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff684034000 tx=0x7ff68402e000 comp rx=0 tx=0).stop 2024-04-25T01:25:54.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.357+0000 7ff69a9ce640 1 -- 172.21.15.112:0/1871852935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6940ca8a0 msgr2=0x7ff69414cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:54.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.357+0000 7ff69a9ce640 1 --2- 172.21.15.112:0/1871852935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6940ca8a0 0x7ff69414cc40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00cdb0 tx=0x7ff68c00a650 comp rx=0 tx=0).stop 2024-04-25T01:25:54.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.357+0000 7ff69a9ce640 1 -- 172.21.15.112:0/1871852935 shutdown_connections 2024-04-25T01:25:54.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.357+0000 7ff69a9ce640 1 --2- 172.21.15.112:0/1871852935 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff68006be30 0x7ff68006e2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:54.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.357+0000 7ff69a9ce640 1 --2- 172.21.15.112:0/1871852935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6940ca8a0 0x7ff69414cc40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:54.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.357+0000 7ff69a9ce640 1 --2- 172.21.15.112:0/1871852935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6940ca370 0x7ff6941e14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:54.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.357+0000 7ff69a9ce640 1 --2- 172.21.15.112:0/1871852935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6940c9250 0x7ff6941e0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:54.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.357+0000 7ff69a9ce640 1 -- 172.21.15.112:0/1871852935 >> 172.21.15.112:0/1871852935 conn(0x7ff6940cb310 msgr2=0x7ff6940671a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:54.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.357+0000 7ff69a9ce640 1 -- 172.21.15.112:0/1871852935 shutdown_connections 2024-04-25T01:25:54.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.357+0000 7ff69a9ce640 1 -- 172.21.15.112:0/1871852935 wait complete. 2024-04-25T01:25:54.349 DEBUG:tasks.cephfs.filesystem:fs_config: {'session_timeout': 300} 2024-04-25T01:25:54.350 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set a allow_standby_replay false 2024-04-25T01:25:54.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.468+0000 7f115a231640 1 Processor -- start 2024-04-25T01:25:54.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.468+0000 7f115a231640 1 -- start start 2024-04-25T01:25:54.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.469+0000 7f115a231640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f115406d460 0x7f1154071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:54.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.469+0000 7f115a231640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f115406f290 con 0x7f11540724f0 2024-04-25T01:25:54.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.469+0000 7f115a231640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f115406efb0 con 0x7f115414fbe0 2024-04-25T01:25:54.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.469+0000 7f115a231640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f115414ffc0 con 0x7f115406d460 2024-04-25T01:25:54.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.469+0000 7f1152ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f115406d460 0x7f1154071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:54.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.469+0000 7f1153fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f11540724f0 0x7f115415ce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50698/0 (socket says 172.21.15.112:50698) 2024-04-25T01:25:54.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.469+0000 7f1153fff640 1 -- 172.21.15.112:0/3993847471 learned_addr learned my addr 172.21.15.112:0/3993847471 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:25:54.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.469+0000 7f1152ffd640 1 -- 172.21.15.112:0/3993847471 >> v1:172.21.15.187:6789/0 conn(0x7f115414fbe0 legacy=0x7f11540c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:54.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.469+0000 7f1152ffd640 1 -- 172.21.15.112:0/3993847471 >> v1:172.21.15.112:6789/0 conn(0x7f11540724f0 legacy=0x7f115415ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:54.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.469+0000 7f1152ffd640 1 -- 172.21.15.112:0/3993847471 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11540a9530 con 0x7f115406d460 2024-04-25T01:25:54.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.470+0000 7f1152ffd640 1 --2- 172.21.15.112:0/3993847471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f115406d460 0x7f1154071fb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f11480097a0 tx=0x7f114802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bbc9b26e7ffe82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:54.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.470+0000 7f11527fc640 1 -- 172.21.15.112:0/3993847471 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148037070 con 0x7f115406d460 2024-04-25T01:25:54.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.470+0000 7f11527fc640 1 -- 172.21.15.112:0/3993847471 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f114802ae50 con 0x7f115406d460 2024-04-25T01:25:54.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.470+0000 7f11527fc640 1 -- 172.21.15.112:0/3993847471 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148032660 con 0x7f115406d460 2024-04-25T01:25:54.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.470+0000 7f115a231640 1 -- 172.21.15.112:0/3993847471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f115406d460 msgr2=0x7f1154071fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.470+0000 7f115a231640 1 --2- 172.21.15.112:0/3993847471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f115406d460 0x7f1154071fb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f11480097a0 tx=0x7f114802a140 comp rx=0 tx=0).stop 2024-04-25T01:25:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.470+0000 7f115a231640 1 -- 172.21.15.112:0/3993847471 shutdown_connections 2024-04-25T01:25:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.470+0000 7f115a231640 1 --2- 172.21.15.112:0/3993847471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f115406d460 0x7f1154071fb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.470+0000 7f115a231640 1 -- 172.21.15.112:0/3993847471 >> 172.21.15.112:0/3993847471 conn(0x7f11540c3a10 msgr2=0x7f11540c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.470+0000 7f115a231640 1 -- 172.21.15.112:0/3993847471 shutdown_connections 2024-04-25T01:25:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.471+0000 7f115a231640 1 -- 172.21.15.112:0/3993847471 wait complete. 2024-04-25T01:25:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.472+0000 7f115a231640 1 Processor -- start 2024-04-25T01:25:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.472+0000 7f115a231640 1 -- start start 2024-04-25T01:25:54.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.472+0000 7f115a231640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f115406d460 0x7f11541de8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:54.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.472+0000 7f115a231640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11540724f0 0x7f1154145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:54.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.472+0000 7f115a231640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f115414fbe0 0x7f11540c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:54.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.472+0000 7f115a231640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11540c3030 con 0x7f11540724f0 2024-04-25T01:25:54.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.472+0000 7f115a231640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f11541533a0 con 0x7f115406d460 2024-04-25T01:25:54.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.472+0000 7f115a231640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f11540c31b0 con 0x7f115414fbe0 2024-04-25T01:25:54.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.472+0000 7f1152ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11540724f0 0x7f1154145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:54.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.473+0000 7f1152ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11540724f0 0x7f1154145cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41212/0 (socket says 172.21.15.112:41212) 2024-04-25T01:25:54.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.473+0000 7f1152ffd640 1 -- 172.21.15.112:0/1693707022 learned_addr learned my addr 172.21.15.112:0/1693707022 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:54.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.473+0000 7f11537fe640 1 --2- 172.21.15.112:0/1693707022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f115406d460 0x7f11541de8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:54.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.473+0000 7f1153fff640 1 --2- 172.21.15.112:0/1693707022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f115414fbe0 0x7f11540c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:54.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.473+0000 7f1152ffd640 1 -- 172.21.15.112:0/1693707022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f115414fbe0 msgr2=0x7f11540c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:54.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.473+0000 7f1152ffd640 1 --2- 172.21.15.112:0/1693707022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f115414fbe0 0x7f11540c56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:54.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.473+0000 7f1152ffd640 1 -- 172.21.15.112:0/1693707022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f115406d460 msgr2=0x7f11541de8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:54.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.473+0000 7f1152ffd640 1 --2- 172.21.15.112:0/1693707022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f115406d460 0x7f11541de8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:54.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.473+0000 7f1152ffd640 1 -- 172.21.15.112:0/1693707022 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11541ed3a0 con 0x7f11540724f0 2024-04-25T01:25:54.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.473+0000 7f1152ffd640 1 --2- 172.21.15.112:0/1693707022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11540724f0 0x7f1154145cb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f11480098c0 tx=0x7f11480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:54.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.473+0000 7f113bfff640 1 -- 172.21.15.112:0/1693707022 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148042070 con 0x7f11540724f0 2024-04-25T01:25:54.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.473+0000 7f113bfff640 1 -- 172.21.15.112:0/1693707022 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f114803e670 con 0x7f11540724f0 2024-04-25T01:25:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.473+0000 7f115a231640 1 -- 172.21.15.112:0/1693707022 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11540c4c30 con 0x7f11540724f0 2024-04-25T01:25:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.473+0000 7f113bfff640 1 -- 172.21.15.112:0/1693707022 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148037040 con 0x7f11540724f0 2024-04-25T01:25:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.473+0000 7f115a231640 1 -- 172.21.15.112:0/1693707022 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1154152250 con 0x7f11540724f0 2024-04-25T01:25:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.474+0000 7f115a231640 1 -- 172.21.15.112:0/1693707022 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f115404f500 con 0x7f11540724f0 2024-04-25T01:25:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.475+0000 7f113bfff640 1 -- 172.21.15.112:0/1693707022 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1148045070 con 0x7f11540724f0 2024-04-25T01:25:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.475+0000 7f113bfff640 1 --2- 172.21.15.112:0/1693707022 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f114006be30 0x7f114006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.475+0000 7f113bfff640 1 -- 172.21.15.112:0/1693707022 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7f11480ab570 con 0x7f11540724f0 2024-04-25T01:25:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.475+0000 7f11537fe640 1 --2- 172.21.15.112:0/1693707022 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f114006be30 0x7f114006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.475+0000 7f11537fe640 1 --2- 172.21.15.112:0/1693707022 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f114006be30 0x7f114006e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1144006fd0 tx=0x7f1144008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:54.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.479+0000 7f113bfff640 1 -- 172.21.15.112:0/1693707022 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f114807a0e0 con 0x7f11540724f0 2024-04-25T01:25:54.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:54.693+0000 7f115a231640 1 -- 172.21.15.112:0/1693707022 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "a", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f11540c5480 con 0x7f11540724f0 2024-04-25T01:25:55.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.362+0000 7f113bfff640 1 -- 172.21.15.112:0/1693707022 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "a", "var": "allow_standby_replay", "val": "false"}]=0 v6) ==== 117+0+0 (secure 0 0 0) 0x7f11480799e0 con 0x7f11540724f0 2024-04-25T01:25:55.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.365+0000 7f115a231640 1 -- 172.21.15.112:0/1693707022 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f114006be30 msgr2=0x7f114006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:55.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.365+0000 7f115a231640 1 --2- 172.21.15.112:0/1693707022 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f114006be30 0x7f114006e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1144006fd0 tx=0x7f1144008040 comp rx=0 tx=0).stop 2024-04-25T01:25:55.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.365+0000 7f115a231640 1 -- 172.21.15.112:0/1693707022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11540724f0 msgr2=0x7f1154145cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:55.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.365+0000 7f115a231640 1 --2- 172.21.15.112:0/1693707022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11540724f0 0x7f1154145cb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f11480098c0 tx=0x7f11480046c0 comp rx=0 tx=0).stop 2024-04-25T01:25:55.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.365+0000 7f115a231640 1 -- 172.21.15.112:0/1693707022 shutdown_connections 2024-04-25T01:25:55.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.365+0000 7f115a231640 1 --2- 172.21.15.112:0/1693707022 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f114006be30 0x7f114006e2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:55.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.365+0000 7f115a231640 1 --2- 172.21.15.112:0/1693707022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f115414fbe0 0x7f11540c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:55.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.365+0000 7f115a231640 1 --2- 172.21.15.112:0/1693707022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11540724f0 0x7f1154145cb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:55.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.365+0000 7f115a231640 1 --2- 172.21.15.112:0/1693707022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f115406d460 0x7f11541de8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:55.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.365+0000 7f115a231640 1 -- 172.21.15.112:0/1693707022 >> 172.21.15.112:0/1693707022 conn(0x7f11540c3a10 msgr2=0x7f11540c1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:55.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.365+0000 7f115a231640 1 -- 172.21.15.112:0/1693707022 shutdown_connections 2024-04-25T01:25:55.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.365+0000 7f115a231640 1 -- 172.21.15.112:0/1693707022 wait complete. 2024-04-25T01:25:55.356 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set a session_timeout 300 2024-04-25T01:25:55.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.476+0000 7fe6b8363640 1 Processor -- start 2024-04-25T01:25:55.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.476+0000 7fe6b8363640 1 -- start start 2024-04-25T01:25:55.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.477+0000 7fe6b8363640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b014d8c0 0x7fe6b006b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:55.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.477+0000 7fe6b8363640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6b01539e0 con 0x7fe6b00aa400 2024-04-25T01:25:55.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.477+0000 7fe6b8363640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6b0153cc0 con 0x7fe6b006b160 2024-04-25T01:25:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.477+0000 7fe6b8363640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b01545f0 con 0x7fe6b014d8c0 2024-04-25T01:25:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.477+0000 7fe6b6900640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe6b00aa400 0x7fe6b00aa870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50700/0 (socket says 172.21.15.112:50700) 2024-04-25T01:25:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.477+0000 7fe6b6900640 1 -- 172.21.15.112:0/1855041828 learned_addr learned my addr 172.21.15.112:0/1855041828 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:25:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.477+0000 7fe6b60ff640 1 --2- 172.21.15.112:0/1855041828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b014d8c0 0x7fe6b006b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.477+0000 7fe6b50fd640 1 -- 172.21.15.112:0/1855041828 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2398771768 0 0) 0x7fe6b01539e0 con 0x7fe6b00aa400 2024-04-25T01:25:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.477+0000 7fe6b50fd640 1 -- 172.21.15.112:0/1855041828 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe69c003520 con 0x7fe6b00aa400 2024-04-25T01:25:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.477+0000 7fe6b50fd640 1 -- 172.21.15.112:0/1855041828 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2681384167 0 0) 0x7fe69c003520 con 0x7fe6b00aa400 2024-04-25T01:25:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.477+0000 7fe6b50fd640 1 -- 172.21.15.112:0/1855041828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b014d8c0 msgr2=0x7fe6b006b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.477+0000 7fe6b50fd640 1 --2- 172.21.15.112:0/1855041828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b014d8c0 0x7fe6b006b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.477+0000 7fe6b50fd640 1 -- 172.21.15.112:0/1855041828 >> v1:172.21.15.187:6789/0 conn(0x7fe6b006b160 legacy=0x7fe6b00a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.477+0000 7fe6b50fd640 1 -- 172.21.15.112:0/1855041828 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b0153fa0 con 0x7fe6b00aa400 2024-04-25T01:25:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.477+0000 7fe6b50fd640 1 -- 172.21.15.112:0/1855041828 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fe6a80019f0 con 0x7fe6b00aa400 2024-04-25T01:25:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.477+0000 7fe6b50fd640 1 -- 172.21.15.112:0/1855041828 >> v1:172.21.15.112:6789/0 conn(0x7fe6b00aa400 legacy=0x7fe6b00aa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.477+0000 7fe6b50fd640 1 --2- 172.21.15.112:0/1855041828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe69c0039e0 0x7fe69c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.477+0000 7fe6b50fd640 1 --2- 172.21.15.112:0/1855041828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe69c004800 0x7fe69c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.478+0000 7fe6b50fd640 1 --2- 172.21.15.112:0/1855041828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b014d8c0 0x7fe69c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.478+0000 7fe6b50fd640 1 -- 172.21.15.112:0/1855041828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b01545f0 con 0x7fe69c0039e0 2024-04-25T01:25:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.478+0000 7fe6b50fd640 1 -- 172.21.15.112:0/1855041828 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c00bb00 con 0x7fe69c004800 2024-04-25T01:25:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.478+0000 7fe6b50fd640 1 -- 172.21.15.112:0/1855041828 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe69c00bd00 con 0x7fe6b014d8c0 2024-04-25T01:25:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.478+0000 7fe6b60ff640 1 --2- 172.21.15.112:0/1855041828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe69c0039e0 0x7fe69c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.478+0000 7fe6b58fe640 1 --2- 172.21.15.112:0/1855041828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe69c004800 0x7fe69c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.478+0000 7fe6b6900640 1 --2- 172.21.15.112:0/1855041828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b014d8c0 0x7fe69c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.478+0000 7fe6b60ff640 1 -- 172.21.15.112:0/1855041828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b014d8c0 msgr2=0x7fe69c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.478+0000 7fe6b60ff640 1 --2- 172.21.15.112:0/1855041828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b014d8c0 0x7fe69c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.478+0000 7fe6b60ff640 1 -- 172.21.15.112:0/1855041828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe69c004800 msgr2=0x7fe69c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.478+0000 7fe6b60ff640 1 --2- 172.21.15.112:0/1855041828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe69c004800 0x7fe69c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.478+0000 7fe6b60ff640 1 -- 172.21.15.112:0/1855041828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69c003520 con 0x7fe69c0039e0 2024-04-25T01:25:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.478+0000 7fe6b58fe640 1 --2- 172.21.15.112:0/1855041828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe69c004800 0x7fe69c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:25:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.478+0000 7fe6b60ff640 1 --2- 172.21.15.112:0/1855041828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe69c0039e0 0x7fe69c003e00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009a90 tx=0x7fe69c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5a0110fe926080 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:55.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.479+0000 7fe6b50fd640 1 -- 172.21.15.112:0/1855041828 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac036070 con 0x7fe69c0039e0 2024-04-25T01:25:55.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.479+0000 7fe6b50fd640 1 -- 172.21.15.112:0/1855041828 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6ac031400 con 0x7fe69c0039e0 2024-04-25T01:25:55.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.479+0000 7fe6b50fd640 1 -- 172.21.15.112:0/1855041828 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac009690 con 0x7fe69c0039e0 2024-04-25T01:25:55.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.479+0000 7fe6b8363640 1 -- 172.21.15.112:0/1855041828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe69c0039e0 msgr2=0x7fe69c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:55.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.479+0000 7fe6b8363640 1 --2- 172.21.15.112:0/1855041828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe69c0039e0 0x7fe69c003e00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009a90 tx=0x7fe69c005830 comp rx=0 tx=0).stop 2024-04-25T01:25:55.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.479+0000 7fe6b6900640 1 -- 172.21.15.112:0/1855041828 reap_dead start 2024-04-25T01:25:55.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.479+0000 7fe6b8363640 1 -- 172.21.15.112:0/1855041828 shutdown_connections 2024-04-25T01:25:55.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.479+0000 7fe6b8363640 1 -- 172.21.15.112:0/1855041828 >> 172.21.15.112:0/1855041828 conn(0x7fe6b00c8100 msgr2=0x7fe6b00c84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:55.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.479+0000 7fe6b8363640 1 -- 172.21.15.112:0/1855041828 shutdown_connections 2024-04-25T01:25:55.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.479+0000 7fe6b8363640 1 -- 172.21.15.112:0/1855041828 wait complete. 2024-04-25T01:25:55.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.480+0000 7fe6b8363640 1 Processor -- start 2024-04-25T01:25:55.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.480+0000 7fe6b8363640 1 -- start start 2024-04-25T01:25:55.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.481+0000 7fe6b8363640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe69c0039e0 0x7fe6b01eb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:55.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.481+0000 7fe6b8363640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b01ee6b0 0x7fe6b00c17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:55.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.481+0000 7fe6b8363640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b00abb80 0x7fe6b00ac000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:55.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.481+0000 7fe6b8363640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b00a9ba0 con 0x7fe6b01ee6b0 2024-04-25T01:25:55.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.481+0000 7fe6b8363640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b0153690 con 0x7fe6b00abb80 2024-04-25T01:25:55.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.481+0000 7fe6b8363640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b00a9d20 con 0x7fe69c0039e0 2024-04-25T01:25:55.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.481+0000 7fe6b58fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b01ee6b0 0x7fe6b00c17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:55.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.481+0000 7fe6b58fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b01ee6b0 0x7fe6b00c17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41230/0 (socket says 172.21.15.112:41230) 2024-04-25T01:25:55.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.481+0000 7fe6b58fe640 1 -- 172.21.15.112:0/1235125637 learned_addr learned my addr 172.21.15.112:0/1235125637 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:55.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.481+0000 7fe6b58fe640 1 -- 172.21.15.112:0/1235125637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe69c0039e0 msgr2=0x7fe6b01eb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:55.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.481+0000 7fe6b60ff640 1 --2- 172.21.15.112:0/1235125637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe69c0039e0 0x7fe6b01eb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:55.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.481+0000 7fe6b6900640 1 --2- 172.21.15.112:0/1235125637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b00abb80 0x7fe6b00ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:55.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.481+0000 7fe6b58fe640 1 --2- 172.21.15.112:0/1235125637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe69c0039e0 0x7fe6b01eb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:55.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.481+0000 7fe6b58fe640 1 -- 172.21.15.112:0/1235125637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b00abb80 msgr2=0x7fe6b00ac000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:55.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.481+0000 7fe6b58fe640 1 --2- 172.21.15.112:0/1235125637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b00abb80 0x7fe6b00ac000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:55.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.481+0000 7fe6b58fe640 1 -- 172.21.15.112:0/1235125637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b01eb790 con 0x7fe6b01ee6b0 2024-04-25T01:25:55.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.482+0000 7fe6b58fe640 1 --2- 172.21.15.112:0/1235125637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b01ee6b0 0x7fe6b00c17a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000e600 tx=0x7fe6a000eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:55.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.482+0000 7fe6a6ffd640 1 -- 172.21.15.112:0/1235125637 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a00052e0 con 0x7fe6b01ee6b0 2024-04-25T01:25:55.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.482+0000 7fe6a6ffd640 1 -- 172.21.15.112:0/1235125637 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a0018070 con 0x7fe6b01ee6b0 2024-04-25T01:25:55.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.482+0000 7fe6a6ffd640 1 -- 172.21.15.112:0/1235125637 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0003810 con 0x7fe6b01ee6b0 2024-04-25T01:25:55.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.482+0000 7fe6b8363640 1 -- 172.21.15.112:0/1235125637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b01eba70 con 0x7fe6b01ee6b0 2024-04-25T01:25:55.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.482+0000 7fe6b8363640 1 -- 172.21.15.112:0/1235125637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b01ec030 con 0x7fe6b01ee6b0 2024-04-25T01:25:55.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.483+0000 7fe6b8363640 1 -- 172.21.15.112:0/1235125637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b004f500 con 0x7fe6b01ee6b0 2024-04-25T01:25:55.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.483+0000 7fe6a6ffd640 1 -- 172.21.15.112:0/1235125637 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe6a00039b0 con 0x7fe6b01ee6b0 2024-04-25T01:25:55.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.483+0000 7fe6a6ffd640 1 --2- 172.21.15.112:0/1235125637 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe69406be60 0x7fe69406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:55.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.484+0000 7fe6a6ffd640 1 -- 172.21.15.112:0/1235125637 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7fe6a008aac0 con 0x7fe6b01ee6b0 2024-04-25T01:25:55.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.484+0000 7fe6b60ff640 1 --2- 172.21.15.112:0/1235125637 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe69406be60 0x7fe69406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:55.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.484+0000 7fe6b60ff640 1 --2- 172.21.15.112:0/1235125637 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe69406be60 0x7fe69406e320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac033000 tx=0x7fe6ac008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:55.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.488+0000 7fe6a6ffd640 1 -- 172.21.15.112:0/1235125637 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe6a0059510 con 0x7fe6b01ee6b0 2024-04-25T01:25:55.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:55.701+0000 7fe6b8363640 1 -- 172.21.15.112:0/1235125637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "a", "var": "session_timeout", "val": "300"} v 0) -- 0x7fe6b006bdb0 con 0x7fe6b01ee6b0 2024-04-25T01:25:56.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.369+0000 7fe6a6ffd640 1 -- 172.21.15.112:0/1235125637 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "a", "var": "session_timeout", "val": "300"}]=0 v7) ==== 110+0+0 (secure 0 0 0) 0x7fe6a0058e10 con 0x7fe6b01ee6b0 2024-04-25T01:25:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.372+0000 7fe6b8363640 1 -- 172.21.15.112:0/1235125637 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe69406be60 msgr2=0x7fe69406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.372+0000 7fe6b8363640 1 --2- 172.21.15.112:0/1235125637 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe69406be60 0x7fe69406e320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac033000 tx=0x7fe6ac008030 comp rx=0 tx=0).stop 2024-04-25T01:25:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.372+0000 7fe6b8363640 1 -- 172.21.15.112:0/1235125637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b01ee6b0 msgr2=0x7fe6b00c17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.372+0000 7fe6b8363640 1 --2- 172.21.15.112:0/1235125637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b01ee6b0 0x7fe6b00c17a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000e600 tx=0x7fe6a000eac0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.372+0000 7fe6b8363640 1 -- 172.21.15.112:0/1235125637 shutdown_connections 2024-04-25T01:25:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.372+0000 7fe6b8363640 1 --2- 172.21.15.112:0/1235125637 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe69406be60 0x7fe69406e320 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.372+0000 7fe6b8363640 1 --2- 172.21.15.112:0/1235125637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b00abb80 0x7fe6b00ac000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.372+0000 7fe6b8363640 1 --2- 172.21.15.112:0/1235125637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b01ee6b0 0x7fe6b00c17a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.372+0000 7fe6b8363640 1 --2- 172.21.15.112:0/1235125637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe69c0039e0 0x7fe6b01eb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.372+0000 7fe6b8363640 1 -- 172.21.15.112:0/1235125637 >> 172.21.15.112:0/1235125637 conn(0x7fe6b00c8100 msgr2=0x7fe6b01512b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.372+0000 7fe6b8363640 1 -- 172.21.15.112:0/1235125637 shutdown_connections 2024-04-25T01:25:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.372+0000 7fe6b8363640 1 -- 172.21.15.112:0/1235125637 wait complete. 2024-04-25T01:25:56.363 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'a' 2024-04-25T01:25:56.363 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:25:56.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.481+0000 7fce0b2df640 1 Processor -- start 2024-04-25T01:25:56.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.482+0000 7fce0b2df640 1 -- start start 2024-04-25T01:25:56.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.482+0000 7fce0b2df640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce041592c0 0x7fce040c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:56.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.482+0000 7fce0b2df640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fce040c72b0 0x7fce04160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.482+0000 7fce0b2df640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce04159b90 con 0x7fce041596a0 2024-04-25T01:25:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.482+0000 7fce0b2df640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fce040c6e20 con 0x7fce040c72b0 2024-04-25T01:25:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.482+0000 7fce0b2df640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fce0406c970 con 0x7fce041592c0 2024-04-25T01:25:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.482+0000 7fce0987c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fce040c72b0 0x7fce04160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.483+0000 7fce0987c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fce040c72b0 0x7fce04160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41242/0 (socket says 172.21.15.112:41242) 2024-04-25T01:25:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.483+0000 7fce0987c640 1 -- 172.21.15.112:0/2049341723 learned_addr learned my addr 172.21.15.112:0/2049341723 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.483+0000 7fce0907b640 1 --2- 172.21.15.112:0/2049341723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce041592c0 0x7fce040c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.483+0000 7fce0987c640 1 -- 172.21.15.112:0/2049341723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce041592c0 msgr2=0x7fce040c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.483+0000 7fce0987c640 1 --2- 172.21.15.112:0/2049341723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce041592c0 0x7fce040c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.483+0000 7fce0987c640 1 -- 172.21.15.112:0/2049341723 >> v1:172.21.15.187:6789/0 conn(0x7fce041596a0 legacy=0x7fce04152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.483+0000 7fce0987c640 1 -- 172.21.15.112:0/2049341723 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0406cc50 con 0x7fce040c72b0 2024-04-25T01:25:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.483+0000 7fce0987c640 1 --2- 172.21.15.112:0/2049341723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce040c72b0 0x7fce04160380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009e50 tx=0x7fcdfc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc49529272977dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.484+0000 7fcdfbfff640 1 -- 172.21.15.112:0/2049341723 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc036070 con 0x7fce040c72b0 2024-04-25T01:25:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.484+0000 7fcdfbfff640 1 -- 172.21.15.112:0/2049341723 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdfc031600 con 0x7fce040c72b0 2024-04-25T01:25:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.484+0000 7fcdfbfff640 1 -- 172.21.15.112:0/2049341723 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0319d0 con 0x7fce040c72b0 2024-04-25T01:25:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.484+0000 7fce0b2df640 1 -- 172.21.15.112:0/2049341723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce040c72b0 msgr2=0x7fce04160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.484+0000 7fce0b2df640 1 --2- 172.21.15.112:0/2049341723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce040c72b0 0x7fce04160380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009e50 tx=0x7fcdfc02a5f0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.484+0000 7fce0b2df640 1 -- 172.21.15.112:0/2049341723 shutdown_connections 2024-04-25T01:25:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.484+0000 7fce0b2df640 1 --2- 172.21.15.112:0/2049341723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce040c72b0 0x7fce04160380 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.484+0000 7fce0b2df640 1 --2- 172.21.15.112:0/2049341723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce041592c0 0x7fce040c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.484+0000 7fce0b2df640 1 -- 172.21.15.112:0/2049341723 >> 172.21.15.112:0/2049341723 conn(0x7fce040cb170 msgr2=0x7fce040cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.484+0000 7fce0b2df640 1 -- 172.21.15.112:0/2049341723 shutdown_connections 2024-04-25T01:25:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.484+0000 7fce0b2df640 1 -- 172.21.15.112:0/2049341723 wait complete. 2024-04-25T01:25:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.485+0000 7fce0b2df640 1 Processor -- start 2024-04-25T01:25:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.485+0000 7fce0b2df640 1 -- start start 2024-04-25T01:25:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.486+0000 7fce0b2df640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce040c72b0 0x7fce040c1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.486+0000 7fce0b2df640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce041592c0 0x7fce040c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.486+0000 7fce0b2df640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce041596a0 0x7fce0414c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.486+0000 7fce0b2df640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fce04153590 con 0x7fce041596a0 2024-04-25T01:25:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.486+0000 7fce0b2df640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fce04153410 con 0x7fce041592c0 2024-04-25T01:25:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.486+0000 7fce0b2df640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fce0406db20 con 0x7fce040c72b0 2024-04-25T01:25:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.486+0000 7fce0987c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce041596a0 0x7fce0414c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.486+0000 7fce0987c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce041596a0 0x7fce0414c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41258/0 (socket says 172.21.15.112:41258) 2024-04-25T01:25:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.486+0000 7fce0987c640 1 -- 172.21.15.112:0/776331678 learned_addr learned my addr 172.21.15.112:0/776331678 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.486+0000 7fce0907b640 1 --2- 172.21.15.112:0/776331678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce040c72b0 0x7fce040c1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.486+0000 7fce0887a640 1 --2- 172.21.15.112:0/776331678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce041592c0 0x7fce040c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.486+0000 7fce0987c640 1 -- 172.21.15.112:0/776331678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce040c72b0 msgr2=0x7fce040c1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.486+0000 7fce0987c640 1 --2- 172.21.15.112:0/776331678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce040c72b0 0x7fce040c1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.486+0000 7fce0987c640 1 -- 172.21.15.112:0/776331678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce041592c0 msgr2=0x7fce040c4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.486+0000 7fce0987c640 1 --2- 172.21.15.112:0/776331678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce041592c0 0x7fce040c4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.486+0000 7fce0987c640 1 -- 172.21.15.112:0/776331678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce040c2da0 con 0x7fce041596a0 2024-04-25T01:25:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.486+0000 7fce0887a640 1 --2- 172.21.15.112:0/776331678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce041592c0 0x7fce040c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:25:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.487+0000 7fce0987c640 1 --2- 172.21.15.112:0/776331678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce041596a0 0x7fce0414c550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc034040 tx=0x7fcdfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.487+0000 7fcdf9ffb640 1 -- 172.21.15.112:0/776331678 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc036050 con 0x7fce041596a0 2024-04-25T01:25:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.487+0000 7fcdf9ffb640 1 -- 172.21.15.112:0/776331678 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdfc042070 con 0x7fce041596a0 2024-04-25T01:25:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.487+0000 7fcdf9ffb640 1 -- 172.21.15.112:0/776331678 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc009510 con 0x7fce041596a0 2024-04-25T01:25:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.487+0000 7fce0b2df640 1 -- 172.21.15.112:0/776331678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0414f1f0 con 0x7fce041596a0 2024-04-25T01:25:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.487+0000 7fce0b2df640 1 -- 172.21.15.112:0/776331678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce040c2a50 con 0x7fce041596a0 2024-04-25T01:25:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.488+0000 7fce0b2df640 1 -- 172.21.15.112:0/776331678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce040679a0 con 0x7fce041596a0 2024-04-25T01:25:56.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.488+0000 7fcdf9ffb640 1 -- 172.21.15.112:0/776331678 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcdfc007bd0 con 0x7fce041596a0 2024-04-25T01:25:56.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.489+0000 7fcdf9ffb640 1 --2- 172.21.15.112:0/776331678 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcde806be30 0x7fcde806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:56.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.489+0000 7fcdf9ffb640 1 -- 172.21.15.112:0/776331678 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7fcdfc0abcc0 con 0x7fce041596a0 2024-04-25T01:25:56.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.489+0000 7fce0907b640 1 --2- 172.21.15.112:0/776331678 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcde806be30 0x7fcde806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:56.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.489+0000 7fce0907b640 1 --2- 172.21.15.112:0/776331678 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcde806be30 0x7fcde806e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4006fd0 tx=0x7fcdf4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:56.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.493+0000 7fcdf9ffb640 1 -- 172.21.15.112:0/776331678 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcdfc07a680 con 0x7fce041596a0 2024-04-25T01:25:56.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.702+0000 7fce0b2df640 1 -- 172.21.15.112:0/776331678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce040682a0 con 0x7fce041596a0 2024-04-25T01:25:56.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.702+0000 7fcdf9ffb640 1 -- 172.21.15.112:0/776331678 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5180 (secure 0 0 0) 0x7fcdfc079f80 con 0x7fce041596a0 2024-04-25T01:25:56.682 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:25:56.682 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/4047219598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4047219598},{"type":"v1","addr":"172.21.15.187:6834","nonce":4047219598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2204640907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2204640907},{"type":"v1","addr":"172.21.15.187:6835","nonce":2204640907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/278686477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":278686477},{"type":"v1","addr":"172.21.15.112:6835","nonce":278686477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:25:52.687260+0000","modified":"2024-04-25T01:25:55.702719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/2115984531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2115984531},{"type":"v1","addr":"172.21.15.112:6837","nonce":2115984531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-25T01:25:56.682 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 7 2024-04-25T01:25:56.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.704+0000 7fce0b2df640 1 -- 172.21.15.112:0/776331678 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcde806be30 msgr2=0x7fcde806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:56.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.704+0000 7fce0b2df640 1 --2- 172.21.15.112:0/776331678 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcde806be30 0x7fcde806e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4006fd0 tx=0x7fcdf4008040 comp rx=0 tx=0).stop 2024-04-25T01:25:56.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.704+0000 7fce0b2df640 1 -- 172.21.15.112:0/776331678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce041596a0 msgr2=0x7fce0414c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:56.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.704+0000 7fce0b2df640 1 --2- 172.21.15.112:0/776331678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce041596a0 0x7fce0414c550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc034040 tx=0x7fcdfc004a40 comp rx=0 tx=0).stop 2024-04-25T01:25:56.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.704+0000 7fce0b2df640 1 -- 172.21.15.112:0/776331678 shutdown_connections 2024-04-25T01:25:56.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.704+0000 7fce0b2df640 1 --2- 172.21.15.112:0/776331678 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcde806be30 0x7fcde806e2f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.704+0000 7fce0b2df640 1 --2- 172.21.15.112:0/776331678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce041596a0 0x7fce0414c550 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.704+0000 7fce0b2df640 1 --2- 172.21.15.112:0/776331678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce041592c0 0x7fce040c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.704+0000 7fce0b2df640 1 --2- 172.21.15.112:0/776331678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce040c72b0 0x7fce040c1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.704+0000 7fce0b2df640 1 -- 172.21.15.112:0/776331678 >> 172.21.15.112:0/776331678 conn(0x7fce040cb170 msgr2=0x7fce040cb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:56.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.704+0000 7fce0b2df640 1 -- 172.21.15.112:0/776331678 shutdown_connections 2024-04-25T01:25:56.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.704+0000 7fce0b2df640 1 -- 172.21.15.112:0/776331678 wait complete. 2024-04-25T01:25:56.694 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:25:56.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.810+0000 7fa6fcf29640 1 Processor -- start 2024-04-25T01:25:56.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.810+0000 7fa6fcf29640 1 -- start start 2024-04-25T01:25:56.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.811+0000 7fa6fcf29640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa6f81524a0 0x7fa6f815de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:56.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.811+0000 7fa6fcf29640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6f81529b0 con 0x7fa6f814c3f0 2024-04-25T01:25:56.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.811+0000 7fa6fcf29640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6f8152c70 con 0x7fa6f80ac3b0 2024-04-25T01:25:56.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.811+0000 7fa6fcf29640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa6f8155a90 con 0x7fa6f81524a0 2024-04-25T01:25:56.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.811+0000 7fa6f659c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa6f814c3f0 0x7fa6f80c2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50704/0 (socket says 172.21.15.112:50704) 2024-04-25T01:25:56.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.811+0000 7fa6f659c640 1 -- 172.21.15.112:0/4078314552 learned_addr learned my addr 172.21.15.112:0/4078314552 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:25:56.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.811+0000 7fa6f6d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa6f81524a0 0x7fa6f815de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:56.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.811+0000 7fa6f6d9d640 1 -- 172.21.15.112:0/4078314552 >> v1:172.21.15.187:6789/0 conn(0x7fa6f80ac3b0 legacy=0x7fa6f8155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:56.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.811+0000 7fa6f5d9b640 1 -- 172.21.15.112:0/4078314552 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 299522796 0 0) 0x7fa6f81529b0 con 0x7fa6f814c3f0 2024-04-25T01:25:56.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.811+0000 7fa6f6d9d640 1 -- 172.21.15.112:0/4078314552 >> v1:172.21.15.112:6789/0 conn(0x7fa6f814c3f0 legacy=0x7fa6f80c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:56.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.811+0000 7fa6f6d9d640 1 -- 172.21.15.112:0/4078314552 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f8152f50 con 0x7fa6f81524a0 2024-04-25T01:25:56.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.812+0000 7fa6f6d9d640 1 --2- 172.21.15.112:0/4078314552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f81524a0 0x7fa6f815de30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009d90 tx=0x7fa6f002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8555a1abdc7dd262 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:56.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.812+0000 7fa6f5d9b640 1 -- 172.21.15.112:0/4078314552 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0031600 con 0x7fa6f81524a0 2024-04-25T01:25:56.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.812+0000 7fa6f5d9b640 1 -- 172.21.15.112:0/4078314552 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6f00317a0 con 0x7fa6f81524a0 2024-04-25T01:25:56.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.812+0000 7fa6f5d9b640 1 -- 172.21.15.112:0/4078314552 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0031b70 con 0x7fa6f81524a0 2024-04-25T01:25:56.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.813+0000 7fa6fcf29640 1 -- 172.21.15.112:0/4078314552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f81524a0 msgr2=0x7fa6f815de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:56.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.813+0000 7fa6fcf29640 1 --2- 172.21.15.112:0/4078314552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f81524a0 0x7fa6f815de30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009d90 tx=0x7fa6f002a610 comp rx=0 tx=0).stop 2024-04-25T01:25:56.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.813+0000 7fa6fcf29640 1 -- 172.21.15.112:0/4078314552 shutdown_connections 2024-04-25T01:25:56.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.813+0000 7fa6fcf29640 1 --2- 172.21.15.112:0/4078314552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f81524a0 0x7fa6f815de30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.813+0000 7fa6fcf29640 1 -- 172.21.15.112:0/4078314552 >> 172.21.15.112:0/4078314552 conn(0x7fa6f806b160 msgr2=0x7fa6f806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:56.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.813+0000 7fa6fcf29640 1 -- 172.21.15.112:0/4078314552 shutdown_connections 2024-04-25T01:25:56.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.813+0000 7fa6fcf29640 1 -- 172.21.15.112:0/4078314552 wait complete. 2024-04-25T01:25:56.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.814+0000 7fa6fcf29640 1 Processor -- start 2024-04-25T01:25:56.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.814+0000 7fa6fcf29640 1 -- start start 2024-04-25T01:25:56.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.814+0000 7fa6fcf29640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f80ac3b0 0x7fa6f813e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:56.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6fcf29640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f814c3f0 0x7fa6f8137370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:56.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6fcf29640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f81524a0 0x7fa6f81378b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:56.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6fcf29640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f80c32a0 con 0x7fa6f80ac3b0 2024-04-25T01:25:56.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6fcf29640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f8070d60 con 0x7fa6f814c3f0 2024-04-25T01:25:56.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6fcf29640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f80c3420 con 0x7fa6f81524a0 2024-04-25T01:25:56.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6f659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f80ac3b0 0x7fa6f813e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:56.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6f659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f80ac3b0 0x7fa6f813e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41286/0 (socket says 172.21.15.112:41286) 2024-04-25T01:25:56.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6f659c640 1 -- 172.21.15.112:0/3742146516 learned_addr learned my addr 172.21.15.112:0/3742146516 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:56.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6f6d9d640 1 --2- 172.21.15.112:0/3742146516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f81524a0 0x7fa6f81378b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:56.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6effff640 1 --2- 172.21.15.112:0/3742146516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f814c3f0 0x7fa6f8137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:56.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6f659c640 1 -- 172.21.15.112:0/3742146516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f81524a0 msgr2=0x7fa6f81378b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:56.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6f659c640 1 --2- 172.21.15.112:0/3742146516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f81524a0 0x7fa6f81378b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6f659c640 1 -- 172.21.15.112:0/3742146516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f814c3f0 msgr2=0x7fa6f8137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:56.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6f659c640 1 --2- 172.21.15.112:0/3742146516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f814c3f0 0x7fa6f8137370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6f659c640 1 -- 172.21.15.112:0/3742146516 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f81ec370 con 0x7fa6f80ac3b0 2024-04-25T01:25:56.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6f6d9d640 1 --2- 172.21.15.112:0/3742146516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f81524a0 0x7fa6f81378b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:25:56.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6effff640 1 --2- 172.21.15.112:0/3742146516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f814c3f0 0x7fa6f8137370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:25:56.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6f659c640 1 --2- 172.21.15.112:0/3742146516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f80ac3b0 0x7fa6f813e750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800c500 tx=0x7fa6e800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:56.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6eeffd640 1 -- 172.21.15.112:0/3742146516 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8018070 con 0x7fa6f80ac3b0 2024-04-25T01:25:56.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6eeffd640 1 -- 172.21.15.112:0/3742146516 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e8007580 con 0x7fa6f80ac3b0 2024-04-25T01:25:56.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6fcf29640 1 -- 172.21.15.112:0/3742146516 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f81ec650 con 0x7fa6f80ac3b0 2024-04-25T01:25:56.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6eeffd640 1 -- 172.21.15.112:0/3742146516 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8004a20 con 0x7fa6f80ac3b0 2024-04-25T01:25:56.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.815+0000 7fa6fcf29640 1 -- 172.21.15.112:0/3742146516 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f81ecc10 con 0x7fa6f80ac3b0 2024-04-25T01:25:56.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.816+0000 7fa6fcf29640 1 -- 172.21.15.112:0/3742146516 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f80682a0 con 0x7fa6f80ac3b0 2024-04-25T01:25:56.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.817+0000 7fa6eeffd640 1 -- 172.21.15.112:0/3742146516 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa6e801b070 con 0x7fa6f80ac3b0 2024-04-25T01:25:56.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.817+0000 7fa6eeffd640 1 --2- 172.21.15.112:0/3742146516 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa6d806be30 0x7fa6d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:56.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.817+0000 7fa6eeffd640 1 -- 172.21.15.112:0/3742146516 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7fa6e808a130 con 0x7fa6f80ac3b0 2024-04-25T01:25:56.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.817+0000 7fa6effff640 1 --2- 172.21.15.112:0/3742146516 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa6d806be30 0x7fa6d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:56.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.817+0000 7fa6effff640 1 --2- 172.21.15.112:0/3742146516 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa6d806be30 0x7fa6d806e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80abb70 tx=0x7fa6e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:56.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.821+0000 7fa6eeffd640 1 -- 172.21.15.112:0/3742146516 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa6e8058b80 con 0x7fa6f80ac3b0 2024-04-25T01:25:56.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.995+0000 7fa6fcf29640 1 -- 172.21.15.112:0/3742146516 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6f813f8c0 con 0x7fa6f80ac3b0 2024-04-25T01:25:56.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.996+0000 7fa6eeffd640 1 -- 172.21.15.112:0/3742146516 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16609 (secure 0 0 0) 0x7fa6e8058480 con 0x7fa6f80ac3b0 2024-04-25T01:25:56.976 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:25:56.976 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:25:54.355478+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:25:56.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.998+0000 7fa6fcf29640 1 -- 172.21.15.112:0/3742146516 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa6d806be30 msgr2=0x7fa6d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:56.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.998+0000 7fa6fcf29640 1 --2- 172.21.15.112:0/3742146516 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa6d806be30 0x7fa6d806e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80abb70 tx=0x7fa6e0009040 comp rx=0 tx=0).stop 2024-04-25T01:25:56.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.998+0000 7fa6fcf29640 1 -- 172.21.15.112:0/3742146516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f80ac3b0 msgr2=0x7fa6f813e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:56.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.998+0000 7fa6fcf29640 1 --2- 172.21.15.112:0/3742146516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f80ac3b0 0x7fa6f813e750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800c500 tx=0x7fa6e800c9c0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.998+0000 7fa6fcf29640 1 -- 172.21.15.112:0/3742146516 shutdown_connections 2024-04-25T01:25:56.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.998+0000 7fa6fcf29640 1 --2- 172.21.15.112:0/3742146516 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa6d806be30 0x7fa6d806e2f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.998+0000 7fa6fcf29640 1 --2- 172.21.15.112:0/3742146516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f81524a0 0x7fa6f81378b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.998+0000 7fa6fcf29640 1 --2- 172.21.15.112:0/3742146516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f814c3f0 0x7fa6f8137370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.998+0000 7fa6fcf29640 1 --2- 172.21.15.112:0/3742146516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f80ac3b0 0x7fa6f813e750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:56.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.998+0000 7fa6fcf29640 1 -- 172.21.15.112:0/3742146516 >> 172.21.15.112:0/3742146516 conn(0x7fa6f806b160 msgr2=0x7fa6f8145e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:56.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.999+0000 7fa6fcf29640 1 -- 172.21.15.112:0/3742146516 shutdown_connections 2024-04-25T01:25:56.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:56.999+0000 7fa6fcf29640 1 -- 172.21.15.112:0/3742146516 wait complete. 2024-04-25T01:25:56.989 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T01:25:56.989 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:25:57.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.106+0000 7f5b8c838640 1 Processor -- start 2024-04-25T01:25:57.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.106+0000 7f5b8c838640 1 -- start start 2024-04-25T01:25:57.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.107+0000 7f5b8c838640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5b880743d0 0x7f5b880747b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:57.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.107+0000 7f5b8c838640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5b8806fb80 0x7f5b88155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:57.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.107+0000 7f5b8c838640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b880c1e40 con 0x7f5b88155f40 2024-04-25T01:25:57.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.107+0000 7f5b8c838640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f5b880c40e0 con 0x7f5b8806fb80 2024-04-25T01:25:57.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.107+0000 7f5b8c838640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5b881534b0 con 0x7f5b880743d0 2024-04-25T01:25:57.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.107+0000 7f5b85d9b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5b8806fb80 0x7f5b88155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:57.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.107+0000 7f5b85d9b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5b8806fb80 0x7f5b88155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41302/0 (socket says 172.21.15.112:41302) 2024-04-25T01:25:57.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.107+0000 7f5b85d9b640 1 -- 172.21.15.112:0/4204487247 learned_addr learned my addr 172.21.15.112:0/4204487247 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:57.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.107+0000 7f5b8659c640 1 --2- 172.21.15.112:0/4204487247 >> v2:172.21.15.187:3300/0 conn(0x7f5b880743d0 0x7f5b880747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:57.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.107+0000 7f5b85d9b640 1 -- 172.21.15.112:0/4204487247 >> v2:172.21.15.187:3300/0 conn(0x7f5b880743d0 msgr2=0x7f5b880747b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:57.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.107+0000 7f5b85d9b640 1 --2- 172.21.15.112:0/4204487247 >> v2:172.21.15.187:3300/0 conn(0x7f5b880743d0 0x7f5b880747b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.107+0000 7f5b85d9b640 1 -- 172.21.15.112:0/4204487247 >> v1:172.21.15.187:6789/0 conn(0x7f5b88155f40 legacy=0x7f5b880c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:57.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.107+0000 7f5b85d9b640 1 -- 172.21.15.112:0/4204487247 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b88159a20 con 0x7f5b8806fb80 2024-04-25T01:25:57.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.108+0000 7f5b85d9b640 1 --2- 172.21.15.112:0/4204487247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b8806fb80 0x7f5b88155a00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5b88067f20 tx=0x7f5b7c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c889b798746e40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:57.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.108+0000 7f5b8559a640 1 -- 172.21.15.112:0/4204487247 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c037070 con 0x7f5b8806fb80 2024-04-25T01:25:57.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.108+0000 7f5b8559a640 1 -- 172.21.15.112:0/4204487247 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7c032440 con 0x7f5b8806fb80 2024-04-25T01:25:57.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.108+0000 7f5b8559a640 1 -- 172.21.15.112:0/4204487247 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c032810 con 0x7f5b8806fb80 2024-04-25T01:25:57.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.109+0000 7f5b8c838640 1 -- 172.21.15.112:0/4204487247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b8806fb80 msgr2=0x7f5b88155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:57.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.109+0000 7f5b8c838640 1 --2- 172.21.15.112:0/4204487247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b8806fb80 0x7f5b88155a00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5b88067f20 tx=0x7f5b7c02a1a0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.109+0000 7f5b8c838640 1 -- 172.21.15.112:0/4204487247 shutdown_connections 2024-04-25T01:25:57.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.109+0000 7f5b8c838640 1 --2- 172.21.15.112:0/4204487247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b8806fb80 0x7f5b88155a00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.109+0000 7f5b8c838640 1 --2- 172.21.15.112:0/4204487247 >> v2:172.21.15.187:3300/0 conn(0x7f5b880743d0 0x7f5b880747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.109+0000 7f5b8c838640 1 -- 172.21.15.112:0/4204487247 >> 172.21.15.112:0/4204487247 conn(0x7f5b8806b160 msgr2=0x7f5b8806b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:57.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.109+0000 7f5b8c838640 1 -- 172.21.15.112:0/4204487247 shutdown_connections 2024-04-25T01:25:57.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.109+0000 7f5b8c838640 1 -- 172.21.15.112:0/4204487247 wait complete. 2024-04-25T01:25:57.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.110+0000 7f5b8c838640 1 Processor -- start 2024-04-25T01:25:57.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.110+0000 7f5b8c838640 1 -- start start 2024-04-25T01:25:57.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.111+0000 7f5b8c838640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8806fb80 0x7f5b881ee510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:57.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.111+0000 7f5b8c838640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b880743d0 0x7f5b881eb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:57.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.111+0000 7f5b8c838640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b88155f40 0x7f5b8806e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:57.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.111+0000 7f5b8c838640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b880c19b0 con 0x7f5b880743d0 2024-04-25T01:25:57.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.111+0000 7f5b8c838640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5b880c16d0 con 0x7f5b8806fb80 2024-04-25T01:25:57.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.111+0000 7f5b8c838640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5b880a7070 con 0x7f5b88155f40 2024-04-25T01:25:57.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.111+0000 7f5b85d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b880743d0 0x7f5b881eb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:57.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.111+0000 7f5b85d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b880743d0 0x7f5b881eb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41308/0 (socket says 172.21.15.112:41308) 2024-04-25T01:25:57.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.111+0000 7f5b85d9b640 1 -- 172.21.15.112:0/1558295462 learned_addr learned my addr 172.21.15.112:0/1558295462 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:57.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.111+0000 7f5b86d9d640 1 --2- 172.21.15.112:0/1558295462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b88155f40 0x7f5b8806e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:57.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.111+0000 7f5b8659c640 1 --2- 172.21.15.112:0/1558295462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8806fb80 0x7f5b881ee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:57.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.111+0000 7f5b85d9b640 1 -- 172.21.15.112:0/1558295462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b88155f40 msgr2=0x7f5b8806e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:57.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.111+0000 7f5b85d9b640 1 --2- 172.21.15.112:0/1558295462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b88155f40 0x7f5b8806e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.111+0000 7f5b85d9b640 1 -- 172.21.15.112:0/1558295462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8806fb80 msgr2=0x7f5b881ee510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:57.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.111+0000 7f5b85d9b640 1 --2- 172.21.15.112:0/1558295462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8806fb80 0x7f5b881ee510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.111+0000 7f5b85d9b640 1 -- 172.21.15.112:0/1558295462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b881eafa0 con 0x7f5b880743d0 2024-04-25T01:25:57.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.111+0000 7f5b8659c640 1 --2- 172.21.15.112:0/1558295462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8806fb80 0x7f5b881ee510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:25:57.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.111+0000 7f5b85d9b640 1 --2- 172.21.15.112:0/1558295462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b880743d0 0x7f5b881eb740 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c000c00 tx=0x7f5b7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:57.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.112+0000 7f5b76ffd640 1 -- 172.21.15.112:0/1558295462 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c041070 con 0x7f5b880743d0 2024-04-25T01:25:57.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.112+0000 7f5b76ffd640 1 -- 172.21.15.112:0/1558295462 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7c03c870 con 0x7f5b880743d0 2024-04-25T01:25:57.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.112+0000 7f5b8c838640 1 -- 172.21.15.112:0/1558295462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b88152570 con 0x7f5b880743d0 2024-04-25T01:25:57.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.112+0000 7f5b76ffd640 1 -- 172.21.15.112:0/1558295462 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c037040 con 0x7f5b880743d0 2024-04-25T01:25:57.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.112+0000 7f5b8c838640 1 -- 172.21.15.112:0/1558295462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b88152b30 con 0x7f5b880743d0 2024-04-25T01:25:57.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.113+0000 7f5b8c838640 1 -- 172.21.15.112:0/1558295462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b880682a0 con 0x7f5b880743d0 2024-04-25T01:25:57.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.113+0000 7f5b76ffd640 1 -- 172.21.15.112:0/1558295462 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b7c044070 con 0x7f5b880743d0 2024-04-25T01:25:57.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.113+0000 7f5b76ffd640 1 --2- 172.21.15.112:0/1558295462 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b6806be30 0x7f5b6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:57.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.113+0000 7f5b76ffd640 1 -- 172.21.15.112:0/1558295462 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7f5b7c0abb80 con 0x7f5b880743d0 2024-04-25T01:25:57.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.113+0000 7f5b8659c640 1 --2- 172.21.15.112:0/1558295462 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b6806be30 0x7f5b6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:57.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.114+0000 7f5b8659c640 1 --2- 172.21.15.112:0/1558295462 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b6806be30 0x7f5b6806e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5b78006fd0 tx=0x7f5b78008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:57.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.118+0000 7f5b76ffd640 1 -- 172.21.15.112:0/1558295462 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5b881eafa0 con 0x7f5b880743d0 2024-04-25T01:25:57.187 INFO:tasks.ceph.mds.c.smithi112.stderr:2024-04-25T01:25:57.208+0000 7fceb8b48640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T01:25:57.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.288+0000 7f5b8c838640 1 -- 172.21.15.112:0/1558295462 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5b880c2d30 con 0x7f5b6806be30 2024-04-25T01:25:57.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.294+0000 7f5b76ffd640 1 -- 172.21.15.112:0/1558295462 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7f5b880c2d30 con 0x7f5b6806be30 2024-04-25T01:25:57.276 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:25:57.278 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T01:25:57.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.297+0000 7f5b8c838640 1 -- 172.21.15.112:0/1558295462 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b6806be30 msgr2=0x7f5b6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:57.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.297+0000 7f5b8c838640 1 --2- 172.21.15.112:0/1558295462 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b6806be30 0x7f5b6806e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5b78006fd0 tx=0x7f5b78008040 comp rx=0 tx=0).stop 2024-04-25T01:25:57.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.297+0000 7f5b8c838640 1 -- 172.21.15.112:0/1558295462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b880743d0 msgr2=0x7f5b881eb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:57.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.297+0000 7f5b8c838640 1 --2- 172.21.15.112:0/1558295462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b880743d0 0x7f5b881eb740 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c000c00 tx=0x7f5b7c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.297+0000 7f5b8c838640 1 -- 172.21.15.112:0/1558295462 shutdown_connections 2024-04-25T01:25:57.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.297+0000 7f5b8c838640 1 --2- 172.21.15.112:0/1558295462 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b6806be30 0x7f5b6806e2f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.297+0000 7f5b8c838640 1 --2- 172.21.15.112:0/1558295462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b88155f40 0x7f5b8806e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.297+0000 7f5b8c838640 1 --2- 172.21.15.112:0/1558295462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b880743d0 0x7f5b881eb740 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.297+0000 7f5b8c838640 1 --2- 172.21.15.112:0/1558295462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8806fb80 0x7f5b881ee510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.297+0000 7f5b8c838640 1 -- 172.21.15.112:0/1558295462 >> 172.21.15.112:0/1558295462 conn(0x7f5b8806b160 msgr2=0x7f5b88072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:57.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.297+0000 7f5b8c838640 1 -- 172.21.15.112:0/1558295462 shutdown_connections 2024-04-25T01:25:57.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.297+0000 7f5b8c838640 1 -- 172.21.15.112:0/1558295462 wait complete. 2024-04-25T01:25:57.287 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-25T01:25:56.649037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219016,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755704,"statfs":{"total":773094113280,"available":772869840896,"internally_reserved":0,"allocated":2891776,"data_stored":1752388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.314757"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581611+0000","last_change":"2024-04-25T01:25:53.367762+0000","last_active":"2024-04-25T01:25:54.581611+0000","last_peered":"2024-04-25T01:25:54.581611+0000","last_clean":"2024-04-25T01:25:54.581611+0000","last_became_active":"2024-04-25T01:25:53.367383+0000","last_became_peered":"2024-04-25T01:25:53.367383+0000","last_unstale":"2024-04-25T01:25:54.581611+0000","last_undegraded":"2024-04-25T01:25:54.581611+0000","last_fullsized":"2024-04-25T01:25:54.581611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:05.589744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366023+0000","last_change":"2024-04-25T01:25:53.365057+0000","last_active":"2024-04-25T01:25:54.366023+0000","last_peered":"2024-04-25T01:25:54.366023+0000","last_clean":"2024-04-25T01:25:54.366023+0000","last_became_active":"2024-04-25T01:25:53.364793+0000","last_became_peered":"2024-04-25T01:25:53.364793+0000","last_unstale":"2024-04-25T01:25:54.366023+0000","last_undegraded":"2024-04-25T01:25:54.366023+0000","last_fullsized":"2024-04-25T01:25:54.366023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361426+0000","last_change":"2024-04-25T01:25:53.363829+0000","last_active":"2024-04-25T01:25:54.361426+0000","last_peered":"2024-04-25T01:25:54.361426+0000","last_clean":"2024-04-25T01:25:54.361426+0000","last_became_active":"2024-04-25T01:25:53.363525+0000","last_became_peered":"2024-04-25T01:25:53.363525+0000","last_unstale":"2024-04-25T01:25:54.361426+0000","last_undegraded":"2024-04-25T01:25:54.361426+0000","last_fullsized":"2024-04-25T01:25:54.361426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579956+0000","last_change":"2024-04-25T01:25:53.371513+0000","last_active":"2024-04-25T01:25:54.579956+0000","last_peered":"2024-04-25T01:25:54.579956+0000","last_clean":"2024-04-25T01:25:54.579956+0000","last_became_active":"2024-04-25T01:25:53.370421+0000","last_became_peered":"2024-04-25T01:25:53.370421+0000","last_unstale":"2024-04-25T01:25:54.579956+0000","last_undegraded":"2024-04-25T01:25:54.579956+0000","last_fullsized":"2024-04-25T01:25:54.579956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581049+0000","last_change":"2024-04-25T01:25:53.367953+0000","last_active":"2024-04-25T01:25:54.581049+0000","last_peered":"2024-04-25T01:25:54.581049+0000","last_clean":"2024-04-25T01:25:54.581049+0000","last_became_active":"2024-04-25T01:25:53.367358+0000","last_became_peered":"2024-04-25T01:25:53.367358+0000","last_unstale":"2024-04-25T01:25:54.581049+0000","last_undegraded":"2024-04-25T01:25:54.581049+0000","last_fullsized":"2024-04-25T01:25:54.581049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579967+0000","last_change":"2024-04-25T01:25:53.375324+0000","last_active":"2024-04-25T01:25:54.579967+0000","last_peered":"2024-04-25T01:25:54.579967+0000","last_clean":"2024-04-25T01:25:54.579967+0000","last_became_active":"2024-04-25T01:25:53.374392+0000","last_became_peered":"2024-04-25T01:25:53.374392+0000","last_unstale":"2024-04-25T01:25:54.579967+0000","last_undegraded":"2024-04-25T01:25:54.579967+0000","last_fullsized":"2024-04-25T01:25:54.579967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579900+0000","last_change":"2024-04-25T01:25:53.364698+0000","last_active":"2024-04-25T01:25:54.579900+0000","last_peered":"2024-04-25T01:25:54.579900+0000","last_clean":"2024-04-25T01:25:54.579900+0000","last_became_active":"2024-04-25T01:25:53.364435+0000","last_became_peered":"2024-04-25T01:25:53.364435+0000","last_unstale":"2024-04-25T01:25:54.579900+0000","last_undegraded":"2024-04-25T01:25:54.579900+0000","last_fullsized":"2024-04-25T01:25:54.579900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366265+0000","last_change":"2024-04-25T01:25:53.374061+0000","last_active":"2024-04-25T01:25:54.366265+0000","last_peered":"2024-04-25T01:25:54.366265+0000","last_clean":"2024-04-25T01:25:54.366265+0000","last_became_active":"2024-04-25T01:25:53.373848+0000","last_became_peered":"2024-04-25T01:25:53.373848+0000","last_unstale":"2024-04-25T01:25:54.366265+0000","last_undegraded":"2024-04-25T01:25:54.366265+0000","last_fullsized":"2024-04-25T01:25:54.366265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581623+0000","last_change":"2024-04-25T01:25:53.376081+0000","last_active":"2024-04-25T01:25:54.581623+0000","last_peered":"2024-04-25T01:25:54.581623+0000","last_clean":"2024-04-25T01:25:54.581623+0000","last_became_active":"2024-04-25T01:25:53.375814+0000","last_became_peered":"2024-04-25T01:25:53.375814+0000","last_unstale":"2024-04-25T01:25:54.581623+0000","last_undegraded":"2024-04-25T01:25:54.581623+0000","last_fullsized":"2024-04-25T01:25:54.581623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:29.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368588+0000","last_change":"2024-04-25T01:25:53.372842+0000","last_active":"2024-04-25T01:25:54.368588+0000","last_peered":"2024-04-25T01:25:54.368588+0000","last_clean":"2024-04-25T01:25:54.368588+0000","last_became_active":"2024-04-25T01:25:53.372580+0000","last_became_peered":"2024-04-25T01:25:53.372580+0000","last_unstale":"2024-04-25T01:25:54.368588+0000","last_undegraded":"2024-04-25T01:25:54.368588+0000","last_fullsized":"2024-04-25T01:25:54.368588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368507+0000","last_change":"2024-04-25T01:25:53.371647+0000","last_active":"2024-04-25T01:25:54.368507+0000","last_peered":"2024-04-25T01:25:54.368507+0000","last_clean":"2024-04-25T01:25:54.368507+0000","last_became_active":"2024-04-25T01:25:53.371353+0000","last_became_peered":"2024-04-25T01:25:53.371353+0000","last_unstale":"2024-04-25T01:25:54.368507+0000","last_undegraded":"2024-04-25T01:25:54.368507+0000","last_fullsized":"2024-04-25T01:25:54.368507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580179+0000","last_change":"2024-04-25T01:25:53.364457+0000","last_active":"2024-04-25T01:25:54.580179+0000","last_peered":"2024-04-25T01:25:54.580179+0000","last_clean":"2024-04-25T01:25:54.580179+0000","last_became_active":"2024-04-25T01:25:53.364103+0000","last_became_peered":"2024-04-25T01:25:53.364103+0000","last_unstale":"2024-04-25T01:25:54.580179+0000","last_undegraded":"2024-04-25T01:25:54.580179+0000","last_fullsized":"2024-04-25T01:25:54.580179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:11.336752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366544+0000","last_change":"2024-04-25T01:25:53.363512+0000","last_active":"2024-04-25T01:25:54.366544+0000","last_peered":"2024-04-25T01:25:54.366544+0000","last_clean":"2024-04-25T01:25:54.366544+0000","last_became_active":"2024-04-25T01:25:53.363246+0000","last_became_peered":"2024-04-25T01:25:53.363246+0000","last_unstale":"2024-04-25T01:25:54.366544+0000","last_undegraded":"2024-04-25T01:25:54.366544+0000","last_fullsized":"2024-04-25T01:25:54.366544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361273+0000","last_change":"2024-04-25T01:25:53.370996+0000","last_active":"2024-04-25T01:25:54.361273+0000","last_peered":"2024-04-25T01:25:54.361273+0000","last_clean":"2024-04-25T01:25:54.361273+0000","last_became_active":"2024-04-25T01:25:53.370479+0000","last_became_peered":"2024-04-25T01:25:53.370479+0000","last_unstale":"2024-04-25T01:25:54.361273+0000","last_undegraded":"2024-04-25T01:25:54.361273+0000","last_fullsized":"2024-04-25T01:25:54.361273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:57.636067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366008+0000","last_change":"2024-04-25T01:25:53.371602+0000","last_active":"2024-04-25T01:25:54.366008+0000","last_peered":"2024-04-25T01:25:54.366008+0000","last_clean":"2024-04-25T01:25:54.366008+0000","last_became_active":"2024-04-25T01:25:53.371362+0000","last_became_peered":"2024-04-25T01:25:53.371362+0000","last_unstale":"2024-04-25T01:25:54.366008+0000","last_undegraded":"2024-04-25T01:25:54.366008+0000","last_fullsized":"2024-04-25T01:25:54.366008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579564+0000","last_change":"2024-04-25T01:25:53.374186+0000","last_active":"2024-04-25T01:25:54.579564+0000","last_peered":"2024-04-25T01:25:54.579564+0000","last_clean":"2024-04-25T01:25:54.579564+0000","last_became_active":"2024-04-25T01:25:53.373766+0000","last_became_peered":"2024-04-25T01:25:53.373766+0000","last_unstale":"2024-04-25T01:25:54.579564+0000","last_undegraded":"2024-04-25T01:25:54.579564+0000","last_fullsized":"2024-04-25T01:25:54.579564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580993+0000","last_change":"2024-04-25T01:25:53.372048+0000","last_active":"2024-04-25T01:25:54.580993+0000","last_peered":"2024-04-25T01:25:54.580993+0000","last_clean":"2024-04-25T01:25:54.580993+0000","last_became_active":"2024-04-25T01:25:53.371767+0000","last_became_peered":"2024-04-25T01:25:53.371767+0000","last_unstale":"2024-04-25T01:25:54.580993+0000","last_undegraded":"2024-04-25T01:25:54.580993+0000","last_fullsized":"2024-04-25T01:25:54.580993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:05.589744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579175+0000","last_change":"2024-04-25T01:25:53.370666+0000","last_active":"2024-04-25T01:25:54.579175+0000","last_peered":"2024-04-25T01:25:54.579175+0000","last_clean":"2024-04-25T01:25:54.579175+0000","last_became_active":"2024-04-25T01:25:53.369733+0000","last_became_peered":"2024-04-25T01:25:53.369733+0000","last_unstale":"2024-04-25T01:25:54.579175+0000","last_undegraded":"2024-04-25T01:25:54.579175+0000","last_fullsized":"2024-04-25T01:25:54.579175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581475+0000","last_change":"2024-04-25T01:25:53.369585+0000","last_active":"2024-04-25T01:25:54.581475+0000","last_peered":"2024-04-25T01:25:54.581475+0000","last_clean":"2024-04-25T01:25:54.581475+0000","last_became_active":"2024-04-25T01:25:53.368499+0000","last_became_peered":"2024-04-25T01:25:53.368499+0000","last_unstale":"2024-04-25T01:25:54.581475+0000","last_undegraded":"2024-04-25T01:25:54.581475+0000","last_fullsized":"2024-04-25T01:25:54.581475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368327+0000","last_change":"2024-04-25T01:25:53.370314+0000","last_active":"2024-04-25T01:25:54.368327+0000","last_peered":"2024-04-25T01:25:54.368327+0000","last_clean":"2024-04-25T01:25:54.368327+0000","last_became_active":"2024-04-25T01:25:53.370049+0000","last_became_peered":"2024-04-25T01:25:53.370049+0000","last_unstale":"2024-04-25T01:25:54.368327+0000","last_undegraded":"2024-04-25T01:25:54.368327+0000","last_fullsized":"2024-04-25T01:25:54.368327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361655+0000","last_change":"2024-04-25T01:25:53.376415+0000","last_active":"2024-04-25T01:25:54.361655+0000","last_peered":"2024-04-25T01:25:54.361655+0000","last_clean":"2024-04-25T01:25:54.361655+0000","last_became_active":"2024-04-25T01:25:53.376104+0000","last_became_peered":"2024-04-25T01:25:53.376104+0000","last_unstale":"2024-04-25T01:25:54.361655+0000","last_undegraded":"2024-04-25T01:25:54.361655+0000","last_fullsized":"2024-04-25T01:25:54.361655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580264+0000","last_change":"2024-04-25T01:25:53.371240+0000","last_active":"2024-04-25T01:25:54.580264+0000","last_peered":"2024-04-25T01:25:54.580264+0000","last_clean":"2024-04-25T01:25:54.580264+0000","last_became_active":"2024-04-25T01:25:53.370164+0000","last_became_peered":"2024-04-25T01:25:53.370164+0000","last_unstale":"2024-04-25T01:25:54.580264+0000","last_undegraded":"2024-04-25T01:25:54.580264+0000","last_fullsized":"2024-04-25T01:25:54.580264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366205+0000","last_change":"2024-04-25T01:25:53.355667+0000","last_active":"2024-04-25T01:25:54.366205+0000","last_peered":"2024-04-25T01:25:54.366205+0000","last_clean":"2024-04-25T01:25:54.366205+0000","last_became_active":"2024-04-25T01:25:53.355417+0000","last_became_peered":"2024-04-25T01:25:53.355417+0000","last_unstale":"2024-04-25T01:25:54.366205+0000","last_undegraded":"2024-04-25T01:25:54.366205+0000","last_fullsized":"2024-04-25T01:25:54.366205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579870+0000","last_change":"2024-04-25T01:25:53.365391+0000","last_active":"2024-04-25T01:25:54.579870+0000","last_peered":"2024-04-25T01:25:54.579870+0000","last_clean":"2024-04-25T01:25:54.579870+0000","last_became_active":"2024-04-25T01:25:53.364829+0000","last_became_peered":"2024-04-25T01:25:53.364829+0000","last_unstale":"2024-04-25T01:25:54.579870+0000","last_undegraded":"2024-04-25T01:25:54.579870+0000","last_fullsized":"2024-04-25T01:25:54.579870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366392+0000","last_change":"2024-04-25T01:25:53.365509+0000","last_active":"2024-04-25T01:25:54.366392+0000","last_peered":"2024-04-25T01:25:54.366392+0000","last_clean":"2024-04-25T01:25:54.366392+0000","last_became_active":"2024-04-25T01:25:53.365245+0000","last_became_peered":"2024-04-25T01:25:53.365245+0000","last_unstale":"2024-04-25T01:25:54.366392+0000","last_undegraded":"2024-04-25T01:25:54.366392+0000","last_fullsized":"2024-04-25T01:25:54.366392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368629+0000","last_change":"2024-04-25T01:25:53.372638+0000","last_active":"2024-04-25T01:25:54.368629+0000","last_peered":"2024-04-25T01:25:54.368629+0000","last_clean":"2024-04-25T01:25:54.368629+0000","last_became_active":"2024-04-25T01:25:53.372001+0000","last_became_peered":"2024-04-25T01:25:53.372001+0000","last_unstale":"2024-04-25T01:25:54.368629+0000","last_undegraded":"2024-04-25T01:25:54.368629+0000","last_fullsized":"2024-04-25T01:25:54.368629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.365990+0000","last_change":"2024-04-25T01:25:53.368413+0000","last_active":"2024-04-25T01:25:54.365990+0000","last_peered":"2024-04-25T01:25:54.365990+0000","last_clean":"2024-04-25T01:25:54.365990+0000","last_became_active":"2024-04-25T01:25:53.368176+0000","last_became_peered":"2024-04-25T01:25:53.368176+0000","last_unstale":"2024-04-25T01:25:54.365990+0000","last_undegraded":"2024-04-25T01:25:54.365990+0000","last_fullsized":"2024-04-25T01:25:54.365990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366985+0000","last_change":"2024-04-25T01:25:53.364575+0000","last_active":"2024-04-25T01:25:54.366985+0000","last_peered":"2024-04-25T01:25:54.366985+0000","last_clean":"2024-04-25T01:25:54.366985+0000","last_became_active":"2024-04-25T01:25:53.364311+0000","last_became_peered":"2024-04-25T01:25:53.364311+0000","last_unstale":"2024-04-25T01:25:54.366985+0000","last_undegraded":"2024-04-25T01:25:54.366985+0000","last_fullsized":"2024-04-25T01:25:54.366985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361693+0000","last_change":"2024-04-25T01:25:53.357142+0000","last_active":"2024-04-25T01:25:54.361693+0000","last_peered":"2024-04-25T01:25:54.361693+0000","last_clean":"2024-04-25T01:25:54.361693+0000","last_became_active":"2024-04-25T01:25:53.356854+0000","last_became_peered":"2024-04-25T01:25:53.356854+0000","last_unstale":"2024-04-25T01:25:54.361693+0000","last_undegraded":"2024-04-25T01:25:54.361693+0000","last_fullsized":"2024-04-25T01:25:54.361693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580456+0000","last_change":"2024-04-25T01:25:53.369850+0000","last_active":"2024-04-25T01:25:54.580456+0000","last_peered":"2024-04-25T01:25:54.580456+0000","last_clean":"2024-04-25T01:25:54.580456+0000","last_became_active":"2024-04-25T01:25:53.369350+0000","last_became_peered":"2024-04-25T01:25:53.369350+0000","last_unstale":"2024-04-25T01:25:54.580456+0000","last_undegraded":"2024-04-25T01:25:54.580456+0000","last_fullsized":"2024-04-25T01:25:54.580456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579112+0000","last_change":"2024-04-25T01:25:53.371345+0000","last_active":"2024-04-25T01:25:54.579112+0000","last_peered":"2024-04-25T01:25:54.579112+0000","last_clean":"2024-04-25T01:25:54.579112+0000","last_became_active":"2024-04-25T01:25:53.371080+0000","last_became_peered":"2024-04-25T01:25:53.371080+0000","last_unstale":"2024-04-25T01:25:54.579112+0000","last_undegraded":"2024-04-25T01:25:54.579112+0000","last_fullsized":"2024-04-25T01:25:54.579112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:09.855763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368224+0000","last_change":"2024-04-25T01:25:53.367143+0000","last_active":"2024-04-25T01:25:54.368224+0000","last_peered":"2024-04-25T01:25:54.368224+0000","last_clean":"2024-04-25T01:25:54.368224+0000","last_became_active":"2024-04-25T01:25:53.366847+0000","last_became_peered":"2024-04-25T01:25:53.366847+0000","last_unstale":"2024-04-25T01:25:54.368224+0000","last_undegraded":"2024-04-25T01:25:54.368224+0000","last_fullsized":"2024-04-25T01:25:54.368224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579943+0000","last_change":"2024-04-25T01:25:53.366586+0000","last_active":"2024-04-25T01:25:54.579943+0000","last_peered":"2024-04-25T01:25:54.579943+0000","last_clean":"2024-04-25T01:25:54.579943+0000","last_became_active":"2024-04-25T01:25:53.366302+0000","last_became_peered":"2024-04-25T01:25:53.366302+0000","last_unstale":"2024-04-25T01:25:54.579943+0000","last_undegraded":"2024-04-25T01:25:54.579943+0000","last_fullsized":"2024-04-25T01:25:54.579943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366663+0000","last_change":"2024-04-25T01:25:52.353955+0000","last_active":"2024-04-25T01:25:54.366663+0000","last_peered":"2024-04-25T01:25:54.366663+0000","last_clean":"2024-04-25T01:25:54.366663+0000","last_became_active":"2024-04-25T01:25:52.353659+0000","last_became_peered":"2024-04-25T01:25:52.353659+0000","last_unstale":"2024-04-25T01:25:54.366663+0000","last_undegraded":"2024-04-25T01:25:54.366663+0000","last_fullsized":"2024-04-25T01:25:54.366663+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361779+0000","last_change":"2024-04-25T01:25:53.356610+0000","last_active":"2024-04-25T01:25:54.361779+0000","last_peered":"2024-04-25T01:25:54.361779+0000","last_clean":"2024-04-25T01:25:54.361779+0000","last_became_active":"2024-04-25T01:25:53.356015+0000","last_became_peered":"2024-04-25T01:25:53.356015+0000","last_unstale":"2024-04-25T01:25:54.361779+0000","last_undegraded":"2024-04-25T01:25:54.361779+0000","last_fullsized":"2024-04-25T01:25:54.361779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368267+0000","last_change":"2024-04-25T01:25:52.349989+0000","last_active":"2024-04-25T01:25:54.368267+0000","last_peered":"2024-04-25T01:25:54.368267+0000","last_clean":"2024-04-25T01:25:54.368267+0000","last_became_active":"2024-04-25T01:25:52.349672+0000","last_became_peered":"2024-04-25T01:25:52.349672+0000","last_unstale":"2024-04-25T01:25:54.368267+0000","last_undegraded":"2024-04-25T01:25:54.368267+0000","last_fullsized":"2024-04-25T01:25:54.368267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366294+0000","last_change":"2024-04-25T01:25:52.347464+0000","last_active":"2024-04-25T01:25:54.366294+0000","last_peered":"2024-04-25T01:25:54.366294+0000","last_clean":"2024-04-25T01:25:54.366294+0000","last_became_active":"2024-04-25T01:25:52.347138+0000","last_became_peered":"2024-04-25T01:25:52.347138+0000","last_unstale":"2024-04-25T01:25:54.366294+0000","last_undegraded":"2024-04-25T01:25:54.366294+0000","last_fullsized":"2024-04-25T01:25:54.366294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366446+0000","last_change":"2024-04-25T01:25:53.363794+0000","last_active":"2024-04-25T01:25:54.366446+0000","last_peered":"2024-04-25T01:25:54.366446+0000","last_clean":"2024-04-25T01:25:54.366446+0000","last_became_active":"2024-04-25T01:25:53.363509+0000","last_became_peered":"2024-04-25T01:25:53.363509+0000","last_unstale":"2024-04-25T01:25:54.366446+0000","last_undegraded":"2024-04-25T01:25:54.366446+0000","last_fullsized":"2024-04-25T01:25:54.366446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579760+0000","last_change":"2024-04-25T01:25:52.358081+0000","last_active":"2024-04-25T01:25:54.579760+0000","last_peered":"2024-04-25T01:25:54.579760+0000","last_clean":"2024-04-25T01:25:54.579760+0000","last_became_active":"2024-04-25T01:25:52.357365+0000","last_became_peered":"2024-04-25T01:25:52.357365+0000","last_unstale":"2024-04-25T01:25:54.579760+0000","last_undegraded":"2024-04-25T01:25:54.579760+0000","last_fullsized":"2024-04-25T01:25:54.579760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.365858+0000","last_change":"2024-04-25T01:25:53.363043+0000","last_active":"2024-04-25T01:25:54.365858+0000","last_peered":"2024-04-25T01:25:54.365858+0000","last_clean":"2024-04-25T01:25:54.365858+0000","last_became_active":"2024-04-25T01:25:53.362601+0000","last_became_peered":"2024-04-25T01:25:53.362601+0000","last_unstale":"2024-04-25T01:25:54.365858+0000","last_undegraded":"2024-04-25T01:25:54.365858+0000","last_fullsized":"2024-04-25T01:25:54.365858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368458+0000","last_change":"2024-04-25T01:25:52.357459+0000","last_active":"2024-04-25T01:25:54.368458+0000","last_peered":"2024-04-25T01:25:54.368458+0000","last_clean":"2024-04-25T01:25:54.368458+0000","last_became_active":"2024-04-25T01:25:52.357182+0000","last_became_peered":"2024-04-25T01:25:52.357182+0000","last_unstale":"2024-04-25T01:25:54.368458+0000","last_undegraded":"2024-04-25T01:25:54.368458+0000","last_fullsized":"2024-04-25T01:25:54.368458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580762+0000","last_change":"2024-04-25T01:25:53.371445+0000","last_active":"2024-04-25T01:25:54.580762+0000","last_peered":"2024-04-25T01:25:54.580762+0000","last_clean":"2024-04-25T01:25:54.580762+0000","last_became_active":"2024-04-25T01:25:53.371086+0000","last_became_peered":"2024-04-25T01:25:53.371086+0000","last_unstale":"2024-04-25T01:25:54.580762+0000","last_undegraded":"2024-04-25T01:25:54.580762+0000","last_fullsized":"2024-04-25T01:25:54.580762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580764+0000","last_change":"2024-04-25T01:25:52.356231+0000","last_active":"2024-04-25T01:25:54.580764+0000","last_peered":"2024-04-25T01:25:54.580764+0000","last_clean":"2024-04-25T01:25:54.580764+0000","last_became_active":"2024-04-25T01:25:52.355913+0000","last_became_peered":"2024-04-25T01:25:52.355913+0000","last_unstale":"2024-04-25T01:25:54.580764+0000","last_undegraded":"2024-04-25T01:25:54.580764+0000","last_fullsized":"2024-04-25T01:25:54.580764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361600+0000","last_change":"2024-04-25T01:25:53.374311+0000","last_active":"2024-04-25T01:25:54.361600+0000","last_peered":"2024-04-25T01:25:54.361600+0000","last_clean":"2024-04-25T01:25:54.361600+0000","last_became_active":"2024-04-25T01:25:53.373890+0000","last_became_peered":"2024-04-25T01:25:53.373890+0000","last_unstale":"2024-04-25T01:25:54.361600+0000","last_undegraded":"2024-04-25T01:25:54.361600+0000","last_fullsized":"2024-04-25T01:25:54.361600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366048+0000","last_change":"2024-04-25T01:25:52.357610+0000","last_active":"2024-04-25T01:25:54.366048+0000","last_peered":"2024-04-25T01:25:54.366048+0000","last_clean":"2024-04-25T01:25:54.366048+0000","last_became_active":"2024-04-25T01:25:52.357355+0000","last_became_peered":"2024-04-25T01:25:52.357355+0000","last_unstale":"2024-04-25T01:25:54.366048+0000","last_undegraded":"2024-04-25T01:25:54.366048+0000","last_fullsized":"2024-04-25T01:25:54.366048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580344+0000","last_change":"2024-04-25T01:25:53.366416+0000","last_active":"2024-04-25T01:25:54.580344+0000","last_peered":"2024-04-25T01:25:54.580344+0000","last_clean":"2024-04-25T01:25:54.580344+0000","last_became_active":"2024-04-25T01:25:53.366157+0000","last_became_peered":"2024-04-25T01:25:53.366157+0000","last_unstale":"2024-04-25T01:25:54.580344+0000","last_undegraded":"2024-04-25T01:25:54.580344+0000","last_fullsized":"2024-04-25T01:25:54.580344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:26.277566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368293+0000","last_change":"2024-04-25T01:25:52.350446+0000","last_active":"2024-04-25T01:25:54.368293+0000","last_peered":"2024-04-25T01:25:54.368293+0000","last_clean":"2024-04-25T01:25:54.368293+0000","last_became_active":"2024-04-25T01:25:52.349448+0000","last_became_peered":"2024-04-25T01:25:52.349448+0000","last_unstale":"2024-04-25T01:25:54.368293+0000","last_undegraded":"2024-04-25T01:25:54.368293+0000","last_fullsized":"2024-04-25T01:25:54.368293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581200+0000","last_change":"2024-04-25T01:25:53.373572+0000","last_active":"2024-04-25T01:25:54.581200+0000","last_peered":"2024-04-25T01:25:54.581200+0000","last_clean":"2024-04-25T01:25:54.581200+0000","last_became_active":"2024-04-25T01:25:53.373129+0000","last_became_peered":"2024-04-25T01:25:53.373129+0000","last_unstale":"2024-04-25T01:25:54.581200+0000","last_undegraded":"2024-04-25T01:25:54.581200+0000","last_fullsized":"2024-04-25T01:25:54.581200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361780+0000","last_change":"2024-04-25T01:25:52.349092+0000","last_active":"2024-04-25T01:25:54.361780+0000","last_peered":"2024-04-25T01:25:54.361780+0000","last_clean":"2024-04-25T01:25:54.361780+0000","last_became_active":"2024-04-25T01:25:52.348846+0000","last_became_peered":"2024-04-25T01:25:52.348846+0000","last_unstale":"2024-04-25T01:25:54.361780+0000","last_undegraded":"2024-04-25T01:25:54.361780+0000","last_fullsized":"2024-04-25T01:25:54.361780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:21.982440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361411+0000","last_change":"2024-04-25T01:25:53.370720+0000","last_active":"2024-04-25T01:25:54.361411+0000","last_peered":"2024-04-25T01:25:54.361411+0000","last_clean":"2024-04-25T01:25:54.361411+0000","last_became_active":"2024-04-25T01:25:53.370142+0000","last_became_peered":"2024-04-25T01:25:53.370142+0000","last_unstale":"2024-04-25T01:25:54.361411+0000","last_undegraded":"2024-04-25T01:25:54.361411+0000","last_fullsized":"2024-04-25T01:25:54.361411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:52.833379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580473+0000","last_change":"2024-04-25T01:25:52.357397+0000","last_active":"2024-04-25T01:25:54.580473+0000","last_peered":"2024-04-25T01:25:54.580473+0000","last_clean":"2024-04-25T01:25:54.580473+0000","last_became_active":"2024-04-25T01:25:52.356881+0000","last_became_peered":"2024-04-25T01:25:52.356881+0000","last_unstale":"2024-04-25T01:25:54.580473+0000","last_undegraded":"2024-04-25T01:25:54.580473+0000","last_fullsized":"2024-04-25T01:25:54.580473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581103+0000","last_change":"2024-04-25T01:25:53.368187+0000","last_active":"2024-04-25T01:25:54.581103+0000","last_peered":"2024-04-25T01:25:54.581103+0000","last_clean":"2024-04-25T01:25:54.581103+0000","last_became_active":"2024-04-25T01:25:53.367664+0000","last_became_peered":"2024-04-25T01:25:53.367664+0000","last_unstale":"2024-04-25T01:25:54.581103+0000","last_undegraded":"2024-04-25T01:25:54.581103+0000","last_fullsized":"2024-04-25T01:25:54.581103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368140+0000","last_change":"2024-04-25T01:25:52.351205+0000","last_active":"2024-04-25T01:25:54.368140+0000","last_peered":"2024-04-25T01:25:54.368140+0000","last_clean":"2024-04-25T01:25:54.368140+0000","last_became_active":"2024-04-25T01:25:52.350946+0000","last_became_peered":"2024-04-25T01:25:52.350946+0000","last_unstale":"2024-04-25T01:25:54.368140+0000","last_undegraded":"2024-04-25T01:25:54.368140+0000","last_fullsized":"2024-04-25T01:25:54.368140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368302+0000","last_change":"2024-04-25T01:25:53.364987+0000","last_active":"2024-04-25T01:25:54.368302+0000","last_peered":"2024-04-25T01:25:54.368302+0000","last_clean":"2024-04-25T01:25:54.368302+0000","last_became_active":"2024-04-25T01:25:53.364431+0000","last_became_peered":"2024-04-25T01:25:53.364431+0000","last_unstale":"2024-04-25T01:25:54.368302+0000","last_undegraded":"2024-04-25T01:25:54.368302+0000","last_fullsized":"2024-04-25T01:25:54.368302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579031+0000","last_change":"2024-04-25T01:25:52.347413+0000","last_active":"2024-04-25T01:25:54.579031+0000","last_peered":"2024-04-25T01:25:54.579031+0000","last_clean":"2024-04-25T01:25:54.579031+0000","last_became_active":"2024-04-25T01:25:52.347144+0000","last_became_peered":"2024-04-25T01:25:52.347144+0000","last_unstale":"2024-04-25T01:25:54.579031+0000","last_undegraded":"2024-04-25T01:25:54.579031+0000","last_fullsized":"2024-04-25T01:25:54.579031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580057+0000","last_change":"2024-04-25T01:25:53.366376+0000","last_active":"2024-04-25T01:25:54.580057+0000","last_peered":"2024-04-25T01:25:54.580057+0000","last_clean":"2024-04-25T01:25:54.580057+0000","last_became_active":"2024-04-25T01:25:53.365715+0000","last_became_peered":"2024-04-25T01:25:53.365715+0000","last_unstale":"2024-04-25T01:25:54.580057+0000","last_undegraded":"2024-04-25T01:25:54.580057+0000","last_fullsized":"2024-04-25T01:25:54.580057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580408+0000","last_change":"2024-04-25T01:25:52.350688+0000","last_active":"2024-04-25T01:25:54.580408+0000","last_peered":"2024-04-25T01:25:54.580408+0000","last_clean":"2024-04-25T01:25:54.580408+0000","last_became_active":"2024-04-25T01:25:52.350357+0000","last_became_peered":"2024-04-25T01:25:52.350357+0000","last_unstale":"2024-04-25T01:25:54.580408+0000","last_undegraded":"2024-04-25T01:25:54.580408+0000","last_fullsized":"2024-04-25T01:25:54.580408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581500+0000","last_change":"2024-04-25T01:25:53.365171+0000","last_active":"2024-04-25T01:25:54.581500+0000","last_peered":"2024-04-25T01:25:54.581500+0000","last_clean":"2024-04-25T01:25:54.581500+0000","last_became_active":"2024-04-25T01:25:53.364049+0000","last_became_peered":"2024-04-25T01:25:53.364049+0000","last_unstale":"2024-04-25T01:25:54.581500+0000","last_undegraded":"2024-04-25T01:25:54.581500+0000","last_fullsized":"2024-04-25T01:25:54.581500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579222+0000","last_change":"2024-04-25T01:25:52.345943+0000","last_active":"2024-04-25T01:25:54.579222+0000","last_peered":"2024-04-25T01:25:54.579222+0000","last_clean":"2024-04-25T01:25:54.579222+0000","last_became_active":"2024-04-25T01:25:52.345717+0000","last_became_peered":"2024-04-25T01:25:52.345717+0000","last_unstale":"2024-04-25T01:25:54.579222+0000","last_undegraded":"2024-04-25T01:25:54.579222+0000","last_fullsized":"2024-04-25T01:25:54.579222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581766+0000","last_change":"2024-04-25T01:25:53.370754+0000","last_active":"2024-04-25T01:25:54.581766+0000","last_peered":"2024-04-25T01:25:54.581766+0000","last_clean":"2024-04-25T01:25:54.581766+0000","last_became_active":"2024-04-25T01:25:53.370074+0000","last_became_peered":"2024-04-25T01:25:53.370074+0000","last_unstale":"2024-04-25T01:25:54.581766+0000","last_undegraded":"2024-04-25T01:25:54.581766+0000","last_fullsized":"2024-04-25T01:25:54.581766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366101+0000","last_change":"2024-04-25T01:25:52.355133+0000","last_active":"2024-04-25T01:25:54.366101+0000","last_peered":"2024-04-25T01:25:54.366101+0000","last_clean":"2024-04-25T01:25:54.366101+0000","last_became_active":"2024-04-25T01:25:52.354836+0000","last_became_peered":"2024-04-25T01:25:52.354836+0000","last_unstale":"2024-04-25T01:25:54.366101+0000","last_undegraded":"2024-04-25T01:25:54.366101+0000","last_fullsized":"2024-04-25T01:25:54.366101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:10.331704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580338+0000","last_change":"2024-04-25T01:25:53.371018+0000","last_active":"2024-04-25T01:25:54.580338+0000","last_peered":"2024-04-25T01:25:54.580338+0000","last_clean":"2024-04-25T01:25:54.580338+0000","last_became_active":"2024-04-25T01:25:53.370539+0000","last_became_peered":"2024-04-25T01:25:53.370539+0000","last_unstale":"2024-04-25T01:25:54.580338+0000","last_undegraded":"2024-04-25T01:25:54.580338+0000","last_fullsized":"2024-04-25T01:25:54.580338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579131+0000","last_change":"2024-04-25T01:25:52.352994+0000","last_active":"2024-04-25T01:25:54.579131+0000","last_peered":"2024-04-25T01:25:54.579131+0000","last_clean":"2024-04-25T01:25:54.579131+0000","last_became_active":"2024-04-25T01:25:52.352644+0000","last_became_peered":"2024-04-25T01:25:52.352644+0000","last_unstale":"2024-04-25T01:25:54.579131+0000","last_undegraded":"2024-04-25T01:25:54.579131+0000","last_fullsized":"2024-04-25T01:25:54.579131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581219+0000","last_change":"2024-04-25T01:25:53.375523+0000","last_active":"2024-04-25T01:25:54.581219+0000","last_peered":"2024-04-25T01:25:54.581219+0000","last_clean":"2024-04-25T01:25:54.581219+0000","last_became_active":"2024-04-25T01:25:53.375219+0000","last_became_peered":"2024-04-25T01:25:53.375219+0000","last_unstale":"2024-04-25T01:25:54.581219+0000","last_undegraded":"2024-04-25T01:25:54.581219+0000","last_fullsized":"2024-04-25T01:25:54.581219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580132+0000","last_change":"2024-04-25T01:25:41.719670+0000","last_active":"2024-04-25T01:25:54.580132+0000","last_peered":"2024-04-25T01:25:54.580132+0000","last_clean":"2024-04-25T01:25:54.580132+0000","last_became_active":"2024-04-25T01:25:41.718721+0000","last_became_peered":"2024-04-25T01:25:41.718721+0000","last_unstale":"2024-04-25T01:25:54.580132+0000","last_undegraded":"2024-04-25T01:25:54.580132+0000","last_fullsized":"2024-04-25T01:25:54.580132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_clean_scrub_stamp":"2024-04-25T01:25:40.688097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:27.456746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580582+0000","last_change":"2024-04-25T01:25:52.353155+0000","last_active":"2024-04-25T01:25:54.580582+0000","last_peered":"2024-04-25T01:25:54.580582+0000","last_clean":"2024-04-25T01:25:54.580582+0000","last_became_active":"2024-04-25T01:25:52.352826+0000","last_became_peered":"2024-04-25T01:25:52.352826+0000","last_unstale":"2024-04-25T01:25:54.580582+0000","last_undegraded":"2024-04-25T01:25:54.580582+0000","last_fullsized":"2024-04-25T01:25:54.580582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366734+0000","last_change":"2024-04-25T01:25:53.374719+0000","last_active":"2024-04-25T01:25:54.366734+0000","last_peered":"2024-04-25T01:25:54.366734+0000","last_clean":"2024-04-25T01:25:54.366734+0000","last_became_active":"2024-04-25T01:25:53.374425+0000","last_became_peered":"2024-04-25T01:25:53.374425+0000","last_unstale":"2024-04-25T01:25:54.366734+0000","last_undegraded":"2024-04-25T01:25:54.366734+0000","last_fullsized":"2024-04-25T01:25:54.366734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580106+0000","last_change":"2024-04-25T01:25:52.357683+0000","last_active":"2024-04-25T01:25:54.580106+0000","last_peered":"2024-04-25T01:25:54.580106+0000","last_clean":"2024-04-25T01:25:54.580106+0000","last_became_active":"2024-04-25T01:25:52.356924+0000","last_became_peered":"2024-04-25T01:25:52.356924+0000","last_unstale":"2024-04-25T01:25:54.580106+0000","last_undegraded":"2024-04-25T01:25:54.580106+0000","last_fullsized":"2024-04-25T01:25:54.580106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361256+0000","last_change":"2024-04-25T01:25:53.355950+0000","last_active":"2024-04-25T01:25:54.361256+0000","last_peered":"2024-04-25T01:25:54.361256+0000","last_clean":"2024-04-25T01:25:54.361256+0000","last_became_active":"2024-04-25T01:25:53.355648+0000","last_became_peered":"2024-04-25T01:25:53.355648+0000","last_unstale":"2024-04-25T01:25:54.361256+0000","last_undegraded":"2024-04-25T01:25:54.361256+0000","last_fullsized":"2024-04-25T01:25:54.361256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579084+0000","last_change":"2024-04-25T01:25:52.345321+0000","last_active":"2024-04-25T01:25:54.579084+0000","last_peered":"2024-04-25T01:25:54.579084+0000","last_clean":"2024-04-25T01:25:54.579084+0000","last_became_active":"2024-04-25T01:25:52.345061+0000","last_became_peered":"2024-04-25T01:25:52.345061+0000","last_unstale":"2024-04-25T01:25:54.579084+0000","last_undegraded":"2024-04-25T01:25:54.579084+0000","last_fullsized":"2024-04-25T01:25:54.579084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366196+0000","last_change":"2024-04-25T01:25:53.362922+0000","last_active":"2024-04-25T01:25:54.366196+0000","last_peered":"2024-04-25T01:25:54.366196+0000","last_clean":"2024-04-25T01:25:54.366196+0000","last_became_active":"2024-04-25T01:25:53.362533+0000","last_became_peered":"2024-04-25T01:25:53.362533+0000","last_unstale":"2024-04-25T01:25:54.366196+0000","last_undegraded":"2024-04-25T01:25:54.366196+0000","last_fullsized":"2024-04-25T01:25:54.366196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580540+0000","last_change":"2024-04-25T01:25:52.354499+0000","last_active":"2024-04-25T01:25:54.580540+0000","last_peered":"2024-04-25T01:25:54.580540+0000","last_clean":"2024-04-25T01:25:54.580540+0000","last_became_active":"2024-04-25T01:25:52.354193+0000","last_became_peered":"2024-04-25T01:25:52.354193+0000","last_unstale":"2024-04-25T01:25:54.580540+0000","last_undegraded":"2024-04-25T01:25:54.580540+0000","last_fullsized":"2024-04-25T01:25:54.580540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580437+0000","last_change":"2024-04-25T01:25:53.361696+0000","last_active":"2024-04-25T01:25:54.580437+0000","last_peered":"2024-04-25T01:25:54.580437+0000","last_clean":"2024-04-25T01:25:54.580437+0000","last_became_active":"2024-04-25T01:25:53.361156+0000","last_became_peered":"2024-04-25T01:25:53.361156+0000","last_unstale":"2024-04-25T01:25:54.580437+0000","last_undegraded":"2024-04-25T01:25:54.580437+0000","last_fullsized":"2024-04-25T01:25:54.580437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:27.472315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.365972+0000","last_change":"2024-04-25T01:25:52.350128+0000","last_active":"2024-04-25T01:25:54.365972+0000","last_peered":"2024-04-25T01:25:54.365972+0000","last_clean":"2024-04-25T01:25:54.365972+0000","last_became_active":"2024-04-25T01:25:52.349874+0000","last_became_peered":"2024-04-25T01:25:52.349874+0000","last_unstale":"2024-04-25T01:25:54.365972+0000","last_undegraded":"2024-04-25T01:25:54.365972+0000","last_fullsized":"2024-04-25T01:25:54.365972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581110+0000","last_change":"2024-04-25T01:25:53.368164+0000","last_active":"2024-04-25T01:25:54.581110+0000","last_peered":"2024-04-25T01:25:54.581110+0000","last_clean":"2024-04-25T01:25:54.581110+0000","last_became_active":"2024-04-25T01:25:53.367854+0000","last_became_peered":"2024-04-25T01:25:53.367854+0000","last_unstale":"2024-04-25T01:25:54.581110+0000","last_undegraded":"2024-04-25T01:25:54.581110+0000","last_fullsized":"2024-04-25T01:25:54.581110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581462+0000","last_change":"2024-04-25T01:25:52.356842+0000","last_active":"2024-04-25T01:25:54.581462+0000","last_peered":"2024-04-25T01:25:54.581462+0000","last_clean":"2024-04-25T01:25:54.581462+0000","last_became_active":"2024-04-25T01:25:52.356396+0000","last_became_peered":"2024-04-25T01:25:52.356396+0000","last_unstale":"2024-04-25T01:25:54.581462+0000","last_undegraded":"2024-04-25T01:25:54.581462+0000","last_fullsized":"2024-04-25T01:25:54.581462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366106+0000","last_change":"2024-04-25T01:25:53.366944+0000","last_active":"2024-04-25T01:25:54.366106+0000","last_peered":"2024-04-25T01:25:54.366106+0000","last_clean":"2024-04-25T01:25:54.366106+0000","last_became_active":"2024-04-25T01:25:53.366693+0000","last_became_peered":"2024-04-25T01:25:53.366693+0000","last_unstale":"2024-04-25T01:25:54.366106+0000","last_undegraded":"2024-04-25T01:25:54.366106+0000","last_fullsized":"2024-04-25T01:25:54.366106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.367714+0000","last_change":"2024-04-25T01:25:52.355395+0000","last_active":"2024-04-25T01:25:54.367714+0000","last_peered":"2024-04-25T01:25:54.367714+0000","last_clean":"2024-04-25T01:25:54.367714+0000","last_became_active":"2024-04-25T01:25:52.355137+0000","last_became_peered":"2024-04-25T01:25:52.355137+0000","last_unstale":"2024-04-25T01:25:54.367714+0000","last_undegraded":"2024-04-25T01:25:54.367714+0000","last_fullsized":"2024-04-25T01:25:54.367714+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:26.467968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366578+0000","last_change":"2024-04-25T01:25:53.364967+0000","last_active":"2024-04-25T01:25:54.366578+0000","last_peered":"2024-04-25T01:25:54.366578+0000","last_clean":"2024-04-25T01:25:54.366578+0000","last_became_active":"2024-04-25T01:25:53.364694+0000","last_became_peered":"2024-04-25T01:25:53.364694+0000","last_unstale":"2024-04-25T01:25:54.366578+0000","last_undegraded":"2024-04-25T01:25:54.366578+0000","last_fullsized":"2024-04-25T01:25:54.366578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368537+0000","last_change":"2024-04-25T01:25:52.353269+0000","last_active":"2024-04-25T01:25:54.368537+0000","last_peered":"2024-04-25T01:25:54.368537+0000","last_clean":"2024-04-25T01:25:54.368537+0000","last_became_active":"2024-04-25T01:25:52.352991+0000","last_became_peered":"2024-04-25T01:25:52.352991+0000","last_unstale":"2024-04-25T01:25:54.368537+0000","last_undegraded":"2024-04-25T01:25:54.368537+0000","last_fullsized":"2024-04-25T01:25:54.368537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361579+0000","last_change":"2024-04-25T01:25:53.370111+0000","last_active":"2024-04-25T01:25:54.361579+0000","last_peered":"2024-04-25T01:25:54.361579+0000","last_clean":"2024-04-25T01:25:54.361579+0000","last_became_active":"2024-04-25T01:25:53.369837+0000","last_became_peered":"2024-04-25T01:25:53.369837+0000","last_unstale":"2024-04-25T01:25:54.361579+0000","last_undegraded":"2024-04-25T01:25:54.361579+0000","last_fullsized":"2024-04-25T01:25:54.361579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580024+0000","last_change":"2024-04-25T01:25:52.353836+0000","last_active":"2024-04-25T01:25:54.580024+0000","last_peered":"2024-04-25T01:25:54.580024+0000","last_clean":"2024-04-25T01:25:54.580024+0000","last_became_active":"2024-04-25T01:25:52.353552+0000","last_became_peered":"2024-04-25T01:25:52.353552+0000","last_unstale":"2024-04-25T01:25:54.580024+0000","last_undegraded":"2024-04-25T01:25:54.580024+0000","last_fullsized":"2024-04-25T01:25:54.580024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580513+0000","last_change":"2024-04-25T01:25:53.361653+0000","last_active":"2024-04-25T01:25:54.580513+0000","last_peered":"2024-04-25T01:25:54.580513+0000","last_clean":"2024-04-25T01:25:54.580513+0000","last_became_active":"2024-04-25T01:25:53.361265+0000","last_became_peered":"2024-04-25T01:25:53.361265+0000","last_unstale":"2024-04-25T01:25:54.580513+0000","last_undegraded":"2024-04-25T01:25:54.580513+0000","last_fullsized":"2024-04-25T01:25:54.580513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366249+0000","last_change":"2024-04-25T01:25:52.353926+0000","last_active":"2024-04-25T01:25:54.366249+0000","last_peered":"2024-04-25T01:25:54.366249+0000","last_clean":"2024-04-25T01:25:54.366249+0000","last_became_active":"2024-04-25T01:25:52.353656+0000","last_became_peered":"2024-04-25T01:25:52.353656+0000","last_unstale":"2024-04-25T01:25:54.366249+0000","last_undegraded":"2024-04-25T01:25:54.366249+0000","last_fullsized":"2024-04-25T01:25:54.366249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366408+0000","last_change":"2024-04-25T01:25:53.361790+0000","last_active":"2024-04-25T01:25:54.366408+0000","last_peered":"2024-04-25T01:25:54.366408+0000","last_clean":"2024-04-25T01:25:54.366408+0000","last_became_active":"2024-04-25T01:25:53.361542+0000","last_became_peered":"2024-04-25T01:25:53.361542+0000","last_unstale":"2024-04-25T01:25:54.366408+0000","last_undegraded":"2024-04-25T01:25:54.366408+0000","last_fullsized":"2024-04-25T01:25:54.366408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366210+0000","last_change":"2024-04-25T01:25:52.348841+0000","last_active":"2024-04-25T01:25:54.366210+0000","last_peered":"2024-04-25T01:25:54.366210+0000","last_clean":"2024-04-25T01:25:54.366210+0000","last_became_active":"2024-04-25T01:25:52.348550+0000","last_became_peered":"2024-04-25T01:25:52.348550+0000","last_unstale":"2024-04-25T01:25:54.366210+0000","last_undegraded":"2024-04-25T01:25:54.366210+0000","last_fullsized":"2024-04-25T01:25:54.366210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366707+0000","last_change":"2024-04-25T01:25:53.361074+0000","last_active":"2024-04-25T01:25:54.366707+0000","last_peered":"2024-04-25T01:25:54.366707+0000","last_clean":"2024-04-25T01:25:54.366707+0000","last_became_active":"2024-04-25T01:25:53.360802+0000","last_became_peered":"2024-04-25T01:25:53.360802+0000","last_unstale":"2024-04-25T01:25:54.366707+0000","last_undegraded":"2024-04-25T01:25:54.366707+0000","last_fullsized":"2024-04-25T01:25:54.366707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581335+0000","last_change":"2024-04-25T01:25:52.358616+0000","last_active":"2024-04-25T01:25:54.581335+0000","last_peered":"2024-04-25T01:25:54.581335+0000","last_clean":"2024-04-25T01:25:54.581335+0000","last_became_active":"2024-04-25T01:25:52.358318+0000","last_became_peered":"2024-04-25T01:25:52.358318+0000","last_unstale":"2024-04-25T01:25:54.581335+0000","last_undegraded":"2024-04-25T01:25:54.581335+0000","last_fullsized":"2024-04-25T01:25:54.581335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368226+0000","last_change":"2024-04-25T01:25:53.372385+0000","last_active":"2024-04-25T01:25:54.368226+0000","last_peered":"2024-04-25T01:25:54.368226+0000","last_clean":"2024-04-25T01:25:54.368226+0000","last_became_active":"2024-04-25T01:25:53.372064+0000","last_became_peered":"2024-04-25T01:25:53.372064+0000","last_unstale":"2024-04-25T01:25:54.368226+0000","last_undegraded":"2024-04-25T01:25:54.368226+0000","last_fullsized":"2024-04-25T01:25:54.368226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581025+0000","last_change":"2024-04-25T01:25:53.368739+0000","last_active":"2024-04-25T01:25:54.581025+0000","last_peered":"2024-04-25T01:25:54.581025+0000","last_clean":"2024-04-25T01:25:54.581025+0000","last_became_active":"2024-04-25T01:25:53.368470+0000","last_became_peered":"2024-04-25T01:25:53.368470+0000","last_unstale":"2024-04-25T01:25:54.581025+0000","last_undegraded":"2024-04-25T01:25:54.581025+0000","last_fullsized":"2024-04-25T01:25:54.581025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:29.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581195+0000","last_change":"2024-04-25T01:25:52.359139+0000","last_active":"2024-04-25T01:25:54.581195+0000","last_peered":"2024-04-25T01:25:54.581195+0000","last_clean":"2024-04-25T01:25:54.581195+0000","last_became_active":"2024-04-25T01:25:52.358841+0000","last_became_peered":"2024-04-25T01:25:52.358841+0000","last_unstale":"2024-04-25T01:25:54.581195+0000","last_undegraded":"2024-04-25T01:25:54.581195+0000","last_fullsized":"2024-04-25T01:25:54.581195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580118+0000","last_change":"2024-04-25T01:25:53.370061+0000","last_active":"2024-04-25T01:25:54.580118+0000","last_peered":"2024-04-25T01:25:54.580118+0000","last_clean":"2024-04-25T01:25:54.580118+0000","last_became_active":"2024-04-25T01:25:53.369721+0000","last_became_peered":"2024-04-25T01:25:53.369721+0000","last_unstale":"2024-04-25T01:25:54.580118+0000","last_undegraded":"2024-04-25T01:25:54.580118+0000","last_fullsized":"2024-04-25T01:25:54.580118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581355+0000","last_change":"2024-04-25T01:25:52.356403+0000","last_active":"2024-04-25T01:25:54.581355+0000","last_peered":"2024-04-25T01:25:54.581355+0000","last_clean":"2024-04-25T01:25:54.581355+0000","last_became_active":"2024-04-25T01:25:52.355822+0000","last_became_peered":"2024-04-25T01:25:52.355822+0000","last_unstale":"2024-04-25T01:25:54.581355+0000","last_undegraded":"2024-04-25T01:25:54.581355+0000","last_fullsized":"2024-04-25T01:25:54.581355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361454+0000","last_change":"2024-04-25T01:25:53.374150+0000","last_active":"2024-04-25T01:25:54.361454+0000","last_peered":"2024-04-25T01:25:54.361454+0000","last_clean":"2024-04-25T01:25:54.361454+0000","last_became_active":"2024-04-25T01:25:53.373755+0000","last_became_peered":"2024-04-25T01:25:53.373755+0000","last_unstale":"2024-04-25T01:25:54.361454+0000","last_undegraded":"2024-04-25T01:25:54.361454+0000","last_fullsized":"2024-04-25T01:25:54.361454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366968+0000","last_change":"2024-04-25T01:25:52.349185+0000","last_active":"2024-04-25T01:25:54.366968+0000","last_peered":"2024-04-25T01:25:54.366968+0000","last_clean":"2024-04-25T01:25:54.366968+0000","last_became_active":"2024-04-25T01:25:52.348931+0000","last_became_peered":"2024-04-25T01:25:52.348931+0000","last_unstale":"2024-04-25T01:25:54.366968+0000","last_undegraded":"2024-04-25T01:25:54.366968+0000","last_fullsized":"2024-04-25T01:25:54.366968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366936+0000","last_change":"2024-04-25T01:25:53.364257+0000","last_active":"2024-04-25T01:25:54.366936+0000","last_peered":"2024-04-25T01:25:54.366936+0000","last_clean":"2024-04-25T01:25:54.366936+0000","last_became_active":"2024-04-25T01:25:53.363409+0000","last_became_peered":"2024-04-25T01:25:53.363409+0000","last_unstale":"2024-04-25T01:25:54.366936+0000","last_undegraded":"2024-04-25T01:25:54.366936+0000","last_fullsized":"2024-04-25T01:25:54.366936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361657+0000","last_change":"2024-04-25T01:25:52.353978+0000","last_active":"2024-04-25T01:25:54.361657+0000","last_peered":"2024-04-25T01:25:54.361657+0000","last_clean":"2024-04-25T01:25:54.361657+0000","last_became_active":"2024-04-25T01:25:52.353649+0000","last_became_peered":"2024-04-25T01:25:52.353649+0000","last_unstale":"2024-04-25T01:25:54.361657+0000","last_undegraded":"2024-04-25T01:25:54.361657+0000","last_fullsized":"2024-04-25T01:25:54.361657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":70944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":70980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:25:57.292 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:25:57.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.444+0000 7fba0b2ee640 1 Processor -- start 2024-04-25T01:25:57.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.445+0000 7fba0b2ee640 1 -- start start 2024-04-25T01:25:57.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.445+0000 7fba0b2ee640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fba04071520 0x7fba04154b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:57.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.445+0000 7fba0b2ee640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba04154120 con 0x7fba040c7c80 2024-04-25T01:25:57.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.445+0000 7fba0b2ee640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba04154400 con 0x7fba040c7800 2024-04-25T01:25:57.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.445+0000 7fba0b2ee640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fba040c7000 con 0x7fba04071520 2024-04-25T01:25:57.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.445+0000 7fba0908a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fba04071520 0x7fba04154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:57.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.445+0000 7fba0988b640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fba040c7c80 0x7fba04153660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50718/0 (socket says 172.21.15.112:50718) 2024-04-25T01:25:57.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.445+0000 7fba0988b640 1 -- 172.21.15.112:0/183918761 learned_addr learned my addr 172.21.15.112:0/183918761 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:25:57.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.446+0000 7fba0908a640 1 -- 172.21.15.112:0/183918761 >> v1:172.21.15.187:6789/0 conn(0x7fba040c7800 legacy=0x7fba040c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:57.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.446+0000 7fba0908a640 1 -- 172.21.15.112:0/183918761 >> v1:172.21.15.112:6789/0 conn(0x7fba040c7c80 legacy=0x7fba04153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:57.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.446+0000 7fb9fbfff640 1 -- 172.21.15.112:0/183918761 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2494031961 0 0) 0x7fba04154120 con 0x7fba040c7c80 2024-04-25T01:25:57.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.446+0000 7fba0908a640 1 -- 172.21.15.112:0/183918761 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba040c72e0 con 0x7fba04071520 2024-04-25T01:25:57.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.446+0000 7fba0908a640 1 --2- 172.21.15.112:0/183918761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba04071520 0x7fba04154b50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40097a0 tx=0x7fb9f402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e394a0f2d756592c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:57.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.446+0000 7fb9fbfff640 1 -- 172.21.15.112:0/183918761 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4037070 con 0x7fba04071520 2024-04-25T01:25:57.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.446+0000 7fb9fbfff640 1 -- 172.21.15.112:0/183918761 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9f402adc0 con 0x7fba04071520 2024-04-25T01:25:57.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.447+0000 7fb9fbfff640 1 -- 172.21.15.112:0/183918761 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4032660 con 0x7fba04071520 2024-04-25T01:25:57.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.447+0000 7fba0b2ee640 1 -- 172.21.15.112:0/183918761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba04071520 msgr2=0x7fba04154b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:57.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.447+0000 7fba0b2ee640 1 --2- 172.21.15.112:0/183918761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba04071520 0x7fba04154b50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40097a0 tx=0x7fb9f402a0b0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.447+0000 7fba0b2ee640 1 -- 172.21.15.112:0/183918761 shutdown_connections 2024-04-25T01:25:57.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.447+0000 7fba0b2ee640 1 --2- 172.21.15.112:0/183918761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba04071520 0x7fba04154b50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.447+0000 7fba0b2ee640 1 -- 172.21.15.112:0/183918761 >> 172.21.15.112:0/183918761 conn(0x7fba04074a00 msgr2=0x7fba040c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:57.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.447+0000 7fba0b2ee640 1 -- 172.21.15.112:0/183918761 shutdown_connections 2024-04-25T01:25:57.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.447+0000 7fba0b2ee640 1 -- 172.21.15.112:0/183918761 wait complete. 2024-04-25T01:25:57.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.448+0000 7fba0b2ee640 1 Processor -- start 2024-04-25T01:25:57.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.448+0000 7fba0b2ee640 1 -- start start 2024-04-25T01:25:57.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fba0b2ee640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba04071520 0x7fba040c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:57.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fba0b2ee640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba040c7800 0x7fba041e93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:57.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fba0b2ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba040c7c80 0x7fba041e0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:57.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fba0b2ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fba04153d70 con 0x7fba040c7c80 2024-04-25T01:25:57.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fba0b2ee640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fba040ad930 con 0x7fba040c7800 2024-04-25T01:25:57.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fba0b2ee640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fba0414a110 con 0x7fba04071520 2024-04-25T01:25:57.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fba0988b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba040c7c80 0x7fba041e0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:57.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fba0988b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba040c7c80 0x7fba041e0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41312/0 (socket says 172.21.15.112:41312) 2024-04-25T01:25:57.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fba0988b640 1 -- 172.21.15.112:0/1446269620 learned_addr learned my addr 172.21.15.112:0/1446269620 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:57.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fba0908a640 1 --2- 172.21.15.112:0/1446269620 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba04071520 0x7fba040c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:57.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fba08889640 1 --2- 172.21.15.112:0/1446269620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba040c7800 0x7fba041e93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:57.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fba0988b640 1 -- 172.21.15.112:0/1446269620 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba04071520 msgr2=0x7fba040c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:57.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fba0988b640 1 --2- 172.21.15.112:0/1446269620 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba04071520 0x7fba040c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fba0988b640 1 -- 172.21.15.112:0/1446269620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba040c7800 msgr2=0x7fba041e93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:57.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fba0988b640 1 --2- 172.21.15.112:0/1446269620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba040c7800 0x7fba041e93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fba0988b640 1 -- 172.21.15.112:0/1446269620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0414a370 con 0x7fba040c7c80 2024-04-25T01:25:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fba0988b640 1 --2- 172.21.15.112:0/1446269620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba040c7c80 0x7fba041e0020 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00c7a0 tx=0x7fb9fc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fb9f9ffb640 1 -- 172.21.15.112:0/1446269620 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc018070 con 0x7fba040c7c80 2024-04-25T01:25:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fb9f9ffb640 1 -- 172.21.15.112:0/1446269620 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9fc00a640 con 0x7fba040c7c80 2024-04-25T01:25:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fba0b2ee640 1 -- 172.21.15.112:0/1446269620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba041ed430 con 0x7fba040c7c80 2024-04-25T01:25:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.449+0000 7fb9f9ffb640 1 -- 172.21.15.112:0/1446269620 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc00aab0 con 0x7fba040c7c80 2024-04-25T01:25:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.450+0000 7fba0b2ee640 1 -- 172.21.15.112:0/1446269620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba041ed710 con 0x7fba040c7c80 2024-04-25T01:25:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.451+0000 7fba0b2ee640 1 -- 172.21.15.112:0/1446269620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba040682a0 con 0x7fba040c7c80 2024-04-25T01:25:57.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.451+0000 7fb9f9ffb640 1 -- 172.21.15.112:0/1446269620 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb9fc01b070 con 0x7fba040c7c80 2024-04-25T01:25:57.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.451+0000 7fb9f9ffb640 1 --2- 172.21.15.112:0/1446269620 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb9e806be30 0x7fb9e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:57.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.451+0000 7fb9f9ffb640 1 -- 172.21.15.112:0/1446269620 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7fb9fc08a400 con 0x7fba040c7c80 2024-04-25T01:25:57.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.451+0000 7fba0908a640 1 --2- 172.21.15.112:0/1446269620 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb9e806be30 0x7fb9e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:57.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.452+0000 7fba0908a640 1 --2- 172.21.15.112:0/1446269620 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb9e806be30 0x7fb9e806e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4034000 tx=0x7fb9f402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:57.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.455+0000 7fb9f9ffb640 1 -- 172.21.15.112:0/1446269620 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb9fc058e50 con 0x7fba040c7c80 2024-04-25T01:25:57.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.625+0000 7fba0b2ee640 1 -- 172.21.15.112:0/1446269620 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fba0414a650 con 0x7fb9e806be30 2024-04-25T01:25:57.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.631+0000 7fb9f9ffb640 1 -- 172.21.15.112:0/1446269620 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7fba0414a650 con 0x7fb9e806be30 2024-04-25T01:25:57.613 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:25:57.615 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T01:25:57.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.634+0000 7fba0b2ee640 1 -- 172.21.15.112:0/1446269620 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb9e806be30 msgr2=0x7fb9e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:57.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.634+0000 7fba0b2ee640 1 --2- 172.21.15.112:0/1446269620 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb9e806be30 0x7fb9e806e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4034000 tx=0x7fb9f402e000 comp rx=0 tx=0).stop 2024-04-25T01:25:57.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.634+0000 7fba0b2ee640 1 -- 172.21.15.112:0/1446269620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba040c7c80 msgr2=0x7fba041e0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:57.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.634+0000 7fba0b2ee640 1 --2- 172.21.15.112:0/1446269620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba040c7c80 0x7fba041e0020 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00c7a0 tx=0x7fb9fc00cc60 comp rx=0 tx=0).stop 2024-04-25T01:25:57.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.634+0000 7fba0b2ee640 1 -- 172.21.15.112:0/1446269620 shutdown_connections 2024-04-25T01:25:57.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.634+0000 7fba0b2ee640 1 --2- 172.21.15.112:0/1446269620 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb9e806be30 0x7fb9e806e2f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.634+0000 7fba0b2ee640 1 --2- 172.21.15.112:0/1446269620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba040c7c80 0x7fba041e0020 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.634+0000 7fba0b2ee640 1 --2- 172.21.15.112:0/1446269620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba040c7800 0x7fba041e93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.634+0000 7fba0b2ee640 1 --2- 172.21.15.112:0/1446269620 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba04071520 0x7fba040c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.634+0000 7fba0b2ee640 1 -- 172.21.15.112:0/1446269620 >> 172.21.15.112:0/1446269620 conn(0x7fba04074a00 msgr2=0x7fba041563d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:57.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.634+0000 7fba0b2ee640 1 -- 172.21.15.112:0/1446269620 shutdown_connections 2024-04-25T01:25:57.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.634+0000 7fba0b2ee640 1 -- 172.21.15.112:0/1446269620 wait complete. 2024-04-25T01:25:57.625 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-25T01:25:56.649037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219016,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755704,"statfs":{"total":773094113280,"available":772869840896,"internally_reserved":0,"allocated":2891776,"data_stored":1752388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.314757"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581611+0000","last_change":"2024-04-25T01:25:53.367762+0000","last_active":"2024-04-25T01:25:54.581611+0000","last_peered":"2024-04-25T01:25:54.581611+0000","last_clean":"2024-04-25T01:25:54.581611+0000","last_became_active":"2024-04-25T01:25:53.367383+0000","last_became_peered":"2024-04-25T01:25:53.367383+0000","last_unstale":"2024-04-25T01:25:54.581611+0000","last_undegraded":"2024-04-25T01:25:54.581611+0000","last_fullsized":"2024-04-25T01:25:54.581611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:05.589744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366023+0000","last_change":"2024-04-25T01:25:53.365057+0000","last_active":"2024-04-25T01:25:54.366023+0000","last_peered":"2024-04-25T01:25:54.366023+0000","last_clean":"2024-04-25T01:25:54.366023+0000","last_became_active":"2024-04-25T01:25:53.364793+0000","last_became_peered":"2024-04-25T01:25:53.364793+0000","last_unstale":"2024-04-25T01:25:54.366023+0000","last_undegraded":"2024-04-25T01:25:54.366023+0000","last_fullsized":"2024-04-25T01:25:54.366023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361426+0000","last_change":"2024-04-25T01:25:53.363829+0000","last_active":"2024-04-25T01:25:54.361426+0000","last_peered":"2024-04-25T01:25:54.361426+0000","last_clean":"2024-04-25T01:25:54.361426+0000","last_became_active":"2024-04-25T01:25:53.363525+0000","last_became_peered":"2024-04-25T01:25:53.363525+0000","last_unstale":"2024-04-25T01:25:54.361426+0000","last_undegraded":"2024-04-25T01:25:54.361426+0000","last_fullsized":"2024-04-25T01:25:54.361426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579956+0000","last_change":"2024-04-25T01:25:53.371513+0000","last_active":"2024-04-25T01:25:54.579956+0000","last_peered":"2024-04-25T01:25:54.579956+0000","last_clean":"2024-04-25T01:25:54.579956+0000","last_became_active":"2024-04-25T01:25:53.370421+0000","last_became_peered":"2024-04-25T01:25:53.370421+0000","last_unstale":"2024-04-25T01:25:54.579956+0000","last_undegraded":"2024-04-25T01:25:54.579956+0000","last_fullsized":"2024-04-25T01:25:54.579956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581049+0000","last_change":"2024-04-25T01:25:53.367953+0000","last_active":"2024-04-25T01:25:54.581049+0000","last_peered":"2024-04-25T01:25:54.581049+0000","last_clean":"2024-04-25T01:25:54.581049+0000","last_became_active":"2024-04-25T01:25:53.367358+0000","last_became_peered":"2024-04-25T01:25:53.367358+0000","last_unstale":"2024-04-25T01:25:54.581049+0000","last_undegraded":"2024-04-25T01:25:54.581049+0000","last_fullsized":"2024-04-25T01:25:54.581049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579967+0000","last_change":"2024-04-25T01:25:53.375324+0000","last_active":"2024-04-25T01:25:54.579967+0000","last_peered":"2024-04-25T01:25:54.579967+0000","last_clean":"2024-04-25T01:25:54.579967+0000","last_became_active":"2024-04-25T01:25:53.374392+0000","last_became_peered":"2024-04-25T01:25:53.374392+0000","last_unstale":"2024-04-25T01:25:54.579967+0000","last_undegraded":"2024-04-25T01:25:54.579967+0000","last_fullsized":"2024-04-25T01:25:54.579967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579900+0000","last_change":"2024-04-25T01:25:53.364698+0000","last_active":"2024-04-25T01:25:54.579900+0000","last_peered":"2024-04-25T01:25:54.579900+0000","last_clean":"2024-04-25T01:25:54.579900+0000","last_became_active":"2024-04-25T01:25:53.364435+0000","last_became_peered":"2024-04-25T01:25:53.364435+0000","last_unstale":"2024-04-25T01:25:54.579900+0000","last_undegraded":"2024-04-25T01:25:54.579900+0000","last_fullsized":"2024-04-25T01:25:54.579900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366265+0000","last_change":"2024-04-25T01:25:53.374061+0000","last_active":"2024-04-25T01:25:54.366265+0000","last_peered":"2024-04-25T01:25:54.366265+0000","last_clean":"2024-04-25T01:25:54.366265+0000","last_became_active":"2024-04-25T01:25:53.373848+0000","last_became_peered":"2024-04-25T01:25:53.373848+0000","last_unstale":"2024-04-25T01:25:54.366265+0000","last_undegraded":"2024-04-25T01:25:54.366265+0000","last_fullsized":"2024-04-25T01:25:54.366265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581623+0000","last_change":"2024-04-25T01:25:53.376081+0000","last_active":"2024-04-25T01:25:54.581623+0000","last_peered":"2024-04-25T01:25:54.581623+0000","last_clean":"2024-04-25T01:25:54.581623+0000","last_became_active":"2024-04-25T01:25:53.375814+0000","last_became_peered":"2024-04-25T01:25:53.375814+0000","last_unstale":"2024-04-25T01:25:54.581623+0000","last_undegraded":"2024-04-25T01:25:54.581623+0000","last_fullsized":"2024-04-25T01:25:54.581623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:29.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368588+0000","last_change":"2024-04-25T01:25:53.372842+0000","last_active":"2024-04-25T01:25:54.368588+0000","last_peered":"2024-04-25T01:25:54.368588+0000","last_clean":"2024-04-25T01:25:54.368588+0000","last_became_active":"2024-04-25T01:25:53.372580+0000","last_became_peered":"2024-04-25T01:25:53.372580+0000","last_unstale":"2024-04-25T01:25:54.368588+0000","last_undegraded":"2024-04-25T01:25:54.368588+0000","last_fullsized":"2024-04-25T01:25:54.368588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368507+0000","last_change":"2024-04-25T01:25:53.371647+0000","last_active":"2024-04-25T01:25:54.368507+0000","last_peered":"2024-04-25T01:25:54.368507+0000","last_clean":"2024-04-25T01:25:54.368507+0000","last_became_active":"2024-04-25T01:25:53.371353+0000","last_became_peered":"2024-04-25T01:25:53.371353+0000","last_unstale":"2024-04-25T01:25:54.368507+0000","last_undegraded":"2024-04-25T01:25:54.368507+0000","last_fullsized":"2024-04-25T01:25:54.368507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580179+0000","last_change":"2024-04-25T01:25:53.364457+0000","last_active":"2024-04-25T01:25:54.580179+0000","last_peered":"2024-04-25T01:25:54.580179+0000","last_clean":"2024-04-25T01:25:54.580179+0000","last_became_active":"2024-04-25T01:25:53.364103+0000","last_became_peered":"2024-04-25T01:25:53.364103+0000","last_unstale":"2024-04-25T01:25:54.580179+0000","last_undegraded":"2024-04-25T01:25:54.580179+0000","last_fullsized":"2024-04-25T01:25:54.580179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:11.336752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366544+0000","last_change":"2024-04-25T01:25:53.363512+0000","last_active":"2024-04-25T01:25:54.366544+0000","last_peered":"2024-04-25T01:25:54.366544+0000","last_clean":"2024-04-25T01:25:54.366544+0000","last_became_active":"2024-04-25T01:25:53.363246+0000","last_became_peered":"2024-04-25T01:25:53.363246+0000","last_unstale":"2024-04-25T01:25:54.366544+0000","last_undegraded":"2024-04-25T01:25:54.366544+0000","last_fullsized":"2024-04-25T01:25:54.366544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361273+0000","last_change":"2024-04-25T01:25:53.370996+0000","last_active":"2024-04-25T01:25:54.361273+0000","last_peered":"2024-04-25T01:25:54.361273+0000","last_clean":"2024-04-25T01:25:54.361273+0000","last_became_active":"2024-04-25T01:25:53.370479+0000","last_became_peered":"2024-04-25T01:25:53.370479+0000","last_unstale":"2024-04-25T01:25:54.361273+0000","last_undegraded":"2024-04-25T01:25:54.361273+0000","last_fullsized":"2024-04-25T01:25:54.361273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:57.636067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366008+0000","last_change":"2024-04-25T01:25:53.371602+0000","last_active":"2024-04-25T01:25:54.366008+0000","last_peered":"2024-04-25T01:25:54.366008+0000","last_clean":"2024-04-25T01:25:54.366008+0000","last_became_active":"2024-04-25T01:25:53.371362+0000","last_became_peered":"2024-04-25T01:25:53.371362+0000","last_unstale":"2024-04-25T01:25:54.366008+0000","last_undegraded":"2024-04-25T01:25:54.366008+0000","last_fullsized":"2024-04-25T01:25:54.366008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579564+0000","last_change":"2024-04-25T01:25:53.374186+0000","last_active":"2024-04-25T01:25:54.579564+0000","last_peered":"2024-04-25T01:25:54.579564+0000","last_clean":"2024-04-25T01:25:54.579564+0000","last_became_active":"2024-04-25T01:25:53.373766+0000","last_became_peered":"2024-04-25T01:25:53.373766+0000","last_unstale":"2024-04-25T01:25:54.579564+0000","last_undegraded":"2024-04-25T01:25:54.579564+0000","last_fullsized":"2024-04-25T01:25:54.579564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580993+0000","last_change":"2024-04-25T01:25:53.372048+0000","last_active":"2024-04-25T01:25:54.580993+0000","last_peered":"2024-04-25T01:25:54.580993+0000","last_clean":"2024-04-25T01:25:54.580993+0000","last_became_active":"2024-04-25T01:25:53.371767+0000","last_became_peered":"2024-04-25T01:25:53.371767+0000","last_unstale":"2024-04-25T01:25:54.580993+0000","last_undegraded":"2024-04-25T01:25:54.580993+0000","last_fullsized":"2024-04-25T01:25:54.580993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:05.589744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579175+0000","last_change":"2024-04-25T01:25:53.370666+0000","last_active":"2024-04-25T01:25:54.579175+0000","last_peered":"2024-04-25T01:25:54.579175+0000","last_clean":"2024-04-25T01:25:54.579175+0000","last_became_active":"2024-04-25T01:25:53.369733+0000","last_became_peered":"2024-04-25T01:25:53.369733+0000","last_unstale":"2024-04-25T01:25:54.579175+0000","last_undegraded":"2024-04-25T01:25:54.579175+0000","last_fullsized":"2024-04-25T01:25:54.579175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581475+0000","last_change":"2024-04-25T01:25:53.369585+0000","last_active":"2024-04-25T01:25:54.581475+0000","last_peered":"2024-04-25T01:25:54.581475+0000","last_clean":"2024-04-25T01:25:54.581475+0000","last_became_active":"2024-04-25T01:25:53.368499+0000","last_became_peered":"2024-04-25T01:25:53.368499+0000","last_unstale":"2024-04-25T01:25:54.581475+0000","last_undegraded":"2024-04-25T01:25:54.581475+0000","last_fullsized":"2024-04-25T01:25:54.581475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368327+0000","last_change":"2024-04-25T01:25:53.370314+0000","last_active":"2024-04-25T01:25:54.368327+0000","last_peered":"2024-04-25T01:25:54.368327+0000","last_clean":"2024-04-25T01:25:54.368327+0000","last_became_active":"2024-04-25T01:25:53.370049+0000","last_became_peered":"2024-04-25T01:25:53.370049+0000","last_unstale":"2024-04-25T01:25:54.368327+0000","last_undegraded":"2024-04-25T01:25:54.368327+0000","last_fullsized":"2024-04-25T01:25:54.368327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361655+0000","last_change":"2024-04-25T01:25:53.376415+0000","last_active":"2024-04-25T01:25:54.361655+0000","last_peered":"2024-04-25T01:25:54.361655+0000","last_clean":"2024-04-25T01:25:54.361655+0000","last_became_active":"2024-04-25T01:25:53.376104+0000","last_became_peered":"2024-04-25T01:25:53.376104+0000","last_unstale":"2024-04-25T01:25:54.361655+0000","last_undegraded":"2024-04-25T01:25:54.361655+0000","last_fullsized":"2024-04-25T01:25:54.361655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580264+0000","last_change":"2024-04-25T01:25:53.371240+0000","last_active":"2024-04-25T01:25:54.580264+0000","last_peered":"2024-04-25T01:25:54.580264+0000","last_clean":"2024-04-25T01:25:54.580264+0000","last_became_active":"2024-04-25T01:25:53.370164+0000","last_became_peered":"2024-04-25T01:25:53.370164+0000","last_unstale":"2024-04-25T01:25:54.580264+0000","last_undegraded":"2024-04-25T01:25:54.580264+0000","last_fullsized":"2024-04-25T01:25:54.580264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366205+0000","last_change":"2024-04-25T01:25:53.355667+0000","last_active":"2024-04-25T01:25:54.366205+0000","last_peered":"2024-04-25T01:25:54.366205+0000","last_clean":"2024-04-25T01:25:54.366205+0000","last_became_active":"2024-04-25T01:25:53.355417+0000","last_became_peered":"2024-04-25T01:25:53.355417+0000","last_unstale":"2024-04-25T01:25:54.366205+0000","last_undegraded":"2024-04-25T01:25:54.366205+0000","last_fullsized":"2024-04-25T01:25:54.366205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579870+0000","last_change":"2024-04-25T01:25:53.365391+0000","last_active":"2024-04-25T01:25:54.579870+0000","last_peered":"2024-04-25T01:25:54.579870+0000","last_clean":"2024-04-25T01:25:54.579870+0000","last_became_active":"2024-04-25T01:25:53.364829+0000","last_became_peered":"2024-04-25T01:25:53.364829+0000","last_unstale":"2024-04-25T01:25:54.579870+0000","last_undegraded":"2024-04-25T01:25:54.579870+0000","last_fullsized":"2024-04-25T01:25:54.579870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366392+0000","last_change":"2024-04-25T01:25:53.365509+0000","last_active":"2024-04-25T01:25:54.366392+0000","last_peered":"2024-04-25T01:25:54.366392+0000","last_clean":"2024-04-25T01:25:54.366392+0000","last_became_active":"2024-04-25T01:25:53.365245+0000","last_became_peered":"2024-04-25T01:25:53.365245+0000","last_unstale":"2024-04-25T01:25:54.366392+0000","last_undegraded":"2024-04-25T01:25:54.366392+0000","last_fullsized":"2024-04-25T01:25:54.366392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368629+0000","last_change":"2024-04-25T01:25:53.372638+0000","last_active":"2024-04-25T01:25:54.368629+0000","last_peered":"2024-04-25T01:25:54.368629+0000","last_clean":"2024-04-25T01:25:54.368629+0000","last_became_active":"2024-04-25T01:25:53.372001+0000","last_became_peered":"2024-04-25T01:25:53.372001+0000","last_unstale":"2024-04-25T01:25:54.368629+0000","last_undegraded":"2024-04-25T01:25:54.368629+0000","last_fullsized":"2024-04-25T01:25:54.368629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.365990+0000","last_change":"2024-04-25T01:25:53.368413+0000","last_active":"2024-04-25T01:25:54.365990+0000","last_peered":"2024-04-25T01:25:54.365990+0000","last_clean":"2024-04-25T01:25:54.365990+0000","last_became_active":"2024-04-25T01:25:53.368176+0000","last_became_peered":"2024-04-25T01:25:53.368176+0000","last_unstale":"2024-04-25T01:25:54.365990+0000","last_undegraded":"2024-04-25T01:25:54.365990+0000","last_fullsized":"2024-04-25T01:25:54.365990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366985+0000","last_change":"2024-04-25T01:25:53.364575+0000","last_active":"2024-04-25T01:25:54.366985+0000","last_peered":"2024-04-25T01:25:54.366985+0000","last_clean":"2024-04-25T01:25:54.366985+0000","last_became_active":"2024-04-25T01:25:53.364311+0000","last_became_peered":"2024-04-25T01:25:53.364311+0000","last_unstale":"2024-04-25T01:25:54.366985+0000","last_undegraded":"2024-04-25T01:25:54.366985+0000","last_fullsized":"2024-04-25T01:25:54.366985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361693+0000","last_change":"2024-04-25T01:25:53.357142+0000","last_active":"2024-04-25T01:25:54.361693+0000","last_peered":"2024-04-25T01:25:54.361693+0000","last_clean":"2024-04-25T01:25:54.361693+0000","last_became_active":"2024-04-25T01:25:53.356854+0000","last_became_peered":"2024-04-25T01:25:53.356854+0000","last_unstale":"2024-04-25T01:25:54.361693+0000","last_undegraded":"2024-04-25T01:25:54.361693+0000","last_fullsized":"2024-04-25T01:25:54.361693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580456+0000","last_change":"2024-04-25T01:25:53.369850+0000","last_active":"2024-04-25T01:25:54.580456+0000","last_peered":"2024-04-25T01:25:54.580456+0000","last_clean":"2024-04-25T01:25:54.580456+0000","last_became_active":"2024-04-25T01:25:53.369350+0000","last_became_peered":"2024-04-25T01:25:53.369350+0000","last_unstale":"2024-04-25T01:25:54.580456+0000","last_undegraded":"2024-04-25T01:25:54.580456+0000","last_fullsized":"2024-04-25T01:25:54.580456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579112+0000","last_change":"2024-04-25T01:25:53.371345+0000","last_active":"2024-04-25T01:25:54.579112+0000","last_peered":"2024-04-25T01:25:54.579112+0000","last_clean":"2024-04-25T01:25:54.579112+0000","last_became_active":"2024-04-25T01:25:53.371080+0000","last_became_peered":"2024-04-25T01:25:53.371080+0000","last_unstale":"2024-04-25T01:25:54.579112+0000","last_undegraded":"2024-04-25T01:25:54.579112+0000","last_fullsized":"2024-04-25T01:25:54.579112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:09.855763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368224+0000","last_change":"2024-04-25T01:25:53.367143+0000","last_active":"2024-04-25T01:25:54.368224+0000","last_peered":"2024-04-25T01:25:54.368224+0000","last_clean":"2024-04-25T01:25:54.368224+0000","last_became_active":"2024-04-25T01:25:53.366847+0000","last_became_peered":"2024-04-25T01:25:53.366847+0000","last_unstale":"2024-04-25T01:25:54.368224+0000","last_undegraded":"2024-04-25T01:25:54.368224+0000","last_fullsized":"2024-04-25T01:25:54.368224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579943+0000","last_change":"2024-04-25T01:25:53.366586+0000","last_active":"2024-04-25T01:25:54.579943+0000","last_peered":"2024-04-25T01:25:54.579943+0000","last_clean":"2024-04-25T01:25:54.579943+0000","last_became_active":"2024-04-25T01:25:53.366302+0000","last_became_peered":"2024-04-25T01:25:53.366302+0000","last_unstale":"2024-04-25T01:25:54.579943+0000","last_undegraded":"2024-04-25T01:25:54.579943+0000","last_fullsized":"2024-04-25T01:25:54.579943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366663+0000","last_change":"2024-04-25T01:25:52.353955+0000","last_active":"2024-04-25T01:25:54.366663+0000","last_peered":"2024-04-25T01:25:54.366663+0000","last_clean":"2024-04-25T01:25:54.366663+0000","last_became_active":"2024-04-25T01:25:52.353659+0000","last_became_peered":"2024-04-25T01:25:52.353659+0000","last_unstale":"2024-04-25T01:25:54.366663+0000","last_undegraded":"2024-04-25T01:25:54.366663+0000","last_fullsized":"2024-04-25T01:25:54.366663+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361779+0000","last_change":"2024-04-25T01:25:53.356610+0000","last_active":"2024-04-25T01:25:54.361779+0000","last_peered":"2024-04-25T01:25:54.361779+0000","last_clean":"2024-04-25T01:25:54.361779+0000","last_became_active":"2024-04-25T01:25:53.356015+0000","last_became_peered":"2024-04-25T01:25:53.356015+0000","last_unstale":"2024-04-25T01:25:54.361779+0000","last_undegraded":"2024-04-25T01:25:54.361779+0000","last_fullsized":"2024-04-25T01:25:54.361779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368267+0000","last_change":"2024-04-25T01:25:52.349989+0000","last_active":"2024-04-25T01:25:54.368267+0000","last_peered":"2024-04-25T01:25:54.368267+0000","last_clean":"2024-04-25T01:25:54.368267+0000","last_became_active":"2024-04-25T01:25:52.349672+0000","last_became_peered":"2024-04-25T01:25:52.349672+0000","last_unstale":"2024-04-25T01:25:54.368267+0000","last_undegraded":"2024-04-25T01:25:54.368267+0000","last_fullsized":"2024-04-25T01:25:54.368267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366294+0000","last_change":"2024-04-25T01:25:52.347464+0000","last_active":"2024-04-25T01:25:54.366294+0000","last_peered":"2024-04-25T01:25:54.366294+0000","last_clean":"2024-04-25T01:25:54.366294+0000","last_became_active":"2024-04-25T01:25:52.347138+0000","last_became_peered":"2024-04-25T01:25:52.347138+0000","last_unstale":"2024-04-25T01:25:54.366294+0000","last_undegraded":"2024-04-25T01:25:54.366294+0000","last_fullsized":"2024-04-25T01:25:54.366294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366446+0000","last_change":"2024-04-25T01:25:53.363794+0000","last_active":"2024-04-25T01:25:54.366446+0000","last_peered":"2024-04-25T01:25:54.366446+0000","last_clean":"2024-04-25T01:25:54.366446+0000","last_became_active":"2024-04-25T01:25:53.363509+0000","last_became_peered":"2024-04-25T01:25:53.363509+0000","last_unstale":"2024-04-25T01:25:54.366446+0000","last_undegraded":"2024-04-25T01:25:54.366446+0000","last_fullsized":"2024-04-25T01:25:54.366446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579760+0000","last_change":"2024-04-25T01:25:52.358081+0000","last_active":"2024-04-25T01:25:54.579760+0000","last_peered":"2024-04-25T01:25:54.579760+0000","last_clean":"2024-04-25T01:25:54.579760+0000","last_became_active":"2024-04-25T01:25:52.357365+0000","last_became_peered":"2024-04-25T01:25:52.357365+0000","last_unstale":"2024-04-25T01:25:54.579760+0000","last_undegraded":"2024-04-25T01:25:54.579760+0000","last_fullsized":"2024-04-25T01:25:54.579760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.365858+0000","last_change":"2024-04-25T01:25:53.363043+0000","last_active":"2024-04-25T01:25:54.365858+0000","last_peered":"2024-04-25T01:25:54.365858+0000","last_clean":"2024-04-25T01:25:54.365858+0000","last_became_active":"2024-04-25T01:25:53.362601+0000","last_became_peered":"2024-04-25T01:25:53.362601+0000","last_unstale":"2024-04-25T01:25:54.365858+0000","last_undegraded":"2024-04-25T01:25:54.365858+0000","last_fullsized":"2024-04-25T01:25:54.365858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368458+0000","last_change":"2024-04-25T01:25:52.357459+0000","last_active":"2024-04-25T01:25:54.368458+0000","last_peered":"2024-04-25T01:25:54.368458+0000","last_clean":"2024-04-25T01:25:54.368458+0000","last_became_active":"2024-04-25T01:25:52.357182+0000","last_became_peered":"2024-04-25T01:25:52.357182+0000","last_unstale":"2024-04-25T01:25:54.368458+0000","last_undegraded":"2024-04-25T01:25:54.368458+0000","last_fullsized":"2024-04-25T01:25:54.368458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580762+0000","last_change":"2024-04-25T01:25:53.371445+0000","last_active":"2024-04-25T01:25:54.580762+0000","last_peered":"2024-04-25T01:25:54.580762+0000","last_clean":"2024-04-25T01:25:54.580762+0000","last_became_active":"2024-04-25T01:25:53.371086+0000","last_became_peered":"2024-04-25T01:25:53.371086+0000","last_unstale":"2024-04-25T01:25:54.580762+0000","last_undegraded":"2024-04-25T01:25:54.580762+0000","last_fullsized":"2024-04-25T01:25:54.580762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580764+0000","last_change":"2024-04-25T01:25:52.356231+0000","last_active":"2024-04-25T01:25:54.580764+0000","last_peered":"2024-04-25T01:25:54.580764+0000","last_clean":"2024-04-25T01:25:54.580764+0000","last_became_active":"2024-04-25T01:25:52.355913+0000","last_became_peered":"2024-04-25T01:25:52.355913+0000","last_unstale":"2024-04-25T01:25:54.580764+0000","last_undegraded":"2024-04-25T01:25:54.580764+0000","last_fullsized":"2024-04-25T01:25:54.580764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361600+0000","last_change":"2024-04-25T01:25:53.374311+0000","last_active":"2024-04-25T01:25:54.361600+0000","last_peered":"2024-04-25T01:25:54.361600+0000","last_clean":"2024-04-25T01:25:54.361600+0000","last_became_active":"2024-04-25T01:25:53.373890+0000","last_became_peered":"2024-04-25T01:25:53.373890+0000","last_unstale":"2024-04-25T01:25:54.361600+0000","last_undegraded":"2024-04-25T01:25:54.361600+0000","last_fullsized":"2024-04-25T01:25:54.361600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366048+0000","last_change":"2024-04-25T01:25:52.357610+0000","last_active":"2024-04-25T01:25:54.366048+0000","last_peered":"2024-04-25T01:25:54.366048+0000","last_clean":"2024-04-25T01:25:54.366048+0000","last_became_active":"2024-04-25T01:25:52.357355+0000","last_became_peered":"2024-04-25T01:25:52.357355+0000","last_unstale":"2024-04-25T01:25:54.366048+0000","last_undegraded":"2024-04-25T01:25:54.366048+0000","last_fullsized":"2024-04-25T01:25:54.366048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580344+0000","last_change":"2024-04-25T01:25:53.366416+0000","last_active":"2024-04-25T01:25:54.580344+0000","last_peered":"2024-04-25T01:25:54.580344+0000","last_clean":"2024-04-25T01:25:54.580344+0000","last_became_active":"2024-04-25T01:25:53.366157+0000","last_became_peered":"2024-04-25T01:25:53.366157+0000","last_unstale":"2024-04-25T01:25:54.580344+0000","last_undegraded":"2024-04-25T01:25:54.580344+0000","last_fullsized":"2024-04-25T01:25:54.580344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:26.277566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368293+0000","last_change":"2024-04-25T01:25:52.350446+0000","last_active":"2024-04-25T01:25:54.368293+0000","last_peered":"2024-04-25T01:25:54.368293+0000","last_clean":"2024-04-25T01:25:54.368293+0000","last_became_active":"2024-04-25T01:25:52.349448+0000","last_became_peered":"2024-04-25T01:25:52.349448+0000","last_unstale":"2024-04-25T01:25:54.368293+0000","last_undegraded":"2024-04-25T01:25:54.368293+0000","last_fullsized":"2024-04-25T01:25:54.368293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581200+0000","last_change":"2024-04-25T01:25:53.373572+0000","last_active":"2024-04-25T01:25:54.581200+0000","last_peered":"2024-04-25T01:25:54.581200+0000","last_clean":"2024-04-25T01:25:54.581200+0000","last_became_active":"2024-04-25T01:25:53.373129+0000","last_became_peered":"2024-04-25T01:25:53.373129+0000","last_unstale":"2024-04-25T01:25:54.581200+0000","last_undegraded":"2024-04-25T01:25:54.581200+0000","last_fullsized":"2024-04-25T01:25:54.581200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361780+0000","last_change":"2024-04-25T01:25:52.349092+0000","last_active":"2024-04-25T01:25:54.361780+0000","last_peered":"2024-04-25T01:25:54.361780+0000","last_clean":"2024-04-25T01:25:54.361780+0000","last_became_active":"2024-04-25T01:25:52.348846+0000","last_became_peered":"2024-04-25T01:25:52.348846+0000","last_unstale":"2024-04-25T01:25:54.361780+0000","last_undegraded":"2024-04-25T01:25:54.361780+0000","last_fullsized":"2024-04-25T01:25:54.361780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:21.982440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361411+0000","last_change":"2024-04-25T01:25:53.370720+0000","last_active":"2024-04-25T01:25:54.361411+0000","last_peered":"2024-04-25T01:25:54.361411+0000","last_clean":"2024-04-25T01:25:54.361411+0000","last_became_active":"2024-04-25T01:25:53.370142+0000","last_became_peered":"2024-04-25T01:25:53.370142+0000","last_unstale":"2024-04-25T01:25:54.361411+0000","last_undegraded":"2024-04-25T01:25:54.361411+0000","last_fullsized":"2024-04-25T01:25:54.361411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:52.833379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580473+0000","last_change":"2024-04-25T01:25:52.357397+0000","last_active":"2024-04-25T01:25:54.580473+0000","last_peered":"2024-04-25T01:25:54.580473+0000","last_clean":"2024-04-25T01:25:54.580473+0000","last_became_active":"2024-04-25T01:25:52.356881+0000","last_became_peered":"2024-04-25T01:25:52.356881+0000","last_unstale":"2024-04-25T01:25:54.580473+0000","last_undegraded":"2024-04-25T01:25:54.580473+0000","last_fullsized":"2024-04-25T01:25:54.580473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581103+0000","last_change":"2024-04-25T01:25:53.368187+0000","last_active":"2024-04-25T01:25:54.581103+0000","last_peered":"2024-04-25T01:25:54.581103+0000","last_clean":"2024-04-25T01:25:54.581103+0000","last_became_active":"2024-04-25T01:25:53.367664+0000","last_became_peered":"2024-04-25T01:25:53.367664+0000","last_unstale":"2024-04-25T01:25:54.581103+0000","last_undegraded":"2024-04-25T01:25:54.581103+0000","last_fullsized":"2024-04-25T01:25:54.581103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368140+0000","last_change":"2024-04-25T01:25:52.351205+0000","last_active":"2024-04-25T01:25:54.368140+0000","last_peered":"2024-04-25T01:25:54.368140+0000","last_clean":"2024-04-25T01:25:54.368140+0000","last_became_active":"2024-04-25T01:25:52.350946+0000","last_became_peered":"2024-04-25T01:25:52.350946+0000","last_unstale":"2024-04-25T01:25:54.368140+0000","last_undegraded":"2024-04-25T01:25:54.368140+0000","last_fullsized":"2024-04-25T01:25:54.368140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368302+0000","last_change":"2024-04-25T01:25:53.364987+0000","last_active":"2024-04-25T01:25:54.368302+0000","last_peered":"2024-04-25T01:25:54.368302+0000","last_clean":"2024-04-25T01:25:54.368302+0000","last_became_active":"2024-04-25T01:25:53.364431+0000","last_became_peered":"2024-04-25T01:25:53.364431+0000","last_unstale":"2024-04-25T01:25:54.368302+0000","last_undegraded":"2024-04-25T01:25:54.368302+0000","last_fullsized":"2024-04-25T01:25:54.368302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579031+0000","last_change":"2024-04-25T01:25:52.347413+0000","last_active":"2024-04-25T01:25:54.579031+0000","last_peered":"2024-04-25T01:25:54.579031+0000","last_clean":"2024-04-25T01:25:54.579031+0000","last_became_active":"2024-04-25T01:25:52.347144+0000","last_became_peered":"2024-04-25T01:25:52.347144+0000","last_unstale":"2024-04-25T01:25:54.579031+0000","last_undegraded":"2024-04-25T01:25:54.579031+0000","last_fullsized":"2024-04-25T01:25:54.579031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580057+0000","last_change":"2024-04-25T01:25:53.366376+0000","last_active":"2024-04-25T01:25:54.580057+0000","last_peered":"2024-04-25T01:25:54.580057+0000","last_clean":"2024-04-25T01:25:54.580057+0000","last_became_active":"2024-04-25T01:25:53.365715+0000","last_became_peered":"2024-04-25T01:25:53.365715+0000","last_unstale":"2024-04-25T01:25:54.580057+0000","last_undegraded":"2024-04-25T01:25:54.580057+0000","last_fullsized":"2024-04-25T01:25:54.580057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580408+0000","last_change":"2024-04-25T01:25:52.350688+0000","last_active":"2024-04-25T01:25:54.580408+0000","last_peered":"2024-04-25T01:25:54.580408+0000","last_clean":"2024-04-25T01:25:54.580408+0000","last_became_active":"2024-04-25T01:25:52.350357+0000","last_became_peered":"2024-04-25T01:25:52.350357+0000","last_unstale":"2024-04-25T01:25:54.580408+0000","last_undegraded":"2024-04-25T01:25:54.580408+0000","last_fullsized":"2024-04-25T01:25:54.580408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581500+0000","last_change":"2024-04-25T01:25:53.365171+0000","last_active":"2024-04-25T01:25:54.581500+0000","last_peered":"2024-04-25T01:25:54.581500+0000","last_clean":"2024-04-25T01:25:54.581500+0000","last_became_active":"2024-04-25T01:25:53.364049+0000","last_became_peered":"2024-04-25T01:25:53.364049+0000","last_unstale":"2024-04-25T01:25:54.581500+0000","last_undegraded":"2024-04-25T01:25:54.581500+0000","last_fullsized":"2024-04-25T01:25:54.581500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579222+0000","last_change":"2024-04-25T01:25:52.345943+0000","last_active":"2024-04-25T01:25:54.579222+0000","last_peered":"2024-04-25T01:25:54.579222+0000","last_clean":"2024-04-25T01:25:54.579222+0000","last_became_active":"2024-04-25T01:25:52.345717+0000","last_became_peered":"2024-04-25T01:25:52.345717+0000","last_unstale":"2024-04-25T01:25:54.579222+0000","last_undegraded":"2024-04-25T01:25:54.579222+0000","last_fullsized":"2024-04-25T01:25:54.579222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581766+0000","last_change":"2024-04-25T01:25:53.370754+0000","last_active":"2024-04-25T01:25:54.581766+0000","last_peered":"2024-04-25T01:25:54.581766+0000","last_clean":"2024-04-25T01:25:54.581766+0000","last_became_active":"2024-04-25T01:25:53.370074+0000","last_became_peered":"2024-04-25T01:25:53.370074+0000","last_unstale":"2024-04-25T01:25:54.581766+0000","last_undegraded":"2024-04-25T01:25:54.581766+0000","last_fullsized":"2024-04-25T01:25:54.581766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366101+0000","last_change":"2024-04-25T01:25:52.355133+0000","last_active":"2024-04-25T01:25:54.366101+0000","last_peered":"2024-04-25T01:25:54.366101+0000","last_clean":"2024-04-25T01:25:54.366101+0000","last_became_active":"2024-04-25T01:25:52.354836+0000","last_became_peered":"2024-04-25T01:25:52.354836+0000","last_unstale":"2024-04-25T01:25:54.366101+0000","last_undegraded":"2024-04-25T01:25:54.366101+0000","last_fullsized":"2024-04-25T01:25:54.366101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:10.331704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580338+0000","last_change":"2024-04-25T01:25:53.371018+0000","last_active":"2024-04-25T01:25:54.580338+0000","last_peered":"2024-04-25T01:25:54.580338+0000","last_clean":"2024-04-25T01:25:54.580338+0000","last_became_active":"2024-04-25T01:25:53.370539+0000","last_became_peered":"2024-04-25T01:25:53.370539+0000","last_unstale":"2024-04-25T01:25:54.580338+0000","last_undegraded":"2024-04-25T01:25:54.580338+0000","last_fullsized":"2024-04-25T01:25:54.580338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579131+0000","last_change":"2024-04-25T01:25:52.352994+0000","last_active":"2024-04-25T01:25:54.579131+0000","last_peered":"2024-04-25T01:25:54.579131+0000","last_clean":"2024-04-25T01:25:54.579131+0000","last_became_active":"2024-04-25T01:25:52.352644+0000","last_became_peered":"2024-04-25T01:25:52.352644+0000","last_unstale":"2024-04-25T01:25:54.579131+0000","last_undegraded":"2024-04-25T01:25:54.579131+0000","last_fullsized":"2024-04-25T01:25:54.579131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581219+0000","last_change":"2024-04-25T01:25:53.375523+0000","last_active":"2024-04-25T01:25:54.581219+0000","last_peered":"2024-04-25T01:25:54.581219+0000","last_clean":"2024-04-25T01:25:54.581219+0000","last_became_active":"2024-04-25T01:25:53.375219+0000","last_became_peered":"2024-04-25T01:25:53.375219+0000","last_unstale":"2024-04-25T01:25:54.581219+0000","last_undegraded":"2024-04-25T01:25:54.581219+0000","last_fullsized":"2024-04-25T01:25:54.581219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580132+0000","last_change":"2024-04-25T01:25:41.719670+0000","last_active":"2024-04-25T01:25:54.580132+0000","last_peered":"2024-04-25T01:25:54.580132+0000","last_clean":"2024-04-25T01:25:54.580132+0000","last_became_active":"2024-04-25T01:25:41.718721+0000","last_became_peered":"2024-04-25T01:25:41.718721+0000","last_unstale":"2024-04-25T01:25:54.580132+0000","last_undegraded":"2024-04-25T01:25:54.580132+0000","last_fullsized":"2024-04-25T01:25:54.580132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_clean_scrub_stamp":"2024-04-25T01:25:40.688097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:27.456746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580582+0000","last_change":"2024-04-25T01:25:52.353155+0000","last_active":"2024-04-25T01:25:54.580582+0000","last_peered":"2024-04-25T01:25:54.580582+0000","last_clean":"2024-04-25T01:25:54.580582+0000","last_became_active":"2024-04-25T01:25:52.352826+0000","last_became_peered":"2024-04-25T01:25:52.352826+0000","last_unstale":"2024-04-25T01:25:54.580582+0000","last_undegraded":"2024-04-25T01:25:54.580582+0000","last_fullsized":"2024-04-25T01:25:54.580582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366734+0000","last_change":"2024-04-25T01:25:53.374719+0000","last_active":"2024-04-25T01:25:54.366734+0000","last_peered":"2024-04-25T01:25:54.366734+0000","last_clean":"2024-04-25T01:25:54.366734+0000","last_became_active":"2024-04-25T01:25:53.374425+0000","last_became_peered":"2024-04-25T01:25:53.374425+0000","last_unstale":"2024-04-25T01:25:54.366734+0000","last_undegraded":"2024-04-25T01:25:54.366734+0000","last_fullsized":"2024-04-25T01:25:54.366734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580106+0000","last_change":"2024-04-25T01:25:52.357683+0000","last_active":"2024-04-25T01:25:54.580106+0000","last_peered":"2024-04-25T01:25:54.580106+0000","last_clean":"2024-04-25T01:25:54.580106+0000","last_became_active":"2024-04-25T01:25:52.356924+0000","last_became_peered":"2024-04-25T01:25:52.356924+0000","last_unstale":"2024-04-25T01:25:54.580106+0000","last_undegraded":"2024-04-25T01:25:54.580106+0000","last_fullsized":"2024-04-25T01:25:54.580106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361256+0000","last_change":"2024-04-25T01:25:53.355950+0000","last_active":"2024-04-25T01:25:54.361256+0000","last_peered":"2024-04-25T01:25:54.361256+0000","last_clean":"2024-04-25T01:25:54.361256+0000","last_became_active":"2024-04-25T01:25:53.355648+0000","last_became_peered":"2024-04-25T01:25:53.355648+0000","last_unstale":"2024-04-25T01:25:54.361256+0000","last_undegraded":"2024-04-25T01:25:54.361256+0000","last_fullsized":"2024-04-25T01:25:54.361256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.579084+0000","last_change":"2024-04-25T01:25:52.345321+0000","last_active":"2024-04-25T01:25:54.579084+0000","last_peered":"2024-04-25T01:25:54.579084+0000","last_clean":"2024-04-25T01:25:54.579084+0000","last_became_active":"2024-04-25T01:25:52.345061+0000","last_became_peered":"2024-04-25T01:25:52.345061+0000","last_unstale":"2024-04-25T01:25:54.579084+0000","last_undegraded":"2024-04-25T01:25:54.579084+0000","last_fullsized":"2024-04-25T01:25:54.579084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366196+0000","last_change":"2024-04-25T01:25:53.362922+0000","last_active":"2024-04-25T01:25:54.366196+0000","last_peered":"2024-04-25T01:25:54.366196+0000","last_clean":"2024-04-25T01:25:54.366196+0000","last_became_active":"2024-04-25T01:25:53.362533+0000","last_became_peered":"2024-04-25T01:25:53.362533+0000","last_unstale":"2024-04-25T01:25:54.366196+0000","last_undegraded":"2024-04-25T01:25:54.366196+0000","last_fullsized":"2024-04-25T01:25:54.366196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580540+0000","last_change":"2024-04-25T01:25:52.354499+0000","last_active":"2024-04-25T01:25:54.580540+0000","last_peered":"2024-04-25T01:25:54.580540+0000","last_clean":"2024-04-25T01:25:54.580540+0000","last_became_active":"2024-04-25T01:25:52.354193+0000","last_became_peered":"2024-04-25T01:25:52.354193+0000","last_unstale":"2024-04-25T01:25:54.580540+0000","last_undegraded":"2024-04-25T01:25:54.580540+0000","last_fullsized":"2024-04-25T01:25:54.580540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580437+0000","last_change":"2024-04-25T01:25:53.361696+0000","last_active":"2024-04-25T01:25:54.580437+0000","last_peered":"2024-04-25T01:25:54.580437+0000","last_clean":"2024-04-25T01:25:54.580437+0000","last_became_active":"2024-04-25T01:25:53.361156+0000","last_became_peered":"2024-04-25T01:25:53.361156+0000","last_unstale":"2024-04-25T01:25:54.580437+0000","last_undegraded":"2024-04-25T01:25:54.580437+0000","last_fullsized":"2024-04-25T01:25:54.580437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:27.472315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.365972+0000","last_change":"2024-04-25T01:25:52.350128+0000","last_active":"2024-04-25T01:25:54.365972+0000","last_peered":"2024-04-25T01:25:54.365972+0000","last_clean":"2024-04-25T01:25:54.365972+0000","last_became_active":"2024-04-25T01:25:52.349874+0000","last_became_peered":"2024-04-25T01:25:52.349874+0000","last_unstale":"2024-04-25T01:25:54.365972+0000","last_undegraded":"2024-04-25T01:25:54.365972+0000","last_fullsized":"2024-04-25T01:25:54.365972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581110+0000","last_change":"2024-04-25T01:25:53.368164+0000","last_active":"2024-04-25T01:25:54.581110+0000","last_peered":"2024-04-25T01:25:54.581110+0000","last_clean":"2024-04-25T01:25:54.581110+0000","last_became_active":"2024-04-25T01:25:53.367854+0000","last_became_peered":"2024-04-25T01:25:53.367854+0000","last_unstale":"2024-04-25T01:25:54.581110+0000","last_undegraded":"2024-04-25T01:25:54.581110+0000","last_fullsized":"2024-04-25T01:25:54.581110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581462+0000","last_change":"2024-04-25T01:25:52.356842+0000","last_active":"2024-04-25T01:25:54.581462+0000","last_peered":"2024-04-25T01:25:54.581462+0000","last_clean":"2024-04-25T01:25:54.581462+0000","last_became_active":"2024-04-25T01:25:52.356396+0000","last_became_peered":"2024-04-25T01:25:52.356396+0000","last_unstale":"2024-04-25T01:25:54.581462+0000","last_undegraded":"2024-04-25T01:25:54.581462+0000","last_fullsized":"2024-04-25T01:25:54.581462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366106+0000","last_change":"2024-04-25T01:25:53.366944+0000","last_active":"2024-04-25T01:25:54.366106+0000","last_peered":"2024-04-25T01:25:54.366106+0000","last_clean":"2024-04-25T01:25:54.366106+0000","last_became_active":"2024-04-25T01:25:53.366693+0000","last_became_peered":"2024-04-25T01:25:53.366693+0000","last_unstale":"2024-04-25T01:25:54.366106+0000","last_undegraded":"2024-04-25T01:25:54.366106+0000","last_fullsized":"2024-04-25T01:25:54.366106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.367714+0000","last_change":"2024-04-25T01:25:52.355395+0000","last_active":"2024-04-25T01:25:54.367714+0000","last_peered":"2024-04-25T01:25:54.367714+0000","last_clean":"2024-04-25T01:25:54.367714+0000","last_became_active":"2024-04-25T01:25:52.355137+0000","last_became_peered":"2024-04-25T01:25:52.355137+0000","last_unstale":"2024-04-25T01:25:54.367714+0000","last_undegraded":"2024-04-25T01:25:54.367714+0000","last_fullsized":"2024-04-25T01:25:54.367714+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:26.467968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366578+0000","last_change":"2024-04-25T01:25:53.364967+0000","last_active":"2024-04-25T01:25:54.366578+0000","last_peered":"2024-04-25T01:25:54.366578+0000","last_clean":"2024-04-25T01:25:54.366578+0000","last_became_active":"2024-04-25T01:25:53.364694+0000","last_became_peered":"2024-04-25T01:25:53.364694+0000","last_unstale":"2024-04-25T01:25:54.366578+0000","last_undegraded":"2024-04-25T01:25:54.366578+0000","last_fullsized":"2024-04-25T01:25:54.366578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368537+0000","last_change":"2024-04-25T01:25:52.353269+0000","last_active":"2024-04-25T01:25:54.368537+0000","last_peered":"2024-04-25T01:25:54.368537+0000","last_clean":"2024-04-25T01:25:54.368537+0000","last_became_active":"2024-04-25T01:25:52.352991+0000","last_became_peered":"2024-04-25T01:25:52.352991+0000","last_unstale":"2024-04-25T01:25:54.368537+0000","last_undegraded":"2024-04-25T01:25:54.368537+0000","last_fullsized":"2024-04-25T01:25:54.368537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361579+0000","last_change":"2024-04-25T01:25:53.370111+0000","last_active":"2024-04-25T01:25:54.361579+0000","last_peered":"2024-04-25T01:25:54.361579+0000","last_clean":"2024-04-25T01:25:54.361579+0000","last_became_active":"2024-04-25T01:25:53.369837+0000","last_became_peered":"2024-04-25T01:25:53.369837+0000","last_unstale":"2024-04-25T01:25:54.361579+0000","last_undegraded":"2024-04-25T01:25:54.361579+0000","last_fullsized":"2024-04-25T01:25:54.361579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580024+0000","last_change":"2024-04-25T01:25:52.353836+0000","last_active":"2024-04-25T01:25:54.580024+0000","last_peered":"2024-04-25T01:25:54.580024+0000","last_clean":"2024-04-25T01:25:54.580024+0000","last_became_active":"2024-04-25T01:25:52.353552+0000","last_became_peered":"2024-04-25T01:25:52.353552+0000","last_unstale":"2024-04-25T01:25:54.580024+0000","last_undegraded":"2024-04-25T01:25:54.580024+0000","last_fullsized":"2024-04-25T01:25:54.580024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580513+0000","last_change":"2024-04-25T01:25:53.361653+0000","last_active":"2024-04-25T01:25:54.580513+0000","last_peered":"2024-04-25T01:25:54.580513+0000","last_clean":"2024-04-25T01:25:54.580513+0000","last_became_active":"2024-04-25T01:25:53.361265+0000","last_became_peered":"2024-04-25T01:25:53.361265+0000","last_unstale":"2024-04-25T01:25:54.580513+0000","last_undegraded":"2024-04-25T01:25:54.580513+0000","last_fullsized":"2024-04-25T01:25:54.580513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366249+0000","last_change":"2024-04-25T01:25:52.353926+0000","last_active":"2024-04-25T01:25:54.366249+0000","last_peered":"2024-04-25T01:25:54.366249+0000","last_clean":"2024-04-25T01:25:54.366249+0000","last_became_active":"2024-04-25T01:25:52.353656+0000","last_became_peered":"2024-04-25T01:25:52.353656+0000","last_unstale":"2024-04-25T01:25:54.366249+0000","last_undegraded":"2024-04-25T01:25:54.366249+0000","last_fullsized":"2024-04-25T01:25:54.366249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366408+0000","last_change":"2024-04-25T01:25:53.361790+0000","last_active":"2024-04-25T01:25:54.366408+0000","last_peered":"2024-04-25T01:25:54.366408+0000","last_clean":"2024-04-25T01:25:54.366408+0000","last_became_active":"2024-04-25T01:25:53.361542+0000","last_became_peered":"2024-04-25T01:25:53.361542+0000","last_unstale":"2024-04-25T01:25:54.366408+0000","last_undegraded":"2024-04-25T01:25:54.366408+0000","last_fullsized":"2024-04-25T01:25:54.366408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366210+0000","last_change":"2024-04-25T01:25:52.348841+0000","last_active":"2024-04-25T01:25:54.366210+0000","last_peered":"2024-04-25T01:25:54.366210+0000","last_clean":"2024-04-25T01:25:54.366210+0000","last_became_active":"2024-04-25T01:25:52.348550+0000","last_became_peered":"2024-04-25T01:25:52.348550+0000","last_unstale":"2024-04-25T01:25:54.366210+0000","last_undegraded":"2024-04-25T01:25:54.366210+0000","last_fullsized":"2024-04-25T01:25:54.366210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366707+0000","last_change":"2024-04-25T01:25:53.361074+0000","last_active":"2024-04-25T01:25:54.366707+0000","last_peered":"2024-04-25T01:25:54.366707+0000","last_clean":"2024-04-25T01:25:54.366707+0000","last_became_active":"2024-04-25T01:25:53.360802+0000","last_became_peered":"2024-04-25T01:25:53.360802+0000","last_unstale":"2024-04-25T01:25:54.366707+0000","last_undegraded":"2024-04-25T01:25:54.366707+0000","last_fullsized":"2024-04-25T01:25:54.366707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581335+0000","last_change":"2024-04-25T01:25:52.358616+0000","last_active":"2024-04-25T01:25:54.581335+0000","last_peered":"2024-04-25T01:25:54.581335+0000","last_clean":"2024-04-25T01:25:54.581335+0000","last_became_active":"2024-04-25T01:25:52.358318+0000","last_became_peered":"2024-04-25T01:25:52.358318+0000","last_unstale":"2024-04-25T01:25:54.581335+0000","last_undegraded":"2024-04-25T01:25:54.581335+0000","last_fullsized":"2024-04-25T01:25:54.581335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.368226+0000","last_change":"2024-04-25T01:25:53.372385+0000","last_active":"2024-04-25T01:25:54.368226+0000","last_peered":"2024-04-25T01:25:54.368226+0000","last_clean":"2024-04-25T01:25:54.368226+0000","last_became_active":"2024-04-25T01:25:53.372064+0000","last_became_peered":"2024-04-25T01:25:53.372064+0000","last_unstale":"2024-04-25T01:25:54.368226+0000","last_undegraded":"2024-04-25T01:25:54.368226+0000","last_fullsized":"2024-04-25T01:25:54.368226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581025+0000","last_change":"2024-04-25T01:25:53.368739+0000","last_active":"2024-04-25T01:25:54.581025+0000","last_peered":"2024-04-25T01:25:54.581025+0000","last_clean":"2024-04-25T01:25:54.581025+0000","last_became_active":"2024-04-25T01:25:53.368470+0000","last_became_peered":"2024-04-25T01:25:53.368470+0000","last_unstale":"2024-04-25T01:25:54.581025+0000","last_undegraded":"2024-04-25T01:25:54.581025+0000","last_fullsized":"2024-04-25T01:25:54.581025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:29.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581195+0000","last_change":"2024-04-25T01:25:52.359139+0000","last_active":"2024-04-25T01:25:54.581195+0000","last_peered":"2024-04-25T01:25:54.581195+0000","last_clean":"2024-04-25T01:25:54.581195+0000","last_became_active":"2024-04-25T01:25:52.358841+0000","last_became_peered":"2024-04-25T01:25:52.358841+0000","last_unstale":"2024-04-25T01:25:54.581195+0000","last_undegraded":"2024-04-25T01:25:54.581195+0000","last_fullsized":"2024-04-25T01:25:54.581195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.580118+0000","last_change":"2024-04-25T01:25:53.370061+0000","last_active":"2024-04-25T01:25:54.580118+0000","last_peered":"2024-04-25T01:25:54.580118+0000","last_clean":"2024-04-25T01:25:54.580118+0000","last_became_active":"2024-04-25T01:25:53.369721+0000","last_became_peered":"2024-04-25T01:25:53.369721+0000","last_unstale":"2024-04-25T01:25:54.580118+0000","last_undegraded":"2024-04-25T01:25:54.580118+0000","last_fullsized":"2024-04-25T01:25:54.580118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.581355+0000","last_change":"2024-04-25T01:25:52.356403+0000","last_active":"2024-04-25T01:25:54.581355+0000","last_peered":"2024-04-25T01:25:54.581355+0000","last_clean":"2024-04-25T01:25:54.581355+0000","last_became_active":"2024-04-25T01:25:52.355822+0000","last_became_peered":"2024-04-25T01:25:52.355822+0000","last_unstale":"2024-04-25T01:25:54.581355+0000","last_undegraded":"2024-04-25T01:25:54.581355+0000","last_fullsized":"2024-04-25T01:25:54.581355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361454+0000","last_change":"2024-04-25T01:25:53.374150+0000","last_active":"2024-04-25T01:25:54.361454+0000","last_peered":"2024-04-25T01:25:54.361454+0000","last_clean":"2024-04-25T01:25:54.361454+0000","last_became_active":"2024-04-25T01:25:53.373755+0000","last_became_peered":"2024-04-25T01:25:53.373755+0000","last_unstale":"2024-04-25T01:25:54.361454+0000","last_undegraded":"2024-04-25T01:25:54.361454+0000","last_fullsized":"2024-04-25T01:25:54.361454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366968+0000","last_change":"2024-04-25T01:25:52.349185+0000","last_active":"2024-04-25T01:25:54.366968+0000","last_peered":"2024-04-25T01:25:54.366968+0000","last_clean":"2024-04-25T01:25:54.366968+0000","last_became_active":"2024-04-25T01:25:52.348931+0000","last_became_peered":"2024-04-25T01:25:52.348931+0000","last_unstale":"2024-04-25T01:25:54.366968+0000","last_undegraded":"2024-04-25T01:25:54.366968+0000","last_fullsized":"2024-04-25T01:25:54.366968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.366936+0000","last_change":"2024-04-25T01:25:53.364257+0000","last_active":"2024-04-25T01:25:54.366936+0000","last_peered":"2024-04-25T01:25:54.366936+0000","last_clean":"2024-04-25T01:25:54.366936+0000","last_became_active":"2024-04-25T01:25:53.363409+0000","last_became_peered":"2024-04-25T01:25:53.363409+0000","last_unstale":"2024-04-25T01:25:54.366936+0000","last_undegraded":"2024-04-25T01:25:54.366936+0000","last_fullsized":"2024-04-25T01:25:54.366936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T01:25:54.361657+0000","last_change":"2024-04-25T01:25:52.353978+0000","last_active":"2024-04-25T01:25:54.361657+0000","last_peered":"2024-04-25T01:25:54.361657+0000","last_clean":"2024-04-25T01:25:54.361657+0000","last_became_active":"2024-04-25T01:25:52.353649+0000","last_became_peered":"2024-04-25T01:25:52.353649+0000","last_unstale":"2024-04-25T01:25:54.361657+0000","last_undegraded":"2024-04-25T01:25:54.361657+0000","last_fullsized":"2024-04-25T01:25:54.361657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":70944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":70980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:25:57.628 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T01:25:57.629 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:25:57.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.782+0000 7fa84f6a5640 1 Processor -- start 2024-04-25T01:25:57.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.782+0000 7fa84f6a5640 1 -- start start 2024-04-25T01:25:57.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.783+0000 7fa84f6a5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa8481596a0 0x7fa848152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:57.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.783+0000 7fa84f6a5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa8480c72b0 0x7fa848160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:57.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.783+0000 7fa84f6a5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa848159b90 con 0x7fa8481592c0 2024-04-25T01:25:57.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.783+0000 7fa84f6a5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa8480c6e20 con 0x7fa8480c72b0 2024-04-25T01:25:57.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.783+0000 7fa84f6a5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa84806c970 con 0x7fa8481596a0 2024-04-25T01:25:57.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.783+0000 7fa84dc42640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa8480c72b0 0x7fa848160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:57.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.783+0000 7fa84dc42640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa8480c72b0 0x7fa848160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58072/0 (socket says 172.21.15.112:58072) 2024-04-25T01:25:57.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.783+0000 7fa84dc42640 1 -- 172.21.15.112:0/2065282582 learned_addr learned my addr 172.21.15.112:0/2065282582 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:57.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.783+0000 7fa84cc40640 1 --2- 172.21.15.112:0/2065282582 >> v2:172.21.15.187:3300/0 conn(0x7fa8481596a0 0x7fa848152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:57.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.784+0000 7fa84dc42640 1 -- 172.21.15.112:0/2065282582 >> v2:172.21.15.187:3300/0 conn(0x7fa8481596a0 msgr2=0x7fa848152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:57.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.784+0000 7fa84dc42640 1 --2- 172.21.15.112:0/2065282582 >> v2:172.21.15.187:3300/0 conn(0x7fa8481596a0 0x7fa848152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.784+0000 7fa84dc42640 1 -- 172.21.15.112:0/2065282582 >> v1:172.21.15.187:6789/0 conn(0x7fa8481592c0 legacy=0x7fa8480c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:57.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.784+0000 7fa84dc42640 1 -- 172.21.15.112:0/2065282582 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84806cc50 con 0x7fa8480c72b0 2024-04-25T01:25:57.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.784+0000 7fa84cc40640 1 --2- 172.21.15.112:0/2065282582 >> v2:172.21.15.187:3300/0 conn(0x7fa8481596a0 0x7fa848152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:25:57.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.784+0000 7fa84dc42640 1 --2- 172.21.15.112:0/2065282582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8480c72b0 0x7fa848160380 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa8480bea40 tx=0x7fa84002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3140bd07dd727222 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:57.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.784+0000 7fa847fff640 1 -- 172.21.15.112:0/2065282582 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840037070 con 0x7fa8480c72b0 2024-04-25T01:25:57.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.784+0000 7fa847fff640 1 -- 172.21.15.112:0/2065282582 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa840032440 con 0x7fa8480c72b0 2024-04-25T01:25:57.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.784+0000 7fa847fff640 1 -- 172.21.15.112:0/2065282582 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840032810 con 0x7fa8480c72b0 2024-04-25T01:25:57.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.785+0000 7fa84f6a5640 1 -- 172.21.15.112:0/2065282582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8480c72b0 msgr2=0x7fa848160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:57.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.785+0000 7fa84f6a5640 1 --2- 172.21.15.112:0/2065282582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8480c72b0 0x7fa848160380 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa8480bea40 tx=0x7fa84002a150 comp rx=0 tx=0).stop 2024-04-25T01:25:57.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.785+0000 7fa84f6a5640 1 -- 172.21.15.112:0/2065282582 shutdown_connections 2024-04-25T01:25:57.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.785+0000 7fa84f6a5640 1 --2- 172.21.15.112:0/2065282582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8480c72b0 0x7fa848160380 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.785+0000 7fa84f6a5640 1 --2- 172.21.15.112:0/2065282582 >> v2:172.21.15.187:3300/0 conn(0x7fa8481596a0 0x7fa848152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.785+0000 7fa84f6a5640 1 -- 172.21.15.112:0/2065282582 >> 172.21.15.112:0/2065282582 conn(0x7fa8480cb170 msgr2=0x7fa8480cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:57.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.785+0000 7fa84f6a5640 1 -- 172.21.15.112:0/2065282582 shutdown_connections 2024-04-25T01:25:57.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.785+0000 7fa84f6a5640 1 -- 172.21.15.112:0/2065282582 wait complete. 2024-04-25T01:25:57.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.786+0000 7fa84f6a5640 1 Processor -- start 2024-04-25T01:25:57.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.786+0000 7fa84f6a5640 1 -- start start 2024-04-25T01:25:57.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa84f6a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8480c72b0 0x7fa848151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:57.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa84f6a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8481592c0 0x7fa848154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:57.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa84f6a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8481596a0 0x7fa84814c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:57.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa84f6a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa84806c510 con 0x7fa8481596a0 2024-04-25T01:25:57.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa84f6a5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa84806c390 con 0x7fa8480c72b0 2024-04-25T01:25:57.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa84f6a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa84806db20 con 0x7fa8481592c0 2024-04-25T01:25:57.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa84dc42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8481596a0 0x7fa84814c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:57.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa84dc42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8481596a0 0x7fa84814c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58088/0 (socket says 172.21.15.112:58088) 2024-04-25T01:25:57.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa84dc42640 1 -- 172.21.15.112:0/515067329 learned_addr learned my addr 172.21.15.112:0/515067329 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:57.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa84cc40640 1 --2- 172.21.15.112:0/515067329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8481592c0 0x7fa848154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:57.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa84dc42640 1 -- 172.21.15.112:0/515067329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8481592c0 msgr2=0x7fa848154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:57.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa84dc42640 1 --2- 172.21.15.112:0/515067329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8481592c0 0x7fa848154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa84dc42640 1 -- 172.21.15.112:0/515067329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8480c72b0 msgr2=0x7fa848151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:57.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa84d441640 1 --2- 172.21.15.112:0/515067329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8480c72b0 0x7fa848151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:57.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa84dc42640 1 --2- 172.21.15.112:0/515067329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8480c72b0 0x7fa848151a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa84dc42640 1 -- 172.21.15.112:0/515067329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8481eebd0 con 0x7fa8481596a0 2024-04-25T01:25:57.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa84dc42640 1 --2- 172.21.15.112:0/515067329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8481596a0 0x7fa84814c470 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa84002a190 tx=0x7fa8400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:57.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa845ffb640 1 -- 172.21.15.112:0/515067329 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840041020 con 0x7fa8481596a0 2024-04-25T01:25:57.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa845ffb640 1 -- 172.21.15.112:0/515067329 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa840030740 con 0x7fa8481596a0 2024-04-25T01:25:57.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa845ffb640 1 -- 172.21.15.112:0/515067329 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840037040 con 0x7fa8481596a0 2024-04-25T01:25:57.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.787+0000 7fa84f6a5640 1 -- 172.21.15.112:0/515067329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8481eeeb0 con 0x7fa8481596a0 2024-04-25T01:25:57.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.788+0000 7fa84f6a5640 1 -- 172.21.15.112:0/515067329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8481ef470 con 0x7fa8481596a0 2024-04-25T01:25:57.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.788+0000 7fa84f6a5640 1 -- 172.21.15.112:0/515067329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8480679a0 con 0x7fa8481596a0 2024-04-25T01:25:57.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.789+0000 7fa845ffb640 1 -- 172.21.15.112:0/515067329 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa840044020 con 0x7fa8481596a0 2024-04-25T01:25:57.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.789+0000 7fa845ffb640 1 --2- 172.21.15.112:0/515067329 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa83406be30 0x7fa83406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:57.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.789+0000 7fa845ffb640 1 -- 172.21.15.112:0/515067329 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7fa8400ab6a0 con 0x7fa8481596a0 2024-04-25T01:25:57.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.789+0000 7fa84d441640 1 --2- 172.21.15.112:0/515067329 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa83406be30 0x7fa83406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:57.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.790+0000 7fa84d441640 1 --2- 172.21.15.112:0/515067329 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa83406be30 0x7fa83406e2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa838007fd0 tx=0x7fa838009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:57.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.793+0000 7fa845ffb640 1 -- 172.21.15.112:0/515067329 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa84004b030 con 0x7fa8481596a0 2024-04-25T01:25:57.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.969+0000 7fa84f6a5640 1 -- 172.21.15.112:0/515067329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8480682a0 con 0x7fa8481596a0 2024-04-25T01:25:57.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.970+0000 7fa845ffb640 1 -- 172.21.15.112:0/515067329 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16609 (secure 0 0 0) 0x7fa84007a060 con 0x7fa8481596a0 2024-04-25T01:25:57.950 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:25:57.950 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:25:54.355478+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:25:57.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.972+0000 7fa84f6a5640 1 -- 172.21.15.112:0/515067329 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa83406be30 msgr2=0x7fa83406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.972+0000 7fa84f6a5640 1 --2- 172.21.15.112:0/515067329 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa83406be30 0x7fa83406e2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa838007fd0 tx=0x7fa838009040 comp rx=0 tx=0).stop 2024-04-25T01:25:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.972+0000 7fa84f6a5640 1 -- 172.21.15.112:0/515067329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8481596a0 msgr2=0x7fa84814c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.972+0000 7fa84f6a5640 1 --2- 172.21.15.112:0/515067329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8481596a0 0x7fa84814c470 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa84002a190 tx=0x7fa8400046c0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.972+0000 7fa84f6a5640 1 -- 172.21.15.112:0/515067329 shutdown_connections 2024-04-25T01:25:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.972+0000 7fa84f6a5640 1 --2- 172.21.15.112:0/515067329 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa83406be30 0x7fa83406e2f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.972+0000 7fa84f6a5640 1 --2- 172.21.15.112:0/515067329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8481596a0 0x7fa84814c470 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.972+0000 7fa84f6a5640 1 --2- 172.21.15.112:0/515067329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8481592c0 0x7fa848154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.972+0000 7fa84f6a5640 1 --2- 172.21.15.112:0/515067329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8480c72b0 0x7fa848151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.972+0000 7fa84f6a5640 1 -- 172.21.15.112:0/515067329 >> 172.21.15.112:0/515067329 conn(0x7fa8480cb170 msgr2=0x7fa848155e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.972+0000 7fa84f6a5640 1 -- 172.21.15.112:0/515067329 shutdown_connections 2024-04-25T01:25:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:57.972+0000 7fa84f6a5640 1 -- 172.21.15.112:0/515067329 wait complete. 2024-04-25T01:25:57.962 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:25:40.679119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-04-25T01:25:50.593045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-04-25T01:25:51.664930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:25:57.963 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:25:58.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.082+0000 7fe3013b0640 1 Processor -- start 2024-04-25T01:25:58.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.082+0000 7fe3013b0640 1 -- start start 2024-04-25T01:25:58.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.083+0000 7fe3013b0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2fc0ca8a0 0x7fe2fc06c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.083+0000 7fe3013b0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2fc1534a0 con 0x7fe2fc0ca370 2024-04-25T01:25:58.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.083+0000 7fe3013b0640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2fc0cae30 con 0x7fe2fc0c9250 2024-04-25T01:25:58.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.083+0000 7fe3013b0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2fc156990 con 0x7fe2fc0ca8a0 2024-04-25T01:25:58.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.083+0000 7fe2f3fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe2fc0ca370 0x7fe2fc15e2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37160/0 (socket says 172.21.15.112:37160) 2024-04-25T01:25:58.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.083+0000 7fe2f3fff640 1 -- 172.21.15.112:0/2949795996 learned_addr learned my addr 172.21.15.112:0/2949795996 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:25:58.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.083+0000 7fe2fb7fe640 1 --2- 172.21.15.112:0/2949795996 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2fc0ca8a0 0x7fe2fc06c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:58.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.083+0000 7fe2fa7fc640 1 -- 172.21.15.112:0/2949795996 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1236715572 0 0) 0x7fe2fc1534a0 con 0x7fe2fc0ca370 2024-04-25T01:25:58.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.083+0000 7fe2fa7fc640 1 -- 172.21.15.112:0/2949795996 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2e8003520 con 0x7fe2fc0ca370 2024-04-25T01:25:58.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.083+0000 7fe2fa7fc640 1 -- 172.21.15.112:0/2949795996 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1136713322 0 0) 0x7fe2fc0cae30 con 0x7fe2fc0c9250 2024-04-25T01:25:58.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.083+0000 7fe2fa7fc640 1 -- 172.21.15.112:0/2949795996 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2fc1534a0 con 0x7fe2fc0c9250 2024-04-25T01:25:58.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.083+0000 7fe2fb7fe640 1 -- 172.21.15.112:0/2949795996 >> v1:172.21.15.187:6789/0 conn(0x7fe2fc0c9250 legacy=0x7fe2fc06bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.083+0000 7fe2fb7fe640 1 -- 172.21.15.112:0/2949795996 >> v1:172.21.15.112:6789/0 conn(0x7fe2fc0ca370 legacy=0x7fe2fc15e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.083+0000 7fe2fb7fe640 1 -- 172.21.15.112:0/2949795996 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2fc0c9cd0 con 0x7fe2fc0ca8a0 2024-04-25T01:25:58.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.084+0000 7fe2fb7fe640 1 --2- 172.21.15.112:0/2949795996 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2fc0ca8a0 0x7fe2fc06c420 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4009dc0 tx=0x7fe2f402a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7992f8e959dd34f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:58.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.084+0000 7fe2fa7fc640 1 -- 172.21.15.112:0/2949795996 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f402ae60 con 0x7fe2fc0ca8a0 2024-04-25T01:25:58.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.084+0000 7fe2fa7fc640 1 -- 172.21.15.112:0/2949795996 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f4007740 con 0x7fe2fc0ca8a0 2024-04-25T01:25:58.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.084+0000 7fe2fa7fc640 1 -- 172.21.15.112:0/2949795996 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4007af0 con 0x7fe2fc0ca8a0 2024-04-25T01:25:58.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.085+0000 7fe3013b0640 1 -- 172.21.15.112:0/2949795996 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2fc0ca8a0 msgr2=0x7fe2fc06c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.085+0000 7fe3013b0640 1 --2- 172.21.15.112:0/2949795996 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2fc0ca8a0 0x7fe2fc06c420 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4009dc0 tx=0x7fe2f402a640 comp rx=0 tx=0).stop 2024-04-25T01:25:58.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.085+0000 7fe3013b0640 1 -- 172.21.15.112:0/2949795996 shutdown_connections 2024-04-25T01:25:58.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.085+0000 7fe3013b0640 1 --2- 172.21.15.112:0/2949795996 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2fc0ca8a0 0x7fe2fc06c420 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.085+0000 7fe3013b0640 1 -- 172.21.15.112:0/2949795996 >> 172.21.15.112:0/2949795996 conn(0x7fe2fc0cb310 msgr2=0x7fe2fc0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:58.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.085+0000 7fe3013b0640 1 -- 172.21.15.112:0/2949795996 shutdown_connections 2024-04-25T01:25:58.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.085+0000 7fe3013b0640 1 -- 172.21.15.112:0/2949795996 wait complete. 2024-04-25T01:25:58.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.086+0000 7fe3013b0640 1 Processor -- start 2024-04-25T01:25:58.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.086+0000 7fe3013b0640 1 -- start start 2024-04-25T01:25:58.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.086+0000 7fe3013b0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2fc0c9250 0x7fe2fc0c0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.086+0000 7fe3013b0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2fc0ca370 0x7fe2fc0ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.086+0000 7fe3013b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2fc0ca8a0 0x7fe2fc1eedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.086+0000 7fe3013b0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc155eb0 con 0x7fe2fc0ca8a0 2024-04-25T01:25:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.086+0000 7fe3013b0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc156500 con 0x7fe2fc0ca370 2024-04-25T01:25:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.086+0000 7fe3013b0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2fc156190 con 0x7fe2fc0c9250 2024-04-25T01:25:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.087+0000 7fe2fb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2fc0ca8a0 0x7fe2fc1eedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.087+0000 7fe2fb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2fc0ca8a0 0x7fe2fc1eedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58104/0 (socket says 172.21.15.112:58104) 2024-04-25T01:25:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.087+0000 7fe2fb7fe640 1 -- 172.21.15.112:0/1932595871 learned_addr learned my addr 172.21.15.112:0/1932595871 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.087+0000 7fe2faffd640 1 --2- 172.21.15.112:0/1932595871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2fc0c9250 0x7fe2fc0c0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.087+0000 7fe2f3fff640 1 --2- 172.21.15.112:0/1932595871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2fc0ca370 0x7fe2fc0ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.087+0000 7fe2fb7fe640 1 -- 172.21.15.112:0/1932595871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2fc0c9250 msgr2=0x7fe2fc0c0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.087+0000 7fe2fb7fe640 1 --2- 172.21.15.112:0/1932595871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2fc0c9250 0x7fe2fc0c0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.087+0000 7fe2fb7fe640 1 -- 172.21.15.112:0/1932595871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2fc0ca370 msgr2=0x7fe2fc0ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.087+0000 7fe2fb7fe640 1 --2- 172.21.15.112:0/1932595871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2fc0ca370 0x7fe2fc0ac160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.087+0000 7fe2fb7fe640 1 -- 172.21.15.112:0/1932595871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2fc0c06b0 con 0x7fe2fc0ca8a0 2024-04-25T01:25:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.087+0000 7fe2fb7fe640 1 --2- 172.21.15.112:0/1932595871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2fc0ca8a0 0x7fe2fc1eedf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe2f402ac30 tx=0x7fe2f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.087+0000 7fe2f37fe640 1 -- 172.21.15.112:0/1932595871 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f403f070 con 0x7fe2fc0ca8a0 2024-04-25T01:25:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.087+0000 7fe2f37fe640 1 -- 172.21.15.112:0/1932595871 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f4031a40 con 0x7fe2fc0ca8a0 2024-04-25T01:25:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.087+0000 7fe3013b0640 1 -- 172.21.15.112:0/1932595871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2fc0bf380 con 0x7fe2fc0ca8a0 2024-04-25T01:25:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.087+0000 7fe2f37fe640 1 -- 172.21.15.112:0/1932595871 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4009450 con 0x7fe2fc0ca8a0 2024-04-25T01:25:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.087+0000 7fe3013b0640 1 -- 172.21.15.112:0/1932595871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2fc1ee8d0 con 0x7fe2fc0ca8a0 2024-04-25T01:25:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.088+0000 7fe3013b0640 1 -- 172.21.15.112:0/1932595871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2fc04f500 con 0x7fe2fc0ca8a0 2024-04-25T01:25:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.088+0000 7fe2f37fe640 1 -- 172.21.15.112:0/1932595871 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe2f4031c30 con 0x7fe2fc0ca8a0 2024-04-25T01:25:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.089+0000 7fe2f37fe640 1 --2- 172.21.15.112:0/1932595871 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe2dc06be30 0x7fe2dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.089+0000 7fe2f37fe640 1 -- 172.21.15.112:0/1932595871 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7fe2f40ac320 con 0x7fe2fc0ca8a0 2024-04-25T01:25:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.089+0000 7fe2faffd640 1 --2- 172.21.15.112:0/1932595871 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe2dc06be30 0x7fe2dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.089+0000 7fe2faffd640 1 --2- 172.21.15.112:0/1932595871 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe2dc06be30 0x7fe2dc06e2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0011e0 tx=0x7fe2ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:58.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.093+0000 7fe2f37fe640 1 -- 172.21.15.112:0/1932595871 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe2f407a820 con 0x7fe2fc0ca8a0 2024-04-25T01:25:58.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.267+0000 7fe3013b0640 1 -- 172.21.15.112:0/1932595871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe2fc06d970 con 0x7fe2fc0ca8a0 2024-04-25T01:25:58.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.267+0000 7fe2f37fe640 1 -- 172.21.15.112:0/1932595871 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fe2f407eb20 con 0x7fe2fc0ca8a0 2024-04-25T01:25:58.247 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T01:25:58.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.269+0000 7fe3013b0640 1 -- 172.21.15.112:0/1932595871 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe2dc06be30 msgr2=0x7fe2dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.269+0000 7fe3013b0640 1 --2- 172.21.15.112:0/1932595871 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe2dc06be30 0x7fe2dc06e2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0011e0 tx=0x7fe2ec009040 comp rx=0 tx=0).stop 2024-04-25T01:25:58.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.269+0000 7fe3013b0640 1 -- 172.21.15.112:0/1932595871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2fc0ca8a0 msgr2=0x7fe2fc1eedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.269+0000 7fe3013b0640 1 --2- 172.21.15.112:0/1932595871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2fc0ca8a0 0x7fe2fc1eedf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe2f402ac30 tx=0x7fe2f4004a40 comp rx=0 tx=0).stop 2024-04-25T01:25:58.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.269+0000 7fe3013b0640 1 -- 172.21.15.112:0/1932595871 shutdown_connections 2024-04-25T01:25:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.269+0000 7fe3013b0640 1 --2- 172.21.15.112:0/1932595871 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe2dc06be30 0x7fe2dc06e2f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.269+0000 7fe3013b0640 1 --2- 172.21.15.112:0/1932595871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2fc0ca8a0 0x7fe2fc1eedf0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.269+0000 7fe3013b0640 1 --2- 172.21.15.112:0/1932595871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2fc0ca370 0x7fe2fc0ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.269+0000 7fe3013b0640 1 --2- 172.21.15.112:0/1932595871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2fc0c9250 0x7fe2fc0c0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.269+0000 7fe3013b0640 1 -- 172.21.15.112:0/1932595871 >> 172.21.15.112:0/1932595871 conn(0x7fe2fc0cb310 msgr2=0x7fe2fc1553b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.269+0000 7fe3013b0640 1 -- 172.21.15.112:0/1932595871 shutdown_connections 2024-04-25T01:25:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.269+0000 7fe3013b0640 1 -- 172.21.15.112:0/1932595871 wait complete. 2024-04-25T01:25:58.259 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-04-25T01:25:58.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.379+0000 7f8829a10640 1 Processor -- start 2024-04-25T01:25:58.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.379+0000 7f8829a10640 1 -- start start 2024-04-25T01:25:58.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.380+0000 7f8829a10640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f88240ca370 0x7f882406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.380+0000 7f8829a10640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f88240ca8a0 0x7f88240c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.380+0000 7f8829a10640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88241534a0 con 0x7f88240c9250 2024-04-25T01:25:58.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.380+0000 7f8829a10640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f8824156990 con 0x7f88240ca370 2024-04-25T01:25:58.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.380+0000 7f8829a10640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8824156500 con 0x7f88240ca8a0 2024-04-25T01:25:58.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.380+0000 7f881bfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f88240ca370 0x7f882406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:58.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.380+0000 7f881bfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f88240ca370 0x7f882406c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58118/0 (socket says 172.21.15.112:58118) 2024-04-25T01:25:58.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.380+0000 7f881bfff640 1 -- 172.21.15.112:0/1645311631 learned_addr learned my addr 172.21.15.112:0/1645311631 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:58.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.380+0000 7f88237fe640 1 --2- 172.21.15.112:0/1645311631 >> v2:172.21.15.187:3300/0 conn(0x7f88240ca8a0 0x7f88240c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:58.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.380+0000 7f881bfff640 1 -- 172.21.15.112:0/1645311631 >> v2:172.21.15.187:3300/0 conn(0x7f88240ca8a0 msgr2=0x7f88240c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:58.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.380+0000 7f881bfff640 1 --2- 172.21.15.112:0/1645311631 >> v2:172.21.15.187:3300/0 conn(0x7f88240ca8a0 0x7f88240c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.380+0000 7f881bfff640 1 -- 172.21.15.112:0/1645311631 >> v1:172.21.15.187:6789/0 conn(0x7f88240c9250 legacy=0x7f882406bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.380+0000 7f881bfff640 1 -- 172.21.15.112:0/1645311631 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88240cae30 con 0x7f88240ca370 2024-04-25T01:25:58.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.381+0000 7f881bfff640 1 --2- 172.21.15.112:0/1645311631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88240ca370 0x7f882406c420 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f88240c8bc0 tx=0x7f880c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba18248425b30dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:58.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.381+0000 7f88227fc640 1 -- 172.21.15.112:0/1645311631 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c037070 con 0x7f88240ca370 2024-04-25T01:25:58.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.381+0000 7f88227fc640 1 -- 172.21.15.112:0/1645311631 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f880c032440 con 0x7f88240ca370 2024-04-25T01:25:58.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.381+0000 7f88227fc640 1 -- 172.21.15.112:0/1645311631 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c032810 con 0x7f88240ca370 2024-04-25T01:25:58.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.382+0000 7f8829a10640 1 -- 172.21.15.112:0/1645311631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88240ca370 msgr2=0x7f882406c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.382+0000 7f8829a10640 1 --2- 172.21.15.112:0/1645311631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88240ca370 0x7f882406c420 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f88240c8bc0 tx=0x7f880c02a1e0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.382+0000 7f8829a10640 1 -- 172.21.15.112:0/1645311631 shutdown_connections 2024-04-25T01:25:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.382+0000 7f8829a10640 1 --2- 172.21.15.112:0/1645311631 >> v2:172.21.15.187:3300/0 conn(0x7f88240ca8a0 0x7f88240c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.382+0000 7f8829a10640 1 --2- 172.21.15.112:0/1645311631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88240ca370 0x7f882406c420 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.382+0000 7f8829a10640 1 -- 172.21.15.112:0/1645311631 >> 172.21.15.112:0/1645311631 conn(0x7f88240cb310 msgr2=0x7f88240cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.382+0000 7f8829a10640 1 -- 172.21.15.112:0/1645311631 shutdown_connections 2024-04-25T01:25:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.382+0000 7f8829a10640 1 -- 172.21.15.112:0/1645311631 wait complete. 2024-04-25T01:25:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.383+0000 7f8829a10640 1 Processor -- start 2024-04-25T01:25:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.383+0000 7f8829a10640 1 -- start start 2024-04-25T01:25:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.383+0000 7f8829a10640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88240c9250 0x7f8824153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.383+0000 7f8829a10640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88240ca370 0x7f8824150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.383+0000 7f8829a10640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88240ca8a0 0x7f8824150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.383+0000 7f8829a10640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8824155eb0 con 0x7f88240ca8a0 2024-04-25T01:25:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.383+0000 7f8829a10640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8824155bd0 con 0x7f88240ca370 2024-04-25T01:25:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.383+0000 7f8829a10640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8824156190 con 0x7f88240c9250 2024-04-25T01:25:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.383+0000 7f88237fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88240ca8a0 0x7f8824150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.384+0000 7f88237fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88240ca8a0 0x7f8824150940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58128/0 (socket says 172.21.15.112:58128) 2024-04-25T01:25:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.384+0000 7f88237fe640 1 -- 172.21.15.112:0/3155814728 learned_addr learned my addr 172.21.15.112:0/3155814728 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.384+0000 7f881bfff640 1 --2- 172.21.15.112:0/3155814728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88240ca370 0x7f8824150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.384+0000 7f8822ffd640 1 --2- 172.21.15.112:0/3155814728 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88240c9250 0x7f8824153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.384+0000 7f88237fe640 1 -- 172.21.15.112:0/3155814728 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88240c9250 msgr2=0x7f8824153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.384+0000 7f88237fe640 1 --2- 172.21.15.112:0/3155814728 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88240c9250 0x7f8824153b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.384+0000 7f88237fe640 1 -- 172.21.15.112:0/3155814728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88240ca370 msgr2=0x7f8824150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.384+0000 7f88237fe640 1 --2- 172.21.15.112:0/3155814728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88240ca370 0x7f8824150400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.384+0000 7f88237fe640 1 -- 172.21.15.112:0/3155814728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88240bec80 con 0x7f88240ca8a0 2024-04-25T01:25:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.384+0000 7f881bfff640 1 --2- 172.21.15.112:0/3155814728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88240ca370 0x7f8824150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:25:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.384+0000 7f88237fe640 1 --2- 172.21.15.112:0/3155814728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88240ca8a0 0x7f8824150940 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f881c009d30 tx=0x7f881c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.384+0000 7f8820ff9640 1 -- 172.21.15.112:0/3155814728 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c018020 con 0x7f88240ca8a0 2024-04-25T01:25:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.384+0000 7f8820ff9640 1 -- 172.21.15.112:0/3155814728 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f881c014480 con 0x7f88240ca8a0 2024-04-25T01:25:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.384+0000 7f8829a10640 1 -- 172.21.15.112:0/3155814728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88241ef620 con 0x7f88240ca8a0 2024-04-25T01:25:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.384+0000 7f8820ff9640 1 -- 172.21.15.112:0/3155814728 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c014870 con 0x7f88240ca8a0 2024-04-25T01:25:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.384+0000 7f8829a10640 1 -- 172.21.15.112:0/3155814728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88241ef8c0 con 0x7f88240ca8a0 2024-04-25T01:25:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.385+0000 7f8829a10640 1 -- 172.21.15.112:0/3155814728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f882404f500 con 0x7f88240ca8a0 2024-04-25T01:25:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.385+0000 7f8820ff9640 1 -- 172.21.15.112:0/3155814728 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f881c005d10 con 0x7f88240ca8a0 2024-04-25T01:25:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.386+0000 7f8820ff9640 1 --2- 172.21.15.112:0/3155814728 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f880406be30 0x7f880406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.386+0000 7f8822ffd640 1 --2- 172.21.15.112:0/3155814728 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f880406be30 0x7f880406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.386+0000 7f8820ff9640 1 -- 172.21.15.112:0/3155814728 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7f881c08b7c0 con 0x7f88240ca8a0 2024-04-25T01:25:58.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.386+0000 7f8822ffd640 1 --2- 172.21.15.112:0/3155814728 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f880406be30 0x7f880406e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8814000f60 tx=0x7f8814009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:58.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.390+0000 7f8820ff9640 1 -- 172.21.15.112:0/3155814728 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f881c022070 con 0x7f88240ca8a0 2024-04-25T01:25:58.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.562+0000 7f8829a10640 1 -- 172.21.15.112:0/3155814728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) -- 0x7f882406d970 con 0x7f88240ca8a0 2024-04-25T01:25:58.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.563+0000 7f8820ff9640 1 -- 172.21.15.112:0/3155814728 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v20) ==== 99+0+11 (secure 0 0 0) 0x7f88240bec80 con 0x7f88240ca8a0 2024-04-25T01:25:58.543 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T01:25:58.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.565+0000 7f8829a10640 1 -- 172.21.15.112:0/3155814728 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f880406be30 msgr2=0x7f880406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.565+0000 7f8829a10640 1 --2- 172.21.15.112:0/3155814728 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f880406be30 0x7f880406e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8814000f60 tx=0x7f8814009040 comp rx=0 tx=0).stop 2024-04-25T01:25:58.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.565+0000 7f8829a10640 1 -- 172.21.15.112:0/3155814728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88240ca8a0 msgr2=0x7f8824150940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.565+0000 7f8829a10640 1 --2- 172.21.15.112:0/3155814728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88240ca8a0 0x7f8824150940 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f881c009d30 tx=0x7f881c00c490 comp rx=0 tx=0).stop 2024-04-25T01:25:58.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.565+0000 7f8829a10640 1 -- 172.21.15.112:0/3155814728 shutdown_connections 2024-04-25T01:25:58.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.565+0000 7f8829a10640 1 --2- 172.21.15.112:0/3155814728 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f880406be30 0x7f880406e2f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.565+0000 7f8829a10640 1 --2- 172.21.15.112:0/3155814728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88240ca8a0 0x7f8824150940 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.565+0000 7f8829a10640 1 --2- 172.21.15.112:0/3155814728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88240ca370 0x7f8824150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.565+0000 7f8829a10640 1 --2- 172.21.15.112:0/3155814728 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88240c9250 0x7f8824153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.565+0000 7f8829a10640 1 -- 172.21.15.112:0/3155814728 >> 172.21.15.112:0/3155814728 conn(0x7f88240cb310 msgr2=0x7f882406c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:58.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.565+0000 7f8829a10640 1 -- 172.21.15.112:0/3155814728 shutdown_connections 2024-04-25T01:25:58.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.565+0000 7f8829a10640 1 -- 172.21.15.112:0/3155814728 wait complete. 2024-04-25T01:25:58.555 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-04-25T01:25:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.676+0000 7fc910b5c640 1 Processor -- start 2024-04-25T01:25:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.676+0000 7fc910b5c640 1 -- start start 2024-04-25T01:25:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.677+0000 7fc910b5c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc90c0c9250 0x7fc90c0c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.677+0000 7fc910b5c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc90c0ca8a0 0x7fc90c06c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.677+0000 7fc910b5c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc90c1534a0 con 0x7fc90c0ca370 2024-04-25T01:25:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.677+0000 7fc910b5c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fc90c156990 con 0x7fc90c0ca8a0 2024-04-25T01:25:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.677+0000 7fc910b5c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc90c156500 con 0x7fc90c0c9250 2024-04-25T01:25:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.677+0000 7fc90ad9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc90c0ca8a0 0x7fc90c06c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.677+0000 7fc909d9b640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc90c0ca370 0x7fc90c06bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37172/0 (socket says 172.21.15.112:37172) 2024-04-25T01:25:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.677+0000 7fc909d9b640 1 -- 172.21.15.112:0/2605906256 learned_addr learned my addr 172.21.15.112:0/2605906256 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:25:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.677+0000 7fc90a59c640 1 --2- 172.21.15.112:0/2605906256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc90c0c9250 0x7fc90c0c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.678+0000 7fc90ad9d640 1 -- 172.21.15.112:0/2605906256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc90c0c9250 msgr2=0x7fc90c0c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.678+0000 7fc90ad9d640 1 --2- 172.21.15.112:0/2605906256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc90c0c9250 0x7fc90c0c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.678+0000 7fc90ad9d640 1 -- 172.21.15.112:0/2605906256 >> v1:172.21.15.112:6789/0 conn(0x7fc90c0ca370 legacy=0x7fc90c06bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.678+0000 7fc90959a640 1 -- 172.21.15.112:0/2605906256 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2547168307 0 0) 0x7fc90c1534a0 con 0x7fc90c0ca370 2024-04-25T01:25:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.678+0000 7fc90ad9d640 1 -- 172.21.15.112:0/2605906256 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c0cae30 con 0x7fc90c0ca8a0 2024-04-25T01:25:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.678+0000 7fc90a59c640 1 --2- 172.21.15.112:0/2605906256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc90c0c9250 0x7fc90c0c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:25:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.678+0000 7fc90ad9d640 1 --2- 172.21.15.112:0/2605906256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0ca8a0 0x7fc90c06c420 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc9040097a0 tx=0x7fc90402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29fac13d738be32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.678+0000 7fc90959a640 1 -- 172.21.15.112:0/2605906256 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904037070 con 0x7fc90c0ca8a0 2024-04-25T01:25:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.678+0000 7fc90959a640 1 -- 172.21.15.112:0/2605906256 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc90402adb0 con 0x7fc90c0ca8a0 2024-04-25T01:25:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.678+0000 7fc90959a640 1 -- 172.21.15.112:0/2605906256 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904032660 con 0x7fc90c0ca8a0 2024-04-25T01:25:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.679+0000 7fc910b5c640 1 -- 172.21.15.112:0/2605906256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0ca8a0 msgr2=0x7fc90c06c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.679+0000 7fc910b5c640 1 --2- 172.21.15.112:0/2605906256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0ca8a0 0x7fc90c06c420 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc9040097a0 tx=0x7fc90402a060 comp rx=0 tx=0).stop 2024-04-25T01:25:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.679+0000 7fc910b5c640 1 -- 172.21.15.112:0/2605906256 shutdown_connections 2024-04-25T01:25:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.679+0000 7fc910b5c640 1 --2- 172.21.15.112:0/2605906256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0ca8a0 0x7fc90c06c420 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.679+0000 7fc910b5c640 1 --2- 172.21.15.112:0/2605906256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc90c0c9250 0x7fc90c0c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.679+0000 7fc910b5c640 1 -- 172.21.15.112:0/2605906256 >> 172.21.15.112:0/2605906256 conn(0x7fc90c0cb310 msgr2=0x7fc90c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.679+0000 7fc910b5c640 1 -- 172.21.15.112:0/2605906256 shutdown_connections 2024-04-25T01:25:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.679+0000 7fc910b5c640 1 -- 172.21.15.112:0/2605906256 wait complete. 2024-04-25T01:25:58.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.680+0000 7fc910b5c640 1 Processor -- start 2024-04-25T01:25:58.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.680+0000 7fc910b5c640 1 -- start start 2024-04-25T01:25:58.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.681+0000 7fc910b5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0c9250 0x7fc90c146e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.681+0000 7fc910b5c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc90c0ca370 0x7fc90c0c1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.681+0000 7fc910b5c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc90c0ca8a0 0x7fc90c0c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.681+0000 7fc910b5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c155eb0 con 0x7fc90c0c9250 2024-04-25T01:25:58.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.681+0000 7fc910b5c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c155bd0 con 0x7fc90c0ca8a0 2024-04-25T01:25:58.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.681+0000 7fc910b5c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc90c156190 con 0x7fc90c0ca370 2024-04-25T01:25:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.681+0000 7fc90a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0c9250 0x7fc90c146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.681+0000 7fc90a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0c9250 0x7fc90c146e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58142/0 (socket says 172.21.15.112:58142) 2024-04-25T01:25:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.681+0000 7fc90a59c640 1 -- 172.21.15.112:0/4001350867 learned_addr learned my addr 172.21.15.112:0/4001350867 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.681+0000 7fc909d9b640 1 --2- 172.21.15.112:0/4001350867 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc90c0ca370 0x7fc90c0c1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.681+0000 7fc90ad9d640 1 --2- 172.21.15.112:0/4001350867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc90c0ca8a0 0x7fc90c0c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.681+0000 7fc90a59c640 1 -- 172.21.15.112:0/4001350867 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc90c0ca370 msgr2=0x7fc90c0c1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.681+0000 7fc90a59c640 1 --2- 172.21.15.112:0/4001350867 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc90c0ca370 0x7fc90c0c1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.681+0000 7fc90a59c640 1 -- 172.21.15.112:0/4001350867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc90c0ca8a0 msgr2=0x7fc90c0c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.681+0000 7fc90a59c640 1 --2- 172.21.15.112:0/4001350867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc90c0ca8a0 0x7fc90c0c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.681+0000 7fc90a59c640 1 -- 172.21.15.112:0/4001350867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c14b540 con 0x7fc90c0c9250 2024-04-25T01:25:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.681+0000 7fc90a59c640 1 --2- 172.21.15.112:0/4001350867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0c9250 0x7fc90c146e70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00cf60 tx=0x7fc8fc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.682+0000 7fc8faffd640 1 -- 172.21.15.112:0/4001350867 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc01a070 con 0x7fc90c0c9250 2024-04-25T01:25:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.682+0000 7fc8faffd640 1 -- 172.21.15.112:0/4001350867 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8fc00d840 con 0x7fc90c0c9250 2024-04-25T01:25:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.682+0000 7fc8faffd640 1 -- 172.21.15.112:0/4001350867 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc00dc10 con 0x7fc90c0c9250 2024-04-25T01:25:58.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.682+0000 7fc910b5c640 1 -- 172.21.15.112:0/4001350867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90c0c42c0 con 0x7fc90c0c9250 2024-04-25T01:25:58.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.682+0000 7fc910b5c640 1 -- 172.21.15.112:0/4001350867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90c0c4880 con 0x7fc90c0c9250 2024-04-25T01:25:58.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.683+0000 7fc910b5c640 1 -- 172.21.15.112:0/4001350867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc90c0689b0 con 0x7fc90c0c9250 2024-04-25T01:25:58.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.683+0000 7fc8faffd640 1 -- 172.21.15.112:0/4001350867 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc8fc01d070 con 0x7fc90c0c9250 2024-04-25T01:25:58.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.683+0000 7fc8faffd640 1 --2- 172.21.15.112:0/4001350867 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc8ec06be30 0x7fc8ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.683+0000 7fc8faffd640 1 -- 172.21.15.112:0/4001350867 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7fc8fc08b6a0 con 0x7fc90c0c9250 2024-04-25T01:25:58.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.683+0000 7fc909d9b640 1 --2- 172.21.15.112:0/4001350867 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc8ec06be30 0x7fc8ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:58.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.684+0000 7fc909d9b640 1 --2- 172.21.15.112:0/4001350867 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc8ec06be30 0x7fc8ec06e2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc90c152210 tx=0x7fc900007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:58.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.687+0000 7fc8faffd640 1 -- 172.21.15.112:0/4001350867 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc90c0689b0 con 0x7fc90c0c9250 2024-04-25T01:25:58.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.860+0000 7fc910b5c640 1 -- 172.21.15.112:0/4001350867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) -- 0x7fc90c04f500 con 0x7fc90c0c9250 2024-04-25T01:25:58.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.860+0000 7fc8faffd640 1 -- 172.21.15.112:0/4001350867 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v20) ==== 95+0+11 (secure 0 0 0) 0x7fc8fc059a10 con 0x7fc90c0c9250 2024-04-25T01:25:58.840 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T01:25:58.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.862+0000 7fc910b5c640 1 -- 172.21.15.112:0/4001350867 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc8ec06be30 msgr2=0x7fc8ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.862+0000 7fc910b5c640 1 --2- 172.21.15.112:0/4001350867 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc8ec06be30 0x7fc8ec06e2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc90c152210 tx=0x7fc900007b50 comp rx=0 tx=0).stop 2024-04-25T01:25:58.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.862+0000 7fc910b5c640 1 -- 172.21.15.112:0/4001350867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0c9250 msgr2=0x7fc90c146e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.862+0000 7fc910b5c640 1 --2- 172.21.15.112:0/4001350867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0c9250 0x7fc90c146e70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00cf60 tx=0x7fc8fc00cf90 comp rx=0 tx=0).stop 2024-04-25T01:25:58.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.862+0000 7fc910b5c640 1 -- 172.21.15.112:0/4001350867 shutdown_connections 2024-04-25T01:25:58.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.862+0000 7fc910b5c640 1 --2- 172.21.15.112:0/4001350867 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc8ec06be30 0x7fc8ec06e2f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.862+0000 7fc910b5c640 1 --2- 172.21.15.112:0/4001350867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc90c0ca8a0 0x7fc90c0c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.862+0000 7fc910b5c640 1 --2- 172.21.15.112:0/4001350867 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc90c0ca370 0x7fc90c0c1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.862+0000 7fc910b5c640 1 --2- 172.21.15.112:0/4001350867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0c9250 0x7fc90c146e70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.862+0000 7fc910b5c640 1 -- 172.21.15.112:0/4001350867 >> 172.21.15.112:0/4001350867 conn(0x7fc90c0cb310 msgr2=0x7fc90c155360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:58.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.863+0000 7fc910b5c640 1 -- 172.21.15.112:0/4001350867 shutdown_connections 2024-04-25T01:25:58.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.863+0000 7fc910b5c640 1 -- 172.21.15.112:0/4001350867 wait complete. 2024-04-25T01:25:58.855 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T01:25:58.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.972+0000 7f3dd474f640 1 Processor -- start 2024-04-25T01:25:58.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.972+0000 7f3dd474f640 1 -- start start 2024-04-25T01:25:58.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.972+0000 7f3dd474f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3dcc1524a0 0x7f3dcc15de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.972+0000 7f3dd474f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dcc1529b0 con 0x7f3dcc0ac3b0 2024-04-25T01:25:58.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.973+0000 7f3dd474f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dcc152c70 con 0x7f3dcc14c3f0 2024-04-25T01:25:58.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.973+0000 7f3dd474f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f3dcc155a90 con 0x7f3dcc1524a0 2024-04-25T01:25:58.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.973+0000 7f3dd2cec640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3dcc1524a0 0x7f3dcc15de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:58.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.973+0000 7f3dd1cea640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3dcc0ac3b0 0x7f3dcc155f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37176/0 (socket says 172.21.15.112:37176) 2024-04-25T01:25:58.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.973+0000 7f3dd1cea640 1 -- 172.21.15.112:0/58569919 learned_addr learned my addr 172.21.15.112:0/58569919 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:25:58.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.973+0000 7f3dd2cec640 1 -- 172.21.15.112:0/58569919 >> v1:172.21.15.187:6789/0 conn(0x7f3dcc14c3f0 legacy=0x7f3dcc0c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:58.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.973+0000 7f3dd2cec640 1 -- 172.21.15.112:0/58569919 >> v1:172.21.15.112:6789/0 conn(0x7f3dcc0ac3b0 legacy=0x7f3dcc155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.973+0000 7f3dd14e9640 1 -- 172.21.15.112:0/58569919 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2784846746 0 0) 0x7f3dcc1529b0 con 0x7f3dcc0ac3b0 2024-04-25T01:25:58.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.973+0000 7f3dd2cec640 1 -- 172.21.15.112:0/58569919 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc152f50 con 0x7f3dcc1524a0 2024-04-25T01:25:58.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.974+0000 7f3dd2cec640 1 --2- 172.21.15.112:0/58569919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dcc1524a0 0x7f3dcc15de30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009d90 tx=0x7f3dc402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be73d961d5bccce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:58.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.974+0000 7f3dd14e9640 1 -- 172.21.15.112:0/58569919 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4031600 con 0x7f3dcc1524a0 2024-04-25T01:25:58.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.974+0000 7f3dd14e9640 1 -- 172.21.15.112:0/58569919 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc40317a0 con 0x7f3dcc1524a0 2024-04-25T01:25:58.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.974+0000 7f3dd14e9640 1 -- 172.21.15.112:0/58569919 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4031b70 con 0x7f3dcc1524a0 2024-04-25T01:25:58.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.974+0000 7f3dd474f640 1 -- 172.21.15.112:0/58569919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dcc1524a0 msgr2=0x7f3dcc15de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.974+0000 7f3dd474f640 1 --2- 172.21.15.112:0/58569919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dcc1524a0 0x7f3dcc15de30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009d90 tx=0x7f3dc402a610 comp rx=0 tx=0).stop 2024-04-25T01:25:58.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.974+0000 7f3dd474f640 1 -- 172.21.15.112:0/58569919 shutdown_connections 2024-04-25T01:25:58.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.974+0000 7f3dd474f640 1 --2- 172.21.15.112:0/58569919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dcc1524a0 0x7f3dcc15de30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.974+0000 7f3dd474f640 1 -- 172.21.15.112:0/58569919 >> 172.21.15.112:0/58569919 conn(0x7f3dcc06b160 msgr2=0x7f3dcc06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:58.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.974+0000 7f3dd474f640 1 -- 172.21.15.112:0/58569919 shutdown_connections 2024-04-25T01:25:58.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.974+0000 7f3dd474f640 1 -- 172.21.15.112:0/58569919 wait complete. 2024-04-25T01:25:58.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.975+0000 7f3dd474f640 1 Processor -- start 2024-04-25T01:25:58.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.975+0000 7f3dd474f640 1 -- start start 2024-04-25T01:25:58.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.976+0000 7f3dd474f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dcc0ac3b0 0x7f3dcc0c1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.976+0000 7f3dd474f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dcc14c3f0 0x7f3dcc0c04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.976+0000 7f3dd474f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dcc1524a0 0x7f3dcc0c09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.976+0000 7f3dd474f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc0c32a0 con 0x7f3dcc14c3f0 2024-04-25T01:25:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.976+0000 7f3dd474f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc070d60 con 0x7f3dcc1524a0 2024-04-25T01:25:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.976+0000 7f3dd474f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3dcc0c3420 con 0x7f3dcc0ac3b0 2024-04-25T01:25:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.976+0000 7f3dd1cea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dcc14c3f0 0x7f3dcc0c04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.976+0000 7f3dd1cea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dcc14c3f0 0x7f3dcc0c04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58156/0 (socket says 172.21.15.112:58156) 2024-04-25T01:25:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.976+0000 7f3dd1cea640 1 -- 172.21.15.112:0/1598760982 learned_addr learned my addr 172.21.15.112:0/1598760982 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.976+0000 7f3dd1cea640 1 -- 172.21.15.112:0/1598760982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dcc0ac3b0 msgr2=0x7f3dcc0c1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.976+0000 7f3dd1cea640 1 --2- 172.21.15.112:0/1598760982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dcc0ac3b0 0x7f3dcc0c1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.976+0000 7f3dd1cea640 1 -- 172.21.15.112:0/1598760982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dcc1524a0 msgr2=0x7f3dcc0c09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.976+0000 7f3dd1cea640 1 --2- 172.21.15.112:0/1598760982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dcc1524a0 0x7f3dcc0c09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:58.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.976+0000 7f3dd1cea640 1 -- 172.21.15.112:0/1598760982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc1eea20 con 0x7f3dcc14c3f0 2024-04-25T01:25:58.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.977+0000 7f3dd1cea640 1 --2- 172.21.15.112:0/1598760982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dcc14c3f0 0x7f3dcc0c04b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00c500 tx=0x7f3dbc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:58.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.977+0000 7f3dc2ffd640 1 -- 172.21.15.112:0/1598760982 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc018070 con 0x7f3dcc14c3f0 2024-04-25T01:25:58.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.977+0000 7f3dc2ffd640 1 -- 172.21.15.112:0/1598760982 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dbc007580 con 0x7f3dcc14c3f0 2024-04-25T01:25:58.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.977+0000 7f3dc2ffd640 1 -- 172.21.15.112:0/1598760982 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc004a20 con 0x7f3dcc14c3f0 2024-04-25T01:25:58.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.977+0000 7f3dd474f640 1 -- 172.21.15.112:0/1598760982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dcc1eed00 con 0x7f3dcc14c3f0 2024-04-25T01:25:58.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.977+0000 7f3dd474f640 1 -- 172.21.15.112:0/1598760982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dcc1ef2c0 con 0x7f3dcc14c3f0 2024-04-25T01:25:58.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.978+0000 7f3dd474f640 1 -- 172.21.15.112:0/1598760982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dcc0682a0 con 0x7f3dcc14c3f0 2024-04-25T01:25:58.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.978+0000 7f3dc2ffd640 1 -- 172.21.15.112:0/1598760982 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3dbc004bc0 con 0x7f3dcc14c3f0 2024-04-25T01:25:58.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.979+0000 7f3dc2ffd640 1 --2- 172.21.15.112:0/1598760982 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3db006be30 0x7f3db006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:58.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.979+0000 7f3dc2ffd640 1 -- 172.21.15.112:0/1598760982 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7f3dbc08a0d0 con 0x7f3dcc14c3f0 2024-04-25T01:25:58.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.979+0000 7f3dd24eb640 1 --2- 172.21.15.112:0/1598760982 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3db006be30 0x7f3db006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:58.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.979+0000 7f3dd24eb640 1 --2- 172.21.15.112:0/1598760982 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3db006be30 0x7f3db006e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8003cc0 tx=0x7f3dc80079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:58.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:58.983+0000 7f3dc2ffd640 1 -- 172.21.15.112:0/1598760982 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3dbc058b20 con 0x7f3dcc14c3f0 2024-04-25T01:25:59.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.158+0000 7f3dd474f640 1 -- 172.21.15.112:0/1598760982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3dcc0aaf90 con 0x7f3dcc14c3f0 2024-04-25T01:25:59.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.158+0000 7f3dc2ffd640 1 -- 172.21.15.112:0/1598760982 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v20) ==== 84+0+197 (secure 0 0 0) 0x7f3dbc058420 con 0x7f3dcc14c3f0 2024-04-25T01:25:59.138 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:25:59.139 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2024-04-25T01:25:59.139 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:25:59.139 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2024-04-25T01:25:59.139 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2024-04-25T01:25:59.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:25:59.139 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:25:59.139 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 2, 2024-04-25T01:25:59.139 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": "a_metadata" 2024-04-25T01:25:59.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:25:59.139 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:25:59.139 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 3, 2024-04-25T01:25:59.139 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": "a_data" 2024-04-25T01:25:59.139 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:25:59.139 INFO:teuthology.orchestra.run.smithi112.stdout:] 2024-04-25T01:25:59.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.160+0000 7f3dd474f640 1 -- 172.21.15.112:0/1598760982 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3db006be30 msgr2=0x7f3db006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:59.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.160+0000 7f3dd474f640 1 --2- 172.21.15.112:0/1598760982 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3db006be30 0x7f3db006e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8003cc0 tx=0x7f3dc80079b0 comp rx=0 tx=0).stop 2024-04-25T01:25:59.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.160+0000 7f3dd474f640 1 -- 172.21.15.112:0/1598760982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dcc14c3f0 msgr2=0x7f3dcc0c04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:59.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.160+0000 7f3dd474f640 1 --2- 172.21.15.112:0/1598760982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dcc14c3f0 0x7f3dcc0c04b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00c500 tx=0x7f3dbc00c9c0 comp rx=0 tx=0).stop 2024-04-25T01:25:59.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.160+0000 7f3dd474f640 1 -- 172.21.15.112:0/1598760982 shutdown_connections 2024-04-25T01:25:59.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.160+0000 7f3dd474f640 1 --2- 172.21.15.112:0/1598760982 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3db006be30 0x7f3db006e2f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:59.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.160+0000 7f3dd474f640 1 --2- 172.21.15.112:0/1598760982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dcc1524a0 0x7f3dcc0c09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:59.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.160+0000 7f3dd474f640 1 --2- 172.21.15.112:0/1598760982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dcc14c3f0 0x7f3dcc0c04b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:59.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.160+0000 7f3dd474f640 1 --2- 172.21.15.112:0/1598760982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dcc0ac3b0 0x7f3dcc0c1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:59.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.160+0000 7f3dd474f640 1 -- 172.21.15.112:0/1598760982 >> 172.21.15.112:0/1598760982 conn(0x7f3dcc06b160 msgr2=0x7f3dcc0ab1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:59.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.160+0000 7f3dd474f640 1 -- 172.21.15.112:0/1598760982 shutdown_connections 2024-04-25T01:25:59.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.160+0000 7f3dd474f640 1 -- 172.21.15.112:0/1598760982 wait complete. 2024-04-25T01:25:59.151 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'b' 2024-04-25T01:25:59.152 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create b_metadata --pg_num_min 64 2024-04-25T01:25:59.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.273+0000 7f47e422a640 1 Processor -- start 2024-04-25T01:25:59.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.273+0000 7f47e422a640 1 -- start start 2024-04-25T01:25:59.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.274+0000 7f47e422a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f47dc0aaf80 0x7f47dc1541e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:59.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.274+0000 7f47e422a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47dc0c7080 0x7f47dc1509d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:59.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.274+0000 7f47e422a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f47dc150f10 0x7f47dc0c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:59.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.274+0000 7f47e422a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f47dc0c36c0 con 0x7f47dc0aaf80 2024-04-25T01:25:59.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.274+0000 7f47e422a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f47dc156cd0 con 0x7f47dc150f10 2024-04-25T01:25:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.274+0000 7f47e422a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f47dc156f70 con 0x7f47dc0c7080 2024-04-25T01:25:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.274+0000 7f47e1fc6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f47dc0aaf80 0x7f47dc1541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.274+0000 7f47e1fc6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f47dc0aaf80 0x7f47dc1541e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58158/0 (socket says 172.21.15.112:58158) 2024-04-25T01:25:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.274+0000 7f47e1fc6640 1 -- 172.21.15.112:0/2305849108 learned_addr learned my addr 172.21.15.112:0/2305849108 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.274+0000 7f47e17c5640 1 --2- 172.21.15.112:0/2305849108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47dc0c7080 0x7f47dc1509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.274+0000 7f47e27c7640 1 --2- 172.21.15.112:0/2305849108 >> v2:172.21.15.187:3300/0 conn(0x7f47dc150f10 0x7f47dc0c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.274+0000 7f47e1fc6640 1 -- 172.21.15.112:0/2305849108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47dc0c7080 msgr2=0x7f47dc1509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.274+0000 7f47e1fc6640 1 --2- 172.21.15.112:0/2305849108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47dc0c7080 0x7f47dc1509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.274+0000 7f47e1fc6640 1 -- 172.21.15.112:0/2305849108 >> v2:172.21.15.187:3300/0 conn(0x7f47dc150f10 msgr2=0x7f47dc0c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:25:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.274+0000 7f47e1fc6640 1 --2- 172.21.15.112:0/2305849108 >> v2:172.21.15.187:3300/0 conn(0x7f47dc150f10 0x7f47dc0c43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.274+0000 7f47e1fc6640 1 -- 172.21.15.112:0/2305849108 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47dc157250 con 0x7f47dc0aaf80 2024-04-25T01:25:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.275+0000 7f47e1fc6640 1 --2- 172.21.15.112:0/2305849108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47dc0aaf80 0x7f47dc1541e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0c3a80 tx=0x7f47d802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c29ddd410c0da1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.275+0000 7f47e0fc4640 1 -- 172.21.15.112:0/2305849108 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8037070 con 0x7f47dc0aaf80 2024-04-25T01:25:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.275+0000 7f47e0fc4640 1 -- 172.21.15.112:0/2305849108 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47d8032440 con 0x7f47dc0aaf80 2024-04-25T01:25:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.275+0000 7f47e0fc4640 1 -- 172.21.15.112:0/2305849108 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8032810 con 0x7f47dc0aaf80 2024-04-25T01:25:59.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.275+0000 7f47e422a640 1 -- 172.21.15.112:0/2305849108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47dc0aaf80 msgr2=0x7f47dc1541e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:59.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.275+0000 7f47e422a640 1 --2- 172.21.15.112:0/2305849108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47dc0aaf80 0x7f47dc1541e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0c3a80 tx=0x7f47d802a140 comp rx=0 tx=0).stop 2024-04-25T01:25:59.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.275+0000 7f47e422a640 1 -- 172.21.15.112:0/2305849108 shutdown_connections 2024-04-25T01:25:59.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.275+0000 7f47e422a640 1 --2- 172.21.15.112:0/2305849108 >> v2:172.21.15.187:3300/0 conn(0x7f47dc150f10 0x7f47dc0c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:59.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.275+0000 7f47e422a640 1 --2- 172.21.15.112:0/2305849108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47dc0c7080 0x7f47dc1509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:59.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.275+0000 7f47e422a640 1 --2- 172.21.15.112:0/2305849108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47dc0aaf80 0x7f47dc1541e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:59.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.275+0000 7f47e422a640 1 -- 172.21.15.112:0/2305849108 >> 172.21.15.112:0/2305849108 conn(0x7f47dc06f300 msgr2=0x7f47dc0743d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:59.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.275+0000 7f47e422a640 1 -- 172.21.15.112:0/2305849108 shutdown_connections 2024-04-25T01:25:59.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.275+0000 7f47e422a640 1 -- 172.21.15.112:0/2305849108 wait complete. 2024-04-25T01:25:59.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.276+0000 7f47e422a640 1 Processor -- start 2024-04-25T01:25:59.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.277+0000 7f47e422a640 1 -- start start 2024-04-25T01:25:59.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.277+0000 7f47e422a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47dc0aaf80 0x7f47dc0aca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:59.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.277+0000 7f47e422a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47dc0c7080 0x7f47dc0acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:59.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.277+0000 7f47e422a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47dc150f10 0x7f47dc1eb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:59.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.277+0000 7f47e422a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc0ab360 con 0x7f47dc0c7080 2024-04-25T01:25:59.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.277+0000 7f47e422a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc151380 con 0x7f47dc0aaf80 2024-04-25T01:25:59.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.277+0000 7f47e422a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f47dc0ab4e0 con 0x7f47dc150f10 2024-04-25T01:25:59.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.277+0000 7f47e17c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47dc0c7080 0x7f47dc0acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.277+0000 7f47e17c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47dc0c7080 0x7f47dc0acfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58174/0 (socket says 172.21.15.112:58174) 2024-04-25T01:25:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.277+0000 7f47e17c5640 1 -- 172.21.15.112:0/3446701506 learned_addr learned my addr 172.21.15.112:0/3446701506 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:25:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.277+0000 7f47e17c5640 1 -- 172.21.15.112:0/3446701506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47dc150f10 msgr2=0x7f47dc1eb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.277+0000 7f47e17c5640 1 --2- 172.21.15.112:0/3446701506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47dc150f10 0x7f47dc1eb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.277+0000 7f47e17c5640 1 -- 172.21.15.112:0/3446701506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47dc0aaf80 msgr2=0x7f47dc0aca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.277+0000 7f47e17c5640 1 --2- 172.21.15.112:0/3446701506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47dc0aaf80 0x7f47dc0aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.277+0000 7f47e17c5640 1 -- 172.21.15.112:0/3446701506 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47dc1526c0 con 0x7f47dc0c7080 2024-04-25T01:25:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.278+0000 7f47e17c5640 1 --2- 172.21.15.112:0/3446701506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47dc0c7080 0x7f47dc0acfa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00d800 tx=0x7f47cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.278+0000 7f47d27fc640 1 -- 172.21.15.112:0/3446701506 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc018030 con 0x7f47dc0c7080 2024-04-25T01:25:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.278+0000 7f47d27fc640 1 -- 172.21.15.112:0/3446701506 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47cc0096a0 con 0x7f47dc0c7080 2024-04-25T01:25:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.278+0000 7f47d27fc640 1 -- 172.21.15.112:0/3446701506 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc009870 con 0x7f47dc0c7080 2024-04-25T01:25:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.278+0000 7f47e422a640 1 -- 172.21.15.112:0/3446701506 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47dc153060 con 0x7f47dc0c7080 2024-04-25T01:25:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.278+0000 7f47e422a640 1 -- 172.21.15.112:0/3446701506 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47dc06e180 con 0x7f47dc0c7080 2024-04-25T01:25:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.279+0000 7f47e422a640 1 -- 172.21.15.112:0/3446701506 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47dc0682a0 con 0x7f47dc0c7080 2024-04-25T01:25:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.279+0000 7f47d27fc640 1 -- 172.21.15.112:0/3446701506 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f47cc01b070 con 0x7f47dc0c7080 2024-04-25T01:25:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.279+0000 7f47d27fc640 1 --2- 172.21.15.112:0/3446701506 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f47c006be60 0x7f47c006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.280+0000 7f47d27fc640 1 -- 172.21.15.112:0/3446701506 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7f47cc08afe0 con 0x7f47dc0c7080 2024-04-25T01:25:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.280+0000 7f47e1fc6640 1 --2- 172.21.15.112:0/3446701506 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f47c006be60 0x7f47c006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.280+0000 7f47e1fc6640 1 --2- 172.21.15.112:0/3446701506 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f47c006be60 0x7f47c006e320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f47d8034000 tx=0x7f47d802c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:59.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.284+0000 7f47d27fc640 1 -- 172.21.15.112:0/3446701506 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f47cc059a30 con 0x7f47dc0c7080 2024-04-25T01:25:59.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:25:59.459+0000 7f47e422a640 1 -- 172.21.15.112:0/3446701506 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "b_metadata", "pg_num_min": 64} v 0) -- 0x7f47dc0713d0 con 0x7f47dc0c7080 2024-04-25T01:26:00.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.389+0000 7f47d27fc640 1 -- 172.21.15.112:0/3446701506 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "b_metadata", "pg_num_min": 64}]=0 pool 'b_metadata' created v21) ==== 128+0+0 (secure 0 0 0) 0x7f47cc059330 con 0x7f47dc0c7080 2024-04-25T01:26:00.370 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'b_metadata' created 2024-04-25T01:26:00.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.394+0000 7f47e422a640 1 -- 172.21.15.112:0/3446701506 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f47c006be60 msgr2=0x7f47c006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:00.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.394+0000 7f47e422a640 1 --2- 172.21.15.112:0/3446701506 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f47c006be60 0x7f47c006e320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f47d8034000 tx=0x7f47d802c000 comp rx=0 tx=0).stop 2024-04-25T01:26:00.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.394+0000 7f47e422a640 1 -- 172.21.15.112:0/3446701506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47dc0c7080 msgr2=0x7f47dc0acfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:00.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.394+0000 7f47e422a640 1 --2- 172.21.15.112:0/3446701506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47dc0c7080 0x7f47dc0acfa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00d800 tx=0x7f47cc00dcc0 comp rx=0 tx=0).stop 2024-04-25T01:26:00.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.394+0000 7f47e422a640 1 -- 172.21.15.112:0/3446701506 shutdown_connections 2024-04-25T01:26:00.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.394+0000 7f47e422a640 1 --2- 172.21.15.112:0/3446701506 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f47c006be60 0x7f47c006e320 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:00.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.394+0000 7f47e422a640 1 --2- 172.21.15.112:0/3446701506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47dc150f10 0x7f47dc1eb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:00.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.394+0000 7f47e422a640 1 --2- 172.21.15.112:0/3446701506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47dc0c7080 0x7f47dc0acfa0 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00d800 tx=0x7f47cc00dcc0 comp rx=0 tx=0).stop 2024-04-25T01:26:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.394+0000 7f47e422a640 1 --2- 172.21.15.112:0/3446701506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47dc0aaf80 0x7f47dc0aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.394+0000 7f47e422a640 1 -- 172.21.15.112:0/3446701506 >> 172.21.15.112:0/3446701506 conn(0x7f47dc06f300 msgr2=0x7f47dc0c5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.395+0000 7f47e422a640 1 -- 172.21.15.112:0/3446701506 shutdown_connections 2024-04-25T01:26:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.396+0000 7f47e422a640 1 -- 172.21.15.112:0/3446701506 wait complete. 2024-04-25T01:26:00.390 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create b_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T01:26:00.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.515+0000 7fac80c76640 1 Processor -- start 2024-04-25T01:26:00.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.515+0000 7fac80c76640 1 -- start start 2024-04-25T01:26:00.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.516+0000 7fac80c76640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fac7c152fd0 0x7fac7c1562a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:00.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.516+0000 7fac80c76640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac7c071290 0x7fac7c0c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:00.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.516+0000 7fac80c76640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac7c0c8d60 con 0x7fac7c1596c0 2024-04-25T01:26:00.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.516+0000 7fac80c76640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fac7c06e2f0 con 0x7fac7c152fd0 2024-04-25T01:26:00.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.516+0000 7fac80c76640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fac7c0c4590 con 0x7fac7c071290 2024-04-25T01:26:00.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.516+0000 7fac7a59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fac7c152fd0 0x7fac7c1562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:00.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.516+0000 7fac7a59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fac7c152fd0 0x7fac7c1562a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58188/0 (socket says 172.21.15.112:58188) 2024-04-25T01:26:00.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.516+0000 7fac7a59c640 1 -- 172.21.15.112:0/2319663513 learned_addr learned my addr 172.21.15.112:0/2319663513 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:00.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.516+0000 7fac7ad9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fac7c1596c0 0x7fac7c0c59f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37178/0 (socket says 172.21.15.112:37178) 2024-04-25T01:26:00.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.516+0000 7fac73fff640 1 --2- 172.21.15.112:0/2319663513 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac7c071290 0x7fac7c0c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:00.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.516+0000 7fac7a59c640 1 -- 172.21.15.112:0/2319663513 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac7c071290 msgr2=0x7fac7c0c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:00.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.516+0000 7fac7a59c640 1 --2- 172.21.15.112:0/2319663513 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac7c071290 0x7fac7c0c6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:00.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.516+0000 7fac7a59c640 1 -- 172.21.15.112:0/2319663513 >> v1:172.21.15.112:6789/0 conn(0x7fac7c1596c0 legacy=0x7fac7c0c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:00.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.516+0000 7fac79d9b640 1 -- 172.21.15.112:0/2319663513 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2213927524 0 0) 0x7fac7c0c8d60 con 0x7fac7c1596c0 2024-04-25T01:26:00.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.516+0000 7fac7a59c640 1 -- 172.21.15.112:0/2319663513 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7c0c87a0 con 0x7fac7c152fd0 2024-04-25T01:26:00.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.517+0000 7fac7a59c640 1 --2- 172.21.15.112:0/2319663513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac7c152fd0 0x7fac7c1562a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009870 tx=0x7fac6c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292968689666553a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:00.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.517+0000 7fac79d9b640 1 -- 172.21.15.112:0/2319663513 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c037070 con 0x7fac7c152fd0 2024-04-25T01:26:00.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.517+0000 7fac79d9b640 1 -- 172.21.15.112:0/2319663513 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac6c032440 con 0x7fac7c152fd0 2024-04-25T01:26:00.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.517+0000 7fac79d9b640 1 -- 172.21.15.112:0/2319663513 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c032810 con 0x7fac7c152fd0 2024-04-25T01:26:00.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.517+0000 7fac80c76640 1 -- 172.21.15.112:0/2319663513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac7c152fd0 msgr2=0x7fac7c1562a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:00.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.517+0000 7fac80c76640 1 --2- 172.21.15.112:0/2319663513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac7c152fd0 0x7fac7c1562a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009870 tx=0x7fac6c02a130 comp rx=0 tx=0).stop 2024-04-25T01:26:00.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.517+0000 7fac80c76640 1 -- 172.21.15.112:0/2319663513 shutdown_connections 2024-04-25T01:26:00.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.517+0000 7fac80c76640 1 --2- 172.21.15.112:0/2319663513 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac7c071290 0x7fac7c0c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:00.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.517+0000 7fac80c76640 1 --2- 172.21.15.112:0/2319663513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac7c152fd0 0x7fac7c1562a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:00.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.517+0000 7fac80c76640 1 -- 172.21.15.112:0/2319663513 >> 172.21.15.112:0/2319663513 conn(0x7fac7c0c8f60 msgr2=0x7fac7c0c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:00.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.517+0000 7fac80c76640 1 -- 172.21.15.112:0/2319663513 shutdown_connections 2024-04-25T01:26:00.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.517+0000 7fac80c76640 1 -- 172.21.15.112:0/2319663513 wait complete. 2024-04-25T01:26:00.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.519+0000 7fac80c76640 1 Processor -- start 2024-04-25T01:26:00.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.519+0000 7fac80c76640 1 -- start start 2024-04-25T01:26:00.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.519+0000 7fac80c76640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac7c071290 0x7fac7c0ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:00.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.519+0000 7fac80c76640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac7c152fd0 0x7fac7c0a9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:00.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.519+0000 7fac80c76640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac7c1596c0 0x7fac7c0a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:00.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.519+0000 7fac80c76640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c1591a0 con 0x7fac7c152fd0 2024-04-25T01:26:00.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.519+0000 7fac80c76640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c158ec0 con 0x7fac7c1596c0 2024-04-25T01:26:00.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.519+0000 7fac80c76640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fac7c1533b0 con 0x7fac7c071290 2024-04-25T01:26:00.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.519+0000 7fac73fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac7c152fd0 0x7fac7c0a9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:00.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.519+0000 7fac73fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac7c152fd0 0x7fac7c0a9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58190/0 (socket says 172.21.15.112:58190) 2024-04-25T01:26:00.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.519+0000 7fac73fff640 1 -- 172.21.15.112:0/1697259019 learned_addr learned my addr 172.21.15.112:0/1697259019 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:00.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.520+0000 7fac7a59c640 1 --2- 172.21.15.112:0/1697259019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac7c071290 0x7fac7c0ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:00.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.520+0000 7fac7ad9d640 1 --2- 172.21.15.112:0/1697259019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac7c1596c0 0x7fac7c0a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:00.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.520+0000 7fac73fff640 1 -- 172.21.15.112:0/1697259019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac7c071290 msgr2=0x7fac7c0ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:00.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.520+0000 7fac73fff640 1 --2- 172.21.15.112:0/1697259019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac7c071290 0x7fac7c0ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:00.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.520+0000 7fac73fff640 1 -- 172.21.15.112:0/1697259019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac7c1596c0 msgr2=0x7fac7c0a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:00.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.520+0000 7fac73fff640 1 --2- 172.21.15.112:0/1697259019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac7c1596c0 0x7fac7c0a9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:00.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.520+0000 7fac73fff640 1 -- 172.21.15.112:0/1697259019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7c14ed70 con 0x7fac7c152fd0 2024-04-25T01:26:00.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.520+0000 7fac7ad9d640 1 --2- 172.21.15.112:0/1697259019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac7c1596c0 0x7fac7c0a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:00.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.520+0000 7fac73fff640 1 --2- 172.21.15.112:0/1697259019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac7c152fd0 0x7fac7c0a9130 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fac6400c7f0 tx=0x7fac6400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:00.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.520+0000 7fac72ffd640 1 -- 172.21.15.112:0/1697259019 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64015070 con 0x7fac7c152fd0 2024-04-25T01:26:00.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.520+0000 7fac72ffd640 1 -- 172.21.15.112:0/1697259019 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac64004240 con 0x7fac7c152fd0 2024-04-25T01:26:00.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.520+0000 7fac80c76640 1 -- 172.21.15.112:0/1697259019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7c14f050 con 0x7fac7c152fd0 2024-04-25T01:26:00.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.520+0000 7fac72ffd640 1 -- 172.21.15.112:0/1697259019 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64007620 con 0x7fac7c152fd0 2024-04-25T01:26:00.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.520+0000 7fac80c76640 1 -- 172.21.15.112:0/1697259019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac7c14f8f0 con 0x7fac7c152fd0 2024-04-25T01:26:00.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.521+0000 7fac80c76640 1 -- 172.21.15.112:0/1697259019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac7c04f500 con 0x7fac7c152fd0 2024-04-25T01:26:00.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.522+0000 7fac72ffd640 1 -- 172.21.15.112:0/1697259019 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fac640079d0 con 0x7fac7c152fd0 2024-04-25T01:26:00.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.522+0000 7fac72ffd640 1 --2- 172.21.15.112:0/1697259019 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fac5c06be30 0x7fac5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:00.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.522+0000 7fac72ffd640 1 -- 172.21.15.112:0/1697259019 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6054+0+0 (secure 0 0 0) 0x7fac6408b340 con 0x7fac7c152fd0 2024-04-25T01:26:00.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.522+0000 7fac7a59c640 1 --2- 172.21.15.112:0/1697259019 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fac5c06be30 0x7fac5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:00.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.522+0000 7fac7a59c640 1 --2- 172.21.15.112:0/1697259019 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fac5c06be30 0x7fac5c06e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fac6c034000 tx=0x7fac6c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:00.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.526+0000 7fac72ffd640 1 -- 172.21.15.112:0/1697259019 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fac64059d00 con 0x7fac7c152fd0 2024-04-25T01:26:00.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:00.699+0000 7fac80c76640 1 -- 172.21.15.112:0/1697259019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "b_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fac7c1563a0 con 0x7fac7c152fd0 2024-04-25T01:26:01.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.393+0000 7fac72ffd640 1 -- 172.21.15.112:0/1697259019 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "b_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'b_data' created v22) ==== 160+0+0 (secure 0 0 0) 0x7fac7c1563a0 con 0x7fac7c152fd0 2024-04-25T01:26:01.374 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'b_data' created 2024-04-25T01:26:01.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.396+0000 7fac80c76640 1 -- 172.21.15.112:0/1697259019 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fac5c06be30 msgr2=0x7fac5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:01.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.396+0000 7fac80c76640 1 --2- 172.21.15.112:0/1697259019 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fac5c06be30 0x7fac5c06e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fac6c034000 tx=0x7fac6c02e000 comp rx=0 tx=0).stop 2024-04-25T01:26:01.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.396+0000 7fac80c76640 1 -- 172.21.15.112:0/1697259019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac7c152fd0 msgr2=0x7fac7c0a9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:01.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.396+0000 7fac80c76640 1 --2- 172.21.15.112:0/1697259019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac7c152fd0 0x7fac7c0a9130 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fac6400c7f0 tx=0x7fac6400ccb0 comp rx=0 tx=0).stop 2024-04-25T01:26:01.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.399+0000 7fac80c76640 1 -- 172.21.15.112:0/1697259019 shutdown_connections 2024-04-25T01:26:01.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.399+0000 7fac80c76640 1 --2- 172.21.15.112:0/1697259019 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fac5c06be30 0x7fac5c06e2f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:01.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.399+0000 7fac80c76640 1 --2- 172.21.15.112:0/1697259019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac7c1596c0 0x7fac7c0a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:01.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.399+0000 7fac80c76640 1 --2- 172.21.15.112:0/1697259019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac7c152fd0 0x7fac7c0a9130 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:01.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.399+0000 7fac80c76640 1 --2- 172.21.15.112:0/1697259019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac7c071290 0x7fac7c0ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:01.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.399+0000 7fac80c76640 1 -- 172.21.15.112:0/1697259019 >> 172.21.15.112:0/1697259019 conn(0x7fac7c0c8f60 msgr2=0x7fac7c1574f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:01.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.401+0000 7fac80c76640 1 -- 172.21.15.112:0/1697259019 shutdown_connections 2024-04-25T01:26:01.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.401+0000 7fac80c76640 1 -- 172.21.15.112:0/1697259019 wait complete. 2024-04-25T01:26:01.398 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new b b_metadata b_data 2024-04-25T01:26:01.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.517+0000 7f1a928c3640 1 Processor -- start 2024-04-25T01:26:01.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.517+0000 7f1a928c3640 1 -- start start 2024-04-25T01:26:01.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.518+0000 7f1a928c3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1a8c06d460 0x7f1a8c071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:01.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.518+0000 7f1a928c3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a8c0724f0 0x7f1a8c15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:01.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.518+0000 7f1a928c3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a8c06f290 con 0x7f1a8c14fbe0 2024-04-25T01:26:01.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.518+0000 7f1a928c3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1a8c14ffc0 con 0x7f1a8c06d460 2024-04-25T01:26:01.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.518+0000 7f1a928c3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8c1533a0 con 0x7f1a8c0724f0 2024-04-25T01:26:01.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.518+0000 7f1a8b7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1a8c06d460 0x7f1a8c071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:01.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.518+0000 7f1a90e60640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a8c0724f0 0x7f1a8c15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:01.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.518+0000 7f1a90e60640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a8c0724f0 0x7f1a8c15ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:44916/0 (socket says 172.21.15.112:44916) 2024-04-25T01:26:01.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.518+0000 7f1a8bfff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f1a8c14fbe0 0x7f1a8c0c2920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:46794/0 (socket says 172.21.15.112:46794) 2024-04-25T01:26:01.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.518+0000 7f1a90e60640 1 -- 172.21.15.112:0/3609872322 learned_addr learned my addr 172.21.15.112:0/3609872322 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:01.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.518+0000 7f1a8b7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1a8c06d460 0x7f1a8c071fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43928/0 (socket says 172.21.15.112:43928) 2024-04-25T01:26:01.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.518+0000 7f1a8b7fe640 1 -- 172.21.15.112:0/3609872322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a8c0724f0 msgr2=0x7f1a8c15ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:01.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.518+0000 7f1a8b7fe640 1 --2- 172.21.15.112:0/3609872322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a8c0724f0 0x7f1a8c15ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:01.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.518+0000 7f1a8b7fe640 1 -- 172.21.15.112:0/3609872322 >> v1:172.21.15.187:6789/0 conn(0x7f1a8c14fbe0 legacy=0x7f1a8c0c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:01.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.518+0000 7f1a8affd640 1 -- 172.21.15.112:0/3609872322 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1020795006 0 0) 0x7f1a8c06f290 con 0x7f1a8c14fbe0 2024-04-25T01:26:01.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.518+0000 7f1a8b7fe640 1 -- 172.21.15.112:0/3609872322 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8c06efb0 con 0x7f1a8c06d460 2024-04-25T01:26:01.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.518+0000 7f1a90e60640 1 --2- 172.21.15.112:0/3609872322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a8c0724f0 0x7f1a8c15ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:26:01.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.519+0000 7f1a8b7fe640 1 --2- 172.21.15.112:0/3609872322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a8c06d460 0x7f1a8c071fb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1a800097a0 tx=0x7f1a8002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=702624f6eff2cfae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:01.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.519+0000 7f1a8affd640 1 -- 172.21.15.112:0/3609872322 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80037070 con 0x7f1a8c06d460 2024-04-25T01:26:01.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.519+0000 7f1a8affd640 1 -- 172.21.15.112:0/3609872322 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8002ae50 con 0x7f1a8c06d460 2024-04-25T01:26:01.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.519+0000 7f1a8affd640 1 -- 172.21.15.112:0/3609872322 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80032660 con 0x7f1a8c06d460 2024-04-25T01:26:01.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.520+0000 7f1a928c3640 1 -- 172.21.15.112:0/3609872322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a8c06d460 msgr2=0x7f1a8c071fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:01.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.520+0000 7f1a928c3640 1 --2- 172.21.15.112:0/3609872322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a8c06d460 0x7f1a8c071fb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1a800097a0 tx=0x7f1a8002a100 comp rx=0 tx=0).stop 2024-04-25T01:26:01.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.520+0000 7f1a928c3640 1 -- 172.21.15.112:0/3609872322 shutdown_connections 2024-04-25T01:26:01.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.520+0000 7f1a928c3640 1 --2- 172.21.15.112:0/3609872322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a8c0724f0 0x7f1a8c15ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:01.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.520+0000 7f1a928c3640 1 --2- 172.21.15.112:0/3609872322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a8c06d460 0x7f1a8c071fb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:01.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.520+0000 7f1a928c3640 1 -- 172.21.15.112:0/3609872322 >> 172.21.15.112:0/3609872322 conn(0x7f1a8c0c3a10 msgr2=0x7f1a8c0c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:01.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.520+0000 7f1a928c3640 1 -- 172.21.15.112:0/3609872322 shutdown_connections 2024-04-25T01:26:01.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.520+0000 7f1a928c3640 1 -- 172.21.15.112:0/3609872322 wait complete. 2024-04-25T01:26:01.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.521+0000 7f1a928c3640 1 Processor -- start 2024-04-25T01:26:01.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.521+0000 7f1a928c3640 1 -- start start 2024-04-25T01:26:01.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a928c3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a8c06d460 0x7f1a8c0ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:01.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a928c3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a8c0724f0 0x7f1a8c0ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:01.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a928c3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a8c14fbe0 0x7f1a8c1415d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:01.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a928c3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c0c3030 con 0x7f1a8c0724f0 2024-04-25T01:26:01.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a928c3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c153030 con 0x7f1a8c14fbe0 2024-04-25T01:26:01.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a928c3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8c0c31b0 con 0x7f1a8c06d460 2024-04-25T01:26:01.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a8b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a8c0724f0 0x7f1a8c0ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:01.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a8b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a8c0724f0 0x7f1a8c0ac2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58194/0 (socket says 172.21.15.112:58194) 2024-04-25T01:26:01.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a8b7fe640 1 -- 172.21.15.112:0/1462279105 learned_addr learned my addr 172.21.15.112:0/1462279105 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:01.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a90e60640 1 --2- 172.21.15.112:0/1462279105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a8c14fbe0 0x7f1a8c1415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:01.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a8bfff640 1 --2- 172.21.15.112:0/1462279105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a8c06d460 0x7f1a8c0ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:01.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a8b7fe640 1 -- 172.21.15.112:0/1462279105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a8c06d460 msgr2=0x7f1a8c0ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:01.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a8b7fe640 1 --2- 172.21.15.112:0/1462279105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a8c06d460 0x7f1a8c0ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:01.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a8b7fe640 1 -- 172.21.15.112:0/1462279105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a8c14fbe0 msgr2=0x7f1a8c1415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:01.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a8b7fe640 1 --2- 172.21.15.112:0/1462279105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a8c14fbe0 0x7f1a8c1415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:01.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a8b7fe640 1 -- 172.21.15.112:0/1462279105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8c0abe20 con 0x7f1a8c0724f0 2024-04-25T01:26:01.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a8bfff640 1 --2- 172.21.15.112:0/1462279105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a8c06d460 0x7f1a8c0ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:01.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a8b7fe640 1 --2- 172.21.15.112:0/1462279105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a8c0724f0 0x7f1a8c0ac2e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1a800097a0 tx=0x7f1a800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:01.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a88ff9640 1 -- 172.21.15.112:0/1462279105 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80037040 con 0x7f1a8c0724f0 2024-04-25T01:26:01.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a88ff9640 1 -- 172.21.15.112:0/1462279105 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a80004030 con 0x7f1a8c0724f0 2024-04-25T01:26:01.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a928c3640 1 -- 172.21.15.112:0/1462279105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a8c0c5c80 con 0x7f1a8c0724f0 2024-04-25T01:26:01.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.522+0000 7f1a88ff9640 1 -- 172.21.15.112:0/1462279105 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80004420 con 0x7f1a8c0724f0 2024-04-25T01:26:01.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.523+0000 7f1a928c3640 1 -- 172.21.15.112:0/1462279105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a8c0c6240 con 0x7f1a8c0724f0 2024-04-25T01:26:01.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.524+0000 7f1a928c3640 1 -- 172.21.15.112:0/1462279105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a8c0c1be0 con 0x7f1a8c0724f0 2024-04-25T01:26:01.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.524+0000 7f1a88ff9640 1 -- 172.21.15.112:0/1462279105 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a80003610 con 0x7f1a8c0724f0 2024-04-25T01:26:01.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.524+0000 7f1a88ff9640 1 --2- 172.21.15.112:0/1462279105 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1a7806be30 0x7f1a7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:01.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.524+0000 7f1a8bfff640 1 --2- 172.21.15.112:0/1462279105 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1a7806be30 0x7f1a7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:01.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.524+0000 7f1a88ff9640 1 -- 172.21.15.112:0/1462279105 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6431+0+0 (secure 0 0 0) 0x7f1a800abdc0 con 0x7f1a8c0724f0 2024-04-25T01:26:01.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.525+0000 7f1a8bfff640 1 --2- 172.21.15.112:0/1462279105 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1a7806be30 0x7f1a7806e2f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c000f60 tx=0x7f1a7c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:01.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.529+0000 7f1a88ff9640 1 -- 172.21.15.112:0/1462279105 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1a8c0c1be0 con 0x7f1a8c0724f0 2024-04-25T01:26:01.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:01.737+0000 7f1a928c3640 1 -- 172.21.15.112:0/1462279105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "b", "metadata": "b_metadata", "data": "b_data"} v 0) -- 0x7f1a8c06e030 con 0x7f1a8c0724f0 2024-04-25T01:26:02.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.394+0000 7f1a88ff9640 1 -- 172.21.15.112:0/1462279105 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "b", "metadata": "b_metadata", "data": "b_data"}]=0 new fs with metadata pool 4 and data pool 5 v8) ==== 157+0+0 (secure 0 0 0) 0x7f1a8007a530 con 0x7f1a8c0724f0 2024-04-25T01:26:02.375 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 4 and data pool 5 2024-04-25T01:26:02.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.400+0000 7f1a928c3640 1 -- 172.21.15.112:0/1462279105 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1a7806be30 msgr2=0x7f1a7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:02.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.400+0000 7f1a928c3640 1 --2- 172.21.15.112:0/1462279105 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1a7806be30 0x7f1a7806e2f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c000f60 tx=0x7f1a7c009040 comp rx=0 tx=0).stop 2024-04-25T01:26:02.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.400+0000 7f1a928c3640 1 -- 172.21.15.112:0/1462279105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a8c0724f0 msgr2=0x7f1a8c0ac2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:02.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.400+0000 7f1a928c3640 1 --2- 172.21.15.112:0/1462279105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a8c0724f0 0x7f1a8c0ac2e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1a800097a0 tx=0x7f1a800046c0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.401+0000 7f1a928c3640 1 -- 172.21.15.112:0/1462279105 shutdown_connections 2024-04-25T01:26:02.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.401+0000 7f1a928c3640 1 --2- 172.21.15.112:0/1462279105 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1a7806be30 0x7f1a7806e2f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.401+0000 7f1a928c3640 1 --2- 172.21.15.112:0/1462279105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a8c14fbe0 0x7f1a8c1415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.401+0000 7f1a928c3640 1 --2- 172.21.15.112:0/1462279105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a8c0724f0 0x7f1a8c0ac2e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.401+0000 7f1a928c3640 1 --2- 172.21.15.112:0/1462279105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a8c06d460 0x7f1a8c0ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.401+0000 7f1a928c3640 1 -- 172.21.15.112:0/1462279105 >> 172.21.15.112:0/1462279105 conn(0x7f1a8c0c3a10 msgr2=0x7f1a8c0c0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:02.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.402+0000 7f1a928c3640 1 -- 172.21.15.112:0/1462279105 shutdown_connections 2024-04-25T01:26:02.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.407+0000 7f1a928c3640 1 -- 172.21.15.112:0/1462279105 wait complete. 2024-04-25T01:26:02.414 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:26:02.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.531+0000 7faaf9c7d640 1 Processor -- start 2024-04-25T01:26:02.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.531+0000 7faaf9c7d640 1 -- start start 2024-04-25T01:26:02.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.531+0000 7faaf9c7d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7faaf40ac3b0 0x7faaf4155f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:02.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.531+0000 7faaf9c7d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaf41529b0 con 0x7faaf414c3f0 2024-04-25T01:26:02.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.532+0000 7faaf9c7d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaf4152c70 con 0x7faaf41524a0 2024-04-25T01:26:02.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.532+0000 7faaf9c7d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7faaf4155a90 con 0x7faaf40ac3b0 2024-04-25T01:26:02.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.532+0000 7faaf2ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7faaf40ac3b0 0x7faaf4155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:02.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.532+0000 7faaf37fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7faaf414c3f0 0x7faaf40c2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37184/0 (socket says 172.21.15.112:37184) 2024-04-25T01:26:02.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.532+0000 7faaf37fe640 1 -- 172.21.15.112:0/3553444238 learned_addr learned my addr 172.21.15.112:0/3553444238 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:02.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.532+0000 7faaf2ffd640 1 -- 172.21.15.112:0/3553444238 >> v1:172.21.15.187:6789/0 conn(0x7faaf41524a0 legacy=0x7faaf415ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:02.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.532+0000 7faaf2ffd640 1 -- 172.21.15.112:0/3553444238 >> v1:172.21.15.112:6789/0 conn(0x7faaf414c3f0 legacy=0x7faaf40c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:02.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.532+0000 7faaf27fc640 1 -- 172.21.15.112:0/3553444238 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3999689535 0 0) 0x7faaf41529b0 con 0x7faaf414c3f0 2024-04-25T01:26:02.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.532+0000 7faaf2ffd640 1 -- 172.21.15.112:0/3553444238 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf4152f50 con 0x7faaf40ac3b0 2024-04-25T01:26:02.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.533+0000 7faaf2ffd640 1 --2- 172.21.15.112:0/3553444238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf40ac3b0 0x7faaf4155f20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faae80097a0 tx=0x7faae802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77da6301e5cc67c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:02.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.533+0000 7faaf27fc640 1 -- 172.21.15.112:0/3553444238 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae8037070 con 0x7faaf40ac3b0 2024-04-25T01:26:02.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.533+0000 7faaf27fc640 1 -- 172.21.15.112:0/3553444238 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae802ae50 con 0x7faaf40ac3b0 2024-04-25T01:26:02.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.533+0000 7faaf27fc640 1 -- 172.21.15.112:0/3553444238 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae8032660 con 0x7faaf40ac3b0 2024-04-25T01:26:02.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.533+0000 7faaf9c7d640 1 -- 172.21.15.112:0/3553444238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf40ac3b0 msgr2=0x7faaf4155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:02.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.533+0000 7faaf9c7d640 1 --2- 172.21.15.112:0/3553444238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf40ac3b0 0x7faaf4155f20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faae80097a0 tx=0x7faae802a140 comp rx=0 tx=0).stop 2024-04-25T01:26:02.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.533+0000 7faaf9c7d640 1 -- 172.21.15.112:0/3553444238 shutdown_connections 2024-04-25T01:26:02.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.533+0000 7faaf9c7d640 1 --2- 172.21.15.112:0/3553444238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf40ac3b0 0x7faaf4155f20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.533+0000 7faaf9c7d640 1 -- 172.21.15.112:0/3553444238 >> 172.21.15.112:0/3553444238 conn(0x7faaf406b160 msgr2=0x7faaf406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:02.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.533+0000 7faaf9c7d640 1 -- 172.21.15.112:0/3553444238 shutdown_connections 2024-04-25T01:26:02.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.533+0000 7faaf9c7d640 1 -- 172.21.15.112:0/3553444238 wait complete. 2024-04-25T01:26:02.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.534+0000 7faaf9c7d640 1 Processor -- start 2024-04-25T01:26:02.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.535+0000 7faaf9c7d640 1 -- start start 2024-04-25T01:26:02.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.535+0000 7faaf9c7d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faaf40ac3b0 0x7faaf40c0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:02.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.535+0000 7faaf9c7d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf414c3f0 0x7faaf41eaf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:02.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.535+0000 7faaf9c7d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faaf41524a0 0x7faaf41eb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:02.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.535+0000 7faaf9c7d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faaf40c32a0 con 0x7faaf414c3f0 2024-04-25T01:26:02.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.535+0000 7faaf9c7d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faaf4070d60 con 0x7faaf40ac3b0 2024-04-25T01:26:02.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.535+0000 7faaf9c7d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faaf40c3420 con 0x7faaf41524a0 2024-04-25T01:26:02.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.535+0000 7faaf2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf414c3f0 0x7faaf41eaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:02.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.535+0000 7faaf2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf414c3f0 0x7faaf41eaf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58214/0 (socket says 172.21.15.112:58214) 2024-04-25T01:26:02.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.535+0000 7faaf2ffd640 1 -- 172.21.15.112:0/668365108 learned_addr learned my addr 172.21.15.112:0/668365108 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:02.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.535+0000 7faaf3fff640 1 --2- 172.21.15.112:0/668365108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faaf41524a0 0x7faaf41eb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:02.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.535+0000 7faaf2ffd640 1 -- 172.21.15.112:0/668365108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faaf41524a0 msgr2=0x7faaf41eb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:02.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.535+0000 7faaf37fe640 1 --2- 172.21.15.112:0/668365108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faaf40ac3b0 0x7faaf40c0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:02.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.536+0000 7faaf2ffd640 1 --2- 172.21.15.112:0/668365108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faaf41524a0 0x7faaf41eb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.536+0000 7faaf2ffd640 1 -- 172.21.15.112:0/668365108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faaf40ac3b0 msgr2=0x7faaf40c0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:02.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.536+0000 7faaf2ffd640 1 --2- 172.21.15.112:0/668365108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faaf40ac3b0 0x7faaf40c0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.536+0000 7faaf2ffd640 1 -- 172.21.15.112:0/668365108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf406f300 con 0x7faaf414c3f0 2024-04-25T01:26:02.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.536+0000 7faaf2ffd640 1 --2- 172.21.15.112:0/668365108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf414c3f0 0x7faaf41eaf20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7faae80097a0 tx=0x7faae80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:02.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.536+0000 7faadbfff640 1 -- 172.21.15.112:0/668365108 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae8037040 con 0x7faaf414c3f0 2024-04-25T01:26:02.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.536+0000 7faadbfff640 1 -- 172.21.15.112:0/668365108 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae8004030 con 0x7faaf414c3f0 2024-04-25T01:26:02.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.536+0000 7faaf9c7d640 1 -- 172.21.15.112:0/668365108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf406eda0 con 0x7faaf414c3f0 2024-04-25T01:26:02.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.536+0000 7faadbfff640 1 -- 172.21.15.112:0/668365108 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae8006e10 con 0x7faaf414c3f0 2024-04-25T01:26:02.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.536+0000 7faaf9c7d640 1 -- 172.21.15.112:0/668365108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf41de0e0 con 0x7faaf414c3f0 2024-04-25T01:26:02.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.537+0000 7faaf9c7d640 1 -- 172.21.15.112:0/668365108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf40682a0 con 0x7faaf414c3f0 2024-04-25T01:26:02.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.537+0000 7faadbfff640 1 -- 172.21.15.112:0/668365108 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faae803d650 con 0x7faaf414c3f0 2024-04-25T01:26:02.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.538+0000 7faadbfff640 1 --2- 172.21.15.112:0/668365108 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faae006be30 0x7faae006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:02.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.538+0000 7faadbfff640 1 -- 172.21.15.112:0/668365108 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6521+0+0 (secure 0 0 0) 0x7faae80ab9f0 con 0x7faaf414c3f0 2024-04-25T01:26:02.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.538+0000 7faaf37fe640 1 --2- 172.21.15.112:0/668365108 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faae006be30 0x7faae006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:02.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.538+0000 7faaf37fe640 1 --2- 172.21.15.112:0/668365108 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faae006be30 0x7faae006e2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faae4007fd0 tx=0x7faae4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:02.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.542+0000 7faadbfff640 1 -- 172.21.15.112:0/668365108 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faaf406eda0 con 0x7faaf414c3f0 2024-04-25T01:26:02.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.719+0000 7faaf9c7d640 1 -- 172.21.15.112:0/668365108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faaf40aaf90 con 0x7faaf414c3f0 2024-04-25T01:26:02.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.721+0000 7faadbfff640 1 -- 172.21.15.112:0/668365108 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+20380 (secure 0 0 0) 0x7faae8042020 con 0x7faaf414c3f0 2024-04-25T01:26:02.702 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:02.702 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":23,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:26:01.740210+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-25T01:25:59.478275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-25T01:26:00.719358+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:26:02.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.723+0000 7faaf9c7d640 1 -- 172.21.15.112:0/668365108 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faae006be30 msgr2=0x7faae006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:02.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.723+0000 7faaf9c7d640 1 --2- 172.21.15.112:0/668365108 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faae006be30 0x7faae006e2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faae4007fd0 tx=0x7faae4009040 comp rx=0 tx=0).stop 2024-04-25T01:26:02.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.723+0000 7faaf9c7d640 1 -- 172.21.15.112:0/668365108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf414c3f0 msgr2=0x7faaf41eaf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:02.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.723+0000 7faaf9c7d640 1 --2- 172.21.15.112:0/668365108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf414c3f0 0x7faaf41eaf20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7faae80097a0 tx=0x7faae80046c0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.723+0000 7faaf9c7d640 1 -- 172.21.15.112:0/668365108 shutdown_connections 2024-04-25T01:26:02.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.723+0000 7faaf9c7d640 1 --2- 172.21.15.112:0/668365108 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faae006be30 0x7faae006e2f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.723+0000 7faaf9c7d640 1 --2- 172.21.15.112:0/668365108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faaf41524a0 0x7faaf41eb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.723+0000 7faaf9c7d640 1 --2- 172.21.15.112:0/668365108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf414c3f0 0x7faaf41eaf20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.723+0000 7faaf9c7d640 1 --2- 172.21.15.112:0/668365108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faaf40ac3b0 0x7faaf40c0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.723+0000 7faaf9c7d640 1 -- 172.21.15.112:0/668365108 >> 172.21.15.112:0/668365108 conn(0x7faaf406b160 msgr2=0x7faaf40ab1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:02.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.723+0000 7faaf9c7d640 1 -- 172.21.15.112:0/668365108 shutdown_connections 2024-04-25T01:26:02.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.723+0000 7faaf9c7d640 1 -- 172.21.15.112:0/668365108 wait complete. 2024-04-25T01:26:02.715 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:26:02.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.830+0000 7f9f1c9a5640 1 Processor -- start 2024-04-25T01:26:02.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.830+0000 7f9f1c9a5640 1 -- start start 2024-04-25T01:26:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.831+0000 7f9f1c9a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f180ac3b0 0x7f9f18155f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.831+0000 7f9f1c9a5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9f181524a0 0x7f9f18156460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.831+0000 7f9f1c9a5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f181529b0 con 0x7f9f1814c3f0 2024-04-25T01:26:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.831+0000 7f9f1c9a5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f9f18155a90 con 0x7f9f181524a0 2024-04-25T01:26:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.831+0000 7f9f1c9a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9f181569a0 con 0x7f9f180ac3b0 2024-04-25T01:26:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.831+0000 7f9f16d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9f181524a0 0x7f9f18156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.831+0000 7f9f16d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9f181524a0 0x7f9f18156460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58216/0 (socket says 172.21.15.112:58216) 2024-04-25T01:26:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.831+0000 7f9f16d9d640 1 -- 172.21.15.112:0/2305574151 learned_addr learned my addr 172.21.15.112:0/2305574151 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.831+0000 7f9f0ffff640 1 --2- 172.21.15.112:0/2305574151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f180ac3b0 0x7f9f18155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.831+0000 7f9f16d9d640 1 -- 172.21.15.112:0/2305574151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f180ac3b0 msgr2=0x7f9f18155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.831+0000 7f9f16d9d640 1 --2- 172.21.15.112:0/2305574151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f180ac3b0 0x7f9f18155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.831+0000 7f9f16d9d640 1 -- 172.21.15.112:0/2305574151 >> v1:172.21.15.187:6789/0 conn(0x7f9f1814c3f0 legacy=0x7f9f180c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.831+0000 7f9f16d9d640 1 -- 172.21.15.112:0/2305574151 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f18152c70 con 0x7f9f181524a0 2024-04-25T01:26:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.832+0000 7f9f16d9d640 1 --2- 172.21.15.112:0/2305574151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f181524a0 0x7f9f18156460 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009e50 tx=0x7f9f1002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221d9153fdeaa493 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.832+0000 7f9f15d9b640 1 -- 172.21.15.112:0/2305574151 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10036070 con 0x7f9f181524a0 2024-04-25T01:26:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.832+0000 7f9f15d9b640 1 -- 172.21.15.112:0/2305574151 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f10031600 con 0x7f9f181524a0 2024-04-25T01:26:02.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.832+0000 7f9f15d9b640 1 -- 172.21.15.112:0/2305574151 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f100319d0 con 0x7f9f181524a0 2024-04-25T01:26:02.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.832+0000 7f9f1c9a5640 1 -- 172.21.15.112:0/2305574151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f181524a0 msgr2=0x7f9f18156460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:02.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.832+0000 7f9f1c9a5640 1 --2- 172.21.15.112:0/2305574151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f181524a0 0x7f9f18156460 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009e50 tx=0x7f9f1002a630 comp rx=0 tx=0).stop 2024-04-25T01:26:02.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.832+0000 7f9f1c9a5640 1 -- 172.21.15.112:0/2305574151 shutdown_connections 2024-04-25T01:26:02.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.832+0000 7f9f1c9a5640 1 --2- 172.21.15.112:0/2305574151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f181524a0 0x7f9f18156460 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.832+0000 7f9f1c9a5640 1 --2- 172.21.15.112:0/2305574151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f180ac3b0 0x7f9f18155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.832+0000 7f9f1c9a5640 1 -- 172.21.15.112:0/2305574151 >> 172.21.15.112:0/2305574151 conn(0x7f9f1806b160 msgr2=0x7f9f1806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:02.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.832+0000 7f9f1c9a5640 1 -- 172.21.15.112:0/2305574151 shutdown_connections 2024-04-25T01:26:02.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.832+0000 7f9f1c9a5640 1 -- 172.21.15.112:0/2305574151 wait complete. 2024-04-25T01:26:02.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.834+0000 7f9f1c9a5640 1 Processor -- start 2024-04-25T01:26:02.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.834+0000 7f9f1c9a5640 1 -- start start 2024-04-25T01:26:02.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.834+0000 7f9f1c9a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f180ac3b0 0x7f9f18147a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:02.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.834+0000 7f9f1c9a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f1814c3f0 0x7f9f180c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:02.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.834+0000 7f9f1c9a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f181524a0 0x7f9f180c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:02.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.834+0000 7f9f1c9a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f18070d60 con 0x7f9f181524a0 2024-04-25T01:26:02.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.834+0000 7f9f1c9a5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9f18153230 con 0x7f9f180ac3b0 2024-04-25T01:26:02.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.834+0000 7f9f1c9a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9f180c32a0 con 0x7f9f1814c3f0 2024-04-25T01:26:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.834+0000 7f9f16d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f181524a0 0x7f9f180c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.834+0000 7f9f16d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f181524a0 0x7f9f180c79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58218/0 (socket says 172.21.15.112:58218) 2024-04-25T01:26:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.834+0000 7f9f16d9d640 1 -- 172.21.15.112:0/3175861547 learned_addr learned my addr 172.21.15.112:0/3175861547 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.835+0000 7f9f16d9d640 1 -- 172.21.15.112:0/3175861547 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f1814c3f0 msgr2=0x7f9f180c7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.835+0000 7f9f16d9d640 1 --2- 172.21.15.112:0/3175861547 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f1814c3f0 0x7f9f180c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.835+0000 7f9f16d9d640 1 -- 172.21.15.112:0/3175861547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f180ac3b0 msgr2=0x7f9f18147a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.835+0000 7f9f1659c640 1 --2- 172.21.15.112:0/3175861547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f180ac3b0 0x7f9f18147a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.835+0000 7f9f16d9d640 1 --2- 172.21.15.112:0/3175861547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f180ac3b0 0x7f9f18147a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.835+0000 7f9f16d9d640 1 -- 172.21.15.112:0/3175861547 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f180c7ee0 con 0x7f9f181524a0 2024-04-25T01:26:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.835+0000 7f9f16d9d640 1 --2- 172.21.15.112:0/3175861547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f181524a0 0x7f9f180c79a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009e50 tx=0x7f9f10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.835+0000 7f9f0effd640 1 -- 172.21.15.112:0/3175861547 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10036050 con 0x7f9f181524a0 2024-04-25T01:26:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.835+0000 7f9f0effd640 1 -- 172.21.15.112:0/3175861547 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f10042070 con 0x7f9f181524a0 2024-04-25T01:26:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.835+0000 7f9f1c9a5640 1 -- 172.21.15.112:0/3175861547 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f181e92e0 con 0x7f9f181524a0 2024-04-25T01:26:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.835+0000 7f9f0effd640 1 -- 172.21.15.112:0/3175861547 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10009510 con 0x7f9f181524a0 2024-04-25T01:26:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.835+0000 7f9f1c9a5640 1 -- 172.21.15.112:0/3175861547 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f181e9860 con 0x7f9f181524a0 2024-04-25T01:26:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.836+0000 7f9f1c9a5640 1 -- 172.21.15.112:0/3175861547 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f180682a0 con 0x7f9f181524a0 2024-04-25T01:26:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.836+0000 7f9f0effd640 1 -- 172.21.15.112:0/3175861547 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9f10007bb0 con 0x7f9f181524a0 2024-04-25T01:26:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.837+0000 7f9f0effd640 1 --2- 172.21.15.112:0/3175861547 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ef806be30 0x7f9ef806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.837+0000 7f9f0effd640 1 -- 172.21.15.112:0/3175861547 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6521+0+0 (secure 0 0 0) 0x7f9f100ac010 con 0x7f9f181524a0 2024-04-25T01:26:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.837+0000 7f9f1659c640 1 --2- 172.21.15.112:0/3175861547 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ef806be30 0x7f9ef806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.837+0000 7f9f1659c640 1 --2- 172.21.15.112:0/3175861547 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ef806be30 0x7f9ef806e2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9f08003d90 tx=0x7f9f08009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:02.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:02.841+0000 7f9f0effd640 1 -- 172.21.15.112:0/3175861547 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9f180c7ee0 con 0x7f9f181524a0 2024-04-25T01:26:02.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.013+0000 7f9f1c9a5640 1 -- 172.21.15.112:0/3175861547 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f18072130 con 0x7f9f181524a0 2024-04-25T01:26:02.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.015+0000 7f9f0effd640 1 -- 172.21.15.112:0/3175861547 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+20380 (secure 0 0 0) 0x7f9f10045360 con 0x7f9f181524a0 2024-04-25T01:26:02.996 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:02.996 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":23,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:26:01.740210+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-25T01:25:59.478275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-25T01:26:00.719358+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:26:02.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.017+0000 7f9f1c9a5640 1 -- 172.21.15.112:0/3175861547 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ef806be30 msgr2=0x7f9ef806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:02.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.017+0000 7f9f1c9a5640 1 --2- 172.21.15.112:0/3175861547 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ef806be30 0x7f9ef806e2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9f08003d90 tx=0x7f9f08009040 comp rx=0 tx=0).stop 2024-04-25T01:26:02.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.017+0000 7f9f1c9a5640 1 -- 172.21.15.112:0/3175861547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f181524a0 msgr2=0x7f9f180c79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:02.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.017+0000 7f9f1c9a5640 1 --2- 172.21.15.112:0/3175861547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f181524a0 0x7f9f180c79a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009e50 tx=0x7f9f10004a40 comp rx=0 tx=0).stop 2024-04-25T01:26:02.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.017+0000 7f9f1c9a5640 1 -- 172.21.15.112:0/3175861547 shutdown_connections 2024-04-25T01:26:02.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.017+0000 7f9f1c9a5640 1 --2- 172.21.15.112:0/3175861547 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ef806be30 0x7f9ef806e2f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.017+0000 7f9f1c9a5640 1 --2- 172.21.15.112:0/3175861547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f181524a0 0x7f9f180c79a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.017+0000 7f9f1c9a5640 1 --2- 172.21.15.112:0/3175861547 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f1814c3f0 0x7f9f180c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.017+0000 7f9f1c9a5640 1 --2- 172.21.15.112:0/3175861547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f180ac3b0 0x7f9f18147a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:02.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.017+0000 7f9f1c9a5640 1 -- 172.21.15.112:0/3175861547 >> 172.21.15.112:0/3175861547 conn(0x7f9f1806b160 msgr2=0x7f9f1806b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:02.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.017+0000 7f9f1c9a5640 1 -- 172.21.15.112:0/3175861547 shutdown_connections 2024-04-25T01:26:02.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.017+0000 7f9f1c9a5640 1 -- 172.21.15.112:0/3175861547 wait complete. 2024-04-25T01:26:03.009 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set b standby_count_wanted 0 2024-04-25T01:26:03.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.129+0000 7f780ed7e640 1 Processor -- start 2024-04-25T01:26:03.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.129+0000 7f780ed7e640 1 -- start start 2024-04-25T01:26:03.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.129+0000 7f780ed7e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f780806d460 0x7f7808071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:03.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.129+0000 7f780ed7e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f78080724f0 0x7f7808072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:03.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.129+0000 7f780ed7e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f780806f290 con 0x7f780814fbe0 2024-04-25T01:26:03.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.129+0000 7f780ed7e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f780814ffc0 con 0x7f78080724f0 2024-04-25T01:26:03.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.129+0000 7f780ed7e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f78081533a0 con 0x7f780806d460 2024-04-25T01:26:03.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.130+0000 7f780d31b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f78080724f0 0x7f7808072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:03.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.130+0000 7f780d31b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f78080724f0 0x7f7808072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58234/0 (socket says 172.21.15.112:58234) 2024-04-25T01:26:03.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.130+0000 7f780d31b640 1 -- 172.21.15.112:0/1929228186 learned_addr learned my addr 172.21.15.112:0/1929228186 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:03.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.130+0000 7f7807fff640 1 --2- 172.21.15.112:0/1929228186 >> v2:172.21.15.187:3300/0 conn(0x7f780806d460 0x7f7808071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:03.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.130+0000 7f780d31b640 1 -- 172.21.15.112:0/1929228186 >> v2:172.21.15.187:3300/0 conn(0x7f780806d460 msgr2=0x7f7808071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:03.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.130+0000 7f780d31b640 1 --2- 172.21.15.112:0/1929228186 >> v2:172.21.15.187:3300/0 conn(0x7f780806d460 0x7f7808071fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:03.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.130+0000 7f780d31b640 1 -- 172.21.15.112:0/1929228186 >> v1:172.21.15.187:6789/0 conn(0x7f780814fbe0 legacy=0x7f78080c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:03.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.130+0000 7f780d31b640 1 -- 172.21.15.112:0/1929228186 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780806efb0 con 0x7f78080724f0 2024-04-25T01:26:03.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.131+0000 7f780d31b640 1 --2- 172.21.15.112:0/1929228186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78080724f0 0x7f7808072900 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f78080c13a0 tx=0x7f780002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08089db2bb6a2b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:03.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.131+0000 7f78077fe640 1 -- 172.21.15.112:0/1929228186 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800036070 con 0x7f78080724f0 2024-04-25T01:26:03.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.131+0000 7f78077fe640 1 -- 172.21.15.112:0/1929228186 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7800031600 con 0x7f78080724f0 2024-04-25T01:26:03.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.131+0000 7f78077fe640 1 -- 172.21.15.112:0/1929228186 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78000319d0 con 0x7f78080724f0 2024-04-25T01:26:03.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.131+0000 7f780ed7e640 1 -- 172.21.15.112:0/1929228186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78080724f0 msgr2=0x7f7808072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:03.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.131+0000 7f780ed7e640 1 --2- 172.21.15.112:0/1929228186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78080724f0 0x7f7808072900 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f78080c13a0 tx=0x7f780002a660 comp rx=0 tx=0).stop 2024-04-25T01:26:03.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.131+0000 7f780ed7e640 1 -- 172.21.15.112:0/1929228186 shutdown_connections 2024-04-25T01:26:03.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.131+0000 7f780ed7e640 1 --2- 172.21.15.112:0/1929228186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78080724f0 0x7f7808072900 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:03.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.131+0000 7f780ed7e640 1 --2- 172.21.15.112:0/1929228186 >> v2:172.21.15.187:3300/0 conn(0x7f780806d460 0x7f7808071fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:03.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.131+0000 7f780ed7e640 1 -- 172.21.15.112:0/1929228186 >> 172.21.15.112:0/1929228186 conn(0x7f78080c3a10 msgr2=0x7f78080c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:03.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.131+0000 7f780ed7e640 1 -- 172.21.15.112:0/1929228186 shutdown_connections 2024-04-25T01:26:03.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.131+0000 7f780ed7e640 1 -- 172.21.15.112:0/1929228186 wait complete. 2024-04-25T01:26:03.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.132+0000 7f780ed7e640 1 Processor -- start 2024-04-25T01:26:03.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.132+0000 7f780ed7e640 1 -- start start 2024-04-25T01:26:03.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.133+0000 7f780ed7e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f780806d460 0x7f78080c2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:03.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.133+0000 7f780ed7e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78080724f0 0x7f7808148de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:03.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.133+0000 7f780ed7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f780814fbe0 0x7f78081428c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:03.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.133+0000 7f780ed7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f78080c3030 con 0x7f780814fbe0 2024-04-25T01:26:03.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.133+0000 7f780ed7e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7808153030 con 0x7f780806d460 2024-04-25T01:26:03.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.133+0000 7f780ed7e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f78080c31b0 con 0x7f78080724f0 2024-04-25T01:26:03.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.133+0000 7f780d31b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f780814fbe0 0x7f78081428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:03.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.133+0000 7f780d31b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f780814fbe0 0x7f78081428c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58244/0 (socket says 172.21.15.112:58244) 2024-04-25T01:26:03.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.133+0000 7f780d31b640 1 -- 172.21.15.112:0/3251754686 learned_addr learned my addr 172.21.15.112:0/3251754686 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:03.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.133+0000 7f780d31b640 1 -- 172.21.15.112:0/3251754686 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78080724f0 msgr2=0x7f7808148de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:03.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.133+0000 7f780d31b640 1 --2- 172.21.15.112:0/3251754686 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78080724f0 0x7f7808148de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:03.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.133+0000 7f780d31b640 1 -- 172.21.15.112:0/3251754686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f780806d460 msgr2=0x7f78080c2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:03.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.133+0000 7f780d31b640 1 --2- 172.21.15.112:0/3251754686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f780806d460 0x7f78080c2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:03.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.133+0000 7f780d31b640 1 -- 172.21.15.112:0/3251754686 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78081ed8b0 con 0x7f780814fbe0 2024-04-25T01:26:03.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.134+0000 7f780d31b640 1 --2- 172.21.15.112:0/3251754686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f780814fbe0 0x7f78081428c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7800009f40 tx=0x7f7800004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:03.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.134+0000 7f78057fa640 1 -- 172.21.15.112:0/3251754686 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800036050 con 0x7f780814fbe0 2024-04-25T01:26:03.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.134+0000 7f78057fa640 1 -- 172.21.15.112:0/3251754686 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7800042070 con 0x7f780814fbe0 2024-04-25T01:26:03.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.134+0000 7f780ed7e640 1 -- 172.21.15.112:0/3251754686 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78081ecaf0 con 0x7f780814fbe0 2024-04-25T01:26:03.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.134+0000 7f78057fa640 1 -- 172.21.15.112:0/3251754686 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800009510 con 0x7f780814fbe0 2024-04-25T01:26:03.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.134+0000 7f780ed7e640 1 -- 172.21.15.112:0/3251754686 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78081ed0b0 con 0x7f780814fbe0 2024-04-25T01:26:03.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.135+0000 7f780ed7e640 1 -- 172.21.15.112:0/3251754686 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f780804f500 con 0x7f780814fbe0 2024-04-25T01:26:03.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.135+0000 7f78057fa640 1 -- 172.21.15.112:0/3251754686 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7800007bb0 con 0x7f780814fbe0 2024-04-25T01:26:03.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.136+0000 7f78057fa640 1 --2- 172.21.15.112:0/3251754686 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f77ec06be30 0x7f77ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:03.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.136+0000 7f78057fa640 1 -- 172.21.15.112:0/3251754686 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6521+0+0 (secure 0 0 0) 0x7f78000ac110 con 0x7f780814fbe0 2024-04-25T01:26:03.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.136+0000 7f780cb1a640 1 --2- 172.21.15.112:0/3251754686 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f77ec06be30 0x7f77ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:03.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.136+0000 7f780cb1a640 1 --2- 172.21.15.112:0/3251754686 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f77ec06be30 0x7f77ec06e2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f78080ad380 tx=0x7f77fc006ba0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:03.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.140+0000 7f78057fa640 1 -- 172.21.15.112:0/3251754686 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f780007a820 con 0x7f780814fbe0 2024-04-25T01:26:03.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.349+0000 7f780ed7e640 1 -- 172.21.15.112:0/3251754686 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "b", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f780814e500 con 0x7f780814fbe0 2024-04-25T01:26:03.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.409+0000 7f78057fa640 1 -- 172.21.15.112:0/3251754686 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "b", "var": "standby_count_wanted", "val": "0"}]=0 v9) ==== 113+0+0 (secure 0 0 0) 0x7f780007a120 con 0x7f780814fbe0 2024-04-25T01:26:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.411+0000 7f780ed7e640 1 -- 172.21.15.112:0/3251754686 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f77ec06be30 msgr2=0x7f77ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.411+0000 7f780ed7e640 1 --2- 172.21.15.112:0/3251754686 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f77ec06be30 0x7f77ec06e2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f78080ad380 tx=0x7f77fc006ba0 comp rx=0 tx=0).stop 2024-04-25T01:26:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.411+0000 7f780ed7e640 1 -- 172.21.15.112:0/3251754686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f780814fbe0 msgr2=0x7f78081428c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.411+0000 7f780ed7e640 1 --2- 172.21.15.112:0/3251754686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f780814fbe0 0x7f78081428c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7800009f40 tx=0x7f7800004a40 comp rx=0 tx=0).stop 2024-04-25T01:26:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.411+0000 7f780ed7e640 1 -- 172.21.15.112:0/3251754686 shutdown_connections 2024-04-25T01:26:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.411+0000 7f780ed7e640 1 --2- 172.21.15.112:0/3251754686 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f77ec06be30 0x7f77ec06e2f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.411+0000 7f780ed7e640 1 --2- 172.21.15.112:0/3251754686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f780814fbe0 0x7f78081428c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.411+0000 7f780ed7e640 1 --2- 172.21.15.112:0/3251754686 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78080724f0 0x7f7808148de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.411+0000 7f780ed7e640 1 --2- 172.21.15.112:0/3251754686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f780806d460 0x7f78080c2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.411+0000 7f780ed7e640 1 -- 172.21.15.112:0/3251754686 >> 172.21.15.112:0/3251754686 conn(0x7f78080c3a10 msgr2=0x7f780806d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.411+0000 7f780ed7e640 1 -- 172.21.15.112:0/3251754686 shutdown_connections 2024-04-25T01:26:03.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.411+0000 7f780ed7e640 1 -- 172.21.15.112:0/3251754686 wait complete. 2024-04-25T01:26:03.404 DEBUG:tasks.cephfs.filesystem:fs_config: {'session_timeout': 300} 2024-04-25T01:26:03.404 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set b allow_standby_replay false 2024-04-25T01:26:03.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.522+0000 7fd8f3dee640 1 Processor -- start 2024-04-25T01:26:03.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.522+0000 7fd8f3dee640 1 -- start start 2024-04-25T01:26:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.523+0000 7fd8f3dee640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8ec1564b0 0x7fd8ec0c4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.523+0000 7fd8f3dee640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8ec06dc20 con 0x7fd8ec14fa40 2024-04-25T01:26:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.523+0000 7fd8f3dee640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8ec06e210 con 0x7fd8ec0add20 2024-04-25T01:26:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.523+0000 7fd8f3dee640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ec06d9d0 con 0x7fd8ec1564b0 2024-04-25T01:26:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.523+0000 7fd8f1b8a640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd8ec14fa40 0x7fd8ec06e830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37192/0 (socket says 172.21.15.112:37192) 2024-04-25T01:26:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.523+0000 7fd8f1b8a640 1 -- 172.21.15.112:0/367256623 learned_addr learned my addr 172.21.15.112:0/367256623 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.523+0000 7fd8f0b88640 1 -- 172.21.15.112:0/367256623 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1316359273 0 0) 0x7fd8ec06dc20 con 0x7fd8ec14fa40 2024-04-25T01:26:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.523+0000 7fd8f0b88640 1 -- 172.21.15.112:0/367256623 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8d8003520 con 0x7fd8ec14fa40 2024-04-25T01:26:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.523+0000 7fd8f238b640 1 --2- 172.21.15.112:0/367256623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8ec1564b0 0x7fd8ec0c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f0b88640 1 -- 172.21.15.112:0/367256623 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2042715013 0 0) 0x7fd8d8003520 con 0x7fd8ec14fa40 2024-04-25T01:26:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f0b88640 1 -- 172.21.15.112:0/367256623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8ec1564b0 msgr2=0x7fd8ec0c4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f0b88640 1 --2- 172.21.15.112:0/367256623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8ec1564b0 0x7fd8ec0c4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f0b88640 1 -- 172.21.15.112:0/367256623 >> v1:172.21.15.187:6789/0 conn(0x7fd8ec0add20 legacy=0x7fd8ec155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f0b88640 1 -- 172.21.15.112:0/367256623 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec06e4f0 con 0x7fd8ec14fa40 2024-04-25T01:26:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f0b88640 1 -- 172.21.15.112:0/367256623 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fd8e8001670 con 0x7fd8ec14fa40 2024-04-25T01:26:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f0b88640 1 -- 172.21.15.112:0/367256623 >> v1:172.21.15.112:6789/0 conn(0x7fd8ec14fa40 legacy=0x7fd8ec06e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f0b88640 1 --2- 172.21.15.112:0/367256623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8d80039e0 0x7fd8d8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:03.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f0b88640 1 --2- 172.21.15.112:0/367256623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d8004800 0x7fd8d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:03.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f0b88640 1 --2- 172.21.15.112:0/367256623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8ec1564b0 0x7fd8d8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:03.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f0b88640 1 -- 172.21.15.112:0/367256623 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec06d9d0 con 0x7fd8d8004800 2024-04-25T01:26:03.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f0b88640 1 -- 172.21.15.112:0/367256623 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d800baf0 con 0x7fd8d80039e0 2024-04-25T01:26:03.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f0b88640 1 -- 172.21.15.112:0/367256623 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd8d800bcf0 con 0x7fd8ec1564b0 2024-04-25T01:26:03.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f1389640 1 --2- 172.21.15.112:0/367256623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d8004800 0x7fd8d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:03.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f238b640 1 --2- 172.21.15.112:0/367256623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8ec1564b0 0x7fd8d8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:03.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f1b8a640 1 --2- 172.21.15.112:0/367256623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8d80039e0 0x7fd8d8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:03.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f1389640 1 -- 172.21.15.112:0/367256623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8ec1564b0 msgr2=0x7fd8d8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:03.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f1389640 1 --2- 172.21.15.112:0/367256623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8ec1564b0 0x7fd8d8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:03.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f1389640 1 -- 172.21.15.112:0/367256623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8d80039e0 msgr2=0x7fd8d8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:03.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f1389640 1 --2- 172.21.15.112:0/367256623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8d80039e0 0x7fd8d8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.524+0000 7fd8f1389640 1 -- 172.21.15.112:0/367256623 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d8003520 con 0x7fd8d8004800 2024-04-25T01:26:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.525+0000 7fd8f1389640 1 --2- 172.21.15.112:0/367256623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d8004800 0x7fd8d8004c80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc001750 tx=0x7fd8d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f864772cb60a4aef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.525+0000 7fd8f0b88640 1 -- 172.21.15.112:0/367256623 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc001370 con 0x7fd8d8004800 2024-04-25T01:26:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.525+0000 7fd8f0b88640 1 -- 172.21.15.112:0/367256623 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8dc0052d0 con 0x7fd8d8004800 2024-04-25T01:26:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.525+0000 7fd8f0b88640 1 -- 172.21.15.112:0/367256623 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc02abe0 con 0x7fd8d8004800 2024-04-25T01:26:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.525+0000 7fd8f3dee640 1 -- 172.21.15.112:0/367256623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d8004800 msgr2=0x7fd8d8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.525+0000 7fd8f3dee640 1 --2- 172.21.15.112:0/367256623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d8004800 0x7fd8d8004c80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc001750 tx=0x7fd8d8005830 comp rx=0 tx=0).stop 2024-04-25T01:26:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.525+0000 7fd8f238b640 1 -- 172.21.15.112:0/367256623 reap_dead start 2024-04-25T01:26:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.525+0000 7fd8f3dee640 1 -- 172.21.15.112:0/367256623 shutdown_connections 2024-04-25T01:26:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.525+0000 7fd8f3dee640 1 -- 172.21.15.112:0/367256623 >> 172.21.15.112:0/367256623 conn(0x7fd8ec0c49b0 msgr2=0x7fd8ec0c4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.526+0000 7fd8f3dee640 1 -- 172.21.15.112:0/367256623 shutdown_connections 2024-04-25T01:26:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.526+0000 7fd8f3dee640 1 -- 172.21.15.112:0/367256623 wait complete. 2024-04-25T01:26:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.527+0000 7fd8f3dee640 1 Processor -- start 2024-04-25T01:26:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.527+0000 7fd8f3dee640 1 -- start start 2024-04-25T01:26:03.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.527+0000 7fd8f3dee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d8004800 0x7fd8ec14d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:03.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.527+0000 7fd8f3dee640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8ec1462f0 0x7fd8ec0c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:03.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.527+0000 7fd8f3dee640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8ec146820 0x7fd8ec0c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:03.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.527+0000 7fd8f3dee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec06ef40 con 0x7fd8d8004800 2024-04-25T01:26:03.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.527+0000 7fd8f3dee640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec0c28b0 con 0x7fd8ec1462f0 2024-04-25T01:26:03.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.527+0000 7fd8f3dee640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ec06f0c0 con 0x7fd8ec146820 2024-04-25T01:26:03.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.527+0000 7fd8f1b8a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d8004800 0x7fd8ec14d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:03.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.527+0000 7fd8f1b8a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d8004800 0x7fd8ec14d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58256/0 (socket says 172.21.15.112:58256) 2024-04-25T01:26:03.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.527+0000 7fd8f1b8a640 1 -- 172.21.15.112:0/369685543 learned_addr learned my addr 172.21.15.112:0/369685543 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:03.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.528+0000 7fd8f238b640 1 --2- 172.21.15.112:0/369685543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8ec146820 0x7fd8ec0c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:03.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.528+0000 7fd8f1389640 1 --2- 172.21.15.112:0/369685543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8ec1462f0 0x7fd8ec0c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:03.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.528+0000 7fd8f1b8a640 1 -- 172.21.15.112:0/369685543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8ec146820 msgr2=0x7fd8ec0c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:03.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.528+0000 7fd8f1b8a640 1 --2- 172.21.15.112:0/369685543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8ec146820 0x7fd8ec0c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:03.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.528+0000 7fd8f1b8a640 1 -- 172.21.15.112:0/369685543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8ec1462f0 msgr2=0x7fd8ec0c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:03.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.528+0000 7fd8f1b8a640 1 --2- 172.21.15.112:0/369685543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8ec1462f0 0x7fd8ec0c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:03.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.528+0000 7fd8f1b8a640 1 -- 172.21.15.112:0/369685543 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec1e1c90 con 0x7fd8d8004800 2024-04-25T01:26:03.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.528+0000 7fd8f238b640 1 --2- 172.21.15.112:0/369685543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8ec146820 0x7fd8ec0c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:03.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.528+0000 7fd8f1389640 1 --2- 172.21.15.112:0/369685543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8ec1462f0 0x7fd8ec0c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:03.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.528+0000 7fd8f1b8a640 1 --2- 172.21.15.112:0/369685543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d8004800 0x7fd8ec14d560 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80072c0 tx=0x7fd8e8007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:03.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.528+0000 7fd8e27fc640 1 -- 172.21.15.112:0/369685543 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e801a070 con 0x7fd8d8004800 2024-04-25T01:26:03.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.528+0000 7fd8e27fc640 1 -- 172.21.15.112:0/369685543 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8e800f960 con 0x7fd8d8004800 2024-04-25T01:26:03.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.528+0000 7fd8f3dee640 1 -- 172.21.15.112:0/369685543 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ec1e1f70 con 0x7fd8d8004800 2024-04-25T01:26:03.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.528+0000 7fd8e27fc640 1 -- 172.21.15.112:0/369685543 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e800fd30 con 0x7fd8d8004800 2024-04-25T01:26:03.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.528+0000 7fd8f3dee640 1 -- 172.21.15.112:0/369685543 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ec154fa0 con 0x7fd8d8004800 2024-04-25T01:26:03.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.529+0000 7fd8f3dee640 1 -- 172.21.15.112:0/369685543 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ec04f500 con 0x7fd8d8004800 2024-04-25T01:26:03.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.529+0000 7fd8e27fc640 1 -- 172.21.15.112:0/369685543 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd8e801d070 con 0x7fd8d8004800 2024-04-25T01:26:03.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.530+0000 7fd8e27fc640 1 --2- 172.21.15.112:0/369685543 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd8d006be60 0x7fd8d006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:03.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.530+0000 7fd8f1389640 1 --2- 172.21.15.112:0/369685543 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd8d006be60 0x7fd8d006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:03.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.530+0000 7fd8e27fc640 1 -- 172.21.15.112:0/369685543 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fd8e808aac0 con 0x7fd8d8004800 2024-04-25T01:26:03.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.530+0000 7fd8f1389640 1 --2- 172.21.15.112:0/369685543 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd8d006be60 0x7fd8d006e320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc032000 tx=0x7fd8dc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:03.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.535+0000 7fd8e27fc640 1 -- 172.21.15.112:0/369685543 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd8e80591d0 con 0x7fd8d8004800 2024-04-25T01:26:03.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:03.748+0000 7fd8f3dee640 1 -- 172.21.15.112:0/369685543 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "b", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd8ec14c4d0 con 0x7fd8d8004800 2024-04-25T01:26:04.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.411+0000 7fd8e27fc640 1 -- 172.21.15.112:0/369685543 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "b", "var": "allow_standby_replay", "val": "false"}]=0 v10) ==== 117+0+0 (secure 0 0 0) 0x7fd8e8058ad0 con 0x7fd8d8004800 2024-04-25T01:26:04.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.413+0000 7fd8f3dee640 1 -- 172.21.15.112:0/369685543 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd8d006be60 msgr2=0x7fd8d006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:04.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.413+0000 7fd8f3dee640 1 --2- 172.21.15.112:0/369685543 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd8d006be60 0x7fd8d006e320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc032000 tx=0x7fd8dc02e000 comp rx=0 tx=0).stop 2024-04-25T01:26:04.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.413+0000 7fd8f3dee640 1 -- 172.21.15.112:0/369685543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d8004800 msgr2=0x7fd8ec14d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:04.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.413+0000 7fd8f3dee640 1 --2- 172.21.15.112:0/369685543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d8004800 0x7fd8ec14d560 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80072c0 tx=0x7fd8e8007780 comp rx=0 tx=0).stop 2024-04-25T01:26:04.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.413+0000 7fd8f3dee640 1 -- 172.21.15.112:0/369685543 shutdown_connections 2024-04-25T01:26:04.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.413+0000 7fd8f3dee640 1 --2- 172.21.15.112:0/369685543 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd8d006be60 0x7fd8d006e320 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:04.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.413+0000 7fd8f3dee640 1 --2- 172.21.15.112:0/369685543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8ec146820 0x7fd8ec0c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:04.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.413+0000 7fd8f3dee640 1 --2- 172.21.15.112:0/369685543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8ec1462f0 0x7fd8ec0c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:04.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.413+0000 7fd8f3dee640 1 --2- 172.21.15.112:0/369685543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d8004800 0x7fd8ec14d560 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:04.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.413+0000 7fd8f3dee640 1 -- 172.21.15.112:0/369685543 >> 172.21.15.112:0/369685543 conn(0x7fd8ec0c49b0 msgr2=0x7fd8ec14ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:04.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.413+0000 7fd8f3dee640 1 -- 172.21.15.112:0/369685543 shutdown_connections 2024-04-25T01:26:04.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.413+0000 7fd8f3dee640 1 -- 172.21.15.112:0/369685543 wait complete. 2024-04-25T01:26:04.404 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set b session_timeout 300 2024-04-25T01:26:04.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.525+0000 7f9780822640 1 Processor -- start 2024-04-25T01:26:04.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.525+0000 7f9780822640 1 -- start start 2024-04-25T01:26:04.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.525+0000 7f9780822640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f977c06d460 0x7f977c071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:04.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.525+0000 7f9780822640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f977c0724f0 0x7f977c15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:04.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.525+0000 7f9780822640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f977c06f290 con 0x7f977c14fbe0 2024-04-25T01:26:04.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.526+0000 7f9780822640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f977c14ffc0 con 0x7f977c06d460 2024-04-25T01:26:04.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.526+0000 7f9780822640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f977c1533a0 con 0x7f977c0724f0 2024-04-25T01:26:04.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.526+0000 7f977ad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f977c0724f0 0x7f977c15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:04.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.526+0000 7f9779d9b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f977c06d460 0x7f977c071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:04.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.526+0000 7f977ad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f977c0724f0 0x7f977c15ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:44984/0 (socket says 172.21.15.112:44984) 2024-04-25T01:26:04.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.526+0000 7f977ad9d640 1 -- 172.21.15.112:0/873371723 learned_addr learned my addr 172.21.15.112:0/873371723 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:04.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.526+0000 7f977a59c640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f977c14fbe0 0x7f977c0c2920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:46862/0 (socket says 172.21.15.112:46862) 2024-04-25T01:26:04.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.526+0000 7f9779d9b640 1 -- 172.21.15.112:0/873371723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f977c0724f0 msgr2=0x7f977c15ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:04.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.526+0000 7f9779d9b640 1 --2- 172.21.15.112:0/873371723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f977c0724f0 0x7f977c15ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:04.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.526+0000 7f9779d9b640 1 -- 172.21.15.112:0/873371723 >> v1:172.21.15.187:6789/0 conn(0x7f977c14fbe0 legacy=0x7f977c0c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:04.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.526+0000 7f977959a640 1 -- 172.21.15.112:0/873371723 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 724524995 0 0) 0x7f977c06f290 con 0x7f977c14fbe0 2024-04-25T01:26:04.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.526+0000 7f9779d9b640 1 -- 172.21.15.112:0/873371723 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c06efb0 con 0x7f977c06d460 2024-04-25T01:26:04.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.526+0000 7f977ad9d640 1 --2- 172.21.15.112:0/873371723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f977c0724f0 0x7f977c15ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:26:04.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.527+0000 7f9779d9b640 1 --2- 172.21.15.112:0/873371723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f977c06d460 0x7f977c071fb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f977000ba40 tx=0x7f977002c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3048ac37a522e35e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:04.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.527+0000 7f977959a640 1 -- 172.21.15.112:0/873371723 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770031710 con 0x7f977c06d460 2024-04-25T01:26:04.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.527+0000 7f977959a640 1 -- 172.21.15.112:0/873371723 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97700318b0 con 0x7f977c06d460 2024-04-25T01:26:04.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.527+0000 7f977959a640 1 -- 172.21.15.112:0/873371723 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770031c80 con 0x7f977c06d460 2024-04-25T01:26:04.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.528+0000 7f9780822640 1 -- 172.21.15.112:0/873371723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f977c06d460 msgr2=0x7f977c071fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:04.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.528+0000 7f9780822640 1 --2- 172.21.15.112:0/873371723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f977c06d460 0x7f977c071fb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f977000ba40 tx=0x7f977002c7b0 comp rx=0 tx=0).stop 2024-04-25T01:26:04.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.528+0000 7f9780822640 1 -- 172.21.15.112:0/873371723 shutdown_connections 2024-04-25T01:26:04.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.528+0000 7f9780822640 1 --2- 172.21.15.112:0/873371723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f977c0724f0 0x7f977c15ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:04.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.528+0000 7f9780822640 1 --2- 172.21.15.112:0/873371723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f977c06d460 0x7f977c071fb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:04.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.528+0000 7f9780822640 1 -- 172.21.15.112:0/873371723 >> 172.21.15.112:0/873371723 conn(0x7f977c0c3a10 msgr2=0x7f977c0c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:04.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.528+0000 7f9780822640 1 -- 172.21.15.112:0/873371723 shutdown_connections 2024-04-25T01:26:04.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.528+0000 7f9780822640 1 -- 172.21.15.112:0/873371723 wait complete. 2024-04-25T01:26:04.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.529+0000 7f9780822640 1 Processor -- start 2024-04-25T01:26:04.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.529+0000 7f9780822640 1 -- start start 2024-04-25T01:26:04.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.530+0000 7f9780822640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f977c06d460 0x7f977c06dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:04.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.530+0000 7f9780822640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f977c0724f0 0x7f977c1dcf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:04.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.530+0000 7f9780822640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f977c14fbe0 0x7f977c1dd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:04.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.530+0000 7f9780822640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f977c0c3030 con 0x7f977c0724f0 2024-04-25T01:26:04.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.530+0000 7f9780822640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f977c153030 con 0x7f977c14fbe0 2024-04-25T01:26:04.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.530+0000 7f9780822640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f977c0c31b0 con 0x7f977c06d460 2024-04-25T01:26:04.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.530+0000 7f9779d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f977c0724f0 0x7f977c1dcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:04.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.530+0000 7f9779d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f977c0724f0 0x7f977c1dcf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58270/0 (socket says 172.21.15.112:58270) 2024-04-25T01:26:04.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.530+0000 7f9779d9b640 1 -- 172.21.15.112:0/3382226199 learned_addr learned my addr 172.21.15.112:0/3382226199 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:04.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.530+0000 7f977a59c640 1 --2- 172.21.15.112:0/3382226199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f977c06d460 0x7f977c06dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:04.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.530+0000 7f977ad9d640 1 --2- 172.21.15.112:0/3382226199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f977c14fbe0 0x7f977c1dd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:04.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.530+0000 7f9779d9b640 1 -- 172.21.15.112:0/3382226199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f977c06d460 msgr2=0x7f977c06dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:04.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.530+0000 7f9779d9b640 1 --2- 172.21.15.112:0/3382226199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f977c06d460 0x7f977c06dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:04.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.530+0000 7f9779d9b640 1 -- 172.21.15.112:0/3382226199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f977c14fbe0 msgr2=0x7f977c1dd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:04.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.530+0000 7f9779d9b640 1 --2- 172.21.15.112:0/3382226199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f977c14fbe0 0x7f977c1dd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:04.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.530+0000 7f9779d9b640 1 -- 172.21.15.112:0/3382226199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c1de750 con 0x7f977c0724f0 2024-04-25T01:26:04.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.531+0000 7f9779d9b640 1 --2- 172.21.15.112:0/3382226199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f977c0724f0 0x7f977c1dcf90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9770000c00 tx=0x7f97700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:04.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.531+0000 7f976affd640 1 -- 172.21.15.112:0/3382226199 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770041070 con 0x7f977c0724f0 2024-04-25T01:26:04.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.531+0000 7f976affd640 1 -- 172.21.15.112:0/3382226199 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9770033c40 con 0x7f977c0724f0 2024-04-25T01:26:04.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.531+0000 7f9780822640 1 -- 172.21.15.112:0/3382226199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977c148ee0 con 0x7f977c0724f0 2024-04-25T01:26:04.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.531+0000 7f976affd640 1 -- 172.21.15.112:0/3382226199 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770006e10 con 0x7f977c0724f0 2024-04-25T01:26:04.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.531+0000 7f9780822640 1 -- 172.21.15.112:0/3382226199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977c1494a0 con 0x7f977c0724f0 2024-04-25T01:26:04.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.532+0000 7f9780822640 1 -- 172.21.15.112:0/3382226199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f977c1496a0 con 0x7f977c0724f0 2024-04-25T01:26:04.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.532+0000 7f976affd640 1 -- 172.21.15.112:0/3382226199 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9770044070 con 0x7f977c0724f0 2024-04-25T01:26:04.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.533+0000 7f976affd640 1 --2- 172.21.15.112:0/3382226199 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f975c06be30 0x7f975c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:04.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.533+0000 7f976affd640 1 -- 172.21.15.112:0/3382226199 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f97700abf20 con 0x7f977c0724f0 2024-04-25T01:26:04.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.533+0000 7f977a59c640 1 --2- 172.21.15.112:0/3382226199 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f975c06be30 0x7f975c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:04.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.533+0000 7f977a59c640 1 --2- 172.21.15.112:0/3382226199 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f975c06be30 0x7f975c06e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f976c0012e0 tx=0x7f976c005eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:04.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.537+0000 7f976affd640 1 -- 172.21.15.112:0/3382226199 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f977c1496a0 con 0x7f977c0724f0 2024-04-25T01:26:04.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:04.745+0000 7f9780822640 1 -- 172.21.15.112:0/3382226199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "b", "var": "session_timeout", "val": "300"} v 0) -- 0x7f977c04f500 con 0x7f977c0724f0 2024-04-25T01:26:05.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.424+0000 7f976affd640 1 -- 172.21.15.112:0/3382226199 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "b", "var": "session_timeout", "val": "300"}]=0 v11) ==== 110+0+0 (secure 0 0 0) 0x7f977007a6d0 con 0x7f977c0724f0 2024-04-25T01:26:05.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.426+0000 7f9780822640 1 -- 172.21.15.112:0/3382226199 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f975c06be30 msgr2=0x7f975c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:05.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.426+0000 7f9780822640 1 --2- 172.21.15.112:0/3382226199 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f975c06be30 0x7f975c06e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f976c0012e0 tx=0x7f976c005eb0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.426+0000 7f9780822640 1 -- 172.21.15.112:0/3382226199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f977c0724f0 msgr2=0x7f977c1dcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:05.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.426+0000 7f9780822640 1 --2- 172.21.15.112:0/3382226199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f977c0724f0 0x7f977c1dcf90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9770000c00 tx=0x7f97700046c0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.426+0000 7f9780822640 1 -- 172.21.15.112:0/3382226199 shutdown_connections 2024-04-25T01:26:05.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.426+0000 7f9780822640 1 --2- 172.21.15.112:0/3382226199 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f975c06be30 0x7f975c06e2f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.426+0000 7f9780822640 1 --2- 172.21.15.112:0/3382226199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f977c14fbe0 0x7f977c1dd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.426+0000 7f9780822640 1 --2- 172.21.15.112:0/3382226199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f977c0724f0 0x7f977c1dcf90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.426+0000 7f9780822640 1 --2- 172.21.15.112:0/3382226199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f977c06d460 0x7f977c06dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.426+0000 7f9780822640 1 -- 172.21.15.112:0/3382226199 >> 172.21.15.112:0/3382226199 conn(0x7f977c0c3a10 msgr2=0x7f977c14dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:05.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.426+0000 7f9780822640 1 -- 172.21.15.112:0/3382226199 shutdown_connections 2024-04-25T01:26:05.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.426+0000 7f9780822640 1 -- 172.21.15.112:0/3382226199 wait complete. 2024-04-25T01:26:05.418 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'b' 2024-04-25T01:26:05.418 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:26:05.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.535+0000 7fe2cb509640 1 Processor -- start 2024-04-25T01:26:05.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.535+0000 7fe2cb509640 1 -- start start 2024-04-25T01:26:05.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2cb509640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c406b160 0x7fe2c406b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:05.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2cb509640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe2c4158ff0 0x7fe2c4159480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:05.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2cb509640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2c4156460 con 0x7fe2c40a9490 2024-04-25T01:26:05.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2cb509640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe2c40c4a50 con 0x7fe2c4158ff0 2024-04-25T01:26:05.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2cb509640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c4156740 con 0x7fe2c406b160 2024-04-25T01:26:05.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2c8aa4640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe2c40a9490 0x7fe2c406ba80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37202/0 (socket says 172.21.15.112:37202) 2024-04-25T01:26:05.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2c8aa4640 1 -- 172.21.15.112:0/4122216960 learned_addr learned my addr 172.21.15.112:0/4122216960 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:05.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2bbfff640 1 -- 172.21.15.112:0/4122216960 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3781150510 0 0) 0x7fe2c4156460 con 0x7fe2c40a9490 2024-04-25T01:26:05.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2bbfff640 1 -- 172.21.15.112:0/4122216960 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2b0003520 con 0x7fe2c40a9490 2024-04-25T01:26:05.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2c9aa6640 1 --2- 172.21.15.112:0/4122216960 >> v2:172.21.15.187:3300/0 conn(0x7fe2c4158ff0 0x7fe2c4159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:05.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2c92a5640 1 --2- 172.21.15.112:0/4122216960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c406b160 0x7fe2c406b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:05.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2bbfff640 1 -- 172.21.15.112:0/4122216960 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 355550860 0 0) 0x7fe2b0003520 con 0x7fe2c40a9490 2024-04-25T01:26:05.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2bbfff640 1 -- 172.21.15.112:0/4122216960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c406b160 msgr2=0x7fe2c406b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:05.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2bbfff640 1 --2- 172.21.15.112:0/4122216960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c406b160 0x7fe2c406b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2bbfff640 1 -- 172.21.15.112:0/4122216960 >> v2:172.21.15.187:3300/0 conn(0x7fe2c4158ff0 msgr2=0x7fe2c4159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:05.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2bbfff640 1 --2- 172.21.15.112:0/4122216960 >> v2:172.21.15.187:3300/0 conn(0x7fe2c4158ff0 0x7fe2c4159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2bbfff640 1 -- 172.21.15.112:0/4122216960 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c40c2c80 con 0x7fe2c40a9490 2024-04-25T01:26:05.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2bbfff640 1 -- 172.21.15.112:0/4122216960 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fe2b4001670 con 0x7fe2c40a9490 2024-04-25T01:26:05.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2c92a5640 1 --2- 172.21.15.112:0/4122216960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c406b160 0x7fe2c406b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:05.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2bbfff640 1 -- 172.21.15.112:0/4122216960 >> v1:172.21.15.112:6789/0 conn(0x7fe2c40a9490 legacy=0x7fe2c406ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:05.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2bbfff640 1 --2- 172.21.15.112:0/4122216960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2b00039b0 0x7fe2b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:05.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2bbfff640 1 --2- 172.21.15.112:0/4122216960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b0004800 0x7fe2b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:05.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2bbfff640 1 --2- 172.21.15.112:0/4122216960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c406b160 0x7fe2b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:05.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2bbfff640 1 -- 172.21.15.112:0/4122216960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c40c4a50 con 0x7fe2b0004800 2024-04-25T01:26:05.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2bbfff640 1 -- 172.21.15.112:0/4122216960 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c4156740 con 0x7fe2b00039b0 2024-04-25T01:26:05.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.536+0000 7fe2bbfff640 1 -- 172.21.15.112:0/4122216960 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b000baf0 con 0x7fe2c406b160 2024-04-25T01:26:05.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.537+0000 7fe2c8aa4640 1 --2- 172.21.15.112:0/4122216960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b0004800 0x7fe2b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:05.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.537+0000 7fe2c9aa6640 1 --2- 172.21.15.112:0/4122216960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c406b160 0x7fe2b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:05.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.537+0000 7fe2c92a5640 1 --2- 172.21.15.112:0/4122216960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2b00039b0 0x7fe2b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:05.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.537+0000 7fe2c8aa4640 1 -- 172.21.15.112:0/4122216960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c406b160 msgr2=0x7fe2b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:05.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.537+0000 7fe2c8aa4640 1 --2- 172.21.15.112:0/4122216960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c406b160 0x7fe2b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.537+0000 7fe2c8aa4640 1 -- 172.21.15.112:0/4122216960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2b00039b0 msgr2=0x7fe2b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:05.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.537+0000 7fe2c8aa4640 1 --2- 172.21.15.112:0/4122216960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2b00039b0 0x7fe2b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.537+0000 7fe2c8aa4640 1 -- 172.21.15.112:0/4122216960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b0003520 con 0x7fe2b0004800 2024-04-25T01:26:05.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.537+0000 7fe2c8aa4640 1 --2- 172.21.15.112:0/4122216960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b0004800 0x7fe2b0004c80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4007fd0 tx=0x7fe2b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ea5156855b74cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:05.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.538+0000 7fe2bbfff640 1 -- 172.21.15.112:0/4122216960 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4007d50 con 0x7fe2b0004800 2024-04-25T01:26:05.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.538+0000 7fe2bbfff640 1 -- 172.21.15.112:0/4122216960 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2b40058d0 con 0x7fe2b0004800 2024-04-25T01:26:05.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.538+0000 7fe2bbfff640 1 -- 172.21.15.112:0/4122216960 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4005ce0 con 0x7fe2b0004800 2024-04-25T01:26:05.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.538+0000 7fe2cb509640 1 -- 172.21.15.112:0/4122216960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b0004800 msgr2=0x7fe2b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:05.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.538+0000 7fe2cb509640 1 --2- 172.21.15.112:0/4122216960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b0004800 0x7fe2b0004c80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4007fd0 tx=0x7fe2b0005830 comp rx=0 tx=0).stop 2024-04-25T01:26:05.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.538+0000 7fe2c9aa6640 1 -- 172.21.15.112:0/4122216960 reap_dead start 2024-04-25T01:26:05.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.538+0000 7fe2cb509640 1 -- 172.21.15.112:0/4122216960 shutdown_connections 2024-04-25T01:26:05.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.538+0000 7fe2cb509640 1 -- 172.21.15.112:0/4122216960 >> 172.21.15.112:0/4122216960 conn(0x7fe2c40c4c00 msgr2=0x7fe2c40c4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:05.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.538+0000 7fe2cb509640 1 -- 172.21.15.112:0/4122216960 shutdown_connections 2024-04-25T01:26:05.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.538+0000 7fe2cb509640 1 -- 172.21.15.112:0/4122216960 wait complete. 2024-04-25T01:26:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.539+0000 7fe2cb509640 1 Processor -- start 2024-04-25T01:26:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.539+0000 7fe2cb509640 1 -- start start 2024-04-25T01:26:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.540+0000 7fe2cb509640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2b0004800 0x7fe2c41eec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.540+0000 7fe2cb509640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c40c6d90 0x7fe2c41e25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.540+0000 7fe2cb509640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c413dce0 0x7fe2c413e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.540+0000 7fe2cb509640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c413f150 con 0x7fe2c40c6d90 2024-04-25T01:26:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.540+0000 7fe2cb509640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c413efd0 con 0x7fe2b0004800 2024-04-25T01:26:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.540+0000 7fe2cb509640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c41599c0 con 0x7fe2c413dce0 2024-04-25T01:26:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.540+0000 7fe2c8aa4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c40c6d90 0x7fe2c41e25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.540+0000 7fe2c8aa4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c40c6d90 0x7fe2c41e25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58302/0 (socket says 172.21.15.112:58302) 2024-04-25T01:26:05.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.540+0000 7fe2c8aa4640 1 -- 172.21.15.112:0/4047041529 learned_addr learned my addr 172.21.15.112:0/4047041529 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:05.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.540+0000 7fe2c9aa6640 1 --2- 172.21.15.112:0/4047041529 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c413dce0 0x7fe2c413e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:05.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.540+0000 7fe2c92a5640 1 --2- 172.21.15.112:0/4047041529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2b0004800 0x7fe2c41eec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:05.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.540+0000 7fe2c8aa4640 1 -- 172.21.15.112:0/4047041529 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c413dce0 msgr2=0x7fe2c413e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:05.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.540+0000 7fe2c8aa4640 1 --2- 172.21.15.112:0/4047041529 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c413dce0 0x7fe2c413e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.540+0000 7fe2c8aa4640 1 -- 172.21.15.112:0/4047041529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2b0004800 msgr2=0x7fe2c41eec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:05.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.540+0000 7fe2c8aa4640 1 --2- 172.21.15.112:0/4047041529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2b0004800 0x7fe2c41eec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.540+0000 7fe2c8aa4640 1 -- 172.21.15.112:0/4047041529 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c41394a0 con 0x7fe2c40c6d90 2024-04-25T01:26:05.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.540+0000 7fe2c9aa6640 1 --2- 172.21.15.112:0/4047041529 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c413dce0 0x7fe2c413e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:05.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.540+0000 7fe2c8aa4640 1 --2- 172.21.15.112:0/4047041529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c40c6d90 0x7fe2c41e25c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400c380 tx=0x7fe2c4155940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:05.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.541+0000 7fe2b9ffb640 1 -- 172.21.15.112:0/4047041529 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4001c80 con 0x7fe2c40c6d90 2024-04-25T01:26:05.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.541+0000 7fe2b9ffb640 1 -- 172.21.15.112:0/4047041529 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2b4041070 con 0x7fe2c40c6d90 2024-04-25T01:26:05.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.541+0000 7fe2cb509640 1 -- 172.21.15.112:0/4047041529 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c41539c0 con 0x7fe2c40c6d90 2024-04-25T01:26:05.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.541+0000 7fe2b9ffb640 1 -- 172.21.15.112:0/4047041529 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40049f0 con 0x7fe2c40c6d90 2024-04-25T01:26:05.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.541+0000 7fe2cb509640 1 -- 172.21.15.112:0/4047041529 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c41e2b00 con 0x7fe2c40c6d90 2024-04-25T01:26:05.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.542+0000 7fe2cb509640 1 -- 172.21.15.112:0/4047041529 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c41537b0 con 0x7fe2c40c6d90 2024-04-25T01:26:05.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.542+0000 7fe2b9ffb640 1 -- 172.21.15.112:0/4047041529 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe2b403b070 con 0x7fe2c40c6d90 2024-04-25T01:26:05.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.543+0000 7fe2b9ffb640 1 --2- 172.21.15.112:0/4047041529 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe2a806be30 0x7fe2a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:05.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.543+0000 7fe2c92a5640 1 --2- 172.21.15.112:0/4047041529 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe2a806be30 0x7fe2a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:05.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.543+0000 7fe2b9ffb640 1 -- 172.21.15.112:0/4047041529 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fe2b40aa950 con 0x7fe2c40c6d90 2024-04-25T01:26:05.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.543+0000 7fe2c92a5640 1 --2- 172.21.15.112:0/4047041529 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe2a806be30 0x7fe2a806e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40c78e0 tx=0x7fe2c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:05.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.547+0000 7fe2b9ffb640 1 -- 172.21.15.112:0/4047041529 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe2b4079060 con 0x7fe2c40c6d90 2024-04-25T01:26:05.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.757+0000 7fe2cb509640 1 -- 172.21.15.112:0/4047041529 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2c40682a0 con 0x7fe2c40c6d90 2024-04-25T01:26:05.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.758+0000 7fe2b9ffb640 1 -- 172.21.15.112:0/4047041529 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+6491 (secure 0 0 0) 0x7fe2b4078960 con 0x7fe2c40c6d90 2024-04-25T01:26:05.738 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:05.738 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/4047219598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4047219598},{"type":"v1","addr":"172.21.15.187:6834","nonce":4047219598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2204640907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2204640907},{"type":"v1","addr":"172.21.15.187:6835","nonce":2204640907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:25:52.687260+0000","modified":"2024-04-25T01:25:55.702719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/2115984531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2115984531},{"type":"v1","addr":"172.21.15.112:6837","nonce":2115984531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1},{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:26:01.740521+0000","modified":"2024-04-25T01:26:04.746955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":8,"state":"up:active","state_seq":6,"addr":"172.21.15.112:6835/278686477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":278686477},{"type":"v1","addr":"172.21.15.112:6835","nonce":278686477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":2}]} 2024-04-25T01:26:05.738 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 11 2024-04-25T01:26:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.760+0000 7fe2cb509640 1 -- 172.21.15.112:0/4047041529 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe2a806be30 msgr2=0x7fe2a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.760+0000 7fe2cb509640 1 --2- 172.21.15.112:0/4047041529 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe2a806be30 0x7fe2a806e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40c78e0 tx=0x7fe2c0008040 comp rx=0 tx=0).stop 2024-04-25T01:26:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.760+0000 7fe2cb509640 1 -- 172.21.15.112:0/4047041529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c40c6d90 msgr2=0x7fe2c41e25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.760+0000 7fe2cb509640 1 --2- 172.21.15.112:0/4047041529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c40c6d90 0x7fe2c41e25c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400c380 tx=0x7fe2c4155940 comp rx=0 tx=0).stop 2024-04-25T01:26:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.760+0000 7fe2cb509640 1 -- 172.21.15.112:0/4047041529 shutdown_connections 2024-04-25T01:26:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.760+0000 7fe2cb509640 1 --2- 172.21.15.112:0/4047041529 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe2a806be30 0x7fe2a806e2f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.760+0000 7fe2cb509640 1 --2- 172.21.15.112:0/4047041529 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c413dce0 0x7fe2c413e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.760+0000 7fe2cb509640 1 --2- 172.21.15.112:0/4047041529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c40c6d90 0x7fe2c41e25c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.760+0000 7fe2cb509640 1 --2- 172.21.15.112:0/4047041529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2b0004800 0x7fe2c41eec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.760+0000 7fe2cb509640 1 -- 172.21.15.112:0/4047041529 >> 172.21.15.112:0/4047041529 conn(0x7fe2c40c4c00 msgr2=0x7fe2c406b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.760+0000 7fe2cb509640 1 -- 172.21.15.112:0/4047041529 shutdown_connections 2024-04-25T01:26:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.760+0000 7fe2cb509640 1 -- 172.21.15.112:0/4047041529 wait complete. 2024-04-25T01:26:05.750 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:26:05.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.867+0000 7f8f5c495640 1 Processor -- start 2024-04-25T01:26:05.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.867+0000 7f8f5c495640 1 -- start start 2024-04-25T01:26:05.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.868+0000 7f8f5c495640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8f54153030 0x7f8f540c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:05.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.868+0000 7f8f5c495640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8f54156370 0x7f8f540c5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:05.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.868+0000 7f8f5c495640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f540c8910 con 0x7f8f540c9af0 2024-04-25T01:26:05.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.868+0000 7f8f5c495640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f8f54159160 con 0x7f8f54153030 2024-04-25T01:26:05.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.868+0000 7f8f5c495640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8f540c8bf0 con 0x7f8f54156370 2024-04-25T01:26:05.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.868+0000 7f8f5a231640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8f54153030 0x7f8f540c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:05.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.868+0000 7f8f5a231640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8f54153030 0x7f8f540c70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58314/0 (socket says 172.21.15.112:58314) 2024-04-25T01:26:05.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.868+0000 7f8f5a231640 1 -- 172.21.15.112:0/2328073096 learned_addr learned my addr 172.21.15.112:0/2328073096 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:05.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.869+0000 7f8f5aa32640 1 --2- 172.21.15.112:0/2328073096 >> v2:172.21.15.187:3300/0 conn(0x7f8f54156370 0x7f8f540c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:05.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.869+0000 7f8f5a231640 1 -- 172.21.15.112:0/2328073096 >> v2:172.21.15.187:3300/0 conn(0x7f8f54156370 msgr2=0x7f8f540c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:05.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.869+0000 7f8f5a231640 1 --2- 172.21.15.112:0/2328073096 >> v2:172.21.15.187:3300/0 conn(0x7f8f54156370 0x7f8f540c5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.869+0000 7f8f5a231640 1 -- 172.21.15.112:0/2328073096 >> v1:172.21.15.187:6789/0 conn(0x7f8f540c9af0 legacy=0x7f8f541595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:05.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.869+0000 7f8f5a231640 1 -- 172.21.15.112:0/2328073096 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f54159d00 con 0x7f8f54153030 2024-04-25T01:26:05.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.869+0000 7f8f5aa32640 1 --2- 172.21.15.112:0/2328073096 >> v2:172.21.15.187:3300/0 conn(0x7f8f54156370 0x7f8f540c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:05.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.869+0000 7f8f5a231640 1 --2- 172.21.15.112:0/2328073096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f54153030 0x7f8f540c70a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8f540ae6a0 tx=0x7f8f5002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a30d898b0f65ac0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:05.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.869+0000 7f8f5922f640 1 -- 172.21.15.112:0/2328073096 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50036070 con 0x7f8f54153030 2024-04-25T01:26:05.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.869+0000 7f8f5922f640 1 -- 172.21.15.112:0/2328073096 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f50031600 con 0x7f8f54153030 2024-04-25T01:26:05.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.869+0000 7f8f5922f640 1 -- 172.21.15.112:0/2328073096 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f500319d0 con 0x7f8f54153030 2024-04-25T01:26:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.870+0000 7f8f5c495640 1 -- 172.21.15.112:0/2328073096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f54153030 msgr2=0x7f8f540c70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.870+0000 7f8f5c495640 1 --2- 172.21.15.112:0/2328073096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f54153030 0x7f8f540c70a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8f540ae6a0 tx=0x7f8f5002a750 comp rx=0 tx=0).stop 2024-04-25T01:26:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.870+0000 7f8f5c495640 1 -- 172.21.15.112:0/2328073096 shutdown_connections 2024-04-25T01:26:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.870+0000 7f8f5c495640 1 --2- 172.21.15.112:0/2328073096 >> v2:172.21.15.187:3300/0 conn(0x7f8f54156370 0x7f8f540c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.870+0000 7f8f5c495640 1 --2- 172.21.15.112:0/2328073096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f54153030 0x7f8f540c70a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.870+0000 7f8f5c495640 1 -- 172.21.15.112:0/2328073096 >> 172.21.15.112:0/2328073096 conn(0x7f8f54074a00 msgr2=0x7f8f540c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.870+0000 7f8f5c495640 1 -- 172.21.15.112:0/2328073096 shutdown_connections 2024-04-25T01:26:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.870+0000 7f8f5c495640 1 -- 172.21.15.112:0/2328073096 wait complete. 2024-04-25T01:26:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.871+0000 7f8f5c495640 1 Processor -- start 2024-04-25T01:26:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.871+0000 7f8f5c495640 1 -- start start 2024-04-25T01:26:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f5c495640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f540c9af0 0x7f8f541eea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f5c495640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f54153030 0x7f8f541eefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f5c495640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f54156370 0x7f8f541e0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f5c495640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f540c6020 con 0x7f8f540c9af0 2024-04-25T01:26:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f5c495640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8f540c5ea0 con 0x7f8f54156370 2024-04-25T01:26:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f5c495640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8f540c61a0 con 0x7f8f54153030 2024-04-25T01:26:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f5a231640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f540c9af0 0x7f8f541eea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f5a231640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f540c9af0 0x7f8f541eea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58320/0 (socket says 172.21.15.112:58320) 2024-04-25T01:26:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f5a231640 1 -- 172.21.15.112:0/3285333358 learned_addr learned my addr 172.21.15.112:0/3285333358 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f59a30640 1 --2- 172.21.15.112:0/3285333358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f54153030 0x7f8f541eefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f5aa32640 1 --2- 172.21.15.112:0/3285333358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f54156370 0x7f8f541e0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f5a231640 1 -- 172.21.15.112:0/3285333358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f54153030 msgr2=0x7f8f541eefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f5a231640 1 --2- 172.21.15.112:0/3285333358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f54153030 0x7f8f541eefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f5a231640 1 -- 172.21.15.112:0/3285333358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f54156370 msgr2=0x7f8f541e0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:05.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f5a231640 1 --2- 172.21.15.112:0/3285333358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f54156370 0x7f8f541e0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:05.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f5a231640 1 -- 172.21.15.112:0/3285333358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f540ae0f0 con 0x7f8f540c9af0 2024-04-25T01:26:05.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f59a30640 1 --2- 172.21.15.112:0/3285333358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f54153030 0x7f8f541eefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:05.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f5a231640 1 --2- 172.21.15.112:0/3285333358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f540c9af0 0x7f8f541eea80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8f50000f80 tx=0x7f8f50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:05.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f4affd640 1 -- 172.21.15.112:0/3285333358 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50036050 con 0x7f8f540c9af0 2024-04-25T01:26:05.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f4affd640 1 -- 172.21.15.112:0/3285333358 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f50042070 con 0x7f8f540c9af0 2024-04-25T01:26:05.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.872+0000 7f8f5c495640 1 -- 172.21.15.112:0/3285333358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f540abb60 con 0x7f8f540c9af0 2024-04-25T01:26:05.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.873+0000 7f8f4affd640 1 -- 172.21.15.112:0/3285333358 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50009510 con 0x7f8f540c9af0 2024-04-25T01:26:05.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.873+0000 7f8f5c495640 1 -- 172.21.15.112:0/3285333358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f541ef650 con 0x7f8f540c9af0 2024-04-25T01:26:05.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.874+0000 7f8f5c495640 1 -- 172.21.15.112:0/3285333358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f540c63e0 con 0x7f8f540c9af0 2024-04-25T01:26:05.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.874+0000 7f8f4affd640 1 -- 172.21.15.112:0/3285333358 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f50007c70 con 0x7f8f540c9af0 2024-04-25T01:26:05.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.874+0000 7f8f4affd640 1 --2- 172.21.15.112:0/3285333358 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8f3806be30 0x7f8f3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:05.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.874+0000 7f8f4affd640 1 -- 172.21.15.112:0/3285333358 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f8f500ac0f0 con 0x7f8f540c9af0 2024-04-25T01:26:05.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.874+0000 7f8f59a30640 1 --2- 172.21.15.112:0/3285333358 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8f3806be30 0x7f8f3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:05.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.875+0000 7f8f59a30640 1 --2- 172.21.15.112:0/3285333358 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8f3806be30 0x7f8f3806e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8f44006fd0 tx=0x7f8f44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:05.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:05.879+0000 7f8f4affd640 1 -- 172.21.15.112:0/3285333358 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8f540c63e0 con 0x7f8f540c9af0 2024-04-25T01:26:06.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.054+0000 7f8f5c495640 1 -- 172.21.15.112:0/3285333358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f5406ff70 con 0x7f8f540c9af0 2024-04-25T01:26:06.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.056+0000 7f8f4affd640 1 -- 172.21.15.112:0/3285333358 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20367 (secure 0 0 0) 0x7f8f5406ff70 con 0x7f8f540c9af0 2024-04-25T01:26:06.036 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:06.036 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:26:03.405271+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-25T01:25:59.478275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-25T01:26:00.719358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:26:06.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.058+0000 7f8f5c495640 1 -- 172.21.15.112:0/3285333358 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8f3806be30 msgr2=0x7f8f3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:06.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.058+0000 7f8f5c495640 1 --2- 172.21.15.112:0/3285333358 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8f3806be30 0x7f8f3806e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8f44006fd0 tx=0x7f8f44008040 comp rx=0 tx=0).stop 2024-04-25T01:26:06.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.058+0000 7f8f5c495640 1 -- 172.21.15.112:0/3285333358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f540c9af0 msgr2=0x7f8f541eea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:06.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.058+0000 7f8f5c495640 1 --2- 172.21.15.112:0/3285333358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f540c9af0 0x7f8f541eea80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8f50000f80 tx=0x7f8f50004a40 comp rx=0 tx=0).stop 2024-04-25T01:26:06.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.058+0000 7f8f5c495640 1 -- 172.21.15.112:0/3285333358 shutdown_connections 2024-04-25T01:26:06.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.058+0000 7f8f5c495640 1 --2- 172.21.15.112:0/3285333358 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8f3806be30 0x7f8f3806e2f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.058+0000 7f8f5c495640 1 --2- 172.21.15.112:0/3285333358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f54156370 0x7f8f541e0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.058+0000 7f8f5c495640 1 --2- 172.21.15.112:0/3285333358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f54153030 0x7f8f541eefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.058+0000 7f8f5c495640 1 --2- 172.21.15.112:0/3285333358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f540c9af0 0x7f8f541eea80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.058+0000 7f8f5c495640 1 -- 172.21.15.112:0/3285333358 >> 172.21.15.112:0/3285333358 conn(0x7f8f54074a00 msgr2=0x7f8f54156cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:06.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.058+0000 7f8f5c495640 1 -- 172.21.15.112:0/3285333358 shutdown_connections 2024-04-25T01:26:06.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.058+0000 7f8f5c495640 1 -- 172.21.15.112:0/3285333358 wait complete. 2024-04-25T01:26:06.049 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T01:26:06.049 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:26:06.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.165+0000 7f2265a76640 1 Processor -- start 2024-04-25T01:26:06.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.165+0000 7f2265a76640 1 -- start start 2024-04-25T01:26:06.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.166+0000 7f2265a76640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f226006c760 0x7f22600c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.166+0000 7f2265a76640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2260073140 0x7f22600c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.166+0000 7f2265a76640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22600716a0 con 0x7f226014d7c0 2024-04-25T01:26:06.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.166+0000 7f2265a76640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2260153b20 con 0x7f226006c760 2024-04-25T01:26:06.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.166+0000 7f2265a76640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2260071980 con 0x7f2260073140 2024-04-25T01:26:06.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.166+0000 7f225ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2260073140 0x7f22600c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:06.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.166+0000 7f225ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2260073140 0x7f22600c21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:45034/0 (socket says 172.21.15.112:45034) 2024-04-25T01:26:06.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.166+0000 7f225effd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f226006c760 0x7f22600c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:06.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.166+0000 7f225ffff640 1 -- 172.21.15.112:0/802389524 learned_addr learned my addr 172.21.15.112:0/802389524 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:06.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.167+0000 7f225ffff640 1 -- 172.21.15.112:0/802389524 >> v2:172.21.15.187:3300/0 conn(0x7f226006c760 msgr2=0x7f22600c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:06.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.167+0000 7f225ffff640 1 --2- 172.21.15.112:0/802389524 >> v2:172.21.15.187:3300/0 conn(0x7f226006c760 0x7f22600c66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.167+0000 7f225ffff640 1 -- 172.21.15.112:0/802389524 >> v1:172.21.15.187:6789/0 conn(0x7f226014d7c0 legacy=0x7f2260153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:06.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.167+0000 7f225ffff640 1 -- 172.21.15.112:0/802389524 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22601537b0 con 0x7f2260073140 2024-04-25T01:26:06.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.167+0000 7f225effd640 1 --2- 172.21.15.112:0/802389524 >> v2:172.21.15.187:3300/0 conn(0x7f226006c760 0x7f22600c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:26:06.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.167+0000 7f225ffff640 1 --2- 172.21.15.112:0/802389524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2260073140 0x7f22600c21d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2258009e50 tx=0x7f225802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ef8f00276c8624c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:06.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.167+0000 7f225e7fc640 1 -- 172.21.15.112:0/802389524 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258036070 con 0x7f2260073140 2024-04-25T01:26:06.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.168+0000 7f225e7fc640 1 -- 172.21.15.112:0/802389524 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2258031600 con 0x7f2260073140 2024-04-25T01:26:06.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.168+0000 7f225e7fc640 1 -- 172.21.15.112:0/802389524 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22580319d0 con 0x7f2260073140 2024-04-25T01:26:06.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.168+0000 7f2265a76640 1 -- 172.21.15.112:0/802389524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2260073140 msgr2=0x7f22600c21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:06.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.168+0000 7f2265a76640 1 --2- 172.21.15.112:0/802389524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2260073140 0x7f22600c21d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2258009e50 tx=0x7f225802a710 comp rx=0 tx=0).stop 2024-04-25T01:26:06.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.168+0000 7f2265a76640 1 -- 172.21.15.112:0/802389524 shutdown_connections 2024-04-25T01:26:06.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.168+0000 7f2265a76640 1 --2- 172.21.15.112:0/802389524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2260073140 0x7f22600c21d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.168+0000 7f2265a76640 1 --2- 172.21.15.112:0/802389524 >> v2:172.21.15.187:3300/0 conn(0x7f226006c760 0x7f22600c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.168+0000 7f2265a76640 1 -- 172.21.15.112:0/802389524 >> 172.21.15.112:0/802389524 conn(0x7f226006b160 msgr2=0x7f226006b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:06.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.168+0000 7f2265a76640 1 -- 172.21.15.112:0/802389524 shutdown_connections 2024-04-25T01:26:06.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.168+0000 7f2265a76640 1 -- 172.21.15.112:0/802389524 wait complete. 2024-04-25T01:26:06.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.169+0000 7f2265a76640 1 Processor -- start 2024-04-25T01:26:06.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.169+0000 7f2265a76640 1 -- start start 2024-04-25T01:26:06.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.170+0000 7f2265a76640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f226006c760 0x7f2260148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.170+0000 7f2265a76640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2260073140 0x7f22601e7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.170+0000 7f2265a76640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f226014b6b0 0x7f226014a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.170+0000 7f2265a76640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2260073750 con 0x7f2260073140 2024-04-25T01:26:06.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.170+0000 7f2265a76640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f22600735d0 con 0x7f226014b6b0 2024-04-25T01:26:06.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.170+0000 7f2265a76640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f22600c2710 con 0x7f226006c760 2024-04-25T01:26:06.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.170+0000 7f225effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2260073140 0x7f22601e7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:06.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.170+0000 7f225effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2260073140 0x7f22601e7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58326/0 (socket says 172.21.15.112:58326) 2024-04-25T01:26:06.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.170+0000 7f225effd640 1 -- 172.21.15.112:0/364011028 learned_addr learned my addr 172.21.15.112:0/364011028 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:06.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.170+0000 7f225f7fe640 1 --2- 172.21.15.112:0/364011028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f226006c760 0x7f2260148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:06.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.170+0000 7f225ffff640 1 --2- 172.21.15.112:0/364011028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f226014b6b0 0x7f226014a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:06.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.170+0000 7f225effd640 1 -- 172.21.15.112:0/364011028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f226006c760 msgr2=0x7f2260148b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:06.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.170+0000 7f225effd640 1 --2- 172.21.15.112:0/364011028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f226006c760 0x7f2260148b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.170+0000 7f225effd640 1 -- 172.21.15.112:0/364011028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f226014b6b0 msgr2=0x7f226014a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:06.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.170+0000 7f225effd640 1 --2- 172.21.15.112:0/364011028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f226014b6b0 0x7f226014a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.170+0000 7f225effd640 1 -- 172.21.15.112:0/364011028 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2260149b40 con 0x7f2260073140 2024-04-25T01:26:06.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.171+0000 7f225effd640 1 --2- 172.21.15.112:0/364011028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2260073140 0x7f22601e7200 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f225400e960 tx=0x7f225400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:06.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.171+0000 7f2247fff640 1 -- 172.21.15.112:0/364011028 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254017070 con 0x7f2260073140 2024-04-25T01:26:06.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.171+0000 7f2247fff640 1 -- 172.21.15.112:0/364011028 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f225400ca00 con 0x7f2260073140 2024-04-25T01:26:06.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.171+0000 7f2247fff640 1 -- 172.21.15.112:0/364011028 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225400cdd0 con 0x7f2260073140 2024-04-25T01:26:06.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.171+0000 7f2265a76640 1 -- 172.21.15.112:0/364011028 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22601e7740 con 0x7f2260073140 2024-04-25T01:26:06.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.171+0000 7f2265a76640 1 -- 172.21.15.112:0/364011028 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22601e9e40 con 0x7f2260073140 2024-04-25T01:26:06.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.172+0000 7f2265a76640 1 -- 172.21.15.112:0/364011028 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22600682a0 con 0x7f2260073140 2024-04-25T01:26:06.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.172+0000 7f2247fff640 1 -- 172.21.15.112:0/364011028 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f225401a070 con 0x7f2260073140 2024-04-25T01:26:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.173+0000 7f2247fff640 1 --2- 172.21.15.112:0/364011028 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f224c06be30 0x7f224c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.173+0000 7f225f7fe640 1 --2- 172.21.15.112:0/364011028 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f224c06be30 0x7f224c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.173+0000 7f2247fff640 1 -- 172.21.15.112:0/364011028 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f225408af90 con 0x7f2260073140 2024-04-25T01:26:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.173+0000 7f225f7fe640 1 --2- 172.21.15.112:0/364011028 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f224c06be30 0x7f224c06e2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f226014e6e0 tx=0x7f2250009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:06.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.177+0000 7f2247fff640 1 -- 172.21.15.112:0/364011028 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2254059050 con 0x7f2260073140 2024-04-25T01:26:06.186 INFO:tasks.ceph.mds.a.smithi112.stderr:2024-04-25T01:26:06.207+0000 7f069b5c6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T01:26:06.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.348+0000 7f2265a76640 1 -- 172.21.15.112:0/364011028 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f226006c540 con 0x7f224c06be30 2024-04-25T01:26:06.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.359+0000 7f2247fff640 1 -- 172.21.15.112:0/364011028 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+478292 (secure 0 0 0) 0x7f226006c540 con 0x7f224c06be30 2024-04-25T01:26:06.343 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:06.347 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T01:26:06.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.363+0000 7f2265a76640 1 -- 172.21.15.112:0/364011028 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f224c06be30 msgr2=0x7f224c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:06.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.363+0000 7f2265a76640 1 --2- 172.21.15.112:0/364011028 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f224c06be30 0x7f224c06e2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f226014e6e0 tx=0x7f2250009040 comp rx=0 tx=0).stop 2024-04-25T01:26:06.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.363+0000 7f2265a76640 1 -- 172.21.15.112:0/364011028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2260073140 msgr2=0x7f22601e7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:06.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.363+0000 7f2265a76640 1 --2- 172.21.15.112:0/364011028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2260073140 0x7f22601e7200 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f225400e960 tx=0x7f225400ee20 comp rx=0 tx=0).stop 2024-04-25T01:26:06.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.363+0000 7f2265a76640 1 -- 172.21.15.112:0/364011028 shutdown_connections 2024-04-25T01:26:06.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.363+0000 7f2265a76640 1 --2- 172.21.15.112:0/364011028 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f224c06be30 0x7f224c06e2f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.363+0000 7f2265a76640 1 --2- 172.21.15.112:0/364011028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f226014b6b0 0x7f226014a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.363+0000 7f2265a76640 1 --2- 172.21.15.112:0/364011028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2260073140 0x7f22601e7200 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.363+0000 7f2265a76640 1 --2- 172.21.15.112:0/364011028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f226006c760 0x7f2260148b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.363+0000 7f2265a76640 1 -- 172.21.15.112:0/364011028 >> 172.21.15.112:0/364011028 conn(0x7f226006b160 msgr2=0x7f226006b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:06.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.363+0000 7f2265a76640 1 -- 172.21.15.112:0/364011028 shutdown_connections 2024-04-25T01:26:06.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.363+0000 7f2265a76640 1 -- 172.21.15.112:0/364011028 wait complete. 2024-04-25T01:26:06.353 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-04-25T01:26:04.650314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":201,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219024,"kb_used_data":2836,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755696,"statfs":{"total":773094113280,"available":772869832704,"internally_reserved":0,"allocated":2904064,"data_stored":1759027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.258553"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.407176+0000","last_change":"2024-04-25T01:25:52.353978+0000","last_active":"2024-04-25T01:26:00.407176+0000","last_peered":"2024-04-25T01:26:00.407176+0000","last_clean":"2024-04-25T01:26:00.407176+0000","last_became_active":"2024-04-25T01:25:52.353649+0000","last_became_peered":"2024-04-25T01:25:52.353649+0000","last_unstale":"2024-04-25T01:26:00.407176+0000","last_undegraded":"2024-04-25T01:26:00.407176+0000","last_fullsized":"2024-04-25T01:26:00.407176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.396308+0000","last_change":"2024-04-25T01:25:53.364257+0000","last_active":"2024-04-25T01:26:00.396308+0000","last_peered":"2024-04-25T01:26:00.396308+0000","last_clean":"2024-04-25T01:26:00.396308+0000","last_became_active":"2024-04-25T01:25:53.363409+0000","last_became_peered":"2024-04-25T01:25:53.363409+0000","last_unstale":"2024-04-25T01:26:00.396308+0000","last_undegraded":"2024-04-25T01:26:00.396308+0000","last_fullsized":"2024-04-25T01:26:00.396308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.394429+0000","last_change":"2024-04-25T01:26:00.394429+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.394429+0000","last_undegraded":"2024-04-25T01:26:00.394429+0000","last_fullsized":"2024-04-25T01:26:00.394429+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.396411+0000","last_change":"2024-04-25T01:25:52.349185+0000","last_active":"2024-04-25T01:26:00.396411+0000","last_peered":"2024-04-25T01:26:00.396411+0000","last_clean":"2024-04-25T01:26:00.396411+0000","last_became_active":"2024-04-25T01:25:52.348931+0000","last_became_peered":"2024-04-25T01:25:52.348931+0000","last_unstale":"2024-04-25T01:26:00.396411+0000","last_undegraded":"2024-04-25T01:26:00.396411+0000","last_fullsized":"2024-04-25T01:26:00.396411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.406854+0000","last_change":"2024-04-25T01:25:53.374150+0000","last_active":"2024-04-25T01:26:00.406854+0000","last_peered":"2024-04-25T01:26:00.406854+0000","last_clean":"2024-04-25T01:26:00.406854+0000","last_became_active":"2024-04-25T01:25:53.373755+0000","last_became_peered":"2024-04-25T01:25:53.373755+0000","last_unstale":"2024-04-25T01:26:00.406854+0000","last_undegraded":"2024-04-25T01:26:00.406854+0000","last_fullsized":"2024-04-25T01:26:00.406854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.399622+0000","last_change":"2024-04-25T01:26:00.399622+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.399622+0000","last_undegraded":"2024-04-25T01:26:00.399622+0000","last_fullsized":"2024-04-25T01:26:00.399622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397720+0000","last_change":"2024-04-25T01:25:52.356403+0000","last_active":"2024-04-25T01:26:00.397720+0000","last_peered":"2024-04-25T01:26:00.397720+0000","last_clean":"2024-04-25T01:26:00.397720+0000","last_became_active":"2024-04-25T01:25:52.355822+0000","last_became_peered":"2024-04-25T01:25:52.355822+0000","last_unstale":"2024-04-25T01:26:00.397720+0000","last_undegraded":"2024-04-25T01:26:00.397720+0000","last_fullsized":"2024-04-25T01:26:00.397720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395791+0000","last_change":"2024-04-25T01:25:53.370061+0000","last_active":"2024-04-25T01:26:00.395791+0000","last_peered":"2024-04-25T01:26:00.395791+0000","last_clean":"2024-04-25T01:26:00.395791+0000","last_became_active":"2024-04-25T01:25:53.369721+0000","last_became_peered":"2024-04-25T01:25:53.369721+0000","last_unstale":"2024-04-25T01:26:00.395791+0000","last_undegraded":"2024-04-25T01:26:00.395791+0000","last_fullsized":"2024-04-25T01:26:00.395791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.400393+0000","last_change":"2024-04-25T01:26:00.400393+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.400393+0000","last_undegraded":"2024-04-25T01:26:00.400393+0000","last_fullsized":"2024-04-25T01:26:00.400393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397159+0000","last_change":"2024-04-25T01:25:52.359139+0000","last_active":"2024-04-25T01:26:00.397159+0000","last_peered":"2024-04-25T01:26:00.397159+0000","last_clean":"2024-04-25T01:26:00.397159+0000","last_became_active":"2024-04-25T01:25:52.358841+0000","last_became_peered":"2024-04-25T01:25:52.358841+0000","last_unstale":"2024-04-25T01:26:00.397159+0000","last_undegraded":"2024-04-25T01:26:00.397159+0000","last_fullsized":"2024-04-25T01:26:00.397159+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395904+0000","last_change":"2024-04-25T01:25:53.368739+0000","last_active":"2024-04-25T01:26:00.395904+0000","last_peered":"2024-04-25T01:26:00.395904+0000","last_clean":"2024-04-25T01:26:00.395904+0000","last_became_active":"2024-04-25T01:25:53.368470+0000","last_became_peered":"2024-04-25T01:25:53.368470+0000","last_unstale":"2024-04-25T01:26:00.395904+0000","last_undegraded":"2024-04-25T01:26:00.395904+0000","last_fullsized":"2024-04-25T01:26:00.395904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:29.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.399661+0000","last_change":"2024-04-25T01:26:00.399661+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.399661+0000","last_undegraded":"2024-04-25T01:26:00.399661+0000","last_fullsized":"2024-04-25T01:26:00.399661+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397409+0000","last_change":"2024-04-25T01:25:53.372385+0000","last_active":"2024-04-25T01:26:00.397409+0000","last_peered":"2024-04-25T01:26:00.397409+0000","last_clean":"2024-04-25T01:26:00.397409+0000","last_became_active":"2024-04-25T01:25:53.372064+0000","last_became_peered":"2024-04-25T01:25:53.372064+0000","last_unstale":"2024-04-25T01:26:00.397409+0000","last_undegraded":"2024-04-25T01:26:00.397409+0000","last_fullsized":"2024-04-25T01:26:00.397409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397315+0000","last_change":"2024-04-25T01:25:52.358616+0000","last_active":"2024-04-25T01:26:00.397315+0000","last_peered":"2024-04-25T01:26:00.397315+0000","last_clean":"2024-04-25T01:26:00.397315+0000","last_became_active":"2024-04-25T01:25:52.358318+0000","last_became_peered":"2024-04-25T01:25:52.358318+0000","last_unstale":"2024-04-25T01:26:00.397315+0000","last_undegraded":"2024-04-25T01:26:00.397315+0000","last_fullsized":"2024-04-25T01:26:00.397315+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.399686+0000","last_change":"2024-04-25T01:26:00.399686+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.399686+0000","last_undegraded":"2024-04-25T01:26:00.399686+0000","last_fullsized":"2024-04-25T01:26:00.399686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395938+0000","last_change":"2024-04-25T01:25:53.361074+0000","last_active":"2024-04-25T01:26:00.395938+0000","last_peered":"2024-04-25T01:26:00.395938+0000","last_clean":"2024-04-25T01:26:00.395938+0000","last_became_active":"2024-04-25T01:25:53.360802+0000","last_became_peered":"2024-04-25T01:25:53.360802+0000","last_unstale":"2024-04-25T01:26:00.395938+0000","last_undegraded":"2024-04-25T01:26:00.395938+0000","last_fullsized":"2024-04-25T01:26:00.395938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397389+0000","last_change":"2024-04-25T01:25:52.348841+0000","last_active":"2024-04-25T01:26:00.397389+0000","last_peered":"2024-04-25T01:26:00.397389+0000","last_clean":"2024-04-25T01:26:00.397389+0000","last_became_active":"2024-04-25T01:25:52.348550+0000","last_became_peered":"2024-04-25T01:25:52.348550+0000","last_unstale":"2024-04-25T01:26:00.397389+0000","last_undegraded":"2024-04-25T01:26:00.397389+0000","last_fullsized":"2024-04-25T01:26:00.397389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.399685+0000","last_change":"2024-04-25T01:26:00.399685+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.399685+0000","last_undegraded":"2024-04-25T01:26:00.399685+0000","last_fullsized":"2024-04-25T01:26:00.399685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395643+0000","last_change":"2024-04-25T01:25:53.361790+0000","last_active":"2024-04-25T01:26:00.395643+0000","last_peered":"2024-04-25T01:26:00.395643+0000","last_clean":"2024-04-25T01:26:00.395643+0000","last_became_active":"2024-04-25T01:25:53.361542+0000","last_became_peered":"2024-04-25T01:25:53.361542+0000","last_unstale":"2024-04-25T01:26:00.395643+0000","last_undegraded":"2024-04-25T01:26:00.395643+0000","last_fullsized":"2024-04-25T01:26:00.395643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.396893+0000","last_change":"2024-04-25T01:25:52.353926+0000","last_active":"2024-04-25T01:26:00.396893+0000","last_peered":"2024-04-25T01:26:00.396893+0000","last_clean":"2024-04-25T01:26:00.396893+0000","last_became_active":"2024-04-25T01:25:52.353656+0000","last_became_peered":"2024-04-25T01:25:52.353656+0000","last_unstale":"2024-04-25T01:26:00.396893+0000","last_undegraded":"2024-04-25T01:26:00.396893+0000","last_fullsized":"2024-04-25T01:26:00.396893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.397203+0000","last_change":"2024-04-25T01:26:00.397203+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.397203+0000","last_undegraded":"2024-04-25T01:26:00.397203+0000","last_fullsized":"2024-04-25T01:26:00.397203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.401205+0000","last_change":"2024-04-25T01:25:53.361653+0000","last_active":"2024-04-25T01:26:00.401205+0000","last_peered":"2024-04-25T01:26:00.401205+0000","last_clean":"2024-04-25T01:26:00.401205+0000","last_became_active":"2024-04-25T01:25:53.361265+0000","last_became_peered":"2024-04-25T01:25:53.361265+0000","last_unstale":"2024-04-25T01:26:00.401205+0000","last_undegraded":"2024-04-25T01:26:00.401205+0000","last_fullsized":"2024-04-25T01:26:00.401205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397415+0000","last_change":"2024-04-25T01:25:52.353836+0000","last_active":"2024-04-25T01:26:00.397415+0000","last_peered":"2024-04-25T01:26:00.397415+0000","last_clean":"2024-04-25T01:26:00.397415+0000","last_became_active":"2024-04-25T01:25:52.353552+0000","last_became_peered":"2024-04-25T01:25:52.353552+0000","last_unstale":"2024-04-25T01:26:00.397415+0000","last_undegraded":"2024-04-25T01:26:00.397415+0000","last_fullsized":"2024-04-25T01:26:00.397415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.399687+0000","last_change":"2024-04-25T01:26:00.399687+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.399687+0000","last_undegraded":"2024-04-25T01:26:00.399687+0000","last_fullsized":"2024-04-25T01:26:00.399687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.407075+0000","last_change":"2024-04-25T01:25:53.370111+0000","last_active":"2024-04-25T01:26:00.407075+0000","last_peered":"2024-04-25T01:26:00.407075+0000","last_clean":"2024-04-25T01:26:00.407075+0000","last_became_active":"2024-04-25T01:25:53.369837+0000","last_became_peered":"2024-04-25T01:25:53.369837+0000","last_unstale":"2024-04-25T01:26:00.407075+0000","last_undegraded":"2024-04-25T01:26:00.407075+0000","last_fullsized":"2024-04-25T01:26:00.407075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.398688+0000","last_change":"2024-04-25T01:25:52.353269+0000","last_active":"2024-04-25T01:26:00.398688+0000","last_peered":"2024-04-25T01:26:00.398688+0000","last_clean":"2024-04-25T01:26:00.398688+0000","last_became_active":"2024-04-25T01:25:52.352991+0000","last_became_peered":"2024-04-25T01:25:52.352991+0000","last_unstale":"2024-04-25T01:26:00.398688+0000","last_undegraded":"2024-04-25T01:26:00.398688+0000","last_fullsized":"2024-04-25T01:26:00.398688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.399650+0000","last_change":"2024-04-25T01:26:00.399650+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.399650+0000","last_undegraded":"2024-04-25T01:26:00.399650+0000","last_fullsized":"2024-04-25T01:26:00.399650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.396538+0000","last_change":"2024-04-25T01:25:53.364967+0000","last_active":"2024-04-25T01:26:00.396538+0000","last_peered":"2024-04-25T01:26:00.396538+0000","last_clean":"2024-04-25T01:26:00.396538+0000","last_became_active":"2024-04-25T01:25:53.364694+0000","last_became_peered":"2024-04-25T01:25:53.364694+0000","last_unstale":"2024-04-25T01:26:00.396538+0000","last_undegraded":"2024-04-25T01:26:00.396538+0000","last_fullsized":"2024-04-25T01:26:00.396538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397127+0000","last_change":"2024-04-25T01:25:52.355395+0000","last_active":"2024-04-25T01:26:00.397127+0000","last_peered":"2024-04-25T01:26:00.397127+0000","last_clean":"2024-04-25T01:26:00.397127+0000","last_became_active":"2024-04-25T01:25:52.355137+0000","last_became_peered":"2024-04-25T01:25:52.355137+0000","last_unstale":"2024-04-25T01:26:00.397127+0000","last_undegraded":"2024-04-25T01:26:00.397127+0000","last_fullsized":"2024-04-25T01:26:00.397127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:26.467968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.394675+0000","last_change":"2024-04-25T01:26:00.394675+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.394675+0000","last_undegraded":"2024-04-25T01:26:00.394675+0000","last_fullsized":"2024-04-25T01:26:00.394675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397049+0000","last_change":"2024-04-25T01:25:53.366944+0000","last_active":"2024-04-25T01:26:00.397049+0000","last_peered":"2024-04-25T01:26:00.397049+0000","last_clean":"2024-04-25T01:26:00.397049+0000","last_became_active":"2024-04-25T01:25:53.366693+0000","last_became_peered":"2024-04-25T01:25:53.366693+0000","last_unstale":"2024-04-25T01:26:00.397049+0000","last_undegraded":"2024-04-25T01:26:00.397049+0000","last_fullsized":"2024-04-25T01:26:00.397049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397911+0000","last_change":"2024-04-25T01:25:52.356842+0000","last_active":"2024-04-25T01:26:00.397911+0000","last_peered":"2024-04-25T01:26:00.397911+0000","last_clean":"2024-04-25T01:26:00.397911+0000","last_became_active":"2024-04-25T01:25:52.356396+0000","last_became_peered":"2024-04-25T01:25:52.356396+0000","last_unstale":"2024-04-25T01:26:00.397911+0000","last_undegraded":"2024-04-25T01:26:00.397911+0000","last_fullsized":"2024-04-25T01:26:00.397911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.405375+0000","last_change":"2024-04-25T01:26:00.405375+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.405375+0000","last_undegraded":"2024-04-25T01:26:00.405375+0000","last_fullsized":"2024-04-25T01:26:00.405375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395999+0000","last_change":"2024-04-25T01:25:53.368164+0000","last_active":"2024-04-25T01:26:00.395999+0000","last_peered":"2024-04-25T01:26:00.395999+0000","last_clean":"2024-04-25T01:26:00.395999+0000","last_became_active":"2024-04-25T01:25:53.367854+0000","last_became_peered":"2024-04-25T01:25:53.367854+0000","last_unstale":"2024-04-25T01:26:00.395999+0000","last_undegraded":"2024-04-25T01:26:00.395999+0000","last_fullsized":"2024-04-25T01:26:00.395999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397081+0000","last_change":"2024-04-25T01:25:52.350128+0000","last_active":"2024-04-25T01:26:00.397081+0000","last_peered":"2024-04-25T01:26:00.397081+0000","last_clean":"2024-04-25T01:26:00.397081+0000","last_became_active":"2024-04-25T01:25:52.349874+0000","last_became_peered":"2024-04-25T01:25:52.349874+0000","last_unstale":"2024-04-25T01:26:00.397081+0000","last_undegraded":"2024-04-25T01:26:00.397081+0000","last_fullsized":"2024-04-25T01:26:00.397081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.396231+0000","last_change":"2024-04-25T01:26:00.396231+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.396231+0000","last_undegraded":"2024-04-25T01:26:00.396231+0000","last_fullsized":"2024-04-25T01:26:00.396231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.401247+0000","last_change":"2024-04-25T01:25:53.361696+0000","last_active":"2024-04-25T01:26:00.401247+0000","last_peered":"2024-04-25T01:26:00.401247+0000","last_clean":"2024-04-25T01:26:00.401247+0000","last_became_active":"2024-04-25T01:25:53.361156+0000","last_became_peered":"2024-04-25T01:25:53.361156+0000","last_unstale":"2024-04-25T01:26:00.401247+0000","last_undegraded":"2024-04-25T01:26:00.401247+0000","last_fullsized":"2024-04-25T01:26:00.401247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:27.472315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395584+0000","last_change":"2024-04-25T01:25:52.354499+0000","last_active":"2024-04-25T01:26:00.395584+0000","last_peered":"2024-04-25T01:26:00.395584+0000","last_clean":"2024-04-25T01:26:00.395584+0000","last_became_active":"2024-04-25T01:25:52.354193+0000","last_became_peered":"2024-04-25T01:25:52.354193+0000","last_unstale":"2024-04-25T01:26:00.395584+0000","last_undegraded":"2024-04-25T01:26:00.395584+0000","last_fullsized":"2024-04-25T01:26:00.395584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.396480+0000","last_change":"2024-04-25T01:26:00.396480+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.396480+0000","last_undegraded":"2024-04-25T01:26:00.396480+0000","last_fullsized":"2024-04-25T01:26:00.396480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.396052+0000","last_change":"2024-04-25T01:25:53.362922+0000","last_active":"2024-04-25T01:26:00.396052+0000","last_peered":"2024-04-25T01:26:00.396052+0000","last_clean":"2024-04-25T01:26:00.396052+0000","last_became_active":"2024-04-25T01:25:53.362533+0000","last_became_peered":"2024-04-25T01:25:53.362533+0000","last_unstale":"2024-04-25T01:26:00.396052+0000","last_undegraded":"2024-04-25T01:26:00.396052+0000","last_fullsized":"2024-04-25T01:26:00.396052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.393528+0000","last_change":"2024-04-25T01:25:52.345321+0000","last_active":"2024-04-25T01:26:00.393528+0000","last_peered":"2024-04-25T01:26:00.393528+0000","last_clean":"2024-04-25T01:26:00.393528+0000","last_became_active":"2024-04-25T01:25:52.345061+0000","last_became_peered":"2024-04-25T01:25:52.345061+0000","last_unstale":"2024-04-25T01:26:00.393528+0000","last_undegraded":"2024-04-25T01:26:00.393528+0000","last_fullsized":"2024-04-25T01:26:00.393528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.396440+0000","last_change":"2024-04-25T01:26:00.396440+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.396440+0000","last_undegraded":"2024-04-25T01:26:00.396440+0000","last_fullsized":"2024-04-25T01:26:00.396440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.406683+0000","last_change":"2024-04-25T01:25:53.355950+0000","last_active":"2024-04-25T01:26:00.406683+0000","last_peered":"2024-04-25T01:26:00.406683+0000","last_clean":"2024-04-25T01:26:00.406683+0000","last_became_active":"2024-04-25T01:25:53.355648+0000","last_became_peered":"2024-04-25T01:25:53.355648+0000","last_unstale":"2024-04-25T01:26:00.406683+0000","last_undegraded":"2024-04-25T01:26:00.406683+0000","last_fullsized":"2024-04-25T01:26:00.406683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.396968+0000","last_change":"2024-04-25T01:25:52.357683+0000","last_active":"2024-04-25T01:26:00.396968+0000","last_peered":"2024-04-25T01:26:00.396968+0000","last_clean":"2024-04-25T01:26:00.396968+0000","last_became_active":"2024-04-25T01:25:52.356924+0000","last_became_peered":"2024-04-25T01:25:52.356924+0000","last_unstale":"2024-04-25T01:26:00.396968+0000","last_undegraded":"2024-04-25T01:26:00.396968+0000","last_fullsized":"2024-04-25T01:26:00.396968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.405941+0000","last_change":"2024-04-25T01:26:00.405941+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.405941+0000","last_undegraded":"2024-04-25T01:26:00.405941+0000","last_fullsized":"2024-04-25T01:26:00.405941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395622+0000","last_change":"2024-04-25T01:25:53.374719+0000","last_active":"2024-04-25T01:26:00.395622+0000","last_peered":"2024-04-25T01:26:00.395622+0000","last_clean":"2024-04-25T01:26:00.395622+0000","last_became_active":"2024-04-25T01:25:53.374425+0000","last_became_peered":"2024-04-25T01:25:53.374425+0000","last_unstale":"2024-04-25T01:26:00.395622+0000","last_undegraded":"2024-04-25T01:26:00.395622+0000","last_fullsized":"2024-04-25T01:26:00.395622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395197+0000","last_change":"2024-04-25T01:25:52.353155+0000","last_active":"2024-04-25T01:26:00.395197+0000","last_peered":"2024-04-25T01:26:00.395197+0000","last_clean":"2024-04-25T01:26:00.395197+0000","last_became_active":"2024-04-25T01:25:52.352826+0000","last_became_peered":"2024-04-25T01:25:52.352826+0000","last_unstale":"2024-04-25T01:26:00.395197+0000","last_undegraded":"2024-04-25T01:26:00.395197+0000","last_fullsized":"2024-04-25T01:26:00.395197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":172,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397105+0000","last_change":"2024-04-25T01:25:41.719670+0000","last_active":"2024-04-25T01:26:00.397105+0000","last_peered":"2024-04-25T01:26:00.397105+0000","last_clean":"2024-04-25T01:26:00.397105+0000","last_became_active":"2024-04-25T01:25:41.718721+0000","last_became_peered":"2024-04-25T01:25:41.718721+0000","last_unstale":"2024-04-25T01:26:00.397105+0000","last_undegraded":"2024-04-25T01:26:00.397105+0000","last_fullsized":"2024-04-25T01:26:00.397105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_clean_scrub_stamp":"2024-04-25T01:25:40.688097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:27.456746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.396647+0000","last_change":"2024-04-25T01:26:00.396647+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.396647+0000","last_undegraded":"2024-04-25T01:26:00.396647+0000","last_fullsized":"2024-04-25T01:26:00.396647+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397608+0000","last_change":"2024-04-25T01:25:53.375523+0000","last_active":"2024-04-25T01:26:00.397608+0000","last_peered":"2024-04-25T01:26:00.397608+0000","last_clean":"2024-04-25T01:26:00.397608+0000","last_became_active":"2024-04-25T01:25:53.375219+0000","last_became_peered":"2024-04-25T01:25:53.375219+0000","last_unstale":"2024-04-25T01:26:00.397608+0000","last_undegraded":"2024-04-25T01:26:00.397608+0000","last_fullsized":"2024-04-25T01:26:00.397608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.393456+0000","last_change":"2024-04-25T01:25:52.352994+0000","last_active":"2024-04-25T01:26:00.393456+0000","last_peered":"2024-04-25T01:26:00.393456+0000","last_clean":"2024-04-25T01:26:00.393456+0000","last_became_active":"2024-04-25T01:25:52.352644+0000","last_became_peered":"2024-04-25T01:25:52.352644+0000","last_unstale":"2024-04-25T01:26:00.393456+0000","last_undegraded":"2024-04-25T01:26:00.393456+0000","last_fullsized":"2024-04-25T01:26:00.393456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.396645+0000","last_change":"2024-04-25T01:26:00.396645+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.396645+0000","last_undegraded":"2024-04-25T01:26:00.396645+0000","last_fullsized":"2024-04-25T01:26:00.396645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395593+0000","last_change":"2024-04-25T01:25:53.371018+0000","last_active":"2024-04-25T01:26:00.395593+0000","last_peered":"2024-04-25T01:26:00.395593+0000","last_clean":"2024-04-25T01:26:00.395593+0000","last_became_active":"2024-04-25T01:25:53.370539+0000","last_became_peered":"2024-04-25T01:25:53.370539+0000","last_unstale":"2024-04-25T01:26:00.395593+0000","last_undegraded":"2024-04-25T01:26:00.395593+0000","last_fullsized":"2024-04-25T01:26:00.395593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397676+0000","last_change":"2024-04-25T01:25:52.355133+0000","last_active":"2024-04-25T01:26:00.397676+0000","last_peered":"2024-04-25T01:26:00.397676+0000","last_clean":"2024-04-25T01:26:00.397676+0000","last_became_active":"2024-04-25T01:25:52.354836+0000","last_became_peered":"2024-04-25T01:25:52.354836+0000","last_unstale":"2024-04-25T01:26:00.397676+0000","last_undegraded":"2024-04-25T01:26:00.397676+0000","last_fullsized":"2024-04-25T01:26:00.397676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:10.331704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.405796+0000","last_change":"2024-04-25T01:26:00.405796+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.405796+0000","last_undegraded":"2024-04-25T01:26:00.405796+0000","last_fullsized":"2024-04-25T01:26:00.405796+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397833+0000","last_change":"2024-04-25T01:25:53.370754+0000","last_active":"2024-04-25T01:26:00.397833+0000","last_peered":"2024-04-25T01:26:00.397833+0000","last_clean":"2024-04-25T01:26:00.397833+0000","last_became_active":"2024-04-25T01:25:53.370074+0000","last_became_peered":"2024-04-25T01:25:53.370074+0000","last_unstale":"2024-04-25T01:26:00.397833+0000","last_undegraded":"2024-04-25T01:26:00.397833+0000","last_fullsized":"2024-04-25T01:26:00.397833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.393621+0000","last_change":"2024-04-25T01:25:52.345943+0000","last_active":"2024-04-25T01:26:00.393621+0000","last_peered":"2024-04-25T01:26:00.393621+0000","last_clean":"2024-04-25T01:26:00.393621+0000","last_became_active":"2024-04-25T01:25:52.345717+0000","last_became_peered":"2024-04-25T01:25:52.345717+0000","last_unstale":"2024-04-25T01:26:00.393621+0000","last_undegraded":"2024-04-25T01:26:00.393621+0000","last_fullsized":"2024-04-25T01:26:00.393621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.394211+0000","last_change":"2024-04-25T01:26:00.394211+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.394211+0000","last_undegraded":"2024-04-25T01:26:00.394211+0000","last_fullsized":"2024-04-25T01:26:00.394211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397437+0000","last_change":"2024-04-25T01:25:53.365171+0000","last_active":"2024-04-25T01:26:00.397437+0000","last_peered":"2024-04-25T01:26:00.397437+0000","last_clean":"2024-04-25T01:26:00.397437+0000","last_became_active":"2024-04-25T01:25:53.364049+0000","last_became_peered":"2024-04-25T01:25:53.364049+0000","last_unstale":"2024-04-25T01:26:00.397437+0000","last_undegraded":"2024-04-25T01:26:00.397437+0000","last_fullsized":"2024-04-25T01:26:00.397437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.401739+0000","last_change":"2024-04-25T01:25:52.350688+0000","last_active":"2024-04-25T01:26:00.401739+0000","last_peered":"2024-04-25T01:26:00.401739+0000","last_clean":"2024-04-25T01:26:00.401739+0000","last_became_active":"2024-04-25T01:25:52.350357+0000","last_became_peered":"2024-04-25T01:25:52.350357+0000","last_unstale":"2024-04-25T01:26:00.401739+0000","last_undegraded":"2024-04-25T01:26:00.401739+0000","last_fullsized":"2024-04-25T01:26:00.401739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.400287+0000","last_change":"2024-04-25T01:26:00.400287+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.400287+0000","last_undegraded":"2024-04-25T01:26:00.400287+0000","last_fullsized":"2024-04-25T01:26:00.400287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397305+0000","last_change":"2024-04-25T01:25:53.366376+0000","last_active":"2024-04-25T01:26:00.397305+0000","last_peered":"2024-04-25T01:26:00.397305+0000","last_clean":"2024-04-25T01:26:00.397305+0000","last_became_active":"2024-04-25T01:25:53.365715+0000","last_became_peered":"2024-04-25T01:25:53.365715+0000","last_unstale":"2024-04-25T01:26:00.397305+0000","last_undegraded":"2024-04-25T01:26:00.397305+0000","last_fullsized":"2024-04-25T01:26:00.397305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.393434+0000","last_change":"2024-04-25T01:25:52.347413+0000","last_active":"2024-04-25T01:26:00.393434+0000","last_peered":"2024-04-25T01:26:00.393434+0000","last_clean":"2024-04-25T01:26:00.393434+0000","last_became_active":"2024-04-25T01:25:52.347144+0000","last_became_peered":"2024-04-25T01:25:52.347144+0000","last_unstale":"2024-04-25T01:26:00.393434+0000","last_undegraded":"2024-04-25T01:26:00.393434+0000","last_fullsized":"2024-04-25T01:26:00.393434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.405460+0000","last_change":"2024-04-25T01:26:00.405460+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.405460+0000","last_undegraded":"2024-04-25T01:26:00.405460+0000","last_fullsized":"2024-04-25T01:26:00.405460+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397203+0000","last_change":"2024-04-25T01:25:53.364987+0000","last_active":"2024-04-25T01:26:00.397203+0000","last_peered":"2024-04-25T01:26:00.397203+0000","last_clean":"2024-04-25T01:26:00.397203+0000","last_became_active":"2024-04-25T01:25:53.364431+0000","last_became_peered":"2024-04-25T01:25:53.364431+0000","last_unstale":"2024-04-25T01:26:00.397203+0000","last_undegraded":"2024-04-25T01:26:00.397203+0000","last_fullsized":"2024-04-25T01:26:00.397203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397441+0000","last_change":"2024-04-25T01:25:52.351205+0000","last_active":"2024-04-25T01:26:00.397441+0000","last_peered":"2024-04-25T01:26:00.397441+0000","last_clean":"2024-04-25T01:26:00.397441+0000","last_became_active":"2024-04-25T01:25:52.350946+0000","last_became_peered":"2024-04-25T01:25:52.350946+0000","last_unstale":"2024-04-25T01:26:00.397441+0000","last_undegraded":"2024-04-25T01:26:00.397441+0000","last_fullsized":"2024-04-25T01:26:00.397441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.395127+0000","last_change":"2024-04-25T01:26:00.395127+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.395127+0000","last_undegraded":"2024-04-25T01:26:00.395127+0000","last_fullsized":"2024-04-25T01:26:00.395127+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397169+0000","last_change":"2024-04-25T01:25:53.368187+0000","last_active":"2024-04-25T01:26:00.397169+0000","last_peered":"2024-04-25T01:26:00.397169+0000","last_clean":"2024-04-25T01:26:00.397169+0000","last_became_active":"2024-04-25T01:25:53.367664+0000","last_became_peered":"2024-04-25T01:25:53.367664+0000","last_unstale":"2024-04-25T01:26:00.397169+0000","last_undegraded":"2024-04-25T01:26:00.397169+0000","last_fullsized":"2024-04-25T01:26:00.397169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395485+0000","last_change":"2024-04-25T01:25:52.357397+0000","last_active":"2024-04-25T01:26:00.395485+0000","last_peered":"2024-04-25T01:26:00.395485+0000","last_clean":"2024-04-25T01:26:00.395485+0000","last_became_active":"2024-04-25T01:25:52.356881+0000","last_became_peered":"2024-04-25T01:25:52.356881+0000","last_unstale":"2024-04-25T01:26:00.395485+0000","last_undegraded":"2024-04-25T01:26:00.395485+0000","last_fullsized":"2024-04-25T01:26:00.395485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.393870+0000","last_change":"2024-04-25T01:26:00.393870+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.393870+0000","last_undegraded":"2024-04-25T01:26:00.393870+0000","last_fullsized":"2024-04-25T01:26:00.393870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.406904+0000","last_change":"2024-04-25T01:25:53.370720+0000","last_active":"2024-04-25T01:26:00.406904+0000","last_peered":"2024-04-25T01:26:00.406904+0000","last_clean":"2024-04-25T01:26:00.406904+0000","last_became_active":"2024-04-25T01:25:53.370142+0000","last_became_peered":"2024-04-25T01:25:53.370142+0000","last_unstale":"2024-04-25T01:26:00.406904+0000","last_undegraded":"2024-04-25T01:26:00.406904+0000","last_fullsized":"2024-04-25T01:26:00.406904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:52.833379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.407268+0000","last_change":"2024-04-25T01:25:52.349092+0000","last_active":"2024-04-25T01:26:00.407268+0000","last_peered":"2024-04-25T01:26:00.407268+0000","last_clean":"2024-04-25T01:26:00.407268+0000","last_became_active":"2024-04-25T01:25:52.348846+0000","last_became_peered":"2024-04-25T01:25:52.348846+0000","last_unstale":"2024-04-25T01:26:00.407268+0000","last_undegraded":"2024-04-25T01:26:00.407268+0000","last_fullsized":"2024-04-25T01:26:00.407268+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:21.982440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.394201+0000","last_change":"2024-04-25T01:26:00.394201+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.394201+0000","last_undegraded":"2024-04-25T01:26:00.394201+0000","last_fullsized":"2024-04-25T01:26:00.394201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397713+0000","last_change":"2024-04-25T01:25:53.373572+0000","last_active":"2024-04-25T01:26:00.397713+0000","last_peered":"2024-04-25T01:26:00.397713+0000","last_clean":"2024-04-25T01:26:00.397713+0000","last_became_active":"2024-04-25T01:25:53.373129+0000","last_became_peered":"2024-04-25T01:25:53.373129+0000","last_unstale":"2024-04-25T01:26:00.397713+0000","last_undegraded":"2024-04-25T01:26:00.397713+0000","last_fullsized":"2024-04-25T01:26:00.397713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397712+0000","last_change":"2024-04-25T01:25:52.350446+0000","last_active":"2024-04-25T01:26:00.397712+0000","last_peered":"2024-04-25T01:26:00.397712+0000","last_clean":"2024-04-25T01:26:00.397712+0000","last_became_active":"2024-04-25T01:25:52.349448+0000","last_became_peered":"2024-04-25T01:25:52.349448+0000","last_unstale":"2024-04-25T01:26:00.397712+0000","last_undegraded":"2024-04-25T01:26:00.397712+0000","last_fullsized":"2024-04-25T01:26:00.397712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.397026+0000","last_change":"2024-04-25T01:26:00.397026+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.397026+0000","last_undegraded":"2024-04-25T01:26:00.397026+0000","last_fullsized":"2024-04-25T01:26:00.397026+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.401442+0000","last_change":"2024-04-25T01:25:53.366416+0000","last_active":"2024-04-25T01:26:00.401442+0000","last_peered":"2024-04-25T01:26:00.401442+0000","last_clean":"2024-04-25T01:26:00.401442+0000","last_became_active":"2024-04-25T01:25:53.366157+0000","last_became_peered":"2024-04-25T01:25:53.366157+0000","last_unstale":"2024-04-25T01:26:00.401442+0000","last_undegraded":"2024-04-25T01:26:00.401442+0000","last_fullsized":"2024-04-25T01:26:00.401442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:26.277566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397585+0000","last_change":"2024-04-25T01:25:52.357610+0000","last_active":"2024-04-25T01:26:00.397585+0000","last_peered":"2024-04-25T01:26:00.397585+0000","last_clean":"2024-04-25T01:26:00.397585+0000","last_became_active":"2024-04-25T01:25:52.357355+0000","last_became_peered":"2024-04-25T01:25:52.357355+0000","last_unstale":"2024-04-25T01:26:00.397585+0000","last_undegraded":"2024-04-25T01:26:00.397585+0000","last_fullsized":"2024-04-25T01:26:00.397585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.399683+0000","last_change":"2024-04-25T01:26:00.399683+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.399683+0000","last_undegraded":"2024-04-25T01:26:00.399683+0000","last_fullsized":"2024-04-25T01:26:00.399683+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.407072+0000","last_change":"2024-04-25T01:25:53.374311+0000","last_active":"2024-04-25T01:26:00.407072+0000","last_peered":"2024-04-25T01:26:00.407072+0000","last_clean":"2024-04-25T01:26:00.407072+0000","last_became_active":"2024-04-25T01:25:53.373890+0000","last_became_peered":"2024-04-25T01:25:53.373890+0000","last_unstale":"2024-04-25T01:26:00.407072+0000","last_undegraded":"2024-04-25T01:26:00.407072+0000","last_fullsized":"2024-04-25T01:26:00.407072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395954+0000","last_change":"2024-04-25T01:25:52.356231+0000","last_active":"2024-04-25T01:26:00.395954+0000","last_peered":"2024-04-25T01:26:00.395954+0000","last_clean":"2024-04-25T01:26:00.395954+0000","last_became_active":"2024-04-25T01:25:52.355913+0000","last_became_peered":"2024-04-25T01:25:52.355913+0000","last_unstale":"2024-04-25T01:26:00.395954+0000","last_undegraded":"2024-04-25T01:26:00.395954+0000","last_fullsized":"2024-04-25T01:26:00.395954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.393714+0000","last_change":"2024-04-25T01:26:00.393714+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.393714+0000","last_undegraded":"2024-04-25T01:26:00.393714+0000","last_fullsized":"2024-04-25T01:26:00.393714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395885+0000","last_change":"2024-04-25T01:25:53.371445+0000","last_active":"2024-04-25T01:26:00.395885+0000","last_peered":"2024-04-25T01:26:00.395885+0000","last_clean":"2024-04-25T01:26:00.395885+0000","last_became_active":"2024-04-25T01:25:53.371086+0000","last_became_peered":"2024-04-25T01:25:53.371086+0000","last_unstale":"2024-04-25T01:26:00.395885+0000","last_undegraded":"2024-04-25T01:26:00.395885+0000","last_fullsized":"2024-04-25T01:26:00.395885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.398579+0000","last_change":"2024-04-25T01:25:52.357459+0000","last_active":"2024-04-25T01:26:00.398579+0000","last_peered":"2024-04-25T01:26:00.398579+0000","last_clean":"2024-04-25T01:26:00.398579+0000","last_became_active":"2024-04-25T01:25:52.357182+0000","last_became_peered":"2024-04-25T01:25:52.357182+0000","last_unstale":"2024-04-25T01:26:00.398579+0000","last_undegraded":"2024-04-25T01:26:00.398579+0000","last_fullsized":"2024-04-25T01:26:00.398579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.394879+0000","last_change":"2024-04-25T01:26:00.394879+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.394879+0000","last_undegraded":"2024-04-25T01:26:00.394879+0000","last_fullsized":"2024-04-25T01:26:00.394879+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.406352+0000","last_change":"2024-04-25T01:26:00.406352+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.406352+0000","last_undegraded":"2024-04-25T01:26:00.406352+0000","last_fullsized":"2024-04-25T01:26:00.406352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.396714+0000","last_change":"2024-04-25T01:25:53.363043+0000","last_active":"2024-04-25T01:26:00.396714+0000","last_peered":"2024-04-25T01:26:00.396714+0000","last_clean":"2024-04-25T01:26:00.396714+0000","last_became_active":"2024-04-25T01:25:53.362601+0000","last_became_peered":"2024-04-25T01:25:53.362601+0000","last_unstale":"2024-04-25T01:26:00.396714+0000","last_undegraded":"2024-04-25T01:26:00.396714+0000","last_fullsized":"2024-04-25T01:26:00.396714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395223+0000","last_change":"2024-04-25T01:25:52.358081+0000","last_active":"2024-04-25T01:26:00.395223+0000","last_peered":"2024-04-25T01:26:00.395223+0000","last_clean":"2024-04-25T01:26:00.395223+0000","last_became_active":"2024-04-25T01:25:52.357365+0000","last_became_peered":"2024-04-25T01:25:52.357365+0000","last_unstale":"2024-04-25T01:26:00.395223+0000","last_undegraded":"2024-04-25T01:26:00.395223+0000","last_fullsized":"2024-04-25T01:26:00.395223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.396689+0000","last_change":"2024-04-25T01:26:00.396689+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.396689+0000","last_undegraded":"2024-04-25T01:26:00.396689+0000","last_fullsized":"2024-04-25T01:26:00.396689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395534+0000","last_change":"2024-04-25T01:25:53.363794+0000","last_active":"2024-04-25T01:26:00.395534+0000","last_peered":"2024-04-25T01:26:00.395534+0000","last_clean":"2024-04-25T01:26:00.395534+0000","last_became_active":"2024-04-25T01:25:53.363509+0000","last_became_peered":"2024-04-25T01:25:53.363509+0000","last_unstale":"2024-04-25T01:26:00.395534+0000","last_undegraded":"2024-04-25T01:26:00.395534+0000","last_fullsized":"2024-04-25T01:26:00.395534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395763+0000","last_change":"2024-04-25T01:25:52.347464+0000","last_active":"2024-04-25T01:26:00.395763+0000","last_peered":"2024-04-25T01:26:00.395763+0000","last_clean":"2024-04-25T01:26:00.395763+0000","last_became_active":"2024-04-25T01:25:52.347138+0000","last_became_peered":"2024-04-25T01:25:52.347138+0000","last_unstale":"2024-04-25T01:26:00.395763+0000","last_undegraded":"2024-04-25T01:26:00.395763+0000","last_fullsized":"2024-04-25T01:26:00.395763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397478+0000","last_change":"2024-04-25T01:25:52.349989+0000","last_active":"2024-04-25T01:26:00.397478+0000","last_peered":"2024-04-25T01:26:00.397478+0000","last_clean":"2024-04-25T01:26:00.397478+0000","last_became_active":"2024-04-25T01:25:52.349672+0000","last_became_peered":"2024-04-25T01:25:52.349672+0000","last_unstale":"2024-04-25T01:26:00.397478+0000","last_undegraded":"2024-04-25T01:26:00.397478+0000","last_fullsized":"2024-04-25T01:26:00.397478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.407163+0000","last_change":"2024-04-25T01:25:53.356610+0000","last_active":"2024-04-25T01:26:00.407163+0000","last_peered":"2024-04-25T01:26:00.407163+0000","last_clean":"2024-04-25T01:26:00.407163+0000","last_became_active":"2024-04-25T01:25:53.356015+0000","last_became_peered":"2024-04-25T01:25:53.356015+0000","last_unstale":"2024-04-25T01:26:00.407163+0000","last_undegraded":"2024-04-25T01:26:00.407163+0000","last_fullsized":"2024-04-25T01:26:00.407163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.395250+0000","last_change":"2024-04-25T01:26:00.395250+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.395250+0000","last_undegraded":"2024-04-25T01:26:00.395250+0000","last_fullsized":"2024-04-25T01:26:00.395250+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.396037+0000","last_change":"2024-04-25T01:25:52.353955+0000","last_active":"2024-04-25T01:26:00.396037+0000","last_peered":"2024-04-25T01:26:00.396037+0000","last_clean":"2024-04-25T01:26:00.396037+0000","last_became_active":"2024-04-25T01:25:52.353659+0000","last_became_peered":"2024-04-25T01:25:52.353659+0000","last_unstale":"2024-04-25T01:26:00.396037+0000","last_undegraded":"2024-04-25T01:26:00.396037+0000","last_fullsized":"2024-04-25T01:26:00.396037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.401056+0000","last_change":"2024-04-25T01:25:53.366586+0000","last_active":"2024-04-25T01:26:00.401056+0000","last_peered":"2024-04-25T01:26:00.401056+0000","last_clean":"2024-04-25T01:26:00.401056+0000","last_became_active":"2024-04-25T01:25:53.366302+0000","last_became_peered":"2024-04-25T01:25:53.366302+0000","last_unstale":"2024-04-25T01:26:00.401056+0000","last_undegraded":"2024-04-25T01:26:00.401056+0000","last_fullsized":"2024-04-25T01:26:00.401056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T01:26:00.393991+0000","last_change":"2024-04-25T01:26:00.393991+0000","last_active":"2024-04-25T01:26:00.384370+0000","last_peered":"2024-04-25T01:26:00.384370+0000","last_clean":"2024-04-25T01:26:00.384370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:00.393991+0000","last_undegraded":"2024-04-25T01:26:00.393991+0000","last_fullsized":"2024-04-25T01:26:00.393991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397190+0000","last_change":"2024-04-25T01:25:53.367143+0000","last_active":"2024-04-25T01:26:00.397190+0000","last_peered":"2024-04-25T01:26:00.397190+0000","last_clean":"2024-04-25T01:26:00.397190+0000","last_became_active":"2024-04-25T01:25:53.366847+0000","last_became_peered":"2024-04-25T01:25:53.366847+0000","last_unstale":"2024-04-25T01:26:00.397190+0000","last_undegraded":"2024-04-25T01:26:00.397190+0000","last_fullsized":"2024-04-25T01:26:00.397190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.393357+0000","last_change":"2024-04-25T01:25:53.371345+0000","last_active":"2024-04-25T01:26:00.393357+0000","last_peered":"2024-04-25T01:26:00.393357+0000","last_clean":"2024-04-25T01:26:00.393357+0000","last_became_active":"2024-04-25T01:25:53.371080+0000","last_became_peered":"2024-04-25T01:25:53.371080+0000","last_unstale":"2024-04-25T01:26:00.393357+0000","last_undegraded":"2024-04-25T01:26:00.393357+0000","last_fullsized":"2024-04-25T01:26:00.393357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:09.855763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395468+0000","last_change":"2024-04-25T01:25:53.369850+0000","last_active":"2024-04-25T01:26:00.395468+0000","last_peered":"2024-04-25T01:26:00.395468+0000","last_clean":"2024-04-25T01:26:00.395468+0000","last_became_active":"2024-04-25T01:25:53.369350+0000","last_became_peered":"2024-04-25T01:25:53.369350+0000","last_unstale":"2024-04-25T01:26:00.395468+0000","last_undegraded":"2024-04-25T01:26:00.395468+0000","last_fullsized":"2024-04-25T01:26:00.395468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.407143+0000","last_change":"2024-04-25T01:25:53.357142+0000","last_active":"2024-04-25T01:26:00.407143+0000","last_peered":"2024-04-25T01:26:00.407143+0000","last_clean":"2024-04-25T01:26:00.407143+0000","last_became_active":"2024-04-25T01:25:53.356854+0000","last_became_peered":"2024-04-25T01:25:53.356854+0000","last_unstale":"2024-04-25T01:26:00.407143+0000","last_undegraded":"2024-04-25T01:26:00.407143+0000","last_fullsized":"2024-04-25T01:26:00.407143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.396011+0000","last_change":"2024-04-25T01:25:53.364575+0000","last_active":"2024-04-25T01:26:00.396011+0000","last_peered":"2024-04-25T01:26:00.396011+0000","last_clean":"2024-04-25T01:26:00.396011+0000","last_became_active":"2024-04-25T01:25:53.364311+0000","last_became_peered":"2024-04-25T01:25:53.364311+0000","last_unstale":"2024-04-25T01:26:00.396011+0000","last_undegraded":"2024-04-25T01:26:00.396011+0000","last_fullsized":"2024-04-25T01:26:00.396011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.396756+0000","last_change":"2024-04-25T01:25:53.368413+0000","last_active":"2024-04-25T01:26:00.396756+0000","last_peered":"2024-04-25T01:26:00.396756+0000","last_clean":"2024-04-25T01:26:00.396756+0000","last_became_active":"2024-04-25T01:25:53.368176+0000","last_became_peered":"2024-04-25T01:25:53.368176+0000","last_unstale":"2024-04-25T01:26:00.396756+0000","last_undegraded":"2024-04-25T01:26:00.396756+0000","last_fullsized":"2024-04-25T01:26:00.396756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.398425+0000","last_change":"2024-04-25T01:25:53.372638+0000","last_active":"2024-04-25T01:26:00.398425+0000","last_peered":"2024-04-25T01:26:00.398425+0000","last_clean":"2024-04-25T01:26:00.398425+0000","last_became_active":"2024-04-25T01:25:53.372001+0000","last_became_peered":"2024-04-25T01:25:53.372001+0000","last_unstale":"2024-04-25T01:26:00.398425+0000","last_undegraded":"2024-04-25T01:26:00.398425+0000","last_fullsized":"2024-04-25T01:26:00.398425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.396389+0000","last_change":"2024-04-25T01:25:53.365509+0000","last_active":"2024-04-25T01:26:00.396389+0000","last_peered":"2024-04-25T01:26:00.396389+0000","last_clean":"2024-04-25T01:26:00.396389+0000","last_became_active":"2024-04-25T01:25:53.365245+0000","last_became_peered":"2024-04-25T01:25:53.365245+0000","last_unstale":"2024-04-25T01:26:00.396389+0000","last_undegraded":"2024-04-25T01:26:00.396389+0000","last_fullsized":"2024-04-25T01:26:00.396389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395105+0000","last_change":"2024-04-25T01:25:53.365391+0000","last_active":"2024-04-25T01:26:00.395105+0000","last_peered":"2024-04-25T01:26:00.395105+0000","last_clean":"2024-04-25T01:26:00.395105+0000","last_became_active":"2024-04-25T01:25:53.364829+0000","last_became_peered":"2024-04-25T01:25:53.364829+0000","last_unstale":"2024-04-25T01:26:00.395105+0000","last_undegraded":"2024-04-25T01:26:00.395105+0000","last_fullsized":"2024-04-25T01:26:00.395105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395668+0000","last_change":"2024-04-25T01:25:53.355667+0000","last_active":"2024-04-25T01:26:00.395668+0000","last_peered":"2024-04-25T01:26:00.395668+0000","last_clean":"2024-04-25T01:26:00.395668+0000","last_became_active":"2024-04-25T01:25:53.355417+0000","last_became_peered":"2024-04-25T01:25:53.355417+0000","last_unstale":"2024-04-25T01:26:00.395668+0000","last_undegraded":"2024-04-25T01:26:00.395668+0000","last_fullsized":"2024-04-25T01:26:00.395668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395356+0000","last_change":"2024-04-25T01:25:53.371240+0000","last_active":"2024-04-25T01:26:00.395356+0000","last_peered":"2024-04-25T01:26:00.395356+0000","last_clean":"2024-04-25T01:26:00.395356+0000","last_became_active":"2024-04-25T01:25:53.370164+0000","last_became_peered":"2024-04-25T01:25:53.370164+0000","last_unstale":"2024-04-25T01:26:00.395356+0000","last_undegraded":"2024-04-25T01:26:00.395356+0000","last_fullsized":"2024-04-25T01:26:00.395356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.407018+0000","last_change":"2024-04-25T01:25:53.376415+0000","last_active":"2024-04-25T01:26:00.407018+0000","last_peered":"2024-04-25T01:26:00.407018+0000","last_clean":"2024-04-25T01:26:00.407018+0000","last_became_active":"2024-04-25T01:25:53.376104+0000","last_became_peered":"2024-04-25T01:25:53.376104+0000","last_unstale":"2024-04-25T01:26:00.407018+0000","last_undegraded":"2024-04-25T01:26:00.407018+0000","last_fullsized":"2024-04-25T01:26:00.407018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397599+0000","last_change":"2024-04-25T01:25:53.370314+0000","last_active":"2024-04-25T01:26:00.397599+0000","last_peered":"2024-04-25T01:26:00.397599+0000","last_clean":"2024-04-25T01:26:00.397599+0000","last_became_active":"2024-04-25T01:25:53.370049+0000","last_became_peered":"2024-04-25T01:25:53.370049+0000","last_unstale":"2024-04-25T01:26:00.397599+0000","last_undegraded":"2024-04-25T01:26:00.397599+0000","last_fullsized":"2024-04-25T01:26:00.397599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397836+0000","last_change":"2024-04-25T01:25:53.369585+0000","last_active":"2024-04-25T01:26:00.397836+0000","last_peered":"2024-04-25T01:26:00.397836+0000","last_clean":"2024-04-25T01:26:00.397836+0000","last_became_active":"2024-04-25T01:25:53.368499+0000","last_became_peered":"2024-04-25T01:25:53.368499+0000","last_unstale":"2024-04-25T01:26:00.397836+0000","last_undegraded":"2024-04-25T01:26:00.397836+0000","last_fullsized":"2024-04-25T01:26:00.397836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.393569+0000","last_change":"2024-04-25T01:25:53.370666+0000","last_active":"2024-04-25T01:26:00.393569+0000","last_peered":"2024-04-25T01:26:00.393569+0000","last_clean":"2024-04-25T01:26:00.393569+0000","last_became_active":"2024-04-25T01:25:53.369733+0000","last_became_peered":"2024-04-25T01:25:53.369733+0000","last_unstale":"2024-04-25T01:26:00.393569+0000","last_undegraded":"2024-04-25T01:26:00.393569+0000","last_fullsized":"2024-04-25T01:26:00.393569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.396077+0000","last_change":"2024-04-25T01:25:53.372048+0000","last_active":"2024-04-25T01:26:00.396077+0000","last_peered":"2024-04-25T01:26:00.396077+0000","last_clean":"2024-04-25T01:26:00.396077+0000","last_became_active":"2024-04-25T01:25:53.371767+0000","last_became_peered":"2024-04-25T01:25:53.371767+0000","last_unstale":"2024-04-25T01:26:00.396077+0000","last_undegraded":"2024-04-25T01:26:00.396077+0000","last_fullsized":"2024-04-25T01:26:00.396077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:05.589744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395079+0000","last_change":"2024-04-25T01:25:53.374186+0000","last_active":"2024-04-25T01:26:00.395079+0000","last_peered":"2024-04-25T01:26:00.395079+0000","last_clean":"2024-04-25T01:26:00.395079+0000","last_became_active":"2024-04-25T01:25:53.373766+0000","last_became_peered":"2024-04-25T01:25:53.373766+0000","last_unstale":"2024-04-25T01:26:00.395079+0000","last_undegraded":"2024-04-25T01:26:00.395079+0000","last_fullsized":"2024-04-25T01:26:00.395079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.396816+0000","last_change":"2024-04-25T01:25:53.371602+0000","last_active":"2024-04-25T01:26:00.396816+0000","last_peered":"2024-04-25T01:26:00.396816+0000","last_clean":"2024-04-25T01:26:00.396816+0000","last_became_active":"2024-04-25T01:25:53.371362+0000","last_became_peered":"2024-04-25T01:25:53.371362+0000","last_unstale":"2024-04-25T01:26:00.396816+0000","last_undegraded":"2024-04-25T01:26:00.396816+0000","last_fullsized":"2024-04-25T01:26:00.396816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.406854+0000","last_change":"2024-04-25T01:25:53.370996+0000","last_active":"2024-04-25T01:26:00.406854+0000","last_peered":"2024-04-25T01:26:00.406854+0000","last_clean":"2024-04-25T01:26:00.406854+0000","last_became_active":"2024-04-25T01:25:53.370479+0000","last_became_peered":"2024-04-25T01:25:53.370479+0000","last_unstale":"2024-04-25T01:26:00.406854+0000","last_undegraded":"2024-04-25T01:26:00.406854+0000","last_fullsized":"2024-04-25T01:26:00.406854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:57.636067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395821+0000","last_change":"2024-04-25T01:25:53.363512+0000","last_active":"2024-04-25T01:26:00.395821+0000","last_peered":"2024-04-25T01:26:00.395821+0000","last_clean":"2024-04-25T01:26:00.395821+0000","last_became_active":"2024-04-25T01:25:53.363246+0000","last_became_peered":"2024-04-25T01:25:53.363246+0000","last_unstale":"2024-04-25T01:26:00.395821+0000","last_undegraded":"2024-04-25T01:26:00.395821+0000","last_fullsized":"2024-04-25T01:26:00.395821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.401451+0000","last_change":"2024-04-25T01:25:53.364457+0000","last_active":"2024-04-25T01:26:00.401451+0000","last_peered":"2024-04-25T01:26:00.401451+0000","last_clean":"2024-04-25T01:26:00.401451+0000","last_became_active":"2024-04-25T01:25:53.364103+0000","last_became_peered":"2024-04-25T01:25:53.364103+0000","last_unstale":"2024-04-25T01:26:00.401451+0000","last_undegraded":"2024-04-25T01:26:00.401451+0000","last_fullsized":"2024-04-25T01:26:00.401451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:11.336752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.398051+0000","last_change":"2024-04-25T01:25:53.371647+0000","last_active":"2024-04-25T01:26:00.398051+0000","last_peered":"2024-04-25T01:26:00.398051+0000","last_clean":"2024-04-25T01:26:00.398051+0000","last_became_active":"2024-04-25T01:25:53.371353+0000","last_became_peered":"2024-04-25T01:25:53.371353+0000","last_unstale":"2024-04-25T01:26:00.398051+0000","last_undegraded":"2024-04-25T01:26:00.398051+0000","last_fullsized":"2024-04-25T01:26:00.398051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.398380+0000","last_change":"2024-04-25T01:25:53.372842+0000","last_active":"2024-04-25T01:26:00.398380+0000","last_peered":"2024-04-25T01:26:00.398380+0000","last_clean":"2024-04-25T01:26:00.398380+0000","last_became_active":"2024-04-25T01:25:53.372580+0000","last_became_peered":"2024-04-25T01:25:53.372580+0000","last_unstale":"2024-04-25T01:26:00.398380+0000","last_undegraded":"2024-04-25T01:26:00.398380+0000","last_fullsized":"2024-04-25T01:26:00.398380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397698+0000","last_change":"2024-04-25T01:25:53.376081+0000","last_active":"2024-04-25T01:26:00.397698+0000","last_peered":"2024-04-25T01:26:00.397698+0000","last_clean":"2024-04-25T01:26:00.397698+0000","last_became_active":"2024-04-25T01:25:53.375814+0000","last_became_peered":"2024-04-25T01:25:53.375814+0000","last_unstale":"2024-04-25T01:26:00.397698+0000","last_undegraded":"2024-04-25T01:26:00.397698+0000","last_fullsized":"2024-04-25T01:26:00.397698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:29.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395843+0000","last_change":"2024-04-25T01:25:53.374061+0000","last_active":"2024-04-25T01:26:00.395843+0000","last_peered":"2024-04-25T01:26:00.395843+0000","last_clean":"2024-04-25T01:26:00.395843+0000","last_became_active":"2024-04-25T01:25:53.373848+0000","last_became_peered":"2024-04-25T01:25:53.373848+0000","last_unstale":"2024-04-25T01:26:00.395843+0000","last_undegraded":"2024-04-25T01:26:00.395843+0000","last_fullsized":"2024-04-25T01:26:00.395843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397212+0000","last_change":"2024-04-25T01:25:53.364698+0000","last_active":"2024-04-25T01:26:00.397212+0000","last_peered":"2024-04-25T01:26:00.397212+0000","last_clean":"2024-04-25T01:26:00.397212+0000","last_became_active":"2024-04-25T01:25:53.364435+0000","last_became_peered":"2024-04-25T01:25:53.364435+0000","last_unstale":"2024-04-25T01:26:00.397212+0000","last_undegraded":"2024-04-25T01:26:00.397212+0000","last_fullsized":"2024-04-25T01:26:00.397212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395329+0000","last_change":"2024-04-25T01:25:53.375324+0000","last_active":"2024-04-25T01:26:00.395329+0000","last_peered":"2024-04-25T01:26:00.395329+0000","last_clean":"2024-04-25T01:26:00.395329+0000","last_became_active":"2024-04-25T01:25:53.374392+0000","last_became_peered":"2024-04-25T01:25:53.374392+0000","last_unstale":"2024-04-25T01:26:00.395329+0000","last_undegraded":"2024-04-25T01:26:00.395329+0000","last_fullsized":"2024-04-25T01:26:00.395329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397081+0000","last_change":"2024-04-25T01:25:53.367953+0000","last_active":"2024-04-25T01:26:00.397081+0000","last_peered":"2024-04-25T01:26:00.397081+0000","last_clean":"2024-04-25T01:26:00.397081+0000","last_became_active":"2024-04-25T01:25:53.367358+0000","last_became_peered":"2024-04-25T01:25:53.367358+0000","last_unstale":"2024-04-25T01:26:00.397081+0000","last_undegraded":"2024-04-25T01:26:00.397081+0000","last_fullsized":"2024-04-25T01:26:00.397081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.395330+0000","last_change":"2024-04-25T01:25:53.371513+0000","last_active":"2024-04-25T01:26:00.395330+0000","last_peered":"2024-04-25T01:26:00.395330+0000","last_clean":"2024-04-25T01:26:00.395330+0000","last_became_active":"2024-04-25T01:25:53.370421+0000","last_became_peered":"2024-04-25T01:25:53.370421+0000","last_unstale":"2024-04-25T01:26:00.395330+0000","last_undegraded":"2024-04-25T01:26:00.395330+0000","last_fullsized":"2024-04-25T01:26:00.395330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.406823+0000","last_change":"2024-04-25T01:25:53.363829+0000","last_active":"2024-04-25T01:26:00.406823+0000","last_peered":"2024-04-25T01:26:00.406823+0000","last_clean":"2024-04-25T01:26:00.406823+0000","last_became_active":"2024-04-25T01:25:53.363525+0000","last_became_peered":"2024-04-25T01:25:53.363525+0000","last_unstale":"2024-04-25T01:26:00.406823+0000","last_undegraded":"2024-04-25T01:26:00.406823+0000","last_fullsized":"2024-04-25T01:26:00.406823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.396999+0000","last_change":"2024-04-25T01:25:53.365057+0000","last_active":"2024-04-25T01:26:00.396999+0000","last_peered":"2024-04-25T01:26:00.396999+0000","last_clean":"2024-04-25T01:26:00.396999+0000","last_became_active":"2024-04-25T01:25:53.364793+0000","last_became_peered":"2024-04-25T01:25:53.364793+0000","last_unstale":"2024-04-25T01:26:00.396999+0000","last_undegraded":"2024-04-25T01:26:00.396999+0000","last_fullsized":"2024-04-25T01:26:00.396999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T01:26:01.391184+0000","last_change":"2024-04-25T01:26:01.391184+0000","last_active":"2024-04-25T01:26:01.391184+0000","last_peered":"2024-04-25T01:26:01.391184+0000","last_clean":"2024-04-25T01:26:01.391184+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T01:26:01.391184+0000","last_undegraded":"2024-04-25T01:26:01.391184+0000","last_fullsized":"2024-04-25T01:26:01.391184+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T01:26:00.397684+0000","last_change":"2024-04-25T01:25:53.367762+0000","last_active":"2024-04-25T01:26:00.397684+0000","last_peered":"2024-04-25T01:26:00.397684+0000","last_clean":"2024-04-25T01:26:00.397684+0000","last_became_active":"2024-04-25T01:25:53.367383+0000","last_became_peered":"2024-04-25T01:25:53.367383+0000","last_unstale":"2024-04-25T01:26:00.397684+0000","last_undegraded":"2024-04-25T01:26:00.397684+0000","last_fullsized":"2024-04-25T01:26:00.397684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:05.589744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":77583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":70980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:26:06.361 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:26:06.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.511+0000 7f2628276640 1 Processor -- start 2024-04-25T01:26:06.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.511+0000 7f2628276640 1 -- start start 2024-04-25T01:26:06.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.512+0000 7f2628276640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2620154120 0x7f26200708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.512+0000 7f2628276640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2620156c10 con 0x7f2620150d80 2024-04-25T01:26:06.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.512+0000 7f2628276640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2620156ef0 con 0x7f26200c70a0 2024-04-25T01:26:06.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.512+0000 7f2628276640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f26201571d0 con 0x7f2620154120 2024-04-25T01:26:06.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.512+0000 7f2626012640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2620150d80 0x7f26200aacc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37218/0 (socket says 172.21.15.112:37218) 2024-04-25T01:26:06.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.512+0000 7f2626012640 1 -- 172.21.15.112:0/521475443 learned_addr learned my addr 172.21.15.112:0/521475443 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:06.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.512+0000 7f2625010640 1 -- 172.21.15.112:0/521475443 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3189118755 0 0) 0x7f2620156c10 con 0x7f2620150d80 2024-04-25T01:26:06.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.512+0000 7f2625010640 1 -- 172.21.15.112:0/521475443 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f260c003520 con 0x7f2620150d80 2024-04-25T01:26:06.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.512+0000 7f2626813640 1 --2- 172.21.15.112:0/521475443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2620154120 0x7f26200708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:06.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2625010640 1 -- 172.21.15.112:0/521475443 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2100766480 0 0) 0x7f260c003520 con 0x7f2620150d80 2024-04-25T01:26:06.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2625010640 1 -- 172.21.15.112:0/521475443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2620154120 msgr2=0x7f26200708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:06.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2625010640 1 --2- 172.21.15.112:0/521475443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2620154120 0x7f26200708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2625010640 1 -- 172.21.15.112:0/521475443 >> v1:172.21.15.187:6789/0 conn(0x7f26200c70a0 legacy=0x7f26200701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:06.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2625010640 1 -- 172.21.15.112:0/521475443 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26201574b0 con 0x7f2620150d80 2024-04-25T01:26:06.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2625010640 1 -- 172.21.15.112:0/521475443 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f261c001670 con 0x7f2620150d80 2024-04-25T01:26:06.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2625010640 1 -- 172.21.15.112:0/521475443 >> v1:172.21.15.112:6789/0 conn(0x7f2620150d80 legacy=0x7f26200aacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:06.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2625010640 1 --2- 172.21.15.112:0/521475443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2620154120 0x7f260c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2625010640 1 --2- 172.21.15.112:0/521475443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f260c004010 0x7f260c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2625010640 1 --2- 172.21.15.112:0/521475443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f260c004d40 0x7f260c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2625010640 1 -- 172.21.15.112:0/521475443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f26201571d0 con 0x7f260c004d40 2024-04-25T01:26:06.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2625010640 1 -- 172.21.15.112:0/521475443 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f260c00bc00 con 0x7f260c004010 2024-04-25T01:26:06.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2625010640 1 -- 172.21.15.112:0/521475443 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f260c00be00 con 0x7f2620154120 2024-04-25T01:26:06.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2626813640 1 --2- 172.21.15.112:0/521475443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f260c004d40 0x7f260c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:06.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2626012640 1 --2- 172.21.15.112:0/521475443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2620154120 0x7f260c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:06.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2625811640 1 --2- 172.21.15.112:0/521475443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f260c004010 0x7f260c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:06.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2626813640 1 -- 172.21.15.112:0/521475443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2620154120 msgr2=0x7f260c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:06.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2626813640 1 --2- 172.21.15.112:0/521475443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2620154120 0x7f260c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2626813640 1 -- 172.21.15.112:0/521475443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f260c004010 msgr2=0x7f260c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:06.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2626813640 1 --2- 172.21.15.112:0/521475443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f260c004010 0x7f260c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.513+0000 7f2626813640 1 -- 172.21.15.112:0/521475443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260c003520 con 0x7f260c004d40 2024-04-25T01:26:06.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.514+0000 7f2626813640 1 --2- 172.21.15.112:0/521475443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f260c004d40 0x7f260c0051c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2618007360 tx=0x7f260c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b824a2f379fc7ef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:06.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.514+0000 7f2625010640 1 -- 172.21.15.112:0/521475443 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618009be0 con 0x7f260c004d40 2024-04-25T01:26:06.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.514+0000 7f2625010640 1 -- 172.21.15.112:0/521475443 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2618036600 con 0x7f260c004d40 2024-04-25T01:26:06.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.514+0000 7f2625010640 1 -- 172.21.15.112:0/521475443 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26180369d0 con 0x7f260c004d40 2024-04-25T01:26:06.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.514+0000 7f2628276640 1 -- 172.21.15.112:0/521475443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f260c004d40 msgr2=0x7f260c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:06.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.514+0000 7f2628276640 1 --2- 172.21.15.112:0/521475443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f260c004d40 0x7f260c0051c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2618007360 tx=0x7f260c005830 comp rx=0 tx=0).stop 2024-04-25T01:26:06.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.514+0000 7f2626813640 1 -- 172.21.15.112:0/521475443 reap_dead start 2024-04-25T01:26:06.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.514+0000 7f2628276640 1 -- 172.21.15.112:0/521475443 shutdown_connections 2024-04-25T01:26:06.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.514+0000 7f2628276640 1 -- 172.21.15.112:0/521475443 >> 172.21.15.112:0/521475443 conn(0x7f262006f310 msgr2=0x7f26200743d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:06.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.514+0000 7f2628276640 1 -- 172.21.15.112:0/521475443 shutdown_connections 2024-04-25T01:26:06.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.514+0000 7f2628276640 1 -- 172.21.15.112:0/521475443 wait complete. 2024-04-25T01:26:06.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.515+0000 7f2628276640 1 Processor -- start 2024-04-25T01:26:06.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.516+0000 7f2628276640 1 -- start start 2024-04-25T01:26:06.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.516+0000 7f2628276640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f260c004d40 0x7f26200ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.516+0000 7f2628276640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26200c5270 0x7f262014a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.516+0000 7f2628276640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26200c5760 0x7f262014a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.516+0000 7f2628276640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f26201545b0 con 0x7f26200c5270 2024-04-25T01:26:06.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.516+0000 7f2628276640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2620157a70 con 0x7f26200c5760 2024-04-25T01:26:06.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.516+0000 7f2628276640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f26200ab3d0 con 0x7f260c004d40 2024-04-25T01:26:06.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.516+0000 7f2625811640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26200c5270 0x7f262014a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:06.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.516+0000 7f2625811640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26200c5270 0x7f262014a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58348/0 (socket says 172.21.15.112:58348) 2024-04-25T01:26:06.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.516+0000 7f2625811640 1 -- 172.21.15.112:0/2940350435 learned_addr learned my addr 172.21.15.112:0/2940350435 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:06.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.516+0000 7f2625811640 1 -- 172.21.15.112:0/2940350435 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f260c004d40 msgr2=0x7f26200ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:06.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.516+0000 7f2625811640 1 --2- 172.21.15.112:0/2940350435 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f260c004d40 0x7f26200ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.516+0000 7f2625811640 1 -- 172.21.15.112:0/2940350435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26200c5760 msgr2=0x7f262014a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:06.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.516+0000 7f2625811640 1 --2- 172.21.15.112:0/2940350435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26200c5760 0x7f262014a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.516+0000 7f2625811640 1 -- 172.21.15.112:0/2940350435 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262014abb0 con 0x7f26200c5270 2024-04-25T01:26:06.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.517+0000 7f2625811640 1 --2- 172.21.15.112:0/2940350435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26200c5270 0x7f262014a130 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f261000f5f0 tx=0x7f261000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:06.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.517+0000 7f2616ffd640 1 -- 172.21.15.112:0/2940350435 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26100052e0 con 0x7f26200c5270 2024-04-25T01:26:06.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.517+0000 7f2616ffd640 1 -- 172.21.15.112:0/2940350435 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2610018070 con 0x7f26200c5270 2024-04-25T01:26:06.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.517+0000 7f2628276640 1 -- 172.21.15.112:0/2940350435 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26201ec720 con 0x7f26200c5270 2024-04-25T01:26:06.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.517+0000 7f2616ffd640 1 -- 172.21.15.112:0/2940350435 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610003810 con 0x7f26200c5270 2024-04-25T01:26:06.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.517+0000 7f2628276640 1 -- 172.21.15.112:0/2940350435 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26201ecce0 con 0x7f26200c5270 2024-04-25T01:26:06.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.518+0000 7f2628276640 1 -- 172.21.15.112:0/2940350435 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26200c6b70 con 0x7f26200c5270 2024-04-25T01:26:06.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.518+0000 7f2616ffd640 1 -- 172.21.15.112:0/2940350435 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f26100039b0 con 0x7f26200c5270 2024-04-25T01:26:06.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.519+0000 7f2616ffd640 1 --2- 172.21.15.112:0/2940350435 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f260406be60 0x7f260406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.519+0000 7f2626012640 1 --2- 172.21.15.112:0/2940350435 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f260406be60 0x7f260406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:06.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.519+0000 7f2616ffd640 1 -- 172.21.15.112:0/2940350435 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f261008a9a0 con 0x7f26200c5270 2024-04-25T01:26:06.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.519+0000 7f2626012640 1 --2- 172.21.15.112:0/2940350435 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f260406be60 0x7f260406e320 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f261c001550 tx=0x7f261c007c50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:06.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.523+0000 7f2616ffd640 1 -- 172.21.15.112:0/2940350435 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f26200c6b70 con 0x7f26200c5270 2024-04-25T01:26:06.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.694+0000 7f2628276640 1 -- 172.21.15.112:0/2940350435 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2620075980 con 0x7f260406be60 2024-04-25T01:26:06.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.704+0000 7f2616ffd640 1 -- 172.21.15.112:0/2940350435 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+490417 (secure 0 0 0) 0x7f261c0016a0 con 0x7f260406be60 2024-04-25T01:26:06.689 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:06.692 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T01:26:06.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.708+0000 7f2628276640 1 -- 172.21.15.112:0/2940350435 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f260406be60 msgr2=0x7f260406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:06.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.709+0000 7f2628276640 1 --2- 172.21.15.112:0/2940350435 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f260406be60 0x7f260406e320 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f261c001550 tx=0x7f261c007c50 comp rx=0 tx=0).stop 2024-04-25T01:26:06.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.709+0000 7f2628276640 1 -- 172.21.15.112:0/2940350435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26200c5270 msgr2=0x7f262014a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:06.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.709+0000 7f2628276640 1 --2- 172.21.15.112:0/2940350435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26200c5270 0x7f262014a130 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f261000f5f0 tx=0x7f261000fab0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.709+0000 7f2628276640 1 -- 172.21.15.112:0/2940350435 shutdown_connections 2024-04-25T01:26:06.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.709+0000 7f2628276640 1 --2- 172.21.15.112:0/2940350435 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f260406be60 0x7f260406e320 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.709+0000 7f2628276640 1 --2- 172.21.15.112:0/2940350435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26200c5760 0x7f262014a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.709+0000 7f2628276640 1 --2- 172.21.15.112:0/2940350435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26200c5270 0x7f262014a130 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.709+0000 7f2628276640 1 --2- 172.21.15.112:0/2940350435 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f260c004d40 0x7f26200ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.709+0000 7f2628276640 1 -- 172.21.15.112:0/2940350435 >> 172.21.15.112:0/2940350435 conn(0x7f262006f310 msgr2=0x7f2620152c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:06.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.709+0000 7f2628276640 1 -- 172.21.15.112:0/2940350435 shutdown_connections 2024-04-25T01:26:06.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.709+0000 7f2628276640 1 -- 172.21.15.112:0/2940350435 wait complete. 2024-04-25T01:26:06.699 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-04-25T01:26:06.650636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":379,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3350528,"data_stored":1984464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000918"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415591+0000","last_change":"2024-04-25T01:26:02.412272+0000","last_active":"2024-04-25T01:26:03.415591+0000","last_peered":"2024-04-25T01:26:03.415591+0000","last_clean":"2024-04-25T01:26:03.415591+0000","last_became_active":"2024-04-25T01:26:02.411176+0000","last_became_peered":"2024-04-25T01:26:02.411176+0000","last_unstale":"2024-04-25T01:26:03.415591+0000","last_undegraded":"2024-04-25T01:26:03.415591+0000","last_fullsized":"2024-04-25T01:26:03.415591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:49.498275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716276+0000","last_change":"2024-04-25T01:25:52.353978+0000","last_active":"2024-04-25T01:26:03.716276+0000","last_peered":"2024-04-25T01:26:03.716276+0000","last_clean":"2024-04-25T01:26:03.716276+0000","last_became_active":"2024-04-25T01:25:52.353649+0000","last_became_peered":"2024-04-25T01:25:52.353649+0000","last_unstale":"2024-04-25T01:26:03.716276+0000","last_undegraded":"2024-04-25T01:26:03.716276+0000","last_fullsized":"2024-04-25T01:26:03.716276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416695+0000","last_change":"2024-04-25T01:25:53.364257+0000","last_active":"2024-04-25T01:26:03.416695+0000","last_peered":"2024-04-25T01:26:03.416695+0000","last_clean":"2024-04-25T01:26:03.416695+0000","last_became_active":"2024-04-25T01:25:53.363409+0000","last_became_peered":"2024-04-25T01:25:53.363409+0000","last_unstale":"2024-04-25T01:26:03.416695+0000","last_undegraded":"2024-04-25T01:26:03.416695+0000","last_fullsized":"2024-04-25T01:26:03.416695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415484+0000","last_change":"2024-04-25T01:26:01.407136+0000","last_active":"2024-04-25T01:26:03.415484+0000","last_peered":"2024-04-25T01:26:03.415484+0000","last_clean":"2024-04-25T01:26:03.415484+0000","last_became_active":"2024-04-25T01:26:01.406855+0000","last_became_peered":"2024-04-25T01:26:01.406855+0000","last_unstale":"2024-04-25T01:26:03.415484+0000","last_undegraded":"2024-04-25T01:26:03.415484+0000","last_fullsized":"2024-04-25T01:26:03.415484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:02.180826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716643+0000","last_change":"2024-04-25T01:26:02.425367+0000","last_active":"2024-04-25T01:26:03.716643+0000","last_peered":"2024-04-25T01:26:03.716643+0000","last_clean":"2024-04-25T01:26:03.716643+0000","last_became_active":"2024-04-25T01:26:02.425071+0000","last_became_peered":"2024-04-25T01:26:02.425071+0000","last_unstale":"2024-04-25T01:26:03.716643+0000","last_undegraded":"2024-04-25T01:26:03.716643+0000","last_fullsized":"2024-04-25T01:26:03.716643+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:47.810943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416737+0000","last_change":"2024-04-25T01:25:52.349185+0000","last_active":"2024-04-25T01:26:03.416737+0000","last_peered":"2024-04-25T01:26:03.416737+0000","last_clean":"2024-04-25T01:26:03.416737+0000","last_became_active":"2024-04-25T01:25:52.348931+0000","last_became_peered":"2024-04-25T01:25:52.348931+0000","last_unstale":"2024-04-25T01:26:03.416737+0000","last_undegraded":"2024-04-25T01:26:03.416737+0000","last_fullsized":"2024-04-25T01:26:03.416737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716082+0000","last_change":"2024-04-25T01:25:53.374150+0000","last_active":"2024-04-25T01:26:03.716082+0000","last_peered":"2024-04-25T01:26:03.716082+0000","last_clean":"2024-04-25T01:26:03.716082+0000","last_became_active":"2024-04-25T01:25:53.373755+0000","last_became_peered":"2024-04-25T01:25:53.373755+0000","last_unstale":"2024-04-25T01:26:03.716082+0000","last_undegraded":"2024-04-25T01:26:03.716082+0000","last_fullsized":"2024-04-25T01:26:03.716082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415473+0000","last_change":"2024-04-25T01:26:01.406334+0000","last_active":"2024-04-25T01:26:03.415473+0000","last_peered":"2024-04-25T01:26:03.415473+0000","last_clean":"2024-04-25T01:26:03.415473+0000","last_became_active":"2024-04-25T01:26:01.405994+0000","last_became_peered":"2024-04-25T01:26:01.405994+0000","last_unstale":"2024-04-25T01:26:03.415473+0000","last_undegraded":"2024-04-25T01:26:03.415473+0000","last_fullsized":"2024-04-25T01:26:03.415473+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:43.953437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507172+0000","last_change":"2024-04-25T01:26:02.427695+0000","last_active":"2024-04-25T01:26:03.507172+0000","last_peered":"2024-04-25T01:26:03.507172+0000","last_clean":"2024-04-25T01:26:03.507172+0000","last_became_active":"2024-04-25T01:26:02.427443+0000","last_became_peered":"2024-04-25T01:26:02.427443+0000","last_unstale":"2024-04-25T01:26:03.507172+0000","last_undegraded":"2024-04-25T01:26:03.507172+0000","last_fullsized":"2024-04-25T01:26:03.507172+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507616+0000","last_change":"2024-04-25T01:25:52.356403+0000","last_active":"2024-04-25T01:26:03.507616+0000","last_peered":"2024-04-25T01:26:03.507616+0000","last_clean":"2024-04-25T01:26:03.507616+0000","last_became_active":"2024-04-25T01:25:52.355822+0000","last_became_peered":"2024-04-25T01:25:52.355822+0000","last_unstale":"2024-04-25T01:26:03.507616+0000","last_undegraded":"2024-04-25T01:26:03.507616+0000","last_fullsized":"2024-04-25T01:26:03.507616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717449+0000","last_change":"2024-04-25T01:25:53.370061+0000","last_active":"2024-04-25T01:26:03.717449+0000","last_peered":"2024-04-25T01:26:03.717449+0000","last_clean":"2024-04-25T01:26:03.717449+0000","last_became_active":"2024-04-25T01:25:53.369721+0000","last_became_peered":"2024-04-25T01:25:53.369721+0000","last_unstale":"2024-04-25T01:26:03.717449+0000","last_undegraded":"2024-04-25T01:26:03.717449+0000","last_fullsized":"2024-04-25T01:26:03.717449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415934+0000","last_change":"2024-04-25T01:26:01.408179+0000","last_active":"2024-04-25T01:26:03.415934+0000","last_peered":"2024-04-25T01:26:03.415934+0000","last_clean":"2024-04-25T01:26:03.415934+0000","last_became_active":"2024-04-25T01:26:01.407909+0000","last_became_peered":"2024-04-25T01:26:01.407909+0000","last_unstale":"2024-04-25T01:26:03.415934+0000","last_undegraded":"2024-04-25T01:26:03.415934+0000","last_fullsized":"2024-04-25T01:26:03.415934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:48.865940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717234+0000","last_change":"2024-04-25T01:26:02.424615+0000","last_active":"2024-04-25T01:26:03.717234+0000","last_peered":"2024-04-25T01:26:03.717234+0000","last_clean":"2024-04-25T01:26:03.717234+0000","last_became_active":"2024-04-25T01:26:02.424325+0000","last_became_peered":"2024-04-25T01:26:02.424325+0000","last_unstale":"2024-04-25T01:26:03.717234+0000","last_undegraded":"2024-04-25T01:26:03.717234+0000","last_fullsized":"2024-04-25T01:26:03.717234+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507443+0000","last_change":"2024-04-25T01:25:52.359139+0000","last_active":"2024-04-25T01:26:03.507443+0000","last_peered":"2024-04-25T01:26:03.507443+0000","last_clean":"2024-04-25T01:26:03.507443+0000","last_became_active":"2024-04-25T01:25:52.358841+0000","last_became_peered":"2024-04-25T01:25:52.358841+0000","last_unstale":"2024-04-25T01:26:03.507443+0000","last_undegraded":"2024-04-25T01:26:03.507443+0000","last_fullsized":"2024-04-25T01:26:03.507443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717907+0000","last_change":"2024-04-25T01:25:53.368739+0000","last_active":"2024-04-25T01:26:03.717907+0000","last_peered":"2024-04-25T01:26:03.717907+0000","last_clean":"2024-04-25T01:26:03.717907+0000","last_became_active":"2024-04-25T01:25:53.368470+0000","last_became_peered":"2024-04-25T01:25:53.368470+0000","last_unstale":"2024-04-25T01:26:03.717907+0000","last_undegraded":"2024-04-25T01:26:03.717907+0000","last_fullsized":"2024-04-25T01:26:03.717907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:29.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415820+0000","last_change":"2024-04-25T01:26:01.407969+0000","last_active":"2024-04-25T01:26:03.415820+0000","last_peered":"2024-04-25T01:26:03.415820+0000","last_clean":"2024-04-25T01:26:03.415820+0000","last_became_active":"2024-04-25T01:26:01.407655+0000","last_became_peered":"2024-04-25T01:26:01.407655+0000","last_unstale":"2024-04-25T01:26:03.415820+0000","last_undegraded":"2024-04-25T01:26:03.415820+0000","last_fullsized":"2024-04-25T01:26:03.415820+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:58.485864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717005+0000","last_change":"2024-04-25T01:26:02.417776+0000","last_active":"2024-04-25T01:26:03.717005+0000","last_peered":"2024-04-25T01:26:03.717005+0000","last_clean":"2024-04-25T01:26:03.717005+0000","last_became_active":"2024-04-25T01:26:02.417524+0000","last_became_peered":"2024-04-25T01:26:02.417524+0000","last_unstale":"2024-04-25T01:26:03.717005+0000","last_undegraded":"2024-04-25T01:26:03.717005+0000","last_fullsized":"2024-04-25T01:26:03.717005+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.688469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716814+0000","last_change":"2024-04-25T01:25:53.372385+0000","last_active":"2024-04-25T01:26:03.716814+0000","last_peered":"2024-04-25T01:26:03.716814+0000","last_clean":"2024-04-25T01:26:03.716814+0000","last_became_active":"2024-04-25T01:25:53.372064+0000","last_became_peered":"2024-04-25T01:25:53.372064+0000","last_unstale":"2024-04-25T01:26:03.716814+0000","last_undegraded":"2024-04-25T01:26:03.716814+0000","last_fullsized":"2024-04-25T01:26:03.716814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507411+0000","last_change":"2024-04-25T01:25:52.358616+0000","last_active":"2024-04-25T01:26:03.507411+0000","last_peered":"2024-04-25T01:26:03.507411+0000","last_clean":"2024-04-25T01:26:03.507411+0000","last_became_active":"2024-04-25T01:25:52.358318+0000","last_became_peered":"2024-04-25T01:25:52.358318+0000","last_unstale":"2024-04-25T01:26:03.507411+0000","last_undegraded":"2024-04-25T01:26:03.507411+0000","last_fullsized":"2024-04-25T01:26:03.507411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416542+0000","last_change":"2024-04-25T01:26:01.400599+0000","last_active":"2024-04-25T01:26:03.416542+0000","last_peered":"2024-04-25T01:26:03.416542+0000","last_clean":"2024-04-25T01:26:03.416542+0000","last_became_active":"2024-04-25T01:26:01.400266+0000","last_became_peered":"2024-04-25T01:26:01.400266+0000","last_unstale":"2024-04-25T01:26:03.416542+0000","last_undegraded":"2024-04-25T01:26:03.416542+0000","last_fullsized":"2024-04-25T01:26:03.416542+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416953+0000","last_change":"2024-04-25T01:26:02.420269+0000","last_active":"2024-04-25T01:26:03.416953+0000","last_peered":"2024-04-25T01:26:03.416953+0000","last_clean":"2024-04-25T01:26:03.416953+0000","last_became_active":"2024-04-25T01:26:02.420001+0000","last_became_peered":"2024-04-25T01:26:02.420001+0000","last_unstale":"2024-04-25T01:26:03.416953+0000","last_undegraded":"2024-04-25T01:26:03.416953+0000","last_fullsized":"2024-04-25T01:26:03.416953+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:38.149069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415549+0000","last_change":"2024-04-25T01:25:53.361074+0000","last_active":"2024-04-25T01:26:03.415549+0000","last_peered":"2024-04-25T01:26:03.415549+0000","last_clean":"2024-04-25T01:26:03.415549+0000","last_became_active":"2024-04-25T01:25:53.360802+0000","last_became_peered":"2024-04-25T01:25:53.360802+0000","last_unstale":"2024-04-25T01:26:03.415549+0000","last_undegraded":"2024-04-25T01:26:03.415549+0000","last_fullsized":"2024-04-25T01:26:03.415549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717195+0000","last_change":"2024-04-25T01:25:52.348841+0000","last_active":"2024-04-25T01:26:03.717195+0000","last_peered":"2024-04-25T01:26:03.717195+0000","last_clean":"2024-04-25T01:26:03.717195+0000","last_became_active":"2024-04-25T01:25:52.348550+0000","last_became_peered":"2024-04-25T01:25:52.348550+0000","last_unstale":"2024-04-25T01:26:03.717195+0000","last_undegraded":"2024-04-25T01:26:03.717195+0000","last_fullsized":"2024-04-25T01:26:03.717195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416126+0000","last_change":"2024-04-25T01:26:01.408529+0000","last_active":"2024-04-25T01:26:03.416126+0000","last_peered":"2024-04-25T01:26:03.416126+0000","last_clean":"2024-04-25T01:26:03.416126+0000","last_became_active":"2024-04-25T01:26:01.407738+0000","last_became_peered":"2024-04-25T01:26:01.407738+0000","last_unstale":"2024-04-25T01:26:03.416126+0000","last_undegraded":"2024-04-25T01:26:03.416126+0000","last_fullsized":"2024-04-25T01:26:03.416126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:37.142255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416507+0000","last_change":"2024-04-25T01:26:02.417731+0000","last_active":"2024-04-25T01:26:03.416507+0000","last_peered":"2024-04-25T01:26:03.416507+0000","last_clean":"2024-04-25T01:26:03.416507+0000","last_became_active":"2024-04-25T01:26:02.417438+0000","last_became_peered":"2024-04-25T01:26:02.417438+0000","last_unstale":"2024-04-25T01:26:03.416507+0000","last_undegraded":"2024-04-25T01:26:03.416507+0000","last_fullsized":"2024-04-25T01:26:03.416507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415373+0000","last_change":"2024-04-25T01:25:53.361790+0000","last_active":"2024-04-25T01:26:03.415373+0000","last_peered":"2024-04-25T01:26:03.415373+0000","last_clean":"2024-04-25T01:26:03.415373+0000","last_became_active":"2024-04-25T01:25:53.361542+0000","last_became_peered":"2024-04-25T01:25:53.361542+0000","last_unstale":"2024-04-25T01:26:03.415373+0000","last_undegraded":"2024-04-25T01:26:03.415373+0000","last_fullsized":"2024-04-25T01:26:03.415373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716946+0000","last_change":"2024-04-25T01:25:52.353926+0000","last_active":"2024-04-25T01:26:03.716946+0000","last_peered":"2024-04-25T01:26:03.716946+0000","last_clean":"2024-04-25T01:26:03.716946+0000","last_became_active":"2024-04-25T01:25:52.353656+0000","last_became_peered":"2024-04-25T01:25:52.353656+0000","last_unstale":"2024-04-25T01:26:03.716946+0000","last_undegraded":"2024-04-25T01:26:03.716946+0000","last_fullsized":"2024-04-25T01:26:03.716946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716923+0000","last_change":"2024-04-25T01:26:01.418556+0000","last_active":"2024-04-25T01:26:03.716923+0000","last_peered":"2024-04-25T01:26:03.716923+0000","last_clean":"2024-04-25T01:26:03.716923+0000","last_became_active":"2024-04-25T01:26:01.418263+0000","last_became_peered":"2024-04-25T01:26:01.418263+0000","last_unstale":"2024-04-25T01:26:03.716923+0000","last_undegraded":"2024-04-25T01:26:03.716923+0000","last_fullsized":"2024-04-25T01:26:03.716923+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:28.139183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717839+0000","last_change":"2024-04-25T01:26:02.413429+0000","last_active":"2024-04-25T01:26:03.717839+0000","last_peered":"2024-04-25T01:26:03.717839+0000","last_clean":"2024-04-25T01:26:03.717839+0000","last_became_active":"2024-04-25T01:26:02.412632+0000","last_became_peered":"2024-04-25T01:26:02.412632+0000","last_unstale":"2024-04-25T01:26:03.717839+0000","last_undegraded":"2024-04-25T01:26:03.717839+0000","last_fullsized":"2024-04-25T01:26:03.717839+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:10.049310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416067+0000","last_change":"2024-04-25T01:25:53.361653+0000","last_active":"2024-04-25T01:26:03.416067+0000","last_peered":"2024-04-25T01:26:03.416067+0000","last_clean":"2024-04-25T01:26:03.416067+0000","last_became_active":"2024-04-25T01:25:53.361265+0000","last_became_peered":"2024-04-25T01:25:53.361265+0000","last_unstale":"2024-04-25T01:26:03.416067+0000","last_undegraded":"2024-04-25T01:26:03.416067+0000","last_fullsized":"2024-04-25T01:26:03.416067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507177+0000","last_change":"2024-04-25T01:25:52.353836+0000","last_active":"2024-04-25T01:26:03.507177+0000","last_peered":"2024-04-25T01:26:03.507177+0000","last_clean":"2024-04-25T01:26:03.507177+0000","last_became_active":"2024-04-25T01:25:52.353552+0000","last_became_peered":"2024-04-25T01:25:52.353552+0000","last_unstale":"2024-04-25T01:26:03.507177+0000","last_undegraded":"2024-04-25T01:26:03.507177+0000","last_fullsized":"2024-04-25T01:26:03.507177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416325+0000","last_change":"2024-04-25T01:26:01.411323+0000","last_active":"2024-04-25T01:26:03.416325+0000","last_peered":"2024-04-25T01:26:03.416325+0000","last_clean":"2024-04-25T01:26:03.416325+0000","last_became_active":"2024-04-25T01:26:01.410378+0000","last_became_peered":"2024-04-25T01:26:01.410378+0000","last_unstale":"2024-04-25T01:26:03.416325+0000","last_undegraded":"2024-04-25T01:26:03.416325+0000","last_fullsized":"2024-04-25T01:26:03.416325+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415000+0000","last_change":"2024-04-25T01:26:02.416646+0000","last_active":"2024-04-25T01:26:03.415000+0000","last_peered":"2024-04-25T01:26:03.415000+0000","last_clean":"2024-04-25T01:26:03.415000+0000","last_became_active":"2024-04-25T01:26:02.416395+0000","last_became_peered":"2024-04-25T01:26:02.416395+0000","last_unstale":"2024-04-25T01:26:03.415000+0000","last_undegraded":"2024-04-25T01:26:03.415000+0000","last_fullsized":"2024-04-25T01:26:03.415000+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716795+0000","last_change":"2024-04-25T01:25:53.370111+0000","last_active":"2024-04-25T01:26:03.716795+0000","last_peered":"2024-04-25T01:26:03.716795+0000","last_clean":"2024-04-25T01:26:03.716795+0000","last_became_active":"2024-04-25T01:25:53.369837+0000","last_became_peered":"2024-04-25T01:25:53.369837+0000","last_unstale":"2024-04-25T01:26:03.716795+0000","last_undegraded":"2024-04-25T01:26:03.716795+0000","last_fullsized":"2024-04-25T01:26:03.716795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717961+0000","last_change":"2024-04-25T01:25:52.353269+0000","last_active":"2024-04-25T01:26:03.717961+0000","last_peered":"2024-04-25T01:26:03.717961+0000","last_clean":"2024-04-25T01:26:03.717961+0000","last_became_active":"2024-04-25T01:25:52.352991+0000","last_became_peered":"2024-04-25T01:25:52.352991+0000","last_unstale":"2024-04-25T01:26:03.717961+0000","last_undegraded":"2024-04-25T01:26:03.717961+0000","last_fullsized":"2024-04-25T01:26:03.717961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415651+0000","last_change":"2024-04-25T01:26:01.410360+0000","last_active":"2024-04-25T01:26:03.415651+0000","last_peered":"2024-04-25T01:26:03.415651+0000","last_clean":"2024-04-25T01:26:03.415651+0000","last_became_active":"2024-04-25T01:26:01.410076+0000","last_became_peered":"2024-04-25T01:26:01.410076+0000","last_unstale":"2024-04-25T01:26:03.415651+0000","last_undegraded":"2024-04-25T01:26:03.415651+0000","last_fullsized":"2024-04-25T01:26:03.415651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:52.561251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415577+0000","last_change":"2024-04-25T01:26:02.422182+0000","last_active":"2024-04-25T01:26:03.415577+0000","last_peered":"2024-04-25T01:26:03.415577+0000","last_clean":"2024-04-25T01:26:03.415577+0000","last_became_active":"2024-04-25T01:26:02.421885+0000","last_became_peered":"2024-04-25T01:26:02.421885+0000","last_unstale":"2024-04-25T01:26:03.415577+0000","last_undegraded":"2024-04-25T01:26:03.415577+0000","last_fullsized":"2024-04-25T01:26:03.415577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:03.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416628+0000","last_change":"2024-04-25T01:25:53.364967+0000","last_active":"2024-04-25T01:26:03.416628+0000","last_peered":"2024-04-25T01:26:03.416628+0000","last_clean":"2024-04-25T01:26:03.416628+0000","last_became_active":"2024-04-25T01:25:53.364694+0000","last_became_peered":"2024-04-25T01:25:53.364694+0000","last_unstale":"2024-04-25T01:26:03.416628+0000","last_undegraded":"2024-04-25T01:26:03.416628+0000","last_fullsized":"2024-04-25T01:26:03.416628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717049+0000","last_change":"2024-04-25T01:25:52.355395+0000","last_active":"2024-04-25T01:26:03.717049+0000","last_peered":"2024-04-25T01:26:03.717049+0000","last_clean":"2024-04-25T01:26:03.717049+0000","last_became_active":"2024-04-25T01:25:52.355137+0000","last_became_peered":"2024-04-25T01:25:52.355137+0000","last_unstale":"2024-04-25T01:26:03.717049+0000","last_undegraded":"2024-04-25T01:26:03.717049+0000","last_fullsized":"2024-04-25T01:26:03.717049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:26.467968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.414943+0000","last_change":"2024-04-25T01:26:01.416108+0000","last_active":"2024-04-25T01:26:03.414943+0000","last_peered":"2024-04-25T01:26:03.414943+0000","last_clean":"2024-04-25T01:26:03.414943+0000","last_became_active":"2024-04-25T01:26:01.415824+0000","last_became_peered":"2024-04-25T01:26:01.415824+0000","last_unstale":"2024-04-25T01:26:03.414943+0000","last_undegraded":"2024-04-25T01:26:03.414943+0000","last_fullsized":"2024-04-25T01:26:03.414943+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:28.139183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716568+0000","last_change":"2024-04-25T01:26:02.416208+0000","last_active":"2024-04-25T01:26:03.716568+0000","last_peered":"2024-04-25T01:26:03.716568+0000","last_clean":"2024-04-25T01:26:03.716568+0000","last_became_active":"2024-04-25T01:26:02.415949+0000","last_became_peered":"2024-04-25T01:26:02.415949+0000","last_unstale":"2024-04-25T01:26:03.716568+0000","last_undegraded":"2024-04-25T01:26:03.716568+0000","last_fullsized":"2024-04-25T01:26:03.716568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:21.097225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717249+0000","last_change":"2024-04-25T01:25:53.366944+0000","last_active":"2024-04-25T01:26:03.717249+0000","last_peered":"2024-04-25T01:26:03.717249+0000","last_clean":"2024-04-25T01:26:03.717249+0000","last_became_active":"2024-04-25T01:25:53.366693+0000","last_became_peered":"2024-04-25T01:25:53.366693+0000","last_unstale":"2024-04-25T01:26:03.717249+0000","last_undegraded":"2024-04-25T01:26:03.717249+0000","last_fullsized":"2024-04-25T01:26:03.717249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507732+0000","last_change":"2024-04-25T01:25:52.356842+0000","last_active":"2024-04-25T01:26:03.507732+0000","last_peered":"2024-04-25T01:26:03.507732+0000","last_clean":"2024-04-25T01:26:03.507732+0000","last_became_active":"2024-04-25T01:25:52.356396+0000","last_became_peered":"2024-04-25T01:25:52.356396+0000","last_unstale":"2024-04-25T01:26:03.507732+0000","last_undegraded":"2024-04-25T01:26:03.507732+0000","last_fullsized":"2024-04-25T01:26:03.507732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716730+0000","last_change":"2024-04-25T01:26:01.407934+0000","last_active":"2024-04-25T01:26:03.716730+0000","last_peered":"2024-04-25T01:26:03.716730+0000","last_clean":"2024-04-25T01:26:03.716730+0000","last_became_active":"2024-04-25T01:26:01.407659+0000","last_became_peered":"2024-04-25T01:26:01.407659+0000","last_unstale":"2024-04-25T01:26:03.716730+0000","last_undegraded":"2024-04-25T01:26:03.716730+0000","last_fullsized":"2024-04-25T01:26:03.716730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:19.629335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716666+0000","last_change":"2024-04-25T01:26:02.418087+0000","last_active":"2024-04-25T01:26:03.716666+0000","last_peered":"2024-04-25T01:26:03.716666+0000","last_clean":"2024-04-25T01:26:03.716666+0000","last_became_active":"2024-04-25T01:26:02.417622+0000","last_became_peered":"2024-04-25T01:26:02.417622+0000","last_unstale":"2024-04-25T01:26:03.716666+0000","last_undegraded":"2024-04-25T01:26:03.716666+0000","last_fullsized":"2024-04-25T01:26:03.716666+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:03.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717962+0000","last_change":"2024-04-25T01:25:53.368164+0000","last_active":"2024-04-25T01:26:03.717962+0000","last_peered":"2024-04-25T01:26:03.717962+0000","last_clean":"2024-04-25T01:26:03.717962+0000","last_became_active":"2024-04-25T01:25:53.367854+0000","last_became_peered":"2024-04-25T01:25:53.367854+0000","last_unstale":"2024-04-25T01:26:03.717962+0000","last_undegraded":"2024-04-25T01:26:03.717962+0000","last_fullsized":"2024-04-25T01:26:03.717962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717294+0000","last_change":"2024-04-25T01:25:52.350128+0000","last_active":"2024-04-25T01:26:03.717294+0000","last_peered":"2024-04-25T01:26:03.717294+0000","last_clean":"2024-04-25T01:26:03.717294+0000","last_became_active":"2024-04-25T01:25:52.349874+0000","last_became_peered":"2024-04-25T01:25:52.349874+0000","last_unstale":"2024-04-25T01:26:03.717294+0000","last_undegraded":"2024-04-25T01:26:03.717294+0000","last_fullsized":"2024-04-25T01:26:03.717294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716937+0000","last_change":"2024-04-25T01:26:01.418196+0000","last_active":"2024-04-25T01:26:03.716937+0000","last_peered":"2024-04-25T01:26:03.716937+0000","last_clean":"2024-04-25T01:26:03.716937+0000","last_became_active":"2024-04-25T01:26:01.417820+0000","last_became_peered":"2024-04-25T01:26:01.417820+0000","last_unstale":"2024-04-25T01:26:03.716937+0000","last_undegraded":"2024-04-25T01:26:03.716937+0000","last_fullsized":"2024-04-25T01:26:03.716937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716324+0000","last_change":"2024-04-25T01:26:02.422761+0000","last_active":"2024-04-25T01:26:03.716324+0000","last_peered":"2024-04-25T01:26:03.716324+0000","last_clean":"2024-04-25T01:26:03.716324+0000","last_became_active":"2024-04-25T01:26:02.422503+0000","last_became_peered":"2024-04-25T01:26:02.422503+0000","last_unstale":"2024-04-25T01:26:03.716324+0000","last_undegraded":"2024-04-25T01:26:03.716324+0000","last_fullsized":"2024-04-25T01:26:03.716324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:55.040946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416617+0000","last_change":"2024-04-25T01:25:53.361696+0000","last_active":"2024-04-25T01:26:03.416617+0000","last_peered":"2024-04-25T01:26:03.416617+0000","last_clean":"2024-04-25T01:26:03.416617+0000","last_became_active":"2024-04-25T01:25:53.361156+0000","last_became_peered":"2024-04-25T01:25:53.361156+0000","last_unstale":"2024-04-25T01:26:03.416617+0000","last_undegraded":"2024-04-25T01:26:03.416617+0000","last_fullsized":"2024-04-25T01:26:03.416617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:27.472315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717721+0000","last_change":"2024-04-25T01:25:52.354499+0000","last_active":"2024-04-25T01:26:03.717721+0000","last_peered":"2024-04-25T01:26:03.717721+0000","last_clean":"2024-04-25T01:26:03.717721+0000","last_became_active":"2024-04-25T01:25:52.354193+0000","last_became_peered":"2024-04-25T01:25:52.354193+0000","last_unstale":"2024-04-25T01:26:03.717721+0000","last_undegraded":"2024-04-25T01:26:03.717721+0000","last_fullsized":"2024-04-25T01:26:03.717721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717190+0000","last_change":"2024-04-25T01:26:01.418186+0000","last_active":"2024-04-25T01:26:03.717190+0000","last_peered":"2024-04-25T01:26:03.717190+0000","last_clean":"2024-04-25T01:26:03.717190+0000","last_became_active":"2024-04-25T01:26:01.417805+0000","last_became_peered":"2024-04-25T01:26:01.417805+0000","last_unstale":"2024-04-25T01:26:03.717190+0000","last_undegraded":"2024-04-25T01:26:03.717190+0000","last_fullsized":"2024-04-25T01:26:03.717190+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:19.629335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416697+0000","last_change":"2024-04-25T01:26:02.415877+0000","last_active":"2024-04-25T01:26:03.416697+0000","last_peered":"2024-04-25T01:26:03.416697+0000","last_clean":"2024-04-25T01:26:03.416697+0000","last_became_active":"2024-04-25T01:26:02.415619+0000","last_became_peered":"2024-04-25T01:26:02.415619+0000","last_unstale":"2024-04-25T01:26:03.416697+0000","last_undegraded":"2024-04-25T01:26:03.416697+0000","last_fullsized":"2024-04-25T01:26:03.416697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:14.647083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716434+0000","last_change":"2024-04-25T01:25:53.362922+0000","last_active":"2024-04-25T01:26:03.716434+0000","last_peered":"2024-04-25T01:26:03.716434+0000","last_clean":"2024-04-25T01:26:03.716434+0000","last_became_active":"2024-04-25T01:25:53.362533+0000","last_became_peered":"2024-04-25T01:25:53.362533+0000","last_unstale":"2024-04-25T01:26:03.716434+0000","last_undegraded":"2024-04-25T01:26:03.716434+0000","last_fullsized":"2024-04-25T01:26:03.716434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415851+0000","last_change":"2024-04-25T01:25:52.345321+0000","last_active":"2024-04-25T01:26:03.415851+0000","last_peered":"2024-04-25T01:26:03.415851+0000","last_clean":"2024-04-25T01:26:03.415851+0000","last_became_active":"2024-04-25T01:25:52.345061+0000","last_became_peered":"2024-04-25T01:25:52.345061+0000","last_unstale":"2024-04-25T01:26:03.415851+0000","last_undegraded":"2024-04-25T01:26:03.415851+0000","last_fullsized":"2024-04-25T01:26:03.415851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717072+0000","last_change":"2024-04-25T01:26:01.418516+0000","last_active":"2024-04-25T01:26:03.717072+0000","last_peered":"2024-04-25T01:26:03.717072+0000","last_clean":"2024-04-25T01:26:03.717072+0000","last_became_active":"2024-04-25T01:26:01.418173+0000","last_became_peered":"2024-04-25T01:26:01.418173+0000","last_unstale":"2024-04-25T01:26:03.717072+0000","last_undegraded":"2024-04-25T01:26:03.717072+0000","last_fullsized":"2024-04-25T01:26:03.717072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:02.568472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.718145+0000","last_change":"2024-04-25T01:26:02.418115+0000","last_active":"2024-04-25T01:26:03.718145+0000","last_peered":"2024-04-25T01:26:03.718145+0000","last_clean":"2024-04-25T01:26:03.718145+0000","last_became_active":"2024-04-25T01:26:02.417355+0000","last_became_peered":"2024-04-25T01:26:02.417355+0000","last_unstale":"2024-04-25T01:26:03.718145+0000","last_undegraded":"2024-04-25T01:26:03.718145+0000","last_fullsized":"2024-04-25T01:26:03.718145+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:19.644468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716066+0000","last_change":"2024-04-25T01:25:53.355950+0000","last_active":"2024-04-25T01:26:03.716066+0000","last_peered":"2024-04-25T01:26:03.716066+0000","last_clean":"2024-04-25T01:26:03.716066+0000","last_became_active":"2024-04-25T01:25:53.355648+0000","last_became_peered":"2024-04-25T01:25:53.355648+0000","last_unstale":"2024-04-25T01:26:03.716066+0000","last_undegraded":"2024-04-25T01:26:03.716066+0000","last_fullsized":"2024-04-25T01:26:03.716066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507424+0000","last_change":"2024-04-25T01:25:52.357683+0000","last_active":"2024-04-25T01:26:03.507424+0000","last_peered":"2024-04-25T01:26:03.507424+0000","last_clean":"2024-04-25T01:26:03.507424+0000","last_became_active":"2024-04-25T01:25:52.356924+0000","last_became_peered":"2024-04-25T01:25:52.356924+0000","last_unstale":"2024-04-25T01:26:03.507424+0000","last_undegraded":"2024-04-25T01:26:03.507424+0000","last_fullsized":"2024-04-25T01:26:03.507424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716612+0000","last_change":"2024-04-25T01:26:01.421260+0000","last_active":"2024-04-25T01:26:03.716612+0000","last_peered":"2024-04-25T01:26:03.716612+0000","last_clean":"2024-04-25T01:26:03.716612+0000","last_became_active":"2024-04-25T01:26:01.420831+0000","last_became_peered":"2024-04-25T01:26:01.420831+0000","last_unstale":"2024-04-25T01:26:03.716612+0000","last_undegraded":"2024-04-25T01:26:03.716612+0000","last_fullsized":"2024-04-25T01:26:03.716612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507519+0000","last_change":"2024-04-25T01:26:02.421386+0000","last_active":"2024-04-25T01:26:03.507519+0000","last_peered":"2024-04-25T01:26:03.507519+0000","last_clean":"2024-04-25T01:26:03.507519+0000","last_became_active":"2024-04-25T01:26:02.421079+0000","last_became_peered":"2024-04-25T01:26:02.421079+0000","last_unstale":"2024-04-25T01:26:03.507519+0000","last_undegraded":"2024-04-25T01:26:03.507519+0000","last_fullsized":"2024-04-25T01:26:03.507519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416038+0000","last_change":"2024-04-25T01:25:53.374719+0000","last_active":"2024-04-25T01:26:03.416038+0000","last_peered":"2024-04-25T01:26:03.416038+0000","last_clean":"2024-04-25T01:26:03.416038+0000","last_became_active":"2024-04-25T01:25:53.374425+0000","last_became_peered":"2024-04-25T01:25:53.374425+0000","last_unstale":"2024-04-25T01:26:03.416038+0000","last_undegraded":"2024-04-25T01:26:03.416038+0000","last_fullsized":"2024-04-25T01:26:03.416038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717309+0000","last_change":"2024-04-25T01:25:52.353155+0000","last_active":"2024-04-25T01:26:03.717309+0000","last_peered":"2024-04-25T01:26:03.717309+0000","last_clean":"2024-04-25T01:26:03.717309+0000","last_became_active":"2024-04-25T01:25:52.352826+0000","last_became_peered":"2024-04-25T01:25:52.352826+0000","last_unstale":"2024-04-25T01:26:03.717309+0000","last_undegraded":"2024-04-25T01:26:03.717309+0000","last_fullsized":"2024-04-25T01:26:03.717309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507554+0000","last_change":"2024-04-25T01:25:41.719670+0000","last_active":"2024-04-25T01:26:03.507554+0000","last_peered":"2024-04-25T01:26:03.507554+0000","last_clean":"2024-04-25T01:26:03.507554+0000","last_became_active":"2024-04-25T01:25:41.718721+0000","last_became_peered":"2024-04-25T01:25:41.718721+0000","last_unstale":"2024-04-25T01:26:03.507554+0000","last_undegraded":"2024-04-25T01:26:03.507554+0000","last_fullsized":"2024-04-25T01:26:03.507554+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_clean_scrub_stamp":"2024-04-25T01:25:40.688097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:27.456746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717378+0000","last_change":"2024-04-25T01:26:01.409339+0000","last_active":"2024-04-25T01:26:03.717378+0000","last_peered":"2024-04-25T01:26:03.717378+0000","last_clean":"2024-04-25T01:26:03.717378+0000","last_became_active":"2024-04-25T01:26:01.409062+0000","last_became_peered":"2024-04-25T01:26:01.409062+0000","last_unstale":"2024-04-25T01:26:03.717378+0000","last_undegraded":"2024-04-25T01:26:03.717378+0000","last_fullsized":"2024-04-25T01:26:03.717378+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:52.561251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415532+0000","last_change":"2024-04-25T01:26:02.418574+0000","last_active":"2024-04-25T01:26:03.415532+0000","last_peered":"2024-04-25T01:26:03.415532+0000","last_clean":"2024-04-25T01:26:03.415532+0000","last_became_active":"2024-04-25T01:26:02.416546+0000","last_became_peered":"2024-04-25T01:26:02.416546+0000","last_unstale":"2024-04-25T01:26:03.415532+0000","last_undegraded":"2024-04-25T01:26:03.415532+0000","last_fullsized":"2024-04-25T01:26:03.415532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:43.994141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507591+0000","last_change":"2024-04-25T01:25:53.375523+0000","last_active":"2024-04-25T01:26:03.507591+0000","last_peered":"2024-04-25T01:26:03.507591+0000","last_clean":"2024-04-25T01:26:03.507591+0000","last_became_active":"2024-04-25T01:25:53.375219+0000","last_became_peered":"2024-04-25T01:25:53.375219+0000","last_unstale":"2024-04-25T01:26:03.507591+0000","last_undegraded":"2024-04-25T01:26:03.507591+0000","last_fullsized":"2024-04-25T01:26:03.507591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.417443+0000","last_change":"2024-04-25T01:25:52.352994+0000","last_active":"2024-04-25T01:26:03.417443+0000","last_peered":"2024-04-25T01:26:03.417443+0000","last_clean":"2024-04-25T01:26:03.417443+0000","last_became_active":"2024-04-25T01:25:52.352644+0000","last_became_peered":"2024-04-25T01:25:52.352644+0000","last_unstale":"2024-04-25T01:26:03.417443+0000","last_undegraded":"2024-04-25T01:26:03.417443+0000","last_fullsized":"2024-04-25T01:26:03.417443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716482+0000","last_change":"2024-04-25T01:26:01.420891+0000","last_active":"2024-04-25T01:26:03.716482+0000","last_peered":"2024-04-25T01:26:03.716482+0000","last_clean":"2024-04-25T01:26:03.716482+0000","last_became_active":"2024-04-25T01:26:01.420664+0000","last_became_peered":"2024-04-25T01:26:01.420664+0000","last_unstale":"2024-04-25T01:26:03.716482+0000","last_undegraded":"2024-04-25T01:26:03.716482+0000","last_fullsized":"2024-04-25T01:26:03.716482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415682+0000","last_change":"2024-04-25T01:26:02.403489+0000","last_active":"2024-04-25T01:26:03.415682+0000","last_peered":"2024-04-25T01:26:03.415682+0000","last_clean":"2024-04-25T01:26:03.415682+0000","last_became_active":"2024-04-25T01:26:02.403206+0000","last_became_peered":"2024-04-25T01:26:02.403206+0000","last_unstale":"2024-04-25T01:26:03.415682+0000","last_undegraded":"2024-04-25T01:26:03.415682+0000","last_fullsized":"2024-04-25T01:26:03.415682+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:49.872754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.718041+0000","last_change":"2024-04-25T01:25:53.371018+0000","last_active":"2024-04-25T01:26:03.718041+0000","last_peered":"2024-04-25T01:26:03.718041+0000","last_clean":"2024-04-25T01:26:03.718041+0000","last_became_active":"2024-04-25T01:25:53.370539+0000","last_became_peered":"2024-04-25T01:25:53.370539+0000","last_unstale":"2024-04-25T01:26:03.718041+0000","last_undegraded":"2024-04-25T01:26:03.718041+0000","last_fullsized":"2024-04-25T01:26:03.718041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717910+0000","last_change":"2024-04-25T01:25:52.355133+0000","last_active":"2024-04-25T01:26:03.717910+0000","last_peered":"2024-04-25T01:26:03.717910+0000","last_clean":"2024-04-25T01:26:03.717910+0000","last_became_active":"2024-04-25T01:25:52.354836+0000","last_became_peered":"2024-04-25T01:25:52.354836+0000","last_unstale":"2024-04-25T01:26:03.717910+0000","last_undegraded":"2024-04-25T01:26:03.717910+0000","last_fullsized":"2024-04-25T01:26:03.717910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:10.331704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716238+0000","last_change":"2024-04-25T01:26:01.419285+0000","last_active":"2024-04-25T01:26:03.716238+0000","last_peered":"2024-04-25T01:26:03.716238+0000","last_clean":"2024-04-25T01:26:03.716238+0000","last_became_active":"2024-04-25T01:26:01.418987+0000","last_became_peered":"2024-04-25T01:26:01.418987+0000","last_unstale":"2024-04-25T01:26:03.716238+0000","last_undegraded":"2024-04-25T01:26:03.716238+0000","last_fullsized":"2024-04-25T01:26:03.716238+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.506958+0000","last_change":"2024-04-25T01:26:02.416544+0000","last_active":"2024-04-25T01:26:03.506958+0000","last_peered":"2024-04-25T01:26:03.506958+0000","last_clean":"2024-04-25T01:26:03.506958+0000","last_became_active":"2024-04-25T01:26:02.416189+0000","last_became_peered":"2024-04-25T01:26:02.416189+0000","last_unstale":"2024-04-25T01:26:03.506958+0000","last_undegraded":"2024-04-25T01:26:03.506958+0000","last_fullsized":"2024-04-25T01:26:03.506958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.688469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.508134+0000","last_change":"2024-04-25T01:25:53.370754+0000","last_active":"2024-04-25T01:26:03.508134+0000","last_peered":"2024-04-25T01:26:03.508134+0000","last_clean":"2024-04-25T01:26:03.508134+0000","last_became_active":"2024-04-25T01:25:53.370074+0000","last_became_peered":"2024-04-25T01:25:53.370074+0000","last_unstale":"2024-04-25T01:26:03.508134+0000","last_undegraded":"2024-04-25T01:26:03.508134+0000","last_fullsized":"2024-04-25T01:26:03.508134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416836+0000","last_change":"2024-04-25T01:25:52.345943+0000","last_active":"2024-04-25T01:26:03.416836+0000","last_peered":"2024-04-25T01:26:03.416836+0000","last_clean":"2024-04-25T01:26:03.416836+0000","last_became_active":"2024-04-25T01:25:52.345717+0000","last_became_peered":"2024-04-25T01:25:52.345717+0000","last_unstale":"2024-04-25T01:26:03.416836+0000","last_undegraded":"2024-04-25T01:26:03.416836+0000","last_fullsized":"2024-04-25T01:26:03.416836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716622+0000","last_change":"2024-04-25T01:26:01.416027+0000","last_active":"2024-04-25T01:26:03.716622+0000","last_peered":"2024-04-25T01:26:03.716622+0000","last_clean":"2024-04-25T01:26:03.716622+0000","last_became_active":"2024-04-25T01:26:01.415750+0000","last_became_peered":"2024-04-25T01:26:01.415750+0000","last_unstale":"2024-04-25T01:26:03.716622+0000","last_undegraded":"2024-04-25T01:26:03.716622+0000","last_fullsized":"2024-04-25T01:26:03.716622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:27.042376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416127+0000","last_change":"2024-04-25T01:26:02.414929+0000","last_active":"2024-04-25T01:26:03.416127+0000","last_peered":"2024-04-25T01:26:03.416127+0000","last_clean":"2024-04-25T01:26:03.416127+0000","last_became_active":"2024-04-25T01:26:02.414316+0000","last_became_peered":"2024-04-25T01:26:02.414316+0000","last_unstale":"2024-04-25T01:26:03.416127+0000","last_undegraded":"2024-04-25T01:26:03.416127+0000","last_fullsized":"2024-04-25T01:26:03.416127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:53.568065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507875+0000","last_change":"2024-04-25T01:25:53.365171+0000","last_active":"2024-04-25T01:26:03.507875+0000","last_peered":"2024-04-25T01:26:03.507875+0000","last_clean":"2024-04-25T01:26:03.507875+0000","last_became_active":"2024-04-25T01:25:53.364049+0000","last_became_peered":"2024-04-25T01:25:53.364049+0000","last_unstale":"2024-04-25T01:26:03.507875+0000","last_undegraded":"2024-04-25T01:26:03.507875+0000","last_fullsized":"2024-04-25T01:26:03.507875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416635+0000","last_change":"2024-04-25T01:25:52.350688+0000","last_active":"2024-04-25T01:26:03.416635+0000","last_peered":"2024-04-25T01:26:03.416635+0000","last_clean":"2024-04-25T01:26:03.416635+0000","last_became_active":"2024-04-25T01:25:52.350357+0000","last_became_peered":"2024-04-25T01:25:52.350357+0000","last_unstale":"2024-04-25T01:26:03.416635+0000","last_undegraded":"2024-04-25T01:26:03.416635+0000","last_fullsized":"2024-04-25T01:26:03.416635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415491+0000","last_change":"2024-04-25T01:26:01.418580+0000","last_active":"2024-04-25T01:26:03.415491+0000","last_peered":"2024-04-25T01:26:03.415491+0000","last_clean":"2024-04-25T01:26:03.415491+0000","last_became_active":"2024-04-25T01:26:01.418179+0000","last_became_peered":"2024-04-25T01:26:01.418179+0000","last_unstale":"2024-04-25T01:26:03.415491+0000","last_undegraded":"2024-04-25T01:26:03.415491+0000","last_fullsized":"2024-04-25T01:26:03.415491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:42.987327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416438+0000","last_change":"2024-04-25T01:26:02.418424+0000","last_active":"2024-04-25T01:26:03.416438+0000","last_peered":"2024-04-25T01:26:03.416438+0000","last_clean":"2024-04-25T01:26:03.416438+0000","last_became_active":"2024-04-25T01:26:02.418090+0000","last_became_peered":"2024-04-25T01:26:02.418090+0000","last_unstale":"2024-04-25T01:26:03.416438+0000","last_undegraded":"2024-04-25T01:26:03.416438+0000","last_fullsized":"2024-04-25T01:26:03.416438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:10.049310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507217+0000","last_change":"2024-04-25T01:25:53.366376+0000","last_active":"2024-04-25T01:26:03.507217+0000","last_peered":"2024-04-25T01:26:03.507217+0000","last_clean":"2024-04-25T01:26:03.507217+0000","last_became_active":"2024-04-25T01:25:53.365715+0000","last_became_peered":"2024-04-25T01:25:53.365715+0000","last_unstale":"2024-04-25T01:26:03.507217+0000","last_undegraded":"2024-04-25T01:26:03.507217+0000","last_fullsized":"2024-04-25T01:26:03.507217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416219+0000","last_change":"2024-04-25T01:25:52.347413+0000","last_active":"2024-04-25T01:26:03.416219+0000","last_peered":"2024-04-25T01:26:03.416219+0000","last_clean":"2024-04-25T01:26:03.416219+0000","last_became_active":"2024-04-25T01:25:52.347144+0000","last_became_peered":"2024-04-25T01:25:52.347144+0000","last_unstale":"2024-04-25T01:26:03.416219+0000","last_undegraded":"2024-04-25T01:26:03.416219+0000","last_fullsized":"2024-04-25T01:26:03.416219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716710+0000","last_change":"2024-04-25T01:26:01.409917+0000","last_active":"2024-04-25T01:26:03.716710+0000","last_peered":"2024-04-25T01:26:03.716710+0000","last_clean":"2024-04-25T01:26:03.716710+0000","last_became_active":"2024-04-25T01:26:01.409654+0000","last_became_peered":"2024-04-25T01:26:01.409654+0000","last_unstale":"2024-04-25T01:26:03.716710+0000","last_undegraded":"2024-04-25T01:26:03.716710+0000","last_fullsized":"2024-04-25T01:26:03.716710+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:37.142255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717898+0000","last_change":"2024-04-25T01:26:02.425103+0000","last_active":"2024-04-25T01:26:03.717898+0000","last_peered":"2024-04-25T01:26:03.717898+0000","last_clean":"2024-04-25T01:26:03.717898+0000","last_became_active":"2024-04-25T01:26:02.424819+0000","last_became_peered":"2024-04-25T01:26:02.424819+0000","last_unstale":"2024-04-25T01:26:03.717898+0000","last_undegraded":"2024-04-25T01:26:03.717898+0000","last_fullsized":"2024-04-25T01:26:03.717898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716406+0000","last_change":"2024-04-25T01:25:53.364987+0000","last_active":"2024-04-25T01:26:03.716406+0000","last_peered":"2024-04-25T01:26:03.716406+0000","last_clean":"2024-04-25T01:26:03.716406+0000","last_became_active":"2024-04-25T01:25:53.364431+0000","last_became_peered":"2024-04-25T01:25:53.364431+0000","last_unstale":"2024-04-25T01:26:03.716406+0000","last_undegraded":"2024-04-25T01:26:03.716406+0000","last_fullsized":"2024-04-25T01:26:03.716406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716731+0000","last_change":"2024-04-25T01:25:52.351205+0000","last_active":"2024-04-25T01:26:03.716731+0000","last_peered":"2024-04-25T01:26:03.716731+0000","last_clean":"2024-04-25T01:26:03.716731+0000","last_became_active":"2024-04-25T01:25:52.350946+0000","last_became_peered":"2024-04-25T01:25:52.350946+0000","last_unstale":"2024-04-25T01:26:03.716731+0000","last_undegraded":"2024-04-25T01:26:03.716731+0000","last_fullsized":"2024-04-25T01:26:03.716731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416482+0000","last_change":"2024-04-25T01:26:01.415799+0000","last_active":"2024-04-25T01:26:03.416482+0000","last_peered":"2024-04-25T01:26:03.416482+0000","last_clean":"2024-04-25T01:26:03.416482+0000","last_became_active":"2024-04-25T01:26:01.415572+0000","last_became_peered":"2024-04-25T01:26:01.415572+0000","last_unstale":"2024-04-25T01:26:03.416482+0000","last_undegraded":"2024-04-25T01:26:03.416482+0000","last_fullsized":"2024-04-25T01:26:03.416482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:54.034132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717227+0000","last_change":"2024-04-25T01:26:02.422959+0000","last_active":"2024-04-25T01:26:03.717227+0000","last_peered":"2024-04-25T01:26:03.717227+0000","last_clean":"2024-04-25T01:26:03.717227+0000","last_became_active":"2024-04-25T01:26:02.422210+0000","last_became_peered":"2024-04-25T01:26:02.422210+0000","last_unstale":"2024-04-25T01:26:03.717227+0000","last_undegraded":"2024-04-25T01:26:03.717227+0000","last_fullsized":"2024-04-25T01:26:03.717227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507159+0000","last_change":"2024-04-25T01:25:53.368187+0000","last_active":"2024-04-25T01:26:03.507159+0000","last_peered":"2024-04-25T01:26:03.507159+0000","last_clean":"2024-04-25T01:26:03.507159+0000","last_became_active":"2024-04-25T01:25:53.367664+0000","last_became_peered":"2024-04-25T01:25:53.367664+0000","last_unstale":"2024-04-25T01:26:03.507159+0000","last_undegraded":"2024-04-25T01:26:03.507159+0000","last_fullsized":"2024-04-25T01:26:03.507159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717680+0000","last_change":"2024-04-25T01:25:52.357397+0000","last_active":"2024-04-25T01:26:03.717680+0000","last_peered":"2024-04-25T01:26:03.717680+0000","last_clean":"2024-04-25T01:26:03.717680+0000","last_became_active":"2024-04-25T01:25:52.356881+0000","last_became_peered":"2024-04-25T01:25:52.356881+0000","last_unstale":"2024-04-25T01:26:03.717680+0000","last_undegraded":"2024-04-25T01:26:03.717680+0000","last_fullsized":"2024-04-25T01:26:03.717680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"23'6","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416132+0000","last_change":"2024-04-25T01:26:01.420103+0000","last_active":"2024-04-25T01:26:03.416132+0000","last_peered":"2024-04-25T01:26:03.416132+0000","last_clean":"2024-04-25T01:26:03.416132+0000","last_became_active":"2024-04-25T01:26:01.419526+0000","last_became_peered":"2024-04-25T01:26:01.419526+0000","last_unstale":"2024-04-25T01:26:03.416132+0000","last_undegraded":"2024-04-25T01:26:03.416132+0000","last_fullsized":"2024-04-25T01:26:03.416132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:27.042376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416191+0000","last_change":"2024-04-25T01:26:02.421488+0000","last_active":"2024-04-25T01:26:03.416191+0000","last_peered":"2024-04-25T01:26:03.416191+0000","last_clean":"2024-04-25T01:26:03.416191+0000","last_became_active":"2024-04-25T01:26:02.419308+0000","last_became_peered":"2024-04-25T01:26:02.419308+0000","last_unstale":"2024-04-25T01:26:03.416191+0000","last_undegraded":"2024-04-25T01:26:03.416191+0000","last_fullsized":"2024-04-25T01:26:03.416191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716019+0000","last_change":"2024-04-25T01:25:53.370720+0000","last_active":"2024-04-25T01:26:03.716019+0000","last_peered":"2024-04-25T01:26:03.716019+0000","last_clean":"2024-04-25T01:26:03.716019+0000","last_became_active":"2024-04-25T01:25:53.370142+0000","last_became_peered":"2024-04-25T01:25:53.370142+0000","last_unstale":"2024-04-25T01:26:03.716019+0000","last_undegraded":"2024-04-25T01:26:03.716019+0000","last_fullsized":"2024-04-25T01:26:03.716019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:52.833379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716951+0000","last_change":"2024-04-25T01:25:52.349092+0000","last_active":"2024-04-25T01:26:03.716951+0000","last_peered":"2024-04-25T01:26:03.716951+0000","last_clean":"2024-04-25T01:26:03.716951+0000","last_became_active":"2024-04-25T01:25:52.348846+0000","last_became_peered":"2024-04-25T01:25:52.348846+0000","last_unstale":"2024-04-25T01:26:03.716951+0000","last_undegraded":"2024-04-25T01:26:03.716951+0000","last_fullsized":"2024-04-25T01:26:03.716951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:21.982440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717840+0000","last_change":"2024-04-25T01:26:01.414517+0000","last_active":"2024-04-25T01:26:03.717840+0000","last_peered":"2024-04-25T01:26:03.717840+0000","last_clean":"2024-04-25T01:26:03.717840+0000","last_became_active":"2024-04-25T01:26:01.414243+0000","last_became_peered":"2024-04-25T01:26:01.414243+0000","last_unstale":"2024-04-25T01:26:03.717840+0000","last_undegraded":"2024-04-25T01:26:03.717840+0000","last_fullsized":"2024-04-25T01:26:03.717840+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:48.491461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415418+0000","last_change":"2024-04-25T01:26:02.415893+0000","last_active":"2024-04-25T01:26:03.415418+0000","last_peered":"2024-04-25T01:26:03.415418+0000","last_clean":"2024-04-25T01:26:03.415418+0000","last_became_active":"2024-04-25T01:26:02.415596+0000","last_became_peered":"2024-04-25T01:26:02.415596+0000","last_unstale":"2024-04-25T01:26:03.415418+0000","last_undegraded":"2024-04-25T01:26:03.415418+0000","last_fullsized":"2024-04-25T01:26:03.415418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507428+0000","last_change":"2024-04-25T01:25:53.373572+0000","last_active":"2024-04-25T01:26:03.507428+0000","last_peered":"2024-04-25T01:26:03.507428+0000","last_clean":"2024-04-25T01:26:03.507428+0000","last_became_active":"2024-04-25T01:25:53.373129+0000","last_became_peered":"2024-04-25T01:25:53.373129+0000","last_unstale":"2024-04-25T01:26:03.507428+0000","last_undegraded":"2024-04-25T01:26:03.507428+0000","last_fullsized":"2024-04-25T01:26:03.507428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":28,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717081+0000","last_change":"2024-04-25T01:25:52.350446+0000","last_active":"2024-04-25T01:26:03.717081+0000","last_peered":"2024-04-25T01:26:03.717081+0000","last_clean":"2024-04-25T01:26:03.717081+0000","last_became_active":"2024-04-25T01:25:52.349448+0000","last_became_peered":"2024-04-25T01:25:52.349448+0000","last_unstale":"2024-04-25T01:26:03.717081+0000","last_undegraded":"2024-04-25T01:26:03.717081+0000","last_fullsized":"2024-04-25T01:26:03.717081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716463+0000","last_change":"2024-04-25T01:26:01.418794+0000","last_active":"2024-04-25T01:26:03.716463+0000","last_peered":"2024-04-25T01:26:03.716463+0000","last_clean":"2024-04-25T01:26:03.716463+0000","last_became_active":"2024-04-25T01:26:01.418541+0000","last_became_peered":"2024-04-25T01:26:01.418541+0000","last_unstale":"2024-04-25T01:26:03.716463+0000","last_undegraded":"2024-04-25T01:26:03.716463+0000","last_fullsized":"2024-04-25T01:26:03.716463+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717164+0000","last_change":"2024-04-25T01:26:02.425391+0000","last_active":"2024-04-25T01:26:03.717164+0000","last_peered":"2024-04-25T01:26:03.717164+0000","last_clean":"2024-04-25T01:26:03.717164+0000","last_became_active":"2024-04-25T01:26:02.425106+0000","last_became_peered":"2024-04-25T01:26:02.425106+0000","last_unstale":"2024-04-25T01:26:03.717164+0000","last_undegraded":"2024-04-25T01:26:03.717164+0000","last_fullsized":"2024-04-25T01:26:03.717164+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416706+0000","last_change":"2024-04-25T01:25:53.366416+0000","last_active":"2024-04-25T01:26:03.416706+0000","last_peered":"2024-04-25T01:26:03.416706+0000","last_clean":"2024-04-25T01:26:03.416706+0000","last_became_active":"2024-04-25T01:25:53.366157+0000","last_became_peered":"2024-04-25T01:25:53.366157+0000","last_unstale":"2024-04-25T01:26:03.416706+0000","last_undegraded":"2024-04-25T01:26:03.416706+0000","last_fullsized":"2024-04-25T01:26:03.416706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:26.277566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717854+0000","last_change":"2024-04-25T01:25:52.357610+0000","last_active":"2024-04-25T01:26:03.717854+0000","last_peered":"2024-04-25T01:26:03.717854+0000","last_clean":"2024-04-25T01:26:03.717854+0000","last_became_active":"2024-04-25T01:25:52.357355+0000","last_became_peered":"2024-04-25T01:25:52.357355+0000","last_unstale":"2024-04-25T01:26:03.717854+0000","last_undegraded":"2024-04-25T01:26:03.717854+0000","last_fullsized":"2024-04-25T01:26:03.717854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416422+0000","last_change":"2024-04-25T01:26:01.412666+0000","last_active":"2024-04-25T01:26:03.416422+0000","last_peered":"2024-04-25T01:26:03.416422+0000","last_clean":"2024-04-25T01:26:03.416422+0000","last_became_active":"2024-04-25T01:26:01.412132+0000","last_became_peered":"2024-04-25T01:26:01.412132+0000","last_unstale":"2024-04-25T01:26:03.416422+0000","last_undegraded":"2024-04-25T01:26:03.416422+0000","last_fullsized":"2024-04-25T01:26:03.416422+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716704+0000","last_change":"2024-04-25T01:26:02.413106+0000","last_active":"2024-04-25T01:26:03.716704+0000","last_peered":"2024-04-25T01:26:03.716704+0000","last_clean":"2024-04-25T01:26:03.716704+0000","last_became_active":"2024-04-25T01:26:02.412759+0000","last_became_peered":"2024-04-25T01:26:02.412759+0000","last_unstale":"2024-04-25T01:26:03.716704+0000","last_undegraded":"2024-04-25T01:26:03.716704+0000","last_fullsized":"2024-04-25T01:26:03.716704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:28.049190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716160+0000","last_change":"2024-04-25T01:25:53.374311+0000","last_active":"2024-04-25T01:26:03.716160+0000","last_peered":"2024-04-25T01:26:03.716160+0000","last_clean":"2024-04-25T01:26:03.716160+0000","last_became_active":"2024-04-25T01:25:53.373890+0000","last_became_peered":"2024-04-25T01:25:53.373890+0000","last_unstale":"2024-04-25T01:26:03.716160+0000","last_undegraded":"2024-04-25T01:26:03.716160+0000","last_fullsized":"2024-04-25T01:26:03.716160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717907+0000","last_change":"2024-04-25T01:25:52.356231+0000","last_active":"2024-04-25T01:26:03.717907+0000","last_peered":"2024-04-25T01:26:03.717907+0000","last_clean":"2024-04-25T01:26:03.717907+0000","last_became_active":"2024-04-25T01:25:52.355913+0000","last_became_peered":"2024-04-25T01:25:52.355913+0000","last_unstale":"2024-04-25T01:26:03.717907+0000","last_undegraded":"2024-04-25T01:26:03.717907+0000","last_fullsized":"2024-04-25T01:26:03.717907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415065+0000","last_change":"2024-04-25T01:26:01.419513+0000","last_active":"2024-04-25T01:26:03.415065+0000","last_peered":"2024-04-25T01:26:03.415065+0000","last_clean":"2024-04-25T01:26:03.415065+0000","last_became_active":"2024-04-25T01:26:01.419015+0000","last_became_peered":"2024-04-25T01:26:01.419015+0000","last_unstale":"2024-04-25T01:26:03.415065+0000","last_undegraded":"2024-04-25T01:26:03.415065+0000","last_fullsized":"2024-04-25T01:26:03.415065+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:10.486367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507391+0000","last_change":"2024-04-25T01:26:02.425241+0000","last_active":"2024-04-25T01:26:03.507391+0000","last_peered":"2024-04-25T01:26:03.507391+0000","last_clean":"2024-04-25T01:26:03.507391+0000","last_became_active":"2024-04-25T01:26:02.424960+0000","last_became_peered":"2024-04-25T01:26:02.424960+0000","last_unstale":"2024-04-25T01:26:03.507391+0000","last_undegraded":"2024-04-25T01:26:03.507391+0000","last_fullsized":"2024-04-25T01:26:03.507391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717765+0000","last_change":"2024-04-25T01:25:53.371445+0000","last_active":"2024-04-25T01:26:03.717765+0000","last_peered":"2024-04-25T01:26:03.717765+0000","last_clean":"2024-04-25T01:26:03.717765+0000","last_became_active":"2024-04-25T01:25:53.371086+0000","last_became_peered":"2024-04-25T01:25:53.371086+0000","last_unstale":"2024-04-25T01:26:03.717765+0000","last_undegraded":"2024-04-25T01:26:03.717765+0000","last_fullsized":"2024-04-25T01:26:03.717765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.718056+0000","last_change":"2024-04-25T01:25:52.357459+0000","last_active":"2024-04-25T01:26:03.718056+0000","last_peered":"2024-04-25T01:26:03.718056+0000","last_clean":"2024-04-25T01:26:03.718056+0000","last_became_active":"2024-04-25T01:25:52.357182+0000","last_became_peered":"2024-04-25T01:25:52.357182+0000","last_unstale":"2024-04-25T01:26:03.718056+0000","last_undegraded":"2024-04-25T01:26:03.718056+0000","last_fullsized":"2024-04-25T01:26:03.718056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415347+0000","last_change":"2024-04-25T01:26:01.415321+0000","last_active":"2024-04-25T01:26:03.415347+0000","last_peered":"2024-04-25T01:26:03.415347+0000","last_clean":"2024-04-25T01:26:03.415347+0000","last_became_active":"2024-04-25T01:26:01.415020+0000","last_became_peered":"2024-04-25T01:26:01.415020+0000","last_unstale":"2024-04-25T01:26:03.415347+0000","last_undegraded":"2024-04-25T01:26:03.415347+0000","last_fullsized":"2024-04-25T01:26:03.415347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:20.090411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717822+0000","last_change":"2024-04-25T01:26:02.416917+0000","last_active":"2024-04-25T01:26:03.717822+0000","last_peered":"2024-04-25T01:26:03.717822+0000","last_clean":"2024-04-25T01:26:03.717822+0000","last_became_active":"2024-04-25T01:26:02.416623+0000","last_became_peered":"2024-04-25T01:26:02.416623+0000","last_unstale":"2024-04-25T01:26:03.717822+0000","last_undegraded":"2024-04-25T01:26:03.717822+0000","last_fullsized":"2024-04-25T01:26:03.717822+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:55.040946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716446+0000","last_change":"2024-04-25T01:26:01.415472+0000","last_active":"2024-04-25T01:26:03.716446+0000","last_peered":"2024-04-25T01:26:03.716446+0000","last_clean":"2024-04-25T01:26:03.716446+0000","last_became_active":"2024-04-25T01:26:01.415184+0000","last_became_peered":"2024-04-25T01:26:01.415184+0000","last_unstale":"2024-04-25T01:26:03.716446+0000","last_undegraded":"2024-04-25T01:26:03.716446+0000","last_fullsized":"2024-04-25T01:26:03.716446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716757+0000","last_change":"2024-04-25T01:25:53.363043+0000","last_active":"2024-04-25T01:26:03.716757+0000","last_peered":"2024-04-25T01:26:03.716757+0000","last_clean":"2024-04-25T01:26:03.716757+0000","last_became_active":"2024-04-25T01:25:53.362601+0000","last_became_peered":"2024-04-25T01:25:53.362601+0000","last_unstale":"2024-04-25T01:26:03.716757+0000","last_undegraded":"2024-04-25T01:26:03.716757+0000","last_fullsized":"2024-04-25T01:26:03.716757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716830+0000","last_change":"2024-04-25T01:25:52.358081+0000","last_active":"2024-04-25T01:26:03.716830+0000","last_peered":"2024-04-25T01:26:03.716830+0000","last_clean":"2024-04-25T01:26:03.716830+0000","last_became_active":"2024-04-25T01:25:52.357365+0000","last_became_peered":"2024-04-25T01:25:52.357365+0000","last_unstale":"2024-04-25T01:26:03.716830+0000","last_undegraded":"2024-04-25T01:26:03.716830+0000","last_fullsized":"2024-04-25T01:26:03.716830+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716855+0000","last_change":"2024-04-25T01:26:02.418872+0000","last_active":"2024-04-25T01:26:03.716855+0000","last_peered":"2024-04-25T01:26:03.716855+0000","last_clean":"2024-04-25T01:26:03.716855+0000","last_became_active":"2024-04-25T01:26:02.418614+0000","last_became_peered":"2024-04-25T01:26:02.418614+0000","last_unstale":"2024-04-25T01:26:03.716855+0000","last_undegraded":"2024-04-25T01:26:03.716855+0000","last_fullsized":"2024-04-25T01:26:03.716855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:49.498275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717818+0000","last_change":"2024-04-25T01:26:01.410258+0000","last_active":"2024-04-25T01:26:03.717818+0000","last_peered":"2024-04-25T01:26:03.717818+0000","last_clean":"2024-04-25T01:26:03.717818+0000","last_became_active":"2024-04-25T01:26:01.410004+0000","last_became_peered":"2024-04-25T01:26:01.410004+0000","last_unstale":"2024-04-25T01:26:03.717818+0000","last_undegraded":"2024-04-25T01:26:03.717818+0000","last_fullsized":"2024-04-25T01:26:03.717818+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:48.865940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415284+0000","last_change":"2024-04-25T01:25:53.363794+0000","last_active":"2024-04-25T01:26:03.415284+0000","last_peered":"2024-04-25T01:26:03.415284+0000","last_clean":"2024-04-25T01:26:03.415284+0000","last_became_active":"2024-04-25T01:25:53.363509+0000","last_became_peered":"2024-04-25T01:25:53.363509+0000","last_unstale":"2024-04-25T01:26:03.415284+0000","last_undegraded":"2024-04-25T01:26:03.415284+0000","last_fullsized":"2024-04-25T01:26:03.415284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415685+0000","last_change":"2024-04-25T01:25:52.347464+0000","last_active":"2024-04-25T01:26:03.415685+0000","last_peered":"2024-04-25T01:26:03.415685+0000","last_clean":"2024-04-25T01:26:03.415685+0000","last_became_active":"2024-04-25T01:25:52.347138+0000","last_became_peered":"2024-04-25T01:25:52.347138+0000","last_unstale":"2024-04-25T01:26:03.415685+0000","last_undegraded":"2024-04-25T01:26:03.415685+0000","last_fullsized":"2024-04-25T01:26:03.415685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717780+0000","last_change":"2024-04-25T01:26:02.425948+0000","last_active":"2024-04-25T01:26:03.717780+0000","last_peered":"2024-04-25T01:26:03.717780+0000","last_clean":"2024-04-25T01:26:03.717780+0000","last_became_active":"2024-04-25T01:26:02.424947+0000","last_became_peered":"2024-04-25T01:26:02.424947+0000","last_unstale":"2024-04-25T01:26:03.717780+0000","last_undegraded":"2024-04-25T01:26:03.717780+0000","last_fullsized":"2024-04-25T01:26:03.717780+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716700+0000","last_change":"2024-04-25T01:25:52.349989+0000","last_active":"2024-04-25T01:26:03.716700+0000","last_peered":"2024-04-25T01:26:03.716700+0000","last_clean":"2024-04-25T01:26:03.716700+0000","last_became_active":"2024-04-25T01:25:52.349672+0000","last_became_peered":"2024-04-25T01:25:52.349672+0000","last_unstale":"2024-04-25T01:26:03.716700+0000","last_undegraded":"2024-04-25T01:26:03.716700+0000","last_fullsized":"2024-04-25T01:26:03.716700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716805+0000","last_change":"2024-04-25T01:25:53.356610+0000","last_active":"2024-04-25T01:26:03.716805+0000","last_peered":"2024-04-25T01:26:03.716805+0000","last_clean":"2024-04-25T01:26:03.716805+0000","last_became_active":"2024-04-25T01:25:53.356015+0000","last_became_peered":"2024-04-25T01:25:53.356015+0000","last_unstale":"2024-04-25T01:26:03.716805+0000","last_undegraded":"2024-04-25T01:26:03.716805+0000","last_fullsized":"2024-04-25T01:26:03.716805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.417192+0000","last_change":"2024-04-25T01:26:01.420261+0000","last_active":"2024-04-25T01:26:03.417192+0000","last_peered":"2024-04-25T01:26:03.417192+0000","last_clean":"2024-04-25T01:26:03.417192+0000","last_became_active":"2024-04-25T01:26:01.420003+0000","last_became_peered":"2024-04-25T01:26:01.420003+0000","last_unstale":"2024-04-25T01:26:03.417192+0000","last_undegraded":"2024-04-25T01:26:03.417192+0000","last_fullsized":"2024-04-25T01:26:03.417192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:28.695853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416039+0000","last_change":"2024-04-25T01:26:02.426327+0000","last_active":"2024-04-25T01:26:03.416039+0000","last_peered":"2024-04-25T01:26:03.416039+0000","last_clean":"2024-04-25T01:26:03.416039+0000","last_became_active":"2024-04-25T01:26:02.426054+0000","last_became_peered":"2024-04-25T01:26:02.426054+0000","last_unstale":"2024-04-25T01:26:03.416039+0000","last_undegraded":"2024-04-25T01:26:03.416039+0000","last_fullsized":"2024-04-25T01:26:03.416039+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:17.809738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415686+0000","last_change":"2024-04-25T01:25:52.353955+0000","last_active":"2024-04-25T01:26:03.415686+0000","last_peered":"2024-04-25T01:26:03.415686+0000","last_clean":"2024-04-25T01:26:03.415686+0000","last_became_active":"2024-04-25T01:25:52.353659+0000","last_became_peered":"2024-04-25T01:25:52.353659+0000","last_unstale":"2024-04-25T01:26:03.415686+0000","last_undegraded":"2024-04-25T01:26:03.415686+0000","last_fullsized":"2024-04-25T01:26:03.415686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416142+0000","last_change":"2024-04-25T01:25:53.366586+0000","last_active":"2024-04-25T01:26:03.416142+0000","last_peered":"2024-04-25T01:26:03.416142+0000","last_clean":"2024-04-25T01:26:03.416142+0000","last_became_active":"2024-04-25T01:25:53.366302+0000","last_became_peered":"2024-04-25T01:25:53.366302+0000","last_unstale":"2024-04-25T01:26:03.416142+0000","last_undegraded":"2024-04-25T01:26:03.416142+0000","last_fullsized":"2024-04-25T01:26:03.416142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416415+0000","last_change":"2024-04-25T01:26:01.420960+0000","last_active":"2024-04-25T01:26:03.416415+0000","last_peered":"2024-04-25T01:26:03.416415+0000","last_clean":"2024-04-25T01:26:03.416415+0000","last_became_active":"2024-04-25T01:26:01.420713+0000","last_became_peered":"2024-04-25T01:26:01.420713+0000","last_unstale":"2024-04-25T01:26:03.416415+0000","last_undegraded":"2024-04-25T01:26:03.416415+0000","last_fullsized":"2024-04-25T01:26:03.416415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:29.489275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416652+0000","last_change":"2024-04-25T01:26:02.419282+0000","last_active":"2024-04-25T01:26:03.416652+0000","last_peered":"2024-04-25T01:26:03.416652+0000","last_clean":"2024-04-25T01:26:03.416652+0000","last_became_active":"2024-04-25T01:26:02.418802+0000","last_became_peered":"2024-04-25T01:26:02.418802+0000","last_unstale":"2024-04-25T01:26:03.416652+0000","last_undegraded":"2024-04-25T01:26:03.416652+0000","last_fullsized":"2024-04-25T01:26:03.416652+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:46.807175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716343+0000","last_change":"2024-04-25T01:25:53.367143+0000","last_active":"2024-04-25T01:26:03.716343+0000","last_peered":"2024-04-25T01:26:03.716343+0000","last_clean":"2024-04-25T01:26:03.716343+0000","last_became_active":"2024-04-25T01:25:53.366847+0000","last_became_peered":"2024-04-25T01:25:53.366847+0000","last_unstale":"2024-04-25T01:26:03.716343+0000","last_undegraded":"2024-04-25T01:26:03.716343+0000","last_fullsized":"2024-04-25T01:26:03.716343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716783+0000","last_change":"2024-04-25T01:26:02.416438+0000","last_active":"2024-04-25T01:26:03.716783+0000","last_peered":"2024-04-25T01:26:03.716783+0000","last_clean":"2024-04-25T01:26:03.716783+0000","last_became_active":"2024-04-25T01:26:02.416170+0000","last_became_peered":"2024-04-25T01:26:02.416170+0000","last_unstale":"2024-04-25T01:26:03.716783+0000","last_undegraded":"2024-04-25T01:26:03.716783+0000","last_fullsized":"2024-04-25T01:26:03.716783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415637+0000","last_change":"2024-04-25T01:25:53.371345+0000","last_active":"2024-04-25T01:26:03.415637+0000","last_peered":"2024-04-25T01:26:03.415637+0000","last_clean":"2024-04-25T01:26:03.415637+0000","last_became_active":"2024-04-25T01:25:53.371080+0000","last_became_peered":"2024-04-25T01:25:53.371080+0000","last_unstale":"2024-04-25T01:26:03.415637+0000","last_undegraded":"2024-04-25T01:26:03.415637+0000","last_fullsized":"2024-04-25T01:26:03.415637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:09.855763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717353+0000","last_change":"2024-04-25T01:26:02.419688+0000","last_active":"2024-04-25T01:26:03.717353+0000","last_peered":"2024-04-25T01:26:03.717353+0000","last_clean":"2024-04-25T01:26:03.717353+0000","last_became_active":"2024-04-25T01:26:02.419417+0000","last_became_peered":"2024-04-25T01:26:02.419417+0000","last_unstale":"2024-04-25T01:26:03.717353+0000","last_undegraded":"2024-04-25T01:26:03.717353+0000","last_fullsized":"2024-04-25T01:26:03.717353+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717442+0000","last_change":"2024-04-25T01:25:53.369850+0000","last_active":"2024-04-25T01:26:03.717442+0000","last_peered":"2024-04-25T01:26:03.717442+0000","last_clean":"2024-04-25T01:26:03.717442+0000","last_became_active":"2024-04-25T01:25:53.369350+0000","last_became_peered":"2024-04-25T01:25:53.369350+0000","last_unstale":"2024-04-25T01:26:03.717442+0000","last_undegraded":"2024-04-25T01:26:03.717442+0000","last_fullsized":"2024-04-25T01:26:03.717442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717742+0000","last_change":"2024-04-25T01:26:02.425528+0000","last_active":"2024-04-25T01:26:03.717742+0000","last_peered":"2024-04-25T01:26:03.717742+0000","last_clean":"2024-04-25T01:26:03.717742+0000","last_became_active":"2024-04-25T01:26:02.425160+0000","last_became_peered":"2024-04-25T01:26:02.425160+0000","last_unstale":"2024-04-25T01:26:03.717742+0000","last_undegraded":"2024-04-25T01:26:03.717742+0000","last_fullsized":"2024-04-25T01:26:03.717742+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716710+0000","last_change":"2024-04-25T01:25:53.357142+0000","last_active":"2024-04-25T01:26:03.716710+0000","last_peered":"2024-04-25T01:26:03.716710+0000","last_clean":"2024-04-25T01:26:03.716710+0000","last_became_active":"2024-04-25T01:25:53.356854+0000","last_became_peered":"2024-04-25T01:25:53.356854+0000","last_unstale":"2024-04-25T01:26:03.716710+0000","last_undegraded":"2024-04-25T01:26:03.716710+0000","last_fullsized":"2024-04-25T01:26:03.716710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507419+0000","last_change":"2024-04-25T01:26:02.417492+0000","last_active":"2024-04-25T01:26:03.507419+0000","last_peered":"2024-04-25T01:26:03.507419+0000","last_clean":"2024-04-25T01:26:03.507419+0000","last_became_active":"2024-04-25T01:26:02.417234+0000","last_became_peered":"2024-04-25T01:26:02.417234+0000","last_unstale":"2024-04-25T01:26:03.507419+0000","last_undegraded":"2024-04-25T01:26:03.507419+0000","last_fullsized":"2024-04-25T01:26:03.507419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416287+0000","last_change":"2024-04-25T01:25:53.364575+0000","last_active":"2024-04-25T01:26:03.416287+0000","last_peered":"2024-04-25T01:26:03.416287+0000","last_clean":"2024-04-25T01:26:03.416287+0000","last_became_active":"2024-04-25T01:25:53.364311+0000","last_became_peered":"2024-04-25T01:25:53.364311+0000","last_unstale":"2024-04-25T01:26:03.416287+0000","last_undegraded":"2024-04-25T01:26:03.416287+0000","last_fullsized":"2024-04-25T01:26:03.416287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415942+0000","last_change":"2024-04-25T01:26:02.426813+0000","last_active":"2024-04-25T01:26:03.415942+0000","last_peered":"2024-04-25T01:26:03.415942+0000","last_clean":"2024-04-25T01:26:03.415942+0000","last_became_active":"2024-04-25T01:26:02.426552+0000","last_became_peered":"2024-04-25T01:26:02.426552+0000","last_unstale":"2024-04-25T01:26:03.415942+0000","last_undegraded":"2024-04-25T01:26:03.415942+0000","last_fullsized":"2024-04-25T01:26:03.415942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:03.687174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717821+0000","last_change":"2024-04-25T01:25:53.368413+0000","last_active":"2024-04-25T01:26:03.717821+0000","last_peered":"2024-04-25T01:26:03.717821+0000","last_clean":"2024-04-25T01:26:03.717821+0000","last_became_active":"2024-04-25T01:25:53.368176+0000","last_became_peered":"2024-04-25T01:25:53.368176+0000","last_unstale":"2024-04-25T01:26:03.717821+0000","last_undegraded":"2024-04-25T01:26:03.717821+0000","last_fullsized":"2024-04-25T01:26:03.717821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.506869+0000","last_change":"2024-04-25T01:26:02.425185+0000","last_active":"2024-04-25T01:26:03.506869+0000","last_peered":"2024-04-25T01:26:03.506869+0000","last_clean":"2024-04-25T01:26:03.506869+0000","last_became_active":"2024-04-25T01:26:02.424903+0000","last_became_peered":"2024-04-25T01:26:02.424903+0000","last_unstale":"2024-04-25T01:26:03.506869+0000","last_undegraded":"2024-04-25T01:26:03.506869+0000","last_fullsized":"2024-04-25T01:26:03.506869+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717561+0000","last_change":"2024-04-25T01:25:53.372638+0000","last_active":"2024-04-25T01:26:03.717561+0000","last_peered":"2024-04-25T01:26:03.717561+0000","last_clean":"2024-04-25T01:26:03.717561+0000","last_became_active":"2024-04-25T01:25:53.372001+0000","last_became_peered":"2024-04-25T01:25:53.372001+0000","last_unstale":"2024-04-25T01:26:03.717561+0000","last_undegraded":"2024-04-25T01:26:03.717561+0000","last_fullsized":"2024-04-25T01:26:03.717561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716617+0000","last_change":"2024-04-25T01:26:02.420867+0000","last_active":"2024-04-25T01:26:03.716617+0000","last_peered":"2024-04-25T01:26:03.716617+0000","last_clean":"2024-04-25T01:26:03.716617+0000","last_became_active":"2024-04-25T01:26:02.420621+0000","last_became_peered":"2024-04-25T01:26:02.420621+0000","last_unstale":"2024-04-25T01:26:03.716617+0000","last_undegraded":"2024-04-25T01:26:03.716617+0000","last_fullsized":"2024-04-25T01:26:03.716617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:11.493181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416565+0000","last_change":"2024-04-25T01:25:53.365509+0000","last_active":"2024-04-25T01:26:03.416565+0000","last_peered":"2024-04-25T01:26:03.416565+0000","last_clean":"2024-04-25T01:26:03.416565+0000","last_became_active":"2024-04-25T01:25:53.365245+0000","last_became_peered":"2024-04-25T01:25:53.365245+0000","last_unstale":"2024-04-25T01:26:03.416565+0000","last_undegraded":"2024-04-25T01:26:03.416565+0000","last_fullsized":"2024-04-25T01:26:03.416565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416549+0000","last_change":"2024-04-25T01:26:02.415672+0000","last_active":"2024-04-25T01:26:03.416549+0000","last_peered":"2024-04-25T01:26:03.416549+0000","last_clean":"2024-04-25T01:26:03.416549+0000","last_became_active":"2024-04-25T01:26:02.415389+0000","last_became_peered":"2024-04-25T01:26:02.415389+0000","last_unstale":"2024-04-25T01:26:03.416549+0000","last_undegraded":"2024-04-25T01:26:03.416549+0000","last_fullsized":"2024-04-25T01:26:03.416549+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:52.342053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716741+0000","last_change":"2024-04-25T01:25:53.365391+0000","last_active":"2024-04-25T01:26:03.716741+0000","last_peered":"2024-04-25T01:26:03.716741+0000","last_clean":"2024-04-25T01:26:03.716741+0000","last_became_active":"2024-04-25T01:25:53.364829+0000","last_became_peered":"2024-04-25T01:25:53.364829+0000","last_unstale":"2024-04-25T01:26:03.716741+0000","last_undegraded":"2024-04-25T01:26:03.716741+0000","last_fullsized":"2024-04-25T01:26:03.716741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507955+0000","last_change":"2024-04-25T01:26:02.418292+0000","last_active":"2024-04-25T01:26:03.507955+0000","last_peered":"2024-04-25T01:26:03.507955+0000","last_clean":"2024-04-25T01:26:03.507955+0000","last_became_active":"2024-04-25T01:26:02.418036+0000","last_became_peered":"2024-04-25T01:26:02.418036+0000","last_unstale":"2024-04-25T01:26:03.507955+0000","last_undegraded":"2024-04-25T01:26:03.507955+0000","last_fullsized":"2024-04-25T01:26:03.507955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415608+0000","last_change":"2024-04-25T01:25:53.355667+0000","last_active":"2024-04-25T01:26:03.415608+0000","last_peered":"2024-04-25T01:26:03.415608+0000","last_clean":"2024-04-25T01:26:03.415608+0000","last_became_active":"2024-04-25T01:25:53.355417+0000","last_became_peered":"2024-04-25T01:25:53.355417+0000","last_unstale":"2024-04-25T01:26:03.415608+0000","last_undegraded":"2024-04-25T01:26:03.415608+0000","last_fullsized":"2024-04-25T01:26:03.415608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415307+0000","last_change":"2024-04-25T01:26:02.414994+0000","last_active":"2024-04-25T01:26:03.415307+0000","last_peered":"2024-04-25T01:26:03.415307+0000","last_clean":"2024-04-25T01:26:03.415307+0000","last_became_active":"2024-04-25T01:26:02.414735+0000","last_became_peered":"2024-04-25T01:26:02.414735+0000","last_unstale":"2024-04-25T01:26:03.415307+0000","last_undegraded":"2024-04-25T01:26:03.415307+0000","last_fullsized":"2024-04-25T01:26:03.415307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:44.960251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717184+0000","last_change":"2024-04-25T01:25:53.371240+0000","last_active":"2024-04-25T01:26:03.717184+0000","last_peered":"2024-04-25T01:26:03.717184+0000","last_clean":"2024-04-25T01:26:03.717184+0000","last_became_active":"2024-04-25T01:25:53.370164+0000","last_became_peered":"2024-04-25T01:25:53.370164+0000","last_unstale":"2024-04-25T01:26:03.717184+0000","last_undegraded":"2024-04-25T01:26:03.717184+0000","last_fullsized":"2024-04-25T01:26:03.717184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416367+0000","last_change":"2024-04-25T01:26:02.418272+0000","last_active":"2024-04-25T01:26:03.416367+0000","last_peered":"2024-04-25T01:26:03.416367+0000","last_clean":"2024-04-25T01:26:03.416367+0000","last_became_active":"2024-04-25T01:26:02.417977+0000","last_became_peered":"2024-04-25T01:26:02.417977+0000","last_unstale":"2024-04-25T01:26:03.416367+0000","last_undegraded":"2024-04-25T01:26:03.416367+0000","last_fullsized":"2024-04-25T01:26:03.416367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716511+0000","last_change":"2024-04-25T01:25:53.376415+0000","last_active":"2024-04-25T01:26:03.716511+0000","last_peered":"2024-04-25T01:26:03.716511+0000","last_clean":"2024-04-25T01:26:03.716511+0000","last_became_active":"2024-04-25T01:25:53.376104+0000","last_became_peered":"2024-04-25T01:25:53.376104+0000","last_unstale":"2024-04-25T01:26:03.716511+0000","last_undegraded":"2024-04-25T01:26:03.716511+0000","last_fullsized":"2024-04-25T01:26:03.716511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416512+0000","last_change":"2024-04-25T01:26:02.426333+0000","last_active":"2024-04-25T01:26:03.416512+0000","last_peered":"2024-04-25T01:26:03.416512+0000","last_clean":"2024-04-25T01:26:03.416512+0000","last_became_active":"2024-04-25T01:26:02.426069+0000","last_became_peered":"2024-04-25T01:26:02.426069+0000","last_unstale":"2024-04-25T01:26:03.416512+0000","last_undegraded":"2024-04-25T01:26:03.416512+0000","last_fullsized":"2024-04-25T01:26:03.416512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716975+0000","last_change":"2024-04-25T01:25:53.370314+0000","last_active":"2024-04-25T01:26:03.716975+0000","last_peered":"2024-04-25T01:26:03.716975+0000","last_clean":"2024-04-25T01:26:03.716975+0000","last_became_active":"2024-04-25T01:25:53.370049+0000","last_became_peered":"2024-04-25T01:25:53.370049+0000","last_unstale":"2024-04-25T01:26:03.716975+0000","last_undegraded":"2024-04-25T01:26:03.716975+0000","last_fullsized":"2024-04-25T01:26:03.716975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415866+0000","last_change":"2024-04-25T01:26:02.415464+0000","last_active":"2024-04-25T01:26:03.415866+0000","last_peered":"2024-04-25T01:26:03.415866+0000","last_clean":"2024-04-25T01:26:03.415866+0000","last_became_active":"2024-04-25T01:26:02.415173+0000","last_became_peered":"2024-04-25T01:26:02.415173+0000","last_unstale":"2024-04-25T01:26:03.415866+0000","last_undegraded":"2024-04-25T01:26:03.415866+0000","last_fullsized":"2024-04-25T01:26:03.415866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.343169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507691+0000","last_change":"2024-04-25T01:25:53.369585+0000","last_active":"2024-04-25T01:26:03.507691+0000","last_peered":"2024-04-25T01:26:03.507691+0000","last_clean":"2024-04-25T01:26:03.507691+0000","last_became_active":"2024-04-25T01:25:53.368499+0000","last_became_peered":"2024-04-25T01:25:53.368499+0000","last_unstale":"2024-04-25T01:26:03.507691+0000","last_undegraded":"2024-04-25T01:26:03.507691+0000","last_fullsized":"2024-04-25T01:26:03.507691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716525+0000","last_change":"2024-04-25T01:26:02.421975+0000","last_active":"2024-04-25T01:26:03.716525+0000","last_peered":"2024-04-25T01:26:03.716525+0000","last_clean":"2024-04-25T01:26:03.716525+0000","last_became_active":"2024-04-25T01:26:02.421698+0000","last_became_peered":"2024-04-25T01:26:02.421698+0000","last_unstale":"2024-04-25T01:26:03.716525+0000","last_undegraded":"2024-04-25T01:26:03.716525+0000","last_fullsized":"2024-04-25T01:26:03.716525+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416746+0000","last_change":"2024-04-25T01:25:53.370666+0000","last_active":"2024-04-25T01:26:03.416746+0000","last_peered":"2024-04-25T01:26:03.416746+0000","last_clean":"2024-04-25T01:26:03.416746+0000","last_became_active":"2024-04-25T01:25:53.369733+0000","last_became_peered":"2024-04-25T01:25:53.369733+0000","last_unstale":"2024-04-25T01:26:03.416746+0000","last_undegraded":"2024-04-25T01:26:03.416746+0000","last_fullsized":"2024-04-25T01:26:03.416746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.715962+0000","last_change":"2024-04-25T01:26:02.418114+0000","last_active":"2024-04-25T01:26:03.715962+0000","last_peered":"2024-04-25T01:26:03.715962+0000","last_clean":"2024-04-25T01:26:03.715962+0000","last_became_active":"2024-04-25T01:26:02.417810+0000","last_became_peered":"2024-04-25T01:26:02.417810+0000","last_unstale":"2024-04-25T01:26:03.715962+0000","last_undegraded":"2024-04-25T01:26:03.715962+0000","last_fullsized":"2024-04-25T01:26:03.715962+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:30.496089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.718049+0000","last_change":"2024-04-25T01:25:53.372048+0000","last_active":"2024-04-25T01:26:03.718049+0000","last_peered":"2024-04-25T01:26:03.718049+0000","last_clean":"2024-04-25T01:26:03.718049+0000","last_became_active":"2024-04-25T01:25:53.371767+0000","last_became_peered":"2024-04-25T01:25:53.371767+0000","last_unstale":"2024-04-25T01:26:03.718049+0000","last_undegraded":"2024-04-25T01:26:03.718049+0000","last_fullsized":"2024-04-25T01:26:03.718049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:05.589744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716672+0000","last_change":"2024-04-25T01:26:02.426289+0000","last_active":"2024-04-25T01:26:03.716672+0000","last_peered":"2024-04-25T01:26:03.716672+0000","last_clean":"2024-04-25T01:26:03.716672+0000","last_became_active":"2024-04-25T01:26:02.426015+0000","last_became_peered":"2024-04-25T01:26:02.426015+0000","last_unstale":"2024-04-25T01:26:03.716672+0000","last_undegraded":"2024-04-25T01:26:03.716672+0000","last_fullsized":"2024-04-25T01:26:03.716672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716678+0000","last_change":"2024-04-25T01:25:53.374186+0000","last_active":"2024-04-25T01:26:03.716678+0000","last_peered":"2024-04-25T01:26:03.716678+0000","last_clean":"2024-04-25T01:26:03.716678+0000","last_became_active":"2024-04-25T01:25:53.373766+0000","last_became_peered":"2024-04-25T01:25:53.373766+0000","last_unstale":"2024-04-25T01:26:03.716678+0000","last_undegraded":"2024-04-25T01:26:03.716678+0000","last_fullsized":"2024-04-25T01:26:03.716678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416320+0000","last_change":"2024-04-25T01:26:02.416345+0000","last_active":"2024-04-25T01:26:03.416320+0000","last_peered":"2024-04-25T01:26:03.416320+0000","last_clean":"2024-04-25T01:26:03.416320+0000","last_became_active":"2024-04-25T01:26:02.416082+0000","last_became_peered":"2024-04-25T01:26:02.416082+0000","last_unstale":"2024-04-25T01:26:03.416320+0000","last_undegraded":"2024-04-25T01:26:03.416320+0000","last_fullsized":"2024-04-25T01:26:03.416320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:47.810943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716847+0000","last_change":"2024-04-25T01:25:53.371602+0000","last_active":"2024-04-25T01:26:03.716847+0000","last_peered":"2024-04-25T01:26:03.716847+0000","last_clean":"2024-04-25T01:26:03.716847+0000","last_became_active":"2024-04-25T01:25:53.371362+0000","last_became_peered":"2024-04-25T01:25:53.371362+0000","last_unstale":"2024-04-25T01:26:03.716847+0000","last_undegraded":"2024-04-25T01:26:03.716847+0000","last_fullsized":"2024-04-25T01:26:03.716847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717204+0000","last_change":"2024-04-25T01:26:02.425888+0000","last_active":"2024-04-25T01:26:03.717204+0000","last_peered":"2024-04-25T01:26:03.717204+0000","last_clean":"2024-04-25T01:26:03.717204+0000","last_became_active":"2024-04-25T01:26:02.425577+0000","last_became_peered":"2024-04-25T01:26:02.425577+0000","last_unstale":"2024-04-25T01:26:03.717204+0000","last_undegraded":"2024-04-25T01:26:03.717204+0000","last_fullsized":"2024-04-25T01:26:03.717204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.715963+0000","last_change":"2024-04-25T01:25:53.370996+0000","last_active":"2024-04-25T01:26:03.715963+0000","last_peered":"2024-04-25T01:26:03.715963+0000","last_clean":"2024-04-25T01:26:03.715963+0000","last_became_active":"2024-04-25T01:25:53.370479+0000","last_became_peered":"2024-04-25T01:25:53.370479+0000","last_unstale":"2024-04-25T01:26:03.715963+0000","last_undegraded":"2024-04-25T01:26:03.715963+0000","last_fullsized":"2024-04-25T01:26:03.715963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:57.636067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507529+0000","last_change":"2024-04-25T01:26:02.423420+0000","last_active":"2024-04-25T01:26:03.507529+0000","last_peered":"2024-04-25T01:26:03.507529+0000","last_clean":"2024-04-25T01:26:03.507529+0000","last_became_active":"2024-04-25T01:26:02.423146+0000","last_became_peered":"2024-04-25T01:26:02.423146+0000","last_unstale":"2024-04-25T01:26:03.507529+0000","last_undegraded":"2024-04-25T01:26:03.507529+0000","last_fullsized":"2024-04-25T01:26:03.507529+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415469+0000","last_change":"2024-04-25T01:25:53.363512+0000","last_active":"2024-04-25T01:26:03.415469+0000","last_peered":"2024-04-25T01:26:03.415469+0000","last_clean":"2024-04-25T01:26:03.415469+0000","last_became_active":"2024-04-25T01:25:53.363246+0000","last_became_peered":"2024-04-25T01:25:53.363246+0000","last_unstale":"2024-04-25T01:26:03.415469+0000","last_undegraded":"2024-04-25T01:26:03.415469+0000","last_fullsized":"2024-04-25T01:26:03.415469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415970+0000","last_change":"2024-04-25T01:26:02.413346+0000","last_active":"2024-04-25T01:26:03.415970+0000","last_peered":"2024-04-25T01:26:03.415970+0000","last_clean":"2024-04-25T01:26:03.415970+0000","last_became_active":"2024-04-25T01:26:02.413036+0000","last_became_peered":"2024-04-25T01:26:02.413036+0000","last_unstale":"2024-04-25T01:26:03.415970+0000","last_undegraded":"2024-04-25T01:26:03.415970+0000","last_fullsized":"2024-04-25T01:26:03.415970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416642+0000","last_change":"2024-04-25T01:25:53.364457+0000","last_active":"2024-04-25T01:26:03.416642+0000","last_peered":"2024-04-25T01:26:03.416642+0000","last_clean":"2024-04-25T01:26:03.416642+0000","last_became_active":"2024-04-25T01:25:53.364103+0000","last_became_peered":"2024-04-25T01:25:53.364103+0000","last_unstale":"2024-04-25T01:26:03.416642+0000","last_undegraded":"2024-04-25T01:26:03.416642+0000","last_fullsized":"2024-04-25T01:26:03.416642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:11.336752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415365+0000","last_change":"2024-04-25T01:26:02.418792+0000","last_active":"2024-04-25T01:26:03.415365+0000","last_peered":"2024-04-25T01:26:03.415365+0000","last_clean":"2024-04-25T01:26:03.415365+0000","last_became_active":"2024-04-25T01:26:02.416942+0000","last_became_peered":"2024-04-25T01:26:02.416942+0000","last_unstale":"2024-04-25T01:26:03.415365+0000","last_undegraded":"2024-04-25T01:26:03.415365+0000","last_fullsized":"2024-04-25T01:26:03.415365+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:59.492678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717288+0000","last_change":"2024-04-25T01:25:53.371647+0000","last_active":"2024-04-25T01:26:03.717288+0000","last_peered":"2024-04-25T01:26:03.717288+0000","last_clean":"2024-04-25T01:26:03.717288+0000","last_became_active":"2024-04-25T01:25:53.371353+0000","last_became_peered":"2024-04-25T01:25:53.371353+0000","last_unstale":"2024-04-25T01:26:03.717288+0000","last_undegraded":"2024-04-25T01:26:03.717288+0000","last_fullsized":"2024-04-25T01:26:03.717288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507578+0000","last_change":"2024-04-25T01:26:02.426335+0000","last_active":"2024-04-25T01:26:03.507578+0000","last_peered":"2024-04-25T01:26:03.507578+0000","last_clean":"2024-04-25T01:26:03.507578+0000","last_became_active":"2024-04-25T01:26:02.426024+0000","last_became_peered":"2024-04-25T01:26:02.426024+0000","last_unstale":"2024-04-25T01:26:03.507578+0000","last_undegraded":"2024-04-25T01:26:03.507578+0000","last_fullsized":"2024-04-25T01:26:03.507578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717460+0000","last_change":"2024-04-25T01:25:53.372842+0000","last_active":"2024-04-25T01:26:03.717460+0000","last_peered":"2024-04-25T01:26:03.717460+0000","last_clean":"2024-04-25T01:26:03.717460+0000","last_became_active":"2024-04-25T01:25:53.372580+0000","last_became_peered":"2024-04-25T01:25:53.372580+0000","last_unstale":"2024-04-25T01:26:03.717460+0000","last_undegraded":"2024-04-25T01:26:03.717460+0000","last_fullsized":"2024-04-25T01:26:03.717460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716748+0000","last_change":"2024-04-25T01:26:02.414632+0000","last_active":"2024-04-25T01:26:03.716748+0000","last_peered":"2024-04-25T01:26:03.716748+0000","last_clean":"2024-04-25T01:26:03.716748+0000","last_became_active":"2024-04-25T01:26:02.414388+0000","last_became_peered":"2024-04-25T01:26:02.414388+0000","last_unstale":"2024-04-25T01:26:03.716748+0000","last_undegraded":"2024-04-25T01:26:03.716748+0000","last_fullsized":"2024-04-25T01:26:03.716748+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.508086+0000","last_change":"2024-04-25T01:25:53.376081+0000","last_active":"2024-04-25T01:26:03.508086+0000","last_peered":"2024-04-25T01:26:03.508086+0000","last_clean":"2024-04-25T01:26:03.508086+0000","last_became_active":"2024-04-25T01:25:53.375814+0000","last_became_peered":"2024-04-25T01:25:53.375814+0000","last_unstale":"2024-04-25T01:26:03.508086+0000","last_undegraded":"2024-04-25T01:26:03.508086+0000","last_fullsized":"2024-04-25T01:26:03.508086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:29.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416509+0000","last_change":"2024-04-25T01:26:02.414366+0000","last_active":"2024-04-25T01:26:03.416509+0000","last_peered":"2024-04-25T01:26:03.416509+0000","last_clean":"2024-04-25T01:26:03.416509+0000","last_became_active":"2024-04-25T01:26:02.414061+0000","last_became_peered":"2024-04-25T01:26:02.414061+0000","last_unstale":"2024-04-25T01:26:03.416509+0000","last_undegraded":"2024-04-25T01:26:03.416509+0000","last_fullsized":"2024-04-25T01:26:03.416509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:19.644468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415110+0000","last_change":"2024-04-25T01:25:53.374061+0000","last_active":"2024-04-25T01:26:03.415110+0000","last_peered":"2024-04-25T01:26:03.415110+0000","last_clean":"2024-04-25T01:26:03.415110+0000","last_became_active":"2024-04-25T01:25:53.373848+0000","last_became_peered":"2024-04-25T01:25:53.373848+0000","last_unstale":"2024-04-25T01:26:03.415110+0000","last_undegraded":"2024-04-25T01:26:03.415110+0000","last_fullsized":"2024-04-25T01:26:03.415110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717762+0000","last_change":"2024-04-25T01:26:02.414620+0000","last_active":"2024-04-25T01:26:03.717762+0000","last_peered":"2024-04-25T01:26:03.717762+0000","last_clean":"2024-04-25T01:26:03.717762+0000","last_became_active":"2024-04-25T01:26:02.414326+0000","last_became_peered":"2024-04-25T01:26:02.414326+0000","last_unstale":"2024-04-25T01:26:03.717762+0000","last_undegraded":"2024-04-25T01:26:03.717762+0000","last_fullsized":"2024-04-25T01:26:03.717762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:30.496089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507095+0000","last_change":"2024-04-25T01:25:53.364698+0000","last_active":"2024-04-25T01:26:03.507095+0000","last_peered":"2024-04-25T01:26:03.507095+0000","last_clean":"2024-04-25T01:26:03.507095+0000","last_became_active":"2024-04-25T01:25:53.364435+0000","last_became_peered":"2024-04-25T01:25:53.364435+0000","last_unstale":"2024-04-25T01:26:03.507095+0000","last_undegraded":"2024-04-25T01:26:03.507095+0000","last_fullsized":"2024-04-25T01:26:03.507095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716509+0000","last_change":"2024-04-25T01:26:02.418331+0000","last_active":"2024-04-25T01:26:03.716509+0000","last_peered":"2024-04-25T01:26:03.716509+0000","last_clean":"2024-04-25T01:26:03.716509+0000","last_became_active":"2024-04-25T01:26:02.418057+0000","last_became_peered":"2024-04-25T01:26:02.418057+0000","last_unstale":"2024-04-25T01:26:03.716509+0000","last_undegraded":"2024-04-25T01:26:03.716509+0000","last_fullsized":"2024-04-25T01:26:03.716509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:17.809738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717064+0000","last_change":"2024-04-25T01:25:53.375324+0000","last_active":"2024-04-25T01:26:03.717064+0000","last_peered":"2024-04-25T01:26:03.717064+0000","last_clean":"2024-04-25T01:26:03.717064+0000","last_became_active":"2024-04-25T01:25:53.374392+0000","last_became_peered":"2024-04-25T01:25:53.374392+0000","last_unstale":"2024-04-25T01:26:03.717064+0000","last_undegraded":"2024-04-25T01:26:03.717064+0000","last_fullsized":"2024-04-25T01:26:03.717064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717818+0000","last_change":"2024-04-25T01:26:02.422720+0000","last_active":"2024-04-25T01:26:03.717818+0000","last_peered":"2024-04-25T01:26:03.717818+0000","last_clean":"2024-04-25T01:26:03.717818+0000","last_became_active":"2024-04-25T01:26:02.421848+0000","last_became_peered":"2024-04-25T01:26:02.421848+0000","last_unstale":"2024-04-25T01:26:03.717818+0000","last_undegraded":"2024-04-25T01:26:03.717818+0000","last_fullsized":"2024-04-25T01:26:03.717818+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507255+0000","last_change":"2024-04-25T01:25:53.367953+0000","last_active":"2024-04-25T01:26:03.507255+0000","last_peered":"2024-04-25T01:26:03.507255+0000","last_clean":"2024-04-25T01:26:03.507255+0000","last_became_active":"2024-04-25T01:25:53.367358+0000","last_became_peered":"2024-04-25T01:25:53.367358+0000","last_unstale":"2024-04-25T01:26:03.507255+0000","last_undegraded":"2024-04-25T01:26:03.507255+0000","last_fullsized":"2024-04-25T01:26:03.507255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415875+0000","last_change":"2024-04-25T01:26:02.422300+0000","last_active":"2024-04-25T01:26:03.415875+0000","last_peered":"2024-04-25T01:26:03.415875+0000","last_clean":"2024-04-25T01:26:03.415875+0000","last_became_active":"2024-04-25T01:26:02.421702+0000","last_became_peered":"2024-04-25T01:26:02.421702+0000","last_unstale":"2024-04-25T01:26:03.415875+0000","last_undegraded":"2024-04-25T01:26:03.415875+0000","last_fullsized":"2024-04-25T01:26:03.415875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717355+0000","last_change":"2024-04-25T01:25:53.371513+0000","last_active":"2024-04-25T01:26:03.717355+0000","last_peered":"2024-04-25T01:26:03.717355+0000","last_clean":"2024-04-25T01:26:03.717355+0000","last_became_active":"2024-04-25T01:25:53.370421+0000","last_became_peered":"2024-04-25T01:25:53.370421+0000","last_unstale":"2024-04-25T01:26:03.717355+0000","last_undegraded":"2024-04-25T01:26:03.717355+0000","last_fullsized":"2024-04-25T01:26:03.717355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415236+0000","last_change":"2024-04-25T01:26:02.411784+0000","last_active":"2024-04-25T01:26:03.415236+0000","last_peered":"2024-04-25T01:26:03.415236+0000","last_clean":"2024-04-25T01:26:03.415236+0000","last_became_active":"2024-04-25T01:26:02.411478+0000","last_became_peered":"2024-04-25T01:26:02.411478+0000","last_unstale":"2024-04-25T01:26:03.415236+0000","last_undegraded":"2024-04-25T01:26:03.415236+0000","last_fullsized":"2024-04-25T01:26:03.415236+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:20.194816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716214+0000","last_change":"2024-04-25T01:25:53.363829+0000","last_active":"2024-04-25T01:26:03.716214+0000","last_peered":"2024-04-25T01:26:03.716214+0000","last_clean":"2024-04-25T01:26:03.716214+0000","last_became_active":"2024-04-25T01:25:53.363525+0000","last_became_peered":"2024-04-25T01:25:53.363525+0000","last_unstale":"2024-04-25T01:26:03.716214+0000","last_undegraded":"2024-04-25T01:26:03.716214+0000","last_fullsized":"2024-04-25T01:26:03.716214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.506843+0000","last_change":"2024-04-25T01:26:02.421208+0000","last_active":"2024-04-25T01:26:03.506843+0000","last_peered":"2024-04-25T01:26:03.506843+0000","last_clean":"2024-04-25T01:26:03.506843+0000","last_became_active":"2024-04-25T01:26:02.418796+0000","last_became_peered":"2024-04-25T01:26:02.418796+0000","last_unstale":"2024-04-25T01:26:03.506843+0000","last_undegraded":"2024-04-25T01:26:03.506843+0000","last_fullsized":"2024-04-25T01:26:03.506843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716784+0000","last_change":"2024-04-25T01:25:53.365057+0000","last_active":"2024-04-25T01:26:03.716784+0000","last_peered":"2024-04-25T01:26:03.716784+0000","last_clean":"2024-04-25T01:26:03.716784+0000","last_became_active":"2024-04-25T01:25:53.364793+0000","last_became_peered":"2024-04-25T01:25:53.364793+0000","last_unstale":"2024-04-25T01:26:03.716784+0000","last_undegraded":"2024-04-25T01:26:03.716784+0000","last_fullsized":"2024-04-25T01:26:03.716784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507060+0000","last_change":"2024-04-25T01:26:02.417606+0000","last_active":"2024-04-25T01:26:03.507060+0000","last_peered":"2024-04-25T01:26:03.507060+0000","last_clean":"2024-04-25T01:26:03.507060+0000","last_became_active":"2024-04-25T01:26:02.417350+0000","last_became_peered":"2024-04-25T01:26:02.417350+0000","last_unstale":"2024-04-25T01:26:03.507060+0000","last_undegraded":"2024-04-25T01:26:03.507060+0000","last_fullsized":"2024-04-25T01:26:03.507060+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.508077+0000","last_change":"2024-04-25T01:25:53.367762+0000","last_active":"2024-04-25T01:26:03.508077+0000","last_peered":"2024-04-25T01:26:03.508077+0000","last_clean":"2024-04-25T01:26:03.508077+0000","last_became_active":"2024-04-25T01:25:53.367383+0000","last_became_peered":"2024-04-25T01:25:53.367383+0000","last_unstale":"2024-04-25T01:26:03.508077+0000","last_undegraded":"2024-04-25T01:26:03.508077+0000","last_fullsized":"2024-04-25T01:26:03.508077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:05.589744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":690301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":101489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":99491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":102507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":99331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":101055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":690267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:26:06.706 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T01:26:06.706 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T01:26:06.706 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T01:26:06.706 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T01:26:06.707 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T01:26:06.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.856+0000 7f5350942640 1 Processor -- start 2024-04-25T01:26:06.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.857+0000 7f5350942640 1 -- start start 2024-04-25T01:26:06.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.857+0000 7f5350942640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f534c153280 0x7f534c0c1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.857+0000 7f5350942640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f534c06f680 0x7f534c155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.857+0000 7f5350942640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f534c06fbb0 0x7f534c0743d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.857+0000 7f5350942640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f534c0a6d00 con 0x7f534c06f680 2024-04-25T01:26:06.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.857+0000 7f5350942640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f534c0a6f50 con 0x7f534c153280 2024-04-25T01:26:06.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.857+0000 7f5350942640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f534c159b90 con 0x7f534c06fbb0 2024-04-25T01:26:06.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.858+0000 7f5349d9b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f534c06f680 0x7f534c155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:06.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.858+0000 7f5349d9b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f534c06f680 0x7f534c155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58356/0 (socket says 172.21.15.112:58356) 2024-04-25T01:26:06.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.858+0000 7f5349d9b640 1 -- 172.21.15.112:0/891761962 learned_addr learned my addr 172.21.15.112:0/891761962 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:06.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.858+0000 7f534ad9d640 1 --2- 172.21.15.112:0/891761962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f534c06fbb0 0x7f534c0743d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:06.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.858+0000 7f534a59c640 1 --2- 172.21.15.112:0/891761962 >> v2:172.21.15.187:3300/0 conn(0x7f534c153280 0x7f534c0c1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:06.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.858+0000 7f5349d9b640 1 -- 172.21.15.112:0/891761962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f534c06fbb0 msgr2=0x7f534c0743d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:06.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.858+0000 7f5349d9b640 1 --2- 172.21.15.112:0/891761962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f534c06fbb0 0x7f534c0743d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.858+0000 7f5349d9b640 1 -- 172.21.15.112:0/891761962 >> v2:172.21.15.187:3300/0 conn(0x7f534c153280 msgr2=0x7f534c0c1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:06.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.858+0000 7f5349d9b640 1 --2- 172.21.15.112:0/891761962 >> v2:172.21.15.187:3300/0 conn(0x7f534c153280 0x7f534c0c1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.858+0000 7f5349d9b640 1 -- 172.21.15.112:0/891761962 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534c155f40 con 0x7f534c06f680 2024-04-25T01:26:06.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.858+0000 7f534ad9d640 1 --2- 172.21.15.112:0/891761962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f534c06fbb0 0x7f534c0743d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:06.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.858+0000 7f5349d9b640 1 --2- 172.21.15.112:0/891761962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f534c06f680 0x7f534c155a00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f534c0c1330 tx=0x7f533c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6937ae41e8d533 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:06.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.859+0000 7f534959a640 1 -- 172.21.15.112:0/891761962 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c037070 con 0x7f534c06f680 2024-04-25T01:26:06.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.859+0000 7f534959a640 1 -- 172.21.15.112:0/891761962 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f533c02ae00 con 0x7f534c06f680 2024-04-25T01:26:06.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.859+0000 7f534959a640 1 -- 172.21.15.112:0/891761962 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c032660 con 0x7f534c06f680 2024-04-25T01:26:06.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.859+0000 7f5350942640 1 -- 172.21.15.112:0/891761962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f534c06f680 msgr2=0x7f534c155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:06.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.859+0000 7f5350942640 1 --2- 172.21.15.112:0/891761962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f534c06f680 0x7f534c155a00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f534c0c1330 tx=0x7f533c02a0b0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.859+0000 7f5350942640 1 -- 172.21.15.112:0/891761962 shutdown_connections 2024-04-25T01:26:06.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.859+0000 7f5350942640 1 --2- 172.21.15.112:0/891761962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f534c06fbb0 0x7f534c0743d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.859+0000 7f5350942640 1 --2- 172.21.15.112:0/891761962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f534c06f680 0x7f534c155a00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.859+0000 7f5350942640 1 --2- 172.21.15.112:0/891761962 >> v2:172.21.15.187:3300/0 conn(0x7f534c153280 0x7f534c0c1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.859+0000 7f5350942640 1 -- 172.21.15.112:0/891761962 >> 172.21.15.112:0/891761962 conn(0x7f534c06b160 msgr2=0x7f534c06b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.859+0000 7f5350942640 1 -- 172.21.15.112:0/891761962 shutdown_connections 2024-04-25T01:26:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.859+0000 7f5350942640 1 -- 172.21.15.112:0/891761962 wait complete. 2024-04-25T01:26:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.860+0000 7f5350942640 1 Processor -- start 2024-04-25T01:26:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.861+0000 7f5350942640 1 -- start start 2024-04-25T01:26:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.861+0000 7f5350942640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f534c06fbb0 0x7f534c1ec350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.861+0000 7f5350942640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f534c153280 0x7f534c0a7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.861+0000 7f5350942640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f534c152770 0x7f534c151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.861+0000 7f5350942640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f534c074a90 con 0x7f534c153280 2024-04-25T01:26:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.861+0000 7f5350942640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f534c074910 con 0x7f534c152770 2024-04-25T01:26:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.861+0000 7f5350942640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f534c074c10 con 0x7f534c06fbb0 2024-04-25T01:26:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.861+0000 7f5349d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f534c153280 0x7f534c0a7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.861+0000 7f5349d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f534c153280 0x7f534c0a7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58358/0 (socket says 172.21.15.112:58358) 2024-04-25T01:26:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.861+0000 7f5349d9b640 1 -- 172.21.15.112:0/2281125335 learned_addr learned my addr 172.21.15.112:0/2281125335 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.861+0000 7f5349d9b640 1 -- 172.21.15.112:0/2281125335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f534c06fbb0 msgr2=0x7f534c1ec350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.862+0000 7f5349d9b640 1 --2- 172.21.15.112:0/2281125335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f534c06fbb0 0x7f534c1ec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.862+0000 7f5349d9b640 1 -- 172.21.15.112:0/2281125335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f534c152770 msgr2=0x7f534c151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.862+0000 7f5349d9b640 1 --2- 172.21.15.112:0/2281125335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f534c152770 0x7f534c151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.862+0000 7f5349d9b640 1 -- 172.21.15.112:0/2281125335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534c1eb890 con 0x7f534c153280 2024-04-25T01:26:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.862+0000 7f5349d9b640 1 --2- 172.21.15.112:0/2281125335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f534c153280 0x7f534c0a7970 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f533c000c00 tx=0x7f533c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.862+0000 7f533affd640 1 -- 172.21.15.112:0/2281125335 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c041030 con 0x7f534c153280 2024-04-25T01:26:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.862+0000 7f533affd640 1 -- 172.21.15.112:0/2281125335 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f533c03c670 con 0x7f534c153280 2024-04-25T01:26:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.862+0000 7f533affd640 1 -- 172.21.15.112:0/2281125335 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c037040 con 0x7f534c153280 2024-04-25T01:26:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.862+0000 7f5350942640 1 -- 172.21.15.112:0/2281125335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f534c1ef8b0 con 0x7f534c153280 2024-04-25T01:26:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.862+0000 7f5350942640 1 -- 172.21.15.112:0/2281125335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f534c1ebb50 con 0x7f534c153280 2024-04-25T01:26:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.863+0000 7f5350942640 1 -- 172.21.15.112:0/2281125335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f534c0c4000 con 0x7f534c153280 2024-04-25T01:26:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.863+0000 7f533affd640 1 -- 172.21.15.112:0/2281125335 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f533c044070 con 0x7f534c153280 2024-04-25T01:26:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.864+0000 7f533affd640 1 --2- 172.21.15.112:0/2281125335 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f532c06be60 0x7f532c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.864+0000 7f533affd640 1 -- 172.21.15.112:0/2281125335 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f533c0ab480 con 0x7f534c153280 2024-04-25T01:26:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.864+0000 7f534a59c640 1 --2- 172.21.15.112:0/2281125335 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f532c06be60 0x7f532c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.864+0000 7f534a59c640 1 --2- 172.21.15.112:0/2281125335 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f532c06be60 0x7f532c06e320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f534c150b70 tx=0x7f5340008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:06.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:06.868+0000 7f533affd640 1 -- 172.21.15.112:0/2281125335 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f533c04b070 con 0x7f534c153280 2024-04-25T01:26:07.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.088+0000 7f5350942640 1 -- 172.21.15.112:0/2281125335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f534c06c6d0 con 0x7f534c153280 2024-04-25T01:26:07.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.091+0000 7f533affd640 1 -- 172.21.15.112:0/2281125335 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172946 (secure 0 0 0) 0x7f533c044e40 con 0x7f534c153280 2024-04-25T01:26:07.073 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:07.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.093+0000 7f5350942640 1 -- 172.21.15.112:0/2281125335 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f532c06be60 msgr2=0x7f532c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.093+0000 7f5350942640 1 --2- 172.21.15.112:0/2281125335 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f532c06be60 0x7f532c06e320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f534c150b70 tx=0x7f5340008040 comp rx=0 tx=0).stop 2024-04-25T01:26:07.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.094+0000 7f5350942640 1 -- 172.21.15.112:0/2281125335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f534c153280 msgr2=0x7f534c0a7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.094+0000 7f5350942640 1 --2- 172.21.15.112:0/2281125335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f534c153280 0x7f534c0a7970 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f533c000c00 tx=0x7f533c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.094+0000 7f5350942640 1 -- 172.21.15.112:0/2281125335 shutdown_connections 2024-04-25T01:26:07.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.094+0000 7f5350942640 1 --2- 172.21.15.112:0/2281125335 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f532c06be60 0x7f532c06e320 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.094+0000 7f5350942640 1 --2- 172.21.15.112:0/2281125335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f534c152770 0x7f534c151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.094+0000 7f5350942640 1 --2- 172.21.15.112:0/2281125335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f534c153280 0x7f534c0a7970 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.094+0000 7f5350942640 1 --2- 172.21.15.112:0/2281125335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f534c06fbb0 0x7f534c1ec350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.094+0000 7f5350942640 1 -- 172.21.15.112:0/2281125335 >> 172.21.15.112:0/2281125335 conn(0x7f534c06b160 msgr2=0x7f534c1def80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.094+0000 7f5350942640 1 -- 172.21.15.112:0/2281125335 shutdown_connections 2024-04-25T01:26:07.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.094+0000 7f5350942640 1 -- 172.21.15.112:0/2281125335 wait complete. 2024-04-25T01:26:07.084 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3661599222},{"type":"v1","addr":"172.21.15.112:6833","nonce":3661599222}]},"active_addr":"172.21.15.112:6833/3661599222","active_change":"2024-04-25T01:25:32.638349+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4111,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":1789878409}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":2403242074}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":3247594227}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":3843229034}]}]} 2024-04-25T01:26:07.088 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T01:26:07.088 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T01:26:07.088 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:26:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.205+0000 7fb18b177640 1 Processor -- start 2024-04-25T01:26:07.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.205+0000 7fb18b177640 1 -- start start 2024-04-25T01:26:07.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.206+0000 7fb18b177640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb18414c3f0 0x7fb1840c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.206+0000 7fb18b177640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb1840ac3b0 0x7fb184155f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.206+0000 7fb18b177640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb1841524a0 0x7fb184156460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.206+0000 7fb18b177640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fb184155a90 con 0x7fb1840ac3b0 2024-04-25T01:26:07.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.206+0000 7fb18b177640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb1841569a0 con 0x7fb1841524a0 2024-04-25T01:26:07.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.206+0000 7fb18b177640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb1841529b0 con 0x7fb18414c3f0 2024-04-25T01:26:07.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.206+0000 7fb183fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb1840ac3b0 0x7fb184155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.206+0000 7fb183fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb1840ac3b0 0x7fb184155f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58370/0 (socket says 172.21.15.112:58370) 2024-04-25T01:26:07.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.206+0000 7fb183fff640 1 -- 172.21.15.112:0/4269228143 learned_addr learned my addr 172.21.15.112:0/4269228143 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:07.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.206+0000 7fb188f13640 1 --2- 172.21.15.112:0/4269228143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb18414c3f0 0x7fb1840c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.206+0000 7fb189714640 1 --2- 172.21.15.112:0/4269228143 >> v2:172.21.15.187:3300/0 conn(0x7fb1841524a0 0x7fb184156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.206+0000 7fb183fff640 1 -- 172.21.15.112:0/4269228143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb18414c3f0 msgr2=0x7fb1840c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.206+0000 7fb183fff640 1 --2- 172.21.15.112:0/4269228143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb18414c3f0 0x7fb1840c2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.206+0000 7fb183fff640 1 -- 172.21.15.112:0/4269228143 >> v2:172.21.15.187:3300/0 conn(0x7fb1841524a0 msgr2=0x7fb184156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.206+0000 7fb183fff640 1 --2- 172.21.15.112:0/4269228143 >> v2:172.21.15.187:3300/0 conn(0x7fb1841524a0 0x7fb184156460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.206+0000 7fb183fff640 1 -- 172.21.15.112:0/4269228143 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb184152c70 con 0x7fb1840ac3b0 2024-04-25T01:26:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.207+0000 7fb183fff640 1 --2- 172.21.15.112:0/4269228143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1840ac3b0 0x7fb184155f20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb184153510 tx=0x7fb17802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2193d523dacd6c80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.207+0000 7fb1837fe640 1 -- 172.21.15.112:0/4269228143 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb178037070 con 0x7fb1840ac3b0 2024-04-25T01:26:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.207+0000 7fb1837fe640 1 -- 172.21.15.112:0/4269228143 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb178032440 con 0x7fb1840ac3b0 2024-04-25T01:26:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.207+0000 7fb1837fe640 1 -- 172.21.15.112:0/4269228143 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb178032810 con 0x7fb1840ac3b0 2024-04-25T01:26:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.207+0000 7fb18b177640 1 -- 172.21.15.112:0/4269228143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1840ac3b0 msgr2=0x7fb184155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.207+0000 7fb18b177640 1 --2- 172.21.15.112:0/4269228143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1840ac3b0 0x7fb184155f20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb184153510 tx=0x7fb17802a140 comp rx=0 tx=0).stop 2024-04-25T01:26:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.207+0000 7fb18b177640 1 -- 172.21.15.112:0/4269228143 shutdown_connections 2024-04-25T01:26:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.207+0000 7fb18b177640 1 --2- 172.21.15.112:0/4269228143 >> v2:172.21.15.187:3300/0 conn(0x7fb1841524a0 0x7fb184156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.207+0000 7fb18b177640 1 --2- 172.21.15.112:0/4269228143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1840ac3b0 0x7fb184155f20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.207+0000 7fb18b177640 1 --2- 172.21.15.112:0/4269228143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb18414c3f0 0x7fb1840c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.207+0000 7fb18b177640 1 -- 172.21.15.112:0/4269228143 >> 172.21.15.112:0/4269228143 conn(0x7fb18406b160 msgr2=0x7fb18406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.207+0000 7fb18b177640 1 -- 172.21.15.112:0/4269228143 shutdown_connections 2024-04-25T01:26:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.207+0000 7fb18b177640 1 -- 172.21.15.112:0/4269228143 wait complete. 2024-04-25T01:26:07.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.208+0000 7fb18b177640 1 Processor -- start 2024-04-25T01:26:07.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.208+0000 7fb18b177640 1 -- start start 2024-04-25T01:26:07.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.209+0000 7fb18b177640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1840ac3b0 0x7fb1840c0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.209+0000 7fb18b177640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb18414c3f0 0x7fb1841ee840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.209+0000 7fb18b177640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1841524a0 0x7fb1841eed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.209+0000 7fb18b177640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb184070d60 con 0x7fb18414c3f0 2024-04-25T01:26:07.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.209+0000 7fb18b177640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb184153230 con 0x7fb1840ac3b0 2024-04-25T01:26:07.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.209+0000 7fb18b177640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb18414c7d0 con 0x7fb1841524a0 2024-04-25T01:26:07.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.209+0000 7fb183fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb18414c3f0 0x7fb1841ee840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.209+0000 7fb183fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb18414c3f0 0x7fb1841ee840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58374/0 (socket says 172.21.15.112:58374) 2024-04-25T01:26:07.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.209+0000 7fb183fff640 1 -- 172.21.15.112:0/2681782790 learned_addr learned my addr 172.21.15.112:0/2681782790 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:07.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.210+0000 7fb189714640 1 --2- 172.21.15.112:0/2681782790 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1841524a0 0x7fb1841eed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.210+0000 7fb183fff640 1 -- 172.21.15.112:0/2681782790 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1841524a0 msgr2=0x7fb1841eed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.210+0000 7fb188f13640 1 --2- 172.21.15.112:0/2681782790 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1840ac3b0 0x7fb1840c0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.210+0000 7fb183fff640 1 --2- 172.21.15.112:0/2681782790 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1841524a0 0x7fb1841eed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.210+0000 7fb183fff640 1 -- 172.21.15.112:0/2681782790 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1840ac3b0 msgr2=0x7fb1840c0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.210+0000 7fb183fff640 1 --2- 172.21.15.112:0/2681782790 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1840ac3b0 0x7fb1840c0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.210+0000 7fb183fff640 1 -- 172.21.15.112:0/2681782790 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1841ddb60 con 0x7fb18414c3f0 2024-04-25T01:26:07.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.210+0000 7fb183fff640 1 --2- 172.21.15.112:0/2681782790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb18414c3f0 0x7fb1841ee840 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb178000c00 tx=0x7fb1780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.210+0000 7fb1817fa640 1 -- 172.21.15.112:0/2681782790 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb178037040 con 0x7fb18414c3f0 2024-04-25T01:26:07.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.210+0000 7fb1817fa640 1 -- 172.21.15.112:0/2681782790 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb178042070 con 0x7fb18414c3f0 2024-04-25T01:26:07.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.210+0000 7fb1817fa640 1 -- 172.21.15.112:0/2681782790 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb178030bb0 con 0x7fb18414c3f0 2024-04-25T01:26:07.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.210+0000 7fb18b177640 1 -- 172.21.15.112:0/2681782790 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1841dde40 con 0x7fb18414c3f0 2024-04-25T01:26:07.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.210+0000 7fb18b177640 1 -- 172.21.15.112:0/2681782790 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1841de400 con 0x7fb18414c3f0 2024-04-25T01:26:07.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.211+0000 7fb18b177640 1 -- 172.21.15.112:0/2681782790 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1840682a0 con 0x7fb18414c3f0 2024-04-25T01:26:07.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.211+0000 7fb1817fa640 1 -- 172.21.15.112:0/2681782790 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb178047460 con 0x7fb18414c3f0 2024-04-25T01:26:07.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.212+0000 7fb1817fa640 1 --2- 172.21.15.112:0/2681782790 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb17006be30 0x7fb17006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.212+0000 7fb188f13640 1 --2- 172.21.15.112:0/2681782790 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb17006be30 0x7fb17006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.212+0000 7fb1817fa640 1 -- 172.21.15.112:0/2681782790 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fb1780abd60 con 0x7fb18414c3f0 2024-04-25T01:26:07.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.212+0000 7fb188f13640 1 --2- 172.21.15.112:0/2681782790 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb17006be30 0x7fb17006e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb174008ef0 tx=0x7fb174008440 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.216+0000 7fb1817fa640 1 -- 172.21.15.112:0/2681782790 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb178045360 con 0x7fb18414c3f0 2024-04-25T01:26:07.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.390+0000 7fb18b177640 1 -- 172.21.15.112:0/2681782790 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1840aaf90 con 0x7fb18414c3f0 2024-04-25T01:26:07.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.391+0000 7fb1817fa640 1 -- 172.21.15.112:0/2681782790 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20367 (secure 0 0 0) 0x7fb17807a470 con 0x7fb18414c3f0 2024-04-25T01:26:07.371 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:07.371 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:26:03.405271+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-25T01:25:59.478275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-25T01:26:00.719358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:26:07.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.393+0000 7fb18b177640 1 -- 172.21.15.112:0/2681782790 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb17006be30 msgr2=0x7fb17006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.393+0000 7fb18b177640 1 --2- 172.21.15.112:0/2681782790 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb17006be30 0x7fb17006e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb174008ef0 tx=0x7fb174008440 comp rx=0 tx=0).stop 2024-04-25T01:26:07.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.393+0000 7fb18b177640 1 -- 172.21.15.112:0/2681782790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb18414c3f0 msgr2=0x7fb1841ee840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.393+0000 7fb18b177640 1 --2- 172.21.15.112:0/2681782790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb18414c3f0 0x7fb1841ee840 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb178000c00 tx=0x7fb1780046c0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.393+0000 7fb18b177640 1 -- 172.21.15.112:0/2681782790 shutdown_connections 2024-04-25T01:26:07.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.393+0000 7fb18b177640 1 --2- 172.21.15.112:0/2681782790 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb17006be30 0x7fb17006e2f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.393+0000 7fb18b177640 1 --2- 172.21.15.112:0/2681782790 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1841524a0 0x7fb1841eed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.393+0000 7fb18b177640 1 --2- 172.21.15.112:0/2681782790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb18414c3f0 0x7fb1841ee840 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.393+0000 7fb18b177640 1 --2- 172.21.15.112:0/2681782790 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1840ac3b0 0x7fb1840c0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.393+0000 7fb18b177640 1 -- 172.21.15.112:0/2681782790 >> 172.21.15.112:0/2681782790 conn(0x7fb18406b160 msgr2=0x7fb1840c2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.393+0000 7fb18b177640 1 -- 172.21.15.112:0/2681782790 shutdown_connections 2024-04-25T01:26:07.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.393+0000 7fb18b177640 1 -- 172.21.15.112:0/2681782790 wait complete. 2024-04-25T01:26:07.384 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T01:26:07.384 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:26:07.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.501+0000 7f60dae9a640 1 Processor -- start 2024-04-25T01:26:07.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.501+0000 7f60dae9a640 1 -- start start 2024-04-25T01:26:07.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.502+0000 7f60dae9a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f60d40ac3b0 0x7f60d4155f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.502+0000 7f60dae9a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f60d41524a0 0x7f60d4156460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.502+0000 7f60dae9a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60d41529b0 con 0x7f60d414c3f0 2024-04-25T01:26:07.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.502+0000 7f60dae9a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f60d4155a90 con 0x7f60d41524a0 2024-04-25T01:26:07.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.502+0000 7f60dae9a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f60d41569a0 con 0x7f60d40ac3b0 2024-04-25T01:26:07.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.502+0000 7f60d9437640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f60d41524a0 0x7f60d4156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.502+0000 7f60d9437640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f60d41524a0 0x7f60d4156460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43164/0 (socket says 172.21.15.112:43164) 2024-04-25T01:26:07.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.502+0000 7f60d9437640 1 -- 172.21.15.112:0/593840833 learned_addr learned my addr 172.21.15.112:0/593840833 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:07.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.502+0000 7f60d9437640 1 -- 172.21.15.112:0/593840833 >> v2:172.21.15.187:3300/0 conn(0x7f60d40ac3b0 msgr2=0x7f60d4155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:07.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.502+0000 7f60d9437640 1 --2- 172.21.15.112:0/593840833 >> v2:172.21.15.187:3300/0 conn(0x7f60d40ac3b0 0x7f60d4155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.502+0000 7f60d9437640 1 -- 172.21.15.112:0/593840833 >> v1:172.21.15.187:6789/0 conn(0x7f60d414c3f0 legacy=0x7f60d40c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:07.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.502+0000 7f60d9437640 1 -- 172.21.15.112:0/593840833 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d4152c70 con 0x7f60d41524a0 2024-04-25T01:26:07.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.503+0000 7f60d9437640 1 --2- 172.21.15.112:0/593840833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d41524a0 0x7f60d4156460 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f60d4155c10 tx=0x7f60cc02c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4b6e6f075280d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.503+0000 7f60d37fe640 1 -- 172.21.15.112:0/593840833 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc039070 con 0x7f60d41524a0 2024-04-25T01:26:07.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.503+0000 7f60d37fe640 1 -- 172.21.15.112:0/593840833 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60cc00b2e0 con 0x7f60d41524a0 2024-04-25T01:26:07.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.503+0000 7f60d37fe640 1 -- 172.21.15.112:0/593840833 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc009820 con 0x7f60d41524a0 2024-04-25T01:26:07.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.503+0000 7f60dae9a640 1 -- 172.21.15.112:0/593840833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d41524a0 msgr2=0x7f60d4156460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.503+0000 7f60dae9a640 1 --2- 172.21.15.112:0/593840833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d41524a0 0x7f60d4156460 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f60d4155c10 tx=0x7f60cc02c860 comp rx=0 tx=0).stop 2024-04-25T01:26:07.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.503+0000 7f60dae9a640 1 -- 172.21.15.112:0/593840833 shutdown_connections 2024-04-25T01:26:07.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.503+0000 7f60dae9a640 1 --2- 172.21.15.112:0/593840833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d41524a0 0x7f60d4156460 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.503+0000 7f60dae9a640 1 --2- 172.21.15.112:0/593840833 >> v2:172.21.15.187:3300/0 conn(0x7f60d40ac3b0 0x7f60d4155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.503+0000 7f60dae9a640 1 -- 172.21.15.112:0/593840833 >> 172.21.15.112:0/593840833 conn(0x7f60d406b160 msgr2=0x7f60d406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.503+0000 7f60dae9a640 1 -- 172.21.15.112:0/593840833 shutdown_connections 2024-04-25T01:26:07.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.503+0000 7f60dae9a640 1 -- 172.21.15.112:0/593840833 wait complete. 2024-04-25T01:26:07.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.504+0000 7f60dae9a640 1 Processor -- start 2024-04-25T01:26:07.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.505+0000 7f60dae9a640 1 -- start start 2024-04-25T01:26:07.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.505+0000 7f60dae9a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60d40ac3b0 0x7f60d414b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.505+0000 7f60dae9a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60d414c3f0 0x7f60d4149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.505+0000 7f60dae9a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d41524a0 0x7f60d406d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.505+0000 7f60dae9a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60d4070d60 con 0x7f60d41524a0 2024-04-25T01:26:07.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.505+0000 7f60dae9a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f60d4153230 con 0x7f60d40ac3b0 2024-04-25T01:26:07.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.505+0000 7f60dae9a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f60d40c32a0 con 0x7f60d414c3f0 2024-04-25T01:26:07.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.505+0000 7f60d9437640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d41524a0 0x7f60d406d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.505+0000 7f60d9437640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d41524a0 0x7f60d406d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43168/0 (socket says 172.21.15.112:43168) 2024-04-25T01:26:07.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.505+0000 7f60d9437640 1 -- 172.21.15.112:0/276301118 learned_addr learned my addr 172.21.15.112:0/276301118 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:07.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.506+0000 7f60d3fff640 1 --2- 172.21.15.112:0/276301118 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60d414c3f0 0x7f60d4149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.506+0000 7f60d8c36640 1 --2- 172.21.15.112:0/276301118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60d40ac3b0 0x7f60d414b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.506+0000 7f60d9437640 1 -- 172.21.15.112:0/276301118 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60d414c3f0 msgr2=0x7f60d4149c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.506+0000 7f60d9437640 1 --2- 172.21.15.112:0/276301118 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60d414c3f0 0x7f60d4149c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.506+0000 7f60d9437640 1 -- 172.21.15.112:0/276301118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60d40ac3b0 msgr2=0x7f60d414b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.506+0000 7f60d9437640 1 --2- 172.21.15.112:0/276301118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60d40ac3b0 0x7f60d414b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.506+0000 7f60d9437640 1 -- 172.21.15.112:0/276301118 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d406f1e0 con 0x7f60d41524a0 2024-04-25T01:26:07.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.506+0000 7f60d9437640 1 --2- 172.21.15.112:0/276301118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d41524a0 0x7f60d406d240 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f60cc02ce50 tx=0x7f60cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.506+0000 7f60d17fa640 1 -- 172.21.15.112:0/276301118 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc044020 con 0x7f60d41524a0 2024-04-25T01:26:07.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.506+0000 7f60d17fa640 1 -- 172.21.15.112:0/276301118 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60cc034a60 con 0x7f60d41524a0 2024-04-25T01:26:07.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.506+0000 7f60d17fa640 1 -- 172.21.15.112:0/276301118 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc039040 con 0x7f60d41524a0 2024-04-25T01:26:07.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.506+0000 7f60dae9a640 1 -- 172.21.15.112:0/276301118 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d414f550 con 0x7f60d41524a0 2024-04-25T01:26:07.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.506+0000 7f60dae9a640 1 -- 172.21.15.112:0/276301118 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d41e6fd0 con 0x7f60d41524a0 2024-04-25T01:26:07.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.507+0000 7f60dae9a640 1 -- 172.21.15.112:0/276301118 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d40682a0 con 0x7f60d41524a0 2024-04-25T01:26:07.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.508+0000 7f60d17fa640 1 -- 172.21.15.112:0/276301118 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60cc047020 con 0x7f60d41524a0 2024-04-25T01:26:07.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.508+0000 7f60d17fa640 1 --2- 172.21.15.112:0/276301118 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f60b806be30 0x7f60b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.508+0000 7f60d17fa640 1 -- 172.21.15.112:0/276301118 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f60cc0add90 con 0x7f60d41524a0 2024-04-25T01:26:07.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.508+0000 7f60d8c36640 1 --2- 172.21.15.112:0/276301118 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f60b806be30 0x7f60b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.508+0000 7f60d8c36640 1 --2- 172.21.15.112:0/276301118 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f60b806be30 0x7f60b806e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f60c8007fd0 tx=0x7f60c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.512+0000 7f60d17fa640 1 -- 172.21.15.112:0/276301118 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f60cc04e030 con 0x7f60d41524a0 2024-04-25T01:26:07.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.688+0000 7f60dae9a640 1 -- 172.21.15.112:0/276301118 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60d40aaf90 con 0x7f60d41524a0 2024-04-25T01:26:07.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.689+0000 7f60d17fa640 1 -- 172.21.15.112:0/276301118 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20367 (secure 0 0 0) 0x7f60cc07c4d0 con 0x7f60d41524a0 2024-04-25T01:26:07.669 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:07.669 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:26:03.405271+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-25T01:25:59.478275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-25T01:26:00.719358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:26:07.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.691+0000 7f60dae9a640 1 -- 172.21.15.112:0/276301118 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f60b806be30 msgr2=0x7f60b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.691+0000 7f60dae9a640 1 --2- 172.21.15.112:0/276301118 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f60b806be30 0x7f60b806e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f60c8007fd0 tx=0x7f60c8009040 comp rx=0 tx=0).stop 2024-04-25T01:26:07.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.692+0000 7f60dae9a640 1 -- 172.21.15.112:0/276301118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d41524a0 msgr2=0x7f60d406d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.692+0000 7f60dae9a640 1 --2- 172.21.15.112:0/276301118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d41524a0 0x7f60d406d240 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f60cc02ce50 tx=0x7f60cc004a40 comp rx=0 tx=0).stop 2024-04-25T01:26:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.692+0000 7f60dae9a640 1 -- 172.21.15.112:0/276301118 shutdown_connections 2024-04-25T01:26:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.692+0000 7f60dae9a640 1 --2- 172.21.15.112:0/276301118 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f60b806be30 0x7f60b806e2f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.692+0000 7f60dae9a640 1 --2- 172.21.15.112:0/276301118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d41524a0 0x7f60d406d240 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.692+0000 7f60dae9a640 1 --2- 172.21.15.112:0/276301118 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60d414c3f0 0x7f60d4149c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.692+0000 7f60dae9a640 1 --2- 172.21.15.112:0/276301118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60d40ac3b0 0x7f60d414b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.692+0000 7f60dae9a640 1 -- 172.21.15.112:0/276301118 >> 172.21.15.112:0/276301118 conn(0x7f60d406b160 msgr2=0x7f60d4150b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.692+0000 7f60dae9a640 1 -- 172.21.15.112:0/276301118 shutdown_connections 2024-04-25T01:26:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.692+0000 7f60dae9a640 1 -- 172.21.15.112:0/276301118 wait complete. 2024-04-25T01:26:07.683 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T01:26:07.683 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T01:26:07.684 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T01:26:07.684 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T01:26:07.684 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T01:26:07.685 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T01:26:07.685 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T01:26:07.685 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T01:26:07.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.872+0000 7f0ad20ff640 1 Processor -- start 2024-04-25T01:26:07.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.872+0000 7faf9e651640 1 Processor -- start 2024-04-25T01:26:07.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.872+0000 7f3b13597640 1 Processor -- start 2024-04-25T01:26:07.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.872+0000 7f3b13597640 1 -- start start 2024-04-25T01:26:07.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.872+0000 7faf9e651640 1 -- start start 2024-04-25T01:26:07.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.872+0000 7fd5782bd640 1 Processor -- start 2024-04-25T01:26:07.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.872+0000 7fd5782bd640 1 -- start start 2024-04-25T01:26:07.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.872+0000 7f0ad20ff640 1 -- start start 2024-04-25T01:26:07.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.873+0000 7f3b13597640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3b0c0c16d0 0x7f3b0c0c8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.873+0000 7f3b13597640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b0c15a470 0x7f3b0c06ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.873+0000 7f3b13597640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b0c15a950 con 0x7f3b0c0c6580 2024-04-25T01:26:07.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.873+0000 7f3b13597640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3b0c15d0d0 con 0x7f3b0c0c16d0 2024-04-25T01:26:07.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.873+0000 7f3b13597640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3b0c15d3b0 con 0x7f3b0c15a470 2024-04-25T01:26:07.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.873+0000 7fd6bffff640 1 Processor -- start 2024-04-25T01:26:07.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.873+0000 7faf9e651640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7faf90049970 0x7faf900d7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.873+0000 7faf9e651640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf90046c10 0x7faf90006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.873+0000 7faf9e651640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf90049ea0 con 0x7faf90009fb0 2024-04-25T01:26:07.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.873+0000 7faf9e651640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7faf9004a210 con 0x7faf90049970 2024-04-25T01:26:07.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.873+0000 7faf9e651640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faf9004a4f0 con 0x7faf90046c10 2024-04-25T01:26:07.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7faf9d64f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7faf90049970 0x7faf900d7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7faf9d64f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7faf90049970 0x7faf900d7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43174/0 (socket says 172.21.15.112:43174) 2024-04-25T01:26:07.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7faf9d64f640 1 -- 172.21.15.112:0/3937093757 learned_addr learned my addr 172.21.15.112:0/3937093757 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:07.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.873+0000 7f0ad20ff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac4050720 0x7f0ac40062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.873+0000 7f0ad20ff640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ac40d7d00 con 0x7f0ac40d7920 2024-04-25T01:26:07.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.873+0000 7f0ad20ff640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ac40db250 con 0x7f0ac400fcf0 2024-04-25T01:26:07.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.873+0000 7f0ad20ff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac404ba00 con 0x7f0ac4050720 2024-04-25T01:26:07.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7f0ad10fd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0ac40d7920 0x7f0ac400edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37944/0 (socket says 172.21.15.112:37944) 2024-04-25T01:26:07.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7f0ad10fd640 1 -- 172.21.15.112:0/716037144 learned_addr learned my addr 172.21.15.112:0/716037144 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:07.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.873+0000 7fc626147640 1 Processor -- start 2024-04-25T01:26:07.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7faf9de50640 1 --2- 172.21.15.112:0/3937093757 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf90046c10 0x7faf90006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.873+0000 7fc626147640 1 -- start start 2024-04-25T01:26:07.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7f3b12595640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3b0c0c16d0 0x7f3b0c0c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7f0ad08fc640 1 --2- 172.21.15.112:0/716037144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac4050720 0x7f0ac40062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.872+0000 7f1aaa17e640 1 Processor -- start 2024-04-25T01:26:07.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.873+0000 7f1aaa17e640 1 -- start start 2024-04-25T01:26:07.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7faf9d64f640 1 -- 172.21.15.112:0/3937093757 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf90046c10 msgr2=0x7faf90006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:07.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7faf9d64f640 1 --2- 172.21.15.112:0/3937093757 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf90046c10 0x7faf90006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7faf9d64f640 1 -- 172.21.15.112:0/3937093757 >> v1:172.21.15.187:6789/0 conn(0x7faf90009fb0 legacy=0x7faf9004afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7f0ac3fff640 1 -- 172.21.15.112:0/716037144 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2914850204 0 0) 0x7f0ac40d7d00 con 0x7f0ac40d7920 2024-04-25T01:26:07.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7f0ac3fff640 1 -- 172.21.15.112:0/716037144 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ab8003520 con 0x7f0ac40d7920 2024-04-25T01:26:07.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.873+0000 7fd5782bd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd570152fa0 0x7fd5700c7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7fd5782bd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd570158e30 0x7fd5701592e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7fd5782bd640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd570155ab0 con 0x7fd57006c840 2024-04-25T01:26:07.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7fd5782bd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fd570156870 con 0x7fd570158e30 2024-04-25T01:26:07.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7fd5782bd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd5701563e0 con 0x7fd570152fa0 2024-04-25T01:26:07.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7fd57685a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd570158e30 0x7fd5701592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7fd57685a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd570158e30 0x7fd5701592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43178/0 (socket says 172.21.15.112:43178) 2024-04-25T01:26:07.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7fd57685a640 1 -- 172.21.15.112:0/1794243 learned_addr learned my addr 172.21.15.112:0/1794243 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:07.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7faf9d64f640 1 -- 172.21.15.112:0/3937093757 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9004a7d0 con 0x7faf90049970 2024-04-25T01:26:07.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7fac8d7e4640 1 Processor -- start 2024-04-25T01:26:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7f0ac3fff640 1 -- 172.21.15.112:0/716037144 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3473062832 0 0) 0x7f0ac40db250 con 0x7f0ac400fcf0 2024-04-25T01:26:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f0ac3fff640 1 -- 172.21.15.112:0/716037144 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ac40d7d00 con 0x7f0ac400fcf0 2024-04-25T01:26:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7fc626147640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc62006d830 0x7fc62006e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7fc626147640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc620073490 0x7fc620152380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fc626147640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc62006bf60 con 0x7fc6200c1ff0 2024-04-25T01:26:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fc626147640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fc620074ae0 con 0x7fc620073490 2024-04-25T01:26:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fc626147640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc620151ef0 con 0x7fc62006d830 2024-04-25T01:26:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.873+0000 7fd6bffff640 1 -- start start 2024-04-25T01:26:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fd6bffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd6c00c16d0 0x7fd6c00c8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fd6bffff640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6c015a950 con 0x7fd6c00c6580 2024-04-25T01:26:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fd6bffff640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6c015d0d0 con 0x7fd6c015a470 2024-04-25T01:26:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fd6bffff640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fd6c015d3b0 con 0x7fd6c00c16d0 2024-04-25T01:26:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fd6be7fc640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd6c00c6580 0x7fd6c006c730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37946/0 (socket says 172.21.15.112:37946) 2024-04-25T01:26:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fd6be7fc640 1 -- 172.21.15.112:0/1759699204 learned_addr learned my addr 172.21.15.112:0/1759699204 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fd6beffd640 1 --2- 172.21.15.112:0/1759699204 >> v2:172.21.15.112:3300/0 conn(0x7fd6c00c16d0 0x7fd6c00c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fac8d7e4640 1 -- start start 2024-04-25T01:26:07.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fc625946640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fc6200c1ff0 0x7fc6200743d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:59552/0 (socket says 172.21.15.112:59552) 2024-04-25T01:26:07.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fc625946640 1 -- 172.21.15.112:0/1770221761 learned_addr learned my addr 172.21.15.112:0/1770221761 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:07.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fc625145640 1 --2- 172.21.15.112:0/1770221761 >> v2:172.21.15.187:3300/0 conn(0x7fc62006d830 0x7fc62006e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fc624944640 1 --2- 172.21.15.112:0/1770221761 >> v2:172.21.15.112:3300/0 conn(0x7fc620073490 0x7fc620152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fc617fff640 1 -- 172.21.15.112:0/1770221761 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 99496444 0 0) 0x7fc62006bf60 con 0x7fc6200c1ff0 2024-04-25T01:26:07.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7f3b12595640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3b0c0c16d0 0x7f3b0c0c8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:48902/0 (socket says 172.21.15.112:48902) 2024-04-25T01:26:07.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7f3b12595640 1 -- 172.21.15.112:0/1129921969 learned_addr learned my addr 172.21.15.112:0/1129921969 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:07.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7f3b12d96640 1 --2- 172.21.15.112:0/1129921969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b0c15a470 0x7f3b0c06ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fc617fff640 1 -- 172.21.15.112:0/1770221761 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc60c003520 con 0x7fc6200c1ff0 2024-04-25T01:26:07.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7f3b12595640 1 -- 172.21.15.112:0/1129921969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b0c15a470 msgr2=0x7f3b0c06ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:07.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7f3b12595640 1 --2- 172.21.15.112:0/1129921969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b0c15a470 0x7f3b0c06ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7f3b12595640 1 -- 172.21.15.112:0/1129921969 >> v1:172.21.15.187:6789/0 conn(0x7f3b0c0c6580 legacy=0x7f3b0c06c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:07.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7f3b12595640 1 -- 172.21.15.112:0/1129921969 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0c15d690 con 0x7f3b0c0c16d0 2024-04-25T01:26:07.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f0ac3fff640 1 -- 172.21.15.112:0/716037144 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3157051785 0 0) 0x7f0ac40d7d00 con 0x7f0ac400fcf0 2024-04-25T01:26:07.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f0ac3fff640 1 -- 172.21.15.112:0/716037144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac4050720 msgr2=0x7f0ac40062c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:07.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f0ac3fff640 1 --2- 172.21.15.112:0/716037144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac4050720 0x7f0ac40062c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f0ac3fff640 1 -- 172.21.15.112:0/716037144 >> v1:172.21.15.112:6789/0 conn(0x7f0ac40d7920 legacy=0x7f0ac400edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f0ac3fff640 1 -- 172.21.15.112:0/716037144 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac404bce0 con 0x7f0ac400fcf0 2024-04-25T01:26:07.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f0ac3fff640 1 -- 172.21.15.112:0/716037144 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f0acc04fd50 con 0x7f0ac400fcf0 2024-04-25T01:26:07.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f0ac3fff640 1 -- 172.21.15.112:0/716037144 >> v1:172.21.15.187:6789/0 conn(0x7f0ac400fcf0 legacy=0x7f0ac4006800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f0ac3fff640 1 --2- 172.21.15.112:0/716037144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ab8003b80 0x7f0ab8003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f0ac3fff640 1 --2- 172.21.15.112:0/716037144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ab8004800 0x7f0ab8004c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f0ac3fff640 1 --2- 172.21.15.112:0/716037144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ab80051a0 0x7f0ab800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f0ac3fff640 1 -- 172.21.15.112:0/716037144 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac404ba00 con 0x7f0ab8003b80 2024-04-25T01:26:07.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f0ac3fff640 1 -- 172.21.15.112:0/716037144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab800bf80 con 0x7f0ab8004800 2024-04-25T01:26:07.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f0ac3fff640 1 -- 172.21.15.112:0/716037144 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0ab800c180 con 0x7f0ab80051a0 2024-04-25T01:26:07.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7faf9d64f640 1 --2- 172.21.15.112:0/3937093757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf90049970 0x7faf900d7450 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faf94009870 tx=0x7faf9402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22ca11ad75a26c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7f1aaa17e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1a9c00bd40 0x7f1a9c0d3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7f1aaa17e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1a9c04b100 0x7f1a9c046b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7f1aaa17e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a9c006e90 0x7f1a9c007320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7f1aaa17e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f1a9c0d4070 con 0x7f1a9c00bd40 2024-04-25T01:26:07.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7f1aaa17e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1a9c047040 con 0x7f1a9c04b100 2024-04-25T01:26:07.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7f1aaa17e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1a9c0d9c50 con 0x7f1a9c006e90 2024-04-25T01:26:07.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f1aa37fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1a9c00bd40 0x7f1a9c0d3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f1aa37fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1a9c00bd40 0x7f1a9c0d3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43186/0 (socket says 172.21.15.112:43186) 2024-04-25T01:26:07.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f1aa37fe640 1 -- 172.21.15.112:0/2801691776 learned_addr learned my addr 172.21.15.112:0/2801691776 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:07.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f1aa2ffd640 1 --2- 172.21.15.112:0/2801691776 >> v2:172.21.15.187:3300/0 conn(0x7f1a9c04b100 0x7f1a9c046b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f1aa3fff640 1 --2- 172.21.15.112:0/2801691776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a9c006e90 0x7f1a9c007320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f0ad10fd640 1 --2- 172.21.15.112:0/716037144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ab8003b80 0x7f0ab8003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f0ad18fe640 1 --2- 172.21.15.112:0/716037144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ab80051a0 0x7f0ab800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7f0ad08fc640 1 --2- 172.21.15.112:0/716037144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ab8004800 0x7f0ab8004c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7fd57685a640 1 -- 172.21.15.112:0/1794243 >> v2:172.21.15.187:3300/0 conn(0x7fd570152fa0 msgr2=0x7fd5700c7300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:26:07.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.874+0000 7fd57685a640 1 --2- 172.21.15.112:0/1794243 >> v2:172.21.15.187:3300/0 conn(0x7fd570152fa0 0x7fd5700c7300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fd57685a640 1 -- 172.21.15.112:0/1794243 >> v1:172.21.15.187:6789/0 conn(0x7fd57006c840 legacy=0x7fd5700c8190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:26:07.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fd57685a640 1 -- 172.21.15.112:0/1794243 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd570155d90 con 0x7fd570158e30 2024-04-25T01:26:07.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fd6beffd640 1 -- 172.21.15.112:0/1759699204 >> v1:172.21.15.187:6789/0 conn(0x7fd6c015a470 legacy=0x7fd6c01643c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fc617fff640 1 -- 172.21.15.112:0/1770221761 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 700291150 0 0) 0x7fc60c003520 con 0x7fc6200c1ff0 2024-04-25T01:26:07.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fc617fff640 1 -- 172.21.15.112:0/1770221761 >> v2:172.21.15.187:3300/0 conn(0x7fc62006d830 msgr2=0x7fc62006e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:07.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fc617fff640 1 --2- 172.21.15.112:0/1770221761 >> v2:172.21.15.187:3300/0 conn(0x7fc62006d830 0x7fc62006e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fc617fff640 1 -- 172.21.15.112:0/1770221761 >> v2:172.21.15.112:3300/0 conn(0x7fc620073490 msgr2=0x7fc620152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:07.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.875+0000 7fc617fff640 1 --2- 172.21.15.112:0/1770221761 >> v2:172.21.15.112:3300/0 conn(0x7fc620073490 0x7fc620152380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fc617fff640 1 -- 172.21.15.112:0/1770221761 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62006c240 con 0x7fc6200c1ff0 2024-04-25T01:26:07.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fc617fff640 1 -- 172.21.15.112:0/1770221761 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fc6180019f0 con 0x7fc6200c1ff0 2024-04-25T01:26:07.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fc617fff640 1 -- 172.21.15.112:0/1770221761 >> v1:172.21.15.187:6789/0 conn(0x7fc6200c1ff0 legacy=0x7fc6200743d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fd6beffd640 1 -- 172.21.15.112:0/1759699204 >> v1:172.21.15.112:6789/0 conn(0x7fd6c00c6580 legacy=0x7fd6c006c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fd6beffd640 1 -- 172.21.15.112:0/1759699204 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c015d690 con 0x7fd6c00c16d0 2024-04-25T01:26:07.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fd6bdffb640 1 -- 172.21.15.112:0/1759699204 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1870348421 0 0) 0x7fd6c015a950 con 0x7fd6c00c6580 2024-04-25T01:26:07.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fd57685a640 1 --2- 172.21.15.112:0/1794243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd570158e30 0x7fd5701592e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009ad0 tx=0x7fd56c02a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f67e07fdc0d831 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7f3b12595640 1 --2- 172.21.15.112:0/1129921969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c0c16d0 0x7f3b0c0c8110 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0adc20 tx=0x7f3b0402a6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d6dedffe292847e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fc617fff640 1 --2- 172.21.15.112:0/1770221761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc60c003a00 0x7fc60c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fc617fff640 1 --2- 172.21.15.112:0/1770221761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc60c004800 0x7fc60c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fc617fff640 1 --2- 172.21.15.112:0/1770221761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc60c0051c0 0x7fc60c0141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fc617fff640 1 -- 172.21.15.112:0/1770221761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc620074ae0 con 0x7fc60c003a00 2024-04-25T01:26:07.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fc617fff640 1 -- 172.21.15.112:0/1770221761 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c0148d0 con 0x7fc60c004800 2024-04-25T01:26:07.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fc617fff640 1 -- 172.21.15.112:0/1770221761 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc60c014ad0 con 0x7fc60c0051c0 2024-04-25T01:26:07.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fc624944640 1 --2- 172.21.15.112:0/1770221761 >> v2:172.21.15.112:3300/0 conn(0x7fc620073490 0x7fc620152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:07.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7f1aa3fff640 1 -- 172.21.15.112:0/2801691776 >> v2:172.21.15.187:3300/0 conn(0x7f1a9c04b100 msgr2=0x7f1a9c046b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:07.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7f1aa3fff640 1 --2- 172.21.15.112:0/2801691776 >> v2:172.21.15.187:3300/0 conn(0x7f1a9c04b100 0x7f1a9c046b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7f1aa3fff640 1 -- 172.21.15.112:0/2801691776 >> v2:172.21.15.112:3300/0 conn(0x7f1a9c00bd40 msgr2=0x7f1a9c0d3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:07.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7f1aa3fff640 1 --2- 172.21.15.112:0/2801691776 >> v2:172.21.15.112:3300/0 conn(0x7f1a9c00bd40 0x7f1a9c0d3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7f1aa3fff640 1 -- 172.21.15.112:0/2801691776 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9c0d9f30 con 0x7f1a9c006e90 2024-04-25T01:26:07.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7f1aa2ffd640 1 --2- 172.21.15.112:0/2801691776 >> v2:172.21.15.187:3300/0 conn(0x7f1a9c04b100 0x7f1a9c046b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:07.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fc625946640 1 --2- 172.21.15.112:0/1770221761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc60c0051c0 0x7fc60c0141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7f0ad10fd640 1 -- 172.21.15.112:0/716037144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ab80051a0 msgr2=0x7f0ab800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:07.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7f0ad10fd640 1 --2- 172.21.15.112:0/716037144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ab80051a0 0x7f0ab800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7f0ad10fd640 1 -- 172.21.15.112:0/716037144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ab8004800 msgr2=0x7f0ab8004c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:07.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7f0ad10fd640 1 --2- 172.21.15.112:0/716037144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ab8004800 0x7f0ab8004c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7f0ad10fd640 1 -- 172.21.15.112:0/716037144 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac404bce0 con 0x7f0ab8003b80 2024-04-25T01:26:07.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fac8d7e4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fac800dbe80 0x7fac800df200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fc624944640 1 --2- 172.21.15.112:0/1770221761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc60c004800 0x7fc60c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fac8d7e4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac8000ea00 0x7fac8000ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fac8d7e4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac800df740 con 0x7fac8004dae0 2024-04-25T01:26:07.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fac8d7e4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fac8004a560 con 0x7fac800dbe80 2024-04-25T01:26:07.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fac8d7e4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fac8004a840 con 0x7fac8000ea00 2024-04-25T01:26:07.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fac87fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fac800dbe80 0x7fac800df200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fac87fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fac800dbe80 0x7fac800df200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43240/0 (socket says 172.21.15.112:43240) 2024-04-25T01:26:07.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7fac87fff640 1 -- 172.21.15.112:0/3665650458 learned_addr learned my addr 172.21.15.112:0/3665650458 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:07.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7f3b11593640 1 -- 172.21.15.112:0/1129921969 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04036070 con 0x7f3b0c0c16d0 2024-04-25T01:26:07.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7f3b11593640 1 -- 172.21.15.112:0/1129921969 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b04031600 con 0x7f3b0c0c16d0 2024-04-25T01:26:07.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.876+0000 7f3b11593640 1 -- 172.21.15.112:0/1129921969 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04031a10 con 0x7f3b0c0c16d0 2024-04-25T01:26:07.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7faf8bfff640 1 -- 172.21.15.112:0/3937093757 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf94037070 con 0x7faf90049970 2024-04-25T01:26:07.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7faf8bfff640 1 -- 172.21.15.112:0/3937093757 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf94032440 con 0x7faf90049970 2024-04-25T01:26:07.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7fac8cfe3640 1 --2- 172.21.15.112:0/3665650458 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac8000ea00 0x7fac8000ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7fc624944640 1 -- 172.21.15.112:0/1770221761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc60c0051c0 msgr2=0x7fc60c0141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:07.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7fc624944640 1 --2- 172.21.15.112:0/1770221761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc60c0051c0 0x7fc60c0141d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7fc624944640 1 -- 172.21.15.112:0/1770221761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc60c003a00 msgr2=0x7fc60c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:07.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7fc624944640 1 --2- 172.21.15.112:0/1770221761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc60c003a00 0x7fc60c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7fc624944640 1 -- 172.21.15.112:0/1770221761 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60c003520 con 0x7fc60c004800 2024-04-25T01:26:07.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7fc625946640 1 -- 172.21.15.112:0/1770221761 reap_dead start 2024-04-25T01:26:07.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7f3b13597640 1 -- 172.21.15.112:0/1129921969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c0c16d0 msgr2=0x7f3b0c0c8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7f3b13597640 1 --2- 172.21.15.112:0/1129921969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c0c16d0 0x7f3b0c0c8110 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0adc20 tx=0x7f3b0402a6d0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7fd6beffd640 1 --2- 172.21.15.112:0/1759699204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6c00c16d0 0x7fd6c00c8110 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00097a0 tx=0x7fd6b002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104d3457b81a3a2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7fd6bdffb640 1 -- 172.21.15.112:0/1759699204 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0037070 con 0x7fd6c00c16d0 2024-04-25T01:26:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7fac87fff640 1 -- 172.21.15.112:0/3665650458 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac8000ea00 msgr2=0x7fac8000ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7fac87fff640 1 --2- 172.21.15.112:0/3665650458 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac8000ea00 0x7fac8000ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7fac87fff640 1 -- 172.21.15.112:0/3665650458 >> v1:172.21.15.112:6789/0 conn(0x7fac8004dae0 legacy=0x7fac800071b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7fd575057640 1 -- 172.21.15.112:0/1794243 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c036070 con 0x7fd570158e30 2024-04-25T01:26:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7fd575057640 1 -- 172.21.15.112:0/1794243 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd56c031600 con 0x7fd570158e30 2024-04-25T01:26:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7fd575057640 1 -- 172.21.15.112:0/1794243 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c0319d0 con 0x7fd570158e30 2024-04-25T01:26:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7fac87fff640 1 -- 172.21.15.112:0/3665650458 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8004ab20 con 0x7fac800dbe80 2024-04-25T01:26:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7f3b13597640 1 -- 172.21.15.112:0/1129921969 shutdown_connections 2024-04-25T01:26:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7f3b13597640 1 --2- 172.21.15.112:0/1129921969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b0c15a470 0x7f3b0c06ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7f3b13597640 1 --2- 172.21.15.112:0/1129921969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c0c16d0 0x7f3b0c0c8110 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7f3b13597640 1 -- 172.21.15.112:0/1129921969 >> 172.21.15.112:0/1129921969 conn(0x7f3b0c0ad1c0 msgr2=0x7f3b0c0c7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7fd6bdffb640 1 -- 172.21.15.112:0/1759699204 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b0032440 con 0x7fd6c00c16d0 2024-04-25T01:26:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7fd6bdffb640 1 -- 172.21.15.112:0/1759699204 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0032810 con 0x7fd6c00c16d0 2024-04-25T01:26:07.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7f3b13597640 1 -- 172.21.15.112:0/1129921969 shutdown_connections 2024-04-25T01:26:07.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7faf8bfff640 1 -- 172.21.15.112:0/3937093757 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf94032810 con 0x7faf90049970 2024-04-25T01:26:07.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7f3b13597640 1 -- 172.21.15.112:0/1129921969 wait complete. 2024-04-25T01:26:07.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7fc624944640 1 --2- 172.21.15.112:0/1770221761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc60c004800 0x7fc60c004c80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00aa80 tx=0x7fc60c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcf40a8ebff5ad78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7faf9e651640 1 -- 172.21.15.112:0/3937093757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf90049970 msgr2=0x7faf900d7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7faf9e651640 1 --2- 172.21.15.112:0/3937093757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf90049970 0x7faf900d7450 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faf94009870 tx=0x7faf9402a250 comp rx=0 tx=0).stop 2024-04-25T01:26:07.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7faf9e651640 1 -- 172.21.15.112:0/3937093757 shutdown_connections 2024-04-25T01:26:07.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7faf9e651640 1 --2- 172.21.15.112:0/3937093757 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf90046c10 0x7faf90006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7faf9e651640 1 --2- 172.21.15.112:0/3937093757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf90049970 0x7faf900d7450 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7faf9e651640 1 -- 172.21.15.112:0/3937093757 >> 172.21.15.112:0/3937093757 conn(0x7faf90048240 msgr2=0x7faf90048650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.877+0000 7f1aa3fff640 1 --2- 172.21.15.112:0/2801691776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a9c006e90 0x7f1a9c007320 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1aa402d700 tx=0x7f1aa40a5e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3b866b0afddbc3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7f1aa27fc640 1 -- 172.21.15.112:0/2801691776 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa40b3070 con 0x7f1a9c006e90 2024-04-25T01:26:07.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7f1aa27fc640 1 -- 172.21.15.112:0/2801691776 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa40a6c30 con 0x7f1a9c006e90 2024-04-25T01:26:07.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7f1aa27fc640 1 -- 172.21.15.112:0/2801691776 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa40a6e60 con 0x7f1a9c006e90 2024-04-25T01:26:07.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7fc617fff640 1 -- 172.21.15.112:0/1770221761 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c037070 con 0x7fc60c004800 2024-04-25T01:26:07.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7fc617fff640 1 -- 172.21.15.112:0/1770221761 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc61c02abf0 con 0x7fc60c004800 2024-04-25T01:26:07.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7fc617fff640 1 -- 172.21.15.112:0/1770221761 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c00a650 con 0x7fc60c004800 2024-04-25T01:26:07.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7f0ad10fd640 1 --2- 172.21.15.112:0/716037144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ab8003b80 0x7f0ab8003fc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0abc0019c0 tx=0x7f0ab8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea5db015b03c615 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7fd5782bd640 1 -- 172.21.15.112:0/1794243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd570158e30 msgr2=0x7fd5701592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7fd5782bd640 1 --2- 172.21.15.112:0/1794243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd570158e30 0x7fd5701592e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009ad0 tx=0x7fd56c02a440 comp rx=0 tx=0).stop 2024-04-25T01:26:07.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7fd6bffff640 1 -- 172.21.15.112:0/1759699204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6c00c16d0 msgr2=0x7fd6c00c8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7fd6bffff640 1 --2- 172.21.15.112:0/1759699204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6c00c16d0 0x7fd6c00c8110 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00097a0 tx=0x7fd6b002a230 comp rx=0 tx=0).stop 2024-04-25T01:26:07.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7fd5782bd640 1 -- 172.21.15.112:0/1794243 shutdown_connections 2024-04-25T01:26:07.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7fd5782bd640 1 --2- 172.21.15.112:0/1794243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd570158e30 0x7fd5701592e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7fd5782bd640 1 --2- 172.21.15.112:0/1794243 >> v2:172.21.15.187:3300/0 conn(0x7fd570152fa0 0x7fd5700c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7fd5782bd640 1 -- 172.21.15.112:0/1794243 >> 172.21.15.112:0/1794243 conn(0x7fd5700ad150 msgr2=0x7fd5700ad560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7fd5782bd640 1 -- 172.21.15.112:0/1794243 shutdown_connections 2024-04-25T01:26:07.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7fd6bffff640 1 -- 172.21.15.112:0/1759699204 shutdown_connections 2024-04-25T01:26:07.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7fd6bffff640 1 --2- 172.21.15.112:0/1759699204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6c00c16d0 0x7fd6c00c8110 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7fd6bffff640 1 -- 172.21.15.112:0/1759699204 >> 172.21.15.112:0/1759699204 conn(0x7fd6c00ad1c0 msgr2=0x7fd6c00c7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7fac87fff640 1 --2- 172.21.15.112:0/3665650458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac800dbe80 0x7fac800df200 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fac74009870 tx=0x7fac7402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea377216139f8a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7f1aaa17e640 1 -- 172.21.15.112:0/2801691776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a9c006e90 msgr2=0x7f1a9c007320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7f1aaa17e640 1 --2- 172.21.15.112:0/2801691776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a9c006e90 0x7f1a9c007320 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1aa402d700 tx=0x7f1aa40a5e20 comp rx=0 tx=0).stop 2024-04-25T01:26:07.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7fd6bffff640 1 -- 172.21.15.112:0/1759699204 shutdown_connections 2024-04-25T01:26:07.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7faf9e651640 1 -- 172.21.15.112:0/3937093757 shutdown_connections 2024-04-25T01:26:07.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.878+0000 7faf9e651640 1 -- 172.21.15.112:0/3937093757 wait complete. 2024-04-25T01:26:07.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7fd5782bd640 1 -- 172.21.15.112:0/1794243 wait complete. 2024-04-25T01:26:07.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7fc626147640 1 -- 172.21.15.112:0/1770221761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc60c004800 msgr2=0x7fc60c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7fc626147640 1 --2- 172.21.15.112:0/1770221761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc60c004800 0x7fc60c004c80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00aa80 tx=0x7fc60c005830 comp rx=0 tx=0).stop 2024-04-25T01:26:07.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7f0ac3fff640 1 -- 172.21.15.112:0/716037144 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc001610 con 0x7f0ab8003b80 2024-04-25T01:26:07.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7fc626147640 1 -- 172.21.15.112:0/1770221761 shutdown_connections 2024-04-25T01:26:07.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7fc626147640 1 --2- 172.21.15.112:0/1770221761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc60c004800 0x7fc60c004c80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7fc626147640 1 -- 172.21.15.112:0/1770221761 >> 172.21.15.112:0/1770221761 conn(0x7fc6200c7a10 msgr2=0x7fc6200c7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7f1aaa17e640 1 -- 172.21.15.112:0/2801691776 shutdown_connections 2024-04-25T01:26:07.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7f1aaa17e640 1 --2- 172.21.15.112:0/2801691776 >> v2:172.21.15.187:3300/0 conn(0x7f1a9c04b100 0x7f1a9c046b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7f1aaa17e640 1 --2- 172.21.15.112:0/2801691776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a9c006e90 0x7f1a9c007320 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7f1aaa17e640 1 --2- 172.21.15.112:0/2801691776 >> v2:172.21.15.112:3300/0 conn(0x7f1a9c00bd40 0x7f1a9c0d3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7f1aaa17e640 1 -- 172.21.15.112:0/2801691776 >> 172.21.15.112:0/2801691776 conn(0x7f1a9c048240 msgr2=0x7f1a9c048650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7f3b13597640 1 Processor -- start 2024-04-25T01:26:07.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7f3b13597640 1 -- start start 2024-04-25T01:26:07.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7fac86ffd640 1 -- 172.21.15.112:0/3665650458 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74037070 con 0x7fac800dbe80 2024-04-25T01:26:07.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7f0ac3fff640 1 -- 172.21.15.112:0/716037144 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0abc02e070 con 0x7f0ab8003b80 2024-04-25T01:26:07.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7fac86ffd640 1 -- 172.21.15.112:0/3665650458 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac74032440 con 0x7fac800dbe80 2024-04-25T01:26:07.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7fac86ffd640 1 -- 172.21.15.112:0/3665650458 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74032810 con 0x7fac800dbe80 2024-04-25T01:26:07.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7f3b13597640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b0c0c16d0 0x7f3b0c15b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7f3b13597640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c0c6580 0x7f3b0c15bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7f3b13597640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c15a470 0x7f3b0c1fb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7f3b13597640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c15de90 con 0x7f3b0c15a470 2024-04-25T01:26:07.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7f3b13597640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c0c1ab0 con 0x7f3b0c0c6580 2024-04-25T01:26:07.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7f3b13597640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3b0c0c8650 con 0x7f3b0c0c16d0 2024-04-25T01:26:07.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7fd6bffff640 1 -- 172.21.15.112:0/1759699204 wait complete. 2024-04-25T01:26:07.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7fd6bffff640 1 Processor -- start 2024-04-25T01:26:07.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7faf9e651640 1 Processor -- start 2024-04-25T01:26:07.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7faf9e651640 1 -- start start 2024-04-25T01:26:07.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.879+0000 7f1aaa17e640 1 -- 172.21.15.112:0/2801691776 shutdown_connections 2024-04-25T01:26:07.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7fac8d7e4640 1 -- 172.21.15.112:0/3665650458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac800dbe80 msgr2=0x7fac800df200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7fac8d7e4640 1 --2- 172.21.15.112:0/3665650458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac800dbe80 0x7fac800df200 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fac74009870 tx=0x7fac7402a1c0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7fac8d7e4640 1 -- 172.21.15.112:0/3665650458 shutdown_connections 2024-04-25T01:26:07.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7fac8d7e4640 1 --2- 172.21.15.112:0/3665650458 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac8000ea00 0x7fac8000ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7fac8d7e4640 1 --2- 172.21.15.112:0/3665650458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac800dbe80 0x7fac800df200 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7fac8d7e4640 1 -- 172.21.15.112:0/3665650458 >> 172.21.15.112:0/3665650458 conn(0x7fac8004c660 msgr2=0x7fac8004ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7fac8d7e4640 1 -- 172.21.15.112:0/3665650458 shutdown_connections 2024-04-25T01:26:07.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7fac8d7e4640 1 -- 172.21.15.112:0/3665650458 wait complete. 2024-04-25T01:26:07.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7f0ac3fff640 1 -- 172.21.15.112:0/716037144 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc0048a0 con 0x7f0ab8003b80 2024-04-25T01:26:07.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7fd5782bd640 1 Processor -- start 2024-04-25T01:26:07.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7fd5782bd640 1 -- start start 2024-04-25T01:26:07.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7fc626147640 1 -- 172.21.15.112:0/1770221761 shutdown_connections 2024-04-25T01:26:07.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7f0ad20ff640 1 -- 172.21.15.112:0/716037144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ab8003b80 msgr2=0x7f0ab8003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7f0ad20ff640 1 --2- 172.21.15.112:0/716037144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ab8003b80 0x7f0ab8003fc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0abc0019c0 tx=0x7f0ab8005830 comp rx=0 tx=0).stop 2024-04-25T01:26:07.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7faf9e651640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf90009fb0 0x7faf900d5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7f0ad18fe640 1 -- 172.21.15.112:0/716037144 reap_dead start 2024-04-25T01:26:07.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7f0ad20ff640 1 -- 172.21.15.112:0/716037144 shutdown_connections 2024-04-25T01:26:07.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7f0ad20ff640 1 -- 172.21.15.112:0/716037144 >> 172.21.15.112:0/716037144 conn(0x7f0ac404fc00 msgr2=0x7f0ac4050010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7fd5782bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd57006c840 0x7fd570154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7faf9e651640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf90046c10 0x7faf90041e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7f0ad20ff640 1 -- 172.21.15.112:0/716037144 shutdown_connections 2024-04-25T01:26:07.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7f1aaa17e640 1 -- 172.21.15.112:0/2801691776 wait complete. 2024-04-25T01:26:07.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7fd576059640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd57006c840 0x7fd570154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7fd576059640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd57006c840 0x7fd570154b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43266/0 (socket says 172.21.15.112:43266) 2024-04-25T01:26:07.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7f1aaa17e640 1 Processor -- start 2024-04-25T01:26:07.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7fd576059640 1 -- 172.21.15.112:0/3875146841 learned_addr learned my addr 172.21.15.112:0/3875146841 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:07.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7fd5782bd640 1 --2- 172.21.15.112:0/3875146841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd570152fa0 0x7fd5701ee8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7faf9d64f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf90009fb0 0x7faf900d5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7fd5782bd640 1 --2- 172.21.15.112:0/3875146841 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd570158e30 0x7fd5701eedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7fd5782bd640 1 -- 172.21.15.112:0/3875146841 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5700c88a0 con 0x7fd57006c840 2024-04-25T01:26:07.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7fd5782bd640 1 -- 172.21.15.112:0/3875146841 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd57006cdb0 con 0x7fd570152fa0 2024-04-25T01:26:07.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7fd5782bd640 1 -- 172.21.15.112:0/3875146841 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd5700c8a20 con 0x7fd570158e30 2024-04-25T01:26:07.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7faf9d64f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf90009fb0 0x7faf900d5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:48952/0 (socket says 172.21.15.112:48952) 2024-04-25T01:26:07.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7f3b12d96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c15a470 0x7f3b0c1fb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7f3b11d94640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c0c6580 0x7f3b0c15bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7f3b12595640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b0c0c16d0 0x7f3b0c15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7f3b12595640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b0c0c16d0 0x7f3b0c15b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:56428/0 (socket says 172.21.15.112:56428) 2024-04-25T01:26:07.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7f3b11d94640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c0c6580 0x7f3b0c15bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:48938/0 (socket says 172.21.15.112:48938) 2024-04-25T01:26:07.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.880+0000 7f3b12595640 1 -- 172.21.15.112:0/3524113971 learned_addr learned my addr 172.21.15.112:0/3524113971 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:07.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7f3b12595640 1 -- 172.21.15.112:0/3524113971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c0c6580 msgr2=0x7f3b0c15bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7f3b12595640 1 --2- 172.21.15.112:0/3524113971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c0c6580 0x7f3b0c15bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7f3b12595640 1 -- 172.21.15.112:0/3524113971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c15a470 msgr2=0x7f3b0c1fb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7f3b12595640 1 --2- 172.21.15.112:0/3524113971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c15a470 0x7f3b0c1fb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7f3b12595640 1 -- 172.21.15.112:0/3524113971 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0c15b810 con 0x7f3b0c0c16d0 2024-04-25T01:26:07.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7f3b12d96640 1 --2- 172.21.15.112:0/3524113971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c15a470 0x7f3b0c1fb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:26:07.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7f3b11d94640 1 --2- 172.21.15.112:0/3524113971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c0c6580 0x7f3b0c15bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-25T01:26:07.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7fd575858640 1 --2- 172.21.15.112:0/3875146841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd570152fa0 0x7fd5701ee8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7fd6bffff640 1 -- start start 2024-04-25T01:26:07.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7fd6bffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6c00c16d0 0x7fd6c0153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7fd6bffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6c00c6580 0x7fd6c01f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7fd6bffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6c015a470 0x7fd6c01f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7fd6bffff640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c015de90 con 0x7fd6c015a470 2024-04-25T01:26:07.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7fd6bffff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c00c1ab0 con 0x7fd6c00c16d0 2024-04-25T01:26:07.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7fd6bffff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c00c8650 con 0x7fd6c00c6580 2024-04-25T01:26:07.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7faf9d64f640 1 -- 172.21.15.112:0/1509197802 learned_addr learned my addr 172.21.15.112:0/1509197802 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:07.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7faf9e651640 1 --2- 172.21.15.112:0/1509197802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf90049970 0x7faf900d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7fd575858640 1 -- 172.21.15.112:0/3875146841 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd570158e30 msgr2=0x7fd5701eedf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:26:07.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7faf9ce4e640 1 --2- 172.21.15.112:0/1509197802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf90046c10 0x7faf90041e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.881+0000 7f0ad20ff640 1 -- 172.21.15.112:0/716037144 wait complete. 2024-04-25T01:26:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7faf9e651640 1 -- 172.21.15.112:0/1509197802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faf900314d0 con 0x7faf90046c10 2024-04-25T01:26:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7faf9e651640 1 -- 172.21.15.112:0/1509197802 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faf9000a440 con 0x7faf90009fb0 2024-04-25T01:26:07.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7fd575858640 1 --2- 172.21.15.112:0/3875146841 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd570158e30 0x7fd5701eedf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7fd575858640 1 -- 172.21.15.112:0/3875146841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd57006c840 msgr2=0x7fd570154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7fd575858640 1 --2- 172.21.15.112:0/3875146841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd57006c840 0x7fd570154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7faf9e651640 1 -- 172.21.15.112:0/1509197802 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faf900073d0 con 0x7faf90049970 2024-04-25T01:26:07.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7fd575858640 1 -- 172.21.15.112:0/3875146841 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5700c2070 con 0x7fd570152fa0 2024-04-25T01:26:07.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7fac8d7e4640 1 Processor -- start 2024-04-25T01:26:07.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7f3b12595640 1 --2- 172.21.15.112:0/3524113971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b0c0c16d0 0x7f3b0c15b210 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3b04000f80 tx=0x7f3b04004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7f3b02ffd640 1 -- 172.21.15.112:0/3524113971 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04041020 con 0x7f3b0c0c16d0 2024-04-25T01:26:07.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7f3b13597640 1 -- 172.21.15.112:0/3524113971 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b0c0c9580 con 0x7f3b0c0c16d0 2024-04-25T01:26:07.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7f3b13597640 1 -- 172.21.15.112:0/3524113971 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b0c1fba80 con 0x7f3b0c0c16d0 2024-04-25T01:26:07.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7fd6be7fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6c00c6580 0x7fd6c01f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7fd6be7fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6c00c6580 0x7fd6c01f51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:56448/0 (socket says 172.21.15.112:56448) 2024-04-25T01:26:07.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7fd6be7fc640 1 -- 172.21.15.112:0/99841031 learned_addr learned my addr 172.21.15.112:0/99841031 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:07.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7fd576059640 1 --2- 172.21.15.112:0/3875146841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd57006c840 0x7fd570154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:26:07.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7fd6beffd640 1 --2- 172.21.15.112:0/99841031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6c00c16d0 0x7fd6c0153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7fac8d7e4640 1 -- start start 2024-04-25T01:26:07.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7fd575858640 1 --2- 172.21.15.112:0/3875146841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd570152fa0 0x7fd5701ee8b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd56000b3e0 tx=0x7fd56000b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7fd6be7fc640 1 -- 172.21.15.112:0/99841031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6c00c16d0 msgr2=0x7fd6c0153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7fd6be7fc640 1 --2- 172.21.15.112:0/99841031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6c00c16d0 0x7fd6c0153f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.882+0000 7f1aaa17e640 1 -- start start 2024-04-25T01:26:07.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7f1aaa17e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a9c006e90 0x7f1a9c0d7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7f1aaa17e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a9c00bd40 0x7f1a9c0d8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7f1aaa17e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a9c04b100 0x7f1a9c0d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7faf9ce4e640 1 -- 172.21.15.112:0/1509197802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf90049970 msgr2=0x7faf900d18a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:26:07.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7faf9ce4e640 1 --2- 172.21.15.112:0/1509197802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf90049970 0x7faf900d18a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7f0ad20ff640 1 Processor -- start 2024-04-25T01:26:07.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7faf9ce4e640 1 -- 172.21.15.112:0/1509197802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf90009fb0 msgr2=0x7faf900d5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7faf9ce4e640 1 --2- 172.21.15.112:0/1509197802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf90009fb0 0x7faf900d5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7fd566ffd640 1 -- 172.21.15.112:0/3875146841 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd560015020 con 0x7fd570152fa0 2024-04-25T01:26:07.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7fd566ffd640 1 -- 172.21.15.112:0/3875146841 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd560011830 con 0x7fd570152fa0 2024-04-25T01:26:07.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7fd566ffd640 1 -- 172.21.15.112:0/3875146841 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd560011c00 con 0x7fd570152fa0 2024-04-25T01:26:07.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7faf9ce4e640 1 -- 172.21.15.112:0/1509197802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf90170fe0 con 0x7faf90046c10 2024-04-25T01:26:07.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7f1aaa17e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9c0da7d0 con 0x7f1a9c04b100 2024-04-25T01:26:07.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7f1aaa17e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9c0da4f0 con 0x7f1a9c00bd40 2024-04-25T01:26:07.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7f1aaa17e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1a9c0d9920 con 0x7f1a9c006e90 2024-04-25T01:26:07.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7f1aa2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a9c00bd40 0x7f1a9c0d8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7faf9ce4e640 1 --2- 172.21.15.112:0/1509197802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf90046c10 0x7faf90041e40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00c390 tx=0x7faf8c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7fd6be7fc640 1 -- 172.21.15.112:0/99841031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6c015a470 msgr2=0x7fd6c01f5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7fd6bf7fe640 1 --2- 172.21.15.112:0/99841031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6c015a470 0x7fd6c01f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7f3b02ffd640 1 -- 172.21.15.112:0/3524113971 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b04007820 con 0x7f3b0c0c16d0 2024-04-25T01:26:07.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7f3b02ffd640 1 -- 172.21.15.112:0/3524113971 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04036040 con 0x7f3b0c0c16d0 2024-04-25T01:26:07.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7fac8d7e4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac8000ea00 0x7fac801754d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7faf89ffb640 1 -- 172.21.15.112:0/1509197802 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c005220 con 0x7faf90046c10 2024-04-25T01:26:07.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7faf89ffb640 1 -- 172.21.15.112:0/1509197802 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf8c0053c0 con 0x7faf90046c10 2024-04-25T01:26:07.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7faf89ffb640 1 -- 172.21.15.112:0/1509197802 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c007230 con 0x7faf90046c10 2024-04-25T01:26:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7faf9e651640 1 -- 172.21.15.112:0/1509197802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf901712c0 con 0x7faf90046c10 2024-04-25T01:26:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7fac8d7e4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac8004dae0 0x7fac80046cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7fac8d7e4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac800dbe80 0x7fac800e0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7fac8d7e4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac8004ae00 con 0x7fac8000ea00 2024-04-25T01:26:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7fac8d7e4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fac8000e4e0 con 0x7fac8004dae0 2024-04-25T01:26:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7fac8d7e4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fac800078c0 con 0x7fac800dbe80 2024-04-25T01:26:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7fac87fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac8000ea00 0x7fac801754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7f0ad20ff640 1 -- start start 2024-04-25T01:26:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7f0ad20ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ac40cd020 0x7f0ac40cf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7f0ad20ff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac40cf8c0 0x7f0ac40cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7f0ad20ff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ac40cd940 0x7f0ac41743d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7f0ad20ff640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac4006f10 con 0x7f0ac40cd020 2024-04-25T01:26:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7f0ad20ff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac404c2a0 con 0x7f0ac40cd940 2024-04-25T01:26:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7f0ad20ff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac400f4c0 con 0x7f0ac40cf8c0 2024-04-25T01:26:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7fd5782bd640 1 -- 172.21.15.112:0/3875146841 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57014d2b0 con 0x7fd570152fa0 2024-04-25T01:26:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7fd5782bd640 1 -- 172.21.15.112:0/3875146841 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd570151ac0 con 0x7fd570152fa0 2024-04-25T01:26:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7fd6be7fc640 1 --2- 172.21.15.112:0/99841031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6c015a470 0x7fd6c01f5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7fd6be7fc640 1 -- 172.21.15.112:0/99841031 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c0154460 con 0x7fd6c00c6580 2024-04-25T01:26:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7fd6bf7fe640 1 --2- 172.21.15.112:0/99841031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6c015a470 0x7fd6c01f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7fd6be7fc640 1 --2- 172.21.15.112:0/99841031 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6c00c6580 0x7fd6c01f51c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800c8f0 tx=0x7fd6b800cdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7f0ad10fd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ac40cd020 0x7f0ac40cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7fd6a7fff640 1 -- 172.21.15.112:0/99841031 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8018070 con 0x7fd6c00c6580 2024-04-25T01:26:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7fd6a7fff640 1 -- 172.21.15.112:0/99841031 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b800a640 con 0x7fd6c00c6580 2024-04-25T01:26:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7fd6a7fff640 1 -- 172.21.15.112:0/99841031 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b800aa10 con 0x7fd6c00c6580 2024-04-25T01:26:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7fc626147640 1 -- 172.21.15.112:0/1770221761 wait complete. 2024-04-25T01:26:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7fc626147640 1 Processor -- start 2024-04-25T01:26:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7f1aa2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a9c00bd40 0x7f1a9c0d8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:48982/0 (socket says 172.21.15.112:48982) 2024-04-25T01:26:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7f1aa2ffd640 1 -- 172.21.15.112:0/3782953576 learned_addr learned my addr 172.21.15.112:0/3782953576 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7f1aa3fff640 1 --2- 172.21.15.112:0/3782953576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a9c04b100 0x7f1a9c0d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.883+0000 7f1aa37fe640 1 --2- 172.21.15.112:0/3782953576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a9c006e90 0x7f1a9c0d7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7f1aa3fff640 1 -- 172.21.15.112:0/3782953576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a9c006e90 msgr2=0x7f1a9c0d7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7f1aa3fff640 1 --2- 172.21.15.112:0/3782953576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a9c006e90 0x7f1a9c0d7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7f1aa3fff640 1 -- 172.21.15.112:0/3782953576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a9c00bd40 msgr2=0x7f1a9c0d8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7f1aa3fff640 1 --2- 172.21.15.112:0/3782953576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a9c00bd40 0x7f1a9c0d8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7f1aa3fff640 1 -- 172.21.15.112:0/3782953576 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9c0301b0 con 0x7f1a9c04b100 2024-04-25T01:26:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7f1aa3fff640 1 --2- 172.21.15.112:0/3782953576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a9c04b100 0x7f1a9c0d2b70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1aa406a130 tx=0x7f1aa4064fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7fc626147640 1 -- start start 2024-04-25T01:26:07.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7f0ad10fd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ac40cd020 0x7f0ac40cf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43304/0 (socket says 172.21.15.112:43304) 2024-04-25T01:26:07.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7f0ad10fd640 1 -- 172.21.15.112:0/3498069269 learned_addr learned my addr 172.21.15.112:0/3498069269 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:07.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7f0ad08fc640 1 --2- 172.21.15.112:0/3498069269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac40cf8c0 0x7f0ac40cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7f0ad18fe640 1 --2- 172.21.15.112:0/3498069269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ac40cd940 0x7f0ac41743d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7fac87fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac8000ea00 0x7fac801754d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43320/0 (socket says 172.21.15.112:43320) 2024-04-25T01:26:07.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7fac87fff640 1 -- 172.21.15.112:0/3361187594 learned_addr learned my addr 172.21.15.112:0/3361187594 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:07.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7fac8cfe3640 1 --2- 172.21.15.112:0/3361187594 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac800dbe80 0x7fac800e0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7fac877fe640 1 --2- 172.21.15.112:0/3361187594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac8004dae0 0x7fac80046cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7fac87fff640 1 -- 172.21.15.112:0/3361187594 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac800dbe80 msgr2=0x7fac800e0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7fac87fff640 1 --2- 172.21.15.112:0/3361187594 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac800dbe80 0x7fac800e0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7fac87fff640 1 -- 172.21.15.112:0/3361187594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac8004dae0 msgr2=0x7fac80046cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7fac87fff640 1 --2- 172.21.15.112:0/3361187594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac8004dae0 0x7fac80046cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7fac87fff640 1 -- 172.21.15.112:0/3361187594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac800dfc80 con 0x7fac8000ea00 2024-04-25T01:26:07.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7fac87fff640 1 --2- 172.21.15.112:0/3361187594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac8000ea00 0x7fac801754d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fac74009870 tx=0x7fac740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7fd6bffff640 1 -- 172.21.15.112:0/99841031 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c00c4120 con 0x7fd6c00c6580 2024-04-25T01:26:07.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.886+0000 7fd6bffff640 1 -- 172.21.15.112:0/99841031 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c00c46e0 con 0x7fd6c00c6580 2024-04-25T01:26:07.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.886+0000 7fac84ff9640 1 -- 172.21.15.112:0/3361187594 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74041070 con 0x7fac8000ea00 2024-04-25T01:26:07.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.886+0000 7fac84ff9640 1 -- 172.21.15.112:0/3361187594 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac7403c870 con 0x7fac8000ea00 2024-04-25T01:26:07.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.886+0000 7fac8d7e4640 1 -- 172.21.15.112:0/3361187594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac80175a10 con 0x7fac8000ea00 2024-04-25T01:26:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.884+0000 7faf9e651640 1 -- 172.21.15.112:0/1509197802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf90171880 con 0x7faf90046c10 2024-04-25T01:26:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7faf7b7fe640 1 -- 172.21.15.112:0/1509197802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faf90008c60 con 0x7faf90046c10 2024-04-25T01:26:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.886+0000 7fc626147640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc60c004800 0x7fc6200c38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7f0ad08fc640 1 -- 172.21.15.112:0/3498069269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ac40cd940 msgr2=0x7f0ac41743d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7f0ad08fc640 1 --2- 172.21.15.112:0/3498069269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ac40cd940 0x7f0ac41743d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7f0ad08fc640 1 -- 172.21.15.112:0/3498069269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ac40cd020 msgr2=0x7f0ac40cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7f0ad08fc640 1 --2- 172.21.15.112:0/3498069269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ac40cd020 0x7f0ac40cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.886+0000 7f0ad08fc640 1 -- 172.21.15.112:0/3498069269 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac4175ba0 con 0x7f0ac40cf8c0 2024-04-25T01:26:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.886+0000 7f0ad10fd640 1 --2- 172.21.15.112:0/3498069269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ac40cd020 0x7f0ac40cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7fd564ff9640 1 -- 172.21.15.112:0/3875146841 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd570075980 con 0x7fd570152fa0 2024-04-25T01:26:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.886+0000 7fc626147640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6201f83e0 0x7fc62014e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.886+0000 7f0ad08fc640 1 --2- 172.21.15.112:0/3498069269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac40cf8c0 0x7f0ac40cd400 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8011520 tx=0x7f0ac80119e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.886+0000 7fc626147640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc620148410 0x7fc62014ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.886+0000 7fc626147640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc62006bba0 con 0x7fc6201f83e0 2024-04-25T01:26:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.886+0000 7fc626147640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc620151b10 con 0x7fc620148410 2024-04-25T01:26:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.886+0000 7fc626147640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc62006bd20 con 0x7fc60c004800 2024-04-25T01:26:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7fc625145640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc60c004800 0x7fc6200c38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7fd6bffff640 1 -- 172.21.15.112:0/99841031 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd6c004f500 con 0x7fd6c00c6580 2024-04-25T01:26:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7f0ac1ffb640 1 -- 172.21.15.112:0/3498069269 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8019070 con 0x7f0ac40cf8c0 2024-04-25T01:26:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7fc625145640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc60c004800 0x7fc6200c38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:56490/0 (socket says 172.21.15.112:56490) 2024-04-25T01:26:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7fc625145640 1 -- 172.21.15.112:0/642178130 learned_addr learned my addr 172.21.15.112:0/642178130 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7fc625946640 1 --2- 172.21.15.112:0/642178130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc620148410 0x7fc62014ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7f0ad20ff640 1 -- 172.21.15.112:0/3498069269 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac40d0280 con 0x7f0ac40cf8c0 2024-04-25T01:26:07.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7f0ac1ffb640 1 -- 172.21.15.112:0/3498069269 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ac8004440 con 0x7f0ac40cf8c0 2024-04-25T01:26:07.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7f0ac1ffb640 1 -- 172.21.15.112:0/3498069269 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8003830 con 0x7f0ac40cf8c0 2024-04-25T01:26:07.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7f0ad20ff640 1 -- 172.21.15.112:0/3498069269 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac40d0840 con 0x7f0ac40cf8c0 2024-04-25T01:26:07.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7f3b02ffd640 1 -- 172.21.15.112:0/3524113971 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3b04044020 con 0x7f3b0c0c16d0 2024-04-25T01:26:07.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7f3b02ffd640 1 --2- 172.21.15.112:0/3524113971 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3af006bf00 0x7f3af006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7f3b02ffd640 1 -- 172.21.15.112:0/3524113971 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f3b040abbe0 con 0x7f3b0c0c16d0 2024-04-25T01:26:07.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7f1a8ffff640 1 -- 172.21.15.112:0/3782953576 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa40bd020 con 0x7f1a9c04b100 2024-04-25T01:26:07.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7f1a8ffff640 1 -- 172.21.15.112:0/3782953576 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa40afc30 con 0x7f1a9c04b100 2024-04-25T01:26:07.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.885+0000 7f1a8ffff640 1 -- 172.21.15.112:0/3782953576 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa40b3040 con 0x7f1a9c04b100 2024-04-25T01:26:07.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7fc625946640 1 -- 172.21.15.112:0/642178130 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc60c004800 msgr2=0x7fc6200c38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7fc625946640 1 --2- 172.21.15.112:0/642178130 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc60c004800 0x7fc6200c38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7fc625946640 1 -- 172.21.15.112:0/642178130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6201f83e0 msgr2=0x7fc62014e8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:26:07.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7fc625946640 1 --2- 172.21.15.112:0/642178130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6201f83e0 0x7fc62014e8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7fc625946640 1 -- 172.21.15.112:0/642178130 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62014d0e0 con 0x7fc620148410 2024-04-25T01:26:07.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7f3b13597640 1 --2- 172.21.15.112:0/3524113971 >> [v2:172.21.15.112:6816/2045679399,v1:172.21.15.112:6817/2045679399] conn(0x7f3af4001650 0x7f3af4003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.888+0000 7f3b13597640 1 -- 172.21.15.112:0/3524113971 --> [v2:172.21.15.112:6816/2045679399,v1:172.21.15.112:6817/2045679399] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3af4006bb0 con 0x7f3af4001650 2024-04-25T01:26:07.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.888+0000 7f1aaa17e640 1 -- 172.21.15.112:0/3782953576 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a9c030450 con 0x7f1a9c04b100 2024-04-25T01:26:07.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.888+0000 7fc625946640 1 --2- 172.21.15.112:0/642178130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc620148410 0x7fc62014ee10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc62014c0d0 tx=0x7fc618007a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.888+0000 7f3b11d94640 1 --2- 172.21.15.112:0/3524113971 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3af006bf00 0x7f3af006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.888+0000 7f1aaa17e640 1 -- 172.21.15.112:0/3782953576 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a9c0d8dc0 con 0x7f1a9c04b100 2024-04-25T01:26:07.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.888+0000 7fc615ffb640 1 -- 172.21.15.112:0/642178130 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61800aaf0 con 0x7fc620148410 2024-04-25T01:26:07.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.888+0000 7fc626147640 1 -- 172.21.15.112:0/642178130 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6200c44b0 con 0x7fc620148410 2024-04-25T01:26:07.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.888+0000 7f3b12d96640 1 --2- 172.21.15.112:0/3524113971 >> [v2:172.21.15.112:6816/2045679399,v1:172.21.15.112:6817/2045679399] conn(0x7f3af4001650 0x7f3af4003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.888+0000 7f0ad20ff640 1 -- 172.21.15.112:0/3498069269 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0aac001030 con 0x7f0ac40cf8c0 2024-04-25T01:26:07.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.888+0000 7fc626147640 1 -- 172.21.15.112:0/642178130 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6201f7be0 con 0x7fc620148410 2024-04-25T01:26:07.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.888+0000 7fd6a7fff640 1 -- 172.21.15.112:0/99841031 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd6b801b070 con 0x7fd6c00c6580 2024-04-25T01:26:07.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.888+0000 7fc615ffb640 1 -- 172.21.15.112:0/642178130 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc618018070 con 0x7fc620148410 2024-04-25T01:26:07.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.888+0000 7fc615ffb640 1 -- 172.21.15.112:0/642178130 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6180133f0 con 0x7fc620148410 2024-04-25T01:26:07.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7fac8d7e4640 1 -- 172.21.15.112:0/3361187594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac800dff20 con 0x7fac8000ea00 2024-04-25T01:26:07.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.888+0000 7fd6a7fff640 1 --2- 172.21.15.112:0/99841031 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd6ac06bf00 0x7fd6ac06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.889+0000 7fd6a7fff640 1 -- 172.21.15.112:0/99841031 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fd6b808ade0 con 0x7fd6c00c6580 2024-04-25T01:26:07.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.889+0000 7fd6a7fff640 1 --2- 172.21.15.112:0/99841031 >> [v2:172.21.15.112:6800/1032830648,v1:172.21.15.112:6801/1032830648] conn(0x7fd6ac076790 0x7fd6ac078bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.889+0000 7fd6a7fff640 1 -- 172.21.15.112:0/99841031 --> [v2:172.21.15.112:6800/1032830648,v1:172.21.15.112:6801/1032830648] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd6b8015ec0 con 0x7fd6ac076790 2024-04-25T01:26:07.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.889+0000 7fd6a7fff640 1 -- 172.21.15.112:0/99841031 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd6b808b1d0 con 0x7fd6c00c6580 2024-04-25T01:26:07.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.889+0000 7f1aaa17e640 1 -- 172.21.15.112:0/3782953576 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1a9c005380 con 0x7f1a9c04b100 2024-04-25T01:26:07.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7fd566ffd640 1 -- 172.21.15.112:0/3875146841 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd560018020 con 0x7fd570152fa0 2024-04-25T01:26:07.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.887+0000 7fd566ffd640 1 --2- 172.21.15.112:0/3875146841 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd55406bf00 0x7fd55406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.888+0000 7fd576059640 1 --2- 172.21.15.112:0/3875146841 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd55406bf00 0x7fd55406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.889+0000 7fd6beffd640 1 --2- 172.21.15.112:0/99841031 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd6ac06bf00 0x7fd6ac06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.889+0000 7fd566ffd640 1 -- 172.21.15.112:0/3875146841 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fd56008aee0 con 0x7fd570152fa0 2024-04-25T01:26:07.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.889+0000 7fd566ffd640 1 --2- 172.21.15.112:0/3875146841 >> [v2:172.21.15.187:6808/931860956,v1:172.21.15.187:6809/931860956] conn(0x7fd5540767a0 0x7fd554078bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.889+0000 7fd566ffd640 1 -- 172.21.15.112:0/3875146841 --> [v2:172.21.15.187:6808/931860956,v1:172.21.15.187:6809/931860956] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd560004450 con 0x7fd5540767a0 2024-04-25T01:26:07.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.889+0000 7fd566ffd640 1 -- 172.21.15.112:0/3875146841 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd56008d540 con 0x7fd570152fa0 2024-04-25T01:26:07.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.889+0000 7fc626147640 1 -- 172.21.15.112:0/642178130 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc600001030 con 0x7fc620148410 2024-04-25T01:26:07.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.889+0000 7faf89ffb640 1 -- 172.21.15.112:0/1509197802 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faf8c019020 con 0x7faf90046c10 2024-04-25T01:26:07.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.890+0000 7fd57685a640 1 --2- 172.21.15.112:0/3875146841 >> [v2:172.21.15.187:6808/931860956,v1:172.21.15.187:6809/931860956] conn(0x7fd5540767a0 0x7fd554078bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.889+0000 7faf89ffb640 1 --2- 172.21.15.112:0/1509197802 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faf7c06bf00 0x7faf7c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.889+0000 7fd6bf7fe640 1 --2- 172.21.15.112:0/99841031 >> [v2:172.21.15.112:6800/1032830648,v1:172.21.15.112:6801/1032830648] conn(0x7fd6ac076790 0x7fd6ac078bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.889+0000 7faf89ffb640 1 -- 172.21.15.112:0/1509197802 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7faf8c089ca0 con 0x7faf90046c10 2024-04-25T01:26:07.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.890+0000 7faf9d64f640 1 --2- 172.21.15.112:0/1509197802 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faf7c06bf00 0x7faf7c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.890+0000 7faf89ffb640 1 --2- 172.21.15.112:0/1509197802 >> [v2:172.21.15.112:6824/2172872078,v1:172.21.15.112:6825/2172872078] conn(0x7faf7c076790 0x7faf7c078bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.890+0000 7faf89ffb640 1 -- 172.21.15.112:0/1509197802 --> [v2:172.21.15.112:6824/2172872078,v1:172.21.15.112:6825/2172872078] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faf7c079260 con 0x7faf7c076790 2024-04-25T01:26:07.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.890+0000 7faf89ffb640 1 -- 172.21.15.112:0/1509197802 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faf8c089fe0 con 0x7faf90046c10 2024-04-25T01:26:07.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.888+0000 7f3b12d96640 1 --2- 172.21.15.112:0/3524113971 >> [v2:172.21.15.112:6816/2045679399,v1:172.21.15.112:6817/2045679399] conn(0x7f3af4001650 0x7f3af4003b10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.889+0000 7f3b11d94640 1 --2- 172.21.15.112:0/3524113971 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3af006bf00 0x7f3af006e3c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3afc007fd0 tx=0x7f3afc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.890+0000 7fd6bf7fe640 1 --2- 172.21.15.112:0/99841031 >> [v2:172.21.15.112:6800/1032830648,v1:172.21.15.112:6801/1032830648] conn(0x7fd6ac076790 0x7fd6ac078bb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.890+0000 7fac84ff9640 1 -- 172.21.15.112:0/3361187594 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74037040 con 0x7fac8000ea00 2024-04-25T01:26:07.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.890+0000 7fac727fc640 1 -- 172.21.15.112:0/3361187594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fac68001030 con 0x7fac8000ea00 2024-04-25T01:26:07.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.890+0000 7fd6beffd640 1 --2- 172.21.15.112:0/99841031 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd6ac06bf00 0x7fd6ac06e3c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0034000 tx=0x7fd6b002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.890+0000 7fac84ff9640 1 -- 172.21.15.112:0/3361187594 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fac74044070 con 0x7fac8000ea00 2024-04-25T01:26:07.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.890+0000 7faf9d64f640 1 --2- 172.21.15.112:0/1509197802 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faf7c06bf00 0x7faf7c06e3c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faf9402a290 tx=0x7faf9402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.890+0000 7fd576059640 1 --2- 172.21.15.112:0/3875146841 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd55406bf00 0x7fd55406e3c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd56800c570 tx=0x7fd56800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.891+0000 7f0ac1ffb640 1 -- 172.21.15.112:0/3498069269 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ac801c070 con 0x7f0ac40cf8c0 2024-04-25T01:26:07.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.891+0000 7faf9de50640 1 --2- 172.21.15.112:0/1509197802 >> [v2:172.21.15.112:6824/2172872078,v1:172.21.15.112:6825/2172872078] conn(0x7faf7c076790 0x7faf7c078bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.890+0000 7f3b02ffd640 1 -- 172.21.15.112:0/3524113971 <== osd.0 v2:172.21.15.112:6816/2045679399 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3af4006bb0 con 0x7f3af4001650 2024-04-25T01:26:07.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.891+0000 7fc615ffb640 1 -- 172.21.15.112:0/642178130 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc618013590 con 0x7fc620148410 2024-04-25T01:26:07.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.891+0000 7fac84ff9640 1 --2- 172.21.15.112:0/3361187594 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fac6c06beb0 0x7fac6c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.891+0000 7faf9de50640 1 --2- 172.21.15.112:0/1509197802 >> [v2:172.21.15.112:6824/2172872078,v1:172.21.15.112:6825/2172872078] conn(0x7faf7c076790 0x7faf7c078bb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.891+0000 7fac877fe640 1 --2- 172.21.15.112:0/3361187594 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fac6c06beb0 0x7fac6c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.891+0000 7fac84ff9640 1 -- 172.21.15.112:0/3361187594 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fac740abc40 con 0x7fac8000ea00 2024-04-25T01:26:07.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.891+0000 7f0ac1ffb640 1 --2- 172.21.15.112:0/3498069269 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0ab006be60 0x7f0ab006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.891+0000 7fd6a7fff640 1 -- 172.21.15.112:0/99841031 <== osd.1 v2:172.21.15.112:6800/1032830648 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd6b8015ec0 con 0x7fd6ac076790 2024-04-25T01:26:07.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.891+0000 7fac84ff9640 1 --2- 172.21.15.112:0/3361187594 >> [v2:172.21.15.187:6800/876892580,v1:172.21.15.187:6801/876892580] conn(0x7fac6c0766c0 0x7fac6c078ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.891+0000 7fac84ff9640 1 -- 172.21.15.112:0/3361187594 --> [v2:172.21.15.187:6800/876892580,v1:172.21.15.187:6801/876892580] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac74032440 con 0x7fac6c0766c0 2024-04-25T01:26:07.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7fac8cfe3640 1 --2- 172.21.15.112:0/3361187594 >> [v2:172.21.15.187:6800/876892580,v1:172.21.15.187:6801/876892580] conn(0x7fac6c0766c0 0x7fac6c078ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7fac84ff9640 1 -- 172.21.15.112:0/3361187594 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fac74076110 con 0x7fac8000ea00 2024-04-25T01:26:07.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.891+0000 7f0ac1ffb640 1 -- 172.21.15.112:0/3498069269 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f0ac808b390 con 0x7f0ac40cf8c0 2024-04-25T01:26:07.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7f0ac1ffb640 1 --2- 172.21.15.112:0/3498069269 >> [v2:172.21.15.187:6824/1463958731,v1:172.21.15.187:6825/1463958731] conn(0x7f0ab0076700 0x7f0ab0078b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7f0ac1ffb640 1 -- 172.21.15.112:0/3498069269 --> [v2:172.21.15.187:6824/1463958731,v1:172.21.15.187:6825/1463958731] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ac8012e90 con 0x7f0ab0076700 2024-04-25T01:26:07.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7f0ac1ffb640 1 -- 172.21.15.112:0/3498069269 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0ac8092050 con 0x7f0ac40cf8c0 2024-04-25T01:26:07.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7fd57685a640 1 --2- 172.21.15.112:0/3875146841 >> [v2:172.21.15.187:6808/931860956,v1:172.21.15.187:6809/931860956] conn(0x7fd5540767a0 0x7fd554078bc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7fc615ffb640 1 --2- 172.21.15.112:0/642178130 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc60406bf30 0x7fc60406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7f0ad10fd640 1 --2- 172.21.15.112:0/3498069269 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0ab006be60 0x7f0ab006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7f0ad18fe640 1 --2- 172.21.15.112:0/3498069269 >> [v2:172.21.15.187:6824/1463958731,v1:172.21.15.187:6825/1463958731] conn(0x7f0ab0076700 0x7f0ab0078b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7fc615ffb640 1 -- 172.21.15.112:0/642178130 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fc61808b380 con 0x7fc620148410 2024-04-25T01:26:07.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7fc625145640 1 --2- 172.21.15.112:0/642178130 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc60406bf30 0x7fc60406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7fc615ffb640 1 --2- 172.21.15.112:0/642178130 >> [v2:172.21.15.187:6816/155270368,v1:172.21.15.187:6817/155270368] conn(0x7fc6040767c0 0x7fc604078be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7fc615ffb640 1 -- 172.21.15.112:0/642178130 --> [v2:172.21.15.187:6816/155270368,v1:172.21.15.187:6817/155270368] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc618021ec0 con 0x7fc6040767c0 2024-04-25T01:26:07.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7fc615ffb640 1 -- 172.21.15.112:0/642178130 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc618092050 con 0x7fc620148410 2024-04-25T01:26:07.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7faf89ffb640 1 -- 172.21.15.112:0/1509197802 <== osd.2 v2:172.21.15.112:6824/2172872078 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faf7c079260 con 0x7faf7c076790 2024-04-25T01:26:07.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7fc624944640 1 --2- 172.21.15.112:0/642178130 >> [v2:172.21.15.187:6816/155270368,v1:172.21.15.187:6817/155270368] conn(0x7fc6040767c0 0x7fc604078be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.893+0000 7fac8cfe3640 1 --2- 172.21.15.112:0/3361187594 >> [v2:172.21.15.187:6800/876892580,v1:172.21.15.187:6801/876892580] conn(0x7fac6c0766c0 0x7fac6c078ae0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7f1a8ffff640 1 -- 172.21.15.112:0/3782953576 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1aa40c0020 con 0x7f1a9c04b100 2024-04-25T01:26:07.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7f1a8ffff640 1 --2- 172.21.15.112:0/3782953576 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1a8806bf00 0x7f1a8806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7f1a8ffff640 1 -- 172.21.15.112:0/3782953576 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f1aa4127270 con 0x7f1a9c04b100 2024-04-25T01:26:07.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7f1aa37fe640 1 --2- 172.21.15.112:0/3782953576 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1a8806bf00 0x7f1a8806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.892+0000 7f1a8ffff640 1 --2- 172.21.15.112:0/3782953576 >> [v2:172.21.15.112:6808/1418784374,v1:172.21.15.112:6809/1418784374] conn(0x7f1a88076790 0x7f1a88078bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:07.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.893+0000 7f1a8ffff640 1 -- 172.21.15.112:0/3782953576 --> [v2:172.21.15.112:6808/1418784374,v1:172.21.15.112:6809/1418784374] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a88079260 con 0x7f1a88076790 2024-04-25T01:26:07.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.893+0000 7f1a8ffff640 1 -- 172.21.15.112:0/3782953576 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1aa412e050 con 0x7f1a9c04b100 2024-04-25T01:26:07.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.893+0000 7fac877fe640 1 --2- 172.21.15.112:0/3361187594 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fac6c06beb0 0x7fac6c06e370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fac8804fa20 tx=0x7fac88069c30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.893+0000 7fd566ffd640 1 -- 172.21.15.112:0/3875146841 <== osd.6 v2:172.21.15.187:6808/931860956 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd560004450 con 0x7fd5540767a0 2024-04-25T01:26:07.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.893+0000 7f0ad10fd640 1 --2- 172.21.15.112:0/3498069269 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0ab006be60 0x7f0ab006e320 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0abc004840 tx=0x7f0abc007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.893+0000 7fc624944640 1 --2- 172.21.15.112:0/642178130 >> [v2:172.21.15.187:6816/155270368,v1:172.21.15.187:6817/155270368] conn(0x7fc6040767c0 0x7fc604078be0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.893+0000 7f1aa2ffd640 1 --2- 172.21.15.112:0/3782953576 >> [v2:172.21.15.112:6808/1418784374,v1:172.21.15.112:6809/1418784374] conn(0x7f1a88076790 0x7f1a88078bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:07.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.893+0000 7f0ad18fe640 1 --2- 172.21.15.112:0/3498069269 >> [v2:172.21.15.187:6824/1463958731,v1:172.21.15.187:6825/1463958731] conn(0x7f0ab0076700 0x7f0ab0078b20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.893+0000 7fc625145640 1 --2- 172.21.15.112:0/642178130 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc60406bf30 0x7fc60406e3f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc6100047b0 tx=0x7fc610009590 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.893+0000 7fac84ff9640 1 -- 172.21.15.112:0/3361187594 <== osd.4 v2:172.21.15.187:6800/876892580 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fac74032440 con 0x7fac6c0766c0 2024-04-25T01:26:07.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.894+0000 7f1aa37fe640 1 --2- 172.21.15.112:0/3782953576 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1a8806bf00 0x7f1a8806e3c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1a98006cc0 tx=0x7f1a98009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.894+0000 7f1aa2ffd640 1 --2- 172.21.15.112:0/3782953576 >> [v2:172.21.15.112:6808/1418784374,v1:172.21.15.112:6809/1418784374] conn(0x7f1a88076790 0x7f1a88078bb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:07.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.894+0000 7fc615ffb640 1 -- 172.21.15.112:0/642178130 <== osd.7 v2:172.21.15.187:6816/155270368 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc618021ec0 con 0x7fc6040767c0 2024-04-25T01:26:07.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.895+0000 7f0ac1ffb640 1 -- 172.21.15.112:0/3498069269 <== osd.5 v2:172.21.15.187:6824/1463958731 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0ac8012e90 con 0x7f0ab0076700 2024-04-25T01:26:07.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.895+0000 7f1a8ffff640 1 -- 172.21.15.112:0/3782953576 <== osd.3 v2:172.21.15.112:6808/1418784374 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1a88079260 con 0x7f1a88076790 2024-04-25T01:26:07.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.922+0000 7f3b13597640 1 -- 172.21.15.112:0/3524113971 --> [v2:172.21.15.112:6816/2045679399,v1:172.21.15.112:6817/2045679399] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3af4005c90 con 0x7f3af4001650 2024-04-25T01:26:07.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.923+0000 7f3b02ffd640 1 -- 172.21.15.112:0/3524113971 <== osd.0 v2:172.21.15.112:6816/2045679399 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3af4005c90 con 0x7f3af4001650 2024-04-25T01:26:07.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.923+0000 7f3b00ff9640 1 -- 172.21.15.112:0/3524113971 >> [v2:172.21.15.112:6816/2045679399,v1:172.21.15.112:6817/2045679399] conn(0x7f3af4001650 msgr2=0x7f3af4003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.924+0000 7f3b00ff9640 1 --2- 172.21.15.112:0/3524113971 >> [v2:172.21.15.112:6816/2045679399,v1:172.21.15.112:6817/2045679399] conn(0x7f3af4001650 0x7f3af4003b10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.924+0000 7f3b00ff9640 1 -- 172.21.15.112:0/3524113971 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3af006bf00 msgr2=0x7f3af006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.924+0000 7f3b00ff9640 1 --2- 172.21.15.112:0/3524113971 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3af006bf00 0x7f3af006e3c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3afc007fd0 tx=0x7f3afc009040 comp rx=0 tx=0).stop 2024-04-25T01:26:07.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.924+0000 7f3b00ff9640 1 -- 172.21.15.112:0/3524113971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b0c0c16d0 msgr2=0x7f3b0c15b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.924+0000 7f3b00ff9640 1 --2- 172.21.15.112:0/3524113971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b0c0c16d0 0x7f3b0c15b210 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3b04000f80 tx=0x7f3b04004a40 comp rx=0 tx=0).stop 2024-04-25T01:26:07.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.924+0000 7f3b12d96640 1 -- 172.21.15.112:0/3524113971 reap_dead start 2024-04-25T01:26:07.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.924+0000 7f3b00ff9640 1 -- 172.21.15.112:0/3524113971 shutdown_connections 2024-04-25T01:26:07.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.924+0000 7fd6bffff640 1 -- 172.21.15.112:0/99841031 --> [v2:172.21.15.112:6800/1032830648,v1:172.21.15.112:6801/1032830648] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd6c00c69b0 con 0x7fd6ac076790 2024-04-25T01:26:07.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.924+0000 7f3b00ff9640 1 -- 172.21.15.112:0/3524113971 >> 172.21.15.112:0/3524113971 conn(0x7f3b0c0ad1c0 msgr2=0x7f3b0c152010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.925+0000 7f3b00ff9640 1 -- 172.21.15.112:0/3524113971 shutdown_connections 2024-04-25T01:26:07.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.925+0000 7f3b00ff9640 1 -- 172.21.15.112:0/3524113971 wait complete. 2024-04-25T01:26:07.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.925+0000 7fd6a7fff640 1 -- 172.21.15.112:0/99841031 <== osd.1 v2:172.21.15.112:6800/1032830648 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd6c00c69b0 con 0x7fd6ac076790 2024-04-25T01:26:07.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.925+0000 7fd6bffff640 1 -- 172.21.15.112:0/99841031 >> [v2:172.21.15.112:6800/1032830648,v1:172.21.15.112:6801/1032830648] conn(0x7fd6ac076790 msgr2=0x7fd6ac078bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.925+0000 7fd6bffff640 1 --2- 172.21.15.112:0/99841031 >> [v2:172.21.15.112:6800/1032830648,v1:172.21.15.112:6801/1032830648] conn(0x7fd6ac076790 0x7fd6ac078bb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.926+0000 7faf7b7fe640 1 -- 172.21.15.112:0/1509197802 --> [v2:172.21.15.112:6824/2172872078,v1:172.21.15.112:6825/2172872078] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faf90005380 con 0x7faf7c076790 2024-04-25T01:26:07.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.926+0000 7fd6bffff640 1 -- 172.21.15.112:0/99841031 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd6ac06bf00 msgr2=0x7fd6ac06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.926+0000 7fd6bffff640 1 --2- 172.21.15.112:0/99841031 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd6ac06bf00 0x7fd6ac06e3c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0034000 tx=0x7fd6b002e000 comp rx=0 tx=0).stop 2024-04-25T01:26:07.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.926+0000 7fd6bffff640 1 -- 172.21.15.112:0/99841031 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6c00c6580 msgr2=0x7fd6c01f51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.926+0000 7fd6bffff640 1 --2- 172.21.15.112:0/99841031 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6c00c6580 0x7fd6c01f51c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800c8f0 tx=0x7fd6b800cdb0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.926+0000 7fd6bf7fe640 1 -- 172.21.15.112:0/99841031 reap_dead start 2024-04-25T01:26:07.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.926+0000 7fac727fc640 1 -- 172.21.15.112:0/3361187594 --> [v2:172.21.15.187:6800/876892580,v1:172.21.15.187:6801/876892580] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fac68002dd0 con 0x7fac6c0766c0 2024-04-25T01:26:07.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.926+0000 7fd6bffff640 1 -- 172.21.15.112:0/99841031 shutdown_connections 2024-04-25T01:26:07.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.926+0000 7fd6bffff640 1 -- 172.21.15.112:0/99841031 >> 172.21.15.112:0/99841031 conn(0x7fd6c00ad1c0 msgr2=0x7fd6c00c7520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.926+0000 7f0ad20ff640 1 -- 172.21.15.112:0/3498069269 --> [v2:172.21.15.187:6824/1463958731,v1:172.21.15.187:6825/1463958731] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0aac002dd0 con 0x7f0ab0076700 2024-04-25T01:26:07.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.926+0000 7faf89ffb640 1 -- 172.21.15.112:0/1509197802 <== osd.2 v2:172.21.15.112:6824/2172872078 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faf90005380 con 0x7faf7c076790 2024-04-25T01:26:07.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.926+0000 7fd6bffff640 1 -- 172.21.15.112:0/99841031 shutdown_connections 2024-04-25T01:26:07.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.926+0000 7fd6bffff640 1 -- 172.21.15.112:0/99841031 wait complete. 2024-04-25T01:26:07.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.927+0000 7fc626147640 1 -- 172.21.15.112:0/642178130 --> [v2:172.21.15.187:6816/155270368,v1:172.21.15.187:6817/155270368] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc600002dd0 con 0x7fc6040767c0 2024-04-25T01:26:07.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.927+0000 7fac84ff9640 1 -- 172.21.15.112:0/3361187594 <== osd.4 v2:172.21.15.187:6800/876892580 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fac68002dd0 con 0x7fac6c0766c0 2024-04-25T01:26:07.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.927+0000 7f0ac1ffb640 1 -- 172.21.15.112:0/3498069269 <== osd.5 v2:172.21.15.187:6824/1463958731 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0aac002dd0 con 0x7f0ab0076700 2024-04-25T01:26:07.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.927+0000 7faf7b7fe640 1 -- 172.21.15.112:0/1509197802 >> [v2:172.21.15.112:6824/2172872078,v1:172.21.15.112:6825/2172872078] conn(0x7faf7c076790 msgr2=0x7faf7c078bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.927+0000 7faf7b7fe640 1 --2- 172.21.15.112:0/1509197802 >> [v2:172.21.15.112:6824/2172872078,v1:172.21.15.112:6825/2172872078] conn(0x7faf7c076790 0x7faf7c078bb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.927+0000 7faf7b7fe640 1 -- 172.21.15.112:0/1509197802 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faf7c06bf00 msgr2=0x7faf7c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.927+0000 7faf7b7fe640 1 --2- 172.21.15.112:0/1509197802 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faf7c06bf00 0x7faf7c06e3c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faf9402a290 tx=0x7faf9402e000 comp rx=0 tx=0).stop 2024-04-25T01:26:07.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.927+0000 7faf7b7fe640 1 -- 172.21.15.112:0/1509197802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf90046c10 msgr2=0x7faf90041e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.927+0000 7faf7b7fe640 1 --2- 172.21.15.112:0/1509197802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf90046c10 0x7faf90041e40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00c390 tx=0x7faf8c00c850 comp rx=0 tx=0).stop 2024-04-25T01:26:07.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.927+0000 7fac727fc640 1 -- 172.21.15.112:0/3361187594 >> [v2:172.21.15.187:6800/876892580,v1:172.21.15.187:6801/876892580] conn(0x7fac6c0766c0 msgr2=0x7fac6c078ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.927+0000 7fac727fc640 1 --2- 172.21.15.112:0/3361187594 >> [v2:172.21.15.187:6800/876892580,v1:172.21.15.187:6801/876892580] conn(0x7fac6c0766c0 0x7fac6c078ae0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.927+0000 7f0ad20ff640 1 -- 172.21.15.112:0/3498069269 >> [v2:172.21.15.187:6824/1463958731,v1:172.21.15.187:6825/1463958731] conn(0x7f0ab0076700 msgr2=0x7f0ab0078b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.927+0000 7f0ad20ff640 1 --2- 172.21.15.112:0/3498069269 >> [v2:172.21.15.187:6824/1463958731,v1:172.21.15.187:6825/1463958731] conn(0x7f0ab0076700 0x7f0ab0078b20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.927+0000 7fc615ffb640 1 -- 172.21.15.112:0/642178130 <== osd.7 v2:172.21.15.187:6816/155270368 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc600002dd0 con 0x7fc6040767c0 2024-04-25T01:26:07.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.927+0000 7faf9de50640 1 -- 172.21.15.112:0/1509197802 reap_dead start 2024-04-25T01:26:07.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.927+0000 7faf7b7fe640 1 -- 172.21.15.112:0/1509197802 shutdown_connections 2024-04-25T01:26:07.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.927+0000 7faf7b7fe640 1 -- 172.21.15.112:0/1509197802 >> 172.21.15.112:0/1509197802 conn(0x7faf90048240 msgr2=0x7faf900d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.927+0000 7fd564ff9640 1 -- 172.21.15.112:0/3875146841 --> [v2:172.21.15.187:6808/931860956,v1:172.21.15.187:6809/931860956] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd5700689b0 con 0x7fd5540767a0 2024-04-25T01:26:07.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.927+0000 7fac727fc640 1 -- 172.21.15.112:0/3361187594 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fac6c06beb0 msgr2=0x7fac6c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7fac727fc640 1 --2- 172.21.15.112:0/3361187594 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fac6c06beb0 0x7fac6c06e370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fac8804fa20 tx=0x7fac88069c30 comp rx=0 tx=0).stop 2024-04-25T01:26:07.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7fac727fc640 1 -- 172.21.15.112:0/3361187594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac8000ea00 msgr2=0x7fac801754d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7fac727fc640 1 --2- 172.21.15.112:0/3361187594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac8000ea00 0x7fac801754d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fac74009870 tx=0x7fac740046c0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7fac8cfe3640 1 -- 172.21.15.112:0/3361187594 reap_dead start 2024-04-25T01:26:07.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7faf7b7fe640 1 -- 172.21.15.112:0/1509197802 shutdown_connections 2024-04-25T01:26:07.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7f1aaa17e640 1 -- 172.21.15.112:0/3782953576 --> [v2:172.21.15.112:6808/1418784374,v1:172.21.15.112:6809/1418784374] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1a9c000e80 con 0x7f1a88076790 2024-04-25T01:26:07.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7f0ad20ff640 1 -- 172.21.15.112:0/3498069269 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0ab006be60 msgr2=0x7f0ab006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7f0ad20ff640 1 --2- 172.21.15.112:0/3498069269 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0ab006be60 0x7f0ab006e320 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0abc004840 tx=0x7f0abc007000 comp rx=0 tx=0).stop 2024-04-25T01:26:07.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7f0ad20ff640 1 -- 172.21.15.112:0/3498069269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac40cf8c0 msgr2=0x7f0ac40cd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7f0ad20ff640 1 --2- 172.21.15.112:0/3498069269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac40cf8c0 0x7f0ac40cd400 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8011520 tx=0x7f0ac80119e0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7f0ad18fe640 1 -- 172.21.15.112:0/3498069269 reap_dead start 2024-04-25T01:26:07.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7fac727fc640 1 -- 172.21.15.112:0/3361187594 shutdown_connections 2024-04-25T01:26:07.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7fac727fc640 1 -- 172.21.15.112:0/3361187594 >> 172.21.15.112:0/3361187594 conn(0x7fac8004c660 msgr2=0x7fac80006a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7f0ad20ff640 1 -- 172.21.15.112:0/3498069269 shutdown_connections 2024-04-25T01:26:07.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7f0ad20ff640 1 -- 172.21.15.112:0/3498069269 >> 172.21.15.112:0/3498069269 conn(0x7f0ac404fc00 msgr2=0x7f0ac40d6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7faf7b7fe640 1 -- 172.21.15.112:0/1509197802 wait complete. 2024-04-25T01:26:07.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7fd566ffd640 1 -- 172.21.15.112:0/3875146841 <== osd.6 v2:172.21.15.187:6808/931860956 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd5700689b0 con 0x7fd5540767a0 2024-04-25T01:26:07.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7f0ad20ff640 1 -- 172.21.15.112:0/3498069269 shutdown_connections 2024-04-25T01:26:07.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7f0ad20ff640 1 -- 172.21.15.112:0/3498069269 wait complete. 2024-04-25T01:26:07.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7fac727fc640 1 -- 172.21.15.112:0/3361187594 shutdown_connections 2024-04-25T01:26:07.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7fac727fc640 1 -- 172.21.15.112:0/3361187594 wait complete. 2024-04-25T01:26:07.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7f1a8ffff640 1 -- 172.21.15.112:0/3782953576 <== osd.3 v2:172.21.15.112:6808/1418784374 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1a9c000e80 con 0x7f1a88076790 2024-04-25T01:26:07.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7fd5782bd640 1 -- 172.21.15.112:0/3875146841 >> [v2:172.21.15.187:6808/931860956,v1:172.21.15.187:6809/931860956] conn(0x7fd5540767a0 msgr2=0x7fd554078bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7fd5782bd640 1 --2- 172.21.15.112:0/3875146841 >> [v2:172.21.15.187:6808/931860956,v1:172.21.15.187:6809/931860956] conn(0x7fd5540767a0 0x7fd554078bc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7fd5782bd640 1 -- 172.21.15.112:0/3875146841 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd55406bf00 msgr2=0x7fd55406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7fc626147640 1 -- 172.21.15.112:0/642178130 >> [v2:172.21.15.187:6816/155270368,v1:172.21.15.187:6817/155270368] conn(0x7fc6040767c0 msgr2=0x7fc604078be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.928+0000 7fc626147640 1 --2- 172.21.15.112:0/642178130 >> [v2:172.21.15.187:6816/155270368,v1:172.21.15.187:6817/155270368] conn(0x7fc6040767c0 0x7fc604078be0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7fd5782bd640 1 --2- 172.21.15.112:0/3875146841 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd55406bf00 0x7fd55406e3c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd56800c570 tx=0x7fd56800b040 comp rx=0 tx=0).stop 2024-04-25T01:26:07.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7fd5782bd640 1 -- 172.21.15.112:0/3875146841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd570152fa0 msgr2=0x7fd5701ee8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7fd5782bd640 1 --2- 172.21.15.112:0/3875146841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd570152fa0 0x7fd5701ee8b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd56000b3e0 tx=0x7fd56000b8a0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7fd57685a640 1 -- 172.21.15.112:0/3875146841 reap_dead start 2024-04-25T01:26:07.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7fc626147640 1 -- 172.21.15.112:0/642178130 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc60406bf30 msgr2=0x7fc60406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7fc626147640 1 --2- 172.21.15.112:0/642178130 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc60406bf30 0x7fc60406e3f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc6100047b0 tx=0x7fc610009590 comp rx=0 tx=0).stop 2024-04-25T01:26:07.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7fc626147640 1 -- 172.21.15.112:0/642178130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc620148410 msgr2=0x7fc62014ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7fc626147640 1 --2- 172.21.15.112:0/642178130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc620148410 0x7fc62014ee10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc62014c0d0 tx=0x7fc618007a90 comp rx=0 tx=0).stop 2024-04-25T01:26:07.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7fd5782bd640 1 -- 172.21.15.112:0/3875146841 shutdown_connections 2024-04-25T01:26:07.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7fd5782bd640 1 -- 172.21.15.112:0/3875146841 >> 172.21.15.112:0/3875146841 conn(0x7fd5700ad150 msgr2=0x7fd5700c63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7f1aaa17e640 1 -- 172.21.15.112:0/3782953576 >> [v2:172.21.15.112:6808/1418784374,v1:172.21.15.112:6809/1418784374] conn(0x7f1a88076790 msgr2=0x7f1a88078bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7fd5782bd640 1 -- 172.21.15.112:0/3875146841 shutdown_connections 2024-04-25T01:26:07.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7fd5782bd640 1 -- 172.21.15.112:0/3875146841 wait complete. 2024-04-25T01:26:07.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7fc625946640 1 -- 172.21.15.112:0/642178130 reap_dead start 2024-04-25T01:26:07.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7fc626147640 1 -- 172.21.15.112:0/642178130 shutdown_connections 2024-04-25T01:26:07.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7fc626147640 1 -- 172.21.15.112:0/642178130 >> 172.21.15.112:0/642178130 conn(0x7fc6200c7a10 msgr2=0x7fc620152280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7f1aaa17e640 1 --2- 172.21.15.112:0/3782953576 >> [v2:172.21.15.112:6808/1418784374,v1:172.21.15.112:6809/1418784374] conn(0x7f1a88076790 0x7f1a88078bb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7fc626147640 1 -- 172.21.15.112:0/642178130 shutdown_connections 2024-04-25T01:26:07.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.929+0000 7f1aaa17e640 1 -- 172.21.15.112:0/3782953576 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1a8806bf00 msgr2=0x7f1a8806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.930+0000 7f1aaa17e640 1 --2- 172.21.15.112:0/3782953576 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1a8806bf00 0x7f1a8806e3c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1a98006cc0 tx=0x7f1a98009240 comp rx=0 tx=0).stop 2024-04-25T01:26:07.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.930+0000 7f1aaa17e640 1 -- 172.21.15.112:0/3782953576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a9c04b100 msgr2=0x7f1a9c0d2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:07.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.930+0000 7f1aaa17e640 1 --2- 172.21.15.112:0/3782953576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a9c04b100 0x7f1a9c0d2b70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1aa406a130 tx=0x7f1aa4064fe0 comp rx=0 tx=0).stop 2024-04-25T01:26:07.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.930+0000 7fc626147640 1 -- 172.21.15.112:0/642178130 wait complete. 2024-04-25T01:26:07.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.930+0000 7f1aa3fff640 1 -- 172.21.15.112:0/3782953576 reap_dead start 2024-04-25T01:26:07.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.930+0000 7f1aaa17e640 1 -- 172.21.15.112:0/3782953576 shutdown_connections 2024-04-25T01:26:07.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.930+0000 7f1aaa17e640 1 -- 172.21.15.112:0/3782953576 >> 172.21.15.112:0/3782953576 conn(0x7f1a9c048240 msgr2=0x7f1a9c0dac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:07.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.930+0000 7f1aaa17e640 1 -- 172.21.15.112:0/3782953576 shutdown_connections 2024-04-25T01:26:07.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:07.930+0000 7f1aaa17e640 1 -- 172.21.15.112:0/3782953576 wait complete. 2024-04-25T01:26:07.970 INFO:teuthology.orchestra.run.smithi112.stdout:55834574856 2024-04-25T01:26:07.970 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T01:26:07.971 INFO:teuthology.orchestra.run.smithi112.stdout:55834574857 2024-04-25T01:26:07.971 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T01:26:07.972 INFO:teuthology.orchestra.run.smithi112.stdout:55834574857 2024-04-25T01:26:07.972 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T01:26:07.973 INFO:teuthology.orchestra.run.smithi112.stdout:55834574857 2024-04-25T01:26:07.973 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T01:26:07.974 INFO:teuthology.orchestra.run.smithi112.stdout:55834574857 2024-04-25T01:26:07.974 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T01:26:07.975 INFO:teuthology.orchestra.run.smithi112.stdout:55834574856 2024-04-25T01:26:07.975 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T01:26:07.976 INFO:teuthology.orchestra.run.smithi112.stdout:55834574858 2024-04-25T01:26:07.976 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T01:26:07.977 INFO:teuthology.orchestra.run.smithi112.stdout:55834574857 2024-04-25T01:26:07.977 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T01:26:08.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.193+0000 7ff4db5c1640 1 Processor -- start 2024-04-25T01:26:08.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.194+0000 7ff4db5c1640 1 -- start start 2024-04-25T01:26:08.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.195+0000 7ff4db5c1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4d414edb0 0x7ff4d40c2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.195+0000 7ff4db5c1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff4d4072ed0 0x7ff4d41516c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.195+0000 7ff4db5c1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff4d40c2ed0 0x7ff4d4151c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.195+0000 7ff4db5c1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7ff4d406efd0 con 0x7ff4d40c2ed0 2024-04-25T01:26:08.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.195+0000 7ff4db5c1640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff4d4072540 con 0x7ff4d4072ed0 2024-04-25T01:26:08.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.195+0000 7ff4db5c1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff4d41559d0 con 0x7ff4d414edb0 2024-04-25T01:26:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.195+0000 7ff4d935d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4d414edb0 0x7ff4d40c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.195+0000 7ff4d8b5c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff4d4072ed0 0x7ff4d41516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.195+0000 7ff4d9b5e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff4d40c2ed0 0x7ff4d4151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.195+0000 7ff4d9b5e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff4d40c2ed0 0x7ff4d4151c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43334/0 (socket says 172.21.15.112:43334) 2024-04-25T01:26:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.195+0000 7ff4d9b5e640 1 -- 172.21.15.112:0/1965921347 learned_addr learned my addr 172.21.15.112:0/1965921347 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.196+0000 7ff4d8b5c640 1 -- 172.21.15.112:0/1965921347 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4d414edb0 msgr2=0x7ff4d40c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.196+0000 7ff4d8b5c640 1 --2- 172.21.15.112:0/1965921347 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4d414edb0 0x7ff4d40c2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.196+0000 7ff4d8b5c640 1 -- 172.21.15.112:0/1965921347 >> v2:172.21.15.112:3300/0 conn(0x7ff4d40c2ed0 msgr2=0x7ff4d4151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.196+0000 7ff4d8b5c640 1 --2- 172.21.15.112:0/1965921347 >> v2:172.21.15.112:3300/0 conn(0x7ff4d40c2ed0 0x7ff4d4151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.196+0000 7ff4d8b5c640 1 -- 172.21.15.112:0/1965921347 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d406e4f0 con 0x7ff4d4072ed0 2024-04-25T01:26:08.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.197+0000 7f2ef1d66640 1 Processor -- start 2024-04-25T01:26:08.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.197+0000 7ff4d8b5c640 1 --2- 172.21.15.112:0/1965921347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4d4072ed0 0x7ff4d41516c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40c4a40 tx=0x7ff4cc02a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ee43f229a4a3efe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.197+0000 7f2ef1d66640 1 -- start start 2024-04-25T01:26:08.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.197+0000 7f23d359e640 1 Processor -- start 2024-04-25T01:26:08.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.198+0000 7f23d359e640 1 -- start start 2024-04-25T01:26:08.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.198+0000 7ff4cbfff640 1 -- 172.21.15.112:0/1965921347 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc036070 con 0x7ff4d4072ed0 2024-04-25T01:26:08.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.198+0000 7f1bd953c640 1 Processor -- start 2024-04-25T01:26:08.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.198+0000 7ff4cbfff640 1 -- 172.21.15.112:0/1965921347 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4cc003b90 con 0x7ff4d4072ed0 2024-04-25T01:26:08.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.198+0000 7f1bd953c640 1 -- start start 2024-04-25T01:26:08.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.198+0000 7f31b7407640 1 Processor -- start 2024-04-25T01:26:08.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.198+0000 7f2ef1d66640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2ee40474a0 0x7f2ee4006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.198+0000 7ffae759e640 1 Processor -- start 2024-04-25T01:26:08.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.198+0000 7f31b7407640 1 -- start start 2024-04-25T01:26:08.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.198+0000 7f2ef1d66640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ee400bb50 con 0x7f2ee40dbce0 2024-04-25T01:26:08.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.198+0000 7f2ef0d64640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2ee40dbce0 0x7f2ee40df180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37962/0 (socket says 172.21.15.112:37962) 2024-04-25T01:26:08.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7ff4cbfff640 1 -- 172.21.15.112:0/1965921347 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc00c040 con 0x7ff4d4072ed0 2024-04-25T01:26:08.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.198+0000 7f2ef0d64640 1 -- 172.21.15.112:0/148401649 learned_addr learned my addr 172.21.15.112:0/148401649 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:08.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7fda42ae6640 1 Processor -- start 2024-04-25T01:26:08.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7f2eebfff640 1 --2- 172.21.15.112:0/148401649 >> v2:172.21.15.112:3300/0 conn(0x7f2ee40474a0 0x7f2ee4006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7f2eeb7fe640 1 -- 172.21.15.112:0/148401649 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 466602440 0 0) 0x7f2ee400bb50 con 0x7f2ee40dbce0 2024-04-25T01:26:08.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7ff4db5c1640 1 -- 172.21.15.112:0/1965921347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4d4072ed0 msgr2=0x7ff4d41516c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7ff4db5c1640 1 --2- 172.21.15.112:0/1965921347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4d4072ed0 0x7ff4d41516c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40c4a40 tx=0x7ff4cc02a6a0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7f2ef1d66640 1 -- 172.21.15.112:0/148401649 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ee4048060 con 0x7f2ee40073e0 2024-04-25T01:26:08.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7f2ef1d66640 1 -- 172.21.15.112:0/148401649 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f2ee400a420 con 0x7f2ee40474a0 2024-04-25T01:26:08.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7fda42ae6640 1 -- start start 2024-04-25T01:26:08.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7f2eeb7fe640 1 -- 172.21.15.112:0/148401649 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2edc003520 con 0x7f2ee40dbce0 2024-04-25T01:26:08.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7ffae759e640 1 -- start start 2024-04-25T01:26:08.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7f1bd953c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1bcc0d45b0 0x7f1bcc007970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7f2eeb7fe640 1 -- 172.21.15.112:0/148401649 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4278371977 0 0) 0x7f2ee4048060 con 0x7f2ee40073e0 2024-04-25T01:26:08.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7f2eeb7fe640 1 -- 172.21.15.112:0/148401649 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ee400bb50 con 0x7f2ee40073e0 2024-04-25T01:26:08.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7ff4db5c1640 1 -- 172.21.15.112:0/1965921347 shutdown_connections 2024-04-25T01:26:08.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7ff4db5c1640 1 --2- 172.21.15.112:0/1965921347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4d4072ed0 0x7ff4d41516c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7ff4db5c1640 1 --2- 172.21.15.112:0/1965921347 >> v2:172.21.15.112:3300/0 conn(0x7ff4d40c2ed0 0x7ff4d4151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7ff4db5c1640 1 --2- 172.21.15.112:0/1965921347 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4d414edb0 0x7ff4d40c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7ff4db5c1640 1 -- 172.21.15.112:0/1965921347 >> 172.21.15.112:0/1965921347 conn(0x7ff4d40c3a80 msgr2=0x7ff4d40c3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:08.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7f23d359e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d40c8ef0 0x7f23d40caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7f23d359e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f23d40cb4c0 0x7f23d41556a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7f2eeb7fe640 1 -- 172.21.15.112:0/148401649 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3697483835 0 0) 0x7f2edc003520 con 0x7f2ee40dbce0 2024-04-25T01:26:08.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7f2eebfff640 1 -- 172.21.15.112:0/148401649 >> v1:172.21.15.187:6789/0 conn(0x7f2ee40073e0 legacy=0x7f2ee400a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f2eebfff640 1 -- 172.21.15.112:0/148401649 >> v1:172.21.15.112:6789/0 conn(0x7f2ee40dbce0 legacy=0x7f2ee40df180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f2eebfff640 1 -- 172.21.15.112:0/148401649 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee400a0f0 con 0x7f2ee40474a0 2024-04-25T01:26:08.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7ffae759e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffae000a8b0 0x7ffae000ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7f31b7407640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f31b006fbe0 0x7f31b006e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7f31b7407640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f31b00c5660 0x7f31b006eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7f31b7407640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31b01578a0 con 0x7f31b0150f40 2024-04-25T01:26:08.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7f31b7407640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f31b00c3a70 con 0x7f31b006fbe0 2024-04-25T01:26:08.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7f31b7407640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f31b0151320 con 0x7f31b00c5660 2024-04-25T01:26:08.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f31b59a4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f31b00c5660 0x7f31b006eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f31b59a4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f31b00c5660 0x7f31b006eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49050/0 (socket says 172.21.15.112:49050) 2024-04-25T01:26:08.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f31b59a4640 1 -- 172.21.15.112:0/667977259 learned_addr learned my addr 172.21.15.112:0/667977259 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:08.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f31b49a2640 1 --2- 172.21.15.112:0/667977259 >> v2:172.21.15.112:3300/0 conn(0x7f31b006fbe0 0x7f31b006e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f31b59a4640 1 -- 172.21.15.112:0/667977259 >> v2:172.21.15.112:3300/0 conn(0x7f31b006fbe0 msgr2=0x7f31b006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:08.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f31b59a4640 1 --2- 172.21.15.112:0/667977259 >> v2:172.21.15.112:3300/0 conn(0x7f31b006fbe0 0x7f31b006e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f31b59a4640 1 -- 172.21.15.112:0/667977259 >> v1:172.21.15.112:6789/0 conn(0x7f31b0150f40 legacy=0x7f31b006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:08.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.199+0000 7f1bd953c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1bcc0de2a0 0x7f1bcc00b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f1bd3fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1bcc0d45b0 0x7f1bcc007970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f1bd3fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1bcc0d45b0 0x7f1bcc007970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49040/0 (socket says 172.21.15.112:49040) 2024-04-25T01:26:08.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f1bd3fff640 1 -- 172.21.15.112:0/1530791430 learned_addr learned my addr 172.21.15.112:0/1530791430 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:08.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f1bd953c640 1 -- 172.21.15.112:0/1530791430 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bcc04a0e0 con 0x7f1bcc00b5c0 2024-04-25T01:26:08.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f1bd953c640 1 -- 172.21.15.112:0/1530791430 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f1bcc0de800 con 0x7f1bcc0de2a0 2024-04-25T01:26:08.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f1bd953c640 1 -- 172.21.15.112:0/1530791430 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1bcc04aa10 con 0x7f1bcc0d45b0 2024-04-25T01:26:08.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7ff4db5c1640 1 -- 172.21.15.112:0/1965921347 shutdown_connections 2024-04-25T01:26:08.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f23d359e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23d40ad380 con 0x7f23d40b03d0 2024-04-25T01:26:08.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f23d359e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f23d40c6210 con 0x7f23d40cb4c0 2024-04-25T01:26:08.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f23d359e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f23d40ad660 con 0x7f23d40c8ef0 2024-04-25T01:26:08.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f23d2d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f23d40cb4c0 0x7f23d41556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f23d2d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f23d40cb4c0 0x7f23d41556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43358/0 (socket says 172.21.15.112:43358) 2024-04-25T01:26:08.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f23d2d9d640 1 -- 172.21.15.112:0/1174094265 learned_addr learned my addr 172.21.15.112:0/1174094265 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:08.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7ffae759e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffae000bb50 con 0x7ffae00dbce0 2024-04-25T01:26:08.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7ffae759e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffae0048060 con 0x7ffae00474a0 2024-04-25T01:26:08.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7ffae759e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffae000a420 con 0x7ffae000a8b0 2024-04-25T01:26:08.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f1bd3fff640 1 -- 172.21.15.112:0/1530791430 >> v2:172.21.15.112:3300/0 conn(0x7f1bcc0de2a0 msgr2=0x7f1bcc00b080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:26:08.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f1bd3fff640 1 --2- 172.21.15.112:0/1530791430 >> v2:172.21.15.112:3300/0 conn(0x7f1bcc0de2a0 0x7f1bcc00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f1bd3fff640 1 -- 172.21.15.112:0/1530791430 >> v1:172.21.15.187:6789/0 conn(0x7f1bcc00b5c0 legacy=0x7f1bcc04aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:08.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f1bd3fff640 1 -- 172.21.15.112:0/1530791430 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bcc04a3c0 con 0x7f1bcc0d45b0 2024-04-25T01:26:08.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7fda42ae6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fda3c070b10 0x7fda3c13a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7fda42ae6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda3c0c7890 0x7fda3c06eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7fda42ae6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fda3c072db0 0x7fda3c1517c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7fda42ae6640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fda3c1525b0 con 0x7fda3c072db0 2024-04-25T01:26:08.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7fda42ae6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fda3c0c2830 con 0x7fda3c070b10 2024-04-25T01:26:08.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7fda42ae6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fda3c0c2b10 con 0x7fda3c0c7890 2024-04-25T01:26:08.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f31b59a4640 1 -- 172.21.15.112:0/667977259 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b0157b80 con 0x7f31b00c5660 2024-04-25T01:26:08.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f31b49a2640 1 --2- 172.21.15.112:0/667977259 >> v2:172.21.15.112:3300/0 conn(0x7f31b006fbe0 0x7f31b006e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:08.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7f23d1d9b640 1 --2- 172.21.15.112:0/1174094265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d40c8ef0 0x7f23d40caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7ffae6d9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffae000a8b0 0x7ffae000ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.200+0000 7ff4db5c1640 1 -- 172.21.15.112:0/1965921347 wait complete. 2024-04-25T01:26:08.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7f23d159a640 1 -- 172.21.15.112:0/1174094265 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2177281741 0 0) 0x7f23d40ad380 con 0x7f23d40b03d0 2024-04-25T01:26:08.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7ffae5d9b640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7ffae00474a0 0x7ffae0006ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:59584/0 (socket says 172.21.15.112:59584) 2024-04-25T01:26:08.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7ffae659c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ffae00dbce0 0x7ffae00df180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37978/0 (socket says 172.21.15.112:37978) 2024-04-25T01:26:08.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7ffae659c640 1 -- 172.21.15.112:0/3325650743 learned_addr learned my addr 172.21.15.112:0/3325650743 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:08.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7fda422e5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fda3c072db0 0x7fda3c1517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7fda422e5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fda3c072db0 0x7fda3c1517c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43366/0 (socket says 172.21.15.112:43366) 2024-04-25T01:26:08.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7fda422e5640 1 -- 172.21.15.112:0/2804021875 learned_addr learned my addr 172.21.15.112:0/2804021875 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:08.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7f23d159a640 1 -- 172.21.15.112:0/1174094265 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23bc003520 con 0x7f23d40b03d0 2024-04-25T01:26:08.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7fda412e3640 1 --2- 172.21.15.112:0/2804021875 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda3c0c7890 0x7fda3c06eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7fda41ae4640 1 --2- 172.21.15.112:0/2804021875 >> v2:172.21.15.187:3300/0 conn(0x7fda3c070b10 0x7fda3c13a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7f23d2d9d640 1 -- 172.21.15.112:0/1174094265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d40c8ef0 msgr2=0x7f23d40caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:08.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7f23d2d9d640 1 --2- 172.21.15.112:0/1174094265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d40c8ef0 0x7f23d40caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7f23d2d9d640 1 -- 172.21.15.112:0/1174094265 >> v1:172.21.15.187:6789/0 conn(0x7f23d40b03d0 legacy=0x7f23d41596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7f23d2d9d640 1 -- 172.21.15.112:0/1174094265 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d40c5a10 con 0x7f23d40cb4c0 2024-04-25T01:26:08.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7ffae559a640 1 -- 172.21.15.112:0/3325650743 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1591699634 0 0) 0x7ffae000bb50 con 0x7ffae00dbce0 2024-04-25T01:26:08.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7ffae559a640 1 -- 172.21.15.112:0/3325650743 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffad0003520 con 0x7ffae00dbce0 2024-04-25T01:26:08.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7ffae6d9d640 1 -- 172.21.15.112:0/3325650743 >> v1:172.21.15.187:6789/0 conn(0x7ffae00474a0 legacy=0x7ffae0006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7ffae6d9d640 1 -- 172.21.15.112:0/3325650743 >> v1:172.21.15.112:6789/0 conn(0x7ffae00dbce0 legacy=0x7ffae00df180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7ff4db5c1640 1 Processor -- start 2024-04-25T01:26:08.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7ff4db5c1640 1 -- start start 2024-04-25T01:26:08.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7f1bd3fff640 1 --2- 172.21.15.112:0/1530791430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bcc0d45b0 0x7f1bcc007970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc048a20 tx=0x7f1bc002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33864ba09d85b60b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.202+0000 7f2eebfff640 1 --2- 172.21.15.112:0/148401649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee40474a0 0x7f2ee4006ea0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009720 tx=0x7f2ee0029fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf40fccaaa28171d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.202+0000 7f31b59a4640 1 --2- 172.21.15.112:0/667977259 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31b00c5660 0x7f31b006eb50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f31b00a72b0 tx=0x7f31a802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=694671527a5bad94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.202+0000 7f1bd2ffd640 1 -- 172.21.15.112:0/1530791430 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0037070 con 0x7f1bcc0d45b0 2024-04-25T01:26:08.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7ffae6d9d640 1 -- 172.21.15.112:0/3325650743 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae000a0f0 con 0x7ffae000a8b0 2024-04-25T01:26:08.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.202+0000 7f1bd2ffd640 1 -- 172.21.15.112:0/1530791430 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc0032440 con 0x7f1bcc0d45b0 2024-04-25T01:26:08.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.202+0000 7f1bd2ffd640 1 -- 172.21.15.112:0/1530791430 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0032810 con 0x7f1bcc0d45b0 2024-04-25T01:26:08.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7fda422e5640 1 -- 172.21.15.112:0/2804021875 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda3c0c7890 msgr2=0x7fda3c06eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:08.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.202+0000 7f23d2d9d640 1 --2- 172.21.15.112:0/1174094265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d40cb4c0 0x7f23d41556a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009e50 tx=0x7f23cc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd145d4093ab1b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7fda422e5640 1 --2- 172.21.15.112:0/2804021875 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda3c0c7890 0x7fda3c06eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.202+0000 7f2eeb7fe640 1 -- 172.21.15.112:0/148401649 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0037070 con 0x7f2ee40474a0 2024-04-25T01:26:08.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.202+0000 7f2eeb7fe640 1 -- 172.21.15.112:0/148401649 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ee002aa70 con 0x7f2ee40474a0 2024-04-25T01:26:08.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.202+0000 7f2eeb7fe640 1 -- 172.21.15.112:0/148401649 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8001670 con 0x7f2ee40474a0 2024-04-25T01:26:08.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.202+0000 7ffae6d9d640 1 --2- 172.21.15.112:0/3325650743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffae000a8b0 0x7ffae000ad00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffae806a090 tx=0x7ffae80a5db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1d7ab0684df2b1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7fda422e5640 1 -- 172.21.15.112:0/2804021875 >> v2:172.21.15.187:3300/0 conn(0x7fda3c070b10 msgr2=0x7fda3c13a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:08.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7fda422e5640 1 --2- 172.21.15.112:0/2804021875 >> v2:172.21.15.187:3300/0 conn(0x7fda3c070b10 0x7fda3c13a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.201+0000 7fda422e5640 1 -- 172.21.15.112:0/2804021875 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c06f060 con 0x7fda3c072db0 2024-04-25T01:26:08.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.202+0000 7fda422e5640 1 --2- 172.21.15.112:0/2804021875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c072db0 0x7fda3c1517c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fda3c06f690 tx=0x7fda3402ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ec5e6436a319d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.202+0000 7ff4db5c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4d4072ed0 0x7ff4d41ebad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7ffae559a640 1 -- 172.21.15.112:0/3325650743 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae80b3070 con 0x7ffae000a8b0 2024-04-25T01:26:08.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7f31a7fff640 1 -- 172.21.15.112:0/667977259 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8036070 con 0x7f31b00c5660 2024-04-25T01:26:08.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7f31a7fff640 1 -- 172.21.15.112:0/667977259 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a8031600 con 0x7f31b00c5660 2024-04-25T01:26:08.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7f31a7fff640 1 -- 172.21.15.112:0/667977259 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a80319d0 con 0x7f31b00c5660 2024-04-25T01:26:08.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7ffae559a640 1 -- 172.21.15.112:0/3325650743 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffae80a6bc0 con 0x7ffae000a8b0 2024-04-25T01:26:08.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7ffae559a640 1 -- 172.21.15.112:0/3325650743 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae80ae440 con 0x7ffae000a8b0 2024-04-25T01:26:08.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7ff4d935d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4d4072ed0 0x7ff4d41ebad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7ff4d935d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4d4072ed0 0x7ff4d41ebad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43372/0 (socket says 172.21.15.112:43372) 2024-04-25T01:26:08.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7ff4d935d640 1 -- 172.21.15.112:0/41429143 learned_addr learned my addr 172.21.15.112:0/41429143 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:08.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7ff4db5c1640 1 --2- 172.21.15.112:0/41429143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4d40c2ed0 0x7ff4d41ee6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7f1bd953c640 1 -- 172.21.15.112:0/1530791430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bcc0d45b0 msgr2=0x7f1bcc007970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7ff4db5c1640 1 --2- 172.21.15.112:0/41429143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4d414edb0 0x7ff4d4151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7f23d159a640 1 -- 172.21.15.112:0/1174094265 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc036070 con 0x7f23d40cb4c0 2024-04-25T01:26:08.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7f1bd953c640 1 --2- 172.21.15.112:0/1530791430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bcc0d45b0 0x7f1bcc007970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc048a20 tx=0x7f1bc002a250 comp rx=0 tx=0).stop 2024-04-25T01:26:08.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7fda40ae2640 1 -- 172.21.15.112:0/2804021875 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34037020 con 0x7fda3c072db0 2024-04-25T01:26:08.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7fda40ae2640 1 -- 172.21.15.112:0/2804021875 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda34009600 con 0x7fda3c072db0 2024-04-25T01:26:08.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7ff4d8b5c640 1 --2- 172.21.15.112:0/41429143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4d40c2ed0 0x7ff4d41ee6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7fda40ae2640 1 -- 172.21.15.112:0/2804021875 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda340099d0 con 0x7fda3c072db0 2024-04-25T01:26:08.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7ff4db5c1640 1 -- 172.21.15.112:0/41429143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d41556c0 con 0x7ff4d4072ed0 2024-04-25T01:26:08.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7f23d159a640 1 -- 172.21.15.112:0/1174094265 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23cc031600 con 0x7f23d40cb4c0 2024-04-25T01:26:08.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7f23d159a640 1 -- 172.21.15.112:0/1174094265 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc0319d0 con 0x7f23d40cb4c0 2024-04-25T01:26:08.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7ff4d9b5e640 1 --2- 172.21.15.112:0/41429143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4d414edb0 0x7ff4d4151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7ff4db5c1640 1 -- 172.21.15.112:0/41429143 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d406eab0 con 0x7ff4d40c2ed0 2024-04-25T01:26:08.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7ff4db5c1640 1 -- 172.21.15.112:0/41429143 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff4d4152140 con 0x7ff4d414edb0 2024-04-25T01:26:08.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7f2ef1d66640 1 -- 172.21.15.112:0/148401649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee40474a0 msgr2=0x7f2ee4006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.203+0000 7f2ef1d66640 1 --2- 172.21.15.112:0/148401649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee40474a0 0x7f2ee4006ea0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009720 tx=0x7f2ee0029fe0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f31b7407640 1 -- 172.21.15.112:0/667977259 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31b00c5660 msgr2=0x7f31b006eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f31b7407640 1 --2- 172.21.15.112:0/667977259 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31b00c5660 0x7f31b006eb50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f31b00a72b0 tx=0x7f31a802a5d0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f2ef1d66640 1 -- 172.21.15.112:0/148401649 shutdown_connections 2024-04-25T01:26:08.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f2ef1d66640 1 --2- 172.21.15.112:0/148401649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee40474a0 0x7f2ee4006ea0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f2ef1d66640 1 -- 172.21.15.112:0/148401649 >> 172.21.15.112:0/148401649 conn(0x7f2ee4048250 msgr2=0x7f2ee4048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:08.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7ffae759e640 1 -- 172.21.15.112:0/3325650743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffae000a8b0 msgr2=0x7ffae000ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7ffae759e640 1 --2- 172.21.15.112:0/3325650743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffae000a8b0 0x7ffae000ad00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffae806a090 tx=0x7ffae80a5db0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7ffae759e640 1 -- 172.21.15.112:0/3325650743 shutdown_connections 2024-04-25T01:26:08.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7ffae759e640 1 --2- 172.21.15.112:0/3325650743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffae000a8b0 0x7ffae000ad00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7ffae759e640 1 -- 172.21.15.112:0/3325650743 >> 172.21.15.112:0/3325650743 conn(0x7ffae0048250 msgr2=0x7ffae0048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:08.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f2ef1d66640 1 -- 172.21.15.112:0/148401649 shutdown_connections 2024-04-25T01:26:08.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f31b7407640 1 -- 172.21.15.112:0/667977259 shutdown_connections 2024-04-25T01:26:08.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f31b7407640 1 --2- 172.21.15.112:0/667977259 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31b00c5660 0x7f31b006eb50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f31b7407640 1 --2- 172.21.15.112:0/667977259 >> v2:172.21.15.112:3300/0 conn(0x7f31b006fbe0 0x7f31b006e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f31b7407640 1 -- 172.21.15.112:0/667977259 >> 172.21.15.112:0/667977259 conn(0x7f31b00c4260 msgr2=0x7f31b00c4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:08.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7ff4d935d640 1 -- 172.21.15.112:0/41429143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4d414edb0 msgr2=0x7ff4d4151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7ff4d935d640 1 --2- 172.21.15.112:0/41429143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4d414edb0 0x7ff4d4151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7ff4d935d640 1 -- 172.21.15.112:0/41429143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4d40c2ed0 msgr2=0x7ff4d41ee6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7ff4d935d640 1 --2- 172.21.15.112:0/41429143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4d40c2ed0 0x7ff4d41ee6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7ff4d935d640 1 -- 172.21.15.112:0/41429143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d41ec010 con 0x7ff4d4072ed0 2024-04-25T01:26:08.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7fda42ae6640 1 -- 172.21.15.112:0/2804021875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c072db0 msgr2=0x7fda3c1517c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7fda42ae6640 1 --2- 172.21.15.112:0/2804021875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c072db0 0x7fda3c1517c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fda3c06f690 tx=0x7fda3402ea10 comp rx=0 tx=0).stop 2024-04-25T01:26:08.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f1bd953c640 1 -- 172.21.15.112:0/1530791430 shutdown_connections 2024-04-25T01:26:08.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f1bd953c640 1 --2- 172.21.15.112:0/1530791430 >> v2:172.21.15.112:3300/0 conn(0x7f1bcc0de2a0 0x7f1bcc00b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f1bd953c640 1 --2- 172.21.15.112:0/1530791430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bcc0d45b0 0x7f1bcc007970 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f1bd953c640 1 -- 172.21.15.112:0/1530791430 >> 172.21.15.112:0/1530791430 conn(0x7f1bcc048ac0 msgr2=0x7f1bcc048ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:08.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7fda42ae6640 1 -- 172.21.15.112:0/2804021875 shutdown_connections 2024-04-25T01:26:08.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7fda42ae6640 1 --2- 172.21.15.112:0/2804021875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c072db0 0x7fda3c1517c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7fda42ae6640 1 --2- 172.21.15.112:0/2804021875 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda3c0c7890 0x7fda3c06eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7fda42ae6640 1 --2- 172.21.15.112:0/2804021875 >> v2:172.21.15.187:3300/0 conn(0x7fda3c070b10 0x7fda3c13a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7fda42ae6640 1 -- 172.21.15.112:0/2804021875 >> 172.21.15.112:0/2804021875 conn(0x7fda3c0c6da0 msgr2=0x7fda3c0c7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:08.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f31b7407640 1 -- 172.21.15.112:0/667977259 shutdown_connections 2024-04-25T01:26:08.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f23d359e640 1 -- 172.21.15.112:0/1174094265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d40cb4c0 msgr2=0x7f23d41556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f23d359e640 1 --2- 172.21.15.112:0/1174094265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d40cb4c0 0x7f23d41556a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009e50 tx=0x7f23cc02a680 comp rx=0 tx=0).stop 2024-04-25T01:26:08.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f1bd953c640 1 -- 172.21.15.112:0/1530791430 shutdown_connections 2024-04-25T01:26:08.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f31b7407640 1 -- 172.21.15.112:0/667977259 wait complete. 2024-04-25T01:26:08.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f1bd953c640 1 -- 172.21.15.112:0/1530791430 wait complete. 2024-04-25T01:26:08.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f23d359e640 1 -- 172.21.15.112:0/1174094265 shutdown_connections 2024-04-25T01:26:08.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7ff4d935d640 1 --2- 172.21.15.112:0/41429143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4d4072ed0 0x7ff4d41ebad0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000ec10 tx=0x7ff4d000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7fda42ae6640 1 -- 172.21.15.112:0/2804021875 shutdown_connections 2024-04-25T01:26:08.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.205+0000 7ff4c9ffb640 1 -- 172.21.15.112:0/41429143 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0017070 con 0x7ff4d4072ed0 2024-04-25T01:26:08.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.205+0000 7f23d359e640 1 --2- 172.21.15.112:0/1174094265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d40cb4c0 0x7f23d41556a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.204+0000 7f2ef1d66640 1 -- 172.21.15.112:0/148401649 wait complete. 2024-04-25T01:26:08.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.205+0000 7ff4c9ffb640 1 -- 172.21.15.112:0/41429143 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d000ed00 con 0x7ff4d4072ed0 2024-04-25T01:26:08.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.205+0000 7ff4c9ffb640 1 -- 172.21.15.112:0/41429143 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d000ce70 con 0x7ff4d4072ed0 2024-04-25T01:26:08.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.205+0000 7f23d359e640 1 --2- 172.21.15.112:0/1174094265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d40c8ef0 0x7f23d40caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.205+0000 7f23d359e640 1 -- 172.21.15.112:0/1174094265 >> 172.21.15.112:0/1174094265 conn(0x7f23d40743d0 msgr2=0x7f23d40747e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:08.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.205+0000 7ff4db5c1640 1 -- 172.21.15.112:0/41429143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d414f500 con 0x7ff4d4072ed0 2024-04-25T01:26:08.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.205+0000 7f23d359e640 1 -- 172.21.15.112:0/1174094265 shutdown_connections 2024-04-25T01:26:08.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.205+0000 7f23d359e640 1 -- 172.21.15.112:0/1174094265 wait complete. 2024-04-25T01:26:08.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.205+0000 7ffae759e640 1 -- 172.21.15.112:0/3325650743 shutdown_connections 2024-04-25T01:26:08.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.205+0000 7ffae759e640 1 -- 172.21.15.112:0/3325650743 wait complete. 2024-04-25T01:26:08.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.206+0000 7f31b7407640 1 Processor -- start 2024-04-25T01:26:08.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.206+0000 7f1bd953c640 1 Processor -- start 2024-04-25T01:26:08.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.206+0000 7f1bd953c640 1 -- start start 2024-04-25T01:26:08.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.205+0000 7ff4db5c1640 1 -- 172.21.15.112:0/41429143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d414fac0 con 0x7ff4d4072ed0 2024-04-25T01:26:08.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.205+0000 7fda42ae6640 1 -- 172.21.15.112:0/2804021875 wait complete. 2024-04-25T01:26:08.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.206+0000 7fda42ae6640 1 Processor -- start 2024-04-25T01:26:08.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.206+0000 7f2ef1d66640 1 Processor -- start 2024-04-25T01:26:08.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.206+0000 7f2ef1d66640 1 -- start start 2024-04-25T01:26:08.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7f23d359e640 1 Processor -- start 2024-04-25T01:26:08.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7f23d359e640 1 -- start start 2024-04-25T01:26:08.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7ffae759e640 1 Processor -- start 2024-04-25T01:26:08.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.206+0000 7f31b7407640 1 -- start start 2024-04-25T01:26:08.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7f31b7407640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31b006fbe0 0x7f31b00aad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7f31b7407640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31b00c5660 0x7f31b0156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7f31b7407640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b0150f40 0x7f31b01f99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7f31b7407640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31b00c1b60 con 0x7f31b0150f40 2024-04-25T01:26:08.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7f31b7407640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f31b00c19e0 con 0x7f31b00c5660 2024-04-25T01:26:08.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7f31b7407640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f31b006f090 con 0x7f31b006fbe0 2024-04-25T01:26:08.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7ffae759e640 1 -- start start 2024-04-25T01:26:08.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7f2ef1d66640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ee40073e0 0x7f2ee40de9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7f2ef1d66640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ee40474a0 0x7f2ee40d98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7f2ef1d66640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee40dbce0 0x7f2ee40d9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7f2ef1d66640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee40dc0c0 con 0x7f2ee40dbce0 2024-04-25T01:26:08.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7f2ef1d66640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4047d80 con 0x7f2ee40073e0 2024-04-25T01:26:08.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7f2ef1d66640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2ee40dfbf0 con 0x7f2ee40474a0 2024-04-25T01:26:08.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.206+0000 7fda42ae6640 1 -- start start 2024-04-25T01:26:08.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7fda42ae6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda3c070b10 0x7fda3c1effe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7fda42ae6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c072db0 0x7fda3c0ac570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7f1bd953c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bcc00b5c0 0x7f1bcc0dcd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f1bd953c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bcc0d45b0 0x7f1bcc0dd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7fda42ae6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda3c0c7890 0x7fda3c0acab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f31b59a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b0150f40 0x7f31b01f99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f31b59a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b0150f40 0x7f31b01f99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43402/0 (socket says 172.21.15.112:43402) 2024-04-25T01:26:08.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f31b59a4640 1 -- 172.21.15.112:0/392519154 learned_addr learned my addr 172.21.15.112:0/392519154 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:08.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7fda42ae6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c13ac40 con 0x7fda3c072db0 2024-04-25T01:26:08.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7fda42ae6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c13aac0 con 0x7fda3c070b10 2024-04-25T01:26:08.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7fda42ae6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fda3c13adc0 con 0x7fda3c0c7890 2024-04-25T01:26:08.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7fc08d15a640 1 Processor -- start 2024-04-25T01:26:08.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.207+0000 7fc08d15a640 1 -- start start 2024-04-25T01:26:08.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f1bd953c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bcc0de2a0 0x7f1bcc030210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7fda41ae4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda3c070b10 0x7fda3c1effe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7fda41ae4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda3c070b10 0x7fda3c1effe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49096/0 (socket says 172.21.15.112:49096) 2024-04-25T01:26:08.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7fda41ae4640 1 -- 172.21.15.112:0/25896361 learned_addr learned my addr 172.21.15.112:0/25896361 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:08.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f1bd3fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bcc00b5c0 0x7f1bcc0dcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7fda422e5640 1 --2- 172.21.15.112:0/25896361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda3c0c7890 0x7fda3c0acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7fda412e3640 1 --2- 172.21.15.112:0/25896361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c072db0 0x7fda3c0ac570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f1bd3fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bcc00b5c0 0x7f1bcc0dcd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:56540/0 (socket says 172.21.15.112:56540) 2024-04-25T01:26:08.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f1bd3fff640 1 -- 172.21.15.112:0/2410534142 learned_addr learned my addr 172.21.15.112:0/2410534142 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:08.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f1bd8d3b640 1 --2- 172.21.15.112:0/2410534142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bcc0de2a0 0x7f1bcc030210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f2eebfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ee40474a0 0x7f2ee40d98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f1bd953c640 1 -- 172.21.15.112:0/2410534142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc046df0 con 0x7f1bcc0d45b0 2024-04-25T01:26:08.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f2eebfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ee40474a0 0x7f2ee40d98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:56528/0 (socket says 172.21.15.112:56528) 2024-04-25T01:26:08.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f2eebfff640 1 -- 172.21.15.112:0/2070554888 learned_addr learned my addr 172.21.15.112:0/2070554888 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:08.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f31b59a4640 1 -- 172.21.15.112:0/392519154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31b006fbe0 msgr2=0x7f31b00aad90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:26:08.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f1bd953c640 1 -- 172.21.15.112:0/2410534142 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc0d4990 con 0x7f1bcc0de2a0 2024-04-25T01:26:08.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f1bd953c640 1 -- 172.21.15.112:0/2410534142 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1bcc046f70 con 0x7f1bcc00b5c0 2024-04-25T01:26:08.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f31b59a4640 1 --2- 172.21.15.112:0/392519154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31b006fbe0 0x7f31b00aad90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7ff4db5c1640 1 -- 172.21.15.112:0/41429143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d404f570 con 0x7ff4d4072ed0 2024-04-25T01:26:08.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f1bd37fe640 1 --2- 172.21.15.112:0/2410534142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bcc0d45b0 0x7f1bcc0dd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f31b59a4640 1 -- 172.21.15.112:0/392519154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31b00c5660 msgr2=0x7f31b0156280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:26:08.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7fda422e5640 1 -- 172.21.15.112:0/25896361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda3c070b10 msgr2=0x7fda3c1effe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7fda422e5640 1 --2- 172.21.15.112:0/25896361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda3c070b10 0x7fda3c1effe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7fda422e5640 1 -- 172.21.15.112:0/25896361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c072db0 msgr2=0x7fda3c0ac570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7fda422e5640 1 --2- 172.21.15.112:0/25896361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c072db0 0x7fda3c0ac570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7fda422e5640 1 -- 172.21.15.112:0/25896361 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c136ce0 con 0x7fda3c0c7890 2024-04-25T01:26:08.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7ff4c9ffb640 1 -- 172.21.15.112:0/41429143 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff4d001a070 con 0x7ff4d4072ed0 2024-04-25T01:26:08.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f31b59a4640 1 --2- 172.21.15.112:0/392519154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31b00c5660 0x7f31b0156280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f31b59a4640 1 -- 172.21.15.112:0/392519154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b00ab2d0 con 0x7f31b0150f40 2024-04-25T01:26:08.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.208+0000 7f2ef0d64640 1 --2- 172.21.15.112:0/2070554888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ee40073e0 0x7f2ee40de9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7fda422e5640 1 --2- 172.21.15.112:0/25896361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda3c0c7890 0x7fda3c0acab0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fda3c068f60 tx=0x7fda34004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7ffae759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffae000a8b0 0x7ffae0175200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7ffae759e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffae00474a0 0x7ffae0178440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7f2ef1565640 1 --2- 172.21.15.112:0/2070554888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee40dbce0 0x7f2ee40d9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7f2eebfff640 1 -- 172.21.15.112:0/2070554888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ee40073e0 msgr2=0x7f2ee40de9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7f2eebfff640 1 --2- 172.21.15.112:0/2070554888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ee40073e0 0x7f2ee40de9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7f2eebfff640 1 -- 172.21.15.112:0/2070554888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee40dbce0 msgr2=0x7f2ee40d9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7f2eebfff640 1 --2- 172.21.15.112:0/2070554888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee40dbce0 0x7f2ee40d9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7f2eebfff640 1 -- 172.21.15.112:0/2070554888 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee4042940 con 0x7f2ee40474a0 2024-04-25T01:26:08.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7ffae759e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffae00dbce0 0x7ffae00deb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7ffae759e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffae000b240 con 0x7ffae000a8b0 2024-04-25T01:26:08.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7ffae759e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffae0047d80 con 0x7ffae00dbce0 2024-04-25T01:26:08.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7ffae6d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffae00dbce0 0x7ffae00deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7ffae6d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffae00dbce0 0x7ffae00deb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49120/0 (socket says 172.21.15.112:49120) 2024-04-25T01:26:08.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7ffae6d9d640 1 -- 172.21.15.112:0/2421983751 learned_addr learned my addr 172.21.15.112:0/2421983751 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:08.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7ffae759e640 1 -- 172.21.15.112:0/2421983751 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffae00dc0c0 con 0x7ffae00474a0 2024-04-25T01:26:08.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7ff4c9ffb640 1 --2- 172.21.15.112:0/41429143 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff4b806be30 0x7ff4b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7ff4c9ffb640 1 -- 172.21.15.112:0/41429143 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7ff4d008aff0 con 0x7ff4d4072ed0 2024-04-25T01:26:08.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7fc08d15a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc0880c7f40 0x7fc0880707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7fc08d15a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc0880c4e40 0x7fc0880c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7fc08d15a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0880c5620 con 0x7fc0880701b0 2024-04-25T01:26:08.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7fc08d15a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fc08815d740 con 0x7fc0880c7f40 2024-04-25T01:26:08.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7fc08d15a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc08815e2e0 con 0x7fc0880c4e40 2024-04-25T01:26:08.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7fda327fc640 1 -- 172.21.15.112:0/25896361 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34033460 con 0x7fda3c0c7890 2024-04-25T01:26:08.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7fda42ae6640 1 -- 172.21.15.112:0/25896361 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3c136fc0 con 0x7fda3c0c7890 2024-04-25T01:26:08.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7f31b59a4640 1 --2- 172.21.15.112:0/392519154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b0150f40 0x7f31b01f99a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009f40 tx=0x7f31a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7f23d359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d40b03d0 0x7f23d4150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7fda42ae6640 1 -- 172.21.15.112:0/25896361 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fda3c155ad0 con 0x7fda3c0c7890 2024-04-25T01:26:08.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7ffae659c640 1 --2- 172.21.15.112:0/2421983751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffae000a8b0 0x7ffae0175200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7ff4d8b5c640 1 --2- 172.21.15.112:0/41429143 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff4b806be30 0x7ff4b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7f23d359e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d40c8ef0 0x7f23d414c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7ffae5d9b640 1 --2- 172.21.15.112:0/2421983751 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffae00474a0 0x7ffae0178440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7f23d259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d40b03d0 0x7f23d4150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.209+0000 7f2eebfff640 1 --2- 172.21.15.112:0/2070554888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ee40474a0 0x7f2ee40d98c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2ee00098c0 tx=0x7f2ee00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7fc08759e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc0880c4e40 0x7fc0880c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7fc08759e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc0880c4e40 0x7fc0880c6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49110/0 (socket says 172.21.15.112:49110) 2024-04-25T01:26:08.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7fc08759e640 1 -- 172.21.15.112:0/2737528535 learned_addr learned my addr 172.21.15.112:0/2737528535 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:08.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7f1bd37fe640 1 -- 172.21.15.112:0/2410534142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bcc00b5c0 msgr2=0x7f1bcc0dcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7f23d359e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d40cb4c0 0x7f23d414c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7f1bd37fe640 1 --2- 172.21.15.112:0/2410534142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bcc00b5c0 0x7f1bcc0dcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7f23d359e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23d40b07b0 con 0x7f23d40b03d0 2024-04-25T01:26:08.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7f1bd37fe640 1 -- 172.21.15.112:0/2410534142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bcc0de2a0 msgr2=0x7f1bcc030210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7f23d359e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f23d40ad940 con 0x7f23d40c8ef0 2024-04-25T01:26:08.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7f23d359e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f23d40b0930 con 0x7f23d40cb4c0 2024-04-25T01:26:08.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7fc086d9d640 1 --2- 172.21.15.112:0/2737528535 >> v2:172.21.15.112:3300/0 conn(0x7fc0880c7f40 0x7fc0880707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7f1bd37fe640 1 --2- 172.21.15.112:0/2410534142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bcc0de2a0 0x7f1bcc030210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7f23d1d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d40c8ef0 0x7f23d414c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7f23d1d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d40c8ef0 0x7f23d414c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49134/0 (socket says 172.21.15.112:49134) 2024-04-25T01:26:08.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7f23d1d9b640 1 -- 172.21.15.112:0/222110957 learned_addr learned my addr 172.21.15.112:0/222110957 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:08.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7f1bd37fe640 1 -- 172.21.15.112:0/2410534142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bcc0d02d0 con 0x7f1bcc0d45b0 2024-04-25T01:26:08.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7f1bd3fff640 1 --2- 172.21.15.112:0/2410534142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bcc00b5c0 0x7f1bcc0dcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:08.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7f1bd8d3b640 1 --2- 172.21.15.112:0/2410534142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bcc0de2a0 0x7f1bcc030210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:08.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7f1bd37fe640 1 --2- 172.21.15.112:0/2410534142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bcc0d45b0 0x7f1bcc0dd2d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800b410 tx=0x7f1bc800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7fda327fc640 1 -- 172.21.15.112:0/25896361 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda34033600 con 0x7fda3c0c7890 2024-04-25T01:26:08.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7f23d1d9b640 1 -- 172.21.15.112:0/222110957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d40cb4c0 msgr2=0x7f23d414c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7f23d1d9b640 1 --2- 172.21.15.112:0/222110957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d40cb4c0 0x7f23d414c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7f23d1d9b640 1 -- 172.21.15.112:0/222110957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d40b03d0 msgr2=0x7f23d4150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7f23d1d9b640 1 --2- 172.21.15.112:0/222110957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d40b03d0 0x7f23d4150360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.211+0000 7ff4d8b5c640 1 --2- 172.21.15.112:0/41429143 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff4b806be30 0x7ff4b806e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc033000 tx=0x7ff4cc02f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.211+0000 7f23d1d9b640 1 -- 172.21.15.112:0/222110957 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d41eace0 con 0x7f23d40c8ef0 2024-04-25T01:26:08.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7ffae6d9d640 1 -- 172.21.15.112:0/2421983751 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffae00474a0 msgr2=0x7ffae0178440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.211+0000 7fc08759e640 1 -- 172.21.15.112:0/2737528535 >> v2:172.21.15.112:3300/0 conn(0x7fc0880c7f40 msgr2=0x7fc0880707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:08.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.211+0000 7fc08759e640 1 --2- 172.21.15.112:0/2737528535 >> v2:172.21.15.112:3300/0 conn(0x7fc0880c7f40 0x7fc0880707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7ffae6d9d640 1 --2- 172.21.15.112:0/2421983751 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffae00474a0 0x7ffae0178440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.211+0000 7f2ee97fa640 1 -- 172.21.15.112:0/2070554888 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0042070 con 0x7f2ee40474a0 2024-04-25T01:26:08.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.211+0000 7f2ee97fa640 1 -- 172.21.15.112:0/2070554888 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ee0032dd0 con 0x7f2ee40474a0 2024-04-25T01:26:08.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.211+0000 7f2ee97fa640 1 -- 172.21.15.112:0/2070554888 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0037040 con 0x7f2ee40474a0 2024-04-25T01:26:08.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.211+0000 7fc08759e640 1 -- 172.21.15.112:0/2737528535 >> v1:172.21.15.112:6789/0 conn(0x7fc0880701b0 legacy=0x7fc08815dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.211+0000 7fc08759e640 1 -- 172.21.15.112:0/2737528535 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0880c5900 con 0x7fc0880c4e40 2024-04-25T01:26:08.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7ffae6d9d640 1 -- 172.21.15.112:0/2421983751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffae000a8b0 msgr2=0x7ffae0175200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.211+0000 7fc086d9d640 1 --2- 172.21.15.112:0/2737528535 >> v2:172.21.15.112:3300/0 conn(0x7fc0880c7f40 0x7fc0880707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:26:08.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.211+0000 7f2ef1d66640 1 -- 172.21.15.112:0/2070554888 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee4042c20 con 0x7f2ee40474a0 2024-04-25T01:26:08.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7ffae6d9d640 1 --2- 172.21.15.112:0/2421983751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffae000a8b0 0x7ffae0175200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.211+0000 7f23d1d9b640 1 --2- 172.21.15.112:0/222110957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d40c8ef0 0x7f23d414c280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f23c400e9b0 tx=0x7f23c400ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.210+0000 7ffae6d9d640 1 -- 172.21.15.112:0/2421983751 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae0043e50 con 0x7ffae00dbce0 2024-04-25T01:26:08.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.211+0000 7f2ef1d66640 1 -- 172.21.15.112:0/2070554888 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee40def20 con 0x7f2ee40474a0 2024-04-25T01:26:08.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.211+0000 7f23c2ffd640 1 -- 172.21.15.112:0/222110957 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c4017070 con 0x7f23d40c8ef0 2024-04-25T01:26:08.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.211+0000 7fda327fc640 1 -- 172.21.15.112:0/25896361 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda340339d0 con 0x7fda3c0c7890 2024-04-25T01:26:08.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.211+0000 7fda42ae6640 1 -- 172.21.15.112:0/25896361 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda3c04f500 con 0x7fda3c0c7890 2024-04-25T01:26:08.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.211+0000 7fda327fc640 1 -- 172.21.15.112:0/25896361 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fda3403a020 con 0x7fda3c0c7890 2024-04-25T01:26:08.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.211+0000 7f23c2ffd640 1 -- 172.21.15.112:0/222110957 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c400c9a0 con 0x7f23d40c8ef0 2024-04-25T01:26:08.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.211+0000 7f23d359e640 1 -- 172.21.15.112:0/222110957 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d41eafc0 con 0x7f23d40c8ef0 2024-04-25T01:26:08.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.211+0000 7f23d359e640 1 -- 172.21.15.112:0/222110957 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d41eb580 con 0x7f23d40c8ef0 2024-04-25T01:26:08.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.212+0000 7fda327fc640 1 --2- 172.21.15.112:0/25896361 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fda2006be80 0x7fda2006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.212+0000 7f23c2ffd640 1 -- 172.21.15.112:0/222110957 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c4017070 con 0x7f23d40c8ef0 2024-04-25T01:26:08.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.212+0000 7ffae6d9d640 1 --2- 172.21.15.112:0/2421983751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffae00dbce0 0x7ffae00deb80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffae8073d70 tx=0x7ffae8064fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.212+0000 7fda327fc640 1 -- 172.21.15.112:0/25896361 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fda340ab800 con 0x7fda3c0c7890 2024-04-25T01:26:08.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.212+0000 7fda41ae4640 1 --2- 172.21.15.112:0/25896361 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fda2006be80 0x7fda2006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.212+0000 7f2ef1d66640 1 -- 172.21.15.112:0/2070554888 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee4005390 con 0x7f2ee40474a0 2024-04-25T01:26:08.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.212+0000 7fc08759e640 1 --2- 172.21.15.112:0/2737528535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0880c4e40 0x7fc0880c6fd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc088071b00 tx=0x7fc08002a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d012c83041e21571 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.213+0000 7fda41ae4640 1 --2- 172.21.15.112:0/25896361 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fda2006be80 0x7fda2006e340 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fda2c006fd0 tx=0x7fda2c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.213+0000 7f23d359e640 1 -- 172.21.15.112:0/222110957 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d40ca2f0 con 0x7f23d40c8ef0 2024-04-25T01:26:08.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.213+0000 7fc085d9b640 1 -- 172.21.15.112:0/2737528535 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080036070 con 0x7fc0880c4e40 2024-04-25T01:26:08.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.213+0000 7fc085d9b640 1 -- 172.21.15.112:0/2737528535 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc080031600 con 0x7fc0880c4e40 2024-04-25T01:26:08.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.213+0000 7fc085d9b640 1 -- 172.21.15.112:0/2737528535 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0800319d0 con 0x7fc0880c4e40 2024-04-25T01:26:08.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.213+0000 7f2ee97fa640 1 -- 172.21.15.112:0/2070554888 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ee0045070 con 0x7f2ee40474a0 2024-04-25T01:26:08.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.213+0000 7f2ee97fa640 1 --2- 172.21.15.112:0/2070554888 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ed006be80 0x7f2ed006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.213+0000 7ffad6ffd640 1 -- 172.21.15.112:0/2421983751 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae80bd020 con 0x7ffae00dbce0 2024-04-25T01:26:08.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.213+0000 7ffad6ffd640 1 -- 172.21.15.112:0/2421983751 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffae80afce0 con 0x7ffae00dbce0 2024-04-25T01:26:08.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.213+0000 7ffad6ffd640 1 -- 172.21.15.112:0/2421983751 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae80b3040 con 0x7ffae00dbce0 2024-04-25T01:26:08.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.213+0000 7ffae759e640 1 -- 172.21.15.112:0/2421983751 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae0044130 con 0x7ffae00dbce0 2024-04-25T01:26:08.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.213+0000 7f2ee97fa640 1 -- 172.21.15.112:0/2070554888 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f2ee00ab1d0 con 0x7f2ee40474a0 2024-04-25T01:26:08.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.213+0000 7f2ef0d64640 1 --2- 172.21.15.112:0/2070554888 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ed006be80 0x7f2ed006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.213+0000 7ffae759e640 1 -- 172.21.15.112:0/2421983751 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae00d89b0 con 0x7ffae00dbce0 2024-04-25T01:26:08.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.214+0000 7fc08d15a640 1 -- 172.21.15.112:0/2737528535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0880c4e40 msgr2=0x7fc0880c6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.214+0000 7fc08d15a640 1 --2- 172.21.15.112:0/2737528535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0880c4e40 0x7fc0880c6fd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc088071b00 tx=0x7fc08002a610 comp rx=0 tx=0).stop 2024-04-25T01:26:08.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.214+0000 7f2ef0d64640 1 --2- 172.21.15.112:0/2070554888 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ed006be80 0x7f2ed006e340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80011e0 tx=0x7f2ed80016d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.214+0000 7f23c2ffd640 1 -- 172.21.15.112:0/222110957 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f23c401a070 con 0x7f23d40c8ef0 2024-04-25T01:26:08.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.214+0000 7f23c2ffd640 1 --2- 172.21.15.112:0/222110957 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f23b406c030 0x7f23b406e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.214+0000 7f23c2ffd640 1 -- 172.21.15.112:0/222110957 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f23c408b4e0 con 0x7f23d40c8ef0 2024-04-25T01:26:08.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.214+0000 7fc08d15a640 1 -- 172.21.15.112:0/2737528535 shutdown_connections 2024-04-25T01:26:08.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.214+0000 7fc08d15a640 1 --2- 172.21.15.112:0/2737528535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0880c4e40 0x7fc0880c6fd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.214+0000 7fc08d15a640 1 --2- 172.21.15.112:0/2737528535 >> v2:172.21.15.112:3300/0 conn(0x7fc0880c7f40 0x7fc0880707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.214+0000 7fc08d15a640 1 -- 172.21.15.112:0/2737528535 >> 172.21.15.112:0/2737528535 conn(0x7fc08806f2e0 msgr2=0x7fc08806f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:08.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.214+0000 7fc08d15a640 1 -- 172.21.15.112:0/2737528535 shutdown_connections 2024-04-25T01:26:08.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.214+0000 7f23d259c640 1 --2- 172.21.15.112:0/222110957 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f23b406c030 0x7f23b406e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.214+0000 7fc08d15a640 1 -- 172.21.15.112:0/2737528535 wait complete. 2024-04-25T01:26:08.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.215+0000 7ffae759e640 1 -- 172.21.15.112:0/2421983751 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffae0177d10 con 0x7ffae00dbce0 2024-04-25T01:26:08.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.215+0000 7f23d259c640 1 --2- 172.21.15.112:0/222110957 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f23b406c030 0x7f23b406e4f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f23c80011e0 tx=0x7f23c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.215+0000 7f31a5ffb640 1 -- 172.21.15.112:0/392519154 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8041020 con 0x7f31b0150f40 2024-04-25T01:26:08.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.215+0000 7f31a5ffb640 1 -- 172.21.15.112:0/392519154 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a8008710 con 0x7f31b0150f40 2024-04-25T01:26:08.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.215+0000 7f31a5ffb640 1 -- 172.21.15.112:0/392519154 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8036040 con 0x7f31b0150f40 2024-04-25T01:26:08.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.215+0000 7f1bd0ff9640 1 -- 172.21.15.112:0/2410534142 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8015020 con 0x7f1bcc0d45b0 2024-04-25T01:26:08.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.216+0000 7f1bd953c640 1 -- 172.21.15.112:0/2410534142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bcc170e20 con 0x7f1bcc0d45b0 2024-04-25T01:26:08.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.216+0000 7f1bd0ff9640 1 -- 172.21.15.112:0/2410534142 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc8011830 con 0x7f1bcc0d45b0 2024-04-25T01:26:08.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.216+0000 7ff4c9ffb640 1 -- 172.21.15.112:0/41429143 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff4d0059010 con 0x7ff4d4072ed0 2024-04-25T01:26:08.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.216+0000 7f1bd0ff9640 1 -- 172.21.15.112:0/2410534142 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8015020 con 0x7f1bcc0d45b0 2024-04-25T01:26:08.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.216+0000 7f31b7407640 1 -- 172.21.15.112:0/392519154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b00aa810 con 0x7f31b0150f40 2024-04-25T01:26:08.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.216+0000 7f1bd953c640 1 -- 172.21.15.112:0/2410534142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bcc0dc8a0 con 0x7f1bcc0d45b0 2024-04-25T01:26:08.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.216+0000 7f31b7407640 1 -- 172.21.15.112:0/392519154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b00aaab0 con 0x7f31b0150f40 2024-04-25T01:26:08.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.216+0000 7fc08d15a640 1 Processor -- start 2024-04-25T01:26:08.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.216+0000 7fc08d15a640 1 -- start start 2024-04-25T01:26:08.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.217+0000 7f31b7407640 1 -- 172.21.15.112:0/392519154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31b00c1dc0 con 0x7f31b0150f40 2024-04-25T01:26:08.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.217+0000 7fc08d15a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0880c4e40 0x7fc0880aa260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.217+0000 7fda327fc640 1 -- 172.21.15.112:0/25896361 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fda34040030 con 0x7fda3c0c7890 2024-04-25T01:26:08.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.217+0000 7fc08d15a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0880c7f40 0x7fc0880ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.218+0000 7f1bd953c640 1 -- 172.21.15.112:0/2410534142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bcc0057e0 con 0x7f1bcc0d45b0 2024-04-25T01:26:08.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.217+0000 7fc08d15a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0880ab900 0x7fc0880a8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.217+0000 7fc08d15a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0880c9690 con 0x7fc0880c7f40 2024-04-25T01:26:08.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.217+0000 7fc08d15a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc0880c8320 con 0x7fc0880ab900 2024-04-25T01:26:08.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.217+0000 7fc08d15a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc0880c9810 con 0x7fc0880c4e40 2024-04-25T01:26:08.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.218+0000 7fc08659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0880c7f40 0x7fc0880ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.218+0000 7fc08659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0880c7f40 0x7fc0880ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43456/0 (socket says 172.21.15.112:43456) 2024-04-25T01:26:08.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.218+0000 7fc08659c640 1 -- 172.21.15.112:0/2995983968 learned_addr learned my addr 172.21.15.112:0/2995983968 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:08.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.218+0000 7fc086d9d640 1 --2- 172.21.15.112:0/2995983968 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0880c4e40 0x7fc0880aa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.218+0000 7f1bd0ff9640 1 -- 172.21.15.112:0/2410534142 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1bc8018020 con 0x7f1bcc0d45b0 2024-04-25T01:26:08.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.218+0000 7fc08759e640 1 --2- 172.21.15.112:0/2995983968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0880ab900 0x7fc0880a8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.218+0000 7fc08659c640 1 -- 172.21.15.112:0/2995983968 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0880c4e40 msgr2=0x7fc0880aa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.218+0000 7f31a5ffb640 1 -- 172.21.15.112:0/392519154 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f31a8044020 con 0x7f31b0150f40 2024-04-25T01:26:08.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.218+0000 7fc08659c640 1 --2- 172.21.15.112:0/2995983968 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0880c4e40 0x7fc0880aa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.218+0000 7fc08659c640 1 -- 172.21.15.112:0/2995983968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0880ab900 msgr2=0x7fc0880a8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.218+0000 7fc08659c640 1 --2- 172.21.15.112:0/2995983968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0880ab900 0x7fc0880a8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.218+0000 7fc08659c640 1 -- 172.21.15.112:0/2995983968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0880adbd0 con 0x7fc0880c7f40 2024-04-25T01:26:08.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.218+0000 7fc08659c640 1 --2- 172.21.15.112:0/2995983968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0880c7f40 0x7fc0880ad1a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00c450 tx=0x7fc07c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.218+0000 7f1bd0ff9640 1 --2- 172.21.15.112:0/2410534142 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1bb806c030 0x7f1bb806e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.218+0000 7f23c2ffd640 1 -- 172.21.15.112:0/222110957 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f23d40ca2f0 con 0x7f23d40c8ef0 2024-04-25T01:26:08.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.219+0000 7f31a5ffb640 1 --2- 172.21.15.112:0/392519154 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f319406be80 0x7f319406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.219+0000 7f1bd0ff9640 1 -- 172.21.15.112:0/2410534142 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f1bc808a8e0 con 0x7f1bcc0d45b0 2024-04-25T01:26:08.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.219+0000 7f1bd3fff640 1 --2- 172.21.15.112:0/2410534142 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1bb806c030 0x7f1bb806e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.219+0000 7f31a5ffb640 1 -- 172.21.15.112:0/392519154 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f31a80ac170 con 0x7f31b0150f40 2024-04-25T01:26:08.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.219+0000 7f31b51a3640 1 --2- 172.21.15.112:0/392519154 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f319406be80 0x7f319406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.219+0000 7f1bd3fff640 1 --2- 172.21.15.112:0/2410534142 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1bb806c030 0x7f1bb806e4f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1bc00090b0 tx=0x7f1bc002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.220+0000 7ffad6ffd640 1 -- 172.21.15.112:0/2421983751 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffae80c0020 con 0x7ffae00dbce0 2024-04-25T01:26:08.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.220+0000 7f31b51a3640 1 --2- 172.21.15.112:0/392519154 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f319406be80 0x7f319406e340 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0095f0 tx=0x7f31ac006b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.220+0000 7ffad6ffd640 1 --2- 172.21.15.112:0/2421983751 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ffac806be80 0x7ffac806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.220+0000 7ffad6ffd640 1 -- 172.21.15.112:0/2421983751 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7ffae8127a10 con 0x7ffae00dbce0 2024-04-25T01:26:08.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.220+0000 7ffae659c640 1 --2- 172.21.15.112:0/2421983751 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ffac806be80 0x7ffac806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.220+0000 7ffae659c640 1 --2- 172.21.15.112:0/2421983751 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ffac806be80 0x7ffac806e340 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffad8000f60 tx=0x7ffad8007e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.222+0000 7f2ee97fa640 1 -- 172.21.15.112:0/2070554888 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2ee0079490 con 0x7f2ee40474a0 2024-04-25T01:26:08.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.223+0000 7ffad6ffd640 1 -- 172.21.15.112:0/2421983751 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffae80c7030 con 0x7ffae00dbce0 2024-04-25T01:26:08.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.227+0000 7f1bd0ff9640 1 -- 172.21.15.112:0/2410534142 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1bc801f030 con 0x7f1bcc0d45b0 2024-04-25T01:26:08.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.236+0000 7f31a5ffb640 1 -- 172.21.15.112:0/392519154 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f31b00c1dc0 con 0x7f31b0150f40 2024-04-25T01:26:08.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.236+0000 7fc06f7fe640 1 -- 172.21.15.112:0/2995983968 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c017070 con 0x7fc0880c7f40 2024-04-25T01:26:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.237+0000 7fc06f7fe640 1 -- 172.21.15.112:0/2995983968 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc07c0052d0 con 0x7fc0880c7f40 2024-04-25T01:26:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.238+0000 7fc06f7fe640 1 -- 172.21.15.112:0/2995983968 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c003830 con 0x7fc0880c7f40 2024-04-25T01:26:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.238+0000 7fc08d15a640 1 -- 172.21.15.112:0/2995983968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0880aa9d0 con 0x7fc0880c7f40 2024-04-25T01:26:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.238+0000 7fc08d15a640 1 -- 172.21.15.112:0/2995983968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0881efe50 con 0x7fc0880c7f40 2024-04-25T01:26:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.240+0000 7fc08d15a640 1 -- 172.21.15.112:0/2995983968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0880689b0 con 0x7fc0880c7f40 2024-04-25T01:26:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.252+0000 7fc06f7fe640 1 -- 172.21.15.112:0/2995983968 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc07c01a070 con 0x7fc0880c7f40 2024-04-25T01:26:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.253+0000 7fc06f7fe640 1 --2- 172.21.15.112:0/2995983968 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc07406bf30 0x7fc07406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.253+0000 7fc06f7fe640 1 -- 172.21.15.112:0/2995983968 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fc07c08a8c0 con 0x7fc0880c7f40 2024-04-25T01:26:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.253+0000 7fc06f7fe640 1 -- 172.21.15.112:0/2995983968 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc07c021020 con 0x7fc0880c7f40 2024-04-25T01:26:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.254+0000 7fc086d9d640 1 --2- 172.21.15.112:0/2995983968 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc07406bf30 0x7fc07406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.255+0000 7fc086d9d640 1 --2- 172.21.15.112:0/2995983968 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc07406bf30 0x7fc07406e3f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc078009a30 tx=0x7fc078008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:08.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.635+0000 7fda42ae6640 1 -- 172.21.15.112:0/25896361 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fda3c0c2400 con 0x7fda3c0c7890 2024-04-25T01:26:08.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.636+0000 7fda327fc640 1 -- 172.21.15.112:0/25896361 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fda340799a0 con 0x7fda3c0c7890 2024-04-25T01:26:08.616 INFO:teuthology.orchestra.run.smithi112.stdout:55834574856 2024-04-25T01:26:08.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.639+0000 7f1bd953c640 1 -- 172.21.15.112:0/2410534142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1bcc005d90 con 0x7f1bcc0d45b0 2024-04-25T01:26:08.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.639+0000 7f23d359e640 1 -- 172.21.15.112:0/222110957 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f23d404f500 con 0x7f23d40c8ef0 2024-04-25T01:26:08.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.639+0000 7fda42ae6640 1 -- 172.21.15.112:0/25896361 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fda2006be80 msgr2=0x7fda2006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.639+0000 7fda42ae6640 1 --2- 172.21.15.112:0/25896361 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fda2006be80 0x7fda2006e340 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fda2c006fd0 tx=0x7fda2c008040 comp rx=0 tx=0).stop 2024-04-25T01:26:08.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.639+0000 7fda42ae6640 1 -- 172.21.15.112:0/25896361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda3c0c7890 msgr2=0x7fda3c0acab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.639+0000 7fda42ae6640 1 --2- 172.21.15.112:0/25896361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda3c0c7890 0x7fda3c0acab0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fda3c068f60 tx=0x7fda34004a40 comp rx=0 tx=0).stop 2024-04-25T01:26:08.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.639+0000 7fda42ae6640 1 -- 172.21.15.112:0/25896361 shutdown_connections 2024-04-25T01:26:08.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.639+0000 7fda42ae6640 1 --2- 172.21.15.112:0/25896361 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fda2006be80 0x7fda2006e340 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.639+0000 7fda42ae6640 1 --2- 172.21.15.112:0/25896361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda3c0c7890 0x7fda3c0acab0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.639+0000 7fda42ae6640 1 --2- 172.21.15.112:0/25896361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c072db0 0x7fda3c0ac570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.639+0000 7fda42ae6640 1 --2- 172.21.15.112:0/25896361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda3c070b10 0x7fda3c1effe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.639+0000 7fda42ae6640 1 -- 172.21.15.112:0/25896361 >> 172.21.15.112:0/25896361 conn(0x7fda3c0c6da0 msgr2=0x7fda3c0c7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:08.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.639+0000 7f1bd0ff9640 1 -- 172.21.15.112:0/2410534142 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1bc801f030 con 0x7f1bcc0d45b0 2024-04-25T01:26:08.620 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2024-04-25T01:26:08.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.639+0000 7f23c2ffd640 1 -- 172.21.15.112:0/222110957 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f23c4059500 con 0x7f23d40c8ef0 2024-04-25T01:26:08.622 INFO:teuthology.orchestra.run.smithi112.stdout:55834574856 2024-04-25T01:26:08.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.640+0000 7fda42ae6640 1 -- 172.21.15.112:0/25896361 shutdown_connections 2024-04-25T01:26:08.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.640+0000 7fda42ae6640 1 -- 172.21.15.112:0/25896361 wait complete. 2024-04-25T01:26:08.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f23d359e640 1 -- 172.21.15.112:0/222110957 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f23b406c030 msgr2=0x7f23b406e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f23d359e640 1 --2- 172.21.15.112:0/222110957 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f23b406c030 0x7f23b406e4f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f23c80011e0 tx=0x7f23c8009040 comp rx=0 tx=0).stop 2024-04-25T01:26:08.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f23d359e640 1 -- 172.21.15.112:0/222110957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d40c8ef0 msgr2=0x7f23d414c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f23d359e640 1 --2- 172.21.15.112:0/222110957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d40c8ef0 0x7f23d414c280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f23c400e9b0 tx=0x7f23c400ee70 comp rx=0 tx=0).stop 2024-04-25T01:26:08.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f23d359e640 1 -- 172.21.15.112:0/222110957 shutdown_connections 2024-04-25T01:26:08.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f23d359e640 1 --2- 172.21.15.112:0/222110957 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f23b406c030 0x7f23b406e4f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f23d359e640 1 --2- 172.21.15.112:0/222110957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d40cb4c0 0x7f23d414c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f23d359e640 1 --2- 172.21.15.112:0/222110957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d40c8ef0 0x7f23d414c280 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f23d359e640 1 --2- 172.21.15.112:0/222110957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d40b03d0 0x7f23d4150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f23d359e640 1 -- 172.21.15.112:0/222110957 >> 172.21.15.112:0/222110957 conn(0x7f23d40743d0 msgr2=0x7f23d41523f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:08.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f1bd953c640 1 -- 172.21.15.112:0/2410534142 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1bb806c030 msgr2=0x7f1bb806e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f1bd953c640 1 --2- 172.21.15.112:0/2410534142 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1bb806c030 0x7f1bb806e4f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1bc00090b0 tx=0x7f1bc002e000 comp rx=0 tx=0).stop 2024-04-25T01:26:08.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f1bd953c640 1 -- 172.21.15.112:0/2410534142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bcc0d45b0 msgr2=0x7f1bcc0dd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f1bd953c640 1 --2- 172.21.15.112:0/2410534142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bcc0d45b0 0x7f1bcc0dd2d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800b410 tx=0x7f1bc800b8d0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f1bd953c640 1 -- 172.21.15.112:0/2410534142 shutdown_connections 2024-04-25T01:26:08.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f1bd953c640 1 --2- 172.21.15.112:0/2410534142 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1bb806c030 0x7f1bb806e4f0 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1bc00090b0 tx=0x7f1bc002e000 comp rx=0 tx=0).stop 2024-04-25T01:26:08.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f1bd953c640 1 --2- 172.21.15.112:0/2410534142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bcc0de2a0 0x7f1bcc030210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f1bd953c640 1 --2- 172.21.15.112:0/2410534142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bcc0d45b0 0x7f1bcc0dd2d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f1bd953c640 1 --2- 172.21.15.112:0/2410534142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bcc00b5c0 0x7f1bcc0dcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f1bd953c640 1 -- 172.21.15.112:0/2410534142 >> 172.21.15.112:0/2410534142 conn(0x7f1bcc048ac0 msgr2=0x7f1bcc02db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:08.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f23d359e640 1 -- 172.21.15.112:0/222110957 shutdown_connections 2024-04-25T01:26:08.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f23d359e640 1 -- 172.21.15.112:0/222110957 wait complete. 2024-04-25T01:26:08.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.643+0000 7f1bd953c640 1 -- 172.21.15.112:0/2410534142 shutdown_connections 2024-04-25T01:26:08.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.644+0000 7f1bd953c640 1 -- 172.21.15.112:0/2410534142 wait complete. 2024-04-25T01:26:08.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.644+0000 7ff4db5c1640 1 -- 172.21.15.112:0/41429143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff4d40726f0 con 0x7ff4d4072ed0 2024-04-25T01:26:08.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.645+0000 7ff4c9ffb640 1 -- 172.21.15.112:0/41429143 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff4d005d310 con 0x7ff4d4072ed0 2024-04-25T01:26:08.626 INFO:teuthology.orchestra.run.smithi112.stdout:55834574856 2024-04-25T01:26:08.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.649+0000 7ffae759e640 1 -- 172.21.15.112:0/2421983751 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ffae000b590 con 0x7ffae00dbce0 2024-04-25T01:26:08.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.649+0000 7f2ef1d66640 1 -- 172.21.15.112:0/2070554888 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f2ee400b590 con 0x7f2ee40474a0 2024-04-25T01:26:08.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.650+0000 7ffad6ffd640 1 -- 172.21.15.112:0/2421983751 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffae80f5ab0 con 0x7ffae00dbce0 2024-04-25T01:26:08.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.650+0000 7f2ee97fa640 1 -- 172.21.15.112:0/2070554888 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2ee007d790 con 0x7f2ee40474a0 2024-04-25T01:26:08.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.650+0000 7ff4db5c1640 1 -- 172.21.15.112:0/41429143 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff4b806be30 msgr2=0x7ff4b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.650+0000 7ff4db5c1640 1 --2- 172.21.15.112:0/41429143 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff4b806be30 0x7ff4b806e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc033000 tx=0x7ff4cc02f000 comp rx=0 tx=0).stop 2024-04-25T01:26:08.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.650+0000 7ff4db5c1640 1 -- 172.21.15.112:0/41429143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4d4072ed0 msgr2=0x7ff4d41ebad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.650+0000 7ff4db5c1640 1 --2- 172.21.15.112:0/41429143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4d4072ed0 0x7ff4d41ebad0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000ec10 tx=0x7ff4d000c520 comp rx=0 tx=0).stop 2024-04-25T01:26:08.630 INFO:teuthology.orchestra.run.smithi112.stdout:55834574856 2024-04-25T01:26:08.632 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2024-04-25T01:26:08.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.652+0000 7ff4db5c1640 1 -- 172.21.15.112:0/41429143 shutdown_connections 2024-04-25T01:26:08.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.652+0000 7ff4db5c1640 1 --2- 172.21.15.112:0/41429143 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff4b806be30 0x7ff4b806e2f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.652+0000 7ff4db5c1640 1 --2- 172.21.15.112:0/41429143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4d414edb0 0x7ff4d4151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.652+0000 7ff4db5c1640 1 --2- 172.21.15.112:0/41429143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4d40c2ed0 0x7ff4d41ee6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.652+0000 7ff4db5c1640 1 --2- 172.21.15.112:0/41429143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4d4072ed0 0x7ff4d41ebad0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.652+0000 7ff4db5c1640 1 -- 172.21.15.112:0/41429143 >> 172.21.15.112:0/41429143 conn(0x7ff4d40c3a80 msgr2=0x7ff4d40c3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:08.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.652+0000 7ff4db5c1640 1 -- 172.21.15.112:0/41429143 shutdown_connections 2024-04-25T01:26:08.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.652+0000 7ff4db5c1640 1 -- 172.21.15.112:0/41429143 wait complete. 2024-04-25T01:26:08.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.653+0000 7f31b7407640 1 -- 172.21.15.112:0/392519154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f31b00689b0 con 0x7f31b0150f40 2024-04-25T01:26:08.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.654+0000 7ffae759e640 1 -- 172.21.15.112:0/2421983751 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ffac806be80 msgr2=0x7ffac806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.654+0000 7ffae759e640 1 --2- 172.21.15.112:0/2421983751 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ffac806be80 0x7ffac806e340 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffad8000f60 tx=0x7ffad8007e30 comp rx=0 tx=0).stop 2024-04-25T01:26:08.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.654+0000 7ffae759e640 1 -- 172.21.15.112:0/2421983751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffae00dbce0 msgr2=0x7ffae00deb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.654+0000 7ffae759e640 1 --2- 172.21.15.112:0/2421983751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffae00dbce0 0x7ffae00deb80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffae8073d70 tx=0x7ffae8064fe0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.654+0000 7ffae759e640 1 -- 172.21.15.112:0/2421983751 shutdown_connections 2024-04-25T01:26:08.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.654+0000 7ffae759e640 1 --2- 172.21.15.112:0/2421983751 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ffac806be80 0x7ffac806e340 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.654+0000 7ffae759e640 1 --2- 172.21.15.112:0/2421983751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffae00dbce0 0x7ffae00deb80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.654+0000 7ffae759e640 1 --2- 172.21.15.112:0/2421983751 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffae00474a0 0x7ffae0178440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.654+0000 7ffae759e640 1 --2- 172.21.15.112:0/2421983751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffae000a8b0 0x7ffae0175200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.654+0000 7ffae759e640 1 -- 172.21.15.112:0/2421983751 >> 172.21.15.112:0/2421983751 conn(0x7ffae0048250 msgr2=0x7ffae002bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:08.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.654+0000 7ffae759e640 1 -- 172.21.15.112:0/2421983751 shutdown_connections 2024-04-25T01:26:08.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.654+0000 7ffae759e640 1 -- 172.21.15.112:0/2421983751 wait complete. 2024-04-25T01:26:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.654+0000 7f31a5ffb640 1 -- 172.21.15.112:0/392519154 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f31a804b020 con 0x7f31b0150f40 2024-04-25T01:26:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.655+0000 7f2ef1d66640 1 -- 172.21.15.112:0/2070554888 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ed006be80 msgr2=0x7f2ed006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.655+0000 7f2ef1d66640 1 --2- 172.21.15.112:0/2070554888 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ed006be80 0x7f2ed006e340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80011e0 tx=0x7f2ed80016d0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.655+0000 7f2ef1d66640 1 -- 172.21.15.112:0/2070554888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ee40474a0 msgr2=0x7f2ee40d98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.655+0000 7f2ef1d66640 1 --2- 172.21.15.112:0/2070554888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ee40474a0 0x7f2ee40d98c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2ee00098c0 tx=0x7f2ee00046c0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.656+0000 7f2ef1d66640 1 -- 172.21.15.112:0/2070554888 shutdown_connections 2024-04-25T01:26:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.656+0000 7f2ef1d66640 1 --2- 172.21.15.112:0/2070554888 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ed006be80 0x7f2ed006e340 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.656+0000 7f2ef1d66640 1 --2- 172.21.15.112:0/2070554888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee40dbce0 0x7f2ee40d9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.656+0000 7f2ef1d66640 1 --2- 172.21.15.112:0/2070554888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ee40474a0 0x7f2ee40d98c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.656+0000 7f2ef1d66640 1 --2- 172.21.15.112:0/2070554888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ee40073e0 0x7f2ee40de9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.656+0000 7f2ef1d66640 1 -- 172.21.15.112:0/2070554888 >> 172.21.15.112:0/2070554888 conn(0x7f2ee4048250 msgr2=0x7f2ee40055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:08.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.656+0000 7f2ef1d66640 1 -- 172.21.15.112:0/2070554888 shutdown_connections 2024-04-25T01:26:08.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.656+0000 7f2ef1d66640 1 -- 172.21.15.112:0/2070554888 wait complete. 2024-04-25T01:26:08.637 INFO:teuthology.orchestra.run.smithi112.stdout:55834574856 2024-04-25T01:26:08.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.659+0000 7f31b7407640 1 -- 172.21.15.112:0/392519154 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f319406be80 msgr2=0x7f319406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.659+0000 7f31b7407640 1 --2- 172.21.15.112:0/392519154 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f319406be80 0x7f319406e340 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0095f0 tx=0x7f31ac006b50 comp rx=0 tx=0).stop 2024-04-25T01:26:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.659+0000 7f31b7407640 1 -- 172.21.15.112:0/392519154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b0150f40 msgr2=0x7f31b01f99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.659+0000 7f31b7407640 1 --2- 172.21.15.112:0/392519154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b0150f40 0x7f31b01f99a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009f40 tx=0x7f31a8004a40 comp rx=0 tx=0).stop 2024-04-25T01:26:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.660+0000 7f31b7407640 1 -- 172.21.15.112:0/392519154 shutdown_connections 2024-04-25T01:26:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.660+0000 7f31b7407640 1 --2- 172.21.15.112:0/392519154 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f319406be80 0x7f319406e340 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.660+0000 7f31b7407640 1 --2- 172.21.15.112:0/392519154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b0150f40 0x7f31b01f99a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.660+0000 7f31b7407640 1 --2- 172.21.15.112:0/392519154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31b00c5660 0x7f31b0156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.660+0000 7f31b7407640 1 --2- 172.21.15.112:0/392519154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31b006fbe0 0x7f31b00aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.660+0000 7f31b7407640 1 -- 172.21.15.112:0/392519154 >> 172.21.15.112:0/392519154 conn(0x7f31b00c4260 msgr2=0x7f31b00c63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:08.641 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.3 2024-04-25T01:26:08.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.660+0000 7f31b7407640 1 -- 172.21.15.112:0/392519154 shutdown_connections 2024-04-25T01:26:08.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.660+0000 7f31b7407640 1 -- 172.21.15.112:0/392519154 wait complete. 2024-04-25T01:26:08.644 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.6 2024-04-25T01:26:08.645 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2024-04-25T01:26:08.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.669+0000 7fc08d15a640 1 -- 172.21.15.112:0/2995983968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc0880ae420 con 0x7fc0880c7f40 2024-04-25T01:26:08.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.670+0000 7fc06f7fe640 1 -- 172.21.15.112:0/2995983968 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc0880adbd0 con 0x7fc0880c7f40 2024-04-25T01:26:08.649 INFO:teuthology.orchestra.run.smithi112.stdout:55834574857 2024-04-25T01:26:08.651 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2024-04-25T01:26:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.673+0000 7fc08d15a640 1 -- 172.21.15.112:0/2995983968 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc07406bf30 msgr2=0x7fc07406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.673+0000 7fc08d15a640 1 --2- 172.21.15.112:0/2995983968 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc07406bf30 0x7fc07406e3f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc078009a30 tx=0x7fc078008040 comp rx=0 tx=0).stop 2024-04-25T01:26:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.673+0000 7fc08d15a640 1 -- 172.21.15.112:0/2995983968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0880c7f40 msgr2=0x7fc0880ad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.673+0000 7fc08d15a640 1 --2- 172.21.15.112:0/2995983968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0880c7f40 0x7fc0880ad1a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00c450 tx=0x7fc07c00c910 comp rx=0 tx=0).stop 2024-04-25T01:26:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.673+0000 7fc08d15a640 1 -- 172.21.15.112:0/2995983968 shutdown_connections 2024-04-25T01:26:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.673+0000 7fc08d15a640 1 --2- 172.21.15.112:0/2995983968 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc07406bf30 0x7fc07406e3f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.673+0000 7fc08d15a640 1 --2- 172.21.15.112:0/2995983968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0880ab900 0x7fc0880a8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.673+0000 7fc08d15a640 1 --2- 172.21.15.112:0/2995983968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0880c7f40 0x7fc0880ad1a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.673+0000 7fc08d15a640 1 --2- 172.21.15.112:0/2995983968 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0880c4e40 0x7fc0880aa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.673+0000 7fc08d15a640 1 -- 172.21.15.112:0/2995983968 >> 172.21.15.112:0/2995983968 conn(0x7fc08806f2e0 msgr2=0x7fc08815a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.673+0000 7fc08d15a640 1 -- 172.21.15.112:0/2995983968 shutdown_connections 2024-04-25T01:26:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:08.673+0000 7fc08d15a640 1 -- 172.21.15.112:0/2995983968 wait complete. 2024-04-25T01:26:08.654 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.0 2024-04-25T01:26:08.655 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.4 2024-04-25T01:26:08.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2024-04-25T01:26:08.663 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2024-04-25T01:26:09.642 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T01:26:09.645 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T01:26:09.646 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T01:26:09.652 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T01:26:09.655 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T01:26:09.656 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T01:26:09.657 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T01:26:09.666 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T01:26:09.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.810+0000 7fe9d11ce640 1 Processor -- start 2024-04-25T01:26:09.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.810+0000 7fbeece72640 1 Processor -- start 2024-04-25T01:26:09.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.810+0000 7fbeece72640 1 -- start start 2024-04-25T01:26:09.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.810+0000 7f7521c9e640 1 Processor -- start 2024-04-25T01:26:09.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.810+0000 7f7521c9e640 1 -- start start 2024-04-25T01:26:09.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.811+0000 7fbeece72640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbee004bdf0 0x7fbee00e3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.811+0000 7fbeece72640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbee00d82f0 con 0x7fbee00d58e0 2024-04-25T01:26:09.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.811+0000 7fbeece72640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbee00d8590 con 0x7fbee004b8e0 2024-04-25T01:26:09.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.811+0000 7fbeece72640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbee0049dd0 con 0x7fbee004bdf0 2024-04-25T01:26:09.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.811+0000 7fe9d11ce640 1 -- start start 2024-04-25T01:26:09.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.811+0000 7fbee7fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbee004bdf0 0x7fbee00e3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.811+0000 7fbee6ffd640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fbee004b8e0 0x7fbee000cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:59592/0 (socket says 172.21.15.112:59592) 2024-04-25T01:26:09.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.811+0000 7fbee7fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbee004bdf0 0x7fbee00e3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49158/0 (socket says 172.21.15.112:49158) 2024-04-25T01:26:09.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.811+0000 7fbee6ffd640 1 -- 172.21.15.112:0/3797418901 learned_addr learned my addr 172.21.15.112:0/3797418901 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:09.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.812+0000 7f7521c9e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f75140dbce0 0x7f75140474a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.812+0000 7f7521c9e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75140df180 0x7f7514006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.812+0000 7f7521c9e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f751400bb50 con 0x7f75140073e0 2024-04-25T01:26:09.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.812+0000 7fbee67fc640 1 -- 172.21.15.112:0/3797418901 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 56248629 0 0) 0x7fbee00d8590 con 0x7fbee004b8e0 2024-04-25T01:26:09.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.812+0000 7f7521c9e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f751400a420 con 0x7f75140dbce0 2024-04-25T01:26:09.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.812+0000 7f7521c9e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7514048060 con 0x7f75140df180 2024-04-25T01:26:09.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.812+0000 7fbee67fc640 1 -- 172.21.15.112:0/3797418901 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbed8003520 con 0x7fbee004b8e0 2024-04-25T01:26:09.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.812+0000 7f751b7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f75140dbce0 0x7f75140474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.812+0000 7fe9d11ce640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9cc0c7f20 0x7fe9cc0c7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.813+0000 7fe9d11ce640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe9cc155c70 0x7fe9cc156100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.813+0000 7fe9d11ce640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9cc071660 con 0x7fe9cc153290 2024-04-25T01:26:09.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.813+0000 7fe9d11ce640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe9cc0c4f90 con 0x7fe9cc155c70 2024-04-25T01:26:09.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.813+0000 7fe9d11ce640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe9cc0c8790 con 0x7fe9cc0c7f20 2024-04-25T01:26:09.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.812+0000 7fbee7fff640 1 -- 172.21.15.112:0/3797418901 >> v1:172.21.15.187:6789/0 conn(0x7fbee004b8e0 legacy=0x7fbee000cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.812+0000 7fbee7fff640 1 -- 172.21.15.112:0/3797418901 >> v1:172.21.15.112:6789/0 conn(0x7fbee00d58e0 legacy=0x7fbee0048360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:09.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.813+0000 7fe9ca59c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9cc0c7f20 0x7fe9cc0c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.813+0000 7fe9ca59c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9cc0c7f20 0x7fe9cc0c7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:56598/0 (socket says 172.21.15.112:56598) 2024-04-25T01:26:09.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.813+0000 7fe9ca59c640 1 -- 172.21.15.112:0/2152063478 learned_addr learned my addr 172.21.15.112:0/2152063478 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:09.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.813+0000 7fe9cb59e640 1 --2- 172.21.15.112:0/2152063478 >> v2:172.21.15.187:3300/0 conn(0x7fe9cc155c70 0x7fe9cc156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.812+0000 7fbee7fff640 1 -- 172.21.15.112:0/3797418901 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee00d8870 con 0x7fbee004bdf0 2024-04-25T01:26:09.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.812+0000 7f751b7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f75140dbce0 0x7f75140474a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49168/0 (socket says 172.21.15.112:49168) 2024-04-25T01:26:09.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.813+0000 7fbee7fff640 1 --2- 172.21.15.112:0/3797418901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbee004bdf0 0x7fbee00e3b70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbee0049f50 tx=0x7fbedc02a150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ae357bbdb9e56a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.812+0000 7f751b7fe640 1 -- 172.21.15.112:0/2481077827 learned_addr learned my addr 172.21.15.112:0/2481077827 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:09.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.814+0000 7fbee67fc640 1 -- 172.21.15.112:0/3797418901 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc037070 con 0x7fbee004bdf0 2024-04-25T01:26:09.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.814+0000 7fbee67fc640 1 -- 172.21.15.112:0/3797418901 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbedc032440 con 0x7fbee004bdf0 2024-04-25T01:26:09.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.814+0000 7fbee67fc640 1 -- 172.21.15.112:0/3797418901 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc02ae60 con 0x7fbee004bdf0 2024-04-25T01:26:09.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.813+0000 7f751a7fc640 1 -- 172.21.15.112:0/2481077827 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1375097341 0 0) 0x7f751400bb50 con 0x7f75140073e0 2024-04-25T01:26:09.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.813+0000 7f751a7fc640 1 -- 172.21.15.112:0/2481077827 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f750c003520 con 0x7f75140073e0 2024-04-25T01:26:09.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.813+0000 7f751affd640 1 --2- 172.21.15.112:0/2481077827 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75140df180 0x7f7514006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.813+0000 7f751a7fc640 1 -- 172.21.15.112:0/2481077827 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2800063671 0 0) 0x7f750c003520 con 0x7f75140073e0 2024-04-25T01:26:09.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.813+0000 7f751a7fc640 1 -- 172.21.15.112:0/2481077827 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75140df180 msgr2=0x7f7514006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:09.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.813+0000 7f751a7fc640 1 --2- 172.21.15.112:0/2481077827 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75140df180 0x7f7514006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.813+0000 7f751a7fc640 1 -- 172.21.15.112:0/2481077827 >> v2:172.21.15.187:3300/0 conn(0x7f75140dbce0 msgr2=0x7f75140474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:09.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.813+0000 7f751a7fc640 1 --2- 172.21.15.112:0/2481077827 >> v2:172.21.15.187:3300/0 conn(0x7f75140dbce0 0x7f75140474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.813+0000 7f751a7fc640 1 -- 172.21.15.112:0/2481077827 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751400a0f0 con 0x7f75140073e0 2024-04-25T01:26:09.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.813+0000 7f751a7fc640 1 -- 172.21.15.112:0/2481077827 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f751c066ea0 con 0x7f75140073e0 2024-04-25T01:26:09.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.813+0000 7f751a7fc640 1 -- 172.21.15.112:0/2481077827 >> v1:172.21.15.112:6789/0 conn(0x7f75140073e0 legacy=0x7f751400a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.814+0000 7f751a7fc640 1 --2- 172.21.15.112:0/2481077827 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f750c0039b0 0x7f750c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.814+0000 7f751a7fc640 1 --2- 172.21.15.112:0/2481077827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f750c004350 0x7f750c004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.814+0000 7f751a7fc640 1 --2- 172.21.15.112:0/2481077827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f750c004f40 0x7f750c01c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.814+0000 7f751a7fc640 1 -- 172.21.15.112:0/2481077827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f751400a420 con 0x7f750c004350 2024-04-25T01:26:09.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.814+0000 7f751a7fc640 1 -- 172.21.15.112:0/2481077827 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7514048060 con 0x7f750c004f40 2024-04-25T01:26:09.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.814+0000 7fe9ca59c640 1 -- 172.21.15.112:0/2152063478 >> v2:172.21.15.187:3300/0 conn(0x7fe9cc155c70 msgr2=0x7fe9cc156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:09.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.814+0000 7fe9ca59c640 1 --2- 172.21.15.112:0/2152063478 >> v2:172.21.15.187:3300/0 conn(0x7fe9cc155c70 0x7fe9cc156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.814+0000 7fe9ca59c640 1 -- 172.21.15.112:0/2152063478 >> v1:172.21.15.112:6789/0 conn(0x7fe9cc153290 legacy=0x7fe9cc0c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:09.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.814+0000 7fe9ca59c640 1 -- 172.21.15.112:0/2152063478 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9cc0c8a70 con 0x7fe9cc0c7f20 2024-04-25T01:26:09.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.814+0000 7f751a7fc640 1 -- 172.21.15.112:0/2481077827 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f750c01cf10 con 0x7f750c0039b0 2024-04-25T01:26:09.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.814+0000 7f751bfff640 1 --2- 172.21.15.112:0/2481077827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f750c004f40 0x7f750c01c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.814+0000 7f751affd640 1 --2- 172.21.15.112:0/2481077827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f750c004350 0x7f750c004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.814+0000 7f751bfff640 1 -- 172.21.15.112:0/2481077827 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f750c0039b0 msgr2=0x7f750c003dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:26:09.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.814+0000 7f751bfff640 1 --2- 172.21.15.112:0/2481077827 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f750c0039b0 0x7f750c003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.814+0000 7f751bfff640 1 -- 172.21.15.112:0/2481077827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f750c004350 msgr2=0x7f750c004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:09.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.814+0000 7f751bfff640 1 --2- 172.21.15.112:0/2481077827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f750c004350 0x7f750c004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.814+0000 7f751bfff640 1 -- 172.21.15.112:0/2481077827 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751400a0f0 con 0x7f750c004f40 2024-04-25T01:26:09.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.815+0000 7fe9ca59c640 1 --2- 172.21.15.112:0/2152063478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9cc0c7f20 0x7fe9cc0c7900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4009e50 tx=0x7fe9c402a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98bb36d7f330caa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.815+0000 7fe9c9d9b640 1 -- 172.21.15.112:0/2152063478 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4036070 con 0x7fe9cc0c7f20 2024-04-25T01:26:09.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.815+0000 7fe9c9d9b640 1 -- 172.21.15.112:0/2152063478 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c4031600 con 0x7fe9cc0c7f20 2024-04-25T01:26:09.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.815+0000 7fe9c9d9b640 1 -- 172.21.15.112:0/2152063478 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c40319b0 con 0x7fe9cc0c7f20 2024-04-25T01:26:09.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.815+0000 7f751bfff640 1 --2- 172.21.15.112:0/2481077827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f750c004f40 0x7f750c01c9d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f751c06a130 tx=0x7f750c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7748b2c42e7ffce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.816+0000 7f751a7fc640 1 -- 172.21.15.112:0/2481077827 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c0753e0 con 0x7f750c004f40 2024-04-25T01:26:09.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.816+0000 7f751a7fc640 1 -- 172.21.15.112:0/2481077827 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f751c0ad600 con 0x7f750c004f40 2024-04-25T01:26:09.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.816+0000 7f751a7fc640 1 -- 172.21.15.112:0/2481077827 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c0ada70 con 0x7f750c004f40 2024-04-25T01:26:09.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.816+0000 7fe9d11ce640 1 -- 172.21.15.112:0/2152063478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9cc0c7f20 msgr2=0x7fe9cc0c7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.816+0000 7fe9d11ce640 1 --2- 172.21.15.112:0/2152063478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9cc0c7f20 0x7fe9cc0c7900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4009e50 tx=0x7fe9c402a6f0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.816+0000 7fe9d11ce640 1 -- 172.21.15.112:0/2152063478 shutdown_connections 2024-04-25T01:26:09.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.816+0000 7fe9d11ce640 1 --2- 172.21.15.112:0/2152063478 >> v2:172.21.15.187:3300/0 conn(0x7fe9cc155c70 0x7fe9cc156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.816+0000 7fe9d11ce640 1 --2- 172.21.15.112:0/2152063478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9cc0c7f20 0x7fe9cc0c7900 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.816+0000 7fe9d11ce640 1 -- 172.21.15.112:0/2152063478 >> 172.21.15.112:0/2152063478 conn(0x7fe9cc074a00 msgr2=0x7fe9cc0c8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:09.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.817+0000 7fe9d11ce640 1 -- 172.21.15.112:0/2152063478 shutdown_connections 2024-04-25T01:26:09.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.817+0000 7f7521c9e640 1 -- 172.21.15.112:0/2481077827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f750c004f40 msgr2=0x7f750c01c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.817+0000 7f7521c9e640 1 --2- 172.21.15.112:0/2481077827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f750c004f40 0x7f750c01c9d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f751c06a130 tx=0x7f750c005830 comp rx=0 tx=0).stop 2024-04-25T01:26:09.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.816+0000 7fbeece72640 1 -- 172.21.15.112:0/3797418901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbee004bdf0 msgr2=0x7fbee00e3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.816+0000 7fbeece72640 1 --2- 172.21.15.112:0/3797418901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbee004bdf0 0x7fbee00e3b70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbee0049f50 tx=0x7fbedc02a150 comp rx=0 tx=0).stop 2024-04-25T01:26:09.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.816+0000 7fbeece72640 1 -- 172.21.15.112:0/3797418901 shutdown_connections 2024-04-25T01:26:09.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.817+0000 7f751bfff640 1 -- 172.21.15.112:0/2481077827 reap_dead start 2024-04-25T01:26:09.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.816+0000 7fbeece72640 1 --2- 172.21.15.112:0/3797418901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbee004bdf0 0x7fbee00e3b70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.816+0000 7fbeece72640 1 -- 172.21.15.112:0/3797418901 >> 172.21.15.112:0/3797418901 conn(0x7fbee002f5d0 msgr2=0x7fbee002f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:09.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.816+0000 7fbeece72640 1 -- 172.21.15.112:0/3797418901 shutdown_connections 2024-04-25T01:26:09.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.816+0000 7fbeece72640 1 -- 172.21.15.112:0/3797418901 wait complete. 2024-04-25T01:26:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.817+0000 7f7521c9e640 1 -- 172.21.15.112:0/2481077827 shutdown_connections 2024-04-25T01:26:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.817+0000 7f7521c9e640 1 -- 172.21.15.112:0/2481077827 >> 172.21.15.112:0/2481077827 conn(0x7f7514048250 msgr2=0x7f7514048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.817+0000 7f7521c9e640 1 -- 172.21.15.112:0/2481077827 shutdown_connections 2024-04-25T01:26:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.818+0000 7fbeece72640 1 Processor -- start 2024-04-25T01:26:09.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.817+0000 7fe9d11ce640 1 -- 172.21.15.112:0/2152063478 wait complete. 2024-04-25T01:26:09.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.819+0000 7fe9d11ce640 1 Processor -- start 2024-04-25T01:26:09.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.818+0000 7fbeece72640 1 -- start start 2024-04-25T01:26:09.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.817+0000 7f7521c9e640 1 -- 172.21.15.112:0/2481077827 wait complete. 2024-04-25T01:26:09.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.819+0000 7f7521c9e640 1 Processor -- start 2024-04-25T01:26:09.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.820+0000 7fbeece72640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbee004b8e0 0x7fbee00c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.820+0000 7fbeece72640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee004bdf0 0x7fbee00c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.819+0000 7f7521c9e640 1 -- start start 2024-04-25T01:26:09.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.819+0000 7fe9d11ce640 1 -- start start 2024-04-25T01:26:09.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.820+0000 7fe9d11ce640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9cc0c7f20 0x7fe9cc1e1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.820+0000 7fe9d11ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9cc153290 0x7fe9cc0adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.820+0000 7fe9d11ce640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9cc155c70 0x7fe9cc14e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.820+0000 7fe9d11ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc0c6100 con 0x7fe9cc153290 2024-04-25T01:26:09.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.820+0000 7fe9d11ce640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc0712f0 con 0x7fe9cc0c7f20 2024-04-25T01:26:09.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.820+0000 7fe9d11ce640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe9cc0c6280 con 0x7fe9cc155c70 2024-04-25T01:26:09.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.820+0000 7fe9ca59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9cc153290 0x7fe9cc0adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.820+0000 7fe9ca59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9cc153290 0x7fe9cc0adc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43486/0 (socket says 172.21.15.112:43486) 2024-04-25T01:26:09.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.820+0000 7fe9ca59c640 1 -- 172.21.15.112:0/2886806375 learned_addr learned my addr 172.21.15.112:0/2886806375 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:09.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.820+0000 7fbee77fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbee004b8e0 0x7fbee00c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.820+0000 7fbee77fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbee004b8e0 0x7fbee00c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49182/0 (socket says 172.21.15.112:49182) 2024-04-25T01:26:09.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.820+0000 7fbee6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee004bdf0 0x7fbee00c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.820+0000 7fbee6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee004bdf0 0x7fbee00c51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43492/0 (socket says 172.21.15.112:43492) 2024-04-25T01:26:09.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.820+0000 7fbee77fe640 1 -- 172.21.15.112:0/236070368 learned_addr learned my addr 172.21.15.112:0/236070368 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:09.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.820+0000 7fbeece72640 1 --2- 172.21.15.112:0/236070368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbee00d58e0 0x7fbee00c5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.821+0000 7fbeece72640 1 -- 172.21.15.112:0/236070368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbee004b200 con 0x7fbee004bdf0 2024-04-25T01:26:09.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.821+0000 7fbeece72640 1 -- 172.21.15.112:0/236070368 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbee004c2d0 con 0x7fbee004b8e0 2024-04-25T01:26:09.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.821+0000 7fbeece72640 1 -- 172.21.15.112:0/236070368 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbee0048a70 con 0x7fbee00d58e0 2024-04-25T01:26:09.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.821+0000 7fbee7fff640 1 --2- 172.21.15.112:0/236070368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbee00d58e0 0x7fbee00c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.821+0000 7fe9ca59c640 1 -- 172.21.15.112:0/2886806375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9cc155c70 msgr2=0x7fe9cc14e9c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:26:09.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.821+0000 7fe9ca59c640 1 --2- 172.21.15.112:0/2886806375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9cc155c70 0x7fe9cc14e9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.821+0000 7fe9ca59c640 1 -- 172.21.15.112:0/2886806375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9cc0c7f20 msgr2=0x7fe9cc1e1590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:26:09.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.821+0000 7fe9ca59c640 1 --2- 172.21.15.112:0/2886806375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9cc0c7f20 0x7fe9cc1e1590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.821+0000 7fe9ca59c640 1 -- 172.21.15.112:0/2886806375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9cc06ef90 con 0x7fe9cc153290 2024-04-25T01:26:09.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.821+0000 7fe9ca59c640 1 --2- 172.21.15.112:0/2886806375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9cc153290 0x7fe9cc0adc00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4009f70 tx=0x7fe9c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.821+0000 7fbee77fe640 1 -- 172.21.15.112:0/236070368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbee00d58e0 msgr2=0x7fbee00c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.821+0000 7fbee77fe640 1 --2- 172.21.15.112:0/236070368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbee00d58e0 0x7fbee00c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.821+0000 7fbee77fe640 1 -- 172.21.15.112:0/236070368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee004bdf0 msgr2=0x7fbee00c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.821+0000 7fbee77fe640 1 --2- 172.21.15.112:0/236070368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee004bdf0 0x7fbee00c51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.821+0000 7fbee77fe640 1 -- 172.21.15.112:0/236070368 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee00d6010 con 0x7fbee004b8e0 2024-04-25T01:26:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.821+0000 7fbee6ffd640 1 --2- 172.21.15.112:0/236070368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee004bdf0 0x7fbee00c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:26:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.821+0000 7f7521c9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f750c004f40 0x7f751416c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.822+0000 7fe9b37fe640 1 -- 172.21.15.112:0/2886806375 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4036050 con 0x7fe9cc153290 2024-04-25T01:26:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.822+0000 7f7521c9e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7514049380 0x7f751402c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.822+0000 7f751b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f750c004f40 0x7f751416c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.822+0000 7f751b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f750c004f40 0x7f751416c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43494/0 (socket says 172.21.15.112:43494) 2024-04-25T01:26:09.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.822+0000 7fe9b37fe640 1 -- 172.21.15.112:0/2886806375 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c4042070 con 0x7fe9cc153290 2024-04-25T01:26:09.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.822+0000 7fe9b37fe640 1 -- 172.21.15.112:0/2886806375 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4009510 con 0x7fe9cc153290 2024-04-25T01:26:09.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.822+0000 7fe9d11ce640 1 -- 172.21.15.112:0/2886806375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9cc0ac2b0 con 0x7fe9cc153290 2024-04-25T01:26:09.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.822+0000 7fe9d11ce640 1 -- 172.21.15.112:0/2886806375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9cc0aa890 con 0x7fe9cc153290 2024-04-25T01:26:09.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.822+0000 7fbee77fe640 1 --2- 172.21.15.112:0/236070368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbee004b8e0 0x7fbee00c9da0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbed400c410 tx=0x7fbed400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.822+0000 7f751b7fe640 1 -- 172.21.15.112:0/3424102426 learned_addr learned my addr 172.21.15.112:0/3424102426 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.822+0000 7f7521c9e640 1 --2- 172.21.15.112:0/3424102426 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f751402c7a0 0x7f75140c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.822+0000 7f751affd640 1 --2- 172.21.15.112:0/3424102426 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7514049380 0x7f751402c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.822+0000 7f7521c9e640 1 -- 172.21.15.112:0/3424102426 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75140dc0c0 con 0x7f750c004f40 2024-04-25T01:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.822+0000 7f7521c9e640 1 -- 172.21.15.112:0/3424102426 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7514047d80 con 0x7f7514049380 2024-04-25T01:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.823+0000 7f751bfff640 1 --2- 172.21.15.112:0/3424102426 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f751402c7a0 0x7f75140c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.823+0000 7f7521c9e640 1 -- 172.21.15.112:0/3424102426 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f75140df5c0 con 0x7f751402c7a0 2024-04-25T01:26:09.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.823+0000 7fbed3fff640 1 -- 172.21.15.112:0/236070368 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4017070 con 0x7fbee004b8e0 2024-04-25T01:26:09.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.823+0000 7fbed3fff640 1 -- 172.21.15.112:0/236070368 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed40035f0 con 0x7fbee004b8e0 2024-04-25T01:26:09.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.824+0000 7fe9b37fe640 1 -- 172.21.15.112:0/2886806375 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe9c4004240 con 0x7fe9cc153290 2024-04-25T01:26:09.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.825+0000 7fe9b37fe640 1 --2- 172.21.15.112:0/2886806375 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe9b806bf00 0x7fe9b806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.825+0000 7fe9b37fe640 1 -- 172.21.15.112:0/2886806375 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fe9c40aba60 con 0x7fe9cc153290 2024-04-25T01:26:09.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.824+0000 7fbeece72640 1 -- 172.21.15.112:0/236070368 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee002d220 con 0x7fbee004b8e0 2024-04-25T01:26:09.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.824+0000 7fbeece72640 1 -- 172.21.15.112:0/236070368 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee002f060 con 0x7fbee004b8e0 2024-04-25T01:26:09.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.825+0000 7fbed1ffb640 1 -- 172.21.15.112:0/236070368 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee00057e0 con 0x7fbee004b8e0 2024-04-25T01:26:09.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.823+0000 7f751b7fe640 1 -- 172.21.15.112:0/3424102426 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f751402c7a0 msgr2=0x7f75140c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.823+0000 7f751b7fe640 1 --2- 172.21.15.112:0/3424102426 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f751402c7a0 0x7f75140c7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.823+0000 7f751b7fe640 1 -- 172.21.15.112:0/3424102426 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7514049380 msgr2=0x7f751402c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.823+0000 7f751b7fe640 1 --2- 172.21.15.112:0/3424102426 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7514049380 0x7f751402c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.823+0000 7f751b7fe640 1 -- 172.21.15.112:0/3424102426 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751402d0b0 con 0x7f750c004f40 2024-04-25T01:26:09.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.824+0000 7f751b7fe640 1 --2- 172.21.15.112:0/3424102426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f750c004f40 0x7f751416c0b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f751000e810 tx=0x7f751000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.824+0000 7f751affd640 1 --2- 172.21.15.112:0/3424102426 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7514049380 0x7f751402c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:09.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.824+0000 7f751bfff640 1 --2- 172.21.15.112:0/3424102426 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f751402c7a0 0x7f75140c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:09.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.824+0000 7f7507fff640 1 -- 172.21.15.112:0/3424102426 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510017020 con 0x7f750c004f40 2024-04-25T01:26:09.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.825+0000 7f7521c9e640 1 -- 172.21.15.112:0/3424102426 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75140498f0 con 0x7f750c004f40 2024-04-25T01:26:09.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.826+0000 7fe9cad9d640 1 --2- 172.21.15.112:0/2886806375 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe9b806bf00 0x7fe9b806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.825+0000 7fbed3fff640 1 -- 172.21.15.112:0/236070368 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed40039e0 con 0x7fbee004b8e0 2024-04-25T01:26:09.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.826+0000 7f7507fff640 1 -- 172.21.15.112:0/3424102426 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7510004030 con 0x7f750c004f40 2024-04-25T01:26:09.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.827+0000 7f7507fff640 1 -- 172.21.15.112:0/3424102426 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751000f040 con 0x7f750c004f40 2024-04-25T01:26:09.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.827+0000 7fe9cad9d640 1 --2- 172.21.15.112:0/2886806375 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe9b806bf00 0x7fe9b806e3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0014f0 tx=0x7fe9bc007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.827+0000 7fe9d11ce640 1 -- 172.21.15.112:0/2886806375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9cc0ac0a0 con 0x7fe9cc153290 2024-04-25T01:26:09.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.827+0000 7f7521c9e640 1 -- 172.21.15.112:0/3424102426 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7514049eb0 con 0x7f750c004f40 2024-04-25T01:26:09.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.828+0000 7fbed3fff640 1 -- 172.21.15.112:0/236070368 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbed401a070 con 0x7fbee004b8e0 2024-04-25T01:26:09.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.828+0000 7fbed3fff640 1 --2- 172.21.15.112:0/236070368 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbed806c320 0x7fbed806e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.829+0000 7fbed3fff640 1 -- 172.21.15.112:0/236070368 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fbed408a7a0 con 0x7fbee004b8e0 2024-04-25T01:26:09.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.829+0000 7fbee6ffd640 1 --2- 172.21.15.112:0/236070368 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbed806c320 0x7fbed806e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.830+0000 7fbee6ffd640 1 --2- 172.21.15.112:0/236070368 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbed806c320 0x7fbed806e7e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbee8069430 tx=0x7fbee8076b30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.830+0000 7f7507fff640 1 -- 172.21.15.112:0/3424102426 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f751001a020 con 0x7f750c004f40 2024-04-25T01:26:09.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.830+0000 7f7507fff640 1 --2- 172.21.15.112:0/3424102426 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f750006beb0 0x7f750006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.831+0000 7fbed3fff640 1 -- 172.21.15.112:0/236070368 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbed4021020 con 0x7fbee004b8e0 2024-04-25T01:26:09.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.831+0000 7f7521c9e640 1 -- 172.21.15.112:0/3424102426 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74fc005290 con 0x7f750c004f40 2024-04-25T01:26:09.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.832+0000 7f751affd640 1 --2- 172.21.15.112:0/3424102426 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f750006beb0 0x7f750006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.832+0000 7f751affd640 1 --2- 172.21.15.112:0/3424102426 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f750006beb0 0x7f750006e370 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f75140c9e20 tx=0x7f7508008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.837+0000 7fe9b37fe640 1 -- 172.21.15.112:0/2886806375 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe9c404b020 con 0x7fe9cc153290 2024-04-25T01:26:09.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.846+0000 7f7507fff640 1 -- 172.21.15.112:0/3424102426 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f7510056290 con 0x7f750c004f40 2024-04-25T01:26:09.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.846+0000 7f7507fff640 1 -- 172.21.15.112:0/3424102426 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f751004e2d0 con 0x7f750c004f40 2024-04-25T01:26:09.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.878+0000 7f3089563640 1 Processor -- start 2024-04-25T01:26:09.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.878+0000 7f3089563640 1 -- start start 2024-04-25T01:26:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.879+0000 7f3089563640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3084150d80 0x7f3084154470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.879+0000 7f76845f9640 1 Processor -- start 2024-04-25T01:26:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.879+0000 7f9078488640 1 Processor -- start 2024-04-25T01:26:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.879+0000 7f9078488640 1 -- start start 2024-04-25T01:26:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.879+0000 7f6bb24b2640 1 Processor -- start 2024-04-25T01:26:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.879+0000 7f70e3812640 1 Processor -- start 2024-04-25T01:26:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.880+0000 7f3089563640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3084071850 con 0x7f3084071000 2024-04-25T01:26:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.880+0000 7f3089563640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3084071570 con 0x7f30840c3900 2024-04-25T01:26:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.880+0000 7f3082ffd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3084150d80 0x7f3084154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.880+0000 7f3082ffd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3084150d80 0x7f3084154470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49198/0 (socket says 172.21.15.112:49198) 2024-04-25T01:26:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.880+0000 7f3082ffd640 1 -- 172.21.15.112:0/1419563518 learned_addr learned my addr 172.21.15.112:0/1419563518 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.880+0000 7f3089563640 1 -- 172.21.15.112:0/1419563518 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3084153990 con 0x7f3084150d80 2024-04-25T01:26:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f3082ffd640 1 -- 172.21.15.112:0/1419563518 >> v1:172.21.15.187:6789/0 conn(0x7f30840c3900 legacy=0x7f30840c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f3082ffd640 1 -- 172.21.15.112:0/1419563518 >> v1:172.21.15.112:6789/0 conn(0x7f3084071000 legacy=0x7f30840c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f3082ffd640 1 -- 172.21.15.112:0/1419563518 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3084153c70 con 0x7f3084150d80 2024-04-25T01:26:09.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.880+0000 7f6bb24b2640 1 -- start start 2024-04-25T01:26:09.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f6bb24b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bac0ac050 0x7f6bac070b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f6bb24b2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6bac155960 0x7f6bac15c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f6bb24b2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bac0ac430 con 0x7f6bac155500 2024-04-25T01:26:09.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f6bb24b2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f6bac0c2840 con 0x7f6bac155960 2024-04-25T01:26:09.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f6bb24b2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6bac154d90 con 0x7f6bac0ac050 2024-04-25T01:26:09.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f6bb1cb1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6bac155960 0x7f6bac15c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.882+0000 7f6bb1cb1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6bac155960 0x7f6bac15c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43506/0 (socket says 172.21.15.112:43506) 2024-04-25T01:26:09.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.882+0000 7f6bb1cb1640 1 -- 172.21.15.112:0/969851964 learned_addr learned my addr 172.21.15.112:0/969851964 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:09.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.880+0000 7f70e3812640 1 -- start start 2024-04-25T01:26:09.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f70e3812640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f70d40474d0 0x7f70d4006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f70e3812640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d4007410 0x7f70d400a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f70e3812640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70d400bb80 con 0x7f70d40dbd10 2024-04-25T01:26:09.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f70e3812640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f70d400a450 con 0x7f70d40474d0 2024-04-25T01:26:09.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f70e3812640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f70d4048090 con 0x7f70d4007410 2024-04-25T01:26:09.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f70e1daf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d4007410 0x7f70d400a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f70e1daf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d4007410 0x7f70d400a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:56642/0 (socket says 172.21.15.112:56642) 2024-04-25T01:26:09.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.880+0000 7f76845f9640 1 -- start start 2024-04-25T01:26:09.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f76845f9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f767c0730b0 0x7f767c0c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f76845f9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f767c155c70 con 0x7f767c06f510 2024-04-25T01:26:09.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f76845f9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f767c0ac450 con 0x7f767c0c1fc0 2024-04-25T01:26:09.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f76845f9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f767c0ac200 con 0x7f767c0730b0 2024-04-25T01:26:09.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f7681b94640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f767c0730b0 0x7f767c0c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f9078488640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f907008b9d0 0x7f907006f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f9078488640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90701633e0 con 0x7f9070072d50 2024-04-25T01:26:09.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f9078488640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90700ab500 con 0x7f90700abfe0 2024-04-25T01:26:09.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f9078488640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f90700c3990 con 0x7f907008b9d0 2024-04-25T01:26:09.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f9075a23640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f907008b9d0 0x7f907006f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f9076a25640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f90700abfe0 0x7f9070070d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:59614/0 (socket says 172.21.15.112:59614) 2024-04-25T01:26:09.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f9076a25640 1 -- 172.21.15.112:0/3921570154 learned_addr learned my addr 172.21.15.112:0/3921570154 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:09.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f70e1daf640 1 -- 172.21.15.112:0/3146112441 learned_addr learned my addr 172.21.15.112:0/3146112441 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:09.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.882+0000 7f70e1daf640 1 -- 172.21.15.112:0/3146112441 >> v2:172.21.15.112:3300/0 conn(0x7f70d40474d0 msgr2=0x7f70d4006ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:09.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.882+0000 7f70e1daf640 1 --2- 172.21.15.112:0/3146112441 >> v2:172.21.15.112:3300/0 conn(0x7f70d40474d0 0x7f70d4006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.882+0000 7f70e1daf640 1 -- 172.21.15.112:0/3146112441 >> v1:172.21.15.112:6789/0 conn(0x7f70d40dbd10 legacy=0x7f70d40df1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:09.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.882+0000 7f6bb14b0640 1 --2- 172.21.15.112:0/969851964 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bac0ac050 0x7f6bac070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.882+0000 7f7682395640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f767c06f510 0x7f767c071170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38058/0 (socket says 172.21.15.112:38058) 2024-04-25T01:26:09.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.882+0000 7f7682395640 1 -- 172.21.15.112:0/2900611891 learned_addr learned my addr 172.21.15.112:0/2900611891 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:09.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.882+0000 7f3082ffd640 1 --2- 172.21.15.112:0/1419563518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3084150d80 0x7f3084154470 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3074009870 tx=0x7f307402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb9fba7f989e88b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.881+0000 7f7681b94640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f767c0730b0 0x7f767c0c1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43500/0 (socket says 172.21.15.112:43500) 2024-04-25T01:26:09.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.882+0000 7f7681393640 1 -- 172.21.15.112:0/2900611891 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2538422346 0 0) 0x7f767c0ac450 con 0x7f767c0c1fc0 2024-04-25T01:26:09.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.882+0000 7f7681393640 1 -- 172.21.15.112:0/2900611891 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7668003520 con 0x7f767c0c1fc0 2024-04-25T01:26:09.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.882+0000 7f7681393640 1 -- 172.21.15.112:0/2900611891 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1062856602 0 0) 0x7f767c155c70 con 0x7f767c06f510 2024-04-25T01:26:09.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.882+0000 7f7681393640 1 -- 172.21.15.112:0/2900611891 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f767c0ac450 con 0x7f767c06f510 2024-04-25T01:26:09.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.883+0000 7f3081ffb640 1 -- 172.21.15.112:0/1419563518 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3074037070 con 0x7f3084150d80 2024-04-25T01:26:09.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.882+0000 7f6bb14b0640 1 -- 172.21.15.112:0/969851964 >> v2:172.21.15.112:3300/0 conn(0x7f6bac155960 msgr2=0x7f6bac15c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:09.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.882+0000 7f6bb14b0640 1 --2- 172.21.15.112:0/969851964 >> v2:172.21.15.112:3300/0 conn(0x7f6bac155960 0x7f6bac15c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.882+0000 7f6bb14b0640 1 -- 172.21.15.112:0/969851964 >> v1:172.21.15.187:6789/0 conn(0x7f6bac155500 legacy=0x7f6bac06cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:09.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.882+0000 7f6bb14b0640 1 -- 172.21.15.112:0/969851964 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bac0c2a90 con 0x7f6bac0ac050 2024-04-25T01:26:09.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.883+0000 7f3081ffb640 1 -- 172.21.15.112:0/1419563518 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3074032440 con 0x7f3084150d80 2024-04-25T01:26:09.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.882+0000 7f70e1daf640 1 -- 172.21.15.112:0/3146112441 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d400a120 con 0x7f70d4007410 2024-04-25T01:26:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.883+0000 7f9075222640 1 -- 172.21.15.112:0/3921570154 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1317120641 0 0) 0x7f90701633e0 con 0x7f9070072d50 2024-04-25T01:26:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.883+0000 7f9075222640 1 -- 172.21.15.112:0/3921570154 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f905c003520 con 0x7f9070072d50 2024-04-25T01:26:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.883+0000 7f9075a23640 1 -- 172.21.15.112:0/3921570154 >> v1:172.21.15.187:6789/0 conn(0x7f90700abfe0 legacy=0x7f9070070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.883+0000 7f9075a23640 1 -- 172.21.15.112:0/3921570154 >> v1:172.21.15.112:6789/0 conn(0x7f9070072d50 legacy=0x7f90700c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.883+0000 7f9075a23640 1 -- 172.21.15.112:0/3921570154 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90700ab7e0 con 0x7f907008b9d0 2024-04-25T01:26:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.883+0000 7f3081ffb640 1 -- 172.21.15.112:0/1419563518 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3074032810 con 0x7f3084150d80 2024-04-25T01:26:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.883+0000 7f70e1daf640 1 --2- 172.21.15.112:0/3146112441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d4007410 0x7f70d400a8e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f70dc04fa20 tx=0x7f70dc0a5ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1876b5b5514cc1b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.883+0000 7f7681393640 1 -- 172.21.15.112:0/2900611891 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1490962782 0 0) 0x7f7668003520 con 0x7f767c0c1fc0 2024-04-25T01:26:09.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.883+0000 7f7681393640 1 -- 172.21.15.112:0/2900611891 >> v2:172.21.15.112:3300/0 conn(0x7f767c0730b0 msgr2=0x7f767c0c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:09.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f7681393640 1 --2- 172.21.15.112:0/2900611891 >> v2:172.21.15.112:3300/0 conn(0x7f767c0730b0 0x7f767c0c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f70d3fff640 1 -- 172.21.15.112:0/3146112441 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc0b3070 con 0x7f70d4007410 2024-04-25T01:26:09.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f3089563640 1 -- 172.21.15.112:0/1419563518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3084150d80 msgr2=0x7f3084154470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f3089563640 1 --2- 172.21.15.112:0/1419563518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3084150d80 0x7f3084154470 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3074009870 tx=0x7f307402a210 comp rx=0 tx=0).stop 2024-04-25T01:26:09.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f70d3fff640 1 -- 172.21.15.112:0/3146112441 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70dc0a6c90 con 0x7f70d4007410 2024-04-25T01:26:09.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f70d3fff640 1 -- 172.21.15.112:0/3146112441 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc0ae660 con 0x7f70d4007410 2024-04-25T01:26:09.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f3089563640 1 -- 172.21.15.112:0/1419563518 shutdown_connections 2024-04-25T01:26:09.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f3089563640 1 --2- 172.21.15.112:0/1419563518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3084150d80 0x7f3084154470 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f3089563640 1 -- 172.21.15.112:0/1419563518 >> 172.21.15.112:0/1419563518 conn(0x7f3084074a90 msgr2=0x7f30840c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:09.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f6bb14b0640 1 --2- 172.21.15.112:0/969851964 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bac0ac050 0x7f6bac070b60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80097a0 tx=0x7f6ba802a0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a6a5f3af17d1194 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f3089563640 1 -- 172.21.15.112:0/1419563518 shutdown_connections 2024-04-25T01:26:09.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f7681393640 1 -- 172.21.15.112:0/2900611891 >> v1:172.21.15.112:6789/0 conn(0x7f767c06f510 legacy=0x7f767c071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f7681393640 1 -- 172.21.15.112:0/2900611891 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767c154eb0 con 0x7f767c0c1fc0 2024-04-25T01:26:09.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f3089563640 1 -- 172.21.15.112:0/1419563518 wait complete. 2024-04-25T01:26:09.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f7681393640 1 -- 172.21.15.112:0/2900611891 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f76740019f0 con 0x7f767c0c1fc0 2024-04-25T01:26:09.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f6ba3fff640 1 -- 172.21.15.112:0/969851964 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8037070 con 0x7f6bac0ac050 2024-04-25T01:26:09.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f6ba3fff640 1 -- 172.21.15.112:0/969851964 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba802adc0 con 0x7f6bac0ac050 2024-04-25T01:26:09.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f7681393640 1 -- 172.21.15.112:0/2900611891 >> v1:172.21.15.187:6789/0 conn(0x7f767c0c1fc0 legacy=0x7f767c14e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f7681393640 1 --2- 172.21.15.112:0/2900611891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7668004bd0 0x7f7668004fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f7681393640 1 --2- 172.21.15.112:0/2900611891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76680037a0 0x7f7668003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f7681393640 1 --2- 172.21.15.112:0/2900611891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7668004160 0x7f766800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f7681393640 1 -- 172.21.15.112:0/2900611891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f767c0ac200 con 0x7f7668004bd0 2024-04-25T01:26:09.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f7681393640 1 -- 172.21.15.112:0/2900611891 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f766800c0c0 con 0x7f7668004160 2024-04-25T01:26:09.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f7681393640 1 -- 172.21.15.112:0/2900611891 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f766800c2c0 con 0x7f76680037a0 2024-04-25T01:26:09.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f7682395640 1 --2- 172.21.15.112:0/2900611891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7668004bd0 0x7f7668004fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f7681b94640 1 --2- 172.21.15.112:0/2900611891 >> v2:172.21.15.112:3300/0 conn(0x7f767c0730b0 0x7f767c0c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:09.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.884+0000 7f7682b96640 1 --2- 172.21.15.112:0/2900611891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7668004160 0x7f766800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f70e3812640 1 -- 172.21.15.112:0/3146112441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d4007410 msgr2=0x7f70d400a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f70e3812640 1 --2- 172.21.15.112:0/3146112441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d4007410 0x7f70d400a8e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f70dc04fa20 tx=0x7f70dc0a5ec0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f7681b94640 1 --2- 172.21.15.112:0/2900611891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76680037a0 0x7f7668003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f70e3812640 1 -- 172.21.15.112:0/3146112441 shutdown_connections 2024-04-25T01:26:09.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f70e3812640 1 --2- 172.21.15.112:0/3146112441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d4007410 0x7f70d400a8e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f70e3812640 1 --2- 172.21.15.112:0/3146112441 >> v2:172.21.15.112:3300/0 conn(0x7f70d40474d0 0x7f70d4006ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f70e3812640 1 -- 172.21.15.112:0/3146112441 >> 172.21.15.112:0/3146112441 conn(0x7f70d4048280 msgr2=0x7f70d4048690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:09.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f6bb24b2640 1 -- 172.21.15.112:0/969851964 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bac0ac050 msgr2=0x7f6bac070b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f6bb24b2640 1 --2- 172.21.15.112:0/969851964 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bac0ac050 0x7f6bac070b60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80097a0 tx=0x7f6ba802a0b0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f7682b96640 1 -- 172.21.15.112:0/2900611891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76680037a0 msgr2=0x7f7668003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:09.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f7682b96640 1 --2- 172.21.15.112:0/2900611891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76680037a0 0x7f7668003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f7682b96640 1 -- 172.21.15.112:0/2900611891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7668004bd0 msgr2=0x7f7668004fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:09.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f7682b96640 1 --2- 172.21.15.112:0/2900611891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7668004bd0 0x7f7668004fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f7682b96640 1 -- 172.21.15.112:0/2900611891 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767c154eb0 con 0x7f7668004160 2024-04-25T01:26:09.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f6bb24b2640 1 -- 172.21.15.112:0/969851964 shutdown_connections 2024-04-25T01:26:09.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f6bb24b2640 1 --2- 172.21.15.112:0/969851964 >> v2:172.21.15.112:3300/0 conn(0x7f6bac155960 0x7f6bac15c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f6bb24b2640 1 --2- 172.21.15.112:0/969851964 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bac0ac050 0x7f6bac070b60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f6bb24b2640 1 -- 172.21.15.112:0/969851964 >> 172.21.15.112:0/969851964 conn(0x7f6bac0c3930 msgr2=0x7f6bac0c3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:09.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f70e3812640 1 -- 172.21.15.112:0/3146112441 shutdown_connections 2024-04-25T01:26:09.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f6bb24b2640 1 -- 172.21.15.112:0/969851964 shutdown_connections 2024-04-25T01:26:09.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.886+0000 7f3089563640 1 Processor -- start 2024-04-25T01:26:09.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.886+0000 7f7682b96640 1 -- 172.21.15.112:0/2900611891 reap_dead start 2024-04-25T01:26:09.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f70e3812640 1 -- 172.21.15.112:0/3146112441 wait complete. 2024-04-25T01:26:09.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.885+0000 7f6bb24b2640 1 -- 172.21.15.112:0/969851964 wait complete. 2024-04-25T01:26:09.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.886+0000 7f3089563640 1 -- start start 2024-04-25T01:26:09.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.886+0000 7f7682b96640 1 --2- 172.21.15.112:0/2900611891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7668004160 0x7f766800b9c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f767400c4e0 tx=0x7f7668005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea71c686fd7e3aa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.886+0000 7f9075a23640 1 --2- 172.21.15.112:0/3921570154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f907008b9d0 0x7f907006f380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9068009d90 tx=0x7f906802a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8dcb730418d3e46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.887+0000 7f7681393640 1 -- 172.21.15.112:0/2900611891 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767400c0e0 con 0x7f7668004160 2024-04-25T01:26:09.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.887+0000 7f7681393640 1 -- 172.21.15.112:0/2900611891 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7674006d50 con 0x7f7668004160 2024-04-25T01:26:09.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.887+0000 7f3089563640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3084071000 0x7f3084154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.887+0000 7f3089563640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30840c3900 0x7f30841ec6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.887+0000 7f3089563640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3084150d80 0x7f30840c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.887+0000 7f3089563640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f30840c8bb0 con 0x7f30840c3900 2024-04-25T01:26:09.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.887+0000 7f3089563640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f30840c7590 con 0x7f3084071000 2024-04-25T01:26:09.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.887+0000 7f3089563640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f30840c8d30 con 0x7f3084150d80 2024-04-25T01:26:09.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.887+0000 7f6bb24b2640 1 Processor -- start 2024-04-25T01:26:09.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.887+0000 7f30827fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30840c3900 0x7f30841ec6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.887+0000 7f70e3812640 1 Processor -- start 2024-04-25T01:26:09.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.887+0000 7f70e3812640 1 -- start start 2024-04-25T01:26:09.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.887+0000 7f30827fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30840c3900 0x7f30841ec6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43550/0 (socket says 172.21.15.112:43550) 2024-04-25T01:26:09.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.887+0000 7f30827fc640 1 -- 172.21.15.112:0/3519939443 learned_addr learned my addr 172.21.15.112:0/3519939443 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:09.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.887+0000 7f7681393640 1 -- 172.21.15.112:0/2900611891 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674007260 con 0x7f7668004160 2024-04-25T01:26:09.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.887+0000 7f9075222640 1 -- 172.21.15.112:0/3921570154 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068007740 con 0x7f907008b9d0 2024-04-25T01:26:09.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.887+0000 7f9075222640 1 -- 172.21.15.112:0/3921570154 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90680078e0 con 0x7f907008b9d0 2024-04-25T01:26:09.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.887+0000 7f9075222640 1 -- 172.21.15.112:0/3921570154 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068007cb0 con 0x7f907008b9d0 2024-04-25T01:26:09.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f76845f9640 1 -- 172.21.15.112:0/2900611891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7668004160 msgr2=0x7f766800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f30827fc640 1 -- 172.21.15.112:0/3519939443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3084150d80 msgr2=0x7f30840c3dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:26:09.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f76845f9640 1 --2- 172.21.15.112:0/2900611891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7668004160 0x7f766800b9c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f767400c4e0 tx=0x7f7668005830 comp rx=0 tx=0).stop 2024-04-25T01:26:09.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f30827fc640 1 --2- 172.21.15.112:0/3519939443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3084150d80 0x7f30840c3dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f30827fc640 1 -- 172.21.15.112:0/3519939443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3084071000 msgr2=0x7f3084154f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:26:09.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f30827fc640 1 --2- 172.21.15.112:0/3519939443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3084071000 0x7f3084154f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f30827fc640 1 -- 172.21.15.112:0/3519939443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3084155460 con 0x7f30840c3900 2024-04-25T01:26:09.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f76845f9640 1 -- 172.21.15.112:0/2900611891 shutdown_connections 2024-04-25T01:26:09.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f76845f9640 1 --2- 172.21.15.112:0/2900611891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7668004160 0x7f766800b9c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.887+0000 7f6bb24b2640 1 -- start start 2024-04-25T01:26:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f76845f9640 1 -- 172.21.15.112:0/2900611891 >> 172.21.15.112:0/2900611891 conn(0x7f767c0c7080 msgr2=0x7f767c0c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f6bb24b2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6bac0ac050 0x7f6bac0c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f30827fc640 1 --2- 172.21.15.112:0/3519939443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30840c3900 0x7f30841ec6f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f307c00caf0 tx=0x7f307c002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f6bb24b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bac155500 0x7f6bac06d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f6bb24b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bac155960 0x7f6bac1efb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f6bb24b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac154fe0 con 0x7f6bac155500 2024-04-25T01:26:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f6bb24b2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac0c3050 con 0x7f6bac0ac050 2024-04-25T01:26:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f6bb24b2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6bac072a90 con 0x7f6bac155960 2024-04-25T01:26:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f6bb1cb1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bac155960 0x7f6bac1efb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f6bb1cb1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bac155960 0x7f6bac1efb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:56662/0 (socket says 172.21.15.112:56662) 2024-04-25T01:26:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f6bb1cb1640 1 -- 172.21.15.112:0/4212673911 learned_addr learned my addr 172.21.15.112:0/4212673911 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f6bb0caf640 1 --2- 172.21.15.112:0/4212673911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bac155500 0x7f6bac06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f6bb14b0640 1 --2- 172.21.15.112:0/4212673911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6bac0ac050 0x7f6bac0c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f70e3812640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70d4007410 0x7f70d402ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f70e3812640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d40474d0 0x7f70d40de0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f70e3812640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70d40dbd10 0x7f70d40de5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f70e3812640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f70d40dc0f0 con 0x7f70d40dbd10 2024-04-25T01:26:09.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f70e3812640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f70d4047db0 con 0x7f70d4007410 2024-04-25T01:26:09.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f70e3812640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f70d40dfc20 con 0x7f70d40474d0 2024-04-25T01:26:09.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f70e15ae640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70d4007410 0x7f70d402ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f70e0dad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d40474d0 0x7f70d40de0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f70e15ae640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70d4007410 0x7f70d402ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49228/0 (socket says 172.21.15.112:49228) 2024-04-25T01:26:09.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f70e15ae640 1 -- 172.21.15.112:0/2860338533 learned_addr learned my addr 172.21.15.112:0/2860338533 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:09.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f70e1daf640 1 --2- 172.21.15.112:0/2860338533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70d40dbd10 0x7f70d40de5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f76845f9640 1 -- 172.21.15.112:0/2900611891 shutdown_connections 2024-04-25T01:26:09.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f70e15ae640 1 -- 172.21.15.112:0/2860338533 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d40474d0 msgr2=0x7f70d40de0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f6bb1cb1640 1 -- 172.21.15.112:0/4212673911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6bac0ac050 msgr2=0x7f6bac0c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f6bb1cb1640 1 --2- 172.21.15.112:0/4212673911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6bac0ac050 0x7f6bac0c6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f6bb1cb1640 1 -- 172.21.15.112:0/4212673911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bac155500 msgr2=0x7f6bac06d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f6bb1cb1640 1 --2- 172.21.15.112:0/4212673911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bac155500 0x7f6bac06d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f9078488640 1 -- 172.21.15.112:0/3921570154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f907008b9d0 msgr2=0x7f907006f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f306b7fe640 1 -- 172.21.15.112:0/3519939443 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c017070 con 0x7f30840c3900 2024-04-25T01:26:09.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f9078488640 1 --2- 172.21.15.112:0/3921570154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f907008b9d0 0x7f907006f380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9068009d90 tx=0x7f906802a730 comp rx=0 tx=0).stop 2024-04-25T01:26:09.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f6bb1cb1640 1 -- 172.21.15.112:0/4212673911 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bac1f3a00 con 0x7f6bac155960 2024-04-25T01:26:09.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f6bb14b0640 1 --2- 172.21.15.112:0/4212673911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6bac0ac050 0x7f6bac0c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:09.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f6bb0caf640 1 --2- 172.21.15.112:0/4212673911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bac155500 0x7f6bac06d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:09.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f9078488640 1 -- 172.21.15.112:0/3921570154 shutdown_connections 2024-04-25T01:26:09.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f9078488640 1 --2- 172.21.15.112:0/3921570154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f907008b9d0 0x7f907006f380 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f9078488640 1 -- 172.21.15.112:0/3921570154 >> 172.21.15.112:0/3921570154 conn(0x7f90700c7080 msgr2=0x7f90700c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:09.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f6bb1cb1640 1 --2- 172.21.15.112:0/4212673911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bac155960 0x7f6bac1efb60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00e870 tx=0x7f6b9c00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f70e15ae640 1 --2- 172.21.15.112:0/2860338533 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d40474d0 0x7f70d40de0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f70e15ae640 1 -- 172.21.15.112:0/2860338533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70d40dbd10 msgr2=0x7f70d40de5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f70e15ae640 1 --2- 172.21.15.112:0/2860338533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70d40dbd10 0x7f70d40de5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f70e15ae640 1 -- 172.21.15.112:0/2860338533 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d40dae80 con 0x7f70d4007410 2024-04-25T01:26:09.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f70e1daf640 1 --2- 172.21.15.112:0/2860338533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70d40dbd10 0x7f70d40de5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:09.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f70e0dad640 1 --2- 172.21.15.112:0/2860338533 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d40474d0 0x7f70d40de0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:26:09.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f3089563640 1 -- 172.21.15.112:0/3519939443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3084146270 con 0x7f30840c3900 2024-04-25T01:26:09.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.889+0000 7f3089563640 1 -- 172.21.15.112:0/3519939443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f308406f200 con 0x7f30840c3900 2024-04-25T01:26:09.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.890+0000 7f70e15ae640 1 --2- 172.21.15.112:0/2860338533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70d4007410 0x7f70d402ea10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f70d800c430 tx=0x7f70d800c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.890+0000 7f6ba1ffb640 1 -- 172.21.15.112:0/4212673911 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c017070 con 0x7f6bac155960 2024-04-25T01:26:09.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.890+0000 7f70d1ffb640 1 -- 172.21.15.112:0/2860338533 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d8011790 con 0x7f70d4007410 2024-04-25T01:26:09.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.890+0000 7f6ba1ffb640 1 -- 172.21.15.112:0/4212673911 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b9c009e00 con 0x7f6bac155960 2024-04-25T01:26:09.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.891+0000 7f6ba1ffb640 1 -- 172.21.15.112:0/4212673911 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c017070 con 0x7f6bac155960 2024-04-25T01:26:09.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.890+0000 7f70d1ffb640 1 -- 172.21.15.112:0/2860338533 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70d8017070 con 0x7f70d4007410 2024-04-25T01:26:09.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.891+0000 7f70d1ffb640 1 -- 172.21.15.112:0/2860338533 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d8011790 con 0x7f70d4007410 2024-04-25T01:26:09.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.888+0000 7f76845f9640 1 -- 172.21.15.112:0/2900611891 wait complete. 2024-04-25T01:26:09.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.890+0000 7f76845f9640 1 Processor -- start 2024-04-25T01:26:09.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.890+0000 7f76845f9640 1 -- start start 2024-04-25T01:26:09.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.891+0000 7f9078488640 1 -- 172.21.15.112:0/3921570154 shutdown_connections 2024-04-25T01:26:09.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.891+0000 7f76845f9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7668004160 0x7f767c0c0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.891+0000 7f76845f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f767c0c1080 0x7f767c14cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.891+0000 7f76845f9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f767c14d070 0x7f767c1f5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.891+0000 7f70e3812640 1 -- 172.21.15.112:0/2860338533 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d40d93d0 con 0x7f70d4007410 2024-04-25T01:26:09.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.891+0000 7f76845f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f767c155470 con 0x7f767c0c1080 2024-04-25T01:26:09.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.891+0000 7f76845f9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f767c1557e0 con 0x7f767c14d070 2024-04-25T01:26:09.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.891+0000 7f70e3812640 1 -- 172.21.15.112:0/2860338533 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d40db400 con 0x7f70d4007410 2024-04-25T01:26:09.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.891+0000 7f76845f9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f767c0c2430 con 0x7f7668004160 2024-04-25T01:26:09.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.891+0000 7f7682b96640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f767c14d070 0x7f767c1f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.891+0000 7f7682b96640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f767c14d070 0x7f767c1f5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49244/0 (socket says 172.21.15.112:49244) 2024-04-25T01:26:09.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.891+0000 7f7682b96640 1 -- 172.21.15.112:0/2518058476 learned_addr learned my addr 172.21.15.112:0/2518058476 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:09.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.892+0000 7f7682b96640 1 -- 172.21.15.112:0/2518058476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7668004160 msgr2=0x7f767c0c0b40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:26:09.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.892+0000 7f7682b96640 1 --2- 172.21.15.112:0/2518058476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7668004160 0x7f767c0c0b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.892+0000 7f7682b96640 1 -- 172.21.15.112:0/2518058476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f767c0c1080 msgr2=0x7f767c14cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.891+0000 7f9078488640 1 -- 172.21.15.112:0/3921570154 wait complete. 2024-04-25T01:26:09.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.892+0000 7f7682b96640 1 --2- 172.21.15.112:0/2518058476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f767c0c1080 0x7f767c14cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.892+0000 7f7682b96640 1 -- 172.21.15.112:0/2518058476 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767c151300 con 0x7f767c14d070 2024-04-25T01:26:09.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.892+0000 7f7682b96640 1 --2- 172.21.15.112:0/2518058476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f767c14d070 0x7f767c1f5020 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f767400c4e0 tx=0x7f7674001cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.892+0000 7f3089563640 1 -- 172.21.15.112:0/3519939443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30840689b0 con 0x7f30840c3900 2024-04-25T01:26:09.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.892+0000 7f6bb24b2640 1 -- 172.21.15.112:0/4212673911 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bac0c59a0 con 0x7f6bac155960 2024-04-25T01:26:09.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.892+0000 7f6bb24b2640 1 -- 172.21.15.112:0/4212673911 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bac06d640 con 0x7f6bac155960 2024-04-25T01:26:09.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.893+0000 7f7672ffd640 1 -- 172.21.15.112:0/2518058476 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674006b50 con 0x7f767c14d070 2024-04-25T01:26:09.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.893+0000 7f76845f9640 1 -- 172.21.15.112:0/2518058476 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767c1f7bf0 con 0x7f767c14d070 2024-04-25T01:26:09.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.893+0000 7f76845f9640 1 -- 172.21.15.112:0/2518058476 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f767c1f8190 con 0x7f767c14d070 2024-04-25T01:26:09.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.893+0000 7f7672ffd640 1 -- 172.21.15.112:0/2518058476 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7674005510 con 0x7f767c14d070 2024-04-25T01:26:09.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.893+0000 7f9078488640 1 Processor -- start 2024-04-25T01:26:09.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.893+0000 7f9078488640 1 -- start start 2024-04-25T01:26:09.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.894+0000 7f9078488640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9070072d50 0x7f90701fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.894+0000 7f9078488640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f907008b9d0 0x7f90701660c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.894+0000 7f9078488640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f90700abfe0 0x7f907015b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.894+0000 7f9078488640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9070071470 con 0x7f907008b9d0 2024-04-25T01:26:09.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.894+0000 7f9078488640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f90700aa7f0 con 0x7f90700abfe0 2024-04-25T01:26:09.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.894+0000 7f9078488640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f90700715f0 con 0x7f9070072d50 2024-04-25T01:26:09.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.894+0000 7f9076a25640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f90700abfe0 0x7f907015b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.894+0000 7f9076a25640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f90700abfe0 0x7f907015b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49260/0 (socket says 172.21.15.112:49260) 2024-04-25T01:26:09.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.894+0000 7f9076a25640 1 -- 172.21.15.112:0/1505761381 learned_addr learned my addr 172.21.15.112:0/1505761381 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:09.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.894+0000 7f9075a23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f907008b9d0 0x7f90701660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.894+0000 7f7672ffd640 1 -- 172.21.15.112:0/2518058476 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767402c420 con 0x7f767c14d070 2024-04-25T01:26:09.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.895+0000 7f7672ffd640 1 -- 172.21.15.112:0/2518058476 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f767403f030 con 0x7f767c14d070 2024-04-25T01:26:09.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.893+0000 7f70d1ffb640 1 -- 172.21.15.112:0/2860338533 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f70d8011930 con 0x7f70d4007410 2024-04-25T01:26:09.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.894+0000 7f70d1ffb640 1 --2- 172.21.15.112:0/2860338533 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f70c006c060 0x7f70c006e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.894+0000 7f70d1ffb640 1 -- 172.21.15.112:0/2860338533 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f70d808a970 con 0x7f70d4007410 2024-04-25T01:26:09.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.895+0000 7f70e0dad640 1 --2- 172.21.15.112:0/2860338533 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f70c006c060 0x7f70c006e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.894+0000 7f6bb24b2640 1 -- 172.21.15.112:0/4212673911 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bac0c5ec0 con 0x7f6bac155960 2024-04-25T01:26:09.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.895+0000 7f70e3812640 1 -- 172.21.15.112:0/2860338533 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d40ddbc0 con 0x7f70d4007410 2024-04-25T01:26:09.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.895+0000 7f76845f9640 1 -- 172.21.15.112:0/2518058476 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7664005290 con 0x7f767c14d070 2024-04-25T01:26:09.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.895+0000 7f306b7fe640 1 -- 172.21.15.112:0/3519939443 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f307c003b70 con 0x7f30840c3900 2024-04-25T01:26:09.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.896+0000 7f70e0dad640 1 --2- 172.21.15.112:0/2860338533 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f70c006c060 0x7f70c006e520 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f70d40e92e0 tx=0x7f70cc009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.896+0000 7f7672ffd640 1 --2- 172.21.15.112:0/2518058476 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f766006bf00 0x7f766006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.896+0000 7f7682395640 1 --2- 172.21.15.112:0/2518058476 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f766006bf00 0x7f766006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.896+0000 7f9076a25640 1 -- 172.21.15.112:0/1505761381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9070072d50 msgr2=0x7f90701fb720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:26:09.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.896+0000 7f9076a25640 1 --2- 172.21.15.112:0/1505761381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9070072d50 0x7f90701fb720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.896+0000 7f9076a25640 1 -- 172.21.15.112:0/1505761381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f907008b9d0 msgr2=0x7f90701660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:09.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.896+0000 7f9076a25640 1 --2- 172.21.15.112:0/1505761381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f907008b9d0 0x7f90701660c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:09.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.896+0000 7f9076a25640 1 -- 172.21.15.112:0/1505761381 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9070162270 con 0x7f90700abfe0 2024-04-25T01:26:09.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.896+0000 7f9075a23640 1 --2- 172.21.15.112:0/1505761381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f907008b9d0 0x7f90701660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:09.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.896+0000 7f9076a25640 1 --2- 172.21.15.112:0/1505761381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f90700abfe0 0x7f907015b2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9060015be0 tx=0x7f9060013560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.896+0000 7f306b7fe640 1 -- 172.21.15.112:0/3519939443 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c00a7a0 con 0x7f30840c3900 2024-04-25T01:26:09.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.896+0000 7f306b7fe640 1 -- 172.21.15.112:0/3519939443 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f307c01a070 con 0x7f30840c3900 2024-04-25T01:26:09.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.897+0000 7f6ba1ffb640 1 -- 172.21.15.112:0/4212673911 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b9c01a070 con 0x7f6bac155960 2024-04-25T01:26:09.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.897+0000 7f7672ffd640 1 -- 172.21.15.112:0/2518058476 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f76740aae40 con 0x7f767c14d070 2024-04-25T01:26:09.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.897+0000 7f306b7fe640 1 --2- 172.21.15.112:0/3519939443 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f307006bf30 0x7f307006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.897+0000 7f6ba1ffb640 1 --2- 172.21.15.112:0/4212673911 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6b9006c030 0x7f6b9006e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.897+0000 7f306b7fe640 1 -- 172.21.15.112:0/3519939443 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f307c08a930 con 0x7f30840c3900 2024-04-25T01:26:09.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.897+0000 7f6ba1ffb640 1 -- 172.21.15.112:0/4212673911 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f6b9c08b360 con 0x7f6bac155960 2024-04-25T01:26:09.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.897+0000 7f6bb14b0640 1 --2- 172.21.15.112:0/4212673911 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6b9006c030 0x7f6b9006e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.897+0000 7f7682395640 1 --2- 172.21.15.112:0/2518058476 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f766006bf00 0x7f766006e3c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7678001a70 tx=0x7f76780019e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.897+0000 7f3082ffd640 1 --2- 172.21.15.112:0/3519939443 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f307006bf30 0x7f307006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.898+0000 7f6bb14b0640 1 --2- 172.21.15.112:0/4212673911 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6b9006c030 0x7f6b9006e4f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8034000 tx=0x7f6ba802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.898+0000 7f3082ffd640 1 --2- 172.21.15.112:0/3519939443 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f307006bf30 0x7f307006e3f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3074034000 tx=0x7f307402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.900+0000 7f6ba1ffb640 1 -- 172.21.15.112:0/4212673911 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6b9c059a70 con 0x7f6bac155960 2024-04-25T01:26:09.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.900+0000 7f306b7fe640 1 -- 172.21.15.112:0/3519939443 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3084155460 con 0x7f30840c3900 2024-04-25T01:26:09.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.900+0000 7f70d1ffb640 1 -- 172.21.15.112:0/2860338533 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f70d8021020 con 0x7f70d4007410 2024-04-25T01:26:09.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.905+0000 7f9066ffd640 1 -- 172.21.15.112:0/1505761381 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906001d020 con 0x7f90700abfe0 2024-04-25T01:26:09.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.905+0000 7f9078488640 1 -- 172.21.15.112:0/1505761381 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90701ebc80 con 0x7f90700abfe0 2024-04-25T01:26:09.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.905+0000 7f9066ffd640 1 -- 172.21.15.112:0/1505761381 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9060019430 con 0x7f90700abfe0 2024-04-25T01:26:09.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.905+0000 7f9078488640 1 -- 172.21.15.112:0/1505761381 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90701ebf40 con 0x7f90700abfe0 2024-04-25T01:26:09.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.905+0000 7f9066ffd640 1 -- 172.21.15.112:0/1505761381 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060012030 con 0x7f90700abfe0 2024-04-25T01:26:09.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.906+0000 7f7672ffd640 1 -- 172.21.15.112:0/2518058476 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7674032030 con 0x7f767c14d070 2024-04-25T01:26:09.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.906+0000 7f9064ff9640 1 -- 172.21.15.112:0/1505761381 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9070073650 con 0x7f90700abfe0 2024-04-25T01:26:09.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.908+0000 7f9066ffd640 1 -- 172.21.15.112:0/1505761381 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9060020020 con 0x7f90700abfe0 2024-04-25T01:26:09.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.909+0000 7f9066ffd640 1 --2- 172.21.15.112:0/1505761381 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f905406bf00 0x7f905406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:09.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.909+0000 7f9076224640 1 --2- 172.21.15.112:0/1505761381 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f905406bf00 0x7f905406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:09.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.910+0000 7f9076224640 1 --2- 172.21.15.112:0/1505761381 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f905406bf00 0x7f905406e3c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f906c0015d0 tx=0x7f906c007890 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:09.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.910+0000 7f9066ffd640 1 -- 172.21.15.112:0/1505761381 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f9060094110 con 0x7f90700abfe0 2024-04-25T01:26:09.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:09.911+0000 7f9066ffd640 1 -- 172.21.15.112:0/1505761381 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9070073650 con 0x7f90700abfe0 2024-04-25T01:26:10.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.243+0000 7fbed1ffb640 1 -- 172.21.15.112:0/236070368 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fbee004afe0 con 0x7fbee004b8e0 2024-04-25T01:26:10.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.244+0000 7fbed3fff640 1 -- 172.21.15.112:0/236070368 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbed4058940 con 0x7fbee004b8e0 2024-04-25T01:26:10.223 INFO:teuthology.orchestra.run.smithi112.stdout:55834574856 2024-04-25T01:26:10.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.247+0000 7fbeece72640 1 -- 172.21.15.112:0/236070368 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbed806c320 msgr2=0x7fbed806e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.247+0000 7fbeece72640 1 --2- 172.21.15.112:0/236070368 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbed806c320 0x7fbed806e7e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbee8069430 tx=0x7fbee8076b30 comp rx=0 tx=0).stop 2024-04-25T01:26:10.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.247+0000 7fbeece72640 1 -- 172.21.15.112:0/236070368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbee004b8e0 msgr2=0x7fbee00c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.248+0000 7fbeece72640 1 --2- 172.21.15.112:0/236070368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbee004b8e0 0x7fbee00c9da0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbed400c410 tx=0x7fbed400c8d0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.248+0000 7f7521c9e640 1 -- 172.21.15.112:0/3424102426 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f74fc005850 con 0x7f750c004f40 2024-04-25T01:26:10.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.248+0000 7fbeece72640 1 -- 172.21.15.112:0/236070368 shutdown_connections 2024-04-25T01:26:10.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.248+0000 7fbeece72640 1 --2- 172.21.15.112:0/236070368 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbed806c320 0x7fbed806e7e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.248+0000 7fbeece72640 1 --2- 172.21.15.112:0/236070368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbee00d58e0 0x7fbee00c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.248+0000 7fbeece72640 1 --2- 172.21.15.112:0/236070368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee004bdf0 0x7fbee00c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.248+0000 7fbeece72640 1 --2- 172.21.15.112:0/236070368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbee004b8e0 0x7fbee00c9da0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.248+0000 7fbeece72640 1 -- 172.21.15.112:0/236070368 >> 172.21.15.112:0/236070368 conn(0x7fbee002f5d0 msgr2=0x7fbee00d4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:10.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.248+0000 7fbeece72640 1 -- 172.21.15.112:0/236070368 shutdown_connections 2024-04-25T01:26:10.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.248+0000 7fbeece72640 1 -- 172.21.15.112:0/236070368 wait complete. 2024-04-25T01:26:10.229 INFO:teuthology.orchestra.run.smithi112.stdout:55834574857 2024-04-25T01:26:10.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.249+0000 7f7507fff640 1 -- 172.21.15.112:0/3424102426 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f75100588f0 con 0x7f750c004f40 2024-04-25T01:26:10.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.252+0000 7f7505ffb640 1 -- 172.21.15.112:0/3424102426 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f750006beb0 msgr2=0x7f750006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.252+0000 7f7505ffb640 1 --2- 172.21.15.112:0/3424102426 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f750006beb0 0x7f750006e370 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f75140c9e20 tx=0x7f7508008040 comp rx=0 tx=0).stop 2024-04-25T01:26:10.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.252+0000 7f7505ffb640 1 -- 172.21.15.112:0/3424102426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f750c004f40 msgr2=0x7f751416c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.252+0000 7f7505ffb640 1 --2- 172.21.15.112:0/3424102426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f750c004f40 0x7f751416c0b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f751000e810 tx=0x7f751000ecd0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.252+0000 7f7505ffb640 1 -- 172.21.15.112:0/3424102426 shutdown_connections 2024-04-25T01:26:10.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.253+0000 7f7505ffb640 1 --2- 172.21.15.112:0/3424102426 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f750006beb0 0x7f750006e370 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.253+0000 7f7505ffb640 1 --2- 172.21.15.112:0/3424102426 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f751402c7a0 0x7f75140c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.253+0000 7f7505ffb640 1 --2- 172.21.15.112:0/3424102426 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7514049380 0x7f751402c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.253+0000 7f7505ffb640 1 --2- 172.21.15.112:0/3424102426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f750c004f40 0x7f751416c0b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.253+0000 7f7505ffb640 1 -- 172.21.15.112:0/3424102426 >> 172.21.15.112:0/3424102426 conn(0x7f7514048250 msgr2=0x7f7514048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:10.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.253+0000 7f7505ffb640 1 -- 172.21.15.112:0/3424102426 shutdown_connections 2024-04-25T01:26:10.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.253+0000 7f7505ffb640 1 -- 172.21.15.112:0/3424102426 wait complete. 2024-04-25T01:26:10.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.256+0000 7fe9d11ce640 1 -- 172.21.15.112:0/2886806375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe9cc0689b0 con 0x7fe9cc153290 2024-04-25T01:26:10.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.257+0000 7fe9b37fe640 1 -- 172.21.15.112:0/2886806375 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe9c407a0c0 con 0x7fe9cc153290 2024-04-25T01:26:10.236 INFO:teuthology.orchestra.run.smithi112.stdout:55834574857 2024-04-25T01:26:10.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.260+0000 7fe9d11ce640 1 -- 172.21.15.112:0/2886806375 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe9b806bf00 msgr2=0x7fe9b806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.260+0000 7fe9d11ce640 1 --2- 172.21.15.112:0/2886806375 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe9b806bf00 0x7fe9b806e3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0014f0 tx=0x7fe9bc007990 comp rx=0 tx=0).stop 2024-04-25T01:26:10.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.261+0000 7fe9d11ce640 1 -- 172.21.15.112:0/2886806375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9cc153290 msgr2=0x7fe9cc0adc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.261+0000 7fe9d11ce640 1 --2- 172.21.15.112:0/2886806375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9cc153290 0x7fe9cc0adc00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4009f70 tx=0x7fe9c4004a40 comp rx=0 tx=0).stop 2024-04-25T01:26:10.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.261+0000 7fe9d11ce640 1 -- 172.21.15.112:0/2886806375 shutdown_connections 2024-04-25T01:26:10.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.261+0000 7fe9d11ce640 1 --2- 172.21.15.112:0/2886806375 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe9b806bf00 0x7fe9b806e3c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.261+0000 7fe9d11ce640 1 --2- 172.21.15.112:0/2886806375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9cc155c70 0x7fe9cc14e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.261+0000 7fe9d11ce640 1 --2- 172.21.15.112:0/2886806375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9cc153290 0x7fe9cc0adc00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.261+0000 7fe9d11ce640 1 --2- 172.21.15.112:0/2886806375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9cc0c7f20 0x7fe9cc1e1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.261+0000 7fe9d11ce640 1 -- 172.21.15.112:0/2886806375 >> 172.21.15.112:0/2886806375 conn(0x7fe9cc074a00 msgr2=0x7fe9cc1523b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:10.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.262+0000 7fe9d11ce640 1 -- 172.21.15.112:0/2886806375 shutdown_connections 2024-04-25T01:26:10.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.262+0000 7fe9d11ce640 1 -- 172.21.15.112:0/2886806375 wait complete. 2024-04-25T01:26:10.245 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2024-04-25T01:26:10.245 DEBUG:teuthology.parallel:result is None 2024-04-25T01:26:10.251 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2024-04-25T01:26:10.251 DEBUG:teuthology.parallel:result is None 2024-04-25T01:26:10.254 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.3 2024-04-25T01:26:10.254 DEBUG:teuthology.parallel:result is None 2024-04-25T01:26:10.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.289+0000 7f3089563640 1 -- 172.21.15.112:0/3519939443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f30840c8280 con 0x7f30840c3900 2024-04-25T01:26:10.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.289+0000 7f306b7fe640 1 -- 172.21.15.112:0/3519939443 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f307c059020 con 0x7f30840c3900 2024-04-25T01:26:10.269 INFO:teuthology.orchestra.run.smithi112.stdout:55834574857 2024-04-25T01:26:10.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.291+0000 7f3089563640 1 -- 172.21.15.112:0/3519939443 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f307006bf30 msgr2=0x7f307006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.291+0000 7f3089563640 1 --2- 172.21.15.112:0/3519939443 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f307006bf30 0x7f307006e3f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3074034000 tx=0x7f307402e000 comp rx=0 tx=0).stop 2024-04-25T01:26:10.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.291+0000 7f3089563640 1 -- 172.21.15.112:0/3519939443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30840c3900 msgr2=0x7f30841ec6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.291+0000 7f3089563640 1 --2- 172.21.15.112:0/3519939443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30840c3900 0x7f30841ec6f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f307c00caf0 tx=0x7f307c002170 comp rx=0 tx=0).stop 2024-04-25T01:26:10.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.291+0000 7f3089563640 1 -- 172.21.15.112:0/3519939443 shutdown_connections 2024-04-25T01:26:10.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.291+0000 7f3089563640 1 --2- 172.21.15.112:0/3519939443 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f307006bf30 0x7f307006e3f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.291+0000 7f3089563640 1 --2- 172.21.15.112:0/3519939443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3084150d80 0x7f30840c3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.291+0000 7f3089563640 1 --2- 172.21.15.112:0/3519939443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30840c3900 0x7f30841ec6f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.291+0000 7f3089563640 1 --2- 172.21.15.112:0/3519939443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3084071000 0x7f3084154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.291+0000 7f3089563640 1 -- 172.21.15.112:0/3519939443 >> 172.21.15.112:0/3519939443 conn(0x7f3084074a90 msgr2=0x7f30840addd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:10.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.291+0000 7f3089563640 1 -- 172.21.15.112:0/3519939443 shutdown_connections 2024-04-25T01:26:10.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.291+0000 7f3089563640 1 -- 172.21.15.112:0/3519939443 wait complete. 2024-04-25T01:26:10.281 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2024-04-25T01:26:10.281 DEBUG:teuthology.parallel:result is None 2024-04-25T01:26:10.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.320+0000 7f70e3812640 1 -- 172.21.15.112:0/2860338533 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f70d40df6d0 con 0x7f70d4007410 2024-04-25T01:26:10.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.320+0000 7f70d1ffb640 1 -- 172.21.15.112:0/2860338533 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f70d8058f60 con 0x7f70d4007410 2024-04-25T01:26:10.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.321+0000 7f6bb24b2640 1 -- 172.21.15.112:0/4212673911 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f6bac04f500 con 0x7f6bac155960 2024-04-25T01:26:10.300 INFO:teuthology.orchestra.run.smithi112.stdout:55834574857 2024-04-25T01:26:10.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.321+0000 7f6ba1ffb640 1 -- 172.21.15.112:0/4212673911 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6b9c059a70 con 0x7f6bac155960 2024-04-25T01:26:10.301 INFO:teuthology.orchestra.run.smithi112.stdout:55834574856 2024-04-25T01:26:10.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f70e3812640 1 -- 172.21.15.112:0/2860338533 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f70c006c060 msgr2=0x7f70c006e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f70e3812640 1 --2- 172.21.15.112:0/2860338533 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f70c006c060 0x7f70c006e520 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f70d40e92e0 tx=0x7f70cc009110 comp rx=0 tx=0).stop 2024-04-25T01:26:10.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f70e3812640 1 -- 172.21.15.112:0/2860338533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70d4007410 msgr2=0x7f70d402ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f70e3812640 1 --2- 172.21.15.112:0/2860338533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70d4007410 0x7f70d402ea10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f70d800c430 tx=0x7f70d800c830 comp rx=0 tx=0).stop 2024-04-25T01:26:10.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f70e3812640 1 -- 172.21.15.112:0/2860338533 shutdown_connections 2024-04-25T01:26:10.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f70e3812640 1 --2- 172.21.15.112:0/2860338533 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f70c006c060 0x7f70c006e520 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f70e3812640 1 --2- 172.21.15.112:0/2860338533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70d40dbd10 0x7f70d40de5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f70e3812640 1 --2- 172.21.15.112:0/2860338533 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d40474d0 0x7f70d40de0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f70e3812640 1 --2- 172.21.15.112:0/2860338533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70d4007410 0x7f70d402ea10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f70e3812640 1 -- 172.21.15.112:0/2860338533 >> 172.21.15.112:0/2860338533 conn(0x7f70d4048280 msgr2=0x7f70d404a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:10.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f70e3812640 1 -- 172.21.15.112:0/2860338533 shutdown_connections 2024-04-25T01:26:10.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f70e3812640 1 -- 172.21.15.112:0/2860338533 wait complete. 2024-04-25T01:26:10.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f6bb24b2640 1 -- 172.21.15.112:0/4212673911 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6b9006c030 msgr2=0x7f6b9006e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f6bb24b2640 1 --2- 172.21.15.112:0/4212673911 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6b9006c030 0x7f6b9006e4f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8034000 tx=0x7f6ba802e000 comp rx=0 tx=0).stop 2024-04-25T01:26:10.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f6bb24b2640 1 -- 172.21.15.112:0/4212673911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bac155960 msgr2=0x7f6bac1efb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f6bb24b2640 1 --2- 172.21.15.112:0/4212673911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bac155960 0x7f6bac1efb60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00e870 tx=0x7f6b9c00ed30 comp rx=0 tx=0).stop 2024-04-25T01:26:10.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f6bb24b2640 1 -- 172.21.15.112:0/4212673911 shutdown_connections 2024-04-25T01:26:10.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f6bb24b2640 1 --2- 172.21.15.112:0/4212673911 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6b9006c030 0x7f6b9006e4f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f6bb24b2640 1 --2- 172.21.15.112:0/4212673911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bac155960 0x7f6bac1efb60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f6bb24b2640 1 --2- 172.21.15.112:0/4212673911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bac155500 0x7f6bac06d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f6bb24b2640 1 --2- 172.21.15.112:0/4212673911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6bac0ac050 0x7f6bac0c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.323+0000 7f6bb24b2640 1 -- 172.21.15.112:0/4212673911 >> 172.21.15.112:0/4212673911 conn(0x7f6bac0c3930 msgr2=0x7f6bac0aa290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:10.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.324+0000 7f6bb24b2640 1 -- 172.21.15.112:0/4212673911 shutdown_connections 2024-04-25T01:26:10.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.324+0000 7f6bb24b2640 1 -- 172.21.15.112:0/4212673911 wait complete. 2024-04-25T01:26:10.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.329+0000 7f76845f9640 1 -- 172.21.15.112:0/2518058476 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f7664005580 con 0x7f767c14d070 2024-04-25T01:26:10.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.330+0000 7f7672ffd640 1 -- 172.21.15.112:0/2518058476 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7674079550 con 0x7f767c14d070 2024-04-25T01:26:10.309 INFO:teuthology.orchestra.run.smithi112.stdout:55834574858 2024-04-25T01:26:10.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.331+0000 7f9064ff9640 1 -- 172.21.15.112:0/1505761381 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f90700682a0 con 0x7f90700abfe0 2024-04-25T01:26:10.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.332+0000 7f9066ffd640 1 -- 172.21.15.112:0/1505761381 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9060027020 con 0x7f90700abfe0 2024-04-25T01:26:10.311 INFO:teuthology.orchestra.run.smithi112.stdout:55834574857 2024-04-25T01:26:10.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.333+0000 7f76845f9640 1 -- 172.21.15.112:0/2518058476 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f766006bf00 msgr2=0x7f766006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.333+0000 7f76845f9640 1 --2- 172.21.15.112:0/2518058476 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f766006bf00 0x7f766006e3c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7678001a70 tx=0x7f76780019e0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.333+0000 7f76845f9640 1 -- 172.21.15.112:0/2518058476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f767c14d070 msgr2=0x7f767c1f5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.333+0000 7f76845f9640 1 --2- 172.21.15.112:0/2518058476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f767c14d070 0x7f767c1f5020 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f767400c4e0 tx=0x7f7674001cf0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.333+0000 7f76845f9640 1 -- 172.21.15.112:0/2518058476 shutdown_connections 2024-04-25T01:26:10.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.333+0000 7f76845f9640 1 --2- 172.21.15.112:0/2518058476 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f766006bf00 0x7f766006e3c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.333+0000 7f76845f9640 1 --2- 172.21.15.112:0/2518058476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f767c14d070 0x7f767c1f5020 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.333+0000 7f76845f9640 1 --2- 172.21.15.112:0/2518058476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f767c0c1080 0x7f767c14cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.333+0000 7f76845f9640 1 --2- 172.21.15.112:0/2518058476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7668004160 0x7f767c0c0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.333+0000 7f76845f9640 1 -- 172.21.15.112:0/2518058476 >> 172.21.15.112:0/2518058476 conn(0x7f767c0c7080 msgr2=0x7f767c14ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:10.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.333+0000 7f76845f9640 1 -- 172.21.15.112:0/2518058476 shutdown_connections 2024-04-25T01:26:10.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.333+0000 7f76845f9640 1 -- 172.21.15.112:0/2518058476 wait complete. 2024-04-25T01:26:10.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.334+0000 7f9064ff9640 1 -- 172.21.15.112:0/1505761381 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f905406bf00 msgr2=0x7f905406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.334+0000 7f9064ff9640 1 --2- 172.21.15.112:0/1505761381 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f905406bf00 0x7f905406e3c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f906c0015d0 tx=0x7f906c007890 comp rx=0 tx=0).stop 2024-04-25T01:26:10.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.334+0000 7f9064ff9640 1 -- 172.21.15.112:0/1505761381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f90700abfe0 msgr2=0x7f907015b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.334+0000 7f9064ff9640 1 --2- 172.21.15.112:0/1505761381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f90700abfe0 0x7f907015b2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9060015be0 tx=0x7f9060013560 comp rx=0 tx=0).stop 2024-04-25T01:26:10.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.334+0000 7f9064ff9640 1 -- 172.21.15.112:0/1505761381 shutdown_connections 2024-04-25T01:26:10.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.334+0000 7f9064ff9640 1 --2- 172.21.15.112:0/1505761381 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f905406bf00 0x7f905406e3c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.334+0000 7f9064ff9640 1 --2- 172.21.15.112:0/1505761381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f90700abfe0 0x7f907015b2f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.334+0000 7f9064ff9640 1 --2- 172.21.15.112:0/1505761381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f907008b9d0 0x7f90701660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.334+0000 7f9064ff9640 1 --2- 172.21.15.112:0/1505761381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9070072d50 0x7f90701fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.334+0000 7f9064ff9640 1 -- 172.21.15.112:0/1505761381 >> 172.21.15.112:0/1505761381 conn(0x7f90700c7080 msgr2=0x7f90700c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:10.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.335+0000 7f9064ff9640 1 -- 172.21.15.112:0/1505761381 shutdown_connections 2024-04-25T01:26:10.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.335+0000 7f9064ff9640 1 -- 172.21.15.112:0/1505761381 wait complete. 2024-04-25T01:26:10.316 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.2 2024-04-25T01:26:10.317 DEBUG:teuthology.parallel:result is None 2024-04-25T01:26:10.317 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2024-04-25T01:26:10.317 DEBUG:teuthology.parallel:result is None 2024-04-25T01:26:10.325 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2024-04-25T01:26:10.325 DEBUG:teuthology.parallel:result is None 2024-04-25T01:26:10.325 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2024-04-25T01:26:10.325 DEBUG:teuthology.parallel:result is None 2024-04-25T01:26:10.326 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T01:26:10.326 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:26:10.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.484+0000 7fedef441640 1 Processor -- start 2024-04-25T01:26:10.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.484+0000 7fedef441640 1 -- start start 2024-04-25T01:26:10.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.485+0000 7fedef441640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede80ac3b0 0x7fede8155f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:10.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.485+0000 7fedef441640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fede81524a0 0x7fede8156460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:10.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.485+0000 7fedef441640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fede81529b0 con 0x7fede814c3f0 2024-04-25T01:26:10.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.485+0000 7fedef441640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fede8155a90 con 0x7fede81524a0 2024-04-25T01:26:10.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.485+0000 7fedef441640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fede81569a0 con 0x7fede80ac3b0 2024-04-25T01:26:10.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.485+0000 7feded9de640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fede81524a0 0x7fede8156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:10.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.485+0000 7feded9de640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fede81524a0 0x7fede8156460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43592/0 (socket says 172.21.15.112:43592) 2024-04-25T01:26:10.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.485+0000 7feded9de640 1 -- 172.21.15.112:0/3145185308 learned_addr learned my addr 172.21.15.112:0/3145185308 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:10.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.485+0000 7fedec9dc640 1 --2- 172.21.15.112:0/3145185308 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede80ac3b0 0x7fede8155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:10.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.485+0000 7feded9de640 1 -- 172.21.15.112:0/3145185308 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede80ac3b0 msgr2=0x7fede8155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:10.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.485+0000 7feded9de640 1 --2- 172.21.15.112:0/3145185308 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede80ac3b0 0x7fede8155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.485+0000 7feded9de640 1 -- 172.21.15.112:0/3145185308 >> v1:172.21.15.112:6789/0 conn(0x7fede814c3f0 legacy=0x7fede80c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.485+0000 7feded9de640 1 -- 172.21.15.112:0/3145185308 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede8152c70 con 0x7fede81524a0 2024-04-25T01:26:10.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.486+0000 7feded9de640 1 --2- 172.21.15.112:0/3145185308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fede81524a0 0x7fede8156460 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fede0009e50 tx=0x7fede002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4839542836a41c3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:10.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.486+0000 7feddffff640 1 -- 172.21.15.112:0/3145185308 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0036070 con 0x7fede81524a0 2024-04-25T01:26:10.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.486+0000 7feddffff640 1 -- 172.21.15.112:0/3145185308 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fede0031600 con 0x7fede81524a0 2024-04-25T01:26:10.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.486+0000 7feddffff640 1 -- 172.21.15.112:0/3145185308 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede00319d0 con 0x7fede81524a0 2024-04-25T01:26:10.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.486+0000 7fedef441640 1 -- 172.21.15.112:0/3145185308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fede81524a0 msgr2=0x7fede8156460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.486+0000 7fedef441640 1 --2- 172.21.15.112:0/3145185308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fede81524a0 0x7fede8156460 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fede0009e50 tx=0x7fede002a680 comp rx=0 tx=0).stop 2024-04-25T01:26:10.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.486+0000 7fedef441640 1 -- 172.21.15.112:0/3145185308 shutdown_connections 2024-04-25T01:26:10.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.486+0000 7fedef441640 1 --2- 172.21.15.112:0/3145185308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fede81524a0 0x7fede8156460 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.486+0000 7fedef441640 1 --2- 172.21.15.112:0/3145185308 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede80ac3b0 0x7fede8155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.486+0000 7fedef441640 1 -- 172.21.15.112:0/3145185308 >> 172.21.15.112:0/3145185308 conn(0x7fede806b160 msgr2=0x7fede806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:10.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.487+0000 7fedef441640 1 -- 172.21.15.112:0/3145185308 shutdown_connections 2024-04-25T01:26:10.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.487+0000 7fedef441640 1 -- 172.21.15.112:0/3145185308 wait complete. 2024-04-25T01:26:10.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.488+0000 7fedef441640 1 Processor -- start 2024-04-25T01:26:10.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.488+0000 7fedef441640 1 -- start start 2024-04-25T01:26:10.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.488+0000 7fedef441640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fede80ac3b0 0x7fede806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:10.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.488+0000 7fedef441640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fede814c3f0 0x7fede8145f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:10.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.488+0000 7fedef441640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede81524a0 0x7fede8144120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:10.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.488+0000 7fedef441640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fede8070d60 con 0x7fede80ac3b0 2024-04-25T01:26:10.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.488+0000 7fedef441640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fede8153230 con 0x7fede814c3f0 2024-04-25T01:26:10.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.488+0000 7fedef441640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fede80c32a0 con 0x7fede81524a0 2024-04-25T01:26:10.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.488+0000 7feded1dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fede80ac3b0 0x7fede806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:10.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.488+0000 7feded1dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fede80ac3b0 0x7fede806e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43608/0 (socket says 172.21.15.112:43608) 2024-04-25T01:26:10.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.488+0000 7feded1dd640 1 -- 172.21.15.112:0/1032494861 learned_addr learned my addr 172.21.15.112:0/1032494861 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:10.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.488+0000 7feded9de640 1 --2- 172.21.15.112:0/1032494861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede81524a0 0x7fede8144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:10.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.489+0000 7fedec9dc640 1 --2- 172.21.15.112:0/1032494861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fede814c3f0 0x7fede8145f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:10.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.489+0000 7feded1dd640 1 -- 172.21.15.112:0/1032494861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede81524a0 msgr2=0x7fede8144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.489+0000 7feded1dd640 1 --2- 172.21.15.112:0/1032494861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede81524a0 0x7fede8144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.489+0000 7feded1dd640 1 -- 172.21.15.112:0/1032494861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fede814c3f0 msgr2=0x7fede8145f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.489+0000 7feded1dd640 1 --2- 172.21.15.112:0/1032494861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fede814c3f0 0x7fede8145f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.489+0000 7feded1dd640 1 -- 172.21.15.112:0/1032494861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede806e600 con 0x7fede80ac3b0 2024-04-25T01:26:10.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.489+0000 7feded1dd640 1 --2- 172.21.15.112:0/1032494861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fede80ac3b0 0x7fede806e0c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fede400c400 tx=0x7fede400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:10.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.489+0000 7fedddffb640 1 -- 172.21.15.112:0/1032494861 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede40052d0 con 0x7fede80ac3b0 2024-04-25T01:26:10.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.489+0000 7fedddffb640 1 -- 172.21.15.112:0/1032494861 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fede4005490 con 0x7fede80ac3b0 2024-04-25T01:26:10.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.489+0000 7fedef441640 1 -- 172.21.15.112:0/1032494861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede81ec050 con 0x7fede80ac3b0 2024-04-25T01:26:10.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.489+0000 7fedddffb640 1 -- 172.21.15.112:0/1032494861 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4007260 con 0x7fede80ac3b0 2024-04-25T01:26:10.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.489+0000 7fedef441640 1 -- 172.21.15.112:0/1032494861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede806d8f0 con 0x7fede80ac3b0 2024-04-25T01:26:10.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.490+0000 7fedef441640 1 -- 172.21.15.112:0/1032494861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede80682a0 con 0x7fede80ac3b0 2024-04-25T01:26:10.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.490+0000 7fedddffb640 1 -- 172.21.15.112:0/1032494861 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fede4007400 con 0x7fede80ac3b0 2024-04-25T01:26:10.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.491+0000 7fedddffb640 1 --2- 172.21.15.112:0/1032494861 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fedcc06be30 0x7fedcc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:10.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.491+0000 7fedddffb640 1 -- 172.21.15.112:0/1032494861 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fede408a5e0 con 0x7fede80ac3b0 2024-04-25T01:26:10.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.491+0000 7fedec9dc640 1 --2- 172.21.15.112:0/1032494861 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fedcc06be30 0x7fedcc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:10.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.491+0000 7fedec9dc640 1 --2- 172.21.15.112:0/1032494861 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fedcc06be30 0x7fedcc06e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fede815c920 tx=0x7fedd8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:10.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.495+0000 7fedddffb640 1 -- 172.21.15.112:0/1032494861 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fede40586a0 con 0x7fede80ac3b0 2024-04-25T01:26:10.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.666+0000 7fedef441640 1 -- 172.21.15.112:0/1032494861 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fede814f7a0 con 0x7fedcc06be30 2024-04-25T01:26:10.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.677+0000 7fedddffb640 1 -- 172.21.15.112:0/1032494861 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+490305 (secure 0 0 0) 0x7fede814f7a0 con 0x7fedcc06be30 2024-04-25T01:26:10.662 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:10.665 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T01:26:10.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.682+0000 7fedef441640 1 -- 172.21.15.112:0/1032494861 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fedcc06be30 msgr2=0x7fedcc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.682+0000 7fedef441640 1 --2- 172.21.15.112:0/1032494861 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fedcc06be30 0x7fedcc06e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fede815c920 tx=0x7fedd8008040 comp rx=0 tx=0).stop 2024-04-25T01:26:10.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.682+0000 7fedef441640 1 -- 172.21.15.112:0/1032494861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fede80ac3b0 msgr2=0x7fede806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.682+0000 7fedef441640 1 --2- 172.21.15.112:0/1032494861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fede80ac3b0 0x7fede806e0c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fede400c400 tx=0x7fede400c8c0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.682+0000 7fedef441640 1 -- 172.21.15.112:0/1032494861 shutdown_connections 2024-04-25T01:26:10.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.682+0000 7fedef441640 1 --2- 172.21.15.112:0/1032494861 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fedcc06be30 0x7fedcc06e2f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.682+0000 7fedef441640 1 --2- 172.21.15.112:0/1032494861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fede81524a0 0x7fede8144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.682+0000 7fedef441640 1 --2- 172.21.15.112:0/1032494861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fede814c3f0 0x7fede8145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.682+0000 7fedef441640 1 --2- 172.21.15.112:0/1032494861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fede80ac3b0 0x7fede806e0c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.682+0000 7fedef441640 1 -- 172.21.15.112:0/1032494861 >> 172.21.15.112:0/1032494861 conn(0x7fede806b160 msgr2=0x7fede80c2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:10.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.682+0000 7fedef441640 1 -- 172.21.15.112:0/1032494861 shutdown_connections 2024-04-25T01:26:10.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.682+0000 7fedef441640 1 -- 172.21.15.112:0/1032494861 wait complete. 2024-04-25T01:26:10.672 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-04-25T01:26:10.652028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3350528,"data_stored":1984464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.254461"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415591+0000","last_change":"2024-04-25T01:26:02.412272+0000","last_active":"2024-04-25T01:26:03.415591+0000","last_peered":"2024-04-25T01:26:03.415591+0000","last_clean":"2024-04-25T01:26:03.415591+0000","last_became_active":"2024-04-25T01:26:02.411176+0000","last_became_peered":"2024-04-25T01:26:02.411176+0000","last_unstale":"2024-04-25T01:26:03.415591+0000","last_undegraded":"2024-04-25T01:26:03.415591+0000","last_fullsized":"2024-04-25T01:26:03.415591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:49.498275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716276+0000","last_change":"2024-04-25T01:25:52.353978+0000","last_active":"2024-04-25T01:26:03.716276+0000","last_peered":"2024-04-25T01:26:03.716276+0000","last_clean":"2024-04-25T01:26:03.716276+0000","last_became_active":"2024-04-25T01:25:52.353649+0000","last_became_peered":"2024-04-25T01:25:52.353649+0000","last_unstale":"2024-04-25T01:26:03.716276+0000","last_undegraded":"2024-04-25T01:26:03.716276+0000","last_fullsized":"2024-04-25T01:26:03.716276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416695+0000","last_change":"2024-04-25T01:25:53.364257+0000","last_active":"2024-04-25T01:26:03.416695+0000","last_peered":"2024-04-25T01:26:03.416695+0000","last_clean":"2024-04-25T01:26:03.416695+0000","last_became_active":"2024-04-25T01:25:53.363409+0000","last_became_peered":"2024-04-25T01:25:53.363409+0000","last_unstale":"2024-04-25T01:26:03.416695+0000","last_undegraded":"2024-04-25T01:26:03.416695+0000","last_fullsized":"2024-04-25T01:26:03.416695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415484+0000","last_change":"2024-04-25T01:26:01.407136+0000","last_active":"2024-04-25T01:26:03.415484+0000","last_peered":"2024-04-25T01:26:03.415484+0000","last_clean":"2024-04-25T01:26:03.415484+0000","last_became_active":"2024-04-25T01:26:01.406855+0000","last_became_peered":"2024-04-25T01:26:01.406855+0000","last_unstale":"2024-04-25T01:26:03.415484+0000","last_undegraded":"2024-04-25T01:26:03.415484+0000","last_fullsized":"2024-04-25T01:26:03.415484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:02.180826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716643+0000","last_change":"2024-04-25T01:26:02.425367+0000","last_active":"2024-04-25T01:26:03.716643+0000","last_peered":"2024-04-25T01:26:03.716643+0000","last_clean":"2024-04-25T01:26:03.716643+0000","last_became_active":"2024-04-25T01:26:02.425071+0000","last_became_peered":"2024-04-25T01:26:02.425071+0000","last_unstale":"2024-04-25T01:26:03.716643+0000","last_undegraded":"2024-04-25T01:26:03.716643+0000","last_fullsized":"2024-04-25T01:26:03.716643+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:47.810943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416737+0000","last_change":"2024-04-25T01:25:52.349185+0000","last_active":"2024-04-25T01:26:03.416737+0000","last_peered":"2024-04-25T01:26:03.416737+0000","last_clean":"2024-04-25T01:26:03.416737+0000","last_became_active":"2024-04-25T01:25:52.348931+0000","last_became_peered":"2024-04-25T01:25:52.348931+0000","last_unstale":"2024-04-25T01:26:03.416737+0000","last_undegraded":"2024-04-25T01:26:03.416737+0000","last_fullsized":"2024-04-25T01:26:03.416737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716082+0000","last_change":"2024-04-25T01:25:53.374150+0000","last_active":"2024-04-25T01:26:03.716082+0000","last_peered":"2024-04-25T01:26:03.716082+0000","last_clean":"2024-04-25T01:26:03.716082+0000","last_became_active":"2024-04-25T01:25:53.373755+0000","last_became_peered":"2024-04-25T01:25:53.373755+0000","last_unstale":"2024-04-25T01:26:03.716082+0000","last_undegraded":"2024-04-25T01:26:03.716082+0000","last_fullsized":"2024-04-25T01:26:03.716082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415473+0000","last_change":"2024-04-25T01:26:01.406334+0000","last_active":"2024-04-25T01:26:03.415473+0000","last_peered":"2024-04-25T01:26:03.415473+0000","last_clean":"2024-04-25T01:26:03.415473+0000","last_became_active":"2024-04-25T01:26:01.405994+0000","last_became_peered":"2024-04-25T01:26:01.405994+0000","last_unstale":"2024-04-25T01:26:03.415473+0000","last_undegraded":"2024-04-25T01:26:03.415473+0000","last_fullsized":"2024-04-25T01:26:03.415473+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:43.953437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507172+0000","last_change":"2024-04-25T01:26:02.427695+0000","last_active":"2024-04-25T01:26:03.507172+0000","last_peered":"2024-04-25T01:26:03.507172+0000","last_clean":"2024-04-25T01:26:03.507172+0000","last_became_active":"2024-04-25T01:26:02.427443+0000","last_became_peered":"2024-04-25T01:26:02.427443+0000","last_unstale":"2024-04-25T01:26:03.507172+0000","last_undegraded":"2024-04-25T01:26:03.507172+0000","last_fullsized":"2024-04-25T01:26:03.507172+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507616+0000","last_change":"2024-04-25T01:25:52.356403+0000","last_active":"2024-04-25T01:26:03.507616+0000","last_peered":"2024-04-25T01:26:03.507616+0000","last_clean":"2024-04-25T01:26:03.507616+0000","last_became_active":"2024-04-25T01:25:52.355822+0000","last_became_peered":"2024-04-25T01:25:52.355822+0000","last_unstale":"2024-04-25T01:26:03.507616+0000","last_undegraded":"2024-04-25T01:26:03.507616+0000","last_fullsized":"2024-04-25T01:26:03.507616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717449+0000","last_change":"2024-04-25T01:25:53.370061+0000","last_active":"2024-04-25T01:26:03.717449+0000","last_peered":"2024-04-25T01:26:03.717449+0000","last_clean":"2024-04-25T01:26:03.717449+0000","last_became_active":"2024-04-25T01:25:53.369721+0000","last_became_peered":"2024-04-25T01:25:53.369721+0000","last_unstale":"2024-04-25T01:26:03.717449+0000","last_undegraded":"2024-04-25T01:26:03.717449+0000","last_fullsized":"2024-04-25T01:26:03.717449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415934+0000","last_change":"2024-04-25T01:26:01.408179+0000","last_active":"2024-04-25T01:26:03.415934+0000","last_peered":"2024-04-25T01:26:03.415934+0000","last_clean":"2024-04-25T01:26:03.415934+0000","last_became_active":"2024-04-25T01:26:01.407909+0000","last_became_peered":"2024-04-25T01:26:01.407909+0000","last_unstale":"2024-04-25T01:26:03.415934+0000","last_undegraded":"2024-04-25T01:26:03.415934+0000","last_fullsized":"2024-04-25T01:26:03.415934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:48.865940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717234+0000","last_change":"2024-04-25T01:26:02.424615+0000","last_active":"2024-04-25T01:26:03.717234+0000","last_peered":"2024-04-25T01:26:03.717234+0000","last_clean":"2024-04-25T01:26:03.717234+0000","last_became_active":"2024-04-25T01:26:02.424325+0000","last_became_peered":"2024-04-25T01:26:02.424325+0000","last_unstale":"2024-04-25T01:26:03.717234+0000","last_undegraded":"2024-04-25T01:26:03.717234+0000","last_fullsized":"2024-04-25T01:26:03.717234+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507443+0000","last_change":"2024-04-25T01:25:52.359139+0000","last_active":"2024-04-25T01:26:03.507443+0000","last_peered":"2024-04-25T01:26:03.507443+0000","last_clean":"2024-04-25T01:26:03.507443+0000","last_became_active":"2024-04-25T01:25:52.358841+0000","last_became_peered":"2024-04-25T01:25:52.358841+0000","last_unstale":"2024-04-25T01:26:03.507443+0000","last_undegraded":"2024-04-25T01:26:03.507443+0000","last_fullsized":"2024-04-25T01:26:03.507443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717907+0000","last_change":"2024-04-25T01:25:53.368739+0000","last_active":"2024-04-25T01:26:03.717907+0000","last_peered":"2024-04-25T01:26:03.717907+0000","last_clean":"2024-04-25T01:26:03.717907+0000","last_became_active":"2024-04-25T01:25:53.368470+0000","last_became_peered":"2024-04-25T01:25:53.368470+0000","last_unstale":"2024-04-25T01:26:03.717907+0000","last_undegraded":"2024-04-25T01:26:03.717907+0000","last_fullsized":"2024-04-25T01:26:03.717907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:29.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415820+0000","last_change":"2024-04-25T01:26:01.407969+0000","last_active":"2024-04-25T01:26:03.415820+0000","last_peered":"2024-04-25T01:26:03.415820+0000","last_clean":"2024-04-25T01:26:03.415820+0000","last_became_active":"2024-04-25T01:26:01.407655+0000","last_became_peered":"2024-04-25T01:26:01.407655+0000","last_unstale":"2024-04-25T01:26:03.415820+0000","last_undegraded":"2024-04-25T01:26:03.415820+0000","last_fullsized":"2024-04-25T01:26:03.415820+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:58.485864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717005+0000","last_change":"2024-04-25T01:26:02.417776+0000","last_active":"2024-04-25T01:26:03.717005+0000","last_peered":"2024-04-25T01:26:03.717005+0000","last_clean":"2024-04-25T01:26:03.717005+0000","last_became_active":"2024-04-25T01:26:02.417524+0000","last_became_peered":"2024-04-25T01:26:02.417524+0000","last_unstale":"2024-04-25T01:26:03.717005+0000","last_undegraded":"2024-04-25T01:26:03.717005+0000","last_fullsized":"2024-04-25T01:26:03.717005+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.688469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716814+0000","last_change":"2024-04-25T01:25:53.372385+0000","last_active":"2024-04-25T01:26:03.716814+0000","last_peered":"2024-04-25T01:26:03.716814+0000","last_clean":"2024-04-25T01:26:03.716814+0000","last_became_active":"2024-04-25T01:25:53.372064+0000","last_became_peered":"2024-04-25T01:25:53.372064+0000","last_unstale":"2024-04-25T01:26:03.716814+0000","last_undegraded":"2024-04-25T01:26:03.716814+0000","last_fullsized":"2024-04-25T01:26:03.716814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507411+0000","last_change":"2024-04-25T01:25:52.358616+0000","last_active":"2024-04-25T01:26:03.507411+0000","last_peered":"2024-04-25T01:26:03.507411+0000","last_clean":"2024-04-25T01:26:03.507411+0000","last_became_active":"2024-04-25T01:25:52.358318+0000","last_became_peered":"2024-04-25T01:25:52.358318+0000","last_unstale":"2024-04-25T01:26:03.507411+0000","last_undegraded":"2024-04-25T01:26:03.507411+0000","last_fullsized":"2024-04-25T01:26:03.507411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416542+0000","last_change":"2024-04-25T01:26:01.400599+0000","last_active":"2024-04-25T01:26:03.416542+0000","last_peered":"2024-04-25T01:26:03.416542+0000","last_clean":"2024-04-25T01:26:03.416542+0000","last_became_active":"2024-04-25T01:26:01.400266+0000","last_became_peered":"2024-04-25T01:26:01.400266+0000","last_unstale":"2024-04-25T01:26:03.416542+0000","last_undegraded":"2024-04-25T01:26:03.416542+0000","last_fullsized":"2024-04-25T01:26:03.416542+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416953+0000","last_change":"2024-04-25T01:26:02.420269+0000","last_active":"2024-04-25T01:26:03.416953+0000","last_peered":"2024-04-25T01:26:03.416953+0000","last_clean":"2024-04-25T01:26:03.416953+0000","last_became_active":"2024-04-25T01:26:02.420001+0000","last_became_peered":"2024-04-25T01:26:02.420001+0000","last_unstale":"2024-04-25T01:26:03.416953+0000","last_undegraded":"2024-04-25T01:26:03.416953+0000","last_fullsized":"2024-04-25T01:26:03.416953+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:38.149069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415549+0000","last_change":"2024-04-25T01:25:53.361074+0000","last_active":"2024-04-25T01:26:03.415549+0000","last_peered":"2024-04-25T01:26:03.415549+0000","last_clean":"2024-04-25T01:26:03.415549+0000","last_became_active":"2024-04-25T01:25:53.360802+0000","last_became_peered":"2024-04-25T01:25:53.360802+0000","last_unstale":"2024-04-25T01:26:03.415549+0000","last_undegraded":"2024-04-25T01:26:03.415549+0000","last_fullsized":"2024-04-25T01:26:03.415549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717195+0000","last_change":"2024-04-25T01:25:52.348841+0000","last_active":"2024-04-25T01:26:03.717195+0000","last_peered":"2024-04-25T01:26:03.717195+0000","last_clean":"2024-04-25T01:26:03.717195+0000","last_became_active":"2024-04-25T01:25:52.348550+0000","last_became_peered":"2024-04-25T01:25:52.348550+0000","last_unstale":"2024-04-25T01:26:03.717195+0000","last_undegraded":"2024-04-25T01:26:03.717195+0000","last_fullsized":"2024-04-25T01:26:03.717195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416126+0000","last_change":"2024-04-25T01:26:01.408529+0000","last_active":"2024-04-25T01:26:03.416126+0000","last_peered":"2024-04-25T01:26:03.416126+0000","last_clean":"2024-04-25T01:26:03.416126+0000","last_became_active":"2024-04-25T01:26:01.407738+0000","last_became_peered":"2024-04-25T01:26:01.407738+0000","last_unstale":"2024-04-25T01:26:03.416126+0000","last_undegraded":"2024-04-25T01:26:03.416126+0000","last_fullsized":"2024-04-25T01:26:03.416126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:37.142255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416507+0000","last_change":"2024-04-25T01:26:02.417731+0000","last_active":"2024-04-25T01:26:03.416507+0000","last_peered":"2024-04-25T01:26:03.416507+0000","last_clean":"2024-04-25T01:26:03.416507+0000","last_became_active":"2024-04-25T01:26:02.417438+0000","last_became_peered":"2024-04-25T01:26:02.417438+0000","last_unstale":"2024-04-25T01:26:03.416507+0000","last_undegraded":"2024-04-25T01:26:03.416507+0000","last_fullsized":"2024-04-25T01:26:03.416507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415373+0000","last_change":"2024-04-25T01:25:53.361790+0000","last_active":"2024-04-25T01:26:03.415373+0000","last_peered":"2024-04-25T01:26:03.415373+0000","last_clean":"2024-04-25T01:26:03.415373+0000","last_became_active":"2024-04-25T01:25:53.361542+0000","last_became_peered":"2024-04-25T01:25:53.361542+0000","last_unstale":"2024-04-25T01:26:03.415373+0000","last_undegraded":"2024-04-25T01:26:03.415373+0000","last_fullsized":"2024-04-25T01:26:03.415373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716946+0000","last_change":"2024-04-25T01:25:52.353926+0000","last_active":"2024-04-25T01:26:03.716946+0000","last_peered":"2024-04-25T01:26:03.716946+0000","last_clean":"2024-04-25T01:26:03.716946+0000","last_became_active":"2024-04-25T01:25:52.353656+0000","last_became_peered":"2024-04-25T01:25:52.353656+0000","last_unstale":"2024-04-25T01:26:03.716946+0000","last_undegraded":"2024-04-25T01:26:03.716946+0000","last_fullsized":"2024-04-25T01:26:03.716946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716923+0000","last_change":"2024-04-25T01:26:01.418556+0000","last_active":"2024-04-25T01:26:03.716923+0000","last_peered":"2024-04-25T01:26:03.716923+0000","last_clean":"2024-04-25T01:26:03.716923+0000","last_became_active":"2024-04-25T01:26:01.418263+0000","last_became_peered":"2024-04-25T01:26:01.418263+0000","last_unstale":"2024-04-25T01:26:03.716923+0000","last_undegraded":"2024-04-25T01:26:03.716923+0000","last_fullsized":"2024-04-25T01:26:03.716923+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:28.139183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717839+0000","last_change":"2024-04-25T01:26:02.413429+0000","last_active":"2024-04-25T01:26:03.717839+0000","last_peered":"2024-04-25T01:26:03.717839+0000","last_clean":"2024-04-25T01:26:03.717839+0000","last_became_active":"2024-04-25T01:26:02.412632+0000","last_became_peered":"2024-04-25T01:26:02.412632+0000","last_unstale":"2024-04-25T01:26:03.717839+0000","last_undegraded":"2024-04-25T01:26:03.717839+0000","last_fullsized":"2024-04-25T01:26:03.717839+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:10.049310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416067+0000","last_change":"2024-04-25T01:25:53.361653+0000","last_active":"2024-04-25T01:26:03.416067+0000","last_peered":"2024-04-25T01:26:03.416067+0000","last_clean":"2024-04-25T01:26:03.416067+0000","last_became_active":"2024-04-25T01:25:53.361265+0000","last_became_peered":"2024-04-25T01:25:53.361265+0000","last_unstale":"2024-04-25T01:26:03.416067+0000","last_undegraded":"2024-04-25T01:26:03.416067+0000","last_fullsized":"2024-04-25T01:26:03.416067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507177+0000","last_change":"2024-04-25T01:25:52.353836+0000","last_active":"2024-04-25T01:26:03.507177+0000","last_peered":"2024-04-25T01:26:03.507177+0000","last_clean":"2024-04-25T01:26:03.507177+0000","last_became_active":"2024-04-25T01:25:52.353552+0000","last_became_peered":"2024-04-25T01:25:52.353552+0000","last_unstale":"2024-04-25T01:26:03.507177+0000","last_undegraded":"2024-04-25T01:26:03.507177+0000","last_fullsized":"2024-04-25T01:26:03.507177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416325+0000","last_change":"2024-04-25T01:26:01.411323+0000","last_active":"2024-04-25T01:26:03.416325+0000","last_peered":"2024-04-25T01:26:03.416325+0000","last_clean":"2024-04-25T01:26:03.416325+0000","last_became_active":"2024-04-25T01:26:01.410378+0000","last_became_peered":"2024-04-25T01:26:01.410378+0000","last_unstale":"2024-04-25T01:26:03.416325+0000","last_undegraded":"2024-04-25T01:26:03.416325+0000","last_fullsized":"2024-04-25T01:26:03.416325+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415000+0000","last_change":"2024-04-25T01:26:02.416646+0000","last_active":"2024-04-25T01:26:03.415000+0000","last_peered":"2024-04-25T01:26:03.415000+0000","last_clean":"2024-04-25T01:26:03.415000+0000","last_became_active":"2024-04-25T01:26:02.416395+0000","last_became_peered":"2024-04-25T01:26:02.416395+0000","last_unstale":"2024-04-25T01:26:03.415000+0000","last_undegraded":"2024-04-25T01:26:03.415000+0000","last_fullsized":"2024-04-25T01:26:03.415000+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716795+0000","last_change":"2024-04-25T01:25:53.370111+0000","last_active":"2024-04-25T01:26:03.716795+0000","last_peered":"2024-04-25T01:26:03.716795+0000","last_clean":"2024-04-25T01:26:03.716795+0000","last_became_active":"2024-04-25T01:25:53.369837+0000","last_became_peered":"2024-04-25T01:25:53.369837+0000","last_unstale":"2024-04-25T01:26:03.716795+0000","last_undegraded":"2024-04-25T01:26:03.716795+0000","last_fullsized":"2024-04-25T01:26:03.716795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717961+0000","last_change":"2024-04-25T01:25:52.353269+0000","last_active":"2024-04-25T01:26:03.717961+0000","last_peered":"2024-04-25T01:26:03.717961+0000","last_clean":"2024-04-25T01:26:03.717961+0000","last_became_active":"2024-04-25T01:25:52.352991+0000","last_became_peered":"2024-04-25T01:25:52.352991+0000","last_unstale":"2024-04-25T01:26:03.717961+0000","last_undegraded":"2024-04-25T01:26:03.717961+0000","last_fullsized":"2024-04-25T01:26:03.717961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415651+0000","last_change":"2024-04-25T01:26:01.410360+0000","last_active":"2024-04-25T01:26:03.415651+0000","last_peered":"2024-04-25T01:26:03.415651+0000","last_clean":"2024-04-25T01:26:03.415651+0000","last_became_active":"2024-04-25T01:26:01.410076+0000","last_became_peered":"2024-04-25T01:26:01.410076+0000","last_unstale":"2024-04-25T01:26:03.415651+0000","last_undegraded":"2024-04-25T01:26:03.415651+0000","last_fullsized":"2024-04-25T01:26:03.415651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:52.561251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415577+0000","last_change":"2024-04-25T01:26:02.422182+0000","last_active":"2024-04-25T01:26:03.415577+0000","last_peered":"2024-04-25T01:26:03.415577+0000","last_clean":"2024-04-25T01:26:03.415577+0000","last_became_active":"2024-04-25T01:26:02.421885+0000","last_became_peered":"2024-04-25T01:26:02.421885+0000","last_unstale":"2024-04-25T01:26:03.415577+0000","last_undegraded":"2024-04-25T01:26:03.415577+0000","last_fullsized":"2024-04-25T01:26:03.415577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:03.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416628+0000","last_change":"2024-04-25T01:25:53.364967+0000","last_active":"2024-04-25T01:26:03.416628+0000","last_peered":"2024-04-25T01:26:03.416628+0000","last_clean":"2024-04-25T01:26:03.416628+0000","last_became_active":"2024-04-25T01:25:53.364694+0000","last_became_peered":"2024-04-25T01:25:53.364694+0000","last_unstale":"2024-04-25T01:26:03.416628+0000","last_undegraded":"2024-04-25T01:26:03.416628+0000","last_fullsized":"2024-04-25T01:26:03.416628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717049+0000","last_change":"2024-04-25T01:25:52.355395+0000","last_active":"2024-04-25T01:26:03.717049+0000","last_peered":"2024-04-25T01:26:03.717049+0000","last_clean":"2024-04-25T01:26:03.717049+0000","last_became_active":"2024-04-25T01:25:52.355137+0000","last_became_peered":"2024-04-25T01:25:52.355137+0000","last_unstale":"2024-04-25T01:26:03.717049+0000","last_undegraded":"2024-04-25T01:26:03.717049+0000","last_fullsized":"2024-04-25T01:26:03.717049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:26.467968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.414943+0000","last_change":"2024-04-25T01:26:01.416108+0000","last_active":"2024-04-25T01:26:03.414943+0000","last_peered":"2024-04-25T01:26:03.414943+0000","last_clean":"2024-04-25T01:26:03.414943+0000","last_became_active":"2024-04-25T01:26:01.415824+0000","last_became_peered":"2024-04-25T01:26:01.415824+0000","last_unstale":"2024-04-25T01:26:03.414943+0000","last_undegraded":"2024-04-25T01:26:03.414943+0000","last_fullsized":"2024-04-25T01:26:03.414943+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:28.139183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716568+0000","last_change":"2024-04-25T01:26:02.416208+0000","last_active":"2024-04-25T01:26:03.716568+0000","last_peered":"2024-04-25T01:26:03.716568+0000","last_clean":"2024-04-25T01:26:03.716568+0000","last_became_active":"2024-04-25T01:26:02.415949+0000","last_became_peered":"2024-04-25T01:26:02.415949+0000","last_unstale":"2024-04-25T01:26:03.716568+0000","last_undegraded":"2024-04-25T01:26:03.716568+0000","last_fullsized":"2024-04-25T01:26:03.716568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:21.097225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717249+0000","last_change":"2024-04-25T01:25:53.366944+0000","last_active":"2024-04-25T01:26:03.717249+0000","last_peered":"2024-04-25T01:26:03.717249+0000","last_clean":"2024-04-25T01:26:03.717249+0000","last_became_active":"2024-04-25T01:25:53.366693+0000","last_became_peered":"2024-04-25T01:25:53.366693+0000","last_unstale":"2024-04-25T01:26:03.717249+0000","last_undegraded":"2024-04-25T01:26:03.717249+0000","last_fullsized":"2024-04-25T01:26:03.717249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507732+0000","last_change":"2024-04-25T01:25:52.356842+0000","last_active":"2024-04-25T01:26:03.507732+0000","last_peered":"2024-04-25T01:26:03.507732+0000","last_clean":"2024-04-25T01:26:03.507732+0000","last_became_active":"2024-04-25T01:25:52.356396+0000","last_became_peered":"2024-04-25T01:25:52.356396+0000","last_unstale":"2024-04-25T01:26:03.507732+0000","last_undegraded":"2024-04-25T01:26:03.507732+0000","last_fullsized":"2024-04-25T01:26:03.507732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716730+0000","last_change":"2024-04-25T01:26:01.407934+0000","last_active":"2024-04-25T01:26:03.716730+0000","last_peered":"2024-04-25T01:26:03.716730+0000","last_clean":"2024-04-25T01:26:03.716730+0000","last_became_active":"2024-04-25T01:26:01.407659+0000","last_became_peered":"2024-04-25T01:26:01.407659+0000","last_unstale":"2024-04-25T01:26:03.716730+0000","last_undegraded":"2024-04-25T01:26:03.716730+0000","last_fullsized":"2024-04-25T01:26:03.716730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:19.629335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716666+0000","last_change":"2024-04-25T01:26:02.418087+0000","last_active":"2024-04-25T01:26:03.716666+0000","last_peered":"2024-04-25T01:26:03.716666+0000","last_clean":"2024-04-25T01:26:03.716666+0000","last_became_active":"2024-04-25T01:26:02.417622+0000","last_became_peered":"2024-04-25T01:26:02.417622+0000","last_unstale":"2024-04-25T01:26:03.716666+0000","last_undegraded":"2024-04-25T01:26:03.716666+0000","last_fullsized":"2024-04-25T01:26:03.716666+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:03.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717962+0000","last_change":"2024-04-25T01:25:53.368164+0000","last_active":"2024-04-25T01:26:03.717962+0000","last_peered":"2024-04-25T01:26:03.717962+0000","last_clean":"2024-04-25T01:26:03.717962+0000","last_became_active":"2024-04-25T01:25:53.367854+0000","last_became_peered":"2024-04-25T01:25:53.367854+0000","last_unstale":"2024-04-25T01:26:03.717962+0000","last_undegraded":"2024-04-25T01:26:03.717962+0000","last_fullsized":"2024-04-25T01:26:03.717962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717294+0000","last_change":"2024-04-25T01:25:52.350128+0000","last_active":"2024-04-25T01:26:03.717294+0000","last_peered":"2024-04-25T01:26:03.717294+0000","last_clean":"2024-04-25T01:26:03.717294+0000","last_became_active":"2024-04-25T01:25:52.349874+0000","last_became_peered":"2024-04-25T01:25:52.349874+0000","last_unstale":"2024-04-25T01:26:03.717294+0000","last_undegraded":"2024-04-25T01:26:03.717294+0000","last_fullsized":"2024-04-25T01:26:03.717294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716937+0000","last_change":"2024-04-25T01:26:01.418196+0000","last_active":"2024-04-25T01:26:03.716937+0000","last_peered":"2024-04-25T01:26:03.716937+0000","last_clean":"2024-04-25T01:26:03.716937+0000","last_became_active":"2024-04-25T01:26:01.417820+0000","last_became_peered":"2024-04-25T01:26:01.417820+0000","last_unstale":"2024-04-25T01:26:03.716937+0000","last_undegraded":"2024-04-25T01:26:03.716937+0000","last_fullsized":"2024-04-25T01:26:03.716937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716324+0000","last_change":"2024-04-25T01:26:02.422761+0000","last_active":"2024-04-25T01:26:03.716324+0000","last_peered":"2024-04-25T01:26:03.716324+0000","last_clean":"2024-04-25T01:26:03.716324+0000","last_became_active":"2024-04-25T01:26:02.422503+0000","last_became_peered":"2024-04-25T01:26:02.422503+0000","last_unstale":"2024-04-25T01:26:03.716324+0000","last_undegraded":"2024-04-25T01:26:03.716324+0000","last_fullsized":"2024-04-25T01:26:03.716324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:55.040946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416617+0000","last_change":"2024-04-25T01:25:53.361696+0000","last_active":"2024-04-25T01:26:03.416617+0000","last_peered":"2024-04-25T01:26:03.416617+0000","last_clean":"2024-04-25T01:26:03.416617+0000","last_became_active":"2024-04-25T01:25:53.361156+0000","last_became_peered":"2024-04-25T01:25:53.361156+0000","last_unstale":"2024-04-25T01:26:03.416617+0000","last_undegraded":"2024-04-25T01:26:03.416617+0000","last_fullsized":"2024-04-25T01:26:03.416617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:27.472315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717721+0000","last_change":"2024-04-25T01:25:52.354499+0000","last_active":"2024-04-25T01:26:03.717721+0000","last_peered":"2024-04-25T01:26:03.717721+0000","last_clean":"2024-04-25T01:26:03.717721+0000","last_became_active":"2024-04-25T01:25:52.354193+0000","last_became_peered":"2024-04-25T01:25:52.354193+0000","last_unstale":"2024-04-25T01:26:03.717721+0000","last_undegraded":"2024-04-25T01:26:03.717721+0000","last_fullsized":"2024-04-25T01:26:03.717721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717190+0000","last_change":"2024-04-25T01:26:01.418186+0000","last_active":"2024-04-25T01:26:03.717190+0000","last_peered":"2024-04-25T01:26:03.717190+0000","last_clean":"2024-04-25T01:26:03.717190+0000","last_became_active":"2024-04-25T01:26:01.417805+0000","last_became_peered":"2024-04-25T01:26:01.417805+0000","last_unstale":"2024-04-25T01:26:03.717190+0000","last_undegraded":"2024-04-25T01:26:03.717190+0000","last_fullsized":"2024-04-25T01:26:03.717190+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:19.629335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416697+0000","last_change":"2024-04-25T01:26:02.415877+0000","last_active":"2024-04-25T01:26:03.416697+0000","last_peered":"2024-04-25T01:26:03.416697+0000","last_clean":"2024-04-25T01:26:03.416697+0000","last_became_active":"2024-04-25T01:26:02.415619+0000","last_became_peered":"2024-04-25T01:26:02.415619+0000","last_unstale":"2024-04-25T01:26:03.416697+0000","last_undegraded":"2024-04-25T01:26:03.416697+0000","last_fullsized":"2024-04-25T01:26:03.416697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:14.647083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716434+0000","last_change":"2024-04-25T01:25:53.362922+0000","last_active":"2024-04-25T01:26:03.716434+0000","last_peered":"2024-04-25T01:26:03.716434+0000","last_clean":"2024-04-25T01:26:03.716434+0000","last_became_active":"2024-04-25T01:25:53.362533+0000","last_became_peered":"2024-04-25T01:25:53.362533+0000","last_unstale":"2024-04-25T01:26:03.716434+0000","last_undegraded":"2024-04-25T01:26:03.716434+0000","last_fullsized":"2024-04-25T01:26:03.716434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415851+0000","last_change":"2024-04-25T01:25:52.345321+0000","last_active":"2024-04-25T01:26:03.415851+0000","last_peered":"2024-04-25T01:26:03.415851+0000","last_clean":"2024-04-25T01:26:03.415851+0000","last_became_active":"2024-04-25T01:25:52.345061+0000","last_became_peered":"2024-04-25T01:25:52.345061+0000","last_unstale":"2024-04-25T01:26:03.415851+0000","last_undegraded":"2024-04-25T01:26:03.415851+0000","last_fullsized":"2024-04-25T01:26:03.415851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717072+0000","last_change":"2024-04-25T01:26:01.418516+0000","last_active":"2024-04-25T01:26:03.717072+0000","last_peered":"2024-04-25T01:26:03.717072+0000","last_clean":"2024-04-25T01:26:03.717072+0000","last_became_active":"2024-04-25T01:26:01.418173+0000","last_became_peered":"2024-04-25T01:26:01.418173+0000","last_unstale":"2024-04-25T01:26:03.717072+0000","last_undegraded":"2024-04-25T01:26:03.717072+0000","last_fullsized":"2024-04-25T01:26:03.717072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:02.568472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.718145+0000","last_change":"2024-04-25T01:26:02.418115+0000","last_active":"2024-04-25T01:26:03.718145+0000","last_peered":"2024-04-25T01:26:03.718145+0000","last_clean":"2024-04-25T01:26:03.718145+0000","last_became_active":"2024-04-25T01:26:02.417355+0000","last_became_peered":"2024-04-25T01:26:02.417355+0000","last_unstale":"2024-04-25T01:26:03.718145+0000","last_undegraded":"2024-04-25T01:26:03.718145+0000","last_fullsized":"2024-04-25T01:26:03.718145+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:19.644468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716066+0000","last_change":"2024-04-25T01:25:53.355950+0000","last_active":"2024-04-25T01:26:03.716066+0000","last_peered":"2024-04-25T01:26:03.716066+0000","last_clean":"2024-04-25T01:26:03.716066+0000","last_became_active":"2024-04-25T01:25:53.355648+0000","last_became_peered":"2024-04-25T01:25:53.355648+0000","last_unstale":"2024-04-25T01:26:03.716066+0000","last_undegraded":"2024-04-25T01:26:03.716066+0000","last_fullsized":"2024-04-25T01:26:03.716066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507424+0000","last_change":"2024-04-25T01:25:52.357683+0000","last_active":"2024-04-25T01:26:03.507424+0000","last_peered":"2024-04-25T01:26:03.507424+0000","last_clean":"2024-04-25T01:26:03.507424+0000","last_became_active":"2024-04-25T01:25:52.356924+0000","last_became_peered":"2024-04-25T01:25:52.356924+0000","last_unstale":"2024-04-25T01:26:03.507424+0000","last_undegraded":"2024-04-25T01:26:03.507424+0000","last_fullsized":"2024-04-25T01:26:03.507424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716612+0000","last_change":"2024-04-25T01:26:01.421260+0000","last_active":"2024-04-25T01:26:03.716612+0000","last_peered":"2024-04-25T01:26:03.716612+0000","last_clean":"2024-04-25T01:26:03.716612+0000","last_became_active":"2024-04-25T01:26:01.420831+0000","last_became_peered":"2024-04-25T01:26:01.420831+0000","last_unstale":"2024-04-25T01:26:03.716612+0000","last_undegraded":"2024-04-25T01:26:03.716612+0000","last_fullsized":"2024-04-25T01:26:03.716612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507519+0000","last_change":"2024-04-25T01:26:02.421386+0000","last_active":"2024-04-25T01:26:03.507519+0000","last_peered":"2024-04-25T01:26:03.507519+0000","last_clean":"2024-04-25T01:26:03.507519+0000","last_became_active":"2024-04-25T01:26:02.421079+0000","last_became_peered":"2024-04-25T01:26:02.421079+0000","last_unstale":"2024-04-25T01:26:03.507519+0000","last_undegraded":"2024-04-25T01:26:03.507519+0000","last_fullsized":"2024-04-25T01:26:03.507519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416038+0000","last_change":"2024-04-25T01:25:53.374719+0000","last_active":"2024-04-25T01:26:03.416038+0000","last_peered":"2024-04-25T01:26:03.416038+0000","last_clean":"2024-04-25T01:26:03.416038+0000","last_became_active":"2024-04-25T01:25:53.374425+0000","last_became_peered":"2024-04-25T01:25:53.374425+0000","last_unstale":"2024-04-25T01:26:03.416038+0000","last_undegraded":"2024-04-25T01:26:03.416038+0000","last_fullsized":"2024-04-25T01:26:03.416038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717309+0000","last_change":"2024-04-25T01:25:52.353155+0000","last_active":"2024-04-25T01:26:03.717309+0000","last_peered":"2024-04-25T01:26:03.717309+0000","last_clean":"2024-04-25T01:26:03.717309+0000","last_became_active":"2024-04-25T01:25:52.352826+0000","last_became_peered":"2024-04-25T01:25:52.352826+0000","last_unstale":"2024-04-25T01:26:03.717309+0000","last_undegraded":"2024-04-25T01:26:03.717309+0000","last_fullsized":"2024-04-25T01:26:03.717309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507554+0000","last_change":"2024-04-25T01:25:41.719670+0000","last_active":"2024-04-25T01:26:03.507554+0000","last_peered":"2024-04-25T01:26:03.507554+0000","last_clean":"2024-04-25T01:26:03.507554+0000","last_became_active":"2024-04-25T01:25:41.718721+0000","last_became_peered":"2024-04-25T01:25:41.718721+0000","last_unstale":"2024-04-25T01:26:03.507554+0000","last_undegraded":"2024-04-25T01:26:03.507554+0000","last_fullsized":"2024-04-25T01:26:03.507554+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_clean_scrub_stamp":"2024-04-25T01:25:40.688097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:27.456746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717378+0000","last_change":"2024-04-25T01:26:01.409339+0000","last_active":"2024-04-25T01:26:03.717378+0000","last_peered":"2024-04-25T01:26:03.717378+0000","last_clean":"2024-04-25T01:26:03.717378+0000","last_became_active":"2024-04-25T01:26:01.409062+0000","last_became_peered":"2024-04-25T01:26:01.409062+0000","last_unstale":"2024-04-25T01:26:03.717378+0000","last_undegraded":"2024-04-25T01:26:03.717378+0000","last_fullsized":"2024-04-25T01:26:03.717378+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:52.561251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415532+0000","last_change":"2024-04-25T01:26:02.418574+0000","last_active":"2024-04-25T01:26:03.415532+0000","last_peered":"2024-04-25T01:26:03.415532+0000","last_clean":"2024-04-25T01:26:03.415532+0000","last_became_active":"2024-04-25T01:26:02.416546+0000","last_became_peered":"2024-04-25T01:26:02.416546+0000","last_unstale":"2024-04-25T01:26:03.415532+0000","last_undegraded":"2024-04-25T01:26:03.415532+0000","last_fullsized":"2024-04-25T01:26:03.415532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:43.994141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507591+0000","last_change":"2024-04-25T01:25:53.375523+0000","last_active":"2024-04-25T01:26:03.507591+0000","last_peered":"2024-04-25T01:26:03.507591+0000","last_clean":"2024-04-25T01:26:03.507591+0000","last_became_active":"2024-04-25T01:25:53.375219+0000","last_became_peered":"2024-04-25T01:25:53.375219+0000","last_unstale":"2024-04-25T01:26:03.507591+0000","last_undegraded":"2024-04-25T01:26:03.507591+0000","last_fullsized":"2024-04-25T01:26:03.507591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.417443+0000","last_change":"2024-04-25T01:25:52.352994+0000","last_active":"2024-04-25T01:26:03.417443+0000","last_peered":"2024-04-25T01:26:03.417443+0000","last_clean":"2024-04-25T01:26:03.417443+0000","last_became_active":"2024-04-25T01:25:52.352644+0000","last_became_peered":"2024-04-25T01:25:52.352644+0000","last_unstale":"2024-04-25T01:26:03.417443+0000","last_undegraded":"2024-04-25T01:26:03.417443+0000","last_fullsized":"2024-04-25T01:26:03.417443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716482+0000","last_change":"2024-04-25T01:26:01.420891+0000","last_active":"2024-04-25T01:26:03.716482+0000","last_peered":"2024-04-25T01:26:03.716482+0000","last_clean":"2024-04-25T01:26:03.716482+0000","last_became_active":"2024-04-25T01:26:01.420664+0000","last_became_peered":"2024-04-25T01:26:01.420664+0000","last_unstale":"2024-04-25T01:26:03.716482+0000","last_undegraded":"2024-04-25T01:26:03.716482+0000","last_fullsized":"2024-04-25T01:26:03.716482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415682+0000","last_change":"2024-04-25T01:26:02.403489+0000","last_active":"2024-04-25T01:26:03.415682+0000","last_peered":"2024-04-25T01:26:03.415682+0000","last_clean":"2024-04-25T01:26:03.415682+0000","last_became_active":"2024-04-25T01:26:02.403206+0000","last_became_peered":"2024-04-25T01:26:02.403206+0000","last_unstale":"2024-04-25T01:26:03.415682+0000","last_undegraded":"2024-04-25T01:26:03.415682+0000","last_fullsized":"2024-04-25T01:26:03.415682+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:49.872754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.718041+0000","last_change":"2024-04-25T01:25:53.371018+0000","last_active":"2024-04-25T01:26:03.718041+0000","last_peered":"2024-04-25T01:26:03.718041+0000","last_clean":"2024-04-25T01:26:03.718041+0000","last_became_active":"2024-04-25T01:25:53.370539+0000","last_became_peered":"2024-04-25T01:25:53.370539+0000","last_unstale":"2024-04-25T01:26:03.718041+0000","last_undegraded":"2024-04-25T01:26:03.718041+0000","last_fullsized":"2024-04-25T01:26:03.718041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717910+0000","last_change":"2024-04-25T01:25:52.355133+0000","last_active":"2024-04-25T01:26:03.717910+0000","last_peered":"2024-04-25T01:26:03.717910+0000","last_clean":"2024-04-25T01:26:03.717910+0000","last_became_active":"2024-04-25T01:25:52.354836+0000","last_became_peered":"2024-04-25T01:25:52.354836+0000","last_unstale":"2024-04-25T01:26:03.717910+0000","last_undegraded":"2024-04-25T01:26:03.717910+0000","last_fullsized":"2024-04-25T01:26:03.717910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:10.331704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716238+0000","last_change":"2024-04-25T01:26:01.419285+0000","last_active":"2024-04-25T01:26:03.716238+0000","last_peered":"2024-04-25T01:26:03.716238+0000","last_clean":"2024-04-25T01:26:03.716238+0000","last_became_active":"2024-04-25T01:26:01.418987+0000","last_became_peered":"2024-04-25T01:26:01.418987+0000","last_unstale":"2024-04-25T01:26:03.716238+0000","last_undegraded":"2024-04-25T01:26:03.716238+0000","last_fullsized":"2024-04-25T01:26:03.716238+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.506958+0000","last_change":"2024-04-25T01:26:02.416544+0000","last_active":"2024-04-25T01:26:03.506958+0000","last_peered":"2024-04-25T01:26:03.506958+0000","last_clean":"2024-04-25T01:26:03.506958+0000","last_became_active":"2024-04-25T01:26:02.416189+0000","last_became_peered":"2024-04-25T01:26:02.416189+0000","last_unstale":"2024-04-25T01:26:03.506958+0000","last_undegraded":"2024-04-25T01:26:03.506958+0000","last_fullsized":"2024-04-25T01:26:03.506958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.688469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.508134+0000","last_change":"2024-04-25T01:25:53.370754+0000","last_active":"2024-04-25T01:26:03.508134+0000","last_peered":"2024-04-25T01:26:03.508134+0000","last_clean":"2024-04-25T01:26:03.508134+0000","last_became_active":"2024-04-25T01:25:53.370074+0000","last_became_peered":"2024-04-25T01:25:53.370074+0000","last_unstale":"2024-04-25T01:26:03.508134+0000","last_undegraded":"2024-04-25T01:26:03.508134+0000","last_fullsized":"2024-04-25T01:26:03.508134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416836+0000","last_change":"2024-04-25T01:25:52.345943+0000","last_active":"2024-04-25T01:26:03.416836+0000","last_peered":"2024-04-25T01:26:03.416836+0000","last_clean":"2024-04-25T01:26:03.416836+0000","last_became_active":"2024-04-25T01:25:52.345717+0000","last_became_peered":"2024-04-25T01:25:52.345717+0000","last_unstale":"2024-04-25T01:26:03.416836+0000","last_undegraded":"2024-04-25T01:26:03.416836+0000","last_fullsized":"2024-04-25T01:26:03.416836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716622+0000","last_change":"2024-04-25T01:26:01.416027+0000","last_active":"2024-04-25T01:26:03.716622+0000","last_peered":"2024-04-25T01:26:03.716622+0000","last_clean":"2024-04-25T01:26:03.716622+0000","last_became_active":"2024-04-25T01:26:01.415750+0000","last_became_peered":"2024-04-25T01:26:01.415750+0000","last_unstale":"2024-04-25T01:26:03.716622+0000","last_undegraded":"2024-04-25T01:26:03.716622+0000","last_fullsized":"2024-04-25T01:26:03.716622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:27.042376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416127+0000","last_change":"2024-04-25T01:26:02.414929+0000","last_active":"2024-04-25T01:26:03.416127+0000","last_peered":"2024-04-25T01:26:03.416127+0000","last_clean":"2024-04-25T01:26:03.416127+0000","last_became_active":"2024-04-25T01:26:02.414316+0000","last_became_peered":"2024-04-25T01:26:02.414316+0000","last_unstale":"2024-04-25T01:26:03.416127+0000","last_undegraded":"2024-04-25T01:26:03.416127+0000","last_fullsized":"2024-04-25T01:26:03.416127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:53.568065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507875+0000","last_change":"2024-04-25T01:25:53.365171+0000","last_active":"2024-04-25T01:26:03.507875+0000","last_peered":"2024-04-25T01:26:03.507875+0000","last_clean":"2024-04-25T01:26:03.507875+0000","last_became_active":"2024-04-25T01:25:53.364049+0000","last_became_peered":"2024-04-25T01:25:53.364049+0000","last_unstale":"2024-04-25T01:26:03.507875+0000","last_undegraded":"2024-04-25T01:26:03.507875+0000","last_fullsized":"2024-04-25T01:26:03.507875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416635+0000","last_change":"2024-04-25T01:25:52.350688+0000","last_active":"2024-04-25T01:26:03.416635+0000","last_peered":"2024-04-25T01:26:03.416635+0000","last_clean":"2024-04-25T01:26:03.416635+0000","last_became_active":"2024-04-25T01:25:52.350357+0000","last_became_peered":"2024-04-25T01:25:52.350357+0000","last_unstale":"2024-04-25T01:26:03.416635+0000","last_undegraded":"2024-04-25T01:26:03.416635+0000","last_fullsized":"2024-04-25T01:26:03.416635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415491+0000","last_change":"2024-04-25T01:26:01.418580+0000","last_active":"2024-04-25T01:26:03.415491+0000","last_peered":"2024-04-25T01:26:03.415491+0000","last_clean":"2024-04-25T01:26:03.415491+0000","last_became_active":"2024-04-25T01:26:01.418179+0000","last_became_peered":"2024-04-25T01:26:01.418179+0000","last_unstale":"2024-04-25T01:26:03.415491+0000","last_undegraded":"2024-04-25T01:26:03.415491+0000","last_fullsized":"2024-04-25T01:26:03.415491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:42.987327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416438+0000","last_change":"2024-04-25T01:26:02.418424+0000","last_active":"2024-04-25T01:26:03.416438+0000","last_peered":"2024-04-25T01:26:03.416438+0000","last_clean":"2024-04-25T01:26:03.416438+0000","last_became_active":"2024-04-25T01:26:02.418090+0000","last_became_peered":"2024-04-25T01:26:02.418090+0000","last_unstale":"2024-04-25T01:26:03.416438+0000","last_undegraded":"2024-04-25T01:26:03.416438+0000","last_fullsized":"2024-04-25T01:26:03.416438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:10.049310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507217+0000","last_change":"2024-04-25T01:25:53.366376+0000","last_active":"2024-04-25T01:26:03.507217+0000","last_peered":"2024-04-25T01:26:03.507217+0000","last_clean":"2024-04-25T01:26:03.507217+0000","last_became_active":"2024-04-25T01:25:53.365715+0000","last_became_peered":"2024-04-25T01:25:53.365715+0000","last_unstale":"2024-04-25T01:26:03.507217+0000","last_undegraded":"2024-04-25T01:26:03.507217+0000","last_fullsized":"2024-04-25T01:26:03.507217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416219+0000","last_change":"2024-04-25T01:25:52.347413+0000","last_active":"2024-04-25T01:26:03.416219+0000","last_peered":"2024-04-25T01:26:03.416219+0000","last_clean":"2024-04-25T01:26:03.416219+0000","last_became_active":"2024-04-25T01:25:52.347144+0000","last_became_peered":"2024-04-25T01:25:52.347144+0000","last_unstale":"2024-04-25T01:26:03.416219+0000","last_undegraded":"2024-04-25T01:26:03.416219+0000","last_fullsized":"2024-04-25T01:26:03.416219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716710+0000","last_change":"2024-04-25T01:26:01.409917+0000","last_active":"2024-04-25T01:26:03.716710+0000","last_peered":"2024-04-25T01:26:03.716710+0000","last_clean":"2024-04-25T01:26:03.716710+0000","last_became_active":"2024-04-25T01:26:01.409654+0000","last_became_peered":"2024-04-25T01:26:01.409654+0000","last_unstale":"2024-04-25T01:26:03.716710+0000","last_undegraded":"2024-04-25T01:26:03.716710+0000","last_fullsized":"2024-04-25T01:26:03.716710+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:37.142255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717898+0000","last_change":"2024-04-25T01:26:02.425103+0000","last_active":"2024-04-25T01:26:03.717898+0000","last_peered":"2024-04-25T01:26:03.717898+0000","last_clean":"2024-04-25T01:26:03.717898+0000","last_became_active":"2024-04-25T01:26:02.424819+0000","last_became_peered":"2024-04-25T01:26:02.424819+0000","last_unstale":"2024-04-25T01:26:03.717898+0000","last_undegraded":"2024-04-25T01:26:03.717898+0000","last_fullsized":"2024-04-25T01:26:03.717898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716406+0000","last_change":"2024-04-25T01:25:53.364987+0000","last_active":"2024-04-25T01:26:03.716406+0000","last_peered":"2024-04-25T01:26:03.716406+0000","last_clean":"2024-04-25T01:26:03.716406+0000","last_became_active":"2024-04-25T01:25:53.364431+0000","last_became_peered":"2024-04-25T01:25:53.364431+0000","last_unstale":"2024-04-25T01:26:03.716406+0000","last_undegraded":"2024-04-25T01:26:03.716406+0000","last_fullsized":"2024-04-25T01:26:03.716406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716731+0000","last_change":"2024-04-25T01:25:52.351205+0000","last_active":"2024-04-25T01:26:03.716731+0000","last_peered":"2024-04-25T01:26:03.716731+0000","last_clean":"2024-04-25T01:26:03.716731+0000","last_became_active":"2024-04-25T01:25:52.350946+0000","last_became_peered":"2024-04-25T01:25:52.350946+0000","last_unstale":"2024-04-25T01:26:03.716731+0000","last_undegraded":"2024-04-25T01:26:03.716731+0000","last_fullsized":"2024-04-25T01:26:03.716731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416482+0000","last_change":"2024-04-25T01:26:01.415799+0000","last_active":"2024-04-25T01:26:03.416482+0000","last_peered":"2024-04-25T01:26:03.416482+0000","last_clean":"2024-04-25T01:26:03.416482+0000","last_became_active":"2024-04-25T01:26:01.415572+0000","last_became_peered":"2024-04-25T01:26:01.415572+0000","last_unstale":"2024-04-25T01:26:03.416482+0000","last_undegraded":"2024-04-25T01:26:03.416482+0000","last_fullsized":"2024-04-25T01:26:03.416482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:54.034132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717227+0000","last_change":"2024-04-25T01:26:02.422959+0000","last_active":"2024-04-25T01:26:03.717227+0000","last_peered":"2024-04-25T01:26:03.717227+0000","last_clean":"2024-04-25T01:26:03.717227+0000","last_became_active":"2024-04-25T01:26:02.422210+0000","last_became_peered":"2024-04-25T01:26:02.422210+0000","last_unstale":"2024-04-25T01:26:03.717227+0000","last_undegraded":"2024-04-25T01:26:03.717227+0000","last_fullsized":"2024-04-25T01:26:03.717227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507159+0000","last_change":"2024-04-25T01:25:53.368187+0000","last_active":"2024-04-25T01:26:03.507159+0000","last_peered":"2024-04-25T01:26:03.507159+0000","last_clean":"2024-04-25T01:26:03.507159+0000","last_became_active":"2024-04-25T01:25:53.367664+0000","last_became_peered":"2024-04-25T01:25:53.367664+0000","last_unstale":"2024-04-25T01:26:03.507159+0000","last_undegraded":"2024-04-25T01:26:03.507159+0000","last_fullsized":"2024-04-25T01:26:03.507159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717680+0000","last_change":"2024-04-25T01:25:52.357397+0000","last_active":"2024-04-25T01:26:03.717680+0000","last_peered":"2024-04-25T01:26:03.717680+0000","last_clean":"2024-04-25T01:26:03.717680+0000","last_became_active":"2024-04-25T01:25:52.356881+0000","last_became_peered":"2024-04-25T01:25:52.356881+0000","last_unstale":"2024-04-25T01:26:03.717680+0000","last_undegraded":"2024-04-25T01:26:03.717680+0000","last_fullsized":"2024-04-25T01:26:03.717680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"23'6","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416132+0000","last_change":"2024-04-25T01:26:01.420103+0000","last_active":"2024-04-25T01:26:03.416132+0000","last_peered":"2024-04-25T01:26:03.416132+0000","last_clean":"2024-04-25T01:26:03.416132+0000","last_became_active":"2024-04-25T01:26:01.419526+0000","last_became_peered":"2024-04-25T01:26:01.419526+0000","last_unstale":"2024-04-25T01:26:03.416132+0000","last_undegraded":"2024-04-25T01:26:03.416132+0000","last_fullsized":"2024-04-25T01:26:03.416132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:27.042376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416191+0000","last_change":"2024-04-25T01:26:02.421488+0000","last_active":"2024-04-25T01:26:03.416191+0000","last_peered":"2024-04-25T01:26:03.416191+0000","last_clean":"2024-04-25T01:26:03.416191+0000","last_became_active":"2024-04-25T01:26:02.419308+0000","last_became_peered":"2024-04-25T01:26:02.419308+0000","last_unstale":"2024-04-25T01:26:03.416191+0000","last_undegraded":"2024-04-25T01:26:03.416191+0000","last_fullsized":"2024-04-25T01:26:03.416191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716019+0000","last_change":"2024-04-25T01:25:53.370720+0000","last_active":"2024-04-25T01:26:03.716019+0000","last_peered":"2024-04-25T01:26:03.716019+0000","last_clean":"2024-04-25T01:26:03.716019+0000","last_became_active":"2024-04-25T01:25:53.370142+0000","last_became_peered":"2024-04-25T01:25:53.370142+0000","last_unstale":"2024-04-25T01:26:03.716019+0000","last_undegraded":"2024-04-25T01:26:03.716019+0000","last_fullsized":"2024-04-25T01:26:03.716019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:52.833379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716951+0000","last_change":"2024-04-25T01:25:52.349092+0000","last_active":"2024-04-25T01:26:03.716951+0000","last_peered":"2024-04-25T01:26:03.716951+0000","last_clean":"2024-04-25T01:26:03.716951+0000","last_became_active":"2024-04-25T01:25:52.348846+0000","last_became_peered":"2024-04-25T01:25:52.348846+0000","last_unstale":"2024-04-25T01:26:03.716951+0000","last_undegraded":"2024-04-25T01:26:03.716951+0000","last_fullsized":"2024-04-25T01:26:03.716951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:21.982440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717840+0000","last_change":"2024-04-25T01:26:01.414517+0000","last_active":"2024-04-25T01:26:03.717840+0000","last_peered":"2024-04-25T01:26:03.717840+0000","last_clean":"2024-04-25T01:26:03.717840+0000","last_became_active":"2024-04-25T01:26:01.414243+0000","last_became_peered":"2024-04-25T01:26:01.414243+0000","last_unstale":"2024-04-25T01:26:03.717840+0000","last_undegraded":"2024-04-25T01:26:03.717840+0000","last_fullsized":"2024-04-25T01:26:03.717840+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:48.491461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415418+0000","last_change":"2024-04-25T01:26:02.415893+0000","last_active":"2024-04-25T01:26:03.415418+0000","last_peered":"2024-04-25T01:26:03.415418+0000","last_clean":"2024-04-25T01:26:03.415418+0000","last_became_active":"2024-04-25T01:26:02.415596+0000","last_became_peered":"2024-04-25T01:26:02.415596+0000","last_unstale":"2024-04-25T01:26:03.415418+0000","last_undegraded":"2024-04-25T01:26:03.415418+0000","last_fullsized":"2024-04-25T01:26:03.415418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507428+0000","last_change":"2024-04-25T01:25:53.373572+0000","last_active":"2024-04-25T01:26:03.507428+0000","last_peered":"2024-04-25T01:26:03.507428+0000","last_clean":"2024-04-25T01:26:03.507428+0000","last_became_active":"2024-04-25T01:25:53.373129+0000","last_became_peered":"2024-04-25T01:25:53.373129+0000","last_unstale":"2024-04-25T01:26:03.507428+0000","last_undegraded":"2024-04-25T01:26:03.507428+0000","last_fullsized":"2024-04-25T01:26:03.507428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":28,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717081+0000","last_change":"2024-04-25T01:25:52.350446+0000","last_active":"2024-04-25T01:26:03.717081+0000","last_peered":"2024-04-25T01:26:03.717081+0000","last_clean":"2024-04-25T01:26:03.717081+0000","last_became_active":"2024-04-25T01:25:52.349448+0000","last_became_peered":"2024-04-25T01:25:52.349448+0000","last_unstale":"2024-04-25T01:26:03.717081+0000","last_undegraded":"2024-04-25T01:26:03.717081+0000","last_fullsized":"2024-04-25T01:26:03.717081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716463+0000","last_change":"2024-04-25T01:26:01.418794+0000","last_active":"2024-04-25T01:26:03.716463+0000","last_peered":"2024-04-25T01:26:03.716463+0000","last_clean":"2024-04-25T01:26:03.716463+0000","last_became_active":"2024-04-25T01:26:01.418541+0000","last_became_peered":"2024-04-25T01:26:01.418541+0000","last_unstale":"2024-04-25T01:26:03.716463+0000","last_undegraded":"2024-04-25T01:26:03.716463+0000","last_fullsized":"2024-04-25T01:26:03.716463+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717164+0000","last_change":"2024-04-25T01:26:02.425391+0000","last_active":"2024-04-25T01:26:03.717164+0000","last_peered":"2024-04-25T01:26:03.717164+0000","last_clean":"2024-04-25T01:26:03.717164+0000","last_became_active":"2024-04-25T01:26:02.425106+0000","last_became_peered":"2024-04-25T01:26:02.425106+0000","last_unstale":"2024-04-25T01:26:03.717164+0000","last_undegraded":"2024-04-25T01:26:03.717164+0000","last_fullsized":"2024-04-25T01:26:03.717164+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416706+0000","last_change":"2024-04-25T01:25:53.366416+0000","last_active":"2024-04-25T01:26:03.416706+0000","last_peered":"2024-04-25T01:26:03.416706+0000","last_clean":"2024-04-25T01:26:03.416706+0000","last_became_active":"2024-04-25T01:25:53.366157+0000","last_became_peered":"2024-04-25T01:25:53.366157+0000","last_unstale":"2024-04-25T01:26:03.416706+0000","last_undegraded":"2024-04-25T01:26:03.416706+0000","last_fullsized":"2024-04-25T01:26:03.416706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:26.277566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717854+0000","last_change":"2024-04-25T01:25:52.357610+0000","last_active":"2024-04-25T01:26:03.717854+0000","last_peered":"2024-04-25T01:26:03.717854+0000","last_clean":"2024-04-25T01:26:03.717854+0000","last_became_active":"2024-04-25T01:25:52.357355+0000","last_became_peered":"2024-04-25T01:25:52.357355+0000","last_unstale":"2024-04-25T01:26:03.717854+0000","last_undegraded":"2024-04-25T01:26:03.717854+0000","last_fullsized":"2024-04-25T01:26:03.717854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416422+0000","last_change":"2024-04-25T01:26:01.412666+0000","last_active":"2024-04-25T01:26:03.416422+0000","last_peered":"2024-04-25T01:26:03.416422+0000","last_clean":"2024-04-25T01:26:03.416422+0000","last_became_active":"2024-04-25T01:26:01.412132+0000","last_became_peered":"2024-04-25T01:26:01.412132+0000","last_unstale":"2024-04-25T01:26:03.416422+0000","last_undegraded":"2024-04-25T01:26:03.416422+0000","last_fullsized":"2024-04-25T01:26:03.416422+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716704+0000","last_change":"2024-04-25T01:26:02.413106+0000","last_active":"2024-04-25T01:26:03.716704+0000","last_peered":"2024-04-25T01:26:03.716704+0000","last_clean":"2024-04-25T01:26:03.716704+0000","last_became_active":"2024-04-25T01:26:02.412759+0000","last_became_peered":"2024-04-25T01:26:02.412759+0000","last_unstale":"2024-04-25T01:26:03.716704+0000","last_undegraded":"2024-04-25T01:26:03.716704+0000","last_fullsized":"2024-04-25T01:26:03.716704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:28.049190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716160+0000","last_change":"2024-04-25T01:25:53.374311+0000","last_active":"2024-04-25T01:26:03.716160+0000","last_peered":"2024-04-25T01:26:03.716160+0000","last_clean":"2024-04-25T01:26:03.716160+0000","last_became_active":"2024-04-25T01:25:53.373890+0000","last_became_peered":"2024-04-25T01:25:53.373890+0000","last_unstale":"2024-04-25T01:26:03.716160+0000","last_undegraded":"2024-04-25T01:26:03.716160+0000","last_fullsized":"2024-04-25T01:26:03.716160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717907+0000","last_change":"2024-04-25T01:25:52.356231+0000","last_active":"2024-04-25T01:26:03.717907+0000","last_peered":"2024-04-25T01:26:03.717907+0000","last_clean":"2024-04-25T01:26:03.717907+0000","last_became_active":"2024-04-25T01:25:52.355913+0000","last_became_peered":"2024-04-25T01:25:52.355913+0000","last_unstale":"2024-04-25T01:26:03.717907+0000","last_undegraded":"2024-04-25T01:26:03.717907+0000","last_fullsized":"2024-04-25T01:26:03.717907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415065+0000","last_change":"2024-04-25T01:26:01.419513+0000","last_active":"2024-04-25T01:26:03.415065+0000","last_peered":"2024-04-25T01:26:03.415065+0000","last_clean":"2024-04-25T01:26:03.415065+0000","last_became_active":"2024-04-25T01:26:01.419015+0000","last_became_peered":"2024-04-25T01:26:01.419015+0000","last_unstale":"2024-04-25T01:26:03.415065+0000","last_undegraded":"2024-04-25T01:26:03.415065+0000","last_fullsized":"2024-04-25T01:26:03.415065+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:10.486367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507391+0000","last_change":"2024-04-25T01:26:02.425241+0000","last_active":"2024-04-25T01:26:03.507391+0000","last_peered":"2024-04-25T01:26:03.507391+0000","last_clean":"2024-04-25T01:26:03.507391+0000","last_became_active":"2024-04-25T01:26:02.424960+0000","last_became_peered":"2024-04-25T01:26:02.424960+0000","last_unstale":"2024-04-25T01:26:03.507391+0000","last_undegraded":"2024-04-25T01:26:03.507391+0000","last_fullsized":"2024-04-25T01:26:03.507391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717765+0000","last_change":"2024-04-25T01:25:53.371445+0000","last_active":"2024-04-25T01:26:03.717765+0000","last_peered":"2024-04-25T01:26:03.717765+0000","last_clean":"2024-04-25T01:26:03.717765+0000","last_became_active":"2024-04-25T01:25:53.371086+0000","last_became_peered":"2024-04-25T01:25:53.371086+0000","last_unstale":"2024-04-25T01:26:03.717765+0000","last_undegraded":"2024-04-25T01:26:03.717765+0000","last_fullsized":"2024-04-25T01:26:03.717765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.718056+0000","last_change":"2024-04-25T01:25:52.357459+0000","last_active":"2024-04-25T01:26:03.718056+0000","last_peered":"2024-04-25T01:26:03.718056+0000","last_clean":"2024-04-25T01:26:03.718056+0000","last_became_active":"2024-04-25T01:25:52.357182+0000","last_became_peered":"2024-04-25T01:25:52.357182+0000","last_unstale":"2024-04-25T01:26:03.718056+0000","last_undegraded":"2024-04-25T01:26:03.718056+0000","last_fullsized":"2024-04-25T01:26:03.718056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415347+0000","last_change":"2024-04-25T01:26:01.415321+0000","last_active":"2024-04-25T01:26:03.415347+0000","last_peered":"2024-04-25T01:26:03.415347+0000","last_clean":"2024-04-25T01:26:03.415347+0000","last_became_active":"2024-04-25T01:26:01.415020+0000","last_became_peered":"2024-04-25T01:26:01.415020+0000","last_unstale":"2024-04-25T01:26:03.415347+0000","last_undegraded":"2024-04-25T01:26:03.415347+0000","last_fullsized":"2024-04-25T01:26:03.415347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:20.090411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717822+0000","last_change":"2024-04-25T01:26:02.416917+0000","last_active":"2024-04-25T01:26:03.717822+0000","last_peered":"2024-04-25T01:26:03.717822+0000","last_clean":"2024-04-25T01:26:03.717822+0000","last_became_active":"2024-04-25T01:26:02.416623+0000","last_became_peered":"2024-04-25T01:26:02.416623+0000","last_unstale":"2024-04-25T01:26:03.717822+0000","last_undegraded":"2024-04-25T01:26:03.717822+0000","last_fullsized":"2024-04-25T01:26:03.717822+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:55.040946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716446+0000","last_change":"2024-04-25T01:26:01.415472+0000","last_active":"2024-04-25T01:26:03.716446+0000","last_peered":"2024-04-25T01:26:03.716446+0000","last_clean":"2024-04-25T01:26:03.716446+0000","last_became_active":"2024-04-25T01:26:01.415184+0000","last_became_peered":"2024-04-25T01:26:01.415184+0000","last_unstale":"2024-04-25T01:26:03.716446+0000","last_undegraded":"2024-04-25T01:26:03.716446+0000","last_fullsized":"2024-04-25T01:26:03.716446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716757+0000","last_change":"2024-04-25T01:25:53.363043+0000","last_active":"2024-04-25T01:26:03.716757+0000","last_peered":"2024-04-25T01:26:03.716757+0000","last_clean":"2024-04-25T01:26:03.716757+0000","last_became_active":"2024-04-25T01:25:53.362601+0000","last_became_peered":"2024-04-25T01:25:53.362601+0000","last_unstale":"2024-04-25T01:26:03.716757+0000","last_undegraded":"2024-04-25T01:26:03.716757+0000","last_fullsized":"2024-04-25T01:26:03.716757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716830+0000","last_change":"2024-04-25T01:25:52.358081+0000","last_active":"2024-04-25T01:26:03.716830+0000","last_peered":"2024-04-25T01:26:03.716830+0000","last_clean":"2024-04-25T01:26:03.716830+0000","last_became_active":"2024-04-25T01:25:52.357365+0000","last_became_peered":"2024-04-25T01:25:52.357365+0000","last_unstale":"2024-04-25T01:26:03.716830+0000","last_undegraded":"2024-04-25T01:26:03.716830+0000","last_fullsized":"2024-04-25T01:26:03.716830+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716855+0000","last_change":"2024-04-25T01:26:02.418872+0000","last_active":"2024-04-25T01:26:03.716855+0000","last_peered":"2024-04-25T01:26:03.716855+0000","last_clean":"2024-04-25T01:26:03.716855+0000","last_became_active":"2024-04-25T01:26:02.418614+0000","last_became_peered":"2024-04-25T01:26:02.418614+0000","last_unstale":"2024-04-25T01:26:03.716855+0000","last_undegraded":"2024-04-25T01:26:03.716855+0000","last_fullsized":"2024-04-25T01:26:03.716855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:49.498275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717818+0000","last_change":"2024-04-25T01:26:01.410258+0000","last_active":"2024-04-25T01:26:03.717818+0000","last_peered":"2024-04-25T01:26:03.717818+0000","last_clean":"2024-04-25T01:26:03.717818+0000","last_became_active":"2024-04-25T01:26:01.410004+0000","last_became_peered":"2024-04-25T01:26:01.410004+0000","last_unstale":"2024-04-25T01:26:03.717818+0000","last_undegraded":"2024-04-25T01:26:03.717818+0000","last_fullsized":"2024-04-25T01:26:03.717818+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:48.865940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415284+0000","last_change":"2024-04-25T01:25:53.363794+0000","last_active":"2024-04-25T01:26:03.415284+0000","last_peered":"2024-04-25T01:26:03.415284+0000","last_clean":"2024-04-25T01:26:03.415284+0000","last_became_active":"2024-04-25T01:25:53.363509+0000","last_became_peered":"2024-04-25T01:25:53.363509+0000","last_unstale":"2024-04-25T01:26:03.415284+0000","last_undegraded":"2024-04-25T01:26:03.415284+0000","last_fullsized":"2024-04-25T01:26:03.415284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415685+0000","last_change":"2024-04-25T01:25:52.347464+0000","last_active":"2024-04-25T01:26:03.415685+0000","last_peered":"2024-04-25T01:26:03.415685+0000","last_clean":"2024-04-25T01:26:03.415685+0000","last_became_active":"2024-04-25T01:25:52.347138+0000","last_became_peered":"2024-04-25T01:25:52.347138+0000","last_unstale":"2024-04-25T01:26:03.415685+0000","last_undegraded":"2024-04-25T01:26:03.415685+0000","last_fullsized":"2024-04-25T01:26:03.415685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717780+0000","last_change":"2024-04-25T01:26:02.425948+0000","last_active":"2024-04-25T01:26:03.717780+0000","last_peered":"2024-04-25T01:26:03.717780+0000","last_clean":"2024-04-25T01:26:03.717780+0000","last_became_active":"2024-04-25T01:26:02.424947+0000","last_became_peered":"2024-04-25T01:26:02.424947+0000","last_unstale":"2024-04-25T01:26:03.717780+0000","last_undegraded":"2024-04-25T01:26:03.717780+0000","last_fullsized":"2024-04-25T01:26:03.717780+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716700+0000","last_change":"2024-04-25T01:25:52.349989+0000","last_active":"2024-04-25T01:26:03.716700+0000","last_peered":"2024-04-25T01:26:03.716700+0000","last_clean":"2024-04-25T01:26:03.716700+0000","last_became_active":"2024-04-25T01:25:52.349672+0000","last_became_peered":"2024-04-25T01:25:52.349672+0000","last_unstale":"2024-04-25T01:26:03.716700+0000","last_undegraded":"2024-04-25T01:26:03.716700+0000","last_fullsized":"2024-04-25T01:26:03.716700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716805+0000","last_change":"2024-04-25T01:25:53.356610+0000","last_active":"2024-04-25T01:26:03.716805+0000","last_peered":"2024-04-25T01:26:03.716805+0000","last_clean":"2024-04-25T01:26:03.716805+0000","last_became_active":"2024-04-25T01:25:53.356015+0000","last_became_peered":"2024-04-25T01:25:53.356015+0000","last_unstale":"2024-04-25T01:26:03.716805+0000","last_undegraded":"2024-04-25T01:26:03.716805+0000","last_fullsized":"2024-04-25T01:26:03.716805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.417192+0000","last_change":"2024-04-25T01:26:01.420261+0000","last_active":"2024-04-25T01:26:03.417192+0000","last_peered":"2024-04-25T01:26:03.417192+0000","last_clean":"2024-04-25T01:26:03.417192+0000","last_became_active":"2024-04-25T01:26:01.420003+0000","last_became_peered":"2024-04-25T01:26:01.420003+0000","last_unstale":"2024-04-25T01:26:03.417192+0000","last_undegraded":"2024-04-25T01:26:03.417192+0000","last_fullsized":"2024-04-25T01:26:03.417192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:28.695853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416039+0000","last_change":"2024-04-25T01:26:02.426327+0000","last_active":"2024-04-25T01:26:03.416039+0000","last_peered":"2024-04-25T01:26:03.416039+0000","last_clean":"2024-04-25T01:26:03.416039+0000","last_became_active":"2024-04-25T01:26:02.426054+0000","last_became_peered":"2024-04-25T01:26:02.426054+0000","last_unstale":"2024-04-25T01:26:03.416039+0000","last_undegraded":"2024-04-25T01:26:03.416039+0000","last_fullsized":"2024-04-25T01:26:03.416039+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:17.809738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415686+0000","last_change":"2024-04-25T01:25:52.353955+0000","last_active":"2024-04-25T01:26:03.415686+0000","last_peered":"2024-04-25T01:26:03.415686+0000","last_clean":"2024-04-25T01:26:03.415686+0000","last_became_active":"2024-04-25T01:25:52.353659+0000","last_became_peered":"2024-04-25T01:25:52.353659+0000","last_unstale":"2024-04-25T01:26:03.415686+0000","last_undegraded":"2024-04-25T01:26:03.415686+0000","last_fullsized":"2024-04-25T01:26:03.415686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416142+0000","last_change":"2024-04-25T01:25:53.366586+0000","last_active":"2024-04-25T01:26:03.416142+0000","last_peered":"2024-04-25T01:26:03.416142+0000","last_clean":"2024-04-25T01:26:03.416142+0000","last_became_active":"2024-04-25T01:25:53.366302+0000","last_became_peered":"2024-04-25T01:25:53.366302+0000","last_unstale":"2024-04-25T01:26:03.416142+0000","last_undegraded":"2024-04-25T01:26:03.416142+0000","last_fullsized":"2024-04-25T01:26:03.416142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416415+0000","last_change":"2024-04-25T01:26:01.420960+0000","last_active":"2024-04-25T01:26:03.416415+0000","last_peered":"2024-04-25T01:26:03.416415+0000","last_clean":"2024-04-25T01:26:03.416415+0000","last_became_active":"2024-04-25T01:26:01.420713+0000","last_became_peered":"2024-04-25T01:26:01.420713+0000","last_unstale":"2024-04-25T01:26:03.416415+0000","last_undegraded":"2024-04-25T01:26:03.416415+0000","last_fullsized":"2024-04-25T01:26:03.416415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:29.489275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416652+0000","last_change":"2024-04-25T01:26:02.419282+0000","last_active":"2024-04-25T01:26:03.416652+0000","last_peered":"2024-04-25T01:26:03.416652+0000","last_clean":"2024-04-25T01:26:03.416652+0000","last_became_active":"2024-04-25T01:26:02.418802+0000","last_became_peered":"2024-04-25T01:26:02.418802+0000","last_unstale":"2024-04-25T01:26:03.416652+0000","last_undegraded":"2024-04-25T01:26:03.416652+0000","last_fullsized":"2024-04-25T01:26:03.416652+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:46.807175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716343+0000","last_change":"2024-04-25T01:25:53.367143+0000","last_active":"2024-04-25T01:26:03.716343+0000","last_peered":"2024-04-25T01:26:03.716343+0000","last_clean":"2024-04-25T01:26:03.716343+0000","last_became_active":"2024-04-25T01:25:53.366847+0000","last_became_peered":"2024-04-25T01:25:53.366847+0000","last_unstale":"2024-04-25T01:26:03.716343+0000","last_undegraded":"2024-04-25T01:26:03.716343+0000","last_fullsized":"2024-04-25T01:26:03.716343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716783+0000","last_change":"2024-04-25T01:26:02.416438+0000","last_active":"2024-04-25T01:26:03.716783+0000","last_peered":"2024-04-25T01:26:03.716783+0000","last_clean":"2024-04-25T01:26:03.716783+0000","last_became_active":"2024-04-25T01:26:02.416170+0000","last_became_peered":"2024-04-25T01:26:02.416170+0000","last_unstale":"2024-04-25T01:26:03.716783+0000","last_undegraded":"2024-04-25T01:26:03.716783+0000","last_fullsized":"2024-04-25T01:26:03.716783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415637+0000","last_change":"2024-04-25T01:25:53.371345+0000","last_active":"2024-04-25T01:26:03.415637+0000","last_peered":"2024-04-25T01:26:03.415637+0000","last_clean":"2024-04-25T01:26:03.415637+0000","last_became_active":"2024-04-25T01:25:53.371080+0000","last_became_peered":"2024-04-25T01:25:53.371080+0000","last_unstale":"2024-04-25T01:26:03.415637+0000","last_undegraded":"2024-04-25T01:26:03.415637+0000","last_fullsized":"2024-04-25T01:26:03.415637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:09.855763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717353+0000","last_change":"2024-04-25T01:26:02.419688+0000","last_active":"2024-04-25T01:26:03.717353+0000","last_peered":"2024-04-25T01:26:03.717353+0000","last_clean":"2024-04-25T01:26:03.717353+0000","last_became_active":"2024-04-25T01:26:02.419417+0000","last_became_peered":"2024-04-25T01:26:02.419417+0000","last_unstale":"2024-04-25T01:26:03.717353+0000","last_undegraded":"2024-04-25T01:26:03.717353+0000","last_fullsized":"2024-04-25T01:26:03.717353+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717442+0000","last_change":"2024-04-25T01:25:53.369850+0000","last_active":"2024-04-25T01:26:03.717442+0000","last_peered":"2024-04-25T01:26:03.717442+0000","last_clean":"2024-04-25T01:26:03.717442+0000","last_became_active":"2024-04-25T01:25:53.369350+0000","last_became_peered":"2024-04-25T01:25:53.369350+0000","last_unstale":"2024-04-25T01:26:03.717442+0000","last_undegraded":"2024-04-25T01:26:03.717442+0000","last_fullsized":"2024-04-25T01:26:03.717442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717742+0000","last_change":"2024-04-25T01:26:02.425528+0000","last_active":"2024-04-25T01:26:03.717742+0000","last_peered":"2024-04-25T01:26:03.717742+0000","last_clean":"2024-04-25T01:26:03.717742+0000","last_became_active":"2024-04-25T01:26:02.425160+0000","last_became_peered":"2024-04-25T01:26:02.425160+0000","last_unstale":"2024-04-25T01:26:03.717742+0000","last_undegraded":"2024-04-25T01:26:03.717742+0000","last_fullsized":"2024-04-25T01:26:03.717742+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716710+0000","last_change":"2024-04-25T01:25:53.357142+0000","last_active":"2024-04-25T01:26:03.716710+0000","last_peered":"2024-04-25T01:26:03.716710+0000","last_clean":"2024-04-25T01:26:03.716710+0000","last_became_active":"2024-04-25T01:25:53.356854+0000","last_became_peered":"2024-04-25T01:25:53.356854+0000","last_unstale":"2024-04-25T01:26:03.716710+0000","last_undegraded":"2024-04-25T01:26:03.716710+0000","last_fullsized":"2024-04-25T01:26:03.716710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507419+0000","last_change":"2024-04-25T01:26:02.417492+0000","last_active":"2024-04-25T01:26:03.507419+0000","last_peered":"2024-04-25T01:26:03.507419+0000","last_clean":"2024-04-25T01:26:03.507419+0000","last_became_active":"2024-04-25T01:26:02.417234+0000","last_became_peered":"2024-04-25T01:26:02.417234+0000","last_unstale":"2024-04-25T01:26:03.507419+0000","last_undegraded":"2024-04-25T01:26:03.507419+0000","last_fullsized":"2024-04-25T01:26:03.507419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416287+0000","last_change":"2024-04-25T01:25:53.364575+0000","last_active":"2024-04-25T01:26:03.416287+0000","last_peered":"2024-04-25T01:26:03.416287+0000","last_clean":"2024-04-25T01:26:03.416287+0000","last_became_active":"2024-04-25T01:25:53.364311+0000","last_became_peered":"2024-04-25T01:25:53.364311+0000","last_unstale":"2024-04-25T01:26:03.416287+0000","last_undegraded":"2024-04-25T01:26:03.416287+0000","last_fullsized":"2024-04-25T01:26:03.416287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415942+0000","last_change":"2024-04-25T01:26:02.426813+0000","last_active":"2024-04-25T01:26:03.415942+0000","last_peered":"2024-04-25T01:26:03.415942+0000","last_clean":"2024-04-25T01:26:03.415942+0000","last_became_active":"2024-04-25T01:26:02.426552+0000","last_became_peered":"2024-04-25T01:26:02.426552+0000","last_unstale":"2024-04-25T01:26:03.415942+0000","last_undegraded":"2024-04-25T01:26:03.415942+0000","last_fullsized":"2024-04-25T01:26:03.415942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:03.687174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717821+0000","last_change":"2024-04-25T01:25:53.368413+0000","last_active":"2024-04-25T01:26:03.717821+0000","last_peered":"2024-04-25T01:26:03.717821+0000","last_clean":"2024-04-25T01:26:03.717821+0000","last_became_active":"2024-04-25T01:25:53.368176+0000","last_became_peered":"2024-04-25T01:25:53.368176+0000","last_unstale":"2024-04-25T01:26:03.717821+0000","last_undegraded":"2024-04-25T01:26:03.717821+0000","last_fullsized":"2024-04-25T01:26:03.717821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.506869+0000","last_change":"2024-04-25T01:26:02.425185+0000","last_active":"2024-04-25T01:26:03.506869+0000","last_peered":"2024-04-25T01:26:03.506869+0000","last_clean":"2024-04-25T01:26:03.506869+0000","last_became_active":"2024-04-25T01:26:02.424903+0000","last_became_peered":"2024-04-25T01:26:02.424903+0000","last_unstale":"2024-04-25T01:26:03.506869+0000","last_undegraded":"2024-04-25T01:26:03.506869+0000","last_fullsized":"2024-04-25T01:26:03.506869+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717561+0000","last_change":"2024-04-25T01:25:53.372638+0000","last_active":"2024-04-25T01:26:03.717561+0000","last_peered":"2024-04-25T01:26:03.717561+0000","last_clean":"2024-04-25T01:26:03.717561+0000","last_became_active":"2024-04-25T01:25:53.372001+0000","last_became_peered":"2024-04-25T01:25:53.372001+0000","last_unstale":"2024-04-25T01:26:03.717561+0000","last_undegraded":"2024-04-25T01:26:03.717561+0000","last_fullsized":"2024-04-25T01:26:03.717561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716617+0000","last_change":"2024-04-25T01:26:02.420867+0000","last_active":"2024-04-25T01:26:03.716617+0000","last_peered":"2024-04-25T01:26:03.716617+0000","last_clean":"2024-04-25T01:26:03.716617+0000","last_became_active":"2024-04-25T01:26:02.420621+0000","last_became_peered":"2024-04-25T01:26:02.420621+0000","last_unstale":"2024-04-25T01:26:03.716617+0000","last_undegraded":"2024-04-25T01:26:03.716617+0000","last_fullsized":"2024-04-25T01:26:03.716617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:11.493181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416565+0000","last_change":"2024-04-25T01:25:53.365509+0000","last_active":"2024-04-25T01:26:03.416565+0000","last_peered":"2024-04-25T01:26:03.416565+0000","last_clean":"2024-04-25T01:26:03.416565+0000","last_became_active":"2024-04-25T01:25:53.365245+0000","last_became_peered":"2024-04-25T01:25:53.365245+0000","last_unstale":"2024-04-25T01:26:03.416565+0000","last_undegraded":"2024-04-25T01:26:03.416565+0000","last_fullsized":"2024-04-25T01:26:03.416565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416549+0000","last_change":"2024-04-25T01:26:02.415672+0000","last_active":"2024-04-25T01:26:03.416549+0000","last_peered":"2024-04-25T01:26:03.416549+0000","last_clean":"2024-04-25T01:26:03.416549+0000","last_became_active":"2024-04-25T01:26:02.415389+0000","last_became_peered":"2024-04-25T01:26:02.415389+0000","last_unstale":"2024-04-25T01:26:03.416549+0000","last_undegraded":"2024-04-25T01:26:03.416549+0000","last_fullsized":"2024-04-25T01:26:03.416549+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:52.342053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716741+0000","last_change":"2024-04-25T01:25:53.365391+0000","last_active":"2024-04-25T01:26:03.716741+0000","last_peered":"2024-04-25T01:26:03.716741+0000","last_clean":"2024-04-25T01:26:03.716741+0000","last_became_active":"2024-04-25T01:25:53.364829+0000","last_became_peered":"2024-04-25T01:25:53.364829+0000","last_unstale":"2024-04-25T01:26:03.716741+0000","last_undegraded":"2024-04-25T01:26:03.716741+0000","last_fullsized":"2024-04-25T01:26:03.716741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507955+0000","last_change":"2024-04-25T01:26:02.418292+0000","last_active":"2024-04-25T01:26:03.507955+0000","last_peered":"2024-04-25T01:26:03.507955+0000","last_clean":"2024-04-25T01:26:03.507955+0000","last_became_active":"2024-04-25T01:26:02.418036+0000","last_became_peered":"2024-04-25T01:26:02.418036+0000","last_unstale":"2024-04-25T01:26:03.507955+0000","last_undegraded":"2024-04-25T01:26:03.507955+0000","last_fullsized":"2024-04-25T01:26:03.507955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415608+0000","last_change":"2024-04-25T01:25:53.355667+0000","last_active":"2024-04-25T01:26:03.415608+0000","last_peered":"2024-04-25T01:26:03.415608+0000","last_clean":"2024-04-25T01:26:03.415608+0000","last_became_active":"2024-04-25T01:25:53.355417+0000","last_became_peered":"2024-04-25T01:25:53.355417+0000","last_unstale":"2024-04-25T01:26:03.415608+0000","last_undegraded":"2024-04-25T01:26:03.415608+0000","last_fullsized":"2024-04-25T01:26:03.415608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415307+0000","last_change":"2024-04-25T01:26:02.414994+0000","last_active":"2024-04-25T01:26:03.415307+0000","last_peered":"2024-04-25T01:26:03.415307+0000","last_clean":"2024-04-25T01:26:03.415307+0000","last_became_active":"2024-04-25T01:26:02.414735+0000","last_became_peered":"2024-04-25T01:26:02.414735+0000","last_unstale":"2024-04-25T01:26:03.415307+0000","last_undegraded":"2024-04-25T01:26:03.415307+0000","last_fullsized":"2024-04-25T01:26:03.415307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:44.960251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717184+0000","last_change":"2024-04-25T01:25:53.371240+0000","last_active":"2024-04-25T01:26:03.717184+0000","last_peered":"2024-04-25T01:26:03.717184+0000","last_clean":"2024-04-25T01:26:03.717184+0000","last_became_active":"2024-04-25T01:25:53.370164+0000","last_became_peered":"2024-04-25T01:25:53.370164+0000","last_unstale":"2024-04-25T01:26:03.717184+0000","last_undegraded":"2024-04-25T01:26:03.717184+0000","last_fullsized":"2024-04-25T01:26:03.717184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416367+0000","last_change":"2024-04-25T01:26:02.418272+0000","last_active":"2024-04-25T01:26:03.416367+0000","last_peered":"2024-04-25T01:26:03.416367+0000","last_clean":"2024-04-25T01:26:03.416367+0000","last_became_active":"2024-04-25T01:26:02.417977+0000","last_became_peered":"2024-04-25T01:26:02.417977+0000","last_unstale":"2024-04-25T01:26:03.416367+0000","last_undegraded":"2024-04-25T01:26:03.416367+0000","last_fullsized":"2024-04-25T01:26:03.416367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716511+0000","last_change":"2024-04-25T01:25:53.376415+0000","last_active":"2024-04-25T01:26:03.716511+0000","last_peered":"2024-04-25T01:26:03.716511+0000","last_clean":"2024-04-25T01:26:03.716511+0000","last_became_active":"2024-04-25T01:25:53.376104+0000","last_became_peered":"2024-04-25T01:25:53.376104+0000","last_unstale":"2024-04-25T01:26:03.716511+0000","last_undegraded":"2024-04-25T01:26:03.716511+0000","last_fullsized":"2024-04-25T01:26:03.716511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416512+0000","last_change":"2024-04-25T01:26:02.426333+0000","last_active":"2024-04-25T01:26:03.416512+0000","last_peered":"2024-04-25T01:26:03.416512+0000","last_clean":"2024-04-25T01:26:03.416512+0000","last_became_active":"2024-04-25T01:26:02.426069+0000","last_became_peered":"2024-04-25T01:26:02.426069+0000","last_unstale":"2024-04-25T01:26:03.416512+0000","last_undegraded":"2024-04-25T01:26:03.416512+0000","last_fullsized":"2024-04-25T01:26:03.416512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716975+0000","last_change":"2024-04-25T01:25:53.370314+0000","last_active":"2024-04-25T01:26:03.716975+0000","last_peered":"2024-04-25T01:26:03.716975+0000","last_clean":"2024-04-25T01:26:03.716975+0000","last_became_active":"2024-04-25T01:25:53.370049+0000","last_became_peered":"2024-04-25T01:25:53.370049+0000","last_unstale":"2024-04-25T01:26:03.716975+0000","last_undegraded":"2024-04-25T01:26:03.716975+0000","last_fullsized":"2024-04-25T01:26:03.716975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415866+0000","last_change":"2024-04-25T01:26:02.415464+0000","last_active":"2024-04-25T01:26:03.415866+0000","last_peered":"2024-04-25T01:26:03.415866+0000","last_clean":"2024-04-25T01:26:03.415866+0000","last_became_active":"2024-04-25T01:26:02.415173+0000","last_became_peered":"2024-04-25T01:26:02.415173+0000","last_unstale":"2024-04-25T01:26:03.415866+0000","last_undegraded":"2024-04-25T01:26:03.415866+0000","last_fullsized":"2024-04-25T01:26:03.415866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.343169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507691+0000","last_change":"2024-04-25T01:25:53.369585+0000","last_active":"2024-04-25T01:26:03.507691+0000","last_peered":"2024-04-25T01:26:03.507691+0000","last_clean":"2024-04-25T01:26:03.507691+0000","last_became_active":"2024-04-25T01:25:53.368499+0000","last_became_peered":"2024-04-25T01:25:53.368499+0000","last_unstale":"2024-04-25T01:26:03.507691+0000","last_undegraded":"2024-04-25T01:26:03.507691+0000","last_fullsized":"2024-04-25T01:26:03.507691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716525+0000","last_change":"2024-04-25T01:26:02.421975+0000","last_active":"2024-04-25T01:26:03.716525+0000","last_peered":"2024-04-25T01:26:03.716525+0000","last_clean":"2024-04-25T01:26:03.716525+0000","last_became_active":"2024-04-25T01:26:02.421698+0000","last_became_peered":"2024-04-25T01:26:02.421698+0000","last_unstale":"2024-04-25T01:26:03.716525+0000","last_undegraded":"2024-04-25T01:26:03.716525+0000","last_fullsized":"2024-04-25T01:26:03.716525+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416746+0000","last_change":"2024-04-25T01:25:53.370666+0000","last_active":"2024-04-25T01:26:03.416746+0000","last_peered":"2024-04-25T01:26:03.416746+0000","last_clean":"2024-04-25T01:26:03.416746+0000","last_became_active":"2024-04-25T01:25:53.369733+0000","last_became_peered":"2024-04-25T01:25:53.369733+0000","last_unstale":"2024-04-25T01:26:03.416746+0000","last_undegraded":"2024-04-25T01:26:03.416746+0000","last_fullsized":"2024-04-25T01:26:03.416746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.715962+0000","last_change":"2024-04-25T01:26:02.418114+0000","last_active":"2024-04-25T01:26:03.715962+0000","last_peered":"2024-04-25T01:26:03.715962+0000","last_clean":"2024-04-25T01:26:03.715962+0000","last_became_active":"2024-04-25T01:26:02.417810+0000","last_became_peered":"2024-04-25T01:26:02.417810+0000","last_unstale":"2024-04-25T01:26:03.715962+0000","last_undegraded":"2024-04-25T01:26:03.715962+0000","last_fullsized":"2024-04-25T01:26:03.715962+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:30.496089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.718049+0000","last_change":"2024-04-25T01:25:53.372048+0000","last_active":"2024-04-25T01:26:03.718049+0000","last_peered":"2024-04-25T01:26:03.718049+0000","last_clean":"2024-04-25T01:26:03.718049+0000","last_became_active":"2024-04-25T01:25:53.371767+0000","last_became_peered":"2024-04-25T01:25:53.371767+0000","last_unstale":"2024-04-25T01:26:03.718049+0000","last_undegraded":"2024-04-25T01:26:03.718049+0000","last_fullsized":"2024-04-25T01:26:03.718049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:05.589744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716672+0000","last_change":"2024-04-25T01:26:02.426289+0000","last_active":"2024-04-25T01:26:03.716672+0000","last_peered":"2024-04-25T01:26:03.716672+0000","last_clean":"2024-04-25T01:26:03.716672+0000","last_became_active":"2024-04-25T01:26:02.426015+0000","last_became_peered":"2024-04-25T01:26:02.426015+0000","last_unstale":"2024-04-25T01:26:03.716672+0000","last_undegraded":"2024-04-25T01:26:03.716672+0000","last_fullsized":"2024-04-25T01:26:03.716672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716678+0000","last_change":"2024-04-25T01:25:53.374186+0000","last_active":"2024-04-25T01:26:03.716678+0000","last_peered":"2024-04-25T01:26:03.716678+0000","last_clean":"2024-04-25T01:26:03.716678+0000","last_became_active":"2024-04-25T01:25:53.373766+0000","last_became_peered":"2024-04-25T01:25:53.373766+0000","last_unstale":"2024-04-25T01:26:03.716678+0000","last_undegraded":"2024-04-25T01:26:03.716678+0000","last_fullsized":"2024-04-25T01:26:03.716678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416320+0000","last_change":"2024-04-25T01:26:02.416345+0000","last_active":"2024-04-25T01:26:03.416320+0000","last_peered":"2024-04-25T01:26:03.416320+0000","last_clean":"2024-04-25T01:26:03.416320+0000","last_became_active":"2024-04-25T01:26:02.416082+0000","last_became_peered":"2024-04-25T01:26:02.416082+0000","last_unstale":"2024-04-25T01:26:03.416320+0000","last_undegraded":"2024-04-25T01:26:03.416320+0000","last_fullsized":"2024-04-25T01:26:03.416320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:47.810943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716847+0000","last_change":"2024-04-25T01:25:53.371602+0000","last_active":"2024-04-25T01:26:03.716847+0000","last_peered":"2024-04-25T01:26:03.716847+0000","last_clean":"2024-04-25T01:26:03.716847+0000","last_became_active":"2024-04-25T01:25:53.371362+0000","last_became_peered":"2024-04-25T01:25:53.371362+0000","last_unstale":"2024-04-25T01:26:03.716847+0000","last_undegraded":"2024-04-25T01:26:03.716847+0000","last_fullsized":"2024-04-25T01:26:03.716847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717204+0000","last_change":"2024-04-25T01:26:02.425888+0000","last_active":"2024-04-25T01:26:03.717204+0000","last_peered":"2024-04-25T01:26:03.717204+0000","last_clean":"2024-04-25T01:26:03.717204+0000","last_became_active":"2024-04-25T01:26:02.425577+0000","last_became_peered":"2024-04-25T01:26:02.425577+0000","last_unstale":"2024-04-25T01:26:03.717204+0000","last_undegraded":"2024-04-25T01:26:03.717204+0000","last_fullsized":"2024-04-25T01:26:03.717204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.715963+0000","last_change":"2024-04-25T01:25:53.370996+0000","last_active":"2024-04-25T01:26:03.715963+0000","last_peered":"2024-04-25T01:26:03.715963+0000","last_clean":"2024-04-25T01:26:03.715963+0000","last_became_active":"2024-04-25T01:25:53.370479+0000","last_became_peered":"2024-04-25T01:25:53.370479+0000","last_unstale":"2024-04-25T01:26:03.715963+0000","last_undegraded":"2024-04-25T01:26:03.715963+0000","last_fullsized":"2024-04-25T01:26:03.715963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:57.636067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507529+0000","last_change":"2024-04-25T01:26:02.423420+0000","last_active":"2024-04-25T01:26:03.507529+0000","last_peered":"2024-04-25T01:26:03.507529+0000","last_clean":"2024-04-25T01:26:03.507529+0000","last_became_active":"2024-04-25T01:26:02.423146+0000","last_became_peered":"2024-04-25T01:26:02.423146+0000","last_unstale":"2024-04-25T01:26:03.507529+0000","last_undegraded":"2024-04-25T01:26:03.507529+0000","last_fullsized":"2024-04-25T01:26:03.507529+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415469+0000","last_change":"2024-04-25T01:25:53.363512+0000","last_active":"2024-04-25T01:26:03.415469+0000","last_peered":"2024-04-25T01:26:03.415469+0000","last_clean":"2024-04-25T01:26:03.415469+0000","last_became_active":"2024-04-25T01:25:53.363246+0000","last_became_peered":"2024-04-25T01:25:53.363246+0000","last_unstale":"2024-04-25T01:26:03.415469+0000","last_undegraded":"2024-04-25T01:26:03.415469+0000","last_fullsized":"2024-04-25T01:26:03.415469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415970+0000","last_change":"2024-04-25T01:26:02.413346+0000","last_active":"2024-04-25T01:26:03.415970+0000","last_peered":"2024-04-25T01:26:03.415970+0000","last_clean":"2024-04-25T01:26:03.415970+0000","last_became_active":"2024-04-25T01:26:02.413036+0000","last_became_peered":"2024-04-25T01:26:02.413036+0000","last_unstale":"2024-04-25T01:26:03.415970+0000","last_undegraded":"2024-04-25T01:26:03.415970+0000","last_fullsized":"2024-04-25T01:26:03.415970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416642+0000","last_change":"2024-04-25T01:25:53.364457+0000","last_active":"2024-04-25T01:26:03.416642+0000","last_peered":"2024-04-25T01:26:03.416642+0000","last_clean":"2024-04-25T01:26:03.416642+0000","last_became_active":"2024-04-25T01:25:53.364103+0000","last_became_peered":"2024-04-25T01:25:53.364103+0000","last_unstale":"2024-04-25T01:26:03.416642+0000","last_undegraded":"2024-04-25T01:26:03.416642+0000","last_fullsized":"2024-04-25T01:26:03.416642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:11.336752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415365+0000","last_change":"2024-04-25T01:26:02.418792+0000","last_active":"2024-04-25T01:26:03.415365+0000","last_peered":"2024-04-25T01:26:03.415365+0000","last_clean":"2024-04-25T01:26:03.415365+0000","last_became_active":"2024-04-25T01:26:02.416942+0000","last_became_peered":"2024-04-25T01:26:02.416942+0000","last_unstale":"2024-04-25T01:26:03.415365+0000","last_undegraded":"2024-04-25T01:26:03.415365+0000","last_fullsized":"2024-04-25T01:26:03.415365+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:59.492678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717288+0000","last_change":"2024-04-25T01:25:53.371647+0000","last_active":"2024-04-25T01:26:03.717288+0000","last_peered":"2024-04-25T01:26:03.717288+0000","last_clean":"2024-04-25T01:26:03.717288+0000","last_became_active":"2024-04-25T01:25:53.371353+0000","last_became_peered":"2024-04-25T01:25:53.371353+0000","last_unstale":"2024-04-25T01:26:03.717288+0000","last_undegraded":"2024-04-25T01:26:03.717288+0000","last_fullsized":"2024-04-25T01:26:03.717288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507578+0000","last_change":"2024-04-25T01:26:02.426335+0000","last_active":"2024-04-25T01:26:03.507578+0000","last_peered":"2024-04-25T01:26:03.507578+0000","last_clean":"2024-04-25T01:26:03.507578+0000","last_became_active":"2024-04-25T01:26:02.426024+0000","last_became_peered":"2024-04-25T01:26:02.426024+0000","last_unstale":"2024-04-25T01:26:03.507578+0000","last_undegraded":"2024-04-25T01:26:03.507578+0000","last_fullsized":"2024-04-25T01:26:03.507578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717460+0000","last_change":"2024-04-25T01:25:53.372842+0000","last_active":"2024-04-25T01:26:03.717460+0000","last_peered":"2024-04-25T01:26:03.717460+0000","last_clean":"2024-04-25T01:26:03.717460+0000","last_became_active":"2024-04-25T01:25:53.372580+0000","last_became_peered":"2024-04-25T01:25:53.372580+0000","last_unstale":"2024-04-25T01:26:03.717460+0000","last_undegraded":"2024-04-25T01:26:03.717460+0000","last_fullsized":"2024-04-25T01:26:03.717460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716748+0000","last_change":"2024-04-25T01:26:02.414632+0000","last_active":"2024-04-25T01:26:03.716748+0000","last_peered":"2024-04-25T01:26:03.716748+0000","last_clean":"2024-04-25T01:26:03.716748+0000","last_became_active":"2024-04-25T01:26:02.414388+0000","last_became_peered":"2024-04-25T01:26:02.414388+0000","last_unstale":"2024-04-25T01:26:03.716748+0000","last_undegraded":"2024-04-25T01:26:03.716748+0000","last_fullsized":"2024-04-25T01:26:03.716748+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.508086+0000","last_change":"2024-04-25T01:25:53.376081+0000","last_active":"2024-04-25T01:26:03.508086+0000","last_peered":"2024-04-25T01:26:03.508086+0000","last_clean":"2024-04-25T01:26:03.508086+0000","last_became_active":"2024-04-25T01:25:53.375814+0000","last_became_peered":"2024-04-25T01:25:53.375814+0000","last_unstale":"2024-04-25T01:26:03.508086+0000","last_undegraded":"2024-04-25T01:26:03.508086+0000","last_fullsized":"2024-04-25T01:26:03.508086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:29.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416509+0000","last_change":"2024-04-25T01:26:02.414366+0000","last_active":"2024-04-25T01:26:03.416509+0000","last_peered":"2024-04-25T01:26:03.416509+0000","last_clean":"2024-04-25T01:26:03.416509+0000","last_became_active":"2024-04-25T01:26:02.414061+0000","last_became_peered":"2024-04-25T01:26:02.414061+0000","last_unstale":"2024-04-25T01:26:03.416509+0000","last_undegraded":"2024-04-25T01:26:03.416509+0000","last_fullsized":"2024-04-25T01:26:03.416509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:19.644468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415110+0000","last_change":"2024-04-25T01:25:53.374061+0000","last_active":"2024-04-25T01:26:03.415110+0000","last_peered":"2024-04-25T01:26:03.415110+0000","last_clean":"2024-04-25T01:26:03.415110+0000","last_became_active":"2024-04-25T01:25:53.373848+0000","last_became_peered":"2024-04-25T01:25:53.373848+0000","last_unstale":"2024-04-25T01:26:03.415110+0000","last_undegraded":"2024-04-25T01:26:03.415110+0000","last_fullsized":"2024-04-25T01:26:03.415110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717762+0000","last_change":"2024-04-25T01:26:02.414620+0000","last_active":"2024-04-25T01:26:03.717762+0000","last_peered":"2024-04-25T01:26:03.717762+0000","last_clean":"2024-04-25T01:26:03.717762+0000","last_became_active":"2024-04-25T01:26:02.414326+0000","last_became_peered":"2024-04-25T01:26:02.414326+0000","last_unstale":"2024-04-25T01:26:03.717762+0000","last_undegraded":"2024-04-25T01:26:03.717762+0000","last_fullsized":"2024-04-25T01:26:03.717762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:30.496089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507095+0000","last_change":"2024-04-25T01:25:53.364698+0000","last_active":"2024-04-25T01:26:03.507095+0000","last_peered":"2024-04-25T01:26:03.507095+0000","last_clean":"2024-04-25T01:26:03.507095+0000","last_became_active":"2024-04-25T01:25:53.364435+0000","last_became_peered":"2024-04-25T01:25:53.364435+0000","last_unstale":"2024-04-25T01:26:03.507095+0000","last_undegraded":"2024-04-25T01:26:03.507095+0000","last_fullsized":"2024-04-25T01:26:03.507095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716509+0000","last_change":"2024-04-25T01:26:02.418331+0000","last_active":"2024-04-25T01:26:03.716509+0000","last_peered":"2024-04-25T01:26:03.716509+0000","last_clean":"2024-04-25T01:26:03.716509+0000","last_became_active":"2024-04-25T01:26:02.418057+0000","last_became_peered":"2024-04-25T01:26:02.418057+0000","last_unstale":"2024-04-25T01:26:03.716509+0000","last_undegraded":"2024-04-25T01:26:03.716509+0000","last_fullsized":"2024-04-25T01:26:03.716509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:17.809738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717064+0000","last_change":"2024-04-25T01:25:53.375324+0000","last_active":"2024-04-25T01:26:03.717064+0000","last_peered":"2024-04-25T01:26:03.717064+0000","last_clean":"2024-04-25T01:26:03.717064+0000","last_became_active":"2024-04-25T01:25:53.374392+0000","last_became_peered":"2024-04-25T01:25:53.374392+0000","last_unstale":"2024-04-25T01:26:03.717064+0000","last_undegraded":"2024-04-25T01:26:03.717064+0000","last_fullsized":"2024-04-25T01:26:03.717064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717818+0000","last_change":"2024-04-25T01:26:02.422720+0000","last_active":"2024-04-25T01:26:03.717818+0000","last_peered":"2024-04-25T01:26:03.717818+0000","last_clean":"2024-04-25T01:26:03.717818+0000","last_became_active":"2024-04-25T01:26:02.421848+0000","last_became_peered":"2024-04-25T01:26:02.421848+0000","last_unstale":"2024-04-25T01:26:03.717818+0000","last_undegraded":"2024-04-25T01:26:03.717818+0000","last_fullsized":"2024-04-25T01:26:03.717818+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507255+0000","last_change":"2024-04-25T01:25:53.367953+0000","last_active":"2024-04-25T01:26:03.507255+0000","last_peered":"2024-04-25T01:26:03.507255+0000","last_clean":"2024-04-25T01:26:03.507255+0000","last_became_active":"2024-04-25T01:25:53.367358+0000","last_became_peered":"2024-04-25T01:25:53.367358+0000","last_unstale":"2024-04-25T01:26:03.507255+0000","last_undegraded":"2024-04-25T01:26:03.507255+0000","last_fullsized":"2024-04-25T01:26:03.507255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415875+0000","last_change":"2024-04-25T01:26:02.422300+0000","last_active":"2024-04-25T01:26:03.415875+0000","last_peered":"2024-04-25T01:26:03.415875+0000","last_clean":"2024-04-25T01:26:03.415875+0000","last_became_active":"2024-04-25T01:26:02.421702+0000","last_became_peered":"2024-04-25T01:26:02.421702+0000","last_unstale":"2024-04-25T01:26:03.415875+0000","last_undegraded":"2024-04-25T01:26:03.415875+0000","last_fullsized":"2024-04-25T01:26:03.415875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717355+0000","last_change":"2024-04-25T01:25:53.371513+0000","last_active":"2024-04-25T01:26:03.717355+0000","last_peered":"2024-04-25T01:26:03.717355+0000","last_clean":"2024-04-25T01:26:03.717355+0000","last_became_active":"2024-04-25T01:25:53.370421+0000","last_became_peered":"2024-04-25T01:25:53.370421+0000","last_unstale":"2024-04-25T01:26:03.717355+0000","last_undegraded":"2024-04-25T01:26:03.717355+0000","last_fullsized":"2024-04-25T01:26:03.717355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415236+0000","last_change":"2024-04-25T01:26:02.411784+0000","last_active":"2024-04-25T01:26:03.415236+0000","last_peered":"2024-04-25T01:26:03.415236+0000","last_clean":"2024-04-25T01:26:03.415236+0000","last_became_active":"2024-04-25T01:26:02.411478+0000","last_became_peered":"2024-04-25T01:26:02.411478+0000","last_unstale":"2024-04-25T01:26:03.415236+0000","last_undegraded":"2024-04-25T01:26:03.415236+0000","last_fullsized":"2024-04-25T01:26:03.415236+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:20.194816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716214+0000","last_change":"2024-04-25T01:25:53.363829+0000","last_active":"2024-04-25T01:26:03.716214+0000","last_peered":"2024-04-25T01:26:03.716214+0000","last_clean":"2024-04-25T01:26:03.716214+0000","last_became_active":"2024-04-25T01:25:53.363525+0000","last_became_peered":"2024-04-25T01:25:53.363525+0000","last_unstale":"2024-04-25T01:26:03.716214+0000","last_undegraded":"2024-04-25T01:26:03.716214+0000","last_fullsized":"2024-04-25T01:26:03.716214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.506843+0000","last_change":"2024-04-25T01:26:02.421208+0000","last_active":"2024-04-25T01:26:03.506843+0000","last_peered":"2024-04-25T01:26:03.506843+0000","last_clean":"2024-04-25T01:26:03.506843+0000","last_became_active":"2024-04-25T01:26:02.418796+0000","last_became_peered":"2024-04-25T01:26:02.418796+0000","last_unstale":"2024-04-25T01:26:03.506843+0000","last_undegraded":"2024-04-25T01:26:03.506843+0000","last_fullsized":"2024-04-25T01:26:03.506843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716784+0000","last_change":"2024-04-25T01:25:53.365057+0000","last_active":"2024-04-25T01:26:03.716784+0000","last_peered":"2024-04-25T01:26:03.716784+0000","last_clean":"2024-04-25T01:26:03.716784+0000","last_became_active":"2024-04-25T01:25:53.364793+0000","last_became_peered":"2024-04-25T01:25:53.364793+0000","last_unstale":"2024-04-25T01:26:03.716784+0000","last_undegraded":"2024-04-25T01:26:03.716784+0000","last_fullsized":"2024-04-25T01:26:03.716784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507060+0000","last_change":"2024-04-25T01:26:02.417606+0000","last_active":"2024-04-25T01:26:03.507060+0000","last_peered":"2024-04-25T01:26:03.507060+0000","last_clean":"2024-04-25T01:26:03.507060+0000","last_became_active":"2024-04-25T01:26:02.417350+0000","last_became_peered":"2024-04-25T01:26:02.417350+0000","last_unstale":"2024-04-25T01:26:03.507060+0000","last_undegraded":"2024-04-25T01:26:03.507060+0000","last_fullsized":"2024-04-25T01:26:03.507060+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.508077+0000","last_change":"2024-04-25T01:25:53.367762+0000","last_active":"2024-04-25T01:26:03.508077+0000","last_peered":"2024-04-25T01:26:03.508077+0000","last_clean":"2024-04-25T01:26:03.508077+0000","last_became_active":"2024-04-25T01:25:53.367383+0000","last_became_peered":"2024-04-25T01:25:53.367383+0000","last_unstale":"2024-04-25T01:26:03.508077+0000","last_undegraded":"2024-04-25T01:26:03.508077+0000","last_fullsized":"2024-04-25T01:26:03.508077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:05.589744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":690301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":101489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":99491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":102507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":99331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":101055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":690267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:26:10.679 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:26:10.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.829+0000 7ffb98cbd640 1 Processor -- start 2024-04-25T01:26:10.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.829+0000 7ffb98cbd640 1 -- start start 2024-04-25T01:26:10.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.830+0000 7ffb98cbd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb940a9490 0x7ffb9406bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:10.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.830+0000 7ffb98cbd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffb94158ff0 0x7ffb94159480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:10.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.830+0000 7ffb98cbd640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb94156460 con 0x7ffb9406b160 2024-04-25T01:26:10.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.830+0000 7ffb98cbd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7ffb940c4a50 con 0x7ffb94158ff0 2024-04-25T01:26:10.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.830+0000 7ffb98cbd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffb94156740 con 0x7ffb940a9490 2024-04-25T01:26:10.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.830+0000 7ffb92d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffb94158ff0 0x7ffb94159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:10.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.830+0000 7ffb92d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffb94158ff0 0x7ffb94159480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43618/0 (socket says 172.21.15.112:43618) 2024-04-25T01:26:10.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.830+0000 7ffb92d9d640 1 -- 172.21.15.112:0/2863559061 learned_addr learned my addr 172.21.15.112:0/2863559061 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:10.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.830+0000 7ffb91d9b640 1 --2- 172.21.15.112:0/2863559061 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb940a9490 0x7ffb9406bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:10.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.830+0000 7ffb92d9d640 1 -- 172.21.15.112:0/2863559061 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb940a9490 msgr2=0x7ffb9406bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:10.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.830+0000 7ffb92d9d640 1 --2- 172.21.15.112:0/2863559061 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb940a9490 0x7ffb9406bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.830+0000 7ffb92d9d640 1 -- 172.21.15.112:0/2863559061 >> v1:172.21.15.187:6789/0 conn(0x7ffb9406b160 legacy=0x7ffb9406b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:10.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.830+0000 7ffb92d9d640 1 -- 172.21.15.112:0/2863559061 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb940c2c80 con 0x7ffb94158ff0 2024-04-25T01:26:10.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.831+0000 7ffb92d9d640 1 --2- 172.21.15.112:0/2863559061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb94158ff0 0x7ffb94159480 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009870 tx=0x7ffb8c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb992cbd7c2fab14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:10.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.831+0000 7ffb9159a640 1 -- 172.21.15.112:0/2863559061 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c037070 con 0x7ffb94158ff0 2024-04-25T01:26:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.831+0000 7ffb9159a640 1 -- 172.21.15.112:0/2863559061 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb8c032440 con 0x7ffb94158ff0 2024-04-25T01:26:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.831+0000 7ffb9159a640 1 -- 172.21.15.112:0/2863559061 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c032810 con 0x7ffb94158ff0 2024-04-25T01:26:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.832+0000 7ffb98cbd640 1 -- 172.21.15.112:0/2863559061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb94158ff0 msgr2=0x7ffb94159480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.832+0000 7ffb98cbd640 1 --2- 172.21.15.112:0/2863559061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb94158ff0 0x7ffb94159480 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009870 tx=0x7ffb8c02a180 comp rx=0 tx=0).stop 2024-04-25T01:26:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.832+0000 7ffb98cbd640 1 -- 172.21.15.112:0/2863559061 shutdown_connections 2024-04-25T01:26:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.832+0000 7ffb98cbd640 1 --2- 172.21.15.112:0/2863559061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb94158ff0 0x7ffb94159480 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.832+0000 7ffb98cbd640 1 --2- 172.21.15.112:0/2863559061 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb940a9490 0x7ffb9406bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.832+0000 7ffb98cbd640 1 -- 172.21.15.112:0/2863559061 >> 172.21.15.112:0/2863559061 conn(0x7ffb940c4c00 msgr2=0x7ffb940c4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.832+0000 7ffb98cbd640 1 -- 172.21.15.112:0/2863559061 shutdown_connections 2024-04-25T01:26:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.832+0000 7ffb98cbd640 1 -- 172.21.15.112:0/2863559061 wait complete. 2024-04-25T01:26:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.833+0000 7ffb98cbd640 1 Processor -- start 2024-04-25T01:26:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.833+0000 7ffb98cbd640 1 -- start start 2024-04-25T01:26:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb98cbd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb9406b160 0x7ffb94138c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb98cbd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb940a9490 0x7ffb94152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb98cbd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb94158ff0 0x7ffb94152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb98cbd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9413f150 con 0x7ffb9406b160 2024-04-25T01:26:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb98cbd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9413efd0 con 0x7ffb94158ff0 2024-04-25T01:26:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb98cbd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffb941599c0 con 0x7ffb940a9490 2024-04-25T01:26:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb9259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb9406b160 0x7ffb94138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb9259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb9406b160 0x7ffb94138c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43624/0 (socket says 172.21.15.112:43624) 2024-04-25T01:26:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb9259c640 1 -- 172.21.15.112:0/3435443974 learned_addr learned my addr 172.21.15.112:0/3435443974 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb91d9b640 1 --2- 172.21.15.112:0/3435443974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb940a9490 0x7ffb94152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb92d9d640 1 --2- 172.21.15.112:0/3435443974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb94158ff0 0x7ffb94152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb9259c640 1 -- 172.21.15.112:0/3435443974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb940a9490 msgr2=0x7ffb94152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb9259c640 1 --2- 172.21.15.112:0/3435443974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb940a9490 0x7ffb94152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb9259c640 1 -- 172.21.15.112:0/3435443974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb94158ff0 msgr2=0x7ffb94152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb9259c640 1 --2- 172.21.15.112:0/3435443974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb94158ff0 0x7ffb94152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb9259c640 1 -- 172.21.15.112:0/3435443974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb941eb640 con 0x7ffb9406b160 2024-04-25T01:26:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb91d9b640 1 --2- 172.21.15.112:0/3435443974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb940a9490 0x7ffb94152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb9259c640 1 --2- 172.21.15.112:0/3435443974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb9406b160 0x7ffb94138c00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400c7c0 tx=0x7ffb8400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb82ffd640 1 -- 172.21.15.112:0/3435443974 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84016020 con 0x7ffb9406b160 2024-04-25T01:26:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb82ffd640 1 -- 172.21.15.112:0/3435443974 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb8400aa50 con 0x7ffb9406b160 2024-04-25T01:26:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb82ffd640 1 -- 172.21.15.112:0/3435443974 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8400ae40 con 0x7ffb9406b160 2024-04-25T01:26:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb98cbd640 1 -- 172.21.15.112:0/3435443974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb941ef910 con 0x7ffb9406b160 2024-04-25T01:26:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.834+0000 7ffb98cbd640 1 -- 172.21.15.112:0/3435443974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb941ebba0 con 0x7ffb9406b160 2024-04-25T01:26:10.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.836+0000 7ffb98cbd640 1 -- 172.21.15.112:0/3435443974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb940aafc0 con 0x7ffb9406b160 2024-04-25T01:26:10.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.836+0000 7ffb82ffd640 1 -- 172.21.15.112:0/3435443974 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffb84019020 con 0x7ffb9406b160 2024-04-25T01:26:10.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.836+0000 7ffb82ffd640 1 --2- 172.21.15.112:0/3435443974 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ffb7406be30 0x7ffb7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:10.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.836+0000 7ffb91d9b640 1 --2- 172.21.15.112:0/3435443974 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ffb7406be30 0x7ffb7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:10.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.836+0000 7ffb82ffd640 1 -- 172.21.15.112:0/3435443974 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7ffb8408b150 con 0x7ffb9406b160 2024-04-25T01:26:10.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.837+0000 7ffb91d9b640 1 --2- 172.21.15.112:0/3435443974 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ffb7406be30 0x7ffb7406e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffb940c5f70 tx=0x7ffb88008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:10.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:10.841+0000 7ffb82ffd640 1 -- 172.21.15.112:0/3435443974 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffb84020030 con 0x7ffb9406b160 2024-04-25T01:26:10.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.009+0000 7ffb98cbd640 1 -- 172.21.15.112:0/3435443974 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffb94075980 con 0x7ffb7406be30 2024-04-25T01:26:10.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.020+0000 7ffb82ffd640 1 -- 172.21.15.112:0/3435443974 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+490305 (secure 0 0 0) 0x7ffb8800f830 con 0x7ffb7406be30 2024-04-25T01:26:11.005 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:11.008 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T01:26:11.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.024+0000 7ffb98cbd640 1 -- 172.21.15.112:0/3435443974 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ffb7406be30 msgr2=0x7ffb7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:11.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.024+0000 7ffb98cbd640 1 --2- 172.21.15.112:0/3435443974 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ffb7406be30 0x7ffb7406e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffb940c5f70 tx=0x7ffb88008040 comp rx=0 tx=0).stop 2024-04-25T01:26:11.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.024+0000 7ffb98cbd640 1 -- 172.21.15.112:0/3435443974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb9406b160 msgr2=0x7ffb94138c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:11.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.024+0000 7ffb98cbd640 1 --2- 172.21.15.112:0/3435443974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb9406b160 0x7ffb94138c00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400c7c0 tx=0x7ffb8400cc80 comp rx=0 tx=0).stop 2024-04-25T01:26:11.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.024+0000 7ffb98cbd640 1 -- 172.21.15.112:0/3435443974 shutdown_connections 2024-04-25T01:26:11.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.024+0000 7ffb98cbd640 1 --2- 172.21.15.112:0/3435443974 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ffb7406be30 0x7ffb7406e2f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.024+0000 7ffb98cbd640 1 --2- 172.21.15.112:0/3435443974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb94158ff0 0x7ffb94152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.024+0000 7ffb98cbd640 1 --2- 172.21.15.112:0/3435443974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb940a9490 0x7ffb94152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.024+0000 7ffb98cbd640 1 --2- 172.21.15.112:0/3435443974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb9406b160 0x7ffb94138c00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.024+0000 7ffb98cbd640 1 -- 172.21.15.112:0/3435443974 >> 172.21.15.112:0/3435443974 conn(0x7ffb940c4c00 msgr2=0x7ffb94139ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:11.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.024+0000 7ffb98cbd640 1 -- 172.21.15.112:0/3435443974 shutdown_connections 2024-04-25T01:26:11.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.024+0000 7ffb98cbd640 1 -- 172.21.15.112:0/3435443974 wait complete. 2024-04-25T01:26:11.014 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-04-25T01:26:10.652028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3350528,"data_stored":1984464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.254461"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415591+0000","last_change":"2024-04-25T01:26:02.412272+0000","last_active":"2024-04-25T01:26:03.415591+0000","last_peered":"2024-04-25T01:26:03.415591+0000","last_clean":"2024-04-25T01:26:03.415591+0000","last_became_active":"2024-04-25T01:26:02.411176+0000","last_became_peered":"2024-04-25T01:26:02.411176+0000","last_unstale":"2024-04-25T01:26:03.415591+0000","last_undegraded":"2024-04-25T01:26:03.415591+0000","last_fullsized":"2024-04-25T01:26:03.415591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:49.498275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716276+0000","last_change":"2024-04-25T01:25:52.353978+0000","last_active":"2024-04-25T01:26:03.716276+0000","last_peered":"2024-04-25T01:26:03.716276+0000","last_clean":"2024-04-25T01:26:03.716276+0000","last_became_active":"2024-04-25T01:25:52.353649+0000","last_became_peered":"2024-04-25T01:25:52.353649+0000","last_unstale":"2024-04-25T01:26:03.716276+0000","last_undegraded":"2024-04-25T01:26:03.716276+0000","last_fullsized":"2024-04-25T01:26:03.716276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416695+0000","last_change":"2024-04-25T01:25:53.364257+0000","last_active":"2024-04-25T01:26:03.416695+0000","last_peered":"2024-04-25T01:26:03.416695+0000","last_clean":"2024-04-25T01:26:03.416695+0000","last_became_active":"2024-04-25T01:25:53.363409+0000","last_became_peered":"2024-04-25T01:25:53.363409+0000","last_unstale":"2024-04-25T01:26:03.416695+0000","last_undegraded":"2024-04-25T01:26:03.416695+0000","last_fullsized":"2024-04-25T01:26:03.416695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415484+0000","last_change":"2024-04-25T01:26:01.407136+0000","last_active":"2024-04-25T01:26:03.415484+0000","last_peered":"2024-04-25T01:26:03.415484+0000","last_clean":"2024-04-25T01:26:03.415484+0000","last_became_active":"2024-04-25T01:26:01.406855+0000","last_became_peered":"2024-04-25T01:26:01.406855+0000","last_unstale":"2024-04-25T01:26:03.415484+0000","last_undegraded":"2024-04-25T01:26:03.415484+0000","last_fullsized":"2024-04-25T01:26:03.415484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:02.180826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716643+0000","last_change":"2024-04-25T01:26:02.425367+0000","last_active":"2024-04-25T01:26:03.716643+0000","last_peered":"2024-04-25T01:26:03.716643+0000","last_clean":"2024-04-25T01:26:03.716643+0000","last_became_active":"2024-04-25T01:26:02.425071+0000","last_became_peered":"2024-04-25T01:26:02.425071+0000","last_unstale":"2024-04-25T01:26:03.716643+0000","last_undegraded":"2024-04-25T01:26:03.716643+0000","last_fullsized":"2024-04-25T01:26:03.716643+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:47.810943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416737+0000","last_change":"2024-04-25T01:25:52.349185+0000","last_active":"2024-04-25T01:26:03.416737+0000","last_peered":"2024-04-25T01:26:03.416737+0000","last_clean":"2024-04-25T01:26:03.416737+0000","last_became_active":"2024-04-25T01:25:52.348931+0000","last_became_peered":"2024-04-25T01:25:52.348931+0000","last_unstale":"2024-04-25T01:26:03.416737+0000","last_undegraded":"2024-04-25T01:26:03.416737+0000","last_fullsized":"2024-04-25T01:26:03.416737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716082+0000","last_change":"2024-04-25T01:25:53.374150+0000","last_active":"2024-04-25T01:26:03.716082+0000","last_peered":"2024-04-25T01:26:03.716082+0000","last_clean":"2024-04-25T01:26:03.716082+0000","last_became_active":"2024-04-25T01:25:53.373755+0000","last_became_peered":"2024-04-25T01:25:53.373755+0000","last_unstale":"2024-04-25T01:26:03.716082+0000","last_undegraded":"2024-04-25T01:26:03.716082+0000","last_fullsized":"2024-04-25T01:26:03.716082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415473+0000","last_change":"2024-04-25T01:26:01.406334+0000","last_active":"2024-04-25T01:26:03.415473+0000","last_peered":"2024-04-25T01:26:03.415473+0000","last_clean":"2024-04-25T01:26:03.415473+0000","last_became_active":"2024-04-25T01:26:01.405994+0000","last_became_peered":"2024-04-25T01:26:01.405994+0000","last_unstale":"2024-04-25T01:26:03.415473+0000","last_undegraded":"2024-04-25T01:26:03.415473+0000","last_fullsized":"2024-04-25T01:26:03.415473+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:43.953437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507172+0000","last_change":"2024-04-25T01:26:02.427695+0000","last_active":"2024-04-25T01:26:03.507172+0000","last_peered":"2024-04-25T01:26:03.507172+0000","last_clean":"2024-04-25T01:26:03.507172+0000","last_became_active":"2024-04-25T01:26:02.427443+0000","last_became_peered":"2024-04-25T01:26:02.427443+0000","last_unstale":"2024-04-25T01:26:03.507172+0000","last_undegraded":"2024-04-25T01:26:03.507172+0000","last_fullsized":"2024-04-25T01:26:03.507172+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507616+0000","last_change":"2024-04-25T01:25:52.356403+0000","last_active":"2024-04-25T01:26:03.507616+0000","last_peered":"2024-04-25T01:26:03.507616+0000","last_clean":"2024-04-25T01:26:03.507616+0000","last_became_active":"2024-04-25T01:25:52.355822+0000","last_became_peered":"2024-04-25T01:25:52.355822+0000","last_unstale":"2024-04-25T01:26:03.507616+0000","last_undegraded":"2024-04-25T01:26:03.507616+0000","last_fullsized":"2024-04-25T01:26:03.507616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717449+0000","last_change":"2024-04-25T01:25:53.370061+0000","last_active":"2024-04-25T01:26:03.717449+0000","last_peered":"2024-04-25T01:26:03.717449+0000","last_clean":"2024-04-25T01:26:03.717449+0000","last_became_active":"2024-04-25T01:25:53.369721+0000","last_became_peered":"2024-04-25T01:25:53.369721+0000","last_unstale":"2024-04-25T01:26:03.717449+0000","last_undegraded":"2024-04-25T01:26:03.717449+0000","last_fullsized":"2024-04-25T01:26:03.717449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415934+0000","last_change":"2024-04-25T01:26:01.408179+0000","last_active":"2024-04-25T01:26:03.415934+0000","last_peered":"2024-04-25T01:26:03.415934+0000","last_clean":"2024-04-25T01:26:03.415934+0000","last_became_active":"2024-04-25T01:26:01.407909+0000","last_became_peered":"2024-04-25T01:26:01.407909+0000","last_unstale":"2024-04-25T01:26:03.415934+0000","last_undegraded":"2024-04-25T01:26:03.415934+0000","last_fullsized":"2024-04-25T01:26:03.415934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:48.865940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717234+0000","last_change":"2024-04-25T01:26:02.424615+0000","last_active":"2024-04-25T01:26:03.717234+0000","last_peered":"2024-04-25T01:26:03.717234+0000","last_clean":"2024-04-25T01:26:03.717234+0000","last_became_active":"2024-04-25T01:26:02.424325+0000","last_became_peered":"2024-04-25T01:26:02.424325+0000","last_unstale":"2024-04-25T01:26:03.717234+0000","last_undegraded":"2024-04-25T01:26:03.717234+0000","last_fullsized":"2024-04-25T01:26:03.717234+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507443+0000","last_change":"2024-04-25T01:25:52.359139+0000","last_active":"2024-04-25T01:26:03.507443+0000","last_peered":"2024-04-25T01:26:03.507443+0000","last_clean":"2024-04-25T01:26:03.507443+0000","last_became_active":"2024-04-25T01:25:52.358841+0000","last_became_peered":"2024-04-25T01:25:52.358841+0000","last_unstale":"2024-04-25T01:26:03.507443+0000","last_undegraded":"2024-04-25T01:26:03.507443+0000","last_fullsized":"2024-04-25T01:26:03.507443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717907+0000","last_change":"2024-04-25T01:25:53.368739+0000","last_active":"2024-04-25T01:26:03.717907+0000","last_peered":"2024-04-25T01:26:03.717907+0000","last_clean":"2024-04-25T01:26:03.717907+0000","last_became_active":"2024-04-25T01:25:53.368470+0000","last_became_peered":"2024-04-25T01:25:53.368470+0000","last_unstale":"2024-04-25T01:26:03.717907+0000","last_undegraded":"2024-04-25T01:26:03.717907+0000","last_fullsized":"2024-04-25T01:26:03.717907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:29.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415820+0000","last_change":"2024-04-25T01:26:01.407969+0000","last_active":"2024-04-25T01:26:03.415820+0000","last_peered":"2024-04-25T01:26:03.415820+0000","last_clean":"2024-04-25T01:26:03.415820+0000","last_became_active":"2024-04-25T01:26:01.407655+0000","last_became_peered":"2024-04-25T01:26:01.407655+0000","last_unstale":"2024-04-25T01:26:03.415820+0000","last_undegraded":"2024-04-25T01:26:03.415820+0000","last_fullsized":"2024-04-25T01:26:03.415820+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:58.485864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717005+0000","last_change":"2024-04-25T01:26:02.417776+0000","last_active":"2024-04-25T01:26:03.717005+0000","last_peered":"2024-04-25T01:26:03.717005+0000","last_clean":"2024-04-25T01:26:03.717005+0000","last_became_active":"2024-04-25T01:26:02.417524+0000","last_became_peered":"2024-04-25T01:26:02.417524+0000","last_unstale":"2024-04-25T01:26:03.717005+0000","last_undegraded":"2024-04-25T01:26:03.717005+0000","last_fullsized":"2024-04-25T01:26:03.717005+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.688469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716814+0000","last_change":"2024-04-25T01:25:53.372385+0000","last_active":"2024-04-25T01:26:03.716814+0000","last_peered":"2024-04-25T01:26:03.716814+0000","last_clean":"2024-04-25T01:26:03.716814+0000","last_became_active":"2024-04-25T01:25:53.372064+0000","last_became_peered":"2024-04-25T01:25:53.372064+0000","last_unstale":"2024-04-25T01:26:03.716814+0000","last_undegraded":"2024-04-25T01:26:03.716814+0000","last_fullsized":"2024-04-25T01:26:03.716814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507411+0000","last_change":"2024-04-25T01:25:52.358616+0000","last_active":"2024-04-25T01:26:03.507411+0000","last_peered":"2024-04-25T01:26:03.507411+0000","last_clean":"2024-04-25T01:26:03.507411+0000","last_became_active":"2024-04-25T01:25:52.358318+0000","last_became_peered":"2024-04-25T01:25:52.358318+0000","last_unstale":"2024-04-25T01:26:03.507411+0000","last_undegraded":"2024-04-25T01:26:03.507411+0000","last_fullsized":"2024-04-25T01:26:03.507411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416542+0000","last_change":"2024-04-25T01:26:01.400599+0000","last_active":"2024-04-25T01:26:03.416542+0000","last_peered":"2024-04-25T01:26:03.416542+0000","last_clean":"2024-04-25T01:26:03.416542+0000","last_became_active":"2024-04-25T01:26:01.400266+0000","last_became_peered":"2024-04-25T01:26:01.400266+0000","last_unstale":"2024-04-25T01:26:03.416542+0000","last_undegraded":"2024-04-25T01:26:03.416542+0000","last_fullsized":"2024-04-25T01:26:03.416542+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416953+0000","last_change":"2024-04-25T01:26:02.420269+0000","last_active":"2024-04-25T01:26:03.416953+0000","last_peered":"2024-04-25T01:26:03.416953+0000","last_clean":"2024-04-25T01:26:03.416953+0000","last_became_active":"2024-04-25T01:26:02.420001+0000","last_became_peered":"2024-04-25T01:26:02.420001+0000","last_unstale":"2024-04-25T01:26:03.416953+0000","last_undegraded":"2024-04-25T01:26:03.416953+0000","last_fullsized":"2024-04-25T01:26:03.416953+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:38.149069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415549+0000","last_change":"2024-04-25T01:25:53.361074+0000","last_active":"2024-04-25T01:26:03.415549+0000","last_peered":"2024-04-25T01:26:03.415549+0000","last_clean":"2024-04-25T01:26:03.415549+0000","last_became_active":"2024-04-25T01:25:53.360802+0000","last_became_peered":"2024-04-25T01:25:53.360802+0000","last_unstale":"2024-04-25T01:26:03.415549+0000","last_undegraded":"2024-04-25T01:26:03.415549+0000","last_fullsized":"2024-04-25T01:26:03.415549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717195+0000","last_change":"2024-04-25T01:25:52.348841+0000","last_active":"2024-04-25T01:26:03.717195+0000","last_peered":"2024-04-25T01:26:03.717195+0000","last_clean":"2024-04-25T01:26:03.717195+0000","last_became_active":"2024-04-25T01:25:52.348550+0000","last_became_peered":"2024-04-25T01:25:52.348550+0000","last_unstale":"2024-04-25T01:26:03.717195+0000","last_undegraded":"2024-04-25T01:26:03.717195+0000","last_fullsized":"2024-04-25T01:26:03.717195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416126+0000","last_change":"2024-04-25T01:26:01.408529+0000","last_active":"2024-04-25T01:26:03.416126+0000","last_peered":"2024-04-25T01:26:03.416126+0000","last_clean":"2024-04-25T01:26:03.416126+0000","last_became_active":"2024-04-25T01:26:01.407738+0000","last_became_peered":"2024-04-25T01:26:01.407738+0000","last_unstale":"2024-04-25T01:26:03.416126+0000","last_undegraded":"2024-04-25T01:26:03.416126+0000","last_fullsized":"2024-04-25T01:26:03.416126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:37.142255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416507+0000","last_change":"2024-04-25T01:26:02.417731+0000","last_active":"2024-04-25T01:26:03.416507+0000","last_peered":"2024-04-25T01:26:03.416507+0000","last_clean":"2024-04-25T01:26:03.416507+0000","last_became_active":"2024-04-25T01:26:02.417438+0000","last_became_peered":"2024-04-25T01:26:02.417438+0000","last_unstale":"2024-04-25T01:26:03.416507+0000","last_undegraded":"2024-04-25T01:26:03.416507+0000","last_fullsized":"2024-04-25T01:26:03.416507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415373+0000","last_change":"2024-04-25T01:25:53.361790+0000","last_active":"2024-04-25T01:26:03.415373+0000","last_peered":"2024-04-25T01:26:03.415373+0000","last_clean":"2024-04-25T01:26:03.415373+0000","last_became_active":"2024-04-25T01:25:53.361542+0000","last_became_peered":"2024-04-25T01:25:53.361542+0000","last_unstale":"2024-04-25T01:26:03.415373+0000","last_undegraded":"2024-04-25T01:26:03.415373+0000","last_fullsized":"2024-04-25T01:26:03.415373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716946+0000","last_change":"2024-04-25T01:25:52.353926+0000","last_active":"2024-04-25T01:26:03.716946+0000","last_peered":"2024-04-25T01:26:03.716946+0000","last_clean":"2024-04-25T01:26:03.716946+0000","last_became_active":"2024-04-25T01:25:52.353656+0000","last_became_peered":"2024-04-25T01:25:52.353656+0000","last_unstale":"2024-04-25T01:26:03.716946+0000","last_undegraded":"2024-04-25T01:26:03.716946+0000","last_fullsized":"2024-04-25T01:26:03.716946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716923+0000","last_change":"2024-04-25T01:26:01.418556+0000","last_active":"2024-04-25T01:26:03.716923+0000","last_peered":"2024-04-25T01:26:03.716923+0000","last_clean":"2024-04-25T01:26:03.716923+0000","last_became_active":"2024-04-25T01:26:01.418263+0000","last_became_peered":"2024-04-25T01:26:01.418263+0000","last_unstale":"2024-04-25T01:26:03.716923+0000","last_undegraded":"2024-04-25T01:26:03.716923+0000","last_fullsized":"2024-04-25T01:26:03.716923+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:28.139183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717839+0000","last_change":"2024-04-25T01:26:02.413429+0000","last_active":"2024-04-25T01:26:03.717839+0000","last_peered":"2024-04-25T01:26:03.717839+0000","last_clean":"2024-04-25T01:26:03.717839+0000","last_became_active":"2024-04-25T01:26:02.412632+0000","last_became_peered":"2024-04-25T01:26:02.412632+0000","last_unstale":"2024-04-25T01:26:03.717839+0000","last_undegraded":"2024-04-25T01:26:03.717839+0000","last_fullsized":"2024-04-25T01:26:03.717839+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:10.049310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416067+0000","last_change":"2024-04-25T01:25:53.361653+0000","last_active":"2024-04-25T01:26:03.416067+0000","last_peered":"2024-04-25T01:26:03.416067+0000","last_clean":"2024-04-25T01:26:03.416067+0000","last_became_active":"2024-04-25T01:25:53.361265+0000","last_became_peered":"2024-04-25T01:25:53.361265+0000","last_unstale":"2024-04-25T01:26:03.416067+0000","last_undegraded":"2024-04-25T01:26:03.416067+0000","last_fullsized":"2024-04-25T01:26:03.416067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507177+0000","last_change":"2024-04-25T01:25:52.353836+0000","last_active":"2024-04-25T01:26:03.507177+0000","last_peered":"2024-04-25T01:26:03.507177+0000","last_clean":"2024-04-25T01:26:03.507177+0000","last_became_active":"2024-04-25T01:25:52.353552+0000","last_became_peered":"2024-04-25T01:25:52.353552+0000","last_unstale":"2024-04-25T01:26:03.507177+0000","last_undegraded":"2024-04-25T01:26:03.507177+0000","last_fullsized":"2024-04-25T01:26:03.507177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416325+0000","last_change":"2024-04-25T01:26:01.411323+0000","last_active":"2024-04-25T01:26:03.416325+0000","last_peered":"2024-04-25T01:26:03.416325+0000","last_clean":"2024-04-25T01:26:03.416325+0000","last_became_active":"2024-04-25T01:26:01.410378+0000","last_became_peered":"2024-04-25T01:26:01.410378+0000","last_unstale":"2024-04-25T01:26:03.416325+0000","last_undegraded":"2024-04-25T01:26:03.416325+0000","last_fullsized":"2024-04-25T01:26:03.416325+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415000+0000","last_change":"2024-04-25T01:26:02.416646+0000","last_active":"2024-04-25T01:26:03.415000+0000","last_peered":"2024-04-25T01:26:03.415000+0000","last_clean":"2024-04-25T01:26:03.415000+0000","last_became_active":"2024-04-25T01:26:02.416395+0000","last_became_peered":"2024-04-25T01:26:02.416395+0000","last_unstale":"2024-04-25T01:26:03.415000+0000","last_undegraded":"2024-04-25T01:26:03.415000+0000","last_fullsized":"2024-04-25T01:26:03.415000+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716795+0000","last_change":"2024-04-25T01:25:53.370111+0000","last_active":"2024-04-25T01:26:03.716795+0000","last_peered":"2024-04-25T01:26:03.716795+0000","last_clean":"2024-04-25T01:26:03.716795+0000","last_became_active":"2024-04-25T01:25:53.369837+0000","last_became_peered":"2024-04-25T01:25:53.369837+0000","last_unstale":"2024-04-25T01:26:03.716795+0000","last_undegraded":"2024-04-25T01:26:03.716795+0000","last_fullsized":"2024-04-25T01:26:03.716795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717961+0000","last_change":"2024-04-25T01:25:52.353269+0000","last_active":"2024-04-25T01:26:03.717961+0000","last_peered":"2024-04-25T01:26:03.717961+0000","last_clean":"2024-04-25T01:26:03.717961+0000","last_became_active":"2024-04-25T01:25:52.352991+0000","last_became_peered":"2024-04-25T01:25:52.352991+0000","last_unstale":"2024-04-25T01:26:03.717961+0000","last_undegraded":"2024-04-25T01:26:03.717961+0000","last_fullsized":"2024-04-25T01:26:03.717961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415651+0000","last_change":"2024-04-25T01:26:01.410360+0000","last_active":"2024-04-25T01:26:03.415651+0000","last_peered":"2024-04-25T01:26:03.415651+0000","last_clean":"2024-04-25T01:26:03.415651+0000","last_became_active":"2024-04-25T01:26:01.410076+0000","last_became_peered":"2024-04-25T01:26:01.410076+0000","last_unstale":"2024-04-25T01:26:03.415651+0000","last_undegraded":"2024-04-25T01:26:03.415651+0000","last_fullsized":"2024-04-25T01:26:03.415651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:52.561251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415577+0000","last_change":"2024-04-25T01:26:02.422182+0000","last_active":"2024-04-25T01:26:03.415577+0000","last_peered":"2024-04-25T01:26:03.415577+0000","last_clean":"2024-04-25T01:26:03.415577+0000","last_became_active":"2024-04-25T01:26:02.421885+0000","last_became_peered":"2024-04-25T01:26:02.421885+0000","last_unstale":"2024-04-25T01:26:03.415577+0000","last_undegraded":"2024-04-25T01:26:03.415577+0000","last_fullsized":"2024-04-25T01:26:03.415577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:03.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416628+0000","last_change":"2024-04-25T01:25:53.364967+0000","last_active":"2024-04-25T01:26:03.416628+0000","last_peered":"2024-04-25T01:26:03.416628+0000","last_clean":"2024-04-25T01:26:03.416628+0000","last_became_active":"2024-04-25T01:25:53.364694+0000","last_became_peered":"2024-04-25T01:25:53.364694+0000","last_unstale":"2024-04-25T01:26:03.416628+0000","last_undegraded":"2024-04-25T01:26:03.416628+0000","last_fullsized":"2024-04-25T01:26:03.416628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717049+0000","last_change":"2024-04-25T01:25:52.355395+0000","last_active":"2024-04-25T01:26:03.717049+0000","last_peered":"2024-04-25T01:26:03.717049+0000","last_clean":"2024-04-25T01:26:03.717049+0000","last_became_active":"2024-04-25T01:25:52.355137+0000","last_became_peered":"2024-04-25T01:25:52.355137+0000","last_unstale":"2024-04-25T01:26:03.717049+0000","last_undegraded":"2024-04-25T01:26:03.717049+0000","last_fullsized":"2024-04-25T01:26:03.717049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:26.467968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.414943+0000","last_change":"2024-04-25T01:26:01.416108+0000","last_active":"2024-04-25T01:26:03.414943+0000","last_peered":"2024-04-25T01:26:03.414943+0000","last_clean":"2024-04-25T01:26:03.414943+0000","last_became_active":"2024-04-25T01:26:01.415824+0000","last_became_peered":"2024-04-25T01:26:01.415824+0000","last_unstale":"2024-04-25T01:26:03.414943+0000","last_undegraded":"2024-04-25T01:26:03.414943+0000","last_fullsized":"2024-04-25T01:26:03.414943+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:28.139183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716568+0000","last_change":"2024-04-25T01:26:02.416208+0000","last_active":"2024-04-25T01:26:03.716568+0000","last_peered":"2024-04-25T01:26:03.716568+0000","last_clean":"2024-04-25T01:26:03.716568+0000","last_became_active":"2024-04-25T01:26:02.415949+0000","last_became_peered":"2024-04-25T01:26:02.415949+0000","last_unstale":"2024-04-25T01:26:03.716568+0000","last_undegraded":"2024-04-25T01:26:03.716568+0000","last_fullsized":"2024-04-25T01:26:03.716568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:21.097225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717249+0000","last_change":"2024-04-25T01:25:53.366944+0000","last_active":"2024-04-25T01:26:03.717249+0000","last_peered":"2024-04-25T01:26:03.717249+0000","last_clean":"2024-04-25T01:26:03.717249+0000","last_became_active":"2024-04-25T01:25:53.366693+0000","last_became_peered":"2024-04-25T01:25:53.366693+0000","last_unstale":"2024-04-25T01:26:03.717249+0000","last_undegraded":"2024-04-25T01:26:03.717249+0000","last_fullsized":"2024-04-25T01:26:03.717249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507732+0000","last_change":"2024-04-25T01:25:52.356842+0000","last_active":"2024-04-25T01:26:03.507732+0000","last_peered":"2024-04-25T01:26:03.507732+0000","last_clean":"2024-04-25T01:26:03.507732+0000","last_became_active":"2024-04-25T01:25:52.356396+0000","last_became_peered":"2024-04-25T01:25:52.356396+0000","last_unstale":"2024-04-25T01:26:03.507732+0000","last_undegraded":"2024-04-25T01:26:03.507732+0000","last_fullsized":"2024-04-25T01:26:03.507732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716730+0000","last_change":"2024-04-25T01:26:01.407934+0000","last_active":"2024-04-25T01:26:03.716730+0000","last_peered":"2024-04-25T01:26:03.716730+0000","last_clean":"2024-04-25T01:26:03.716730+0000","last_became_active":"2024-04-25T01:26:01.407659+0000","last_became_peered":"2024-04-25T01:26:01.407659+0000","last_unstale":"2024-04-25T01:26:03.716730+0000","last_undegraded":"2024-04-25T01:26:03.716730+0000","last_fullsized":"2024-04-25T01:26:03.716730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:19.629335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716666+0000","last_change":"2024-04-25T01:26:02.418087+0000","last_active":"2024-04-25T01:26:03.716666+0000","last_peered":"2024-04-25T01:26:03.716666+0000","last_clean":"2024-04-25T01:26:03.716666+0000","last_became_active":"2024-04-25T01:26:02.417622+0000","last_became_peered":"2024-04-25T01:26:02.417622+0000","last_unstale":"2024-04-25T01:26:03.716666+0000","last_undegraded":"2024-04-25T01:26:03.716666+0000","last_fullsized":"2024-04-25T01:26:03.716666+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:03.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717962+0000","last_change":"2024-04-25T01:25:53.368164+0000","last_active":"2024-04-25T01:26:03.717962+0000","last_peered":"2024-04-25T01:26:03.717962+0000","last_clean":"2024-04-25T01:26:03.717962+0000","last_became_active":"2024-04-25T01:25:53.367854+0000","last_became_peered":"2024-04-25T01:25:53.367854+0000","last_unstale":"2024-04-25T01:26:03.717962+0000","last_undegraded":"2024-04-25T01:26:03.717962+0000","last_fullsized":"2024-04-25T01:26:03.717962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717294+0000","last_change":"2024-04-25T01:25:52.350128+0000","last_active":"2024-04-25T01:26:03.717294+0000","last_peered":"2024-04-25T01:26:03.717294+0000","last_clean":"2024-04-25T01:26:03.717294+0000","last_became_active":"2024-04-25T01:25:52.349874+0000","last_became_peered":"2024-04-25T01:25:52.349874+0000","last_unstale":"2024-04-25T01:26:03.717294+0000","last_undegraded":"2024-04-25T01:26:03.717294+0000","last_fullsized":"2024-04-25T01:26:03.717294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716937+0000","last_change":"2024-04-25T01:26:01.418196+0000","last_active":"2024-04-25T01:26:03.716937+0000","last_peered":"2024-04-25T01:26:03.716937+0000","last_clean":"2024-04-25T01:26:03.716937+0000","last_became_active":"2024-04-25T01:26:01.417820+0000","last_became_peered":"2024-04-25T01:26:01.417820+0000","last_unstale":"2024-04-25T01:26:03.716937+0000","last_undegraded":"2024-04-25T01:26:03.716937+0000","last_fullsized":"2024-04-25T01:26:03.716937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716324+0000","last_change":"2024-04-25T01:26:02.422761+0000","last_active":"2024-04-25T01:26:03.716324+0000","last_peered":"2024-04-25T01:26:03.716324+0000","last_clean":"2024-04-25T01:26:03.716324+0000","last_became_active":"2024-04-25T01:26:02.422503+0000","last_became_peered":"2024-04-25T01:26:02.422503+0000","last_unstale":"2024-04-25T01:26:03.716324+0000","last_undegraded":"2024-04-25T01:26:03.716324+0000","last_fullsized":"2024-04-25T01:26:03.716324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:55.040946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416617+0000","last_change":"2024-04-25T01:25:53.361696+0000","last_active":"2024-04-25T01:26:03.416617+0000","last_peered":"2024-04-25T01:26:03.416617+0000","last_clean":"2024-04-25T01:26:03.416617+0000","last_became_active":"2024-04-25T01:25:53.361156+0000","last_became_peered":"2024-04-25T01:25:53.361156+0000","last_unstale":"2024-04-25T01:26:03.416617+0000","last_undegraded":"2024-04-25T01:26:03.416617+0000","last_fullsized":"2024-04-25T01:26:03.416617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:27.472315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717721+0000","last_change":"2024-04-25T01:25:52.354499+0000","last_active":"2024-04-25T01:26:03.717721+0000","last_peered":"2024-04-25T01:26:03.717721+0000","last_clean":"2024-04-25T01:26:03.717721+0000","last_became_active":"2024-04-25T01:25:52.354193+0000","last_became_peered":"2024-04-25T01:25:52.354193+0000","last_unstale":"2024-04-25T01:26:03.717721+0000","last_undegraded":"2024-04-25T01:26:03.717721+0000","last_fullsized":"2024-04-25T01:26:03.717721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717190+0000","last_change":"2024-04-25T01:26:01.418186+0000","last_active":"2024-04-25T01:26:03.717190+0000","last_peered":"2024-04-25T01:26:03.717190+0000","last_clean":"2024-04-25T01:26:03.717190+0000","last_became_active":"2024-04-25T01:26:01.417805+0000","last_became_peered":"2024-04-25T01:26:01.417805+0000","last_unstale":"2024-04-25T01:26:03.717190+0000","last_undegraded":"2024-04-25T01:26:03.717190+0000","last_fullsized":"2024-04-25T01:26:03.717190+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:19.629335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416697+0000","last_change":"2024-04-25T01:26:02.415877+0000","last_active":"2024-04-25T01:26:03.416697+0000","last_peered":"2024-04-25T01:26:03.416697+0000","last_clean":"2024-04-25T01:26:03.416697+0000","last_became_active":"2024-04-25T01:26:02.415619+0000","last_became_peered":"2024-04-25T01:26:02.415619+0000","last_unstale":"2024-04-25T01:26:03.416697+0000","last_undegraded":"2024-04-25T01:26:03.416697+0000","last_fullsized":"2024-04-25T01:26:03.416697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:14.647083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716434+0000","last_change":"2024-04-25T01:25:53.362922+0000","last_active":"2024-04-25T01:26:03.716434+0000","last_peered":"2024-04-25T01:26:03.716434+0000","last_clean":"2024-04-25T01:26:03.716434+0000","last_became_active":"2024-04-25T01:25:53.362533+0000","last_became_peered":"2024-04-25T01:25:53.362533+0000","last_unstale":"2024-04-25T01:26:03.716434+0000","last_undegraded":"2024-04-25T01:26:03.716434+0000","last_fullsized":"2024-04-25T01:26:03.716434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415851+0000","last_change":"2024-04-25T01:25:52.345321+0000","last_active":"2024-04-25T01:26:03.415851+0000","last_peered":"2024-04-25T01:26:03.415851+0000","last_clean":"2024-04-25T01:26:03.415851+0000","last_became_active":"2024-04-25T01:25:52.345061+0000","last_became_peered":"2024-04-25T01:25:52.345061+0000","last_unstale":"2024-04-25T01:26:03.415851+0000","last_undegraded":"2024-04-25T01:26:03.415851+0000","last_fullsized":"2024-04-25T01:26:03.415851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717072+0000","last_change":"2024-04-25T01:26:01.418516+0000","last_active":"2024-04-25T01:26:03.717072+0000","last_peered":"2024-04-25T01:26:03.717072+0000","last_clean":"2024-04-25T01:26:03.717072+0000","last_became_active":"2024-04-25T01:26:01.418173+0000","last_became_peered":"2024-04-25T01:26:01.418173+0000","last_unstale":"2024-04-25T01:26:03.717072+0000","last_undegraded":"2024-04-25T01:26:03.717072+0000","last_fullsized":"2024-04-25T01:26:03.717072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:02.568472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.718145+0000","last_change":"2024-04-25T01:26:02.418115+0000","last_active":"2024-04-25T01:26:03.718145+0000","last_peered":"2024-04-25T01:26:03.718145+0000","last_clean":"2024-04-25T01:26:03.718145+0000","last_became_active":"2024-04-25T01:26:02.417355+0000","last_became_peered":"2024-04-25T01:26:02.417355+0000","last_unstale":"2024-04-25T01:26:03.718145+0000","last_undegraded":"2024-04-25T01:26:03.718145+0000","last_fullsized":"2024-04-25T01:26:03.718145+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:19.644468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716066+0000","last_change":"2024-04-25T01:25:53.355950+0000","last_active":"2024-04-25T01:26:03.716066+0000","last_peered":"2024-04-25T01:26:03.716066+0000","last_clean":"2024-04-25T01:26:03.716066+0000","last_became_active":"2024-04-25T01:25:53.355648+0000","last_became_peered":"2024-04-25T01:25:53.355648+0000","last_unstale":"2024-04-25T01:26:03.716066+0000","last_undegraded":"2024-04-25T01:26:03.716066+0000","last_fullsized":"2024-04-25T01:26:03.716066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507424+0000","last_change":"2024-04-25T01:25:52.357683+0000","last_active":"2024-04-25T01:26:03.507424+0000","last_peered":"2024-04-25T01:26:03.507424+0000","last_clean":"2024-04-25T01:26:03.507424+0000","last_became_active":"2024-04-25T01:25:52.356924+0000","last_became_peered":"2024-04-25T01:25:52.356924+0000","last_unstale":"2024-04-25T01:26:03.507424+0000","last_undegraded":"2024-04-25T01:26:03.507424+0000","last_fullsized":"2024-04-25T01:26:03.507424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716612+0000","last_change":"2024-04-25T01:26:01.421260+0000","last_active":"2024-04-25T01:26:03.716612+0000","last_peered":"2024-04-25T01:26:03.716612+0000","last_clean":"2024-04-25T01:26:03.716612+0000","last_became_active":"2024-04-25T01:26:01.420831+0000","last_became_peered":"2024-04-25T01:26:01.420831+0000","last_unstale":"2024-04-25T01:26:03.716612+0000","last_undegraded":"2024-04-25T01:26:03.716612+0000","last_fullsized":"2024-04-25T01:26:03.716612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507519+0000","last_change":"2024-04-25T01:26:02.421386+0000","last_active":"2024-04-25T01:26:03.507519+0000","last_peered":"2024-04-25T01:26:03.507519+0000","last_clean":"2024-04-25T01:26:03.507519+0000","last_became_active":"2024-04-25T01:26:02.421079+0000","last_became_peered":"2024-04-25T01:26:02.421079+0000","last_unstale":"2024-04-25T01:26:03.507519+0000","last_undegraded":"2024-04-25T01:26:03.507519+0000","last_fullsized":"2024-04-25T01:26:03.507519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416038+0000","last_change":"2024-04-25T01:25:53.374719+0000","last_active":"2024-04-25T01:26:03.416038+0000","last_peered":"2024-04-25T01:26:03.416038+0000","last_clean":"2024-04-25T01:26:03.416038+0000","last_became_active":"2024-04-25T01:25:53.374425+0000","last_became_peered":"2024-04-25T01:25:53.374425+0000","last_unstale":"2024-04-25T01:26:03.416038+0000","last_undegraded":"2024-04-25T01:26:03.416038+0000","last_fullsized":"2024-04-25T01:26:03.416038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717309+0000","last_change":"2024-04-25T01:25:52.353155+0000","last_active":"2024-04-25T01:26:03.717309+0000","last_peered":"2024-04-25T01:26:03.717309+0000","last_clean":"2024-04-25T01:26:03.717309+0000","last_became_active":"2024-04-25T01:25:52.352826+0000","last_became_peered":"2024-04-25T01:25:52.352826+0000","last_unstale":"2024-04-25T01:26:03.717309+0000","last_undegraded":"2024-04-25T01:26:03.717309+0000","last_fullsized":"2024-04-25T01:26:03.717309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507554+0000","last_change":"2024-04-25T01:25:41.719670+0000","last_active":"2024-04-25T01:26:03.507554+0000","last_peered":"2024-04-25T01:26:03.507554+0000","last_clean":"2024-04-25T01:26:03.507554+0000","last_became_active":"2024-04-25T01:25:41.718721+0000","last_became_peered":"2024-04-25T01:25:41.718721+0000","last_unstale":"2024-04-25T01:26:03.507554+0000","last_undegraded":"2024-04-25T01:26:03.507554+0000","last_fullsized":"2024-04-25T01:26:03.507554+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_clean_scrub_stamp":"2024-04-25T01:25:40.688097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:27.456746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717378+0000","last_change":"2024-04-25T01:26:01.409339+0000","last_active":"2024-04-25T01:26:03.717378+0000","last_peered":"2024-04-25T01:26:03.717378+0000","last_clean":"2024-04-25T01:26:03.717378+0000","last_became_active":"2024-04-25T01:26:01.409062+0000","last_became_peered":"2024-04-25T01:26:01.409062+0000","last_unstale":"2024-04-25T01:26:03.717378+0000","last_undegraded":"2024-04-25T01:26:03.717378+0000","last_fullsized":"2024-04-25T01:26:03.717378+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:52.561251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415532+0000","last_change":"2024-04-25T01:26:02.418574+0000","last_active":"2024-04-25T01:26:03.415532+0000","last_peered":"2024-04-25T01:26:03.415532+0000","last_clean":"2024-04-25T01:26:03.415532+0000","last_became_active":"2024-04-25T01:26:02.416546+0000","last_became_peered":"2024-04-25T01:26:02.416546+0000","last_unstale":"2024-04-25T01:26:03.415532+0000","last_undegraded":"2024-04-25T01:26:03.415532+0000","last_fullsized":"2024-04-25T01:26:03.415532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:43.994141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507591+0000","last_change":"2024-04-25T01:25:53.375523+0000","last_active":"2024-04-25T01:26:03.507591+0000","last_peered":"2024-04-25T01:26:03.507591+0000","last_clean":"2024-04-25T01:26:03.507591+0000","last_became_active":"2024-04-25T01:25:53.375219+0000","last_became_peered":"2024-04-25T01:25:53.375219+0000","last_unstale":"2024-04-25T01:26:03.507591+0000","last_undegraded":"2024-04-25T01:26:03.507591+0000","last_fullsized":"2024-04-25T01:26:03.507591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.417443+0000","last_change":"2024-04-25T01:25:52.352994+0000","last_active":"2024-04-25T01:26:03.417443+0000","last_peered":"2024-04-25T01:26:03.417443+0000","last_clean":"2024-04-25T01:26:03.417443+0000","last_became_active":"2024-04-25T01:25:52.352644+0000","last_became_peered":"2024-04-25T01:25:52.352644+0000","last_unstale":"2024-04-25T01:26:03.417443+0000","last_undegraded":"2024-04-25T01:26:03.417443+0000","last_fullsized":"2024-04-25T01:26:03.417443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716482+0000","last_change":"2024-04-25T01:26:01.420891+0000","last_active":"2024-04-25T01:26:03.716482+0000","last_peered":"2024-04-25T01:26:03.716482+0000","last_clean":"2024-04-25T01:26:03.716482+0000","last_became_active":"2024-04-25T01:26:01.420664+0000","last_became_peered":"2024-04-25T01:26:01.420664+0000","last_unstale":"2024-04-25T01:26:03.716482+0000","last_undegraded":"2024-04-25T01:26:03.716482+0000","last_fullsized":"2024-04-25T01:26:03.716482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415682+0000","last_change":"2024-04-25T01:26:02.403489+0000","last_active":"2024-04-25T01:26:03.415682+0000","last_peered":"2024-04-25T01:26:03.415682+0000","last_clean":"2024-04-25T01:26:03.415682+0000","last_became_active":"2024-04-25T01:26:02.403206+0000","last_became_peered":"2024-04-25T01:26:02.403206+0000","last_unstale":"2024-04-25T01:26:03.415682+0000","last_undegraded":"2024-04-25T01:26:03.415682+0000","last_fullsized":"2024-04-25T01:26:03.415682+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:49.872754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.718041+0000","last_change":"2024-04-25T01:25:53.371018+0000","last_active":"2024-04-25T01:26:03.718041+0000","last_peered":"2024-04-25T01:26:03.718041+0000","last_clean":"2024-04-25T01:26:03.718041+0000","last_became_active":"2024-04-25T01:25:53.370539+0000","last_became_peered":"2024-04-25T01:25:53.370539+0000","last_unstale":"2024-04-25T01:26:03.718041+0000","last_undegraded":"2024-04-25T01:26:03.718041+0000","last_fullsized":"2024-04-25T01:26:03.718041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717910+0000","last_change":"2024-04-25T01:25:52.355133+0000","last_active":"2024-04-25T01:26:03.717910+0000","last_peered":"2024-04-25T01:26:03.717910+0000","last_clean":"2024-04-25T01:26:03.717910+0000","last_became_active":"2024-04-25T01:25:52.354836+0000","last_became_peered":"2024-04-25T01:25:52.354836+0000","last_unstale":"2024-04-25T01:26:03.717910+0000","last_undegraded":"2024-04-25T01:26:03.717910+0000","last_fullsized":"2024-04-25T01:26:03.717910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:10.331704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716238+0000","last_change":"2024-04-25T01:26:01.419285+0000","last_active":"2024-04-25T01:26:03.716238+0000","last_peered":"2024-04-25T01:26:03.716238+0000","last_clean":"2024-04-25T01:26:03.716238+0000","last_became_active":"2024-04-25T01:26:01.418987+0000","last_became_peered":"2024-04-25T01:26:01.418987+0000","last_unstale":"2024-04-25T01:26:03.716238+0000","last_undegraded":"2024-04-25T01:26:03.716238+0000","last_fullsized":"2024-04-25T01:26:03.716238+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.506958+0000","last_change":"2024-04-25T01:26:02.416544+0000","last_active":"2024-04-25T01:26:03.506958+0000","last_peered":"2024-04-25T01:26:03.506958+0000","last_clean":"2024-04-25T01:26:03.506958+0000","last_became_active":"2024-04-25T01:26:02.416189+0000","last_became_peered":"2024-04-25T01:26:02.416189+0000","last_unstale":"2024-04-25T01:26:03.506958+0000","last_undegraded":"2024-04-25T01:26:03.506958+0000","last_fullsized":"2024-04-25T01:26:03.506958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.688469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.508134+0000","last_change":"2024-04-25T01:25:53.370754+0000","last_active":"2024-04-25T01:26:03.508134+0000","last_peered":"2024-04-25T01:26:03.508134+0000","last_clean":"2024-04-25T01:26:03.508134+0000","last_became_active":"2024-04-25T01:25:53.370074+0000","last_became_peered":"2024-04-25T01:25:53.370074+0000","last_unstale":"2024-04-25T01:26:03.508134+0000","last_undegraded":"2024-04-25T01:26:03.508134+0000","last_fullsized":"2024-04-25T01:26:03.508134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416836+0000","last_change":"2024-04-25T01:25:52.345943+0000","last_active":"2024-04-25T01:26:03.416836+0000","last_peered":"2024-04-25T01:26:03.416836+0000","last_clean":"2024-04-25T01:26:03.416836+0000","last_became_active":"2024-04-25T01:25:52.345717+0000","last_became_peered":"2024-04-25T01:25:52.345717+0000","last_unstale":"2024-04-25T01:26:03.416836+0000","last_undegraded":"2024-04-25T01:26:03.416836+0000","last_fullsized":"2024-04-25T01:26:03.416836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716622+0000","last_change":"2024-04-25T01:26:01.416027+0000","last_active":"2024-04-25T01:26:03.716622+0000","last_peered":"2024-04-25T01:26:03.716622+0000","last_clean":"2024-04-25T01:26:03.716622+0000","last_became_active":"2024-04-25T01:26:01.415750+0000","last_became_peered":"2024-04-25T01:26:01.415750+0000","last_unstale":"2024-04-25T01:26:03.716622+0000","last_undegraded":"2024-04-25T01:26:03.716622+0000","last_fullsized":"2024-04-25T01:26:03.716622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:27.042376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416127+0000","last_change":"2024-04-25T01:26:02.414929+0000","last_active":"2024-04-25T01:26:03.416127+0000","last_peered":"2024-04-25T01:26:03.416127+0000","last_clean":"2024-04-25T01:26:03.416127+0000","last_became_active":"2024-04-25T01:26:02.414316+0000","last_became_peered":"2024-04-25T01:26:02.414316+0000","last_unstale":"2024-04-25T01:26:03.416127+0000","last_undegraded":"2024-04-25T01:26:03.416127+0000","last_fullsized":"2024-04-25T01:26:03.416127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:53.568065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507875+0000","last_change":"2024-04-25T01:25:53.365171+0000","last_active":"2024-04-25T01:26:03.507875+0000","last_peered":"2024-04-25T01:26:03.507875+0000","last_clean":"2024-04-25T01:26:03.507875+0000","last_became_active":"2024-04-25T01:25:53.364049+0000","last_became_peered":"2024-04-25T01:25:53.364049+0000","last_unstale":"2024-04-25T01:26:03.507875+0000","last_undegraded":"2024-04-25T01:26:03.507875+0000","last_fullsized":"2024-04-25T01:26:03.507875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416635+0000","last_change":"2024-04-25T01:25:52.350688+0000","last_active":"2024-04-25T01:26:03.416635+0000","last_peered":"2024-04-25T01:26:03.416635+0000","last_clean":"2024-04-25T01:26:03.416635+0000","last_became_active":"2024-04-25T01:25:52.350357+0000","last_became_peered":"2024-04-25T01:25:52.350357+0000","last_unstale":"2024-04-25T01:26:03.416635+0000","last_undegraded":"2024-04-25T01:26:03.416635+0000","last_fullsized":"2024-04-25T01:26:03.416635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415491+0000","last_change":"2024-04-25T01:26:01.418580+0000","last_active":"2024-04-25T01:26:03.415491+0000","last_peered":"2024-04-25T01:26:03.415491+0000","last_clean":"2024-04-25T01:26:03.415491+0000","last_became_active":"2024-04-25T01:26:01.418179+0000","last_became_peered":"2024-04-25T01:26:01.418179+0000","last_unstale":"2024-04-25T01:26:03.415491+0000","last_undegraded":"2024-04-25T01:26:03.415491+0000","last_fullsized":"2024-04-25T01:26:03.415491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:42.987327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416438+0000","last_change":"2024-04-25T01:26:02.418424+0000","last_active":"2024-04-25T01:26:03.416438+0000","last_peered":"2024-04-25T01:26:03.416438+0000","last_clean":"2024-04-25T01:26:03.416438+0000","last_became_active":"2024-04-25T01:26:02.418090+0000","last_became_peered":"2024-04-25T01:26:02.418090+0000","last_unstale":"2024-04-25T01:26:03.416438+0000","last_undegraded":"2024-04-25T01:26:03.416438+0000","last_fullsized":"2024-04-25T01:26:03.416438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:10.049310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507217+0000","last_change":"2024-04-25T01:25:53.366376+0000","last_active":"2024-04-25T01:26:03.507217+0000","last_peered":"2024-04-25T01:26:03.507217+0000","last_clean":"2024-04-25T01:26:03.507217+0000","last_became_active":"2024-04-25T01:25:53.365715+0000","last_became_peered":"2024-04-25T01:25:53.365715+0000","last_unstale":"2024-04-25T01:26:03.507217+0000","last_undegraded":"2024-04-25T01:26:03.507217+0000","last_fullsized":"2024-04-25T01:26:03.507217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416219+0000","last_change":"2024-04-25T01:25:52.347413+0000","last_active":"2024-04-25T01:26:03.416219+0000","last_peered":"2024-04-25T01:26:03.416219+0000","last_clean":"2024-04-25T01:26:03.416219+0000","last_became_active":"2024-04-25T01:25:52.347144+0000","last_became_peered":"2024-04-25T01:25:52.347144+0000","last_unstale":"2024-04-25T01:26:03.416219+0000","last_undegraded":"2024-04-25T01:26:03.416219+0000","last_fullsized":"2024-04-25T01:26:03.416219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716710+0000","last_change":"2024-04-25T01:26:01.409917+0000","last_active":"2024-04-25T01:26:03.716710+0000","last_peered":"2024-04-25T01:26:03.716710+0000","last_clean":"2024-04-25T01:26:03.716710+0000","last_became_active":"2024-04-25T01:26:01.409654+0000","last_became_peered":"2024-04-25T01:26:01.409654+0000","last_unstale":"2024-04-25T01:26:03.716710+0000","last_undegraded":"2024-04-25T01:26:03.716710+0000","last_fullsized":"2024-04-25T01:26:03.716710+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:37.142255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717898+0000","last_change":"2024-04-25T01:26:02.425103+0000","last_active":"2024-04-25T01:26:03.717898+0000","last_peered":"2024-04-25T01:26:03.717898+0000","last_clean":"2024-04-25T01:26:03.717898+0000","last_became_active":"2024-04-25T01:26:02.424819+0000","last_became_peered":"2024-04-25T01:26:02.424819+0000","last_unstale":"2024-04-25T01:26:03.717898+0000","last_undegraded":"2024-04-25T01:26:03.717898+0000","last_fullsized":"2024-04-25T01:26:03.717898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716406+0000","last_change":"2024-04-25T01:25:53.364987+0000","last_active":"2024-04-25T01:26:03.716406+0000","last_peered":"2024-04-25T01:26:03.716406+0000","last_clean":"2024-04-25T01:26:03.716406+0000","last_became_active":"2024-04-25T01:25:53.364431+0000","last_became_peered":"2024-04-25T01:25:53.364431+0000","last_unstale":"2024-04-25T01:26:03.716406+0000","last_undegraded":"2024-04-25T01:26:03.716406+0000","last_fullsized":"2024-04-25T01:26:03.716406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716731+0000","last_change":"2024-04-25T01:25:52.351205+0000","last_active":"2024-04-25T01:26:03.716731+0000","last_peered":"2024-04-25T01:26:03.716731+0000","last_clean":"2024-04-25T01:26:03.716731+0000","last_became_active":"2024-04-25T01:25:52.350946+0000","last_became_peered":"2024-04-25T01:25:52.350946+0000","last_unstale":"2024-04-25T01:26:03.716731+0000","last_undegraded":"2024-04-25T01:26:03.716731+0000","last_fullsized":"2024-04-25T01:26:03.716731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416482+0000","last_change":"2024-04-25T01:26:01.415799+0000","last_active":"2024-04-25T01:26:03.416482+0000","last_peered":"2024-04-25T01:26:03.416482+0000","last_clean":"2024-04-25T01:26:03.416482+0000","last_became_active":"2024-04-25T01:26:01.415572+0000","last_became_peered":"2024-04-25T01:26:01.415572+0000","last_unstale":"2024-04-25T01:26:03.416482+0000","last_undegraded":"2024-04-25T01:26:03.416482+0000","last_fullsized":"2024-04-25T01:26:03.416482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:54.034132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717227+0000","last_change":"2024-04-25T01:26:02.422959+0000","last_active":"2024-04-25T01:26:03.717227+0000","last_peered":"2024-04-25T01:26:03.717227+0000","last_clean":"2024-04-25T01:26:03.717227+0000","last_became_active":"2024-04-25T01:26:02.422210+0000","last_became_peered":"2024-04-25T01:26:02.422210+0000","last_unstale":"2024-04-25T01:26:03.717227+0000","last_undegraded":"2024-04-25T01:26:03.717227+0000","last_fullsized":"2024-04-25T01:26:03.717227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507159+0000","last_change":"2024-04-25T01:25:53.368187+0000","last_active":"2024-04-25T01:26:03.507159+0000","last_peered":"2024-04-25T01:26:03.507159+0000","last_clean":"2024-04-25T01:26:03.507159+0000","last_became_active":"2024-04-25T01:25:53.367664+0000","last_became_peered":"2024-04-25T01:25:53.367664+0000","last_unstale":"2024-04-25T01:26:03.507159+0000","last_undegraded":"2024-04-25T01:26:03.507159+0000","last_fullsized":"2024-04-25T01:26:03.507159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717680+0000","last_change":"2024-04-25T01:25:52.357397+0000","last_active":"2024-04-25T01:26:03.717680+0000","last_peered":"2024-04-25T01:26:03.717680+0000","last_clean":"2024-04-25T01:26:03.717680+0000","last_became_active":"2024-04-25T01:25:52.356881+0000","last_became_peered":"2024-04-25T01:25:52.356881+0000","last_unstale":"2024-04-25T01:26:03.717680+0000","last_undegraded":"2024-04-25T01:26:03.717680+0000","last_fullsized":"2024-04-25T01:26:03.717680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"23'6","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416132+0000","last_change":"2024-04-25T01:26:01.420103+0000","last_active":"2024-04-25T01:26:03.416132+0000","last_peered":"2024-04-25T01:26:03.416132+0000","last_clean":"2024-04-25T01:26:03.416132+0000","last_became_active":"2024-04-25T01:26:01.419526+0000","last_became_peered":"2024-04-25T01:26:01.419526+0000","last_unstale":"2024-04-25T01:26:03.416132+0000","last_undegraded":"2024-04-25T01:26:03.416132+0000","last_fullsized":"2024-04-25T01:26:03.416132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:27.042376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416191+0000","last_change":"2024-04-25T01:26:02.421488+0000","last_active":"2024-04-25T01:26:03.416191+0000","last_peered":"2024-04-25T01:26:03.416191+0000","last_clean":"2024-04-25T01:26:03.416191+0000","last_became_active":"2024-04-25T01:26:02.419308+0000","last_became_peered":"2024-04-25T01:26:02.419308+0000","last_unstale":"2024-04-25T01:26:03.416191+0000","last_undegraded":"2024-04-25T01:26:03.416191+0000","last_fullsized":"2024-04-25T01:26:03.416191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716019+0000","last_change":"2024-04-25T01:25:53.370720+0000","last_active":"2024-04-25T01:26:03.716019+0000","last_peered":"2024-04-25T01:26:03.716019+0000","last_clean":"2024-04-25T01:26:03.716019+0000","last_became_active":"2024-04-25T01:25:53.370142+0000","last_became_peered":"2024-04-25T01:25:53.370142+0000","last_unstale":"2024-04-25T01:26:03.716019+0000","last_undegraded":"2024-04-25T01:26:03.716019+0000","last_fullsized":"2024-04-25T01:26:03.716019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:52.833379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716951+0000","last_change":"2024-04-25T01:25:52.349092+0000","last_active":"2024-04-25T01:26:03.716951+0000","last_peered":"2024-04-25T01:26:03.716951+0000","last_clean":"2024-04-25T01:26:03.716951+0000","last_became_active":"2024-04-25T01:25:52.348846+0000","last_became_peered":"2024-04-25T01:25:52.348846+0000","last_unstale":"2024-04-25T01:26:03.716951+0000","last_undegraded":"2024-04-25T01:26:03.716951+0000","last_fullsized":"2024-04-25T01:26:03.716951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:21.982440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717840+0000","last_change":"2024-04-25T01:26:01.414517+0000","last_active":"2024-04-25T01:26:03.717840+0000","last_peered":"2024-04-25T01:26:03.717840+0000","last_clean":"2024-04-25T01:26:03.717840+0000","last_became_active":"2024-04-25T01:26:01.414243+0000","last_became_peered":"2024-04-25T01:26:01.414243+0000","last_unstale":"2024-04-25T01:26:03.717840+0000","last_undegraded":"2024-04-25T01:26:03.717840+0000","last_fullsized":"2024-04-25T01:26:03.717840+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:48.491461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415418+0000","last_change":"2024-04-25T01:26:02.415893+0000","last_active":"2024-04-25T01:26:03.415418+0000","last_peered":"2024-04-25T01:26:03.415418+0000","last_clean":"2024-04-25T01:26:03.415418+0000","last_became_active":"2024-04-25T01:26:02.415596+0000","last_became_peered":"2024-04-25T01:26:02.415596+0000","last_unstale":"2024-04-25T01:26:03.415418+0000","last_undegraded":"2024-04-25T01:26:03.415418+0000","last_fullsized":"2024-04-25T01:26:03.415418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507428+0000","last_change":"2024-04-25T01:25:53.373572+0000","last_active":"2024-04-25T01:26:03.507428+0000","last_peered":"2024-04-25T01:26:03.507428+0000","last_clean":"2024-04-25T01:26:03.507428+0000","last_became_active":"2024-04-25T01:25:53.373129+0000","last_became_peered":"2024-04-25T01:25:53.373129+0000","last_unstale":"2024-04-25T01:26:03.507428+0000","last_undegraded":"2024-04-25T01:26:03.507428+0000","last_fullsized":"2024-04-25T01:26:03.507428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":28,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717081+0000","last_change":"2024-04-25T01:25:52.350446+0000","last_active":"2024-04-25T01:26:03.717081+0000","last_peered":"2024-04-25T01:26:03.717081+0000","last_clean":"2024-04-25T01:26:03.717081+0000","last_became_active":"2024-04-25T01:25:52.349448+0000","last_became_peered":"2024-04-25T01:25:52.349448+0000","last_unstale":"2024-04-25T01:26:03.717081+0000","last_undegraded":"2024-04-25T01:26:03.717081+0000","last_fullsized":"2024-04-25T01:26:03.717081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716463+0000","last_change":"2024-04-25T01:26:01.418794+0000","last_active":"2024-04-25T01:26:03.716463+0000","last_peered":"2024-04-25T01:26:03.716463+0000","last_clean":"2024-04-25T01:26:03.716463+0000","last_became_active":"2024-04-25T01:26:01.418541+0000","last_became_peered":"2024-04-25T01:26:01.418541+0000","last_unstale":"2024-04-25T01:26:03.716463+0000","last_undegraded":"2024-04-25T01:26:03.716463+0000","last_fullsized":"2024-04-25T01:26:03.716463+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717164+0000","last_change":"2024-04-25T01:26:02.425391+0000","last_active":"2024-04-25T01:26:03.717164+0000","last_peered":"2024-04-25T01:26:03.717164+0000","last_clean":"2024-04-25T01:26:03.717164+0000","last_became_active":"2024-04-25T01:26:02.425106+0000","last_became_peered":"2024-04-25T01:26:02.425106+0000","last_unstale":"2024-04-25T01:26:03.717164+0000","last_undegraded":"2024-04-25T01:26:03.717164+0000","last_fullsized":"2024-04-25T01:26:03.717164+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416706+0000","last_change":"2024-04-25T01:25:53.366416+0000","last_active":"2024-04-25T01:26:03.416706+0000","last_peered":"2024-04-25T01:26:03.416706+0000","last_clean":"2024-04-25T01:26:03.416706+0000","last_became_active":"2024-04-25T01:25:53.366157+0000","last_became_peered":"2024-04-25T01:25:53.366157+0000","last_unstale":"2024-04-25T01:26:03.416706+0000","last_undegraded":"2024-04-25T01:26:03.416706+0000","last_fullsized":"2024-04-25T01:26:03.416706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:26.277566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717854+0000","last_change":"2024-04-25T01:25:52.357610+0000","last_active":"2024-04-25T01:26:03.717854+0000","last_peered":"2024-04-25T01:26:03.717854+0000","last_clean":"2024-04-25T01:26:03.717854+0000","last_became_active":"2024-04-25T01:25:52.357355+0000","last_became_peered":"2024-04-25T01:25:52.357355+0000","last_unstale":"2024-04-25T01:26:03.717854+0000","last_undegraded":"2024-04-25T01:26:03.717854+0000","last_fullsized":"2024-04-25T01:26:03.717854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416422+0000","last_change":"2024-04-25T01:26:01.412666+0000","last_active":"2024-04-25T01:26:03.416422+0000","last_peered":"2024-04-25T01:26:03.416422+0000","last_clean":"2024-04-25T01:26:03.416422+0000","last_became_active":"2024-04-25T01:26:01.412132+0000","last_became_peered":"2024-04-25T01:26:01.412132+0000","last_unstale":"2024-04-25T01:26:03.416422+0000","last_undegraded":"2024-04-25T01:26:03.416422+0000","last_fullsized":"2024-04-25T01:26:03.416422+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716704+0000","last_change":"2024-04-25T01:26:02.413106+0000","last_active":"2024-04-25T01:26:03.716704+0000","last_peered":"2024-04-25T01:26:03.716704+0000","last_clean":"2024-04-25T01:26:03.716704+0000","last_became_active":"2024-04-25T01:26:02.412759+0000","last_became_peered":"2024-04-25T01:26:02.412759+0000","last_unstale":"2024-04-25T01:26:03.716704+0000","last_undegraded":"2024-04-25T01:26:03.716704+0000","last_fullsized":"2024-04-25T01:26:03.716704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:28.049190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716160+0000","last_change":"2024-04-25T01:25:53.374311+0000","last_active":"2024-04-25T01:26:03.716160+0000","last_peered":"2024-04-25T01:26:03.716160+0000","last_clean":"2024-04-25T01:26:03.716160+0000","last_became_active":"2024-04-25T01:25:53.373890+0000","last_became_peered":"2024-04-25T01:25:53.373890+0000","last_unstale":"2024-04-25T01:26:03.716160+0000","last_undegraded":"2024-04-25T01:26:03.716160+0000","last_fullsized":"2024-04-25T01:26:03.716160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717907+0000","last_change":"2024-04-25T01:25:52.356231+0000","last_active":"2024-04-25T01:26:03.717907+0000","last_peered":"2024-04-25T01:26:03.717907+0000","last_clean":"2024-04-25T01:26:03.717907+0000","last_became_active":"2024-04-25T01:25:52.355913+0000","last_became_peered":"2024-04-25T01:25:52.355913+0000","last_unstale":"2024-04-25T01:26:03.717907+0000","last_undegraded":"2024-04-25T01:26:03.717907+0000","last_fullsized":"2024-04-25T01:26:03.717907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415065+0000","last_change":"2024-04-25T01:26:01.419513+0000","last_active":"2024-04-25T01:26:03.415065+0000","last_peered":"2024-04-25T01:26:03.415065+0000","last_clean":"2024-04-25T01:26:03.415065+0000","last_became_active":"2024-04-25T01:26:01.419015+0000","last_became_peered":"2024-04-25T01:26:01.419015+0000","last_unstale":"2024-04-25T01:26:03.415065+0000","last_undegraded":"2024-04-25T01:26:03.415065+0000","last_fullsized":"2024-04-25T01:26:03.415065+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:10.486367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507391+0000","last_change":"2024-04-25T01:26:02.425241+0000","last_active":"2024-04-25T01:26:03.507391+0000","last_peered":"2024-04-25T01:26:03.507391+0000","last_clean":"2024-04-25T01:26:03.507391+0000","last_became_active":"2024-04-25T01:26:02.424960+0000","last_became_peered":"2024-04-25T01:26:02.424960+0000","last_unstale":"2024-04-25T01:26:03.507391+0000","last_undegraded":"2024-04-25T01:26:03.507391+0000","last_fullsized":"2024-04-25T01:26:03.507391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717765+0000","last_change":"2024-04-25T01:25:53.371445+0000","last_active":"2024-04-25T01:26:03.717765+0000","last_peered":"2024-04-25T01:26:03.717765+0000","last_clean":"2024-04-25T01:26:03.717765+0000","last_became_active":"2024-04-25T01:25:53.371086+0000","last_became_peered":"2024-04-25T01:25:53.371086+0000","last_unstale":"2024-04-25T01:26:03.717765+0000","last_undegraded":"2024-04-25T01:26:03.717765+0000","last_fullsized":"2024-04-25T01:26:03.717765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.718056+0000","last_change":"2024-04-25T01:25:52.357459+0000","last_active":"2024-04-25T01:26:03.718056+0000","last_peered":"2024-04-25T01:26:03.718056+0000","last_clean":"2024-04-25T01:26:03.718056+0000","last_became_active":"2024-04-25T01:25:52.357182+0000","last_became_peered":"2024-04-25T01:25:52.357182+0000","last_unstale":"2024-04-25T01:26:03.718056+0000","last_undegraded":"2024-04-25T01:26:03.718056+0000","last_fullsized":"2024-04-25T01:26:03.718056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415347+0000","last_change":"2024-04-25T01:26:01.415321+0000","last_active":"2024-04-25T01:26:03.415347+0000","last_peered":"2024-04-25T01:26:03.415347+0000","last_clean":"2024-04-25T01:26:03.415347+0000","last_became_active":"2024-04-25T01:26:01.415020+0000","last_became_peered":"2024-04-25T01:26:01.415020+0000","last_unstale":"2024-04-25T01:26:03.415347+0000","last_undegraded":"2024-04-25T01:26:03.415347+0000","last_fullsized":"2024-04-25T01:26:03.415347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:20.090411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717822+0000","last_change":"2024-04-25T01:26:02.416917+0000","last_active":"2024-04-25T01:26:03.717822+0000","last_peered":"2024-04-25T01:26:03.717822+0000","last_clean":"2024-04-25T01:26:03.717822+0000","last_became_active":"2024-04-25T01:26:02.416623+0000","last_became_peered":"2024-04-25T01:26:02.416623+0000","last_unstale":"2024-04-25T01:26:03.717822+0000","last_undegraded":"2024-04-25T01:26:03.717822+0000","last_fullsized":"2024-04-25T01:26:03.717822+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:55.040946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716446+0000","last_change":"2024-04-25T01:26:01.415472+0000","last_active":"2024-04-25T01:26:03.716446+0000","last_peered":"2024-04-25T01:26:03.716446+0000","last_clean":"2024-04-25T01:26:03.716446+0000","last_became_active":"2024-04-25T01:26:01.415184+0000","last_became_peered":"2024-04-25T01:26:01.415184+0000","last_unstale":"2024-04-25T01:26:03.716446+0000","last_undegraded":"2024-04-25T01:26:03.716446+0000","last_fullsized":"2024-04-25T01:26:03.716446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716757+0000","last_change":"2024-04-25T01:25:53.363043+0000","last_active":"2024-04-25T01:26:03.716757+0000","last_peered":"2024-04-25T01:26:03.716757+0000","last_clean":"2024-04-25T01:26:03.716757+0000","last_became_active":"2024-04-25T01:25:53.362601+0000","last_became_peered":"2024-04-25T01:25:53.362601+0000","last_unstale":"2024-04-25T01:26:03.716757+0000","last_undegraded":"2024-04-25T01:26:03.716757+0000","last_fullsized":"2024-04-25T01:26:03.716757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716830+0000","last_change":"2024-04-25T01:25:52.358081+0000","last_active":"2024-04-25T01:26:03.716830+0000","last_peered":"2024-04-25T01:26:03.716830+0000","last_clean":"2024-04-25T01:26:03.716830+0000","last_became_active":"2024-04-25T01:25:52.357365+0000","last_became_peered":"2024-04-25T01:25:52.357365+0000","last_unstale":"2024-04-25T01:26:03.716830+0000","last_undegraded":"2024-04-25T01:26:03.716830+0000","last_fullsized":"2024-04-25T01:26:03.716830+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716855+0000","last_change":"2024-04-25T01:26:02.418872+0000","last_active":"2024-04-25T01:26:03.716855+0000","last_peered":"2024-04-25T01:26:03.716855+0000","last_clean":"2024-04-25T01:26:03.716855+0000","last_became_active":"2024-04-25T01:26:02.418614+0000","last_became_peered":"2024-04-25T01:26:02.418614+0000","last_unstale":"2024-04-25T01:26:03.716855+0000","last_undegraded":"2024-04-25T01:26:03.716855+0000","last_fullsized":"2024-04-25T01:26:03.716855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:49.498275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717818+0000","last_change":"2024-04-25T01:26:01.410258+0000","last_active":"2024-04-25T01:26:03.717818+0000","last_peered":"2024-04-25T01:26:03.717818+0000","last_clean":"2024-04-25T01:26:03.717818+0000","last_became_active":"2024-04-25T01:26:01.410004+0000","last_became_peered":"2024-04-25T01:26:01.410004+0000","last_unstale":"2024-04-25T01:26:03.717818+0000","last_undegraded":"2024-04-25T01:26:03.717818+0000","last_fullsized":"2024-04-25T01:26:03.717818+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:48.865940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415284+0000","last_change":"2024-04-25T01:25:53.363794+0000","last_active":"2024-04-25T01:26:03.415284+0000","last_peered":"2024-04-25T01:26:03.415284+0000","last_clean":"2024-04-25T01:26:03.415284+0000","last_became_active":"2024-04-25T01:25:53.363509+0000","last_became_peered":"2024-04-25T01:25:53.363509+0000","last_unstale":"2024-04-25T01:26:03.415284+0000","last_undegraded":"2024-04-25T01:26:03.415284+0000","last_fullsized":"2024-04-25T01:26:03.415284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415685+0000","last_change":"2024-04-25T01:25:52.347464+0000","last_active":"2024-04-25T01:26:03.415685+0000","last_peered":"2024-04-25T01:26:03.415685+0000","last_clean":"2024-04-25T01:26:03.415685+0000","last_became_active":"2024-04-25T01:25:52.347138+0000","last_became_peered":"2024-04-25T01:25:52.347138+0000","last_unstale":"2024-04-25T01:26:03.415685+0000","last_undegraded":"2024-04-25T01:26:03.415685+0000","last_fullsized":"2024-04-25T01:26:03.415685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717780+0000","last_change":"2024-04-25T01:26:02.425948+0000","last_active":"2024-04-25T01:26:03.717780+0000","last_peered":"2024-04-25T01:26:03.717780+0000","last_clean":"2024-04-25T01:26:03.717780+0000","last_became_active":"2024-04-25T01:26:02.424947+0000","last_became_peered":"2024-04-25T01:26:02.424947+0000","last_unstale":"2024-04-25T01:26:03.717780+0000","last_undegraded":"2024-04-25T01:26:03.717780+0000","last_fullsized":"2024-04-25T01:26:03.717780+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716700+0000","last_change":"2024-04-25T01:25:52.349989+0000","last_active":"2024-04-25T01:26:03.716700+0000","last_peered":"2024-04-25T01:26:03.716700+0000","last_clean":"2024-04-25T01:26:03.716700+0000","last_became_active":"2024-04-25T01:25:52.349672+0000","last_became_peered":"2024-04-25T01:25:52.349672+0000","last_unstale":"2024-04-25T01:26:03.716700+0000","last_undegraded":"2024-04-25T01:26:03.716700+0000","last_fullsized":"2024-04-25T01:26:03.716700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716805+0000","last_change":"2024-04-25T01:25:53.356610+0000","last_active":"2024-04-25T01:26:03.716805+0000","last_peered":"2024-04-25T01:26:03.716805+0000","last_clean":"2024-04-25T01:26:03.716805+0000","last_became_active":"2024-04-25T01:25:53.356015+0000","last_became_peered":"2024-04-25T01:25:53.356015+0000","last_unstale":"2024-04-25T01:26:03.716805+0000","last_undegraded":"2024-04-25T01:26:03.716805+0000","last_fullsized":"2024-04-25T01:26:03.716805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.417192+0000","last_change":"2024-04-25T01:26:01.420261+0000","last_active":"2024-04-25T01:26:03.417192+0000","last_peered":"2024-04-25T01:26:03.417192+0000","last_clean":"2024-04-25T01:26:03.417192+0000","last_became_active":"2024-04-25T01:26:01.420003+0000","last_became_peered":"2024-04-25T01:26:01.420003+0000","last_unstale":"2024-04-25T01:26:03.417192+0000","last_undegraded":"2024-04-25T01:26:03.417192+0000","last_fullsized":"2024-04-25T01:26:03.417192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:28.695853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416039+0000","last_change":"2024-04-25T01:26:02.426327+0000","last_active":"2024-04-25T01:26:03.416039+0000","last_peered":"2024-04-25T01:26:03.416039+0000","last_clean":"2024-04-25T01:26:03.416039+0000","last_became_active":"2024-04-25T01:26:02.426054+0000","last_became_peered":"2024-04-25T01:26:02.426054+0000","last_unstale":"2024-04-25T01:26:03.416039+0000","last_undegraded":"2024-04-25T01:26:03.416039+0000","last_fullsized":"2024-04-25T01:26:03.416039+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:17.809738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415686+0000","last_change":"2024-04-25T01:25:52.353955+0000","last_active":"2024-04-25T01:26:03.415686+0000","last_peered":"2024-04-25T01:26:03.415686+0000","last_clean":"2024-04-25T01:26:03.415686+0000","last_became_active":"2024-04-25T01:25:52.353659+0000","last_became_peered":"2024-04-25T01:25:52.353659+0000","last_unstale":"2024-04-25T01:26:03.415686+0000","last_undegraded":"2024-04-25T01:26:03.415686+0000","last_fullsized":"2024-04-25T01:26:03.415686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416142+0000","last_change":"2024-04-25T01:25:53.366586+0000","last_active":"2024-04-25T01:26:03.416142+0000","last_peered":"2024-04-25T01:26:03.416142+0000","last_clean":"2024-04-25T01:26:03.416142+0000","last_became_active":"2024-04-25T01:25:53.366302+0000","last_became_peered":"2024-04-25T01:25:53.366302+0000","last_unstale":"2024-04-25T01:26:03.416142+0000","last_undegraded":"2024-04-25T01:26:03.416142+0000","last_fullsized":"2024-04-25T01:26:03.416142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416415+0000","last_change":"2024-04-25T01:26:01.420960+0000","last_active":"2024-04-25T01:26:03.416415+0000","last_peered":"2024-04-25T01:26:03.416415+0000","last_clean":"2024-04-25T01:26:03.416415+0000","last_became_active":"2024-04-25T01:26:01.420713+0000","last_became_peered":"2024-04-25T01:26:01.420713+0000","last_unstale":"2024-04-25T01:26:03.416415+0000","last_undegraded":"2024-04-25T01:26:03.416415+0000","last_fullsized":"2024-04-25T01:26:03.416415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:29.489275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416652+0000","last_change":"2024-04-25T01:26:02.419282+0000","last_active":"2024-04-25T01:26:03.416652+0000","last_peered":"2024-04-25T01:26:03.416652+0000","last_clean":"2024-04-25T01:26:03.416652+0000","last_became_active":"2024-04-25T01:26:02.418802+0000","last_became_peered":"2024-04-25T01:26:02.418802+0000","last_unstale":"2024-04-25T01:26:03.416652+0000","last_undegraded":"2024-04-25T01:26:03.416652+0000","last_fullsized":"2024-04-25T01:26:03.416652+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:46.807175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716343+0000","last_change":"2024-04-25T01:25:53.367143+0000","last_active":"2024-04-25T01:26:03.716343+0000","last_peered":"2024-04-25T01:26:03.716343+0000","last_clean":"2024-04-25T01:26:03.716343+0000","last_became_active":"2024-04-25T01:25:53.366847+0000","last_became_peered":"2024-04-25T01:25:53.366847+0000","last_unstale":"2024-04-25T01:26:03.716343+0000","last_undegraded":"2024-04-25T01:26:03.716343+0000","last_fullsized":"2024-04-25T01:26:03.716343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716783+0000","last_change":"2024-04-25T01:26:02.416438+0000","last_active":"2024-04-25T01:26:03.716783+0000","last_peered":"2024-04-25T01:26:03.716783+0000","last_clean":"2024-04-25T01:26:03.716783+0000","last_became_active":"2024-04-25T01:26:02.416170+0000","last_became_peered":"2024-04-25T01:26:02.416170+0000","last_unstale":"2024-04-25T01:26:03.716783+0000","last_undegraded":"2024-04-25T01:26:03.716783+0000","last_fullsized":"2024-04-25T01:26:03.716783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415637+0000","last_change":"2024-04-25T01:25:53.371345+0000","last_active":"2024-04-25T01:26:03.415637+0000","last_peered":"2024-04-25T01:26:03.415637+0000","last_clean":"2024-04-25T01:26:03.415637+0000","last_became_active":"2024-04-25T01:25:53.371080+0000","last_became_peered":"2024-04-25T01:25:53.371080+0000","last_unstale":"2024-04-25T01:26:03.415637+0000","last_undegraded":"2024-04-25T01:26:03.415637+0000","last_fullsized":"2024-04-25T01:26:03.415637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:09.855763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717353+0000","last_change":"2024-04-25T01:26:02.419688+0000","last_active":"2024-04-25T01:26:03.717353+0000","last_peered":"2024-04-25T01:26:03.717353+0000","last_clean":"2024-04-25T01:26:03.717353+0000","last_became_active":"2024-04-25T01:26:02.419417+0000","last_became_peered":"2024-04-25T01:26:02.419417+0000","last_unstale":"2024-04-25T01:26:03.717353+0000","last_undegraded":"2024-04-25T01:26:03.717353+0000","last_fullsized":"2024-04-25T01:26:03.717353+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717442+0000","last_change":"2024-04-25T01:25:53.369850+0000","last_active":"2024-04-25T01:26:03.717442+0000","last_peered":"2024-04-25T01:26:03.717442+0000","last_clean":"2024-04-25T01:26:03.717442+0000","last_became_active":"2024-04-25T01:25:53.369350+0000","last_became_peered":"2024-04-25T01:25:53.369350+0000","last_unstale":"2024-04-25T01:26:03.717442+0000","last_undegraded":"2024-04-25T01:26:03.717442+0000","last_fullsized":"2024-04-25T01:26:03.717442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717742+0000","last_change":"2024-04-25T01:26:02.425528+0000","last_active":"2024-04-25T01:26:03.717742+0000","last_peered":"2024-04-25T01:26:03.717742+0000","last_clean":"2024-04-25T01:26:03.717742+0000","last_became_active":"2024-04-25T01:26:02.425160+0000","last_became_peered":"2024-04-25T01:26:02.425160+0000","last_unstale":"2024-04-25T01:26:03.717742+0000","last_undegraded":"2024-04-25T01:26:03.717742+0000","last_fullsized":"2024-04-25T01:26:03.717742+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716710+0000","last_change":"2024-04-25T01:25:53.357142+0000","last_active":"2024-04-25T01:26:03.716710+0000","last_peered":"2024-04-25T01:26:03.716710+0000","last_clean":"2024-04-25T01:26:03.716710+0000","last_became_active":"2024-04-25T01:25:53.356854+0000","last_became_peered":"2024-04-25T01:25:53.356854+0000","last_unstale":"2024-04-25T01:26:03.716710+0000","last_undegraded":"2024-04-25T01:26:03.716710+0000","last_fullsized":"2024-04-25T01:26:03.716710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507419+0000","last_change":"2024-04-25T01:26:02.417492+0000","last_active":"2024-04-25T01:26:03.507419+0000","last_peered":"2024-04-25T01:26:03.507419+0000","last_clean":"2024-04-25T01:26:03.507419+0000","last_became_active":"2024-04-25T01:26:02.417234+0000","last_became_peered":"2024-04-25T01:26:02.417234+0000","last_unstale":"2024-04-25T01:26:03.507419+0000","last_undegraded":"2024-04-25T01:26:03.507419+0000","last_fullsized":"2024-04-25T01:26:03.507419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416287+0000","last_change":"2024-04-25T01:25:53.364575+0000","last_active":"2024-04-25T01:26:03.416287+0000","last_peered":"2024-04-25T01:26:03.416287+0000","last_clean":"2024-04-25T01:26:03.416287+0000","last_became_active":"2024-04-25T01:25:53.364311+0000","last_became_peered":"2024-04-25T01:25:53.364311+0000","last_unstale":"2024-04-25T01:26:03.416287+0000","last_undegraded":"2024-04-25T01:26:03.416287+0000","last_fullsized":"2024-04-25T01:26:03.416287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415942+0000","last_change":"2024-04-25T01:26:02.426813+0000","last_active":"2024-04-25T01:26:03.415942+0000","last_peered":"2024-04-25T01:26:03.415942+0000","last_clean":"2024-04-25T01:26:03.415942+0000","last_became_active":"2024-04-25T01:26:02.426552+0000","last_became_peered":"2024-04-25T01:26:02.426552+0000","last_unstale":"2024-04-25T01:26:03.415942+0000","last_undegraded":"2024-04-25T01:26:03.415942+0000","last_fullsized":"2024-04-25T01:26:03.415942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:03.687174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717821+0000","last_change":"2024-04-25T01:25:53.368413+0000","last_active":"2024-04-25T01:26:03.717821+0000","last_peered":"2024-04-25T01:26:03.717821+0000","last_clean":"2024-04-25T01:26:03.717821+0000","last_became_active":"2024-04-25T01:25:53.368176+0000","last_became_peered":"2024-04-25T01:25:53.368176+0000","last_unstale":"2024-04-25T01:26:03.717821+0000","last_undegraded":"2024-04-25T01:26:03.717821+0000","last_fullsized":"2024-04-25T01:26:03.717821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.506869+0000","last_change":"2024-04-25T01:26:02.425185+0000","last_active":"2024-04-25T01:26:03.506869+0000","last_peered":"2024-04-25T01:26:03.506869+0000","last_clean":"2024-04-25T01:26:03.506869+0000","last_became_active":"2024-04-25T01:26:02.424903+0000","last_became_peered":"2024-04-25T01:26:02.424903+0000","last_unstale":"2024-04-25T01:26:03.506869+0000","last_undegraded":"2024-04-25T01:26:03.506869+0000","last_fullsized":"2024-04-25T01:26:03.506869+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717561+0000","last_change":"2024-04-25T01:25:53.372638+0000","last_active":"2024-04-25T01:26:03.717561+0000","last_peered":"2024-04-25T01:26:03.717561+0000","last_clean":"2024-04-25T01:26:03.717561+0000","last_became_active":"2024-04-25T01:25:53.372001+0000","last_became_peered":"2024-04-25T01:25:53.372001+0000","last_unstale":"2024-04-25T01:26:03.717561+0000","last_undegraded":"2024-04-25T01:26:03.717561+0000","last_fullsized":"2024-04-25T01:26:03.717561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716617+0000","last_change":"2024-04-25T01:26:02.420867+0000","last_active":"2024-04-25T01:26:03.716617+0000","last_peered":"2024-04-25T01:26:03.716617+0000","last_clean":"2024-04-25T01:26:03.716617+0000","last_became_active":"2024-04-25T01:26:02.420621+0000","last_became_peered":"2024-04-25T01:26:02.420621+0000","last_unstale":"2024-04-25T01:26:03.716617+0000","last_undegraded":"2024-04-25T01:26:03.716617+0000","last_fullsized":"2024-04-25T01:26:03.716617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:11.493181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416565+0000","last_change":"2024-04-25T01:25:53.365509+0000","last_active":"2024-04-25T01:26:03.416565+0000","last_peered":"2024-04-25T01:26:03.416565+0000","last_clean":"2024-04-25T01:26:03.416565+0000","last_became_active":"2024-04-25T01:25:53.365245+0000","last_became_peered":"2024-04-25T01:25:53.365245+0000","last_unstale":"2024-04-25T01:26:03.416565+0000","last_undegraded":"2024-04-25T01:26:03.416565+0000","last_fullsized":"2024-04-25T01:26:03.416565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416549+0000","last_change":"2024-04-25T01:26:02.415672+0000","last_active":"2024-04-25T01:26:03.416549+0000","last_peered":"2024-04-25T01:26:03.416549+0000","last_clean":"2024-04-25T01:26:03.416549+0000","last_became_active":"2024-04-25T01:26:02.415389+0000","last_became_peered":"2024-04-25T01:26:02.415389+0000","last_unstale":"2024-04-25T01:26:03.416549+0000","last_undegraded":"2024-04-25T01:26:03.416549+0000","last_fullsized":"2024-04-25T01:26:03.416549+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:52.342053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716741+0000","last_change":"2024-04-25T01:25:53.365391+0000","last_active":"2024-04-25T01:26:03.716741+0000","last_peered":"2024-04-25T01:26:03.716741+0000","last_clean":"2024-04-25T01:26:03.716741+0000","last_became_active":"2024-04-25T01:25:53.364829+0000","last_became_peered":"2024-04-25T01:25:53.364829+0000","last_unstale":"2024-04-25T01:26:03.716741+0000","last_undegraded":"2024-04-25T01:26:03.716741+0000","last_fullsized":"2024-04-25T01:26:03.716741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507955+0000","last_change":"2024-04-25T01:26:02.418292+0000","last_active":"2024-04-25T01:26:03.507955+0000","last_peered":"2024-04-25T01:26:03.507955+0000","last_clean":"2024-04-25T01:26:03.507955+0000","last_became_active":"2024-04-25T01:26:02.418036+0000","last_became_peered":"2024-04-25T01:26:02.418036+0000","last_unstale":"2024-04-25T01:26:03.507955+0000","last_undegraded":"2024-04-25T01:26:03.507955+0000","last_fullsized":"2024-04-25T01:26:03.507955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415608+0000","last_change":"2024-04-25T01:25:53.355667+0000","last_active":"2024-04-25T01:26:03.415608+0000","last_peered":"2024-04-25T01:26:03.415608+0000","last_clean":"2024-04-25T01:26:03.415608+0000","last_became_active":"2024-04-25T01:25:53.355417+0000","last_became_peered":"2024-04-25T01:25:53.355417+0000","last_unstale":"2024-04-25T01:26:03.415608+0000","last_undegraded":"2024-04-25T01:26:03.415608+0000","last_fullsized":"2024-04-25T01:26:03.415608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415307+0000","last_change":"2024-04-25T01:26:02.414994+0000","last_active":"2024-04-25T01:26:03.415307+0000","last_peered":"2024-04-25T01:26:03.415307+0000","last_clean":"2024-04-25T01:26:03.415307+0000","last_became_active":"2024-04-25T01:26:02.414735+0000","last_became_peered":"2024-04-25T01:26:02.414735+0000","last_unstale":"2024-04-25T01:26:03.415307+0000","last_undegraded":"2024-04-25T01:26:03.415307+0000","last_fullsized":"2024-04-25T01:26:03.415307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:44.960251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717184+0000","last_change":"2024-04-25T01:25:53.371240+0000","last_active":"2024-04-25T01:26:03.717184+0000","last_peered":"2024-04-25T01:26:03.717184+0000","last_clean":"2024-04-25T01:26:03.717184+0000","last_became_active":"2024-04-25T01:25:53.370164+0000","last_became_peered":"2024-04-25T01:25:53.370164+0000","last_unstale":"2024-04-25T01:26:03.717184+0000","last_undegraded":"2024-04-25T01:26:03.717184+0000","last_fullsized":"2024-04-25T01:26:03.717184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416367+0000","last_change":"2024-04-25T01:26:02.418272+0000","last_active":"2024-04-25T01:26:03.416367+0000","last_peered":"2024-04-25T01:26:03.416367+0000","last_clean":"2024-04-25T01:26:03.416367+0000","last_became_active":"2024-04-25T01:26:02.417977+0000","last_became_peered":"2024-04-25T01:26:02.417977+0000","last_unstale":"2024-04-25T01:26:03.416367+0000","last_undegraded":"2024-04-25T01:26:03.416367+0000","last_fullsized":"2024-04-25T01:26:03.416367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716511+0000","last_change":"2024-04-25T01:25:53.376415+0000","last_active":"2024-04-25T01:26:03.716511+0000","last_peered":"2024-04-25T01:26:03.716511+0000","last_clean":"2024-04-25T01:26:03.716511+0000","last_became_active":"2024-04-25T01:25:53.376104+0000","last_became_peered":"2024-04-25T01:25:53.376104+0000","last_unstale":"2024-04-25T01:26:03.716511+0000","last_undegraded":"2024-04-25T01:26:03.716511+0000","last_fullsized":"2024-04-25T01:26:03.716511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416512+0000","last_change":"2024-04-25T01:26:02.426333+0000","last_active":"2024-04-25T01:26:03.416512+0000","last_peered":"2024-04-25T01:26:03.416512+0000","last_clean":"2024-04-25T01:26:03.416512+0000","last_became_active":"2024-04-25T01:26:02.426069+0000","last_became_peered":"2024-04-25T01:26:02.426069+0000","last_unstale":"2024-04-25T01:26:03.416512+0000","last_undegraded":"2024-04-25T01:26:03.416512+0000","last_fullsized":"2024-04-25T01:26:03.416512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716975+0000","last_change":"2024-04-25T01:25:53.370314+0000","last_active":"2024-04-25T01:26:03.716975+0000","last_peered":"2024-04-25T01:26:03.716975+0000","last_clean":"2024-04-25T01:26:03.716975+0000","last_became_active":"2024-04-25T01:25:53.370049+0000","last_became_peered":"2024-04-25T01:25:53.370049+0000","last_unstale":"2024-04-25T01:26:03.716975+0000","last_undegraded":"2024-04-25T01:26:03.716975+0000","last_fullsized":"2024-04-25T01:26:03.716975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415866+0000","last_change":"2024-04-25T01:26:02.415464+0000","last_active":"2024-04-25T01:26:03.415866+0000","last_peered":"2024-04-25T01:26:03.415866+0000","last_clean":"2024-04-25T01:26:03.415866+0000","last_became_active":"2024-04-25T01:26:02.415173+0000","last_became_peered":"2024-04-25T01:26:02.415173+0000","last_unstale":"2024-04-25T01:26:03.415866+0000","last_undegraded":"2024-04-25T01:26:03.415866+0000","last_fullsized":"2024-04-25T01:26:03.415866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.343169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507691+0000","last_change":"2024-04-25T01:25:53.369585+0000","last_active":"2024-04-25T01:26:03.507691+0000","last_peered":"2024-04-25T01:26:03.507691+0000","last_clean":"2024-04-25T01:26:03.507691+0000","last_became_active":"2024-04-25T01:25:53.368499+0000","last_became_peered":"2024-04-25T01:25:53.368499+0000","last_unstale":"2024-04-25T01:26:03.507691+0000","last_undegraded":"2024-04-25T01:26:03.507691+0000","last_fullsized":"2024-04-25T01:26:03.507691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716525+0000","last_change":"2024-04-25T01:26:02.421975+0000","last_active":"2024-04-25T01:26:03.716525+0000","last_peered":"2024-04-25T01:26:03.716525+0000","last_clean":"2024-04-25T01:26:03.716525+0000","last_became_active":"2024-04-25T01:26:02.421698+0000","last_became_peered":"2024-04-25T01:26:02.421698+0000","last_unstale":"2024-04-25T01:26:03.716525+0000","last_undegraded":"2024-04-25T01:26:03.716525+0000","last_fullsized":"2024-04-25T01:26:03.716525+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416746+0000","last_change":"2024-04-25T01:25:53.370666+0000","last_active":"2024-04-25T01:26:03.416746+0000","last_peered":"2024-04-25T01:26:03.416746+0000","last_clean":"2024-04-25T01:26:03.416746+0000","last_became_active":"2024-04-25T01:25:53.369733+0000","last_became_peered":"2024-04-25T01:25:53.369733+0000","last_unstale":"2024-04-25T01:26:03.416746+0000","last_undegraded":"2024-04-25T01:26:03.416746+0000","last_fullsized":"2024-04-25T01:26:03.416746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.715962+0000","last_change":"2024-04-25T01:26:02.418114+0000","last_active":"2024-04-25T01:26:03.715962+0000","last_peered":"2024-04-25T01:26:03.715962+0000","last_clean":"2024-04-25T01:26:03.715962+0000","last_became_active":"2024-04-25T01:26:02.417810+0000","last_became_peered":"2024-04-25T01:26:02.417810+0000","last_unstale":"2024-04-25T01:26:03.715962+0000","last_undegraded":"2024-04-25T01:26:03.715962+0000","last_fullsized":"2024-04-25T01:26:03.715962+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:30.496089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.718049+0000","last_change":"2024-04-25T01:25:53.372048+0000","last_active":"2024-04-25T01:26:03.718049+0000","last_peered":"2024-04-25T01:26:03.718049+0000","last_clean":"2024-04-25T01:26:03.718049+0000","last_became_active":"2024-04-25T01:25:53.371767+0000","last_became_peered":"2024-04-25T01:25:53.371767+0000","last_unstale":"2024-04-25T01:26:03.718049+0000","last_undegraded":"2024-04-25T01:26:03.718049+0000","last_fullsized":"2024-04-25T01:26:03.718049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:05.589744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716672+0000","last_change":"2024-04-25T01:26:02.426289+0000","last_active":"2024-04-25T01:26:03.716672+0000","last_peered":"2024-04-25T01:26:03.716672+0000","last_clean":"2024-04-25T01:26:03.716672+0000","last_became_active":"2024-04-25T01:26:02.426015+0000","last_became_peered":"2024-04-25T01:26:02.426015+0000","last_unstale":"2024-04-25T01:26:03.716672+0000","last_undegraded":"2024-04-25T01:26:03.716672+0000","last_fullsized":"2024-04-25T01:26:03.716672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716678+0000","last_change":"2024-04-25T01:25:53.374186+0000","last_active":"2024-04-25T01:26:03.716678+0000","last_peered":"2024-04-25T01:26:03.716678+0000","last_clean":"2024-04-25T01:26:03.716678+0000","last_became_active":"2024-04-25T01:25:53.373766+0000","last_became_peered":"2024-04-25T01:25:53.373766+0000","last_unstale":"2024-04-25T01:26:03.716678+0000","last_undegraded":"2024-04-25T01:26:03.716678+0000","last_fullsized":"2024-04-25T01:26:03.716678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416320+0000","last_change":"2024-04-25T01:26:02.416345+0000","last_active":"2024-04-25T01:26:03.416320+0000","last_peered":"2024-04-25T01:26:03.416320+0000","last_clean":"2024-04-25T01:26:03.416320+0000","last_became_active":"2024-04-25T01:26:02.416082+0000","last_became_peered":"2024-04-25T01:26:02.416082+0000","last_unstale":"2024-04-25T01:26:03.416320+0000","last_undegraded":"2024-04-25T01:26:03.416320+0000","last_fullsized":"2024-04-25T01:26:03.416320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:47.810943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716847+0000","last_change":"2024-04-25T01:25:53.371602+0000","last_active":"2024-04-25T01:26:03.716847+0000","last_peered":"2024-04-25T01:26:03.716847+0000","last_clean":"2024-04-25T01:26:03.716847+0000","last_became_active":"2024-04-25T01:25:53.371362+0000","last_became_peered":"2024-04-25T01:25:53.371362+0000","last_unstale":"2024-04-25T01:26:03.716847+0000","last_undegraded":"2024-04-25T01:26:03.716847+0000","last_fullsized":"2024-04-25T01:26:03.716847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717204+0000","last_change":"2024-04-25T01:26:02.425888+0000","last_active":"2024-04-25T01:26:03.717204+0000","last_peered":"2024-04-25T01:26:03.717204+0000","last_clean":"2024-04-25T01:26:03.717204+0000","last_became_active":"2024-04-25T01:26:02.425577+0000","last_became_peered":"2024-04-25T01:26:02.425577+0000","last_unstale":"2024-04-25T01:26:03.717204+0000","last_undegraded":"2024-04-25T01:26:03.717204+0000","last_fullsized":"2024-04-25T01:26:03.717204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.715963+0000","last_change":"2024-04-25T01:25:53.370996+0000","last_active":"2024-04-25T01:26:03.715963+0000","last_peered":"2024-04-25T01:26:03.715963+0000","last_clean":"2024-04-25T01:26:03.715963+0000","last_became_active":"2024-04-25T01:25:53.370479+0000","last_became_peered":"2024-04-25T01:25:53.370479+0000","last_unstale":"2024-04-25T01:26:03.715963+0000","last_undegraded":"2024-04-25T01:26:03.715963+0000","last_fullsized":"2024-04-25T01:26:03.715963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:57.636067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507529+0000","last_change":"2024-04-25T01:26:02.423420+0000","last_active":"2024-04-25T01:26:03.507529+0000","last_peered":"2024-04-25T01:26:03.507529+0000","last_clean":"2024-04-25T01:26:03.507529+0000","last_became_active":"2024-04-25T01:26:02.423146+0000","last_became_peered":"2024-04-25T01:26:02.423146+0000","last_unstale":"2024-04-25T01:26:03.507529+0000","last_undegraded":"2024-04-25T01:26:03.507529+0000","last_fullsized":"2024-04-25T01:26:03.507529+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415469+0000","last_change":"2024-04-25T01:25:53.363512+0000","last_active":"2024-04-25T01:26:03.415469+0000","last_peered":"2024-04-25T01:26:03.415469+0000","last_clean":"2024-04-25T01:26:03.415469+0000","last_became_active":"2024-04-25T01:25:53.363246+0000","last_became_peered":"2024-04-25T01:25:53.363246+0000","last_unstale":"2024-04-25T01:26:03.415469+0000","last_undegraded":"2024-04-25T01:26:03.415469+0000","last_fullsized":"2024-04-25T01:26:03.415469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415970+0000","last_change":"2024-04-25T01:26:02.413346+0000","last_active":"2024-04-25T01:26:03.415970+0000","last_peered":"2024-04-25T01:26:03.415970+0000","last_clean":"2024-04-25T01:26:03.415970+0000","last_became_active":"2024-04-25T01:26:02.413036+0000","last_became_peered":"2024-04-25T01:26:02.413036+0000","last_unstale":"2024-04-25T01:26:03.415970+0000","last_undegraded":"2024-04-25T01:26:03.415970+0000","last_fullsized":"2024-04-25T01:26:03.415970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416642+0000","last_change":"2024-04-25T01:25:53.364457+0000","last_active":"2024-04-25T01:26:03.416642+0000","last_peered":"2024-04-25T01:26:03.416642+0000","last_clean":"2024-04-25T01:26:03.416642+0000","last_became_active":"2024-04-25T01:25:53.364103+0000","last_became_peered":"2024-04-25T01:25:53.364103+0000","last_unstale":"2024-04-25T01:26:03.416642+0000","last_undegraded":"2024-04-25T01:26:03.416642+0000","last_fullsized":"2024-04-25T01:26:03.416642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:11.336752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415365+0000","last_change":"2024-04-25T01:26:02.418792+0000","last_active":"2024-04-25T01:26:03.415365+0000","last_peered":"2024-04-25T01:26:03.415365+0000","last_clean":"2024-04-25T01:26:03.415365+0000","last_became_active":"2024-04-25T01:26:02.416942+0000","last_became_peered":"2024-04-25T01:26:02.416942+0000","last_unstale":"2024-04-25T01:26:03.415365+0000","last_undegraded":"2024-04-25T01:26:03.415365+0000","last_fullsized":"2024-04-25T01:26:03.415365+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:59.492678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717288+0000","last_change":"2024-04-25T01:25:53.371647+0000","last_active":"2024-04-25T01:26:03.717288+0000","last_peered":"2024-04-25T01:26:03.717288+0000","last_clean":"2024-04-25T01:26:03.717288+0000","last_became_active":"2024-04-25T01:25:53.371353+0000","last_became_peered":"2024-04-25T01:25:53.371353+0000","last_unstale":"2024-04-25T01:26:03.717288+0000","last_undegraded":"2024-04-25T01:26:03.717288+0000","last_fullsized":"2024-04-25T01:26:03.717288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507578+0000","last_change":"2024-04-25T01:26:02.426335+0000","last_active":"2024-04-25T01:26:03.507578+0000","last_peered":"2024-04-25T01:26:03.507578+0000","last_clean":"2024-04-25T01:26:03.507578+0000","last_became_active":"2024-04-25T01:26:02.426024+0000","last_became_peered":"2024-04-25T01:26:02.426024+0000","last_unstale":"2024-04-25T01:26:03.507578+0000","last_undegraded":"2024-04-25T01:26:03.507578+0000","last_fullsized":"2024-04-25T01:26:03.507578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717460+0000","last_change":"2024-04-25T01:25:53.372842+0000","last_active":"2024-04-25T01:26:03.717460+0000","last_peered":"2024-04-25T01:26:03.717460+0000","last_clean":"2024-04-25T01:26:03.717460+0000","last_became_active":"2024-04-25T01:25:53.372580+0000","last_became_peered":"2024-04-25T01:25:53.372580+0000","last_unstale":"2024-04-25T01:26:03.717460+0000","last_undegraded":"2024-04-25T01:26:03.717460+0000","last_fullsized":"2024-04-25T01:26:03.717460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716748+0000","last_change":"2024-04-25T01:26:02.414632+0000","last_active":"2024-04-25T01:26:03.716748+0000","last_peered":"2024-04-25T01:26:03.716748+0000","last_clean":"2024-04-25T01:26:03.716748+0000","last_became_active":"2024-04-25T01:26:02.414388+0000","last_became_peered":"2024-04-25T01:26:02.414388+0000","last_unstale":"2024-04-25T01:26:03.716748+0000","last_undegraded":"2024-04-25T01:26:03.716748+0000","last_fullsized":"2024-04-25T01:26:03.716748+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.508086+0000","last_change":"2024-04-25T01:25:53.376081+0000","last_active":"2024-04-25T01:26:03.508086+0000","last_peered":"2024-04-25T01:26:03.508086+0000","last_clean":"2024-04-25T01:26:03.508086+0000","last_became_active":"2024-04-25T01:25:53.375814+0000","last_became_peered":"2024-04-25T01:25:53.375814+0000","last_unstale":"2024-04-25T01:26:03.508086+0000","last_undegraded":"2024-04-25T01:26:03.508086+0000","last_fullsized":"2024-04-25T01:26:03.508086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:29.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416509+0000","last_change":"2024-04-25T01:26:02.414366+0000","last_active":"2024-04-25T01:26:03.416509+0000","last_peered":"2024-04-25T01:26:03.416509+0000","last_clean":"2024-04-25T01:26:03.416509+0000","last_became_active":"2024-04-25T01:26:02.414061+0000","last_became_peered":"2024-04-25T01:26:02.414061+0000","last_unstale":"2024-04-25T01:26:03.416509+0000","last_undegraded":"2024-04-25T01:26:03.416509+0000","last_fullsized":"2024-04-25T01:26:03.416509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:19.644468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415110+0000","last_change":"2024-04-25T01:25:53.374061+0000","last_active":"2024-04-25T01:26:03.415110+0000","last_peered":"2024-04-25T01:26:03.415110+0000","last_clean":"2024-04-25T01:26:03.415110+0000","last_became_active":"2024-04-25T01:25:53.373848+0000","last_became_peered":"2024-04-25T01:25:53.373848+0000","last_unstale":"2024-04-25T01:26:03.415110+0000","last_undegraded":"2024-04-25T01:26:03.415110+0000","last_fullsized":"2024-04-25T01:26:03.415110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717762+0000","last_change":"2024-04-25T01:26:02.414620+0000","last_active":"2024-04-25T01:26:03.717762+0000","last_peered":"2024-04-25T01:26:03.717762+0000","last_clean":"2024-04-25T01:26:03.717762+0000","last_became_active":"2024-04-25T01:26:02.414326+0000","last_became_peered":"2024-04-25T01:26:02.414326+0000","last_unstale":"2024-04-25T01:26:03.717762+0000","last_undegraded":"2024-04-25T01:26:03.717762+0000","last_fullsized":"2024-04-25T01:26:03.717762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:30.496089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507095+0000","last_change":"2024-04-25T01:25:53.364698+0000","last_active":"2024-04-25T01:26:03.507095+0000","last_peered":"2024-04-25T01:26:03.507095+0000","last_clean":"2024-04-25T01:26:03.507095+0000","last_became_active":"2024-04-25T01:25:53.364435+0000","last_became_peered":"2024-04-25T01:25:53.364435+0000","last_unstale":"2024-04-25T01:26:03.507095+0000","last_undegraded":"2024-04-25T01:26:03.507095+0000","last_fullsized":"2024-04-25T01:26:03.507095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716509+0000","last_change":"2024-04-25T01:26:02.418331+0000","last_active":"2024-04-25T01:26:03.716509+0000","last_peered":"2024-04-25T01:26:03.716509+0000","last_clean":"2024-04-25T01:26:03.716509+0000","last_became_active":"2024-04-25T01:26:02.418057+0000","last_became_peered":"2024-04-25T01:26:02.418057+0000","last_unstale":"2024-04-25T01:26:03.716509+0000","last_undegraded":"2024-04-25T01:26:03.716509+0000","last_fullsized":"2024-04-25T01:26:03.716509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:17.809738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717064+0000","last_change":"2024-04-25T01:25:53.375324+0000","last_active":"2024-04-25T01:26:03.717064+0000","last_peered":"2024-04-25T01:26:03.717064+0000","last_clean":"2024-04-25T01:26:03.717064+0000","last_became_active":"2024-04-25T01:25:53.374392+0000","last_became_peered":"2024-04-25T01:25:53.374392+0000","last_unstale":"2024-04-25T01:26:03.717064+0000","last_undegraded":"2024-04-25T01:26:03.717064+0000","last_fullsized":"2024-04-25T01:26:03.717064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717818+0000","last_change":"2024-04-25T01:26:02.422720+0000","last_active":"2024-04-25T01:26:03.717818+0000","last_peered":"2024-04-25T01:26:03.717818+0000","last_clean":"2024-04-25T01:26:03.717818+0000","last_became_active":"2024-04-25T01:26:02.421848+0000","last_became_peered":"2024-04-25T01:26:02.421848+0000","last_unstale":"2024-04-25T01:26:03.717818+0000","last_undegraded":"2024-04-25T01:26:03.717818+0000","last_fullsized":"2024-04-25T01:26:03.717818+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507255+0000","last_change":"2024-04-25T01:25:53.367953+0000","last_active":"2024-04-25T01:26:03.507255+0000","last_peered":"2024-04-25T01:26:03.507255+0000","last_clean":"2024-04-25T01:26:03.507255+0000","last_became_active":"2024-04-25T01:25:53.367358+0000","last_became_peered":"2024-04-25T01:25:53.367358+0000","last_unstale":"2024-04-25T01:26:03.507255+0000","last_undegraded":"2024-04-25T01:26:03.507255+0000","last_fullsized":"2024-04-25T01:26:03.507255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415875+0000","last_change":"2024-04-25T01:26:02.422300+0000","last_active":"2024-04-25T01:26:03.415875+0000","last_peered":"2024-04-25T01:26:03.415875+0000","last_clean":"2024-04-25T01:26:03.415875+0000","last_became_active":"2024-04-25T01:26:02.421702+0000","last_became_peered":"2024-04-25T01:26:02.421702+0000","last_unstale":"2024-04-25T01:26:03.415875+0000","last_undegraded":"2024-04-25T01:26:03.415875+0000","last_fullsized":"2024-04-25T01:26:03.415875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717355+0000","last_change":"2024-04-25T01:25:53.371513+0000","last_active":"2024-04-25T01:26:03.717355+0000","last_peered":"2024-04-25T01:26:03.717355+0000","last_clean":"2024-04-25T01:26:03.717355+0000","last_became_active":"2024-04-25T01:25:53.370421+0000","last_became_peered":"2024-04-25T01:25:53.370421+0000","last_unstale":"2024-04-25T01:26:03.717355+0000","last_undegraded":"2024-04-25T01:26:03.717355+0000","last_fullsized":"2024-04-25T01:26:03.717355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415236+0000","last_change":"2024-04-25T01:26:02.411784+0000","last_active":"2024-04-25T01:26:03.415236+0000","last_peered":"2024-04-25T01:26:03.415236+0000","last_clean":"2024-04-25T01:26:03.415236+0000","last_became_active":"2024-04-25T01:26:02.411478+0000","last_became_peered":"2024-04-25T01:26:02.411478+0000","last_unstale":"2024-04-25T01:26:03.415236+0000","last_undegraded":"2024-04-25T01:26:03.415236+0000","last_fullsized":"2024-04-25T01:26:03.415236+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:20.194816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716214+0000","last_change":"2024-04-25T01:25:53.363829+0000","last_active":"2024-04-25T01:26:03.716214+0000","last_peered":"2024-04-25T01:26:03.716214+0000","last_clean":"2024-04-25T01:26:03.716214+0000","last_became_active":"2024-04-25T01:25:53.363525+0000","last_became_peered":"2024-04-25T01:25:53.363525+0000","last_unstale":"2024-04-25T01:26:03.716214+0000","last_undegraded":"2024-04-25T01:26:03.716214+0000","last_fullsized":"2024-04-25T01:26:03.716214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.506843+0000","last_change":"2024-04-25T01:26:02.421208+0000","last_active":"2024-04-25T01:26:03.506843+0000","last_peered":"2024-04-25T01:26:03.506843+0000","last_clean":"2024-04-25T01:26:03.506843+0000","last_became_active":"2024-04-25T01:26:02.418796+0000","last_became_peered":"2024-04-25T01:26:02.418796+0000","last_unstale":"2024-04-25T01:26:03.506843+0000","last_undegraded":"2024-04-25T01:26:03.506843+0000","last_fullsized":"2024-04-25T01:26:03.506843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716784+0000","last_change":"2024-04-25T01:25:53.365057+0000","last_active":"2024-04-25T01:26:03.716784+0000","last_peered":"2024-04-25T01:26:03.716784+0000","last_clean":"2024-04-25T01:26:03.716784+0000","last_became_active":"2024-04-25T01:25:53.364793+0000","last_became_peered":"2024-04-25T01:25:53.364793+0000","last_unstale":"2024-04-25T01:26:03.716784+0000","last_undegraded":"2024-04-25T01:26:03.716784+0000","last_fullsized":"2024-04-25T01:26:03.716784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507060+0000","last_change":"2024-04-25T01:26:02.417606+0000","last_active":"2024-04-25T01:26:03.507060+0000","last_peered":"2024-04-25T01:26:03.507060+0000","last_clean":"2024-04-25T01:26:03.507060+0000","last_became_active":"2024-04-25T01:26:02.417350+0000","last_became_peered":"2024-04-25T01:26:02.417350+0000","last_unstale":"2024-04-25T01:26:03.507060+0000","last_undegraded":"2024-04-25T01:26:03.507060+0000","last_fullsized":"2024-04-25T01:26:03.507060+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.508077+0000","last_change":"2024-04-25T01:25:53.367762+0000","last_active":"2024-04-25T01:26:03.508077+0000","last_peered":"2024-04-25T01:26:03.508077+0000","last_clean":"2024-04-25T01:26:03.508077+0000","last_became_active":"2024-04-25T01:25:53.367383+0000","last_became_peered":"2024-04-25T01:25:53.367383+0000","last_unstale":"2024-04-25T01:26:03.508077+0000","last_undegraded":"2024-04-25T01:26:03.508077+0000","last_fullsized":"2024-04-25T01:26:03.508077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:05.589744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":690301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":101489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":99491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":102507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":99331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":101055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":690267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:26:11.021 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T01:26:11.021 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T01:26:11.022 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T01:26:11.022 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T01:26:11.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.172+0000 7f92a19f9640 1 Processor -- start 2024-04-25T01:26:11.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.172+0000 7f92a19f9640 1 -- start start 2024-04-25T01:26:11.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.173+0000 7f92a19f9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f929c06b3d0 0x7f929c14d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:11.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.173+0000 7f92a19f9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f929c072dd0 0x7f929c14d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:11.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.173+0000 7f92a19f9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f929c154690 con 0x7f929c06bbe0 2024-04-25T01:26:11.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.173+0000 7f92a19f9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f929c0aa9d0 con 0x7f929c072dd0 2024-04-25T01:26:11.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.173+0000 7f92a19f9640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f929c0aac20 con 0x7f929c06b3d0 2024-04-25T01:26:11.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.173+0000 7f929b7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f929c072dd0 0x7f929c14d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:11.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.173+0000 7f929b7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f929c072dd0 0x7f929c14d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43628/0 (socket says 172.21.15.112:43628) 2024-04-25T01:26:11.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.173+0000 7f929a7fc640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f929c06bbe0 0x7f929c0a98e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38084/0 (socket says 172.21.15.112:38084) 2024-04-25T01:26:11.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.173+0000 7f929b7fe640 1 -- 172.21.15.112:0/2905461500 learned_addr learned my addr 172.21.15.112:0/2905461500 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:11.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.173+0000 7f929b7fe640 1 -- 172.21.15.112:0/2905461500 >> v2:172.21.15.187:3300/0 conn(0x7f929c06b3d0 msgr2=0x7f929c14d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:11.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.173+0000 7f929affd640 1 --2- 172.21.15.112:0/2905461500 >> v2:172.21.15.187:3300/0 conn(0x7f929c06b3d0 0x7f929c14d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:11.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.173+0000 7f929b7fe640 1 --2- 172.21.15.112:0/2905461500 >> v2:172.21.15.187:3300/0 conn(0x7f929c06b3d0 0x7f929c14d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.173+0000 7f9299ffb640 1 -- 172.21.15.112:0/2905461500 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2857411690 0 0) 0x7f929c154690 con 0x7f929c06bbe0 2024-04-25T01:26:11.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.173+0000 7f929b7fe640 1 -- 172.21.15.112:0/2905461500 >> v1:172.21.15.112:6789/0 conn(0x7f929c06bbe0 legacy=0x7f929c0a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:11.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.173+0000 7f929b7fe640 1 -- 172.21.15.112:0/2905461500 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f929c0aaf00 con 0x7f929c072dd0 2024-04-25T01:26:11.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.174+0000 7f929b7fe640 1 --2- 172.21.15.112:0/2905461500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f929c072dd0 0x7f929c14d860 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f929c0c5a00 tx=0x7f929402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df5b013e22088c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:11.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.174+0000 7f9299ffb640 1 -- 172.21.15.112:0/2905461500 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294036070 con 0x7f929c072dd0 2024-04-25T01:26:11.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.174+0000 7f9299ffb640 1 -- 172.21.15.112:0/2905461500 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9294031600 con 0x7f929c072dd0 2024-04-25T01:26:11.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.174+0000 7f9299ffb640 1 -- 172.21.15.112:0/2905461500 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92940319d0 con 0x7f929c072dd0 2024-04-25T01:26:11.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.175+0000 7f92a19f9640 1 -- 172.21.15.112:0/2905461500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f929c072dd0 msgr2=0x7f929c14d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:11.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.175+0000 7f92a19f9640 1 --2- 172.21.15.112:0/2905461500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f929c072dd0 0x7f929c14d860 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f929c0c5a00 tx=0x7f929402a6a0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.175+0000 7f92a19f9640 1 -- 172.21.15.112:0/2905461500 shutdown_connections 2024-04-25T01:26:11.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.175+0000 7f92a19f9640 1 --2- 172.21.15.112:0/2905461500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f929c072dd0 0x7f929c14d860 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.175+0000 7f92a19f9640 1 --2- 172.21.15.112:0/2905461500 >> v2:172.21.15.187:3300/0 conn(0x7f929c06b3d0 0x7f929c14d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.175+0000 7f92a19f9640 1 -- 172.21.15.112:0/2905461500 >> 172.21.15.112:0/2905461500 conn(0x7f929c0c4c00 msgr2=0x7f929c0c5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:11.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.175+0000 7f92a19f9640 1 -- 172.21.15.112:0/2905461500 shutdown_connections 2024-04-25T01:26:11.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.175+0000 7f92a19f9640 1 -- 172.21.15.112:0/2905461500 wait complete. 2024-04-25T01:26:11.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.176+0000 7f92a19f9640 1 Processor -- start 2024-04-25T01:26:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.176+0000 7f92a19f9640 1 -- start start 2024-04-25T01:26:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f92a19f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f929c06b3d0 0x7f929c0c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f92a19f9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f929c06bbe0 0x7f929c1ec990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f92a19f9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f929c072dd0 0x7f929c1eced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f92a19f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f929c06b930 con 0x7f929c06b3d0 2024-04-25T01:26:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f92a19f9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f929c06b7b0 con 0x7f929c072dd0 2024-04-25T01:26:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f92a19f9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f929c0a9ff0 con 0x7f929c06bbe0 2024-04-25T01:26:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f929affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f929c06b3d0 0x7f929c0c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f929affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f929c06b3d0 0x7f929c0c7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43638/0 (socket says 172.21.15.112:43638) 2024-04-25T01:26:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f929affd640 1 -- 172.21.15.112:0/690531718 learned_addr learned my addr 172.21.15.112:0/690531718 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f929b7fe640 1 --2- 172.21.15.112:0/690531718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f929c072dd0 0x7f929c1eced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f929a7fc640 1 --2- 172.21.15.112:0/690531718 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f929c06bbe0 0x7f929c1ec990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f929affd640 1 -- 172.21.15.112:0/690531718 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f929c06bbe0 msgr2=0x7f929c1ec990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f929affd640 1 --2- 172.21.15.112:0/690531718 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f929c06bbe0 0x7f929c1ec990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f929affd640 1 -- 172.21.15.112:0/690531718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f929c072dd0 msgr2=0x7f929c1eced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f929affd640 1 --2- 172.21.15.112:0/690531718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f929c072dd0 0x7f929c1eced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f929affd640 1 -- 172.21.15.112:0/690531718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f929c149de0 con 0x7f929c06b3d0 2024-04-25T01:26:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f929affd640 1 --2- 172.21.15.112:0/690531718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f929c06b3d0 0x7f929c0c7680 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f928c00c7f0 tx=0x7f928c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f9283fff640 1 -- 172.21.15.112:0/690531718 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c015070 con 0x7f929c06b3d0 2024-04-25T01:26:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f9283fff640 1 -- 172.21.15.112:0/690531718 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f928c004240 con 0x7f929c06b3d0 2024-04-25T01:26:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f92a19f9640 1 -- 172.21.15.112:0/690531718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f929c14a0c0 con 0x7f929c06b3d0 2024-04-25T01:26:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f9283fff640 1 -- 172.21.15.112:0/690531718 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c007620 con 0x7f929c06b3d0 2024-04-25T01:26:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.177+0000 7f92a19f9640 1 -- 172.21.15.112:0/690531718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f929c0c7bc0 con 0x7f929c06b3d0 2024-04-25T01:26:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.178+0000 7f92a19f9640 1 -- 172.21.15.112:0/690531718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f929c0682a0 con 0x7f929c06b3d0 2024-04-25T01:26:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.179+0000 7f9283fff640 1 -- 172.21.15.112:0/690531718 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f928c0079d0 con 0x7f929c06b3d0 2024-04-25T01:26:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.179+0000 7f9283fff640 1 --2- 172.21.15.112:0/690531718 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f928806be60 0x7f928806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.179+0000 7f9283fff640 1 -- 172.21.15.112:0/690531718 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f928c08b1e0 con 0x7f929c06b3d0 2024-04-25T01:26:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.179+0000 7f929a7fc640 1 --2- 172.21.15.112:0/690531718 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f928806be60 0x7f928806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.179+0000 7f929a7fc640 1 --2- 172.21.15.112:0/690531718 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f928806be60 0x7f928806e320 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9290007fd0 tx=0x7f9290009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:11.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.183+0000 7f9283fff640 1 -- 172.21.15.112:0/690531718 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f928c0598f0 con 0x7f929c06b3d0 2024-04-25T01:26:11.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.392+0000 7f92a19f9640 1 -- 172.21.15.112:0/690531718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f929c0c5a30 con 0x7f929c06b3d0 2024-04-25T01:26:11.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.393+0000 7f9283fff640 1 -- 172.21.15.112:0/690531718 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f928c0591f0 con 0x7f929c06b3d0 2024-04-25T01:26:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:11.372 INFO:teuthology.orchestra.run.smithi112.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T01:26:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.395+0000 7f92a19f9640 1 -- 172.21.15.112:0/690531718 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f928806be60 msgr2=0x7f928806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.395+0000 7f92a19f9640 1 --2- 172.21.15.112:0/690531718 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f928806be60 0x7f928806e320 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9290007fd0 tx=0x7f9290009040 comp rx=0 tx=0).stop 2024-04-25T01:26:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.395+0000 7f92a19f9640 1 -- 172.21.15.112:0/690531718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f929c06b3d0 msgr2=0x7f929c0c7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.395+0000 7f92a19f9640 1 --2- 172.21.15.112:0/690531718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f929c06b3d0 0x7f929c0c7680 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f928c00c7f0 tx=0x7f928c00ccb0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.395+0000 7f92a19f9640 1 -- 172.21.15.112:0/690531718 shutdown_connections 2024-04-25T01:26:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.395+0000 7f92a19f9640 1 --2- 172.21.15.112:0/690531718 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f928806be60 0x7f928806e320 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.395+0000 7f92a19f9640 1 --2- 172.21.15.112:0/690531718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f929c072dd0 0x7f929c1eced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.395+0000 7f92a19f9640 1 --2- 172.21.15.112:0/690531718 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f929c06bbe0 0x7f929c1ec990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.395+0000 7f92a19f9640 1 --2- 172.21.15.112:0/690531718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f929c06b3d0 0x7f929c0c7680 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.395+0000 7f92a19f9640 1 -- 172.21.15.112:0/690531718 >> 172.21.15.112:0/690531718 conn(0x7f929c0c4c00 msgr2=0x7f929c0ad140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.395+0000 7f92a19f9640 1 -- 172.21.15.112:0/690531718 shutdown_connections 2024-04-25T01:26:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.395+0000 7f92a19f9640 1 -- 172.21.15.112:0/690531718 wait complete. 2024-04-25T01:26:11.385 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T01:26:11.385 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:26:11.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.502+0000 7f7d534ee640 1 Processor -- start 2024-04-25T01:26:11.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.502+0000 7f7d534ee640 1 -- start start 2024-04-25T01:26:11.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.503+0000 7f7d534ee640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7d4c154b50 0x7f7d4c0c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:11.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.503+0000 7f7d534ee640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d4c0c8810 0x7f7d4c153660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:11.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.503+0000 7f7d534ee640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d4c154fd0 con 0x7f7d4c071520 2024-04-25T01:26:11.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.503+0000 7f7d534ee640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f7d4c0c7f10 con 0x7f7d4c154b50 2024-04-25T01:26:11.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.503+0000 7f7d534ee640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7d4c154120 con 0x7f7d4c0c8810 2024-04-25T01:26:11.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.503+0000 7f7d50a89640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7d4c154b50 0x7f7d4c0c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:11.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.503+0000 7f7d5128a640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7d4c071520 0x7f7d4c0c7800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38092/0 (socket says 172.21.15.112:38092) 2024-04-25T01:26:11.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.503+0000 7f7d5128a640 1 -- 172.21.15.112:0/271620800 learned_addr learned my addr 172.21.15.112:0/271620800 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:11.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.503+0000 7f7d51a8b640 1 --2- 172.21.15.112:0/271620800 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d4c0c8810 0x7f7d4c153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:11.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.503+0000 7f7d50a89640 1 -- 172.21.15.112:0/271620800 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d4c0c8810 msgr2=0x7f7d4c153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:11.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.503+0000 7f7d50a89640 1 --2- 172.21.15.112:0/271620800 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d4c0c8810 0x7f7d4c153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.503+0000 7f7d50a89640 1 -- 172.21.15.112:0/271620800 >> v1:172.21.15.112:6789/0 conn(0x7f7d4c071520 legacy=0x7f7d4c0c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:11.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.503+0000 7f7d43fff640 1 -- 172.21.15.112:0/271620800 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 304651202 0 0) 0x7f7d48001110 con 0x7f7d4c071520 2024-04-25T01:26:11.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.503+0000 7f7d50a89640 1 -- 172.21.15.112:0/271620800 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4c154400 con 0x7f7d4c154b50 2024-04-25T01:26:11.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.504+0000 7f7d50a89640 1 --2- 172.21.15.112:0/271620800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4c154b50 0x7f7d4c0c82d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c0097a0 tx=0x7f7d3c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3733762eca5af282 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:11.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.504+0000 7f7d43fff640 1 -- 172.21.15.112:0/271620800 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c037070 con 0x7f7d4c154b50 2024-04-25T01:26:11.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.504+0000 7f7d43fff640 1 -- 172.21.15.112:0/271620800 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d3c02adb0 con 0x7f7d4c154b50 2024-04-25T01:26:11.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.504+0000 7f7d43fff640 1 -- 172.21.15.112:0/271620800 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c032660 con 0x7f7d4c154b50 2024-04-25T01:26:11.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.505+0000 7f7d534ee640 1 -- 172.21.15.112:0/271620800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4c154b50 msgr2=0x7f7d4c0c82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:11.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.505+0000 7f7d534ee640 1 --2- 172.21.15.112:0/271620800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4c154b50 0x7f7d4c0c82d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c0097a0 tx=0x7f7d3c02a060 comp rx=0 tx=0).stop 2024-04-25T01:26:11.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.505+0000 7f7d534ee640 1 -- 172.21.15.112:0/271620800 shutdown_connections 2024-04-25T01:26:11.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.505+0000 7f7d534ee640 1 --2- 172.21.15.112:0/271620800 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d4c0c8810 0x7f7d4c153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.505+0000 7f7d534ee640 1 --2- 172.21.15.112:0/271620800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4c154b50 0x7f7d4c0c82d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.505+0000 7f7d534ee640 1 -- 172.21.15.112:0/271620800 >> 172.21.15.112:0/271620800 conn(0x7f7d4c074a00 msgr2=0x7f7d4c0c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:11.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.505+0000 7f7d534ee640 1 -- 172.21.15.112:0/271620800 shutdown_connections 2024-04-25T01:26:11.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.505+0000 7f7d534ee640 1 -- 172.21.15.112:0/271620800 wait complete. 2024-04-25T01:26:11.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.506+0000 7f7d534ee640 1 Processor -- start 2024-04-25T01:26:11.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.506+0000 7f7d534ee640 1 -- start start 2024-04-25T01:26:11.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d534ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4c071520 0x7f7d4c0705c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:11.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d534ee640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d4c0c8810 0x7f7d4c149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:11.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d534ee640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d4c154b50 0x7f7d4c14a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:11.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d534ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4c0ad930 con 0x7f7d4c071520 2024-04-25T01:26:11.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d534ee640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4c0c72e0 con 0x7f7d4c0c8810 2024-04-25T01:26:11.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d534ee640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7d4c0adab0 con 0x7f7d4c154b50 2024-04-25T01:26:11.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d5128a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4c071520 0x7f7d4c0705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:11.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d5128a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4c071520 0x7f7d4c0705c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43666/0 (socket says 172.21.15.112:43666) 2024-04-25T01:26:11.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d5128a640 1 -- 172.21.15.112:0/1662970362 learned_addr learned my addr 172.21.15.112:0/1662970362 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:11.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d51a8b640 1 --2- 172.21.15.112:0/1662970362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d4c154b50 0x7f7d4c14a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:11.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d50a89640 1 --2- 172.21.15.112:0/1662970362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d4c0c8810 0x7f7d4c149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:11.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d5128a640 1 -- 172.21.15.112:0/1662970362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d4c154b50 msgr2=0x7f7d4c14a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:11.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d5128a640 1 --2- 172.21.15.112:0/1662970362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d4c154b50 0x7f7d4c14a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d5128a640 1 -- 172.21.15.112:0/1662970362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d4c0c8810 msgr2=0x7f7d4c149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:11.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d5128a640 1 --2- 172.21.15.112:0/1662970362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d4c0c8810 0x7f7d4c149ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d5128a640 1 -- 172.21.15.112:0/1662970362 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4c14c0b0 con 0x7f7d4c071520 2024-04-25T01:26:11.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d5128a640 1 --2- 172.21.15.112:0/1662970362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4c071520 0x7f7d4c0705c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800c7b0 tx=0x7f7d4800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:11.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d41ffb640 1 -- 172.21.15.112:0/1662970362 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48018070 con 0x7f7d4c071520 2024-04-25T01:26:11.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d41ffb640 1 -- 172.21.15.112:0/1662970362 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d480072d0 con 0x7f7d4c071520 2024-04-25T01:26:11.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d534ee640 1 -- 172.21.15.112:0/1662970362 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d4c1493a0 con 0x7f7d4c071520 2024-04-25T01:26:11.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d41ffb640 1 -- 172.21.15.112:0/1662970362 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d480047e0 con 0x7f7d4c071520 2024-04-25T01:26:11.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.507+0000 7f7d534ee640 1 -- 172.21.15.112:0/1662970362 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d4c1ec8a0 con 0x7f7d4c071520 2024-04-25T01:26:11.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.508+0000 7f7d534ee640 1 -- 172.21.15.112:0/1662970362 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d4c0682a0 con 0x7f7d4c071520 2024-04-25T01:26:11.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.509+0000 7f7d41ffb640 1 -- 172.21.15.112:0/1662970362 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d4801b070 con 0x7f7d4c071520 2024-04-25T01:26:11.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.509+0000 7f7d41ffb640 1 --2- 172.21.15.112:0/1662970362 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7d3006be60 0x7f7d3006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:11.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.509+0000 7f7d41ffb640 1 -- 172.21.15.112:0/1662970362 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f7d4808a460 con 0x7f7d4c071520 2024-04-25T01:26:11.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.509+0000 7f7d50a89640 1 --2- 172.21.15.112:0/1662970362 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7d3006be60 0x7f7d3006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:11.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.510+0000 7f7d50a89640 1 --2- 172.21.15.112:0/1662970362 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7d3006be60 0x7f7d3006e320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c034000 tx=0x7f7d3c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:11.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.513+0000 7f7d41ffb640 1 -- 172.21.15.112:0/1662970362 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7d4c154fd0 con 0x7f7d4c071520 2024-04-25T01:26:11.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.687+0000 7f7d534ee640 1 -- 172.21.15.112:0/1662970362 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d4c0c80b0 con 0x7f7d4c071520 2024-04-25T01:26:11.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.688+0000 7f7d41ffb640 1 -- 172.21.15.112:0/1662970362 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20367 (secure 0 0 0) 0x7f7d48058b70 con 0x7f7d4c071520 2024-04-25T01:26:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:11.668 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:26:03.405271+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-25T01:25:59.478275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-25T01:26:00.719358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:26:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.690+0000 7f7d534ee640 1 -- 172.21.15.112:0/1662970362 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7d3006be60 msgr2=0x7f7d3006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.690+0000 7f7d534ee640 1 --2- 172.21.15.112:0/1662970362 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7d3006be60 0x7f7d3006e320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c034000 tx=0x7f7d3c02e000 comp rx=0 tx=0).stop 2024-04-25T01:26:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.690+0000 7f7d534ee640 1 -- 172.21.15.112:0/1662970362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4c071520 msgr2=0x7f7d4c0705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.690+0000 7f7d534ee640 1 --2- 172.21.15.112:0/1662970362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4c071520 0x7f7d4c0705c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800c7b0 tx=0x7f7d4800cc70 comp rx=0 tx=0).stop 2024-04-25T01:26:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.690+0000 7f7d534ee640 1 -- 172.21.15.112:0/1662970362 shutdown_connections 2024-04-25T01:26:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.690+0000 7f7d534ee640 1 --2- 172.21.15.112:0/1662970362 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7d3006be60 0x7f7d3006e320 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.690+0000 7f7d534ee640 1 --2- 172.21.15.112:0/1662970362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d4c154b50 0x7f7d4c14a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.690+0000 7f7d534ee640 1 --2- 172.21.15.112:0/1662970362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d4c0c8810 0x7f7d4c149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.690+0000 7f7d534ee640 1 --2- 172.21.15.112:0/1662970362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4c071520 0x7f7d4c0705c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.690+0000 7f7d534ee640 1 -- 172.21.15.112:0/1662970362 >> 172.21.15.112:0/1662970362 conn(0x7f7d4c074a00 msgr2=0x7f7d4c0aa4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:11.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.690+0000 7f7d534ee640 1 -- 172.21.15.112:0/1662970362 shutdown_connections 2024-04-25T01:26:11.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.690+0000 7f7d534ee640 1 -- 172.21.15.112:0/1662970362 wait complete. 2024-04-25T01:26:11.680 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:25:40.679119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-04-25T01:25:50.593045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-04-25T01:25:51.664930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-04-25T01:25:59.478275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-04-25T01:26:00.719358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:26:11.681 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:26:11.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.800+0000 7f08e2831640 1 Processor -- start 2024-04-25T01:26:11.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.800+0000 7f08e2831640 1 -- start start 2024-04-25T01:26:11.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.801+0000 7f08e2831640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f08dc06d460 0x7f08dc071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:11.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.801+0000 7f08e2831640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08dc06f290 con 0x7f08dc0724f0 2024-04-25T01:26:11.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.801+0000 7f08e2831640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08dc06efb0 con 0x7f08dc14fbe0 2024-04-25T01:26:11.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.801+0000 7f08e2831640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f08dc14ffc0 con 0x7f08dc06d460 2024-04-25T01:26:11.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.801+0000 7f08db7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f08dc06d460 0x7f08dc071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:11.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.801+0000 7f08e0dce640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f08dc0724f0 0x7f08dc15ce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38102/0 (socket says 172.21.15.112:38102) 2024-04-25T01:26:11.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.801+0000 7f08e0dce640 1 -- 172.21.15.112:0/1639369513 learned_addr learned my addr 172.21.15.112:0/1639369513 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:11.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.801+0000 7f08db7fe640 1 -- 172.21.15.112:0/1639369513 >> v1:172.21.15.187:6789/0 conn(0x7f08dc14fbe0 legacy=0x7f08dc0c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:11.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.801+0000 7f08db7fe640 1 -- 172.21.15.112:0/1639369513 >> v1:172.21.15.112:6789/0 conn(0x7f08dc0724f0 legacy=0x7f08dc15ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:11.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.801+0000 7f08daffd640 1 -- 172.21.15.112:0/1639369513 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4186120886 0 0) 0x7f08dc06f290 con 0x7f08dc0724f0 2024-04-25T01:26:11.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.801+0000 7f08db7fe640 1 -- 172.21.15.112:0/1639369513 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08dc0a9530 con 0x7f08dc06d460 2024-04-25T01:26:11.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.802+0000 7f08db7fe640 1 --2- 172.21.15.112:0/1639369513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08dc06d460 0x7f08dc071fb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f08d00097a0 tx=0x7f08d002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517e3693a46a9b53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:11.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.802+0000 7f08daffd640 1 -- 172.21.15.112:0/1639369513 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0037070 con 0x7f08dc06d460 2024-04-25T01:26:11.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.802+0000 7f08daffd640 1 -- 172.21.15.112:0/1639369513 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d002ae50 con 0x7f08dc06d460 2024-04-25T01:26:11.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.802+0000 7f08daffd640 1 -- 172.21.15.112:0/1639369513 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0032660 con 0x7f08dc06d460 2024-04-25T01:26:11.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.802+0000 7f08e2831640 1 -- 172.21.15.112:0/1639369513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08dc06d460 msgr2=0x7f08dc071fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:11.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.802+0000 7f08e2831640 1 --2- 172.21.15.112:0/1639369513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08dc06d460 0x7f08dc071fb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f08d00097a0 tx=0x7f08d002a140 comp rx=0 tx=0).stop 2024-04-25T01:26:11.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.802+0000 7f08e2831640 1 -- 172.21.15.112:0/1639369513 shutdown_connections 2024-04-25T01:26:11.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.803+0000 7f08e2831640 1 --2- 172.21.15.112:0/1639369513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08dc06d460 0x7f08dc071fb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.803+0000 7f08e2831640 1 -- 172.21.15.112:0/1639369513 >> 172.21.15.112:0/1639369513 conn(0x7f08dc0c3a10 msgr2=0x7f08dc0c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:11.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.803+0000 7f08e2831640 1 -- 172.21.15.112:0/1639369513 shutdown_connections 2024-04-25T01:26:11.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.803+0000 7f08e2831640 1 -- 172.21.15.112:0/1639369513 wait complete. 2024-04-25T01:26:11.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.804+0000 7f08e2831640 1 Processor -- start 2024-04-25T01:26:11.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.804+0000 7f08e2831640 1 -- start start 2024-04-25T01:26:11.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.804+0000 7f08e2831640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08dc06d460 0x7f08dc1ed050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:11.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.804+0000 7f08e2831640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08dc0724f0 0x7f08dc14ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:11.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.804+0000 7f08e2831640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08dc14fbe0 0x7f08dc1e92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:11.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.804+0000 7f08e2831640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc0c3030 con 0x7f08dc14fbe0 2024-04-25T01:26:11.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.804+0000 7f08e2831640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc1533a0 con 0x7f08dc06d460 2024-04-25T01:26:11.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.804+0000 7f08e2831640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f08dc0c31b0 con 0x7f08dc0724f0 2024-04-25T01:26:11.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.804+0000 7f08e0dce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08dc14fbe0 0x7f08dc1e92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:11.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.805+0000 7f08e0dce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08dc14fbe0 0x7f08dc1e92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43692/0 (socket says 172.21.15.112:43692) 2024-04-25T01:26:11.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.805+0000 7f08e0dce640 1 -- 172.21.15.112:0/1718553315 learned_addr learned my addr 172.21.15.112:0/1718553315 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:11.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.805+0000 7f08db7fe640 1 --2- 172.21.15.112:0/1718553315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08dc0724f0 0x7f08dc14ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:11.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.805+0000 7f08e0dce640 1 -- 172.21.15.112:0/1718553315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08dc0724f0 msgr2=0x7f08dc14ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:11.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.805+0000 7f08e0dce640 1 --2- 172.21.15.112:0/1718553315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08dc0724f0 0x7f08dc14ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.805+0000 7f08e0dce640 1 -- 172.21.15.112:0/1718553315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08dc06d460 msgr2=0x7f08dc1ed050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:11.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.805+0000 7f08dbfff640 1 --2- 172.21.15.112:0/1718553315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08dc06d460 0x7f08dc1ed050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:11.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.805+0000 7f08e0dce640 1 --2- 172.21.15.112:0/1718553315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08dc06d460 0x7f08dc1ed050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.805+0000 7f08e0dce640 1 -- 172.21.15.112:0/1718553315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08dc153d70 con 0x7f08dc14fbe0 2024-04-25T01:26:11.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.805+0000 7f08db7fe640 1 --2- 172.21.15.112:0/1718553315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08dc0724f0 0x7f08dc14ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:11.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.805+0000 7f08dbfff640 1 --2- 172.21.15.112:0/1718553315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08dc06d460 0x7f08dc1ed050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:11.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.805+0000 7f08e0dce640 1 --2- 172.21.15.112:0/1718553315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08dc14fbe0 0x7f08dc1e92f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f08d400c7d0 tx=0x7f08d400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:11.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.805+0000 7f08d8ff9640 1 -- 172.21.15.112:0/1718553315 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d4018070 con 0x7f08dc14fbe0 2024-04-25T01:26:11.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.805+0000 7f08d8ff9640 1 -- 172.21.15.112:0/1718553315 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d400a640 con 0x7f08dc14fbe0 2024-04-25T01:26:11.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.805+0000 7f08e2831640 1 -- 172.21.15.112:0/1718553315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08dc154050 con 0x7f08dc14fbe0 2024-04-25T01:26:11.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.805+0000 7f08d8ff9640 1 -- 172.21.15.112:0/1718553315 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d400aa70 con 0x7f08dc14fbe0 2024-04-25T01:26:11.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.805+0000 7f08e2831640 1 -- 172.21.15.112:0/1718553315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08dc14f290 con 0x7f08dc14fbe0 2024-04-25T01:26:11.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.806+0000 7f08e2831640 1 -- 172.21.15.112:0/1718553315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08dc04f500 con 0x7f08dc14fbe0 2024-04-25T01:26:11.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.807+0000 7f08d8ff9640 1 -- 172.21.15.112:0/1718553315 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f08d401b070 con 0x7f08dc14fbe0 2024-04-25T01:26:11.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.807+0000 7f08d8ff9640 1 --2- 172.21.15.112:0/1718553315 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f08c806be60 0x7f08c806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:11.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.807+0000 7f08dbfff640 1 --2- 172.21.15.112:0/1718553315 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f08c806be60 0x7f08c806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:11.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.807+0000 7f08d8ff9640 1 -- 172.21.15.112:0/1718553315 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f08d408a700 con 0x7f08dc14fbe0 2024-04-25T01:26:11.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.807+0000 7f08dbfff640 1 --2- 172.21.15.112:0/1718553315 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f08c806be60 0x7f08c806e320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f08dc1522f0 tx=0x7f08cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:11.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.811+0000 7f08d8ff9640 1 -- 172.21.15.112:0/1718553315 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f08d4058e10 con 0x7f08dc14fbe0 2024-04-25T01:26:11.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.982+0000 7f08e2831640 1 -- 172.21.15.112:0/1718553315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f08dc14e120 con 0x7f08dc14fbe0 2024-04-25T01:26:11.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.982+0000 7f08d8ff9640 1 -- 172.21.15.112:0/1718553315 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f08d4058710 con 0x7f08dc14fbe0 2024-04-25T01:26:11.962 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T01:26:11.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.984+0000 7f08e2831640 1 -- 172.21.15.112:0/1718553315 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f08c806be60 msgr2=0x7f08c806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:11.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.984+0000 7f08e2831640 1 --2- 172.21.15.112:0/1718553315 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f08c806be60 0x7f08c806e320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f08dc1522f0 tx=0x7f08cc008040 comp rx=0 tx=0).stop 2024-04-25T01:26:11.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.984+0000 7f08e2831640 1 -- 172.21.15.112:0/1718553315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08dc14fbe0 msgr2=0x7f08dc1e92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:11.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.984+0000 7f08e2831640 1 --2- 172.21.15.112:0/1718553315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08dc14fbe0 0x7f08dc1e92f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f08d400c7d0 tx=0x7f08d400cc90 comp rx=0 tx=0).stop 2024-04-25T01:26:11.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.984+0000 7f08e2831640 1 -- 172.21.15.112:0/1718553315 shutdown_connections 2024-04-25T01:26:11.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.984+0000 7f08e2831640 1 --2- 172.21.15.112:0/1718553315 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f08c806be60 0x7f08c806e320 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.984+0000 7f08e2831640 1 --2- 172.21.15.112:0/1718553315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08dc14fbe0 0x7f08dc1e92f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.984+0000 7f08e2831640 1 --2- 172.21.15.112:0/1718553315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08dc0724f0 0x7f08dc14ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.984+0000 7f08e2831640 1 --2- 172.21.15.112:0/1718553315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08dc06d460 0x7f08dc1ed050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:11.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.984+0000 7f08e2831640 1 -- 172.21.15.112:0/1718553315 >> 172.21.15.112:0/1718553315 conn(0x7f08dc0c3a10 msgr2=0x7f08dc14d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:11.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.984+0000 7f08e2831640 1 -- 172.21.15.112:0/1718553315 shutdown_connections 2024-04-25T01:26:11.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:11.984+0000 7f08e2831640 1 -- 172.21.15.112:0/1718553315 wait complete. 2024-04-25T01:26:11.975 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-04-25T01:26:12.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.095+0000 7f2d02cb3640 1 Processor -- start 2024-04-25T01:26:12.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.095+0000 7f2d02cb3640 1 -- start start 2024-04-25T01:26:12.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.096+0000 7f2d02cb3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2cfc06c420 0x7f2cfc0c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.096+0000 7f2d02cb3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2cfc0ca750 0x7f2cfc159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.096+0000 7f2d02cb3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cfc0c7390 con 0x7f2cfc0c5d20 2024-04-25T01:26:12.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.096+0000 7f2d02cb3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f2cfc0cac00 con 0x7f2cfc0ca750 2024-04-25T01:26:12.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.096+0000 7f2d02cb3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2cfc0cae50 con 0x7f2cfc06c420 2024-04-25T01:26:12.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.096+0000 7f2d01250640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2cfc0ca750 0x7f2cfc159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.096+0000 7f2d00a4f640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2cfc0c5d20 0x7f2cfc06bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38106/0 (socket says 172.21.15.112:38106) 2024-04-25T01:26:12.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.096+0000 7f2d01250640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2cfc0ca750 0x7f2cfc159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43696/0 (socket says 172.21.15.112:43696) 2024-04-25T01:26:12.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.096+0000 7f2d00a4f640 1 -- 172.21.15.112:0/2095336210 learned_addr learned my addr 172.21.15.112:0/2095336210 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:12.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.096+0000 7f2cfbfff640 1 --2- 172.21.15.112:0/2095336210 >> v2:172.21.15.187:3300/0 conn(0x7f2cfc06c420 0x7f2cfc0c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.097+0000 7f2d01250640 1 -- 172.21.15.112:0/2095336210 >> v2:172.21.15.187:3300/0 conn(0x7f2cfc06c420 msgr2=0x7f2cfc0c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:12.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.097+0000 7f2d01250640 1 --2- 172.21.15.112:0/2095336210 >> v2:172.21.15.187:3300/0 conn(0x7f2cfc06c420 0x7f2cfc0c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.097+0000 7f2d01250640 1 -- 172.21.15.112:0/2095336210 >> v1:172.21.15.112:6789/0 conn(0x7f2cfc0c5d20 legacy=0x7f2cfc06bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.097+0000 7f2d01250640 1 -- 172.21.15.112:0/2095336210 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cfc15a330 con 0x7f2cfc0ca750 2024-04-25T01:26:12.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.097+0000 7f2cfb7fe640 1 -- 172.21.15.112:0/2095336210 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 88373578 0 0) 0x7f2cf0001110 con 0x7f2cfc0c5d20 2024-04-25T01:26:12.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.097+0000 7f2d01250640 1 --2- 172.21.15.112:0/2095336210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cfc0ca750 0x7f2cfc159420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc068e60 tx=0x7f2cf402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f02da2f6c79c1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:12.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.097+0000 7f2cfb7fe640 1 -- 172.21.15.112:0/2095336210 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4007740 con 0x7f2cfc0ca750 2024-04-25T01:26:12.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.097+0000 7f2cfb7fe640 1 -- 172.21.15.112:0/2095336210 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf40078e0 con 0x7f2cfc0ca750 2024-04-25T01:26:12.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.097+0000 7f2cfb7fe640 1 -- 172.21.15.112:0/2095336210 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4007cb0 con 0x7f2cfc0ca750 2024-04-25T01:26:12.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.098+0000 7f2d02cb3640 1 -- 172.21.15.112:0/2095336210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cfc0ca750 msgr2=0x7f2cfc159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.098+0000 7f2d02cb3640 1 --2- 172.21.15.112:0/2095336210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cfc0ca750 0x7f2cfc159420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc068e60 tx=0x7f2cf402a680 comp rx=0 tx=0).stop 2024-04-25T01:26:12.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.098+0000 7f2d02cb3640 1 -- 172.21.15.112:0/2095336210 shutdown_connections 2024-04-25T01:26:12.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.098+0000 7f2d02cb3640 1 --2- 172.21.15.112:0/2095336210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cfc0ca750 0x7f2cfc159420 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.098+0000 7f2d02cb3640 1 --2- 172.21.15.112:0/2095336210 >> v2:172.21.15.187:3300/0 conn(0x7f2cfc06c420 0x7f2cfc0c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.098+0000 7f2d02cb3640 1 -- 172.21.15.112:0/2095336210 >> 172.21.15.112:0/2095336210 conn(0x7f2cfc0cb310 msgr2=0x7f2cfc0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.098+0000 7f2d02cb3640 1 -- 172.21.15.112:0/2095336210 shutdown_connections 2024-04-25T01:26:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.098+0000 7f2d02cb3640 1 -- 172.21.15.112:0/2095336210 wait complete. 2024-04-25T01:26:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.099+0000 7f2d02cb3640 1 Processor -- start 2024-04-25T01:26:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.099+0000 7f2d02cb3640 1 -- start start 2024-04-25T01:26:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.100+0000 7f2d02cb3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cfc06c420 0x7f2cfc0c1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.100+0000 7f2d02cb3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cfc0c5d20 0x7f2cfc1f06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.100+0000 7f2d02cb3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cfc0ca750 0x7f2cfc0af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.100+0000 7f2d02cb3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc15ae10 con 0x7f2cfc06c420 2024-04-25T01:26:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.100+0000 7f2d02cb3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc15a8f0 con 0x7f2cfc0c5d20 2024-04-25T01:26:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.100+0000 7f2d02cb3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2cfc15af90 con 0x7f2cfc0ca750 2024-04-25T01:26:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.100+0000 7f2d00a4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cfc06c420 0x7f2cfc0c1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.100+0000 7f2d00a4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cfc06c420 0x7f2cfc0c1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43706/0 (socket says 172.21.15.112:43706) 2024-04-25T01:26:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.100+0000 7f2d00a4f640 1 -- 172.21.15.112:0/972988669 learned_addr learned my addr 172.21.15.112:0/972988669 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.100+0000 7f2d01250640 1 --2- 172.21.15.112:0/972988669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cfc0ca750 0x7f2cfc0af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.100+0000 7f2cfbfff640 1 --2- 172.21.15.112:0/972988669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cfc0c5d20 0x7f2cfc1f06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.100+0000 7f2d00a4f640 1 -- 172.21.15.112:0/972988669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cfc0ca750 msgr2=0x7f2cfc0af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.100+0000 7f2d00a4f640 1 --2- 172.21.15.112:0/972988669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cfc0ca750 0x7f2cfc0af250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.100+0000 7f2d00a4f640 1 -- 172.21.15.112:0/972988669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cfc0c5d20 msgr2=0x7f2cfc1f06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.100+0000 7f2d00a4f640 1 --2- 172.21.15.112:0/972988669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cfc0c5d20 0x7f2cfc1f06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.100+0000 7f2d00a4f640 1 -- 172.21.15.112:0/972988669 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cfc1f0c10 con 0x7f2cfc06c420 2024-04-25T01:26:12.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.100+0000 7f2d01250640 1 --2- 172.21.15.112:0/972988669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cfc0ca750 0x7f2cfc0af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:12.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.100+0000 7f2cfbfff640 1 --2- 172.21.15.112:0/972988669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cfc0c5d20 0x7f2cfc1f06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:12.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.100+0000 7f2d00a4f640 1 --2- 172.21.15.112:0/972988669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cfc06c420 0x7f2cfc0c1a60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2cf000c780 tx=0x7f2cf000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:12.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.100+0000 7f2cf97fa640 1 -- 172.21.15.112:0/972988669 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0018070 con 0x7f2cfc06c420 2024-04-25T01:26:12.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.101+0000 7f2cf97fa640 1 -- 172.21.15.112:0/972988669 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf00072d0 con 0x7f2cfc06c420 2024-04-25T01:26:12.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.101+0000 7f2d02cb3640 1 -- 172.21.15.112:0/972988669 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cfc1f3380 con 0x7f2cfc06c420 2024-04-25T01:26:12.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.101+0000 7f2cf97fa640 1 -- 172.21.15.112:0/972988669 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf00047e0 con 0x7f2cfc06c420 2024-04-25T01:26:12.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.101+0000 7f2d02cb3640 1 -- 172.21.15.112:0/972988669 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cfc0af790 con 0x7f2cfc06c420 2024-04-25T01:26:12.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.101+0000 7f2d02cb3640 1 -- 172.21.15.112:0/972988669 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cfc04f500 con 0x7f2cfc06c420 2024-04-25T01:26:12.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.102+0000 7f2cf97fa640 1 -- 172.21.15.112:0/972988669 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2cf001b070 con 0x7f2cfc06c420 2024-04-25T01:26:12.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.102+0000 7f2cf97fa640 1 --2- 172.21.15.112:0/972988669 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ce006be30 0x7f2ce006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.102+0000 7f2cf97fa640 1 -- 172.21.15.112:0/972988669 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f2cf008a430 con 0x7f2cfc06c420 2024-04-25T01:26:12.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.102+0000 7f2cfbfff640 1 --2- 172.21.15.112:0/972988669 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ce006be30 0x7f2ce006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.103+0000 7f2cfbfff640 1 --2- 172.21.15.112:0/972988669 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ce006be30 0x7f2ce006e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0bf200 tx=0x7f2ce8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:12.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.106+0000 7f2cf97fa640 1 -- 172.21.15.112:0/972988669 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2cfc0c7390 con 0x7f2cfc06c420 2024-04-25T01:26:12.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.279+0000 7f2d02cb3640 1 -- 172.21.15.112:0/972988669 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) -- 0x7f2cfc06beb0 con 0x7f2cfc06c420 2024-04-25T01:26:12.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.279+0000 7f2cf97fa640 1 -- 172.21.15.112:0/972988669 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7f2cf0058b40 con 0x7f2cfc06c420 2024-04-25T01:26:12.259 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T01:26:12.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.281+0000 7f2d02cb3640 1 -- 172.21.15.112:0/972988669 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ce006be30 msgr2=0x7f2ce006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.281+0000 7f2d02cb3640 1 --2- 172.21.15.112:0/972988669 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ce006be30 0x7f2ce006e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0bf200 tx=0x7f2ce8008040 comp rx=0 tx=0).stop 2024-04-25T01:26:12.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.281+0000 7f2d02cb3640 1 -- 172.21.15.112:0/972988669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cfc06c420 msgr2=0x7f2cfc0c1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.281+0000 7f2d02cb3640 1 --2- 172.21.15.112:0/972988669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cfc06c420 0x7f2cfc0c1a60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2cf000c780 tx=0x7f2cf000cc40 comp rx=0 tx=0).stop 2024-04-25T01:26:12.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.281+0000 7f2d02cb3640 1 -- 172.21.15.112:0/972988669 shutdown_connections 2024-04-25T01:26:12.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.281+0000 7f2d02cb3640 1 --2- 172.21.15.112:0/972988669 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ce006be30 0x7f2ce006e2f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.281+0000 7f2d02cb3640 1 --2- 172.21.15.112:0/972988669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cfc0ca750 0x7f2cfc0af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.281+0000 7f2d02cb3640 1 --2- 172.21.15.112:0/972988669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cfc0c5d20 0x7f2cfc1f06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.281+0000 7f2d02cb3640 1 --2- 172.21.15.112:0/972988669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cfc06c420 0x7f2cfc0c1a60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.281+0000 7f2d02cb3640 1 -- 172.21.15.112:0/972988669 >> 172.21.15.112:0/972988669 conn(0x7f2cfc0cb310 msgr2=0x7f2cfc0abb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:12.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.281+0000 7f2d02cb3640 1 -- 172.21.15.112:0/972988669 shutdown_connections 2024-04-25T01:26:12.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.281+0000 7f2d02cb3640 1 -- 172.21.15.112:0/972988669 wait complete. 2024-04-25T01:26:12.271 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-04-25T01:26:12.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.391+0000 7f7febc93640 1 Processor -- start 2024-04-25T01:26:12.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.391+0000 7f7febc93640 1 -- start start 2024-04-25T01:26:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.392+0000 7f7febc93640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7fe40ca8a0 0x7f7fe406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.392+0000 7f7febc93640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fe41534a0 con 0x7f7fe40c9250 2024-04-25T01:26:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.392+0000 7f7febc93640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fe40cae30 con 0x7f7fe40ca370 2024-04-25T01:26:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.392+0000 7f7febc93640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7fe4156990 con 0x7f7fe40ca8a0 2024-04-25T01:26:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.392+0000 7f7fe9a2f640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7fe40c9250 0x7f7fe406bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38114/0 (socket says 172.21.15.112:38114) 2024-04-25T01:26:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.392+0000 7f7fe9a2f640 1 -- 172.21.15.112:0/530907211 learned_addr learned my addr 172.21.15.112:0/530907211 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.392+0000 7f7fe8a2d640 1 -- 172.21.15.112:0/530907211 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2712367491 0 0) 0x7f7fe41534a0 con 0x7f7fe40c9250 2024-04-25T01:26:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.392+0000 7f7fe8a2d640 1 -- 172.21.15.112:0/530907211 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fd0003520 con 0x7f7fe40c9250 2024-04-25T01:26:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.392+0000 7f7fea230640 1 --2- 172.21.15.112:0/530907211 >> v2:172.21.15.187:3300/0 conn(0x7f7fe40ca8a0 0x7f7fe406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fe8a2d640 1 -- 172.21.15.112:0/530907211 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2806854220 0 0) 0x7f7fd0003520 con 0x7f7fe40c9250 2024-04-25T01:26:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fe8a2d640 1 -- 172.21.15.112:0/530907211 >> v2:172.21.15.187:3300/0 conn(0x7f7fe40ca8a0 msgr2=0x7f7fe406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fe8a2d640 1 --2- 172.21.15.112:0/530907211 >> v2:172.21.15.187:3300/0 conn(0x7f7fe40ca8a0 0x7f7fe406c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fe8a2d640 1 -- 172.21.15.112:0/530907211 >> v1:172.21.15.187:6789/0 conn(0x7f7fe40ca370 legacy=0x7f7fe415e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fe8a2d640 1 -- 172.21.15.112:0/530907211 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe40c9cd0 con 0x7f7fe40c9250 2024-04-25T01:26:12.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fe8a2d640 1 -- 172.21.15.112:0/530907211 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f7fe00019f0 con 0x7f7fe40c9250 2024-04-25T01:26:12.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fe8a2d640 1 -- 172.21.15.112:0/530907211 >> v1:172.21.15.112:6789/0 conn(0x7f7fe40c9250 legacy=0x7f7fe406bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fe8a2d640 1 --2- 172.21.15.112:0/530907211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fd00039b0 0x7f7fd0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fe8a2d640 1 --2- 172.21.15.112:0/530907211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fd0004800 0x7f7fd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fe8a2d640 1 --2- 172.21.15.112:0/530907211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd00051c0 0x7f7fd000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fe8a2d640 1 -- 172.21.15.112:0/530907211 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe4156990 con 0x7f7fd00039b0 2024-04-25T01:26:12.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fe8a2d640 1 -- 172.21.15.112:0/530907211 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd000c160 con 0x7f7fd00051c0 2024-04-25T01:26:12.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fe8a2d640 1 -- 172.21.15.112:0/530907211 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd000c360 con 0x7f7fd0004800 2024-04-25T01:26:12.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fe9a2f640 1 --2- 172.21.15.112:0/530907211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fd00039b0 0x7f7fd0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fea230640 1 --2- 172.21.15.112:0/530907211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd00051c0 0x7f7fd000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fe922e640 1 --2- 172.21.15.112:0/530907211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fd0004800 0x7f7fd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fe9a2f640 1 -- 172.21.15.112:0/530907211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fd0004800 msgr2=0x7f7fd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fe9a2f640 1 --2- 172.21.15.112:0/530907211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fd0004800 0x7f7fd0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fe9a2f640 1 -- 172.21.15.112:0/530907211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd00051c0 msgr2=0x7f7fd000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fe9a2f640 1 --2- 172.21.15.112:0/530907211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd00051c0 0x7f7fd000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fea230640 1 -- 172.21.15.112:0/530907211 reap_dead start 2024-04-25T01:26:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.393+0000 7f7fe9a2f640 1 -- 172.21.15.112:0/530907211 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd0003520 con 0x7f7fd00039b0 2024-04-25T01:26:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.394+0000 7f7fe9a2f640 1 --2- 172.21.15.112:0/530907211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fd00039b0 0x7f7fd0003dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000a850 tx=0x7f7fd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d17b1072871792b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.394+0000 7f7fe8a2d640 1 -- 172.21.15.112:0/530907211 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe000a4a0 con 0x7f7fd00039b0 2024-04-25T01:26:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.394+0000 7f7fe8a2d640 1 -- 172.21.15.112:0/530907211 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe002f070 con 0x7f7fd00039b0 2024-04-25T01:26:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.394+0000 7f7fe8a2d640 1 -- 172.21.15.112:0/530907211 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe002bcb0 con 0x7f7fd00039b0 2024-04-25T01:26:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.394+0000 7f7febc93640 1 -- 172.21.15.112:0/530907211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fd00039b0 msgr2=0x7f7fd0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.394+0000 7f7febc93640 1 --2- 172.21.15.112:0/530907211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fd00039b0 0x7f7fd0003dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000a850 tx=0x7f7fd0005830 comp rx=0 tx=0).stop 2024-04-25T01:26:12.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.395+0000 7f7febc93640 1 -- 172.21.15.112:0/530907211 shutdown_connections 2024-04-25T01:26:12.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.395+0000 7f7febc93640 1 --2- 172.21.15.112:0/530907211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fd00039b0 0x7f7fd0003dd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.395+0000 7f7febc93640 1 -- 172.21.15.112:0/530907211 >> 172.21.15.112:0/530907211 conn(0x7f7fe40cb310 msgr2=0x7f7fe40cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:12.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.395+0000 7f7febc93640 1 -- 172.21.15.112:0/530907211 shutdown_connections 2024-04-25T01:26:12.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.395+0000 7f7febc93640 1 -- 172.21.15.112:0/530907211 wait complete. 2024-04-25T01:26:12.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.396+0000 7f7febc93640 1 Processor -- start 2024-04-25T01:26:12.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.396+0000 7f7febc93640 1 -- start start 2024-04-25T01:26:12.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.396+0000 7f7febc93640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fd00039b0 0x7f7fe4158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.396+0000 7f7febc93640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fe40c3a80 0x7f7fe414e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.396+0000 7f7febc93640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fe40c3f00 0x7f7fe41eb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.396+0000 7f7febc93640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe4155eb0 con 0x7f7fd00039b0 2024-04-25T01:26:12.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.396+0000 7f7febc93640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe4156500 con 0x7f7fe40c3a80 2024-04-25T01:26:12.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.396+0000 7f7febc93640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7fe4156190 con 0x7f7fe40c3f00 2024-04-25T01:26:12.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.396+0000 7f7fe9a2f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fd00039b0 0x7f7fe4158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.396+0000 7f7fe9a2f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fd00039b0 0x7f7fe4158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43724/0 (socket says 172.21.15.112:43724) 2024-04-25T01:26:12.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.396+0000 7f7fe9a2f640 1 -- 172.21.15.112:0/3580431350 learned_addr learned my addr 172.21.15.112:0/3580431350 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:12.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.397+0000 7f7fea230640 1 --2- 172.21.15.112:0/3580431350 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fe40c3f00 0x7f7fe41eb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.397+0000 7f7fe922e640 1 --2- 172.21.15.112:0/3580431350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fe40c3a80 0x7f7fe414e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.397+0000 7f7fe9a2f640 1 -- 172.21.15.112:0/3580431350 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fe40c3f00 msgr2=0x7f7fe41eb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.397+0000 7f7fe9a2f640 1 --2- 172.21.15.112:0/3580431350 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fe40c3f00 0x7f7fe41eb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.397+0000 7f7fe9a2f640 1 -- 172.21.15.112:0/3580431350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fe40c3a80 msgr2=0x7f7fe414e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.397+0000 7f7fe9a2f640 1 --2- 172.21.15.112:0/3580431350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fe40c3a80 0x7f7fe414e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.397+0000 7f7fe9a2f640 1 -- 172.21.15.112:0/3580431350 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe40c1540 con 0x7f7fd00039b0 2024-04-25T01:26:12.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.397+0000 7f7fe9a2f640 1 --2- 172.21.15.112:0/3580431350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fd00039b0 0x7f7fe4158060 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000a970 tx=0x7f7fe002d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:12.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.397+0000 7f7fda7fc640 1 -- 172.21.15.112:0/3580431350 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0034670 con 0x7f7fd00039b0 2024-04-25T01:26:12.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.397+0000 7f7fda7fc640 1 -- 172.21.15.112:0/3580431350 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe002be50 con 0x7f7fd00039b0 2024-04-25T01:26:12.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.397+0000 7f7febc93640 1 -- 172.21.15.112:0/3580431350 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe40c1820 con 0x7f7fd00039b0 2024-04-25T01:26:12.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.397+0000 7f7fda7fc640 1 -- 172.21.15.112:0/3580431350 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0034a10 con 0x7f7fd00039b0 2024-04-25T01:26:12.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.397+0000 7f7febc93640 1 -- 172.21.15.112:0/3580431350 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe40c1de0 con 0x7f7fd00039b0 2024-04-25T01:26:12.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.398+0000 7f7febc93640 1 -- 172.21.15.112:0/3580431350 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fe404f500 con 0x7f7fd00039b0 2024-04-25T01:26:12.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.398+0000 7f7fda7fc640 1 -- 172.21.15.112:0/3580431350 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7fe0034bb0 con 0x7f7fd00039b0 2024-04-25T01:26:12.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.399+0000 7f7fda7fc640 1 --2- 172.21.15.112:0/3580431350 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7fc806be60 0x7f7fc806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.399+0000 7f7fda7fc640 1 -- 172.21.15.112:0/3580431350 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f7fe00ab860 con 0x7f7fd00039b0 2024-04-25T01:26:12.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.399+0000 7f7fe922e640 1 --2- 172.21.15.112:0/3580431350 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7fc806be60 0x7f7fc806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.399+0000 7f7fe922e640 1 --2- 172.21.15.112:0/3580431350 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7fc806be60 0x7f7fc806e320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4000c00 tx=0x7f7fd4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:12.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.403+0000 7f7fda7fc640 1 -- 172.21.15.112:0/3580431350 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7fe0079f70 con 0x7f7fd00039b0 2024-04-25T01:26:12.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.576+0000 7f7febc93640 1 -- 172.21.15.112:0/3580431350 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) -- 0x7f7fe404f500 con 0x7f7fd00039b0 2024-04-25T01:26:12.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.576+0000 7f7fda7fc640 1 -- 172.21.15.112:0/3580431350 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7f7fe0079870 con 0x7f7fd00039b0 2024-04-25T01:26:12.556 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T01:26:12.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.578+0000 7f7febc93640 1 -- 172.21.15.112:0/3580431350 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7fc806be60 msgr2=0x7f7fc806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.578+0000 7f7febc93640 1 --2- 172.21.15.112:0/3580431350 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7fc806be60 0x7f7fc806e320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4000c00 tx=0x7f7fd4009040 comp rx=0 tx=0).stop 2024-04-25T01:26:12.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.578+0000 7f7febc93640 1 -- 172.21.15.112:0/3580431350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fd00039b0 msgr2=0x7f7fe4158060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.578+0000 7f7febc93640 1 --2- 172.21.15.112:0/3580431350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fd00039b0 0x7f7fe4158060 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000a970 tx=0x7f7fe002d070 comp rx=0 tx=0).stop 2024-04-25T01:26:12.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.578+0000 7f7febc93640 1 -- 172.21.15.112:0/3580431350 shutdown_connections 2024-04-25T01:26:12.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.578+0000 7f7febc93640 1 --2- 172.21.15.112:0/3580431350 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7fc806be60 0x7f7fc806e320 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.578+0000 7f7febc93640 1 --2- 172.21.15.112:0/3580431350 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fe40c3f00 0x7f7fe41eb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.578+0000 7f7febc93640 1 --2- 172.21.15.112:0/3580431350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fe40c3a80 0x7f7fe414e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.578+0000 7f7febc93640 1 --2- 172.21.15.112:0/3580431350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fd00039b0 0x7f7fe4158060 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.578+0000 7f7febc93640 1 -- 172.21.15.112:0/3580431350 >> 172.21.15.112:0/3580431350 conn(0x7f7fe40cb310 msgr2=0x7f7fe406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:12.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.578+0000 7f7febc93640 1 -- 172.21.15.112:0/3580431350 shutdown_connections 2024-04-25T01:26:12.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.578+0000 7f7febc93640 1 -- 172.21.15.112:0/3580431350 wait complete. 2024-04-25T01:26:12.568 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-04-25T01:26:12.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.689+0000 7f290f9a5640 1 Processor -- start 2024-04-25T01:26:12.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.689+0000 7f290f9a5640 1 -- start start 2024-04-25T01:26:12.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.690+0000 7f290f9a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2908152f60 0x7f29080c4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.690+0000 7f290f9a5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29081557d0 con 0x7f2908158f40 2024-04-25T01:26:12.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.690+0000 7f290f9a5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2908155a90 con 0x7f29080c4a40 2024-04-25T01:26:12.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.690+0000 7f290f9a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f29081566a0 con 0x7f2908152f60 2024-04-25T01:26:12.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.690+0000 7f290df42640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2908158f40 0x7f29081593d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38116/0 (socket says 172.21.15.112:38116) 2024-04-25T01:26:12.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.690+0000 7f290df42640 1 -- 172.21.15.112:0/1755195235 learned_addr learned my addr 172.21.15.112:0/1755195235 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:12.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.690+0000 7f28fffff640 1 -- 172.21.15.112:0/1755195235 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2365076646 0 0) 0x7f29081557d0 con 0x7f2908158f40 2024-04-25T01:26:12.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.690+0000 7f290d741640 1 --2- 172.21.15.112:0/1755195235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2908152f60 0x7f29080c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.690+0000 7f28fffff640 1 -- 172.21.15.112:0/1755195235 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28f4003520 con 0x7f2908158f40 2024-04-25T01:26:12.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f28fffff640 1 -- 172.21.15.112:0/1755195235 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2419732210 0 0) 0x7f28f4003520 con 0x7f2908158f40 2024-04-25T01:26:12.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f28fffff640 1 -- 172.21.15.112:0/1755195235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2908152f60 msgr2=0x7f29080c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:12.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f28fffff640 1 --2- 172.21.15.112:0/1755195235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2908152f60 0x7f29080c4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f28fffff640 1 -- 172.21.15.112:0/1755195235 >> v1:172.21.15.187:6789/0 conn(0x7f29080c4a40 legacy=0x7f29080c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f28fffff640 1 -- 172.21.15.112:0/1755195235 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2908155d70 con 0x7f2908158f40 2024-04-25T01:26:12.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f28fffff640 1 -- 172.21.15.112:0/1755195235 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f2900001670 con 0x7f2908158f40 2024-04-25T01:26:12.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f28fffff640 1 -- 172.21.15.112:0/1755195235 >> v1:172.21.15.112:6789/0 conn(0x7f2908158f40 legacy=0x7f29081593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f28fffff640 1 --2- 172.21.15.112:0/1755195235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f40039e0 0x7f28f4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f28fffff640 1 --2- 172.21.15.112:0/1755195235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2908152f60 0x7f28f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f28fffff640 1 --2- 172.21.15.112:0/1755195235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28f4004d40 0x7f28f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f28fffff640 1 -- 172.21.15.112:0/1755195235 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29081566a0 con 0x7f28f40039e0 2024-04-25T01:26:12.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f28fffff640 1 -- 172.21.15.112:0/1755195235 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f28f400baf0 con 0x7f28f4004d40 2024-04-25T01:26:12.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f28fffff640 1 -- 172.21.15.112:0/1755195235 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f28f400bcf0 con 0x7f2908152f60 2024-04-25T01:26:12.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f290d741640 1 --2- 172.21.15.112:0/1755195235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f40039e0 0x7f28f4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f290cf40640 1 --2- 172.21.15.112:0/1755195235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2908152f60 0x7f28f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f290df42640 1 --2- 172.21.15.112:0/1755195235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28f4004d40 0x7f28f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f290d741640 1 -- 172.21.15.112:0/1755195235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2908152f60 msgr2=0x7f28f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:12.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f290d741640 1 --2- 172.21.15.112:0/1755195235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2908152f60 0x7f28f4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f290d741640 1 -- 172.21.15.112:0/1755195235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28f4004d40 msgr2=0x7f28f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:12.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f290d741640 1 --2- 172.21.15.112:0/1755195235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28f4004d40 0x7f28f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.691+0000 7f290d741640 1 -- 172.21.15.112:0/1755195235 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f4003520 con 0x7f28f40039e0 2024-04-25T01:26:12.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.692+0000 7f290d741640 1 --2- 172.21.15.112:0/1755195235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f40039e0 0x7f28f4003e00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2904009fc0 tx=0x7f28f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254b3e2ac8906156 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:12.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.692+0000 7f28fffff640 1 -- 172.21.15.112:0/1755195235 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904037070 con 0x7f28f40039e0 2024-04-25T01:26:12.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.692+0000 7f28fffff640 1 -- 172.21.15.112:0/1755195235 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29040325f0 con 0x7f28f40039e0 2024-04-25T01:26:12.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.692+0000 7f28fffff640 1 -- 172.21.15.112:0/1755195235 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904009c10 con 0x7f28f40039e0 2024-04-25T01:26:12.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.692+0000 7f290f9a5640 1 -- 172.21.15.112:0/1755195235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f40039e0 msgr2=0x7f28f4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.692+0000 7f290f9a5640 1 --2- 172.21.15.112:0/1755195235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f40039e0 0x7f28f4003e00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2904009fc0 tx=0x7f28f4005830 comp rx=0 tx=0).stop 2024-04-25T01:26:12.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.692+0000 7f290df42640 1 -- 172.21.15.112:0/1755195235 reap_dead start 2024-04-25T01:26:12.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.692+0000 7f290f9a5640 1 -- 172.21.15.112:0/1755195235 shutdown_connections 2024-04-25T01:26:12.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.692+0000 7f290f9a5640 1 -- 172.21.15.112:0/1755195235 >> 172.21.15.112:0/1755195235 conn(0x7f29080c8ef0 msgr2=0x7f29080c92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:12.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.693+0000 7f290f9a5640 1 -- 172.21.15.112:0/1755195235 shutdown_connections 2024-04-25T01:26:12.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.693+0000 7f290f9a5640 1 -- 172.21.15.112:0/1755195235 wait complete. 2024-04-25T01:26:12.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.694+0000 7f290f9a5640 1 Processor -- start 2024-04-25T01:26:12.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.694+0000 7f290f9a5640 1 -- start start 2024-04-25T01:26:12.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.694+0000 7f290f9a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28f40039e0 0x7f29080701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.694+0000 7f290f9a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2908140ea0 0x7f2908140360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.694+0000 7f290f9a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29081408a0 0x7f29081ee9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.694+0000 7f290f9a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2908159ae0 con 0x7f2908140ea0 2024-04-25T01:26:12.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.694+0000 7f290f9a5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2908156330 con 0x7f28f40039e0 2024-04-25T01:26:12.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.694+0000 7f290f9a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f29081ebe80 con 0x7f29081408a0 2024-04-25T01:26:12.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.694+0000 7f290cf40640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2908140ea0 0x7f2908140360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.694+0000 7f290cf40640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2908140ea0 0x7f2908140360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43740/0 (socket says 172.21.15.112:43740) 2024-04-25T01:26:12.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.694+0000 7f290cf40640 1 -- 172.21.15.112:0/587014066 learned_addr learned my addr 172.21.15.112:0/587014066 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:12.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.695+0000 7f290df42640 1 --2- 172.21.15.112:0/587014066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29081408a0 0x7f29081ee9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.695+0000 7f290d741640 1 --2- 172.21.15.112:0/587014066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28f40039e0 0x7f29080701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.695+0000 7f290cf40640 1 -- 172.21.15.112:0/587014066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29081408a0 msgr2=0x7f29081ee9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.695+0000 7f290cf40640 1 --2- 172.21.15.112:0/587014066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29081408a0 0x7f29081ee9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.695+0000 7f290cf40640 1 -- 172.21.15.112:0/587014066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28f40039e0 msgr2=0x7f29080701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.695+0000 7f290cf40640 1 --2- 172.21.15.112:0/587014066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28f40039e0 0x7f29080701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.695+0000 7f290cf40640 1 -- 172.21.15.112:0/587014066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29081ec310 con 0x7f2908140ea0 2024-04-25T01:26:12.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.695+0000 7f290df42640 1 --2- 172.21.15.112:0/587014066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29081408a0 0x7f29081ee9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:12.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.695+0000 7f290cf40640 1 --2- 172.21.15.112:0/587014066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2908140ea0 0x7f2908140360 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f28f800e580 tx=0x7f28f800ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:12.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.695+0000 7f28fdffb640 1 -- 172.21.15.112:0/587014066 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f80052e0 con 0x7f2908140ea0 2024-04-25T01:26:12.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.695+0000 7f28fdffb640 1 -- 172.21.15.112:0/587014066 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28f8018070 con 0x7f2908140ea0 2024-04-25T01:26:12.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.695+0000 7f290f9a5640 1 -- 172.21.15.112:0/587014066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29081416f0 con 0x7f2908140ea0 2024-04-25T01:26:12.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.695+0000 7f28fdffb640 1 -- 172.21.15.112:0/587014066 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8003810 con 0x7f2908140ea0 2024-04-25T01:26:12.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.695+0000 7f290f9a5640 1 -- 172.21.15.112:0/587014066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29081eb550 con 0x7f2908140ea0 2024-04-25T01:26:12.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.696+0000 7f290f9a5640 1 -- 172.21.15.112:0/587014066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2908140cb0 con 0x7f2908140ea0 2024-04-25T01:26:12.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.696+0000 7f28fdffb640 1 -- 172.21.15.112:0/587014066 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f28f80039b0 con 0x7f2908140ea0 2024-04-25T01:26:12.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.697+0000 7f28fdffb640 1 --2- 172.21.15.112:0/587014066 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f28ec06be60 0x7f28ec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.697+0000 7f28fdffb640 1 -- 172.21.15.112:0/587014066 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f28f808ad10 con 0x7f2908140ea0 2024-04-25T01:26:12.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.697+0000 7f290d741640 1 --2- 172.21.15.112:0/587014066 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f28ec06be60 0x7f28ec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.697+0000 7f290d741640 1 --2- 172.21.15.112:0/587014066 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f28ec06be60 0x7f28ec06e320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2904034000 tx=0x7f290402f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:12.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.701+0000 7f28fdffb640 1 -- 172.21.15.112:0/587014066 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f28f80593a0 con 0x7f2908140ea0 2024-04-25T01:26:12.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.873+0000 7f290f9a5640 1 -- 172.21.15.112:0/587014066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) -- 0x7f290804f500 con 0x7f2908140ea0 2024-04-25T01:26:12.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.873+0000 7f28fdffb640 1 -- 172.21.15.112:0/587014066 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7f28f8058ca0 con 0x7f2908140ea0 2024-04-25T01:26:12.853 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T01:26:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.875+0000 7f290f9a5640 1 -- 172.21.15.112:0/587014066 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f28ec06be60 msgr2=0x7f28ec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.875+0000 7f290f9a5640 1 --2- 172.21.15.112:0/587014066 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f28ec06be60 0x7f28ec06e320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2904034000 tx=0x7f290402f000 comp rx=0 tx=0).stop 2024-04-25T01:26:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.875+0000 7f290f9a5640 1 -- 172.21.15.112:0/587014066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2908140ea0 msgr2=0x7f2908140360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.875+0000 7f290f9a5640 1 --2- 172.21.15.112:0/587014066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2908140ea0 0x7f2908140360 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f28f800e580 tx=0x7f28f800ea40 comp rx=0 tx=0).stop 2024-04-25T01:26:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.875+0000 7f290f9a5640 1 -- 172.21.15.112:0/587014066 shutdown_connections 2024-04-25T01:26:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.875+0000 7f290f9a5640 1 --2- 172.21.15.112:0/587014066 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f28ec06be60 0x7f28ec06e320 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.875+0000 7f290f9a5640 1 --2- 172.21.15.112:0/587014066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29081408a0 0x7f29081ee9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.875+0000 7f290f9a5640 1 --2- 172.21.15.112:0/587014066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2908140ea0 0x7f2908140360 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.875+0000 7f290f9a5640 1 --2- 172.21.15.112:0/587014066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28f40039e0 0x7f29080701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.875+0000 7f290f9a5640 1 -- 172.21.15.112:0/587014066 >> 172.21.15.112:0/587014066 conn(0x7f29080c8ef0 msgr2=0x7f29080c7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:12.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.875+0000 7f290f9a5640 1 -- 172.21.15.112:0/587014066 shutdown_connections 2024-04-25T01:26:12.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.875+0000 7f290f9a5640 1 -- 172.21.15.112:0/587014066 wait complete. 2024-04-25T01:26:12.866 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-04-25T01:26:12.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.984+0000 7fd9c331d640 1 Processor -- start 2024-04-25T01:26:12.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.985+0000 7fd9c331d640 1 -- start start 2024-04-25T01:26:12.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.985+0000 7fd9c331d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9bc0c7ab0 0x7fd9bc0c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.985+0000 7fd9c331d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd9bc06d680 0x7fd9bc1563c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.985+0000 7fd9c331d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9bc06db10 con 0x7fd9bc0c5a60 2024-04-25T01:26:12.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.985+0000 7fd9c331d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd9bc0c5ea0 con 0x7fd9bc06d680 2024-04-25T01:26:12.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.985+0000 7fd9c331d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd9bc0c6160 con 0x7fd9bc0c7ab0 2024-04-25T01:26:12.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.986+0000 7fd9c10b9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9bc0c7ab0 0x7fd9bc0c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.986+0000 7fd9c18ba640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd9bc06d680 0x7fd9bc1563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.986+0000 7fd9c10b9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9bc0c7ab0 0x7fd9bc0c7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:56832/0 (socket says 172.21.15.112:56832) 2024-04-25T01:26:12.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.986+0000 7fd9c10b9640 1 -- 172.21.15.112:0/724679500 learned_addr learned my addr 172.21.15.112:0/724679500 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:12.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.986+0000 7fd9c10b9640 1 -- 172.21.15.112:0/724679500 >> v2:172.21.15.187:3300/0 conn(0x7fd9bc06d680 msgr2=0x7fd9bc1563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:12.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.986+0000 7fd9c10b9640 1 --2- 172.21.15.112:0/724679500 >> v2:172.21.15.187:3300/0 conn(0x7fd9bc06d680 0x7fd9bc1563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.986+0000 7fd9c10b9640 1 -- 172.21.15.112:0/724679500 >> v1:172.21.15.187:6789/0 conn(0x7fd9bc0c5a60 legacy=0x7fd9bc0c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.986+0000 7fd9c10b9640 1 -- 172.21.15.112:0/724679500 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9bc06cba0 con 0x7fd9bc0c7ab0 2024-04-25T01:26:12.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.986+0000 7fd9c18ba640 1 --2- 172.21.15.112:0/724679500 >> v2:172.21.15.187:3300/0 conn(0x7fd9bc06d680 0x7fd9bc1563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:26:12.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.987+0000 7fd9c10b9640 1 --2- 172.21.15.112:0/724679500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9bc0c7ab0 0x7fd9bc0c7e90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009870 tx=0x7fd9b802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d3e6266a064ef7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:12.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.987+0000 7fd9b3fff640 1 -- 172.21.15.112:0/724679500 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8037070 con 0x7fd9bc0c7ab0 2024-04-25T01:26:12.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.987+0000 7fd9b3fff640 1 -- 172.21.15.112:0/724679500 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b8032440 con 0x7fd9bc0c7ab0 2024-04-25T01:26:12.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.987+0000 7fd9b3fff640 1 -- 172.21.15.112:0/724679500 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8032810 con 0x7fd9bc0c7ab0 2024-04-25T01:26:12.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.987+0000 7fd9c331d640 1 -- 172.21.15.112:0/724679500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9bc0c7ab0 msgr2=0x7fd9bc0c7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.987+0000 7fd9c331d640 1 --2- 172.21.15.112:0/724679500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9bc0c7ab0 0x7fd9bc0c7e90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009870 tx=0x7fd9b802a270 comp rx=0 tx=0).stop 2024-04-25T01:26:12.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.987+0000 7fd9c331d640 1 -- 172.21.15.112:0/724679500 shutdown_connections 2024-04-25T01:26:12.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.987+0000 7fd9c331d640 1 --2- 172.21.15.112:0/724679500 >> v2:172.21.15.187:3300/0 conn(0x7fd9bc06d680 0x7fd9bc1563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.987+0000 7fd9c331d640 1 --2- 172.21.15.112:0/724679500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9bc0c7ab0 0x7fd9bc0c7e90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.987+0000 7fd9c331d640 1 -- 172.21.15.112:0/724679500 >> 172.21.15.112:0/724679500 conn(0x7fd9bc0cb0e0 msgr2=0x7fd9bc0cb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:12.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.987+0000 7fd9c331d640 1 -- 172.21.15.112:0/724679500 shutdown_connections 2024-04-25T01:26:12.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.987+0000 7fd9c331d640 1 -- 172.21.15.112:0/724679500 wait complete. 2024-04-25T01:26:12.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.988+0000 7fd9c331d640 1 Processor -- start 2024-04-25T01:26:12.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.989+0000 7fd9c331d640 1 -- start start 2024-04-25T01:26:12.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.989+0000 7fd9c331d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9bc06d680 0x7fd9bc1eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.989+0000 7fd9c331d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9bc0c5a60 0x7fd9bc147220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.989+0000 7fd9c331d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9bc0c7ab0 0x7fd9bc147760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.989+0000 7fd9c331d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc1532a0 con 0x7fd9bc06d680 2024-04-25T01:26:12.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.989+0000 7fd9c331d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc06d160 con 0x7fd9bc0c5a60 2024-04-25T01:26:12.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.989+0000 7fd9c331d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd9bc0c6440 con 0x7fd9bc0c7ab0 2024-04-25T01:26:12.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.989+0000 7fd9c10b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9bc06d680 0x7fd9bc1eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.989+0000 7fd9c10b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9bc06d680 0x7fd9bc1eeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43754/0 (socket says 172.21.15.112:43754) 2024-04-25T01:26:12.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.989+0000 7fd9c10b9640 1 -- 172.21.15.112:0/612435898 learned_addr learned my addr 172.21.15.112:0/612435898 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:12.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.989+0000 7fd9c18ba640 1 --2- 172.21.15.112:0/612435898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9bc0c7ab0 0x7fd9bc147760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.990+0000 7fd9c08b8640 1 --2- 172.21.15.112:0/612435898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9bc0c5a60 0x7fd9bc147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.990+0000 7fd9c10b9640 1 -- 172.21.15.112:0/612435898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9bc0c7ab0 msgr2=0x7fd9bc147760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.990+0000 7fd9c10b9640 1 --2- 172.21.15.112:0/612435898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9bc0c7ab0 0x7fd9bc147760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.990+0000 7fd9c10b9640 1 -- 172.21.15.112:0/612435898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9bc0c5a60 msgr2=0x7fd9bc147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:12.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.990+0000 7fd9c10b9640 1 --2- 172.21.15.112:0/612435898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9bc0c5a60 0x7fd9bc147220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:12.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.990+0000 7fd9c10b9640 1 -- 172.21.15.112:0/612435898 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9bc0c42e0 con 0x7fd9bc06d680 2024-04-25T01:26:12.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.990+0000 7fd9c08b8640 1 --2- 172.21.15.112:0/612435898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9bc0c5a60 0x7fd9bc147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:12.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.990+0000 7fd9c10b9640 1 --2- 172.21.15.112:0/612435898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9bc06d680 0x7fd9bc1eeee0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd9b802a2b0 tx=0x7fd9b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:12.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.990+0000 7fd9b1ffb640 1 -- 172.21.15.112:0/612435898 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8041030 con 0x7fd9bc06d680 2024-04-25T01:26:12.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.990+0000 7fd9b1ffb640 1 -- 172.21.15.112:0/612435898 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b8030440 con 0x7fd9bc06d680 2024-04-25T01:26:12.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.990+0000 7fd9b1ffb640 1 -- 172.21.15.112:0/612435898 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8037040 con 0x7fd9bc06d680 2024-04-25T01:26:12.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.990+0000 7fd9c331d640 1 -- 172.21.15.112:0/612435898 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9bc0c3a90 con 0x7fd9bc06d680 2024-04-25T01:26:12.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.990+0000 7fd9c331d640 1 -- 172.21.15.112:0/612435898 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9bc0c2cd0 con 0x7fd9bc06d680 2024-04-25T01:26:12.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.991+0000 7fd9c331d640 1 -- 172.21.15.112:0/612435898 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9bc04f500 con 0x7fd9bc06d680 2024-04-25T01:26:12.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.991+0000 7fd9b1ffb640 1 -- 172.21.15.112:0/612435898 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd9b8044070 con 0x7fd9bc06d680 2024-04-25T01:26:12.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.992+0000 7fd9b1ffb640 1 --2- 172.21.15.112:0/612435898 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd9a006be30 0x7fd9a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:12.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.992+0000 7fd9c08b8640 1 --2- 172.21.15.112:0/612435898 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd9a006be30 0x7fd9a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:12.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.992+0000 7fd9b1ffb640 1 -- 172.21.15.112:0/612435898 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fd9b80ab4e0 con 0x7fd9bc06d680 2024-04-25T01:26:12.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.992+0000 7fd9c08b8640 1 --2- 172.21.15.112:0/612435898 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd9a006be30 0x7fd9a006e2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc159cb0 tx=0x7fd9ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:12.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:12.996+0000 7fd9b1ffb640 1 -- 172.21.15.112:0/612435898 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd9b804b070 con 0x7fd9bc06d680 2024-04-25T01:26:13.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.169+0000 7fd9c331d640 1 -- 172.21.15.112:0/612435898 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) -- 0x7fd9bc153420 con 0x7fd9bc06d680 2024-04-25T01:26:13.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.169+0000 7fd9b1ffb640 1 -- 172.21.15.112:0/612435898 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7fd9b8079bf0 con 0x7fd9bc06d680 2024-04-25T01:26:13.149 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T01:26:13.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.171+0000 7fd9c331d640 1 -- 172.21.15.112:0/612435898 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd9a006be30 msgr2=0x7fd9a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:13.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.171+0000 7fd9c331d640 1 --2- 172.21.15.112:0/612435898 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd9a006be30 0x7fd9a006e2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc159cb0 tx=0x7fd9ac009040 comp rx=0 tx=0).stop 2024-04-25T01:26:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.171+0000 7fd9c331d640 1 -- 172.21.15.112:0/612435898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9bc06d680 msgr2=0x7fd9bc1eeee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.171+0000 7fd9c331d640 1 --2- 172.21.15.112:0/612435898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9bc06d680 0x7fd9bc1eeee0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd9b802a2b0 tx=0x7fd9b80046c0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.171+0000 7fd9c331d640 1 -- 172.21.15.112:0/612435898 shutdown_connections 2024-04-25T01:26:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.171+0000 7fd9c331d640 1 --2- 172.21.15.112:0/612435898 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd9a006be30 0x7fd9a006e2f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.171+0000 7fd9c331d640 1 --2- 172.21.15.112:0/612435898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9bc0c7ab0 0x7fd9bc147760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.171+0000 7fd9c331d640 1 --2- 172.21.15.112:0/612435898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9bc0c5a60 0x7fd9bc147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.171+0000 7fd9c331d640 1 --2- 172.21.15.112:0/612435898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9bc06d680 0x7fd9bc1eeee0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.171+0000 7fd9c331d640 1 -- 172.21.15.112:0/612435898 >> 172.21.15.112:0/612435898 conn(0x7fd9bc0cb0e0 msgr2=0x7fd9bc14d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.171+0000 7fd9c331d640 1 -- 172.21.15.112:0/612435898 shutdown_connections 2024-04-25T01:26:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.171+0000 7fd9c331d640 1 -- 172.21.15.112:0/612435898 wait complete. 2024-04-25T01:26:13.162 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:26:13.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.278+0000 7fe43469e640 1 Processor -- start 2024-04-25T01:26:13.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.278+0000 7fe43469e640 1 -- start start 2024-04-25T01:26:13.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.279+0000 7fe43469e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe42c0ac3b0 0x7fe42c155f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:13.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.279+0000 7fe43469e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe42c1524a0 0x7fe42c156460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:13.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.279+0000 7fe43469e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe42c1529b0 con 0x7fe42c14c3f0 2024-04-25T01:26:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.279+0000 7fe43469e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fe42c155a90 con 0x7fe42c0ac3b0 2024-04-25T01:26:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.279+0000 7fe43469e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe42c1569a0 con 0x7fe42c1524a0 2024-04-25T01:26:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.279+0000 7fe431c39640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe42c0ac3b0 0x7fe42c155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.279+0000 7fe431c39640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe42c0ac3b0 0x7fe42c155f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43768/0 (socket says 172.21.15.112:43768) 2024-04-25T01:26:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.279+0000 7fe432c3b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe42c1524a0 0x7fe42c156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.279+0000 7fe431c39640 1 -- 172.21.15.112:0/1146262380 learned_addr learned my addr 172.21.15.112:0/1146262380 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.279+0000 7fe431c39640 1 -- 172.21.15.112:0/1146262380 >> v2:172.21.15.187:3300/0 conn(0x7fe42c1524a0 msgr2=0x7fe42c156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.279+0000 7fe431c39640 1 --2- 172.21.15.112:0/1146262380 >> v2:172.21.15.187:3300/0 conn(0x7fe42c1524a0 0x7fe42c156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.279+0000 7fe431c39640 1 -- 172.21.15.112:0/1146262380 >> v1:172.21.15.187:6789/0 conn(0x7fe42c14c3f0 legacy=0x7fe42c0c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.279+0000 7fe431c39640 1 -- 172.21.15.112:0/1146262380 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42c152c70 con 0x7fe42c0ac3b0 2024-04-25T01:26:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.280+0000 7fe431c39640 1 --2- 172.21.15.112:0/1146262380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe42c0ac3b0 0x7fe42c155f20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe42c155c10 tx=0x7fe41c02c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6698fb99f123002 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.280+0000 7fe431438640 1 -- 172.21.15.112:0/1146262380 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c032790 con 0x7fe42c0ac3b0 2024-04-25T01:26:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.280+0000 7fe431438640 1 -- 172.21.15.112:0/1146262380 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe41c038070 con 0x7fe42c0ac3b0 2024-04-25T01:26:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.280+0000 7fe431438640 1 -- 172.21.15.112:0/1146262380 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c032b50 con 0x7fe42c0ac3b0 2024-04-25T01:26:13.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.280+0000 7fe43469e640 1 -- 172.21.15.112:0/1146262380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe42c0ac3b0 msgr2=0x7fe42c155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:13.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.280+0000 7fe43469e640 1 --2- 172.21.15.112:0/1146262380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe42c0ac3b0 0x7fe42c155f20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe42c155c10 tx=0x7fe41c02c890 comp rx=0 tx=0).stop 2024-04-25T01:26:13.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.281+0000 7fe43469e640 1 -- 172.21.15.112:0/1146262380 shutdown_connections 2024-04-25T01:26:13.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.281+0000 7fe43469e640 1 --2- 172.21.15.112:0/1146262380 >> v2:172.21.15.187:3300/0 conn(0x7fe42c1524a0 0x7fe42c156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.281+0000 7fe43469e640 1 --2- 172.21.15.112:0/1146262380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe42c0ac3b0 0x7fe42c155f20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.281+0000 7fe43469e640 1 -- 172.21.15.112:0/1146262380 >> 172.21.15.112:0/1146262380 conn(0x7fe42c06b160 msgr2=0x7fe42c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:13.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.281+0000 7fe43469e640 1 -- 172.21.15.112:0/1146262380 shutdown_connections 2024-04-25T01:26:13.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.281+0000 7fe43469e640 1 -- 172.21.15.112:0/1146262380 wait complete. 2024-04-25T01:26:13.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.282+0000 7fe43469e640 1 Processor -- start 2024-04-25T01:26:13.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.282+0000 7fe43469e640 1 -- start start 2024-04-25T01:26:13.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.282+0000 7fe43469e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe42c0ac3b0 0x7fe42c14f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:13.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.282+0000 7fe43469e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe42c14c3f0 0x7fe42c1de000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:13.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.282+0000 7fe43469e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe42c1524a0 0x7fe42c14e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:13.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.282+0000 7fe43469e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c070d60 con 0x7fe42c0ac3b0 2024-04-25T01:26:13.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.282+0000 7fe43469e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c153230 con 0x7fe42c14c3f0 2024-04-25T01:26:13.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.282+0000 7fe43469e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe42c0c32a0 con 0x7fe42c1524a0 2024-04-25T01:26:13.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.282+0000 7fe43243a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe42c0ac3b0 0x7fe42c14f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:13.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.282+0000 7fe43243a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe42c0ac3b0 0x7fe42c14f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43784/0 (socket says 172.21.15.112:43784) 2024-04-25T01:26:13.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.282+0000 7fe43243a640 1 -- 172.21.15.112:0/819202956 learned_addr learned my addr 172.21.15.112:0/819202956 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:13.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.283+0000 7fe432c3b640 1 --2- 172.21.15.112:0/819202956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe42c1524a0 0x7fe42c14e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:13.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.283+0000 7fe431c39640 1 --2- 172.21.15.112:0/819202956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe42c14c3f0 0x7fe42c1de000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:13.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.283+0000 7fe43243a640 1 -- 172.21.15.112:0/819202956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe42c1524a0 msgr2=0x7fe42c14e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:13.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.283+0000 7fe43243a640 1 --2- 172.21.15.112:0/819202956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe42c1524a0 0x7fe42c14e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.283+0000 7fe43243a640 1 -- 172.21.15.112:0/819202956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe42c14c3f0 msgr2=0x7fe42c1de000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:13.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.283+0000 7fe43243a640 1 --2- 172.21.15.112:0/819202956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe42c14c3f0 0x7fe42c1de000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.283+0000 7fe43243a640 1 -- 172.21.15.112:0/819202956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42c145950 con 0x7fe42c0ac3b0 2024-04-25T01:26:13.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.283+0000 7fe432c3b640 1 --2- 172.21.15.112:0/819202956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe42c1524a0 0x7fe42c14e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:13.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.283+0000 7fe431c39640 1 --2- 172.21.15.112:0/819202956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe42c14c3f0 0x7fe42c1de000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:13.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.283+0000 7fe43243a640 1 --2- 172.21.15.112:0/819202956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe42c0ac3b0 0x7fe42c14f480 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe42800c450 tx=0x7fe42800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:13.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.283+0000 7fe422ffd640 1 -- 172.21.15.112:0/819202956 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4280052d0 con 0x7fe42c0ac3b0 2024-04-25T01:26:13.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.283+0000 7fe422ffd640 1 -- 172.21.15.112:0/819202956 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe428005470 con 0x7fe42c0ac3b0 2024-04-25T01:26:13.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.283+0000 7fe43469e640 1 -- 172.21.15.112:0/819202956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe42c1e78b0 con 0x7fe42c0ac3b0 2024-04-25T01:26:13.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.283+0000 7fe422ffd640 1 -- 172.21.15.112:0/819202956 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe428012870 con 0x7fe42c0ac3b0 2024-04-25T01:26:13.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.283+0000 7fe43469e640 1 -- 172.21.15.112:0/819202956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe42c1dbc80 con 0x7fe42c0ac3b0 2024-04-25T01:26:13.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.284+0000 7fe43469e640 1 -- 172.21.15.112:0/819202956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe42c0682a0 con 0x7fe42c0ac3b0 2024-04-25T01:26:13.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.284+0000 7fe422ffd640 1 -- 172.21.15.112:0/819202956 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe428012a10 con 0x7fe42c0ac3b0 2024-04-25T01:26:13.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.285+0000 7fe422ffd640 1 --2- 172.21.15.112:0/819202956 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe41006be60 0x7fe41006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:13.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.285+0000 7fe422ffd640 1 -- 172.21.15.112:0/819202956 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fe42808acf0 con 0x7fe42c0ac3b0 2024-04-25T01:26:13.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.285+0000 7fe431c39640 1 --2- 172.21.15.112:0/819202956 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe41006be60 0x7fe41006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:13.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.285+0000 7fe431c39640 1 --2- 172.21.15.112:0/819202956 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe41006be60 0x7fe41006e320 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe41c037000 tx=0x7fe41c032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:13.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.289+0000 7fe422ffd640 1 -- 172.21.15.112:0/819202956 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe42808f070 con 0x7fe42c0ac3b0 2024-04-25T01:26:13.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.499+0000 7fe43469e640 1 -- 172.21.15.112:0/819202956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe42c0731b0 con 0x7fe42c0ac3b0 2024-04-25T01:26:13.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.500+0000 7fe422ffd640 1 -- 172.21.15.112:0/819202956 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+6491 (secure 0 0 0) 0x7fe428059400 con 0x7fe42c0ac3b0 2024-04-25T01:26:13.479 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:13.480 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/4047219598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4047219598},{"type":"v1","addr":"172.21.15.187:6834","nonce":4047219598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2204640907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2204640907},{"type":"v1","addr":"172.21.15.187:6835","nonce":2204640907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:25:52.687260+0000","modified":"2024-04-25T01:25:55.702719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/2115984531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2115984531},{"type":"v1","addr":"172.21.15.112:6837","nonce":2115984531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1},{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:26:01.740521+0000","modified":"2024-04-25T01:26:04.746955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":8,"state":"up:active","state_seq":6,"addr":"172.21.15.112:6835/278686477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":278686477},{"type":"v1","addr":"172.21.15.112:6835","nonce":278686477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":2}]} 2024-04-25T01:26:13.480 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 11 2024-04-25T01:26:13.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.502+0000 7fe43469e640 1 -- 172.21.15.112:0/819202956 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe41006be60 msgr2=0x7fe41006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:13.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.502+0000 7fe43469e640 1 --2- 172.21.15.112:0/819202956 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe41006be60 0x7fe41006e320 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe41c037000 tx=0x7fe41c032000 comp rx=0 tx=0).stop 2024-04-25T01:26:13.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.502+0000 7fe43469e640 1 -- 172.21.15.112:0/819202956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe42c0ac3b0 msgr2=0x7fe42c14f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:13.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.502+0000 7fe43469e640 1 --2- 172.21.15.112:0/819202956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe42c0ac3b0 0x7fe42c14f480 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe42800c450 tx=0x7fe42800c910 comp rx=0 tx=0).stop 2024-04-25T01:26:13.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.502+0000 7fe43469e640 1 -- 172.21.15.112:0/819202956 shutdown_connections 2024-04-25T01:26:13.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.502+0000 7fe43469e640 1 --2- 172.21.15.112:0/819202956 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe41006be60 0x7fe41006e320 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.502+0000 7fe43469e640 1 --2- 172.21.15.112:0/819202956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe42c1524a0 0x7fe42c14e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.502+0000 7fe43469e640 1 --2- 172.21.15.112:0/819202956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe42c14c3f0 0x7fe42c1de000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.502+0000 7fe43469e640 1 --2- 172.21.15.112:0/819202956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe42c0ac3b0 0x7fe42c14f480 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.502+0000 7fe43469e640 1 -- 172.21.15.112:0/819202956 >> 172.21.15.112:0/819202956 conn(0x7fe42c06b160 msgr2=0x7fe42c150bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:13.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.502+0000 7fe43469e640 1 -- 172.21.15.112:0/819202956 shutdown_connections 2024-04-25T01:26:13.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.502+0000 7fe43469e640 1 -- 172.21.15.112:0/819202956 wait complete. 2024-04-25T01:26:13.492 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:26:13.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.609+0000 7fe96ada2640 1 Processor -- start 2024-04-25T01:26:13.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.609+0000 7fe96ada2640 1 -- start start 2024-04-25T01:26:13.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.610+0000 7fe96ada2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9641596a0 0x7fe964152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:13.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.610+0000 7fe96ada2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe9640c72b0 0x7fe964160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:13.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.610+0000 7fe96ada2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe964159b90 con 0x7fe9641592c0 2024-04-25T01:26:13.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.610+0000 7fe96ada2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fe9640c6e20 con 0x7fe9640c72b0 2024-04-25T01:26:13.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.610+0000 7fe96ada2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe96406c970 con 0x7fe9641596a0 2024-04-25T01:26:13.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.610+0000 7fe96933f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe9640c72b0 0x7fe964160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:13.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.610+0000 7fe96933f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe9640c72b0 0x7fe964160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43798/0 (socket says 172.21.15.112:43798) 2024-04-25T01:26:13.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.610+0000 7fe96933f640 1 -- 172.21.15.112:0/759078503 learned_addr learned my addr 172.21.15.112:0/759078503 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:13.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.610+0000 7fe963fff640 1 --2- 172.21.15.112:0/759078503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9641596a0 0x7fe964152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:13.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.610+0000 7fe96933f640 1 -- 172.21.15.112:0/759078503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9641596a0 msgr2=0x7fe964152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:13.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.610+0000 7fe96933f640 1 --2- 172.21.15.112:0/759078503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9641596a0 0x7fe964152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.610+0000 7fe96933f640 1 -- 172.21.15.112:0/759078503 >> v1:172.21.15.187:6789/0 conn(0x7fe9641592c0 legacy=0x7fe9640c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:13.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.610+0000 7fe96933f640 1 -- 172.21.15.112:0/759078503 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96406cc50 con 0x7fe9640c72b0 2024-04-25T01:26:13.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.611+0000 7fe96933f640 1 --2- 172.21.15.112:0/759078503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9640c72b0 0x7fe964160380 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe9640aac40 tx=0x7fe95c02a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee12b6f2650abff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:13.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.611+0000 7fe9637fe640 1 -- 172.21.15.112:0/759078503 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c036070 con 0x7fe9640c72b0 2024-04-25T01:26:13.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.611+0000 7fe9637fe640 1 -- 172.21.15.112:0/759078503 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe95c031600 con 0x7fe9640c72b0 2024-04-25T01:26:13.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.611+0000 7fe9637fe640 1 -- 172.21.15.112:0/759078503 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c0319d0 con 0x7fe9640c72b0 2024-04-25T01:26:13.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.612+0000 7fe96ada2640 1 -- 172.21.15.112:0/759078503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9640c72b0 msgr2=0x7fe964160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:13.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.612+0000 7fe96ada2640 1 --2- 172.21.15.112:0/759078503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9640c72b0 0x7fe964160380 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe9640aac40 tx=0x7fe95c02a570 comp rx=0 tx=0).stop 2024-04-25T01:26:13.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.612+0000 7fe96ada2640 1 -- 172.21.15.112:0/759078503 shutdown_connections 2024-04-25T01:26:13.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.612+0000 7fe96ada2640 1 --2- 172.21.15.112:0/759078503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9640c72b0 0x7fe964160380 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.612+0000 7fe96ada2640 1 --2- 172.21.15.112:0/759078503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9641596a0 0x7fe964152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.612+0000 7fe96ada2640 1 -- 172.21.15.112:0/759078503 >> 172.21.15.112:0/759078503 conn(0x7fe9640cb170 msgr2=0x7fe9640cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:13.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.612+0000 7fe96ada2640 1 -- 172.21.15.112:0/759078503 shutdown_connections 2024-04-25T01:26:13.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.612+0000 7fe96ada2640 1 -- 172.21.15.112:0/759078503 wait complete. 2024-04-25T01:26:13.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.613+0000 7fe96ada2640 1 Processor -- start 2024-04-25T01:26:13.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.613+0000 7fe96ada2640 1 -- start start 2024-04-25T01:26:13.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe96ada2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9640c72b0 0x7fe96414be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:13.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe96ada2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9641592c0 0x7fe9641575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:13.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe96ada2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9641596a0 0x7fe964157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:13.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe96ada2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe96406c510 con 0x7fe9641592c0 2024-04-25T01:26:13.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe96ada2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe96406c390 con 0x7fe9640c72b0 2024-04-25T01:26:13.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe96ada2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe96406db20 con 0x7fe9641596a0 2024-04-25T01:26:13.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe963fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9641592c0 0x7fe9641575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:13.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe963fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9641592c0 0x7fe9641575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43814/0 (socket says 172.21.15.112:43814) 2024-04-25T01:26:13.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe963fff640 1 -- 172.21.15.112:0/2164168111 learned_addr learned my addr 172.21.15.112:0/2164168111 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:13.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe96933f640 1 --2- 172.21.15.112:0/2164168111 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9641596a0 0x7fe964157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:13.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe963fff640 1 -- 172.21.15.112:0/2164168111 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9641596a0 msgr2=0x7fe964157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:13.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe963fff640 1 --2- 172.21.15.112:0/2164168111 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9641596a0 0x7fe964157b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe963fff640 1 -- 172.21.15.112:0/2164168111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9640c72b0 msgr2=0x7fe96414be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:13.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe968b3e640 1 --2- 172.21.15.112:0/2164168111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9640c72b0 0x7fe96414be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:13.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe963fff640 1 --2- 172.21.15.112:0/2164168111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9640c72b0 0x7fe96414be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe963fff640 1 -- 172.21.15.112:0/2164168111 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9640c4d20 con 0x7fe9641592c0 2024-04-25T01:26:13.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe963fff640 1 --2- 172.21.15.112:0/2164168111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9641592c0 0x7fe9641575c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe95800d5c0 tx=0x7fe95800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:13.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe9617fa640 1 -- 172.21.15.112:0/2164168111 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9580125f0 con 0x7fe9641592c0 2024-04-25T01:26:13.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe9617fa640 1 -- 172.21.15.112:0/2164168111 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe958019070 con 0x7fe9641592c0 2024-04-25T01:26:13.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe96ada2640 1 -- 172.21.15.112:0/2164168111 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe964158cf0 con 0x7fe9641592c0 2024-04-25T01:26:13.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe9617fa640 1 -- 172.21.15.112:0/2164168111 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9580129d0 con 0x7fe9641592c0 2024-04-25T01:26:13.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.614+0000 7fe96ada2640 1 -- 172.21.15.112:0/2164168111 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9641ec6f0 con 0x7fe9641592c0 2024-04-25T01:26:13.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.615+0000 7fe96ada2640 1 -- 172.21.15.112:0/2164168111 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9640679a0 con 0x7fe9641592c0 2024-04-25T01:26:13.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.616+0000 7fe9617fa640 1 -- 172.21.15.112:0/2164168111 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe958012b70 con 0x7fe9641592c0 2024-04-25T01:26:13.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.616+0000 7fe9617fa640 1 --2- 172.21.15.112:0/2164168111 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe95006be30 0x7fe95006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:13.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.616+0000 7fe968b3e640 1 --2- 172.21.15.112:0/2164168111 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe95006be30 0x7fe95006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:13.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.616+0000 7fe9617fa640 1 -- 172.21.15.112:0/2164168111 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fe95808b4b0 con 0x7fe9641592c0 2024-04-25T01:26:13.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.617+0000 7fe968b3e640 1 --2- 172.21.15.112:0/2164168111 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe95006be30 0x7fe95006e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe9640b1260 tx=0x7fe954009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:13.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.620+0000 7fe9617fa640 1 -- 172.21.15.112:0/2164168111 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe958059bc0 con 0x7fe9641592c0 2024-04-25T01:26:13.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.797+0000 7fe96ada2640 1 -- 172.21.15.112:0/2164168111 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9640682a0 con 0x7fe9641592c0 2024-04-25T01:26:13.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.799+0000 7fe9617fa640 1 -- 172.21.15.112:0/2164168111 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20367 (secure 0 0 0) 0x7fe9580594c0 con 0x7fe9641592c0 2024-04-25T01:26:13.779 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:13.779 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:26:03.405271+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-25T01:25:59.478275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-25T01:26:00.719358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:26:13.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.801+0000 7fe96ada2640 1 -- 172.21.15.112:0/2164168111 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe95006be30 msgr2=0x7fe95006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:13.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.801+0000 7fe96ada2640 1 --2- 172.21.15.112:0/2164168111 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe95006be30 0x7fe95006e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe9640b1260 tx=0x7fe954009040 comp rx=0 tx=0).stop 2024-04-25T01:26:13.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.801+0000 7fe96ada2640 1 -- 172.21.15.112:0/2164168111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9641592c0 msgr2=0x7fe9641575c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:13.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.801+0000 7fe96ada2640 1 --2- 172.21.15.112:0/2164168111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9641592c0 0x7fe9641575c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe95800d5c0 tx=0x7fe95800da80 comp rx=0 tx=0).stop 2024-04-25T01:26:13.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.801+0000 7fe96ada2640 1 -- 172.21.15.112:0/2164168111 shutdown_connections 2024-04-25T01:26:13.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.801+0000 7fe96ada2640 1 --2- 172.21.15.112:0/2164168111 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe95006be30 0x7fe95006e2f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.801+0000 7fe96ada2640 1 --2- 172.21.15.112:0/2164168111 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9641596a0 0x7fe964157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.801+0000 7fe96ada2640 1 --2- 172.21.15.112:0/2164168111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9641592c0 0x7fe9641575c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.801+0000 7fe96ada2640 1 --2- 172.21.15.112:0/2164168111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9640c72b0 0x7fe96414be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.801+0000 7fe96ada2640 1 -- 172.21.15.112:0/2164168111 >> 172.21.15.112:0/2164168111 conn(0x7fe9640cb170 msgr2=0x7fe9640aac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:13.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.801+0000 7fe96ada2640 1 -- 172.21.15.112:0/2164168111 shutdown_connections 2024-04-25T01:26:13.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.801+0000 7fe96ada2640 1 -- 172.21.15.112:0/2164168111 wait complete. 2024-04-25T01:26:13.791 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:25:40.679119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-04-25T01:25:50.593045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-04-25T01:25:51.664930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-04-25T01:25:59.478275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-04-25T01:26:00.719358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:26:13.792 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:26:13.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.911+0000 7fad398a5640 1 Processor -- start 2024-04-25T01:26:13.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.911+0000 7fad398a5640 1 -- start start 2024-04-25T01:26:13.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.912+0000 7fad398a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad340c9250 0x7fad340c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:13.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.912+0000 7fad398a5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fad340ca370 0x7fad3406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:13.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.912+0000 7fad398a5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fad340ca8a0 0x7fad3406c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.912+0000 7fad398a5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fad34156990 con 0x7fad340ca370 2024-04-25T01:26:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.912+0000 7fad398a5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fad34156500 con 0x7fad340ca8a0 2024-04-25T01:26:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.912+0000 7fad398a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fad341534a0 con 0x7fad340c9250 2024-04-25T01:26:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.912+0000 7fad327fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fad340ca370 0x7fad3406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.912+0000 7fad327fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fad340ca370 0x7fad3406bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43824/0 (socket says 172.21.15.112:43824) 2024-04-25T01:26:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.912+0000 7fad327fc640 1 -- 172.21.15.112:0/2798477023 learned_addr learned my addr 172.21.15.112:0/2798477023 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.912+0000 7fad32ffd640 1 --2- 172.21.15.112:0/2798477023 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad340c9250 0x7fad340c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.912+0000 7fad337fe640 1 --2- 172.21.15.112:0/2798477023 >> v2:172.21.15.187:3300/0 conn(0x7fad340ca8a0 0x7fad3406c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.912+0000 7fad327fc640 1 -- 172.21.15.112:0/2798477023 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad340c9250 msgr2=0x7fad340c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.912+0000 7fad327fc640 1 --2- 172.21.15.112:0/2798477023 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad340c9250 0x7fad340c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.912+0000 7fad327fc640 1 -- 172.21.15.112:0/2798477023 >> v2:172.21.15.187:3300/0 conn(0x7fad340ca8a0 msgr2=0x7fad3406c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.912+0000 7fad327fc640 1 --2- 172.21.15.112:0/2798477023 >> v2:172.21.15.187:3300/0 conn(0x7fad340ca8a0 0x7fad3406c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.912+0000 7fad327fc640 1 -- 172.21.15.112:0/2798477023 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3406c790 con 0x7fad340ca370 2024-04-25T01:26:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.913+0000 7fad327fc640 1 --2- 172.21.15.112:0/2798477023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad340ca370 0x7fad3406bd10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fad34073750 tx=0x7fad2802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d519dd41d5c7130 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:13.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.913+0000 7fad31ffb640 1 -- 172.21.15.112:0/2798477023 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28037070 con 0x7fad340ca370 2024-04-25T01:26:13.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.913+0000 7fad31ffb640 1 -- 172.21.15.112:0/2798477023 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad2802ae00 con 0x7fad340ca370 2024-04-25T01:26:13.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.913+0000 7fad31ffb640 1 -- 172.21.15.112:0/2798477023 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28032660 con 0x7fad340ca370 2024-04-25T01:26:13.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.914+0000 7fad398a5640 1 -- 172.21.15.112:0/2798477023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad340ca370 msgr2=0x7fad3406bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:13.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.914+0000 7fad398a5640 1 --2- 172.21.15.112:0/2798477023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad340ca370 0x7fad3406bd10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fad34073750 tx=0x7fad2802a0b0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.914+0000 7fad398a5640 1 -- 172.21.15.112:0/2798477023 shutdown_connections 2024-04-25T01:26:13.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.914+0000 7fad398a5640 1 --2- 172.21.15.112:0/2798477023 >> v2:172.21.15.187:3300/0 conn(0x7fad340ca8a0 0x7fad3406c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.914+0000 7fad398a5640 1 --2- 172.21.15.112:0/2798477023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad340ca370 0x7fad3406bd10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.914+0000 7fad398a5640 1 --2- 172.21.15.112:0/2798477023 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad340c9250 0x7fad340c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.914+0000 7fad398a5640 1 -- 172.21.15.112:0/2798477023 >> 172.21.15.112:0/2798477023 conn(0x7fad340cb310 msgr2=0x7fad340cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:13.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.914+0000 7fad398a5640 1 -- 172.21.15.112:0/2798477023 shutdown_connections 2024-04-25T01:26:13.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.914+0000 7fad398a5640 1 -- 172.21.15.112:0/2798477023 wait complete. 2024-04-25T01:26:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.915+0000 7fad398a5640 1 Processor -- start 2024-04-25T01:26:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.915+0000 7fad398a5640 1 -- start start 2024-04-25T01:26:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad398a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad340c9250 0x7fad341ee910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad398a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad340ca370 0x7fad340bfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad398a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad340ca8a0 0x7fad340c01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad398a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fad34155bd0 con 0x7fad340c9250 2024-04-25T01:26:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad398a5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fad340c9cd0 con 0x7fad340ca370 2024-04-25T01:26:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad398a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fad34155eb0 con 0x7fad340ca8a0 2024-04-25T01:26:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad32ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad340c9250 0x7fad341ee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:13.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad32ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad340c9250 0x7fad341ee910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43840/0 (socket says 172.21.15.112:43840) 2024-04-25T01:26:13.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad32ffd640 1 -- 172.21.15.112:0/2379630438 learned_addr learned my addr 172.21.15.112:0/2379630438 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:13.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad337fe640 1 --2- 172.21.15.112:0/2379630438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad340ca8a0 0x7fad340c01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:13.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad327fc640 1 --2- 172.21.15.112:0/2379630438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad340ca370 0x7fad340bfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:13.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad32ffd640 1 -- 172.21.15.112:0/2379630438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad340ca8a0 msgr2=0x7fad340c01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:13.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad32ffd640 1 --2- 172.21.15.112:0/2379630438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad340ca8a0 0x7fad340c01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad32ffd640 1 -- 172.21.15.112:0/2379630438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad340ca370 msgr2=0x7fad340bfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:13.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad32ffd640 1 --2- 172.21.15.112:0/2379630438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad340ca370 0x7fad340bfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:13.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad32ffd640 1 -- 172.21.15.112:0/2379630438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad340c0730 con 0x7fad340c9250 2024-04-25T01:26:13.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad327fc640 1 --2- 172.21.15.112:0/2379630438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad340ca370 0x7fad340bfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:13.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad32ffd640 1 --2- 172.21.15.112:0/2379630438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad340c9250 0x7fad341ee910 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fad24007e90 tx=0x7fad2400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:13.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad1bfff640 1 -- 172.21.15.112:0/2379630438 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24014620 con 0x7fad340c9250 2024-04-25T01:26:13.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad1bfff640 1 -- 172.21.15.112:0/2379630438 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad24019070 con 0x7fad340c9250 2024-04-25T01:26:13.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad1bfff640 1 -- 172.21.15.112:0/2379630438 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24014a00 con 0x7fad340c9250 2024-04-25T01:26:13.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad398a5640 1 -- 172.21.15.112:0/2379630438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad341ef540 con 0x7fad340c9250 2024-04-25T01:26:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.916+0000 7fad398a5640 1 -- 172.21.15.112:0/2379630438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad34158fb0 con 0x7fad340c9250 2024-04-25T01:26:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.917+0000 7fad398a5640 1 -- 172.21.15.112:0/2379630438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad3404f500 con 0x7fad340c9250 2024-04-25T01:26:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.918+0000 7fad1bfff640 1 -- 172.21.15.112:0/2379630438 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad24014ba0 con 0x7fad340c9250 2024-04-25T01:26:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.918+0000 7fad1bfff640 1 --2- 172.21.15.112:0/2379630438 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fad2006be30 0x7fad2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.918+0000 7fad327fc640 1 --2- 172.21.15.112:0/2379630438 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fad2006be30 0x7fad2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.918+0000 7fad1bfff640 1 -- 172.21.15.112:0/2379630438 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fad2408b930 con 0x7fad340c9250 2024-04-25T01:26:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.919+0000 7fad327fc640 1 --2- 172.21.15.112:0/2379630438 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fad2006be30 0x7fad2006e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fad28034000 tx=0x7fad2802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:13.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:13.922+0000 7fad1bfff640 1 -- 172.21.15.112:0/2379630438 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fad340c0730 con 0x7fad340c9250 2024-04-25T01:26:14.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.094+0000 7fad398a5640 1 -- 172.21.15.112:0/2379630438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fad340c9030 con 0x7fad340c9250 2024-04-25T01:26:14.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.094+0000 7fad1bfff640 1 -- 172.21.15.112:0/2379630438 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fad2405a040 con 0x7fad340c9250 2024-04-25T01:26:14.075 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T01:26:14.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.096+0000 7fad398a5640 1 -- 172.21.15.112:0/2379630438 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fad2006be30 msgr2=0x7fad2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.096+0000 7fad398a5640 1 --2- 172.21.15.112:0/2379630438 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fad2006be30 0x7fad2006e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fad28034000 tx=0x7fad2802e000 comp rx=0 tx=0).stop 2024-04-25T01:26:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.096+0000 7fad398a5640 1 -- 172.21.15.112:0/2379630438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad340c9250 msgr2=0x7fad341ee910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.096+0000 7fad398a5640 1 --2- 172.21.15.112:0/2379630438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad340c9250 0x7fad341ee910 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fad24007e90 tx=0x7fad2400e4b0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.096+0000 7fad398a5640 1 -- 172.21.15.112:0/2379630438 shutdown_connections 2024-04-25T01:26:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.096+0000 7fad398a5640 1 --2- 172.21.15.112:0/2379630438 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fad2006be30 0x7fad2006e2f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.096+0000 7fad398a5640 1 --2- 172.21.15.112:0/2379630438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad340ca8a0 0x7fad340c01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.096+0000 7fad398a5640 1 --2- 172.21.15.112:0/2379630438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad340ca370 0x7fad340bfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.096+0000 7fad398a5640 1 --2- 172.21.15.112:0/2379630438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad340c9250 0x7fad341ee910 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.096+0000 7fad398a5640 1 -- 172.21.15.112:0/2379630438 >> 172.21.15.112:0/2379630438 conn(0x7fad340cb310 msgr2=0x7fad340c5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.097+0000 7fad398a5640 1 -- 172.21.15.112:0/2379630438 shutdown_connections 2024-04-25T01:26:14.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.097+0000 7fad398a5640 1 -- 172.21.15.112:0/2379630438 wait complete. 2024-04-25T01:26:14.087 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-04-25T01:26:14.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.207+0000 7fb9ac99b640 1 Processor -- start 2024-04-25T01:26:14.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.207+0000 7fb9ac99b640 1 -- start start 2024-04-25T01:26:14.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.208+0000 7fb9ac99b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb9a8151260 0x7fb9a80c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:14.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.208+0000 7fb9ac99b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a8153c10 con 0x7fb9a80c8740 2024-04-25T01:26:14.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.208+0000 7fb9ac99b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a8153ef0 con 0x7fb9a80c5ef0 2024-04-25T01:26:14.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.208+0000 7fb9ac99b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a81541d0 con 0x7fb9a8151260 2024-04-25T01:26:14.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.208+0000 7fb9a659c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb9a8151260 0x7fb9a80c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:14.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.208+0000 7fb9a6d9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb9a80c8740 0x7fb9a80bf120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38128/0 (socket says 172.21.15.112:38128) 2024-04-25T01:26:14.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.208+0000 7fb9a6d9d640 1 -- 172.21.15.112:0/1478596562 learned_addr learned my addr 172.21.15.112:0/1478596562 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:14.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.209+0000 7fb9a659c640 1 -- 172.21.15.112:0/1478596562 >> v1:172.21.15.187:6789/0 conn(0x7fb9a80c5ef0 legacy=0x7fb9a80c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:14.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.209+0000 7fb9a659c640 1 -- 172.21.15.112:0/1478596562 >> v1:172.21.15.112:6789/0 conn(0x7fb9a80c8740 legacy=0x7fb9a80bf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.209+0000 7fb9a659c640 1 -- 172.21.15.112:0/1478596562 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a81544b0 con 0x7fb9a8151260 2024-04-25T01:26:14.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.209+0000 7fb9a559a640 1 -- 172.21.15.112:0/1478596562 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 327203397 0 0) 0x7fb9a8153c10 con 0x7fb9a80c8740 2024-04-25T01:26:14.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.209+0000 7fb9a659c640 1 --2- 172.21.15.112:0/1478596562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9a8151260 0x7fb9a80c59b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb998009d90 tx=0x7fb99802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b39cdad12f5aaa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:14.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.209+0000 7fb9a559a640 1 -- 172.21.15.112:0/1478596562 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998031600 con 0x7fb9a8151260 2024-04-25T01:26:14.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.209+0000 7fb9a559a640 1 -- 172.21.15.112:0/1478596562 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9980317a0 con 0x7fb9a8151260 2024-04-25T01:26:14.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.210+0000 7fb9a559a640 1 -- 172.21.15.112:0/1478596562 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998031b70 con 0x7fb9a8151260 2024-04-25T01:26:14.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.210+0000 7fb9ac99b640 1 -- 172.21.15.112:0/1478596562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9a8151260 msgr2=0x7fb9a80c59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.210+0000 7fb9ac99b640 1 --2- 172.21.15.112:0/1478596562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9a8151260 0x7fb9a80c59b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb998009d90 tx=0x7fb99802a610 comp rx=0 tx=0).stop 2024-04-25T01:26:14.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.210+0000 7fb9ac99b640 1 -- 172.21.15.112:0/1478596562 shutdown_connections 2024-04-25T01:26:14.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.210+0000 7fb9ac99b640 1 --2- 172.21.15.112:0/1478596562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9a8151260 0x7fb9a80c59b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.210+0000 7fb9ac99b640 1 -- 172.21.15.112:0/1478596562 >> 172.21.15.112:0/1478596562 conn(0x7fb9a80c8f80 msgr2=0x7fb9a80c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:14.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.210+0000 7fb9ac99b640 1 -- 172.21.15.112:0/1478596562 shutdown_connections 2024-04-25T01:26:14.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.210+0000 7fb9ac99b640 1 -- 172.21.15.112:0/1478596562 wait complete. 2024-04-25T01:26:14.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.211+0000 7fb9ac99b640 1 Processor -- start 2024-04-25T01:26:14.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.211+0000 7fb9ac99b640 1 -- start start 2024-04-25T01:26:14.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.212+0000 7fb9ac99b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9a80c5ef0 0x7fb9a81eb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:14.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.212+0000 7fb9ac99b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9a80c8740 0x7fb9a80ace70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:14.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.212+0000 7fb9ac99b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9a8151260 0x7fb9a81eed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:14.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.212+0000 7fb9ac99b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a8157ab0 con 0x7fb9a80c8740 2024-04-25T01:26:14.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.212+0000 7fb9ac99b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a8157930 con 0x7fb9a8151260 2024-04-25T01:26:14.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.212+0000 7fb9ac99b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a8157c30 con 0x7fb9a80c5ef0 2024-04-25T01:26:14.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.212+0000 7fb9a5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9a80c8740 0x7fb9a80ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:14.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.212+0000 7fb9a5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9a80c8740 0x7fb9a80ace70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43868/0 (socket says 172.21.15.112:43868) 2024-04-25T01:26:14.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.212+0000 7fb9a5d9b640 1 -- 172.21.15.112:0/2250571267 learned_addr learned my addr 172.21.15.112:0/2250571267 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:14.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.212+0000 7fb9a659c640 1 --2- 172.21.15.112:0/2250571267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9a80c5ef0 0x7fb9a81eb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:14.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.212+0000 7fb9a6d9d640 1 --2- 172.21.15.112:0/2250571267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9a8151260 0x7fb9a81eed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:14.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.212+0000 7fb9a5d9b640 1 -- 172.21.15.112:0/2250571267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9a80c5ef0 msgr2=0x7fb9a81eb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.212+0000 7fb9a5d9b640 1 --2- 172.21.15.112:0/2250571267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9a80c5ef0 0x7fb9a81eb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.212+0000 7fb9a5d9b640 1 -- 172.21.15.112:0/2250571267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9a8151260 msgr2=0x7fb9a81eed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.212+0000 7fb9a5d9b640 1 --2- 172.21.15.112:0/2250571267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9a8151260 0x7fb9a81eed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.212+0000 7fb9a5d9b640 1 -- 172.21.15.112:0/2250571267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a806dd90 con 0x7fb9a80c8740 2024-04-25T01:26:14.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.213+0000 7fb9a5d9b640 1 --2- 172.21.15.112:0/2250571267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9a80c8740 0x7fb9a80ace70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00b3f0 tx=0x7fb99c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:14.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.213+0000 7fb996ffd640 1 -- 172.21.15.112:0/2250571267 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c015020 con 0x7fb9a80c8740 2024-04-25T01:26:14.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.213+0000 7fb996ffd640 1 -- 172.21.15.112:0/2250571267 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb99c011830 con 0x7fb9a80c8740 2024-04-25T01:26:14.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.213+0000 7fb9ac99b640 1 -- 172.21.15.112:0/2250571267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a806e070 con 0x7fb9a80c8740 2024-04-25T01:26:14.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.213+0000 7fb996ffd640 1 -- 172.21.15.112:0/2250571267 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c011c20 con 0x7fb9a80c8740 2024-04-25T01:26:14.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.213+0000 7fb9ac99b640 1 -- 172.21.15.112:0/2250571267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a80c5470 con 0x7fb9a80c8740 2024-04-25T01:26:14.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.214+0000 7fb9ac99b640 1 -- 172.21.15.112:0/2250571267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a804f500 con 0x7fb9a80c8740 2024-04-25T01:26:14.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.214+0000 7fb996ffd640 1 -- 172.21.15.112:0/2250571267 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb99c018020 con 0x7fb9a80c8740 2024-04-25T01:26:14.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.214+0000 7fb996ffd640 1 --2- 172.21.15.112:0/2250571267 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb98806be30 0x7fb98806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:14.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.214+0000 7fb9a659c640 1 --2- 172.21.15.112:0/2250571267 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb98806be30 0x7fb98806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:14.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.214+0000 7fb996ffd640 1 -- 172.21.15.112:0/2250571267 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fb99c08ada0 con 0x7fb9a80c8740 2024-04-25T01:26:14.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.215+0000 7fb9a659c640 1 --2- 172.21.15.112:0/2250571267 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb98806be30 0x7fb98806e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb998034000 tx=0x7fb99802f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:14.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.218+0000 7fb996ffd640 1 -- 172.21.15.112:0/2250571267 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb99c01f030 con 0x7fb9a80c8740 2024-04-25T01:26:14.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.390+0000 7fb9ac99b640 1 -- 172.21.15.112:0/2250571267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) -- 0x7fb9a806e240 con 0x7fb9a80c8740 2024-04-25T01:26:14.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.391+0000 7fb996ffd640 1 -- 172.21.15.112:0/2250571267 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7fb99c0172c0 con 0x7fb9a80c8740 2024-04-25T01:26:14.370 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T01:26:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.392+0000 7fb9ac99b640 1 -- 172.21.15.112:0/2250571267 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb98806be30 msgr2=0x7fb98806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.392+0000 7fb9ac99b640 1 --2- 172.21.15.112:0/2250571267 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb98806be30 0x7fb98806e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb998034000 tx=0x7fb99802f000 comp rx=0 tx=0).stop 2024-04-25T01:26:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.392+0000 7fb9ac99b640 1 -- 172.21.15.112:0/2250571267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9a80c8740 msgr2=0x7fb9a80ace70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.392+0000 7fb9ac99b640 1 --2- 172.21.15.112:0/2250571267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9a80c8740 0x7fb9a80ace70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00b3f0 tx=0x7fb99c00b8b0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.392+0000 7fb9ac99b640 1 -- 172.21.15.112:0/2250571267 shutdown_connections 2024-04-25T01:26:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.392+0000 7fb9ac99b640 1 --2- 172.21.15.112:0/2250571267 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb98806be30 0x7fb98806e2f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.392+0000 7fb9ac99b640 1 --2- 172.21.15.112:0/2250571267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9a8151260 0x7fb9a81eed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.392+0000 7fb9ac99b640 1 --2- 172.21.15.112:0/2250571267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9a80c8740 0x7fb9a80ace70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.392+0000 7fb9ac99b640 1 --2- 172.21.15.112:0/2250571267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9a80c5ef0 0x7fb9a81eb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.392+0000 7fb9ac99b640 1 -- 172.21.15.112:0/2250571267 >> 172.21.15.112:0/2250571267 conn(0x7fb9a80c8f80 msgr2=0x7fb9a80c3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.392+0000 7fb9ac99b640 1 -- 172.21.15.112:0/2250571267 shutdown_connections 2024-04-25T01:26:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.392+0000 7fb9ac99b640 1 -- 172.21.15.112:0/2250571267 wait complete. 2024-04-25T01:26:14.383 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-04-25T01:26:14.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.503+0000 7efec0b10640 1 Processor -- start 2024-04-25T01:26:14.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.503+0000 7efec0b10640 1 -- start start 2024-04-25T01:26:14.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.504+0000 7efec0b10640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efebc06d460 0x7efebc071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:14.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.504+0000 7efec0b10640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efebc06f290 con 0x7efebc0724f0 2024-04-25T01:26:14.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.504+0000 7efec0b10640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efebc06efb0 con 0x7efebc14fbe0 2024-04-25T01:26:14.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.504+0000 7efec0b10640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efebc14ffc0 con 0x7efebc06d460 2024-04-25T01:26:14.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.504+0000 7efebad9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7efebc0724f0 0x7efebc15ce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38142/0 (socket says 172.21.15.112:38142) 2024-04-25T01:26:14.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.504+0000 7efebad9d640 1 -- 172.21.15.112:0/2730822632 learned_addr learned my addr 172.21.15.112:0/2730822632 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:14.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.504+0000 7efeb9d9b640 1 --2- 172.21.15.112:0/2730822632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efebc06d460 0x7efebc071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:14.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.504+0000 7efeb959a640 1 -- 172.21.15.112:0/2730822632 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 971807130 0 0) 0x7efebc06f290 con 0x7efebc0724f0 2024-04-25T01:26:14.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.504+0000 7efeb959a640 1 -- 172.21.15.112:0/2730822632 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efea4005730 con 0x7efebc0724f0 2024-04-25T01:26:14.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.504+0000 7efeb959a640 1 -- 172.21.15.112:0/2730822632 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 949080715 0 0) 0x7efebc06efb0 con 0x7efebc14fbe0 2024-04-25T01:26:14.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.504+0000 7efeb9d9b640 1 -- 172.21.15.112:0/2730822632 >> v1:172.21.15.187:6789/0 conn(0x7efebc14fbe0 legacy=0x7efebc0c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.504+0000 7efeb9d9b640 1 -- 172.21.15.112:0/2730822632 >> v1:172.21.15.112:6789/0 conn(0x7efebc0724f0 legacy=0x7efebc15ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.504+0000 7efeb9d9b640 1 -- 172.21.15.112:0/2730822632 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efebc0a9530 con 0x7efebc06d460 2024-04-25T01:26:14.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.505+0000 7efeb9d9b640 1 --2- 172.21.15.112:0/2730822632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efebc06d460 0x7efebc071fb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efeb00097d0 tx=0x7efeb002a0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e82999d165c06b56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:14.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.505+0000 7efeb959a640 1 -- 172.21.15.112:0/2730822632 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0037070 con 0x7efebc06d460 2024-04-25T01:26:14.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.505+0000 7efeb959a640 1 -- 172.21.15.112:0/2730822632 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeb002ab70 con 0x7efebc06d460 2024-04-25T01:26:14.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.505+0000 7efeb959a640 1 -- 172.21.15.112:0/2730822632 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb4001670 con 0x7efebc06d460 2024-04-25T01:26:14.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.506+0000 7efec0b10640 1 -- 172.21.15.112:0/2730822632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efebc06d460 msgr2=0x7efebc071fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.506+0000 7efec0b10640 1 --2- 172.21.15.112:0/2730822632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efebc06d460 0x7efebc071fb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efeb00097d0 tx=0x7efeb002a0e0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.506+0000 7efec0b10640 1 -- 172.21.15.112:0/2730822632 shutdown_connections 2024-04-25T01:26:14.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.506+0000 7efec0b10640 1 --2- 172.21.15.112:0/2730822632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efebc06d460 0x7efebc071fb0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.506+0000 7efec0b10640 1 -- 172.21.15.112:0/2730822632 >> 172.21.15.112:0/2730822632 conn(0x7efebc0c3a10 msgr2=0x7efebc0c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:14.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.506+0000 7efec0b10640 1 -- 172.21.15.112:0/2730822632 shutdown_connections 2024-04-25T01:26:14.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.506+0000 7efec0b10640 1 -- 172.21.15.112:0/2730822632 wait complete. 2024-04-25T01:26:14.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.507+0000 7efec0b10640 1 Processor -- start 2024-04-25T01:26:14.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.507+0000 7efec0b10640 1 -- start start 2024-04-25T01:26:14.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efec0b10640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efebc06d460 0x7efebc1e6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:14.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efec0b10640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efebc0724f0 0x7efebc14c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:14.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efec0b10640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efebc14fbe0 0x7efebc1546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:14.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efec0b10640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efebc0c3030 con 0x7efebc0724f0 2024-04-25T01:26:14.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efec0b10640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1533a0 con 0x7efebc06d460 2024-04-25T01:26:14.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efec0b10640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efebc0c31b0 con 0x7efebc14fbe0 2024-04-25T01:26:14.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efeb9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efebc0724f0 0x7efebc14c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:14.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efeb9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efebc0724f0 0x7efebc14c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43876/0 (socket says 172.21.15.112:43876) 2024-04-25T01:26:14.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efeb9d9b640 1 -- 172.21.15.112:0/378422534 learned_addr learned my addr 172.21.15.112:0/378422534 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:14.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efebad9d640 1 --2- 172.21.15.112:0/378422534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efebc14fbe0 0x7efebc1546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:14.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efeba59c640 1 --2- 172.21.15.112:0/378422534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efebc06d460 0x7efebc1e6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:14.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efeb9d9b640 1 -- 172.21.15.112:0/378422534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efebc14fbe0 msgr2=0x7efebc1546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efeb9d9b640 1 --2- 172.21.15.112:0/378422534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efebc14fbe0 0x7efebc1546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efeb9d9b640 1 -- 172.21.15.112:0/378422534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efebc06d460 msgr2=0x7efebc1e6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efeb9d9b640 1 --2- 172.21.15.112:0/378422534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efebc06d460 0x7efebc1e6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efeb9d9b640 1 -- 172.21.15.112:0/378422534 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efebc1e7500 con 0x7efebc0724f0 2024-04-25T01:26:14.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efeb9d9b640 1 --2- 172.21.15.112:0/378422534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efebc0724f0 0x7efebc14c2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009720 tx=0x7efeb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:14.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efeaaffd640 1 -- 172.21.15.112:0/378422534 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0037040 con 0x7efebc0724f0 2024-04-25T01:26:14.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efeaaffd640 1 -- 172.21.15.112:0/378422534 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeb0004030 con 0x7efebc0724f0 2024-04-25T01:26:14.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efeaaffd640 1 -- 172.21.15.112:0/378422534 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0004420 con 0x7efebc0724f0 2024-04-25T01:26:14.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efec0b10640 1 -- 172.21.15.112:0/378422534 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efebc155d60 con 0x7efebc0724f0 2024-04-25T01:26:14.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.508+0000 7efec0b10640 1 -- 172.21.15.112:0/378422534 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efebc1e7cc0 con 0x7efebc0724f0 2024-04-25T01:26:14.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.509+0000 7efec0b10640 1 -- 172.21.15.112:0/378422534 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efebc04f500 con 0x7efebc0724f0 2024-04-25T01:26:14.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.510+0000 7efeaaffd640 1 -- 172.21.15.112:0/378422534 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efeb003a040 con 0x7efebc0724f0 2024-04-25T01:26:14.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.510+0000 7efeaaffd640 1 --2- 172.21.15.112:0/378422534 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efe9c06be30 0x7efe9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:14.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.510+0000 7efeaaffd640 1 -- 172.21.15.112:0/378422534 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7efeb00ab5f0 con 0x7efebc0724f0 2024-04-25T01:26:14.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.510+0000 7efeba59c640 1 --2- 172.21.15.112:0/378422534 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efe9c06be30 0x7efe9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:14.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.511+0000 7efeba59c640 1 --2- 172.21.15.112:0/378422534 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efe9c06be30 0x7efe9c06e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efeac007fd0 tx=0x7efeac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:14.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.515+0000 7efeaaffd640 1 -- 172.21.15.112:0/378422534 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efeb0042020 con 0x7efebc0724f0 2024-04-25T01:26:14.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.687+0000 7efec0b10640 1 -- 172.21.15.112:0/378422534 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) -- 0x7efebc0c3470 con 0x7efebc0724f0 2024-04-25T01:26:14.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.688+0000 7efeaaffd640 1 -- 172.21.15.112:0/378422534 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7efeb0079750 con 0x7efebc0724f0 2024-04-25T01:26:14.667 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T01:26:14.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.689+0000 7efec0b10640 1 -- 172.21.15.112:0/378422534 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efe9c06be30 msgr2=0x7efe9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.689+0000 7efec0b10640 1 --2- 172.21.15.112:0/378422534 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efe9c06be30 0x7efe9c06e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efeac007fd0 tx=0x7efeac009040 comp rx=0 tx=0).stop 2024-04-25T01:26:14.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.689+0000 7efec0b10640 1 -- 172.21.15.112:0/378422534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efebc0724f0 msgr2=0x7efebc14c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.689+0000 7efec0b10640 1 --2- 172.21.15.112:0/378422534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efebc0724f0 0x7efebc14c2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009720 tx=0x7efeb00046c0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.690+0000 7efec0b10640 1 -- 172.21.15.112:0/378422534 shutdown_connections 2024-04-25T01:26:14.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.690+0000 7efec0b10640 1 --2- 172.21.15.112:0/378422534 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efe9c06be30 0x7efe9c06e2f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.690+0000 7efec0b10640 1 --2- 172.21.15.112:0/378422534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efebc14fbe0 0x7efebc1546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.690+0000 7efec0b10640 1 --2- 172.21.15.112:0/378422534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efebc0724f0 0x7efebc14c2f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.690+0000 7efec0b10640 1 --2- 172.21.15.112:0/378422534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efebc06d460 0x7efebc1e6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.690+0000 7efec0b10640 1 -- 172.21.15.112:0/378422534 >> 172.21.15.112:0/378422534 conn(0x7efebc0c3a10 msgr2=0x7efebc14d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:14.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.690+0000 7efec0b10640 1 -- 172.21.15.112:0/378422534 shutdown_connections 2024-04-25T01:26:14.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.690+0000 7efec0b10640 1 -- 172.21.15.112:0/378422534 wait complete. 2024-04-25T01:26:14.680 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-04-25T01:26:14.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.799+0000 7fef14b34640 1 Processor -- start 2024-04-25T01:26:14.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.799+0000 7fef14b34640 1 -- start start 2024-04-25T01:26:14.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.799+0000 7fef14b34640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef100726d0 0x7fef10070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:14.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef14b34640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fef10157600 0x7fef100c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:14.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef14b34640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef10157ab0 con 0x7fef10150c80 2024-04-25T01:26:14.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef14b34640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fef10072b60 con 0x7fef10157600 2024-04-25T01:26:14.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef14b34640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fef10072e40 con 0x7fef100726d0 2024-04-25T01:26:14.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0e59c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fef10150c80 0x7fef100ac920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38150/0 (socket says 172.21.15.112:38150) 2024-04-25T01:26:14.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0e59c640 1 -- 172.21.15.112:0/928717053 learned_addr learned my addr 172.21.15.112:0/928717053 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:14.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0dd9b640 1 -- 172.21.15.112:0/928717053 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1547203182 0 0) 0x7fef10157ab0 con 0x7fef10150c80 2024-04-25T01:26:14.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef07fff640 1 --2- 172.21.15.112:0/928717053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef100726d0 0x7fef10070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:14.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0dd9b640 1 -- 172.21.15.112:0/928717053 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feefc003520 con 0x7fef10150c80 2024-04-25T01:26:14.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0ed9d640 1 --2- 172.21.15.112:0/928717053 >> v2:172.21.15.187:3300/0 conn(0x7fef10157600 0x7fef100c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:14.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0dd9b640 1 -- 172.21.15.112:0/928717053 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4196134116 0 0) 0x7feefc003520 con 0x7fef10150c80 2024-04-25T01:26:14.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0dd9b640 1 -- 172.21.15.112:0/928717053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef100726d0 msgr2=0x7fef10070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:14.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0dd9b640 1 --2- 172.21.15.112:0/928717053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef100726d0 0x7fef10070cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0dd9b640 1 -- 172.21.15.112:0/928717053 >> v2:172.21.15.187:3300/0 conn(0x7fef10157600 msgr2=0x7fef100c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:14.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0dd9b640 1 --2- 172.21.15.112:0/928717053 >> v2:172.21.15.187:3300/0 conn(0x7fef10157600 0x7fef100c2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0dd9b640 1 -- 172.21.15.112:0/928717053 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef10073120 con 0x7fef10150c80 2024-04-25T01:26:14.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0dd9b640 1 -- 172.21.15.112:0/928717053 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fef00001670 con 0x7fef10150c80 2024-04-25T01:26:14.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0dd9b640 1 -- 172.21.15.112:0/928717053 >> v1:172.21.15.112:6789/0 conn(0x7fef10150c80 legacy=0x7fef100ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0dd9b640 1 --2- 172.21.15.112:0/928717053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feefc0039b0 0x7feefc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:14.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0dd9b640 1 --2- 172.21.15.112:0/928717053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc004800 0x7feefc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:14.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0dd9b640 1 --2- 172.21.15.112:0/928717053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feefc0051c0 0x7feefc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:14.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0dd9b640 1 -- 172.21.15.112:0/928717053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef10072b60 con 0x7feefc004800 2024-04-25T01:26:14.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0dd9b640 1 -- 172.21.15.112:0/928717053 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fef10072e40 con 0x7feefc0039b0 2024-04-25T01:26:14.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0dd9b640 1 -- 172.21.15.112:0/928717053 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feefc00bf00 con 0x7feefc0051c0 2024-04-25T01:26:14.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef07fff640 1 --2- 172.21.15.112:0/928717053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc004800 0x7feefc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:14.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0e59c640 1 --2- 172.21.15.112:0/928717053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feefc0039b0 0x7feefc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:14.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.800+0000 7fef0ed9d640 1 --2- 172.21.15.112:0/928717053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feefc0051c0 0x7feefc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:14.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.801+0000 7fef07fff640 1 -- 172.21.15.112:0/928717053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feefc0051c0 msgr2=0x7feefc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:14.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.801+0000 7fef07fff640 1 --2- 172.21.15.112:0/928717053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feefc0051c0 0x7feefc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.801+0000 7fef07fff640 1 -- 172.21.15.112:0/928717053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feefc0039b0 msgr2=0x7feefc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:14.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.801+0000 7fef07fff640 1 --2- 172.21.15.112:0/928717053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feefc0039b0 0x7feefc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.801+0000 7fef07fff640 1 -- 172.21.15.112:0/928717053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc003520 con 0x7feefc004800 2024-04-25T01:26:14.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.801+0000 7fef0e59c640 1 --2- 172.21.15.112:0/928717053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feefc0039b0 0x7feefc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:14.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.801+0000 7fef07fff640 1 --2- 172.21.15.112:0/928717053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc004800 0x7feefc004c80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feef8009ac0 tx=0x7feefc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b6a6c248258aed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:14.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.801+0000 7fef0dd9b640 1 -- 172.21.15.112:0/928717053 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef8036070 con 0x7feefc004800 2024-04-25T01:26:14.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.801+0000 7fef0dd9b640 1 -- 172.21.15.112:0/928717053 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef8031400 con 0x7feefc004800 2024-04-25T01:26:14.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.801+0000 7fef0dd9b640 1 -- 172.21.15.112:0/928717053 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef8009690 con 0x7feefc004800 2024-04-25T01:26:14.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.802+0000 7fef14b34640 1 -- 172.21.15.112:0/928717053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc004800 msgr2=0x7feefc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.802+0000 7fef14b34640 1 --2- 172.21.15.112:0/928717053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc004800 0x7feefc004c80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feef8009ac0 tx=0x7feefc005830 comp rx=0 tx=0).stop 2024-04-25T01:26:14.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.802+0000 7fef0ed9d640 1 -- 172.21.15.112:0/928717053 reap_dead start 2024-04-25T01:26:14.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.802+0000 7fef14b34640 1 -- 172.21.15.112:0/928717053 shutdown_connections 2024-04-25T01:26:14.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.802+0000 7fef14b34640 1 -- 172.21.15.112:0/928717053 >> 172.21.15.112:0/928717053 conn(0x7fef100c3a10 msgr2=0x7fef100c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:14.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.802+0000 7fef14b34640 1 -- 172.21.15.112:0/928717053 shutdown_connections 2024-04-25T01:26:14.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.802+0000 7fef14b34640 1 -- 172.21.15.112:0/928717053 wait complete. 2024-04-25T01:26:14.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.803+0000 7fef14b34640 1 Processor -- start 2024-04-25T01:26:14.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.803+0000 7fef14b34640 1 -- start start 2024-04-25T01:26:14.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef14b34640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feefc004800 0x7fef101ec8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:14.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef14b34640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef101539b0 0x7fef101ecdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:14.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef14b34640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef101eb4c0 0x7fef101eb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:14.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef14b34640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef100ab8f0 con 0x7fef101539b0 2024-04-25T01:26:14.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef14b34640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fef100ac400 con 0x7feefc004800 2024-04-25T01:26:14.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef14b34640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fef10151060 con 0x7fef101eb4c0 2024-04-25T01:26:14.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef07fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef101539b0 0x7fef101ecdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:14.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef07fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef101539b0 0x7fef101ecdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43888/0 (socket says 172.21.15.112:43888) 2024-04-25T01:26:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef07fff640 1 -- 172.21.15.112:0/3278036760 learned_addr learned my addr 172.21.15.112:0/3278036760 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef0ed9d640 1 --2- 172.21.15.112:0/3278036760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef101eb4c0 0x7fef101eb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef0e59c640 1 --2- 172.21.15.112:0/3278036760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feefc004800 0x7fef101ec8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef07fff640 1 -- 172.21.15.112:0/3278036760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef101eb4c0 msgr2=0x7fef101eb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef07fff640 1 --2- 172.21.15.112:0/3278036760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef101eb4c0 0x7fef101eb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef07fff640 1 -- 172.21.15.112:0/3278036760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feefc004800 msgr2=0x7fef101ec8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef07fff640 1 --2- 172.21.15.112:0/3278036760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feefc004800 0x7fef101ec8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef07fff640 1 -- 172.21.15.112:0/3278036760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef101ea700 con 0x7fef101539b0 2024-04-25T01:26:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef0e59c640 1 --2- 172.21.15.112:0/3278036760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feefc004800 0x7fef101ec8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef07fff640 1 --2- 172.21.15.112:0/3278036760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef101539b0 0x7fef101ecdf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feef8009ac0 tx=0x7feef8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef06ffd640 1 -- 172.21.15.112:0/3278036760 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef8040020 con 0x7fef101539b0 2024-04-25T01:26:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef06ffd640 1 -- 172.21.15.112:0/3278036760 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef8003bc0 con 0x7fef101539b0 2024-04-25T01:26:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.804+0000 7fef14b34640 1 -- 172.21.15.112:0/3278036760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef101ea9e0 con 0x7fef101539b0 2024-04-25T01:26:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.805+0000 7fef06ffd640 1 -- 172.21.15.112:0/3278036760 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef8036040 con 0x7fef101539b0 2024-04-25T01:26:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.805+0000 7fef14b34640 1 -- 172.21.15.112:0/3278036760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef101eafa0 con 0x7fef101539b0 2024-04-25T01:26:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.806+0000 7fef14b34640 1 -- 172.21.15.112:0/3278036760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef1004f500 con 0x7fef101539b0 2024-04-25T01:26:14.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.806+0000 7fef06ffd640 1 -- 172.21.15.112:0/3278036760 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feef8043020 con 0x7fef101539b0 2024-04-25T01:26:14.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.806+0000 7fef06ffd640 1 --2- 172.21.15.112:0/3278036760 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feef006be30 0x7feef006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:14.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.806+0000 7fef0e59c640 1 --2- 172.21.15.112:0/3278036760 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feef006be30 0x7feef006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:14.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.806+0000 7fef06ffd640 1 -- 172.21.15.112:0/3278036760 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7feef80ab0f0 con 0x7fef101539b0 2024-04-25T01:26:14.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.807+0000 7fef0e59c640 1 --2- 172.21.15.112:0/3278036760 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feef006be30 0x7feef006e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fef00001eb0 tx=0x7fef00009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:14.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.810+0000 7fef06ffd640 1 -- 172.21.15.112:0/3278036760 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feef804a070 con 0x7fef101539b0 2024-04-25T01:26:14.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.982+0000 7fef14b34640 1 -- 172.21.15.112:0/3278036760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) -- 0x7fef100a9500 con 0x7fef101539b0 2024-04-25T01:26:14.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.982+0000 7fef06ffd640 1 -- 172.21.15.112:0/3278036760 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7feef80044e0 con 0x7fef101539b0 2024-04-25T01:26:14.962 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T01:26:14.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.984+0000 7fef14b34640 1 -- 172.21.15.112:0/3278036760 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feef006be30 msgr2=0x7feef006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.984+0000 7fef14b34640 1 --2- 172.21.15.112:0/3278036760 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feef006be30 0x7feef006e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fef00001eb0 tx=0x7fef00009040 comp rx=0 tx=0).stop 2024-04-25T01:26:14.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.984+0000 7fef14b34640 1 -- 172.21.15.112:0/3278036760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef101539b0 msgr2=0x7fef101ecdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:14.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.984+0000 7fef14b34640 1 --2- 172.21.15.112:0/3278036760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef101539b0 0x7fef101ecdf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feef8009ac0 tx=0x7feef8004730 comp rx=0 tx=0).stop 2024-04-25T01:26:14.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.984+0000 7fef14b34640 1 -- 172.21.15.112:0/3278036760 shutdown_connections 2024-04-25T01:26:14.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.984+0000 7fef14b34640 1 --2- 172.21.15.112:0/3278036760 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feef006be30 0x7feef006e2f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.984+0000 7fef14b34640 1 --2- 172.21.15.112:0/3278036760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef101eb4c0 0x7fef101eb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.984+0000 7fef14b34640 1 --2- 172.21.15.112:0/3278036760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef101539b0 0x7fef101ecdf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.984+0000 7fef14b34640 1 --2- 172.21.15.112:0/3278036760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feefc004800 0x7fef101ec8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:14.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.984+0000 7fef14b34640 1 -- 172.21.15.112:0/3278036760 >> 172.21.15.112:0/3278036760 conn(0x7fef100c3a10 msgr2=0x7fef1014ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:14.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.984+0000 7fef14b34640 1 -- 172.21.15.112:0/3278036760 shutdown_connections 2024-04-25T01:26:14.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:14.984+0000 7fef14b34640 1 -- 172.21.15.112:0/3278036760 wait complete. 2024-04-25T01:26:14.975 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-04-25T01:26:15.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.094+0000 7f3b333a2640 1 Processor -- start 2024-04-25T01:26:15.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.094+0000 7f3b333a2640 1 -- start start 2024-04-25T01:26:15.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.095+0000 7f3b333a2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b2c0c9250 0x7f3b2c0c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.095+0000 7f3b333a2640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b2c1534a0 con 0x7f3b2c0ca8a0 2024-04-25T01:26:15.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.095+0000 7f3b333a2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b2c0cae30 con 0x7f3b2c0ca370 2024-04-25T01:26:15.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.095+0000 7f3b333a2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2c156990 con 0x7f3b2c0c9250 2024-04-25T01:26:15.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.095+0000 7f3b3193f640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3b2c0ca8a0 0x7f3b2c1602d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38158/0 (socket says 172.21.15.112:38158) 2024-04-25T01:26:15.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.095+0000 7f3b3193f640 1 -- 172.21.15.112:0/3145855990 learned_addr learned my addr 172.21.15.112:0/3145855990 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:15.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.095+0000 7f3b3113e640 1 --2- 172.21.15.112:0/3145855990 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b2c0c9250 0x7f3b2c0c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.096+0000 7f3b23fff640 1 -- 172.21.15.112:0/3145855990 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 421389320 0 0) 0x7f3b2c1534a0 con 0x7f3b2c0ca8a0 2024-04-25T01:26:15.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.096+0000 7f3b23fff640 1 -- 172.21.15.112:0/3145855990 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b18003520 con 0x7f3b2c0ca8a0 2024-04-25T01:26:15.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.096+0000 7f3b3113e640 1 -- 172.21.15.112:0/3145855990 >> v1:172.21.15.187:6789/0 conn(0x7f3b2c0ca370 legacy=0x7f3b2c06bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:15.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.096+0000 7f3b3113e640 1 -- 172.21.15.112:0/3145855990 >> v1:172.21.15.112:6789/0 conn(0x7f3b2c0ca8a0 legacy=0x7f3b2c1602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:15.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.096+0000 7f3b23fff640 1 -- 172.21.15.112:0/3145855990 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1649217703 0 0) 0x7f3b18003520 con 0x7f3b2c0ca8a0 2024-04-25T01:26:15.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.096+0000 7f3b3113e640 1 -- 172.21.15.112:0/3145855990 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2c0c9cd0 con 0x7f3b2c0c9250 2024-04-25T01:26:15.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.097+0000 7f3b3113e640 1 --2- 172.21.15.112:0/3145855990 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b2c0c9250 0x7f3b2c0c9630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009dc0 tx=0x7f3b2802a540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55d0439be0cdbdb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:15.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.097+0000 7f3b23fff640 1 -- 172.21.15.112:0/3145855990 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24001670 con 0x7f3b2c0c9250 2024-04-25T01:26:15.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.097+0000 7f3b23fff640 1 -- 172.21.15.112:0/3145855990 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b28007540 con 0x7f3b2c0c9250 2024-04-25T01:26:15.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.097+0000 7f3b23fff640 1 -- 172.21.15.112:0/3145855990 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b280078f0 con 0x7f3b2c0c9250 2024-04-25T01:26:15.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.097+0000 7f3b333a2640 1 -- 172.21.15.112:0/3145855990 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b2c0c9250 msgr2=0x7f3b2c0c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:15.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.097+0000 7f3b333a2640 1 --2- 172.21.15.112:0/3145855990 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b2c0c9250 0x7f3b2c0c9630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009dc0 tx=0x7f3b2802a540 comp rx=0 tx=0).stop 2024-04-25T01:26:15.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.097+0000 7f3b333a2640 1 -- 172.21.15.112:0/3145855990 shutdown_connections 2024-04-25T01:26:15.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.097+0000 7f3b333a2640 1 --2- 172.21.15.112:0/3145855990 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b2c0c9250 0x7f3b2c0c9630 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.097+0000 7f3b333a2640 1 -- 172.21.15.112:0/3145855990 >> 172.21.15.112:0/3145855990 conn(0x7f3b2c0cb310 msgr2=0x7f3b2c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:15.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.097+0000 7f3b333a2640 1 -- 172.21.15.112:0/3145855990 shutdown_connections 2024-04-25T01:26:15.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.097+0000 7f3b333a2640 1 -- 172.21.15.112:0/3145855990 wait complete. 2024-04-25T01:26:15.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.098+0000 7f3b333a2640 1 Processor -- start 2024-04-25T01:26:15.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.098+0000 7f3b333a2640 1 -- start start 2024-04-25T01:26:15.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.099+0000 7f3b333a2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b2c0c9250 0x7f3b2c0c4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.099+0000 7f3b333a2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b2c0ca370 0x7f3b2c1ef800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.099+0000 7f3b333a2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b2c0ca8a0 0x7f3b2c158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.099+0000 7f3b333a2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c155eb0 con 0x7f3b2c0ca8a0 2024-04-25T01:26:15.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.099+0000 7f3b333a2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c156500 con 0x7f3b2c0c9250 2024-04-25T01:26:15.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.099+0000 7f3b333a2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2c156190 con 0x7f3b2c0ca370 2024-04-25T01:26:15.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.099+0000 7f3b3193f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b2c0ca8a0 0x7f3b2c158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.099+0000 7f3b3193f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b2c0ca8a0 0x7f3b2c158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43896/0 (socket says 172.21.15.112:43896) 2024-04-25T01:26:15.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.099+0000 7f3b3193f640 1 -- 172.21.15.112:0/1270165029 learned_addr learned my addr 172.21.15.112:0/1270165029 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:15.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.099+0000 7f3b3093d640 1 --2- 172.21.15.112:0/1270165029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b2c0ca370 0x7f3b2c1ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.099+0000 7f3b3113e640 1 --2- 172.21.15.112:0/1270165029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b2c0c9250 0x7f3b2c0c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.099+0000 7f3b3193f640 1 -- 172.21.15.112:0/1270165029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b2c0ca370 msgr2=0x7f3b2c1ef800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:15.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.099+0000 7f3b3193f640 1 --2- 172.21.15.112:0/1270165029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b2c0ca370 0x7f3b2c1ef800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.099+0000 7f3b3193f640 1 -- 172.21.15.112:0/1270165029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b2c0c9250 msgr2=0x7f3b2c0c4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:15.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.099+0000 7f3b3193f640 1 --2- 172.21.15.112:0/1270165029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b2c0c9250 0x7f3b2c0c4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.099+0000 7f3b3193f640 1 -- 172.21.15.112:0/1270165029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2c0c4840 con 0x7f3b2c0ca8a0 2024-04-25T01:26:15.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.099+0000 7f3b3113e640 1 --2- 172.21.15.112:0/1270165029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b2c0c9250 0x7f3b2c0c4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:15.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.100+0000 7f3b3193f640 1 --2- 172.21.15.112:0/1270165029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b2c0ca8a0 0x7f3b2c158060 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3b24007fd0 tx=0x7f3b2400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:15.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.100+0000 7f3b21ffb640 1 -- 172.21.15.112:0/1270165029 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24017070 con 0x7f3b2c0ca8a0 2024-04-25T01:26:15.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.100+0000 7f3b21ffb640 1 -- 172.21.15.112:0/1270165029 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b24005ce0 con 0x7f3b2c0ca8a0 2024-04-25T01:26:15.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.100+0000 7f3b21ffb640 1 -- 172.21.15.112:0/1270165029 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24007290 con 0x7f3b2c0ca8a0 2024-04-25T01:26:15.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.100+0000 7f3b333a2640 1 -- 172.21.15.112:0/1270165029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b2c0bec40 con 0x7f3b2c0ca8a0 2024-04-25T01:26:15.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.100+0000 7f3b333a2640 1 -- 172.21.15.112:0/1270165029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b2c1ef000 con 0x7f3b2c0ca8a0 2024-04-25T01:26:15.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.101+0000 7f3b333a2640 1 -- 172.21.15.112:0/1270165029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b2c04f500 con 0x7f3b2c0ca8a0 2024-04-25T01:26:15.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.101+0000 7f3b21ffb640 1 -- 172.21.15.112:0/1270165029 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3b2401a070 con 0x7f3b2c0ca8a0 2024-04-25T01:26:15.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.102+0000 7f3b21ffb640 1 --2- 172.21.15.112:0/1270165029 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3b1006be60 0x7f3b1006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.102+0000 7f3b3113e640 1 --2- 172.21.15.112:0/1270165029 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3b1006be60 0x7f3b1006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.102+0000 7f3b21ffb640 1 -- 172.21.15.112:0/1270165029 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f3b2408a4e0 con 0x7f3b2c0ca8a0 2024-04-25T01:26:15.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.102+0000 7f3b3113e640 1 --2- 172.21.15.112:0/1270165029 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3b1006be60 0x7f3b1006e320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3b28034000 tx=0x7f3b2802f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:15.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.106+0000 7f3b21ffb640 1 -- 172.21.15.112:0/1270165029 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3b24058bf0 con 0x7f3b2c0ca8a0 2024-04-25T01:26:15.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.277+0000 7f3b333a2640 1 -- 172.21.15.112:0/1270165029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) -- 0x7f3b2c0c1e80 con 0x7f3b2c0ca8a0 2024-04-25T01:26:15.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.277+0000 7f3b21ffb640 1 -- 172.21.15.112:0/1270165029 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7f3b240584f0 con 0x7f3b2c0ca8a0 2024-04-25T01:26:15.257 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T01:26:15.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.279+0000 7f3b333a2640 1 -- 172.21.15.112:0/1270165029 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3b1006be60 msgr2=0x7f3b1006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:15.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.279+0000 7f3b333a2640 1 --2- 172.21.15.112:0/1270165029 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3b1006be60 0x7f3b1006e320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3b28034000 tx=0x7f3b2802f000 comp rx=0 tx=0).stop 2024-04-25T01:26:15.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.279+0000 7f3b333a2640 1 -- 172.21.15.112:0/1270165029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b2c0ca8a0 msgr2=0x7f3b2c158060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:15.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.279+0000 7f3b333a2640 1 --2- 172.21.15.112:0/1270165029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b2c0ca8a0 0x7f3b2c158060 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3b24007fd0 tx=0x7f3b2400c9b0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.279+0000 7f3b333a2640 1 -- 172.21.15.112:0/1270165029 shutdown_connections 2024-04-25T01:26:15.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.279+0000 7f3b333a2640 1 --2- 172.21.15.112:0/1270165029 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3b1006be60 0x7f3b1006e320 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.279+0000 7f3b333a2640 1 --2- 172.21.15.112:0/1270165029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b2c0ca8a0 0x7f3b2c158060 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.279+0000 7f3b333a2640 1 --2- 172.21.15.112:0/1270165029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b2c0ca370 0x7f3b2c1ef800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.279+0000 7f3b333a2640 1 --2- 172.21.15.112:0/1270165029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b2c0c9250 0x7f3b2c0c4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.279+0000 7f3b333a2640 1 -- 172.21.15.112:0/1270165029 >> 172.21.15.112:0/1270165029 conn(0x7f3b2c0cb310 msgr2=0x7f3b2c154f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:15.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.279+0000 7f3b333a2640 1 -- 172.21.15.112:0/1270165029 shutdown_connections 2024-04-25T01:26:15.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.279+0000 7f3b333a2640 1 -- 172.21.15.112:0/1270165029 wait complete. 2024-04-25T01:26:15.270 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:26:15.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.386+0000 7f1c38e1b640 1 Processor -- start 2024-04-25T01:26:15.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.387+0000 7f1c38e1b640 1 -- start start 2024-04-25T01:26:15.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.387+0000 7f1c38e1b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1c341592c0 0x7f1c340c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.387+0000 7f1c38e1b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c341596a0 0x7f1c34152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.387+0000 7f1c38e1b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1c340c72b0 0x7f1c34155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.387+0000 7f1c38e1b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f1c340c6e20 con 0x7f1c340c72b0 2024-04-25T01:26:15.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.387+0000 7f1c38e1b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1c34159b90 con 0x7f1c341592c0 2024-04-25T01:26:15.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.387+0000 7f1c38e1b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1c3406c970 con 0x7f1c341596a0 2024-04-25T01:26:15.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.387+0000 7f1c32d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1c340c72b0 0x7f1c34155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.387+0000 7f1c32d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1c340c72b0 0x7f1c34155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43898/0 (socket says 172.21.15.112:43898) 2024-04-25T01:26:15.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.387+0000 7f1c32d9d640 1 -- 172.21.15.112:0/1805488388 learned_addr learned my addr 172.21.15.112:0/1805488388 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:15.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.388+0000 7f1c3259c640 1 --2- 172.21.15.112:0/1805488388 >> v2:172.21.15.187:3300/0 conn(0x7f1c341592c0 0x7f1c340c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.388+0000 7f1c2bfff640 1 --2- 172.21.15.112:0/1805488388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c341596a0 0x7f1c34152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.388+0000 7f1c32d9d640 1 -- 172.21.15.112:0/1805488388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c341596a0 msgr2=0x7f1c34152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:15.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.388+0000 7f1c32d9d640 1 --2- 172.21.15.112:0/1805488388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c341596a0 0x7f1c34152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.388+0000 7f1c32d9d640 1 -- 172.21.15.112:0/1805488388 >> v2:172.21.15.187:3300/0 conn(0x7f1c341592c0 msgr2=0x7f1c340c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:15.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.388+0000 7f1c32d9d640 1 --2- 172.21.15.112:0/1805488388 >> v2:172.21.15.187:3300/0 conn(0x7f1c341592c0 0x7f1c340c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.388+0000 7f1c32d9d640 1 -- 172.21.15.112:0/1805488388 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3406cc50 con 0x7f1c340c72b0 2024-04-25T01:26:15.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.388+0000 7f1c2bfff640 1 --2- 172.21.15.112:0/1805488388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c341596a0 0x7f1c34152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:15.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.388+0000 7f1c32d9d640 1 --2- 172.21.15.112:0/1805488388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c340c72b0 0x7f1c34155ae0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1c3408cb00 tx=0x7f1c2c02a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a3294e674f8554 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:15.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.388+0000 7f1c31d9b640 1 -- 172.21.15.112:0/1805488388 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c036070 con 0x7f1c340c72b0 2024-04-25T01:26:15.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.388+0000 7f1c31d9b640 1 -- 172.21.15.112:0/1805488388 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c2c031600 con 0x7f1c340c72b0 2024-04-25T01:26:15.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.388+0000 7f1c31d9b640 1 -- 172.21.15.112:0/1805488388 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c0319d0 con 0x7f1c340c72b0 2024-04-25T01:26:15.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.389+0000 7f1c38e1b640 1 -- 172.21.15.112:0/1805488388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c340c72b0 msgr2=0x7f1c34155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:15.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.389+0000 7f1c38e1b640 1 --2- 172.21.15.112:0/1805488388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c340c72b0 0x7f1c34155ae0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1c3408cb00 tx=0x7f1c2c02a570 comp rx=0 tx=0).stop 2024-04-25T01:26:15.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.389+0000 7f1c38e1b640 1 -- 172.21.15.112:0/1805488388 shutdown_connections 2024-04-25T01:26:15.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.389+0000 7f1c38e1b640 1 --2- 172.21.15.112:0/1805488388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c340c72b0 0x7f1c34155ae0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.389+0000 7f1c38e1b640 1 --2- 172.21.15.112:0/1805488388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c341596a0 0x7f1c34152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.389+0000 7f1c38e1b640 1 --2- 172.21.15.112:0/1805488388 >> v2:172.21.15.187:3300/0 conn(0x7f1c341592c0 0x7f1c340c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.389+0000 7f1c38e1b640 1 -- 172.21.15.112:0/1805488388 >> 172.21.15.112:0/1805488388 conn(0x7f1c340cb170 msgr2=0x7f1c340cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:15.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.389+0000 7f1c38e1b640 1 -- 172.21.15.112:0/1805488388 shutdown_connections 2024-04-25T01:26:15.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.389+0000 7f1c38e1b640 1 -- 172.21.15.112:0/1805488388 wait complete. 2024-04-25T01:26:15.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.390+0000 7f1c38e1b640 1 Processor -- start 2024-04-25T01:26:15.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.390+0000 7f1c38e1b640 1 -- start start 2024-04-25T01:26:15.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.391+0000 7f1c38e1b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c340c72b0 0x7f1c3414e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.391+0000 7f1c38e1b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c341592c0 0x7f1c3414c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.391+0000 7f1c38e1b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c341596a0 0x7f1c3414c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.391+0000 7f1c38e1b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3406dca0 con 0x7f1c341592c0 2024-04-25T01:26:15.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.391+0000 7f1c38e1b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3406db20 con 0x7f1c341596a0 2024-04-25T01:26:15.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.391+0000 7f1c38e1b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1c340c95b0 con 0x7f1c340c72b0 2024-04-25T01:26:15.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.391+0000 7f1c2bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c341592c0 0x7f1c3414c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.391+0000 7f1c2bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c341592c0 0x7f1c3414c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43906/0 (socket says 172.21.15.112:43906) 2024-04-25T01:26:15.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.391+0000 7f1c2bfff640 1 -- 172.21.15.112:0/3340747231 learned_addr learned my addr 172.21.15.112:0/3340747231 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:15.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.391+0000 7f1c3259c640 1 --2- 172.21.15.112:0/3340747231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c340c72b0 0x7f1c3414e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.391+0000 7f1c32d9d640 1 --2- 172.21.15.112:0/3340747231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c341596a0 0x7f1c3414c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.391+0000 7f1c2bfff640 1 -- 172.21.15.112:0/3340747231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c340c72b0 msgr2=0x7f1c3414e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:15.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.391+0000 7f1c2bfff640 1 --2- 172.21.15.112:0/3340747231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c340c72b0 0x7f1c3414e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.391+0000 7f1c2bfff640 1 -- 172.21.15.112:0/3340747231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c341596a0 msgr2=0x7f1c3414c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:15.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.391+0000 7f1c2bfff640 1 --2- 172.21.15.112:0/3340747231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c341596a0 0x7f1c3414c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.391+0000 7f1c2bfff640 1 -- 172.21.15.112:0/3340747231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c341500a0 con 0x7f1c341592c0 2024-04-25T01:26:15.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.391+0000 7f1c3259c640 1 --2- 172.21.15.112:0/3340747231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c340c72b0 0x7f1c3414e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:15.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.391+0000 7f1c2bfff640 1 --2- 172.21.15.112:0/3340747231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c341592c0 0x7f1c3414c300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00c7c0 tx=0x7f1c1c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:15.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.392+0000 7f1c2affd640 1 -- 172.21.15.112:0/3340747231 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c015070 con 0x7f1c341592c0 2024-04-25T01:26:15.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.392+0000 7f1c2affd640 1 -- 172.21.15.112:0/3340747231 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c1c007820 con 0x7f1c341592c0 2024-04-25T01:26:15.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.392+0000 7f1c2affd640 1 -- 172.21.15.112:0/3340747231 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c007bf0 con 0x7f1c341592c0 2024-04-25T01:26:15.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.392+0000 7f1c38e1b640 1 -- 172.21.15.112:0/3340747231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c3414d4d0 con 0x7f1c341592c0 2024-04-25T01:26:15.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.392+0000 7f1c38e1b640 1 -- 172.21.15.112:0/3340747231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c3414cd80 con 0x7f1c341592c0 2024-04-25T01:26:15.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.393+0000 7f1c38e1b640 1 -- 172.21.15.112:0/3340747231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c340679a0 con 0x7f1c341592c0 2024-04-25T01:26:15.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.393+0000 7f1c2affd640 1 -- 172.21.15.112:0/3340747231 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c1c018070 con 0x7f1c341592c0 2024-04-25T01:26:15.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.393+0000 7f1c2affd640 1 --2- 172.21.15.112:0/3340747231 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1c1406be30 0x7f1c1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.394+0000 7f1c3259c640 1 --2- 172.21.15.112:0/3340747231 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1c1406be30 0x7f1c1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.394+0000 7f1c2affd640 1 -- 172.21.15.112:0/3340747231 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f1c1c08aee0 con 0x7f1c341592c0 2024-04-25T01:26:15.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.394+0000 7f1c3259c640 1 --2- 172.21.15.112:0/3340747231 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1c1406be30 0x7f1c1406e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1c24006fd0 tx=0x7f1c24008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:15.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.398+0000 7f1c2affd640 1 -- 172.21.15.112:0/3340747231 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1c1c0595f0 con 0x7f1c341592c0 2024-04-25T01:26:15.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.609+0000 7f1c38e1b640 1 -- 172.21.15.112:0/3340747231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c340be800 con 0x7f1c341592c0 2024-04-25T01:26:15.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.610+0000 7f1c2affd640 1 -- 172.21.15.112:0/3340747231 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+6491 (secure 0 0 0) 0x7f1c340be800 con 0x7f1c341592c0 2024-04-25T01:26:15.590 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:15.590 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/4047219598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4047219598},{"type":"v1","addr":"172.21.15.187:6834","nonce":4047219598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2204640907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2204640907},{"type":"v1","addr":"172.21.15.187:6835","nonce":2204640907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:25:52.687260+0000","modified":"2024-04-25T01:25:55.702719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/2115984531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2115984531},{"type":"v1","addr":"172.21.15.112:6837","nonce":2115984531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1},{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:26:01.740521+0000","modified":"2024-04-25T01:26:04.746955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":8,"state":"up:active","state_seq":6,"addr":"172.21.15.112:6835/278686477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":278686477},{"type":"v1","addr":"172.21.15.112:6835","nonce":278686477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":2}]} 2024-04-25T01:26:15.590 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 11 2024-04-25T01:26:15.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.612+0000 7f1c38e1b640 1 -- 172.21.15.112:0/3340747231 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1c1406be30 msgr2=0x7f1c1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:15.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.612+0000 7f1c38e1b640 1 --2- 172.21.15.112:0/3340747231 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1c1406be30 0x7f1c1406e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1c24006fd0 tx=0x7f1c24008040 comp rx=0 tx=0).stop 2024-04-25T01:26:15.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.612+0000 7f1c38e1b640 1 -- 172.21.15.112:0/3340747231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c341592c0 msgr2=0x7f1c3414c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:15.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.612+0000 7f1c38e1b640 1 --2- 172.21.15.112:0/3340747231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c341592c0 0x7f1c3414c300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00c7c0 tx=0x7f1c1c00cc80 comp rx=0 tx=0).stop 2024-04-25T01:26:15.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.612+0000 7f1c38e1b640 1 -- 172.21.15.112:0/3340747231 shutdown_connections 2024-04-25T01:26:15.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.612+0000 7f1c38e1b640 1 --2- 172.21.15.112:0/3340747231 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1c1406be30 0x7f1c1406e2f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.612+0000 7f1c38e1b640 1 --2- 172.21.15.112:0/3340747231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c341596a0 0x7f1c3414c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.612+0000 7f1c38e1b640 1 --2- 172.21.15.112:0/3340747231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c341592c0 0x7f1c3414c300 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.612+0000 7f1c38e1b640 1 --2- 172.21.15.112:0/3340747231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c340c72b0 0x7f1c3414e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.612+0000 7f1c38e1b640 1 -- 172.21.15.112:0/3340747231 >> 172.21.15.112:0/3340747231 conn(0x7f1c340cb170 msgr2=0x7f1c340ca360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:15.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.612+0000 7f1c38e1b640 1 -- 172.21.15.112:0/3340747231 shutdown_connections 2024-04-25T01:26:15.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.612+0000 7f1c38e1b640 1 -- 172.21.15.112:0/3340747231 wait complete. 2024-04-25T01:26:15.602 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:26:15.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.719+0000 7efda2431640 1 Processor -- start 2024-04-25T01:26:15.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.719+0000 7efda2431640 1 -- start start 2024-04-25T01:26:15.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.720+0000 7efda2431640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efd9c154b50 0x7efd9c0c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.720+0000 7efda2431640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efd9c0c8810 0x7efd9c153660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.720+0000 7efda2431640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd9c154fd0 con 0x7efd9c071520 2024-04-25T01:26:15.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.720+0000 7efda2431640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7efd9c0c7f10 con 0x7efd9c0c8810 2024-04-25T01:26:15.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.720+0000 7efda2431640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7efd9c154120 con 0x7efd9c154b50 2024-04-25T01:26:15.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.720+0000 7efda09ce640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efd9c0c8810 0x7efd9c153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.720+0000 7efd9bfff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7efd9c071520 0x7efd9c0c7800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38166/0 (socket says 172.21.15.112:38166) 2024-04-25T01:26:15.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.720+0000 7efd9bfff640 1 -- 172.21.15.112:0/3174376640 learned_addr learned my addr 172.21.15.112:0/3174376640 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:15.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.721+0000 7efda09ce640 1 -- 172.21.15.112:0/3174376640 >> v2:172.21.15.187:3300/0 conn(0x7efd9c154b50 msgr2=0x7efd9c0c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:15.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.721+0000 7efd9b7fe640 1 --2- 172.21.15.112:0/3174376640 >> v2:172.21.15.187:3300/0 conn(0x7efd9c154b50 0x7efd9c0c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.721+0000 7efda09ce640 1 --2- 172.21.15.112:0/3174376640 >> v2:172.21.15.187:3300/0 conn(0x7efd9c154b50 0x7efd9c0c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.721+0000 7efd9affd640 1 -- 172.21.15.112:0/3174376640 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 730827588 0 0) 0x7efd8c001110 con 0x7efd9c071520 2024-04-25T01:26:15.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.721+0000 7efda09ce640 1 -- 172.21.15.112:0/3174376640 >> v1:172.21.15.112:6789/0 conn(0x7efd9c071520 legacy=0x7efd9c0c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:15.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.721+0000 7efda09ce640 1 -- 172.21.15.112:0/3174376640 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9c154400 con 0x7efd9c0c8810 2024-04-25T01:26:15.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.721+0000 7efd9b7fe640 1 --2- 172.21.15.112:0/3174376640 >> v2:172.21.15.187:3300/0 conn(0x7efd9c154b50 0x7efd9c0c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:15.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.721+0000 7efda09ce640 1 --2- 172.21.15.112:0/3174376640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd9c0c8810 0x7efd9c153660 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efd9c06eda0 tx=0x7efd9402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c2a132a2c237ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:15.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.721+0000 7efd9affd640 1 -- 172.21.15.112:0/3174376640 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94007740 con 0x7efd9c0c8810 2024-04-25T01:26:15.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.721+0000 7efd9affd640 1 -- 172.21.15.112:0/3174376640 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd940078e0 con 0x7efd9c0c8810 2024-04-25T01:26:15.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.721+0000 7efd9affd640 1 -- 172.21.15.112:0/3174376640 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94007cb0 con 0x7efd9c0c8810 2024-04-25T01:26:15.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.722+0000 7efda2431640 1 -- 172.21.15.112:0/3174376640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd9c0c8810 msgr2=0x7efd9c153660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:15.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.722+0000 7efda2431640 1 --2- 172.21.15.112:0/3174376640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd9c0c8810 0x7efd9c153660 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efd9c06eda0 tx=0x7efd9402a560 comp rx=0 tx=0).stop 2024-04-25T01:26:15.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.722+0000 7efda2431640 1 -- 172.21.15.112:0/3174376640 shutdown_connections 2024-04-25T01:26:15.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.722+0000 7efda2431640 1 --2- 172.21.15.112:0/3174376640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd9c0c8810 0x7efd9c153660 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.722+0000 7efda2431640 1 --2- 172.21.15.112:0/3174376640 >> v2:172.21.15.187:3300/0 conn(0x7efd9c154b50 0x7efd9c0c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.722+0000 7efda2431640 1 -- 172.21.15.112:0/3174376640 >> 172.21.15.112:0/3174376640 conn(0x7efd9c074a00 msgr2=0x7efd9c0c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:15.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.722+0000 7efda2431640 1 -- 172.21.15.112:0/3174376640 shutdown_connections 2024-04-25T01:26:15.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.722+0000 7efda2431640 1 -- 172.21.15.112:0/3174376640 wait complete. 2024-04-25T01:26:15.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.723+0000 7efda2431640 1 Processor -- start 2024-04-25T01:26:15.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.723+0000 7efda2431640 1 -- start start 2024-04-25T01:26:15.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.724+0000 7efda2431640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd9c071520 0x7efd9c153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.724+0000 7efda2431640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd9c0c8810 0x7efd9c0c2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.724+0000 7efda2431640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd9c154b50 0x7efd9c06d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.724+0000 7efda2431640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c0ad930 con 0x7efd9c0c8810 2024-04-25T01:26:15.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.724+0000 7efda2431640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c0c72e0 con 0x7efd9c071520 2024-04-25T01:26:15.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.724+0000 7efda2431640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efd9c0adab0 con 0x7efd9c154b50 2024-04-25T01:26:15.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.724+0000 7efd9b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd9c0c8810 0x7efd9c0c2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.724+0000 7efd9b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd9c0c8810 0x7efd9c0c2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43928/0 (socket says 172.21.15.112:43928) 2024-04-25T01:26:15.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.724+0000 7efd9b7fe640 1 -- 172.21.15.112:0/2162549248 learned_addr learned my addr 172.21.15.112:0/2162549248 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:15.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.724+0000 7efda09ce640 1 --2- 172.21.15.112:0/2162549248 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd9c154b50 0x7efd9c06d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.724+0000 7efd9bfff640 1 --2- 172.21.15.112:0/2162549248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd9c071520 0x7efd9c153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.724+0000 7efd9b7fe640 1 -- 172.21.15.112:0/2162549248 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd9c154b50 msgr2=0x7efd9c06d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:15.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.724+0000 7efd9b7fe640 1 --2- 172.21.15.112:0/2162549248 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd9c154b50 0x7efd9c06d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.724+0000 7efd9b7fe640 1 -- 172.21.15.112:0/2162549248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd9c071520 msgr2=0x7efd9c153a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.724+0000 7efd9b7fe640 1 --2- 172.21.15.112:0/2162549248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd9c071520 0x7efd9c153a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.724+0000 7efd9b7fe640 1 -- 172.21.15.112:0/2162549248 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9c0a8f20 con 0x7efd9c0c8810 2024-04-25T01:26:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.724+0000 7efd9bfff640 1 --2- 172.21.15.112:0/2162549248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd9c071520 0x7efd9c153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.724+0000 7efd9b7fe640 1 --2- 172.21.15.112:0/2162549248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd9c0c8810 0x7efd9c0c2620 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efd9000c7f0 tx=0x7efd9000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.724+0000 7efd98ff9640 1 -- 172.21.15.112:0/2162549248 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90015070 con 0x7efd9c0c8810 2024-04-25T01:26:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.725+0000 7efd98ff9640 1 -- 172.21.15.112:0/2162549248 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd90007a30 con 0x7efd9c0c8810 2024-04-25T01:26:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.725+0000 7efda2431640 1 -- 172.21.15.112:0/2162549248 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd9c0a91c0 con 0x7efd9c0c8810 2024-04-25T01:26:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.725+0000 7efd98ff9640 1 -- 172.21.15.112:0/2162549248 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90007e00 con 0x7efd9c0c8810 2024-04-25T01:26:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.725+0000 7efda2431640 1 -- 172.21.15.112:0/2162549248 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd9c0c2b60 con 0x7efd9c0c8810 2024-04-25T01:26:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.725+0000 7efda2431640 1 -- 172.21.15.112:0/2162549248 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd9c06cf70 con 0x7efd9c0c8810 2024-04-25T01:26:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.726+0000 7efd98ff9640 1 -- 172.21.15.112:0/2162549248 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd90018070 con 0x7efd9c0c8810 2024-04-25T01:26:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.726+0000 7efd98ff9640 1 --2- 172.21.15.112:0/2162549248 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efd8806be30 0x7efd8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.726+0000 7efd9bfff640 1 --2- 172.21.15.112:0/2162549248 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efd8806be30 0x7efd8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.726+0000 7efd98ff9640 1 -- 172.21.15.112:0/2162549248 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7efd9008b110 con 0x7efd9c0c8810 2024-04-25T01:26:15.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.727+0000 7efd9bfff640 1 --2- 172.21.15.112:0/2162549248 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efd8806be30 0x7efd8806e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efd8c000f60 tx=0x7efd8c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:15.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.730+0000 7efd98ff9640 1 -- 172.21.15.112:0/2162549248 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efd90059790 con 0x7efd9c0c8810 2024-04-25T01:26:15.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.905+0000 7efda2431640 1 -- 172.21.15.112:0/2162549248 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd9c0682a0 con 0x7efd9c0c8810 2024-04-25T01:26:15.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.906+0000 7efd98ff9640 1 -- 172.21.15.112:0/2162549248 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20367 (secure 0 0 0) 0x7efd90059090 con 0x7efd9c0c8810 2024-04-25T01:26:15.886 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:15.886 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:26:03.405271+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-25T01:25:59.478275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-25T01:26:00.719358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:26:15.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.908+0000 7efda2431640 1 -- 172.21.15.112:0/2162549248 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efd8806be30 msgr2=0x7efd8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:15.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.908+0000 7efda2431640 1 --2- 172.21.15.112:0/2162549248 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efd8806be30 0x7efd8806e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efd8c000f60 tx=0x7efd8c009040 comp rx=0 tx=0).stop 2024-04-25T01:26:15.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.908+0000 7efda2431640 1 -- 172.21.15.112:0/2162549248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd9c0c8810 msgr2=0x7efd9c0c2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:15.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.908+0000 7efda2431640 1 --2- 172.21.15.112:0/2162549248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd9c0c8810 0x7efd9c0c2620 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efd9000c7f0 tx=0x7efd9000cbf0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.908+0000 7efda2431640 1 -- 172.21.15.112:0/2162549248 shutdown_connections 2024-04-25T01:26:15.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.908+0000 7efda2431640 1 --2- 172.21.15.112:0/2162549248 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efd8806be30 0x7efd8806e2f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.908+0000 7efda2431640 1 --2- 172.21.15.112:0/2162549248 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd9c154b50 0x7efd9c06d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.908+0000 7efda2431640 1 --2- 172.21.15.112:0/2162549248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd9c0c8810 0x7efd9c0c2620 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.908+0000 7efda2431640 1 --2- 172.21.15.112:0/2162549248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd9c071520 0x7efd9c153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.908+0000 7efda2431640 1 -- 172.21.15.112:0/2162549248 >> 172.21.15.112:0/2162549248 conn(0x7efd9c074a00 msgr2=0x7efd9c1553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:15.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.908+0000 7efda2431640 1 -- 172.21.15.112:0/2162549248 shutdown_connections 2024-04-25T01:26:15.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:15.908+0000 7efda2431640 1 -- 172.21.15.112:0/2162549248 wait complete. 2024-04-25T01:26:15.899 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:26:15.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.016+0000 7ff11cacd640 1 Processor -- start 2024-04-25T01:26:15.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.016+0000 7ff11cacd640 1 -- start start 2024-04-25T01:26:15.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.017+0000 7ff11cacd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff1181596a0 0x7ff118152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.017+0000 7ff11cacd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff1180c72b0 0x7ff118160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.017+0000 7ff11cacd640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff118159b90 con 0x7ff1181592c0 2024-04-25T01:26:15.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.017+0000 7ff11cacd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7ff1180c6e20 con 0x7ff1180c72b0 2024-04-25T01:26:15.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.017+0000 7ff11cacd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff11806c970 con 0x7ff1181596a0 2024-04-25T01:26:15.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.017+0000 7ff116d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff1180c72b0 0x7ff118160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.017+0000 7ff116d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff1180c72b0 0x7ff118160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43932/0 (socket says 172.21.15.112:43932) 2024-04-25T01:26:15.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.017+0000 7ff116d9d640 1 -- 172.21.15.112:0/3048762528 learned_addr learned my addr 172.21.15.112:0/3048762528 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:15.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.017+0000 7ff115d9b640 1 --2- 172.21.15.112:0/3048762528 >> v2:172.21.15.187:3300/0 conn(0x7ff1181596a0 0x7ff118152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.017+0000 7ff116d9d640 1 -- 172.21.15.112:0/3048762528 >> v2:172.21.15.187:3300/0 conn(0x7ff1181596a0 msgr2=0x7ff118152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:15.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.017+0000 7ff116d9d640 1 --2- 172.21.15.112:0/3048762528 >> v2:172.21.15.187:3300/0 conn(0x7ff1181596a0 0x7ff118152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.017+0000 7ff116d9d640 1 -- 172.21.15.112:0/3048762528 >> v1:172.21.15.187:6789/0 conn(0x7ff1181592c0 legacy=0x7ff1180c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:15.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.017+0000 7ff116d9d640 1 -- 172.21.15.112:0/3048762528 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11806cc50 con 0x7ff1180c72b0 2024-04-25T01:26:15.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.017+0000 7ff115d9b640 1 --2- 172.21.15.112:0/3048762528 >> v2:172.21.15.187:3300/0 conn(0x7ff1181596a0 0x7ff118152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:15.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.018+0000 7ff116d9d640 1 --2- 172.21.15.112:0/3048762528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1180c72b0 0x7ff118160380 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff1180bea40 tx=0x7ff11002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e4e65b5e0f3b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:15.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.018+0000 7ff11559a640 1 -- 172.21.15.112:0/3048762528 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff110036070 con 0x7ff1180c72b0 2024-04-25T01:26:15.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.018+0000 7ff11559a640 1 -- 172.21.15.112:0/3048762528 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff110031600 con 0x7ff1180c72b0 2024-04-25T01:26:16.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.018+0000 7ff11559a640 1 -- 172.21.15.112:0/3048762528 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1100319d0 con 0x7ff1180c72b0 2024-04-25T01:26:16.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.019+0000 7ff11cacd640 1 -- 172.21.15.112:0/3048762528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1180c72b0 msgr2=0x7ff118160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:16.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.019+0000 7ff11cacd640 1 --2- 172.21.15.112:0/3048762528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1180c72b0 0x7ff118160380 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff1180bea40 tx=0x7ff11002a5d0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.019+0000 7ff11cacd640 1 -- 172.21.15.112:0/3048762528 shutdown_connections 2024-04-25T01:26:16.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.019+0000 7ff11cacd640 1 --2- 172.21.15.112:0/3048762528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1180c72b0 0x7ff118160380 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.019+0000 7ff11cacd640 1 --2- 172.21.15.112:0/3048762528 >> v2:172.21.15.187:3300/0 conn(0x7ff1181596a0 0x7ff118152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.019+0000 7ff11cacd640 1 -- 172.21.15.112:0/3048762528 >> 172.21.15.112:0/3048762528 conn(0x7ff1180cb170 msgr2=0x7ff1180cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:16.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.019+0000 7ff11cacd640 1 -- 172.21.15.112:0/3048762528 shutdown_connections 2024-04-25T01:26:16.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.019+0000 7ff11cacd640 1 -- 172.21.15.112:0/3048762528 wait complete. 2024-04-25T01:26:16.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.020+0000 7ff11cacd640 1 Processor -- start 2024-04-25T01:26:16.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.020+0000 7ff11cacd640 1 -- start start 2024-04-25T01:26:16.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff11cacd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1180c72b0 0x7ff11814c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff11cacd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1181592c0 0x7ff1180c16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff11cacd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1181596a0 0x7ff1181545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff11cacd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff11806c510 con 0x7ff1181596a0 2024-04-25T01:26:16.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff11cacd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff11806c390 con 0x7ff1180c72b0 2024-04-25T01:26:16.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff11cacd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff11806db20 con 0x7ff1181592c0 2024-04-25T01:26:16.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff116d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1181596a0 0x7ff1181545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff116d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1181596a0 0x7ff1181545a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43946/0 (socket says 172.21.15.112:43946) 2024-04-25T01:26:16.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff116d9d640 1 -- 172.21.15.112:0/4292321697 learned_addr learned my addr 172.21.15.112:0/4292321697 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:16.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff115d9b640 1 --2- 172.21.15.112:0/4292321697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1181592c0 0x7ff1180c16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff11659c640 1 --2- 172.21.15.112:0/4292321697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1180c72b0 0x7ff11814c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff116d9d640 1 -- 172.21.15.112:0/4292321697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1181592c0 msgr2=0x7ff1180c16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:16.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff116d9d640 1 --2- 172.21.15.112:0/4292321697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1181592c0 0x7ff1180c16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff116d9d640 1 -- 172.21.15.112:0/4292321697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1180c72b0 msgr2=0x7ff11814c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:16.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff116d9d640 1 --2- 172.21.15.112:0/4292321697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1180c72b0 0x7ff11814c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff116d9d640 1 -- 172.21.15.112:0/4292321697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11814cde0 con 0x7ff1181596a0 2024-04-25T01:26:16.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff11659c640 1 --2- 172.21.15.112:0/4292321697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1180c72b0 0x7ff11814c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:16.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff116d9d640 1 --2- 172.21.15.112:0/4292321697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1181596a0 0x7ff1181545a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff110009f40 tx=0x7ff110004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:16.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff106ffd640 1 -- 172.21.15.112:0/4292321697 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff110036050 con 0x7ff1181596a0 2024-04-25T01:26:16.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff106ffd640 1 -- 172.21.15.112:0/4292321697 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff110042070 con 0x7ff1181596a0 2024-04-25T01:26:16.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff11cacd640 1 -- 172.21.15.112:0/4292321697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1181eca50 con 0x7ff1181596a0 2024-04-25T01:26:16.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff106ffd640 1 -- 172.21.15.112:0/4292321697 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff110007bd0 con 0x7ff1181596a0 2024-04-25T01:26:16.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.021+0000 7ff11cacd640 1 -- 172.21.15.112:0/4292321697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff118153da0 con 0x7ff1181596a0 2024-04-25T01:26:16.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.022+0000 7ff11cacd640 1 -- 172.21.15.112:0/4292321697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1180679a0 con 0x7ff1181596a0 2024-04-25T01:26:16.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.023+0000 7ff106ffd640 1 -- 172.21.15.112:0/4292321697 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff110047460 con 0x7ff1181596a0 2024-04-25T01:26:16.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.023+0000 7ff106ffd640 1 --2- 172.21.15.112:0/4292321697 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff0f806be30 0x7ff0f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.023+0000 7ff106ffd640 1 -- 172.21.15.112:0/4292321697 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7ff1100ac100 con 0x7ff1181596a0 2024-04-25T01:26:16.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.023+0000 7ff11659c640 1 --2- 172.21.15.112:0/4292321697 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff0f806be30 0x7ff0f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.024+0000 7ff11659c640 1 --2- 172.21.15.112:0/4292321697 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff0f806be30 0x7ff0f806e2f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff118158550 tx=0x7ff108009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:16.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.028+0000 7ff106ffd640 1 -- 172.21.15.112:0/4292321697 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff11814cde0 con 0x7ff1181596a0 2024-04-25T01:26:16.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.237+0000 7ff11cacd640 1 -- 172.21.15.112:0/4292321697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1180682a0 con 0x7ff1181596a0 2024-04-25T01:26:16.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.238+0000 7ff106ffd640 1 -- 172.21.15.112:0/4292321697 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+6491 (secure 0 0 0) 0x7ff11007a810 con 0x7ff1181596a0 2024-04-25T01:26:16.217 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:16.218 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/4047219598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4047219598},{"type":"v1","addr":"172.21.15.187:6834","nonce":4047219598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2204640907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2204640907},{"type":"v1","addr":"172.21.15.187:6835","nonce":2204640907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:25:52.687260+0000","modified":"2024-04-25T01:25:55.702719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/2115984531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2115984531},{"type":"v1","addr":"172.21.15.112:6837","nonce":2115984531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1},{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:26:01.740521+0000","modified":"2024-04-25T01:26:04.746955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":8,"state":"up:active","state_seq":6,"addr":"172.21.15.112:6835/278686477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":278686477},{"type":"v1","addr":"172.21.15.112:6835","nonce":278686477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":2}]} 2024-04-25T01:26:16.218 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 11 2024-04-25T01:26:16.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.240+0000 7ff11cacd640 1 -- 172.21.15.112:0/4292321697 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff0f806be30 msgr2=0x7ff0f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:16.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.240+0000 7ff11cacd640 1 --2- 172.21.15.112:0/4292321697 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff0f806be30 0x7ff0f806e2f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff118158550 tx=0x7ff108009040 comp rx=0 tx=0).stop 2024-04-25T01:26:16.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.240+0000 7ff11cacd640 1 -- 172.21.15.112:0/4292321697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1181596a0 msgr2=0x7ff1181545a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:16.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.240+0000 7ff11cacd640 1 --2- 172.21.15.112:0/4292321697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1181596a0 0x7ff1181545a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff110009f40 tx=0x7ff110004a40 comp rx=0 tx=0).stop 2024-04-25T01:26:16.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.240+0000 7ff11cacd640 1 -- 172.21.15.112:0/4292321697 shutdown_connections 2024-04-25T01:26:16.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.240+0000 7ff11cacd640 1 --2- 172.21.15.112:0/4292321697 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff0f806be30 0x7ff0f806e2f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.240+0000 7ff11cacd640 1 --2- 172.21.15.112:0/4292321697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1181596a0 0x7ff1181545a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.240+0000 7ff11cacd640 1 --2- 172.21.15.112:0/4292321697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1181592c0 0x7ff1180c16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.240+0000 7ff11cacd640 1 --2- 172.21.15.112:0/4292321697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1180c72b0 0x7ff11814c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.240+0000 7ff11cacd640 1 -- 172.21.15.112:0/4292321697 >> 172.21.15.112:0/4292321697 conn(0x7ff1180cb170 msgr2=0x7ff1180cb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:16.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.240+0000 7ff11cacd640 1 -- 172.21.15.112:0/4292321697 shutdown_connections 2024-04-25T01:26:16.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.240+0000 7ff11cacd640 1 -- 172.21.15.112:0/4292321697 wait complete. 2024-04-25T01:26:16.632 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T01:25:52.687260+0000', 'modified': '2024-04-25T01:25:55.702719+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4281}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4281': {'gid': 4281, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6837/2115984531', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6836', 'nonce': 2115984531}, {'type': 'v1', 'addr': '172.21.15.112:6837', 'nonce': 2115984531}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'a', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4281, 'qdb_cluster': [4281]} 2024-04-25T01:26:16.633 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T01:26:16.633 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T01:26:16.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.752+0000 7fba3a55a640 1 Processor -- start 2024-04-25T01:26:16.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.752+0000 7fba3a55a640 1 -- start start 2024-04-25T01:26:16.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba3a55a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fba340724c0 0x7fba340c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba3a55a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba3406e0f0 con 0x7fba34072d80 2024-04-25T01:26:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba3a55a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba340c2180 con 0x7fba341536c0 2024-04-25T01:26:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba3a55a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fba340c2460 con 0x7fba340724c0 2024-04-25T01:26:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba33fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fba34072d80 0x7fba3414d340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38174/0 (socket says 172.21.15.112:38174) 2024-04-25T01:26:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba33fff640 1 -- 172.21.15.112:0/1300818611 learned_addr learned my addr 172.21.15.112:0/1300818611 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba38af7640 1 --2- 172.21.15.112:0/1300818611 >> v2:172.21.15.187:3300/0 conn(0x7fba340724c0 0x7fba340c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba32ffd640 1 -- 172.21.15.112:0/1300818611 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2727438213 0 0) 0x7fba24001110 con 0x7fba34072d80 2024-04-25T01:26:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba32ffd640 1 -- 172.21.15.112:0/1300818611 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba1c003520 con 0x7fba34072d80 2024-04-25T01:26:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba32ffd640 1 -- 172.21.15.112:0/1300818611 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3622961848 0 0) 0x7fba1c003520 con 0x7fba34072d80 2024-04-25T01:26:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba32ffd640 1 -- 172.21.15.112:0/1300818611 >> v2:172.21.15.187:3300/0 conn(0x7fba340724c0 msgr2=0x7fba340c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba32ffd640 1 --2- 172.21.15.112:0/1300818611 >> v2:172.21.15.187:3300/0 conn(0x7fba340724c0 0x7fba340c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba32ffd640 1 -- 172.21.15.112:0/1300818611 >> v1:172.21.15.187:6789/0 conn(0x7fba341536c0 legacy=0x7fba34071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba32ffd640 1 -- 172.21.15.112:0/1300818611 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba340c2740 con 0x7fba34072d80 2024-04-25T01:26:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba32ffd640 1 -- 172.21.15.112:0/1300818611 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fba24001840 con 0x7fba34072d80 2024-04-25T01:26:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba32ffd640 1 -- 172.21.15.112:0/1300818611 >> v1:172.21.15.112:6789/0 conn(0x7fba34072d80 legacy=0x7fba3414d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba32ffd640 1 --2- 172.21.15.112:0/1300818611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1c0039b0 0x7fba1c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba32ffd640 1 --2- 172.21.15.112:0/1300818611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba1c004800 0x7fba1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba32ffd640 1 --2- 172.21.15.112:0/1300818611 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba1c0051c0 0x7fba1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba32ffd640 1 -- 172.21.15.112:0/1300818611 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fba340c2460 con 0x7fba1c004800 2024-04-25T01:26:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba32ffd640 1 -- 172.21.15.112:0/1300818611 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c00c060 con 0x7fba1c0039b0 2024-04-25T01:26:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba32ffd640 1 -- 172.21.15.112:0/1300818611 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fba1c00c260 con 0x7fba1c0051c0 2024-04-25T01:26:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.753+0000 7fba337fe640 1 --2- 172.21.15.112:0/1300818611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba1c004800 0x7fba1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.754+0000 7fba33fff640 1 --2- 172.21.15.112:0/1300818611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1c0039b0 0x7fba1c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.754+0000 7fba38af7640 1 --2- 172.21.15.112:0/1300818611 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba1c0051c0 0x7fba1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.754+0000 7fba337fe640 1 -- 172.21.15.112:0/1300818611 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba1c0051c0 msgr2=0x7fba1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.754+0000 7fba337fe640 1 --2- 172.21.15.112:0/1300818611 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba1c0051c0 0x7fba1c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.754+0000 7fba337fe640 1 -- 172.21.15.112:0/1300818611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1c0039b0 msgr2=0x7fba1c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.754+0000 7fba337fe640 1 --2- 172.21.15.112:0/1300818611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1c0039b0 0x7fba1c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.754+0000 7fba337fe640 1 -- 172.21.15.112:0/1300818611 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1c003520 con 0x7fba1c004800 2024-04-25T01:26:16.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.754+0000 7fba38af7640 1 -- 172.21.15.112:0/1300818611 reap_dead start 2024-04-25T01:26:16.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.754+0000 7fba337fe640 1 --2- 172.21.15.112:0/1300818611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba1c004800 0x7fba1c004c80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fba28001750 tx=0x7fba1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ae323a9e1220c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:16.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.755+0000 7fba32ffd640 1 -- 172.21.15.112:0/1300818611 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba28001370 con 0x7fba1c004800 2024-04-25T01:26:16.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.755+0000 7fba32ffd640 1 -- 172.21.15.112:0/1300818611 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba280052d0 con 0x7fba1c004800 2024-04-25T01:26:16.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.755+0000 7fba32ffd640 1 -- 172.21.15.112:0/1300818611 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2802abe0 con 0x7fba1c004800 2024-04-25T01:26:16.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.755+0000 7fba3a55a640 1 -- 172.21.15.112:0/1300818611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba1c004800 msgr2=0x7fba1c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:16.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.755+0000 7fba3a55a640 1 --2- 172.21.15.112:0/1300818611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba1c004800 0x7fba1c004c80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fba28001750 tx=0x7fba1c005830 comp rx=0 tx=0).stop 2024-04-25T01:26:16.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.755+0000 7fba3a55a640 1 -- 172.21.15.112:0/1300818611 shutdown_connections 2024-04-25T01:26:16.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.755+0000 7fba3a55a640 1 --2- 172.21.15.112:0/1300818611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba1c004800 0x7fba1c004c80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.755+0000 7fba3a55a640 1 -- 172.21.15.112:0/1300818611 >> 172.21.15.112:0/1300818611 conn(0x7fba3406b160 msgr2=0x7fba3406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:16.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.755+0000 7fba3a55a640 1 -- 172.21.15.112:0/1300818611 shutdown_connections 2024-04-25T01:26:16.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.755+0000 7fba3a55a640 1 -- 172.21.15.112:0/1300818611 wait complete. 2024-04-25T01:26:16.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.756+0000 7fba3a55a640 1 Processor -- start 2024-04-25T01:26:16.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.756+0000 7fba3a55a640 1 -- start start 2024-04-25T01:26:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.757+0000 7fba3a55a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba1c004800 0x7fba34145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.757+0000 7fba3a55a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba340c6b40 0x7fba341ef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.757+0000 7fba3a55a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba3414c740 0x7fba341498a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.757+0000 7fba3a55a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fba340728d0 con 0x7fba340c6b40 2024-04-25T01:26:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.757+0000 7fba3a55a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fba34071890 con 0x7fba3414c740 2024-04-25T01:26:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.757+0000 7fba3a55a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fba34073160 con 0x7fba1c004800 2024-04-25T01:26:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.757+0000 7fba337fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba340c6b40 0x7fba341ef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.757+0000 7fba337fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba340c6b40 0x7fba341ef0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43952/0 (socket says 172.21.15.112:43952) 2024-04-25T01:26:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.757+0000 7fba337fe640 1 -- 172.21.15.112:0/318511909 learned_addr learned my addr 172.21.15.112:0/318511909 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.757+0000 7fba33fff640 1 --2- 172.21.15.112:0/318511909 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba1c004800 0x7fba34145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.757+0000 7fba38af7640 1 --2- 172.21.15.112:0/318511909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba3414c740 0x7fba341498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.757+0000 7fba337fe640 1 -- 172.21.15.112:0/318511909 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba1c004800 msgr2=0x7fba34145f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.757+0000 7fba337fe640 1 --2- 172.21.15.112:0/318511909 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba1c004800 0x7fba34145f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.757+0000 7fba337fe640 1 -- 172.21.15.112:0/318511909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba3414c740 msgr2=0x7fba341498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.757+0000 7fba337fe640 1 --2- 172.21.15.112:0/318511909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba3414c740 0x7fba341498a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.757+0000 7fba337fe640 1 -- 172.21.15.112:0/318511909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba340aa340 con 0x7fba340c6b40 2024-04-25T01:26:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.757+0000 7fba38af7640 1 --2- 172.21.15.112:0/318511909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba3414c740 0x7fba341498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.757+0000 7fba337fe640 1 --2- 172.21.15.112:0/318511909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba340c6b40 0x7fba341ef0c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fba28000c00 tx=0x7fba28001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.758+0000 7fba30ff9640 1 -- 172.21.15.112:0/318511909 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2803f030 con 0x7fba340c6b40 2024-04-25T01:26:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.758+0000 7fba30ff9640 1 -- 172.21.15.112:0/318511909 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba28005480 con 0x7fba340c6b40 2024-04-25T01:26:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.758+0000 7fba3a55a640 1 -- 172.21.15.112:0/318511909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba340aa5e0 con 0x7fba340c6b40 2024-04-25T01:26:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.758+0000 7fba30ff9640 1 -- 172.21.15.112:0/318511909 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba28037040 con 0x7fba340c6b40 2024-04-25T01:26:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.758+0000 7fba3a55a640 1 -- 172.21.15.112:0/318511909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba3414cbd0 con 0x7fba340c6b40 2024-04-25T01:26:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.760+0000 7fba30ff9640 1 -- 172.21.15.112:0/318511909 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba28042070 con 0x7fba340c6b40 2024-04-25T01:26:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.760+0000 7fba30ff9640 1 --2- 172.21.15.112:0/318511909 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fba2006be30 0x7fba2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.760+0000 7fba33fff640 1 --2- 172.21.15.112:0/318511909 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fba2006be30 0x7fba2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.760+0000 7fba30ff9640 1 -- 172.21.15.112:0/318511909 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fba280aa6f0 con 0x7fba340c6b40 2024-04-25T01:26:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.760+0000 7fba33fff640 1 --2- 172.21.15.112:0/318511909 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fba2006be30 0x7fba2006e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fba24007fd0 tx=0x7fba24007a70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.761+0000 7fba3d264740 1 Processor -- start 2024-04-25T01:26:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.761+0000 7fba3d264740 1 -- start start 2024-04-25T01:26:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.761+0000 7fba3d264740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264faad0 0x5616264f7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.761+0000 7fba3d264740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264bd8e0 0x5616264f7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.761+0000 7fba3d264740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264de510 0x5616264f8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.761+0000 7fba3d264740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56162641ff00 con 0x5616264faad0 2024-04-25T01:26:16.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.761+0000 7fba3d264740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5616263c8750 con 0x5616264bd8e0 2024-04-25T01:26:16.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.761+0000 7fba3d264740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x56162641bec0 con 0x5616264de510 2024-04-25T01:26:16.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.761+0000 7fba33fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264faad0 0x5616264f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.761+0000 7fba33fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264faad0 0x5616264f7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43968/0 (socket says 172.21.15.112:43968) 2024-04-25T01:26:16.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.761+0000 7fba33fff640 1 -- 172.21.15.112:0/1543284443 learned_addr learned my addr 172.21.15.112:0/1543284443 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:16.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.761+0000 7fba38af7640 1 --2- 172.21.15.112:0/1543284443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264de510 0x5616264f8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.761+0000 7fba337fe640 1 --2- 172.21.15.112:0/1543284443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264bd8e0 0x5616264f7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.762+0000 7fba33fff640 1 -- 172.21.15.112:0/1543284443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264de510 msgr2=0x5616264f8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:16.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.762+0000 7fba33fff640 1 --2- 172.21.15.112:0/1543284443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264de510 0x5616264f8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.762+0000 7fba33fff640 1 -- 172.21.15.112:0/1543284443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264bd8e0 msgr2=0x5616264f7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:16.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.762+0000 7fba33fff640 1 --2- 172.21.15.112:0/1543284443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264bd8e0 0x5616264f7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.762+0000 7fba33fff640 1 -- 172.21.15.112:0/1543284443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616261342d0 con 0x5616264faad0 2024-04-25T01:26:16.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.762+0000 7fba38af7640 1 --2- 172.21.15.112:0/1543284443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264de510 0x5616264f8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:16.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.762+0000 7fba337fe640 1 --2- 172.21.15.112:0/1543284443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264bd8e0 0x5616264f7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:16.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.762+0000 7fba33fff640 1 --2- 172.21.15.112:0/1543284443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264faad0 0x5616264f7940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fba24007820 tx=0x7fba2400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f74fbac149a77b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:16.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.762+0000 7fba19ffb640 1 -- 172.21.15.112:0/1543284443 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24028070 con 0x5616264faad0 2024-04-25T01:26:16.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.762+0000 7fba19ffb640 1 -- 172.21.15.112:0/1543284443 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba240159c0 con 0x5616264faad0 2024-04-25T01:26:16.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.762+0000 7fba19ffb640 1 -- 172.21.15.112:0/1543284443 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24015d70 con 0x5616264faad0 2024-04-25T01:26:16.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.762+0000 7fba3d264740 1 -- 172.21.15.112:0/1543284443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264faad0 msgr2=0x5616264f7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:16.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.762+0000 7fba3d264740 1 --2- 172.21.15.112:0/1543284443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264faad0 0x5616264f7940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fba24007820 tx=0x7fba2400c000 comp rx=0 tx=0).stop 2024-04-25T01:26:16.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.762+0000 7fba3d264740 1 -- 172.21.15.112:0/1543284443 shutdown_connections 2024-04-25T01:26:16.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.762+0000 7fba3d264740 1 --2- 172.21.15.112:0/1543284443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264de510 0x5616264f8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.762+0000 7fba3d264740 1 --2- 172.21.15.112:0/1543284443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264bd8e0 0x5616264f7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.762+0000 7fba3d264740 1 --2- 172.21.15.112:0/1543284443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264faad0 0x5616264f7940 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.762+0000 7fba3d264740 1 -- 172.21.15.112:0/1543284443 >> 172.21.15.112:0/1543284443 conn(0x5616264b9640 msgr2=0x5616264dde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:16.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.763+0000 7fba3d264740 1 -- 172.21.15.112:0/1543284443 shutdown_connections 2024-04-25T01:26:16.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.763+0000 7fba3d264740 1 -- 172.21.15.112:0/1543284443 wait complete. 2024-04-25T01:26:16.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.763+0000 7fba3d264740 1 -- start start 2024-04-25T01:26:16.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.763+0000 7fba3d264740 1 Processor -- start 2024-04-25T01:26:16.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.764+0000 7fba3d264740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264de510 0x5616264bbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.764+0000 7fba3d264740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264faad0 0x56162650c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.764+0000 7fba3d264740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264f6360 0x5616264d8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.764+0000 7fba3d264740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5616264bddc0 con 0x5616264f6360 2024-04-25T01:26:16.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.764+0000 7fba3d264740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5616263dbcf0 con 0x5616264faad0 2024-04-25T01:26:16.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.764+0000 7fba3d264740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5616264f74b0 con 0x5616264de510 2024-04-25T01:26:16.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.764+0000 7fba3d264740 10 client.0 _ll_register_callbacks cb 0x5616263db6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:26:16.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.764+0000 7fba38af7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264f6360 0x5616264d8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.764+0000 7fba38af7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264f6360 0x5616264d8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43970/0 (socket says 172.21.15.112:43970) 2024-04-25T01:26:16.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.764+0000 7fba38af7640 1 -- 172.21.15.112:0/2605161551 learned_addr learned my addr 172.21.15.112:0/2605161551 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:16.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.764+0000 7fba337fe640 1 --2- 172.21.15.112:0/2605161551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264faad0 0x56162650c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.764+0000 7fba33fff640 1 --2- 172.21.15.112:0/2605161551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264de510 0x5616264bbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.764+0000 7fba38af7640 1 -- 172.21.15.112:0/2605161551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264de510 msgr2=0x5616264bbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:16.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.764+0000 7fba38af7640 1 --2- 172.21.15.112:0/2605161551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264de510 0x5616264bbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.764+0000 7fba38af7640 1 -- 172.21.15.112:0/2605161551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264faad0 msgr2=0x56162650c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:16.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.764+0000 7fba38af7640 1 --2- 172.21.15.112:0/2605161551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264faad0 0x56162650c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.764+0000 7fba38af7640 1 -- 172.21.15.112:0/2605161551 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616261fe6b0 con 0x5616264f6360 2024-04-25T01:26:16.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.764+0000 7fba33fff640 1 --2- 172.21.15.112:0/2605161551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264de510 0x5616264bbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:16.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.764+0000 7fba38af7640 1 --2- 172.21.15.112:0/2605161551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264f6360 0x5616264d8c30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fba340c4c40 tx=0x7fba2c004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08205656e5af006 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:16.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.764+0000 7fba12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T01:26:16.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.765+0000 7fba12ffd640 1 -- 172.21.15.112:0/2605161551 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c01b070 con 0x5616264f6360 2024-04-25T01:26:16.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.765+0000 7fba12ffd640 1 -- 172.21.15.112:0/2605161551 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fba2c015c90 con 0x5616264f6360 2024-04-25T01:26:16.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.765+0000 7fba3d264740 1 -- 172.21.15.112:0/2605161551 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba240159c0 con 0x5616264f6360 2024-04-25T01:26:16.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.765+0000 7fba12ffd640 1 -- 172.21.15.112:0/2605161551 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=11) ==== 24+0+0 (secure 0 0 0) 0x7fba2c015e90 con 0x5616264f6360 2024-04-25T01:26:16.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.765+0000 7fba3d264740 10 client.4815 fetch_fsmap learned FSMap version 11 2024-04-25T01:26:16.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.765+0000 7fba3d264740 1 -- 172.21.15.112:0/2605161551 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=11,monmap=2+}) -- 0x5616264bb5b0 con 0x5616264f6360 2024-04-25T01:26:16.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.765+0000 7fba12ffd640 1 -- 172.21.15.112:0/2605161551 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba240159c0 con 0x5616264f6360 2024-04-25T01:26:16.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.765+0000 7fba12ffd640 1 -- 172.21.15.112:0/2605161551 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 11) ==== 4128+0+0 (secure 0 0 0) 0x7fba2c020460 con 0x5616264f6360 2024-04-25T01:26:16.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.765+0000 7fba3d264740 10 client.4815 fetch_fsmap finished waiting for FSMap version 11 2024-04-25T01:26:16.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.765+0000 7fba3d264740 10 client.4815 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T01:26:16.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.765+0000 7fba3d264740 20 client.4815 populate_metadata read hostname 'smithi112' 2024-04-25T01:26:16.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.765+0000 7fba3d264740 1 --2- 172.21.15.112:0/2605161551 >> [v2:172.21.15.112:6836/2115984531,v1:172.21.15.112:6837/2115984531] conn(0x5616265095d0 0x561626517820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.765+0000 7fba3d264740 4 client.4815 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:26:16.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.765+0000 7fba3d264740 1 -- 172.21.15.112:0/2605161551 --> [v2:172.21.15.112:6836/2115984531,v1:172.21.15.112:6837/2115984531] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56162642d440 con 0x5616265095d0 2024-04-25T01:26:16.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.765+0000 7fba33fff640 1 --2- 172.21.15.112:0/2605161551 >> [v2:172.21.15.112:6836/2115984531,v1:172.21.15.112:6837/2115984531] conn(0x5616265095d0 0x561626517820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.765+0000 7fba12ffd640 10 client.4815 ms_handle_connect on v2:172.21.15.112:6836/2115984531 2024-04-25T01:26:16.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.765+0000 7fba33fff640 1 --2- 172.21.15.112:0/2605161551 >> [v2:172.21.15.112:6836/2115984531,v1:172.21.15.112:6837/2115984531] conn(0x5616265095d0 0x561626517820 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffc867b4fd0371d3 server_cookie=bb4b50712df2229e in_seq=0 out_seq=0 2024-04-25T01:26:16.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.766+0000 7fba12ffd640 1 -- 172.21.15.112:0/2605161551 <== mds.0 v2:172.21.15.112:6836/2115984531 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56162642d440 con 0x5616265095d0 2024-04-25T01:26:16.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.766+0000 7fba12ffd640 10 client.4815 handle_command_reply: tid=0 2024-04-25T01:26:16.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.766+0000 7fba3d264740 1 client.4815 shutdown 2024-04-25T01:26:16.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.766+0000 7fba3d264740 1 -- 172.21.15.112:0/2605161551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264f6360 msgr2=0x5616264d8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:16.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.766+0000 7fba3d264740 1 --2- 172.21.15.112:0/2605161551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264f6360 0x5616264d8c30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fba340c4c40 tx=0x7fba2c004700 comp rx=0 tx=0).stop 2024-04-25T01:26:16.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.766+0000 7fba3d264740 1 -- 172.21.15.112:0/2605161551 shutdown_connections 2024-04-25T01:26:16.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.766+0000 7fba3d264740 1 --2- 172.21.15.112:0/2605161551 >> [v2:172.21.15.112:6836/2115984531,v1:172.21.15.112:6837/2115984531] conn(0x5616265095d0 0x561626517820 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.766+0000 7fba3d264740 1 --2- 172.21.15.112:0/2605161551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264f6360 0x5616264d8c30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.766+0000 7fba3d264740 1 --2- 172.21.15.112:0/2605161551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264faad0 0x56162650c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.766+0000 7fba12ffd640 0 client.4815 ms_handle_reset on v2:172.21.15.112:6836/2115984531 2024-04-25T01:26:16.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.766+0000 7fba3d264740 1 --2- 172.21.15.112:0/2605161551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264de510 0x5616264bbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.766+0000 7fba3d264740 1 -- 172.21.15.112:0/2605161551 >> 172.21.15.112:0/2605161551 conn(0x5616264bd8e0 msgr2=0x5616264b6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:16.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.766+0000 7fba3d264740 1 -- 172.21.15.112:0/2605161551 shutdown_connections 2024-04-25T01:26:16.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.766+0000 7fba3d264740 1 -- 172.21.15.112:0/2605161551 wait complete. 2024-04-25T01:26:16.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.767+0000 7fba3d264740 20 client.4815 trim_cache size 0 max 16384 2024-04-25T01:26:16.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.784+0000 7fba3d264740 1 Processor -- start 2024-04-25T01:26:16.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.784+0000 7fba3d264740 1 -- start start 2024-04-25T01:26:16.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.785+0000 7fba3d264740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264de510 0x5616264f9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.785+0000 7fba3d264740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264f6360 0x56162651ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.785+0000 7fba3d264740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264faad0 0x56162651cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.785+0000 7fba3d264740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fba341fb940 con 0x5616264de510 2024-04-25T01:26:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.785+0000 7fba3d264740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fba341f9300 con 0x5616264faad0 2024-04-25T01:26:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.785+0000 7fba3d264740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fba341f9950 con 0x5616264f6360 2024-04-25T01:26:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.785+0000 7fba33fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264de510 0x5616264f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.785+0000 7fba33fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264de510 0x5616264f9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43972/0 (socket says 172.21.15.112:43972) 2024-04-25T01:26:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.785+0000 7fba33fff640 1 -- 172.21.15.112:0/3965653953 learned_addr learned my addr 172.21.15.112:0/3965653953 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.785+0000 7fba337fe640 1 --2- 172.21.15.112:0/3965653953 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264f6360 0x56162651ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.785+0000 7fba33fff640 1 -- 172.21.15.112:0/3965653953 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264f6360 msgr2=0x56162651ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.785+0000 7fba38af7640 1 --2- 172.21.15.112:0/3965653953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264faad0 0x56162651cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.785+0000 7fba33fff640 1 --2- 172.21.15.112:0/3965653953 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264f6360 0x56162651ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.785+0000 7fba33fff640 1 -- 172.21.15.112:0/3965653953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264faad0 msgr2=0x56162651cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.785+0000 7fba33fff640 1 --2- 172.21.15.112:0/3965653953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264faad0 0x56162651cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.785+0000 7fba33fff640 1 -- 172.21.15.112:0/3965653953 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616264f4000 con 0x5616264de510 2024-04-25T01:26:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.785+0000 7fba337fe640 1 --2- 172.21.15.112:0/3965653953 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264f6360 0x56162651ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.785+0000 7fba33fff640 1 --2- 172.21.15.112:0/3965653953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264de510 0x5616264f9730 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fba24015810 tx=0x7fba24017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494663f8b3826117 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.786+0000 7fba197fa640 1 -- 172.21.15.112:0/3965653953 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24028040 con 0x5616264de510 2024-04-25T01:26:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.786+0000 7fba197fa640 1 -- 172.21.15.112:0/3965653953 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba240159c0 con 0x5616264de510 2024-04-25T01:26:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.786+0000 7fba197fa640 1 -- 172.21.15.112:0/3965653953 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24034350 con 0x5616264de510 2024-04-25T01:26:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.786+0000 7fba3d264740 1 -- 172.21.15.112:0/3965653953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264de510 msgr2=0x5616264f9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.786+0000 7fba3d264740 1 --2- 172.21.15.112:0/3965653953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264de510 0x5616264f9730 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fba24015810 tx=0x7fba24017000 comp rx=0 tx=0).stop 2024-04-25T01:26:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.786+0000 7fba3d264740 1 -- 172.21.15.112:0/3965653953 shutdown_connections 2024-04-25T01:26:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.786+0000 7fba3d264740 1 --2- 172.21.15.112:0/3965653953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264faad0 0x56162651cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.786+0000 7fba3d264740 1 --2- 172.21.15.112:0/3965653953 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264f6360 0x56162651ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.786+0000 7fba3d264740 1 --2- 172.21.15.112:0/3965653953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264de510 0x5616264f9730 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.786+0000 7fba3d264740 1 -- 172.21.15.112:0/3965653953 >> 172.21.15.112:0/3965653953 conn(0x5616264bd8e0 msgr2=0x5616264b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.786+0000 7fba3d264740 1 -- 172.21.15.112:0/3965653953 shutdown_connections 2024-04-25T01:26:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.786+0000 7fba3d264740 1 -- 172.21.15.112:0/3965653953 wait complete. 2024-04-25T01:26:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.787+0000 7fba3d264740 1 -- start start 2024-04-25T01:26:16.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.787+0000 7fba3d264740 1 Processor -- start 2024-04-25T01:26:16.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba3d264740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264de510 0x5616264f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba3d264740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264f6360 0x5616264dbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba3d264740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264faad0 0x56162656d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba3d264740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5616264bc760 con 0x5616264f6360 2024-04-25T01:26:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba3d264740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5616264f3890 con 0x5616264de510 2024-04-25T01:26:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba3d264740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x561626509080 con 0x5616264faad0 2024-04-25T01:26:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba3d264740 10 client.0 _ll_register_callbacks cb 0x7fba2c009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:26:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba337fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264f6360 0x5616264dbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba337fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264f6360 0x5616264dbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43978/0 (socket says 172.21.15.112:43978) 2024-04-25T01:26:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba337fe640 1 -- 172.21.15.112:0/1458681519 learned_addr learned my addr 172.21.15.112:0/1458681519 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba33fff640 1 --2- 172.21.15.112:0/1458681519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264de510 0x5616264f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba38af7640 1 --2- 172.21.15.112:0/1458681519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264faad0 0x56162656d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba337fe640 1 -- 172.21.15.112:0/1458681519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264faad0 msgr2=0x56162656d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba337fe640 1 --2- 172.21.15.112:0/1458681519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264faad0 0x56162656d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba337fe640 1 -- 172.21.15.112:0/1458681519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264de510 msgr2=0x5616264f7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba337fe640 1 --2- 172.21.15.112:0/1458681519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264de510 0x5616264f7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba337fe640 1 -- 172.21.15.112:0/1458681519 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561626577a60 con 0x5616264f6360 2024-04-25T01:26:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba337fe640 1 --2- 172.21.15.112:0/1458681519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264f6360 0x5616264dbba0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fba280050a0 tx=0x7fba28007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b715ee1d25360b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T01:26:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba12ffd640 1 -- 172.21.15.112:0/1458681519 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2807c990 con 0x5616264f6360 2024-04-25T01:26:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba12ffd640 1 -- 172.21.15.112:0/1458681519 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fba28077a30 con 0x5616264f6360 2024-04-25T01:26:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba3d264740 1 -- 172.21.15.112:0/1458681519 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba240159c0 con 0x5616264f6360 2024-04-25T01:26:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba12ffd640 1 -- 172.21.15.112:0/1458681519 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=11) ==== 24+0+0 (secure 0 0 0) 0x7fba28081250 con 0x5616264f6360 2024-04-25T01:26:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba3d264740 10 client.4821 fetch_fsmap learned FSMap version 11 2024-04-25T01:26:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba3d264740 1 -- 172.21.15.112:0/1458681519 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=11,monmap=2+}) -- 0x5616264dc450 con 0x5616264f6360 2024-04-25T01:26:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba12ffd640 1 -- 172.21.15.112:0/1458681519 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba240159c0 con 0x5616264f6360 2024-04-25T01:26:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.788+0000 7fba12ffd640 1 -- 172.21.15.112:0/1458681519 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 11) ==== 4128+0+0 (secure 0 0 0) 0x7fba280a4020 con 0x5616264f6360 2024-04-25T01:26:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba3d264740 10 client.4821 fetch_fsmap finished waiting for FSMap version 11 2024-04-25T01:26:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba3d264740 10 client.4821 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T01:26:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba3d264740 20 client.4821 populate_metadata read hostname 'smithi112' 2024-04-25T01:26:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba3d264740 1 --2- 172.21.15.112:0/1458681519 >> [v2:172.21.15.112:6836/2115984531,v1:172.21.15.112:6837/2115984531] conn(0x561626504b70 0x561626504f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba3d264740 4 client.4821 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:26:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba3d264740 1 -- 172.21.15.112:0/1458681519 --> [v2:172.21.15.112:6836/2115984531,v1:172.21.15.112:6837/2115984531] -- command(tid 0: {"prefix": "status"}) -- 0x561626427790 con 0x561626504b70 2024-04-25T01:26:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba33fff640 1 --2- 172.21.15.112:0/1458681519 >> [v2:172.21.15.112:6836/2115984531,v1:172.21.15.112:6837/2115984531] conn(0x561626504b70 0x561626504f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba12ffd640 10 client.4821 ms_handle_connect on v2:172.21.15.112:6836/2115984531 2024-04-25T01:26:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba33fff640 1 --2- 172.21.15.112:0/1458681519 >> [v2:172.21.15.112:6836/2115984531,v1:172.21.15.112:6837/2115984531] conn(0x561626504b70 0x561626504f50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be0bbf18723196b6 server_cookie=7e69e486ba8eedf0 in_seq=0 out_seq=0 2024-04-25T01:26:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba12ffd640 1 -- 172.21.15.112:0/1458681519 <== mds.0 v2:172.21.15.112:6836/2115984531 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x561626427790 con 0x561626504b70 2024-04-25T01:26:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba12ffd640 10 client.4821 handle_command_reply: tid=0 2024-04-25T01:26:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba3d264740 1 client.4821 shutdown 2024-04-25T01:26:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba3d264740 1 -- 172.21.15.112:0/1458681519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264f6360 msgr2=0x5616264dbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba3d264740 1 --2- 172.21.15.112:0/1458681519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264f6360 0x5616264dbba0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fba280050a0 tx=0x7fba28007c80 comp rx=0 tx=0).stop 2024-04-25T01:26:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba3d264740 1 -- 172.21.15.112:0/1458681519 shutdown_connections 2024-04-25T01:26:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba3d264740 1 --2- 172.21.15.112:0/1458681519 >> [v2:172.21.15.112:6836/2115984531,v1:172.21.15.112:6837/2115984531] conn(0x561626504b70 0x561626504f50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba3d264740 1 --2- 172.21.15.112:0/1458681519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5616264faad0 0x56162656d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba3d264740 1 --2- 172.21.15.112:0/1458681519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616264f6360 0x5616264dbba0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba3d264740 1 --2- 172.21.15.112:0/1458681519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616264de510 0x5616264f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba12ffd640 0 client.4821 ms_handle_reset on v2:172.21.15.112:6836/2115984531 2024-04-25T01:26:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba3d264740 1 -- 172.21.15.112:0/1458681519 >> 172.21.15.112:0/1458681519 conn(0x5616264bd8e0 msgr2=0x5616264f5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba3d264740 1 -- 172.21.15.112:0/1458681519 shutdown_connections 2024-04-25T01:26:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba3d264740 1 -- 172.21.15.112:0/1458681519 wait complete. 2024-04-25T01:26:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.789+0000 7fba3d264740 20 client.4821 trim_cache size 0 max 16384 2024-04-25T01:26:16.776 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:26:16.777 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:26:16.777 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-04-25T01:26:16.777 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 4281, 2024-04-25T01:26:16.777 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-25T01:26:16.777 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-25T01:26:16.777 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "a", 2024-04-25T01:26:16.777 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 23.431807254999999, 2024-04-25T01:26:16.777 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 7, 2024-04-25T01:26:16.777 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 24, 2024-04-25T01:26:16.777 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T01:26:16.777 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 30.590354904000002 2024-04-25T01:26:16.777 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:26:16.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.790+0000 7fba3a55a640 1 -- 172.21.15.112:0/318511909 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fba2006be30 msgr2=0x7fba2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.790+0000 7fba3a55a640 1 --2- 172.21.15.112:0/318511909 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fba2006be30 0x7fba2006e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fba24007fd0 tx=0x7fba24007a70 comp rx=0 tx=0).stop 2024-04-25T01:26:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.790+0000 7fba3a55a640 1 -- 172.21.15.112:0/318511909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba340c6b40 msgr2=0x7fba341ef0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.790+0000 7fba3a55a640 1 --2- 172.21.15.112:0/318511909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba340c6b40 0x7fba341ef0c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fba28000c00 tx=0x7fba28001120 comp rx=0 tx=0).stop 2024-04-25T01:26:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.790+0000 7fba3a55a640 1 -- 172.21.15.112:0/318511909 shutdown_connections 2024-04-25T01:26:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.790+0000 7fba3a55a640 1 --2- 172.21.15.112:0/318511909 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fba2006be30 0x7fba2006e2f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.790+0000 7fba3a55a640 1 --2- 172.21.15.112:0/318511909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba3414c740 0x7fba341498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.790+0000 7fba3a55a640 1 --2- 172.21.15.112:0/318511909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba340c6b40 0x7fba341ef0c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.790+0000 7fba3a55a640 1 --2- 172.21.15.112:0/318511909 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba1c004800 0x7fba34145f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.790+0000 7fba3a55a640 1 -- 172.21.15.112:0/318511909 >> 172.21.15.112:0/318511909 conn(0x7fba3406b160 msgr2=0x7fba341526d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.790+0000 7fba3a55a640 1 -- 172.21.15.112:0/318511909 shutdown_connections 2024-04-25T01:26:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.790+0000 7fba3a55a640 1 -- 172.21.15.112:0/318511909 wait complete. 2024-04-25T01:26:16.780 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:26:16.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.898+0000 7fe6b8c38640 1 Processor -- start 2024-04-25T01:26:16.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.898+0000 7fe6b8c38640 1 -- start start 2024-04-25T01:26:16.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.898+0000 7fe6b8c38640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe6b40c3050 0x7fe6b40c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.899+0000 7fe6b8c38640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe6b4071770 0x7fe6b40c1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.899+0000 7fe6b8c38640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b41552d0 0x7fe6b4155740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.899+0000 7fe6b8c38640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fe6b40aa5b0 con 0x7fe6b4071770 2024-04-25T01:26:16.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.899+0000 7fe6b8c38640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe6b40c2500 con 0x7fe6b40c3050 2024-04-25T01:26:16.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.899+0000 7fe6b8c38640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b40abd70 con 0x7fe6b41552d0 2024-04-25T01:26:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.899+0000 7fe6abfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe6b4071770 0x7fe6b40c1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.899+0000 7fe6abfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe6b4071770 0x7fe6b40c1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43994/0 (socket says 172.21.15.112:43994) 2024-04-25T01:26:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.899+0000 7fe6abfff640 1 -- 172.21.15.112:0/2969493281 learned_addr learned my addr 172.21.15.112:0/2969493281 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.899+0000 7fe6b2d9d640 1 --2- 172.21.15.112:0/2969493281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b41552d0 0x7fe6b4155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.899+0000 7fe6b259c640 1 --2- 172.21.15.112:0/2969493281 >> v2:172.21.15.187:3300/0 conn(0x7fe6b40c3050 0x7fe6b40c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.899+0000 7fe6abfff640 1 -- 172.21.15.112:0/2969493281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b41552d0 msgr2=0x7fe6b4155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.899+0000 7fe6abfff640 1 --2- 172.21.15.112:0/2969493281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b41552d0 0x7fe6b4155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.899+0000 7fe6abfff640 1 -- 172.21.15.112:0/2969493281 >> v2:172.21.15.187:3300/0 conn(0x7fe6b40c3050 msgr2=0x7fe6b40c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.899+0000 7fe6abfff640 1 --2- 172.21.15.112:0/2969493281 >> v2:172.21.15.187:3300/0 conn(0x7fe6b40c3050 0x7fe6b40c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.899+0000 7fe6abfff640 1 -- 172.21.15.112:0/2969493281 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b40ac100 con 0x7fe6b4071770 2024-04-25T01:26:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.900+0000 7fe6abfff640 1 --2- 172.21.15.112:0/2969493281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b4071770 0x7fe6b40c1fc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40c1950 tx=0x7fe69c02b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96fad355fedd114 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.900+0000 7fe6b1d9b640 1 -- 172.21.15.112:0/2969493281 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c00b040 con 0x7fe6b4071770 2024-04-25T01:26:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.900+0000 7fe6b1d9b640 1 -- 172.21.15.112:0/2969493281 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe69c030710 con 0x7fe6b4071770 2024-04-25T01:26:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.900+0000 7fe6b1d9b640 1 -- 172.21.15.112:0/2969493281 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c030ae0 con 0x7fe6b4071770 2024-04-25T01:26:16.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.900+0000 7fe6b8c38640 1 -- 172.21.15.112:0/2969493281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b4071770 msgr2=0x7fe6b40c1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:16.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.900+0000 7fe6b8c38640 1 --2- 172.21.15.112:0/2969493281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b4071770 0x7fe6b40c1fc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40c1950 tx=0x7fe69c02b510 comp rx=0 tx=0).stop 2024-04-25T01:26:16.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.900+0000 7fe6b8c38640 1 -- 172.21.15.112:0/2969493281 shutdown_connections 2024-04-25T01:26:16.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.900+0000 7fe6b8c38640 1 --2- 172.21.15.112:0/2969493281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b41552d0 0x7fe6b4155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.900+0000 7fe6b8c38640 1 --2- 172.21.15.112:0/2969493281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b4071770 0x7fe6b40c1fc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.900+0000 7fe6b8c38640 1 --2- 172.21.15.112:0/2969493281 >> v2:172.21.15.187:3300/0 conn(0x7fe6b40c3050 0x7fe6b40c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.900+0000 7fe6b8c38640 1 -- 172.21.15.112:0/2969493281 >> 172.21.15.112:0/2969493281 conn(0x7fe6b406b160 msgr2=0x7fe6b406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:16.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.900+0000 7fe6b8c38640 1 -- 172.21.15.112:0/2969493281 shutdown_connections 2024-04-25T01:26:16.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.900+0000 7fe6b8c38640 1 -- 172.21.15.112:0/2969493281 wait complete. 2024-04-25T01:26:16.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.902+0000 7fe6b8c38640 1 Processor -- start 2024-04-25T01:26:16.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.902+0000 7fe6b8c38640 1 -- start start 2024-04-25T01:26:16.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.902+0000 7fe6b8c38640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b4071770 0x7fe6b40c1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.902+0000 7fe6b8c38640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b40c3050 0x7fe6b4152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.902+0000 7fe6b8c38640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b41552d0 0x7fe6b414cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.902+0000 7fe6b8c38640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b40ab850 con 0x7fe6b4071770 2024-04-25T01:26:16.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.902+0000 7fe6b8c38640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b40ab570 con 0x7fe6b40c3050 2024-04-25T01:26:16.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.902+0000 7fe6b8c38640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b4155c80 con 0x7fe6b41552d0 2024-04-25T01:26:16.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.902+0000 7fe6b259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b4071770 0x7fe6b40c1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.902+0000 7fe6b259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b4071770 0x7fe6b40c1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44002/0 (socket says 172.21.15.112:44002) 2024-04-25T01:26:16.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.902+0000 7fe6b259c640 1 -- 172.21.15.112:0/170027832 learned_addr learned my addr 172.21.15.112:0/170027832 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:16.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.903+0000 7fe6b259c640 1 -- 172.21.15.112:0/170027832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b41552d0 msgr2=0x7fe6b414cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:16.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.903+0000 7fe6b259c640 1 --2- 172.21.15.112:0/170027832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b41552d0 0x7fe6b414cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.903+0000 7fe6b259c640 1 -- 172.21.15.112:0/170027832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b40c3050 msgr2=0x7fe6b4152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:16.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.903+0000 7fe6b259c640 1 --2- 172.21.15.112:0/170027832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b40c3050 0x7fe6b4152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:16.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.903+0000 7fe6b259c640 1 -- 172.21.15.112:0/170027832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b4151410 con 0x7fe6b4071770 2024-04-25T01:26:16.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.903+0000 7fe6b259c640 1 --2- 172.21.15.112:0/170027832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b4071770 0x7fe6b40c1000 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400d800 tx=0x7fe6a400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:16.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.903+0000 7fe6aaffd640 1 -- 172.21.15.112:0/170027832 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a400b6c0 con 0x7fe6b4071770 2024-04-25T01:26:16.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.903+0000 7fe6aaffd640 1 -- 172.21.15.112:0/170027832 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a4019070 con 0x7fe6b4071770 2024-04-25T01:26:16.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.903+0000 7fe6b8c38640 1 -- 172.21.15.112:0/170027832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b40c5e10 con 0x7fe6b4071770 2024-04-25T01:26:16.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.903+0000 7fe6aaffd640 1 -- 172.21.15.112:0/170027832 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a400ba80 con 0x7fe6b4071770 2024-04-25T01:26:16.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.903+0000 7fe6b8c38640 1 -- 172.21.15.112:0/170027832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b41516b0 con 0x7fe6b4071770 2024-04-25T01:26:16.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.904+0000 7fe6b8c38640 1 -- 172.21.15.112:0/170027832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b40682a0 con 0x7fe6b4071770 2024-04-25T01:26:16.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.904+0000 7fe6aaffd640 1 -- 172.21.15.112:0/170027832 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe6a400bc20 con 0x7fe6b4071770 2024-04-25T01:26:16.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.905+0000 7fe6aaffd640 1 --2- 172.21.15.112:0/170027832 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe69406be30 0x7fe69406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:16.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.905+0000 7fe6aaffd640 1 -- 172.21.15.112:0/170027832 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fe6a408b5d0 con 0x7fe6b4071770 2024-04-25T01:26:16.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.905+0000 7fe6abfff640 1 --2- 172.21.15.112:0/170027832 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe69406be30 0x7fe69406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:16.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.905+0000 7fe6abfff640 1 --2- 172.21.15.112:0/170027832 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe69406be30 0x7fe69406e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe69c035000 tx=0x7fe69c02f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:16.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:16.909+0000 7fe6aaffd640 1 -- 172.21.15.112:0/170027832 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe6a4059ce0 con 0x7fe6b4071770 2024-04-25T01:26:17.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.084+0000 7fe6b8c38640 1 -- 172.21.15.112:0/170027832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6b4072c40 con 0x7fe6b4071770 2024-04-25T01:26:17.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.086+0000 7fe6aaffd640 1 -- 172.21.15.112:0/170027832 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20367 (secure 0 0 0) 0x7fe6a40595e0 con 0x7fe6b4071770 2024-04-25T01:26:17.065 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:17.065 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:26:03.405271+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-25T01:25:59.478275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-25T01:26:00.719358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:26:17.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.088+0000 7fe6b8c38640 1 -- 172.21.15.112:0/170027832 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe69406be30 msgr2=0x7fe69406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.088+0000 7fe6b8c38640 1 --2- 172.21.15.112:0/170027832 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe69406be30 0x7fe69406e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe69c035000 tx=0x7fe69c02f000 comp rx=0 tx=0).stop 2024-04-25T01:26:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.088+0000 7fe6b8c38640 1 -- 172.21.15.112:0/170027832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b4071770 msgr2=0x7fe6b40c1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.088+0000 7fe6b8c38640 1 --2- 172.21.15.112:0/170027832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b4071770 0x7fe6b40c1000 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400d800 tx=0x7fe6a400dcc0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.088+0000 7fe6b8c38640 1 -- 172.21.15.112:0/170027832 shutdown_connections 2024-04-25T01:26:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.088+0000 7fe6b8c38640 1 --2- 172.21.15.112:0/170027832 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe69406be30 0x7fe69406e2f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.088+0000 7fe6b8c38640 1 --2- 172.21.15.112:0/170027832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b41552d0 0x7fe6b414cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.088+0000 7fe6b8c38640 1 --2- 172.21.15.112:0/170027832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b40c3050 0x7fe6b4152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.088+0000 7fe6b8c38640 1 --2- 172.21.15.112:0/170027832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b4071770 0x7fe6b40c1000 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.088+0000 7fe6b8c38640 1 -- 172.21.15.112:0/170027832 >> 172.21.15.112:0/170027832 conn(0x7fe6b406b160 msgr2=0x7fe6b41de610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.088+0000 7fe6b8c38640 1 -- 172.21.15.112:0/170027832 shutdown_connections 2024-04-25T01:26:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.088+0000 7fe6b8c38640 1 -- 172.21.15.112:0/170027832 wait complete. 2024-04-25T01:26:17.078 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:25:40.679119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-04-25T01:25:50.593045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-04-25T01:25:51.664930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-04-25T01:25:59.478275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-04-25T01:26:00.719358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:26:17.079 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:26:17.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.198+0000 7fba78238640 1 Processor -- start 2024-04-25T01:26:17.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.199+0000 7fba78238640 1 -- start start 2024-04-25T01:26:17.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.199+0000 7fba78238640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fba7006d460 0x7fba70071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:17.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.199+0000 7fba78238640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba700724f0 0x7fba7015ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:17.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.199+0000 7fba78238640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba7006f290 con 0x7fba7014fbe0 2024-04-25T01:26:17.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.199+0000 7fba78238640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fba7014ffc0 con 0x7fba7006d460 2024-04-25T01:26:17.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.199+0000 7fba78238640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fba701533a0 con 0x7fba700724f0 2024-04-25T01:26:17.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.199+0000 7fba757d3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fba7006d460 0x7fba70071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:17.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.199+0000 7fba75fd4640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fba7014fbe0 0x7fba700c2920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38180/0 (socket says 172.21.15.112:38180) 2024-04-25T01:26:17.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.200+0000 7fba75fd4640 1 -- 172.21.15.112:0/2286471977 learned_addr learned my addr 172.21.15.112:0/2286471977 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:17.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.200+0000 7fba767d5640 1 --2- 172.21.15.112:0/2286471977 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba700724f0 0x7fba7015ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:17.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.200+0000 7fba757d3640 1 -- 172.21.15.112:0/2286471977 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba700724f0 msgr2=0x7fba7015ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:17.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.200+0000 7fba757d3640 1 --2- 172.21.15.112:0/2286471977 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba700724f0 0x7fba7015ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.200+0000 7fba757d3640 1 -- 172.21.15.112:0/2286471977 >> v1:172.21.15.112:6789/0 conn(0x7fba7014fbe0 legacy=0x7fba700c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:17.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.200+0000 7fba757d3640 1 -- 172.21.15.112:0/2286471977 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7006efb0 con 0x7fba7006d460 2024-04-25T01:26:17.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.200+0000 7fba74fd2640 1 -- 172.21.15.112:0/2286471977 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2238052210 0 0) 0x7fba7006f290 con 0x7fba7014fbe0 2024-04-25T01:26:17.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.200+0000 7fba757d3640 1 --2- 172.21.15.112:0/2286471977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba7006d460 0x7fba70071fb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fba600097a0 tx=0x7fba6002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97e61826a715665 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:17.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.201+0000 7fba74fd2640 1 -- 172.21.15.112:0/2286471977 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60037070 con 0x7fba7006d460 2024-04-25T01:26:17.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.201+0000 7fba74fd2640 1 -- 172.21.15.112:0/2286471977 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6002ae50 con 0x7fba7006d460 2024-04-25T01:26:17.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.201+0000 7fba74fd2640 1 -- 172.21.15.112:0/2286471977 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60032660 con 0x7fba7006d460 2024-04-25T01:26:17.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.201+0000 7fba78238640 1 -- 172.21.15.112:0/2286471977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba7006d460 msgr2=0x7fba70071fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:17.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.201+0000 7fba78238640 1 --2- 172.21.15.112:0/2286471977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba7006d460 0x7fba70071fb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fba600097a0 tx=0x7fba6002a100 comp rx=0 tx=0).stop 2024-04-25T01:26:17.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.201+0000 7fba78238640 1 -- 172.21.15.112:0/2286471977 shutdown_connections 2024-04-25T01:26:17.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.201+0000 7fba78238640 1 --2- 172.21.15.112:0/2286471977 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba700724f0 0x7fba7015ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.201+0000 7fba78238640 1 --2- 172.21.15.112:0/2286471977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba7006d460 0x7fba70071fb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.201+0000 7fba78238640 1 -- 172.21.15.112:0/2286471977 >> 172.21.15.112:0/2286471977 conn(0x7fba700c3a10 msgr2=0x7fba700c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:17.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.201+0000 7fba78238640 1 -- 172.21.15.112:0/2286471977 shutdown_connections 2024-04-25T01:26:17.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.201+0000 7fba78238640 1 -- 172.21.15.112:0/2286471977 wait complete. 2024-04-25T01:26:17.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.202+0000 7fba78238640 1 Processor -- start 2024-04-25T01:26:17.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.202+0000 7fba78238640 1 -- start start 2024-04-25T01:26:17.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.203+0000 7fba78238640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba7006d460 0x7fba701ef760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:17.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.203+0000 7fba78238640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba700724f0 0x7fba70070f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:17.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.203+0000 7fba78238640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba7014fbe0 0x7fba700c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:17.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.203+0000 7fba78238640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fba700c3030 con 0x7fba7006d460 2024-04-25T01:26:17.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.203+0000 7fba78238640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fba70153030 con 0x7fba700724f0 2024-04-25T01:26:17.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.203+0000 7fba78238640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fba700c31b0 con 0x7fba7014fbe0 2024-04-25T01:26:17.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.203+0000 7fba75fd4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba7006d460 0x7fba701ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:17.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.203+0000 7fba75fd4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba7006d460 0x7fba701ef760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44024/0 (socket says 172.21.15.112:44024) 2024-04-25T01:26:17.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.203+0000 7fba75fd4640 1 -- 172.21.15.112:0/1161995785 learned_addr learned my addr 172.21.15.112:0/1161995785 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:17.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.203+0000 7fba767d5640 1 --2- 172.21.15.112:0/1161995785 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba7014fbe0 0x7fba700c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:17.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.203+0000 7fba75fd4640 1 -- 172.21.15.112:0/1161995785 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba7014fbe0 msgr2=0x7fba700c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:17.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.203+0000 7fba75fd4640 1 --2- 172.21.15.112:0/1161995785 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba7014fbe0 0x7fba700c4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.203+0000 7fba75fd4640 1 -- 172.21.15.112:0/1161995785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba700724f0 msgr2=0x7fba70070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:17.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.203+0000 7fba757d3640 1 --2- 172.21.15.112:0/1161995785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba700724f0 0x7fba70070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:17.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.203+0000 7fba75fd4640 1 --2- 172.21.15.112:0/1161995785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba700724f0 0x7fba70070f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.203+0000 7fba75fd4640 1 -- 172.21.15.112:0/1161995785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba70149490 con 0x7fba7006d460 2024-04-25T01:26:17.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.203+0000 7fba75fd4640 1 --2- 172.21.15.112:0/1161995785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba7006d460 0x7fba701ef760 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fba6800c430 tx=0x7fba6800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:17.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.204+0000 7fba667fc640 1 -- 172.21.15.112:0/1161995785 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba680074e0 con 0x7fba7006d460 2024-04-25T01:26:17.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.204+0000 7fba667fc640 1 -- 172.21.15.112:0/1161995785 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba680047e0 con 0x7fba7006d460 2024-04-25T01:26:17.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.204+0000 7fba78238640 1 -- 172.21.15.112:0/1161995785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba70149770 con 0x7fba7006d460 2024-04-25T01:26:17.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.204+0000 7fba667fc640 1 -- 172.21.15.112:0/1161995785 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68004bd0 con 0x7fba7006d460 2024-04-25T01:26:17.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.204+0000 7fba78238640 1 -- 172.21.15.112:0/1161995785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba701ef240 con 0x7fba7006d460 2024-04-25T01:26:17.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.205+0000 7fba78238640 1 -- 172.21.15.112:0/1161995785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba7004f500 con 0x7fba7006d460 2024-04-25T01:26:17.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.205+0000 7fba667fc640 1 -- 172.21.15.112:0/1161995785 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba68011070 con 0x7fba7006d460 2024-04-25T01:26:17.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.206+0000 7fba667fc640 1 --2- 172.21.15.112:0/1161995785 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fba5406be30 0x7fba5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:17.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.206+0000 7fba757d3640 1 --2- 172.21.15.112:0/1161995785 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fba5406be30 0x7fba5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:17.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.206+0000 7fba667fc640 1 -- 172.21.15.112:0/1161995785 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fba6808ae30 con 0x7fba7006d460 2024-04-25T01:26:17.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.206+0000 7fba757d3640 1 --2- 172.21.15.112:0/1161995785 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fba5406be30 0x7fba5406e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fba60034000 tx=0x7fba6002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:17.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.210+0000 7fba667fc640 1 -- 172.21.15.112:0/1161995785 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fba680594b0 con 0x7fba7006d460 2024-04-25T01:26:17.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.383+0000 7fba78238640 1 -- 172.21.15.112:0/1161995785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fba700682a0 con 0x7fba7006d460 2024-04-25T01:26:17.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.383+0000 7fba667fc640 1 -- 172.21.15.112:0/1161995785 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fba68058db0 con 0x7fba7006d460 2024-04-25T01:26:17.363 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T01:26:17.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.385+0000 7fba78238640 1 -- 172.21.15.112:0/1161995785 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fba5406be30 msgr2=0x7fba5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:17.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.385+0000 7fba78238640 1 --2- 172.21.15.112:0/1161995785 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fba5406be30 0x7fba5406e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fba60034000 tx=0x7fba6002e000 comp rx=0 tx=0).stop 2024-04-25T01:26:17.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.385+0000 7fba78238640 1 -- 172.21.15.112:0/1161995785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba7006d460 msgr2=0x7fba701ef760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:17.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.385+0000 7fba78238640 1 --2- 172.21.15.112:0/1161995785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba7006d460 0x7fba701ef760 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fba6800c430 tx=0x7fba6800c8f0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.385+0000 7fba78238640 1 -- 172.21.15.112:0/1161995785 shutdown_connections 2024-04-25T01:26:17.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.385+0000 7fba78238640 1 --2- 172.21.15.112:0/1161995785 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fba5406be30 0x7fba5406e2f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.385+0000 7fba78238640 1 --2- 172.21.15.112:0/1161995785 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba7014fbe0 0x7fba700c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.385+0000 7fba78238640 1 --2- 172.21.15.112:0/1161995785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba700724f0 0x7fba70070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.385+0000 7fba78238640 1 --2- 172.21.15.112:0/1161995785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba7006d460 0x7fba701ef760 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.385+0000 7fba78238640 1 -- 172.21.15.112:0/1161995785 >> 172.21.15.112:0/1161995785 conn(0x7fba700c3a10 msgr2=0x7fba7014a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:17.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.386+0000 7fba78238640 1 -- 172.21.15.112:0/1161995785 shutdown_connections 2024-04-25T01:26:17.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.386+0000 7fba78238640 1 -- 172.21.15.112:0/1161995785 wait complete. 2024-04-25T01:26:17.376 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-04-25T01:26:17.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.498+0000 7f1f6fdc8640 1 Processor -- start 2024-04-25T01:26:17.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.498+0000 7f1f6fdc8640 1 -- start start 2024-04-25T01:26:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.499+0000 7f1f6fdc8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1f6814f220 0x7f1f680aa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.499+0000 7f1f6fdc8640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f68155d40 con 0x7f1f680ac830 2024-04-25T01:26:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.499+0000 7f1f6fdc8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f68156020 con 0x7f1f68072c20 2024-04-25T01:26:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.499+0000 7f1f6fdc8640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f1f68071700 con 0x7f1f6814f220 2024-04-25T01:26:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.499+0000 7f1f6db64640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1f6814f220 0x7f1f680aa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.499+0000 7f1f6d363640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1f680ac830 0x7f1f68072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53418/0 (socket says 172.21.15.112:53418) 2024-04-25T01:26:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.499+0000 7f1f6d363640 1 -- 172.21.15.112:0/1764058207 learned_addr learned my addr 172.21.15.112:0/1764058207 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.500+0000 7f1f6db64640 1 -- 172.21.15.112:0/1764058207 >> v1:172.21.15.187:6789/0 conn(0x7f1f68072c20 legacy=0x7f1f68155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:17.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.500+0000 7f1f6db64640 1 -- 172.21.15.112:0/1764058207 >> v1:172.21.15.112:6789/0 conn(0x7f1f680ac830 legacy=0x7f1f68072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:17.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.500+0000 7f1f6db64640 1 -- 172.21.15.112:0/1764058207 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f68071950 con 0x7f1f6814f220 2024-04-25T01:26:17.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.500+0000 7f1f6db64640 1 --2- 172.21.15.112:0/1764058207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f6814f220 0x7f1f680aa640 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1f68071880 tx=0x7f1f6402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966e0addcb612405 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:17.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.500+0000 7f1f6cb62640 1 -- 172.21.15.112:0/1764058207 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64007740 con 0x7f1f6814f220 2024-04-25T01:26:17.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.500+0000 7f1f6cb62640 1 -- 172.21.15.112:0/1764058207 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f640078e0 con 0x7f1f6814f220 2024-04-25T01:26:17.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.500+0000 7f1f6cb62640 1 -- 172.21.15.112:0/1764058207 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64007cb0 con 0x7f1f6814f220 2024-04-25T01:26:17.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.501+0000 7f1f6fdc8640 1 -- 172.21.15.112:0/1764058207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f6814f220 msgr2=0x7f1f680aa640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:17.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.501+0000 7f1f6fdc8640 1 --2- 172.21.15.112:0/1764058207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f6814f220 0x7f1f680aa640 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1f68071880 tx=0x7f1f6402a560 comp rx=0 tx=0).stop 2024-04-25T01:26:17.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.501+0000 7f1f6fdc8640 1 -- 172.21.15.112:0/1764058207 shutdown_connections 2024-04-25T01:26:17.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.501+0000 7f1f6fdc8640 1 --2- 172.21.15.112:0/1764058207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f6814f220 0x7f1f680aa640 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.501+0000 7f1f6fdc8640 1 -- 172.21.15.112:0/1764058207 >> 172.21.15.112:0/1764058207 conn(0x7f1f680c3a10 msgr2=0x7f1f680c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:17.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.501+0000 7f1f6fdc8640 1 -- 172.21.15.112:0/1764058207 shutdown_connections 2024-04-25T01:26:17.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.501+0000 7f1f6fdc8640 1 -- 172.21.15.112:0/1764058207 wait complete. 2024-04-25T01:26:17.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.502+0000 7f1f6fdc8640 1 Processor -- start 2024-04-25T01:26:17.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.502+0000 7f1f6fdc8640 1 -- start start 2024-04-25T01:26:17.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f6fdc8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f68072c20 0x7f1f68137ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f6fdc8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f680ac830 0x7f1f68154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f6fdc8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f6814f220 0x7f1f681e9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f6fdc8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6814f600 con 0x7f1f680ac830 2024-04-25T01:26:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f6fdc8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1f68071cc0 con 0x7f1f6814f220 2024-04-25T01:26:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f6fdc8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1f681559a0 con 0x7f1f68072c20 2024-04-25T01:26:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f6d363640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f680ac830 0x7f1f68154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f6d363640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f680ac830 0x7f1f68154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45976/0 (socket says 172.21.15.112:45976) 2024-04-25T01:26:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f6d363640 1 -- 172.21.15.112:0/676402369 learned_addr learned my addr 172.21.15.112:0/676402369 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f6db64640 1 --2- 172.21.15.112:0/676402369 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f68072c20 0x7f1f68137ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f6e365640 1 --2- 172.21.15.112:0/676402369 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f6814f220 0x7f1f681e9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f6d363640 1 -- 172.21.15.112:0/676402369 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f68072c20 msgr2=0x7f1f68137ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f6d363640 1 --2- 172.21.15.112:0/676402369 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f68072c20 0x7f1f68137ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f6d363640 1 -- 172.21.15.112:0/676402369 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f6814f220 msgr2=0x7f1f681e9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f6d363640 1 --2- 172.21.15.112:0/676402369 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f6814f220 0x7f1f681e9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f6d363640 1 -- 172.21.15.112:0/676402369 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f681e9aa0 con 0x7f1f680ac830 2024-04-25T01:26:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f6d363640 1 --2- 172.21.15.112:0/676402369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f680ac830 0x7f1f68154f60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800c7b0 tx=0x7f1f5800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f5e7fc640 1 -- 172.21.15.112:0/676402369 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58018070 con 0x7f1f680ac830 2024-04-25T01:26:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f5e7fc640 1 -- 172.21.15.112:0/676402369 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f580072d0 con 0x7f1f680ac830 2024-04-25T01:26:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f6fdc8640 1 -- 172.21.15.112:0/676402369 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f68138010 con 0x7f1f680ac830 2024-04-25T01:26:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f5e7fc640 1 -- 172.21.15.112:0/676402369 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f580047e0 con 0x7f1f680ac830 2024-04-25T01:26:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.503+0000 7f1f6fdc8640 1 -- 172.21.15.112:0/676402369 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f681ed780 con 0x7f1f680ac830 2024-04-25T01:26:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.504+0000 7f1f6fdc8640 1 -- 172.21.15.112:0/676402369 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f6804f500 con 0x7f1f680ac830 2024-04-25T01:26:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.505+0000 7f1f5e7fc640 1 -- 172.21.15.112:0/676402369 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f5801b070 con 0x7f1f680ac830 2024-04-25T01:26:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.505+0000 7f1f5e7fc640 1 --2- 172.21.15.112:0/676402369 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1f4c06be30 0x7f1f4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.505+0000 7f1f6db64640 1 --2- 172.21.15.112:0/676402369 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1f4c06be30 0x7f1f4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.505+0000 7f1f5e7fc640 1 -- 172.21.15.112:0/676402369 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f1f5808a600 con 0x7f1f680ac830 2024-04-25T01:26:17.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.506+0000 7f1f6db64640 1 --2- 172.21.15.112:0/676402369 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1f4c06be30 0x7f1f4c06e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1f64034000 tx=0x7f1f6402f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:17.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.509+0000 7f1f5e7fc640 1 -- 172.21.15.112:0/676402369 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1f58058d10 con 0x7f1f680ac830 2024-04-25T01:26:17.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.685+0000 7f1f6fdc8640 1 -- 172.21.15.112:0/676402369 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) -- 0x7f1f680c36e0 con 0x7f1f680ac830 2024-04-25T01:26:17.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.685+0000 7f1f5e7fc640 1 -- 172.21.15.112:0/676402369 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7f1f58058610 con 0x7f1f680ac830 2024-04-25T01:26:17.665 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T01:26:17.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.687+0000 7f1f6fdc8640 1 -- 172.21.15.112:0/676402369 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1f4c06be30 msgr2=0x7f1f4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:17.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.687+0000 7f1f6fdc8640 1 --2- 172.21.15.112:0/676402369 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1f4c06be30 0x7f1f4c06e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1f64034000 tx=0x7f1f6402f000 comp rx=0 tx=0).stop 2024-04-25T01:26:17.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.688+0000 7f1f6fdc8640 1 -- 172.21.15.112:0/676402369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f680ac830 msgr2=0x7f1f68154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:17.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.688+0000 7f1f6fdc8640 1 --2- 172.21.15.112:0/676402369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f680ac830 0x7f1f68154f60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800c7b0 tx=0x7f1f5800cc70 comp rx=0 tx=0).stop 2024-04-25T01:26:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.688+0000 7f1f6fdc8640 1 -- 172.21.15.112:0/676402369 shutdown_connections 2024-04-25T01:26:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.688+0000 7f1f6fdc8640 1 --2- 172.21.15.112:0/676402369 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1f4c06be30 0x7f1f4c06e2f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.688+0000 7f1f6fdc8640 1 --2- 172.21.15.112:0/676402369 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f6814f220 0x7f1f681e9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.688+0000 7f1f6fdc8640 1 --2- 172.21.15.112:0/676402369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f680ac830 0x7f1f68154f60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.688+0000 7f1f6fdc8640 1 --2- 172.21.15.112:0/676402369 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f68072c20 0x7f1f68137ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.688+0000 7f1f6fdc8640 1 -- 172.21.15.112:0/676402369 >> 172.21.15.112:0/676402369 conn(0x7f1f680c3a10 msgr2=0x7f1f6814a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.688+0000 7f1f6fdc8640 1 -- 172.21.15.112:0/676402369 shutdown_connections 2024-04-25T01:26:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.688+0000 7f1f6fdc8640 1 -- 172.21.15.112:0/676402369 wait complete. 2024-04-25T01:26:17.678 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-04-25T01:26:17.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.798+0000 7f84a2d4f640 1 Processor -- start 2024-04-25T01:26:17.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.798+0000 7f84a2d4f640 1 -- start start 2024-04-25T01:26:17.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.799+0000 7f84a2d4f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f849c06ebb0 0x7f849c071200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:17.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.799+0000 7f84a2d4f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f849c06cf50 0x7f849c13dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:17.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.799+0000 7f84a2d4f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f849c158fe0 con 0x7f849c13e1b0 2024-04-25T01:26:17.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.799+0000 7f84a2d4f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f849c158d90 con 0x7f849c06ebb0 2024-04-25T01:26:17.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.799+0000 7f84a2d4f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f849c159bf0 con 0x7f849c06cf50 2024-04-25T01:26:17.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.799+0000 7f84a0aeb640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f849c06ebb0 0x7f849c071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:17.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.799+0000 7f84a0aeb640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f849c06ebb0 0x7f849c071200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45990/0 (socket says 172.21.15.112:45990) 2024-04-25T01:26:17.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.799+0000 7f84a0aeb640 1 -- 172.21.15.112:0/2834101036 learned_addr learned my addr 172.21.15.112:0/2834101036 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:17.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.799+0000 7f84a0aeb640 1 -- 172.21.15.112:0/2834101036 >> v2:172.21.15.187:3300/0 conn(0x7f849c06cf50 msgr2=0x7f849c13dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:17.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.799+0000 7f84a0aeb640 1 --2- 172.21.15.112:0/2834101036 >> v2:172.21.15.187:3300/0 conn(0x7f849c06cf50 0x7f849c13dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.799+0000 7f84a0aeb640 1 -- 172.21.15.112:0/2834101036 >> v1:172.21.15.187:6789/0 conn(0x7f849c13e1b0 legacy=0x7f849c1559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:17.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.799+0000 7f84a0aeb640 1 -- 172.21.15.112:0/2834101036 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849c1592c0 con 0x7f849c06ebb0 2024-04-25T01:26:17.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.800+0000 7f84a0aeb640 1 --2- 172.21.15.112:0/2834101036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f849c06ebb0 0x7f849c071200 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f849c0ad280 tx=0x7f849002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e3b21eb9fb253e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:17.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.800+0000 7f849b7fe640 1 -- 172.21.15.112:0/2834101036 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490037070 con 0x7f849c06ebb0 2024-04-25T01:26:17.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.800+0000 7f849b7fe640 1 -- 172.21.15.112:0/2834101036 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8490032440 con 0x7f849c06ebb0 2024-04-25T01:26:17.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.800+0000 7f849b7fe640 1 -- 172.21.15.112:0/2834101036 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490032810 con 0x7f849c06ebb0 2024-04-25T01:26:17.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.800+0000 7f84a2d4f640 1 -- 172.21.15.112:0/2834101036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f849c06ebb0 msgr2=0x7f849c071200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:17.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.800+0000 7f84a2d4f640 1 --2- 172.21.15.112:0/2834101036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f849c06ebb0 0x7f849c071200 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f849c0ad280 tx=0x7f849002a1a0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.800+0000 7f84a2d4f640 1 -- 172.21.15.112:0/2834101036 shutdown_connections 2024-04-25T01:26:17.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.800+0000 7f84a2d4f640 1 --2- 172.21.15.112:0/2834101036 >> v2:172.21.15.187:3300/0 conn(0x7f849c06cf50 0x7f849c13dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.800+0000 7f84a2d4f640 1 --2- 172.21.15.112:0/2834101036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f849c06ebb0 0x7f849c071200 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.800+0000 7f84a2d4f640 1 -- 172.21.15.112:0/2834101036 >> 172.21.15.112:0/2834101036 conn(0x7f849c0c3a10 msgr2=0x7f849c0c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:17.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.800+0000 7f84a2d4f640 1 -- 172.21.15.112:0/2834101036 shutdown_connections 2024-04-25T01:26:17.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.800+0000 7f84a2d4f640 1 -- 172.21.15.112:0/2834101036 wait complete. 2024-04-25T01:26:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.801+0000 7f84a2d4f640 1 Processor -- start 2024-04-25T01:26:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.801+0000 7f84a2d4f640 1 -- start start 2024-04-25T01:26:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.802+0000 7f84a2d4f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f849c06cf50 0x7f849c0c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.802+0000 7f84a2d4f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f849c06ebb0 0x7f849c154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.802+0000 7f84a2d4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f849c13e1b0 0x7f849c1580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.802+0000 7f84a2d4f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f849c06ef90 con 0x7f849c13e1b0 2024-04-25T01:26:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.802+0000 7f84a2d4f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f849c159880 con 0x7f849c06cf50 2024-04-25T01:26:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.802+0000 7f84a2d4f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f849c06f110 con 0x7f849c06ebb0 2024-04-25T01:26:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.802+0000 7f84a12ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f849c13e1b0 0x7f849c1580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.802+0000 7f84a12ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f849c13e1b0 0x7f849c1580c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45998/0 (socket says 172.21.15.112:45998) 2024-04-25T01:26:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.802+0000 7f84a12ec640 1 -- 172.21.15.112:0/2375068886 learned_addr learned my addr 172.21.15.112:0/2375068886 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.802+0000 7f849bfff640 1 --2- 172.21.15.112:0/2375068886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f849c06ebb0 0x7f849c154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.802+0000 7f84a0aeb640 1 --2- 172.21.15.112:0/2375068886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f849c06cf50 0x7f849c0c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.803+0000 7f84a12ec640 1 -- 172.21.15.112:0/2375068886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f849c06ebb0 msgr2=0x7f849c154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:17.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.803+0000 7f84a12ec640 1 --2- 172.21.15.112:0/2375068886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f849c06ebb0 0x7f849c154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.803+0000 7f84a12ec640 1 -- 172.21.15.112:0/2375068886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f849c06cf50 msgr2=0x7f849c0c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:17.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.803+0000 7f84a12ec640 1 --2- 172.21.15.112:0/2375068886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f849c06cf50 0x7f849c0c5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.803+0000 7f84a12ec640 1 -- 172.21.15.112:0/2375068886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849c1eea60 con 0x7f849c13e1b0 2024-04-25T01:26:17.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.803+0000 7f84a12ec640 1 --2- 172.21.15.112:0/2375068886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f849c13e1b0 0x7f849c1580c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f849400b990 tx=0x7f849400bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:17.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.803+0000 7f84997fa640 1 -- 172.21.15.112:0/2375068886 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494019070 con 0x7f849c13e1b0 2024-04-25T01:26:17.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.803+0000 7f84997fa640 1 -- 172.21.15.112:0/2375068886 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f849400caa0 con 0x7f849c13e1b0 2024-04-25T01:26:17.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.803+0000 7f84a2d4f640 1 -- 172.21.15.112:0/2375068886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f849c1eed40 con 0x7f849c13e1b0 2024-04-25T01:26:17.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.803+0000 7f84997fa640 1 -- 172.21.15.112:0/2375068886 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494014440 con 0x7f849c13e1b0 2024-04-25T01:26:17.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.803+0000 7f84a2d4f640 1 -- 172.21.15.112:0/2375068886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f849c1ef300 con 0x7f849c13e1b0 2024-04-25T01:26:17.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.804+0000 7f84a2d4f640 1 -- 172.21.15.112:0/2375068886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f849c04f500 con 0x7f849c13e1b0 2024-04-25T01:26:17.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.804+0000 7f84997fa640 1 -- 172.21.15.112:0/2375068886 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f849401c070 con 0x7f849c13e1b0 2024-04-25T01:26:17.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.805+0000 7f84997fa640 1 --2- 172.21.15.112:0/2375068886 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f848006be60 0x7f848006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:17.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.805+0000 7f84a0aeb640 1 --2- 172.21.15.112:0/2375068886 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f848006be60 0x7f848006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:17.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.805+0000 7f84997fa640 1 -- 172.21.15.112:0/2375068886 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f849408ad40 con 0x7f849c13e1b0 2024-04-25T01:26:17.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.805+0000 7f84a0aeb640 1 --2- 172.21.15.112:0/2375068886 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f848006be60 0x7f848006e320 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8490034000 tx=0x7f849002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:17.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.809+0000 7f84997fa640 1 -- 172.21.15.112:0/2375068886 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8494058d80 con 0x7f849c13e1b0 2024-04-25T01:26:17.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.982+0000 7f84a2d4f640 1 -- 172.21.15.112:0/2375068886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) -- 0x7f849c156960 con 0x7f849c13e1b0 2024-04-25T01:26:17.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.983+0000 7f84997fa640 1 -- 172.21.15.112:0/2375068886 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7f849405d080 con 0x7f849c13e1b0 2024-04-25T01:26:17.962 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T01:26:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.984+0000 7f84a2d4f640 1 -- 172.21.15.112:0/2375068886 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f848006be60 msgr2=0x7f848006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.984+0000 7f84a2d4f640 1 --2- 172.21.15.112:0/2375068886 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f848006be60 0x7f848006e320 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8490034000 tx=0x7f849002e000 comp rx=0 tx=0).stop 2024-04-25T01:26:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.984+0000 7f84a2d4f640 1 -- 172.21.15.112:0/2375068886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f849c13e1b0 msgr2=0x7f849c1580c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.984+0000 7f84a2d4f640 1 --2- 172.21.15.112:0/2375068886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f849c13e1b0 0x7f849c1580c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f849400b990 tx=0x7f849400bd90 comp rx=0 tx=0).stop 2024-04-25T01:26:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.985+0000 7f84a2d4f640 1 -- 172.21.15.112:0/2375068886 shutdown_connections 2024-04-25T01:26:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.985+0000 7f84a2d4f640 1 --2- 172.21.15.112:0/2375068886 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f848006be60 0x7f848006e320 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.985+0000 7f84a2d4f640 1 --2- 172.21.15.112:0/2375068886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f849c13e1b0 0x7f849c1580c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.985+0000 7f84a2d4f640 1 --2- 172.21.15.112:0/2375068886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f849c06ebb0 0x7f849c154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.985+0000 7f84a2d4f640 1 --2- 172.21.15.112:0/2375068886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f849c06cf50 0x7f849c0c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.985+0000 7f84a2d4f640 1 -- 172.21.15.112:0/2375068886 >> 172.21.15.112:0/2375068886 conn(0x7f849c0c3a10 msgr2=0x7f849c072170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.985+0000 7f84a2d4f640 1 -- 172.21.15.112:0/2375068886 shutdown_connections 2024-04-25T01:26:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:17.985+0000 7f84a2d4f640 1 -- 172.21.15.112:0/2375068886 wait complete. 2024-04-25T01:26:17.975 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-04-25T01:26:18.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.094+0000 7fa42feb3640 1 Processor -- start 2024-04-25T01:26:18.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.094+0000 7fa42feb3640 1 -- start start 2024-04-25T01:26:18.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.095+0000 7fa42feb3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa4280c9250 0x7fa4280c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.095+0000 7fa42feb3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa4280ca8a0 0x7fa42806c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.095+0000 7fa42feb3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4281534a0 con 0x7fa4280ca370 2024-04-25T01:26:18.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.095+0000 7fa42feb3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa428156990 con 0x7fa4280ca8a0 2024-04-25T01:26:18.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.095+0000 7fa42feb3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa428156500 con 0x7fa4280c9250 2024-04-25T01:26:18.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.095+0000 7fa42e450640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa4280ca8a0 0x7fa42806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.095+0000 7fa42e450640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa4280ca8a0 0x7fa42806c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46006/0 (socket says 172.21.15.112:46006) 2024-04-25T01:26:18.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.095+0000 7fa42e450640 1 -- 172.21.15.112:0/2484698199 learned_addr learned my addr 172.21.15.112:0/2484698199 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:18.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.095+0000 7fa42dc4f640 1 --2- 172.21.15.112:0/2484698199 >> v2:172.21.15.187:3300/0 conn(0x7fa4280c9250 0x7fa4280c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.096+0000 7fa42e450640 1 -- 172.21.15.112:0/2484698199 >> v2:172.21.15.187:3300/0 conn(0x7fa4280c9250 msgr2=0x7fa4280c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:18.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.096+0000 7fa42e450640 1 --2- 172.21.15.112:0/2484698199 >> v2:172.21.15.187:3300/0 conn(0x7fa4280c9250 0x7fa4280c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.096+0000 7fa42e450640 1 -- 172.21.15.112:0/2484698199 >> v1:172.21.15.187:6789/0 conn(0x7fa4280ca370 legacy=0x7fa42806bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:18.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.096+0000 7fa42e450640 1 -- 172.21.15.112:0/2484698199 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4280cae30 con 0x7fa4280ca8a0 2024-04-25T01:26:18.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.096+0000 7fa42e450640 1 --2- 172.21.15.112:0/2484698199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4280ca8a0 0x7fa42806c420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa428066f10 tx=0x7fa42002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f914c2519f73f356 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:18.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.096+0000 7fa42cc4d640 1 -- 172.21.15.112:0/2484698199 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420037070 con 0x7fa4280ca8a0 2024-04-25T01:26:18.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.096+0000 7fa42cc4d640 1 -- 172.21.15.112:0/2484698199 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa420032440 con 0x7fa4280ca8a0 2024-04-25T01:26:18.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.096+0000 7fa42cc4d640 1 -- 172.21.15.112:0/2484698199 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420032810 con 0x7fa4280ca8a0 2024-04-25T01:26:18.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.097+0000 7fa42feb3640 1 -- 172.21.15.112:0/2484698199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4280ca8a0 msgr2=0x7fa42806c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:18.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.097+0000 7fa42feb3640 1 --2- 172.21.15.112:0/2484698199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4280ca8a0 0x7fa42806c420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa428066f10 tx=0x7fa42002a1a0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.097+0000 7fa42feb3640 1 -- 172.21.15.112:0/2484698199 shutdown_connections 2024-04-25T01:26:18.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.097+0000 7fa42feb3640 1 --2- 172.21.15.112:0/2484698199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4280ca8a0 0x7fa42806c420 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.097+0000 7fa42feb3640 1 --2- 172.21.15.112:0/2484698199 >> v2:172.21.15.187:3300/0 conn(0x7fa4280c9250 0x7fa4280c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.097+0000 7fa42feb3640 1 -- 172.21.15.112:0/2484698199 >> 172.21.15.112:0/2484698199 conn(0x7fa4280cb310 msgr2=0x7fa4280cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:18.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.097+0000 7fa42feb3640 1 -- 172.21.15.112:0/2484698199 shutdown_connections 2024-04-25T01:26:18.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.097+0000 7fa42feb3640 1 -- 172.21.15.112:0/2484698199 wait complete. 2024-04-25T01:26:18.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.098+0000 7fa42feb3640 1 Processor -- start 2024-04-25T01:26:18.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.098+0000 7fa42feb3640 1 -- start start 2024-04-25T01:26:18.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa42feb3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4280c9250 0x7fa4281e0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa42feb3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4280ca370 0x7fa4280c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa42feb3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4280ca8a0 0x7fa4280c2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa42feb3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa428155eb0 con 0x7fa4280ca8a0 2024-04-25T01:26:18.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa42feb3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa428155bd0 con 0x7fa4280c9250 2024-04-25T01:26:18.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa42feb3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa428156190 con 0x7fa4280ca370 2024-04-25T01:26:18.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa42e450640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4280ca8a0 0x7fa4280c2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa42e450640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4280ca8a0 0x7fa4280c2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46020/0 (socket says 172.21.15.112:46020) 2024-04-25T01:26:18.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa42e450640 1 -- 172.21.15.112:0/1473003219 learned_addr learned my addr 172.21.15.112:0/1473003219 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:18.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa42dc4f640 1 --2- 172.21.15.112:0/1473003219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4280c9250 0x7fa4281e0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa42d44e640 1 --2- 172.21.15.112:0/1473003219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4280ca370 0x7fa4280c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa42e450640 1 -- 172.21.15.112:0/1473003219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4280ca370 msgr2=0x7fa4280c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:18.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa42e450640 1 --2- 172.21.15.112:0/1473003219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4280ca370 0x7fa4280c4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa42e450640 1 -- 172.21.15.112:0/1473003219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4280c9250 msgr2=0x7fa4281e0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa42e450640 1 --2- 172.21.15.112:0/1473003219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4280c9250 0x7fa4281e0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa42e450640 1 -- 172.21.15.112:0/1473003219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4280c44b0 con 0x7fa4280ca8a0 2024-04-25T01:26:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa42e450640 1 --2- 172.21.15.112:0/1473003219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4280ca8a0 0x7fa4280c2200 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa42002a1e0 tx=0x7fa4200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa41e7fc640 1 -- 172.21.15.112:0/1473003219 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420041070 con 0x7fa4280ca8a0 2024-04-25T01:26:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa41e7fc640 1 -- 172.21.15.112:0/1473003219 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa42003c950 con 0x7fa4280ca8a0 2024-04-25T01:26:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa42feb3640 1 -- 172.21.15.112:0/1473003219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4281e1a00 con 0x7fa4280ca8a0 2024-04-25T01:26:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.099+0000 7fa41e7fc640 1 -- 172.21.15.112:0/1473003219 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420037040 con 0x7fa4280ca8a0 2024-04-25T01:26:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.100+0000 7fa42feb3640 1 -- 172.21.15.112:0/1473003219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4281e0290 con 0x7fa4280ca8a0 2024-04-25T01:26:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.100+0000 7fa42feb3640 1 -- 172.21.15.112:0/1473003219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4280682a0 con 0x7fa4280ca8a0 2024-04-25T01:26:18.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.101+0000 7fa41e7fc640 1 -- 172.21.15.112:0/1473003219 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa420044070 con 0x7fa4280ca8a0 2024-04-25T01:26:18.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.101+0000 7fa41e7fc640 1 --2- 172.21.15.112:0/1473003219 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa40c06be30 0x7fa40c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.101+0000 7fa41e7fc640 1 -- 172.21.15.112:0/1473003219 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fa4200abb70 con 0x7fa4280ca8a0 2024-04-25T01:26:18.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.101+0000 7fa42dc4f640 1 --2- 172.21.15.112:0/1473003219 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa40c06be30 0x7fa40c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.102+0000 7fa42dc4f640 1 --2- 172.21.15.112:0/1473003219 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa40c06be30 0x7fa40c06e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa42814df20 tx=0x7fa4240077b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:18.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.105+0000 7fa41e7fc640 1 -- 172.21.15.112:0/1473003219 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa4280682a0 con 0x7fa4280ca8a0 2024-04-25T01:26:18.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.281+0000 7fa42feb3640 1 -- 172.21.15.112:0/1473003219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) -- 0x7fa42804f500 con 0x7fa4280ca8a0 2024-04-25T01:26:18.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.281+0000 7fa41e7fc640 1 -- 172.21.15.112:0/1473003219 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7fa420079d50 con 0x7fa4280ca8a0 2024-04-25T01:26:18.261 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T01:26:18.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.283+0000 7fa42feb3640 1 -- 172.21.15.112:0/1473003219 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa40c06be30 msgr2=0x7fa40c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:18.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.283+0000 7fa42feb3640 1 --2- 172.21.15.112:0/1473003219 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa40c06be30 0x7fa40c06e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa42814df20 tx=0x7fa4240077b0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.283+0000 7fa42feb3640 1 -- 172.21.15.112:0/1473003219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4280ca8a0 msgr2=0x7fa4280c2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:18.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.283+0000 7fa42feb3640 1 --2- 172.21.15.112:0/1473003219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4280ca8a0 0x7fa4280c2200 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa42002a1e0 tx=0x7fa4200046c0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.283+0000 7fa42feb3640 1 -- 172.21.15.112:0/1473003219 shutdown_connections 2024-04-25T01:26:18.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.283+0000 7fa42feb3640 1 --2- 172.21.15.112:0/1473003219 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa40c06be30 0x7fa40c06e2f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.283+0000 7fa42feb3640 1 --2- 172.21.15.112:0/1473003219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4280ca8a0 0x7fa4280c2200 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.283+0000 7fa42feb3640 1 --2- 172.21.15.112:0/1473003219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4280ca370 0x7fa4280c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.283+0000 7fa42feb3640 1 --2- 172.21.15.112:0/1473003219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4280c9250 0x7fa4281e0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.283+0000 7fa42feb3640 1 -- 172.21.15.112:0/1473003219 >> 172.21.15.112:0/1473003219 conn(0x7fa4280cb310 msgr2=0x7fa4281538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:18.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.283+0000 7fa42feb3640 1 -- 172.21.15.112:0/1473003219 shutdown_connections 2024-04-25T01:26:18.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.283+0000 7fa42feb3640 1 -- 172.21.15.112:0/1473003219 wait complete. 2024-04-25T01:26:18.273 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-04-25T01:26:18.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.392+0000 7f4900fc1640 1 Processor -- start 2024-04-25T01:26:18.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.392+0000 7f4900fc1640 1 -- start start 2024-04-25T01:26:18.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.393+0000 7f4900fc1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48fc14fbe0 0x7f48fc0c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.393+0000 7f4900fc1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f48fc0c2e60 0x7f48fc0726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.393+0000 7f4900fc1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48fc06f290 con 0x7f48fc06d460 2024-04-25T01:26:18.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.393+0000 7f4900fc1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f48fc14ffc0 con 0x7f48fc0c2e60 2024-04-25T01:26:18.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.393+0000 7f4900fc1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f48fc1533a0 con 0x7f48fc14fbe0 2024-04-25T01:26:18.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.393+0000 7f48fad9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f48fc0c2e60 0x7f48fc0726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.393+0000 7f48f9d9b640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f48fc06d460 0x7f48fc071fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53424/0 (socket says 172.21.15.112:53424) 2024-04-25T01:26:18.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.393+0000 7f48f9d9b640 1 -- 172.21.15.112:0/3038820736 learned_addr learned my addr 172.21.15.112:0/3038820736 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:18.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.394+0000 7f48fa59c640 1 --2- 172.21.15.112:0/3038820736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48fc14fbe0 0x7f48fc0c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.394+0000 7f48fad9d640 1 -- 172.21.15.112:0/3038820736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48fc14fbe0 msgr2=0x7f48fc0c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:18.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.394+0000 7f48fad9d640 1 --2- 172.21.15.112:0/3038820736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48fc14fbe0 0x7f48fc0c2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.394+0000 7f48fad9d640 1 -- 172.21.15.112:0/3038820736 >> v1:172.21.15.112:6789/0 conn(0x7f48fc06d460 legacy=0x7f48fc071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:18.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.394+0000 7f48fad9d640 1 -- 172.21.15.112:0/3038820736 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48fc06efb0 con 0x7f48fc0c2e60 2024-04-25T01:26:18.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.394+0000 7f48f959a640 1 -- 172.21.15.112:0/3038820736 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 449741164 0 0) 0x7f48fc06f290 con 0x7f48fc06d460 2024-04-25T01:26:18.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.394+0000 7f48fa59c640 1 --2- 172.21.15.112:0/3038820736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48fc14fbe0 0x7f48fc0c2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:18.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.394+0000 7f48fad9d640 1 --2- 172.21.15.112:0/3038820736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48fc0c2e60 0x7f48fc0726c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f48f40097a0 tx=0x7f48f402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13871db9c96feec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:18.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.394+0000 7f48f959a640 1 -- 172.21.15.112:0/3038820736 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4037070 con 0x7f48fc0c2e60 2024-04-25T01:26:18.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.394+0000 7f48f959a640 1 -- 172.21.15.112:0/3038820736 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48f402ae50 con 0x7f48fc0c2e60 2024-04-25T01:26:18.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.394+0000 7f48f959a640 1 -- 172.21.15.112:0/3038820736 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4032660 con 0x7f48fc0c2e60 2024-04-25T01:26:18.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.395+0000 7f4900fc1640 1 -- 172.21.15.112:0/3038820736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48fc0c2e60 msgr2=0x7f48fc0726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:18.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.395+0000 7f4900fc1640 1 --2- 172.21.15.112:0/3038820736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48fc0c2e60 0x7f48fc0726c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f48f40097a0 tx=0x7f48f402a100 comp rx=0 tx=0).stop 2024-04-25T01:26:18.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.395+0000 7f4900fc1640 1 -- 172.21.15.112:0/3038820736 shutdown_connections 2024-04-25T01:26:18.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.395+0000 7f4900fc1640 1 --2- 172.21.15.112:0/3038820736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48fc0c2e60 0x7f48fc0726c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.395+0000 7f4900fc1640 1 --2- 172.21.15.112:0/3038820736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48fc14fbe0 0x7f48fc0c2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.395+0000 7f4900fc1640 1 -- 172.21.15.112:0/3038820736 >> 172.21.15.112:0/3038820736 conn(0x7f48fc0c3a10 msgr2=0x7f48fc0c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:18.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.395+0000 7f4900fc1640 1 -- 172.21.15.112:0/3038820736 shutdown_connections 2024-04-25T01:26:18.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.395+0000 7f4900fc1640 1 -- 172.21.15.112:0/3038820736 wait complete. 2024-04-25T01:26:18.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.396+0000 7f4900fc1640 1 Processor -- start 2024-04-25T01:26:18.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.396+0000 7f4900fc1640 1 -- start start 2024-04-25T01:26:18.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.397+0000 7f4900fc1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48fc06d460 0x7f48fc146900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.397+0000 7f4900fc1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48fc0c2e60 0x7f48fc1eee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.397+0000 7f4900fc1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48fc14fbe0 0x7f48fc0c0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.397+0000 7f4900fc1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f48fc072c00 con 0x7f48fc14fbe0 2024-04-25T01:26:18.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.397+0000 7f4900fc1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f48fc153030 con 0x7f48fc06d460 2024-04-25T01:26:18.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.397+0000 7f4900fc1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f48fc0c3700 con 0x7f48fc0c2e60 2024-04-25T01:26:18.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.397+0000 7f48fad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48fc14fbe0 0x7f48fc0c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.397+0000 7f48fad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48fc14fbe0 0x7f48fc0c0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46030/0 (socket says 172.21.15.112:46030) 2024-04-25T01:26:18.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.397+0000 7f48fad9d640 1 -- 172.21.15.112:0/3784241541 learned_addr learned my addr 172.21.15.112:0/3784241541 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:18.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.397+0000 7f48fa59c640 1 --2- 172.21.15.112:0/3784241541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48fc06d460 0x7f48fc146900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.397+0000 7f48f9d9b640 1 --2- 172.21.15.112:0/3784241541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48fc0c2e60 0x7f48fc1eee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.397+0000 7f48fad9d640 1 -- 172.21.15.112:0/3784241541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48fc0c2e60 msgr2=0x7f48fc1eee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:18.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.397+0000 7f48fad9d640 1 --2- 172.21.15.112:0/3784241541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48fc0c2e60 0x7f48fc1eee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.397+0000 7f48fad9d640 1 -- 172.21.15.112:0/3784241541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48fc06d460 msgr2=0x7f48fc146900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:18.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.397+0000 7f48fad9d640 1 --2- 172.21.15.112:0/3784241541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48fc06d460 0x7f48fc146900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.397+0000 7f48fad9d640 1 -- 172.21.15.112:0/3784241541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48fc146e40 con 0x7f48fc14fbe0 2024-04-25T01:26:18.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.397+0000 7f48fad9d640 1 --2- 172.21.15.112:0/3784241541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48fc14fbe0 0x7f48fc0c0520 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f48f40097a0 tx=0x7f48f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:18.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.398+0000 7f48eaffd640 1 -- 172.21.15.112:0/3784241541 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4037040 con 0x7f48fc14fbe0 2024-04-25T01:26:18.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.398+0000 7f48eaffd640 1 -- 172.21.15.112:0/3784241541 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48f4004230 con 0x7f48fc14fbe0 2024-04-25T01:26:18.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.398+0000 7f4900fc1640 1 -- 172.21.15.112:0/3784241541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48fc1569f0 con 0x7f48fc14fbe0 2024-04-25T01:26:18.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.398+0000 7f48eaffd640 1 -- 172.21.15.112:0/3784241541 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4003610 con 0x7f48fc14fbe0 2024-04-25T01:26:18.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.398+0000 7f4900fc1640 1 -- 172.21.15.112:0/3784241541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48fc149340 con 0x7f48fc14fbe0 2024-04-25T01:26:18.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.399+0000 7f4900fc1640 1 -- 172.21.15.112:0/3784241541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48fc04f500 con 0x7f48fc14fbe0 2024-04-25T01:26:18.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.399+0000 7f48eaffd640 1 -- 172.21.15.112:0/3784241541 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f48f40037b0 con 0x7f48fc14fbe0 2024-04-25T01:26:18.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.399+0000 7f48eaffd640 1 --2- 172.21.15.112:0/3784241541 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f48dc06be60 0x7f48dc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.400+0000 7f48eaffd640 1 -- 172.21.15.112:0/3784241541 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f48f40ac090 con 0x7f48fc14fbe0 2024-04-25T01:26:18.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.400+0000 7f48fa59c640 1 --2- 172.21.15.112:0/3784241541 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f48dc06be60 0x7f48dc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.400+0000 7f48fa59c640 1 --2- 172.21.15.112:0/3784241541 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f48dc06be60 0x7f48dc06e320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0075b0 tx=0x7f48ec007540 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:18.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.404+0000 7f48eaffd640 1 -- 172.21.15.112:0/3784241541 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f48fc1569f0 con 0x7f48fc14fbe0 2024-04-25T01:26:18.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.581+0000 7f4900fc1640 1 -- 172.21.15.112:0/3784241541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) -- 0x7f48fc0c3470 con 0x7f48fc14fbe0 2024-04-25T01:26:18.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.582+0000 7f48eaffd640 1 -- 172.21.15.112:0/3784241541 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7f48f407a7a0 con 0x7f48fc14fbe0 2024-04-25T01:26:18.561 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T01:26:18.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.584+0000 7f4900fc1640 1 -- 172.21.15.112:0/3784241541 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f48dc06be60 msgr2=0x7f48dc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:18.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.584+0000 7f4900fc1640 1 --2- 172.21.15.112:0/3784241541 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f48dc06be60 0x7f48dc06e320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0075b0 tx=0x7f48ec007540 comp rx=0 tx=0).stop 2024-04-25T01:26:18.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.584+0000 7f4900fc1640 1 -- 172.21.15.112:0/3784241541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48fc14fbe0 msgr2=0x7f48fc0c0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:18.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.584+0000 7f4900fc1640 1 --2- 172.21.15.112:0/3784241541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48fc14fbe0 0x7f48fc0c0520 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f48f40097a0 tx=0x7f48f40046c0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.584+0000 7f4900fc1640 1 -- 172.21.15.112:0/3784241541 shutdown_connections 2024-04-25T01:26:18.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.584+0000 7f4900fc1640 1 --2- 172.21.15.112:0/3784241541 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f48dc06be60 0x7f48dc06e320 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.584+0000 7f4900fc1640 1 --2- 172.21.15.112:0/3784241541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48fc14fbe0 0x7f48fc0c0520 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.584+0000 7f4900fc1640 1 --2- 172.21.15.112:0/3784241541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48fc0c2e60 0x7f48fc1eee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.584+0000 7f4900fc1640 1 --2- 172.21.15.112:0/3784241541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48fc06d460 0x7f48fc146900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.584+0000 7f4900fc1640 1 -- 172.21.15.112:0/3784241541 >> 172.21.15.112:0/3784241541 conn(0x7f48fc0c3a10 msgr2=0x7f48fc0acab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:18.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.584+0000 7f4900fc1640 1 -- 172.21.15.112:0/3784241541 shutdown_connections 2024-04-25T01:26:18.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.584+0000 7f4900fc1640 1 -- 172.21.15.112:0/3784241541 wait complete. 2024-04-25T01:26:18.574 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:26:18.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.691+0000 7fe648509640 1 Processor -- start 2024-04-25T01:26:18.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.691+0000 7fe648509640 1 -- start start 2024-04-25T01:26:18.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.692+0000 7fe648509640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64014c3f0 0x7fe6400c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.692+0000 7fe648509640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe6400ac3b0 0x7fe640155f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.692+0000 7fe648509640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6401529b0 con 0x7fe6401524a0 2024-04-25T01:26:18.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.692+0000 7fe648509640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe640155a90 con 0x7fe6400ac3b0 2024-04-25T01:26:18.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.692+0000 7fe648509640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe640152c70 con 0x7fe64014c3f0 2024-04-25T01:26:18.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.692+0000 7fe646aa6640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe6401524a0 0x7fe6400c3b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53436/0 (socket says 172.21.15.112:53436) 2024-04-25T01:26:18.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.692+0000 7fe646aa6640 1 -- 172.21.15.112:0/3908063440 learned_addr learned my addr 172.21.15.112:0/3908063440 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:18.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.692+0000 7fe645aa4640 1 --2- 172.21.15.112:0/3908063440 >> v2:172.21.15.187:3300/0 conn(0x7fe6400ac3b0 0x7fe640155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.692+0000 7fe6462a5640 1 --2- 172.21.15.112:0/3908063440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64014c3f0 0x7fe6400c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.692+0000 7fe6452a3640 1 -- 172.21.15.112:0/3908063440 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1738804923 0 0) 0x7fe6401529b0 con 0x7fe6401524a0 2024-04-25T01:26:18.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.692+0000 7fe6452a3640 1 -- 172.21.15.112:0/3908063440 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe62c003520 con 0x7fe6401524a0 2024-04-25T01:26:18.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe6452a3640 1 -- 172.21.15.112:0/3908063440 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1610175588 0 0) 0x7fe62c003520 con 0x7fe6401524a0 2024-04-25T01:26:18.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe6452a3640 1 -- 172.21.15.112:0/3908063440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64014c3f0 msgr2=0x7fe6400c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:18.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe6452a3640 1 --2- 172.21.15.112:0/3908063440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64014c3f0 0x7fe6400c2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe6452a3640 1 -- 172.21.15.112:0/3908063440 >> v2:172.21.15.187:3300/0 conn(0x7fe6400ac3b0 msgr2=0x7fe640155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:18.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe6452a3640 1 --2- 172.21.15.112:0/3908063440 >> v2:172.21.15.187:3300/0 conn(0x7fe6400ac3b0 0x7fe640155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe6452a3640 1 -- 172.21.15.112:0/3908063440 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe640152f50 con 0x7fe6401524a0 2024-04-25T01:26:18.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe6452a3640 1 -- 172.21.15.112:0/3908063440 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fe638001670 con 0x7fe6401524a0 2024-04-25T01:26:18.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe6462a5640 1 --2- 172.21.15.112:0/3908063440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64014c3f0 0x7fe6400c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:26:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe6452a3640 1 -- 172.21.15.112:0/3908063440 >> v1:172.21.15.112:6789/0 conn(0x7fe6401524a0 legacy=0x7fe6400c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe6452a3640 1 --2- 172.21.15.112:0/3908063440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe62c0039b0 0x7fe62c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe6452a3640 1 --2- 172.21.15.112:0/3908063440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe62c004800 0x7fe62c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe6452a3640 1 --2- 172.21.15.112:0/3908063440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64014c3f0 0x7fe62c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe6452a3640 1 -- 172.21.15.112:0/3908063440 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe640155a90 con 0x7fe62c004800 2024-04-25T01:26:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe6452a3640 1 -- 172.21.15.112:0/3908063440 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe640152c70 con 0x7fe62c0039b0 2024-04-25T01:26:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe6452a3640 1 -- 172.21.15.112:0/3908063440 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe62c00b9c0 con 0x7fe64014c3f0 2024-04-25T01:26:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe645aa4640 1 --2- 172.21.15.112:0/3908063440 >> v2:172.21.15.187:3300/0 conn(0x7fe6400ac3b0 0x7fe640155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:26:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe645aa4640 1 --2- 172.21.15.112:0/3908063440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe62c004800 0x7fe62c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe6462a5640 1 --2- 172.21.15.112:0/3908063440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe62c0039b0 0x7fe62c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe646aa6640 1 --2- 172.21.15.112:0/3908063440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64014c3f0 0x7fe62c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe645aa4640 1 -- 172.21.15.112:0/3908063440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64014c3f0 msgr2=0x7fe62c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe645aa4640 1 --2- 172.21.15.112:0/3908063440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64014c3f0 0x7fe62c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe645aa4640 1 -- 172.21.15.112:0/3908063440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe62c0039b0 msgr2=0x7fe62c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:18.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe645aa4640 1 --2- 172.21.15.112:0/3908063440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe62c0039b0 0x7fe62c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.693+0000 7fe645aa4640 1 -- 172.21.15.112:0/3908063440 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62c003520 con 0x7fe62c004800 2024-04-25T01:26:18.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.694+0000 7fe645aa4640 1 --2- 172.21.15.112:0/3908063440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe62c004800 0x7fe62c004c80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe630009c70 tx=0x7fe62c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee16af186e383c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:18.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.694+0000 7fe6452a3640 1 -- 172.21.15.112:0/3908063440 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630009840 con 0x7fe62c004800 2024-04-25T01:26:18.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.694+0000 7fe6452a3640 1 -- 172.21.15.112:0/3908063440 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe630004020 con 0x7fe62c004800 2024-04-25T01:26:18.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.694+0000 7fe6452a3640 1 -- 172.21.15.112:0/3908063440 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6300043d0 con 0x7fe62c004800 2024-04-25T01:26:18.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.694+0000 7fe648509640 1 -- 172.21.15.112:0/3908063440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe62c004800 msgr2=0x7fe62c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:18.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.694+0000 7fe648509640 1 --2- 172.21.15.112:0/3908063440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe62c004800 0x7fe62c004c80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe630009c70 tx=0x7fe62c005830 comp rx=0 tx=0).stop 2024-04-25T01:26:18.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.694+0000 7fe646aa6640 1 -- 172.21.15.112:0/3908063440 reap_dead start 2024-04-25T01:26:18.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.695+0000 7fe648509640 1 -- 172.21.15.112:0/3908063440 shutdown_connections 2024-04-25T01:26:18.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.695+0000 7fe648509640 1 -- 172.21.15.112:0/3908063440 >> 172.21.15.112:0/3908063440 conn(0x7fe64006b160 msgr2=0x7fe64006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:18.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.695+0000 7fe648509640 1 -- 172.21.15.112:0/3908063440 shutdown_connections 2024-04-25T01:26:18.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.695+0000 7fe648509640 1 -- 172.21.15.112:0/3908063440 wait complete. 2024-04-25T01:26:18.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.696+0000 7fe648509640 1 Processor -- start 2024-04-25T01:26:18.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.696+0000 7fe648509640 1 -- start start 2024-04-25T01:26:18.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.696+0000 7fe648509640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe62c004800 0x7fe640151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.696+0000 7fe648509640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6401e9cb0 0x7fe6400c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.696+0000 7fe648509640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe640143430 0x7fe6401438b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.696+0000 7fe648509640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe64014c7d0 con 0x7fe640143430 2024-04-25T01:26:18.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.696+0000 7fe648509640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe640070d60 con 0x7fe62c004800 2024-04-25T01:26:18.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.696+0000 7fe648509640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe64014c950 con 0x7fe6401e9cb0 2024-04-25T01:26:18.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.696+0000 7fe646aa6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe640143430 0x7fe6401438b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.696+0000 7fe646aa6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe640143430 0x7fe6401438b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46042/0 (socket says 172.21.15.112:46042) 2024-04-25T01:26:18.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.696+0000 7fe646aa6640 1 -- 172.21.15.112:0/3688106926 learned_addr learned my addr 172.21.15.112:0/3688106926 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:18.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.696+0000 7fe645aa4640 1 --2- 172.21.15.112:0/3688106926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6401e9cb0 0x7fe6400c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.697+0000 7fe6462a5640 1 --2- 172.21.15.112:0/3688106926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe62c004800 0x7fe640151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.697+0000 7fe646aa6640 1 -- 172.21.15.112:0/3688106926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6401e9cb0 msgr2=0x7fe6400c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:18.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.697+0000 7fe646aa6640 1 --2- 172.21.15.112:0/3688106926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6401e9cb0 0x7fe6400c73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.697+0000 7fe646aa6640 1 -- 172.21.15.112:0/3688106926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe62c004800 msgr2=0x7fe640151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:18.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.697+0000 7fe646aa6640 1 --2- 172.21.15.112:0/3688106926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe62c004800 0x7fe640151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.697+0000 7fe646aa6640 1 -- 172.21.15.112:0/3688106926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6400c6b10 con 0x7fe640143430 2024-04-25T01:26:18.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.697+0000 7fe645aa4640 1 --2- 172.21.15.112:0/3688106926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6401e9cb0 0x7fe6400c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:18.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.697+0000 7fe646aa6640 1 --2- 172.21.15.112:0/3688106926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe640143430 0x7fe6401438b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe6400c1380 tx=0x7fe638007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:18.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.697+0000 7fe636ffd640 1 -- 172.21.15.112:0/3688106926 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63800f620 con 0x7fe640143430 2024-04-25T01:26:18.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.697+0000 7fe636ffd640 1 -- 172.21.15.112:0/3688106926 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe638018070 con 0x7fe640143430 2024-04-25T01:26:18.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.697+0000 7fe648509640 1 -- 172.21.15.112:0/3688106926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6401492d0 con 0x7fe640143430 2024-04-25T01:26:18.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.697+0000 7fe636ffd640 1 -- 172.21.15.112:0/3688106926 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63800fa20 con 0x7fe640143430 2024-04-25T01:26:18.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.697+0000 7fe648509640 1 -- 172.21.15.112:0/3688106926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe640149850 con 0x7fe640143430 2024-04-25T01:26:18.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.698+0000 7fe648509640 1 -- 172.21.15.112:0/3688106926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6400682a0 con 0x7fe640143430 2024-04-25T01:26:18.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.698+0000 7fe636ffd640 1 -- 172.21.15.112:0/3688106926 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe63800fbc0 con 0x7fe640143430 2024-04-25T01:26:18.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.699+0000 7fe636ffd640 1 --2- 172.21.15.112:0/3688106926 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe62406be30 0x7fe62406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:18.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.699+0000 7fe636ffd640 1 -- 172.21.15.112:0/3688106926 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fe63808af80 con 0x7fe640143430 2024-04-25T01:26:18.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.699+0000 7fe6462a5640 1 --2- 172.21.15.112:0/3688106926 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe62406be30 0x7fe62406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:18.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.699+0000 7fe6462a5640 1 --2- 172.21.15.112:0/3688106926 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe62406be30 0x7fe62406e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009bd0 tx=0x7fe63c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:18.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.703+0000 7fe636ffd640 1 -- 172.21.15.112:0/3688106926 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe638059600 con 0x7fe640143430 2024-04-25T01:26:18.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.911+0000 7fe648509640 1 -- 172.21.15.112:0/3688106926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6400aaf90 con 0x7fe640143430 2024-04-25T01:26:18.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.911+0000 7fe636ffd640 1 -- 172.21.15.112:0/3688106926 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+6491 (secure 0 0 0) 0x7fe638058f00 con 0x7fe640143430 2024-04-25T01:26:18.891 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:18.891 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/4047219598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4047219598},{"type":"v1","addr":"172.21.15.187:6834","nonce":4047219598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2204640907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2204640907},{"type":"v1","addr":"172.21.15.187:6835","nonce":2204640907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:25:52.687260+0000","modified":"2024-04-25T01:25:55.702719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/2115984531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2115984531},{"type":"v1","addr":"172.21.15.112:6837","nonce":2115984531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1},{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:26:01.740521+0000","modified":"2024-04-25T01:26:04.746955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":8,"state":"up:active","state_seq":6,"addr":"172.21.15.112:6835/278686477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":278686477},{"type":"v1","addr":"172.21.15.112:6835","nonce":278686477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":2}]} 2024-04-25T01:26:18.891 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 11 2024-04-25T01:26:18.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.913+0000 7fe648509640 1 -- 172.21.15.112:0/3688106926 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe62406be30 msgr2=0x7fe62406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:18.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.913+0000 7fe648509640 1 --2- 172.21.15.112:0/3688106926 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe62406be30 0x7fe62406e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009bd0 tx=0x7fe63c00b040 comp rx=0 tx=0).stop 2024-04-25T01:26:18.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.913+0000 7fe648509640 1 -- 172.21.15.112:0/3688106926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe640143430 msgr2=0x7fe6401438b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:18.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.913+0000 7fe648509640 1 --2- 172.21.15.112:0/3688106926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe640143430 0x7fe6401438b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe6400c1380 tx=0x7fe638007430 comp rx=0 tx=0).stop 2024-04-25T01:26:18.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.913+0000 7fe648509640 1 -- 172.21.15.112:0/3688106926 shutdown_connections 2024-04-25T01:26:18.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.913+0000 7fe648509640 1 --2- 172.21.15.112:0/3688106926 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe62406be30 0x7fe62406e2f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.913+0000 7fe648509640 1 --2- 172.21.15.112:0/3688106926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe640143430 0x7fe6401438b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.913+0000 7fe648509640 1 --2- 172.21.15.112:0/3688106926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6401e9cb0 0x7fe6400c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.913+0000 7fe648509640 1 --2- 172.21.15.112:0/3688106926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe62c004800 0x7fe640151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:18.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.913+0000 7fe648509640 1 -- 172.21.15.112:0/3688106926 >> 172.21.15.112:0/3688106926 conn(0x7fe64006b160 msgr2=0x7fe6400c27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:18.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.913+0000 7fe648509640 1 -- 172.21.15.112:0/3688106926 shutdown_connections 2024-04-25T01:26:18.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:18.913+0000 7fe648509640 1 -- 172.21.15.112:0/3688106926 wait complete. 2024-04-25T01:26:18.904 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:26:19.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.021+0000 7f0f2cf5a640 1 Processor -- start 2024-04-25T01:26:19.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.021+0000 7f0f2cf5a640 1 -- start start 2024-04-25T01:26:19.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.022+0000 7f0f2cf5a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0f2814c3f0 0x7f0f280c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:19.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.022+0000 7f0f2cf5a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0f280ac3b0 0x7f0f28155f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:19.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.022+0000 7f0f2cf5a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f281529b0 con 0x7f0f281524a0 2024-04-25T01:26:19.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.022+0000 7f0f2cf5a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f0f28155a90 con 0x7f0f280ac3b0 2024-04-25T01:26:19.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.022+0000 7f0f2cf5a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0f28152c70 con 0x7f0f2814c3f0 2024-04-25T01:26:19.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.022+0000 7f0f1ffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0f280ac3b0 0x7f0f28155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:19.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.022+0000 7f0f1ffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0f280ac3b0 0x7f0f28155f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46054/0 (socket says 172.21.15.112:46054) 2024-04-25T01:26:19.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.022+0000 7f0f1ffff640 1 -- 172.21.15.112:0/3218253748 learned_addr learned my addr 172.21.15.112:0/3218253748 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:19.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.022+0000 7f0f2659c640 1 --2- 172.21.15.112:0/3218253748 >> v2:172.21.15.187:3300/0 conn(0x7f0f2814c3f0 0x7f0f280c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:19.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.022+0000 7f0f1ffff640 1 -- 172.21.15.112:0/3218253748 >> v2:172.21.15.187:3300/0 conn(0x7f0f2814c3f0 msgr2=0x7f0f280c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:19.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.022+0000 7f0f1ffff640 1 --2- 172.21.15.112:0/3218253748 >> v2:172.21.15.187:3300/0 conn(0x7f0f2814c3f0 0x7f0f280c2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.022+0000 7f0f1ffff640 1 -- 172.21.15.112:0/3218253748 >> v1:172.21.15.187:6789/0 conn(0x7f0f281524a0 legacy=0x7f0f280c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:19.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.022+0000 7f0f1ffff640 1 -- 172.21.15.112:0/3218253748 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f28152f50 con 0x7f0f280ac3b0 2024-04-25T01:26:19.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.023+0000 7f0f1ffff640 1 --2- 172.21.15.112:0/3218253748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f280ac3b0 0x7f0f28155f20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0f28155c10 tx=0x7f0f1002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9548ccb067cbe131 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:19.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.023+0000 7f0f25d9b640 1 -- 172.21.15.112:0/3218253748 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f10037070 con 0x7f0f280ac3b0 2024-04-25T01:26:19.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.023+0000 7f0f25d9b640 1 -- 172.21.15.112:0/3218253748 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f10032440 con 0x7f0f280ac3b0 2024-04-25T01:26:19.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.023+0000 7f0f25d9b640 1 -- 172.21.15.112:0/3218253748 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f10032810 con 0x7f0f280ac3b0 2024-04-25T01:26:19.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.023+0000 7f0f2cf5a640 1 -- 172.21.15.112:0/3218253748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f280ac3b0 msgr2=0x7f0f28155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:19.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.023+0000 7f0f2cf5a640 1 --2- 172.21.15.112:0/3218253748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f280ac3b0 0x7f0f28155f20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0f28155c10 tx=0x7f0f1002a1e0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.023+0000 7f0f2cf5a640 1 -- 172.21.15.112:0/3218253748 shutdown_connections 2024-04-25T01:26:19.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.023+0000 7f0f2cf5a640 1 --2- 172.21.15.112:0/3218253748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f280ac3b0 0x7f0f28155f20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.023+0000 7f0f2cf5a640 1 --2- 172.21.15.112:0/3218253748 >> v2:172.21.15.187:3300/0 conn(0x7f0f2814c3f0 0x7f0f280c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.023+0000 7f0f2cf5a640 1 -- 172.21.15.112:0/3218253748 >> 172.21.15.112:0/3218253748 conn(0x7f0f2806b160 msgr2=0x7f0f2806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:19.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.023+0000 7f0f2cf5a640 1 -- 172.21.15.112:0/3218253748 shutdown_connections 2024-04-25T01:26:19.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.023+0000 7f0f2cf5a640 1 -- 172.21.15.112:0/3218253748 wait complete. 2024-04-25T01:26:19.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.025+0000 7f0f2cf5a640 1 Processor -- start 2024-04-25T01:26:19.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.025+0000 7f0f2cf5a640 1 -- start start 2024-04-25T01:26:19.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.025+0000 7f0f2cf5a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f280ac3b0 0x7f0f2806cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:19.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.025+0000 7f0f2cf5a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f2814c3f0 0x7f0f28142bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:19.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.025+0000 7f0f2cf5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f281524a0 0x7f0f281eb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:19.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.025+0000 7f0f2cf5a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2814c7d0 con 0x7f0f281524a0 2024-04-25T01:26:19.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.025+0000 7f0f2cf5a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0f28070d60 con 0x7f0f280ac3b0 2024-04-25T01:26:19.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.025+0000 7f0f2cf5a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2814c950 con 0x7f0f2814c3f0 2024-04-25T01:26:19.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.025+0000 7f0f26d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f281524a0 0x7f0f281eb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:19.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.025+0000 7f0f26d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f281524a0 0x7f0f281eb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46070/0 (socket says 172.21.15.112:46070) 2024-04-25T01:26:19.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.025+0000 7f0f26d9d640 1 -- 172.21.15.112:0/1876204957 learned_addr learned my addr 172.21.15.112:0/1876204957 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:19.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.025+0000 7f0f2659c640 1 --2- 172.21.15.112:0/1876204957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f280ac3b0 0x7f0f2806cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:19.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.025+0000 7f0f1ffff640 1 --2- 172.21.15.112:0/1876204957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f2814c3f0 0x7f0f28142bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:19.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.026+0000 7f0f26d9d640 1 -- 172.21.15.112:0/1876204957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f2814c3f0 msgr2=0x7f0f28142bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:19.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.026+0000 7f0f26d9d640 1 --2- 172.21.15.112:0/1876204957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f2814c3f0 0x7f0f28142bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.026+0000 7f0f26d9d640 1 -- 172.21.15.112:0/1876204957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f280ac3b0 msgr2=0x7f0f2806cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:19.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.026+0000 7f0f26d9d640 1 --2- 172.21.15.112:0/1876204957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f280ac3b0 0x7f0f2806cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.026+0000 7f0f26d9d640 1 -- 172.21.15.112:0/1876204957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2806db40 con 0x7f0f281524a0 2024-04-25T01:26:19.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.026+0000 7f0f26d9d640 1 --2- 172.21.15.112:0/1876204957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f281524a0 0x7f0f281eb190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000c6e0 tx=0x7f0f2000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:19.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.026+0000 7f0f1effd640 1 -- 172.21.15.112:0/1876204957 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20003b70 con 0x7f0f281524a0 2024-04-25T01:26:19.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.026+0000 7f0f1effd640 1 -- 172.21.15.112:0/1876204957 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f20003d30 con 0x7f0f281524a0 2024-04-25T01:26:19.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.026+0000 7f0f2cf5a640 1 -- 172.21.15.112:0/1876204957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f281eef90 con 0x7f0f281524a0 2024-04-25T01:26:19.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.026+0000 7f0f1effd640 1 -- 172.21.15.112:0/1876204957 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20005310 con 0x7f0f281524a0 2024-04-25T01:26:19.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.026+0000 7f0f2cf5a640 1 -- 172.21.15.112:0/1876204957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f281468e0 con 0x7f0f281524a0 2024-04-25T01:26:19.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.027+0000 7f0f2cf5a640 1 -- 172.21.15.112:0/1876204957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f280682a0 con 0x7f0f281524a0 2024-04-25T01:26:19.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.027+0000 7f0f1effd640 1 -- 172.21.15.112:0/1876204957 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f200054b0 con 0x7f0f281524a0 2024-04-25T01:26:19.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.028+0000 7f0f1effd640 1 --2- 172.21.15.112:0/1876204957 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0f0806be30 0x7f0f0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:19.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.028+0000 7f0f2659c640 1 --2- 172.21.15.112:0/1876204957 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0f0806be30 0x7f0f0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:19.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.028+0000 7f0f1effd640 1 -- 172.21.15.112:0/1876204957 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f0f2008af40 con 0x7f0f281524a0 2024-04-25T01:26:19.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.028+0000 7f0f2659c640 1 --2- 172.21.15.112:0/1876204957 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0f0806be30 0x7f0f0806e2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0f18006fd0 tx=0x7f0f18008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:19.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.032+0000 7f0f1effd640 1 -- 172.21.15.112:0/1876204957 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0f20059650 con 0x7f0f281524a0 2024-04-25T01:26:19.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.207+0000 7f0f2cf5a640 1 -- 172.21.15.112:0/1876204957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f280aaf90 con 0x7f0f281524a0 2024-04-25T01:26:19.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.209+0000 7f0f1effd640 1 -- 172.21.15.112:0/1876204957 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20367 (secure 0 0 0) 0x7f0f20058f50 con 0x7f0f281524a0 2024-04-25T01:26:19.189 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:19.189 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:26:03.405271+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-25T01:25:59.478275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-25T01:26:00.719358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:26:19.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.211+0000 7f0f2cf5a640 1 -- 172.21.15.112:0/1876204957 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0f0806be30 msgr2=0x7f0f0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:19.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.211+0000 7f0f2cf5a640 1 --2- 172.21.15.112:0/1876204957 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0f0806be30 0x7f0f0806e2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0f18006fd0 tx=0x7f0f18008040 comp rx=0 tx=0).stop 2024-04-25T01:26:19.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.211+0000 7f0f2cf5a640 1 -- 172.21.15.112:0/1876204957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f281524a0 msgr2=0x7f0f281eb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:19.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.211+0000 7f0f2cf5a640 1 --2- 172.21.15.112:0/1876204957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f281524a0 0x7f0f281eb190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000c6e0 tx=0x7f0f2000cba0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.211+0000 7f0f2cf5a640 1 -- 172.21.15.112:0/1876204957 shutdown_connections 2024-04-25T01:26:19.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.211+0000 7f0f2cf5a640 1 --2- 172.21.15.112:0/1876204957 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0f0806be30 0x7f0f0806e2f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.211+0000 7f0f2cf5a640 1 --2- 172.21.15.112:0/1876204957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f281524a0 0x7f0f281eb190 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.211+0000 7f0f2cf5a640 1 --2- 172.21.15.112:0/1876204957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f2814c3f0 0x7f0f28142bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.211+0000 7f0f2cf5a640 1 --2- 172.21.15.112:0/1876204957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f280ac3b0 0x7f0f2806cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.211+0000 7f0f2cf5a640 1 -- 172.21.15.112:0/1876204957 >> 172.21.15.112:0/1876204957 conn(0x7f0f2806b160 msgr2=0x7f0f280c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:19.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.211+0000 7f0f2cf5a640 1 -- 172.21.15.112:0/1876204957 shutdown_connections 2024-04-25T01:26:19.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.211+0000 7f0f2cf5a640 1 -- 172.21.15.112:0/1876204957 wait complete. 2024-04-25T01:26:19.202 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:26:19.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.322+0000 7fa479e31640 1 Processor -- start 2024-04-25T01:26:19.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.322+0000 7fa479e31640 1 -- start start 2024-04-25T01:26:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.323+0000 7fa479e31640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa47414c3f0 0x7fa4740c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.323+0000 7fa479e31640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa4740ac3b0 0x7fa474155f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.323+0000 7fa479e31640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4741529b0 con 0x7fa4741524a0 2024-04-25T01:26:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.323+0000 7fa479e31640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa474155a90 con 0x7fa4740ac3b0 2024-04-25T01:26:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.323+0000 7fa479e31640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa474152c70 con 0x7fa47414c3f0 2024-04-25T01:26:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.323+0000 7fa472ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa4740ac3b0 0x7fa474155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.323+0000 7fa472ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa4740ac3b0 0x7fa474155f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46074/0 (socket says 172.21.15.112:46074) 2024-04-25T01:26:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.323+0000 7fa472ffd640 1 -- 172.21.15.112:0/1776131491 learned_addr learned my addr 172.21.15.112:0/1776131491 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.323+0000 7fa4737fe640 1 --2- 172.21.15.112:0/1776131491 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa47414c3f0 0x7fa4740c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.323+0000 7fa472ffd640 1 -- 172.21.15.112:0/1776131491 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa47414c3f0 msgr2=0x7fa4740c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.323+0000 7fa472ffd640 1 --2- 172.21.15.112:0/1776131491 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa47414c3f0 0x7fa4740c2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.323+0000 7fa472ffd640 1 -- 172.21.15.112:0/1776131491 >> v1:172.21.15.187:6789/0 conn(0x7fa4741524a0 legacy=0x7fa4740c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:19.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.323+0000 7fa472ffd640 1 -- 172.21.15.112:0/1776131491 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa474152f50 con 0x7fa4740ac3b0 2024-04-25T01:26:19.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.324+0000 7fa472ffd640 1 --2- 172.21.15.112:0/1776131491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4740ac3b0 0x7fa474155f20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa468009870 tx=0x7fa46802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299de2b652a75898 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:19.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.324+0000 7fa4727fc640 1 -- 172.21.15.112:0/1776131491 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468037070 con 0x7fa4740ac3b0 2024-04-25T01:26:19.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.324+0000 7fa4727fc640 1 -- 172.21.15.112:0/1776131491 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa468032440 con 0x7fa4740ac3b0 2024-04-25T01:26:19.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.324+0000 7fa4727fc640 1 -- 172.21.15.112:0/1776131491 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468032810 con 0x7fa4740ac3b0 2024-04-25T01:26:19.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.325+0000 7fa479e31640 1 -- 172.21.15.112:0/1776131491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4740ac3b0 msgr2=0x7fa474155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:19.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.325+0000 7fa479e31640 1 --2- 172.21.15.112:0/1776131491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4740ac3b0 0x7fa474155f20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa468009870 tx=0x7fa46802a1c0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.325+0000 7fa479e31640 1 -- 172.21.15.112:0/1776131491 shutdown_connections 2024-04-25T01:26:19.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.325+0000 7fa479e31640 1 --2- 172.21.15.112:0/1776131491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4740ac3b0 0x7fa474155f20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.325+0000 7fa479e31640 1 --2- 172.21.15.112:0/1776131491 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa47414c3f0 0x7fa4740c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.325+0000 7fa479e31640 1 -- 172.21.15.112:0/1776131491 >> 172.21.15.112:0/1776131491 conn(0x7fa47406b160 msgr2=0x7fa47406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:19.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.325+0000 7fa479e31640 1 -- 172.21.15.112:0/1776131491 shutdown_connections 2024-04-25T01:26:19.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.325+0000 7fa479e31640 1 -- 172.21.15.112:0/1776131491 wait complete. 2024-04-25T01:26:19.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.326+0000 7fa479e31640 1 Processor -- start 2024-04-25T01:26:19.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.326+0000 7fa479e31640 1 -- start start 2024-04-25T01:26:19.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.326+0000 7fa479e31640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4740ac3b0 0x7fa4740c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:19.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa479e31640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa47414c3f0 0x7fa4740c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:19.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa479e31640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4741524a0 0x7fa474147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:19.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa479e31640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa47414c7d0 con 0x7fa47414c3f0 2024-04-25T01:26:19.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa479e31640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa474070d60 con 0x7fa4740ac3b0 2024-04-25T01:26:19.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa479e31640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa47414c950 con 0x7fa4741524a0 2024-04-25T01:26:19.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa472ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa47414c3f0 0x7fa4740c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:19.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa472ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa47414c3f0 0x7fa4740c6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46084/0 (socket says 172.21.15.112:46084) 2024-04-25T01:26:19.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa472ffd640 1 -- 172.21.15.112:0/1636800098 learned_addr learned my addr 172.21.15.112:0/1636800098 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:19.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa4737fe640 1 --2- 172.21.15.112:0/1636800098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4740ac3b0 0x7fa4740c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:19.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa473fff640 1 --2- 172.21.15.112:0/1636800098 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4741524a0 0x7fa474147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:19.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa472ffd640 1 -- 172.21.15.112:0/1636800098 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4741524a0 msgr2=0x7fa474147fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:19.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa472ffd640 1 --2- 172.21.15.112:0/1636800098 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4741524a0 0x7fa474147fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa472ffd640 1 -- 172.21.15.112:0/1636800098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4740ac3b0 msgr2=0x7fa4740c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:19.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa472ffd640 1 --2- 172.21.15.112:0/1636800098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4740ac3b0 0x7fa4740c78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa472ffd640 1 -- 172.21.15.112:0/1636800098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47406ee50 con 0x7fa47414c3f0 2024-04-25T01:26:19.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa472ffd640 1 --2- 172.21.15.112:0/1636800098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa47414c3f0 0x7fa4740c6fb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa468000c00 tx=0x7fa4680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:19.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa45bfff640 1 -- 172.21.15.112:0/1636800098 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468037040 con 0x7fa47414c3f0 2024-04-25T01:26:19.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa45bfff640 1 -- 172.21.15.112:0/1636800098 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa468042070 con 0x7fa47414c3f0 2024-04-25T01:26:19.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa479e31640 1 -- 172.21.15.112:0/1636800098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa47406f130 con 0x7fa47414c3f0 2024-04-25T01:26:19.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa45bfff640 1 -- 172.21.15.112:0/1636800098 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468030880 con 0x7fa47414c3f0 2024-04-25T01:26:19.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.327+0000 7fa479e31640 1 -- 172.21.15.112:0/1636800098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa474148500 con 0x7fa47414c3f0 2024-04-25T01:26:19.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.328+0000 7fa479e31640 1 -- 172.21.15.112:0/1636800098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4740720e0 con 0x7fa47414c3f0 2024-04-25T01:26:19.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.329+0000 7fa45bfff640 1 -- 172.21.15.112:0/1636800098 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa46802ad10 con 0x7fa47414c3f0 2024-04-25T01:26:19.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.329+0000 7fa45bfff640 1 --2- 172.21.15.112:0/1636800098 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa46006be60 0x7fa46006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:19.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.329+0000 7fa45bfff640 1 -- 172.21.15.112:0/1636800098 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fa4680ab510 con 0x7fa47414c3f0 2024-04-25T01:26:19.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.329+0000 7fa4737fe640 1 --2- 172.21.15.112:0/1636800098 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa46006be60 0x7fa46006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:19.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.330+0000 7fa4737fe640 1 --2- 172.21.15.112:0/1636800098 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa46006be60 0x7fa46006e320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa464009b40 tx=0x7fa464009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:19.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.333+0000 7fa45bfff640 1 -- 172.21.15.112:0/1636800098 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa4740720e0 con 0x7fa47414c3f0 2024-04-25T01:26:19.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.547+0000 7fa479e31640 1 -- 172.21.15.112:0/1636800098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4740682a0 con 0x7fa47414c3f0 2024-04-25T01:26:19.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.548+0000 7fa45bfff640 1 -- 172.21.15.112:0/1636800098 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+6491 (secure 0 0 0) 0x7fa4680795d0 con 0x7fa47414c3f0 2024-04-25T01:26:19.527 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:19.527 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/4047219598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4047219598},{"type":"v1","addr":"172.21.15.187:6834","nonce":4047219598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2204640907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2204640907},{"type":"v1","addr":"172.21.15.187:6835","nonce":2204640907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:25:52.687260+0000","modified":"2024-04-25T01:25:55.702719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/2115984531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2115984531},{"type":"v1","addr":"172.21.15.112:6837","nonce":2115984531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1},{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:26:01.740521+0000","modified":"2024-04-25T01:26:04.746955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":8,"state":"up:active","state_seq":6,"addr":"172.21.15.112:6835/278686477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":278686477},{"type":"v1","addr":"172.21.15.112:6835","nonce":278686477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":2}]} 2024-04-25T01:26:19.528 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 11 2024-04-25T01:26:19.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.550+0000 7fa479e31640 1 -- 172.21.15.112:0/1636800098 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa46006be60 msgr2=0x7fa46006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:19.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.550+0000 7fa479e31640 1 --2- 172.21.15.112:0/1636800098 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa46006be60 0x7fa46006e320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa464009b40 tx=0x7fa464009190 comp rx=0 tx=0).stop 2024-04-25T01:26:19.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.550+0000 7fa479e31640 1 -- 172.21.15.112:0/1636800098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa47414c3f0 msgr2=0x7fa4740c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:19.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.550+0000 7fa479e31640 1 --2- 172.21.15.112:0/1636800098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa47414c3f0 0x7fa4740c6fb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa468000c00 tx=0x7fa4680046c0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.550+0000 7fa479e31640 1 -- 172.21.15.112:0/1636800098 shutdown_connections 2024-04-25T01:26:19.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.550+0000 7fa479e31640 1 --2- 172.21.15.112:0/1636800098 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa46006be60 0x7fa46006e320 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.550+0000 7fa479e31640 1 --2- 172.21.15.112:0/1636800098 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4741524a0 0x7fa474147fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.550+0000 7fa479e31640 1 --2- 172.21.15.112:0/1636800098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa47414c3f0 0x7fa4740c6fb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.550+0000 7fa479e31640 1 --2- 172.21.15.112:0/1636800098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4740ac3b0 0x7fa4740c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.550+0000 7fa479e31640 1 -- 172.21.15.112:0/1636800098 >> 172.21.15.112:0/1636800098 conn(0x7fa47406b160 msgr2=0x7fa47406b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:19.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.550+0000 7fa479e31640 1 -- 172.21.15.112:0/1636800098 shutdown_connections 2024-04-25T01:26:19.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:19.550+0000 7fa479e31640 1 -- 172.21.15.112:0/1636800098 wait complete. 2024-04-25T01:26:19.943 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 11, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T01:26:01.740521+0000', 'modified': '2024-04-25T01:26:04.746955+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4278}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4278': {'gid': 4278, 'name': 'a', 'rank': 0, 'incarnation': 8, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.112:6835/278686477', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6834', 'nonce': 278686477}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 278686477}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'b', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4278, 'qdb_cluster': [4278]} 2024-04-25T01:26:19.943 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T01:26:19.943 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T01:26:20.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.064+0000 7fe7a4637640 1 Processor -- start 2024-04-25T01:26:20.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.064+0000 7fe7a4637640 1 -- start start 2024-04-25T01:26:20.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.065+0000 7fe7a4637640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe79c1592c0 0x7fe79c0c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.065+0000 7fe7a4637640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe79c1596a0 0x7fe79c152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.065+0000 7fe7a4637640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe79c0c72b0 0x7fe79c155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.065+0000 7fe7a4637640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fe79c0c6e20 con 0x7fe79c1592c0 2024-04-25T01:26:20.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.065+0000 7fe7a4637640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe79c159b90 con 0x7fe79c0c72b0 2024-04-25T01:26:20.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.065+0000 7fe7a4637640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe79c06c970 con 0x7fe79c1596a0 2024-04-25T01:26:20.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.065+0000 7fe7a23d3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe79c1592c0 0x7fe79c0c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.065+0000 7fe7a23d3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe79c1592c0 0x7fe79c0c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46098/0 (socket says 172.21.15.112:46098) 2024-04-25T01:26:20.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.065+0000 7fe7a23d3640 1 -- 172.21.15.112:0/3077349252 learned_addr learned my addr 172.21.15.112:0/3077349252 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:20.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.065+0000 7fe7a23d3640 1 -- 172.21.15.112:0/3077349252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe79c1596a0 msgr2=0x7fe79c152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:20.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.065+0000 7fe7a23d3640 1 --2- 172.21.15.112:0/3077349252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe79c1596a0 0x7fe79c152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.066+0000 7fe7a23d3640 1 -- 172.21.15.112:0/3077349252 >> v2:172.21.15.187:3300/0 conn(0x7fe79c0c72b0 msgr2=0x7fe79c155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:20.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.066+0000 7fe7a23d3640 1 --2- 172.21.15.112:0/3077349252 >> v2:172.21.15.187:3300/0 conn(0x7fe79c0c72b0 0x7fe79c155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.066+0000 7fe7a23d3640 1 -- 172.21.15.112:0/3077349252 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c06cc50 con 0x7fe79c1592c0 2024-04-25T01:26:20.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.066+0000 7fe7a23d3640 1 --2- 172.21.15.112:0/3077349252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79c1592c0 0x7fe79c0c9070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0aac40 tx=0x7fe78c02a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb14e4d64527bd30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:20.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.066+0000 7fe7a13d1640 1 -- 172.21.15.112:0/3077349252 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c036070 con 0x7fe79c1592c0 2024-04-25T01:26:20.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.066+0000 7fe7a13d1640 1 -- 172.21.15.112:0/3077349252 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe78c031600 con 0x7fe79c1592c0 2024-04-25T01:26:20.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.066+0000 7fe7a13d1640 1 -- 172.21.15.112:0/3077349252 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c0319d0 con 0x7fe79c1592c0 2024-04-25T01:26:20.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.067+0000 7fe7a4637640 1 -- 172.21.15.112:0/3077349252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79c1592c0 msgr2=0x7fe79c0c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:20.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.067+0000 7fe7a4637640 1 --2- 172.21.15.112:0/3077349252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79c1592c0 0x7fe79c0c9070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0aac40 tx=0x7fe78c02a570 comp rx=0 tx=0).stop 2024-04-25T01:26:20.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.067+0000 7fe7a4637640 1 -- 172.21.15.112:0/3077349252 shutdown_connections 2024-04-25T01:26:20.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.067+0000 7fe7a4637640 1 --2- 172.21.15.112:0/3077349252 >> v2:172.21.15.187:3300/0 conn(0x7fe79c0c72b0 0x7fe79c155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.067+0000 7fe7a4637640 1 --2- 172.21.15.112:0/3077349252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe79c1596a0 0x7fe79c152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.067+0000 7fe7a4637640 1 --2- 172.21.15.112:0/3077349252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79c1592c0 0x7fe79c0c9070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.067+0000 7fe7a4637640 1 -- 172.21.15.112:0/3077349252 >> 172.21.15.112:0/3077349252 conn(0x7fe79c0cb170 msgr2=0x7fe79c0cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:20.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.067+0000 7fe7a4637640 1 -- 172.21.15.112:0/3077349252 shutdown_connections 2024-04-25T01:26:20.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.067+0000 7fe7a4637640 1 -- 172.21.15.112:0/3077349252 wait complete. 2024-04-25T01:26:20.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.068+0000 7fe7a4637640 1 Processor -- start 2024-04-25T01:26:20.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.068+0000 7fe7a4637640 1 -- start start 2024-04-25T01:26:20.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe7a4637640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe79c0c72b0 0x7fe79c154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe7a4637640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe79c1592c0 0x7fe79c0abb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe7a4637640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79c1596a0 0x7fe79c156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe7a4637640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c06dca0 con 0x7fe79c1596a0 2024-04-25T01:26:20.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe7a4637640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c06db20 con 0x7fe79c0c72b0 2024-04-25T01:26:20.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe7a4637640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe79c0c95b0 con 0x7fe79c1592c0 2024-04-25T01:26:20.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe7a2bd4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79c1596a0 0x7fe79c156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe7a2bd4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79c1596a0 0x7fe79c156e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46114/0 (socket says 172.21.15.112:46114) 2024-04-25T01:26:20.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe7a2bd4640 1 -- 172.21.15.112:0/3715199688 learned_addr learned my addr 172.21.15.112:0/3715199688 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:20.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe7a1bd2640 1 --2- 172.21.15.112:0/3715199688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe79c1592c0 0x7fe79c0abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe7a23d3640 1 --2- 172.21.15.112:0/3715199688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe79c0c72b0 0x7fe79c154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe7a2bd4640 1 -- 172.21.15.112:0/3715199688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe79c1592c0 msgr2=0x7fe79c0abb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:20.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe7a2bd4640 1 --2- 172.21.15.112:0/3715199688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe79c1592c0 0x7fe79c0abb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe7a2bd4640 1 -- 172.21.15.112:0/3715199688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe79c0c72b0 msgr2=0x7fe79c154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:20.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe7a2bd4640 1 --2- 172.21.15.112:0/3715199688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe79c0c72b0 0x7fe79c154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe7a2bd4640 1 -- 172.21.15.112:0/3715199688 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c0ac380 con 0x7fe79c1596a0 2024-04-25T01:26:20.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe7a2bd4640 1 --2- 172.21.15.112:0/3715199688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79c1596a0 0x7fe79c156e50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe794009fd0 tx=0x7fe79400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:20.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe79affd640 1 -- 172.21.15.112:0/3715199688 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794011830 con 0x7fe79c1596a0 2024-04-25T01:26:20.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe79affd640 1 -- 172.21.15.112:0/3715199688 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe794016070 con 0x7fe79c1596a0 2024-04-25T01:26:20.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe79affd640 1 -- 172.21.15.112:0/3715199688 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794011bf0 con 0x7fe79c1596a0 2024-04-25T01:26:20.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe7a4637640 1 -- 172.21.15.112:0/3715199688 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79c1eba10 con 0x7fe79c1596a0 2024-04-25T01:26:20.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.069+0000 7fe7a4637640 1 -- 172.21.15.112:0/3715199688 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79c1ebfd0 con 0x7fe79c1596a0 2024-04-25T01:26:20.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.071+0000 7fe79affd640 1 -- 172.21.15.112:0/3715199688 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe79401b460 con 0x7fe79c1596a0 2024-04-25T01:26:20.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.071+0000 7fe79affd640 1 --2- 172.21.15.112:0/3715199688 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe78806be30 0x7fe78806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.071+0000 7fe79affd640 1 -- 172.21.15.112:0/3715199688 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fe79408acf0 con 0x7fe79c1596a0 2024-04-25T01:26:20.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.071+0000 7fe7a23d3640 1 --2- 172.21.15.112:0/3715199688 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe78806be30 0x7fe78806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.071+0000 7fe7a23d3640 1 --2- 172.21.15.112:0/3715199688 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe78806be30 0x7fe78806e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe78c033000 tx=0x7fe78c009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:20.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.072+0000 7fe7a7416740 1 Processor -- start 2024-04-25T01:26:20.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.072+0000 7fe7a7416740 1 -- start start 2024-04-25T01:26:20.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.073+0000 7fe7a7416740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55913e06dad0 0x55913e06a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.073+0000 7fe7a7416740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55913e0308e0 0x55913e06ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.073+0000 7fe7a7416740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e051510 0x55913e06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.073+0000 7fe7a7416740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55913df92f00 con 0x55913e051510 2024-04-25T01:26:20.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.073+0000 7fe7a7416740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55913df3b750 con 0x55913e06dad0 2024-04-25T01:26:20.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.073+0000 7fe7a7416740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55913df8eec0 con 0x55913e0308e0 2024-04-25T01:26:20.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.073+0000 7fe7a2bd4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e051510 0x55913e06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.073+0000 7fe7a2bd4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e051510 0x55913e06bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46126/0 (socket says 172.21.15.112:46126) 2024-04-25T01:26:20.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.073+0000 7fe7a2bd4640 1 -- 172.21.15.112:0/77124789 learned_addr learned my addr 172.21.15.112:0/77124789 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.073+0000 7fe7a2bd4640 1 -- 172.21.15.112:0/77124789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55913e0308e0 msgr2=0x55913e06ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.073+0000 7fe7a2bd4640 1 --2- 172.21.15.112:0/77124789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55913e0308e0 0x55913e06ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.073+0000 7fe7a2bd4640 1 -- 172.21.15.112:0/77124789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55913e06dad0 msgr2=0x55913e06a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.073+0000 7fe7a2bd4640 1 --2- 172.21.15.112:0/77124789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55913e06dad0 0x55913e06a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.073+0000 7fe7a2bd4640 1 -- 172.21.15.112:0/77124789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55913dca72d0 con 0x55913e051510 2024-04-25T01:26:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.073+0000 7fe7a2bd4640 1 --2- 172.21.15.112:0/77124789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e051510 0x55913e06bbf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe794088ef0 tx=0x7fe79400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38effe7e449b09c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.073+0000 7fe783fff640 1 -- 172.21.15.112:0/77124789 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794087070 con 0x55913e051510 2024-04-25T01:26:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.074+0000 7fe783fff640 1 -- 172.21.15.112:0/77124789 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe794055420 con 0x55913e051510 2024-04-25T01:26:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.074+0000 7fe783fff640 1 -- 172.21.15.112:0/77124789 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794058440 con 0x55913e051510 2024-04-25T01:26:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.074+0000 7fe7a7416740 1 -- 172.21.15.112:0/77124789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e051510 msgr2=0x55913e06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.074+0000 7fe7a7416740 1 --2- 172.21.15.112:0/77124789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e051510 0x55913e06bbf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe794088ef0 tx=0x7fe79400bf90 comp rx=0 tx=0).stop 2024-04-25T01:26:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.074+0000 7fe7a7416740 1 -- 172.21.15.112:0/77124789 shutdown_connections 2024-04-25T01:26:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.074+0000 7fe7a7416740 1 --2- 172.21.15.112:0/77124789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e051510 0x55913e06bbf0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.074+0000 7fe7a7416740 1 --2- 172.21.15.112:0/77124789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55913e0308e0 0x55913e06ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.074+0000 7fe7a7416740 1 --2- 172.21.15.112:0/77124789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55913e06dad0 0x55913e06a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.074+0000 7fe7a7416740 1 -- 172.21.15.112:0/77124789 >> 172.21.15.112:0/77124789 conn(0x55913e02c640 msgr2=0x55913e050e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:20.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.074+0000 7fe7a7416740 1 -- 172.21.15.112:0/77124789 shutdown_connections 2024-04-25T01:26:20.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.074+0000 7fe7a7416740 1 -- 172.21.15.112:0/77124789 wait complete. 2024-04-25T01:26:20.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.075+0000 7fe7a7416740 1 -- start start 2024-04-25T01:26:20.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.075+0000 7fe7a7416740 1 Processor -- start 2024-04-25T01:26:20.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7a7416740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55913e051510 0x55913e02a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7a7416740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e06dad0 0x55913e07c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7a7416740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55913e02ad40 0x55913e068560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7a7416740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55913e030dc0 con 0x55913e06dad0 2024-04-25T01:26:20.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7a7416740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55913df4ecf0 con 0x55913e02ad40 2024-04-25T01:26:20.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7a7416740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55913e06a4b0 con 0x55913e051510 2024-04-25T01:26:20.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7a7416740 10 client.0 _ll_register_callbacks cb 0x55913df4e6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:26:20.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7a1bd2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e06dad0 0x55913e07c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7a1bd2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e06dad0 0x55913e07c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46140/0 (socket says 172.21.15.112:46140) 2024-04-25T01:26:20.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7a1bd2640 1 -- 172.21.15.112:0/3286419452 learned_addr learned my addr 172.21.15.112:0/3286419452 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:20.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7a2bd4640 1 --2- 172.21.15.112:0/3286419452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55913e02ad40 0x55913e068560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7a23d3640 1 --2- 172.21.15.112:0/3286419452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55913e051510 0x55913e02a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7a1bd2640 1 -- 172.21.15.112:0/3286419452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55913e051510 msgr2=0x55913e02a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:20.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7a1bd2640 1 --2- 172.21.15.112:0/3286419452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55913e051510 0x55913e02a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7a1bd2640 1 -- 172.21.15.112:0/3286419452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55913e02ad40 msgr2=0x55913e068560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7a1bd2640 1 --2- 172.21.15.112:0/3286419452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55913e02ad40 0x55913e068560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7a1bd2640 1 -- 172.21.15.112:0/3286419452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55913dd716b0 con 0x55913e06dad0 2024-04-25T01:26:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7a1bd2640 1 --2- 172.21.15.112:0/3286419452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e06dad0 0x55913e07c7c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe79c159ac0 tx=0x7fe79000e660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb20d10b62781cf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7817fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T01:26:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7817fa640 1 -- 172.21.15.112:0/3286419452 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe790017070 con 0x55913e06dad0 2024-04-25T01:26:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7817fa640 1 -- 172.21.15.112:0/3286419452 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fe790013470 con 0x55913e06dad0 2024-04-25T01:26:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7a7416740 1 -- 172.21.15.112:0/3286419452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe794055420 con 0x55913e06dad0 2024-04-25T01:26:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.076+0000 7fe7817fa640 1 -- 172.21.15.112:0/3286419452 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=11) ==== 24+0+0 (secure 0 0 0) 0x7fe790022050 con 0x55913e06dad0 2024-04-25T01:26:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.077+0000 7fe7a7416740 10 client.4896 fetch_fsmap learned FSMap version 11 2024-04-25T01:26:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.077+0000 7fe7a7416740 1 -- 172.21.15.112:0/3286419452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=11,monmap=2+}) -- 0x55913e02ea50 con 0x55913e06dad0 2024-04-25T01:26:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.077+0000 7fe7817fa640 1 -- 172.21.15.112:0/3286419452 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe794055420 con 0x55913e06dad0 2024-04-25T01:26:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.077+0000 7fe7817fa640 1 -- 172.21.15.112:0/3286419452 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 11) ==== 4128+0+0 (secure 0 0 0) 0x7fe790025020 con 0x55913e06dad0 2024-04-25T01:26:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.077+0000 7fe7a7416740 10 client.4896 fetch_fsmap finished waiting for FSMap version 11 2024-04-25T01:26:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.077+0000 7fe7a7416740 10 client.4896 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T01:26:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.077+0000 7fe7a7416740 20 client.4896 populate_metadata read hostname 'smithi112' 2024-04-25T01:26:20.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.077+0000 7fe7a7416740 1 --2- 172.21.15.112:0/3286419452 >> [v2:172.21.15.112:6834/278686477,v1:172.21.15.112:6835/278686477] conn(0x55913e04b4e0 0x55913e068aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.077+0000 7fe7a7416740 4 client.4896 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:26:20.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.077+0000 7fe7a7416740 1 -- 172.21.15.112:0/3286419452 --> [v2:172.21.15.112:6834/278686477,v1:172.21.15.112:6835/278686477] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55913dfa0440 con 0x55913e04b4e0 2024-04-25T01:26:20.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.077+0000 7fe7a23d3640 1 --2- 172.21.15.112:0/3286419452 >> [v2:172.21.15.112:6834/278686477,v1:172.21.15.112:6835/278686477] conn(0x55913e04b4e0 0x55913e068aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.077+0000 7fe7817fa640 10 client.4896 ms_handle_connect on v2:172.21.15.112:6834/278686477 2024-04-25T01:26:20.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.077+0000 7fe7a23d3640 1 --2- 172.21.15.112:0/3286419452 >> [v2:172.21.15.112:6834/278686477,v1:172.21.15.112:6835/278686477] conn(0x55913e04b4e0 0x55913e068aa0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6061f031fa6375b server_cookie=771057614771e90f in_seq=0 out_seq=0 2024-04-25T01:26:20.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.078+0000 7fe7817fa640 1 -- 172.21.15.112:0/3286419452 <== mds.0 v2:172.21.15.112:6834/278686477 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55913dfa0440 con 0x55913e04b4e0 2024-04-25T01:26:20.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.078+0000 7fe7817fa640 10 client.4896 handle_command_reply: tid=0 2024-04-25T01:26:20.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.078+0000 7fe7a7416740 1 client.4896 shutdown 2024-04-25T01:26:20.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.078+0000 7fe7a7416740 1 -- 172.21.15.112:0/3286419452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e06dad0 msgr2=0x55913e07c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:20.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.078+0000 7fe7a7416740 1 --2- 172.21.15.112:0/3286419452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e06dad0 0x55913e07c7c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe79c159ac0 tx=0x7fe79000e660 comp rx=0 tx=0).stop 2024-04-25T01:26:20.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.078+0000 7fe7a7416740 1 -- 172.21.15.112:0/3286419452 shutdown_connections 2024-04-25T01:26:20.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.078+0000 7fe7a7416740 1 --2- 172.21.15.112:0/3286419452 >> [v2:172.21.15.112:6834/278686477,v1:172.21.15.112:6835/278686477] conn(0x55913e04b4e0 0x55913e068aa0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.078+0000 7fe7a7416740 1 --2- 172.21.15.112:0/3286419452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55913e02ad40 0x55913e068560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.078+0000 7fe7817fa640 0 client.4896 ms_handle_reset on v2:172.21.15.112:6834/278686477 2024-04-25T01:26:20.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.078+0000 7fe7a7416740 1 --2- 172.21.15.112:0/3286419452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e06dad0 0x55913e07c7c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.078+0000 7fe7a7416740 1 --2- 172.21.15.112:0/3286419452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55913e051510 0x55913e02a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.078+0000 7fe7a7416740 1 -- 172.21.15.112:0/3286419452 >> 172.21.15.112:0/3286419452 conn(0x55913e0308e0 msgr2=0x55913e04ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:20.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.078+0000 7fe7a7416740 1 -- 172.21.15.112:0/3286419452 shutdown_connections 2024-04-25T01:26:20.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.078+0000 7fe7a7416740 1 -- 172.21.15.112:0/3286419452 wait complete. 2024-04-25T01:26:20.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.078+0000 7fe7a7416740 20 client.4896 trim_cache size 0 max 16384 2024-04-25T01:26:20.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.096+0000 7fe7a7416740 1 Processor -- start 2024-04-25T01:26:20.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.096+0000 7fe7a7416740 1 -- start start 2024-04-25T01:26:20.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7a7416740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e02ad40 0x55913e079720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7a7416740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55913e051510 0x55913e04ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7a7416740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55913e06dad0 0x55913e04f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7a7416740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1fa270 con 0x55913e02ad40 2024-04-25T01:26:20.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7a7416740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1fb300 con 0x55913e051510 2024-04-25T01:26:20.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7a7416740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe79c1f8dd0 con 0x55913e06dad0 2024-04-25T01:26:20.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7a23d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e02ad40 0x55913e079720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7a23d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e02ad40 0x55913e079720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46144/0 (socket says 172.21.15.112:46144) 2024-04-25T01:26:20.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7a23d3640 1 -- 172.21.15.112:0/1304053610 learned_addr learned my addr 172.21.15.112:0/1304053610 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:20.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7a2bd4640 1 --2- 172.21.15.112:0/1304053610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55913e06dad0 0x55913e04f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7a23d3640 1 -- 172.21.15.112:0/1304053610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55913e06dad0 msgr2=0x55913e04f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:20.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7a23d3640 1 --2- 172.21.15.112:0/1304053610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55913e06dad0 0x55913e04f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7a1bd2640 1 --2- 172.21.15.112:0/1304053610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55913e051510 0x55913e04ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7a23d3640 1 -- 172.21.15.112:0/1304053610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55913e051510 msgr2=0x55913e04ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:20.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7a23d3640 1 --2- 172.21.15.112:0/1304053610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55913e051510 0x55913e04ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7a23d3640 1 -- 172.21.15.112:0/1304053610 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55913e069ab0 con 0x55913e02ad40 2024-04-25T01:26:20.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7a1bd2640 1 --2- 172.21.15.112:0/1304053610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55913e051510 0x55913e04ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:20.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7a23d3640 1 --2- 172.21.15.112:0/1304053610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e02ad40 0x55913e079720 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009f40 tx=0x7fe78c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db86e4e57c36483 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:20.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7837fe640 1 -- 172.21.15.112:0/1304053610 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c051070 con 0x55913e02ad40 2024-04-25T01:26:20.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7837fe640 1 -- 172.21.15.112:0/1304053610 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe78c03da30 con 0x55913e02ad40 2024-04-25T01:26:20.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.097+0000 7fe7837fe640 1 -- 172.21.15.112:0/1304053610 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c036040 con 0x55913e02ad40 2024-04-25T01:26:20.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.098+0000 7fe7a7416740 1 -- 172.21.15.112:0/1304053610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e02ad40 msgr2=0x55913e079720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:20.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.098+0000 7fe7a7416740 1 --2- 172.21.15.112:0/1304053610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e02ad40 0x55913e079720 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009f40 tx=0x7fe78c02c000 comp rx=0 tx=0).stop 2024-04-25T01:26:20.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.098+0000 7fe7a7416740 1 -- 172.21.15.112:0/1304053610 shutdown_connections 2024-04-25T01:26:20.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.098+0000 7fe7a7416740 1 --2- 172.21.15.112:0/1304053610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55913e06dad0 0x55913e04f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.098+0000 7fe7a7416740 1 --2- 172.21.15.112:0/1304053610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55913e051510 0x55913e04ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.098+0000 7fe7a7416740 1 --2- 172.21.15.112:0/1304053610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e02ad40 0x55913e079720 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.098+0000 7fe7a7416740 1 -- 172.21.15.112:0/1304053610 >> 172.21.15.112:0/1304053610 conn(0x55913e0308e0 msgr2=0x55913e063c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:20.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.098+0000 7fe7a7416740 1 -- 172.21.15.112:0/1304053610 shutdown_connections 2024-04-25T01:26:20.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.098+0000 7fe7a7416740 1 -- 172.21.15.112:0/1304053610 wait complete. 2024-04-25T01:26:20.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.098+0000 7fe7a7416740 1 -- start start 2024-04-25T01:26:20.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.098+0000 7fe7a7416740 1 Processor -- start 2024-04-25T01:26:20.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.099+0000 7fe7a7416740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55913e02ad40 0x55913e06c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.099+0000 7fe7a7416740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e051510 0x55913e08fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.099+0000 7fe7a7416740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55913e06dad0 0x55913e0e4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.099+0000 7fe7a7416740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55913e02f0e0 con 0x55913e051510 2024-04-25T01:26:20.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.099+0000 7fe7a7416740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1f9370 con 0x55913e02ad40 2024-04-25T01:26:20.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.099+0000 7fe7a7416740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55913e04afb0 con 0x55913e06dad0 2024-04-25T01:26:20.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.099+0000 7fe7a7416740 10 client.0 _ll_register_callbacks cb 0x7fe790009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:26:20.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.099+0000 7fe7a1bd2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e051510 0x55913e08fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.099+0000 7fe7a1bd2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e051510 0x55913e08fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46150/0 (socket says 172.21.15.112:46150) 2024-04-25T01:26:20.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.099+0000 7fe7a1bd2640 1 -- 172.21.15.112:0/3457479332 learned_addr learned my addr 172.21.15.112:0/3457479332 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:20.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.099+0000 7fe7a2bd4640 1 --2- 172.21.15.112:0/3457479332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55913e06dad0 0x55913e0e4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.099+0000 7fe7a23d3640 1 --2- 172.21.15.112:0/3457479332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55913e02ad40 0x55913e06c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7a1bd2640 1 -- 172.21.15.112:0/3457479332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55913e06dad0 msgr2=0x55913e0e4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:20.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7a1bd2640 1 --2- 172.21.15.112:0/3457479332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55913e06dad0 0x55913e0e4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7a1bd2640 1 -- 172.21.15.112:0/3457479332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55913e02ad40 msgr2=0x55913e06c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:20.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7a1bd2640 1 --2- 172.21.15.112:0/3457479332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55913e02ad40 0x55913e06c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7a1bd2640 1 -- 172.21.15.112:0/3457479332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55913e02e0c0 con 0x55913e051510 2024-04-25T01:26:20.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7a1bd2640 1 --2- 172.21.15.112:0/3457479332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e051510 0x55913e08fd70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe790004770 tx=0x7fe7900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2500ba3891816716 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:20.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7817fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T01:26:20.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7817fa640 1 -- 172.21.15.112:0/3457479332 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79002a070 con 0x55913e051510 2024-04-25T01:26:20.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7817fa640 1 -- 172.21.15.112:0/3457479332 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fe790005920 con 0x55913e051510 2024-04-25T01:26:20.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7a7416740 1 -- 172.21.15.112:0/3457479332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe78c03da30 con 0x55913e051510 2024-04-25T01:26:20.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7817fa640 1 -- 172.21.15.112:0/3457479332 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=11) ==== 24+0+0 (secure 0 0 0) 0x7fe790035050 con 0x55913e051510 2024-04-25T01:26:20.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7a7416740 10 client.4902 fetch_fsmap learned FSMap version 11 2024-04-25T01:26:20.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7a7416740 1 -- 172.21.15.112:0/3457479332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=11,monmap=2+}) -- 0x55913e0903b0 con 0x55913e051510 2024-04-25T01:26:20.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7817fa640 1 -- 172.21.15.112:0/3457479332 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe78c03da30 con 0x55913e051510 2024-04-25T01:26:20.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7817fa640 1 -- 172.21.15.112:0/3457479332 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 11) ==== 4128+0+0 (secure 0 0 0) 0x7fe790038020 con 0x55913e051510 2024-04-25T01:26:20.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7a7416740 10 client.4902 fetch_fsmap finished waiting for FSMap version 11 2024-04-25T01:26:20.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7a7416740 10 client.4902 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T01:26:20.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7a7416740 20 client.4902 populate_metadata read hostname 'smithi112' 2024-04-25T01:26:20.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7a7416740 1 --2- 172.21.15.112:0/3457479332 >> [v2:172.21.15.112:6834/278686477,v1:172.21.15.112:6835/278686477] conn(0x55913e0ea7d0 0x55913e0eabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7a7416740 4 client.4902 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:26:20.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7a7416740 1 -- 172.21.15.112:0/3457479332 --> [v2:172.21.15.112:6834/278686477,v1:172.21.15.112:6835/278686477] -- command(tid 0: {"prefix": "status"}) -- 0x55913df9a790 con 0x55913e0ea7d0 2024-04-25T01:26:20.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.100+0000 7fe7a23d3640 1 --2- 172.21.15.112:0/3457479332 >> [v2:172.21.15.112:6834/278686477,v1:172.21.15.112:6835/278686477] conn(0x55913e0ea7d0 0x55913e0eabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.101+0000 7fe7817fa640 10 client.4902 ms_handle_connect on v2:172.21.15.112:6834/278686477 2024-04-25T01:26:20.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.101+0000 7fe7a23d3640 1 --2- 172.21.15.112:0/3457479332 >> [v2:172.21.15.112:6834/278686477,v1:172.21.15.112:6835/278686477] conn(0x55913e0ea7d0 0x55913e0eabb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37543b01988b8efc server_cookie=fa045f6d021c2f19 in_seq=0 out_seq=0 2024-04-25T01:26:20.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.101+0000 7fe7817fa640 1 -- 172.21.15.112:0/3457479332 <== mds.0 v2:172.21.15.112:6834/278686477 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55913df9a790 con 0x55913e0ea7d0 2024-04-25T01:26:20.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.101+0000 7fe7817fa640 10 client.4902 handle_command_reply: tid=0 2024-04-25T01:26:20.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.101+0000 7fe7a7416740 1 client.4902 shutdown 2024-04-25T01:26:20.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.101+0000 7fe7a7416740 1 -- 172.21.15.112:0/3457479332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e051510 msgr2=0x55913e08fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:20.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.101+0000 7fe7a7416740 1 --2- 172.21.15.112:0/3457479332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e051510 0x55913e08fd70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe790004770 tx=0x7fe7900046c0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.101+0000 7fe7a7416740 1 -- 172.21.15.112:0/3457479332 shutdown_connections 2024-04-25T01:26:20.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.101+0000 7fe7a7416740 1 --2- 172.21.15.112:0/3457479332 >> [v2:172.21.15.112:6834/278686477,v1:172.21.15.112:6835/278686477] conn(0x55913e0ea7d0 0x55913e0eabb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.101+0000 7fe7a7416740 1 --2- 172.21.15.112:0/3457479332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55913e06dad0 0x55913e0e4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.101+0000 7fe7a7416740 1 --2- 172.21.15.112:0/3457479332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55913e051510 0x55913e08fd70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.101+0000 7fe7817fa640 0 client.4902 ms_handle_reset on v2:172.21.15.112:6834/278686477 2024-04-25T01:26:20.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.101+0000 7fe7a7416740 1 --2- 172.21.15.112:0/3457479332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55913e02ad40 0x55913e06c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.101+0000 7fe7a7416740 1 -- 172.21.15.112:0/3457479332 >> 172.21.15.112:0/3457479332 conn(0x55913e0308e0 msgr2=0x55913e063c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:20.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.101+0000 7fe7a7416740 1 -- 172.21.15.112:0/3457479332 shutdown_connections 2024-04-25T01:26:20.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.101+0000 7fe7a7416740 1 -- 172.21.15.112:0/3457479332 wait complete. 2024-04-25T01:26:20.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.101+0000 7fe7a7416740 20 client.4902 trim_cache size 0 max 16384 2024-04-25T01:26:20.089 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:26:20.089 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:26:20.089 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-04-25T01:26:20.089 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 4278, 2024-04-25T01:26:20.089 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-25T01:26:20.089 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-25T01:26:20.089 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "b", 2024-04-25T01:26:20.089 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 17.704864754999999, 2024-04-25T01:26:20.090 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 11, 2024-04-25T01:26:20.090 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 24, 2024-04-25T01:26:20.090 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 24, 2024-04-25T01:26:20.090 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 33.903105375000003 2024-04-25T01:26:20.090 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:26:20.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.102+0000 7fe7a4637640 1 -- 172.21.15.112:0/3715199688 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe78806be30 msgr2=0x7fe78806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:20.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.102+0000 7fe7a4637640 1 --2- 172.21.15.112:0/3715199688 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe78806be30 0x7fe78806e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe78c033000 tx=0x7fe78c009000 comp rx=0 tx=0).stop 2024-04-25T01:26:20.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.102+0000 7fe7a4637640 1 -- 172.21.15.112:0/3715199688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79c1596a0 msgr2=0x7fe79c156e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:20.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.102+0000 7fe7a4637640 1 --2- 172.21.15.112:0/3715199688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79c1596a0 0x7fe79c156e50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe794009fd0 tx=0x7fe79400b8e0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.102+0000 7fe7a4637640 1 -- 172.21.15.112:0/3715199688 shutdown_connections 2024-04-25T01:26:20.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.102+0000 7fe7a4637640 1 --2- 172.21.15.112:0/3715199688 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe78806be30 0x7fe78806e2f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.102+0000 7fe7a4637640 1 --2- 172.21.15.112:0/3715199688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79c1596a0 0x7fe79c156e50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.102+0000 7fe7a4637640 1 --2- 172.21.15.112:0/3715199688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe79c1592c0 0x7fe79c0abb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.102+0000 7fe7a4637640 1 --2- 172.21.15.112:0/3715199688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe79c0c72b0 0x7fe79c154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.102+0000 7fe7a4637640 1 -- 172.21.15.112:0/3715199688 >> 172.21.15.112:0/3715199688 conn(0x7fe79c0cb170 msgr2=0x7fe79c152170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:20.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.102+0000 7fe7a4637640 1 -- 172.21.15.112:0/3715199688 shutdown_connections 2024-04-25T01:26:20.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.102+0000 7fe7a4637640 1 -- 172.21.15.112:0/3715199688 wait complete. 2024-04-25T01:26:20.093 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T01:26:20.106 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T01:26:20.106 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T01:26:20.107 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T01:26:20.107 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T01:26:20.167 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T01:26:20.167 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-04-25T01:26:20.167 INFO:tasks.ceph_fuse:client.0 config is {'cephfs_name': 'a'} 2024-04-25T01:26:20.167 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T01:26:20.167 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T01:26:20.168 INFO:tasks.ceph_fuse:client.1 config is {'cephfs_name': 'b'} 2024-04-25T01:26:20.168 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T01:26:20.168 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T01:26:20.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:20.168 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-04-25T01:26:20.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:20.187 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2024-04-25T01:26:20.257 INFO:teuthology.orchestra.run.smithi187.stderr:Cannot find device "ceph-brx" 2024-04-25T01:26:20.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:26:20.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:20.259 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-25T01:26:20.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:20.275 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-04-25T01:26:20.340 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2024-04-25T01:26:20.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:26:20.342 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T01:26:20.342 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T01:26:20.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.501+0000 7f3ec6914640 1 Processor -- start 2024-04-25T01:26:20.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.501+0000 7f3ec6914640 1 -- start start 2024-04-25T01:26:20.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.502+0000 7f3ec6914640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3ec0071ce0 0x7f3ec006f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.502+0000 7f3ec6914640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ec006b160 0x7f3ec006b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.502+0000 7f3ec6914640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ec00c20e0 con 0x7f3ec0072d50 2024-04-25T01:26:20.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.502+0000 7f3ec6914640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f3ec006f3b0 con 0x7f3ec0071ce0 2024-04-25T01:26:20.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.502+0000 7f3ec6914640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec0073130 con 0x7f3ec006b160 2024-04-25T01:26:20.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.502+0000 7f3ebf7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3ec0071ce0 0x7f3ec006f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.502+0000 7f3ebffff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3ec0072d50 0x7f3ec01563e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53448/0 (socket says 172.21.15.112:53448) 2024-04-25T01:26:20.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.502+0000 7f3ebffff640 1 -- 172.21.15.112:0/1808907804 learned_addr learned my addr 172.21.15.112:0/1808907804 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:20.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.502+0000 7f3ec4eb1640 1 --2- 172.21.15.112:0/1808907804 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ec006b160 0x7f3ec006b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.502+0000 7f3ebf7fe640 1 -- 172.21.15.112:0/1808907804 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ec006b160 msgr2=0x7f3ec006b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:20.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.502+0000 7f3ebf7fe640 1 --2- 172.21.15.112:0/1808907804 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ec006b160 0x7f3ec006b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.502+0000 7f3ebf7fe640 1 -- 172.21.15.112:0/1808907804 >> v1:172.21.15.112:6789/0 conn(0x7f3ec0072d50 legacy=0x7f3ec01563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:20.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.502+0000 7f3ebf7fe640 1 -- 172.21.15.112:0/1808907804 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec00c2380 con 0x7f3ec0071ce0 2024-04-25T01:26:20.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.502+0000 7f3ebeffd640 1 -- 172.21.15.112:0/1808907804 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 913755125 0 0) 0x7f3ec00c20e0 con 0x7f3ec0072d50 2024-04-25T01:26:20.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.503+0000 7f3ebf7fe640 1 --2- 172.21.15.112:0/1808907804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec0071ce0 0x7f3ec006f840 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3ec006f530 tx=0x7f3eb002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154cb9bb6b38fa33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:20.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.503+0000 7f3ebeffd640 1 -- 172.21.15.112:0/1808907804 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0037070 con 0x7f3ec0071ce0 2024-04-25T01:26:20.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.503+0000 7f3ebeffd640 1 -- 172.21.15.112:0/1808907804 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb002adb0 con 0x7f3ec0071ce0 2024-04-25T01:26:20.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.503+0000 7f3ebeffd640 1 -- 172.21.15.112:0/1808907804 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0032660 con 0x7f3ec0071ce0 2024-04-25T01:26:20.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.504+0000 7f3ec6914640 1 -- 172.21.15.112:0/1808907804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec0071ce0 msgr2=0x7f3ec006f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:20.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.504+0000 7f3ec6914640 1 --2- 172.21.15.112:0/1808907804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec0071ce0 0x7f3ec006f840 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3ec006f530 tx=0x7f3eb002a060 comp rx=0 tx=0).stop 2024-04-25T01:26:20.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.504+0000 7f3ec6914640 1 -- 172.21.15.112:0/1808907804 shutdown_connections 2024-04-25T01:26:20.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.504+0000 7f3ec6914640 1 --2- 172.21.15.112:0/1808907804 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ec006b160 0x7f3ec006b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.504+0000 7f3ec6914640 1 --2- 172.21.15.112:0/1808907804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec0071ce0 0x7f3ec006f840 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.504+0000 7f3ec6914640 1 -- 172.21.15.112:0/1808907804 >> 172.21.15.112:0/1808907804 conn(0x7f3ec00c8090 msgr2=0x7f3ec00c8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:20.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.504+0000 7f3ec6914640 1 -- 172.21.15.112:0/1808907804 shutdown_connections 2024-04-25T01:26:20.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.504+0000 7f3ec6914640 1 -- 172.21.15.112:0/1808907804 wait complete. 2024-04-25T01:26:20.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.505+0000 7f3ec6914640 1 Processor -- start 2024-04-25T01:26:20.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.505+0000 7f3ec6914640 1 -- start start 2024-04-25T01:26:20.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ec6914640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ec006b160 0x7f3ec01ec5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ec6914640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ec0071ce0 0x7f3ec0143660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ec6914640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec0072d50 0x7f3ec0143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ec6914640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec006f040 con 0x7f3ec0072d50 2024-04-25T01:26:20.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ec6914640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec006ed60 con 0x7f3ec006b160 2024-04-25T01:26:20.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ec6914640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec0072160 con 0x7f3ec0071ce0 2024-04-25T01:26:20.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ec4eb1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec0072d50 0x7f3ec0143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ec4eb1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec0072d50 0x7f3ec0143ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46168/0 (socket says 172.21.15.112:46168) 2024-04-25T01:26:20.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ec4eb1640 1 -- 172.21.15.112:0/3331432986 learned_addr learned my addr 172.21.15.112:0/3331432986 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:20.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ebf7fe640 1 --2- 172.21.15.112:0/3331432986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ec0071ce0 0x7f3ec0143660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ebffff640 1 --2- 172.21.15.112:0/3331432986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ec006b160 0x7f3ec01ec5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ec4eb1640 1 -- 172.21.15.112:0/3331432986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ec0071ce0 msgr2=0x7f3ec0143660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:20.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ec4eb1640 1 --2- 172.21.15.112:0/3331432986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ec0071ce0 0x7f3ec0143660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ec4eb1640 1 -- 172.21.15.112:0/3331432986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ec006b160 msgr2=0x7f3ec01ec5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:20.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ec4eb1640 1 --2- 172.21.15.112:0/3331432986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ec006b160 0x7f3ec01ec5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ec4eb1640 1 -- 172.21.15.112:0/3331432986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec0155f00 con 0x7f3ec0072d50 2024-04-25T01:26:20.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ec4eb1640 1 --2- 172.21.15.112:0/3331432986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec0072d50 0x7f3ec0143ba0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8007e40 tx=0x7f3eb800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:20.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ebcff9640 1 -- 172.21.15.112:0/3331432986 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8014620 con 0x7f3ec0072d50 2024-04-25T01:26:20.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ebcff9640 1 -- 172.21.15.112:0/3331432986 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb8019070 con 0x7f3ec0072d50 2024-04-25T01:26:20.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ec6914640 1 -- 172.21.15.112:0/3331432986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec0154230 con 0x7f3ec0072d50 2024-04-25T01:26:20.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ebcff9640 1 -- 172.21.15.112:0/3331432986 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8014a20 con 0x7f3ec0072d50 2024-04-25T01:26:20.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.506+0000 7f3ec6914640 1 -- 172.21.15.112:0/3331432986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec01547f0 con 0x7f3ec0072d50 2024-04-25T01:26:20.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.507+0000 7f3ec6914640 1 -- 172.21.15.112:0/3331432986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec00689b0 con 0x7f3ec0072d50 2024-04-25T01:26:20.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.508+0000 7f3ebcff9640 1 -- 172.21.15.112:0/3331432986 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3eb8014bc0 con 0x7f3ec0072d50 2024-04-25T01:26:20.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.508+0000 7f3ebcff9640 1 --2- 172.21.15.112:0/3331432986 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3eac06be60 0x7f3eac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:20.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.508+0000 7f3ebcff9640 1 -- 172.21.15.112:0/3331432986 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f3eb808b920 con 0x7f3ec0072d50 2024-04-25T01:26:20.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.508+0000 7f3ebffff640 1 --2- 172.21.15.112:0/3331432986 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3eac06be60 0x7f3eac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:20.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.508+0000 7f3ebffff640 1 --2- 172.21.15.112:0/3331432986 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3eac06be60 0x7f3eac06e320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4000f60 tx=0x7f3eb4007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:20.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.512+0000 7f3ebcff9640 1 -- 172.21.15.112:0/3331432986 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3eb805a030 con 0x7f3ec0072d50 2024-04-25T01:26:20.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.723+0000 7f3ec6914640 1 -- 172.21.15.112:0/3331432986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3ec00c7a70 con 0x7f3ec0072d50 2024-04-25T01:26:20.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.723+0000 7f3ebcff9640 1 -- 172.21.15.112:0/3331432986 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v11) ==== 53+0+116 (secure 0 0 0) 0x7f3eb8059930 con 0x7f3ec0072d50 2024-04-25T01:26:20.703 INFO:teuthology.orchestra.run.smithi112.stdout:name: a, metadata pool: a_metadata, data pools: [a_data ] 2024-04-25T01:26:20.703 INFO:teuthology.orchestra.run.smithi112.stdout:name: b, metadata pool: b_metadata, data pools: [b_data ] 2024-04-25T01:26:20.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.725+0000 7f3ec6914640 1 -- 172.21.15.112:0/3331432986 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3eac06be60 msgr2=0x7f3eac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:20.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.725+0000 7f3ec6914640 1 --2- 172.21.15.112:0/3331432986 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3eac06be60 0x7f3eac06e320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4000f60 tx=0x7f3eb4007c00 comp rx=0 tx=0).stop 2024-04-25T01:26:20.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.725+0000 7f3ec6914640 1 -- 172.21.15.112:0/3331432986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec0072d50 msgr2=0x7f3ec0143ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:20.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.725+0000 7f3ec6914640 1 --2- 172.21.15.112:0/3331432986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec0072d50 0x7f3ec0143ba0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8007e40 tx=0x7f3eb800e4b0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.725+0000 7f3ec6914640 1 -- 172.21.15.112:0/3331432986 shutdown_connections 2024-04-25T01:26:20.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.725+0000 7f3ec6914640 1 --2- 172.21.15.112:0/3331432986 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3eac06be60 0x7f3eac06e320 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.725+0000 7f3ec6914640 1 --2- 172.21.15.112:0/3331432986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec0072d50 0x7f3ec0143ba0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.725+0000 7f3ec6914640 1 --2- 172.21.15.112:0/3331432986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ec0071ce0 0x7f3ec0143660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.725+0000 7f3ec6914640 1 --2- 172.21.15.112:0/3331432986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ec006b160 0x7f3ec01ec5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:20.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.725+0000 7f3ec6914640 1 -- 172.21.15.112:0/3331432986 >> 172.21.15.112:0/3331432986 conn(0x7f3ec00c8090 msgr2=0x7f3ec0067110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:20.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.725+0000 7f3ec6914640 1 -- 172.21.15.112:0/3331432986 shutdown_connections 2024-04-25T01:26:20.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20.725+0000 7f3ec6914640 1 -- 172.21.15.112:0/3331432986 wait complete. 2024-04-25T01:26:20.715 INFO:tasks.cephfs.mount:Mounting Ceph FS a; just confirmed its presence on cluster 2024-04-25T01:26:20.715 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T01:26:20.715 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-04-25T01:26:20.716 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T01:26:20.716 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T01:26:20.716 INFO:tasks.cephfs.mount:self.cephfs_name = a 2024-04-25T01:26:20.716 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T01:26:20.716 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T01:26:20.716 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T01:26:20.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:20.716 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-04-25T01:26:20.731 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T01:26:20.731 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T01:26:20.732 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T01:26:20.732 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:26:20.732 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-04-25T01:26:20.732 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:26:20.732 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:26:20.732 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-04-25T01:26:20.732 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-04-25T01:26:20.732 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:26:20.732 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-04-25T01:26:20.732 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-04-25T01:26:20.733 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:26:20.733 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:26:20.733 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T01:26:20.733 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:26:20.733 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T01:26:20.733 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42718sec preferred_lft 42718sec 2024-04-25T01:26:20.733 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::b792:9ec8:2d3f:85ad/64 scope link noprefixroute 2024-04-25T01:26:20.733 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:26:20.734 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T01:26:20.734 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:26:20.734 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T01:26:20.734 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add name ceph-brx type bridge 2024-04-25T01:26:20.735 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr flush dev ceph-brx 2024-04-25T01:26:20.735 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set ceph-brx up 2024-04-25T01:26:20.735 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T01:26:20.735 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T01:26:20.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:26:20.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:26:20.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:20.940 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T01:26:20.963 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-04-25T01:26:20.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:20.965 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-04-25T01:26:21.040 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-04-25T01:26:21.040 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T01:26:21.041 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T01:26:21.041 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T01:26:21.041 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T01:26:21.041 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:26:21.041 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T01:26:21.041 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T01:26:21.041 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T01:26:21.041 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T01:26:21.042 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T01:26:21.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:26:21.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:26:21.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:21.305 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-25T01:26:21.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:21.322 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2024-04-25T01:26:21.377 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:26:21.377 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T01:26:21.377 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:26:21.377 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T01:26:21.378 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T01:26:21.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:26:21.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:26:21.492 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T01:26:21.492 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:26:21.492 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T01:26:21.492 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T01:26:21.492 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T01:26:21.492 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T01:26:21.492 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T01:26:21.493 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T01:26:21.493 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T01:26:21.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:26:21.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:26:21.614 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:26:21.614 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T01:26:21.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-04-25T01:26:21.615 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T01:26:21.615 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T01:26:21.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:26:21.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:26:21.689 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'cephfs_name': 'a'} 2024-04-25T01:26:21.690 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:26:21.690 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T01:26:21.713 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T01:26:21.713 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:26:21.713 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T01:26:21.767 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2024-04-25T01:26:21.832 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:26:21.888 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2024-04-25T01:26:21.888 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2024-04-25T01:26:21.888 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2024-04-25T01:26:21.888 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2024-04-25T01:26:21.888 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2024-04-25T01:26:21.888 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2024-04-25T01:26:21.888 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2024-04-25T01:26:21.888 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2024-04-25T01:26:21.888 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2024-04-25T01:26:21.888 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2024-04-25T01:26:21.888 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2024-04-25T01:26:21.888 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/selinux 2024-04-25T01:26:21.889 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:26:21.889 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2024-04-25T01:26:21.889 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2024-04-25T01:26:21.889 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2024-04-25T01:26:21.889 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2024-04-25T01:26:21.889 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2024-04-25T01:26:21.889 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2024-04-25T01:26:21.889 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T01:26:21.889 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:26:21.889 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T01:26:21.889 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T01:26:21.889 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T01:26:21.889 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2024-04-25T01:26:21.889 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2024-04-25T01:26:21.890 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T01:26:21.890 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T01:26:21.890 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T01:26:21.890 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T01:26:21.890 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2024-04-25T01:26:21.890 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:26:21.890 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:26:21.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:21.890 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2024-04-25T01:26:21.943 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T01:26:21.943 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_fs=a) 2024-04-25T01:26:21.985 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2024-04-25T01:26:22.008 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:26:22.063 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2024-04-25T01:26:22.064 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2024-04-25T01:26:22.064 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2024-04-25T01:26:22.064 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2024-04-25T01:26:22.064 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2024-04-25T01:26:22.064 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2024-04-25T01:26:22.064 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2024-04-25T01:26:22.064 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2024-04-25T01:26:22.064 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2024-04-25T01:26:22.064 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2024-04-25T01:26:22.064 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2024-04-25T01:26:22.064 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/selinux 2024-04-25T01:26:22.064 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:26:22.064 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2024-04-25T01:26:22.065 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2024-04-25T01:26:22.065 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2024-04-25T01:26:22.065 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2024-04-25T01:26:22.065 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2024-04-25T01:26:22.065 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2024-04-25T01:26:22.065 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T01:26:22.065 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:26:22.065 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T01:26:22.065 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T01:26:22.065 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T01:26:22.065 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2024-04-25T01:26:22.065 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2024-04-25T01:26:22.065 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T01:26:22.066 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T01:26:22.066 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T01:26:22.066 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T01:26:22.066 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2024-04-25T01:26:22.066 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:26:22.066 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:26:22.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:22.066 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2024-04-25T01:26:22.096 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi112.stderr:2024-04-25T01:26:22.117+0000 7fc477721480 -1 init, newargv = 0x55646646d500 newargc=15 2024-04-25T01:26:22.097 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi112.stderr:ceph-fuse[45496]: starting ceph client 2024-04-25T01:26:22.102 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi112.stderr:ceph-fuse[45496]: starting fuse 2024-04-25T01:26:22.111 INFO:teuthology.orchestra.run.smithi112.stdout:42 2024-04-25T01:26:22.111 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-25T01:26:22.111 DEBUG:teuthology.orchestra.run.smithi112:> sudo stdin-killer -- python3 -c ' 2024-04-25T01:26:22.112 DEBUG:teuthology.orchestra.run.smithi112:> import glob 2024-04-25T01:26:22.112 DEBUG:teuthology.orchestra.run.smithi112:> import re 2024-04-25T01:26:22.112 DEBUG:teuthology.orchestra.run.smithi112:> import os 2024-04-25T01:26:22.112 DEBUG:teuthology.orchestra.run.smithi112:> import subprocess 2024-04-25T01:26:22.112 DEBUG:teuthology.orchestra.run.smithi112:> 2024-04-25T01:26:22.112 DEBUG:teuthology.orchestra.run.smithi112:> def _find_admin_socket(client_name): 2024-04-25T01:26:22.112 DEBUG:teuthology.orchestra.run.smithi112:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T01:26:22.112 DEBUG:teuthology.orchestra.run.smithi112:> files = glob.glob(asok_path) 2024-04-25T01:26:22.112 DEBUG:teuthology.orchestra.run.smithi112:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T01:26:22.112 DEBUG:teuthology.orchestra.run.smithi112:> 2024-04-25T01:26:22.112 DEBUG:teuthology.orchestra.run.smithi112:> # Given a non-glob path, it better be there 2024-04-25T01:26:22.112 DEBUG:teuthology.orchestra.run.smithi112:> if "*" not in asok_path: 2024-04-25T01:26:22.112 DEBUG:teuthology.orchestra.run.smithi112:> assert(len(files) == 1) 2024-04-25T01:26:22.112 DEBUG:teuthology.orchestra.run.smithi112:> return files[0] 2024-04-25T01:26:22.112 DEBUG:teuthology.orchestra.run.smithi112:> 2024-04-25T01:26:22.113 DEBUG:teuthology.orchestra.run.smithi112:> for f in files: 2024-04-25T01:26:22.113 DEBUG:teuthology.orchestra.run.smithi112:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T01:26:22.113 DEBUG:teuthology.orchestra.run.smithi112:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T01:26:22.113 DEBUG:teuthology.orchestra.run.smithi112:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T01:26:22.113 DEBUG:teuthology.orchestra.run.smithi112:> contents = proc_f.read() 2024-04-25T01:26:22.113 DEBUG:teuthology.orchestra.run.smithi112:> if mountpoint in contents: 2024-04-25T01:26:22.113 DEBUG:teuthology.orchestra.run.smithi112:> return f 2024-04-25T01:26:22.113 DEBUG:teuthology.orchestra.run.smithi112:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T01:26:22.113 DEBUG:teuthology.orchestra.run.smithi112:> 2024-04-25T01:26:22.113 DEBUG:teuthology.orchestra.run.smithi112:> print(_find_admin_socket("client.0")) 2024-04-25T01:26:22.113 DEBUG:teuthology.orchestra.run.smithi112:> ' 2024-04-25T01:26:22.228 INFO:teuthology.orchestra.run.smithi112.stdout:/var/run/ceph/ceph-client.0.45496.asok 2024-04-25T01:26:22.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:26:22.238 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45496.asok 2024-04-25T01:26:22.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:22.239 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45496.asok status 2024-04-25T01:26:22.370 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:26:22.370 INFO:teuthology.orchestra.run.smithi112.stdout: "metadata": { 2024-04-25T01:26:22.370 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T01:26:22.370 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T01:26:22.370 INFO:teuthology.orchestra.run.smithi112.stdout: "entity_id": "0", 2024-04-25T01:26:22.370 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": "smithi112", 2024-04-25T01:26:22.370 INFO:teuthology.orchestra.run.smithi112.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T01:26:22.370 INFO:teuthology.orchestra.run.smithi112.stdout: "pid": "45496", 2024-04-25T01:26:22.370 INFO:teuthology.orchestra.run.smithi112.stdout: "root": "/" 2024-04-25T01:26:22.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:22.370 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_count": 0, 2024-04-25T01:26:22.371 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_pinned_count": 0, 2024-04-25T01:26:22.371 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 4920, 2024-04-25T01:26:22.371 INFO:teuthology.orchestra.run.smithi112.stdout: "inst": { 2024-04-25T01:26:22.371 INFO:teuthology.orchestra.run.smithi112.stdout: "name": { 2024-04-25T01:26:22.371 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "client", 2024-04-25T01:26:22.371 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4920 2024-04-25T01:26:22.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:22.371 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2024-04-25T01:26:22.371 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:26:22.371 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:26:22.371 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 615043797 2024-04-25T01:26:22.371 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:22.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:22.371 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2024-04-25T01:26:22.372 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:26:22.372 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:26:22.372 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 615043797 2024-04-25T01:26:22.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:22.372 INFO:teuthology.orchestra.run.smithi112.stdout: "inst_str": "client.4920 192.168.0.1:0/615043797", 2024-04-25T01:26:22.372 INFO:teuthology.orchestra.run.smithi112.stdout: "addr_str": "192.168.0.1:0/615043797", 2024-04-25T01:26:22.372 INFO:teuthology.orchestra.run.smithi112.stdout: "inode_count": 1, 2024-04-25T01:26:22.372 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_epoch": 7, 2024-04-25T01:26:22.372 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch": 24, 2024-04-25T01:26:22.372 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch_barrier": 0, 2024-04-25T01:26:22.372 INFO:teuthology.orchestra.run.smithi112.stdout: "blocklisted": false, 2024-04-25T01:26:22.372 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "a" 2024-04-25T01:26:22.372 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:26:22.378 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T01:26:22.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.496+0000 7f5b7addb640 1 Processor -- start 2024-04-25T01:26:22.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.496+0000 7f5b7addb640 1 -- start start 2024-04-25T01:26:22.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.497+0000 7f5b7addb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b740743d0 0x7f5b740747b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:22.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.497+0000 7f5b7addb640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5b7406fb80 0x7f5b74155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:22.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.497+0000 7f5b7addb640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b740c1e40 con 0x7f5b74155f40 2024-04-25T01:26:22.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.497+0000 7f5b7addb640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5b740c40e0 con 0x7f5b7406fb80 2024-04-25T01:26:22.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.497+0000 7f5b7addb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5b741534b0 con 0x7f5b740743d0 2024-04-25T01:26:22.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.497+0000 7f5b78b77640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b740743d0 0x7f5b740747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:22.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.497+0000 7f5b73fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5b7406fb80 0x7f5b74155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:22.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.497+0000 7f5b78b77640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b740743d0 0x7f5b740747b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:40806/0 (socket says 172.21.15.112:40806) 2024-04-25T01:26:22.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.497+0000 7f5b78b77640 1 -- 172.21.15.112:0/1333498181 learned_addr learned my addr 172.21.15.112:0/1333498181 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:22.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.497+0000 7f5b78b77640 1 -- 172.21.15.112:0/1333498181 >> v2:172.21.15.187:3300/0 conn(0x7f5b7406fb80 msgr2=0x7f5b74155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:22.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.497+0000 7f5b78b77640 1 --2- 172.21.15.112:0/1333498181 >> v2:172.21.15.187:3300/0 conn(0x7f5b7406fb80 0x7f5b74155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:22.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.497+0000 7f5b78b77640 1 -- 172.21.15.112:0/1333498181 >> v1:172.21.15.187:6789/0 conn(0x7f5b74155f40 legacy=0x7f5b740c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:22.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.497+0000 7f5b78b77640 1 -- 172.21.15.112:0/1333498181 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b74159a20 con 0x7f5b740743d0 2024-04-25T01:26:22.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.497+0000 7f5b73fff640 1 --2- 172.21.15.112:0/1333498181 >> v2:172.21.15.187:3300/0 conn(0x7f5b7406fb80 0x7f5b74155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:26:22.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.498+0000 7f5b78b77640 1 --2- 172.21.15.112:0/1333498181 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b740743d0 0x7f5b740747b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009870 tx=0x7f5b6802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=648d6580ebd1c807 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:22.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.498+0000 7f5b737fe640 1 -- 172.21.15.112:0/1333498181 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68037070 con 0x7f5b740743d0 2024-04-25T01:26:22.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.498+0000 7f5b737fe640 1 -- 172.21.15.112:0/1333498181 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b68032440 con 0x7f5b740743d0 2024-04-25T01:26:22.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.498+0000 7f5b737fe640 1 -- 172.21.15.112:0/1333498181 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68032810 con 0x7f5b740743d0 2024-04-25T01:26:22.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.499+0000 7f5b7addb640 1 -- 172.21.15.112:0/1333498181 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b740743d0 msgr2=0x7f5b740747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:22.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.499+0000 7f5b7addb640 1 --2- 172.21.15.112:0/1333498181 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b740743d0 0x7f5b740747b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009870 tx=0x7f5b6802a1c0 comp rx=0 tx=0).stop 2024-04-25T01:26:22.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.499+0000 7f5b7addb640 1 -- 172.21.15.112:0/1333498181 shutdown_connections 2024-04-25T01:26:22.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.499+0000 7f5b7addb640 1 --2- 172.21.15.112:0/1333498181 >> v2:172.21.15.187:3300/0 conn(0x7f5b7406fb80 0x7f5b74155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.499+0000 7f5b7addb640 1 --2- 172.21.15.112:0/1333498181 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b740743d0 0x7f5b740747b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.499+0000 7f5b7addb640 1 -- 172.21.15.112:0/1333498181 >> 172.21.15.112:0/1333498181 conn(0x7f5b7406b160 msgr2=0x7f5b7406b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.499+0000 7f5b7addb640 1 -- 172.21.15.112:0/1333498181 shutdown_connections 2024-04-25T01:26:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.499+0000 7f5b7addb640 1 -- 172.21.15.112:0/1333498181 wait complete. 2024-04-25T01:26:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.500+0000 7f5b7addb640 1 Processor -- start 2024-04-25T01:26:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.500+0000 7f5b7addb640 1 -- start start 2024-04-25T01:26:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.500+0000 7f5b7addb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b7406fb80 0x7f5b74158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.500+0000 7f5b7addb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b740743d0 0x7f5b7414bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.500+0000 7f5b7addb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b74155f40 0x7f5b7414c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.500+0000 7f5b7addb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b740c19b0 con 0x7f5b740743d0 2024-04-25T01:26:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.500+0000 7f5b7addb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5b740c16d0 con 0x7f5b7406fb80 2024-04-25T01:26:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.500+0000 7f5b7addb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5b740a7070 con 0x7f5b74155f40 2024-04-25T01:26:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.501+0000 7f5b78b77640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b7406fb80 0x7f5b74158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.501+0000 7f5b73fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b740743d0 0x7f5b7414bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.501+0000 7f5b79378640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b74155f40 0x7f5b7414c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.501+0000 7f5b78b77640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b7406fb80 0x7f5b74158160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:33050/0 (socket says 172.21.15.112:33050) 2024-04-25T01:26:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.501+0000 7f5b78b77640 1 -- 172.21.15.112:0/2458745332 learned_addr learned my addr 172.21.15.112:0/2458745332 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.501+0000 7f5b78b77640 1 -- 172.21.15.112:0/2458745332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b74155f40 msgr2=0x7f5b7414c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.501+0000 7f5b78b77640 1 --2- 172.21.15.112:0/2458745332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b74155f40 0x7f5b7414c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.501+0000 7f5b78b77640 1 -- 172.21.15.112:0/2458745332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b740743d0 msgr2=0x7f5b7414bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.501+0000 7f5b78b77640 1 --2- 172.21.15.112:0/2458745332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b740743d0 0x7f5b7414bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.501+0000 7f5b78b77640 1 -- 172.21.15.112:0/2458745332 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b741522c0 con 0x7f5b7406fb80 2024-04-25T01:26:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.501+0000 7f5b79378640 1 --2- 172.21.15.112:0/2458745332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b74155f40 0x7f5b7414c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:26:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.501+0000 7f5b78b77640 1 --2- 172.21.15.112:0/2458745332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b7406fb80 0x7f5b74158160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5b68000c00 tx=0x7f5b680046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.501+0000 7f5b717fa640 1 -- 172.21.15.112:0/2458745332 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68041020 con 0x7f5b7406fb80 2024-04-25T01:26:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.501+0000 7f5b717fa640 1 -- 172.21.15.112:0/2458745332 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b6803d620 con 0x7f5b7406fb80 2024-04-25T01:26:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.501+0000 7f5b7addb640 1 -- 172.21.15.112:0/2458745332 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b741525a0 con 0x7f5b7406fb80 2024-04-25T01:26:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.501+0000 7f5b717fa640 1 -- 172.21.15.112:0/2458745332 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68037040 con 0x7f5b7406fb80 2024-04-25T01:26:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.501+0000 7f5b7addb640 1 -- 172.21.15.112:0/2458745332 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b741efa50 con 0x7f5b7406fb80 2024-04-25T01:26:22.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.502+0000 7f5b7addb640 1 -- 172.21.15.112:0/2458745332 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b74152ff0 con 0x7f5b7406fb80 2024-04-25T01:26:22.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.503+0000 7f5b717fa640 1 -- 172.21.15.112:0/2458745332 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b68044020 con 0x7f5b7406fb80 2024-04-25T01:26:22.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.503+0000 7f5b717fa640 1 --2- 172.21.15.112:0/2458745332 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b5806be30 0x7f5b5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:22.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.503+0000 7f5b717fa640 1 -- 172.21.15.112:0/2458745332 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f5b680ab7d0 con 0x7f5b7406fb80 2024-04-25T01:26:22.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.503+0000 7f5b73fff640 1 --2- 172.21.15.112:0/2458745332 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b5806be30 0x7f5b5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:22.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.504+0000 7f5b73fff640 1 --2- 172.21.15.112:0/2458745332 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b5806be30 0x7f5b5806e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009560 tx=0x7f5b60009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:22.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.508+0000 7f5b717fa640 1 -- 172.21.15.112:0/2458745332 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5b6804b030 con 0x7f5b7406fb80 2024-04-25T01:26:22.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.720+0000 7f5b7addb640 1 -- 172.21.15.112:0/2458745332 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5b740682a0 con 0x7f5b7406fb80 2024-04-25T01:26:22.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.721+0000 7f5b717fa640 1 -- 172.21.15.112:0/2458745332 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v11) ==== 53+0+116 (secure 0 0 0) 0x7f5b68079810 con 0x7f5b7406fb80 2024-04-25T01:26:22.700 INFO:teuthology.orchestra.run.smithi112.stdout:name: a, metadata pool: a_metadata, data pools: [a_data ] 2024-04-25T01:26:22.701 INFO:teuthology.orchestra.run.smithi112.stdout:name: b, metadata pool: b_metadata, data pools: [b_data ] 2024-04-25T01:26:22.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.723+0000 7f5b7addb640 1 -- 172.21.15.112:0/2458745332 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b5806be30 msgr2=0x7f5b5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:22.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.723+0000 7f5b7addb640 1 --2- 172.21.15.112:0/2458745332 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b5806be30 0x7f5b5806e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009560 tx=0x7f5b60009190 comp rx=0 tx=0).stop 2024-04-25T01:26:22.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.723+0000 7f5b7addb640 1 -- 172.21.15.112:0/2458745332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b7406fb80 msgr2=0x7f5b74158160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:22.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.723+0000 7f5b7addb640 1 --2- 172.21.15.112:0/2458745332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b7406fb80 0x7f5b74158160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5b68000c00 tx=0x7f5b680046c0 comp rx=0 tx=0).stop 2024-04-25T01:26:22.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.723+0000 7f5b7addb640 1 -- 172.21.15.112:0/2458745332 shutdown_connections 2024-04-25T01:26:22.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.723+0000 7f5b7addb640 1 --2- 172.21.15.112:0/2458745332 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b5806be30 0x7f5b5806e2f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:22.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.723+0000 7f5b7addb640 1 --2- 172.21.15.112:0/2458745332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b74155f40 0x7f5b7414c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:22.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.723+0000 7f5b7addb640 1 --2- 172.21.15.112:0/2458745332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b740743d0 0x7f5b7414bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:22.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.723+0000 7f5b7addb640 1 --2- 172.21.15.112:0/2458745332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b7406fb80 0x7f5b74158160 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:22.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.723+0000 7f5b7addb640 1 -- 172.21.15.112:0/2458745332 >> 172.21.15.112:0/2458745332 conn(0x7f5b7406b160 msgr2=0x7f5b740c2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:22.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.723+0000 7f5b7addb640 1 -- 172.21.15.112:0/2458745332 shutdown_connections 2024-04-25T01:26:22.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:22.723+0000 7f5b7addb640 1 -- 172.21.15.112:0/2458745332 wait complete. 2024-04-25T01:26:22.713 INFO:tasks.cephfs.mount:Mounting Ceph FS b; just confirmed its presence on cluster 2024-04-25T01:26:22.713 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T01:26:22.713 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-04-25T01:26:22.713 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T01:26:22.714 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T01:26:22.714 INFO:tasks.cephfs.mount:self.cephfs_name = b 2024-04-25T01:26:22.714 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T01:26:22.714 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T01:26:22.714 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T01:26:22.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:22.714 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-04-25T01:26:22.734 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T01:26:22.734 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T01:26:22.734 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T01:26:22.734 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:26:22.734 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-04-25T01:26:22.734 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:26:22.734 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:26:22.734 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:26:22.734 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-04-25T01:26:22.734 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:26:22.735 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:26:22.735 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-04-25T01:26:22.735 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:26:22.735 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:26:22.735 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T01:26:22.735 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:26:22.735 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T01:26:22.735 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42712sec preferred_lft 42712sec 2024-04-25T01:26:22.735 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::3002:5b55:d2fd:8255/64 scope link noprefixroute 2024-04-25T01:26:22.735 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:26:22.735 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T01:26:22.736 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:26:22.736 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-25T01:26:22.736 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link add name ceph-brx type bridge 2024-04-25T01:26:22.736 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip addr flush dev ceph-brx 2024-04-25T01:26:22.736 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set ceph-brx up 2024-04-25T01:26:22.736 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T01:26:22.736 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-25T01:26:22.828 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:26:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:26:22.919 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:26:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:26:22.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:22.925 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T01:26:22.948 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-04-25T01:26:22.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:22.950 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-04-25T01:26:23.018 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-04-25T01:26:23.018 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T01:26:23.018 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T01:26:23.018 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T01:26:23.018 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T01:26:23.019 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:26:23.019 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-25T01:26:23.019 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T01:26:23.019 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T01:26:23.019 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T01:26:23.019 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-25T01:26:23.105 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:26:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:26:23.176 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:26:23.179 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:26:23.292 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:26:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:26:23.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:23.298 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-04-25T01:26:23.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:23.314 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list-id 2024-04-25T01:26:23.369 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:26:23.369 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-25T01:26:23.369 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:26:23.369 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T01:26:23.369 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-25T01:26:23.456 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:26:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:26:23.478 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:26:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:26:23.485 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T01:26:23.485 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:26:23.485 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-25T01:26:23.485 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T01:26:23.485 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T01:26:23.485 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T01:26:23.485 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T01:26:23.485 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T01:26:23.485 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-25T01:26:23.533 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:26:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:26:23.597 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:26:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:26:23.603 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:26:23.604 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-25T01:26:23.604 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.0 up 2024-04-25T01:26:23.604 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T01:26:23.604 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-25T01:26:23.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:26:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:26:23.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:26:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:26:23.678 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {'cephfs_name': 'b'} 2024-04-25T01:26:23.679 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:26:23.679 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T01:26:23.703 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T01:26:23.703 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:26:23.704 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T01:26:23.758 DEBUG:teuthology.orchestra.run.smithi187:> sudo modprobe fuse 2024-04-25T01:26:23.822 DEBUG:teuthology.orchestra.run.smithi187:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:26:23.878 INFO:teuthology.orchestra.run.smithi187.stdout:/proc 2024-04-25T01:26:23.878 INFO:teuthology.orchestra.run.smithi187.stdout:/sys 2024-04-25T01:26:23.878 INFO:teuthology.orchestra.run.smithi187.stdout:/dev 2024-04-25T01:26:23.878 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/security 2024-04-25T01:26:23.878 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/shm 2024-04-25T01:26:23.878 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/pts 2024-04-25T01:26:23.878 INFO:teuthology.orchestra.run.smithi187.stdout:/run 2024-04-25T01:26:23.878 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/cgroup 2024-04-25T01:26:23.878 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/pstore 2024-04-25T01:26:23.878 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/bpf 2024-04-25T01:26:23.878 INFO:teuthology.orchestra.run.smithi187.stdout:/ 2024-04-25T01:26:23.879 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/selinux 2024-04-25T01:26:23.879 INFO:teuthology.orchestra.run.smithi187.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:26:23.879 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/hugepages 2024-04-25T01:26:23.879 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/mqueue 2024-04-25T01:26:23.879 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/debug 2024-04-25T01:26:23.879 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/tracing 2024-04-25T01:26:23.879 INFO:teuthology.orchestra.run.smithi187.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:26:23.879 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/config 2024-04-25T01:26:23.879 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/fuse/connections 2024-04-25T01:26:23.879 INFO:teuthology.orchestra.run.smithi187.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T01:26:23.879 INFO:teuthology.orchestra.run.smithi187.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T01:26:23.879 INFO:teuthology.orchestra.run.smithi187.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T01:26:23.879 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T01:26:23.879 INFO:teuthology.orchestra.run.smithi187.stdout:/run/user/1000 2024-04-25T01:26:23.879 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph 2024-04-25T01:26:23.880 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T01:26:23.880 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T01:26:23.880 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T01:26:23.880 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T01:26:23.880 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns 2024-04-25T01:26:23.880 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:26:23.880 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:26:23.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:23.880 DEBUG:teuthology.orchestra.run.smithi187:> ls /sys/fs/fuse/connections 2024-04-25T01:26:23.933 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T01:26:23.933 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1 --client_fs=b) 2024-04-25T01:26:23.975 DEBUG:teuthology.orchestra.run.smithi187:> sudo modprobe fuse 2024-04-25T01:26:23.998 DEBUG:teuthology.orchestra.run.smithi187:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:26:24.053 INFO:teuthology.orchestra.run.smithi187.stdout:/proc 2024-04-25T01:26:24.054 INFO:teuthology.orchestra.run.smithi187.stdout:/sys 2024-04-25T01:26:24.054 INFO:teuthology.orchestra.run.smithi187.stdout:/dev 2024-04-25T01:26:24.054 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/security 2024-04-25T01:26:24.054 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/shm 2024-04-25T01:26:24.054 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/pts 2024-04-25T01:26:24.054 INFO:teuthology.orchestra.run.smithi187.stdout:/run 2024-04-25T01:26:24.054 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/cgroup 2024-04-25T01:26:24.054 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/pstore 2024-04-25T01:26:24.055 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/bpf 2024-04-25T01:26:24.055 INFO:teuthology.orchestra.run.smithi187.stdout:/ 2024-04-25T01:26:24.055 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/selinux 2024-04-25T01:26:24.055 INFO:teuthology.orchestra.run.smithi187.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:26:24.055 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/hugepages 2024-04-25T01:26:24.055 INFO:teuthology.orchestra.run.smithi187.stdout:/dev/mqueue 2024-04-25T01:26:24.055 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/debug 2024-04-25T01:26:24.055 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/tracing 2024-04-25T01:26:24.055 INFO:teuthology.orchestra.run.smithi187.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:26:24.055 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/kernel/config 2024-04-25T01:26:24.056 INFO:teuthology.orchestra.run.smithi187.stdout:/sys/fs/fuse/connections 2024-04-25T01:26:24.056 INFO:teuthology.orchestra.run.smithi187.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T01:26:24.056 INFO:teuthology.orchestra.run.smithi187.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T01:26:24.056 INFO:teuthology.orchestra.run.smithi187.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T01:26:24.056 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T01:26:24.056 INFO:teuthology.orchestra.run.smithi187.stdout:/run/user/1000 2024-04-25T01:26:24.056 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph 2024-04-25T01:26:24.056 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T01:26:24.056 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T01:26:24.056 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T01:26:24.057 INFO:teuthology.orchestra.run.smithi187.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T01:26:24.057 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns 2024-04-25T01:26:24.057 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:26:24.057 INFO:teuthology.orchestra.run.smithi187.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:26:24.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:24.057 DEBUG:teuthology.orchestra.run.smithi187:> ls /sys/fs/fuse/connections 2024-04-25T01:26:24.070 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi187.stderr:2024-04-25T01:26:24.090+0000 7febdf87d480 -1 init, newargv = 0x564a1249a420 newargc=15 2024-04-25T01:26:24.070 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi187.stderr:ceph-fuse[41261]: starting ceph client 2024-04-25T01:26:24.075 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi187.stderr:ceph-fuse[41261]: starting fuse 2024-04-25T01:26:24.084 INFO:teuthology.orchestra.run.smithi187.stdout:43 2024-04-25T01:26:24.084 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T01:26:24.084 DEBUG:teuthology.orchestra.run.smithi187:> sudo stdin-killer -- python3 -c ' 2024-04-25T01:26:24.084 DEBUG:teuthology.orchestra.run.smithi187:> import glob 2024-04-25T01:26:24.085 DEBUG:teuthology.orchestra.run.smithi187:> import re 2024-04-25T01:26:24.085 DEBUG:teuthology.orchestra.run.smithi187:> import os 2024-04-25T01:26:24.085 DEBUG:teuthology.orchestra.run.smithi187:> import subprocess 2024-04-25T01:26:24.085 DEBUG:teuthology.orchestra.run.smithi187:> 2024-04-25T01:26:24.085 DEBUG:teuthology.orchestra.run.smithi187:> def _find_admin_socket(client_name): 2024-04-25T01:26:24.085 DEBUG:teuthology.orchestra.run.smithi187:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T01:26:24.085 DEBUG:teuthology.orchestra.run.smithi187:> files = glob.glob(asok_path) 2024-04-25T01:26:24.085 DEBUG:teuthology.orchestra.run.smithi187:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T01:26:24.085 DEBUG:teuthology.orchestra.run.smithi187:> 2024-04-25T01:26:24.085 DEBUG:teuthology.orchestra.run.smithi187:> # Given a non-glob path, it better be there 2024-04-25T01:26:24.085 DEBUG:teuthology.orchestra.run.smithi187:> if "*" not in asok_path: 2024-04-25T01:26:24.085 DEBUG:teuthology.orchestra.run.smithi187:> assert(len(files) == 1) 2024-04-25T01:26:24.085 DEBUG:teuthology.orchestra.run.smithi187:> return files[0] 2024-04-25T01:26:24.085 DEBUG:teuthology.orchestra.run.smithi187:> 2024-04-25T01:26:24.085 DEBUG:teuthology.orchestra.run.smithi187:> for f in files: 2024-04-25T01:26:24.086 DEBUG:teuthology.orchestra.run.smithi187:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T01:26:24.086 DEBUG:teuthology.orchestra.run.smithi187:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T01:26:24.086 DEBUG:teuthology.orchestra.run.smithi187:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T01:26:24.086 DEBUG:teuthology.orchestra.run.smithi187:> contents = proc_f.read() 2024-04-25T01:26:24.086 DEBUG:teuthology.orchestra.run.smithi187:> if mountpoint in contents: 2024-04-25T01:26:24.086 DEBUG:teuthology.orchestra.run.smithi187:> return f 2024-04-25T01:26:24.086 DEBUG:teuthology.orchestra.run.smithi187:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T01:26:24.086 DEBUG:teuthology.orchestra.run.smithi187:> 2024-04-25T01:26:24.086 DEBUG:teuthology.orchestra.run.smithi187:> print(_find_admin_socket("client.1")) 2024-04-25T01:26:24.086 DEBUG:teuthology.orchestra.run.smithi187:> ' 2024-04-25T01:26:24.200 INFO:teuthology.orchestra.run.smithi187.stdout:/var/run/ceph/ceph-client.1.41261.asok 2024-04-25T01:26:24.203 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T01:26:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:26:24.211 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.41261.asok 2024-04-25T01:26:24.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:24.211 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.41261.asok status 2024-04-25T01:26:24.345 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2024-04-25T01:26:24.345 INFO:teuthology.orchestra.run.smithi187.stdout: "metadata": { 2024-04-25T01:26:24.346 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T01:26:24.346 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T01:26:24.346 INFO:teuthology.orchestra.run.smithi187.stdout: "entity_id": "1", 2024-04-25T01:26:24.346 INFO:teuthology.orchestra.run.smithi187.stdout: "hostname": "smithi187", 2024-04-25T01:26:24.346 INFO:teuthology.orchestra.run.smithi187.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T01:26:24.346 INFO:teuthology.orchestra.run.smithi187.stdout: "pid": "41261", 2024-04-25T01:26:24.346 INFO:teuthology.orchestra.run.smithi187.stdout: "root": "/" 2024-04-25T01:26:24.348 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-25T01:26:24.348 INFO:teuthology.orchestra.run.smithi187.stdout: "dentry_count": 0, 2024-04-25T01:26:24.348 INFO:teuthology.orchestra.run.smithi187.stdout: "dentry_pinned_count": 0, 2024-04-25T01:26:24.348 INFO:teuthology.orchestra.run.smithi187.stdout: "id": 4735, 2024-04-25T01:26:24.348 INFO:teuthology.orchestra.run.smithi187.stdout: "inst": { 2024-04-25T01:26:24.348 INFO:teuthology.orchestra.run.smithi187.stdout: "name": { 2024-04-25T01:26:24.348 INFO:teuthology.orchestra.run.smithi187.stdout: "type": "client", 2024-04-25T01:26:24.349 INFO:teuthology.orchestra.run.smithi187.stdout: "num": 4735 2024-04-25T01:26:24.349 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-25T01:26:24.349 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": { 2024-04-25T01:26:24.349 INFO:teuthology.orchestra.run.smithi187.stdout: "type": "v1", 2024-04-25T01:26:24.349 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:26:24.349 INFO:teuthology.orchestra.run.smithi187.stdout: "nonce": 2341136281 2024-04-25T01:26:24.349 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-25T01:26:24.349 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-25T01:26:24.350 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": { 2024-04-25T01:26:24.350 INFO:teuthology.orchestra.run.smithi187.stdout: "type": "v1", 2024-04-25T01:26:24.350 INFO:teuthology.orchestra.run.smithi187.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:26:24.350 INFO:teuthology.orchestra.run.smithi187.stdout: "nonce": 2341136281 2024-04-25T01:26:24.350 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-25T01:26:24.350 INFO:teuthology.orchestra.run.smithi187.stdout: "inst_str": "client.4735 192.168.0.1:0/2341136281", 2024-04-25T01:26:24.350 INFO:teuthology.orchestra.run.smithi187.stdout: "addr_str": "192.168.0.1:0/2341136281", 2024-04-25T01:26:24.350 INFO:teuthology.orchestra.run.smithi187.stdout: "inode_count": 1, 2024-04-25T01:26:24.350 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_epoch": 11, 2024-04-25T01:26:24.351 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_epoch": 24, 2024-04-25T01:26:24.351 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_epoch_barrier": 0, 2024-04-25T01:26:24.351 INFO:teuthology.orchestra.run.smithi187.stdout: "blocklisted": false, 2024-04-25T01:26:24.351 INFO:teuthology.orchestra.run.smithi187.stdout: "fs_name": "b" 2024-04-25T01:26:24.351 INFO:teuthology.orchestra.run.smithi187.stdout:} 2024-04-25T01:26:24.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:24.353 DEBUG:teuthology.orchestra.run.smithi112:> stat --file-system '--printf=%T 2024-04-25T01:26:24.353 DEBUG:teuthology.orchestra.run.smithi112:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T01:26:24.379 INFO:teuthology.orchestra.run.smithi112.stdout:fuseblk 2024-04-25T01:26:24.379 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T01:26:24.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:24.379 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T01:26:24.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:24.456 DEBUG:teuthology.orchestra.run.smithi187:> stat --file-system '--printf=%T 2024-04-25T01:26:24.456 DEBUG:teuthology.orchestra.run.smithi187:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T01:26:24.479 INFO:teuthology.orchestra.run.smithi187.stdout:fuseblk 2024-04-25T01:26:24.479 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T01:26:24.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:26:24.479 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T01:26:24.551 INFO:teuthology.run_tasks:Running task mon_thrash... 2024-04-25T01:26:24.611 INFO:tasks.mon_thrash:Beginning mon_thrash... 2024-04-25T01:26:24.611 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:26:24.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.741+0000 7f134f2a5640 1 Processor -- start 2024-04-25T01:26:24.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.741+0000 7f134f2a5640 1 -- start start 2024-04-25T01:26:24.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.742+0000 7f134f2a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f134814c3f0 0x7f13480c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:24.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.742+0000 7f134f2a5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f13480ac3b0 0x7f1348155f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:24.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.742+0000 7f134f2a5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13481529b0 con 0x7f13481524a0 2024-04-25T01:26:24.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.742+0000 7f134f2a5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f1348155a90 con 0x7f13480ac3b0 2024-04-25T01:26:24.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.742+0000 7f134f2a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1348152c70 con 0x7f134814c3f0 2024-04-25T01:26:24.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.742+0000 7f134c840640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f13480ac3b0 0x7f1348155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:24.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.742+0000 7f134d842640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f13481524a0 0x7f13480c3b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53464/0 (socket says 172.21.15.112:53464) 2024-04-25T01:26:24.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.742+0000 7f134d842640 1 -- 172.21.15.112:0/3029907815 learned_addr learned my addr 172.21.15.112:0/3029907815 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:24.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.742+0000 7f134d041640 1 --2- 172.21.15.112:0/3029907815 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f134814c3f0 0x7f13480c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:24.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.743+0000 7f134c840640 1 -- 172.21.15.112:0/3029907815 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f134814c3f0 msgr2=0x7f13480c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:24.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.743+0000 7f134c840640 1 --2- 172.21.15.112:0/3029907815 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f134814c3f0 0x7f13480c2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:24.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.743+0000 7f134c840640 1 -- 172.21.15.112:0/3029907815 >> v1:172.21.15.112:6789/0 conn(0x7f13481524a0 legacy=0x7f13480c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:24.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.743+0000 7f133ffff640 1 -- 172.21.15.112:0/3029907815 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1682560750 0 0) 0x7f13481529b0 con 0x7f13481524a0 2024-04-25T01:26:24.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.743+0000 7f134c840640 1 -- 172.21.15.112:0/3029907815 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1348152f50 con 0x7f13480ac3b0 2024-04-25T01:26:24.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.743+0000 7f134d041640 1 --2- 172.21.15.112:0/3029907815 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f134814c3f0 0x7f13480c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:24.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.743+0000 7f134c840640 1 --2- 172.21.15.112:0/3029907815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13480ac3b0 0x7f1348155f20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f13380097a0 tx=0x7f133802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be6a2cfb5fcf567 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.743+0000 7f133ffff640 1 -- 172.21.15.112:0/3029907815 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338037070 con 0x7f13480ac3b0 2024-04-25T01:26:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.743+0000 7f133ffff640 1 -- 172.21.15.112:0/3029907815 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133802ae50 con 0x7f13480ac3b0 2024-04-25T01:26:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.743+0000 7f133ffff640 1 -- 172.21.15.112:0/3029907815 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338032660 con 0x7f13480ac3b0 2024-04-25T01:26:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.744+0000 7f134f2a5640 1 -- 172.21.15.112:0/3029907815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13480ac3b0 msgr2=0x7f1348155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.744+0000 7f134f2a5640 1 --2- 172.21.15.112:0/3029907815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13480ac3b0 0x7f1348155f20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f13380097a0 tx=0x7f133802a100 comp rx=0 tx=0).stop 2024-04-25T01:26:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.744+0000 7f134f2a5640 1 -- 172.21.15.112:0/3029907815 shutdown_connections 2024-04-25T01:26:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.744+0000 7f134f2a5640 1 --2- 172.21.15.112:0/3029907815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13480ac3b0 0x7f1348155f20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.744+0000 7f134f2a5640 1 --2- 172.21.15.112:0/3029907815 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f134814c3f0 0x7f13480c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.744+0000 7f134f2a5640 1 -- 172.21.15.112:0/3029907815 >> 172.21.15.112:0/3029907815 conn(0x7f134806b160 msgr2=0x7f134806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.744+0000 7f134f2a5640 1 -- 172.21.15.112:0/3029907815 shutdown_connections 2024-04-25T01:26:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.744+0000 7f134f2a5640 1 -- 172.21.15.112:0/3029907815 wait complete. 2024-04-25T01:26:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.745+0000 7f134f2a5640 1 Processor -- start 2024-04-25T01:26:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.745+0000 7f134f2a5640 1 -- start start 2024-04-25T01:26:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.746+0000 7f134f2a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13480ac3b0 0x7f13481ef000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.746+0000 7f134f2a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f134814c3f0 0x7f13481e0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.746+0000 7f134f2a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13481524a0 0x7f1348142970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.746+0000 7f134f2a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f134814c7d0 con 0x7f134814c3f0 2024-04-25T01:26:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.746+0000 7f134f2a5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1348070d60 con 0x7f13480ac3b0 2024-04-25T01:26:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.746+0000 7f134f2a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f134814c950 con 0x7f13481524a0 2024-04-25T01:26:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.746+0000 7f134c840640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f134814c3f0 0x7f13481e0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.746+0000 7f134c840640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f134814c3f0 0x7f13481e0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46188/0 (socket says 172.21.15.112:46188) 2024-04-25T01:26:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.746+0000 7f134c840640 1 -- 172.21.15.112:0/3141050429 learned_addr learned my addr 172.21.15.112:0/3141050429 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.746+0000 7f134d842640 1 --2- 172.21.15.112:0/3141050429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13481524a0 0x7f1348142970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.746+0000 7f134d041640 1 --2- 172.21.15.112:0/3141050429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13480ac3b0 0x7f13481ef000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.746+0000 7f134c840640 1 -- 172.21.15.112:0/3141050429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13481524a0 msgr2=0x7f1348142970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.746+0000 7f134c840640 1 --2- 172.21.15.112:0/3141050429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13481524a0 0x7f1348142970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.746+0000 7f134c840640 1 -- 172.21.15.112:0/3141050429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13480ac3b0 msgr2=0x7f13481ef000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.746+0000 7f134c840640 1 --2- 172.21.15.112:0/3141050429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13480ac3b0 0x7f13481ef000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.746+0000 7f134c840640 1 -- 172.21.15.112:0/3141050429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1348153ef0 con 0x7f134814c3f0 2024-04-25T01:26:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.746+0000 7f134d842640 1 --2- 172.21.15.112:0/3141050429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13481524a0 0x7f1348142970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.747+0000 7f134c840640 1 --2- 172.21.15.112:0/3141050429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f134814c3f0 0x7f13481e0180 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1338000c00 tx=0x7f13380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.747+0000 7f133dffb640 1 -- 172.21.15.112:0/3141050429 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338041070 con 0x7f134814c3f0 2024-04-25T01:26:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.747+0000 7f133dffb640 1 -- 172.21.15.112:0/3141050429 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133803c670 con 0x7f134814c3f0 2024-04-25T01:26:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.747+0000 7f134f2a5640 1 -- 172.21.15.112:0/3141050429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13481541d0 con 0x7f134814c3f0 2024-04-25T01:26:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.747+0000 7f133dffb640 1 -- 172.21.15.112:0/3141050429 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338037040 con 0x7f134814c3f0 2024-04-25T01:26:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.747+0000 7f134f2a5640 1 -- 172.21.15.112:0/3141050429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f134806ebc0 con 0x7f134814c3f0 2024-04-25T01:26:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.748+0000 7f134f2a5640 1 -- 172.21.15.112:0/3141050429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13480ad420 con 0x7f134814c3f0 2024-04-25T01:26:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.749+0000 7f133dffb640 1 -- 172.21.15.112:0/3141050429 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1338044070 con 0x7f134814c3f0 2024-04-25T01:26:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.749+0000 7f133dffb640 1 --2- 172.21.15.112:0/3141050429 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f132c06be30 0x7f132c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.749+0000 7f133dffb640 1 -- 172.21.15.112:0/3141050429 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f13380ab8c0 con 0x7f134814c3f0 2024-04-25T01:26:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.749+0000 7f134d041640 1 --2- 172.21.15.112:0/3141050429 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f132c06be30 0x7f132c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.749+0000 7f134d041640 1 --2- 172.21.15.112:0/3141050429 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f132c06be30 0x7f132c06e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1344003fd0 tx=0x7f1344003ce0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.753+0000 7f133dffb640 1 -- 172.21.15.112:0/3141050429 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f13480ad420 con 0x7f134814c3f0 2024-04-25T01:26:24.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.929+0000 7f134f2a5640 1 -- 172.21.15.112:0/3141050429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13480682a0 con 0x7f134814c3f0 2024-04-25T01:26:24.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.930+0000 7f133dffb640 1 -- 172.21.15.112:0/3141050429 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20367 (secure 0 0 0) 0x7f133807a0f0 con 0x7f134814c3f0 2024-04-25T01:26:24.910 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:24.910 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:26:03.405271+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-25T01:25:59.478275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-25T01:26:00.719358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:26:24.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.932+0000 7f134f2a5640 1 -- 172.21.15.112:0/3141050429 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f132c06be30 msgr2=0x7f132c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:24.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.932+0000 7f134f2a5640 1 --2- 172.21.15.112:0/3141050429 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f132c06be30 0x7f132c06e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1344003fd0 tx=0x7f1344003ce0 comp rx=0 tx=0).stop 2024-04-25T01:26:24.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.932+0000 7f134f2a5640 1 -- 172.21.15.112:0/3141050429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f134814c3f0 msgr2=0x7f13481e0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:24.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.932+0000 7f134f2a5640 1 --2- 172.21.15.112:0/3141050429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f134814c3f0 0x7f13481e0180 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1338000c00 tx=0x7f13380046c0 comp rx=0 tx=0).stop 2024-04-25T01:26:24.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.932+0000 7f134f2a5640 1 -- 172.21.15.112:0/3141050429 shutdown_connections 2024-04-25T01:26:24.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.932+0000 7f134f2a5640 1 --2- 172.21.15.112:0/3141050429 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f132c06be30 0x7f132c06e2f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:24.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.932+0000 7f134f2a5640 1 --2- 172.21.15.112:0/3141050429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13481524a0 0x7f1348142970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:24.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.932+0000 7f134f2a5640 1 --2- 172.21.15.112:0/3141050429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f134814c3f0 0x7f13481e0180 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:24.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.932+0000 7f134f2a5640 1 --2- 172.21.15.112:0/3141050429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13480ac3b0 0x7f13481ef000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:24.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.932+0000 7f134f2a5640 1 -- 172.21.15.112:0/3141050429 >> 172.21.15.112:0/3141050429 conn(0x7f134806b160 msgr2=0x7f13480c2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:24.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.932+0000 7f134f2a5640 1 -- 172.21.15.112:0/3141050429 shutdown_connections 2024-04-25T01:26:24.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:24.933+0000 7f134f2a5640 1 -- 172.21.15.112:0/3141050429 wait complete. 2024-04-25T01:26:24.924 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:25:40.679119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-04-25T01:25:50.593045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-04-25T01:25:51.664930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-04-25T01:25:59.478275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-04-25T01:26:00.719358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:26:24.924 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:26:25.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.045+0000 7f1f7c815640 1 Processor -- start 2024-04-25T01:26:25.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.046+0000 7f1f7c815640 1 -- start start 2024-04-25T01:26:25.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.046+0000 7f1f7c815640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7814fbe0 0x7f1f780c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.046+0000 7f1f7c815640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1f780c2e60 0x7f1f780726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.046+0000 7f1f7c815640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f7806f290 con 0x7f1f7806d460 2024-04-25T01:26:25.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.046+0000 7f1f7c815640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1f7814ffc0 con 0x7f1f780c2e60 2024-04-25T01:26:25.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.046+0000 7f1f7c815640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1f781533a0 con 0x7f1f7814fbe0 2024-04-25T01:26:25.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.047+0000 7f1f76d9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1f780c2e60 0x7f1f780726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.047+0000 7f1f7659c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7814fbe0 0x7f1f780c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.047+0000 7f1f76d9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1f780c2e60 0x7f1f780726c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:33078/0 (socket says 172.21.15.112:33078) 2024-04-25T01:26:25.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.047+0000 7f1f76d9d640 1 -- 172.21.15.112:0/518229125 learned_addr learned my addr 172.21.15.112:0/518229125 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:25.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.047+0000 7f1f75d9b640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f1f7806d460 0x7f1f78071fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:40696/0 (socket says 172.21.15.112:40696) 2024-04-25T01:26:25.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.047+0000 7f1f76d9d640 1 -- 172.21.15.112:0/518229125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7814fbe0 msgr2=0x7f1f780c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:25.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.047+0000 7f1f76d9d640 1 --2- 172.21.15.112:0/518229125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7814fbe0 0x7f1f780c2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.047+0000 7f1f76d9d640 1 -- 172.21.15.112:0/518229125 >> v1:172.21.15.187:6789/0 conn(0x7f1f7806d460 legacy=0x7f1f78071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.047+0000 7f1f76d9d640 1 -- 172.21.15.112:0/518229125 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f7806efb0 con 0x7f1f780c2e60 2024-04-25T01:26:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.048+0000 7f1f76d9d640 1 --2- 172.21.15.112:0/518229125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f780c2e60 0x7f1f780726c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009e50 tx=0x7f1f7002a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa356599a671410 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.048+0000 7f1f7559a640 1 -- 172.21.15.112:0/518229125 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f70036070 con 0x7f1f780c2e60 2024-04-25T01:26:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.048+0000 7f1f7559a640 1 -- 172.21.15.112:0/518229125 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f70031600 con 0x7f1f780c2e60 2024-04-25T01:26:25.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.048+0000 7f1f7559a640 1 -- 172.21.15.112:0/518229125 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f700319d0 con 0x7f1f780c2e60 2024-04-25T01:26:25.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.048+0000 7f1f7c815640 1 -- 172.21.15.112:0/518229125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f780c2e60 msgr2=0x7f1f780726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.048+0000 7f1f7c815640 1 --2- 172.21.15.112:0/518229125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f780c2e60 0x7f1f780726c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009e50 tx=0x7f1f7002a680 comp rx=0 tx=0).stop 2024-04-25T01:26:25.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.049+0000 7f1f7c815640 1 -- 172.21.15.112:0/518229125 shutdown_connections 2024-04-25T01:26:25.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.049+0000 7f1f7c815640 1 --2- 172.21.15.112:0/518229125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f780c2e60 0x7f1f780726c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.049+0000 7f1f7c815640 1 --2- 172.21.15.112:0/518229125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7814fbe0 0x7f1f780c2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.049+0000 7f1f7c815640 1 -- 172.21.15.112:0/518229125 >> 172.21.15.112:0/518229125 conn(0x7f1f780c3a10 msgr2=0x7f1f780c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:25.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.049+0000 7f1f7c815640 1 -- 172.21.15.112:0/518229125 shutdown_connections 2024-04-25T01:26:25.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.049+0000 7f1f7c815640 1 -- 172.21.15.112:0/518229125 wait complete. 2024-04-25T01:26:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.050+0000 7f1f7c815640 1 Processor -- start 2024-04-25T01:26:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.050+0000 7f1f7c815640 1 -- start start 2024-04-25T01:26:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.050+0000 7f1f7c815640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7806d460 0x7f1f781e7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.050+0000 7f1f7c815640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f780c2e60 0x7f1f7806dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.050+0000 7f1f7c815640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f7814fbe0 0x7f1f780ab590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.050+0000 7f1f7c815640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1f78072c00 con 0x7f1f780c2e60 2024-04-25T01:26:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.050+0000 7f1f7c815640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1f78153030 con 0x7f1f7814fbe0 2024-04-25T01:26:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.051+0000 7f1f7c815640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1f780c3700 con 0x7f1f7806d460 2024-04-25T01:26:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.051+0000 7f1f75d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f780c2e60 0x7f1f7806dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.051+0000 7f1f75d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f780c2e60 0x7f1f7806dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46196/0 (socket says 172.21.15.112:46196) 2024-04-25T01:26:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.051+0000 7f1f75d9b640 1 -- 172.21.15.112:0/2029593985 learned_addr learned my addr 172.21.15.112:0/2029593985 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:25.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.051+0000 7f1f76d9d640 1 --2- 172.21.15.112:0/2029593985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f7814fbe0 0x7f1f780ab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.051+0000 7f1f7659c640 1 --2- 172.21.15.112:0/2029593985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7806d460 0x7f1f781e7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.051+0000 7f1f75d9b640 1 -- 172.21.15.112:0/2029593985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7806d460 msgr2=0x7f1f781e7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.051+0000 7f1f75d9b640 1 --2- 172.21.15.112:0/2029593985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7806d460 0x7f1f781e7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.051+0000 7f1f75d9b640 1 -- 172.21.15.112:0/2029593985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f7814fbe0 msgr2=0x7f1f780ab590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.051+0000 7f1f75d9b640 1 --2- 172.21.15.112:0/2029593985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f7814fbe0 0x7f1f780ab590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.051+0000 7f1f75d9b640 1 -- 172.21.15.112:0/2029593985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f781dea40 con 0x7f1f780c2e60 2024-04-25T01:26:25.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.051+0000 7f1f76d9d640 1 --2- 172.21.15.112:0/2029593985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f7814fbe0 0x7f1f780ab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:26:25.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.051+0000 7f1f75d9b640 1 --2- 172.21.15.112:0/2029593985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f780c2e60 0x7f1f7806dae0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800c4d0 tx=0x7f1f6800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:25.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.051+0000 7f1f66ffd640 1 -- 172.21.15.112:0/2029593985 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f68018070 con 0x7f1f780c2e60 2024-04-25T01:26:25.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.051+0000 7f1f66ffd640 1 -- 172.21.15.112:0/2029593985 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f68007480 con 0x7f1f780c2e60 2024-04-25T01:26:25.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.051+0000 7f1f7c815640 1 -- 172.21.15.112:0/2029593985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f781e86c0 con 0x7f1f780c2e60 2024-04-25T01:26:25.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.051+0000 7f1f66ffd640 1 -- 172.21.15.112:0/2029593985 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f680049e0 con 0x7f1f780c2e60 2024-04-25T01:26:25.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.051+0000 7f1f7c815640 1 -- 172.21.15.112:0/2029593985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f781e8c80 con 0x7f1f780c2e60 2024-04-25T01:26:25.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.052+0000 7f1f7c815640 1 -- 172.21.15.112:0/2029593985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f7804f500 con 0x7f1f780c2e60 2024-04-25T01:26:25.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.053+0000 7f1f66ffd640 1 -- 172.21.15.112:0/2029593985 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f6801b070 con 0x7f1f780c2e60 2024-04-25T01:26:25.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.053+0000 7f1f66ffd640 1 --2- 172.21.15.112:0/2029593985 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1f5806be30 0x7f1f5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.053+0000 7f1f66ffd640 1 -- 172.21.15.112:0/2029593985 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f1f6808a390 con 0x7f1f780c2e60 2024-04-25T01:26:25.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.053+0000 7f1f7659c640 1 --2- 172.21.15.112:0/2029593985 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1f5806be30 0x7f1f5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.054+0000 7f1f7659c640 1 --2- 172.21.15.112:0/2029593985 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1f5806be30 0x7f1f5806e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009aa0 tx=0x7f1f6c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:25.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.058+0000 7f1f66ffd640 1 -- 172.21.15.112:0/2029593985 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1f68007670 con 0x7f1f780c2e60 2024-04-25T01:26:25.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.229+0000 7f1f7c815640 1 -- 172.21.15.112:0/2029593985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1f781e8e80 con 0x7f1f780c2e60 2024-04-25T01:26:25.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.229+0000 7f1f66ffd640 1 -- 172.21.15.112:0/2029593985 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f1f68058450 con 0x7f1f780c2e60 2024-04-25T01:26:25.209 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T01:26:25.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.231+0000 7f1f7c815640 1 -- 172.21.15.112:0/2029593985 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1f5806be30 msgr2=0x7f1f5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.231+0000 7f1f7c815640 1 --2- 172.21.15.112:0/2029593985 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1f5806be30 0x7f1f5806e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009aa0 tx=0x7f1f6c009190 comp rx=0 tx=0).stop 2024-04-25T01:26:25.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.231+0000 7f1f7c815640 1 -- 172.21.15.112:0/2029593985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f780c2e60 msgr2=0x7f1f7806dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.231+0000 7f1f7c815640 1 --2- 172.21.15.112:0/2029593985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f780c2e60 0x7f1f7806dae0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800c4d0 tx=0x7f1f6800c990 comp rx=0 tx=0).stop 2024-04-25T01:26:25.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.231+0000 7f1f7c815640 1 -- 172.21.15.112:0/2029593985 shutdown_connections 2024-04-25T01:26:25.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.231+0000 7f1f7c815640 1 --2- 172.21.15.112:0/2029593985 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1f5806be30 0x7f1f5806e2f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.231+0000 7f1f7c815640 1 --2- 172.21.15.112:0/2029593985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f7814fbe0 0x7f1f780ab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.231+0000 7f1f7c815640 1 --2- 172.21.15.112:0/2029593985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f780c2e60 0x7f1f7806dae0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.231+0000 7f1f7c815640 1 --2- 172.21.15.112:0/2029593985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f7806d460 0x7f1f781e7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.231+0000 7f1f7c815640 1 -- 172.21.15.112:0/2029593985 >> 172.21.15.112:0/2029593985 conn(0x7f1f780c3a10 msgr2=0x7f1f781527d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:25.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.231+0000 7f1f7c815640 1 -- 172.21.15.112:0/2029593985 shutdown_connections 2024-04-25T01:26:25.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.231+0000 7f1f7c815640 1 -- 172.21.15.112:0/2029593985 wait complete. 2024-04-25T01:26:25.222 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-04-25T01:26:25.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.342+0000 7f9f93f90640 1 Processor -- start 2024-04-25T01:26:25.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.343+0000 7f9f93f90640 1 -- start start 2024-04-25T01:26:25.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.343+0000 7f9f93f90640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9f8c06d460 0x7f9f8c071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.343+0000 7f9f93f90640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9f8c0724f0 0x7f9f8c072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.343+0000 7f9f93f90640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f8c06f290 con 0x7f9f8c14fbe0 2024-04-25T01:26:25.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.343+0000 7f9f93f90640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f9f8c14ffc0 con 0x7f9f8c0724f0 2024-04-25T01:26:25.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.343+0000 7f9f93f90640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9f8c1533a0 con 0x7f9f8c06d460 2024-04-25T01:26:25.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.344+0000 7f9f9252d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9f8c0724f0 0x7f9f8c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.344+0000 7f9f9252d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9f8c0724f0 0x7f9f8c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46202/0 (socket says 172.21.15.112:46202) 2024-04-25T01:26:25.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.344+0000 7f9f9252d640 1 -- 172.21.15.112:0/1734471514 learned_addr learned my addr 172.21.15.112:0/1734471514 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:25.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.344+0000 7f9f9152b640 1 --2- 172.21.15.112:0/1734471514 >> v2:172.21.15.187:3300/0 conn(0x7f9f8c06d460 0x7f9f8c071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.344+0000 7f9f9252d640 1 -- 172.21.15.112:0/1734471514 >> v2:172.21.15.187:3300/0 conn(0x7f9f8c06d460 msgr2=0x7f9f8c071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:25.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.344+0000 7f9f9252d640 1 --2- 172.21.15.112:0/1734471514 >> v2:172.21.15.187:3300/0 conn(0x7f9f8c06d460 0x7f9f8c071fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.344+0000 7f9f9252d640 1 -- 172.21.15.112:0/1734471514 >> v1:172.21.15.187:6789/0 conn(0x7f9f8c14fbe0 legacy=0x7f9f8c0c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:25.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.344+0000 7f9f9252d640 1 -- 172.21.15.112:0/1734471514 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c06efb0 con 0x7f9f8c0724f0 2024-04-25T01:26:25.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.344+0000 7f9f9252d640 1 --2- 172.21.15.112:0/1734471514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f8c0724f0 0x7f9f8c072900 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0c13a0 tx=0x7f9f8402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3ecd66195076 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:25.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.345+0000 7f9f90d2a640 1 -- 172.21.15.112:0/1734471514 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84036070 con 0x7f9f8c0724f0 2024-04-25T01:26:25.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.345+0000 7f9f90d2a640 1 -- 172.21.15.112:0/1734471514 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f84031600 con 0x7f9f8c0724f0 2024-04-25T01:26:25.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.345+0000 7f9f90d2a640 1 -- 172.21.15.112:0/1734471514 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f840319d0 con 0x7f9f8c0724f0 2024-04-25T01:26:25.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.345+0000 7f9f93f90640 1 -- 172.21.15.112:0/1734471514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f8c0724f0 msgr2=0x7f9f8c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.345+0000 7f9f93f90640 1 --2- 172.21.15.112:0/1734471514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f8c0724f0 0x7f9f8c072900 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0c13a0 tx=0x7f9f8402a660 comp rx=0 tx=0).stop 2024-04-25T01:26:25.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.345+0000 7f9f93f90640 1 -- 172.21.15.112:0/1734471514 shutdown_connections 2024-04-25T01:26:25.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.345+0000 7f9f93f90640 1 --2- 172.21.15.112:0/1734471514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f8c0724f0 0x7f9f8c072900 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.345+0000 7f9f93f90640 1 --2- 172.21.15.112:0/1734471514 >> v2:172.21.15.187:3300/0 conn(0x7f9f8c06d460 0x7f9f8c071fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.345+0000 7f9f93f90640 1 -- 172.21.15.112:0/1734471514 >> 172.21.15.112:0/1734471514 conn(0x7f9f8c0c3a10 msgr2=0x7f9f8c0c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:25.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.345+0000 7f9f93f90640 1 -- 172.21.15.112:0/1734471514 shutdown_connections 2024-04-25T01:26:25.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.345+0000 7f9f93f90640 1 -- 172.21.15.112:0/1734471514 wait complete. 2024-04-25T01:26:25.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.346+0000 7f9f93f90640 1 Processor -- start 2024-04-25T01:26:25.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.346+0000 7f9f93f90640 1 -- start start 2024-04-25T01:26:25.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.347+0000 7f9f93f90640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f8c06d460 0x7f9f8c1eb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.347+0000 7f9f93f90640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f8c0724f0 0x7f9f8c1492c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.347+0000 7f9f93f90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f8c14fbe0 0x7f9f8c0c2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.347+0000 7f9f93f90640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c0c3030 con 0x7f9f8c14fbe0 2024-04-25T01:26:25.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.347+0000 7f9f93f90640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c153030 con 0x7f9f8c06d460 2024-04-25T01:26:25.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.347+0000 7f9f93f90640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9f8c0c31b0 con 0x7f9f8c0724f0 2024-04-25T01:26:25.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.347+0000 7f9f9252d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f8c14fbe0 0x7f9f8c0c2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.347+0000 7f9f9252d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f8c14fbe0 0x7f9f8c0c2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46210/0 (socket says 172.21.15.112:46210) 2024-04-25T01:26:25.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.347+0000 7f9f9252d640 1 -- 172.21.15.112:0/754213220 learned_addr learned my addr 172.21.15.112:0/754213220 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:25.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.347+0000 7f9f9152b640 1 --2- 172.21.15.112:0/754213220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f8c0724f0 0x7f9f8c1492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.347+0000 7f9f91d2c640 1 --2- 172.21.15.112:0/754213220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f8c06d460 0x7f9f8c1eb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.347+0000 7f9f9252d640 1 -- 172.21.15.112:0/754213220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f8c0724f0 msgr2=0x7f9f8c1492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.347+0000 7f9f9252d640 1 --2- 172.21.15.112:0/754213220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f8c0724f0 0x7f9f8c1492c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.347+0000 7f9f9252d640 1 -- 172.21.15.112:0/754213220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f8c06d460 msgr2=0x7f9f8c1eb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.347+0000 7f9f9252d640 1 --2- 172.21.15.112:0/754213220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f8c06d460 0x7f9f8c1eb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.347+0000 7f9f9252d640 1 -- 172.21.15.112:0/754213220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c146c70 con 0x7f9f8c14fbe0 2024-04-25T01:26:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.347+0000 7f9f9152b640 1 --2- 172.21.15.112:0/754213220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f8c0724f0 0x7f9f8c1492c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.347+0000 7f9f91d2c640 1 --2- 172.21.15.112:0/754213220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f8c06d460 0x7f9f8c1eb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.348+0000 7f9f9252d640 1 --2- 172.21.15.112:0/754213220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f8c14fbe0 0x7f9f8c0c2240 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009f40 tx=0x7f9f84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.348+0000 7f9f7e7fc640 1 -- 172.21.15.112:0/754213220 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84036050 con 0x7f9f8c14fbe0 2024-04-25T01:26:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.348+0000 7f9f7e7fc640 1 -- 172.21.15.112:0/754213220 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f84042070 con 0x7f9f8c14fbe0 2024-04-25T01:26:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.348+0000 7f9f93f90640 1 -- 172.21.15.112:0/754213220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8c146f50 con 0x7f9f8c14fbe0 2024-04-25T01:26:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.348+0000 7f9f7e7fc640 1 -- 172.21.15.112:0/754213220 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84009510 con 0x7f9f8c14fbe0 2024-04-25T01:26:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.348+0000 7f9f93f90640 1 -- 172.21.15.112:0/754213220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f8c1e8700 con 0x7f9f8c14fbe0 2024-04-25T01:26:25.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.349+0000 7f9f93f90640 1 -- 172.21.15.112:0/754213220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f8c04f500 con 0x7f9f8c14fbe0 2024-04-25T01:26:25.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.349+0000 7f9f7e7fc640 1 -- 172.21.15.112:0/754213220 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9f84007bd0 con 0x7f9f8c14fbe0 2024-04-25T01:26:25.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.350+0000 7f9f7e7fc640 1 --2- 172.21.15.112:0/754213220 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9f7006be30 0x7f9f7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.350+0000 7f9f91d2c640 1 --2- 172.21.15.112:0/754213220 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9f7006be30 0x7f9f7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.350+0000 7f9f7e7fc640 1 -- 172.21.15.112:0/754213220 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f9f840ac100 con 0x7f9f8c14fbe0 2024-04-25T01:26:25.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.350+0000 7f9f91d2c640 1 --2- 172.21.15.112:0/754213220 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9f7006be30 0x7f9f7006e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9f80006fd0 tx=0x7f9f80008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:25.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.354+0000 7f9f7e7fc640 1 -- 172.21.15.112:0/754213220 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9f8407a810 con 0x7f9f8c14fbe0 2024-04-25T01:26:25.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.534+0000 7f9f93f90640 1 -- 172.21.15.112:0/754213220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) -- 0x7f9f8c14e500 con 0x7f9f8c14fbe0 2024-04-25T01:26:25.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.534+0000 7f9f7e7fc640 1 -- 172.21.15.112:0/754213220 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7f9f8407a110 con 0x7f9f8c14fbe0 2024-04-25T01:26:25.514 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T01:26:25.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.536+0000 7f9f93f90640 1 -- 172.21.15.112:0/754213220 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9f7006be30 msgr2=0x7f9f7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.536+0000 7f9f93f90640 1 --2- 172.21.15.112:0/754213220 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9f7006be30 0x7f9f7006e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9f80006fd0 tx=0x7f9f80008040 comp rx=0 tx=0).stop 2024-04-25T01:26:25.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.536+0000 7f9f93f90640 1 -- 172.21.15.112:0/754213220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f8c14fbe0 msgr2=0x7f9f8c0c2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.536+0000 7f9f93f90640 1 --2- 172.21.15.112:0/754213220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f8c14fbe0 0x7f9f8c0c2240 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009f40 tx=0x7f9f84004a40 comp rx=0 tx=0).stop 2024-04-25T01:26:25.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.536+0000 7f9f93f90640 1 -- 172.21.15.112:0/754213220 shutdown_connections 2024-04-25T01:26:25.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.536+0000 7f9f93f90640 1 --2- 172.21.15.112:0/754213220 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9f7006be30 0x7f9f7006e2f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.536+0000 7f9f93f90640 1 --2- 172.21.15.112:0/754213220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f8c14fbe0 0x7f9f8c0c2240 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.536+0000 7f9f93f90640 1 --2- 172.21.15.112:0/754213220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f8c0724f0 0x7f9f8c1492c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.536+0000 7f9f93f90640 1 --2- 172.21.15.112:0/754213220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f8c06d460 0x7f9f8c1eb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.536+0000 7f9f93f90640 1 -- 172.21.15.112:0/754213220 >> 172.21.15.112:0/754213220 conn(0x7f9f8c0c3a10 msgr2=0x7f9f8c06d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:25.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.537+0000 7f9f93f90640 1 -- 172.21.15.112:0/754213220 shutdown_connections 2024-04-25T01:26:25.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.537+0000 7f9f93f90640 1 -- 172.21.15.112:0/754213220 wait complete. 2024-04-25T01:26:25.526 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-04-25T01:26:25.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.646+0000 7fe0bdad6640 1 Processor -- start 2024-04-25T01:26:25.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.646+0000 7fe0bdad6640 1 -- start start 2024-04-25T01:26:25.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.647+0000 7fe0bdad6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe0b806d460 0x7fe0b8071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.647+0000 7fe0bdad6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe0b80724f0 0x7fe0b8072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.647+0000 7fe0bdad6640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0b806f290 con 0x7fe0b814fbe0 2024-04-25T01:26:25.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.647+0000 7fe0bdad6640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fe0b814ffc0 con 0x7fe0b806d460 2024-04-25T01:26:25.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.647+0000 7fe0bdad6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe0b81533a0 con 0x7fe0b80724f0 2024-04-25T01:26:25.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.647+0000 7fe0b77fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe0b814fbe0 0x7fe0b80c2920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53472/0 (socket says 172.21.15.112:53472) 2024-04-25T01:26:25.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.647+0000 7fe0b77fe640 1 -- 172.21.15.112:0/1015173100 learned_addr learned my addr 172.21.15.112:0/1015173100 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:25.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.647+0000 7fe0aeffd640 1 --2- 172.21.15.112:0/1015173100 >> v2:172.21.15.112:3300/0 conn(0x7fe0b806d460 0x7fe0b8071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.647+0000 7fe0b7fff640 1 --2- 172.21.15.112:0/1015173100 >> v2:172.21.15.187:3300/0 conn(0x7fe0b80724f0 0x7fe0b8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.648+0000 7fe0b6ffd640 1 -- 172.21.15.112:0/1015173100 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4290984176 0 0) 0x7fe0b806f290 con 0x7fe0b814fbe0 2024-04-25T01:26:25.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.648+0000 7fe0b6ffd640 1 -- 172.21.15.112:0/1015173100 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0a4003520 con 0x7fe0b814fbe0 2024-04-25T01:26:25.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.648+0000 7fe0aeffd640 1 -- 172.21.15.112:0/1015173100 >> v2:172.21.15.187:3300/0 conn(0x7fe0b80724f0 msgr2=0x7fe0b8072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:25.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.648+0000 7fe0aeffd640 1 --2- 172.21.15.112:0/1015173100 >> v2:172.21.15.187:3300/0 conn(0x7fe0b80724f0 0x7fe0b8072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.648+0000 7fe0aeffd640 1 -- 172.21.15.112:0/1015173100 >> v1:172.21.15.112:6789/0 conn(0x7fe0b814fbe0 legacy=0x7fe0b80c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.648+0000 7fe0aeffd640 1 -- 172.21.15.112:0/1015173100 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b806efb0 con 0x7fe0b806d460 2024-04-25T01:26:25.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.648+0000 7fe0b7fff640 1 --2- 172.21.15.112:0/1015173100 >> v2:172.21.15.187:3300/0 conn(0x7fe0b80724f0 0x7fe0b8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:25.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.648+0000 7fe0aeffd640 1 --2- 172.21.15.112:0/1015173100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b806d460 0x7fe0b8071fb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe0b806c4b0 tx=0x7fe0a002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f749bfa43a111a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:25.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.648+0000 7fe0b6ffd640 1 -- 172.21.15.112:0/1015173100 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0037070 con 0x7fe0b806d460 2024-04-25T01:26:25.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.648+0000 7fe0b6ffd640 1 -- 172.21.15.112:0/1015173100 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a0032440 con 0x7fe0b806d460 2024-04-25T01:26:25.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.649+0000 7fe0b6ffd640 1 -- 172.21.15.112:0/1015173100 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a002ae60 con 0x7fe0b806d460 2024-04-25T01:26:25.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.649+0000 7fe0bdad6640 1 -- 172.21.15.112:0/1015173100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b806d460 msgr2=0x7fe0b8071fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.649+0000 7fe0bdad6640 1 --2- 172.21.15.112:0/1015173100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b806d460 0x7fe0b8071fb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe0b806c4b0 tx=0x7fe0a002a110 comp rx=0 tx=0).stop 2024-04-25T01:26:25.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.649+0000 7fe0bdad6640 1 -- 172.21.15.112:0/1015173100 shutdown_connections 2024-04-25T01:26:25.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.649+0000 7fe0bdad6640 1 --2- 172.21.15.112:0/1015173100 >> v2:172.21.15.187:3300/0 conn(0x7fe0b80724f0 0x7fe0b8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.649+0000 7fe0bdad6640 1 --2- 172.21.15.112:0/1015173100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b806d460 0x7fe0b8071fb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.649+0000 7fe0bdad6640 1 -- 172.21.15.112:0/1015173100 >> 172.21.15.112:0/1015173100 conn(0x7fe0b80c3a10 msgr2=0x7fe0b80c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:25.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.649+0000 7fe0bdad6640 1 -- 172.21.15.112:0/1015173100 shutdown_connections 2024-04-25T01:26:25.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.649+0000 7fe0bdad6640 1 -- 172.21.15.112:0/1015173100 wait complete. 2024-04-25T01:26:25.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.650+0000 7fe0bdad6640 1 Processor -- start 2024-04-25T01:26:25.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.650+0000 7fe0bdad6640 1 -- start start 2024-04-25T01:26:25.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.651+0000 7fe0bdad6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0b806d460 0x7fe0b8145930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.651+0000 7fe0bdad6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0b80724f0 0x7fe0b81ecda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.651+0000 7fe0bdad6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b814fbe0 0x7fe0b80c2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.651+0000 7fe0bdad6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b80c3030 con 0x7fe0b814fbe0 2024-04-25T01:26:25.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.651+0000 7fe0bdad6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b8153030 con 0x7fe0b806d460 2024-04-25T01:26:25.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.651+0000 7fe0bdad6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe0b80c31b0 con 0x7fe0b80724f0 2024-04-25T01:26:25.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.651+0000 7fe0b7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b814fbe0 0x7fe0b80c2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.651+0000 7fe0b7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b814fbe0 0x7fe0b80c2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46234/0 (socket says 172.21.15.112:46234) 2024-04-25T01:26:25.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.651+0000 7fe0b7fff640 1 -- 172.21.15.112:0/1713734995 learned_addr learned my addr 172.21.15.112:0/1713734995 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:25.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.651+0000 7fe0aeffd640 1 --2- 172.21.15.112:0/1713734995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0b80724f0 0x7fe0b81ecda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.651+0000 7fe0b77fe640 1 --2- 172.21.15.112:0/1713734995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0b806d460 0x7fe0b8145930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.651+0000 7fe0b7fff640 1 -- 172.21.15.112:0/1713734995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0b80724f0 msgr2=0x7fe0b81ecda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.651+0000 7fe0b7fff640 1 --2- 172.21.15.112:0/1713734995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0b80724f0 0x7fe0b81ecda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.651+0000 7fe0b7fff640 1 -- 172.21.15.112:0/1713734995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0b806d460 msgr2=0x7fe0b8145930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.651+0000 7fe0b7fff640 1 --2- 172.21.15.112:0/1713734995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0b806d460 0x7fe0b8145930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.651+0000 7fe0b7fff640 1 -- 172.21.15.112:0/1713734995 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b8146900 con 0x7fe0b814fbe0 2024-04-25T01:26:25.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.651+0000 7fe0aeffd640 1 --2- 172.21.15.112:0/1713734995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0b80724f0 0x7fe0b81ecda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:25.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.652+0000 7fe0b7fff640 1 --2- 172.21.15.112:0/1713734995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b814fbe0 0x7fe0b80c2530 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0007f20 tx=0x7fe0b000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:25.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.652+0000 7fe0b4ff9640 1 -- 172.21.15.112:0/1713734995 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b001a070 con 0x7fe0b814fbe0 2024-04-25T01:26:25.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.652+0000 7fe0b4ff9640 1 -- 172.21.15.112:0/1713734995 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b0003b90 con 0x7fe0b814fbe0 2024-04-25T01:26:25.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.652+0000 7fe0bdad6640 1 -- 172.21.15.112:0/1713734995 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b81494c0 con 0x7fe0b814fbe0 2024-04-25T01:26:25.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.652+0000 7fe0b4ff9640 1 -- 172.21.15.112:0/1713734995 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0004030 con 0x7fe0b814fbe0 2024-04-25T01:26:25.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.652+0000 7fe0bdad6640 1 -- 172.21.15.112:0/1713734995 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b81ec5a0 con 0x7fe0b814fbe0 2024-04-25T01:26:25.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.653+0000 7fe0bdad6640 1 -- 172.21.15.112:0/1713734995 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0b804f500 con 0x7fe0b814fbe0 2024-04-25T01:26:25.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.653+0000 7fe0b4ff9640 1 -- 172.21.15.112:0/1713734995 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe0b001d070 con 0x7fe0b814fbe0 2024-04-25T01:26:25.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.654+0000 7fe0b4ff9640 1 --2- 172.21.15.112:0/1713734995 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe09806be30 0x7fe09806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.654+0000 7fe0b77fe640 1 --2- 172.21.15.112:0/1713734995 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe09806be30 0x7fe09806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.654+0000 7fe0b4ff9640 1 -- 172.21.15.112:0/1713734995 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fe0b008b4b0 con 0x7fe0b814fbe0 2024-04-25T01:26:25.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.654+0000 7fe0b77fe640 1 --2- 172.21.15.112:0/1713734995 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe09806be30 0x7fe09806e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8007fd0 tx=0x7fe0a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:25.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.658+0000 7fe0b4ff9640 1 -- 172.21.15.112:0/1713734995 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe0b0010070 con 0x7fe0b814fbe0 2024-04-25T01:26:25.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.831+0000 7fe0bdad6640 1 -- 172.21.15.112:0/1713734995 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) -- 0x7fe0b80c4540 con 0x7fe0b814fbe0 2024-04-25T01:26:25.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.832+0000 7fe0b4ff9640 1 -- 172.21.15.112:0/1713734995 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7fe0b0059b30 con 0x7fe0b814fbe0 2024-04-25T01:26:25.811 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T01:26:25.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.833+0000 7fe0bdad6640 1 -- 172.21.15.112:0/1713734995 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe09806be30 msgr2=0x7fe09806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.833+0000 7fe0bdad6640 1 --2- 172.21.15.112:0/1713734995 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe09806be30 0x7fe09806e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8007fd0 tx=0x7fe0a8009040 comp rx=0 tx=0).stop 2024-04-25T01:26:25.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.833+0000 7fe0bdad6640 1 -- 172.21.15.112:0/1713734995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b814fbe0 msgr2=0x7fe0b80c2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.833+0000 7fe0bdad6640 1 --2- 172.21.15.112:0/1713734995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b814fbe0 0x7fe0b80c2530 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0007f20 tx=0x7fe0b000d560 comp rx=0 tx=0).stop 2024-04-25T01:26:25.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.833+0000 7fe0bdad6640 1 -- 172.21.15.112:0/1713734995 shutdown_connections 2024-04-25T01:26:25.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.833+0000 7fe0bdad6640 1 --2- 172.21.15.112:0/1713734995 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe09806be30 0x7fe09806e2f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.833+0000 7fe0bdad6640 1 --2- 172.21.15.112:0/1713734995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b814fbe0 0x7fe0b80c2530 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.833+0000 7fe0bdad6640 1 --2- 172.21.15.112:0/1713734995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0b80724f0 0x7fe0b81ecda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.833+0000 7fe0bdad6640 1 --2- 172.21.15.112:0/1713734995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0b806d460 0x7fe0b8145930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.834+0000 7fe0bdad6640 1 -- 172.21.15.112:0/1713734995 >> 172.21.15.112:0/1713734995 conn(0x7fe0b80c3a10 msgr2=0x7fe0b806e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:25.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.834+0000 7fe0bdad6640 1 -- 172.21.15.112:0/1713734995 shutdown_connections 2024-04-25T01:26:25.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.834+0000 7fe0bdad6640 1 -- 172.21.15.112:0/1713734995 wait complete. 2024-04-25T01:26:25.824 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-04-25T01:26:25.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.944+0000 7febf2476640 1 Processor -- start 2024-04-25T01:26:25.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.944+0000 7febf2476640 1 -- start start 2024-04-25T01:26:25.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.945+0000 7febf2476640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febec14fbe0 0x7febec0c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.945+0000 7febf2476640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7febec0c2e60 0x7febec0726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.945+0000 7febf2476640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febec06f290 con 0x7febec06d460 2024-04-25T01:26:25.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.945+0000 7febf2476640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7febec14ffc0 con 0x7febec0c2e60 2024-04-25T01:26:25.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.945+0000 7febf2476640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7febec1533a0 con 0x7febec14fbe0 2024-04-25T01:26:25.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.945+0000 7febeb7fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7febec06d460 0x7febec071fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53482/0 (socket says 172.21.15.112:53482) 2024-04-25T01:26:25.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.945+0000 7febeb7fe640 1 -- 172.21.15.112:0/2876324201 learned_addr learned my addr 172.21.15.112:0/2876324201 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:25.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.945+0000 7febf0a13640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7febec0c2e60 0x7febec0726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.945+0000 7febebfff640 1 --2- 172.21.15.112:0/2876324201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febec14fbe0 0x7febec0c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.945+0000 7febf0a13640 1 -- 172.21.15.112:0/2876324201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febec14fbe0 msgr2=0x7febec0c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:25.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.945+0000 7febf0a13640 1 --2- 172.21.15.112:0/2876324201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febec14fbe0 0x7febec0c2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.945+0000 7febf0a13640 1 -- 172.21.15.112:0/2876324201 >> v1:172.21.15.112:6789/0 conn(0x7febec06d460 legacy=0x7febec071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.945+0000 7febeaffd640 1 -- 172.21.15.112:0/2876324201 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3009404563 0 0) 0x7febec06f290 con 0x7febec06d460 2024-04-25T01:26:25.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.945+0000 7febf0a13640 1 -- 172.21.15.112:0/2876324201 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febec06efb0 con 0x7febec0c2e60 2024-04-25T01:26:25.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.946+0000 7febf0a13640 1 --2- 172.21.15.112:0/2876324201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febec0c2e60 0x7febec0726c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7febe4009d90 tx=0x7febe402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c115f5d78d29ad57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:25.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.946+0000 7febeaffd640 1 -- 172.21.15.112:0/2876324201 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4031600 con 0x7febec0c2e60 2024-04-25T01:26:25.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.946+0000 7febeaffd640 1 -- 172.21.15.112:0/2876324201 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe40317a0 con 0x7febec0c2e60 2024-04-25T01:26:25.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.946+0000 7febeaffd640 1 -- 172.21.15.112:0/2876324201 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4031b70 con 0x7febec0c2e60 2024-04-25T01:26:25.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.946+0000 7febf2476640 1 -- 172.21.15.112:0/2876324201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febec0c2e60 msgr2=0x7febec0726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.946+0000 7febf2476640 1 --2- 172.21.15.112:0/2876324201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febec0c2e60 0x7febec0726c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7febe4009d90 tx=0x7febe402a5d0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.946+0000 7febf2476640 1 -- 172.21.15.112:0/2876324201 shutdown_connections 2024-04-25T01:26:25.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.946+0000 7febf2476640 1 --2- 172.21.15.112:0/2876324201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febec0c2e60 0x7febec0726c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.946+0000 7febf2476640 1 --2- 172.21.15.112:0/2876324201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febec14fbe0 0x7febec0c2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.946+0000 7febf2476640 1 -- 172.21.15.112:0/2876324201 >> 172.21.15.112:0/2876324201 conn(0x7febec0c3a10 msgr2=0x7febec0c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:25.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.946+0000 7febf2476640 1 -- 172.21.15.112:0/2876324201 shutdown_connections 2024-04-25T01:26:25.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.947+0000 7febf2476640 1 -- 172.21.15.112:0/2876324201 wait complete. 2024-04-25T01:26:25.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.948+0000 7febf2476640 1 Processor -- start 2024-04-25T01:26:25.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.948+0000 7febf2476640 1 -- start start 2024-04-25T01:26:25.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.948+0000 7febf2476640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febec06d460 0x7febec1e9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.948+0000 7febf2476640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febec0c2e60 0x7febec151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.948+0000 7febf2476640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febec14fbe0 0x7febec1474f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.948+0000 7febf2476640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7febec072c00 con 0x7febec14fbe0 2024-04-25T01:26:25.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.948+0000 7febf2476640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7febec153030 con 0x7febec0c2e60 2024-04-25T01:26:25.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.948+0000 7febf2476640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7febec0c3700 con 0x7febec06d460 2024-04-25T01:26:25.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.948+0000 7febf0a13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febec14fbe0 0x7febec1474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.949+0000 7febf0a13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febec14fbe0 0x7febec1474f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46264/0 (socket says 172.21.15.112:46264) 2024-04-25T01:26:25.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.949+0000 7febf0a13640 1 -- 172.21.15.112:0/1217006153 learned_addr learned my addr 172.21.15.112:0/1217006153 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:25.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.949+0000 7febebfff640 1 --2- 172.21.15.112:0/1217006153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febec06d460 0x7febec1e9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.949+0000 7febf0a13640 1 -- 172.21.15.112:0/1217006153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febec06d460 msgr2=0x7febec1e9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.949+0000 7febf0a13640 1 --2- 172.21.15.112:0/1217006153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febec06d460 0x7febec1e9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.949+0000 7febf0a13640 1 -- 172.21.15.112:0/1217006153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febec0c2e60 msgr2=0x7febec151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.949+0000 7febeb7fe640 1 --2- 172.21.15.112:0/1217006153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febec0c2e60 0x7febec151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.949+0000 7febf0a13640 1 --2- 172.21.15.112:0/1217006153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febec0c2e60 0x7febec151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.949+0000 7febf0a13640 1 -- 172.21.15.112:0/1217006153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febec156670 con 0x7febec14fbe0 2024-04-25T01:26:25.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.949+0000 7febeb7fe640 1 --2- 172.21.15.112:0/1217006153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febec0c2e60 0x7febec151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:25.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.949+0000 7febf0a13640 1 --2- 172.21.15.112:0/1217006153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febec14fbe0 0x7febec1474f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7febe4009eb0 tx=0x7febe4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:25.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.949+0000 7febe8ff9640 1 -- 172.21.15.112:0/1217006153 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4040070 con 0x7febec14fbe0 2024-04-25T01:26:25.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.949+0000 7febe8ff9640 1 -- 172.21.15.112:0/1217006153 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe4030b60 con 0x7febec14fbe0 2024-04-25T01:26:25.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.949+0000 7febe8ff9640 1 -- 172.21.15.112:0/1217006153 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4009450 con 0x7febec14fbe0 2024-04-25T01:26:25.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.949+0000 7febf2476640 1 -- 172.21.15.112:0/1217006153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febec1ea350 con 0x7febec14fbe0 2024-04-25T01:26:25.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.949+0000 7febf2476640 1 -- 172.21.15.112:0/1217006153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febec1ed720 con 0x7febec14fbe0 2024-04-25T01:26:25.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.950+0000 7febf2476640 1 -- 172.21.15.112:0/1217006153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febec04f500 con 0x7febec14fbe0 2024-04-25T01:26:25.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.951+0000 7febe8ff9640 1 -- 172.21.15.112:0/1217006153 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7febe4039040 con 0x7febec14fbe0 2024-04-25T01:26:25.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.951+0000 7febe8ff9640 1 --2- 172.21.15.112:0/1217006153 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7febd806be30 0x7febd806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.951+0000 7febe8ff9640 1 -- 172.21.15.112:0/1217006153 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7febe40ac720 con 0x7febec14fbe0 2024-04-25T01:26:25.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.951+0000 7febebfff640 1 --2- 172.21.15.112:0/1217006153 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7febd806be30 0x7febd806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.951+0000 7febebfff640 1 --2- 172.21.15.112:0/1217006153 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7febd806be30 0x7febd806e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7febdc009ad0 tx=0x7febdc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:25.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:25.955+0000 7febe8ff9640 1 -- 172.21.15.112:0/1217006153 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7febe407ae30 con 0x7febec14fbe0 2024-04-25T01:26:26.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.129+0000 7febf2476640 1 -- 172.21.15.112:0/1217006153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) -- 0x7febec0c3470 con 0x7febec14fbe0 2024-04-25T01:26:26.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.129+0000 7febe8ff9640 1 -- 172.21.15.112:0/1217006153 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7febe407a730 con 0x7febec14fbe0 2024-04-25T01:26:26.109 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T01:26:26.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.131+0000 7febf2476640 1 -- 172.21.15.112:0/1217006153 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7febd806be30 msgr2=0x7febd806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:26.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.131+0000 7febf2476640 1 --2- 172.21.15.112:0/1217006153 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7febd806be30 0x7febd806e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7febdc009ad0 tx=0x7febdc008040 comp rx=0 tx=0).stop 2024-04-25T01:26:26.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.131+0000 7febf2476640 1 -- 172.21.15.112:0/1217006153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febec14fbe0 msgr2=0x7febec1474f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:26.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.131+0000 7febf2476640 1 --2- 172.21.15.112:0/1217006153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febec14fbe0 0x7febec1474f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7febe4009eb0 tx=0x7febe4004a40 comp rx=0 tx=0).stop 2024-04-25T01:26:26.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.131+0000 7febf2476640 1 -- 172.21.15.112:0/1217006153 shutdown_connections 2024-04-25T01:26:26.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.131+0000 7febf2476640 1 --2- 172.21.15.112:0/1217006153 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7febd806be30 0x7febd806e2f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.131+0000 7febf2476640 1 --2- 172.21.15.112:0/1217006153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febec14fbe0 0x7febec1474f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.131+0000 7febf2476640 1 --2- 172.21.15.112:0/1217006153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febec0c2e60 0x7febec151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.131+0000 7febf2476640 1 --2- 172.21.15.112:0/1217006153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febec06d460 0x7febec1e9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.131+0000 7febf2476640 1 -- 172.21.15.112:0/1217006153 >> 172.21.15.112:0/1217006153 conn(0x7febec0c3a10 msgr2=0x7febec0c1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:26.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.131+0000 7febf2476640 1 -- 172.21.15.112:0/1217006153 shutdown_connections 2024-04-25T01:26:26.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.131+0000 7febf2476640 1 -- 172.21.15.112:0/1217006153 wait complete. 2024-04-25T01:26:26.121 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-04-25T01:26:26.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.241+0000 7fe2057c4640 1 Processor -- start 2024-04-25T01:26:26.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.242+0000 7fe2057c4640 1 -- start start 2024-04-25T01:26:26.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.242+0000 7fe2057c4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe2000c9250 0x7fe2000c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:26.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.242+0000 7fe2057c4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe2000ca370 0x7fe20006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:26.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.242+0000 7fe2057c4640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2001534a0 con 0x7fe2000ca8a0 2024-04-25T01:26:26.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.242+0000 7fe2057c4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fe200156990 con 0x7fe2000ca370 2024-04-25T01:26:26.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.242+0000 7fe2057c4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe200156500 con 0x7fe2000c9250 2024-04-25T01:26:26.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.242+0000 7fe1fe7fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe2000ca370 0x7fe20006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:26.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.242+0000 7fe1fe7fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe2000ca370 0x7fe20006bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46270/0 (socket says 172.21.15.112:46270) 2024-04-25T01:26:26.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.242+0000 7fe1fe7fc640 1 -- 172.21.15.112:0/2185542418 learned_addr learned my addr 172.21.15.112:0/2185542418 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:26.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.243+0000 7fe1feffd640 1 --2- 172.21.15.112:0/2185542418 >> v2:172.21.15.187:3300/0 conn(0x7fe2000c9250 0x7fe2000c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:26.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.243+0000 7fe1fe7fc640 1 -- 172.21.15.112:0/2185542418 >> v2:172.21.15.187:3300/0 conn(0x7fe2000c9250 msgr2=0x7fe2000c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:26.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.243+0000 7fe1fe7fc640 1 --2- 172.21.15.112:0/2185542418 >> v2:172.21.15.187:3300/0 conn(0x7fe2000c9250 0x7fe2000c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.243+0000 7fe1fe7fc640 1 -- 172.21.15.112:0/2185542418 >> v1:172.21.15.187:6789/0 conn(0x7fe2000ca8a0 legacy=0x7fe20006c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:26.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.243+0000 7fe1fe7fc640 1 -- 172.21.15.112:0/2185542418 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2000cae30 con 0x7fe2000ca370 2024-04-25T01:26:26.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.243+0000 7fe1feffd640 1 --2- 172.21.15.112:0/2185542418 >> v2:172.21.15.187:3300/0 conn(0x7fe2000c9250 0x7fe2000c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:26.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.243+0000 7fe1fe7fc640 1 --2- 172.21.15.112:0/2185542418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2000ca370 0x7fe20006bd10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe200066f10 tx=0x7fe1f402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d699a1ee2649470f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:26.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.243+0000 7fe1fdffb640 1 -- 172.21.15.112:0/2185542418 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4037070 con 0x7fe2000ca370 2024-04-25T01:26:26.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.244+0000 7fe1fdffb640 1 -- 172.21.15.112:0/2185542418 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f4032440 con 0x7fe2000ca370 2024-04-25T01:26:26.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.244+0000 7fe1fdffb640 1 -- 172.21.15.112:0/2185542418 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4032810 con 0x7fe2000ca370 2024-04-25T01:26:26.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.244+0000 7fe2057c4640 1 -- 172.21.15.112:0/2185542418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2000ca370 msgr2=0x7fe20006bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:26.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.244+0000 7fe2057c4640 1 --2- 172.21.15.112:0/2185542418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2000ca370 0x7fe20006bd10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe200066f10 tx=0x7fe1f402a1a0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.244+0000 7fe2057c4640 1 -- 172.21.15.112:0/2185542418 shutdown_connections 2024-04-25T01:26:26.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.244+0000 7fe2057c4640 1 --2- 172.21.15.112:0/2185542418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2000ca370 0x7fe20006bd10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.244+0000 7fe2057c4640 1 --2- 172.21.15.112:0/2185542418 >> v2:172.21.15.187:3300/0 conn(0x7fe2000c9250 0x7fe2000c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.244+0000 7fe2057c4640 1 -- 172.21.15.112:0/2185542418 >> 172.21.15.112:0/2185542418 conn(0x7fe2000cb310 msgr2=0x7fe2000cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:26.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.244+0000 7fe2057c4640 1 -- 172.21.15.112:0/2185542418 shutdown_connections 2024-04-25T01:26:26.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.244+0000 7fe2057c4640 1 -- 172.21.15.112:0/2185542418 wait complete. 2024-04-25T01:26:26.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.245+0000 7fe2057c4640 1 Processor -- start 2024-04-25T01:26:26.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.245+0000 7fe2057c4640 1 -- start start 2024-04-25T01:26:26.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.246+0000 7fe2057c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2000c9250 0x7fe2001e0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:26.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.246+0000 7fe2057c4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2000ca370 0x7fe2001585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:26.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.246+0000 7fe2057c4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2000ca8a0 0x7fe20014fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:26.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.246+0000 7fe2057c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe200155eb0 con 0x7fe2000c9250 2024-04-25T01:26:26.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.246+0000 7fe2057c4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe200155bd0 con 0x7fe2000ca370 2024-04-25T01:26:26.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.246+0000 7fe2057c4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe200156190 con 0x7fe2000ca8a0 2024-04-25T01:26:26.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.246+0000 7fe1feffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2000c9250 0x7fe2001e0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:26.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.246+0000 7fe1feffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2000c9250 0x7fe2001e0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46272/0 (socket says 172.21.15.112:46272) 2024-04-25T01:26:26.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.246+0000 7fe1feffd640 1 -- 172.21.15.112:0/3413902343 learned_addr learned my addr 172.21.15.112:0/3413902343 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:26.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.246+0000 7fe1ff7fe640 1 --2- 172.21.15.112:0/3413902343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2000ca8a0 0x7fe20014fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:26.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.246+0000 7fe1fe7fc640 1 --2- 172.21.15.112:0/3413902343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2000ca370 0x7fe2001585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:26.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.246+0000 7fe1feffd640 1 -- 172.21.15.112:0/3413902343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2000ca8a0 msgr2=0x7fe20014fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:26.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.246+0000 7fe1feffd640 1 --2- 172.21.15.112:0/3413902343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2000ca8a0 0x7fe20014fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.246+0000 7fe1feffd640 1 -- 172.21.15.112:0/3413902343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2000ca370 msgr2=0x7fe2001585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:26.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.246+0000 7fe1feffd640 1 --2- 172.21.15.112:0/3413902343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2000ca370 0x7fe2001585e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.246+0000 7fe1feffd640 1 -- 172.21.15.112:0/3413902343 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2001df9c0 con 0x7fe2000c9250 2024-04-25T01:26:26.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.246+0000 7fe1ff7fe640 1 --2- 172.21.15.112:0/3413902343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2000ca8a0 0x7fe20014fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:26.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.247+0000 7fe1feffd640 1 --2- 172.21.15.112:0/3413902343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2000c9250 0x7fe2001e0be0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000cf60 tx=0x7fe1f000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:26.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.247+0000 7fe1e77fe640 1 -- 172.21.15.112:0/3413902343 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f001a070 con 0x7fe2000c9250 2024-04-25T01:26:26.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.247+0000 7fe1e77fe640 1 -- 172.21.15.112:0/3413902343 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f000d840 con 0x7fe2000c9250 2024-04-25T01:26:26.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.247+0000 7fe1e77fe640 1 -- 172.21.15.112:0/3413902343 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f000dc10 con 0x7fe2000c9250 2024-04-25T01:26:26.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.247+0000 7fe2057c4640 1 -- 172.21.15.112:0/3413902343 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2001dfca0 con 0x7fe2000c9250 2024-04-25T01:26:26.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.247+0000 7fe2057c4640 1 -- 172.21.15.112:0/3413902343 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2000c5460 con 0x7fe2000c9250 2024-04-25T01:26:26.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.248+0000 7fe2057c4640 1 -- 172.21.15.112:0/3413902343 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe20004f500 con 0x7fe2000c9250 2024-04-25T01:26:26.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.248+0000 7fe1e77fe640 1 -- 172.21.15.112:0/3413902343 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe1f001d070 con 0x7fe2000c9250 2024-04-25T01:26:26.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.249+0000 7fe1e77fe640 1 --2- 172.21.15.112:0/3413902343 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe1ec06be60 0x7fe1ec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:26.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.249+0000 7fe1e77fe640 1 -- 172.21.15.112:0/3413902343 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fe1f008b860 con 0x7fe2000c9250 2024-04-25T01:26:26.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.249+0000 7fe1fe7fc640 1 --2- 172.21.15.112:0/3413902343 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe1ec06be60 0x7fe1ec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:26.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.249+0000 7fe1fe7fc640 1 --2- 172.21.15.112:0/3413902343 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe1ec06be60 0x7fe1ec06e320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4034000 tx=0x7fe1f402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:26.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.253+0000 7fe1e77fe640 1 -- 172.21.15.112:0/3413902343 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe1f0059920 con 0x7fe2000c9250 2024-04-25T01:26:26.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.432+0000 7fe2057c4640 1 -- 172.21.15.112:0/3413902343 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) -- 0x7fe2000c9030 con 0x7fe2000c9250 2024-04-25T01:26:26.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.433+0000 7fe1e77fe640 1 -- 172.21.15.112:0/3413902343 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7fe1f005dc20 con 0x7fe2000c9250 2024-04-25T01:26:26.412 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T01:26:26.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.435+0000 7fe2057c4640 1 -- 172.21.15.112:0/3413902343 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe1ec06be60 msgr2=0x7fe1ec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:26.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.435+0000 7fe2057c4640 1 --2- 172.21.15.112:0/3413902343 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe1ec06be60 0x7fe1ec06e320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4034000 tx=0x7fe1f402e000 comp rx=0 tx=0).stop 2024-04-25T01:26:26.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.435+0000 7fe2057c4640 1 -- 172.21.15.112:0/3413902343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2000c9250 msgr2=0x7fe2001e0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:26.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.435+0000 7fe2057c4640 1 --2- 172.21.15.112:0/3413902343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2000c9250 0x7fe2001e0be0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000cf60 tx=0x7fe1f000cf90 comp rx=0 tx=0).stop 2024-04-25T01:26:26.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.435+0000 7fe2057c4640 1 -- 172.21.15.112:0/3413902343 shutdown_connections 2024-04-25T01:26:26.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.435+0000 7fe2057c4640 1 --2- 172.21.15.112:0/3413902343 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe1ec06be60 0x7fe1ec06e320 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.435+0000 7fe2057c4640 1 --2- 172.21.15.112:0/3413902343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2000ca8a0 0x7fe20014fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.435+0000 7fe2057c4640 1 --2- 172.21.15.112:0/3413902343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2000ca370 0x7fe2001585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.435+0000 7fe2057c4640 1 --2- 172.21.15.112:0/3413902343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2000c9250 0x7fe2001e0be0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.435+0000 7fe2057c4640 1 -- 172.21.15.112:0/3413902343 >> 172.21.15.112:0/3413902343 conn(0x7fe2000cb310 msgr2=0x7fe2001541e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:26.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.435+0000 7fe2057c4640 1 -- 172.21.15.112:0/3413902343 shutdown_connections 2024-04-25T01:26:26.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.435+0000 7fe2057c4640 1 -- 172.21.15.112:0/3413902343 wait complete. 2024-04-25T01:26:26.425 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2024-04-25T01:26:26.425 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:26:26.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.542+0000 7f11b4128640 1 Processor -- start 2024-04-25T01:26:26.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.542+0000 7f11b4128640 1 -- start start 2024-04-25T01:26:26.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.543+0000 7f11b4128640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11ac1592c0 0x7f11ac0c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:26.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.543+0000 7f11b4128640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f11ac1596a0 0x7f11ac152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:26.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.543+0000 7f11b4128640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11ac159b90 con 0x7f11ac0c72b0 2024-04-25T01:26:26.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.543+0000 7f11b4128640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f11ac0c6e20 con 0x7f11ac1596a0 2024-04-25T01:26:26.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.543+0000 7f11b4128640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f11ac06c970 con 0x7f11ac1592c0 2024-04-25T01:26:26.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.543+0000 7f11b16c3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f11ac1596a0 0x7f11ac152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:26.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.543+0000 7f11b26c5640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f11ac0c72b0 0x7f11ac155ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53496/0 (socket says 172.21.15.112:53496) 2024-04-25T01:26:26.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.543+0000 7f11b26c5640 1 -- 172.21.15.112:0/1253914175 learned_addr learned my addr 172.21.15.112:0/1253914175 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:26.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.543+0000 7f11b16c3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f11ac1596a0 0x7f11ac152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46276/0 (socket says 172.21.15.112:46276) 2024-04-25T01:26:26.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.544+0000 7f11b1ec4640 1 --2- 172.21.15.112:0/1253914175 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11ac1592c0 0x7f11ac0c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:26.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.544+0000 7f11b16c3640 1 -- 172.21.15.112:0/1253914175 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11ac1592c0 msgr2=0x7f11ac0c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:26.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.544+0000 7f11b16c3640 1 --2- 172.21.15.112:0/1253914175 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11ac1592c0 0x7f11ac0c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.544+0000 7f11b16c3640 1 -- 172.21.15.112:0/1253914175 >> v1:172.21.15.112:6789/0 conn(0x7f11ac0c72b0 legacy=0x7f11ac155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:26.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.544+0000 7f11b0ec2640 1 -- 172.21.15.112:0/1253914175 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 860332759 0 0) 0x7f11ac159b90 con 0x7f11ac0c72b0 2024-04-25T01:26:26.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.544+0000 7f11b16c3640 1 -- 172.21.15.112:0/1253914175 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ac06cc50 con 0x7f11ac1596a0 2024-04-25T01:26:26.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.544+0000 7f11b1ec4640 1 --2- 172.21.15.112:0/1253914175 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11ac1592c0 0x7f11ac0c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:26.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.544+0000 7f11b16c3640 1 --2- 172.21.15.112:0/1253914175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ac1596a0 0x7f11ac152d00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f119c0097a0 tx=0x7f119c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cac8749531a7948 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:26.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.544+0000 7f11b0ec2640 1 -- 172.21.15.112:0/1253914175 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c037070 con 0x7f11ac1596a0 2024-04-25T01:26:26.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.544+0000 7f11b0ec2640 1 -- 172.21.15.112:0/1253914175 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f119c02adb0 con 0x7f11ac1596a0 2024-04-25T01:26:26.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.544+0000 7f11b0ec2640 1 -- 172.21.15.112:0/1253914175 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c032660 con 0x7f11ac1596a0 2024-04-25T01:26:26.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.545+0000 7f11b4128640 1 -- 172.21.15.112:0/1253914175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ac1596a0 msgr2=0x7f11ac152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:26.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.545+0000 7f11b4128640 1 --2- 172.21.15.112:0/1253914175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ac1596a0 0x7f11ac152d00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f119c0097a0 tx=0x7f119c02a060 comp rx=0 tx=0).stop 2024-04-25T01:26:26.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.545+0000 7f11b4128640 1 -- 172.21.15.112:0/1253914175 shutdown_connections 2024-04-25T01:26:26.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.545+0000 7f11b4128640 1 --2- 172.21.15.112:0/1253914175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ac1596a0 0x7f11ac152d00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.545+0000 7f11b4128640 1 --2- 172.21.15.112:0/1253914175 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11ac1592c0 0x7f11ac0c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.545+0000 7f11b4128640 1 -- 172.21.15.112:0/1253914175 >> 172.21.15.112:0/1253914175 conn(0x7f11ac0cb170 msgr2=0x7f11ac0cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:26.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.545+0000 7f11b4128640 1 -- 172.21.15.112:0/1253914175 shutdown_connections 2024-04-25T01:26:26.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.545+0000 7f11b4128640 1 -- 172.21.15.112:0/1253914175 wait complete. 2024-04-25T01:26:26.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.546+0000 7f11b4128640 1 Processor -- start 2024-04-25T01:26:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.546+0000 7f11b4128640 1 -- start start 2024-04-25T01:26:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.547+0000 7f11b4128640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11ac0c72b0 0x7f11ac14fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.547+0000 7f11b4128640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ac1592c0 0x7f11ac14b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.547+0000 7f11b4128640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11ac1596a0 0x7f11ac14b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.547+0000 7f11b4128640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac06dca0 con 0x7f11ac1592c0 2024-04-25T01:26:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.547+0000 7f11b4128640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac06db20 con 0x7f11ac0c72b0 2024-04-25T01:26:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.547+0000 7f11b4128640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f11ac0c95b0 con 0x7f11ac1596a0 2024-04-25T01:26:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.547+0000 7f11b16c3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ac1592c0 0x7f11ac14b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.547+0000 7f11b16c3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ac1592c0 0x7f11ac14b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46288/0 (socket says 172.21.15.112:46288) 2024-04-25T01:26:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.547+0000 7f11b16c3640 1 -- 172.21.15.112:0/1968291220 learned_addr learned my addr 172.21.15.112:0/1968291220 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.547+0000 7f11b26c5640 1 --2- 172.21.15.112:0/1968291220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11ac1596a0 0x7f11ac14b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.547+0000 7f11b1ec4640 1 --2- 172.21.15.112:0/1968291220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11ac0c72b0 0x7f11ac14fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.547+0000 7f11b16c3640 1 -- 172.21.15.112:0/1968291220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11ac1596a0 msgr2=0x7f11ac14b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.547+0000 7f11b16c3640 1 --2- 172.21.15.112:0/1968291220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11ac1596a0 0x7f11ac14b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.547+0000 7f11b16c3640 1 -- 172.21.15.112:0/1968291220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11ac0c72b0 msgr2=0x7f11ac14fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.547+0000 7f11b16c3640 1 --2- 172.21.15.112:0/1968291220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11ac0c72b0 0x7f11ac14fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.547+0000 7f11b16c3640 1 -- 172.21.15.112:0/1968291220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ac14e390 con 0x7f11ac1592c0 2024-04-25T01:26:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.547+0000 7f11b26c5640 1 --2- 172.21.15.112:0/1968291220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11ac1596a0 0x7f11ac14b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.547+0000 7f11b1ec4640 1 --2- 172.21.15.112:0/1968291220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11ac0c72b0 0x7f11ac14fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:26.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.547+0000 7f11b16c3640 1 --2- 172.21.15.112:0/1968291220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ac1592c0 0x7f11ac14b2d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f119c0098c0 tx=0x7f119c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:26.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.548+0000 7f11a27fc640 1 -- 172.21.15.112:0/1968291220 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c041030 con 0x7f11ac1592c0 2024-04-25T01:26:26.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.548+0000 7f11a27fc640 1 -- 172.21.15.112:0/1968291220 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f119c02abb0 con 0x7f11ac1592c0 2024-04-25T01:26:26.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.548+0000 7f11a27fc640 1 -- 172.21.15.112:0/1968291220 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c037040 con 0x7f11ac1592c0 2024-04-25T01:26:26.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.548+0000 7f11b4128640 1 -- 172.21.15.112:0/1968291220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11ac14e670 con 0x7f11ac1592c0 2024-04-25T01:26:26.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.548+0000 7f11b4128640 1 -- 172.21.15.112:0/1968291220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11ac14ec30 con 0x7f11ac1592c0 2024-04-25T01:26:26.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.549+0000 7f11b4128640 1 -- 172.21.15.112:0/1968291220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11ac0679a0 con 0x7f11ac1592c0 2024-04-25T01:26:26.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.549+0000 7f11a27fc640 1 -- 172.21.15.112:0/1968291220 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f119c044070 con 0x7f11ac1592c0 2024-04-25T01:26:26.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.550+0000 7f11a27fc640 1 --2- 172.21.15.112:0/1968291220 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f119006be30 0x7f119006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:26.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.550+0000 7f11b1ec4640 1 --2- 172.21.15.112:0/1968291220 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f119006be30 0x7f119006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:26.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.550+0000 7f11a27fc640 1 -- 172.21.15.112:0/1968291220 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f119c0ab7b0 con 0x7f11ac1592c0 2024-04-25T01:26:26.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.550+0000 7f11b1ec4640 1 --2- 172.21.15.112:0/1968291220 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f119006be30 0x7f119006e2f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f11a800e550 tx=0x7f11a8007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:26.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.554+0000 7f11a27fc640 1 -- 172.21.15.112:0/1968291220 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f119c044e40 con 0x7f11ac1592c0 2024-04-25T01:26:26.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.726+0000 7f11b4128640 1 -- 172.21.15.112:0/1968291220 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f11ac075980 con 0x7f119006be30 2024-04-25T01:26:26.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.736+0000 7f11a27fc640 1 -- 172.21.15.112:0/1968291220 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+490299 (secure 0 0 0) 0x7f11a800c5c0 con 0x7f119006be30 2024-04-25T01:26:26.721 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T01:26:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.740+0000 7f11b4128640 1 -- 172.21.15.112:0/1968291220 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f119006be30 msgr2=0x7f119006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.740+0000 7f11b4128640 1 --2- 172.21.15.112:0/1968291220 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f119006be30 0x7f119006e2f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f11a800e550 tx=0x7f11a8007ab0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.740+0000 7f11b4128640 1 -- 172.21.15.112:0/1968291220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ac1592c0 msgr2=0x7f11ac14b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.740+0000 7f11b4128640 1 --2- 172.21.15.112:0/1968291220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ac1592c0 0x7f11ac14b2d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f119c0098c0 tx=0x7f119c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.741+0000 7f11b4128640 1 -- 172.21.15.112:0/1968291220 shutdown_connections 2024-04-25T01:26:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.741+0000 7f11b4128640 1 --2- 172.21.15.112:0/1968291220 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f119006be30 0x7f119006e2f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.741+0000 7f11b4128640 1 --2- 172.21.15.112:0/1968291220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11ac1596a0 0x7f11ac14b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.741+0000 7f11b4128640 1 --2- 172.21.15.112:0/1968291220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ac1592c0 0x7f11ac14b2d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.741+0000 7f11b4128640 1 --2- 172.21.15.112:0/1968291220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11ac0c72b0 0x7f11ac14fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.741+0000 7f11b4128640 1 -- 172.21.15.112:0/1968291220 >> 172.21.15.112:0/1968291220 conn(0x7f11ac0cb170 msgr2=0x7f11ac0ac190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.741+0000 7f11b4128640 1 -- 172.21.15.112:0/1968291220 shutdown_connections 2024-04-25T01:26:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.741+0000 7f11b4128640 1 -- 172.21.15.112:0/1968291220 wait complete. 2024-04-25T01:26:26.731 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-04-25T01:26:26.655428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596070,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":253,"num_write_kb":2448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219440,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755280,"statfs":{"total":773094113280,"available":772869406720,"internally_reserved":0,"allocated":3350528,"data_stored":1986228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":882,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002499"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415591+0000","last_change":"2024-04-25T01:26:02.412272+0000","last_active":"2024-04-25T01:26:03.415591+0000","last_peered":"2024-04-25T01:26:03.415591+0000","last_clean":"2024-04-25T01:26:03.415591+0000","last_became_active":"2024-04-25T01:26:02.411176+0000","last_became_peered":"2024-04-25T01:26:02.411176+0000","last_unstale":"2024-04-25T01:26:03.415591+0000","last_undegraded":"2024-04-25T01:26:03.415591+0000","last_fullsized":"2024-04-25T01:26:03.415591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:49.498275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716276+0000","last_change":"2024-04-25T01:25:52.353978+0000","last_active":"2024-04-25T01:26:03.716276+0000","last_peered":"2024-04-25T01:26:03.716276+0000","last_clean":"2024-04-25T01:26:03.716276+0000","last_became_active":"2024-04-25T01:25:52.353649+0000","last_became_peered":"2024-04-25T01:25:52.353649+0000","last_unstale":"2024-04-25T01:26:03.716276+0000","last_undegraded":"2024-04-25T01:26:03.716276+0000","last_fullsized":"2024-04-25T01:26:03.716276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416695+0000","last_change":"2024-04-25T01:25:53.364257+0000","last_active":"2024-04-25T01:26:03.416695+0000","last_peered":"2024-04-25T01:26:03.416695+0000","last_clean":"2024-04-25T01:26:03.416695+0000","last_became_active":"2024-04-25T01:25:53.363409+0000","last_became_peered":"2024-04-25T01:25:53.363409+0000","last_unstale":"2024-04-25T01:26:03.416695+0000","last_undegraded":"2024-04-25T01:26:03.416695+0000","last_fullsized":"2024-04-25T01:26:03.416695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415484+0000","last_change":"2024-04-25T01:26:01.407136+0000","last_active":"2024-04-25T01:26:03.415484+0000","last_peered":"2024-04-25T01:26:03.415484+0000","last_clean":"2024-04-25T01:26:03.415484+0000","last_became_active":"2024-04-25T01:26:01.406855+0000","last_became_peered":"2024-04-25T01:26:01.406855+0000","last_unstale":"2024-04-25T01:26:03.415484+0000","last_undegraded":"2024-04-25T01:26:03.415484+0000","last_fullsized":"2024-04-25T01:26:03.415484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:02.180826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716643+0000","last_change":"2024-04-25T01:26:02.425367+0000","last_active":"2024-04-25T01:26:03.716643+0000","last_peered":"2024-04-25T01:26:03.716643+0000","last_clean":"2024-04-25T01:26:03.716643+0000","last_became_active":"2024-04-25T01:26:02.425071+0000","last_became_peered":"2024-04-25T01:26:02.425071+0000","last_unstale":"2024-04-25T01:26:03.716643+0000","last_undegraded":"2024-04-25T01:26:03.716643+0000","last_fullsized":"2024-04-25T01:26:03.716643+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:47.810943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416737+0000","last_change":"2024-04-25T01:25:52.349185+0000","last_active":"2024-04-25T01:26:03.416737+0000","last_peered":"2024-04-25T01:26:03.416737+0000","last_clean":"2024-04-25T01:26:03.416737+0000","last_became_active":"2024-04-25T01:25:52.348931+0000","last_became_peered":"2024-04-25T01:25:52.348931+0000","last_unstale":"2024-04-25T01:26:03.416737+0000","last_undegraded":"2024-04-25T01:26:03.416737+0000","last_fullsized":"2024-04-25T01:26:03.416737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716082+0000","last_change":"2024-04-25T01:25:53.374150+0000","last_active":"2024-04-25T01:26:03.716082+0000","last_peered":"2024-04-25T01:26:03.716082+0000","last_clean":"2024-04-25T01:26:03.716082+0000","last_became_active":"2024-04-25T01:25:53.373755+0000","last_became_peered":"2024-04-25T01:25:53.373755+0000","last_unstale":"2024-04-25T01:26:03.716082+0000","last_undegraded":"2024-04-25T01:26:03.716082+0000","last_fullsized":"2024-04-25T01:26:03.716082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415473+0000","last_change":"2024-04-25T01:26:01.406334+0000","last_active":"2024-04-25T01:26:03.415473+0000","last_peered":"2024-04-25T01:26:03.415473+0000","last_clean":"2024-04-25T01:26:03.415473+0000","last_became_active":"2024-04-25T01:26:01.405994+0000","last_became_peered":"2024-04-25T01:26:01.405994+0000","last_unstale":"2024-04-25T01:26:03.415473+0000","last_undegraded":"2024-04-25T01:26:03.415473+0000","last_fullsized":"2024-04-25T01:26:03.415473+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:43.953437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507172+0000","last_change":"2024-04-25T01:26:02.427695+0000","last_active":"2024-04-25T01:26:03.507172+0000","last_peered":"2024-04-25T01:26:03.507172+0000","last_clean":"2024-04-25T01:26:03.507172+0000","last_became_active":"2024-04-25T01:26:02.427443+0000","last_became_peered":"2024-04-25T01:26:02.427443+0000","last_unstale":"2024-04-25T01:26:03.507172+0000","last_undegraded":"2024-04-25T01:26:03.507172+0000","last_fullsized":"2024-04-25T01:26:03.507172+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507616+0000","last_change":"2024-04-25T01:25:52.356403+0000","last_active":"2024-04-25T01:26:03.507616+0000","last_peered":"2024-04-25T01:26:03.507616+0000","last_clean":"2024-04-25T01:26:03.507616+0000","last_became_active":"2024-04-25T01:25:52.355822+0000","last_became_peered":"2024-04-25T01:25:52.355822+0000","last_unstale":"2024-04-25T01:26:03.507616+0000","last_undegraded":"2024-04-25T01:26:03.507616+0000","last_fullsized":"2024-04-25T01:26:03.507616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717449+0000","last_change":"2024-04-25T01:25:53.370061+0000","last_active":"2024-04-25T01:26:03.717449+0000","last_peered":"2024-04-25T01:26:03.717449+0000","last_clean":"2024-04-25T01:26:03.717449+0000","last_became_active":"2024-04-25T01:25:53.369721+0000","last_became_peered":"2024-04-25T01:25:53.369721+0000","last_unstale":"2024-04-25T01:26:03.717449+0000","last_undegraded":"2024-04-25T01:26:03.717449+0000","last_fullsized":"2024-04-25T01:26:03.717449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415934+0000","last_change":"2024-04-25T01:26:01.408179+0000","last_active":"2024-04-25T01:26:03.415934+0000","last_peered":"2024-04-25T01:26:03.415934+0000","last_clean":"2024-04-25T01:26:03.415934+0000","last_became_active":"2024-04-25T01:26:01.407909+0000","last_became_peered":"2024-04-25T01:26:01.407909+0000","last_unstale":"2024-04-25T01:26:03.415934+0000","last_undegraded":"2024-04-25T01:26:03.415934+0000","last_fullsized":"2024-04-25T01:26:03.415934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:48.865940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717234+0000","last_change":"2024-04-25T01:26:02.424615+0000","last_active":"2024-04-25T01:26:03.717234+0000","last_peered":"2024-04-25T01:26:03.717234+0000","last_clean":"2024-04-25T01:26:03.717234+0000","last_became_active":"2024-04-25T01:26:02.424325+0000","last_became_peered":"2024-04-25T01:26:02.424325+0000","last_unstale":"2024-04-25T01:26:03.717234+0000","last_undegraded":"2024-04-25T01:26:03.717234+0000","last_fullsized":"2024-04-25T01:26:03.717234+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507443+0000","last_change":"2024-04-25T01:25:52.359139+0000","last_active":"2024-04-25T01:26:03.507443+0000","last_peered":"2024-04-25T01:26:03.507443+0000","last_clean":"2024-04-25T01:26:03.507443+0000","last_became_active":"2024-04-25T01:25:52.358841+0000","last_became_peered":"2024-04-25T01:25:52.358841+0000","last_unstale":"2024-04-25T01:26:03.507443+0000","last_undegraded":"2024-04-25T01:26:03.507443+0000","last_fullsized":"2024-04-25T01:26:03.507443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717907+0000","last_change":"2024-04-25T01:25:53.368739+0000","last_active":"2024-04-25T01:26:03.717907+0000","last_peered":"2024-04-25T01:26:03.717907+0000","last_clean":"2024-04-25T01:26:03.717907+0000","last_became_active":"2024-04-25T01:25:53.368470+0000","last_became_peered":"2024-04-25T01:25:53.368470+0000","last_unstale":"2024-04-25T01:26:03.717907+0000","last_undegraded":"2024-04-25T01:26:03.717907+0000","last_fullsized":"2024-04-25T01:26:03.717907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:29.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415820+0000","last_change":"2024-04-25T01:26:01.407969+0000","last_active":"2024-04-25T01:26:03.415820+0000","last_peered":"2024-04-25T01:26:03.415820+0000","last_clean":"2024-04-25T01:26:03.415820+0000","last_became_active":"2024-04-25T01:26:01.407655+0000","last_became_peered":"2024-04-25T01:26:01.407655+0000","last_unstale":"2024-04-25T01:26:03.415820+0000","last_undegraded":"2024-04-25T01:26:03.415820+0000","last_fullsized":"2024-04-25T01:26:03.415820+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:58.485864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717005+0000","last_change":"2024-04-25T01:26:02.417776+0000","last_active":"2024-04-25T01:26:03.717005+0000","last_peered":"2024-04-25T01:26:03.717005+0000","last_clean":"2024-04-25T01:26:03.717005+0000","last_became_active":"2024-04-25T01:26:02.417524+0000","last_became_peered":"2024-04-25T01:26:02.417524+0000","last_unstale":"2024-04-25T01:26:03.717005+0000","last_undegraded":"2024-04-25T01:26:03.717005+0000","last_fullsized":"2024-04-25T01:26:03.717005+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.688469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716814+0000","last_change":"2024-04-25T01:25:53.372385+0000","last_active":"2024-04-25T01:26:03.716814+0000","last_peered":"2024-04-25T01:26:03.716814+0000","last_clean":"2024-04-25T01:26:03.716814+0000","last_became_active":"2024-04-25T01:25:53.372064+0000","last_became_peered":"2024-04-25T01:25:53.372064+0000","last_unstale":"2024-04-25T01:26:03.716814+0000","last_undegraded":"2024-04-25T01:26:03.716814+0000","last_fullsized":"2024-04-25T01:26:03.716814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507411+0000","last_change":"2024-04-25T01:25:52.358616+0000","last_active":"2024-04-25T01:26:03.507411+0000","last_peered":"2024-04-25T01:26:03.507411+0000","last_clean":"2024-04-25T01:26:03.507411+0000","last_became_active":"2024-04-25T01:25:52.358318+0000","last_became_peered":"2024-04-25T01:25:52.358318+0000","last_unstale":"2024-04-25T01:26:03.507411+0000","last_undegraded":"2024-04-25T01:26:03.507411+0000","last_fullsized":"2024-04-25T01:26:03.507411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416542+0000","last_change":"2024-04-25T01:26:01.400599+0000","last_active":"2024-04-25T01:26:03.416542+0000","last_peered":"2024-04-25T01:26:03.416542+0000","last_clean":"2024-04-25T01:26:03.416542+0000","last_became_active":"2024-04-25T01:26:01.400266+0000","last_became_peered":"2024-04-25T01:26:01.400266+0000","last_unstale":"2024-04-25T01:26:03.416542+0000","last_undegraded":"2024-04-25T01:26:03.416542+0000","last_fullsized":"2024-04-25T01:26:03.416542+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416953+0000","last_change":"2024-04-25T01:26:02.420269+0000","last_active":"2024-04-25T01:26:03.416953+0000","last_peered":"2024-04-25T01:26:03.416953+0000","last_clean":"2024-04-25T01:26:03.416953+0000","last_became_active":"2024-04-25T01:26:02.420001+0000","last_became_peered":"2024-04-25T01:26:02.420001+0000","last_unstale":"2024-04-25T01:26:03.416953+0000","last_undegraded":"2024-04-25T01:26:03.416953+0000","last_fullsized":"2024-04-25T01:26:03.416953+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:38.149069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415549+0000","last_change":"2024-04-25T01:25:53.361074+0000","last_active":"2024-04-25T01:26:03.415549+0000","last_peered":"2024-04-25T01:26:03.415549+0000","last_clean":"2024-04-25T01:26:03.415549+0000","last_became_active":"2024-04-25T01:25:53.360802+0000","last_became_peered":"2024-04-25T01:25:53.360802+0000","last_unstale":"2024-04-25T01:26:03.415549+0000","last_undegraded":"2024-04-25T01:26:03.415549+0000","last_fullsized":"2024-04-25T01:26:03.415549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717195+0000","last_change":"2024-04-25T01:25:52.348841+0000","last_active":"2024-04-25T01:26:03.717195+0000","last_peered":"2024-04-25T01:26:03.717195+0000","last_clean":"2024-04-25T01:26:03.717195+0000","last_became_active":"2024-04-25T01:25:52.348550+0000","last_became_peered":"2024-04-25T01:25:52.348550+0000","last_unstale":"2024-04-25T01:26:03.717195+0000","last_undegraded":"2024-04-25T01:26:03.717195+0000","last_fullsized":"2024-04-25T01:26:03.717195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416126+0000","last_change":"2024-04-25T01:26:01.408529+0000","last_active":"2024-04-25T01:26:03.416126+0000","last_peered":"2024-04-25T01:26:03.416126+0000","last_clean":"2024-04-25T01:26:03.416126+0000","last_became_active":"2024-04-25T01:26:01.407738+0000","last_became_peered":"2024-04-25T01:26:01.407738+0000","last_unstale":"2024-04-25T01:26:03.416126+0000","last_undegraded":"2024-04-25T01:26:03.416126+0000","last_fullsized":"2024-04-25T01:26:03.416126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:37.142255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416507+0000","last_change":"2024-04-25T01:26:02.417731+0000","last_active":"2024-04-25T01:26:03.416507+0000","last_peered":"2024-04-25T01:26:03.416507+0000","last_clean":"2024-04-25T01:26:03.416507+0000","last_became_active":"2024-04-25T01:26:02.417438+0000","last_became_peered":"2024-04-25T01:26:02.417438+0000","last_unstale":"2024-04-25T01:26:03.416507+0000","last_undegraded":"2024-04-25T01:26:03.416507+0000","last_fullsized":"2024-04-25T01:26:03.416507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415373+0000","last_change":"2024-04-25T01:25:53.361790+0000","last_active":"2024-04-25T01:26:03.415373+0000","last_peered":"2024-04-25T01:26:03.415373+0000","last_clean":"2024-04-25T01:26:03.415373+0000","last_became_active":"2024-04-25T01:25:53.361542+0000","last_became_peered":"2024-04-25T01:25:53.361542+0000","last_unstale":"2024-04-25T01:26:03.415373+0000","last_undegraded":"2024-04-25T01:26:03.415373+0000","last_fullsized":"2024-04-25T01:26:03.415373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716946+0000","last_change":"2024-04-25T01:25:52.353926+0000","last_active":"2024-04-25T01:26:03.716946+0000","last_peered":"2024-04-25T01:26:03.716946+0000","last_clean":"2024-04-25T01:26:03.716946+0000","last_became_active":"2024-04-25T01:25:52.353656+0000","last_became_peered":"2024-04-25T01:25:52.353656+0000","last_unstale":"2024-04-25T01:26:03.716946+0000","last_undegraded":"2024-04-25T01:26:03.716946+0000","last_fullsized":"2024-04-25T01:26:03.716946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716923+0000","last_change":"2024-04-25T01:26:01.418556+0000","last_active":"2024-04-25T01:26:03.716923+0000","last_peered":"2024-04-25T01:26:03.716923+0000","last_clean":"2024-04-25T01:26:03.716923+0000","last_became_active":"2024-04-25T01:26:01.418263+0000","last_became_peered":"2024-04-25T01:26:01.418263+0000","last_unstale":"2024-04-25T01:26:03.716923+0000","last_undegraded":"2024-04-25T01:26:03.716923+0000","last_fullsized":"2024-04-25T01:26:03.716923+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:28.139183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717839+0000","last_change":"2024-04-25T01:26:02.413429+0000","last_active":"2024-04-25T01:26:03.717839+0000","last_peered":"2024-04-25T01:26:03.717839+0000","last_clean":"2024-04-25T01:26:03.717839+0000","last_became_active":"2024-04-25T01:26:02.412632+0000","last_became_peered":"2024-04-25T01:26:02.412632+0000","last_unstale":"2024-04-25T01:26:03.717839+0000","last_undegraded":"2024-04-25T01:26:03.717839+0000","last_fullsized":"2024-04-25T01:26:03.717839+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:10.049310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416067+0000","last_change":"2024-04-25T01:25:53.361653+0000","last_active":"2024-04-25T01:26:03.416067+0000","last_peered":"2024-04-25T01:26:03.416067+0000","last_clean":"2024-04-25T01:26:03.416067+0000","last_became_active":"2024-04-25T01:25:53.361265+0000","last_became_peered":"2024-04-25T01:25:53.361265+0000","last_unstale":"2024-04-25T01:26:03.416067+0000","last_undegraded":"2024-04-25T01:26:03.416067+0000","last_fullsized":"2024-04-25T01:26:03.416067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507177+0000","last_change":"2024-04-25T01:25:52.353836+0000","last_active":"2024-04-25T01:26:03.507177+0000","last_peered":"2024-04-25T01:26:03.507177+0000","last_clean":"2024-04-25T01:26:03.507177+0000","last_became_active":"2024-04-25T01:25:52.353552+0000","last_became_peered":"2024-04-25T01:25:52.353552+0000","last_unstale":"2024-04-25T01:26:03.507177+0000","last_undegraded":"2024-04-25T01:26:03.507177+0000","last_fullsized":"2024-04-25T01:26:03.507177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416325+0000","last_change":"2024-04-25T01:26:01.411323+0000","last_active":"2024-04-25T01:26:03.416325+0000","last_peered":"2024-04-25T01:26:03.416325+0000","last_clean":"2024-04-25T01:26:03.416325+0000","last_became_active":"2024-04-25T01:26:01.410378+0000","last_became_peered":"2024-04-25T01:26:01.410378+0000","last_unstale":"2024-04-25T01:26:03.416325+0000","last_undegraded":"2024-04-25T01:26:03.416325+0000","last_fullsized":"2024-04-25T01:26:03.416325+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415000+0000","last_change":"2024-04-25T01:26:02.416646+0000","last_active":"2024-04-25T01:26:03.415000+0000","last_peered":"2024-04-25T01:26:03.415000+0000","last_clean":"2024-04-25T01:26:03.415000+0000","last_became_active":"2024-04-25T01:26:02.416395+0000","last_became_peered":"2024-04-25T01:26:02.416395+0000","last_unstale":"2024-04-25T01:26:03.415000+0000","last_undegraded":"2024-04-25T01:26:03.415000+0000","last_fullsized":"2024-04-25T01:26:03.415000+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716795+0000","last_change":"2024-04-25T01:25:53.370111+0000","last_active":"2024-04-25T01:26:03.716795+0000","last_peered":"2024-04-25T01:26:03.716795+0000","last_clean":"2024-04-25T01:26:03.716795+0000","last_became_active":"2024-04-25T01:25:53.369837+0000","last_became_peered":"2024-04-25T01:25:53.369837+0000","last_unstale":"2024-04-25T01:26:03.716795+0000","last_undegraded":"2024-04-25T01:26:03.716795+0000","last_fullsized":"2024-04-25T01:26:03.716795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717961+0000","last_change":"2024-04-25T01:25:52.353269+0000","last_active":"2024-04-25T01:26:03.717961+0000","last_peered":"2024-04-25T01:26:03.717961+0000","last_clean":"2024-04-25T01:26:03.717961+0000","last_became_active":"2024-04-25T01:25:52.352991+0000","last_became_peered":"2024-04-25T01:25:52.352991+0000","last_unstale":"2024-04-25T01:26:03.717961+0000","last_undegraded":"2024-04-25T01:26:03.717961+0000","last_fullsized":"2024-04-25T01:26:03.717961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415651+0000","last_change":"2024-04-25T01:26:01.410360+0000","last_active":"2024-04-25T01:26:03.415651+0000","last_peered":"2024-04-25T01:26:03.415651+0000","last_clean":"2024-04-25T01:26:03.415651+0000","last_became_active":"2024-04-25T01:26:01.410076+0000","last_became_peered":"2024-04-25T01:26:01.410076+0000","last_unstale":"2024-04-25T01:26:03.415651+0000","last_undegraded":"2024-04-25T01:26:03.415651+0000","last_fullsized":"2024-04-25T01:26:03.415651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:52.561251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415577+0000","last_change":"2024-04-25T01:26:02.422182+0000","last_active":"2024-04-25T01:26:03.415577+0000","last_peered":"2024-04-25T01:26:03.415577+0000","last_clean":"2024-04-25T01:26:03.415577+0000","last_became_active":"2024-04-25T01:26:02.421885+0000","last_became_peered":"2024-04-25T01:26:02.421885+0000","last_unstale":"2024-04-25T01:26:03.415577+0000","last_undegraded":"2024-04-25T01:26:03.415577+0000","last_fullsized":"2024-04-25T01:26:03.415577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:03.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416628+0000","last_change":"2024-04-25T01:25:53.364967+0000","last_active":"2024-04-25T01:26:03.416628+0000","last_peered":"2024-04-25T01:26:03.416628+0000","last_clean":"2024-04-25T01:26:03.416628+0000","last_became_active":"2024-04-25T01:25:53.364694+0000","last_became_peered":"2024-04-25T01:25:53.364694+0000","last_unstale":"2024-04-25T01:26:03.416628+0000","last_undegraded":"2024-04-25T01:26:03.416628+0000","last_fullsized":"2024-04-25T01:26:03.416628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717049+0000","last_change":"2024-04-25T01:25:52.355395+0000","last_active":"2024-04-25T01:26:03.717049+0000","last_peered":"2024-04-25T01:26:03.717049+0000","last_clean":"2024-04-25T01:26:03.717049+0000","last_became_active":"2024-04-25T01:25:52.355137+0000","last_became_peered":"2024-04-25T01:25:52.355137+0000","last_unstale":"2024-04-25T01:26:03.717049+0000","last_undegraded":"2024-04-25T01:26:03.717049+0000","last_fullsized":"2024-04-25T01:26:03.717049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:26.467968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.414943+0000","last_change":"2024-04-25T01:26:01.416108+0000","last_active":"2024-04-25T01:26:03.414943+0000","last_peered":"2024-04-25T01:26:03.414943+0000","last_clean":"2024-04-25T01:26:03.414943+0000","last_became_active":"2024-04-25T01:26:01.415824+0000","last_became_peered":"2024-04-25T01:26:01.415824+0000","last_unstale":"2024-04-25T01:26:03.414943+0000","last_undegraded":"2024-04-25T01:26:03.414943+0000","last_fullsized":"2024-04-25T01:26:03.414943+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:28.139183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716568+0000","last_change":"2024-04-25T01:26:02.416208+0000","last_active":"2024-04-25T01:26:03.716568+0000","last_peered":"2024-04-25T01:26:03.716568+0000","last_clean":"2024-04-25T01:26:03.716568+0000","last_became_active":"2024-04-25T01:26:02.415949+0000","last_became_peered":"2024-04-25T01:26:02.415949+0000","last_unstale":"2024-04-25T01:26:03.716568+0000","last_undegraded":"2024-04-25T01:26:03.716568+0000","last_fullsized":"2024-04-25T01:26:03.716568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:21.097225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717249+0000","last_change":"2024-04-25T01:25:53.366944+0000","last_active":"2024-04-25T01:26:03.717249+0000","last_peered":"2024-04-25T01:26:03.717249+0000","last_clean":"2024-04-25T01:26:03.717249+0000","last_became_active":"2024-04-25T01:25:53.366693+0000","last_became_peered":"2024-04-25T01:25:53.366693+0000","last_unstale":"2024-04-25T01:26:03.717249+0000","last_undegraded":"2024-04-25T01:26:03.717249+0000","last_fullsized":"2024-04-25T01:26:03.717249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507732+0000","last_change":"2024-04-25T01:25:52.356842+0000","last_active":"2024-04-25T01:26:03.507732+0000","last_peered":"2024-04-25T01:26:03.507732+0000","last_clean":"2024-04-25T01:26:03.507732+0000","last_became_active":"2024-04-25T01:25:52.356396+0000","last_became_peered":"2024-04-25T01:25:52.356396+0000","last_unstale":"2024-04-25T01:26:03.507732+0000","last_undegraded":"2024-04-25T01:26:03.507732+0000","last_fullsized":"2024-04-25T01:26:03.507732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716730+0000","last_change":"2024-04-25T01:26:01.407934+0000","last_active":"2024-04-25T01:26:03.716730+0000","last_peered":"2024-04-25T01:26:03.716730+0000","last_clean":"2024-04-25T01:26:03.716730+0000","last_became_active":"2024-04-25T01:26:01.407659+0000","last_became_peered":"2024-04-25T01:26:01.407659+0000","last_unstale":"2024-04-25T01:26:03.716730+0000","last_undegraded":"2024-04-25T01:26:03.716730+0000","last_fullsized":"2024-04-25T01:26:03.716730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:19.629335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716666+0000","last_change":"2024-04-25T01:26:02.418087+0000","last_active":"2024-04-25T01:26:03.716666+0000","last_peered":"2024-04-25T01:26:03.716666+0000","last_clean":"2024-04-25T01:26:03.716666+0000","last_became_active":"2024-04-25T01:26:02.417622+0000","last_became_peered":"2024-04-25T01:26:02.417622+0000","last_unstale":"2024-04-25T01:26:03.716666+0000","last_undegraded":"2024-04-25T01:26:03.716666+0000","last_fullsized":"2024-04-25T01:26:03.716666+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:03.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717962+0000","last_change":"2024-04-25T01:25:53.368164+0000","last_active":"2024-04-25T01:26:03.717962+0000","last_peered":"2024-04-25T01:26:03.717962+0000","last_clean":"2024-04-25T01:26:03.717962+0000","last_became_active":"2024-04-25T01:25:53.367854+0000","last_became_peered":"2024-04-25T01:25:53.367854+0000","last_unstale":"2024-04-25T01:26:03.717962+0000","last_undegraded":"2024-04-25T01:26:03.717962+0000","last_fullsized":"2024-04-25T01:26:03.717962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717294+0000","last_change":"2024-04-25T01:25:52.350128+0000","last_active":"2024-04-25T01:26:03.717294+0000","last_peered":"2024-04-25T01:26:03.717294+0000","last_clean":"2024-04-25T01:26:03.717294+0000","last_became_active":"2024-04-25T01:25:52.349874+0000","last_became_peered":"2024-04-25T01:25:52.349874+0000","last_unstale":"2024-04-25T01:26:03.717294+0000","last_undegraded":"2024-04-25T01:26:03.717294+0000","last_fullsized":"2024-04-25T01:26:03.717294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716937+0000","last_change":"2024-04-25T01:26:01.418196+0000","last_active":"2024-04-25T01:26:03.716937+0000","last_peered":"2024-04-25T01:26:03.716937+0000","last_clean":"2024-04-25T01:26:03.716937+0000","last_became_active":"2024-04-25T01:26:01.417820+0000","last_became_peered":"2024-04-25T01:26:01.417820+0000","last_unstale":"2024-04-25T01:26:03.716937+0000","last_undegraded":"2024-04-25T01:26:03.716937+0000","last_fullsized":"2024-04-25T01:26:03.716937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716324+0000","last_change":"2024-04-25T01:26:02.422761+0000","last_active":"2024-04-25T01:26:03.716324+0000","last_peered":"2024-04-25T01:26:03.716324+0000","last_clean":"2024-04-25T01:26:03.716324+0000","last_became_active":"2024-04-25T01:26:02.422503+0000","last_became_peered":"2024-04-25T01:26:02.422503+0000","last_unstale":"2024-04-25T01:26:03.716324+0000","last_undegraded":"2024-04-25T01:26:03.716324+0000","last_fullsized":"2024-04-25T01:26:03.716324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:55.040946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416617+0000","last_change":"2024-04-25T01:25:53.361696+0000","last_active":"2024-04-25T01:26:03.416617+0000","last_peered":"2024-04-25T01:26:03.416617+0000","last_clean":"2024-04-25T01:26:03.416617+0000","last_became_active":"2024-04-25T01:25:53.361156+0000","last_became_peered":"2024-04-25T01:25:53.361156+0000","last_unstale":"2024-04-25T01:26:03.416617+0000","last_undegraded":"2024-04-25T01:26:03.416617+0000","last_fullsized":"2024-04-25T01:26:03.416617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:27.472315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717721+0000","last_change":"2024-04-25T01:25:52.354499+0000","last_active":"2024-04-25T01:26:03.717721+0000","last_peered":"2024-04-25T01:26:03.717721+0000","last_clean":"2024-04-25T01:26:03.717721+0000","last_became_active":"2024-04-25T01:25:52.354193+0000","last_became_peered":"2024-04-25T01:25:52.354193+0000","last_unstale":"2024-04-25T01:26:03.717721+0000","last_undegraded":"2024-04-25T01:26:03.717721+0000","last_fullsized":"2024-04-25T01:26:03.717721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717190+0000","last_change":"2024-04-25T01:26:01.418186+0000","last_active":"2024-04-25T01:26:03.717190+0000","last_peered":"2024-04-25T01:26:03.717190+0000","last_clean":"2024-04-25T01:26:03.717190+0000","last_became_active":"2024-04-25T01:26:01.417805+0000","last_became_peered":"2024-04-25T01:26:01.417805+0000","last_unstale":"2024-04-25T01:26:03.717190+0000","last_undegraded":"2024-04-25T01:26:03.717190+0000","last_fullsized":"2024-04-25T01:26:03.717190+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:19.629335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416697+0000","last_change":"2024-04-25T01:26:02.415877+0000","last_active":"2024-04-25T01:26:03.416697+0000","last_peered":"2024-04-25T01:26:03.416697+0000","last_clean":"2024-04-25T01:26:03.416697+0000","last_became_active":"2024-04-25T01:26:02.415619+0000","last_became_peered":"2024-04-25T01:26:02.415619+0000","last_unstale":"2024-04-25T01:26:03.416697+0000","last_undegraded":"2024-04-25T01:26:03.416697+0000","last_fullsized":"2024-04-25T01:26:03.416697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:14.647083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716434+0000","last_change":"2024-04-25T01:25:53.362922+0000","last_active":"2024-04-25T01:26:03.716434+0000","last_peered":"2024-04-25T01:26:03.716434+0000","last_clean":"2024-04-25T01:26:03.716434+0000","last_became_active":"2024-04-25T01:25:53.362533+0000","last_became_peered":"2024-04-25T01:25:53.362533+0000","last_unstale":"2024-04-25T01:26:03.716434+0000","last_undegraded":"2024-04-25T01:26:03.716434+0000","last_fullsized":"2024-04-25T01:26:03.716434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415851+0000","last_change":"2024-04-25T01:25:52.345321+0000","last_active":"2024-04-25T01:26:03.415851+0000","last_peered":"2024-04-25T01:26:03.415851+0000","last_clean":"2024-04-25T01:26:03.415851+0000","last_became_active":"2024-04-25T01:25:52.345061+0000","last_became_peered":"2024-04-25T01:25:52.345061+0000","last_unstale":"2024-04-25T01:26:03.415851+0000","last_undegraded":"2024-04-25T01:26:03.415851+0000","last_fullsized":"2024-04-25T01:26:03.415851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717072+0000","last_change":"2024-04-25T01:26:01.418516+0000","last_active":"2024-04-25T01:26:03.717072+0000","last_peered":"2024-04-25T01:26:03.717072+0000","last_clean":"2024-04-25T01:26:03.717072+0000","last_became_active":"2024-04-25T01:26:01.418173+0000","last_became_peered":"2024-04-25T01:26:01.418173+0000","last_unstale":"2024-04-25T01:26:03.717072+0000","last_undegraded":"2024-04-25T01:26:03.717072+0000","last_fullsized":"2024-04-25T01:26:03.717072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:02.568472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.718145+0000","last_change":"2024-04-25T01:26:02.418115+0000","last_active":"2024-04-25T01:26:03.718145+0000","last_peered":"2024-04-25T01:26:03.718145+0000","last_clean":"2024-04-25T01:26:03.718145+0000","last_became_active":"2024-04-25T01:26:02.417355+0000","last_became_peered":"2024-04-25T01:26:02.417355+0000","last_unstale":"2024-04-25T01:26:03.718145+0000","last_undegraded":"2024-04-25T01:26:03.718145+0000","last_fullsized":"2024-04-25T01:26:03.718145+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:19.644468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716066+0000","last_change":"2024-04-25T01:25:53.355950+0000","last_active":"2024-04-25T01:26:03.716066+0000","last_peered":"2024-04-25T01:26:03.716066+0000","last_clean":"2024-04-25T01:26:03.716066+0000","last_became_active":"2024-04-25T01:25:53.355648+0000","last_became_peered":"2024-04-25T01:25:53.355648+0000","last_unstale":"2024-04-25T01:26:03.716066+0000","last_undegraded":"2024-04-25T01:26:03.716066+0000","last_fullsized":"2024-04-25T01:26:03.716066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507424+0000","last_change":"2024-04-25T01:25:52.357683+0000","last_active":"2024-04-25T01:26:03.507424+0000","last_peered":"2024-04-25T01:26:03.507424+0000","last_clean":"2024-04-25T01:26:03.507424+0000","last_became_active":"2024-04-25T01:25:52.356924+0000","last_became_peered":"2024-04-25T01:25:52.356924+0000","last_unstale":"2024-04-25T01:26:03.507424+0000","last_undegraded":"2024-04-25T01:26:03.507424+0000","last_fullsized":"2024-04-25T01:26:03.507424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716612+0000","last_change":"2024-04-25T01:26:01.421260+0000","last_active":"2024-04-25T01:26:03.716612+0000","last_peered":"2024-04-25T01:26:03.716612+0000","last_clean":"2024-04-25T01:26:03.716612+0000","last_became_active":"2024-04-25T01:26:01.420831+0000","last_became_peered":"2024-04-25T01:26:01.420831+0000","last_unstale":"2024-04-25T01:26:03.716612+0000","last_undegraded":"2024-04-25T01:26:03.716612+0000","last_fullsized":"2024-04-25T01:26:03.716612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507519+0000","last_change":"2024-04-25T01:26:02.421386+0000","last_active":"2024-04-25T01:26:03.507519+0000","last_peered":"2024-04-25T01:26:03.507519+0000","last_clean":"2024-04-25T01:26:03.507519+0000","last_became_active":"2024-04-25T01:26:02.421079+0000","last_became_peered":"2024-04-25T01:26:02.421079+0000","last_unstale":"2024-04-25T01:26:03.507519+0000","last_undegraded":"2024-04-25T01:26:03.507519+0000","last_fullsized":"2024-04-25T01:26:03.507519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416038+0000","last_change":"2024-04-25T01:25:53.374719+0000","last_active":"2024-04-25T01:26:03.416038+0000","last_peered":"2024-04-25T01:26:03.416038+0000","last_clean":"2024-04-25T01:26:03.416038+0000","last_became_active":"2024-04-25T01:25:53.374425+0000","last_became_peered":"2024-04-25T01:25:53.374425+0000","last_unstale":"2024-04-25T01:26:03.416038+0000","last_undegraded":"2024-04-25T01:26:03.416038+0000","last_fullsized":"2024-04-25T01:26:03.416038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717309+0000","last_change":"2024-04-25T01:25:52.353155+0000","last_active":"2024-04-25T01:26:03.717309+0000","last_peered":"2024-04-25T01:26:03.717309+0000","last_clean":"2024-04-25T01:26:03.717309+0000","last_became_active":"2024-04-25T01:25:52.352826+0000","last_became_peered":"2024-04-25T01:25:52.352826+0000","last_unstale":"2024-04-25T01:26:03.717309+0000","last_undegraded":"2024-04-25T01:26:03.717309+0000","last_fullsized":"2024-04-25T01:26:03.717309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507554+0000","last_change":"2024-04-25T01:25:41.719670+0000","last_active":"2024-04-25T01:26:03.507554+0000","last_peered":"2024-04-25T01:26:03.507554+0000","last_clean":"2024-04-25T01:26:03.507554+0000","last_became_active":"2024-04-25T01:25:41.718721+0000","last_became_peered":"2024-04-25T01:25:41.718721+0000","last_unstale":"2024-04-25T01:26:03.507554+0000","last_undegraded":"2024-04-25T01:26:03.507554+0000","last_fullsized":"2024-04-25T01:26:03.507554+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_clean_scrub_stamp":"2024-04-25T01:25:40.688097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:27.456746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717378+0000","last_change":"2024-04-25T01:26:01.409339+0000","last_active":"2024-04-25T01:26:03.717378+0000","last_peered":"2024-04-25T01:26:03.717378+0000","last_clean":"2024-04-25T01:26:03.717378+0000","last_became_active":"2024-04-25T01:26:01.409062+0000","last_became_peered":"2024-04-25T01:26:01.409062+0000","last_unstale":"2024-04-25T01:26:03.717378+0000","last_undegraded":"2024-04-25T01:26:03.717378+0000","last_fullsized":"2024-04-25T01:26:03.717378+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:52.561251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415532+0000","last_change":"2024-04-25T01:26:02.418574+0000","last_active":"2024-04-25T01:26:03.415532+0000","last_peered":"2024-04-25T01:26:03.415532+0000","last_clean":"2024-04-25T01:26:03.415532+0000","last_became_active":"2024-04-25T01:26:02.416546+0000","last_became_peered":"2024-04-25T01:26:02.416546+0000","last_unstale":"2024-04-25T01:26:03.415532+0000","last_undegraded":"2024-04-25T01:26:03.415532+0000","last_fullsized":"2024-04-25T01:26:03.415532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:43.994141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507591+0000","last_change":"2024-04-25T01:25:53.375523+0000","last_active":"2024-04-25T01:26:03.507591+0000","last_peered":"2024-04-25T01:26:03.507591+0000","last_clean":"2024-04-25T01:26:03.507591+0000","last_became_active":"2024-04-25T01:25:53.375219+0000","last_became_peered":"2024-04-25T01:25:53.375219+0000","last_unstale":"2024-04-25T01:26:03.507591+0000","last_undegraded":"2024-04-25T01:26:03.507591+0000","last_fullsized":"2024-04-25T01:26:03.507591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.417443+0000","last_change":"2024-04-25T01:25:52.352994+0000","last_active":"2024-04-25T01:26:03.417443+0000","last_peered":"2024-04-25T01:26:03.417443+0000","last_clean":"2024-04-25T01:26:03.417443+0000","last_became_active":"2024-04-25T01:25:52.352644+0000","last_became_peered":"2024-04-25T01:25:52.352644+0000","last_unstale":"2024-04-25T01:26:03.417443+0000","last_undegraded":"2024-04-25T01:26:03.417443+0000","last_fullsized":"2024-04-25T01:26:03.417443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716482+0000","last_change":"2024-04-25T01:26:01.420891+0000","last_active":"2024-04-25T01:26:03.716482+0000","last_peered":"2024-04-25T01:26:03.716482+0000","last_clean":"2024-04-25T01:26:03.716482+0000","last_became_active":"2024-04-25T01:26:01.420664+0000","last_became_peered":"2024-04-25T01:26:01.420664+0000","last_unstale":"2024-04-25T01:26:03.716482+0000","last_undegraded":"2024-04-25T01:26:03.716482+0000","last_fullsized":"2024-04-25T01:26:03.716482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415682+0000","last_change":"2024-04-25T01:26:02.403489+0000","last_active":"2024-04-25T01:26:03.415682+0000","last_peered":"2024-04-25T01:26:03.415682+0000","last_clean":"2024-04-25T01:26:03.415682+0000","last_became_active":"2024-04-25T01:26:02.403206+0000","last_became_peered":"2024-04-25T01:26:02.403206+0000","last_unstale":"2024-04-25T01:26:03.415682+0000","last_undegraded":"2024-04-25T01:26:03.415682+0000","last_fullsized":"2024-04-25T01:26:03.415682+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:49.872754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.718041+0000","last_change":"2024-04-25T01:25:53.371018+0000","last_active":"2024-04-25T01:26:03.718041+0000","last_peered":"2024-04-25T01:26:03.718041+0000","last_clean":"2024-04-25T01:26:03.718041+0000","last_became_active":"2024-04-25T01:25:53.370539+0000","last_became_peered":"2024-04-25T01:25:53.370539+0000","last_unstale":"2024-04-25T01:26:03.718041+0000","last_undegraded":"2024-04-25T01:26:03.718041+0000","last_fullsized":"2024-04-25T01:26:03.718041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717910+0000","last_change":"2024-04-25T01:25:52.355133+0000","last_active":"2024-04-25T01:26:03.717910+0000","last_peered":"2024-04-25T01:26:03.717910+0000","last_clean":"2024-04-25T01:26:03.717910+0000","last_became_active":"2024-04-25T01:25:52.354836+0000","last_became_peered":"2024-04-25T01:25:52.354836+0000","last_unstale":"2024-04-25T01:26:03.717910+0000","last_undegraded":"2024-04-25T01:26:03.717910+0000","last_fullsized":"2024-04-25T01:26:03.717910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:10.331704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716238+0000","last_change":"2024-04-25T01:26:01.419285+0000","last_active":"2024-04-25T01:26:03.716238+0000","last_peered":"2024-04-25T01:26:03.716238+0000","last_clean":"2024-04-25T01:26:03.716238+0000","last_became_active":"2024-04-25T01:26:01.418987+0000","last_became_peered":"2024-04-25T01:26:01.418987+0000","last_unstale":"2024-04-25T01:26:03.716238+0000","last_undegraded":"2024-04-25T01:26:03.716238+0000","last_fullsized":"2024-04-25T01:26:03.716238+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.506958+0000","last_change":"2024-04-25T01:26:02.416544+0000","last_active":"2024-04-25T01:26:03.506958+0000","last_peered":"2024-04-25T01:26:03.506958+0000","last_clean":"2024-04-25T01:26:03.506958+0000","last_became_active":"2024-04-25T01:26:02.416189+0000","last_became_peered":"2024-04-25T01:26:02.416189+0000","last_unstale":"2024-04-25T01:26:03.506958+0000","last_undegraded":"2024-04-25T01:26:03.506958+0000","last_fullsized":"2024-04-25T01:26:03.506958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.688469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.508134+0000","last_change":"2024-04-25T01:25:53.370754+0000","last_active":"2024-04-25T01:26:03.508134+0000","last_peered":"2024-04-25T01:26:03.508134+0000","last_clean":"2024-04-25T01:26:03.508134+0000","last_became_active":"2024-04-25T01:25:53.370074+0000","last_became_peered":"2024-04-25T01:25:53.370074+0000","last_unstale":"2024-04-25T01:26:03.508134+0000","last_undegraded":"2024-04-25T01:26:03.508134+0000","last_fullsized":"2024-04-25T01:26:03.508134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416836+0000","last_change":"2024-04-25T01:25:52.345943+0000","last_active":"2024-04-25T01:26:03.416836+0000","last_peered":"2024-04-25T01:26:03.416836+0000","last_clean":"2024-04-25T01:26:03.416836+0000","last_became_active":"2024-04-25T01:25:52.345717+0000","last_became_peered":"2024-04-25T01:25:52.345717+0000","last_unstale":"2024-04-25T01:26:03.416836+0000","last_undegraded":"2024-04-25T01:26:03.416836+0000","last_fullsized":"2024-04-25T01:26:03.416836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716622+0000","last_change":"2024-04-25T01:26:01.416027+0000","last_active":"2024-04-25T01:26:03.716622+0000","last_peered":"2024-04-25T01:26:03.716622+0000","last_clean":"2024-04-25T01:26:03.716622+0000","last_became_active":"2024-04-25T01:26:01.415750+0000","last_became_peered":"2024-04-25T01:26:01.415750+0000","last_unstale":"2024-04-25T01:26:03.716622+0000","last_undegraded":"2024-04-25T01:26:03.716622+0000","last_fullsized":"2024-04-25T01:26:03.716622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:27.042376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416127+0000","last_change":"2024-04-25T01:26:02.414929+0000","last_active":"2024-04-25T01:26:03.416127+0000","last_peered":"2024-04-25T01:26:03.416127+0000","last_clean":"2024-04-25T01:26:03.416127+0000","last_became_active":"2024-04-25T01:26:02.414316+0000","last_became_peered":"2024-04-25T01:26:02.414316+0000","last_unstale":"2024-04-25T01:26:03.416127+0000","last_undegraded":"2024-04-25T01:26:03.416127+0000","last_fullsized":"2024-04-25T01:26:03.416127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:53.568065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507875+0000","last_change":"2024-04-25T01:25:53.365171+0000","last_active":"2024-04-25T01:26:03.507875+0000","last_peered":"2024-04-25T01:26:03.507875+0000","last_clean":"2024-04-25T01:26:03.507875+0000","last_became_active":"2024-04-25T01:25:53.364049+0000","last_became_peered":"2024-04-25T01:25:53.364049+0000","last_unstale":"2024-04-25T01:26:03.507875+0000","last_undegraded":"2024-04-25T01:26:03.507875+0000","last_fullsized":"2024-04-25T01:26:03.507875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416635+0000","last_change":"2024-04-25T01:25:52.350688+0000","last_active":"2024-04-25T01:26:03.416635+0000","last_peered":"2024-04-25T01:26:03.416635+0000","last_clean":"2024-04-25T01:26:03.416635+0000","last_became_active":"2024-04-25T01:25:52.350357+0000","last_became_peered":"2024-04-25T01:25:52.350357+0000","last_unstale":"2024-04-25T01:26:03.416635+0000","last_undegraded":"2024-04-25T01:26:03.416635+0000","last_fullsized":"2024-04-25T01:26:03.416635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415491+0000","last_change":"2024-04-25T01:26:01.418580+0000","last_active":"2024-04-25T01:26:03.415491+0000","last_peered":"2024-04-25T01:26:03.415491+0000","last_clean":"2024-04-25T01:26:03.415491+0000","last_became_active":"2024-04-25T01:26:01.418179+0000","last_became_peered":"2024-04-25T01:26:01.418179+0000","last_unstale":"2024-04-25T01:26:03.415491+0000","last_undegraded":"2024-04-25T01:26:03.415491+0000","last_fullsized":"2024-04-25T01:26:03.415491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:42.987327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416438+0000","last_change":"2024-04-25T01:26:02.418424+0000","last_active":"2024-04-25T01:26:03.416438+0000","last_peered":"2024-04-25T01:26:03.416438+0000","last_clean":"2024-04-25T01:26:03.416438+0000","last_became_active":"2024-04-25T01:26:02.418090+0000","last_became_peered":"2024-04-25T01:26:02.418090+0000","last_unstale":"2024-04-25T01:26:03.416438+0000","last_undegraded":"2024-04-25T01:26:03.416438+0000","last_fullsized":"2024-04-25T01:26:03.416438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:10.049310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507217+0000","last_change":"2024-04-25T01:25:53.366376+0000","last_active":"2024-04-25T01:26:03.507217+0000","last_peered":"2024-04-25T01:26:03.507217+0000","last_clean":"2024-04-25T01:26:03.507217+0000","last_became_active":"2024-04-25T01:25:53.365715+0000","last_became_peered":"2024-04-25T01:25:53.365715+0000","last_unstale":"2024-04-25T01:26:03.507217+0000","last_undegraded":"2024-04-25T01:26:03.507217+0000","last_fullsized":"2024-04-25T01:26:03.507217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416219+0000","last_change":"2024-04-25T01:25:52.347413+0000","last_active":"2024-04-25T01:26:03.416219+0000","last_peered":"2024-04-25T01:26:03.416219+0000","last_clean":"2024-04-25T01:26:03.416219+0000","last_became_active":"2024-04-25T01:25:52.347144+0000","last_became_peered":"2024-04-25T01:25:52.347144+0000","last_unstale":"2024-04-25T01:26:03.416219+0000","last_undegraded":"2024-04-25T01:26:03.416219+0000","last_fullsized":"2024-04-25T01:26:03.416219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716710+0000","last_change":"2024-04-25T01:26:01.409917+0000","last_active":"2024-04-25T01:26:03.716710+0000","last_peered":"2024-04-25T01:26:03.716710+0000","last_clean":"2024-04-25T01:26:03.716710+0000","last_became_active":"2024-04-25T01:26:01.409654+0000","last_became_peered":"2024-04-25T01:26:01.409654+0000","last_unstale":"2024-04-25T01:26:03.716710+0000","last_undegraded":"2024-04-25T01:26:03.716710+0000","last_fullsized":"2024-04-25T01:26:03.716710+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:37.142255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717898+0000","last_change":"2024-04-25T01:26:02.425103+0000","last_active":"2024-04-25T01:26:03.717898+0000","last_peered":"2024-04-25T01:26:03.717898+0000","last_clean":"2024-04-25T01:26:03.717898+0000","last_became_active":"2024-04-25T01:26:02.424819+0000","last_became_peered":"2024-04-25T01:26:02.424819+0000","last_unstale":"2024-04-25T01:26:03.717898+0000","last_undegraded":"2024-04-25T01:26:03.717898+0000","last_fullsized":"2024-04-25T01:26:03.717898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716406+0000","last_change":"2024-04-25T01:25:53.364987+0000","last_active":"2024-04-25T01:26:03.716406+0000","last_peered":"2024-04-25T01:26:03.716406+0000","last_clean":"2024-04-25T01:26:03.716406+0000","last_became_active":"2024-04-25T01:25:53.364431+0000","last_became_peered":"2024-04-25T01:25:53.364431+0000","last_unstale":"2024-04-25T01:26:03.716406+0000","last_undegraded":"2024-04-25T01:26:03.716406+0000","last_fullsized":"2024-04-25T01:26:03.716406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716731+0000","last_change":"2024-04-25T01:25:52.351205+0000","last_active":"2024-04-25T01:26:03.716731+0000","last_peered":"2024-04-25T01:26:03.716731+0000","last_clean":"2024-04-25T01:26:03.716731+0000","last_became_active":"2024-04-25T01:25:52.350946+0000","last_became_peered":"2024-04-25T01:25:52.350946+0000","last_unstale":"2024-04-25T01:26:03.716731+0000","last_undegraded":"2024-04-25T01:26:03.716731+0000","last_fullsized":"2024-04-25T01:26:03.716731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416482+0000","last_change":"2024-04-25T01:26:01.415799+0000","last_active":"2024-04-25T01:26:03.416482+0000","last_peered":"2024-04-25T01:26:03.416482+0000","last_clean":"2024-04-25T01:26:03.416482+0000","last_became_active":"2024-04-25T01:26:01.415572+0000","last_became_peered":"2024-04-25T01:26:01.415572+0000","last_unstale":"2024-04-25T01:26:03.416482+0000","last_undegraded":"2024-04-25T01:26:03.416482+0000","last_fullsized":"2024-04-25T01:26:03.416482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:54.034132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717227+0000","last_change":"2024-04-25T01:26:02.422959+0000","last_active":"2024-04-25T01:26:03.717227+0000","last_peered":"2024-04-25T01:26:03.717227+0000","last_clean":"2024-04-25T01:26:03.717227+0000","last_became_active":"2024-04-25T01:26:02.422210+0000","last_became_peered":"2024-04-25T01:26:02.422210+0000","last_unstale":"2024-04-25T01:26:03.717227+0000","last_undegraded":"2024-04-25T01:26:03.717227+0000","last_fullsized":"2024-04-25T01:26:03.717227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507159+0000","last_change":"2024-04-25T01:25:53.368187+0000","last_active":"2024-04-25T01:26:03.507159+0000","last_peered":"2024-04-25T01:26:03.507159+0000","last_clean":"2024-04-25T01:26:03.507159+0000","last_became_active":"2024-04-25T01:25:53.367664+0000","last_became_peered":"2024-04-25T01:25:53.367664+0000","last_unstale":"2024-04-25T01:26:03.507159+0000","last_undegraded":"2024-04-25T01:26:03.507159+0000","last_fullsized":"2024-04-25T01:26:03.507159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717680+0000","last_change":"2024-04-25T01:25:52.357397+0000","last_active":"2024-04-25T01:26:03.717680+0000","last_peered":"2024-04-25T01:26:03.717680+0000","last_clean":"2024-04-25T01:26:03.717680+0000","last_became_active":"2024-04-25T01:25:52.356881+0000","last_became_peered":"2024-04-25T01:25:52.356881+0000","last_unstale":"2024-04-25T01:26:03.717680+0000","last_undegraded":"2024-04-25T01:26:03.717680+0000","last_fullsized":"2024-04-25T01:26:03.717680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"24'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:24.096754+0000","last_change":"2024-04-25T01:26:01.420103+0000","last_active":"2024-04-25T01:26:24.096754+0000","last_peered":"2024-04-25T01:26:24.096754+0000","last_clean":"2024-04-25T01:26:24.096754+0000","last_became_active":"2024-04-25T01:26:01.419526+0000","last_became_peered":"2024-04-25T01:26:01.419526+0000","last_unstale":"2024-04-25T01:26:24.096754+0000","last_undegraded":"2024-04-25T01:26:24.096754+0000","last_fullsized":"2024-04-25T01:26:24.096754+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:27.042376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2053,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416191+0000","last_change":"2024-04-25T01:26:02.421488+0000","last_active":"2024-04-25T01:26:03.416191+0000","last_peered":"2024-04-25T01:26:03.416191+0000","last_clean":"2024-04-25T01:26:03.416191+0000","last_became_active":"2024-04-25T01:26:02.419308+0000","last_became_peered":"2024-04-25T01:26:02.419308+0000","last_unstale":"2024-04-25T01:26:03.416191+0000","last_undegraded":"2024-04-25T01:26:03.416191+0000","last_fullsized":"2024-04-25T01:26:03.416191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716019+0000","last_change":"2024-04-25T01:25:53.370720+0000","last_active":"2024-04-25T01:26:03.716019+0000","last_peered":"2024-04-25T01:26:03.716019+0000","last_clean":"2024-04-25T01:26:03.716019+0000","last_became_active":"2024-04-25T01:25:53.370142+0000","last_became_peered":"2024-04-25T01:25:53.370142+0000","last_unstale":"2024-04-25T01:26:03.716019+0000","last_undegraded":"2024-04-25T01:26:03.716019+0000","last_fullsized":"2024-04-25T01:26:03.716019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:52.833379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716951+0000","last_change":"2024-04-25T01:25:52.349092+0000","last_active":"2024-04-25T01:26:03.716951+0000","last_peered":"2024-04-25T01:26:03.716951+0000","last_clean":"2024-04-25T01:26:03.716951+0000","last_became_active":"2024-04-25T01:25:52.348846+0000","last_became_peered":"2024-04-25T01:25:52.348846+0000","last_unstale":"2024-04-25T01:26:03.716951+0000","last_undegraded":"2024-04-25T01:26:03.716951+0000","last_fullsized":"2024-04-25T01:26:03.716951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:21.982440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717840+0000","last_change":"2024-04-25T01:26:01.414517+0000","last_active":"2024-04-25T01:26:03.717840+0000","last_peered":"2024-04-25T01:26:03.717840+0000","last_clean":"2024-04-25T01:26:03.717840+0000","last_became_active":"2024-04-25T01:26:01.414243+0000","last_became_peered":"2024-04-25T01:26:01.414243+0000","last_unstale":"2024-04-25T01:26:03.717840+0000","last_undegraded":"2024-04-25T01:26:03.717840+0000","last_fullsized":"2024-04-25T01:26:03.717840+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:48.491461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415418+0000","last_change":"2024-04-25T01:26:02.415893+0000","last_active":"2024-04-25T01:26:03.415418+0000","last_peered":"2024-04-25T01:26:03.415418+0000","last_clean":"2024-04-25T01:26:03.415418+0000","last_became_active":"2024-04-25T01:26:02.415596+0000","last_became_peered":"2024-04-25T01:26:02.415596+0000","last_unstale":"2024-04-25T01:26:03.415418+0000","last_undegraded":"2024-04-25T01:26:03.415418+0000","last_fullsized":"2024-04-25T01:26:03.415418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507428+0000","last_change":"2024-04-25T01:25:53.373572+0000","last_active":"2024-04-25T01:26:03.507428+0000","last_peered":"2024-04-25T01:26:03.507428+0000","last_clean":"2024-04-25T01:26:03.507428+0000","last_became_active":"2024-04-25T01:25:53.373129+0000","last_became_peered":"2024-04-25T01:25:53.373129+0000","last_unstale":"2024-04-25T01:26:03.507428+0000","last_undegraded":"2024-04-25T01:26:03.507428+0000","last_fullsized":"2024-04-25T01:26:03.507428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:22.122293+0000","last_change":"2024-04-25T01:25:52.350446+0000","last_active":"2024-04-25T01:26:22.122293+0000","last_peered":"2024-04-25T01:26:22.122293+0000","last_clean":"2024-04-25T01:26:22.122293+0000","last_became_active":"2024-04-25T01:25:52.349448+0000","last_became_peered":"2024-04-25T01:25:52.349448+0000","last_unstale":"2024-04-25T01:26:22.122293+0000","last_undegraded":"2024-04-25T01:26:22.122293+0000","last_fullsized":"2024-04-25T01:26:22.122293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2053,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716463+0000","last_change":"2024-04-25T01:26:01.418794+0000","last_active":"2024-04-25T01:26:03.716463+0000","last_peered":"2024-04-25T01:26:03.716463+0000","last_clean":"2024-04-25T01:26:03.716463+0000","last_became_active":"2024-04-25T01:26:01.418541+0000","last_became_peered":"2024-04-25T01:26:01.418541+0000","last_unstale":"2024-04-25T01:26:03.716463+0000","last_undegraded":"2024-04-25T01:26:03.716463+0000","last_fullsized":"2024-04-25T01:26:03.716463+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717164+0000","last_change":"2024-04-25T01:26:02.425391+0000","last_active":"2024-04-25T01:26:03.717164+0000","last_peered":"2024-04-25T01:26:03.717164+0000","last_clean":"2024-04-25T01:26:03.717164+0000","last_became_active":"2024-04-25T01:26:02.425106+0000","last_became_peered":"2024-04-25T01:26:02.425106+0000","last_unstale":"2024-04-25T01:26:03.717164+0000","last_undegraded":"2024-04-25T01:26:03.717164+0000","last_fullsized":"2024-04-25T01:26:03.717164+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416706+0000","last_change":"2024-04-25T01:25:53.366416+0000","last_active":"2024-04-25T01:26:03.416706+0000","last_peered":"2024-04-25T01:26:03.416706+0000","last_clean":"2024-04-25T01:26:03.416706+0000","last_became_active":"2024-04-25T01:25:53.366157+0000","last_became_peered":"2024-04-25T01:25:53.366157+0000","last_unstale":"2024-04-25T01:26:03.416706+0000","last_undegraded":"2024-04-25T01:26:03.416706+0000","last_fullsized":"2024-04-25T01:26:03.416706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:26.277566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717854+0000","last_change":"2024-04-25T01:25:52.357610+0000","last_active":"2024-04-25T01:26:03.717854+0000","last_peered":"2024-04-25T01:26:03.717854+0000","last_clean":"2024-04-25T01:26:03.717854+0000","last_became_active":"2024-04-25T01:25:52.357355+0000","last_became_peered":"2024-04-25T01:25:52.357355+0000","last_unstale":"2024-04-25T01:26:03.717854+0000","last_undegraded":"2024-04-25T01:26:03.717854+0000","last_fullsized":"2024-04-25T01:26:03.717854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416422+0000","last_change":"2024-04-25T01:26:01.412666+0000","last_active":"2024-04-25T01:26:03.416422+0000","last_peered":"2024-04-25T01:26:03.416422+0000","last_clean":"2024-04-25T01:26:03.416422+0000","last_became_active":"2024-04-25T01:26:01.412132+0000","last_became_peered":"2024-04-25T01:26:01.412132+0000","last_unstale":"2024-04-25T01:26:03.416422+0000","last_undegraded":"2024-04-25T01:26:03.416422+0000","last_fullsized":"2024-04-25T01:26:03.416422+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716704+0000","last_change":"2024-04-25T01:26:02.413106+0000","last_active":"2024-04-25T01:26:03.716704+0000","last_peered":"2024-04-25T01:26:03.716704+0000","last_clean":"2024-04-25T01:26:03.716704+0000","last_became_active":"2024-04-25T01:26:02.412759+0000","last_became_peered":"2024-04-25T01:26:02.412759+0000","last_unstale":"2024-04-25T01:26:03.716704+0000","last_undegraded":"2024-04-25T01:26:03.716704+0000","last_fullsized":"2024-04-25T01:26:03.716704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:28.049190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716160+0000","last_change":"2024-04-25T01:25:53.374311+0000","last_active":"2024-04-25T01:26:03.716160+0000","last_peered":"2024-04-25T01:26:03.716160+0000","last_clean":"2024-04-25T01:26:03.716160+0000","last_became_active":"2024-04-25T01:25:53.373890+0000","last_became_peered":"2024-04-25T01:25:53.373890+0000","last_unstale":"2024-04-25T01:26:03.716160+0000","last_undegraded":"2024-04-25T01:26:03.716160+0000","last_fullsized":"2024-04-25T01:26:03.716160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717907+0000","last_change":"2024-04-25T01:25:52.356231+0000","last_active":"2024-04-25T01:26:03.717907+0000","last_peered":"2024-04-25T01:26:03.717907+0000","last_clean":"2024-04-25T01:26:03.717907+0000","last_became_active":"2024-04-25T01:25:52.355913+0000","last_became_peered":"2024-04-25T01:25:52.355913+0000","last_unstale":"2024-04-25T01:26:03.717907+0000","last_undegraded":"2024-04-25T01:26:03.717907+0000","last_fullsized":"2024-04-25T01:26:03.717907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415065+0000","last_change":"2024-04-25T01:26:01.419513+0000","last_active":"2024-04-25T01:26:03.415065+0000","last_peered":"2024-04-25T01:26:03.415065+0000","last_clean":"2024-04-25T01:26:03.415065+0000","last_became_active":"2024-04-25T01:26:01.419015+0000","last_became_peered":"2024-04-25T01:26:01.419015+0000","last_unstale":"2024-04-25T01:26:03.415065+0000","last_undegraded":"2024-04-25T01:26:03.415065+0000","last_fullsized":"2024-04-25T01:26:03.415065+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:10.486367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507391+0000","last_change":"2024-04-25T01:26:02.425241+0000","last_active":"2024-04-25T01:26:03.507391+0000","last_peered":"2024-04-25T01:26:03.507391+0000","last_clean":"2024-04-25T01:26:03.507391+0000","last_became_active":"2024-04-25T01:26:02.424960+0000","last_became_peered":"2024-04-25T01:26:02.424960+0000","last_unstale":"2024-04-25T01:26:03.507391+0000","last_undegraded":"2024-04-25T01:26:03.507391+0000","last_fullsized":"2024-04-25T01:26:03.507391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717765+0000","last_change":"2024-04-25T01:25:53.371445+0000","last_active":"2024-04-25T01:26:03.717765+0000","last_peered":"2024-04-25T01:26:03.717765+0000","last_clean":"2024-04-25T01:26:03.717765+0000","last_became_active":"2024-04-25T01:25:53.371086+0000","last_became_peered":"2024-04-25T01:25:53.371086+0000","last_unstale":"2024-04-25T01:26:03.717765+0000","last_undegraded":"2024-04-25T01:26:03.717765+0000","last_fullsized":"2024-04-25T01:26:03.717765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.718056+0000","last_change":"2024-04-25T01:25:52.357459+0000","last_active":"2024-04-25T01:26:03.718056+0000","last_peered":"2024-04-25T01:26:03.718056+0000","last_clean":"2024-04-25T01:26:03.718056+0000","last_became_active":"2024-04-25T01:25:52.357182+0000","last_became_peered":"2024-04-25T01:25:52.357182+0000","last_unstale":"2024-04-25T01:26:03.718056+0000","last_undegraded":"2024-04-25T01:26:03.718056+0000","last_fullsized":"2024-04-25T01:26:03.718056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415347+0000","last_change":"2024-04-25T01:26:01.415321+0000","last_active":"2024-04-25T01:26:03.415347+0000","last_peered":"2024-04-25T01:26:03.415347+0000","last_clean":"2024-04-25T01:26:03.415347+0000","last_became_active":"2024-04-25T01:26:01.415020+0000","last_became_peered":"2024-04-25T01:26:01.415020+0000","last_unstale":"2024-04-25T01:26:03.415347+0000","last_undegraded":"2024-04-25T01:26:03.415347+0000","last_fullsized":"2024-04-25T01:26:03.415347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:20.090411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717822+0000","last_change":"2024-04-25T01:26:02.416917+0000","last_active":"2024-04-25T01:26:03.717822+0000","last_peered":"2024-04-25T01:26:03.717822+0000","last_clean":"2024-04-25T01:26:03.717822+0000","last_became_active":"2024-04-25T01:26:02.416623+0000","last_became_peered":"2024-04-25T01:26:02.416623+0000","last_unstale":"2024-04-25T01:26:03.717822+0000","last_undegraded":"2024-04-25T01:26:03.717822+0000","last_fullsized":"2024-04-25T01:26:03.717822+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:55.040946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716446+0000","last_change":"2024-04-25T01:26:01.415472+0000","last_active":"2024-04-25T01:26:03.716446+0000","last_peered":"2024-04-25T01:26:03.716446+0000","last_clean":"2024-04-25T01:26:03.716446+0000","last_became_active":"2024-04-25T01:26:01.415184+0000","last_became_peered":"2024-04-25T01:26:01.415184+0000","last_unstale":"2024-04-25T01:26:03.716446+0000","last_undegraded":"2024-04-25T01:26:03.716446+0000","last_fullsized":"2024-04-25T01:26:03.716446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716757+0000","last_change":"2024-04-25T01:25:53.363043+0000","last_active":"2024-04-25T01:26:03.716757+0000","last_peered":"2024-04-25T01:26:03.716757+0000","last_clean":"2024-04-25T01:26:03.716757+0000","last_became_active":"2024-04-25T01:25:53.362601+0000","last_became_peered":"2024-04-25T01:25:53.362601+0000","last_unstale":"2024-04-25T01:26:03.716757+0000","last_undegraded":"2024-04-25T01:26:03.716757+0000","last_fullsized":"2024-04-25T01:26:03.716757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716830+0000","last_change":"2024-04-25T01:25:52.358081+0000","last_active":"2024-04-25T01:26:03.716830+0000","last_peered":"2024-04-25T01:26:03.716830+0000","last_clean":"2024-04-25T01:26:03.716830+0000","last_became_active":"2024-04-25T01:25:52.357365+0000","last_became_peered":"2024-04-25T01:25:52.357365+0000","last_unstale":"2024-04-25T01:26:03.716830+0000","last_undegraded":"2024-04-25T01:26:03.716830+0000","last_fullsized":"2024-04-25T01:26:03.716830+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716855+0000","last_change":"2024-04-25T01:26:02.418872+0000","last_active":"2024-04-25T01:26:03.716855+0000","last_peered":"2024-04-25T01:26:03.716855+0000","last_clean":"2024-04-25T01:26:03.716855+0000","last_became_active":"2024-04-25T01:26:02.418614+0000","last_became_peered":"2024-04-25T01:26:02.418614+0000","last_unstale":"2024-04-25T01:26:03.716855+0000","last_undegraded":"2024-04-25T01:26:03.716855+0000","last_fullsized":"2024-04-25T01:26:03.716855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:49.498275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717818+0000","last_change":"2024-04-25T01:26:01.410258+0000","last_active":"2024-04-25T01:26:03.717818+0000","last_peered":"2024-04-25T01:26:03.717818+0000","last_clean":"2024-04-25T01:26:03.717818+0000","last_became_active":"2024-04-25T01:26:01.410004+0000","last_became_peered":"2024-04-25T01:26:01.410004+0000","last_unstale":"2024-04-25T01:26:03.717818+0000","last_undegraded":"2024-04-25T01:26:03.717818+0000","last_fullsized":"2024-04-25T01:26:03.717818+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:48.865940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415284+0000","last_change":"2024-04-25T01:25:53.363794+0000","last_active":"2024-04-25T01:26:03.415284+0000","last_peered":"2024-04-25T01:26:03.415284+0000","last_clean":"2024-04-25T01:26:03.415284+0000","last_became_active":"2024-04-25T01:25:53.363509+0000","last_became_peered":"2024-04-25T01:25:53.363509+0000","last_unstale":"2024-04-25T01:26:03.415284+0000","last_undegraded":"2024-04-25T01:26:03.415284+0000","last_fullsized":"2024-04-25T01:26:03.415284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415685+0000","last_change":"2024-04-25T01:25:52.347464+0000","last_active":"2024-04-25T01:26:03.415685+0000","last_peered":"2024-04-25T01:26:03.415685+0000","last_clean":"2024-04-25T01:26:03.415685+0000","last_became_active":"2024-04-25T01:25:52.347138+0000","last_became_peered":"2024-04-25T01:25:52.347138+0000","last_unstale":"2024-04-25T01:26:03.415685+0000","last_undegraded":"2024-04-25T01:26:03.415685+0000","last_fullsized":"2024-04-25T01:26:03.415685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717780+0000","last_change":"2024-04-25T01:26:02.425948+0000","last_active":"2024-04-25T01:26:03.717780+0000","last_peered":"2024-04-25T01:26:03.717780+0000","last_clean":"2024-04-25T01:26:03.717780+0000","last_became_active":"2024-04-25T01:26:02.424947+0000","last_became_peered":"2024-04-25T01:26:02.424947+0000","last_unstale":"2024-04-25T01:26:03.717780+0000","last_undegraded":"2024-04-25T01:26:03.717780+0000","last_fullsized":"2024-04-25T01:26:03.717780+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716700+0000","last_change":"2024-04-25T01:25:52.349989+0000","last_active":"2024-04-25T01:26:03.716700+0000","last_peered":"2024-04-25T01:26:03.716700+0000","last_clean":"2024-04-25T01:26:03.716700+0000","last_became_active":"2024-04-25T01:25:52.349672+0000","last_became_peered":"2024-04-25T01:25:52.349672+0000","last_unstale":"2024-04-25T01:26:03.716700+0000","last_undegraded":"2024-04-25T01:26:03.716700+0000","last_fullsized":"2024-04-25T01:26:03.716700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716805+0000","last_change":"2024-04-25T01:25:53.356610+0000","last_active":"2024-04-25T01:26:03.716805+0000","last_peered":"2024-04-25T01:26:03.716805+0000","last_clean":"2024-04-25T01:26:03.716805+0000","last_became_active":"2024-04-25T01:25:53.356015+0000","last_became_peered":"2024-04-25T01:25:53.356015+0000","last_unstale":"2024-04-25T01:26:03.716805+0000","last_undegraded":"2024-04-25T01:26:03.716805+0000","last_fullsized":"2024-04-25T01:26:03.716805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.417192+0000","last_change":"2024-04-25T01:26:01.420261+0000","last_active":"2024-04-25T01:26:03.417192+0000","last_peered":"2024-04-25T01:26:03.417192+0000","last_clean":"2024-04-25T01:26:03.417192+0000","last_became_active":"2024-04-25T01:26:01.420003+0000","last_became_peered":"2024-04-25T01:26:01.420003+0000","last_unstale":"2024-04-25T01:26:03.417192+0000","last_undegraded":"2024-04-25T01:26:03.417192+0000","last_fullsized":"2024-04-25T01:26:03.417192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:28.695853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416039+0000","last_change":"2024-04-25T01:26:02.426327+0000","last_active":"2024-04-25T01:26:03.416039+0000","last_peered":"2024-04-25T01:26:03.416039+0000","last_clean":"2024-04-25T01:26:03.416039+0000","last_became_active":"2024-04-25T01:26:02.426054+0000","last_became_peered":"2024-04-25T01:26:02.426054+0000","last_unstale":"2024-04-25T01:26:03.416039+0000","last_undegraded":"2024-04-25T01:26:03.416039+0000","last_fullsized":"2024-04-25T01:26:03.416039+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:17.809738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415686+0000","last_change":"2024-04-25T01:25:52.353955+0000","last_active":"2024-04-25T01:26:03.415686+0000","last_peered":"2024-04-25T01:26:03.415686+0000","last_clean":"2024-04-25T01:26:03.415686+0000","last_became_active":"2024-04-25T01:25:52.353659+0000","last_became_peered":"2024-04-25T01:25:52.353659+0000","last_unstale":"2024-04-25T01:26:03.415686+0000","last_undegraded":"2024-04-25T01:26:03.415686+0000","last_fullsized":"2024-04-25T01:26:03.415686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416142+0000","last_change":"2024-04-25T01:25:53.366586+0000","last_active":"2024-04-25T01:26:03.416142+0000","last_peered":"2024-04-25T01:26:03.416142+0000","last_clean":"2024-04-25T01:26:03.416142+0000","last_became_active":"2024-04-25T01:25:53.366302+0000","last_became_peered":"2024-04-25T01:25:53.366302+0000","last_unstale":"2024-04-25T01:26:03.416142+0000","last_undegraded":"2024-04-25T01:26:03.416142+0000","last_fullsized":"2024-04-25T01:26:03.416142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416415+0000","last_change":"2024-04-25T01:26:01.420960+0000","last_active":"2024-04-25T01:26:03.416415+0000","last_peered":"2024-04-25T01:26:03.416415+0000","last_clean":"2024-04-25T01:26:03.416415+0000","last_became_active":"2024-04-25T01:26:01.420713+0000","last_became_peered":"2024-04-25T01:26:01.420713+0000","last_unstale":"2024-04-25T01:26:03.416415+0000","last_undegraded":"2024-04-25T01:26:03.416415+0000","last_fullsized":"2024-04-25T01:26:03.416415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:29.489275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416652+0000","last_change":"2024-04-25T01:26:02.419282+0000","last_active":"2024-04-25T01:26:03.416652+0000","last_peered":"2024-04-25T01:26:03.416652+0000","last_clean":"2024-04-25T01:26:03.416652+0000","last_became_active":"2024-04-25T01:26:02.418802+0000","last_became_peered":"2024-04-25T01:26:02.418802+0000","last_unstale":"2024-04-25T01:26:03.416652+0000","last_undegraded":"2024-04-25T01:26:03.416652+0000","last_fullsized":"2024-04-25T01:26:03.416652+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:46.807175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716343+0000","last_change":"2024-04-25T01:25:53.367143+0000","last_active":"2024-04-25T01:26:03.716343+0000","last_peered":"2024-04-25T01:26:03.716343+0000","last_clean":"2024-04-25T01:26:03.716343+0000","last_became_active":"2024-04-25T01:25:53.366847+0000","last_became_peered":"2024-04-25T01:25:53.366847+0000","last_unstale":"2024-04-25T01:26:03.716343+0000","last_undegraded":"2024-04-25T01:26:03.716343+0000","last_fullsized":"2024-04-25T01:26:03.716343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716783+0000","last_change":"2024-04-25T01:26:02.416438+0000","last_active":"2024-04-25T01:26:03.716783+0000","last_peered":"2024-04-25T01:26:03.716783+0000","last_clean":"2024-04-25T01:26:03.716783+0000","last_became_active":"2024-04-25T01:26:02.416170+0000","last_became_peered":"2024-04-25T01:26:02.416170+0000","last_unstale":"2024-04-25T01:26:03.716783+0000","last_undegraded":"2024-04-25T01:26:03.716783+0000","last_fullsized":"2024-04-25T01:26:03.716783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415637+0000","last_change":"2024-04-25T01:25:53.371345+0000","last_active":"2024-04-25T01:26:03.415637+0000","last_peered":"2024-04-25T01:26:03.415637+0000","last_clean":"2024-04-25T01:26:03.415637+0000","last_became_active":"2024-04-25T01:25:53.371080+0000","last_became_peered":"2024-04-25T01:25:53.371080+0000","last_unstale":"2024-04-25T01:26:03.415637+0000","last_undegraded":"2024-04-25T01:26:03.415637+0000","last_fullsized":"2024-04-25T01:26:03.415637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:09.855763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717353+0000","last_change":"2024-04-25T01:26:02.419688+0000","last_active":"2024-04-25T01:26:03.717353+0000","last_peered":"2024-04-25T01:26:03.717353+0000","last_clean":"2024-04-25T01:26:03.717353+0000","last_became_active":"2024-04-25T01:26:02.419417+0000","last_became_peered":"2024-04-25T01:26:02.419417+0000","last_unstale":"2024-04-25T01:26:03.717353+0000","last_undegraded":"2024-04-25T01:26:03.717353+0000","last_fullsized":"2024-04-25T01:26:03.717353+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717442+0000","last_change":"2024-04-25T01:25:53.369850+0000","last_active":"2024-04-25T01:26:03.717442+0000","last_peered":"2024-04-25T01:26:03.717442+0000","last_clean":"2024-04-25T01:26:03.717442+0000","last_became_active":"2024-04-25T01:25:53.369350+0000","last_became_peered":"2024-04-25T01:25:53.369350+0000","last_unstale":"2024-04-25T01:26:03.717442+0000","last_undegraded":"2024-04-25T01:26:03.717442+0000","last_fullsized":"2024-04-25T01:26:03.717442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717742+0000","last_change":"2024-04-25T01:26:02.425528+0000","last_active":"2024-04-25T01:26:03.717742+0000","last_peered":"2024-04-25T01:26:03.717742+0000","last_clean":"2024-04-25T01:26:03.717742+0000","last_became_active":"2024-04-25T01:26:02.425160+0000","last_became_peered":"2024-04-25T01:26:02.425160+0000","last_unstale":"2024-04-25T01:26:03.717742+0000","last_undegraded":"2024-04-25T01:26:03.717742+0000","last_fullsized":"2024-04-25T01:26:03.717742+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716710+0000","last_change":"2024-04-25T01:25:53.357142+0000","last_active":"2024-04-25T01:26:03.716710+0000","last_peered":"2024-04-25T01:26:03.716710+0000","last_clean":"2024-04-25T01:26:03.716710+0000","last_became_active":"2024-04-25T01:25:53.356854+0000","last_became_peered":"2024-04-25T01:25:53.356854+0000","last_unstale":"2024-04-25T01:26:03.716710+0000","last_undegraded":"2024-04-25T01:26:03.716710+0000","last_fullsized":"2024-04-25T01:26:03.716710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507419+0000","last_change":"2024-04-25T01:26:02.417492+0000","last_active":"2024-04-25T01:26:03.507419+0000","last_peered":"2024-04-25T01:26:03.507419+0000","last_clean":"2024-04-25T01:26:03.507419+0000","last_became_active":"2024-04-25T01:26:02.417234+0000","last_became_peered":"2024-04-25T01:26:02.417234+0000","last_unstale":"2024-04-25T01:26:03.507419+0000","last_undegraded":"2024-04-25T01:26:03.507419+0000","last_fullsized":"2024-04-25T01:26:03.507419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416287+0000","last_change":"2024-04-25T01:25:53.364575+0000","last_active":"2024-04-25T01:26:03.416287+0000","last_peered":"2024-04-25T01:26:03.416287+0000","last_clean":"2024-04-25T01:26:03.416287+0000","last_became_active":"2024-04-25T01:25:53.364311+0000","last_became_peered":"2024-04-25T01:25:53.364311+0000","last_unstale":"2024-04-25T01:26:03.416287+0000","last_undegraded":"2024-04-25T01:26:03.416287+0000","last_fullsized":"2024-04-25T01:26:03.416287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415942+0000","last_change":"2024-04-25T01:26:02.426813+0000","last_active":"2024-04-25T01:26:03.415942+0000","last_peered":"2024-04-25T01:26:03.415942+0000","last_clean":"2024-04-25T01:26:03.415942+0000","last_became_active":"2024-04-25T01:26:02.426552+0000","last_became_peered":"2024-04-25T01:26:02.426552+0000","last_unstale":"2024-04-25T01:26:03.415942+0000","last_undegraded":"2024-04-25T01:26:03.415942+0000","last_fullsized":"2024-04-25T01:26:03.415942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:03.687174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717821+0000","last_change":"2024-04-25T01:25:53.368413+0000","last_active":"2024-04-25T01:26:03.717821+0000","last_peered":"2024-04-25T01:26:03.717821+0000","last_clean":"2024-04-25T01:26:03.717821+0000","last_became_active":"2024-04-25T01:25:53.368176+0000","last_became_peered":"2024-04-25T01:25:53.368176+0000","last_unstale":"2024-04-25T01:26:03.717821+0000","last_undegraded":"2024-04-25T01:26:03.717821+0000","last_fullsized":"2024-04-25T01:26:03.717821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.506869+0000","last_change":"2024-04-25T01:26:02.425185+0000","last_active":"2024-04-25T01:26:03.506869+0000","last_peered":"2024-04-25T01:26:03.506869+0000","last_clean":"2024-04-25T01:26:03.506869+0000","last_became_active":"2024-04-25T01:26:02.424903+0000","last_became_peered":"2024-04-25T01:26:02.424903+0000","last_unstale":"2024-04-25T01:26:03.506869+0000","last_undegraded":"2024-04-25T01:26:03.506869+0000","last_fullsized":"2024-04-25T01:26:03.506869+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717561+0000","last_change":"2024-04-25T01:25:53.372638+0000","last_active":"2024-04-25T01:26:03.717561+0000","last_peered":"2024-04-25T01:26:03.717561+0000","last_clean":"2024-04-25T01:26:03.717561+0000","last_became_active":"2024-04-25T01:25:53.372001+0000","last_became_peered":"2024-04-25T01:25:53.372001+0000","last_unstale":"2024-04-25T01:26:03.717561+0000","last_undegraded":"2024-04-25T01:26:03.717561+0000","last_fullsized":"2024-04-25T01:26:03.717561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716617+0000","last_change":"2024-04-25T01:26:02.420867+0000","last_active":"2024-04-25T01:26:03.716617+0000","last_peered":"2024-04-25T01:26:03.716617+0000","last_clean":"2024-04-25T01:26:03.716617+0000","last_became_active":"2024-04-25T01:26:02.420621+0000","last_became_peered":"2024-04-25T01:26:02.420621+0000","last_unstale":"2024-04-25T01:26:03.716617+0000","last_undegraded":"2024-04-25T01:26:03.716617+0000","last_fullsized":"2024-04-25T01:26:03.716617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:11.493181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416565+0000","last_change":"2024-04-25T01:25:53.365509+0000","last_active":"2024-04-25T01:26:03.416565+0000","last_peered":"2024-04-25T01:26:03.416565+0000","last_clean":"2024-04-25T01:26:03.416565+0000","last_became_active":"2024-04-25T01:25:53.365245+0000","last_became_peered":"2024-04-25T01:25:53.365245+0000","last_unstale":"2024-04-25T01:26:03.416565+0000","last_undegraded":"2024-04-25T01:26:03.416565+0000","last_fullsized":"2024-04-25T01:26:03.416565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416549+0000","last_change":"2024-04-25T01:26:02.415672+0000","last_active":"2024-04-25T01:26:03.416549+0000","last_peered":"2024-04-25T01:26:03.416549+0000","last_clean":"2024-04-25T01:26:03.416549+0000","last_became_active":"2024-04-25T01:26:02.415389+0000","last_became_peered":"2024-04-25T01:26:02.415389+0000","last_unstale":"2024-04-25T01:26:03.416549+0000","last_undegraded":"2024-04-25T01:26:03.416549+0000","last_fullsized":"2024-04-25T01:26:03.416549+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:52.342053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716741+0000","last_change":"2024-04-25T01:25:53.365391+0000","last_active":"2024-04-25T01:26:03.716741+0000","last_peered":"2024-04-25T01:26:03.716741+0000","last_clean":"2024-04-25T01:26:03.716741+0000","last_became_active":"2024-04-25T01:25:53.364829+0000","last_became_peered":"2024-04-25T01:25:53.364829+0000","last_unstale":"2024-04-25T01:26:03.716741+0000","last_undegraded":"2024-04-25T01:26:03.716741+0000","last_fullsized":"2024-04-25T01:26:03.716741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507955+0000","last_change":"2024-04-25T01:26:02.418292+0000","last_active":"2024-04-25T01:26:03.507955+0000","last_peered":"2024-04-25T01:26:03.507955+0000","last_clean":"2024-04-25T01:26:03.507955+0000","last_became_active":"2024-04-25T01:26:02.418036+0000","last_became_peered":"2024-04-25T01:26:02.418036+0000","last_unstale":"2024-04-25T01:26:03.507955+0000","last_undegraded":"2024-04-25T01:26:03.507955+0000","last_fullsized":"2024-04-25T01:26:03.507955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415608+0000","last_change":"2024-04-25T01:25:53.355667+0000","last_active":"2024-04-25T01:26:03.415608+0000","last_peered":"2024-04-25T01:26:03.415608+0000","last_clean":"2024-04-25T01:26:03.415608+0000","last_became_active":"2024-04-25T01:25:53.355417+0000","last_became_peered":"2024-04-25T01:25:53.355417+0000","last_unstale":"2024-04-25T01:26:03.415608+0000","last_undegraded":"2024-04-25T01:26:03.415608+0000","last_fullsized":"2024-04-25T01:26:03.415608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415307+0000","last_change":"2024-04-25T01:26:02.414994+0000","last_active":"2024-04-25T01:26:03.415307+0000","last_peered":"2024-04-25T01:26:03.415307+0000","last_clean":"2024-04-25T01:26:03.415307+0000","last_became_active":"2024-04-25T01:26:02.414735+0000","last_became_peered":"2024-04-25T01:26:02.414735+0000","last_unstale":"2024-04-25T01:26:03.415307+0000","last_undegraded":"2024-04-25T01:26:03.415307+0000","last_fullsized":"2024-04-25T01:26:03.415307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:44.960251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717184+0000","last_change":"2024-04-25T01:25:53.371240+0000","last_active":"2024-04-25T01:26:03.717184+0000","last_peered":"2024-04-25T01:26:03.717184+0000","last_clean":"2024-04-25T01:26:03.717184+0000","last_became_active":"2024-04-25T01:25:53.370164+0000","last_became_peered":"2024-04-25T01:25:53.370164+0000","last_unstale":"2024-04-25T01:26:03.717184+0000","last_undegraded":"2024-04-25T01:26:03.717184+0000","last_fullsized":"2024-04-25T01:26:03.717184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416367+0000","last_change":"2024-04-25T01:26:02.418272+0000","last_active":"2024-04-25T01:26:03.416367+0000","last_peered":"2024-04-25T01:26:03.416367+0000","last_clean":"2024-04-25T01:26:03.416367+0000","last_became_active":"2024-04-25T01:26:02.417977+0000","last_became_peered":"2024-04-25T01:26:02.417977+0000","last_unstale":"2024-04-25T01:26:03.416367+0000","last_undegraded":"2024-04-25T01:26:03.416367+0000","last_fullsized":"2024-04-25T01:26:03.416367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716511+0000","last_change":"2024-04-25T01:25:53.376415+0000","last_active":"2024-04-25T01:26:03.716511+0000","last_peered":"2024-04-25T01:26:03.716511+0000","last_clean":"2024-04-25T01:26:03.716511+0000","last_became_active":"2024-04-25T01:25:53.376104+0000","last_became_peered":"2024-04-25T01:25:53.376104+0000","last_unstale":"2024-04-25T01:26:03.716511+0000","last_undegraded":"2024-04-25T01:26:03.716511+0000","last_fullsized":"2024-04-25T01:26:03.716511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416512+0000","last_change":"2024-04-25T01:26:02.426333+0000","last_active":"2024-04-25T01:26:03.416512+0000","last_peered":"2024-04-25T01:26:03.416512+0000","last_clean":"2024-04-25T01:26:03.416512+0000","last_became_active":"2024-04-25T01:26:02.426069+0000","last_became_peered":"2024-04-25T01:26:02.426069+0000","last_unstale":"2024-04-25T01:26:03.416512+0000","last_undegraded":"2024-04-25T01:26:03.416512+0000","last_fullsized":"2024-04-25T01:26:03.416512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716975+0000","last_change":"2024-04-25T01:25:53.370314+0000","last_active":"2024-04-25T01:26:03.716975+0000","last_peered":"2024-04-25T01:26:03.716975+0000","last_clean":"2024-04-25T01:26:03.716975+0000","last_became_active":"2024-04-25T01:25:53.370049+0000","last_became_peered":"2024-04-25T01:25:53.370049+0000","last_unstale":"2024-04-25T01:26:03.716975+0000","last_undegraded":"2024-04-25T01:26:03.716975+0000","last_fullsized":"2024-04-25T01:26:03.716975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415866+0000","last_change":"2024-04-25T01:26:02.415464+0000","last_active":"2024-04-25T01:26:03.415866+0000","last_peered":"2024-04-25T01:26:03.415866+0000","last_clean":"2024-04-25T01:26:03.415866+0000","last_became_active":"2024-04-25T01:26:02.415173+0000","last_became_peered":"2024-04-25T01:26:02.415173+0000","last_unstale":"2024-04-25T01:26:03.415866+0000","last_undegraded":"2024-04-25T01:26:03.415866+0000","last_fullsized":"2024-04-25T01:26:03.415866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.343169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507691+0000","last_change":"2024-04-25T01:25:53.369585+0000","last_active":"2024-04-25T01:26:03.507691+0000","last_peered":"2024-04-25T01:26:03.507691+0000","last_clean":"2024-04-25T01:26:03.507691+0000","last_became_active":"2024-04-25T01:25:53.368499+0000","last_became_peered":"2024-04-25T01:25:53.368499+0000","last_unstale":"2024-04-25T01:26:03.507691+0000","last_undegraded":"2024-04-25T01:26:03.507691+0000","last_fullsized":"2024-04-25T01:26:03.507691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716525+0000","last_change":"2024-04-25T01:26:02.421975+0000","last_active":"2024-04-25T01:26:03.716525+0000","last_peered":"2024-04-25T01:26:03.716525+0000","last_clean":"2024-04-25T01:26:03.716525+0000","last_became_active":"2024-04-25T01:26:02.421698+0000","last_became_peered":"2024-04-25T01:26:02.421698+0000","last_unstale":"2024-04-25T01:26:03.716525+0000","last_undegraded":"2024-04-25T01:26:03.716525+0000","last_fullsized":"2024-04-25T01:26:03.716525+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416746+0000","last_change":"2024-04-25T01:25:53.370666+0000","last_active":"2024-04-25T01:26:03.416746+0000","last_peered":"2024-04-25T01:26:03.416746+0000","last_clean":"2024-04-25T01:26:03.416746+0000","last_became_active":"2024-04-25T01:25:53.369733+0000","last_became_peered":"2024-04-25T01:25:53.369733+0000","last_unstale":"2024-04-25T01:26:03.416746+0000","last_undegraded":"2024-04-25T01:26:03.416746+0000","last_fullsized":"2024-04-25T01:26:03.416746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.715962+0000","last_change":"2024-04-25T01:26:02.418114+0000","last_active":"2024-04-25T01:26:03.715962+0000","last_peered":"2024-04-25T01:26:03.715962+0000","last_clean":"2024-04-25T01:26:03.715962+0000","last_became_active":"2024-04-25T01:26:02.417810+0000","last_became_peered":"2024-04-25T01:26:02.417810+0000","last_unstale":"2024-04-25T01:26:03.715962+0000","last_undegraded":"2024-04-25T01:26:03.715962+0000","last_fullsized":"2024-04-25T01:26:03.715962+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:30.496089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.718049+0000","last_change":"2024-04-25T01:25:53.372048+0000","last_active":"2024-04-25T01:26:03.718049+0000","last_peered":"2024-04-25T01:26:03.718049+0000","last_clean":"2024-04-25T01:26:03.718049+0000","last_became_active":"2024-04-25T01:25:53.371767+0000","last_became_peered":"2024-04-25T01:25:53.371767+0000","last_unstale":"2024-04-25T01:26:03.718049+0000","last_undegraded":"2024-04-25T01:26:03.718049+0000","last_fullsized":"2024-04-25T01:26:03.718049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:05.589744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716672+0000","last_change":"2024-04-25T01:26:02.426289+0000","last_active":"2024-04-25T01:26:03.716672+0000","last_peered":"2024-04-25T01:26:03.716672+0000","last_clean":"2024-04-25T01:26:03.716672+0000","last_became_active":"2024-04-25T01:26:02.426015+0000","last_became_peered":"2024-04-25T01:26:02.426015+0000","last_unstale":"2024-04-25T01:26:03.716672+0000","last_undegraded":"2024-04-25T01:26:03.716672+0000","last_fullsized":"2024-04-25T01:26:03.716672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716678+0000","last_change":"2024-04-25T01:25:53.374186+0000","last_active":"2024-04-25T01:26:03.716678+0000","last_peered":"2024-04-25T01:26:03.716678+0000","last_clean":"2024-04-25T01:26:03.716678+0000","last_became_active":"2024-04-25T01:25:53.373766+0000","last_became_peered":"2024-04-25T01:25:53.373766+0000","last_unstale":"2024-04-25T01:26:03.716678+0000","last_undegraded":"2024-04-25T01:26:03.716678+0000","last_fullsized":"2024-04-25T01:26:03.716678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416320+0000","last_change":"2024-04-25T01:26:02.416345+0000","last_active":"2024-04-25T01:26:03.416320+0000","last_peered":"2024-04-25T01:26:03.416320+0000","last_clean":"2024-04-25T01:26:03.416320+0000","last_became_active":"2024-04-25T01:26:02.416082+0000","last_became_peered":"2024-04-25T01:26:02.416082+0000","last_unstale":"2024-04-25T01:26:03.416320+0000","last_undegraded":"2024-04-25T01:26:03.416320+0000","last_fullsized":"2024-04-25T01:26:03.416320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:47.810943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716847+0000","last_change":"2024-04-25T01:25:53.371602+0000","last_active":"2024-04-25T01:26:03.716847+0000","last_peered":"2024-04-25T01:26:03.716847+0000","last_clean":"2024-04-25T01:26:03.716847+0000","last_became_active":"2024-04-25T01:25:53.371362+0000","last_became_peered":"2024-04-25T01:25:53.371362+0000","last_unstale":"2024-04-25T01:26:03.716847+0000","last_undegraded":"2024-04-25T01:26:03.716847+0000","last_fullsized":"2024-04-25T01:26:03.716847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717204+0000","last_change":"2024-04-25T01:26:02.425888+0000","last_active":"2024-04-25T01:26:03.717204+0000","last_peered":"2024-04-25T01:26:03.717204+0000","last_clean":"2024-04-25T01:26:03.717204+0000","last_became_active":"2024-04-25T01:26:02.425577+0000","last_became_peered":"2024-04-25T01:26:02.425577+0000","last_unstale":"2024-04-25T01:26:03.717204+0000","last_undegraded":"2024-04-25T01:26:03.717204+0000","last_fullsized":"2024-04-25T01:26:03.717204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.715963+0000","last_change":"2024-04-25T01:25:53.370996+0000","last_active":"2024-04-25T01:26:03.715963+0000","last_peered":"2024-04-25T01:26:03.715963+0000","last_clean":"2024-04-25T01:26:03.715963+0000","last_became_active":"2024-04-25T01:25:53.370479+0000","last_became_peered":"2024-04-25T01:25:53.370479+0000","last_unstale":"2024-04-25T01:26:03.715963+0000","last_undegraded":"2024-04-25T01:26:03.715963+0000","last_fullsized":"2024-04-25T01:26:03.715963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:57.636067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507529+0000","last_change":"2024-04-25T01:26:02.423420+0000","last_active":"2024-04-25T01:26:03.507529+0000","last_peered":"2024-04-25T01:26:03.507529+0000","last_clean":"2024-04-25T01:26:03.507529+0000","last_became_active":"2024-04-25T01:26:02.423146+0000","last_became_peered":"2024-04-25T01:26:02.423146+0000","last_unstale":"2024-04-25T01:26:03.507529+0000","last_undegraded":"2024-04-25T01:26:03.507529+0000","last_fullsized":"2024-04-25T01:26:03.507529+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415469+0000","last_change":"2024-04-25T01:25:53.363512+0000","last_active":"2024-04-25T01:26:03.415469+0000","last_peered":"2024-04-25T01:26:03.415469+0000","last_clean":"2024-04-25T01:26:03.415469+0000","last_became_active":"2024-04-25T01:25:53.363246+0000","last_became_peered":"2024-04-25T01:25:53.363246+0000","last_unstale":"2024-04-25T01:26:03.415469+0000","last_undegraded":"2024-04-25T01:26:03.415469+0000","last_fullsized":"2024-04-25T01:26:03.415469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415970+0000","last_change":"2024-04-25T01:26:02.413346+0000","last_active":"2024-04-25T01:26:03.415970+0000","last_peered":"2024-04-25T01:26:03.415970+0000","last_clean":"2024-04-25T01:26:03.415970+0000","last_became_active":"2024-04-25T01:26:02.413036+0000","last_became_peered":"2024-04-25T01:26:02.413036+0000","last_unstale":"2024-04-25T01:26:03.415970+0000","last_undegraded":"2024-04-25T01:26:03.415970+0000","last_fullsized":"2024-04-25T01:26:03.415970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416642+0000","last_change":"2024-04-25T01:25:53.364457+0000","last_active":"2024-04-25T01:26:03.416642+0000","last_peered":"2024-04-25T01:26:03.416642+0000","last_clean":"2024-04-25T01:26:03.416642+0000","last_became_active":"2024-04-25T01:25:53.364103+0000","last_became_peered":"2024-04-25T01:25:53.364103+0000","last_unstale":"2024-04-25T01:26:03.416642+0000","last_undegraded":"2024-04-25T01:26:03.416642+0000","last_fullsized":"2024-04-25T01:26:03.416642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:11.336752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415365+0000","last_change":"2024-04-25T01:26:02.418792+0000","last_active":"2024-04-25T01:26:03.415365+0000","last_peered":"2024-04-25T01:26:03.415365+0000","last_clean":"2024-04-25T01:26:03.415365+0000","last_became_active":"2024-04-25T01:26:02.416942+0000","last_became_peered":"2024-04-25T01:26:02.416942+0000","last_unstale":"2024-04-25T01:26:03.415365+0000","last_undegraded":"2024-04-25T01:26:03.415365+0000","last_fullsized":"2024-04-25T01:26:03.415365+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:59.492678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717288+0000","last_change":"2024-04-25T01:25:53.371647+0000","last_active":"2024-04-25T01:26:03.717288+0000","last_peered":"2024-04-25T01:26:03.717288+0000","last_clean":"2024-04-25T01:26:03.717288+0000","last_became_active":"2024-04-25T01:25:53.371353+0000","last_became_peered":"2024-04-25T01:25:53.371353+0000","last_unstale":"2024-04-25T01:26:03.717288+0000","last_undegraded":"2024-04-25T01:26:03.717288+0000","last_fullsized":"2024-04-25T01:26:03.717288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507578+0000","last_change":"2024-04-25T01:26:02.426335+0000","last_active":"2024-04-25T01:26:03.507578+0000","last_peered":"2024-04-25T01:26:03.507578+0000","last_clean":"2024-04-25T01:26:03.507578+0000","last_became_active":"2024-04-25T01:26:02.426024+0000","last_became_peered":"2024-04-25T01:26:02.426024+0000","last_unstale":"2024-04-25T01:26:03.507578+0000","last_undegraded":"2024-04-25T01:26:03.507578+0000","last_fullsized":"2024-04-25T01:26:03.507578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717460+0000","last_change":"2024-04-25T01:25:53.372842+0000","last_active":"2024-04-25T01:26:03.717460+0000","last_peered":"2024-04-25T01:26:03.717460+0000","last_clean":"2024-04-25T01:26:03.717460+0000","last_became_active":"2024-04-25T01:25:53.372580+0000","last_became_peered":"2024-04-25T01:25:53.372580+0000","last_unstale":"2024-04-25T01:26:03.717460+0000","last_undegraded":"2024-04-25T01:26:03.717460+0000","last_fullsized":"2024-04-25T01:26:03.717460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716748+0000","last_change":"2024-04-25T01:26:02.414632+0000","last_active":"2024-04-25T01:26:03.716748+0000","last_peered":"2024-04-25T01:26:03.716748+0000","last_clean":"2024-04-25T01:26:03.716748+0000","last_became_active":"2024-04-25T01:26:02.414388+0000","last_became_peered":"2024-04-25T01:26:02.414388+0000","last_unstale":"2024-04-25T01:26:03.716748+0000","last_undegraded":"2024-04-25T01:26:03.716748+0000","last_fullsized":"2024-04-25T01:26:03.716748+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.508086+0000","last_change":"2024-04-25T01:25:53.376081+0000","last_active":"2024-04-25T01:26:03.508086+0000","last_peered":"2024-04-25T01:26:03.508086+0000","last_clean":"2024-04-25T01:26:03.508086+0000","last_became_active":"2024-04-25T01:25:53.375814+0000","last_became_peered":"2024-04-25T01:25:53.375814+0000","last_unstale":"2024-04-25T01:26:03.508086+0000","last_undegraded":"2024-04-25T01:26:03.508086+0000","last_fullsized":"2024-04-25T01:26:03.508086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:29.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416509+0000","last_change":"2024-04-25T01:26:02.414366+0000","last_active":"2024-04-25T01:26:03.416509+0000","last_peered":"2024-04-25T01:26:03.416509+0000","last_clean":"2024-04-25T01:26:03.416509+0000","last_became_active":"2024-04-25T01:26:02.414061+0000","last_became_peered":"2024-04-25T01:26:02.414061+0000","last_unstale":"2024-04-25T01:26:03.416509+0000","last_undegraded":"2024-04-25T01:26:03.416509+0000","last_fullsized":"2024-04-25T01:26:03.416509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:19.644468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415110+0000","last_change":"2024-04-25T01:25:53.374061+0000","last_active":"2024-04-25T01:26:03.415110+0000","last_peered":"2024-04-25T01:26:03.415110+0000","last_clean":"2024-04-25T01:26:03.415110+0000","last_became_active":"2024-04-25T01:25:53.373848+0000","last_became_peered":"2024-04-25T01:25:53.373848+0000","last_unstale":"2024-04-25T01:26:03.415110+0000","last_undegraded":"2024-04-25T01:26:03.415110+0000","last_fullsized":"2024-04-25T01:26:03.415110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717762+0000","last_change":"2024-04-25T01:26:02.414620+0000","last_active":"2024-04-25T01:26:03.717762+0000","last_peered":"2024-04-25T01:26:03.717762+0000","last_clean":"2024-04-25T01:26:03.717762+0000","last_became_active":"2024-04-25T01:26:02.414326+0000","last_became_peered":"2024-04-25T01:26:02.414326+0000","last_unstale":"2024-04-25T01:26:03.717762+0000","last_undegraded":"2024-04-25T01:26:03.717762+0000","last_fullsized":"2024-04-25T01:26:03.717762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:30.496089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507095+0000","last_change":"2024-04-25T01:25:53.364698+0000","last_active":"2024-04-25T01:26:03.507095+0000","last_peered":"2024-04-25T01:26:03.507095+0000","last_clean":"2024-04-25T01:26:03.507095+0000","last_became_active":"2024-04-25T01:25:53.364435+0000","last_became_peered":"2024-04-25T01:25:53.364435+0000","last_unstale":"2024-04-25T01:26:03.507095+0000","last_undegraded":"2024-04-25T01:26:03.507095+0000","last_fullsized":"2024-04-25T01:26:03.507095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716509+0000","last_change":"2024-04-25T01:26:02.418331+0000","last_active":"2024-04-25T01:26:03.716509+0000","last_peered":"2024-04-25T01:26:03.716509+0000","last_clean":"2024-04-25T01:26:03.716509+0000","last_became_active":"2024-04-25T01:26:02.418057+0000","last_became_peered":"2024-04-25T01:26:02.418057+0000","last_unstale":"2024-04-25T01:26:03.716509+0000","last_undegraded":"2024-04-25T01:26:03.716509+0000","last_fullsized":"2024-04-25T01:26:03.716509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:17.809738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717064+0000","last_change":"2024-04-25T01:25:53.375324+0000","last_active":"2024-04-25T01:26:03.717064+0000","last_peered":"2024-04-25T01:26:03.717064+0000","last_clean":"2024-04-25T01:26:03.717064+0000","last_became_active":"2024-04-25T01:25:53.374392+0000","last_became_peered":"2024-04-25T01:25:53.374392+0000","last_unstale":"2024-04-25T01:26:03.717064+0000","last_undegraded":"2024-04-25T01:26:03.717064+0000","last_fullsized":"2024-04-25T01:26:03.717064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717818+0000","last_change":"2024-04-25T01:26:02.422720+0000","last_active":"2024-04-25T01:26:03.717818+0000","last_peered":"2024-04-25T01:26:03.717818+0000","last_clean":"2024-04-25T01:26:03.717818+0000","last_became_active":"2024-04-25T01:26:02.421848+0000","last_became_peered":"2024-04-25T01:26:02.421848+0000","last_unstale":"2024-04-25T01:26:03.717818+0000","last_undegraded":"2024-04-25T01:26:03.717818+0000","last_fullsized":"2024-04-25T01:26:03.717818+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507255+0000","last_change":"2024-04-25T01:25:53.367953+0000","last_active":"2024-04-25T01:26:03.507255+0000","last_peered":"2024-04-25T01:26:03.507255+0000","last_clean":"2024-04-25T01:26:03.507255+0000","last_became_active":"2024-04-25T01:25:53.367358+0000","last_became_peered":"2024-04-25T01:25:53.367358+0000","last_unstale":"2024-04-25T01:26:03.507255+0000","last_undegraded":"2024-04-25T01:26:03.507255+0000","last_fullsized":"2024-04-25T01:26:03.507255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415875+0000","last_change":"2024-04-25T01:26:02.422300+0000","last_active":"2024-04-25T01:26:03.415875+0000","last_peered":"2024-04-25T01:26:03.415875+0000","last_clean":"2024-04-25T01:26:03.415875+0000","last_became_active":"2024-04-25T01:26:02.421702+0000","last_became_peered":"2024-04-25T01:26:02.421702+0000","last_unstale":"2024-04-25T01:26:03.415875+0000","last_undegraded":"2024-04-25T01:26:03.415875+0000","last_fullsized":"2024-04-25T01:26:03.415875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717355+0000","last_change":"2024-04-25T01:25:53.371513+0000","last_active":"2024-04-25T01:26:03.717355+0000","last_peered":"2024-04-25T01:26:03.717355+0000","last_clean":"2024-04-25T01:26:03.717355+0000","last_became_active":"2024-04-25T01:25:53.370421+0000","last_became_peered":"2024-04-25T01:25:53.370421+0000","last_unstale":"2024-04-25T01:26:03.717355+0000","last_undegraded":"2024-04-25T01:26:03.717355+0000","last_fullsized":"2024-04-25T01:26:03.717355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415236+0000","last_change":"2024-04-25T01:26:02.411784+0000","last_active":"2024-04-25T01:26:03.415236+0000","last_peered":"2024-04-25T01:26:03.415236+0000","last_clean":"2024-04-25T01:26:03.415236+0000","last_became_active":"2024-04-25T01:26:02.411478+0000","last_became_peered":"2024-04-25T01:26:02.411478+0000","last_unstale":"2024-04-25T01:26:03.415236+0000","last_undegraded":"2024-04-25T01:26:03.415236+0000","last_fullsized":"2024-04-25T01:26:03.415236+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:20.194816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716214+0000","last_change":"2024-04-25T01:25:53.363829+0000","last_active":"2024-04-25T01:26:03.716214+0000","last_peered":"2024-04-25T01:26:03.716214+0000","last_clean":"2024-04-25T01:26:03.716214+0000","last_became_active":"2024-04-25T01:25:53.363525+0000","last_became_peered":"2024-04-25T01:25:53.363525+0000","last_unstale":"2024-04-25T01:26:03.716214+0000","last_undegraded":"2024-04-25T01:26:03.716214+0000","last_fullsized":"2024-04-25T01:26:03.716214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.506843+0000","last_change":"2024-04-25T01:26:02.421208+0000","last_active":"2024-04-25T01:26:03.506843+0000","last_peered":"2024-04-25T01:26:03.506843+0000","last_clean":"2024-04-25T01:26:03.506843+0000","last_became_active":"2024-04-25T01:26:02.418796+0000","last_became_peered":"2024-04-25T01:26:02.418796+0000","last_unstale":"2024-04-25T01:26:03.506843+0000","last_undegraded":"2024-04-25T01:26:03.506843+0000","last_fullsized":"2024-04-25T01:26:03.506843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716784+0000","last_change":"2024-04-25T01:25:53.365057+0000","last_active":"2024-04-25T01:26:03.716784+0000","last_peered":"2024-04-25T01:26:03.716784+0000","last_clean":"2024-04-25T01:26:03.716784+0000","last_became_active":"2024-04-25T01:25:53.364793+0000","last_became_peered":"2024-04-25T01:25:53.364793+0000","last_unstale":"2024-04-25T01:26:03.716784+0000","last_undegraded":"2024-04-25T01:26:03.716784+0000","last_fullsized":"2024-04-25T01:26:03.716784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507060+0000","last_change":"2024-04-25T01:26:02.417606+0000","last_active":"2024-04-25T01:26:03.507060+0000","last_peered":"2024-04-25T01:26:03.507060+0000","last_clean":"2024-04-25T01:26:03.507060+0000","last_became_active":"2024-04-25T01:26:02.417350+0000","last_became_peered":"2024-04-25T01:26:02.417350+0000","last_unstale":"2024-04-25T01:26:03.507060+0000","last_undegraded":"2024-04-25T01:26:03.507060+0000","last_fullsized":"2024-04-25T01:26:03.507060+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.508077+0000","last_change":"2024-04-25T01:25:53.367762+0000","last_active":"2024-04-25T01:26:03.508077+0000","last_peered":"2024-04-25T01:26:03.508077+0000","last_clean":"2024-04-25T01:26:03.508077+0000","last_became_active":"2024-04-25T01:25:53.367383+0000","last_became_peered":"2024-04-25T01:25:53.367383+0000","last_unstale":"2024-04-25T01:26:03.508077+0000","last_undegraded":"2024-04-25T01:26:03.508077+0000","last_fullsized":"2024-04-25T01:26:03.508077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:05.589744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2851,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2851,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":690301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":101930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":99491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":103389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":99331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":101496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":690267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:26:26.738 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:26:26.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.889+0000 7f6a5182a640 1 Processor -- start 2024-04-25T01:26:26.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.889+0000 7f6a5182a640 1 -- start start 2024-04-25T01:26:26.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.890+0000 7f6a5182a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a4c0c5b70 0x7f6a4c06c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:26.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.890+0000 7f6a5182a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a4c0c9290 con 0x7f6a4c06bdc0 2024-04-25T01:26:26.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.890+0000 7f6a5182a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a4c1565b0 con 0x7f6a4c0cadf0 2024-04-25T01:26:26.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.890+0000 7f6a5182a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6a4c156890 con 0x7f6a4c0c5b70 2024-04-25T01:26:26.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.890+0000 7f6a4affd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6a4c06bdc0 0x7f6a4c0c9d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53508/0 (socket says 172.21.15.112:53508) 2024-04-25T01:26:26.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.890+0000 7f6a4affd640 1 -- 172.21.15.112:0/2916809120 learned_addr learned my addr 172.21.15.112:0/2916809120 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:26.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.890+0000 7f6a4a7fc640 1 -- 172.21.15.112:0/2916809120 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1550706147 0 0) 0x7f6a4c0c9290 con 0x7f6a4c06bdc0 2024-04-25T01:26:26.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.890+0000 7f6a4b7fe640 1 --2- 172.21.15.112:0/2916809120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a4c0c5b70 0x7f6a4c06c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:26.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.890+0000 7f6a4a7fc640 1 -- 172.21.15.112:0/2916809120 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a38003520 con 0x7f6a4c06bdc0 2024-04-25T01:26:26.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.890+0000 7f6a4a7fc640 1 -- 172.21.15.112:0/2916809120 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2933096157 0 0) 0x7f6a38003520 con 0x7f6a4c06bdc0 2024-04-25T01:26:26.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.890+0000 7f6a4a7fc640 1 -- 172.21.15.112:0/2916809120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a4c0c5b70 msgr2=0x7f6a4c06c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:26.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.890+0000 7f6a4a7fc640 1 --2- 172.21.15.112:0/2916809120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a4c0c5b70 0x7f6a4c06c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.890+0000 7f6a4a7fc640 1 -- 172.21.15.112:0/2916809120 >> v1:172.21.15.187:6789/0 conn(0x7f6a4c0cadf0 legacy=0x7f6a4c15e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:26.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.890+0000 7f6a4a7fc640 1 -- 172.21.15.112:0/2916809120 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4c0c9530 con 0x7f6a4c06bdc0 2024-04-25T01:26:26.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.890+0000 7f6a4a7fc640 1 -- 172.21.15.112:0/2916809120 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f6a3c001670 con 0x7f6a4c06bdc0 2024-04-25T01:26:26.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.890+0000 7f6a4a7fc640 1 -- 172.21.15.112:0/2916809120 >> v1:172.21.15.112:6789/0 conn(0x7f6a4c06bdc0 legacy=0x7f6a4c0c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:26.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.890+0000 7f6a4a7fc640 1 --2- 172.21.15.112:0/2916809120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a380039e0 0x7f6a38003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:26.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.891+0000 7f6a4a7fc640 1 --2- 172.21.15.112:0/2916809120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a38004800 0x7f6a38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:26.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.891+0000 7f6a4a7fc640 1 --2- 172.21.15.112:0/2916809120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a4c0c5b70 0x7f6a38005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:26.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.891+0000 7f6a4a7fc640 1 -- 172.21.15.112:0/2916809120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c156890 con 0x7f6a380039e0 2024-04-25T01:26:26.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.891+0000 7f6a4a7fc640 1 -- 172.21.15.112:0/2916809120 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3800bb80 con 0x7f6a38004800 2024-04-25T01:26:26.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.891+0000 7f6a4a7fc640 1 -- 172.21.15.112:0/2916809120 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6a3800bd80 con 0x7f6a4c0c5b70 2024-04-25T01:26:26.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.891+0000 7f6a4affd640 1 --2- 172.21.15.112:0/2916809120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a380039e0 0x7f6a38003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:26.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.891+0000 7f6a43fff640 1 --2- 172.21.15.112:0/2916809120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a38004800 0x7f6a38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:26.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.891+0000 7f6a4b7fe640 1 --2- 172.21.15.112:0/2916809120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a4c0c5b70 0x7f6a38005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:26.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.891+0000 7f6a4affd640 1 -- 172.21.15.112:0/2916809120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a4c0c5b70 msgr2=0x7f6a38005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:26.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.891+0000 7f6a4affd640 1 --2- 172.21.15.112:0/2916809120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a4c0c5b70 0x7f6a38005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.891+0000 7f6a4affd640 1 -- 172.21.15.112:0/2916809120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a38004800 msgr2=0x7f6a38004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:26.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.891+0000 7f6a4affd640 1 --2- 172.21.15.112:0/2916809120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a38004800 0x7f6a38004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.891+0000 7f6a4affd640 1 -- 172.21.15.112:0/2916809120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a38003520 con 0x7f6a380039e0 2024-04-25T01:26:26.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.892+0000 7f6a4affd640 1 --2- 172.21.15.112:0/2916809120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a380039e0 0x7f6a38003e00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00c240 tx=0x7f6a38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff7acaabaf42f58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:26.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.892+0000 7f6a4a7fc640 1 -- 172.21.15.112:0/2916809120 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c007cf0 con 0x7f6a380039e0 2024-04-25T01:26:26.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.892+0000 7f6a4a7fc640 1 -- 172.21.15.112:0/2916809120 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a3c0058d0 con 0x7f6a380039e0 2024-04-25T01:26:26.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.892+0000 7f6a4a7fc640 1 -- 172.21.15.112:0/2916809120 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c005ca0 con 0x7f6a380039e0 2024-04-25T01:26:26.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.892+0000 7f6a5182a640 1 -- 172.21.15.112:0/2916809120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a380039e0 msgr2=0x7f6a38003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:26.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.892+0000 7f6a5182a640 1 --2- 172.21.15.112:0/2916809120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a380039e0 0x7f6a38003e00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00c240 tx=0x7f6a38005830 comp rx=0 tx=0).stop 2024-04-25T01:26:26.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.892+0000 7f6a4b7fe640 1 -- 172.21.15.112:0/2916809120 reap_dead start 2024-04-25T01:26:26.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.892+0000 7f6a5182a640 1 -- 172.21.15.112:0/2916809120 shutdown_connections 2024-04-25T01:26:26.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.892+0000 7f6a5182a640 1 -- 172.21.15.112:0/2916809120 >> 172.21.15.112:0/2916809120 conn(0x7f6a4c0acd20 msgr2=0x7f6a4c0ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:26.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.892+0000 7f6a5182a640 1 -- 172.21.15.112:0/2916809120 shutdown_connections 2024-04-25T01:26:26.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.892+0000 7f6a5182a640 1 -- 172.21.15.112:0/2916809120 wait complete. 2024-04-25T01:26:26.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.893+0000 7f6a5182a640 1 Processor -- start 2024-04-25T01:26:26.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.893+0000 7f6a5182a640 1 -- start start 2024-04-25T01:26:26.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.894+0000 7f6a5182a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a380039e0 0x7f6a4c156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:26.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.894+0000 7f6a5182a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a4c0c5030 0x7f6a4c1514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:26.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.894+0000 7f6a5182a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a4c1525e0 0x7f6a4c151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:26.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.894+0000 7f6a5182a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c0b0b50 con 0x7f6a4c0c5030 2024-04-25T01:26:26.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.894+0000 7f6a5182a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c156280 con 0x7f6a4c1525e0 2024-04-25T01:26:26.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.894+0000 7f6a5182a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6a4c0b0cd0 con 0x7f6a380039e0 2024-04-25T01:26:26.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.894+0000 7f6a43fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a4c0c5030 0x7f6a4c1514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:26.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.894+0000 7f6a43fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a4c0c5030 0x7f6a4c1514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46308/0 (socket says 172.21.15.112:46308) 2024-04-25T01:26:26.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.894+0000 7f6a43fff640 1 -- 172.21.15.112:0/2559048297 learned_addr learned my addr 172.21.15.112:0/2559048297 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:26.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.894+0000 7f6a4affd640 1 --2- 172.21.15.112:0/2559048297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a380039e0 0x7f6a4c156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:26.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.894+0000 7f6a4b7fe640 1 --2- 172.21.15.112:0/2559048297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a4c1525e0 0x7f6a4c151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:26.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.894+0000 7f6a43fff640 1 -- 172.21.15.112:0/2559048297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a380039e0 msgr2=0x7f6a4c156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:26.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.894+0000 7f6a43fff640 1 --2- 172.21.15.112:0/2559048297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a380039e0 0x7f6a4c156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.894+0000 7f6a43fff640 1 -- 172.21.15.112:0/2559048297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a4c1525e0 msgr2=0x7f6a4c151a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:26.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.894+0000 7f6a43fff640 1 --2- 172.21.15.112:0/2559048297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a4c1525e0 0x7f6a4c151a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:26.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.894+0000 7f6a43fff640 1 -- 172.21.15.112:0/2559048297 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4c0c2000 con 0x7f6a4c0c5030 2024-04-25T01:26:26.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.894+0000 7f6a43fff640 1 --2- 172.21.15.112:0/2559048297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a4c0c5030 0x7f6a4c1514f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400e4d0 tx=0x7f6a3400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:26.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.895+0000 7f6a48ff9640 1 -- 172.21.15.112:0/2559048297 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a340052e0 con 0x7f6a4c0c5030 2024-04-25T01:26:26.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.895+0000 7f6a48ff9640 1 -- 172.21.15.112:0/2559048297 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a34018070 con 0x7f6a4c0c5030 2024-04-25T01:26:26.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.895+0000 7f6a5182a640 1 -- 172.21.15.112:0/2559048297 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a4c152300 con 0x7f6a4c0c5030 2024-04-25T01:26:26.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.895+0000 7f6a48ff9640 1 -- 172.21.15.112:0/2559048297 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34003810 con 0x7f6a4c0c5030 2024-04-25T01:26:26.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.895+0000 7f6a5182a640 1 -- 172.21.15.112:0/2559048297 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a4c0c4b20 con 0x7f6a4c0c5030 2024-04-25T01:26:26.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.896+0000 7f6a5182a640 1 -- 172.21.15.112:0/2559048297 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a4c0682a0 con 0x7f6a4c0c5030 2024-04-25T01:26:26.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.896+0000 7f6a48ff9640 1 -- 172.21.15.112:0/2559048297 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6a340039b0 con 0x7f6a4c0c5030 2024-04-25T01:26:26.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.896+0000 7f6a48ff9640 1 --2- 172.21.15.112:0/2559048297 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6a2c06be60 0x7f6a2c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:26.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.896+0000 7f6a48ff9640 1 -- 172.21.15.112:0/2559048297 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f6a3408ae80 con 0x7f6a4c0c5030 2024-04-25T01:26:26.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.897+0000 7f6a4affd640 1 --2- 172.21.15.112:0/2559048297 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6a2c06be60 0x7f6a2c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:26.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.897+0000 7f6a4affd640 1 --2- 172.21.15.112:0/2559048297 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6a2c06be60 0x7f6a2c06e320 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c033000 tx=0x7f6a3c009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:26.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:26.901+0000 7f6a48ff9640 1 -- 172.21.15.112:0/2559048297 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6a34059500 con 0x7f6a4c0c5030 2024-04-25T01:26:27.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.073+0000 7f6a5182a640 1 -- 172.21.15.112:0/2559048297 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6a4c1520b0 con 0x7f6a2c06be60 2024-04-25T01:26:27.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.084+0000 7f6a48ff9640 1 -- 172.21.15.112:0/2559048297 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+490299 (secure 0 0 0) 0x7f6a4c1520b0 con 0x7f6a2c06be60 2024-04-25T01:26:27.069 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:27.072 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T01:26:27.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.088+0000 7f6a5182a640 1 -- 172.21.15.112:0/2559048297 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6a2c06be60 msgr2=0x7f6a2c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:27.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.088+0000 7f6a5182a640 1 --2- 172.21.15.112:0/2559048297 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6a2c06be60 0x7f6a2c06e320 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c033000 tx=0x7f6a3c009030 comp rx=0 tx=0).stop 2024-04-25T01:26:27.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.088+0000 7f6a5182a640 1 -- 172.21.15.112:0/2559048297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a4c0c5030 msgr2=0x7f6a4c1514f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:27.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.088+0000 7f6a5182a640 1 --2- 172.21.15.112:0/2559048297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a4c0c5030 0x7f6a4c1514f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400e4d0 tx=0x7f6a3400e990 comp rx=0 tx=0).stop 2024-04-25T01:26:27.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.088+0000 7f6a5182a640 1 -- 172.21.15.112:0/2559048297 shutdown_connections 2024-04-25T01:26:27.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.088+0000 7f6a5182a640 1 --2- 172.21.15.112:0/2559048297 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6a2c06be60 0x7f6a2c06e320 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.088+0000 7f6a5182a640 1 --2- 172.21.15.112:0/2559048297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a4c1525e0 0x7f6a4c151a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.088+0000 7f6a5182a640 1 --2- 172.21.15.112:0/2559048297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a4c0c5030 0x7f6a4c1514f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.088+0000 7f6a5182a640 1 --2- 172.21.15.112:0/2559048297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a380039e0 0x7f6a4c156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.088+0000 7f6a5182a640 1 -- 172.21.15.112:0/2559048297 >> 172.21.15.112:0/2559048297 conn(0x7f6a4c0acd20 msgr2=0x7f6a4c1536c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:27.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.088+0000 7f6a5182a640 1 -- 172.21.15.112:0/2559048297 shutdown_connections 2024-04-25T01:26:27.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.088+0000 7f6a5182a640 1 -- 172.21.15.112:0/2559048297 wait complete. 2024-04-25T01:26:27.078 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-04-25T01:26:26.655428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596070,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":253,"num_write_kb":2448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219440,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755280,"statfs":{"total":773094113280,"available":772869406720,"internally_reserved":0,"allocated":3350528,"data_stored":1986228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":882,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002499"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415591+0000","last_change":"2024-04-25T01:26:02.412272+0000","last_active":"2024-04-25T01:26:03.415591+0000","last_peered":"2024-04-25T01:26:03.415591+0000","last_clean":"2024-04-25T01:26:03.415591+0000","last_became_active":"2024-04-25T01:26:02.411176+0000","last_became_peered":"2024-04-25T01:26:02.411176+0000","last_unstale":"2024-04-25T01:26:03.415591+0000","last_undegraded":"2024-04-25T01:26:03.415591+0000","last_fullsized":"2024-04-25T01:26:03.415591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:49.498275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716276+0000","last_change":"2024-04-25T01:25:52.353978+0000","last_active":"2024-04-25T01:26:03.716276+0000","last_peered":"2024-04-25T01:26:03.716276+0000","last_clean":"2024-04-25T01:26:03.716276+0000","last_became_active":"2024-04-25T01:25:52.353649+0000","last_became_peered":"2024-04-25T01:25:52.353649+0000","last_unstale":"2024-04-25T01:26:03.716276+0000","last_undegraded":"2024-04-25T01:26:03.716276+0000","last_fullsized":"2024-04-25T01:26:03.716276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416695+0000","last_change":"2024-04-25T01:25:53.364257+0000","last_active":"2024-04-25T01:26:03.416695+0000","last_peered":"2024-04-25T01:26:03.416695+0000","last_clean":"2024-04-25T01:26:03.416695+0000","last_became_active":"2024-04-25T01:25:53.363409+0000","last_became_peered":"2024-04-25T01:25:53.363409+0000","last_unstale":"2024-04-25T01:26:03.416695+0000","last_undegraded":"2024-04-25T01:26:03.416695+0000","last_fullsized":"2024-04-25T01:26:03.416695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415484+0000","last_change":"2024-04-25T01:26:01.407136+0000","last_active":"2024-04-25T01:26:03.415484+0000","last_peered":"2024-04-25T01:26:03.415484+0000","last_clean":"2024-04-25T01:26:03.415484+0000","last_became_active":"2024-04-25T01:26:01.406855+0000","last_became_peered":"2024-04-25T01:26:01.406855+0000","last_unstale":"2024-04-25T01:26:03.415484+0000","last_undegraded":"2024-04-25T01:26:03.415484+0000","last_fullsized":"2024-04-25T01:26:03.415484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:02.180826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716643+0000","last_change":"2024-04-25T01:26:02.425367+0000","last_active":"2024-04-25T01:26:03.716643+0000","last_peered":"2024-04-25T01:26:03.716643+0000","last_clean":"2024-04-25T01:26:03.716643+0000","last_became_active":"2024-04-25T01:26:02.425071+0000","last_became_peered":"2024-04-25T01:26:02.425071+0000","last_unstale":"2024-04-25T01:26:03.716643+0000","last_undegraded":"2024-04-25T01:26:03.716643+0000","last_fullsized":"2024-04-25T01:26:03.716643+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:47.810943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416737+0000","last_change":"2024-04-25T01:25:52.349185+0000","last_active":"2024-04-25T01:26:03.416737+0000","last_peered":"2024-04-25T01:26:03.416737+0000","last_clean":"2024-04-25T01:26:03.416737+0000","last_became_active":"2024-04-25T01:25:52.348931+0000","last_became_peered":"2024-04-25T01:25:52.348931+0000","last_unstale":"2024-04-25T01:26:03.416737+0000","last_undegraded":"2024-04-25T01:26:03.416737+0000","last_fullsized":"2024-04-25T01:26:03.416737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716082+0000","last_change":"2024-04-25T01:25:53.374150+0000","last_active":"2024-04-25T01:26:03.716082+0000","last_peered":"2024-04-25T01:26:03.716082+0000","last_clean":"2024-04-25T01:26:03.716082+0000","last_became_active":"2024-04-25T01:25:53.373755+0000","last_became_peered":"2024-04-25T01:25:53.373755+0000","last_unstale":"2024-04-25T01:26:03.716082+0000","last_undegraded":"2024-04-25T01:26:03.716082+0000","last_fullsized":"2024-04-25T01:26:03.716082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415473+0000","last_change":"2024-04-25T01:26:01.406334+0000","last_active":"2024-04-25T01:26:03.415473+0000","last_peered":"2024-04-25T01:26:03.415473+0000","last_clean":"2024-04-25T01:26:03.415473+0000","last_became_active":"2024-04-25T01:26:01.405994+0000","last_became_peered":"2024-04-25T01:26:01.405994+0000","last_unstale":"2024-04-25T01:26:03.415473+0000","last_undegraded":"2024-04-25T01:26:03.415473+0000","last_fullsized":"2024-04-25T01:26:03.415473+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:43.953437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507172+0000","last_change":"2024-04-25T01:26:02.427695+0000","last_active":"2024-04-25T01:26:03.507172+0000","last_peered":"2024-04-25T01:26:03.507172+0000","last_clean":"2024-04-25T01:26:03.507172+0000","last_became_active":"2024-04-25T01:26:02.427443+0000","last_became_peered":"2024-04-25T01:26:02.427443+0000","last_unstale":"2024-04-25T01:26:03.507172+0000","last_undegraded":"2024-04-25T01:26:03.507172+0000","last_fullsized":"2024-04-25T01:26:03.507172+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507616+0000","last_change":"2024-04-25T01:25:52.356403+0000","last_active":"2024-04-25T01:26:03.507616+0000","last_peered":"2024-04-25T01:26:03.507616+0000","last_clean":"2024-04-25T01:26:03.507616+0000","last_became_active":"2024-04-25T01:25:52.355822+0000","last_became_peered":"2024-04-25T01:25:52.355822+0000","last_unstale":"2024-04-25T01:26:03.507616+0000","last_undegraded":"2024-04-25T01:26:03.507616+0000","last_fullsized":"2024-04-25T01:26:03.507616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717449+0000","last_change":"2024-04-25T01:25:53.370061+0000","last_active":"2024-04-25T01:26:03.717449+0000","last_peered":"2024-04-25T01:26:03.717449+0000","last_clean":"2024-04-25T01:26:03.717449+0000","last_became_active":"2024-04-25T01:25:53.369721+0000","last_became_peered":"2024-04-25T01:25:53.369721+0000","last_unstale":"2024-04-25T01:26:03.717449+0000","last_undegraded":"2024-04-25T01:26:03.717449+0000","last_fullsized":"2024-04-25T01:26:03.717449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415934+0000","last_change":"2024-04-25T01:26:01.408179+0000","last_active":"2024-04-25T01:26:03.415934+0000","last_peered":"2024-04-25T01:26:03.415934+0000","last_clean":"2024-04-25T01:26:03.415934+0000","last_became_active":"2024-04-25T01:26:01.407909+0000","last_became_peered":"2024-04-25T01:26:01.407909+0000","last_unstale":"2024-04-25T01:26:03.415934+0000","last_undegraded":"2024-04-25T01:26:03.415934+0000","last_fullsized":"2024-04-25T01:26:03.415934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:48.865940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717234+0000","last_change":"2024-04-25T01:26:02.424615+0000","last_active":"2024-04-25T01:26:03.717234+0000","last_peered":"2024-04-25T01:26:03.717234+0000","last_clean":"2024-04-25T01:26:03.717234+0000","last_became_active":"2024-04-25T01:26:02.424325+0000","last_became_peered":"2024-04-25T01:26:02.424325+0000","last_unstale":"2024-04-25T01:26:03.717234+0000","last_undegraded":"2024-04-25T01:26:03.717234+0000","last_fullsized":"2024-04-25T01:26:03.717234+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507443+0000","last_change":"2024-04-25T01:25:52.359139+0000","last_active":"2024-04-25T01:26:03.507443+0000","last_peered":"2024-04-25T01:26:03.507443+0000","last_clean":"2024-04-25T01:26:03.507443+0000","last_became_active":"2024-04-25T01:25:52.358841+0000","last_became_peered":"2024-04-25T01:25:52.358841+0000","last_unstale":"2024-04-25T01:26:03.507443+0000","last_undegraded":"2024-04-25T01:26:03.507443+0000","last_fullsized":"2024-04-25T01:26:03.507443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717907+0000","last_change":"2024-04-25T01:25:53.368739+0000","last_active":"2024-04-25T01:26:03.717907+0000","last_peered":"2024-04-25T01:26:03.717907+0000","last_clean":"2024-04-25T01:26:03.717907+0000","last_became_active":"2024-04-25T01:25:53.368470+0000","last_became_peered":"2024-04-25T01:25:53.368470+0000","last_unstale":"2024-04-25T01:26:03.717907+0000","last_undegraded":"2024-04-25T01:26:03.717907+0000","last_fullsized":"2024-04-25T01:26:03.717907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:29.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415820+0000","last_change":"2024-04-25T01:26:01.407969+0000","last_active":"2024-04-25T01:26:03.415820+0000","last_peered":"2024-04-25T01:26:03.415820+0000","last_clean":"2024-04-25T01:26:03.415820+0000","last_became_active":"2024-04-25T01:26:01.407655+0000","last_became_peered":"2024-04-25T01:26:01.407655+0000","last_unstale":"2024-04-25T01:26:03.415820+0000","last_undegraded":"2024-04-25T01:26:03.415820+0000","last_fullsized":"2024-04-25T01:26:03.415820+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:58.485864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717005+0000","last_change":"2024-04-25T01:26:02.417776+0000","last_active":"2024-04-25T01:26:03.717005+0000","last_peered":"2024-04-25T01:26:03.717005+0000","last_clean":"2024-04-25T01:26:03.717005+0000","last_became_active":"2024-04-25T01:26:02.417524+0000","last_became_peered":"2024-04-25T01:26:02.417524+0000","last_unstale":"2024-04-25T01:26:03.717005+0000","last_undegraded":"2024-04-25T01:26:03.717005+0000","last_fullsized":"2024-04-25T01:26:03.717005+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.688469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716814+0000","last_change":"2024-04-25T01:25:53.372385+0000","last_active":"2024-04-25T01:26:03.716814+0000","last_peered":"2024-04-25T01:26:03.716814+0000","last_clean":"2024-04-25T01:26:03.716814+0000","last_became_active":"2024-04-25T01:25:53.372064+0000","last_became_peered":"2024-04-25T01:25:53.372064+0000","last_unstale":"2024-04-25T01:26:03.716814+0000","last_undegraded":"2024-04-25T01:26:03.716814+0000","last_fullsized":"2024-04-25T01:26:03.716814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507411+0000","last_change":"2024-04-25T01:25:52.358616+0000","last_active":"2024-04-25T01:26:03.507411+0000","last_peered":"2024-04-25T01:26:03.507411+0000","last_clean":"2024-04-25T01:26:03.507411+0000","last_became_active":"2024-04-25T01:25:52.358318+0000","last_became_peered":"2024-04-25T01:25:52.358318+0000","last_unstale":"2024-04-25T01:26:03.507411+0000","last_undegraded":"2024-04-25T01:26:03.507411+0000","last_fullsized":"2024-04-25T01:26:03.507411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416542+0000","last_change":"2024-04-25T01:26:01.400599+0000","last_active":"2024-04-25T01:26:03.416542+0000","last_peered":"2024-04-25T01:26:03.416542+0000","last_clean":"2024-04-25T01:26:03.416542+0000","last_became_active":"2024-04-25T01:26:01.400266+0000","last_became_peered":"2024-04-25T01:26:01.400266+0000","last_unstale":"2024-04-25T01:26:03.416542+0000","last_undegraded":"2024-04-25T01:26:03.416542+0000","last_fullsized":"2024-04-25T01:26:03.416542+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416953+0000","last_change":"2024-04-25T01:26:02.420269+0000","last_active":"2024-04-25T01:26:03.416953+0000","last_peered":"2024-04-25T01:26:03.416953+0000","last_clean":"2024-04-25T01:26:03.416953+0000","last_became_active":"2024-04-25T01:26:02.420001+0000","last_became_peered":"2024-04-25T01:26:02.420001+0000","last_unstale":"2024-04-25T01:26:03.416953+0000","last_undegraded":"2024-04-25T01:26:03.416953+0000","last_fullsized":"2024-04-25T01:26:03.416953+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:38.149069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415549+0000","last_change":"2024-04-25T01:25:53.361074+0000","last_active":"2024-04-25T01:26:03.415549+0000","last_peered":"2024-04-25T01:26:03.415549+0000","last_clean":"2024-04-25T01:26:03.415549+0000","last_became_active":"2024-04-25T01:25:53.360802+0000","last_became_peered":"2024-04-25T01:25:53.360802+0000","last_unstale":"2024-04-25T01:26:03.415549+0000","last_undegraded":"2024-04-25T01:26:03.415549+0000","last_fullsized":"2024-04-25T01:26:03.415549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717195+0000","last_change":"2024-04-25T01:25:52.348841+0000","last_active":"2024-04-25T01:26:03.717195+0000","last_peered":"2024-04-25T01:26:03.717195+0000","last_clean":"2024-04-25T01:26:03.717195+0000","last_became_active":"2024-04-25T01:25:52.348550+0000","last_became_peered":"2024-04-25T01:25:52.348550+0000","last_unstale":"2024-04-25T01:26:03.717195+0000","last_undegraded":"2024-04-25T01:26:03.717195+0000","last_fullsized":"2024-04-25T01:26:03.717195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416126+0000","last_change":"2024-04-25T01:26:01.408529+0000","last_active":"2024-04-25T01:26:03.416126+0000","last_peered":"2024-04-25T01:26:03.416126+0000","last_clean":"2024-04-25T01:26:03.416126+0000","last_became_active":"2024-04-25T01:26:01.407738+0000","last_became_peered":"2024-04-25T01:26:01.407738+0000","last_unstale":"2024-04-25T01:26:03.416126+0000","last_undegraded":"2024-04-25T01:26:03.416126+0000","last_fullsized":"2024-04-25T01:26:03.416126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:37.142255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416507+0000","last_change":"2024-04-25T01:26:02.417731+0000","last_active":"2024-04-25T01:26:03.416507+0000","last_peered":"2024-04-25T01:26:03.416507+0000","last_clean":"2024-04-25T01:26:03.416507+0000","last_became_active":"2024-04-25T01:26:02.417438+0000","last_became_peered":"2024-04-25T01:26:02.417438+0000","last_unstale":"2024-04-25T01:26:03.416507+0000","last_undegraded":"2024-04-25T01:26:03.416507+0000","last_fullsized":"2024-04-25T01:26:03.416507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415373+0000","last_change":"2024-04-25T01:25:53.361790+0000","last_active":"2024-04-25T01:26:03.415373+0000","last_peered":"2024-04-25T01:26:03.415373+0000","last_clean":"2024-04-25T01:26:03.415373+0000","last_became_active":"2024-04-25T01:25:53.361542+0000","last_became_peered":"2024-04-25T01:25:53.361542+0000","last_unstale":"2024-04-25T01:26:03.415373+0000","last_undegraded":"2024-04-25T01:26:03.415373+0000","last_fullsized":"2024-04-25T01:26:03.415373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716946+0000","last_change":"2024-04-25T01:25:52.353926+0000","last_active":"2024-04-25T01:26:03.716946+0000","last_peered":"2024-04-25T01:26:03.716946+0000","last_clean":"2024-04-25T01:26:03.716946+0000","last_became_active":"2024-04-25T01:25:52.353656+0000","last_became_peered":"2024-04-25T01:25:52.353656+0000","last_unstale":"2024-04-25T01:26:03.716946+0000","last_undegraded":"2024-04-25T01:26:03.716946+0000","last_fullsized":"2024-04-25T01:26:03.716946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716923+0000","last_change":"2024-04-25T01:26:01.418556+0000","last_active":"2024-04-25T01:26:03.716923+0000","last_peered":"2024-04-25T01:26:03.716923+0000","last_clean":"2024-04-25T01:26:03.716923+0000","last_became_active":"2024-04-25T01:26:01.418263+0000","last_became_peered":"2024-04-25T01:26:01.418263+0000","last_unstale":"2024-04-25T01:26:03.716923+0000","last_undegraded":"2024-04-25T01:26:03.716923+0000","last_fullsized":"2024-04-25T01:26:03.716923+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:28.139183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717839+0000","last_change":"2024-04-25T01:26:02.413429+0000","last_active":"2024-04-25T01:26:03.717839+0000","last_peered":"2024-04-25T01:26:03.717839+0000","last_clean":"2024-04-25T01:26:03.717839+0000","last_became_active":"2024-04-25T01:26:02.412632+0000","last_became_peered":"2024-04-25T01:26:02.412632+0000","last_unstale":"2024-04-25T01:26:03.717839+0000","last_undegraded":"2024-04-25T01:26:03.717839+0000","last_fullsized":"2024-04-25T01:26:03.717839+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:10.049310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416067+0000","last_change":"2024-04-25T01:25:53.361653+0000","last_active":"2024-04-25T01:26:03.416067+0000","last_peered":"2024-04-25T01:26:03.416067+0000","last_clean":"2024-04-25T01:26:03.416067+0000","last_became_active":"2024-04-25T01:25:53.361265+0000","last_became_peered":"2024-04-25T01:25:53.361265+0000","last_unstale":"2024-04-25T01:26:03.416067+0000","last_undegraded":"2024-04-25T01:26:03.416067+0000","last_fullsized":"2024-04-25T01:26:03.416067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507177+0000","last_change":"2024-04-25T01:25:52.353836+0000","last_active":"2024-04-25T01:26:03.507177+0000","last_peered":"2024-04-25T01:26:03.507177+0000","last_clean":"2024-04-25T01:26:03.507177+0000","last_became_active":"2024-04-25T01:25:52.353552+0000","last_became_peered":"2024-04-25T01:25:52.353552+0000","last_unstale":"2024-04-25T01:26:03.507177+0000","last_undegraded":"2024-04-25T01:26:03.507177+0000","last_fullsized":"2024-04-25T01:26:03.507177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416325+0000","last_change":"2024-04-25T01:26:01.411323+0000","last_active":"2024-04-25T01:26:03.416325+0000","last_peered":"2024-04-25T01:26:03.416325+0000","last_clean":"2024-04-25T01:26:03.416325+0000","last_became_active":"2024-04-25T01:26:01.410378+0000","last_became_peered":"2024-04-25T01:26:01.410378+0000","last_unstale":"2024-04-25T01:26:03.416325+0000","last_undegraded":"2024-04-25T01:26:03.416325+0000","last_fullsized":"2024-04-25T01:26:03.416325+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415000+0000","last_change":"2024-04-25T01:26:02.416646+0000","last_active":"2024-04-25T01:26:03.415000+0000","last_peered":"2024-04-25T01:26:03.415000+0000","last_clean":"2024-04-25T01:26:03.415000+0000","last_became_active":"2024-04-25T01:26:02.416395+0000","last_became_peered":"2024-04-25T01:26:02.416395+0000","last_unstale":"2024-04-25T01:26:03.415000+0000","last_undegraded":"2024-04-25T01:26:03.415000+0000","last_fullsized":"2024-04-25T01:26:03.415000+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716795+0000","last_change":"2024-04-25T01:25:53.370111+0000","last_active":"2024-04-25T01:26:03.716795+0000","last_peered":"2024-04-25T01:26:03.716795+0000","last_clean":"2024-04-25T01:26:03.716795+0000","last_became_active":"2024-04-25T01:25:53.369837+0000","last_became_peered":"2024-04-25T01:25:53.369837+0000","last_unstale":"2024-04-25T01:26:03.716795+0000","last_undegraded":"2024-04-25T01:26:03.716795+0000","last_fullsized":"2024-04-25T01:26:03.716795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717961+0000","last_change":"2024-04-25T01:25:52.353269+0000","last_active":"2024-04-25T01:26:03.717961+0000","last_peered":"2024-04-25T01:26:03.717961+0000","last_clean":"2024-04-25T01:26:03.717961+0000","last_became_active":"2024-04-25T01:25:52.352991+0000","last_became_peered":"2024-04-25T01:25:52.352991+0000","last_unstale":"2024-04-25T01:26:03.717961+0000","last_undegraded":"2024-04-25T01:26:03.717961+0000","last_fullsized":"2024-04-25T01:26:03.717961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415651+0000","last_change":"2024-04-25T01:26:01.410360+0000","last_active":"2024-04-25T01:26:03.415651+0000","last_peered":"2024-04-25T01:26:03.415651+0000","last_clean":"2024-04-25T01:26:03.415651+0000","last_became_active":"2024-04-25T01:26:01.410076+0000","last_became_peered":"2024-04-25T01:26:01.410076+0000","last_unstale":"2024-04-25T01:26:03.415651+0000","last_undegraded":"2024-04-25T01:26:03.415651+0000","last_fullsized":"2024-04-25T01:26:03.415651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:52.561251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415577+0000","last_change":"2024-04-25T01:26:02.422182+0000","last_active":"2024-04-25T01:26:03.415577+0000","last_peered":"2024-04-25T01:26:03.415577+0000","last_clean":"2024-04-25T01:26:03.415577+0000","last_became_active":"2024-04-25T01:26:02.421885+0000","last_became_peered":"2024-04-25T01:26:02.421885+0000","last_unstale":"2024-04-25T01:26:03.415577+0000","last_undegraded":"2024-04-25T01:26:03.415577+0000","last_fullsized":"2024-04-25T01:26:03.415577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:03.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416628+0000","last_change":"2024-04-25T01:25:53.364967+0000","last_active":"2024-04-25T01:26:03.416628+0000","last_peered":"2024-04-25T01:26:03.416628+0000","last_clean":"2024-04-25T01:26:03.416628+0000","last_became_active":"2024-04-25T01:25:53.364694+0000","last_became_peered":"2024-04-25T01:25:53.364694+0000","last_unstale":"2024-04-25T01:26:03.416628+0000","last_undegraded":"2024-04-25T01:26:03.416628+0000","last_fullsized":"2024-04-25T01:26:03.416628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717049+0000","last_change":"2024-04-25T01:25:52.355395+0000","last_active":"2024-04-25T01:26:03.717049+0000","last_peered":"2024-04-25T01:26:03.717049+0000","last_clean":"2024-04-25T01:26:03.717049+0000","last_became_active":"2024-04-25T01:25:52.355137+0000","last_became_peered":"2024-04-25T01:25:52.355137+0000","last_unstale":"2024-04-25T01:26:03.717049+0000","last_undegraded":"2024-04-25T01:26:03.717049+0000","last_fullsized":"2024-04-25T01:26:03.717049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:26.467968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.414943+0000","last_change":"2024-04-25T01:26:01.416108+0000","last_active":"2024-04-25T01:26:03.414943+0000","last_peered":"2024-04-25T01:26:03.414943+0000","last_clean":"2024-04-25T01:26:03.414943+0000","last_became_active":"2024-04-25T01:26:01.415824+0000","last_became_peered":"2024-04-25T01:26:01.415824+0000","last_unstale":"2024-04-25T01:26:03.414943+0000","last_undegraded":"2024-04-25T01:26:03.414943+0000","last_fullsized":"2024-04-25T01:26:03.414943+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:28.139183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716568+0000","last_change":"2024-04-25T01:26:02.416208+0000","last_active":"2024-04-25T01:26:03.716568+0000","last_peered":"2024-04-25T01:26:03.716568+0000","last_clean":"2024-04-25T01:26:03.716568+0000","last_became_active":"2024-04-25T01:26:02.415949+0000","last_became_peered":"2024-04-25T01:26:02.415949+0000","last_unstale":"2024-04-25T01:26:03.716568+0000","last_undegraded":"2024-04-25T01:26:03.716568+0000","last_fullsized":"2024-04-25T01:26:03.716568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:21.097225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717249+0000","last_change":"2024-04-25T01:25:53.366944+0000","last_active":"2024-04-25T01:26:03.717249+0000","last_peered":"2024-04-25T01:26:03.717249+0000","last_clean":"2024-04-25T01:26:03.717249+0000","last_became_active":"2024-04-25T01:25:53.366693+0000","last_became_peered":"2024-04-25T01:25:53.366693+0000","last_unstale":"2024-04-25T01:26:03.717249+0000","last_undegraded":"2024-04-25T01:26:03.717249+0000","last_fullsized":"2024-04-25T01:26:03.717249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507732+0000","last_change":"2024-04-25T01:25:52.356842+0000","last_active":"2024-04-25T01:26:03.507732+0000","last_peered":"2024-04-25T01:26:03.507732+0000","last_clean":"2024-04-25T01:26:03.507732+0000","last_became_active":"2024-04-25T01:25:52.356396+0000","last_became_peered":"2024-04-25T01:25:52.356396+0000","last_unstale":"2024-04-25T01:26:03.507732+0000","last_undegraded":"2024-04-25T01:26:03.507732+0000","last_fullsized":"2024-04-25T01:26:03.507732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716730+0000","last_change":"2024-04-25T01:26:01.407934+0000","last_active":"2024-04-25T01:26:03.716730+0000","last_peered":"2024-04-25T01:26:03.716730+0000","last_clean":"2024-04-25T01:26:03.716730+0000","last_became_active":"2024-04-25T01:26:01.407659+0000","last_became_peered":"2024-04-25T01:26:01.407659+0000","last_unstale":"2024-04-25T01:26:03.716730+0000","last_undegraded":"2024-04-25T01:26:03.716730+0000","last_fullsized":"2024-04-25T01:26:03.716730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:19.629335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716666+0000","last_change":"2024-04-25T01:26:02.418087+0000","last_active":"2024-04-25T01:26:03.716666+0000","last_peered":"2024-04-25T01:26:03.716666+0000","last_clean":"2024-04-25T01:26:03.716666+0000","last_became_active":"2024-04-25T01:26:02.417622+0000","last_became_peered":"2024-04-25T01:26:02.417622+0000","last_unstale":"2024-04-25T01:26:03.716666+0000","last_undegraded":"2024-04-25T01:26:03.716666+0000","last_fullsized":"2024-04-25T01:26:03.716666+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:03.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717962+0000","last_change":"2024-04-25T01:25:53.368164+0000","last_active":"2024-04-25T01:26:03.717962+0000","last_peered":"2024-04-25T01:26:03.717962+0000","last_clean":"2024-04-25T01:26:03.717962+0000","last_became_active":"2024-04-25T01:25:53.367854+0000","last_became_peered":"2024-04-25T01:25:53.367854+0000","last_unstale":"2024-04-25T01:26:03.717962+0000","last_undegraded":"2024-04-25T01:26:03.717962+0000","last_fullsized":"2024-04-25T01:26:03.717962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717294+0000","last_change":"2024-04-25T01:25:52.350128+0000","last_active":"2024-04-25T01:26:03.717294+0000","last_peered":"2024-04-25T01:26:03.717294+0000","last_clean":"2024-04-25T01:26:03.717294+0000","last_became_active":"2024-04-25T01:25:52.349874+0000","last_became_peered":"2024-04-25T01:25:52.349874+0000","last_unstale":"2024-04-25T01:26:03.717294+0000","last_undegraded":"2024-04-25T01:26:03.717294+0000","last_fullsized":"2024-04-25T01:26:03.717294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716937+0000","last_change":"2024-04-25T01:26:01.418196+0000","last_active":"2024-04-25T01:26:03.716937+0000","last_peered":"2024-04-25T01:26:03.716937+0000","last_clean":"2024-04-25T01:26:03.716937+0000","last_became_active":"2024-04-25T01:26:01.417820+0000","last_became_peered":"2024-04-25T01:26:01.417820+0000","last_unstale":"2024-04-25T01:26:03.716937+0000","last_undegraded":"2024-04-25T01:26:03.716937+0000","last_fullsized":"2024-04-25T01:26:03.716937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716324+0000","last_change":"2024-04-25T01:26:02.422761+0000","last_active":"2024-04-25T01:26:03.716324+0000","last_peered":"2024-04-25T01:26:03.716324+0000","last_clean":"2024-04-25T01:26:03.716324+0000","last_became_active":"2024-04-25T01:26:02.422503+0000","last_became_peered":"2024-04-25T01:26:02.422503+0000","last_unstale":"2024-04-25T01:26:03.716324+0000","last_undegraded":"2024-04-25T01:26:03.716324+0000","last_fullsized":"2024-04-25T01:26:03.716324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:55.040946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416617+0000","last_change":"2024-04-25T01:25:53.361696+0000","last_active":"2024-04-25T01:26:03.416617+0000","last_peered":"2024-04-25T01:26:03.416617+0000","last_clean":"2024-04-25T01:26:03.416617+0000","last_became_active":"2024-04-25T01:25:53.361156+0000","last_became_peered":"2024-04-25T01:25:53.361156+0000","last_unstale":"2024-04-25T01:26:03.416617+0000","last_undegraded":"2024-04-25T01:26:03.416617+0000","last_fullsized":"2024-04-25T01:26:03.416617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:27.472315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717721+0000","last_change":"2024-04-25T01:25:52.354499+0000","last_active":"2024-04-25T01:26:03.717721+0000","last_peered":"2024-04-25T01:26:03.717721+0000","last_clean":"2024-04-25T01:26:03.717721+0000","last_became_active":"2024-04-25T01:25:52.354193+0000","last_became_peered":"2024-04-25T01:25:52.354193+0000","last_unstale":"2024-04-25T01:26:03.717721+0000","last_undegraded":"2024-04-25T01:26:03.717721+0000","last_fullsized":"2024-04-25T01:26:03.717721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717190+0000","last_change":"2024-04-25T01:26:01.418186+0000","last_active":"2024-04-25T01:26:03.717190+0000","last_peered":"2024-04-25T01:26:03.717190+0000","last_clean":"2024-04-25T01:26:03.717190+0000","last_became_active":"2024-04-25T01:26:01.417805+0000","last_became_peered":"2024-04-25T01:26:01.417805+0000","last_unstale":"2024-04-25T01:26:03.717190+0000","last_undegraded":"2024-04-25T01:26:03.717190+0000","last_fullsized":"2024-04-25T01:26:03.717190+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:19.629335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416697+0000","last_change":"2024-04-25T01:26:02.415877+0000","last_active":"2024-04-25T01:26:03.416697+0000","last_peered":"2024-04-25T01:26:03.416697+0000","last_clean":"2024-04-25T01:26:03.416697+0000","last_became_active":"2024-04-25T01:26:02.415619+0000","last_became_peered":"2024-04-25T01:26:02.415619+0000","last_unstale":"2024-04-25T01:26:03.416697+0000","last_undegraded":"2024-04-25T01:26:03.416697+0000","last_fullsized":"2024-04-25T01:26:03.416697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:14.647083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716434+0000","last_change":"2024-04-25T01:25:53.362922+0000","last_active":"2024-04-25T01:26:03.716434+0000","last_peered":"2024-04-25T01:26:03.716434+0000","last_clean":"2024-04-25T01:26:03.716434+0000","last_became_active":"2024-04-25T01:25:53.362533+0000","last_became_peered":"2024-04-25T01:25:53.362533+0000","last_unstale":"2024-04-25T01:26:03.716434+0000","last_undegraded":"2024-04-25T01:26:03.716434+0000","last_fullsized":"2024-04-25T01:26:03.716434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415851+0000","last_change":"2024-04-25T01:25:52.345321+0000","last_active":"2024-04-25T01:26:03.415851+0000","last_peered":"2024-04-25T01:26:03.415851+0000","last_clean":"2024-04-25T01:26:03.415851+0000","last_became_active":"2024-04-25T01:25:52.345061+0000","last_became_peered":"2024-04-25T01:25:52.345061+0000","last_unstale":"2024-04-25T01:26:03.415851+0000","last_undegraded":"2024-04-25T01:26:03.415851+0000","last_fullsized":"2024-04-25T01:26:03.415851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717072+0000","last_change":"2024-04-25T01:26:01.418516+0000","last_active":"2024-04-25T01:26:03.717072+0000","last_peered":"2024-04-25T01:26:03.717072+0000","last_clean":"2024-04-25T01:26:03.717072+0000","last_became_active":"2024-04-25T01:26:01.418173+0000","last_became_peered":"2024-04-25T01:26:01.418173+0000","last_unstale":"2024-04-25T01:26:03.717072+0000","last_undegraded":"2024-04-25T01:26:03.717072+0000","last_fullsized":"2024-04-25T01:26:03.717072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:02.568472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.718145+0000","last_change":"2024-04-25T01:26:02.418115+0000","last_active":"2024-04-25T01:26:03.718145+0000","last_peered":"2024-04-25T01:26:03.718145+0000","last_clean":"2024-04-25T01:26:03.718145+0000","last_became_active":"2024-04-25T01:26:02.417355+0000","last_became_peered":"2024-04-25T01:26:02.417355+0000","last_unstale":"2024-04-25T01:26:03.718145+0000","last_undegraded":"2024-04-25T01:26:03.718145+0000","last_fullsized":"2024-04-25T01:26:03.718145+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:19.644468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716066+0000","last_change":"2024-04-25T01:25:53.355950+0000","last_active":"2024-04-25T01:26:03.716066+0000","last_peered":"2024-04-25T01:26:03.716066+0000","last_clean":"2024-04-25T01:26:03.716066+0000","last_became_active":"2024-04-25T01:25:53.355648+0000","last_became_peered":"2024-04-25T01:25:53.355648+0000","last_unstale":"2024-04-25T01:26:03.716066+0000","last_undegraded":"2024-04-25T01:26:03.716066+0000","last_fullsized":"2024-04-25T01:26:03.716066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507424+0000","last_change":"2024-04-25T01:25:52.357683+0000","last_active":"2024-04-25T01:26:03.507424+0000","last_peered":"2024-04-25T01:26:03.507424+0000","last_clean":"2024-04-25T01:26:03.507424+0000","last_became_active":"2024-04-25T01:25:52.356924+0000","last_became_peered":"2024-04-25T01:25:52.356924+0000","last_unstale":"2024-04-25T01:26:03.507424+0000","last_undegraded":"2024-04-25T01:26:03.507424+0000","last_fullsized":"2024-04-25T01:26:03.507424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716612+0000","last_change":"2024-04-25T01:26:01.421260+0000","last_active":"2024-04-25T01:26:03.716612+0000","last_peered":"2024-04-25T01:26:03.716612+0000","last_clean":"2024-04-25T01:26:03.716612+0000","last_became_active":"2024-04-25T01:26:01.420831+0000","last_became_peered":"2024-04-25T01:26:01.420831+0000","last_unstale":"2024-04-25T01:26:03.716612+0000","last_undegraded":"2024-04-25T01:26:03.716612+0000","last_fullsized":"2024-04-25T01:26:03.716612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507519+0000","last_change":"2024-04-25T01:26:02.421386+0000","last_active":"2024-04-25T01:26:03.507519+0000","last_peered":"2024-04-25T01:26:03.507519+0000","last_clean":"2024-04-25T01:26:03.507519+0000","last_became_active":"2024-04-25T01:26:02.421079+0000","last_became_peered":"2024-04-25T01:26:02.421079+0000","last_unstale":"2024-04-25T01:26:03.507519+0000","last_undegraded":"2024-04-25T01:26:03.507519+0000","last_fullsized":"2024-04-25T01:26:03.507519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416038+0000","last_change":"2024-04-25T01:25:53.374719+0000","last_active":"2024-04-25T01:26:03.416038+0000","last_peered":"2024-04-25T01:26:03.416038+0000","last_clean":"2024-04-25T01:26:03.416038+0000","last_became_active":"2024-04-25T01:25:53.374425+0000","last_became_peered":"2024-04-25T01:25:53.374425+0000","last_unstale":"2024-04-25T01:26:03.416038+0000","last_undegraded":"2024-04-25T01:26:03.416038+0000","last_fullsized":"2024-04-25T01:26:03.416038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717309+0000","last_change":"2024-04-25T01:25:52.353155+0000","last_active":"2024-04-25T01:26:03.717309+0000","last_peered":"2024-04-25T01:26:03.717309+0000","last_clean":"2024-04-25T01:26:03.717309+0000","last_became_active":"2024-04-25T01:25:52.352826+0000","last_became_peered":"2024-04-25T01:25:52.352826+0000","last_unstale":"2024-04-25T01:26:03.717309+0000","last_undegraded":"2024-04-25T01:26:03.717309+0000","last_fullsized":"2024-04-25T01:26:03.717309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507554+0000","last_change":"2024-04-25T01:25:41.719670+0000","last_active":"2024-04-25T01:26:03.507554+0000","last_peered":"2024-04-25T01:26:03.507554+0000","last_clean":"2024-04-25T01:26:03.507554+0000","last_became_active":"2024-04-25T01:25:41.718721+0000","last_became_peered":"2024-04-25T01:25:41.718721+0000","last_unstale":"2024-04-25T01:26:03.507554+0000","last_undegraded":"2024-04-25T01:26:03.507554+0000","last_fullsized":"2024-04-25T01:26:03.507554+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_clean_scrub_stamp":"2024-04-25T01:25:40.688097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:27.456746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717378+0000","last_change":"2024-04-25T01:26:01.409339+0000","last_active":"2024-04-25T01:26:03.717378+0000","last_peered":"2024-04-25T01:26:03.717378+0000","last_clean":"2024-04-25T01:26:03.717378+0000","last_became_active":"2024-04-25T01:26:01.409062+0000","last_became_peered":"2024-04-25T01:26:01.409062+0000","last_unstale":"2024-04-25T01:26:03.717378+0000","last_undegraded":"2024-04-25T01:26:03.717378+0000","last_fullsized":"2024-04-25T01:26:03.717378+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:52.561251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415532+0000","last_change":"2024-04-25T01:26:02.418574+0000","last_active":"2024-04-25T01:26:03.415532+0000","last_peered":"2024-04-25T01:26:03.415532+0000","last_clean":"2024-04-25T01:26:03.415532+0000","last_became_active":"2024-04-25T01:26:02.416546+0000","last_became_peered":"2024-04-25T01:26:02.416546+0000","last_unstale":"2024-04-25T01:26:03.415532+0000","last_undegraded":"2024-04-25T01:26:03.415532+0000","last_fullsized":"2024-04-25T01:26:03.415532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:43.994141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507591+0000","last_change":"2024-04-25T01:25:53.375523+0000","last_active":"2024-04-25T01:26:03.507591+0000","last_peered":"2024-04-25T01:26:03.507591+0000","last_clean":"2024-04-25T01:26:03.507591+0000","last_became_active":"2024-04-25T01:25:53.375219+0000","last_became_peered":"2024-04-25T01:25:53.375219+0000","last_unstale":"2024-04-25T01:26:03.507591+0000","last_undegraded":"2024-04-25T01:26:03.507591+0000","last_fullsized":"2024-04-25T01:26:03.507591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.417443+0000","last_change":"2024-04-25T01:25:52.352994+0000","last_active":"2024-04-25T01:26:03.417443+0000","last_peered":"2024-04-25T01:26:03.417443+0000","last_clean":"2024-04-25T01:26:03.417443+0000","last_became_active":"2024-04-25T01:25:52.352644+0000","last_became_peered":"2024-04-25T01:25:52.352644+0000","last_unstale":"2024-04-25T01:26:03.417443+0000","last_undegraded":"2024-04-25T01:26:03.417443+0000","last_fullsized":"2024-04-25T01:26:03.417443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716482+0000","last_change":"2024-04-25T01:26:01.420891+0000","last_active":"2024-04-25T01:26:03.716482+0000","last_peered":"2024-04-25T01:26:03.716482+0000","last_clean":"2024-04-25T01:26:03.716482+0000","last_became_active":"2024-04-25T01:26:01.420664+0000","last_became_peered":"2024-04-25T01:26:01.420664+0000","last_unstale":"2024-04-25T01:26:03.716482+0000","last_undegraded":"2024-04-25T01:26:03.716482+0000","last_fullsized":"2024-04-25T01:26:03.716482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415682+0000","last_change":"2024-04-25T01:26:02.403489+0000","last_active":"2024-04-25T01:26:03.415682+0000","last_peered":"2024-04-25T01:26:03.415682+0000","last_clean":"2024-04-25T01:26:03.415682+0000","last_became_active":"2024-04-25T01:26:02.403206+0000","last_became_peered":"2024-04-25T01:26:02.403206+0000","last_unstale":"2024-04-25T01:26:03.415682+0000","last_undegraded":"2024-04-25T01:26:03.415682+0000","last_fullsized":"2024-04-25T01:26:03.415682+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:49.872754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.718041+0000","last_change":"2024-04-25T01:25:53.371018+0000","last_active":"2024-04-25T01:26:03.718041+0000","last_peered":"2024-04-25T01:26:03.718041+0000","last_clean":"2024-04-25T01:26:03.718041+0000","last_became_active":"2024-04-25T01:25:53.370539+0000","last_became_peered":"2024-04-25T01:25:53.370539+0000","last_unstale":"2024-04-25T01:26:03.718041+0000","last_undegraded":"2024-04-25T01:26:03.718041+0000","last_fullsized":"2024-04-25T01:26:03.718041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717910+0000","last_change":"2024-04-25T01:25:52.355133+0000","last_active":"2024-04-25T01:26:03.717910+0000","last_peered":"2024-04-25T01:26:03.717910+0000","last_clean":"2024-04-25T01:26:03.717910+0000","last_became_active":"2024-04-25T01:25:52.354836+0000","last_became_peered":"2024-04-25T01:25:52.354836+0000","last_unstale":"2024-04-25T01:26:03.717910+0000","last_undegraded":"2024-04-25T01:26:03.717910+0000","last_fullsized":"2024-04-25T01:26:03.717910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:10.331704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716238+0000","last_change":"2024-04-25T01:26:01.419285+0000","last_active":"2024-04-25T01:26:03.716238+0000","last_peered":"2024-04-25T01:26:03.716238+0000","last_clean":"2024-04-25T01:26:03.716238+0000","last_became_active":"2024-04-25T01:26:01.418987+0000","last_became_peered":"2024-04-25T01:26:01.418987+0000","last_unstale":"2024-04-25T01:26:03.716238+0000","last_undegraded":"2024-04-25T01:26:03.716238+0000","last_fullsized":"2024-04-25T01:26:03.716238+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.506958+0000","last_change":"2024-04-25T01:26:02.416544+0000","last_active":"2024-04-25T01:26:03.506958+0000","last_peered":"2024-04-25T01:26:03.506958+0000","last_clean":"2024-04-25T01:26:03.506958+0000","last_became_active":"2024-04-25T01:26:02.416189+0000","last_became_peered":"2024-04-25T01:26:02.416189+0000","last_unstale":"2024-04-25T01:26:03.506958+0000","last_undegraded":"2024-04-25T01:26:03.506958+0000","last_fullsized":"2024-04-25T01:26:03.506958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.688469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.508134+0000","last_change":"2024-04-25T01:25:53.370754+0000","last_active":"2024-04-25T01:26:03.508134+0000","last_peered":"2024-04-25T01:26:03.508134+0000","last_clean":"2024-04-25T01:26:03.508134+0000","last_became_active":"2024-04-25T01:25:53.370074+0000","last_became_peered":"2024-04-25T01:25:53.370074+0000","last_unstale":"2024-04-25T01:26:03.508134+0000","last_undegraded":"2024-04-25T01:26:03.508134+0000","last_fullsized":"2024-04-25T01:26:03.508134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416836+0000","last_change":"2024-04-25T01:25:52.345943+0000","last_active":"2024-04-25T01:26:03.416836+0000","last_peered":"2024-04-25T01:26:03.416836+0000","last_clean":"2024-04-25T01:26:03.416836+0000","last_became_active":"2024-04-25T01:25:52.345717+0000","last_became_peered":"2024-04-25T01:25:52.345717+0000","last_unstale":"2024-04-25T01:26:03.416836+0000","last_undegraded":"2024-04-25T01:26:03.416836+0000","last_fullsized":"2024-04-25T01:26:03.416836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716622+0000","last_change":"2024-04-25T01:26:01.416027+0000","last_active":"2024-04-25T01:26:03.716622+0000","last_peered":"2024-04-25T01:26:03.716622+0000","last_clean":"2024-04-25T01:26:03.716622+0000","last_became_active":"2024-04-25T01:26:01.415750+0000","last_became_peered":"2024-04-25T01:26:01.415750+0000","last_unstale":"2024-04-25T01:26:03.716622+0000","last_undegraded":"2024-04-25T01:26:03.716622+0000","last_fullsized":"2024-04-25T01:26:03.716622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:27.042376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416127+0000","last_change":"2024-04-25T01:26:02.414929+0000","last_active":"2024-04-25T01:26:03.416127+0000","last_peered":"2024-04-25T01:26:03.416127+0000","last_clean":"2024-04-25T01:26:03.416127+0000","last_became_active":"2024-04-25T01:26:02.414316+0000","last_became_peered":"2024-04-25T01:26:02.414316+0000","last_unstale":"2024-04-25T01:26:03.416127+0000","last_undegraded":"2024-04-25T01:26:03.416127+0000","last_fullsized":"2024-04-25T01:26:03.416127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:53.568065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507875+0000","last_change":"2024-04-25T01:25:53.365171+0000","last_active":"2024-04-25T01:26:03.507875+0000","last_peered":"2024-04-25T01:26:03.507875+0000","last_clean":"2024-04-25T01:26:03.507875+0000","last_became_active":"2024-04-25T01:25:53.364049+0000","last_became_peered":"2024-04-25T01:25:53.364049+0000","last_unstale":"2024-04-25T01:26:03.507875+0000","last_undegraded":"2024-04-25T01:26:03.507875+0000","last_fullsized":"2024-04-25T01:26:03.507875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416635+0000","last_change":"2024-04-25T01:25:52.350688+0000","last_active":"2024-04-25T01:26:03.416635+0000","last_peered":"2024-04-25T01:26:03.416635+0000","last_clean":"2024-04-25T01:26:03.416635+0000","last_became_active":"2024-04-25T01:25:52.350357+0000","last_became_peered":"2024-04-25T01:25:52.350357+0000","last_unstale":"2024-04-25T01:26:03.416635+0000","last_undegraded":"2024-04-25T01:26:03.416635+0000","last_fullsized":"2024-04-25T01:26:03.416635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415491+0000","last_change":"2024-04-25T01:26:01.418580+0000","last_active":"2024-04-25T01:26:03.415491+0000","last_peered":"2024-04-25T01:26:03.415491+0000","last_clean":"2024-04-25T01:26:03.415491+0000","last_became_active":"2024-04-25T01:26:01.418179+0000","last_became_peered":"2024-04-25T01:26:01.418179+0000","last_unstale":"2024-04-25T01:26:03.415491+0000","last_undegraded":"2024-04-25T01:26:03.415491+0000","last_fullsized":"2024-04-25T01:26:03.415491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:42.987327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416438+0000","last_change":"2024-04-25T01:26:02.418424+0000","last_active":"2024-04-25T01:26:03.416438+0000","last_peered":"2024-04-25T01:26:03.416438+0000","last_clean":"2024-04-25T01:26:03.416438+0000","last_became_active":"2024-04-25T01:26:02.418090+0000","last_became_peered":"2024-04-25T01:26:02.418090+0000","last_unstale":"2024-04-25T01:26:03.416438+0000","last_undegraded":"2024-04-25T01:26:03.416438+0000","last_fullsized":"2024-04-25T01:26:03.416438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:10.049310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507217+0000","last_change":"2024-04-25T01:25:53.366376+0000","last_active":"2024-04-25T01:26:03.507217+0000","last_peered":"2024-04-25T01:26:03.507217+0000","last_clean":"2024-04-25T01:26:03.507217+0000","last_became_active":"2024-04-25T01:25:53.365715+0000","last_became_peered":"2024-04-25T01:25:53.365715+0000","last_unstale":"2024-04-25T01:26:03.507217+0000","last_undegraded":"2024-04-25T01:26:03.507217+0000","last_fullsized":"2024-04-25T01:26:03.507217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416219+0000","last_change":"2024-04-25T01:25:52.347413+0000","last_active":"2024-04-25T01:26:03.416219+0000","last_peered":"2024-04-25T01:26:03.416219+0000","last_clean":"2024-04-25T01:26:03.416219+0000","last_became_active":"2024-04-25T01:25:52.347144+0000","last_became_peered":"2024-04-25T01:25:52.347144+0000","last_unstale":"2024-04-25T01:26:03.416219+0000","last_undegraded":"2024-04-25T01:26:03.416219+0000","last_fullsized":"2024-04-25T01:26:03.416219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716710+0000","last_change":"2024-04-25T01:26:01.409917+0000","last_active":"2024-04-25T01:26:03.716710+0000","last_peered":"2024-04-25T01:26:03.716710+0000","last_clean":"2024-04-25T01:26:03.716710+0000","last_became_active":"2024-04-25T01:26:01.409654+0000","last_became_peered":"2024-04-25T01:26:01.409654+0000","last_unstale":"2024-04-25T01:26:03.716710+0000","last_undegraded":"2024-04-25T01:26:03.716710+0000","last_fullsized":"2024-04-25T01:26:03.716710+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:37.142255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717898+0000","last_change":"2024-04-25T01:26:02.425103+0000","last_active":"2024-04-25T01:26:03.717898+0000","last_peered":"2024-04-25T01:26:03.717898+0000","last_clean":"2024-04-25T01:26:03.717898+0000","last_became_active":"2024-04-25T01:26:02.424819+0000","last_became_peered":"2024-04-25T01:26:02.424819+0000","last_unstale":"2024-04-25T01:26:03.717898+0000","last_undegraded":"2024-04-25T01:26:03.717898+0000","last_fullsized":"2024-04-25T01:26:03.717898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716406+0000","last_change":"2024-04-25T01:25:53.364987+0000","last_active":"2024-04-25T01:26:03.716406+0000","last_peered":"2024-04-25T01:26:03.716406+0000","last_clean":"2024-04-25T01:26:03.716406+0000","last_became_active":"2024-04-25T01:25:53.364431+0000","last_became_peered":"2024-04-25T01:25:53.364431+0000","last_unstale":"2024-04-25T01:26:03.716406+0000","last_undegraded":"2024-04-25T01:26:03.716406+0000","last_fullsized":"2024-04-25T01:26:03.716406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716731+0000","last_change":"2024-04-25T01:25:52.351205+0000","last_active":"2024-04-25T01:26:03.716731+0000","last_peered":"2024-04-25T01:26:03.716731+0000","last_clean":"2024-04-25T01:26:03.716731+0000","last_became_active":"2024-04-25T01:25:52.350946+0000","last_became_peered":"2024-04-25T01:25:52.350946+0000","last_unstale":"2024-04-25T01:26:03.716731+0000","last_undegraded":"2024-04-25T01:26:03.716731+0000","last_fullsized":"2024-04-25T01:26:03.716731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416482+0000","last_change":"2024-04-25T01:26:01.415799+0000","last_active":"2024-04-25T01:26:03.416482+0000","last_peered":"2024-04-25T01:26:03.416482+0000","last_clean":"2024-04-25T01:26:03.416482+0000","last_became_active":"2024-04-25T01:26:01.415572+0000","last_became_peered":"2024-04-25T01:26:01.415572+0000","last_unstale":"2024-04-25T01:26:03.416482+0000","last_undegraded":"2024-04-25T01:26:03.416482+0000","last_fullsized":"2024-04-25T01:26:03.416482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:54.034132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717227+0000","last_change":"2024-04-25T01:26:02.422959+0000","last_active":"2024-04-25T01:26:03.717227+0000","last_peered":"2024-04-25T01:26:03.717227+0000","last_clean":"2024-04-25T01:26:03.717227+0000","last_became_active":"2024-04-25T01:26:02.422210+0000","last_became_peered":"2024-04-25T01:26:02.422210+0000","last_unstale":"2024-04-25T01:26:03.717227+0000","last_undegraded":"2024-04-25T01:26:03.717227+0000","last_fullsized":"2024-04-25T01:26:03.717227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507159+0000","last_change":"2024-04-25T01:25:53.368187+0000","last_active":"2024-04-25T01:26:03.507159+0000","last_peered":"2024-04-25T01:26:03.507159+0000","last_clean":"2024-04-25T01:26:03.507159+0000","last_became_active":"2024-04-25T01:25:53.367664+0000","last_became_peered":"2024-04-25T01:25:53.367664+0000","last_unstale":"2024-04-25T01:26:03.507159+0000","last_undegraded":"2024-04-25T01:26:03.507159+0000","last_fullsized":"2024-04-25T01:26:03.507159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717680+0000","last_change":"2024-04-25T01:25:52.357397+0000","last_active":"2024-04-25T01:26:03.717680+0000","last_peered":"2024-04-25T01:26:03.717680+0000","last_clean":"2024-04-25T01:26:03.717680+0000","last_became_active":"2024-04-25T01:25:52.356881+0000","last_became_peered":"2024-04-25T01:25:52.356881+0000","last_unstale":"2024-04-25T01:26:03.717680+0000","last_undegraded":"2024-04-25T01:26:03.717680+0000","last_fullsized":"2024-04-25T01:26:03.717680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"24'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:24.096754+0000","last_change":"2024-04-25T01:26:01.420103+0000","last_active":"2024-04-25T01:26:24.096754+0000","last_peered":"2024-04-25T01:26:24.096754+0000","last_clean":"2024-04-25T01:26:24.096754+0000","last_became_active":"2024-04-25T01:26:01.419526+0000","last_became_peered":"2024-04-25T01:26:01.419526+0000","last_unstale":"2024-04-25T01:26:24.096754+0000","last_undegraded":"2024-04-25T01:26:24.096754+0000","last_fullsized":"2024-04-25T01:26:24.096754+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:27.042376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2053,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416191+0000","last_change":"2024-04-25T01:26:02.421488+0000","last_active":"2024-04-25T01:26:03.416191+0000","last_peered":"2024-04-25T01:26:03.416191+0000","last_clean":"2024-04-25T01:26:03.416191+0000","last_became_active":"2024-04-25T01:26:02.419308+0000","last_became_peered":"2024-04-25T01:26:02.419308+0000","last_unstale":"2024-04-25T01:26:03.416191+0000","last_undegraded":"2024-04-25T01:26:03.416191+0000","last_fullsized":"2024-04-25T01:26:03.416191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716019+0000","last_change":"2024-04-25T01:25:53.370720+0000","last_active":"2024-04-25T01:26:03.716019+0000","last_peered":"2024-04-25T01:26:03.716019+0000","last_clean":"2024-04-25T01:26:03.716019+0000","last_became_active":"2024-04-25T01:25:53.370142+0000","last_became_peered":"2024-04-25T01:25:53.370142+0000","last_unstale":"2024-04-25T01:26:03.716019+0000","last_undegraded":"2024-04-25T01:26:03.716019+0000","last_fullsized":"2024-04-25T01:26:03.716019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:52.833379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716951+0000","last_change":"2024-04-25T01:25:52.349092+0000","last_active":"2024-04-25T01:26:03.716951+0000","last_peered":"2024-04-25T01:26:03.716951+0000","last_clean":"2024-04-25T01:26:03.716951+0000","last_became_active":"2024-04-25T01:25:52.348846+0000","last_became_peered":"2024-04-25T01:25:52.348846+0000","last_unstale":"2024-04-25T01:26:03.716951+0000","last_undegraded":"2024-04-25T01:26:03.716951+0000","last_fullsized":"2024-04-25T01:26:03.716951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:21.982440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717840+0000","last_change":"2024-04-25T01:26:01.414517+0000","last_active":"2024-04-25T01:26:03.717840+0000","last_peered":"2024-04-25T01:26:03.717840+0000","last_clean":"2024-04-25T01:26:03.717840+0000","last_became_active":"2024-04-25T01:26:01.414243+0000","last_became_peered":"2024-04-25T01:26:01.414243+0000","last_unstale":"2024-04-25T01:26:03.717840+0000","last_undegraded":"2024-04-25T01:26:03.717840+0000","last_fullsized":"2024-04-25T01:26:03.717840+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:48.491461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415418+0000","last_change":"2024-04-25T01:26:02.415893+0000","last_active":"2024-04-25T01:26:03.415418+0000","last_peered":"2024-04-25T01:26:03.415418+0000","last_clean":"2024-04-25T01:26:03.415418+0000","last_became_active":"2024-04-25T01:26:02.415596+0000","last_became_peered":"2024-04-25T01:26:02.415596+0000","last_unstale":"2024-04-25T01:26:03.415418+0000","last_undegraded":"2024-04-25T01:26:03.415418+0000","last_fullsized":"2024-04-25T01:26:03.415418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507428+0000","last_change":"2024-04-25T01:25:53.373572+0000","last_active":"2024-04-25T01:26:03.507428+0000","last_peered":"2024-04-25T01:26:03.507428+0000","last_clean":"2024-04-25T01:26:03.507428+0000","last_became_active":"2024-04-25T01:25:53.373129+0000","last_became_peered":"2024-04-25T01:25:53.373129+0000","last_unstale":"2024-04-25T01:26:03.507428+0000","last_undegraded":"2024-04-25T01:26:03.507428+0000","last_fullsized":"2024-04-25T01:26:03.507428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:22.122293+0000","last_change":"2024-04-25T01:25:52.350446+0000","last_active":"2024-04-25T01:26:22.122293+0000","last_peered":"2024-04-25T01:26:22.122293+0000","last_clean":"2024-04-25T01:26:22.122293+0000","last_became_active":"2024-04-25T01:25:52.349448+0000","last_became_peered":"2024-04-25T01:25:52.349448+0000","last_unstale":"2024-04-25T01:26:22.122293+0000","last_undegraded":"2024-04-25T01:26:22.122293+0000","last_fullsized":"2024-04-25T01:26:22.122293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2053,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716463+0000","last_change":"2024-04-25T01:26:01.418794+0000","last_active":"2024-04-25T01:26:03.716463+0000","last_peered":"2024-04-25T01:26:03.716463+0000","last_clean":"2024-04-25T01:26:03.716463+0000","last_became_active":"2024-04-25T01:26:01.418541+0000","last_became_peered":"2024-04-25T01:26:01.418541+0000","last_unstale":"2024-04-25T01:26:03.716463+0000","last_undegraded":"2024-04-25T01:26:03.716463+0000","last_fullsized":"2024-04-25T01:26:03.716463+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717164+0000","last_change":"2024-04-25T01:26:02.425391+0000","last_active":"2024-04-25T01:26:03.717164+0000","last_peered":"2024-04-25T01:26:03.717164+0000","last_clean":"2024-04-25T01:26:03.717164+0000","last_became_active":"2024-04-25T01:26:02.425106+0000","last_became_peered":"2024-04-25T01:26:02.425106+0000","last_unstale":"2024-04-25T01:26:03.717164+0000","last_undegraded":"2024-04-25T01:26:03.717164+0000","last_fullsized":"2024-04-25T01:26:03.717164+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416706+0000","last_change":"2024-04-25T01:25:53.366416+0000","last_active":"2024-04-25T01:26:03.416706+0000","last_peered":"2024-04-25T01:26:03.416706+0000","last_clean":"2024-04-25T01:26:03.416706+0000","last_became_active":"2024-04-25T01:25:53.366157+0000","last_became_peered":"2024-04-25T01:25:53.366157+0000","last_unstale":"2024-04-25T01:26:03.416706+0000","last_undegraded":"2024-04-25T01:26:03.416706+0000","last_fullsized":"2024-04-25T01:26:03.416706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:26.277566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717854+0000","last_change":"2024-04-25T01:25:52.357610+0000","last_active":"2024-04-25T01:26:03.717854+0000","last_peered":"2024-04-25T01:26:03.717854+0000","last_clean":"2024-04-25T01:26:03.717854+0000","last_became_active":"2024-04-25T01:25:52.357355+0000","last_became_peered":"2024-04-25T01:25:52.357355+0000","last_unstale":"2024-04-25T01:26:03.717854+0000","last_undegraded":"2024-04-25T01:26:03.717854+0000","last_fullsized":"2024-04-25T01:26:03.717854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416422+0000","last_change":"2024-04-25T01:26:01.412666+0000","last_active":"2024-04-25T01:26:03.416422+0000","last_peered":"2024-04-25T01:26:03.416422+0000","last_clean":"2024-04-25T01:26:03.416422+0000","last_became_active":"2024-04-25T01:26:01.412132+0000","last_became_peered":"2024-04-25T01:26:01.412132+0000","last_unstale":"2024-04-25T01:26:03.416422+0000","last_undegraded":"2024-04-25T01:26:03.416422+0000","last_fullsized":"2024-04-25T01:26:03.416422+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716704+0000","last_change":"2024-04-25T01:26:02.413106+0000","last_active":"2024-04-25T01:26:03.716704+0000","last_peered":"2024-04-25T01:26:03.716704+0000","last_clean":"2024-04-25T01:26:03.716704+0000","last_became_active":"2024-04-25T01:26:02.412759+0000","last_became_peered":"2024-04-25T01:26:02.412759+0000","last_unstale":"2024-04-25T01:26:03.716704+0000","last_undegraded":"2024-04-25T01:26:03.716704+0000","last_fullsized":"2024-04-25T01:26:03.716704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:28.049190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716160+0000","last_change":"2024-04-25T01:25:53.374311+0000","last_active":"2024-04-25T01:26:03.716160+0000","last_peered":"2024-04-25T01:26:03.716160+0000","last_clean":"2024-04-25T01:26:03.716160+0000","last_became_active":"2024-04-25T01:25:53.373890+0000","last_became_peered":"2024-04-25T01:25:53.373890+0000","last_unstale":"2024-04-25T01:26:03.716160+0000","last_undegraded":"2024-04-25T01:26:03.716160+0000","last_fullsized":"2024-04-25T01:26:03.716160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717907+0000","last_change":"2024-04-25T01:25:52.356231+0000","last_active":"2024-04-25T01:26:03.717907+0000","last_peered":"2024-04-25T01:26:03.717907+0000","last_clean":"2024-04-25T01:26:03.717907+0000","last_became_active":"2024-04-25T01:25:52.355913+0000","last_became_peered":"2024-04-25T01:25:52.355913+0000","last_unstale":"2024-04-25T01:26:03.717907+0000","last_undegraded":"2024-04-25T01:26:03.717907+0000","last_fullsized":"2024-04-25T01:26:03.717907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415065+0000","last_change":"2024-04-25T01:26:01.419513+0000","last_active":"2024-04-25T01:26:03.415065+0000","last_peered":"2024-04-25T01:26:03.415065+0000","last_clean":"2024-04-25T01:26:03.415065+0000","last_became_active":"2024-04-25T01:26:01.419015+0000","last_became_peered":"2024-04-25T01:26:01.419015+0000","last_unstale":"2024-04-25T01:26:03.415065+0000","last_undegraded":"2024-04-25T01:26:03.415065+0000","last_fullsized":"2024-04-25T01:26:03.415065+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:10.486367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507391+0000","last_change":"2024-04-25T01:26:02.425241+0000","last_active":"2024-04-25T01:26:03.507391+0000","last_peered":"2024-04-25T01:26:03.507391+0000","last_clean":"2024-04-25T01:26:03.507391+0000","last_became_active":"2024-04-25T01:26:02.424960+0000","last_became_peered":"2024-04-25T01:26:02.424960+0000","last_unstale":"2024-04-25T01:26:03.507391+0000","last_undegraded":"2024-04-25T01:26:03.507391+0000","last_fullsized":"2024-04-25T01:26:03.507391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717765+0000","last_change":"2024-04-25T01:25:53.371445+0000","last_active":"2024-04-25T01:26:03.717765+0000","last_peered":"2024-04-25T01:26:03.717765+0000","last_clean":"2024-04-25T01:26:03.717765+0000","last_became_active":"2024-04-25T01:25:53.371086+0000","last_became_peered":"2024-04-25T01:25:53.371086+0000","last_unstale":"2024-04-25T01:26:03.717765+0000","last_undegraded":"2024-04-25T01:26:03.717765+0000","last_fullsized":"2024-04-25T01:26:03.717765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.718056+0000","last_change":"2024-04-25T01:25:52.357459+0000","last_active":"2024-04-25T01:26:03.718056+0000","last_peered":"2024-04-25T01:26:03.718056+0000","last_clean":"2024-04-25T01:26:03.718056+0000","last_became_active":"2024-04-25T01:25:52.357182+0000","last_became_peered":"2024-04-25T01:25:52.357182+0000","last_unstale":"2024-04-25T01:26:03.718056+0000","last_undegraded":"2024-04-25T01:26:03.718056+0000","last_fullsized":"2024-04-25T01:26:03.718056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415347+0000","last_change":"2024-04-25T01:26:01.415321+0000","last_active":"2024-04-25T01:26:03.415347+0000","last_peered":"2024-04-25T01:26:03.415347+0000","last_clean":"2024-04-25T01:26:03.415347+0000","last_became_active":"2024-04-25T01:26:01.415020+0000","last_became_peered":"2024-04-25T01:26:01.415020+0000","last_unstale":"2024-04-25T01:26:03.415347+0000","last_undegraded":"2024-04-25T01:26:03.415347+0000","last_fullsized":"2024-04-25T01:26:03.415347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:20.090411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717822+0000","last_change":"2024-04-25T01:26:02.416917+0000","last_active":"2024-04-25T01:26:03.717822+0000","last_peered":"2024-04-25T01:26:03.717822+0000","last_clean":"2024-04-25T01:26:03.717822+0000","last_became_active":"2024-04-25T01:26:02.416623+0000","last_became_peered":"2024-04-25T01:26:02.416623+0000","last_unstale":"2024-04-25T01:26:03.717822+0000","last_undegraded":"2024-04-25T01:26:03.717822+0000","last_fullsized":"2024-04-25T01:26:03.717822+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:55.040946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716446+0000","last_change":"2024-04-25T01:26:01.415472+0000","last_active":"2024-04-25T01:26:03.716446+0000","last_peered":"2024-04-25T01:26:03.716446+0000","last_clean":"2024-04-25T01:26:03.716446+0000","last_became_active":"2024-04-25T01:26:01.415184+0000","last_became_peered":"2024-04-25T01:26:01.415184+0000","last_unstale":"2024-04-25T01:26:03.716446+0000","last_undegraded":"2024-04-25T01:26:03.716446+0000","last_fullsized":"2024-04-25T01:26:03.716446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716757+0000","last_change":"2024-04-25T01:25:53.363043+0000","last_active":"2024-04-25T01:26:03.716757+0000","last_peered":"2024-04-25T01:26:03.716757+0000","last_clean":"2024-04-25T01:26:03.716757+0000","last_became_active":"2024-04-25T01:25:53.362601+0000","last_became_peered":"2024-04-25T01:25:53.362601+0000","last_unstale":"2024-04-25T01:26:03.716757+0000","last_undegraded":"2024-04-25T01:26:03.716757+0000","last_fullsized":"2024-04-25T01:26:03.716757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716830+0000","last_change":"2024-04-25T01:25:52.358081+0000","last_active":"2024-04-25T01:26:03.716830+0000","last_peered":"2024-04-25T01:26:03.716830+0000","last_clean":"2024-04-25T01:26:03.716830+0000","last_became_active":"2024-04-25T01:25:52.357365+0000","last_became_peered":"2024-04-25T01:25:52.357365+0000","last_unstale":"2024-04-25T01:26:03.716830+0000","last_undegraded":"2024-04-25T01:26:03.716830+0000","last_fullsized":"2024-04-25T01:26:03.716830+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716855+0000","last_change":"2024-04-25T01:26:02.418872+0000","last_active":"2024-04-25T01:26:03.716855+0000","last_peered":"2024-04-25T01:26:03.716855+0000","last_clean":"2024-04-25T01:26:03.716855+0000","last_became_active":"2024-04-25T01:26:02.418614+0000","last_became_peered":"2024-04-25T01:26:02.418614+0000","last_unstale":"2024-04-25T01:26:03.716855+0000","last_undegraded":"2024-04-25T01:26:03.716855+0000","last_fullsized":"2024-04-25T01:26:03.716855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:49.498275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717818+0000","last_change":"2024-04-25T01:26:01.410258+0000","last_active":"2024-04-25T01:26:03.717818+0000","last_peered":"2024-04-25T01:26:03.717818+0000","last_clean":"2024-04-25T01:26:03.717818+0000","last_became_active":"2024-04-25T01:26:01.410004+0000","last_became_peered":"2024-04-25T01:26:01.410004+0000","last_unstale":"2024-04-25T01:26:03.717818+0000","last_undegraded":"2024-04-25T01:26:03.717818+0000","last_fullsized":"2024-04-25T01:26:03.717818+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:48.865940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415284+0000","last_change":"2024-04-25T01:25:53.363794+0000","last_active":"2024-04-25T01:26:03.415284+0000","last_peered":"2024-04-25T01:26:03.415284+0000","last_clean":"2024-04-25T01:26:03.415284+0000","last_became_active":"2024-04-25T01:25:53.363509+0000","last_became_peered":"2024-04-25T01:25:53.363509+0000","last_unstale":"2024-04-25T01:26:03.415284+0000","last_undegraded":"2024-04-25T01:26:03.415284+0000","last_fullsized":"2024-04-25T01:26:03.415284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415685+0000","last_change":"2024-04-25T01:25:52.347464+0000","last_active":"2024-04-25T01:26:03.415685+0000","last_peered":"2024-04-25T01:26:03.415685+0000","last_clean":"2024-04-25T01:26:03.415685+0000","last_became_active":"2024-04-25T01:25:52.347138+0000","last_became_peered":"2024-04-25T01:25:52.347138+0000","last_unstale":"2024-04-25T01:26:03.415685+0000","last_undegraded":"2024-04-25T01:26:03.415685+0000","last_fullsized":"2024-04-25T01:26:03.415685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717780+0000","last_change":"2024-04-25T01:26:02.425948+0000","last_active":"2024-04-25T01:26:03.717780+0000","last_peered":"2024-04-25T01:26:03.717780+0000","last_clean":"2024-04-25T01:26:03.717780+0000","last_became_active":"2024-04-25T01:26:02.424947+0000","last_became_peered":"2024-04-25T01:26:02.424947+0000","last_unstale":"2024-04-25T01:26:03.717780+0000","last_undegraded":"2024-04-25T01:26:03.717780+0000","last_fullsized":"2024-04-25T01:26:03.717780+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716700+0000","last_change":"2024-04-25T01:25:52.349989+0000","last_active":"2024-04-25T01:26:03.716700+0000","last_peered":"2024-04-25T01:26:03.716700+0000","last_clean":"2024-04-25T01:26:03.716700+0000","last_became_active":"2024-04-25T01:25:52.349672+0000","last_became_peered":"2024-04-25T01:25:52.349672+0000","last_unstale":"2024-04-25T01:26:03.716700+0000","last_undegraded":"2024-04-25T01:26:03.716700+0000","last_fullsized":"2024-04-25T01:26:03.716700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716805+0000","last_change":"2024-04-25T01:25:53.356610+0000","last_active":"2024-04-25T01:26:03.716805+0000","last_peered":"2024-04-25T01:26:03.716805+0000","last_clean":"2024-04-25T01:26:03.716805+0000","last_became_active":"2024-04-25T01:25:53.356015+0000","last_became_peered":"2024-04-25T01:25:53.356015+0000","last_unstale":"2024-04-25T01:26:03.716805+0000","last_undegraded":"2024-04-25T01:26:03.716805+0000","last_fullsized":"2024-04-25T01:26:03.716805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.417192+0000","last_change":"2024-04-25T01:26:01.420261+0000","last_active":"2024-04-25T01:26:03.417192+0000","last_peered":"2024-04-25T01:26:03.417192+0000","last_clean":"2024-04-25T01:26:03.417192+0000","last_became_active":"2024-04-25T01:26:01.420003+0000","last_became_peered":"2024-04-25T01:26:01.420003+0000","last_unstale":"2024-04-25T01:26:03.417192+0000","last_undegraded":"2024-04-25T01:26:03.417192+0000","last_fullsized":"2024-04-25T01:26:03.417192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:28.695853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416039+0000","last_change":"2024-04-25T01:26:02.426327+0000","last_active":"2024-04-25T01:26:03.416039+0000","last_peered":"2024-04-25T01:26:03.416039+0000","last_clean":"2024-04-25T01:26:03.416039+0000","last_became_active":"2024-04-25T01:26:02.426054+0000","last_became_peered":"2024-04-25T01:26:02.426054+0000","last_unstale":"2024-04-25T01:26:03.416039+0000","last_undegraded":"2024-04-25T01:26:03.416039+0000","last_fullsized":"2024-04-25T01:26:03.416039+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:17.809738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415686+0000","last_change":"2024-04-25T01:25:52.353955+0000","last_active":"2024-04-25T01:26:03.415686+0000","last_peered":"2024-04-25T01:26:03.415686+0000","last_clean":"2024-04-25T01:26:03.415686+0000","last_became_active":"2024-04-25T01:25:52.353659+0000","last_became_peered":"2024-04-25T01:25:52.353659+0000","last_unstale":"2024-04-25T01:26:03.415686+0000","last_undegraded":"2024-04-25T01:26:03.415686+0000","last_fullsized":"2024-04-25T01:26:03.415686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416142+0000","last_change":"2024-04-25T01:25:53.366586+0000","last_active":"2024-04-25T01:26:03.416142+0000","last_peered":"2024-04-25T01:26:03.416142+0000","last_clean":"2024-04-25T01:26:03.416142+0000","last_became_active":"2024-04-25T01:25:53.366302+0000","last_became_peered":"2024-04-25T01:25:53.366302+0000","last_unstale":"2024-04-25T01:26:03.416142+0000","last_undegraded":"2024-04-25T01:26:03.416142+0000","last_fullsized":"2024-04-25T01:26:03.416142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416415+0000","last_change":"2024-04-25T01:26:01.420960+0000","last_active":"2024-04-25T01:26:03.416415+0000","last_peered":"2024-04-25T01:26:03.416415+0000","last_clean":"2024-04-25T01:26:03.416415+0000","last_became_active":"2024-04-25T01:26:01.420713+0000","last_became_peered":"2024-04-25T01:26:01.420713+0000","last_unstale":"2024-04-25T01:26:03.416415+0000","last_undegraded":"2024-04-25T01:26:03.416415+0000","last_fullsized":"2024-04-25T01:26:03.416415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:29.489275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416652+0000","last_change":"2024-04-25T01:26:02.419282+0000","last_active":"2024-04-25T01:26:03.416652+0000","last_peered":"2024-04-25T01:26:03.416652+0000","last_clean":"2024-04-25T01:26:03.416652+0000","last_became_active":"2024-04-25T01:26:02.418802+0000","last_became_peered":"2024-04-25T01:26:02.418802+0000","last_unstale":"2024-04-25T01:26:03.416652+0000","last_undegraded":"2024-04-25T01:26:03.416652+0000","last_fullsized":"2024-04-25T01:26:03.416652+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:46.807175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716343+0000","last_change":"2024-04-25T01:25:53.367143+0000","last_active":"2024-04-25T01:26:03.716343+0000","last_peered":"2024-04-25T01:26:03.716343+0000","last_clean":"2024-04-25T01:26:03.716343+0000","last_became_active":"2024-04-25T01:25:53.366847+0000","last_became_peered":"2024-04-25T01:25:53.366847+0000","last_unstale":"2024-04-25T01:26:03.716343+0000","last_undegraded":"2024-04-25T01:26:03.716343+0000","last_fullsized":"2024-04-25T01:26:03.716343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716783+0000","last_change":"2024-04-25T01:26:02.416438+0000","last_active":"2024-04-25T01:26:03.716783+0000","last_peered":"2024-04-25T01:26:03.716783+0000","last_clean":"2024-04-25T01:26:03.716783+0000","last_became_active":"2024-04-25T01:26:02.416170+0000","last_became_peered":"2024-04-25T01:26:02.416170+0000","last_unstale":"2024-04-25T01:26:03.716783+0000","last_undegraded":"2024-04-25T01:26:03.716783+0000","last_fullsized":"2024-04-25T01:26:03.716783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415637+0000","last_change":"2024-04-25T01:25:53.371345+0000","last_active":"2024-04-25T01:26:03.415637+0000","last_peered":"2024-04-25T01:26:03.415637+0000","last_clean":"2024-04-25T01:26:03.415637+0000","last_became_active":"2024-04-25T01:25:53.371080+0000","last_became_peered":"2024-04-25T01:25:53.371080+0000","last_unstale":"2024-04-25T01:26:03.415637+0000","last_undegraded":"2024-04-25T01:26:03.415637+0000","last_fullsized":"2024-04-25T01:26:03.415637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:09.855763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717353+0000","last_change":"2024-04-25T01:26:02.419688+0000","last_active":"2024-04-25T01:26:03.717353+0000","last_peered":"2024-04-25T01:26:03.717353+0000","last_clean":"2024-04-25T01:26:03.717353+0000","last_became_active":"2024-04-25T01:26:02.419417+0000","last_became_peered":"2024-04-25T01:26:02.419417+0000","last_unstale":"2024-04-25T01:26:03.717353+0000","last_undegraded":"2024-04-25T01:26:03.717353+0000","last_fullsized":"2024-04-25T01:26:03.717353+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717442+0000","last_change":"2024-04-25T01:25:53.369850+0000","last_active":"2024-04-25T01:26:03.717442+0000","last_peered":"2024-04-25T01:26:03.717442+0000","last_clean":"2024-04-25T01:26:03.717442+0000","last_became_active":"2024-04-25T01:25:53.369350+0000","last_became_peered":"2024-04-25T01:25:53.369350+0000","last_unstale":"2024-04-25T01:26:03.717442+0000","last_undegraded":"2024-04-25T01:26:03.717442+0000","last_fullsized":"2024-04-25T01:26:03.717442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717742+0000","last_change":"2024-04-25T01:26:02.425528+0000","last_active":"2024-04-25T01:26:03.717742+0000","last_peered":"2024-04-25T01:26:03.717742+0000","last_clean":"2024-04-25T01:26:03.717742+0000","last_became_active":"2024-04-25T01:26:02.425160+0000","last_became_peered":"2024-04-25T01:26:02.425160+0000","last_unstale":"2024-04-25T01:26:03.717742+0000","last_undegraded":"2024-04-25T01:26:03.717742+0000","last_fullsized":"2024-04-25T01:26:03.717742+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716710+0000","last_change":"2024-04-25T01:25:53.357142+0000","last_active":"2024-04-25T01:26:03.716710+0000","last_peered":"2024-04-25T01:26:03.716710+0000","last_clean":"2024-04-25T01:26:03.716710+0000","last_became_active":"2024-04-25T01:25:53.356854+0000","last_became_peered":"2024-04-25T01:25:53.356854+0000","last_unstale":"2024-04-25T01:26:03.716710+0000","last_undegraded":"2024-04-25T01:26:03.716710+0000","last_fullsized":"2024-04-25T01:26:03.716710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507419+0000","last_change":"2024-04-25T01:26:02.417492+0000","last_active":"2024-04-25T01:26:03.507419+0000","last_peered":"2024-04-25T01:26:03.507419+0000","last_clean":"2024-04-25T01:26:03.507419+0000","last_became_active":"2024-04-25T01:26:02.417234+0000","last_became_peered":"2024-04-25T01:26:02.417234+0000","last_unstale":"2024-04-25T01:26:03.507419+0000","last_undegraded":"2024-04-25T01:26:03.507419+0000","last_fullsized":"2024-04-25T01:26:03.507419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416287+0000","last_change":"2024-04-25T01:25:53.364575+0000","last_active":"2024-04-25T01:26:03.416287+0000","last_peered":"2024-04-25T01:26:03.416287+0000","last_clean":"2024-04-25T01:26:03.416287+0000","last_became_active":"2024-04-25T01:25:53.364311+0000","last_became_peered":"2024-04-25T01:25:53.364311+0000","last_unstale":"2024-04-25T01:26:03.416287+0000","last_undegraded":"2024-04-25T01:26:03.416287+0000","last_fullsized":"2024-04-25T01:26:03.416287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415942+0000","last_change":"2024-04-25T01:26:02.426813+0000","last_active":"2024-04-25T01:26:03.415942+0000","last_peered":"2024-04-25T01:26:03.415942+0000","last_clean":"2024-04-25T01:26:03.415942+0000","last_became_active":"2024-04-25T01:26:02.426552+0000","last_became_peered":"2024-04-25T01:26:02.426552+0000","last_unstale":"2024-04-25T01:26:03.415942+0000","last_undegraded":"2024-04-25T01:26:03.415942+0000","last_fullsized":"2024-04-25T01:26:03.415942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:03.687174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717821+0000","last_change":"2024-04-25T01:25:53.368413+0000","last_active":"2024-04-25T01:26:03.717821+0000","last_peered":"2024-04-25T01:26:03.717821+0000","last_clean":"2024-04-25T01:26:03.717821+0000","last_became_active":"2024-04-25T01:25:53.368176+0000","last_became_peered":"2024-04-25T01:25:53.368176+0000","last_unstale":"2024-04-25T01:26:03.717821+0000","last_undegraded":"2024-04-25T01:26:03.717821+0000","last_fullsized":"2024-04-25T01:26:03.717821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.506869+0000","last_change":"2024-04-25T01:26:02.425185+0000","last_active":"2024-04-25T01:26:03.506869+0000","last_peered":"2024-04-25T01:26:03.506869+0000","last_clean":"2024-04-25T01:26:03.506869+0000","last_became_active":"2024-04-25T01:26:02.424903+0000","last_became_peered":"2024-04-25T01:26:02.424903+0000","last_unstale":"2024-04-25T01:26:03.506869+0000","last_undegraded":"2024-04-25T01:26:03.506869+0000","last_fullsized":"2024-04-25T01:26:03.506869+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717561+0000","last_change":"2024-04-25T01:25:53.372638+0000","last_active":"2024-04-25T01:26:03.717561+0000","last_peered":"2024-04-25T01:26:03.717561+0000","last_clean":"2024-04-25T01:26:03.717561+0000","last_became_active":"2024-04-25T01:25:53.372001+0000","last_became_peered":"2024-04-25T01:25:53.372001+0000","last_unstale":"2024-04-25T01:26:03.717561+0000","last_undegraded":"2024-04-25T01:26:03.717561+0000","last_fullsized":"2024-04-25T01:26:03.717561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716617+0000","last_change":"2024-04-25T01:26:02.420867+0000","last_active":"2024-04-25T01:26:03.716617+0000","last_peered":"2024-04-25T01:26:03.716617+0000","last_clean":"2024-04-25T01:26:03.716617+0000","last_became_active":"2024-04-25T01:26:02.420621+0000","last_became_peered":"2024-04-25T01:26:02.420621+0000","last_unstale":"2024-04-25T01:26:03.716617+0000","last_undegraded":"2024-04-25T01:26:03.716617+0000","last_fullsized":"2024-04-25T01:26:03.716617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:11.493181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416565+0000","last_change":"2024-04-25T01:25:53.365509+0000","last_active":"2024-04-25T01:26:03.416565+0000","last_peered":"2024-04-25T01:26:03.416565+0000","last_clean":"2024-04-25T01:26:03.416565+0000","last_became_active":"2024-04-25T01:25:53.365245+0000","last_became_peered":"2024-04-25T01:25:53.365245+0000","last_unstale":"2024-04-25T01:26:03.416565+0000","last_undegraded":"2024-04-25T01:26:03.416565+0000","last_fullsized":"2024-04-25T01:26:03.416565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416549+0000","last_change":"2024-04-25T01:26:02.415672+0000","last_active":"2024-04-25T01:26:03.416549+0000","last_peered":"2024-04-25T01:26:03.416549+0000","last_clean":"2024-04-25T01:26:03.416549+0000","last_became_active":"2024-04-25T01:26:02.415389+0000","last_became_peered":"2024-04-25T01:26:02.415389+0000","last_unstale":"2024-04-25T01:26:03.416549+0000","last_undegraded":"2024-04-25T01:26:03.416549+0000","last_fullsized":"2024-04-25T01:26:03.416549+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:52.342053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716741+0000","last_change":"2024-04-25T01:25:53.365391+0000","last_active":"2024-04-25T01:26:03.716741+0000","last_peered":"2024-04-25T01:26:03.716741+0000","last_clean":"2024-04-25T01:26:03.716741+0000","last_became_active":"2024-04-25T01:25:53.364829+0000","last_became_peered":"2024-04-25T01:25:53.364829+0000","last_unstale":"2024-04-25T01:26:03.716741+0000","last_undegraded":"2024-04-25T01:26:03.716741+0000","last_fullsized":"2024-04-25T01:26:03.716741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507955+0000","last_change":"2024-04-25T01:26:02.418292+0000","last_active":"2024-04-25T01:26:03.507955+0000","last_peered":"2024-04-25T01:26:03.507955+0000","last_clean":"2024-04-25T01:26:03.507955+0000","last_became_active":"2024-04-25T01:26:02.418036+0000","last_became_peered":"2024-04-25T01:26:02.418036+0000","last_unstale":"2024-04-25T01:26:03.507955+0000","last_undegraded":"2024-04-25T01:26:03.507955+0000","last_fullsized":"2024-04-25T01:26:03.507955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415608+0000","last_change":"2024-04-25T01:25:53.355667+0000","last_active":"2024-04-25T01:26:03.415608+0000","last_peered":"2024-04-25T01:26:03.415608+0000","last_clean":"2024-04-25T01:26:03.415608+0000","last_became_active":"2024-04-25T01:25:53.355417+0000","last_became_peered":"2024-04-25T01:25:53.355417+0000","last_unstale":"2024-04-25T01:26:03.415608+0000","last_undegraded":"2024-04-25T01:26:03.415608+0000","last_fullsized":"2024-04-25T01:26:03.415608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415307+0000","last_change":"2024-04-25T01:26:02.414994+0000","last_active":"2024-04-25T01:26:03.415307+0000","last_peered":"2024-04-25T01:26:03.415307+0000","last_clean":"2024-04-25T01:26:03.415307+0000","last_became_active":"2024-04-25T01:26:02.414735+0000","last_became_peered":"2024-04-25T01:26:02.414735+0000","last_unstale":"2024-04-25T01:26:03.415307+0000","last_undegraded":"2024-04-25T01:26:03.415307+0000","last_fullsized":"2024-04-25T01:26:03.415307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:44.960251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717184+0000","last_change":"2024-04-25T01:25:53.371240+0000","last_active":"2024-04-25T01:26:03.717184+0000","last_peered":"2024-04-25T01:26:03.717184+0000","last_clean":"2024-04-25T01:26:03.717184+0000","last_became_active":"2024-04-25T01:25:53.370164+0000","last_became_peered":"2024-04-25T01:25:53.370164+0000","last_unstale":"2024-04-25T01:26:03.717184+0000","last_undegraded":"2024-04-25T01:26:03.717184+0000","last_fullsized":"2024-04-25T01:26:03.717184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416367+0000","last_change":"2024-04-25T01:26:02.418272+0000","last_active":"2024-04-25T01:26:03.416367+0000","last_peered":"2024-04-25T01:26:03.416367+0000","last_clean":"2024-04-25T01:26:03.416367+0000","last_became_active":"2024-04-25T01:26:02.417977+0000","last_became_peered":"2024-04-25T01:26:02.417977+0000","last_unstale":"2024-04-25T01:26:03.416367+0000","last_undegraded":"2024-04-25T01:26:03.416367+0000","last_fullsized":"2024-04-25T01:26:03.416367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716511+0000","last_change":"2024-04-25T01:25:53.376415+0000","last_active":"2024-04-25T01:26:03.716511+0000","last_peered":"2024-04-25T01:26:03.716511+0000","last_clean":"2024-04-25T01:26:03.716511+0000","last_became_active":"2024-04-25T01:25:53.376104+0000","last_became_peered":"2024-04-25T01:25:53.376104+0000","last_unstale":"2024-04-25T01:26:03.716511+0000","last_undegraded":"2024-04-25T01:26:03.716511+0000","last_fullsized":"2024-04-25T01:26:03.716511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416512+0000","last_change":"2024-04-25T01:26:02.426333+0000","last_active":"2024-04-25T01:26:03.416512+0000","last_peered":"2024-04-25T01:26:03.416512+0000","last_clean":"2024-04-25T01:26:03.416512+0000","last_became_active":"2024-04-25T01:26:02.426069+0000","last_became_peered":"2024-04-25T01:26:02.426069+0000","last_unstale":"2024-04-25T01:26:03.416512+0000","last_undegraded":"2024-04-25T01:26:03.416512+0000","last_fullsized":"2024-04-25T01:26:03.416512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716975+0000","last_change":"2024-04-25T01:25:53.370314+0000","last_active":"2024-04-25T01:26:03.716975+0000","last_peered":"2024-04-25T01:26:03.716975+0000","last_clean":"2024-04-25T01:26:03.716975+0000","last_became_active":"2024-04-25T01:25:53.370049+0000","last_became_peered":"2024-04-25T01:25:53.370049+0000","last_unstale":"2024-04-25T01:26:03.716975+0000","last_undegraded":"2024-04-25T01:26:03.716975+0000","last_fullsized":"2024-04-25T01:26:03.716975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415866+0000","last_change":"2024-04-25T01:26:02.415464+0000","last_active":"2024-04-25T01:26:03.415866+0000","last_peered":"2024-04-25T01:26:03.415866+0000","last_clean":"2024-04-25T01:26:03.415866+0000","last_became_active":"2024-04-25T01:26:02.415173+0000","last_became_peered":"2024-04-25T01:26:02.415173+0000","last_unstale":"2024-04-25T01:26:03.415866+0000","last_undegraded":"2024-04-25T01:26:03.415866+0000","last_fullsized":"2024-04-25T01:26:03.415866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.343169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507691+0000","last_change":"2024-04-25T01:25:53.369585+0000","last_active":"2024-04-25T01:26:03.507691+0000","last_peered":"2024-04-25T01:26:03.507691+0000","last_clean":"2024-04-25T01:26:03.507691+0000","last_became_active":"2024-04-25T01:25:53.368499+0000","last_became_peered":"2024-04-25T01:25:53.368499+0000","last_unstale":"2024-04-25T01:26:03.507691+0000","last_undegraded":"2024-04-25T01:26:03.507691+0000","last_fullsized":"2024-04-25T01:26:03.507691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716525+0000","last_change":"2024-04-25T01:26:02.421975+0000","last_active":"2024-04-25T01:26:03.716525+0000","last_peered":"2024-04-25T01:26:03.716525+0000","last_clean":"2024-04-25T01:26:03.716525+0000","last_became_active":"2024-04-25T01:26:02.421698+0000","last_became_peered":"2024-04-25T01:26:02.421698+0000","last_unstale":"2024-04-25T01:26:03.716525+0000","last_undegraded":"2024-04-25T01:26:03.716525+0000","last_fullsized":"2024-04-25T01:26:03.716525+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416746+0000","last_change":"2024-04-25T01:25:53.370666+0000","last_active":"2024-04-25T01:26:03.416746+0000","last_peered":"2024-04-25T01:26:03.416746+0000","last_clean":"2024-04-25T01:26:03.416746+0000","last_became_active":"2024-04-25T01:25:53.369733+0000","last_became_peered":"2024-04-25T01:25:53.369733+0000","last_unstale":"2024-04-25T01:26:03.416746+0000","last_undegraded":"2024-04-25T01:26:03.416746+0000","last_fullsized":"2024-04-25T01:26:03.416746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.715962+0000","last_change":"2024-04-25T01:26:02.418114+0000","last_active":"2024-04-25T01:26:03.715962+0000","last_peered":"2024-04-25T01:26:03.715962+0000","last_clean":"2024-04-25T01:26:03.715962+0000","last_became_active":"2024-04-25T01:26:02.417810+0000","last_became_peered":"2024-04-25T01:26:02.417810+0000","last_unstale":"2024-04-25T01:26:03.715962+0000","last_undegraded":"2024-04-25T01:26:03.715962+0000","last_fullsized":"2024-04-25T01:26:03.715962+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:30.496089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.718049+0000","last_change":"2024-04-25T01:25:53.372048+0000","last_active":"2024-04-25T01:26:03.718049+0000","last_peered":"2024-04-25T01:26:03.718049+0000","last_clean":"2024-04-25T01:26:03.718049+0000","last_became_active":"2024-04-25T01:25:53.371767+0000","last_became_peered":"2024-04-25T01:25:53.371767+0000","last_unstale":"2024-04-25T01:26:03.718049+0000","last_undegraded":"2024-04-25T01:26:03.718049+0000","last_fullsized":"2024-04-25T01:26:03.718049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:05.589744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716672+0000","last_change":"2024-04-25T01:26:02.426289+0000","last_active":"2024-04-25T01:26:03.716672+0000","last_peered":"2024-04-25T01:26:03.716672+0000","last_clean":"2024-04-25T01:26:03.716672+0000","last_became_active":"2024-04-25T01:26:02.426015+0000","last_became_peered":"2024-04-25T01:26:02.426015+0000","last_unstale":"2024-04-25T01:26:03.716672+0000","last_undegraded":"2024-04-25T01:26:03.716672+0000","last_fullsized":"2024-04-25T01:26:03.716672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716678+0000","last_change":"2024-04-25T01:25:53.374186+0000","last_active":"2024-04-25T01:26:03.716678+0000","last_peered":"2024-04-25T01:26:03.716678+0000","last_clean":"2024-04-25T01:26:03.716678+0000","last_became_active":"2024-04-25T01:25:53.373766+0000","last_became_peered":"2024-04-25T01:25:53.373766+0000","last_unstale":"2024-04-25T01:26:03.716678+0000","last_undegraded":"2024-04-25T01:26:03.716678+0000","last_fullsized":"2024-04-25T01:26:03.716678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416320+0000","last_change":"2024-04-25T01:26:02.416345+0000","last_active":"2024-04-25T01:26:03.416320+0000","last_peered":"2024-04-25T01:26:03.416320+0000","last_clean":"2024-04-25T01:26:03.416320+0000","last_became_active":"2024-04-25T01:26:02.416082+0000","last_became_peered":"2024-04-25T01:26:02.416082+0000","last_unstale":"2024-04-25T01:26:03.416320+0000","last_undegraded":"2024-04-25T01:26:03.416320+0000","last_fullsized":"2024-04-25T01:26:03.416320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:47.810943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716847+0000","last_change":"2024-04-25T01:25:53.371602+0000","last_active":"2024-04-25T01:26:03.716847+0000","last_peered":"2024-04-25T01:26:03.716847+0000","last_clean":"2024-04-25T01:26:03.716847+0000","last_became_active":"2024-04-25T01:25:53.371362+0000","last_became_peered":"2024-04-25T01:25:53.371362+0000","last_unstale":"2024-04-25T01:26:03.716847+0000","last_undegraded":"2024-04-25T01:26:03.716847+0000","last_fullsized":"2024-04-25T01:26:03.716847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717204+0000","last_change":"2024-04-25T01:26:02.425888+0000","last_active":"2024-04-25T01:26:03.717204+0000","last_peered":"2024-04-25T01:26:03.717204+0000","last_clean":"2024-04-25T01:26:03.717204+0000","last_became_active":"2024-04-25T01:26:02.425577+0000","last_became_peered":"2024-04-25T01:26:02.425577+0000","last_unstale":"2024-04-25T01:26:03.717204+0000","last_undegraded":"2024-04-25T01:26:03.717204+0000","last_fullsized":"2024-04-25T01:26:03.717204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.715963+0000","last_change":"2024-04-25T01:25:53.370996+0000","last_active":"2024-04-25T01:26:03.715963+0000","last_peered":"2024-04-25T01:26:03.715963+0000","last_clean":"2024-04-25T01:26:03.715963+0000","last_became_active":"2024-04-25T01:25:53.370479+0000","last_became_peered":"2024-04-25T01:25:53.370479+0000","last_unstale":"2024-04-25T01:26:03.715963+0000","last_undegraded":"2024-04-25T01:26:03.715963+0000","last_fullsized":"2024-04-25T01:26:03.715963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:57.636067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507529+0000","last_change":"2024-04-25T01:26:02.423420+0000","last_active":"2024-04-25T01:26:03.507529+0000","last_peered":"2024-04-25T01:26:03.507529+0000","last_clean":"2024-04-25T01:26:03.507529+0000","last_became_active":"2024-04-25T01:26:02.423146+0000","last_became_peered":"2024-04-25T01:26:02.423146+0000","last_unstale":"2024-04-25T01:26:03.507529+0000","last_undegraded":"2024-04-25T01:26:03.507529+0000","last_fullsized":"2024-04-25T01:26:03.507529+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415469+0000","last_change":"2024-04-25T01:25:53.363512+0000","last_active":"2024-04-25T01:26:03.415469+0000","last_peered":"2024-04-25T01:26:03.415469+0000","last_clean":"2024-04-25T01:26:03.415469+0000","last_became_active":"2024-04-25T01:25:53.363246+0000","last_became_peered":"2024-04-25T01:25:53.363246+0000","last_unstale":"2024-04-25T01:26:03.415469+0000","last_undegraded":"2024-04-25T01:26:03.415469+0000","last_fullsized":"2024-04-25T01:26:03.415469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415970+0000","last_change":"2024-04-25T01:26:02.413346+0000","last_active":"2024-04-25T01:26:03.415970+0000","last_peered":"2024-04-25T01:26:03.415970+0000","last_clean":"2024-04-25T01:26:03.415970+0000","last_became_active":"2024-04-25T01:26:02.413036+0000","last_became_peered":"2024-04-25T01:26:02.413036+0000","last_unstale":"2024-04-25T01:26:03.415970+0000","last_undegraded":"2024-04-25T01:26:03.415970+0000","last_fullsized":"2024-04-25T01:26:03.415970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416642+0000","last_change":"2024-04-25T01:25:53.364457+0000","last_active":"2024-04-25T01:26:03.416642+0000","last_peered":"2024-04-25T01:26:03.416642+0000","last_clean":"2024-04-25T01:26:03.416642+0000","last_became_active":"2024-04-25T01:25:53.364103+0000","last_became_peered":"2024-04-25T01:25:53.364103+0000","last_unstale":"2024-04-25T01:26:03.416642+0000","last_undegraded":"2024-04-25T01:26:03.416642+0000","last_fullsized":"2024-04-25T01:26:03.416642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:11.336752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415365+0000","last_change":"2024-04-25T01:26:02.418792+0000","last_active":"2024-04-25T01:26:03.415365+0000","last_peered":"2024-04-25T01:26:03.415365+0000","last_clean":"2024-04-25T01:26:03.415365+0000","last_became_active":"2024-04-25T01:26:02.416942+0000","last_became_peered":"2024-04-25T01:26:02.416942+0000","last_unstale":"2024-04-25T01:26:03.415365+0000","last_undegraded":"2024-04-25T01:26:03.415365+0000","last_fullsized":"2024-04-25T01:26:03.415365+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:59.492678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717288+0000","last_change":"2024-04-25T01:25:53.371647+0000","last_active":"2024-04-25T01:26:03.717288+0000","last_peered":"2024-04-25T01:26:03.717288+0000","last_clean":"2024-04-25T01:26:03.717288+0000","last_became_active":"2024-04-25T01:25:53.371353+0000","last_became_peered":"2024-04-25T01:25:53.371353+0000","last_unstale":"2024-04-25T01:26:03.717288+0000","last_undegraded":"2024-04-25T01:26:03.717288+0000","last_fullsized":"2024-04-25T01:26:03.717288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507578+0000","last_change":"2024-04-25T01:26:02.426335+0000","last_active":"2024-04-25T01:26:03.507578+0000","last_peered":"2024-04-25T01:26:03.507578+0000","last_clean":"2024-04-25T01:26:03.507578+0000","last_became_active":"2024-04-25T01:26:02.426024+0000","last_became_peered":"2024-04-25T01:26:02.426024+0000","last_unstale":"2024-04-25T01:26:03.507578+0000","last_undegraded":"2024-04-25T01:26:03.507578+0000","last_fullsized":"2024-04-25T01:26:03.507578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717460+0000","last_change":"2024-04-25T01:25:53.372842+0000","last_active":"2024-04-25T01:26:03.717460+0000","last_peered":"2024-04-25T01:26:03.717460+0000","last_clean":"2024-04-25T01:26:03.717460+0000","last_became_active":"2024-04-25T01:25:53.372580+0000","last_became_peered":"2024-04-25T01:25:53.372580+0000","last_unstale":"2024-04-25T01:26:03.717460+0000","last_undegraded":"2024-04-25T01:26:03.717460+0000","last_fullsized":"2024-04-25T01:26:03.717460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716748+0000","last_change":"2024-04-25T01:26:02.414632+0000","last_active":"2024-04-25T01:26:03.716748+0000","last_peered":"2024-04-25T01:26:03.716748+0000","last_clean":"2024-04-25T01:26:03.716748+0000","last_became_active":"2024-04-25T01:26:02.414388+0000","last_became_peered":"2024-04-25T01:26:02.414388+0000","last_unstale":"2024-04-25T01:26:03.716748+0000","last_undegraded":"2024-04-25T01:26:03.716748+0000","last_fullsized":"2024-04-25T01:26:03.716748+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.508086+0000","last_change":"2024-04-25T01:25:53.376081+0000","last_active":"2024-04-25T01:26:03.508086+0000","last_peered":"2024-04-25T01:26:03.508086+0000","last_clean":"2024-04-25T01:26:03.508086+0000","last_became_active":"2024-04-25T01:25:53.375814+0000","last_became_peered":"2024-04-25T01:25:53.375814+0000","last_unstale":"2024-04-25T01:26:03.508086+0000","last_undegraded":"2024-04-25T01:26:03.508086+0000","last_fullsized":"2024-04-25T01:26:03.508086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:29.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.416509+0000","last_change":"2024-04-25T01:26:02.414366+0000","last_active":"2024-04-25T01:26:03.416509+0000","last_peered":"2024-04-25T01:26:03.416509+0000","last_clean":"2024-04-25T01:26:03.416509+0000","last_became_active":"2024-04-25T01:26:02.414061+0000","last_became_peered":"2024-04-25T01:26:02.414061+0000","last_unstale":"2024-04-25T01:26:03.416509+0000","last_undegraded":"2024-04-25T01:26:03.416509+0000","last_fullsized":"2024-04-25T01:26:03.416509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:19.644468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415110+0000","last_change":"2024-04-25T01:25:53.374061+0000","last_active":"2024-04-25T01:26:03.415110+0000","last_peered":"2024-04-25T01:26:03.415110+0000","last_clean":"2024-04-25T01:26:03.415110+0000","last_became_active":"2024-04-25T01:25:53.373848+0000","last_became_peered":"2024-04-25T01:25:53.373848+0000","last_unstale":"2024-04-25T01:26:03.415110+0000","last_undegraded":"2024-04-25T01:26:03.415110+0000","last_fullsized":"2024-04-25T01:26:03.415110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717762+0000","last_change":"2024-04-25T01:26:02.414620+0000","last_active":"2024-04-25T01:26:03.717762+0000","last_peered":"2024-04-25T01:26:03.717762+0000","last_clean":"2024-04-25T01:26:03.717762+0000","last_became_active":"2024-04-25T01:26:02.414326+0000","last_became_peered":"2024-04-25T01:26:02.414326+0000","last_unstale":"2024-04-25T01:26:03.717762+0000","last_undegraded":"2024-04-25T01:26:03.717762+0000","last_fullsized":"2024-04-25T01:26:03.717762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:30.496089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507095+0000","last_change":"2024-04-25T01:25:53.364698+0000","last_active":"2024-04-25T01:26:03.507095+0000","last_peered":"2024-04-25T01:26:03.507095+0000","last_clean":"2024-04-25T01:26:03.507095+0000","last_became_active":"2024-04-25T01:25:53.364435+0000","last_became_peered":"2024-04-25T01:25:53.364435+0000","last_unstale":"2024-04-25T01:26:03.507095+0000","last_undegraded":"2024-04-25T01:26:03.507095+0000","last_fullsized":"2024-04-25T01:26:03.507095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716509+0000","last_change":"2024-04-25T01:26:02.418331+0000","last_active":"2024-04-25T01:26:03.716509+0000","last_peered":"2024-04-25T01:26:03.716509+0000","last_clean":"2024-04-25T01:26:03.716509+0000","last_became_active":"2024-04-25T01:26:02.418057+0000","last_became_peered":"2024-04-25T01:26:02.418057+0000","last_unstale":"2024-04-25T01:26:03.716509+0000","last_undegraded":"2024-04-25T01:26:03.716509+0000","last_fullsized":"2024-04-25T01:26:03.716509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:17.809738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717064+0000","last_change":"2024-04-25T01:25:53.375324+0000","last_active":"2024-04-25T01:26:03.717064+0000","last_peered":"2024-04-25T01:26:03.717064+0000","last_clean":"2024-04-25T01:26:03.717064+0000","last_became_active":"2024-04-25T01:25:53.374392+0000","last_became_peered":"2024-04-25T01:25:53.374392+0000","last_unstale":"2024-04-25T01:26:03.717064+0000","last_undegraded":"2024-04-25T01:26:03.717064+0000","last_fullsized":"2024-04-25T01:26:03.717064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717818+0000","last_change":"2024-04-25T01:26:02.422720+0000","last_active":"2024-04-25T01:26:03.717818+0000","last_peered":"2024-04-25T01:26:03.717818+0000","last_clean":"2024-04-25T01:26:03.717818+0000","last_became_active":"2024-04-25T01:26:02.421848+0000","last_became_peered":"2024-04-25T01:26:02.421848+0000","last_unstale":"2024-04-25T01:26:03.717818+0000","last_undegraded":"2024-04-25T01:26:03.717818+0000","last_fullsized":"2024-04-25T01:26:03.717818+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507255+0000","last_change":"2024-04-25T01:25:53.367953+0000","last_active":"2024-04-25T01:26:03.507255+0000","last_peered":"2024-04-25T01:26:03.507255+0000","last_clean":"2024-04-25T01:26:03.507255+0000","last_became_active":"2024-04-25T01:25:53.367358+0000","last_became_peered":"2024-04-25T01:25:53.367358+0000","last_unstale":"2024-04-25T01:26:03.507255+0000","last_undegraded":"2024-04-25T01:26:03.507255+0000","last_fullsized":"2024-04-25T01:26:03.507255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415875+0000","last_change":"2024-04-25T01:26:02.422300+0000","last_active":"2024-04-25T01:26:03.415875+0000","last_peered":"2024-04-25T01:26:03.415875+0000","last_clean":"2024-04-25T01:26:03.415875+0000","last_became_active":"2024-04-25T01:26:02.421702+0000","last_became_peered":"2024-04-25T01:26:02.421702+0000","last_unstale":"2024-04-25T01:26:03.415875+0000","last_undegraded":"2024-04-25T01:26:03.415875+0000","last_fullsized":"2024-04-25T01:26:03.415875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.717355+0000","last_change":"2024-04-25T01:25:53.371513+0000","last_active":"2024-04-25T01:26:03.717355+0000","last_peered":"2024-04-25T01:26:03.717355+0000","last_clean":"2024-04-25T01:26:03.717355+0000","last_became_active":"2024-04-25T01:25:53.370421+0000","last_became_peered":"2024-04-25T01:25:53.370421+0000","last_unstale":"2024-04-25T01:26:03.717355+0000","last_undegraded":"2024-04-25T01:26:03.717355+0000","last_fullsized":"2024-04-25T01:26:03.717355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.415236+0000","last_change":"2024-04-25T01:26:02.411784+0000","last_active":"2024-04-25T01:26:03.415236+0000","last_peered":"2024-04-25T01:26:03.415236+0000","last_clean":"2024-04-25T01:26:03.415236+0000","last_became_active":"2024-04-25T01:26:02.411478+0000","last_became_peered":"2024-04-25T01:26:02.411478+0000","last_unstale":"2024-04-25T01:26:03.415236+0000","last_undegraded":"2024-04-25T01:26:03.415236+0000","last_fullsized":"2024-04-25T01:26:03.415236+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:20.194816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716214+0000","last_change":"2024-04-25T01:25:53.363829+0000","last_active":"2024-04-25T01:26:03.716214+0000","last_peered":"2024-04-25T01:26:03.716214+0000","last_clean":"2024-04-25T01:26:03.716214+0000","last_became_active":"2024-04-25T01:25:53.363525+0000","last_became_peered":"2024-04-25T01:25:53.363525+0000","last_unstale":"2024-04-25T01:26:03.716214+0000","last_undegraded":"2024-04-25T01:26:03.716214+0000","last_fullsized":"2024-04-25T01:26:03.716214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.506843+0000","last_change":"2024-04-25T01:26:02.421208+0000","last_active":"2024-04-25T01:26:03.506843+0000","last_peered":"2024-04-25T01:26:03.506843+0000","last_clean":"2024-04-25T01:26:03.506843+0000","last_became_active":"2024-04-25T01:26:02.418796+0000","last_became_peered":"2024-04-25T01:26:02.418796+0000","last_unstale":"2024-04-25T01:26:03.506843+0000","last_undegraded":"2024-04-25T01:26:03.506843+0000","last_fullsized":"2024-04-25T01:26:03.506843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.716784+0000","last_change":"2024-04-25T01:25:53.365057+0000","last_active":"2024-04-25T01:26:03.716784+0000","last_peered":"2024-04-25T01:26:03.716784+0000","last_clean":"2024-04-25T01:26:03.716784+0000","last_became_active":"2024-04-25T01:25:53.364793+0000","last_became_peered":"2024-04-25T01:25:53.364793+0000","last_unstale":"2024-04-25T01:26:03.716784+0000","last_undegraded":"2024-04-25T01:26:03.716784+0000","last_fullsized":"2024-04-25T01:26:03.716784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.507060+0000","last_change":"2024-04-25T01:26:02.417606+0000","last_active":"2024-04-25T01:26:03.507060+0000","last_peered":"2024-04-25T01:26:03.507060+0000","last_clean":"2024-04-25T01:26:03.507060+0000","last_became_active":"2024-04-25T01:26:02.417350+0000","last_became_peered":"2024-04-25T01:26:02.417350+0000","last_unstale":"2024-04-25T01:26:03.507060+0000","last_undegraded":"2024-04-25T01:26:03.507060+0000","last_fullsized":"2024-04-25T01:26:03.507060+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T01:26:03.508077+0000","last_change":"2024-04-25T01:25:53.367762+0000","last_active":"2024-04-25T01:26:03.508077+0000","last_peered":"2024-04-25T01:26:03.508077+0000","last_clean":"2024-04-25T01:26:03.508077+0000","last_became_active":"2024-04-25T01:25:53.367383+0000","last_became_peered":"2024-04-25T01:25:53.367383+0000","last_unstale":"2024-04-25T01:26:03.508077+0000","last_undegraded":"2024-04-25T01:26:03.508077+0000","last_fullsized":"2024-04-25T01:26:03.508077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:05.589744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2851,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2851,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":690301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":101930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":99491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":103389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":99331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":101496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":690267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:26:27.087 INFO:tasks.mon_thrash.ceph_manager:clean! 2024-04-25T01:26:27.087 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:26:27.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.236+0000 7efe63ca5640 1 Processor -- start 2024-04-25T01:26:27.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.236+0000 7efe63ca5640 1 -- start start 2024-04-25T01:26:27.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.236+0000 7efe63ca5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efe5c0c3400 0x7efe5c158da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:27.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.236+0000 7efe63ca5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe5c155de0 0x7efe5c156230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:27.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.237+0000 7efe63ca5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe5c06f2d0 con 0x7efe5c152d60 2024-04-25T01:26:27.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.237+0000 7efe63ca5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7efe5c156770 con 0x7efe5c0c3400 2024-04-25T01:26:27.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.237+0000 7efe63ca5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efe5c159cd0 con 0x7efe5c155de0 2024-04-25T01:26:27.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.237+0000 7efe61240640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efe5c0c3400 0x7efe5c158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:27.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.237+0000 7efe61a41640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7efe5c152d60 0x7efe5c0c3b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53514/0 (socket says 172.21.15.112:53514) 2024-04-25T01:26:27.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.237+0000 7efe61a41640 1 -- 172.21.15.112:0/2557575728 learned_addr learned my addr 172.21.15.112:0/2557575728 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:27.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.237+0000 7efe61240640 1 -- 172.21.15.112:0/2557575728 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe5c155de0 msgr2=0x7efe5c156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:27.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.237+0000 7efe62242640 1 --2- 172.21.15.112:0/2557575728 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe5c155de0 0x7efe5c156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:27.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.237+0000 7efe60a3f640 1 -- 172.21.15.112:0/2557575728 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1208317084 0 0) 0x7efe5c06f2d0 con 0x7efe5c152d60 2024-04-25T01:26:27.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.237+0000 7efe61240640 1 --2- 172.21.15.112:0/2557575728 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe5c155de0 0x7efe5c156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.237+0000 7efe61240640 1 -- 172.21.15.112:0/2557575728 >> v1:172.21.15.112:6789/0 conn(0x7efe5c152d60 legacy=0x7efe5c0c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:27.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.237+0000 7efe61240640 1 -- 172.21.15.112:0/2557575728 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5c159430 con 0x7efe5c0c3400 2024-04-25T01:26:27.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.237+0000 7efe62242640 1 --2- 172.21.15.112:0/2557575728 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe5c155de0 0x7efe5c156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:27.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.238+0000 7efe61240640 1 --2- 172.21.15.112:0/2557575728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe5c0c3400 0x7efe5c158da0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0097a0 tx=0x7efe4c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446a2019951ba0a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:27.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.238+0000 7efe60a3f640 1 -- 172.21.15.112:0/2557575728 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c037070 con 0x7efe5c0c3400 2024-04-25T01:26:27.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.238+0000 7efe60a3f640 1 -- 172.21.15.112:0/2557575728 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe4c02adb0 con 0x7efe5c0c3400 2024-04-25T01:26:27.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.238+0000 7efe60a3f640 1 -- 172.21.15.112:0/2557575728 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c032660 con 0x7efe5c0c3400 2024-04-25T01:26:27.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.238+0000 7efe63ca5640 1 -- 172.21.15.112:0/2557575728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe5c0c3400 msgr2=0x7efe5c158da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:27.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.238+0000 7efe63ca5640 1 --2- 172.21.15.112:0/2557575728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe5c0c3400 0x7efe5c158da0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0097a0 tx=0x7efe4c02a060 comp rx=0 tx=0).stop 2024-04-25T01:26:27.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.238+0000 7efe63ca5640 1 -- 172.21.15.112:0/2557575728 shutdown_connections 2024-04-25T01:26:27.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.238+0000 7efe63ca5640 1 --2- 172.21.15.112:0/2557575728 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe5c155de0 0x7efe5c156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.238+0000 7efe63ca5640 1 --2- 172.21.15.112:0/2557575728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe5c0c3400 0x7efe5c158da0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.238+0000 7efe63ca5640 1 -- 172.21.15.112:0/2557575728 >> 172.21.15.112:0/2557575728 conn(0x7efe5c06b160 msgr2=0x7efe5c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:27.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.238+0000 7efe63ca5640 1 -- 172.21.15.112:0/2557575728 shutdown_connections 2024-04-25T01:26:27.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.238+0000 7efe63ca5640 1 -- 172.21.15.112:0/2557575728 wait complete. 2024-04-25T01:26:27.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.240+0000 7efe63ca5640 1 Processor -- start 2024-04-25T01:26:27.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.240+0000 7efe63ca5640 1 -- start start 2024-04-25T01:26:27.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.240+0000 7efe63ca5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe5c0c3400 0x7efe5c158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:27.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.240+0000 7efe63ca5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe5c152d60 0x7efe5c06c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:27.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.240+0000 7efe63ca5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe5c155de0 0x7efe5c154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:27.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.240+0000 7efe63ca5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c1558c0 con 0x7efe5c155de0 2024-04-25T01:26:27.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.240+0000 7efe63ca5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c1599f0 con 0x7efe5c0c3400 2024-04-25T01:26:27.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.240+0000 7efe63ca5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efe5c153140 con 0x7efe5c152d60 2024-04-25T01:26:27.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.240+0000 7efe62242640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe5c155de0 0x7efe5c154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:27.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.240+0000 7efe62242640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe5c155de0 0x7efe5c154870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46322/0 (socket says 172.21.15.112:46322) 2024-04-25T01:26:27.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.240+0000 7efe62242640 1 -- 172.21.15.112:0/1989370445 learned_addr learned my addr 172.21.15.112:0/1989370445 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:27.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.241+0000 7efe62242640 1 -- 172.21.15.112:0/1989370445 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe5c152d60 msgr2=0x7efe5c06c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:27.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.241+0000 7efe62242640 1 --2- 172.21.15.112:0/1989370445 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe5c152d60 0x7efe5c06c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.241+0000 7efe62242640 1 -- 172.21.15.112:0/1989370445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe5c0c3400 msgr2=0x7efe5c158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:27.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.241+0000 7efe62242640 1 --2- 172.21.15.112:0/1989370445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe5c0c3400 0x7efe5c158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.241+0000 7efe62242640 1 -- 172.21.15.112:0/1989370445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5c1eb890 con 0x7efe5c155de0 2024-04-25T01:26:27.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.241+0000 7efe62242640 1 --2- 172.21.15.112:0/1989370445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe5c155de0 0x7efe5c154870 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efe5400cd50 tx=0x7efe5400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:27.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.241+0000 7efe527fc640 1 -- 172.21.15.112:0/1989370445 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5400f680 con 0x7efe5c155de0 2024-04-25T01:26:27.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.241+0000 7efe527fc640 1 -- 172.21.15.112:0/1989370445 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe5400f840 con 0x7efe5c155de0 2024-04-25T01:26:27.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.241+0000 7efe63ca5640 1 -- 172.21.15.112:0/1989370445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5c153ab0 con 0x7efe5c155de0 2024-04-25T01:26:27.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.241+0000 7efe527fc640 1 -- 172.21.15.112:0/1989370445 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5400fc30 con 0x7efe5c155de0 2024-04-25T01:26:27.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.241+0000 7efe63ca5640 1 -- 172.21.15.112:0/1989370445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe5c154070 con 0x7efe5c155de0 2024-04-25T01:26:27.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.242+0000 7efe63ca5640 1 -- 172.21.15.112:0/1989370445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe5c0682a0 con 0x7efe5c155de0 2024-04-25T01:26:27.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.242+0000 7efe527fc640 1 -- 172.21.15.112:0/1989370445 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe540213f0 con 0x7efe5c155de0 2024-04-25T01:26:27.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.243+0000 7efe527fc640 1 --2- 172.21.15.112:0/1989370445 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efe4006be30 0x7efe4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:27.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.243+0000 7efe527fc640 1 -- 172.21.15.112:0/1989370445 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7efe5408be80 con 0x7efe5c155de0 2024-04-25T01:26:27.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.243+0000 7efe61a41640 1 --2- 172.21.15.112:0/1989370445 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efe4006be30 0x7efe4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:27.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.243+0000 7efe61a41640 1 --2- 172.21.15.112:0/1989370445 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efe4006be30 0x7efe4006e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efe58007fd0 tx=0x7efe58007960 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:27.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.247+0000 7efe527fc640 1 -- 172.21.15.112:0/1989370445 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efe54014030 con 0x7efe5c155de0 2024-04-25T01:26:27.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.423+0000 7efe63ca5640 1 -- 172.21.15.112:0/1989370445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe5c0c22d0 con 0x7efe5c155de0 2024-04-25T01:26:27.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.425+0000 7efe527fc640 1 -- 172.21.15.112:0/1989370445 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20367 (secure 0 0 0) 0x7efe5405a590 con 0x7efe5c155de0 2024-04-25T01:26:27.404 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:27.404 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:26:03.405271+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-25T01:25:59.478275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-25T01:26:00.719358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:26:27.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.427+0000 7efe63ca5640 1 -- 172.21.15.112:0/1989370445 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efe4006be30 msgr2=0x7efe4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:27.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.427+0000 7efe63ca5640 1 --2- 172.21.15.112:0/1989370445 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efe4006be30 0x7efe4006e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efe58007fd0 tx=0x7efe58007960 comp rx=0 tx=0).stop 2024-04-25T01:26:27.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.427+0000 7efe63ca5640 1 -- 172.21.15.112:0/1989370445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe5c155de0 msgr2=0x7efe5c154870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:27.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.427+0000 7efe63ca5640 1 --2- 172.21.15.112:0/1989370445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe5c155de0 0x7efe5c154870 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efe5400cd50 tx=0x7efe5400a650 comp rx=0 tx=0).stop 2024-04-25T01:26:27.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.427+0000 7efe63ca5640 1 -- 172.21.15.112:0/1989370445 shutdown_connections 2024-04-25T01:26:27.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.427+0000 7efe63ca5640 1 --2- 172.21.15.112:0/1989370445 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efe4006be30 0x7efe4006e2f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.427+0000 7efe63ca5640 1 --2- 172.21.15.112:0/1989370445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe5c155de0 0x7efe5c154870 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.427+0000 7efe63ca5640 1 --2- 172.21.15.112:0/1989370445 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe5c152d60 0x7efe5c06c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.427+0000 7efe63ca5640 1 --2- 172.21.15.112:0/1989370445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe5c0c3400 0x7efe5c158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.427+0000 7efe63ca5640 1 -- 172.21.15.112:0/1989370445 >> 172.21.15.112:0/1989370445 conn(0x7efe5c06b160 msgr2=0x7efe5c06b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:27.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.427+0000 7efe63ca5640 1 -- 172.21.15.112:0/1989370445 shutdown_connections 2024-04-25T01:26:27.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.427+0000 7efe63ca5640 1 -- 172.21.15.112:0/1989370445 wait complete. 2024-04-25T01:26:27.417 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:25:40.679119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-04-25T01:25:50.593045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-04-25T01:25:51.664930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-04-25T01:25:59.478275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-04-25T01:26:00.719358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:26:27.418 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:26:27.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.538+0000 7feac8631640 1 Processor -- start 2024-04-25T01:26:27.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.538+0000 7feac8631640 1 -- start start 2024-04-25T01:26:27.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.539+0000 7feac8631640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feac00ca370 0x7feac006c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:27.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.539+0000 7feac8631640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feac00ca8a0 0x7feac00c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:27.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.539+0000 7feac8631640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feac01534a0 con 0x7feac00c9250 2024-04-25T01:26:27.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.539+0000 7feac8631640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7feac0156990 con 0x7feac00ca8a0 2024-04-25T01:26:27.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.539+0000 7feac8631640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feac0156500 con 0x7feac00ca370 2024-04-25T01:26:27.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.539+0000 7feac6bce640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feac00ca8a0 0x7feac00c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:27.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.539+0000 7feac6bce640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feac00ca8a0 0x7feac00c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58496/0 (socket says 172.21.15.112:58496) 2024-04-25T01:26:27.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.539+0000 7feac6bce640 1 -- 172.21.15.112:0/1937480281 learned_addr learned my addr 172.21.15.112:0/1937480281 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:27.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.540+0000 7feac5bcc640 1 --2- 172.21.15.112:0/1937480281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feac00ca370 0x7feac006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:27.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.540+0000 7feac6bce640 1 -- 172.21.15.112:0/1937480281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feac00ca370 msgr2=0x7feac006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:27.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.540+0000 7feac6bce640 1 --2- 172.21.15.112:0/1937480281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feac00ca370 0x7feac006c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.540+0000 7feac6bce640 1 -- 172.21.15.112:0/1937480281 >> v1:172.21.15.187:6789/0 conn(0x7feac00c9250 legacy=0x7feac006bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:27.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.540+0000 7feac6bce640 1 -- 172.21.15.112:0/1937480281 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac00cae30 con 0x7feac00ca8a0 2024-04-25T01:26:27.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.540+0000 7feac5bcc640 1 --2- 172.21.15.112:0/1937480281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feac00ca370 0x7feac006c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.540+0000 7feac6bce640 1 --2- 172.21.15.112:0/1937480281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac00ca8a0 0x7feac00c9630 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feab8009e50 tx=0x7feab802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7431944ef53e60c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.540+0000 7feac53cb640 1 -- 172.21.15.112:0/1937480281 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8036070 con 0x7feac00ca8a0 2024-04-25T01:26:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.540+0000 7feac53cb640 1 -- 172.21.15.112:0/1937480281 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feab8031600 con 0x7feac00ca8a0 2024-04-25T01:26:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.540+0000 7feac53cb640 1 -- 172.21.15.112:0/1937480281 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab80319d0 con 0x7feac00ca8a0 2024-04-25T01:26:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.541+0000 7feac8631640 1 -- 172.21.15.112:0/1937480281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac00ca8a0 msgr2=0x7feac00c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.541+0000 7feac8631640 1 --2- 172.21.15.112:0/1937480281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac00ca8a0 0x7feac00c9630 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feab8009e50 tx=0x7feab802a630 comp rx=0 tx=0).stop 2024-04-25T01:26:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.541+0000 7feac8631640 1 -- 172.21.15.112:0/1937480281 shutdown_connections 2024-04-25T01:26:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.541+0000 7feac8631640 1 --2- 172.21.15.112:0/1937480281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac00ca8a0 0x7feac00c9630 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.541+0000 7feac8631640 1 --2- 172.21.15.112:0/1937480281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feac00ca370 0x7feac006c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.541+0000 7feac8631640 1 -- 172.21.15.112:0/1937480281 >> 172.21.15.112:0/1937480281 conn(0x7feac00cb310 msgr2=0x7feac00cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.541+0000 7feac8631640 1 -- 172.21.15.112:0/1937480281 shutdown_connections 2024-04-25T01:26:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.541+0000 7feac8631640 1 -- 172.21.15.112:0/1937480281 wait complete. 2024-04-25T01:26:27.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.542+0000 7feac8631640 1 Processor -- start 2024-04-25T01:26:27.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.542+0000 7feac8631640 1 -- start start 2024-04-25T01:26:27.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.543+0000 7feac8631640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac00c9250 0x7feac01eb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:27.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.543+0000 7feac8631640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feac00ca370 0x7feac0150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:27.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.543+0000 7feac8631640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feac00ca8a0 0x7feac01ef170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:27.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.543+0000 7feac8631640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feac0155eb0 con 0x7feac00c9250 2024-04-25T01:26:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.543+0000 7feac8631640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feac0155bd0 con 0x7feac00ca370 2024-04-25T01:26:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.543+0000 7feac8631640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feac0156190 con 0x7feac00ca8a0 2024-04-25T01:26:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.543+0000 7feac63cd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac00c9250 0x7feac01eb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.543+0000 7feac63cd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac00c9250 0x7feac01eb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58498/0 (socket says 172.21.15.112:58498) 2024-04-25T01:26:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.543+0000 7feac63cd640 1 -- 172.21.15.112:0/844077047 learned_addr learned my addr 172.21.15.112:0/844077047 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.543+0000 7feac6bce640 1 --2- 172.21.15.112:0/844077047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feac00ca8a0 0x7feac01ef170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.543+0000 7feac5bcc640 1 --2- 172.21.15.112:0/844077047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feac00ca370 0x7feac0150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.543+0000 7feac63cd640 1 -- 172.21.15.112:0/844077047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feac00ca8a0 msgr2=0x7feac01ef170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.543+0000 7feac63cd640 1 --2- 172.21.15.112:0/844077047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feac00ca8a0 0x7feac01ef170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.543+0000 7feac63cd640 1 -- 172.21.15.112:0/844077047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feac00ca370 msgr2=0x7feac0150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.543+0000 7feac63cd640 1 --2- 172.21.15.112:0/844077047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feac00ca370 0x7feac0150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.543+0000 7feac63cd640 1 -- 172.21.15.112:0/844077047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac01ee9b0 con 0x7feac00c9250 2024-04-25T01:26:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.543+0000 7feac63cd640 1 --2- 172.21.15.112:0/844077047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac00c9250 0x7feac01eb430 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feab400c430 tx=0x7feab400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.544+0000 7feab2ffd640 1 -- 172.21.15.112:0/844077047 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab4011790 con 0x7feac00c9250 2024-04-25T01:26:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.544+0000 7feab2ffd640 1 -- 172.21.15.112:0/844077047 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feab4017070 con 0x7feac00c9250 2024-04-25T01:26:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.544+0000 7feab2ffd640 1 -- 172.21.15.112:0/844077047 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab4011b70 con 0x7feac00c9250 2024-04-25T01:26:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.544+0000 7feac8631640 1 -- 172.21.15.112:0/844077047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac00c2bd0 con 0x7feac00c9250 2024-04-25T01:26:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.544+0000 7feac8631640 1 -- 172.21.15.112:0/844077047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac0150fc0 con 0x7feac00c9250 2024-04-25T01:26:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.545+0000 7feac8631640 1 -- 172.21.15.112:0/844077047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac004f500 con 0x7feac00c9250 2024-04-25T01:26:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.545+0000 7feab2ffd640 1 -- 172.21.15.112:0/844077047 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feab4011d10 con 0x7feac00c9250 2024-04-25T01:26:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.545+0000 7feab2ffd640 1 --2- 172.21.15.112:0/844077047 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feaa406be30 0x7feaa406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.545+0000 7feab2ffd640 1 -- 172.21.15.112:0/844077047 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7feab408af80 con 0x7feac00c9250 2024-04-25T01:26:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.545+0000 7feac5bcc640 1 --2- 172.21.15.112:0/844077047 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feaa406be30 0x7feaa406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.546+0000 7feac5bcc640 1 --2- 172.21.15.112:0/844077047 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feaa406be30 0x7feaa406e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feac0159ff0 tx=0x7feabc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:27.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.550+0000 7feab2ffd640 1 -- 172.21.15.112:0/844077047 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feab4059690 con 0x7feac00c9250 2024-04-25T01:26:27.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.726+0000 7feac8631640 1 -- 172.21.15.112:0/844077047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feac006d970 con 0x7feac00c9250 2024-04-25T01:26:27.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.726+0000 7feab2ffd640 1 -- 172.21.15.112:0/844077047 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7feab4058f90 con 0x7feac00c9250 2024-04-25T01:26:27.705 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T01:26:27.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.728+0000 7feac8631640 1 -- 172.21.15.112:0/844077047 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feaa406be30 msgr2=0x7feaa406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:27.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.728+0000 7feac8631640 1 --2- 172.21.15.112:0/844077047 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feaa406be30 0x7feaa406e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feac0159ff0 tx=0x7feabc008040 comp rx=0 tx=0).stop 2024-04-25T01:26:27.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.728+0000 7feac8631640 1 -- 172.21.15.112:0/844077047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac00c9250 msgr2=0x7feac01eb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:27.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.728+0000 7feac8631640 1 --2- 172.21.15.112:0/844077047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac00c9250 0x7feac01eb430 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feab400c430 tx=0x7feab400c8f0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.728+0000 7feac8631640 1 -- 172.21.15.112:0/844077047 shutdown_connections 2024-04-25T01:26:27.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.728+0000 7feac8631640 1 --2- 172.21.15.112:0/844077047 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feaa406be30 0x7feaa406e2f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.728+0000 7feac8631640 1 --2- 172.21.15.112:0/844077047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feac00ca8a0 0x7feac01ef170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.728+0000 7feac8631640 1 --2- 172.21.15.112:0/844077047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feac00ca370 0x7feac0150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.728+0000 7feac8631640 1 --2- 172.21.15.112:0/844077047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac00c9250 0x7feac01eb430 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.728+0000 7feac8631640 1 -- 172.21.15.112:0/844077047 >> 172.21.15.112:0/844077047 conn(0x7feac00cb310 msgr2=0x7feac00c59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:27.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.728+0000 7feac8631640 1 -- 172.21.15.112:0/844077047 shutdown_connections 2024-04-25T01:26:27.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.728+0000 7feac8631640 1 -- 172.21.15.112:0/844077047 wait complete. 2024-04-25T01:26:27.718 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-04-25T01:26:27.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.838+0000 7f5d87a3f640 1 Processor -- start 2024-04-25T01:26:27.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.839+0000 7f5d87a3f640 1 -- start start 2024-04-25T01:26:27.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.839+0000 7f5d87a3f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d800c9250 0x7f5d800c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:27.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.839+0000 7f5d87a3f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5d800ca370 0x7f5d8006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:27.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.839+0000 7f5d87a3f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d801534a0 con 0x7f5d800ca8a0 2024-04-25T01:26:27.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.839+0000 7f5d87a3f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5d80156990 con 0x7f5d800ca370 2024-04-25T01:26:27.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.839+0000 7f5d87a3f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5d80156500 con 0x7f5d800c9250 2024-04-25T01:26:27.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.840+0000 7f5d857db640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d800c9250 0x7f5d800c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:27.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.840+0000 7f5d857db640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d800c9250 0x7f5d800c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:53338/0 (socket says 172.21.15.112:53338) 2024-04-25T01:26:27.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.840+0000 7f5d84fda640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5d800ca370 0x7f5d8006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:27.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.840+0000 7f5d857db640 1 -- 172.21.15.112:0/1281243629 learned_addr learned my addr 172.21.15.112:0/1281243629 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:27.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.840+0000 7f5d857db640 1 -- 172.21.15.112:0/1281243629 >> v2:172.21.15.187:3300/0 conn(0x7f5d800ca370 msgr2=0x7f5d8006bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:27.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.840+0000 7f5d857db640 1 --2- 172.21.15.112:0/1281243629 >> v2:172.21.15.187:3300/0 conn(0x7f5d800ca370 0x7f5d8006bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.840+0000 7f5d857db640 1 -- 172.21.15.112:0/1281243629 >> v1:172.21.15.187:6789/0 conn(0x7f5d800ca8a0 legacy=0x7f5d8006c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:27.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.840+0000 7f5d857db640 1 -- 172.21.15.112:0/1281243629 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d800cae30 con 0x7f5d800c9250 2024-04-25T01:26:27.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.840+0000 7f5d84fda640 1 --2- 172.21.15.112:0/1281243629 >> v2:172.21.15.187:3300/0 conn(0x7f5d800ca370 0x7f5d8006bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:26:27.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.841+0000 7f5d857db640 1 --2- 172.21.15.112:0/1281243629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d800c9250 0x7f5d800c9630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009870 tx=0x7f5d7c02a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d18c8db0ff25ef99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:27.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.841+0000 7f5d77fff640 1 -- 172.21.15.112:0/1281243629 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c037070 con 0x7f5d800c9250 2024-04-25T01:26:27.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.841+0000 7f5d77fff640 1 -- 172.21.15.112:0/1281243629 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d7c032440 con 0x7f5d800c9250 2024-04-25T01:26:27.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.841+0000 7f5d77fff640 1 -- 172.21.15.112:0/1281243629 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c032810 con 0x7f5d800c9250 2024-04-25T01:26:27.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.841+0000 7f5d87a3f640 1 -- 172.21.15.112:0/1281243629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d800c9250 msgr2=0x7f5d800c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:27.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.841+0000 7f5d87a3f640 1 --2- 172.21.15.112:0/1281243629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d800c9250 0x7f5d800c9630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009870 tx=0x7f5d7c02a130 comp rx=0 tx=0).stop 2024-04-25T01:26:27.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.841+0000 7f5d87a3f640 1 -- 172.21.15.112:0/1281243629 shutdown_connections 2024-04-25T01:26:27.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.841+0000 7f5d87a3f640 1 --2- 172.21.15.112:0/1281243629 >> v2:172.21.15.187:3300/0 conn(0x7f5d800ca370 0x7f5d8006bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.841+0000 7f5d87a3f640 1 --2- 172.21.15.112:0/1281243629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d800c9250 0x7f5d800c9630 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.841+0000 7f5d87a3f640 1 -- 172.21.15.112:0/1281243629 >> 172.21.15.112:0/1281243629 conn(0x7f5d800cb310 msgr2=0x7f5d800cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:27.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.842+0000 7f5d87a3f640 1 -- 172.21.15.112:0/1281243629 shutdown_connections 2024-04-25T01:26:27.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.842+0000 7f5d87a3f640 1 -- 172.21.15.112:0/1281243629 wait complete. 2024-04-25T01:26:27.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.843+0000 7f5d87a3f640 1 Processor -- start 2024-04-25T01:26:27.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.843+0000 7f5d87a3f640 1 -- start start 2024-04-25T01:26:27.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.843+0000 7f5d87a3f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d800c9250 0x7f5d800c2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:27.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.843+0000 7f5d87a3f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d800ca370 0x7f5d800c3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:27.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.843+0000 7f5d87a3f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d800ca8a0 0x7f5d801477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:27.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.843+0000 7f5d87a3f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d80155eb0 con 0x7f5d800ca8a0 2024-04-25T01:26:27.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.843+0000 7f5d87a3f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5d80155bd0 con 0x7f5d800c9250 2024-04-25T01:26:27.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.843+0000 7f5d87a3f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5d80156190 con 0x7f5d800ca370 2024-04-25T01:26:27.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.843+0000 7f5d85fdc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d800ca8a0 0x7f5d801477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:27.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.843+0000 7f5d85fdc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d800ca8a0 0x7f5d801477e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58512/0 (socket says 172.21.15.112:58512) 2024-04-25T01:26:27.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.843+0000 7f5d85fdc640 1 -- 172.21.15.112:0/3938619558 learned_addr learned my addr 172.21.15.112:0/3938619558 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:27.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.843+0000 7f5d84fda640 1 --2- 172.21.15.112:0/3938619558 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d800ca370 0x7f5d800c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:27.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.844+0000 7f5d857db640 1 --2- 172.21.15.112:0/3938619558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d800c9250 0x7f5d800c2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:27.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.844+0000 7f5d85fdc640 1 -- 172.21.15.112:0/3938619558 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d800ca370 msgr2=0x7f5d800c3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:27.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.844+0000 7f5d85fdc640 1 --2- 172.21.15.112:0/3938619558 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d800ca370 0x7f5d800c3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.844+0000 7f5d85fdc640 1 -- 172.21.15.112:0/3938619558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d800c9250 msgr2=0x7f5d800c2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:27.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.844+0000 7f5d85fdc640 1 --2- 172.21.15.112:0/3938619558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d800c9250 0x7f5d800c2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:27.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.844+0000 7f5d85fdc640 1 -- 172.21.15.112:0/3938619558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d800c1f90 con 0x7f5d800ca8a0 2024-04-25T01:26:27.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.844+0000 7f5d85fdc640 1 --2- 172.21.15.112:0/3938619558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d800ca8a0 0x7f5d801477e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800c6b0 tx=0x7f5d7800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:27.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.844+0000 7f5d75ffb640 1 -- 172.21.15.112:0/3938619558 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78003b70 con 0x7f5d800ca8a0 2024-04-25T01:26:27.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.844+0000 7f5d75ffb640 1 -- 172.21.15.112:0/3938619558 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d78003d10 con 0x7f5d800ca8a0 2024-04-25T01:26:27.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.844+0000 7f5d87a3f640 1 -- 172.21.15.112:0/3938619558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d801483b0 con 0x7f5d800ca8a0 2024-04-25T01:26:27.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.844+0000 7f5d75ffb640 1 -- 172.21.15.112:0/3938619558 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78005310 con 0x7f5d800ca8a0 2024-04-25T01:26:27.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.844+0000 7f5d87a3f640 1 -- 172.21.15.112:0/3938619558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d801ec2d0 con 0x7f5d800ca8a0 2024-04-25T01:26:27.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.845+0000 7f5d87a3f640 1 -- 172.21.15.112:0/3938619558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d8004f500 con 0x7f5d800ca8a0 2024-04-25T01:26:27.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.845+0000 7f5d75ffb640 1 -- 172.21.15.112:0/3938619558 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d780054b0 con 0x7f5d800ca8a0 2024-04-25T01:26:27.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.846+0000 7f5d75ffb640 1 --2- 172.21.15.112:0/3938619558 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5d6406be30 0x7f5d6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:27.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.846+0000 7f5d857db640 1 --2- 172.21.15.112:0/3938619558 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5d6406be30 0x7f5d6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:27.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.846+0000 7f5d75ffb640 1 -- 172.21.15.112:0/3938619558 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f5d7808add0 con 0x7f5d800ca8a0 2024-04-25T01:26:27.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.846+0000 7f5d857db640 1 --2- 172.21.15.112:0/3938619558 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5d6406be30 0x7f5d6406e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c034000 tx=0x7f5d7c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:27.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:27.850+0000 7f5d75ffb640 1 -- 172.21.15.112:0/3938619558 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5d78058e90 con 0x7f5d800ca8a0 2024-04-25T01:26:28.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.022+0000 7f5d87a3f640 1 -- 172.21.15.112:0/3938619558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) -- 0x7f5d8006d970 con 0x7f5d800ca8a0 2024-04-25T01:26:28.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.023+0000 7f5d75ffb640 1 -- 172.21.15.112:0/3938619558 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7f5d7808c0a0 con 0x7f5d800ca8a0 2024-04-25T01:26:28.002 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T01:26:28.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.025+0000 7f5d87a3f640 1 -- 172.21.15.112:0/3938619558 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5d6406be30 msgr2=0x7f5d6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.025+0000 7f5d87a3f640 1 --2- 172.21.15.112:0/3938619558 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5d6406be30 0x7f5d6406e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c034000 tx=0x7f5d7c02e000 comp rx=0 tx=0).stop 2024-04-25T01:26:28.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.025+0000 7f5d87a3f640 1 -- 172.21.15.112:0/3938619558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d800ca8a0 msgr2=0x7f5d801477e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.025+0000 7f5d87a3f640 1 --2- 172.21.15.112:0/3938619558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d800ca8a0 0x7f5d801477e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800c6b0 tx=0x7f5d7800cb70 comp rx=0 tx=0).stop 2024-04-25T01:26:28.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.025+0000 7f5d87a3f640 1 -- 172.21.15.112:0/3938619558 shutdown_connections 2024-04-25T01:26:28.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.025+0000 7f5d87a3f640 1 --2- 172.21.15.112:0/3938619558 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5d6406be30 0x7f5d6406e2f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.025+0000 7f5d87a3f640 1 --2- 172.21.15.112:0/3938619558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d800ca8a0 0x7f5d801477e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.025+0000 7f5d87a3f640 1 --2- 172.21.15.112:0/3938619558 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d800ca370 0x7f5d800c3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.025+0000 7f5d87a3f640 1 --2- 172.21.15.112:0/3938619558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d800c9250 0x7f5d800c2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.025+0000 7f5d87a3f640 1 -- 172.21.15.112:0/3938619558 >> 172.21.15.112:0/3938619558 conn(0x7f5d800cb310 msgr2=0x7f5d800aabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:28.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.025+0000 7f5d87a3f640 1 -- 172.21.15.112:0/3938619558 shutdown_connections 2024-04-25T01:26:28.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.025+0000 7f5d87a3f640 1 -- 172.21.15.112:0/3938619558 wait complete. 2024-04-25T01:26:28.015 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-04-25T01:26:28.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.135+0000 7f3112f89640 1 Processor -- start 2024-04-25T01:26:28.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.135+0000 7f3112f89640 1 -- start start 2024-04-25T01:26:28.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.136+0000 7f3112f89640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f310c06ebb0 0x7f310c071200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.136+0000 7f3112f89640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f310c1559c0 0x7f310c155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.136+0000 7f3112f89640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f310c158fe0 con 0x7f310c06cf50 2024-04-25T01:26:28.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.136+0000 7f3112f89640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f310c158d90 con 0x7f310c1559c0 2024-04-25T01:26:28.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.136+0000 7f3112f89640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f310c159bf0 con 0x7f310c06ebb0 2024-04-25T01:26:28.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.136+0000 7f3111526640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f310c1559c0 0x7f310c155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:28.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.136+0000 7f3111526640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f310c1559c0 0x7f310c155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58526/0 (socket says 172.21.15.112:58526) 2024-04-25T01:26:28.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.136+0000 7f3111526640 1 -- 172.21.15.112:0/1570607889 learned_addr learned my addr 172.21.15.112:0/1570607889 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:28.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.136+0000 7f3110d25640 1 --2- 172.21.15.112:0/1570607889 >> v2:172.21.15.187:3300/0 conn(0x7f310c06ebb0 0x7f310c071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:28.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.136+0000 7f3111526640 1 -- 172.21.15.112:0/1570607889 >> v2:172.21.15.187:3300/0 conn(0x7f310c06ebb0 msgr2=0x7f310c071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:28.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.136+0000 7f3111526640 1 --2- 172.21.15.112:0/1570607889 >> v2:172.21.15.187:3300/0 conn(0x7f310c06ebb0 0x7f310c071200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.136+0000 7f3111526640 1 -- 172.21.15.112:0/1570607889 >> v1:172.21.15.187:6789/0 conn(0x7f310c06cf50 legacy=0x7f310c13dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:28.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.136+0000 7f3111526640 1 -- 172.21.15.112:0/1570607889 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310c1592c0 con 0x7f310c1559c0 2024-04-25T01:26:28.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.136+0000 7f3110d25640 1 --2- 172.21.15.112:0/1570607889 >> v2:172.21.15.187:3300/0 conn(0x7f310c06ebb0 0x7f310c071200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:28.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.137+0000 7f3111526640 1 --2- 172.21.15.112:0/1570607889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f310c1559c0 0x7f310c155df0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f310c068020 tx=0x7f310402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75b4d0098ce3c80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:28.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.137+0000 7f31037fe640 1 -- 172.21.15.112:0/1570607889 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3104036070 con 0x7f310c1559c0 2024-04-25T01:26:28.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.137+0000 7f31037fe640 1 -- 172.21.15.112:0/1570607889 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3104031600 con 0x7f310c1559c0 2024-04-25T01:26:28.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.137+0000 7f31037fe640 1 -- 172.21.15.112:0/1570607889 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31040319d0 con 0x7f310c1559c0 2024-04-25T01:26:28.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.137+0000 7f3112f89640 1 -- 172.21.15.112:0/1570607889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f310c1559c0 msgr2=0x7f310c155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.137+0000 7f3112f89640 1 --2- 172.21.15.112:0/1570607889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f310c1559c0 0x7f310c155df0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f310c068020 tx=0x7f310402a660 comp rx=0 tx=0).stop 2024-04-25T01:26:28.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.137+0000 7f3112f89640 1 -- 172.21.15.112:0/1570607889 shutdown_connections 2024-04-25T01:26:28.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.137+0000 7f3112f89640 1 --2- 172.21.15.112:0/1570607889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f310c1559c0 0x7f310c155df0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.137+0000 7f3112f89640 1 --2- 172.21.15.112:0/1570607889 >> v2:172.21.15.187:3300/0 conn(0x7f310c06ebb0 0x7f310c071200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.137+0000 7f3112f89640 1 -- 172.21.15.112:0/1570607889 >> 172.21.15.112:0/1570607889 conn(0x7f310c0c3a10 msgr2=0x7f310c0c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:28.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.138+0000 7f3112f89640 1 -- 172.21.15.112:0/1570607889 shutdown_connections 2024-04-25T01:26:28.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.138+0000 7f3112f89640 1 -- 172.21.15.112:0/1570607889 wait complete. 2024-04-25T01:26:28.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.139+0000 7f3112f89640 1 Processor -- start 2024-04-25T01:26:28.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.139+0000 7f3112f89640 1 -- start start 2024-04-25T01:26:28.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.139+0000 7f3112f89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f310c06cf50 0x7f310c1570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.139+0000 7f3112f89640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f310c06ebb0 0x7f310c1575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.139+0000 7f3112f89640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f310c1559c0 0x7f310c13c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.139+0000 7f3112f89640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f310c13e380 con 0x7f310c06cf50 2024-04-25T01:26:28.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.139+0000 7f3112f89640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f310c159880 con 0x7f310c1559c0 2024-04-25T01:26:28.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.139+0000 7f3112f89640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f310c13e500 con 0x7f310c06ebb0 2024-04-25T01:26:28.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.139+0000 7f3110d25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f310c06cf50 0x7f310c1570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:28.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.139+0000 7f3110d25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f310c06cf50 0x7f310c1570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58530/0 (socket says 172.21.15.112:58530) 2024-04-25T01:26:28.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.140+0000 7f3110d25640 1 -- 172.21.15.112:0/3122125033 learned_addr learned my addr 172.21.15.112:0/3122125033 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:28.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.140+0000 7f3110d25640 1 -- 172.21.15.112:0/3122125033 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f310c06ebb0 msgr2=0x7f310c1575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.140+0000 7f3111526640 1 --2- 172.21.15.112:0/3122125033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f310c1559c0 0x7f310c13c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:28.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.140+0000 7f3103fff640 1 --2- 172.21.15.112:0/3122125033 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f310c06ebb0 0x7f310c1575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:28.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.140+0000 7f3110d25640 1 --2- 172.21.15.112:0/3122125033 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f310c06ebb0 0x7f310c1575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.140+0000 7f3110d25640 1 -- 172.21.15.112:0/3122125033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f310c1559c0 msgr2=0x7f310c13c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.140+0000 7f3110d25640 1 --2- 172.21.15.112:0/3122125033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f310c1559c0 0x7f310c13c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.140+0000 7f3110d25640 1 -- 172.21.15.112:0/3122125033 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310c1eb810 con 0x7f310c06cf50 2024-04-25T01:26:28.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.140+0000 7f3111526640 1 --2- 172.21.15.112:0/3122125033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f310c1559c0 0x7f310c13c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:28.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.140+0000 7f3103fff640 1 --2- 172.21.15.112:0/3122125033 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f310c06ebb0 0x7f310c1575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:28.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.140+0000 7f3110d25640 1 --2- 172.21.15.112:0/3122125033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f310c06cf50 0x7f310c1570a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f310800c7f0 tx=0x7f310800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:28.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.140+0000 7f31017fa640 1 -- 172.21.15.112:0/3122125033 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108015070 con 0x7f310c06cf50 2024-04-25T01:26:28.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.140+0000 7f31017fa640 1 -- 172.21.15.112:0/3122125033 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3108007a30 con 0x7f310c06cf50 2024-04-25T01:26:28.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.140+0000 7f3112f89640 1 -- 172.21.15.112:0/3122125033 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f310c0c4a70 con 0x7f310c06cf50 2024-04-25T01:26:28.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.140+0000 7f31017fa640 1 -- 172.21.15.112:0/3122125033 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108007e00 con 0x7f310c06cf50 2024-04-25T01:26:28.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.140+0000 7f3112f89640 1 -- 172.21.15.112:0/3122125033 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f310c13bf30 con 0x7f310c06cf50 2024-04-25T01:26:28.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.141+0000 7f3112f89640 1 -- 172.21.15.112:0/3122125033 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f310c0c1590 con 0x7f310c06cf50 2024-04-25T01:26:28.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.142+0000 7f31017fa640 1 -- 172.21.15.112:0/3122125033 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3108018070 con 0x7f310c06cf50 2024-04-25T01:26:28.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.142+0000 7f31017fa640 1 --2- 172.21.15.112:0/3122125033 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f30f806be60 0x7f30f806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.142+0000 7f3103fff640 1 --2- 172.21.15.112:0/3122125033 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f30f806be60 0x7f30f806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:28.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.142+0000 7f31017fa640 1 -- 172.21.15.112:0/3122125033 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f310808b080 con 0x7f310c06cf50 2024-04-25T01:26:28.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.142+0000 7f3103fff640 1 --2- 172.21.15.112:0/3122125033 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f30f806be60 0x7f30f806e320 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f310c138bd0 tx=0x7f30fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:28.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.146+0000 7f31017fa640 1 -- 172.21.15.112:0/3122125033 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3108059790 con 0x7f310c06cf50 2024-04-25T01:26:28.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.318+0000 7f3112f89640 1 -- 172.21.15.112:0/3122125033 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) -- 0x7f310c04f500 con 0x7f310c06cf50 2024-04-25T01:26:28.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.319+0000 7f31017fa640 1 -- 172.21.15.112:0/3122125033 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7f3108059090 con 0x7f310c06cf50 2024-04-25T01:26:28.298 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T01:26:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.320+0000 7f3112f89640 1 -- 172.21.15.112:0/3122125033 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f30f806be60 msgr2=0x7f30f806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.320+0000 7f3112f89640 1 --2- 172.21.15.112:0/3122125033 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f30f806be60 0x7f30f806e320 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f310c138bd0 tx=0x7f30fc009040 comp rx=0 tx=0).stop 2024-04-25T01:26:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.320+0000 7f3112f89640 1 -- 172.21.15.112:0/3122125033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f310c06cf50 msgr2=0x7f310c1570a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.320+0000 7f3112f89640 1 --2- 172.21.15.112:0/3122125033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f310c06cf50 0x7f310c1570a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f310800c7f0 tx=0x7f310800cbf0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.320+0000 7f3112f89640 1 -- 172.21.15.112:0/3122125033 shutdown_connections 2024-04-25T01:26:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.320+0000 7f3112f89640 1 --2- 172.21.15.112:0/3122125033 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f30f806be60 0x7f30f806e320 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.320+0000 7f3112f89640 1 --2- 172.21.15.112:0/3122125033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f310c1559c0 0x7f310c13c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.320+0000 7f3112f89640 1 --2- 172.21.15.112:0/3122125033 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f310c06ebb0 0x7f310c1575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.320+0000 7f3112f89640 1 --2- 172.21.15.112:0/3122125033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f310c06cf50 0x7f310c1570a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.320+0000 7f3112f89640 1 -- 172.21.15.112:0/3122125033 >> 172.21.15.112:0/3122125033 conn(0x7f310c0c3a10 msgr2=0x7f310c0c0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.320+0000 7f3112f89640 1 -- 172.21.15.112:0/3122125033 shutdown_connections 2024-04-25T01:26:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.320+0000 7f3112f89640 1 -- 172.21.15.112:0/3122125033 wait complete. 2024-04-25T01:26:28.310 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-04-25T01:26:28.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.431+0000 7f379e68c640 1 Processor -- start 2024-04-25T01:26:28.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.432+0000 7f379e68c640 1 -- start start 2024-04-25T01:26:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.432+0000 7f379e68c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f37980c9250 0x7f37980c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.432+0000 7f379e68c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f37980ca8a0 0x7f379806c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.432+0000 7f379e68c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37981534a0 con 0x7f37980ca370 2024-04-25T01:26:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.432+0000 7f379e68c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f3798156990 con 0x7f37980c9250 2024-04-25T01:26:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.432+0000 7f379e68c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3798156500 con 0x7f37980ca8a0 2024-04-25T01:26:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.432+0000 7f3797fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f37980c9250 0x7f37980c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.432+0000 7f3797fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f37980c9250 0x7f37980c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58532/0 (socket says 172.21.15.112:58532) 2024-04-25T01:26:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.432+0000 7f37977fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f37980ca370 0x7f379806bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:45228/0 (socket says 172.21.15.112:45228) 2024-04-25T01:26:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.432+0000 7f3797fff640 1 -- 172.21.15.112:0/2314878666 learned_addr learned my addr 172.21.15.112:0/2314878666 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.433+0000 7f379cc29640 1 --2- 172.21.15.112:0/2314878666 >> v2:172.21.15.187:3300/0 conn(0x7f37980ca8a0 0x7f379806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.433+0000 7f3797fff640 1 -- 172.21.15.112:0/2314878666 >> v2:172.21.15.187:3300/0 conn(0x7f37980ca8a0 msgr2=0x7f379806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:28.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.433+0000 7f3797fff640 1 --2- 172.21.15.112:0/2314878666 >> v2:172.21.15.187:3300/0 conn(0x7f37980ca8a0 0x7f379806c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.433+0000 7f3797fff640 1 -- 172.21.15.112:0/2314878666 >> v1:172.21.15.112:6789/0 conn(0x7f37980ca370 legacy=0x7f379806bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.433+0000 7f3796ffd640 1 -- 172.21.15.112:0/2314878666 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 941060885 0 0) 0x7f37981534a0 con 0x7f37980ca370 2024-04-25T01:26:28.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.433+0000 7f3797fff640 1 -- 172.21.15.112:0/2314878666 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37980cae30 con 0x7f37980c9250 2024-04-25T01:26:28.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.433+0000 7f379cc29640 1 --2- 172.21.15.112:0/2314878666 >> v2:172.21.15.187:3300/0 conn(0x7f37980ca8a0 0x7f379806c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:28.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.433+0000 7f3797fff640 1 --2- 172.21.15.112:0/2314878666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37980c9250 0x7f37980c9630 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f379806dc20 tx=0x7f378802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed4c2cd89339114 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:28.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.434+0000 7f3796ffd640 1 -- 172.21.15.112:0/2314878666 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788037070 con 0x7f37980c9250 2024-04-25T01:26:28.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.434+0000 7f3796ffd640 1 -- 172.21.15.112:0/2314878666 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3788032440 con 0x7f37980c9250 2024-04-25T01:26:28.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.434+0000 7f3796ffd640 1 -- 172.21.15.112:0/2314878666 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788032810 con 0x7f37980c9250 2024-04-25T01:26:28.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.434+0000 7f379e68c640 1 -- 172.21.15.112:0/2314878666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37980c9250 msgr2=0x7f37980c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.434+0000 7f379e68c640 1 --2- 172.21.15.112:0/2314878666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37980c9250 0x7f37980c9630 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f379806dc20 tx=0x7f378802a1e0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.434+0000 7f379e68c640 1 -- 172.21.15.112:0/2314878666 shutdown_connections 2024-04-25T01:26:28.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.434+0000 7f379e68c640 1 --2- 172.21.15.112:0/2314878666 >> v2:172.21.15.187:3300/0 conn(0x7f37980ca8a0 0x7f379806c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.434+0000 7f379e68c640 1 --2- 172.21.15.112:0/2314878666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37980c9250 0x7f37980c9630 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.434+0000 7f379e68c640 1 -- 172.21.15.112:0/2314878666 >> 172.21.15.112:0/2314878666 conn(0x7f37980cb310 msgr2=0x7f37980cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:28.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.434+0000 7f379e68c640 1 -- 172.21.15.112:0/2314878666 shutdown_connections 2024-04-25T01:26:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.434+0000 7f379e68c640 1 -- 172.21.15.112:0/2314878666 wait complete. 2024-04-25T01:26:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.435+0000 7f379e68c640 1 Processor -- start 2024-04-25T01:26:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.435+0000 7f379e68c640 1 -- start start 2024-04-25T01:26:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.436+0000 7f379e68c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37980c9250 0x7f37980c05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.436+0000 7f379e68c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37980ca370 0x7f37981eb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.436+0000 7f379e68c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37980ca8a0 0x7f37980c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.436+0000 7f379e68c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3798155eb0 con 0x7f37980ca370 2024-04-25T01:26:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.436+0000 7f379e68c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3798155bd0 con 0x7f37980c9250 2024-04-25T01:26:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.436+0000 7f379e68c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3798156190 con 0x7f37980ca8a0 2024-04-25T01:26:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.436+0000 7f37977fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37980ca370 0x7f37981eb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.436+0000 7f37977fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37980ca370 0x7f37981eb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58536/0 (socket says 172.21.15.112:58536) 2024-04-25T01:26:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.436+0000 7f37977fe640 1 -- 172.21.15.112:0/4291794918 learned_addr learned my addr 172.21.15.112:0/4291794918 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.436+0000 7f37977fe640 1 -- 172.21.15.112:0/4291794918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37980ca8a0 msgr2=0x7f37980c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.436+0000 7f379cc29640 1 --2- 172.21.15.112:0/4291794918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37980ca8a0 0x7f37980c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.436+0000 7f37977fe640 1 --2- 172.21.15.112:0/4291794918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37980ca8a0 0x7f37980c4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.436+0000 7f37977fe640 1 -- 172.21.15.112:0/4291794918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37980c9250 msgr2=0x7f37980c05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.436+0000 7f3797fff640 1 --2- 172.21.15.112:0/4291794918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37980c9250 0x7f37980c05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.436+0000 7f37977fe640 1 --2- 172.21.15.112:0/4291794918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37980c9250 0x7f37980c05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.436+0000 7f37977fe640 1 -- 172.21.15.112:0/4291794918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37981ee430 con 0x7f37980ca370 2024-04-25T01:26:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.437+0000 7f37977fe640 1 --2- 172.21.15.112:0/4291794918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37980ca370 0x7f37981eb7f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f378c00c4d0 tx=0x7f378c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.437+0000 7f3794ff9640 1 -- 172.21.15.112:0/4291794918 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c018070 con 0x7f37980ca370 2024-04-25T01:26:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.437+0000 7f3794ff9640 1 -- 172.21.15.112:0/4291794918 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f378c007500 con 0x7f37980ca370 2024-04-25T01:26:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.437+0000 7f379e68c640 1 -- 172.21.15.112:0/4291794918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37981ebd30 con 0x7f37980ca370 2024-04-25T01:26:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.437+0000 7f3794ff9640 1 -- 172.21.15.112:0/4291794918 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c004a20 con 0x7f37980ca370 2024-04-25T01:26:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.437+0000 7f379e68c640 1 -- 172.21.15.112:0/4291794918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37980aeb20 con 0x7f37980ca370 2024-04-25T01:26:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.438+0000 7f379e68c640 1 -- 172.21.15.112:0/4291794918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f379804f500 con 0x7f37980ca370 2024-04-25T01:26:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.438+0000 7f3794ff9640 1 -- 172.21.15.112:0/4291794918 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f378c01b070 con 0x7f37980ca370 2024-04-25T01:26:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.439+0000 7f3794ff9640 1 --2- 172.21.15.112:0/4291794918 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f378406be60 0x7f378406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.439+0000 7f3794ff9640 1 -- 172.21.15.112:0/4291794918 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f378c08a3a0 con 0x7f37980ca370 2024-04-25T01:26:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.439+0000 7f3797fff640 1 --2- 172.21.15.112:0/4291794918 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f378406be60 0x7f378406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.439+0000 7f3797fff640 1 --2- 172.21.15.112:0/4291794918 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f378406be60 0x7f378406e320 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3788034000 tx=0x7f378802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:28.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.443+0000 7f3794ff9640 1 -- 172.21.15.112:0/4291794918 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f378c058ab0 con 0x7f37980ca370 2024-04-25T01:26:28.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.618+0000 7f379e68c640 1 -- 172.21.15.112:0/4291794918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) -- 0x7f379806d970 con 0x7f37980ca370 2024-04-25T01:26:28.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.618+0000 7f3794ff9640 1 -- 172.21.15.112:0/4291794918 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7f378c0583b0 con 0x7f37980ca370 2024-04-25T01:26:28.597 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T01:26:28.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.620+0000 7f379e68c640 1 -- 172.21.15.112:0/4291794918 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f378406be60 msgr2=0x7f378406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.620+0000 7f379e68c640 1 --2- 172.21.15.112:0/4291794918 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f378406be60 0x7f378406e320 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3788034000 tx=0x7f378802e000 comp rx=0 tx=0).stop 2024-04-25T01:26:28.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.620+0000 7f379e68c640 1 -- 172.21.15.112:0/4291794918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37980ca370 msgr2=0x7f37981eb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.620+0000 7f379e68c640 1 --2- 172.21.15.112:0/4291794918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37980ca370 0x7f37981eb7f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f378c00c4d0 tx=0x7f378c00c990 comp rx=0 tx=0).stop 2024-04-25T01:26:28.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.620+0000 7f379e68c640 1 -- 172.21.15.112:0/4291794918 shutdown_connections 2024-04-25T01:26:28.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.620+0000 7f379e68c640 1 --2- 172.21.15.112:0/4291794918 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f378406be60 0x7f378406e320 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.620+0000 7f379e68c640 1 --2- 172.21.15.112:0/4291794918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37980ca8a0 0x7f37980c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.620+0000 7f379e68c640 1 --2- 172.21.15.112:0/4291794918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37980ca370 0x7f37981eb7f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.620+0000 7f379e68c640 1 --2- 172.21.15.112:0/4291794918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37980c9250 0x7f37980c05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.620+0000 7f379e68c640 1 -- 172.21.15.112:0/4291794918 >> 172.21.15.112:0/4291794918 conn(0x7f37980cb310 msgr2=0x7f37981552f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:28.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.620+0000 7f379e68c640 1 -- 172.21.15.112:0/4291794918 shutdown_connections 2024-04-25T01:26:28.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.620+0000 7f379e68c640 1 -- 172.21.15.112:0/4291794918 wait complete. 2024-04-25T01:26:28.610 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-04-25T01:26:28.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.730+0000 7fa573161640 1 Processor -- start 2024-04-25T01:26:28.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.730+0000 7fa573161640 1 -- start start 2024-04-25T01:26:28.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.731+0000 7fa573161640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa56c072c20 0x7fa56c0aa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.731+0000 7fa573161640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa56c155d40 con 0x7fa56c0ac830 2024-04-25T01:26:28.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.731+0000 7fa573161640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa56c156020 con 0x7fa56c14f220 2024-04-25T01:26:28.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.731+0000 7fa573161640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa56c071700 con 0x7fa56c072c20 2024-04-25T01:26:28.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.731+0000 7fa56bfff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa56c0ac830 0x7fa56c155290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:45230/0 (socket says 172.21.15.112:45230) 2024-04-25T01:26:28.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.731+0000 7fa56bfff640 1 -- 172.21.15.112:0/3263827714 learned_addr learned my addr 172.21.15.112:0/3263827714 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:28.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.731+0000 7fa5716fe640 1 --2- 172.21.15.112:0/3263827714 >> v2:172.21.15.187:3300/0 conn(0x7fa56c072c20 0x7fa56c0aa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:28.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.731+0000 7fa56b7fe640 1 -- 172.21.15.112:0/3263827714 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2936326752 0 0) 0x7fa56c155d40 con 0x7fa56c0ac830 2024-04-25T01:26:28.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.731+0000 7fa56b7fe640 1 -- 172.21.15.112:0/3263827714 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa554003520 con 0x7fa56c0ac830 2024-04-25T01:26:28.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.731+0000 7fa56b7fe640 1 -- 172.21.15.112:0/3263827714 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 152703102 0 0) 0x7fa554003520 con 0x7fa56c0ac830 2024-04-25T01:26:28.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.731+0000 7fa56b7fe640 1 -- 172.21.15.112:0/3263827714 >> v2:172.21.15.187:3300/0 conn(0x7fa56c072c20 msgr2=0x7fa56c0aa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:28.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.731+0000 7fa56b7fe640 1 --2- 172.21.15.112:0/3263827714 >> v2:172.21.15.187:3300/0 conn(0x7fa56c072c20 0x7fa56c0aa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.731+0000 7fa56b7fe640 1 -- 172.21.15.112:0/3263827714 >> v1:172.21.15.187:6789/0 conn(0x7fa56c14f220 legacy=0x7fa56c072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.731+0000 7fa56b7fe640 1 -- 172.21.15.112:0/3263827714 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56c071950 con 0x7fa56c0ac830 2024-04-25T01:26:28.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.731+0000 7fa56b7fe640 1 -- 172.21.15.112:0/3263827714 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fa560001670 con 0x7fa56c0ac830 2024-04-25T01:26:28.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.732+0000 7fa56b7fe640 1 -- 172.21.15.112:0/3263827714 >> v1:172.21.15.112:6789/0 conn(0x7fa56c0ac830 legacy=0x7fa56c155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.732+0000 7fa56b7fe640 1 --2- 172.21.15.112:0/3263827714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5540039b0 0x7fa554003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.732+0000 7fa56b7fe640 1 --2- 172.21.15.112:0/3263827714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5540043b0 0x7fa554004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.732+0000 7fa56b7fe640 1 --2- 172.21.15.112:0/3263827714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa554005120 0x7fa55400cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.732+0000 7fa56b7fe640 1 -- 172.21.15.112:0/3263827714 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c071700 con 0x7fa554005120 2024-04-25T01:26:28.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.732+0000 7fa56b7fe640 1 -- 172.21.15.112:0/3263827714 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa55400d170 con 0x7fa5540039b0 2024-04-25T01:26:28.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.732+0000 7fa56b7fe640 1 -- 172.21.15.112:0/3263827714 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa55400d370 con 0x7fa5540043b0 2024-04-25T01:26:28.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.732+0000 7fa5716fe640 1 --2- 172.21.15.112:0/3263827714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa554005120 0x7fa55400cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:28.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.732+0000 7fa570efd640 1 --2- 172.21.15.112:0/3263827714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5540039b0 0x7fa554003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:28.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.732+0000 7fa56bfff640 1 --2- 172.21.15.112:0/3263827714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5540043b0 0x7fa554004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:28.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.732+0000 7fa5716fe640 1 -- 172.21.15.112:0/3263827714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5540043b0 msgr2=0x7fa554004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:28.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.732+0000 7fa5716fe640 1 --2- 172.21.15.112:0/3263827714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5540043b0 0x7fa554004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.732+0000 7fa5716fe640 1 -- 172.21.15.112:0/3263827714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5540039b0 msgr2=0x7fa554003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:28.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.732+0000 7fa5716fe640 1 --2- 172.21.15.112:0/3263827714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5540039b0 0x7fa554003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.732+0000 7fa5716fe640 1 -- 172.21.15.112:0/3263827714 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa554003520 con 0x7fa554005120 2024-04-25T01:26:28.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.732+0000 7fa56bfff640 1 --2- 172.21.15.112:0/3263827714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5540043b0 0x7fa554004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:26:28.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.733+0000 7fa5716fe640 1 -- 172.21.15.112:0/3263827714 reap_dead start 2024-04-25T01:26:28.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.733+0000 7fa5716fe640 1 --2- 172.21.15.112:0/3263827714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa554005120 0x7fa55400cad0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa564009f90 tx=0x7fa554005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5b9fcb4b42b7a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:28.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.733+0000 7fa56b7fe640 1 -- 172.21.15.112:0/3263827714 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564009c10 con 0x7fa554005120 2024-04-25T01:26:28.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.733+0000 7fa56b7fe640 1 -- 172.21.15.112:0/3263827714 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa564031440 con 0x7fa554005120 2024-04-25T01:26:28.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.733+0000 7fa56b7fe640 1 -- 172.21.15.112:0/3263827714 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564031810 con 0x7fa554005120 2024-04-25T01:26:28.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.733+0000 7fa573161640 1 -- 172.21.15.112:0/3263827714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa554005120 msgr2=0x7fa55400cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.733+0000 7fa573161640 1 --2- 172.21.15.112:0/3263827714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa554005120 0x7fa55400cad0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa564009f90 tx=0x7fa554005830 comp rx=0 tx=0).stop 2024-04-25T01:26:28.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.733+0000 7fa573161640 1 -- 172.21.15.112:0/3263827714 shutdown_connections 2024-04-25T01:26:28.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.733+0000 7fa573161640 1 --2- 172.21.15.112:0/3263827714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa554005120 0x7fa55400cad0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.733+0000 7fa573161640 1 -- 172.21.15.112:0/3263827714 >> 172.21.15.112:0/3263827714 conn(0x7fa56c0c3a10 msgr2=0x7fa56c0c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:28.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.733+0000 7fa573161640 1 -- 172.21.15.112:0/3263827714 shutdown_connections 2024-04-25T01:26:28.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.733+0000 7fa573161640 1 -- 172.21.15.112:0/3263827714 wait complete. 2024-04-25T01:26:28.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.734+0000 7fa573161640 1 Processor -- start 2024-04-25T01:26:28.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.734+0000 7fa573161640 1 -- start start 2024-04-25T01:26:28.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.735+0000 7fa573161640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa554005120 0x7fa56c0c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.735+0000 7fa573161640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa56c14e540 0x7fa56c153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.735+0000 7fa573161640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56c1e9620 0x7fa56c1ec3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.735+0000 7fa573161640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c0acf80 con 0x7fa56c1e9620 2024-04-25T01:26:28.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.735+0000 7fa573161640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c071cc0 con 0x7fa56c14e540 2024-04-25T01:26:28.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.735+0000 7fa573161640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa56c0ad100 con 0x7fa554005120 2024-04-25T01:26:28.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.735+0000 7fa5716fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56c1e9620 0x7fa56c1ec3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:28.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.735+0000 7fa5716fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56c1e9620 0x7fa56c1ec3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58558/0 (socket says 172.21.15.112:58558) 2024-04-25T01:26:28.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.735+0000 7fa5716fe640 1 -- 172.21.15.112:0/2393134780 learned_addr learned my addr 172.21.15.112:0/2393134780 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:28.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.736+0000 7fa5716fe640 1 -- 172.21.15.112:0/2393134780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa554005120 msgr2=0x7fa56c0c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.736+0000 7fa5716fe640 1 --2- 172.21.15.112:0/2393134780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa554005120 0x7fa56c0c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.736+0000 7fa5716fe640 1 -- 172.21.15.112:0/2393134780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa56c14e540 msgr2=0x7fa56c153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.736+0000 7fa5716fe640 1 --2- 172.21.15.112:0/2393134780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa56c14e540 0x7fa56c153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.736+0000 7fa5716fe640 1 -- 172.21.15.112:0/2393134780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56c1e8e80 con 0x7fa56c1e9620 2024-04-25T01:26:28.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.736+0000 7fa5716fe640 1 --2- 172.21.15.112:0/2393134780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56c1e9620 0x7fa56c1ec3f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa564033040 tx=0x7fa564009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:28.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.736+0000 7fa5697fa640 1 -- 172.21.15.112:0/2393134780 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564009c10 con 0x7fa56c1e9620 2024-04-25T01:26:28.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.736+0000 7fa5697fa640 1 -- 172.21.15.112:0/2393134780 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa564043070 con 0x7fa56c1e9620 2024-04-25T01:26:28.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.736+0000 7fa573161640 1 -- 172.21.15.112:0/2393134780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56c14d540 con 0x7fa56c1e9620 2024-04-25T01:26:28.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.736+0000 7fa5697fa640 1 -- 172.21.15.112:0/2393134780 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56403c8b0 con 0x7fa56c1e9620 2024-04-25T01:26:28.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.736+0000 7fa573161640 1 -- 172.21.15.112:0/2393134780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa56c14d7e0 con 0x7fa56c1e9620 2024-04-25T01:26:28.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.737+0000 7fa573161640 1 -- 172.21.15.112:0/2393134780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa56c04f500 con 0x7fa56c1e9620 2024-04-25T01:26:28.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.737+0000 7fa5697fa640 1 -- 172.21.15.112:0/2393134780 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa564005d10 con 0x7fa56c1e9620 2024-04-25T01:26:28.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.738+0000 7fa5697fa640 1 --2- 172.21.15.112:0/2393134780 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa55806be30 0x7fa55806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.738+0000 7fa570efd640 1 --2- 172.21.15.112:0/2393134780 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa55806be30 0x7fa55806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:28.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.738+0000 7fa5697fa640 1 -- 172.21.15.112:0/2393134780 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fa5640abbc0 con 0x7fa56c1e9620 2024-04-25T01:26:28.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.738+0000 7fa570efd640 1 --2- 172.21.15.112:0/2393134780 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa55806be30 0x7fa55806e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa55c001880 tx=0x7fa55c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:28.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.742+0000 7fa5697fa640 1 -- 172.21.15.112:0/2393134780 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa56c1e8e80 con 0x7fa56c1e9620 2024-04-25T01:26:28.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.916+0000 7fa573161640 1 -- 172.21.15.112:0/2393134780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) -- 0x7fa56c0682a0 con 0x7fa56c1e9620 2024-04-25T01:26:28.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.916+0000 7fa5697fa640 1 -- 172.21.15.112:0/2393134780 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7fa56407a2d0 con 0x7fa56c1e9620 2024-04-25T01:26:28.896 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T01:26:28.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.918+0000 7fa573161640 1 -- 172.21.15.112:0/2393134780 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa55806be30 msgr2=0x7fa55806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.918+0000 7fa573161640 1 --2- 172.21.15.112:0/2393134780 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa55806be30 0x7fa55806e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa55c001880 tx=0x7fa55c009040 comp rx=0 tx=0).stop 2024-04-25T01:26:28.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.918+0000 7fa573161640 1 -- 172.21.15.112:0/2393134780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56c1e9620 msgr2=0x7fa56c1ec3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.918+0000 7fa573161640 1 --2- 172.21.15.112:0/2393134780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56c1e9620 0x7fa56c1ec3f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa564033040 tx=0x7fa564009750 comp rx=0 tx=0).stop 2024-04-25T01:26:28.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.919+0000 7fa573161640 1 -- 172.21.15.112:0/2393134780 shutdown_connections 2024-04-25T01:26:28.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.919+0000 7fa573161640 1 --2- 172.21.15.112:0/2393134780 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa55806be30 0x7fa55806e2f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.919+0000 7fa573161640 1 --2- 172.21.15.112:0/2393134780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56c1e9620 0x7fa56c1ec3f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.919+0000 7fa573161640 1 --2- 172.21.15.112:0/2393134780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa56c14e540 0x7fa56c153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.919+0000 7fa573161640 1 --2- 172.21.15.112:0/2393134780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa554005120 0x7fa56c0c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.919+0000 7fa573161640 1 -- 172.21.15.112:0/2393134780 >> 172.21.15.112:0/2393134780 conn(0x7fa56c0c3a10 msgr2=0x7fa56c0aabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:28.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.919+0000 7fa573161640 1 -- 172.21.15.112:0/2393134780 shutdown_connections 2024-04-25T01:26:28.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:28.919+0000 7fa573161640 1 -- 172.21.15.112:0/2393134780 wait complete. 2024-04-25T01:26:28.909 DEBUG:tasks.mon_thrash:Yielding 2024-04-25T01:26:28.909 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T01:26:28.922 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:26:28.923 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T01:26:28.923 INFO:tasks.workunit:timeout=3h 2024-04-25T01:26:28.923 INFO:tasks.workunit:cleanup=True 2024-04-25T01:26:28.923 DEBUG:teuthology.orchestra.run.smithi112:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T01:26:28.924 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:26:28.939 INFO:teuthology.orchestra.run.smithi112.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T01:26:28.939 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-25T01:26:28.939 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 2ah/42d Inode: 1 Links: 2 2024-04-25T01:26:28.939 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:26:28.940 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-25T01:26:28.940 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-25T01:26:28.940 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 01:25:53.359266949 +0000 2024-04-25T01:26:28.940 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 01:26:24.473919844 +0000 2024-04-25T01:26:28.940 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: - 2024-04-25T01:26:28.940 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T01:26:28.940 DEBUG:teuthology.orchestra.run.smithi112:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T01:26:29.006 DEBUG:teuthology.orchestra.run.smithi187:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-25T01:26:29.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.043+0000 7fd95e909640 1 Processor -- start 2024-04-25T01:26:29.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.043+0000 7fd95e909640 1 -- start start 2024-04-25T01:26:29.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.044+0000 7fd95e909640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd95814c3f0 0x7fd9580c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.044+0000 7fd95e909640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9580ac3b0 0x7fd958155f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.044+0000 7fd95e909640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9581529b0 con 0x7fd9581524a0 2024-04-25T01:26:29.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.044+0000 7fd95e909640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd958155a90 con 0x7fd95814c3f0 2024-04-25T01:26:29.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.044+0000 7fd95e909640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd958152c70 con 0x7fd9580ac3b0 2024-04-25T01:26:29.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.044+0000 7fd9577fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9580ac3b0 0x7fd958155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.044+0000 7fd957fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd95814c3f0 0x7fd9580c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.044+0000 7fd95cea6640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fd9581524a0 0x7fd9580c3b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:39358/0 (socket says 172.21.15.112:39358) 2024-04-25T01:26:29.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.044+0000 7fd95cea6640 1 -- 172.21.15.112:0/2323329203 learned_addr learned my addr 172.21.15.112:0/2323329203 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:29.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.044+0000 7fd9577fe640 1 -- 172.21.15.112:0/2323329203 >> v2:172.21.15.187:3300/0 conn(0x7fd95814c3f0 msgr2=0x7fd9580c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:29.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.044+0000 7fd9577fe640 1 --2- 172.21.15.112:0/2323329203 >> v2:172.21.15.187:3300/0 conn(0x7fd95814c3f0 0x7fd9580c2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.044+0000 7fd9577fe640 1 -- 172.21.15.112:0/2323329203 >> v1:172.21.15.187:6789/0 conn(0x7fd9581524a0 legacy=0x7fd9580c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.044+0000 7fd9577fe640 1 -- 172.21.15.112:0/2323329203 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd958152f50 con 0x7fd9580ac3b0 2024-04-25T01:26:29.026 INFO:teuthology.orchestra.run.smithi187.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-25T01:26:29.026 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-25T01:26:29.026 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 2bh/43d Inode: 1 Links: 2 2024-04-25T01:26:29.026 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:26:29.026 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-25T01:26:29.026 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-25T01:26:29.027 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-25 01:26:02.398054454 +0000 2024-04-25T01:26:29.027 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-25 01:26:24.570969760 +0000 2024-04-25T01:26:29.027 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2024-04-25T01:26:29.027 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-25T01:26:29.027 DEBUG:teuthology.orchestra.run.smithi187:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-25T01:26:29.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.045+0000 7fd9577fe640 1 --2- 172.21.15.112:0/2323329203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9580ac3b0 0x7fd958155f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd94c0097a0 tx=0x7fd94c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ae1e59ffdcef621 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:29.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.045+0000 7fd956ffd640 1 -- 172.21.15.112:0/2323329203 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c037070 con 0x7fd9580ac3b0 2024-04-25T01:26:29.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.045+0000 7fd956ffd640 1 -- 172.21.15.112:0/2323329203 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd94c02ae50 con 0x7fd9580ac3b0 2024-04-25T01:26:29.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.045+0000 7fd956ffd640 1 -- 172.21.15.112:0/2323329203 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c032660 con 0x7fd9580ac3b0 2024-04-25T01:26:29.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.046+0000 7fd95e909640 1 -- 172.21.15.112:0/2323329203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9580ac3b0 msgr2=0x7fd958155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.046+0000 7fd95e909640 1 --2- 172.21.15.112:0/2323329203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9580ac3b0 0x7fd958155f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd94c0097a0 tx=0x7fd94c02a100 comp rx=0 tx=0).stop 2024-04-25T01:26:29.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.046+0000 7fd95e909640 1 -- 172.21.15.112:0/2323329203 shutdown_connections 2024-04-25T01:26:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.046+0000 7fd95e909640 1 --2- 172.21.15.112:0/2323329203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9580ac3b0 0x7fd958155f20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.046+0000 7fd95e909640 1 --2- 172.21.15.112:0/2323329203 >> v2:172.21.15.187:3300/0 conn(0x7fd95814c3f0 0x7fd9580c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.046+0000 7fd95e909640 1 -- 172.21.15.112:0/2323329203 >> 172.21.15.112:0/2323329203 conn(0x7fd95806b160 msgr2=0x7fd95806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.046+0000 7fd95e909640 1 -- 172.21.15.112:0/2323329203 shutdown_connections 2024-04-25T01:26:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.046+0000 7fd95e909640 1 -- 172.21.15.112:0/2323329203 wait complete. 2024-04-25T01:26:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.047+0000 7fd95e909640 1 Processor -- start 2024-04-25T01:26:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.047+0000 7fd95e909640 1 -- start start 2024-04-25T01:26:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.048+0000 7fd95e909640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9580ac3b0 0x7fd9581470a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.048+0000 7fd95e909640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd95814c3f0 0x7fd958149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.048+0000 7fd95e909640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9581524a0 0x7fd95814a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.048+0000 7fd95e909640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd95814c7d0 con 0x7fd9581524a0 2024-04-25T01:26:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.048+0000 7fd95e909640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd958070d60 con 0x7fd95814c3f0 2024-04-25T01:26:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.048+0000 7fd95e909640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd95814c950 con 0x7fd9580ac3b0 2024-04-25T01:26:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.048+0000 7fd957fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9580ac3b0 0x7fd9581470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.048+0000 7fd95cea6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9581524a0 0x7fd95814a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.048+0000 7fd957fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9580ac3b0 0x7fd9581470a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:53412/0 (socket says 172.21.15.112:53412) 2024-04-25T01:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.048+0000 7fd957fff640 1 -- 172.21.15.112:0/3048289147 learned_addr learned my addr 172.21.15.112:0/3048289147 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.048+0000 7fd9577fe640 1 --2- 172.21.15.112:0/3048289147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd95814c3f0 0x7fd958149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.048+0000 7fd957fff640 1 -- 172.21.15.112:0/3048289147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd95814c3f0 msgr2=0x7fd958149c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.048+0000 7fd957fff640 1 --2- 172.21.15.112:0/3048289147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd95814c3f0 0x7fd958149c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.048+0000 7fd957fff640 1 -- 172.21.15.112:0/3048289147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9581524a0 msgr2=0x7fd95814a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.048+0000 7fd957fff640 1 --2- 172.21.15.112:0/3048289147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9581524a0 0x7fd95814a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.048+0000 7fd957fff640 1 -- 172.21.15.112:0/3048289147 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd958141910 con 0x7fd9580ac3b0 2024-04-25T01:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.048+0000 7fd95cea6640 1 --2- 172.21.15.112:0/3048289147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9581524a0 0x7fd95814a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.049+0000 7fd957fff640 1 --2- 172.21.15.112:0/3048289147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9580ac3b0 0x7fd9581470a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd94800e9b0 tx=0x7fd94800ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.049+0000 7fd954ff9640 1 -- 172.21.15.112:0/3048289147 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948017070 con 0x7fd9580ac3b0 2024-04-25T01:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.049+0000 7fd954ff9640 1 -- 172.21.15.112:0/3048289147 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd94800c9a0 con 0x7fd9580ac3b0 2024-04-25T01:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.049+0000 7fd954ff9640 1 -- 172.21.15.112:0/3048289147 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94800cd90 con 0x7fd9580ac3b0 2024-04-25T01:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.049+0000 7fd95e909640 1 -- 172.21.15.112:0/3048289147 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9580c7e90 con 0x7fd9580ac3b0 2024-04-25T01:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.049+0000 7fd95e909640 1 -- 172.21.15.112:0/3048289147 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd958141610 con 0x7fd9580ac3b0 2024-04-25T01:26:29.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.050+0000 7fd95e909640 1 -- 172.21.15.112:0/3048289147 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9580682a0 con 0x7fd9580ac3b0 2024-04-25T01:26:29.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.050+0000 7fd954ff9640 1 -- 172.21.15.112:0/3048289147 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd94801a070 con 0x7fd9580ac3b0 2024-04-25T01:26:29.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.051+0000 7fd954ff9640 1 --2- 172.21.15.112:0/3048289147 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd94406be60 0x7fd94406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.051+0000 7fd954ff9640 1 -- 172.21.15.112:0/3048289147 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fd94808b030 con 0x7fd9580ac3b0 2024-04-25T01:26:29.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.051+0000 7fd9577fe640 1 --2- 172.21.15.112:0/3048289147 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd94406be60 0x7fd94406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.051+0000 7fd9577fe640 1 --2- 172.21.15.112:0/3048289147 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd94406be60 0x7fd94406e320 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd94c034000 tx=0x7fd94c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:29.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.055+0000 7fd954ff9640 1 -- 172.21.15.112:0/3048289147 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd9480591d0 con 0x7fd9580ac3b0 2024-04-25T01:26:29.099 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:26:29.100 DEBUG:teuthology.orchestra.run.smithi187:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:26:29.115 INFO:tasks.workunit.client.0.smithi112.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T01:26:29.158 INFO:tasks.workunit.client.1.smithi187.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-25T01:26:29.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.266+0000 7fd95e909640 1 -- 172.21.15.112:0/3048289147 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9580ac110 con 0x7fd9580ac3b0 2024-04-25T01:26:29.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.267+0000 7fd954ff9640 1 -- 172.21.15.112:0/3048289147 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+6491 (secure 0 0 0) 0x7fd94805d4d0 con 0x7fd9580ac3b0 2024-04-25T01:26:29.246 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:26:29.246 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/4047219598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4047219598},{"type":"v1","addr":"172.21.15.187:6834","nonce":4047219598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2204640907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2204640907},{"type":"v1","addr":"172.21.15.187:6835","nonce":2204640907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:25:52.687260+0000","modified":"2024-04-25T01:25:55.702719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/2115984531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2115984531},{"type":"v1","addr":"172.21.15.112:6837","nonce":2115984531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1},{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:26:01.740521+0000","modified":"2024-04-25T01:26:04.746955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":8,"state":"up:active","state_seq":6,"addr":"172.21.15.112:6835/278686477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":278686477},{"type":"v1","addr":"172.21.15.112:6835","nonce":278686477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":2}]} 2024-04-25T01:26:29.247 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 11 2024-04-25T01:26:29.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.269+0000 7fd95e909640 1 -- 172.21.15.112:0/3048289147 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd94406be60 msgr2=0x7fd94406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.269+0000 7fd95e909640 1 --2- 172.21.15.112:0/3048289147 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd94406be60 0x7fd94406e320 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd94c034000 tx=0x7fd94c02e000 comp rx=0 tx=0).stop 2024-04-25T01:26:29.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.269+0000 7fd95e909640 1 -- 172.21.15.112:0/3048289147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9580ac3b0 msgr2=0x7fd9581470a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.269+0000 7fd95e909640 1 --2- 172.21.15.112:0/3048289147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9580ac3b0 0x7fd9581470a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd94800e9b0 tx=0x7fd94800ee70 comp rx=0 tx=0).stop 2024-04-25T01:26:29.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.269+0000 7fd95e909640 1 -- 172.21.15.112:0/3048289147 shutdown_connections 2024-04-25T01:26:29.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.269+0000 7fd95e909640 1 --2- 172.21.15.112:0/3048289147 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd94406be60 0x7fd94406e320 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.269+0000 7fd95e909640 1 --2- 172.21.15.112:0/3048289147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9581524a0 0x7fd95814a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.269+0000 7fd95e909640 1 --2- 172.21.15.112:0/3048289147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd95814c3f0 0x7fd958149c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.269+0000 7fd95e909640 1 --2- 172.21.15.112:0/3048289147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9580ac3b0 0x7fd9581470a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.269+0000 7fd95e909640 1 -- 172.21.15.112:0/3048289147 >> 172.21.15.112:0/3048289147 conn(0x7fd95806b160 msgr2=0x7fd9580c42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:29.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.269+0000 7fd95e909640 1 -- 172.21.15.112:0/3048289147 shutdown_connections 2024-04-25T01:26:29.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.269+0000 7fd95e909640 1 -- 172.21.15.112:0/3048289147 wait complete. 2024-04-25T01:26:29.259 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2024-04-25T01:26:29.259 INFO:tasks.mon_thrash.mon_thrasher:seed: 1714008387, revive delay: 20.0, thrash delay: 10.0 thrash many: False, maintain quorum: True store thrash: False, probability: 50 freeze mon: prob 10.0 duration 15.0 2024-04-25T01:26:29.259 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:26:29.259 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:26:29.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.417+0000 7fa97be3f640 1 Processor -- start 2024-04-25T01:26:29.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.417+0000 7fa97be3f640 1 -- start start 2024-04-25T01:26:29.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.418+0000 7fa97be3f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa9740c9570 0x7fa9740c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.418+0000 7fa97be3f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa9740c9060 0x7fa974160200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.418+0000 7fa97be3f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9741532d0 con 0x7fa974152ef0 2024-04-25T01:26:29.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.418+0000 7fa97be3f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa974155a70 con 0x7fa9740c9570 2024-04-25T01:26:29.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.418+0000 7fa97be3f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa9740c9ba0 con 0x7fa9740c9060 2024-04-25T01:26:29.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.418+0000 7fa9793da640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa9740c9570 0x7fa9740c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.418+0000 7fa9793da640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa9740c9570 0x7fa9740c5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58572/0 (socket says 172.21.15.112:58572) 2024-04-25T01:26:29.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.418+0000 7fa9793da640 1 -- 172.21.15.112:0/1174847408 learned_addr learned my addr 172.21.15.112:0/1174847408 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:29.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.418+0000 7fa97a3dc640 1 --2- 172.21.15.112:0/1174847408 >> v2:172.21.15.187:3300/0 conn(0x7fa9740c9060 0x7fa974160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.418+0000 7fa9793da640 1 -- 172.21.15.112:0/1174847408 >> v2:172.21.15.187:3300/0 conn(0x7fa9740c9060 msgr2=0x7fa974160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:29.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.418+0000 7fa9793da640 1 --2- 172.21.15.112:0/1174847408 >> v2:172.21.15.187:3300/0 conn(0x7fa9740c9060 0x7fa974160200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.418+0000 7fa9793da640 1 -- 172.21.15.112:0/1174847408 >> v1:172.21.15.187:6789/0 conn(0x7fa974152ef0 legacy=0x7fa9740ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.418+0000 7fa9793da640 1 -- 172.21.15.112:0/1174847408 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9740c9e80 con 0x7fa9740c9570 2024-04-25T01:26:29.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.419+0000 7fa9793da640 1 --2- 172.21.15.112:0/1174847408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9740c9570 0x7fa9740c5ae0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa9740c2090 tx=0x7fa96402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efb48578aa95977 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:29.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.419+0000 7fa978bd9640 1 -- 172.21.15.112:0/1174847408 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964037070 con 0x7fa9740c9570 2024-04-25T01:26:29.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.419+0000 7fa978bd9640 1 -- 172.21.15.112:0/1174847408 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa964032440 con 0x7fa9740c9570 2024-04-25T01:26:29.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.419+0000 7fa978bd9640 1 -- 172.21.15.112:0/1174847408 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964032810 con 0x7fa9740c9570 2024-04-25T01:26:29.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.420+0000 7fa97be3f640 1 -- 172.21.15.112:0/1174847408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9740c9570 msgr2=0x7fa9740c5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.420+0000 7fa97be3f640 1 --2- 172.21.15.112:0/1174847408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9740c9570 0x7fa9740c5ae0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa9740c2090 tx=0x7fa96402a270 comp rx=0 tx=0).stop 2024-04-25T01:26:29.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.420+0000 7fa97be3f640 1 -- 172.21.15.112:0/1174847408 shutdown_connections 2024-04-25T01:26:29.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.420+0000 7fa97be3f640 1 --2- 172.21.15.112:0/1174847408 >> v2:172.21.15.187:3300/0 conn(0x7fa9740c9060 0x7fa974160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.420+0000 7fa97be3f640 1 --2- 172.21.15.112:0/1174847408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9740c9570 0x7fa9740c5ae0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.420+0000 7fa97be3f640 1 -- 172.21.15.112:0/1174847408 >> 172.21.15.112:0/1174847408 conn(0x7fa9740cb160 msgr2=0x7fa9740cb570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:29.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.420+0000 7fa97be3f640 1 -- 172.21.15.112:0/1174847408 shutdown_connections 2024-04-25T01:26:29.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.420+0000 7fa97be3f640 1 -- 172.21.15.112:0/1174847408 wait complete. 2024-04-25T01:26:29.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.421+0000 7fa97be3f640 1 Processor -- start 2024-04-25T01:26:29.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.421+0000 7fa97be3f640 1 -- start start 2024-04-25T01:26:29.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa97be3f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9740c9060 0x7fa9741597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa97be3f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9740c9570 0x7fa974155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa97be3f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa974152ef0 0x7fa9740c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa97be3f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa97406bf20 con 0x7fa974152ef0 2024-04-25T01:26:29.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa97be3f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa9740be9a0 con 0x7fa9740c9570 2024-04-25T01:26:29.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa97be3f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa97406c0a0 con 0x7fa9740c9060 2024-04-25T01:26:29.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa97a3dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa974152ef0 0x7fa9740c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa97a3dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa974152ef0 0x7fa9740c43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58582/0 (socket says 172.21.15.112:58582) 2024-04-25T01:26:29.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa97a3dc640 1 -- 172.21.15.112:0/3689822997 learned_addr learned my addr 172.21.15.112:0/3689822997 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:29.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa9793da640 1 --2- 172.21.15.112:0/3689822997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9740c9570 0x7fa974155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa979bdb640 1 --2- 172.21.15.112:0/3689822997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9740c9060 0x7fa9741597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa97a3dc640 1 -- 172.21.15.112:0/3689822997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9740c9060 msgr2=0x7fa9741597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa97a3dc640 1 --2- 172.21.15.112:0/3689822997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9740c9060 0x7fa9741597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa97a3dc640 1 -- 172.21.15.112:0/3689822997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9740c9570 msgr2=0x7fa974155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa97a3dc640 1 --2- 172.21.15.112:0/3689822997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9740c9570 0x7fa974155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa97a3dc640 1 -- 172.21.15.112:0/3689822997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9741eee10 con 0x7fa974152ef0 2024-04-25T01:26:29.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa97a3dc640 1 --2- 172.21.15.112:0/3689822997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa974152ef0 0x7fa9740c43f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00d5d0 tx=0x7fa96c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:29.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa96a7fc640 1 -- 172.21.15.112:0/3689822997 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c0125f0 con 0x7fa974152ef0 2024-04-25T01:26:29.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa96a7fc640 1 -- 172.21.15.112:0/3689822997 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa96c019070 con 0x7fa974152ef0 2024-04-25T01:26:29.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa96a7fc640 1 -- 172.21.15.112:0/3689822997 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c0129f0 con 0x7fa974152ef0 2024-04-25T01:26:29.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa97be3f640 1 -- 172.21.15.112:0/3689822997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9740c2a30 con 0x7fa974152ef0 2024-04-25T01:26:29.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.422+0000 7fa97be3f640 1 -- 172.21.15.112:0/3689822997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9741ef0f0 con 0x7fa974152ef0 2024-04-25T01:26:29.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.423+0000 7fa97be3f640 1 -- 172.21.15.112:0/3689822997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9740689b0 con 0x7fa974152ef0 2024-04-25T01:26:29.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.424+0000 7fa96a7fc640 1 -- 172.21.15.112:0/3689822997 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa96c012b90 con 0x7fa974152ef0 2024-04-25T01:26:29.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.424+0000 7fa96a7fc640 1 --2- 172.21.15.112:0/3689822997 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa95806be30 0x7fa95806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.424+0000 7fa96a7fc640 1 -- 172.21.15.112:0/3689822997 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fa96c08b3d0 con 0x7fa974152ef0 2024-04-25T01:26:29.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.424+0000 7fa979bdb640 1 --2- 172.21.15.112:0/3689822997 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa95806be30 0x7fa95806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.425+0000 7fa979bdb640 1 --2- 172.21.15.112:0/3689822997 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa95806be30 0x7fa95806e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa970001f10 tx=0x7fa970009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:29.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.429+0000 7fa96a7fc640 1 -- 172.21.15.112:0/3689822997 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa96c059410 con 0x7fa974152ef0 2024-04-25T01:26:29.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.675+0000 7fa97be3f640 1 -- 172.21.15.112:0/3689822997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa9740cbd80 con 0x7fa974152ef0 2024-04-25T01:26:29.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.676+0000 7fa96a7fc640 1 -- 172.21.15.112:0/3689822997 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1568 (secure 0 0 0) 0x7fa96c05d710 con 0x7fa974152ef0 2024-04-25T01:26:29.655 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":61,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:26:29.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.678+0000 7fa97be3f640 1 -- 172.21.15.112:0/3689822997 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa95806be30 msgr2=0x7fa95806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.678+0000 7fa97be3f640 1 --2- 172.21.15.112:0/3689822997 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa95806be30 0x7fa95806e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa970001f10 tx=0x7fa970009040 comp rx=0 tx=0).stop 2024-04-25T01:26:29.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.678+0000 7fa97be3f640 1 -- 172.21.15.112:0/3689822997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa974152ef0 msgr2=0x7fa9740c43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.678+0000 7fa97be3f640 1 --2- 172.21.15.112:0/3689822997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa974152ef0 0x7fa9740c43f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00d5d0 tx=0x7fa96c00da90 comp rx=0 tx=0).stop 2024-04-25T01:26:29.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.678+0000 7fa97be3f640 1 -- 172.21.15.112:0/3689822997 shutdown_connections 2024-04-25T01:26:29.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.678+0000 7fa97be3f640 1 --2- 172.21.15.112:0/3689822997 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa95806be30 0x7fa95806e2f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.678+0000 7fa97be3f640 1 --2- 172.21.15.112:0/3689822997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa974152ef0 0x7fa9740c43f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.678+0000 7fa97be3f640 1 --2- 172.21.15.112:0/3689822997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9740c9570 0x7fa974155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.678+0000 7fa97be3f640 1 --2- 172.21.15.112:0/3689822997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9740c9060 0x7fa9741597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.678+0000 7fa97be3f640 1 -- 172.21.15.112:0/3689822997 >> 172.21.15.112:0/3689822997 conn(0x7fa9740cb160 msgr2=0x7fa97414bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:29.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.678+0000 7fa97be3f640 1 -- 172.21.15.112:0/3689822997 shutdown_connections 2024-04-25T01:26:29.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.678+0000 7fa97be3f640 1 -- 172.21.15.112:0/3689822997 wait complete. 2024-04-25T01:26:29.668 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:26:29.668 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T01:26:29.668 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:26:29.668 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:26:29.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.787+0000 7fa96a831640 1 Processor -- start 2024-04-25T01:26:29.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.787+0000 7fa96a831640 1 -- start start 2024-04-25T01:26:29.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.788+0000 7fa96a831640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9640aa210 0x7fa964070d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.788+0000 7fa96a831640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa964159bd0 con 0x7fa96415d370 2024-04-25T01:26:29.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.788+0000 7fa96a831640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa964159e90 con 0x7fa9640ac1d0 2024-04-25T01:26:29.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.788+0000 7fa96a831640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa96415a170 con 0x7fa9640aa210 2024-04-25T01:26:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.788+0000 7fa968dce640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa96415d370 0x7fa96415d790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:45246/0 (socket says 172.21.15.112:45246) 2024-04-25T01:26:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.788+0000 7fa968dce640 1 -- 172.21.15.112:0/3581549094 learned_addr learned my addr 172.21.15.112:0/3581549094 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.788+0000 7fa9637fe640 1 --2- 172.21.15.112:0/3581549094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9640aa210 0x7fa964070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.788+0000 7fa962ffd640 1 -- 172.21.15.112:0/3581549094 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 456457037 0 0) 0x7fa964159bd0 con 0x7fa96415d370 2024-04-25T01:26:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.788+0000 7fa962ffd640 1 -- 172.21.15.112:0/3581549094 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa94c003520 con 0x7fa96415d370 2024-04-25T01:26:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.788+0000 7fa962ffd640 1 -- 172.21.15.112:0/3581549094 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3911193592 0 0) 0x7fa964159e90 con 0x7fa9640ac1d0 2024-04-25T01:26:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.788+0000 7fa962ffd640 1 -- 172.21.15.112:0/3581549094 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa964159bd0 con 0x7fa9640ac1d0 2024-04-25T01:26:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.788+0000 7fa9637fe640 1 -- 172.21.15.112:0/3581549094 >> v1:172.21.15.187:6789/0 conn(0x7fa9640ac1d0 legacy=0x7fa9640c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.788+0000 7fa9637fe640 1 -- 172.21.15.112:0/3581549094 >> v1:172.21.15.112:6789/0 conn(0x7fa96415d370 legacy=0x7fa96415d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.788+0000 7fa9637fe640 1 -- 172.21.15.112:0/3581549094 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96415a450 con 0x7fa9640aa210 2024-04-25T01:26:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.789+0000 7fa9637fe640 1 --2- 172.21.15.112:0/3581549094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9640aa210 0x7fa964070d60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa9580097d0 tx=0x7fa95802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4822419c617b5d54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.789+0000 7fa962ffd640 1 -- 172.21.15.112:0/3581549094 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958037070 con 0x7fa9640aa210 2024-04-25T01:26:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.789+0000 7fa962ffd640 1 -- 172.21.15.112:0/3581549094 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa958032440 con 0x7fa9640aa210 2024-04-25T01:26:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.789+0000 7fa962ffd640 1 -- 172.21.15.112:0/3581549094 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9580327f0 con 0x7fa9640aa210 2024-04-25T01:26:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.790+0000 7fa96a831640 1 -- 172.21.15.112:0/3581549094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9640aa210 msgr2=0x7fa964070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.790+0000 7fa96a831640 1 --2- 172.21.15.112:0/3581549094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9640aa210 0x7fa964070d60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa9580097d0 tx=0x7fa95802a180 comp rx=0 tx=0).stop 2024-04-25T01:26:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.790+0000 7fa96a831640 1 -- 172.21.15.112:0/3581549094 shutdown_connections 2024-04-25T01:26:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.790+0000 7fa96a831640 1 --2- 172.21.15.112:0/3581549094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9640aa210 0x7fa964070d60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.790+0000 7fa96a831640 1 -- 172.21.15.112:0/3581549094 >> 172.21.15.112:0/3581549094 conn(0x7fa96406b160 msgr2=0x7fa96406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.790+0000 7fa96a831640 1 -- 172.21.15.112:0/3581549094 shutdown_connections 2024-04-25T01:26:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.790+0000 7fa96a831640 1 -- 172.21.15.112:0/3581549094 wait complete. 2024-04-25T01:26:29.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.791+0000 7fa96a831640 1 Processor -- start 2024-04-25T01:26:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.791+0000 7fa96a831640 1 -- start start 2024-04-25T01:26:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa96a831640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9640aa210 0x7fa964150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa96a831640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9640ac1d0 0x7fa9641ef670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa96a831640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa96415d370 0x7fa96406e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa96a831640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9640c4220 con 0x7fa96415d370 2024-04-25T01:26:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa96a831640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa9640ac5b0 con 0x7fa9640aa210 2024-04-25T01:26:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa96a831640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa9640c43a0 con 0x7fa9640ac1d0 2024-04-25T01:26:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa968dce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa96415d370 0x7fa96406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa968dce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa96415d370 0x7fa96406e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58584/0 (socket says 172.21.15.112:58584) 2024-04-25T01:26:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa968dce640 1 -- 172.21.15.112:0/2282946319 learned_addr learned my addr 172.21.15.112:0/2282946319 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa9637fe640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9640ac1d0 0x7fa9641ef670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa963fff640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9640aa210 0x7fa964150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa968dce640 1 -- 172.21.15.112:0/2282946319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9640ac1d0 msgr2=0x7fa9641ef670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa968dce640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9640ac1d0 0x7fa9641ef670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa968dce640 1 -- 172.21.15.112:0/2282946319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9640aa210 msgr2=0x7fa964150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa968dce640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9640aa210 0x7fa964150ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa968dce640 1 -- 172.21.15.112:0/2282946319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9641543f0 con 0x7fa96415d370 2024-04-25T01:26:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa963fff640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9640aa210 0x7fa964150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa968dce640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa96415d370 0x7fa96406e250 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00cbf0 tx=0x7fa95c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa960ff9640 1 -- 172.21.15.112:0/2282946319 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c017070 con 0x7fa96415d370 2024-04-25T01:26:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa960ff9640 1 -- 172.21.15.112:0/2282946319 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa95c00acf0 con 0x7fa96415d370 2024-04-25T01:26:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa96a831640 1 -- 172.21.15.112:0/2282946319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9641546d0 con 0x7fa96415d370 2024-04-25T01:26:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa960ff9640 1 -- 172.21.15.112:0/2282946319 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c012600 con 0x7fa96415d370 2024-04-25T01:26:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.792+0000 7fa96a831640 1 -- 172.21.15.112:0/2282946319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa96406e790 con 0x7fa96415d370 2024-04-25T01:26:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.793+0000 7fa96a831640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9640c79c0 0x7fa9641f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.793+0000 7fa96a831640 1 -- 172.21.15.112:0/2282946319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa964151190 con 0x7fa9640c79c0 2024-04-25T01:26:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.794+0000 7fa963fff640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9640c79c0 0x7fa9641f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.794+0000 7fa963fff640 1 -- 172.21.15.112:0/2282946319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa964075980 con 0x7fa9640c79c0 2024-04-25T01:26:29.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.794+0000 7fa960ff9640 1 -- 172.21.15.112:0/2282946319 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa95c01a070 con 0x7fa96415d370 2024-04-25T01:26:29.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.794+0000 7fa963fff640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9640c79c0 0x7fa9641f3460 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa95400d450 tx=0x7fa95400d910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:29.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.794+0000 7fa960ff9640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa95006be60 0x7fa95006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.794+0000 7fa9637fe640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa95006be60 0x7fa95006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.794+0000 7fa960ff9640 1 -- 172.21.15.112:0/2282946319 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fa95c08ae10 con 0x7fa96415d370 2024-04-25T01:26:29.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.794+0000 7fa960ff9640 1 -- 172.21.15.112:0/2282946319 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954005340 con 0x7fa9640c79c0 2024-04-25T01:26:29.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.795+0000 7fa960ff9640 1 -- 172.21.15.112:0/2282946319 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa954007020 con 0x7fa9640c79c0 2024-04-25T01:26:29.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.795+0000 7fa960ff9640 1 -- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9640c79c0 msgr2=0x7fa9641f3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.795+0000 7fa960ff9640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9640c79c0 0x7fa9641f3460 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa95400d450 tx=0x7fa95400d910 comp rx=0 tx=0).stop 2024-04-25T01:26:29.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.795+0000 7fa960ff9640 1 -- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9640c79c0 msgr2=0x7fa9641f3460 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:26:29.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.795+0000 7fa960ff9640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9640c79c0 0x7fa9641f3460 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.795+0000 7fa9637fe640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa95006be60 0x7fa95006e320 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa958034000 tx=0x7fa95802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:29.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.801+0000 7fa96a831640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9641e6590 0x7fa9641e6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.801+0000 7fa96a831640 1 -- 172.21.15.112:0/2282946319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9640682a0 con 0x7fa9641e6590 2024-04-25T01:26:29.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.801+0000 7fa963fff640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9641e6590 0x7fa9641e6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.801+0000 7fa963fff640 1 -- 172.21.15.112:0/2282946319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa9640c3750 con 0x7fa9641e6590 2024-04-25T01:26:29.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.802+0000 7fa963fff640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9641e6590 0x7fa9641e6970 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa95400d450 tx=0x7fa954000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:29.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.802+0000 7fa960ff9640 1 -- 172.21.15.112:0/2282946319 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95401b770 con 0x7fa9641e6590 2024-04-25T01:26:29.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.802+0000 7fa960ff9640 1 -- 172.21.15.112:0/2282946319 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5070 (secure 0 0 0) 0x7fa95401b770 con 0x7fa9641e6590 2024-04-25T01:26:29.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.802+0000 7fa960ff9640 1 -- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9641e6590 msgr2=0x7fa9641e6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.802+0000 7fa960ff9640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9641e6590 0x7fa9641e6970 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa95400d450 tx=0x7fa954000ed0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.802+0000 7fa960ff9640 1 -- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9641e6590 msgr2=0x7fa9641e6970 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:26:29.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.802+0000 7fa960ff9640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9641e6590 0x7fa9641e6970 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.782 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:26:29.782 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:26:29.782 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:26:29.782 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:26:29.783 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 4, 2024-04-25T01:26:29.783 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 61430, 2024-04-25T01:26:29.783 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:26:29.783 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:26:29.783 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:26:29.783 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:26:29.783 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:29.783 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 61, 2024-04-25T01:26:29.783 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:26:29.783 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:26:29.783 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:26:29.783 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:26:29.783 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:26:29.783 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:26:29.784 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:26:29.784 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:26:29.784 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:26:29.784 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:26:29.784 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:26:29.784 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:26:29.784 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:26:29.784 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:26:29.784 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:29.784 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:26:29.784 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:26:29.784 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:26:29.784 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:26:29.784 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:26:29.784 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:26:29.785 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:26:29.785 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:26:29.785 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:26:29.785 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:26:29.785 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:26:29.785 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:26:29.785 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:26:29.785 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:29.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.785 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:26:29.785 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:26:29.785 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:26:29.785 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:26:29.785 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:26:29.785 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:26:29.786 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:26:29.786 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:26:29.786 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:26:29.786 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:26:29.786 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:26:29.786 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:26:29.786 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:26:29.786 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:26:29.786 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:26:29.786 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:26:29.786 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:26:29.786 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:26:29.786 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:26:29.786 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:26:29.787 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:26:29.787 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:26:29.787 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:26:29.787 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:26:29.787 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:26:29.787 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:26:29.787 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:26:29.787 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:26:29.787 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:29.787 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:26:29.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.787 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:26:29.787 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.787 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:26:29.787 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:26:29.788 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:26:29.788 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:26:29.788 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.788 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:26:29.788 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:26:29.788 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:29.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.788 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.788 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:26:29.788 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:26:29.788 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:29.788 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:29.788 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:29.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.788 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:26:29.789 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:26:29.789 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:26:29.789 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:26:29.789 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:26:29.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.789 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.789 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:26:29.789 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:26:29.789 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:26:29.789 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:26:29.789 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.789 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:26:29.789 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:26:29.789 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:29.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:26:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:26:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:26:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:26:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:26:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:26:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:26:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 7 2024-04-25T01:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:26:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:26:29.795 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:26:29.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.802+0000 7fa96a831640 1 -- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa95006be60 msgr2=0x7fa95006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.802+0000 7fa96a831640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa95006be60 0x7fa95006e320 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa958034000 tx=0x7fa95802e000 comp rx=0 tx=0).stop 2024-04-25T01:26:29.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.802+0000 7fa96a831640 1 -- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa96415d370 msgr2=0x7fa96406e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.802+0000 7fa96a831640 1 --2- 172.21.15.112:0/2282946319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa96415d370 0x7fa96406e250 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00cbf0 tx=0x7fa95c00a430 comp rx=0 tx=0).stop 2024-04-25T01:26:29.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.802+0000 7fa968dce640 1 -- 172.21.15.112:0/2282946319 reap_dead start 2024-04-25T01:26:29.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.802+0000 7fa968dce640 1 -- 172.21.15.112:0/2282946319 reap_dead start 2024-04-25T01:26:29.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.802+0000 7fa96a831640 1 -- 172.21.15.112:0/2282946319 shutdown_connections 2024-04-25T01:26:29.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.802+0000 7fa96a831640 1 -- 172.21.15.112:0/2282946319 >> 172.21.15.112:0/2282946319 conn(0x7fa96406b160 msgr2=0x7fa96406b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:29.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.802+0000 7fa96a831640 1 -- 172.21.15.112:0/2282946319 shutdown_connections 2024-04-25T01:26:29.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.802+0000 7fa96a831640 1 -- 172.21.15.112:0/2282946319 wait complete. 2024-04-25T01:26:29.797 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:26:29.797 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:26:29.797 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:26:29.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.916+0000 7f407971b640 1 Processor -- start 2024-04-25T01:26:29.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.916+0000 7f407971b640 1 -- start start 2024-04-25T01:26:29.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.916+0000 7f407971b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f40740cadf0 0x7f407406c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.916+0000 7f407971b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f40740c5b70 0x7f4074156d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.916+0000 7f407971b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40740c9290 con 0x7f407406bdc0 2024-04-25T01:26:29.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.916+0000 7f407971b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f4074156890 con 0x7f40740c5b70 2024-04-25T01:26:29.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.916+0000 7f407971b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4074157280 con 0x7f40740cadf0 2024-04-25T01:26:29.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.916+0000 7f40737fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f40740c5b70 0x7f4074156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.916+0000 7f40737fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f40740c5b70 0x7f4074156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58610/0 (socket says 172.21.15.112:58610) 2024-04-25T01:26:29.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.916+0000 7f40737fe640 1 -- 172.21.15.112:0/223073146 learned_addr learned my addr 172.21.15.112:0/223073146 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:29.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.917+0000 7f40737fe640 1 -- 172.21.15.112:0/223073146 >> v2:172.21.15.187:3300/0 conn(0x7f40740cadf0 msgr2=0x7f407406c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:29.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.917+0000 7f40737fe640 1 --2- 172.21.15.112:0/223073146 >> v2:172.21.15.187:3300/0 conn(0x7f40740cadf0 0x7f407406c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.917+0000 7f40737fe640 1 -- 172.21.15.112:0/223073146 >> v1:172.21.15.187:6789/0 conn(0x7f407406bdc0 legacy=0x7f40740c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:29.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.917+0000 7f40737fe640 1 -- 172.21.15.112:0/223073146 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40741565b0 con 0x7f40740c5b70 2024-04-25T01:26:29.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.917+0000 7f40737fe640 1 --2- 172.21.15.112:0/223073146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40740c5b70 0x7f4074156d40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f40740add40 tx=0x7f406c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdb9134d0c7e3b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:29.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.917+0000 7f4071ffb640 1 -- 172.21.15.112:0/223073146 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c036070 con 0x7f40740c5b70 2024-04-25T01:26:29.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.918+0000 7f4071ffb640 1 -- 172.21.15.112:0/223073146 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f406c031600 con 0x7f40740c5b70 2024-04-25T01:26:29.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.918+0000 7f4071ffb640 1 -- 172.21.15.112:0/223073146 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c0319d0 con 0x7f40740c5b70 2024-04-25T01:26:29.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.918+0000 7f407971b640 1 -- 172.21.15.112:0/223073146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40740c5b70 msgr2=0x7f4074156d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.918+0000 7f407971b640 1 --2- 172.21.15.112:0/223073146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40740c5b70 0x7f4074156d40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f40740add40 tx=0x7f406c02a5d0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.918+0000 7f407971b640 1 -- 172.21.15.112:0/223073146 shutdown_connections 2024-04-25T01:26:29.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.918+0000 7f407971b640 1 --2- 172.21.15.112:0/223073146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40740c5b70 0x7f4074156d40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.918+0000 7f407971b640 1 --2- 172.21.15.112:0/223073146 >> v2:172.21.15.187:3300/0 conn(0x7f40740cadf0 0x7f407406c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.918+0000 7f407971b640 1 -- 172.21.15.112:0/223073146 >> 172.21.15.112:0/223073146 conn(0x7f40740acd20 msgr2=0x7f40740ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:29.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.918+0000 7f407971b640 1 -- 172.21.15.112:0/223073146 shutdown_connections 2024-04-25T01:26:29.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.918+0000 7f407971b640 1 -- 172.21.15.112:0/223073146 wait complete. 2024-04-25T01:26:29.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.919+0000 7f407971b640 1 Processor -- start 2024-04-25T01:26:29.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.919+0000 7f407971b640 1 -- start start 2024-04-25T01:26:29.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.920+0000 7f407971b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f407406bdc0 0x7f4074152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.920+0000 7f407971b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40740c5b70 0x7f40741576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.920+0000 7f407971b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40740cadf0 0x7f40740bed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.920+0000 7f407971b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4074156280 con 0x7f407406bdc0 2024-04-25T01:26:29.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.920+0000 7f407971b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f40740c9810 con 0x7f40740cadf0 2024-04-25T01:26:29.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.920+0000 7f407971b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f40740b0b50 con 0x7f40740c5b70 2024-04-25T01:26:29.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.920+0000 7f4072ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f407406bdc0 0x7f4074152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.920+0000 7f4072ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f407406bdc0 0x7f4074152690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58624/0 (socket says 172.21.15.112:58624) 2024-04-25T01:26:29.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.920+0000 7f4072ffd640 1 -- 172.21.15.112:0/2582336739 learned_addr learned my addr 172.21.15.112:0/2582336739 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:29.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.920+0000 7f40727fc640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40740c5b70 0x7f40741576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.920+0000 7f40737fe640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40740cadf0 0x7f40740bed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.920+0000 7f4072ffd640 1 -- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40740c5b70 msgr2=0x7f40741576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.920+0000 7f4072ffd640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40740c5b70 0x7f40741576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.920+0000 7f4072ffd640 1 -- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40740cadf0 msgr2=0x7f40740bed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.920+0000 7f4072ffd640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40740cadf0 0x7f40740bed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.920+0000 7f4072ffd640 1 -- 172.21.15.112:0/2582336739 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40740c4e20 con 0x7f407406bdc0 2024-04-25T01:26:29.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.920+0000 7f40727fc640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40740c5b70 0x7f40741576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:29.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.921+0000 7f4072ffd640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f407406bdc0 0x7f4074152690 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f406800c440 tx=0x7f406800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:29.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.921+0000 7f405b7fe640 1 -- 172.21.15.112:0/2582336739 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068017070 con 0x7f407406bdc0 2024-04-25T01:26:29.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.921+0000 7f405b7fe640 1 -- 172.21.15.112:0/2582336739 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40680035f0 con 0x7f407406bdc0 2024-04-25T01:26:29.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.921+0000 7f405b7fe640 1 -- 172.21.15.112:0/2582336739 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40680039e0 con 0x7f407406bdc0 2024-04-25T01:26:29.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.921+0000 7f407971b640 1 -- 172.21.15.112:0/2582336739 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40740abdb0 con 0x7f407406bdc0 2024-04-25T01:26:29.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.921+0000 7f407971b640 1 -- 172.21.15.112:0/2582336739 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40740c5370 con 0x7f407406bdc0 2024-04-25T01:26:29.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.922+0000 7f407971b640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40740c0390 0x7f40740c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.922+0000 7f407971b640 1 -- 172.21.15.112:0/2582336739 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4074075980 con 0x7f40740c0390 2024-04-25T01:26:29.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.922+0000 7f40727fc640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40740c0390 0x7f40740c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.922+0000 7f40727fc640 1 -- 172.21.15.112:0/2582336739 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f40740682a0 con 0x7f40740c0390 2024-04-25T01:26:29.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.922+0000 7f40727fc640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40740c0390 0x7f40740c18c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f406400afd0 tx=0x7f406400c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:29.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.922+0000 7f405b7fe640 1 -- 172.21.15.112:0/2582336739 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f406801a070 con 0x7f407406bdc0 2024-04-25T01:26:29.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.923+0000 7f405b7fe640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f406006be30 0x7f406006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.923+0000 7f405b7fe640 1 -- 172.21.15.112:0/2582336739 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f406808a490 con 0x7f407406bdc0 2024-04-25T01:26:29.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.923+0000 7f40737fe640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f406006be30 0x7f406006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.923+0000 7f405b7fe640 1 -- 172.21.15.112:0/2582336739 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064015070 con 0x7f40740c0390 2024-04-25T01:26:29.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.923+0000 7f405b7fe640 1 -- 172.21.15.112:0/2582336739 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4064011800 con 0x7f40740c0390 2024-04-25T01:26:29.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.923+0000 7f405b7fe640 1 -- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40740c0390 msgr2=0x7f40740c18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.923+0000 7f405b7fe640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40740c0390 0x7f40740c18c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f406400afd0 tx=0x7f406400c950 comp rx=0 tx=0).stop 2024-04-25T01:26:29.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.923+0000 7f405b7fe640 1 -- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40740c0390 msgr2=0x7f40740c18c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:26:29.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.923+0000 7f405b7fe640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40740c0390 0x7f40740c18c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.923+0000 7f40737fe640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f406006be30 0x7f406006e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f406c033000 tx=0x7f406c009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:29.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.929+0000 7f407971b640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40740bf260 0x7f40741ee760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:29.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.929+0000 7f407971b640 1 -- 172.21.15.112:0/2582336739 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f40740ab200 con 0x7f40740bf260 2024-04-25T01:26:29.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.930+0000 7f40727fc640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40740bf260 0x7f40741ee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:29.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.930+0000 7f40727fc640 1 -- 172.21.15.112:0/2582336739 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f407406b160 con 0x7f40740bf260 2024-04-25T01:26:29.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.930+0000 7f40727fc640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40740bf260 0x7f40741ee760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f406400afd0 tx=0x7f4064004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:29.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.930+0000 7f405b7fe640 1 -- 172.21.15.112:0/2582336739 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064028020 con 0x7f40740bf260 2024-04-25T01:26:29.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.930+0000 7f405b7fe640 1 -- 172.21.15.112:0/2582336739 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4898 (secure 0 0 0) 0x7f4064023350 con 0x7f40740bf260 2024-04-25T01:26:29.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.930+0000 7f405b7fe640 1 -- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40740bf260 msgr2=0x7f40741ee760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.930+0000 7f405b7fe640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40740bf260 0x7f40741ee760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f406400afd0 tx=0x7f4064004730 comp rx=0 tx=0).stop 2024-04-25T01:26:29.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.930+0000 7f405b7fe640 1 -- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40740bf260 msgr2=0x7f40741ee760 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:26:29.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.930+0000 7f405b7fe640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40740bf260 0x7f40741ee760 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:29.911 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:26:29.911 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:26:29.911 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:26:29.911 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:26:29.911 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 4, 2024-04-25T01:26:29.911 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 61519, 2024-04-25T01:26:29.911 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:26:29.911 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:26:29.911 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:26:29.911 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:26:29.911 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 61, 2024-04-25T01:26:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:26:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:26:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:26:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:26:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:26:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:26:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:26:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:26:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:26:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:26:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:26:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:26:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:26:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:26:29.913 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:29.913 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:26:29.913 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:26:29.913 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:26:29.913 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:26:29.913 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:26:29.913 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:26:29.913 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:26:29.913 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:26:29.913 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:26:29.913 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:26:29.913 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:26:29.913 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:26:29.913 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:26:29.913 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:29.914 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.914 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:26:29.914 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:26:29.914 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:26:29.914 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:26:29.914 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:26:29.914 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:26:29.914 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:26:29.914 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:26:29.914 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:26:29.914 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:26:29.914 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:26:29.914 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:26:29.914 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:26:29.915 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:26:29.915 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:26:29.915 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:26:29.915 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:26:29.915 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:26:29.915 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:26:29.915 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:26:29.915 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:26:29.915 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:26:29.915 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:26:29.915 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:26:29.915 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:26:29.915 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:26:29.915 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:26:29.915 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:26:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:26:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:26:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:26:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:26:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:26:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:26:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:26:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:26:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.917 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:26:29.917 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:26:29.917 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:29.917 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:29.917 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:29.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.917 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:26:29.917 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:26:29.917 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:26:29.917 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:26:29.917 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:26:29.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.917 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.917 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:26:29.917 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:26:29.918 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:26:29.918 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:26:29.918 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.918 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:26:29.918 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:26:29.918 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:29.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.918 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.918 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:26:29.918 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:26:29.918 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:29.918 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:29.918 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:29.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.919 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:26:29.919 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:26:29.919 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:26:29.919 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:26:29.919 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:26:29.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.919 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.919 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:26:29.919 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:26:29.919 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:26:29.919 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:26:29.919 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.919 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:26:29.919 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:26:29.919 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:29.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.920 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.920 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:26:29.920 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:26:29.920 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:29.920 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:29.920 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:29.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.920 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:26:29.920 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:26:29.920 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:26:29.920 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:26:29.920 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:26:29.920 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:29.921 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:29.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.921 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:26:29.921 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:26:29.921 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.921 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:29.921 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:29.921 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:26:29.921 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:29.921 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:29.921 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:26:29.921 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.921 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:29.921 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:29.921 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:26:29.922 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:29.922 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:29.922 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:26:29.922 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.922 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:29.922 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:29.922 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:26:29.922 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:29.922 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:29.922 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:26:29.922 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:29.922 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:29.922 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:29.922 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:26:29.923 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:29.923 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:29.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:29.923 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:26:29.923 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:26:29.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.931+0000 7f407971b640 1 -- 172.21.15.112:0/2582336739 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f406006be30 msgr2=0x7f406006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.931+0000 7f407971b640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f406006be30 0x7f406006e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f406c033000 tx=0x7f406c009000 comp rx=0 tx=0).stop 2024-04-25T01:26:29.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.931+0000 7f407971b640 1 -- 172.21.15.112:0/2582336739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f407406bdc0 msgr2=0x7f4074152690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:29.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.931+0000 7f407971b640 1 --2- 172.21.15.112:0/2582336739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f407406bdc0 0x7f4074152690 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f406800c440 tx=0x7f406800c900 comp rx=0 tx=0).stop 2024-04-25T01:26:29.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.931+0000 7f40737fe640 1 -- 172.21.15.112:0/2582336739 reap_dead start 2024-04-25T01:26:29.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.931+0000 7f40737fe640 1 -- 172.21.15.112:0/2582336739 reap_dead start 2024-04-25T01:26:29.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.931+0000 7f407971b640 1 -- 172.21.15.112:0/2582336739 shutdown_connections 2024-04-25T01:26:29.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.931+0000 7f407971b640 1 -- 172.21.15.112:0/2582336739 >> 172.21.15.112:0/2582336739 conn(0x7f40740acd20 msgr2=0x7f407406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:29.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.931+0000 7f407971b640 1 -- 172.21.15.112:0/2582336739 shutdown_connections 2024-04-25T01:26:29.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:29.931+0000 7f407971b640 1 -- 172.21.15.112:0/2582336739 wait complete. 2024-04-25T01:26:29.925 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:26:29.925 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:26:29.925 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:26:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.043+0000 7f0d762cd640 1 Processor -- start 2024-04-25T01:26:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.043+0000 7f0d762cd640 1 -- start start 2024-04-25T01:26:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.044+0000 7f0d762cd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d700724c0 0x7f0d700c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.044+0000 7f0d762cd640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d7006e0f0 con 0x7f0d701536c0 2024-04-25T01:26:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.044+0000 7f0d762cd640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d70072910 con 0x7f0d70072d80 2024-04-25T01:26:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.044+0000 7f0d762cd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0d700c2180 con 0x7f0d700724c0 2024-04-25T01:26:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.044+0000 7f0d6f7fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0d701536c0 0x7f0d70071db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:45254/0 (socket says 172.21.15.112:45254) 2024-04-25T01:26:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.044+0000 7f0d6f7fe640 1 -- 172.21.15.112:0/495023025 learned_addr learned my addr 172.21.15.112:0/495023025 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.044+0000 7f0d7486a640 1 --2- 172.21.15.112:0/495023025 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d700724c0 0x7f0d700c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.044+0000 7f0d6effd640 1 -- 172.21.15.112:0/495023025 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 213508518 0 0) 0x7f0d60001110 con 0x7f0d701536c0 2024-04-25T01:26:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.044+0000 7f0d6effd640 1 -- 172.21.15.112:0/495023025 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d58003520 con 0x7f0d701536c0 2024-04-25T01:26:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.044+0000 7f0d6effd640 1 -- 172.21.15.112:0/495023025 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3896055787 0 0) 0x7f0d70072910 con 0x7f0d70072d80 2024-04-25T01:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.045+0000 7f0d6effd640 1 -- 172.21.15.112:0/495023025 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d60001110 con 0x7f0d70072d80 2024-04-25T01:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.045+0000 7f0d7486a640 1 -- 172.21.15.112:0/495023025 >> v1:172.21.15.187:6789/0 conn(0x7f0d70072d80 legacy=0x7f0d7014d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.045+0000 7f0d7486a640 1 -- 172.21.15.112:0/495023025 >> v1:172.21.15.112:6789/0 conn(0x7f0d701536c0 legacy=0x7f0d70071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.045+0000 7f0d7486a640 1 -- 172.21.15.112:0/495023025 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d700c2460 con 0x7f0d700724c0 2024-04-25T01:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.045+0000 7f0d7486a640 1 --2- 172.21.15.112:0/495023025 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d700724c0 0x7f0d700c1a80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009dc0 tx=0x7f0d6802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffa625716b33bfad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.045+0000 7f0d6effd640 1 -- 172.21.15.112:0/495023025 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68007740 con 0x7f0d700724c0 2024-04-25T01:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.045+0000 7f0d6effd640 1 -- 172.21.15.112:0/495023025 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d680078e0 con 0x7f0d700724c0 2024-04-25T01:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.045+0000 7f0d6effd640 1 -- 172.21.15.112:0/495023025 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68007cb0 con 0x7f0d700724c0 2024-04-25T01:26:30.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.046+0000 7f0d762cd640 1 -- 172.21.15.112:0/495023025 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d700724c0 msgr2=0x7f0d700c1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:30.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.046+0000 7f0d762cd640 1 --2- 172.21.15.112:0/495023025 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d700724c0 0x7f0d700c1a80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009dc0 tx=0x7f0d6802a6d0 comp rx=0 tx=0).stop 2024-04-25T01:26:30.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.046+0000 7f0d762cd640 1 -- 172.21.15.112:0/495023025 shutdown_connections 2024-04-25T01:26:30.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.046+0000 7f0d762cd640 1 --2- 172.21.15.112:0/495023025 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d700724c0 0x7f0d700c1a80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:30.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.046+0000 7f0d762cd640 1 -- 172.21.15.112:0/495023025 >> 172.21.15.112:0/495023025 conn(0x7f0d7006b160 msgr2=0x7f0d7006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:30.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.046+0000 7f0d762cd640 1 -- 172.21.15.112:0/495023025 shutdown_connections 2024-04-25T01:26:30.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.046+0000 7f0d762cd640 1 -- 172.21.15.112:0/495023025 wait complete. 2024-04-25T01:26:30.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.047+0000 7f0d762cd640 1 Processor -- start 2024-04-25T01:26:30.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.047+0000 7f0d762cd640 1 -- start start 2024-04-25T01:26:30.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.048+0000 7f0d762cd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d700724c0 0x7f0d70148ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:30.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.048+0000 7f0d762cd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d70072d80 0x7f0d701428c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:30.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.048+0000 7f0d762cd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d701536c0 0x7f0d70142e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:30.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.048+0000 7f0d762cd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0d70071890 con 0x7f0d701536c0 2024-04-25T01:26:30.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.048+0000 7f0d762cd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0d700c2a20 con 0x7f0d70072d80 2024-04-25T01:26:30.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.048+0000 7f0d762cd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0d7014da50 con 0x7f0d700724c0 2024-04-25T01:26:30.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.048+0000 7f0d7486a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d701536c0 0x7f0d70142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:30.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.048+0000 7f0d7486a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d701536c0 0x7f0d70142e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58636/0 (socket says 172.21.15.112:58636) 2024-04-25T01:26:30.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.048+0000 7f0d7486a640 1 -- 172.21.15.112:0/1691947284 learned_addr learned my addr 172.21.15.112:0/1691947284 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.048+0000 7f0d6f7fe640 1 --2- 172.21.15.112:0/1691947284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d70072d80 0x7f0d701428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.048+0000 7f0d6ffff640 1 --2- 172.21.15.112:0/1691947284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d700724c0 0x7f0d70148ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.048+0000 7f0d7486a640 1 -- 172.21.15.112:0/1691947284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d700724c0 msgr2=0x7f0d70148ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.048+0000 7f0d7486a640 1 --2- 172.21.15.112:0/1691947284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d700724c0 0x7f0d70148ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.048+0000 7f0d7486a640 1 -- 172.21.15.112:0/1691947284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d70072d80 msgr2=0x7f0d701428c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.048+0000 7f0d7486a640 1 --2- 172.21.15.112:0/1691947284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d70072d80 0x7f0d701428c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.048+0000 7f0d7486a640 1 -- 172.21.15.112:0/1691947284 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d701e9d60 con 0x7f0d701536c0 2024-04-25T01:26:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.048+0000 7f0d7486a640 1 --2- 172.21.15.112:0/1691947284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d701536c0 0x7f0d70142e00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0d6802acc0 tx=0x7f0d68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.049+0000 7f0d6cff9640 1 -- 172.21.15.112:0/1691947284 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68040070 con 0x7f0d701536c0 2024-04-25T01:26:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.049+0000 7f0d6cff9640 1 -- 172.21.15.112:0/1691947284 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d68031bc0 con 0x7f0d701536c0 2024-04-25T01:26:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.049+0000 7f0d762cd640 1 -- 172.21.15.112:0/1691947284 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d701ea040 con 0x7f0d701536c0 2024-04-25T01:26:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.049+0000 7f0d6cff9640 1 -- 172.21.15.112:0/1691947284 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68009450 con 0x7f0d701536c0 2024-04-25T01:26:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.049+0000 7f0d762cd640 1 -- 172.21.15.112:0/1691947284 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d700c57e0 con 0x7f0d701536c0 2024-04-25T01:26:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.049+0000 7f0d762cd640 1 --2- 172.21.15.112:0/1691947284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d70149010 0x7f0d701ec840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:30.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.050+0000 7f0d762cd640 1 -- 172.21.15.112:0/1691947284 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0d70075980 con 0x7f0d70149010 2024-04-25T01:26:30.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.050+0000 7f0d6ffff640 1 --2- 172.21.15.112:0/1691947284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d70149010 0x7f0d701ec840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:30.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.050+0000 7f0d6ffff640 1 -- 172.21.15.112:0/1691947284 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0d700c7e90 con 0x7f0d70149010 2024-04-25T01:26:30.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.050+0000 7f0d6cff9640 1 -- 172.21.15.112:0/1691947284 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d68039040 con 0x7f0d701536c0 2024-04-25T01:26:30.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.050+0000 7f0d6ffff640 1 --2- 172.21.15.112:0/1691947284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d70149010 0x7f0d701ec840 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400f560 tx=0x7f0d6400fa20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:30.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.050+0000 7f0d6cff9640 1 --2- 172.21.15.112:0/1691947284 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0d5c06be30 0x7f0d5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:30.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.051+0000 7f0d6f7fe640 1 --2- 172.21.15.112:0/1691947284 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0d5c06be30 0x7f0d5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:30.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.051+0000 7f0d6cff9640 1 -- 172.21.15.112:0/1691947284 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f0d680ac710 con 0x7f0d701536c0 2024-04-25T01:26:30.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.051+0000 7f0d6cff9640 1 -- 172.21.15.112:0/1691947284 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d640052e0 con 0x7f0d70149010 2024-04-25T01:26:30.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.051+0000 7f0d6cff9640 1 -- 172.21.15.112:0/1691947284 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0d64018070 con 0x7f0d70149010 2024-04-25T01:26:30.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.051+0000 7f0d6cff9640 1 -- 172.21.15.112:0/1691947284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d70149010 msgr2=0x7f0d701ec840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:30.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.051+0000 7f0d6cff9640 1 --2- 172.21.15.112:0/1691947284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d70149010 0x7f0d701ec840 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400f560 tx=0x7f0d6400fa20 comp rx=0 tx=0).stop 2024-04-25T01:26:30.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.051+0000 7f0d6cff9640 1 -- 172.21.15.112:0/1691947284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d70149010 msgr2=0x7f0d701ec840 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:26:30.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.051+0000 7f0d6cff9640 1 --2- 172.21.15.112:0/1691947284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d70149010 0x7f0d701ec840 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400f560 tx=0x7f0d6400fa20 comp rx=0 tx=0).stop 2024-04-25T01:26:30.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.051+0000 7f0d6f7fe640 1 --2- 172.21.15.112:0/1691947284 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0d5c06be30 0x7f0d5c06e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0d701511c0 tx=0x7f0d60007d70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:30.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.057+0000 7f0d762cd640 1 --2- 172.21.15.112:0/1691947284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d7014dd40 0x7f0d701ecd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:30.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.057+0000 7f0d762cd640 1 -- 172.21.15.112:0/1691947284 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0d700682a0 con 0x7f0d7014dd40 2024-04-25T01:26:30.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.057+0000 7f0d6ffff640 1 --2- 172.21.15.112:0/1691947284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d7014dd40 0x7f0d701ecd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:30.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.058+0000 7f0d6ffff640 1 -- 172.21.15.112:0/1691947284 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0d7006d3c0 con 0x7f0d7014dd40 2024-04-25T01:26:30.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.058+0000 7f0d6ffff640 1 --2- 172.21.15.112:0/1691947284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d7014dd40 0x7f0d701ecd80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0d64017000 tx=0x7f0d64000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:30.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.058+0000 7f0d6cff9640 1 -- 172.21.15.112:0/1691947284 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64003800 con 0x7f0d7014dd40 2024-04-25T01:26:30.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.058+0000 7f0d6cff9640 1 -- 172.21.15.112:0/1691947284 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4898 (secure 0 0 0) 0x7f0d64003800 con 0x7f0d7014dd40 2024-04-25T01:26:30.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.058+0000 7f0d6cff9640 1 -- 172.21.15.112:0/1691947284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d7014dd40 msgr2=0x7f0d701ecd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:30.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.058+0000 7f0d6cff9640 1 --2- 172.21.15.112:0/1691947284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d7014dd40 0x7f0d701ecd80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0d64017000 tx=0x7f0d64000ed0 comp rx=0 tx=0).stop 2024-04-25T01:26:30.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.058+0000 7f0d6cff9640 1 -- 172.21.15.112:0/1691947284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d7014dd40 msgr2=0x7f0d701ecd80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:26:30.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.058+0000 7f0d6cff9640 1 --2- 172.21.15.112:0/1691947284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d7014dd40 0x7f0d701ecd80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:30.039 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:26:30.039 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:26:30.039 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:26:30.039 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:26:30.039 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 4, 2024-04-25T01:26:30.039 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 61647, 2024-04-25T01:26:30.039 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:26:30.040 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:26:30.040 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:26:30.040 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:26:30.040 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:30.040 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 61, 2024-04-25T01:26:30.040 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:26:30.040 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:26:30.040 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:26:30.040 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:26:30.040 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:26:30.040 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:26:30.040 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:26:30.040 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:26:30.041 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:26:30.041 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:26:30.041 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:26:30.041 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:26:30.041 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:26:30.041 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:26:30.041 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:30.041 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:26:30.041 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:26:30.041 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:26:30.041 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:26:30.041 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:26:30.041 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:26:30.041 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:26:30.042 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:26:30.042 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:26:30.042 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:26:30.042 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:26:30.042 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:26:30.042 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:26:30.042 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:30.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:30.042 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:26:30.042 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:26:30.042 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:26:30.043 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:26:30.043 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:26:30.043 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:26:30.043 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:26:30.043 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:26:30.043 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:26:30.043 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:26:30.043 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:26:30.043 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:26:30.043 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:26:30.043 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:26:30.043 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:26:30.043 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:26:30.044 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:26:30.044 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:26:30.044 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:26:30.044 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:26:30.044 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:26:30.044 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:26:30.044 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:26:30.044 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:26:30.044 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:26:30.044 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:26:30.044 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:26:30.044 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:26:30.044 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:30.044 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:26:30.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:30.045 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:26:30.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:30.045 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:26:30.045 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:26:30.045 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:26:30.045 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:26:30.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:30.045 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:26:30.045 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:26:30.045 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:30.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:30.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:30.045 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:26:30.045 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:26:30.046 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:30.046 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:30.046 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:30.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:30.046 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:26:30.046 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:26:30.046 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:26:30.046 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:26:30.046 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:26:30.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:30.046 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:30.046 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:26:30.046 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:26:30.047 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:26:30.047 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:26:30.047 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:30.047 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:26:30.047 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:26:30.047 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:30.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:30.047 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:30.047 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:26:30.047 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:26:30.047 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:30.047 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:30.047 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:30.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:30.047 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:26:30.048 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:26:30.048 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:26:30.048 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:26:30.048 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:26:30.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:30.048 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:30.048 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:26:30.048 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:26:30.048 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:26:30.048 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:26:30.048 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:30.048 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:26:30.048 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:26:30.048 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:30.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:30.049 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:30.049 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:26:30.049 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:26:30.049 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:30.049 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:30.049 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:30.049 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:30.049 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:26:30.049 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:26:30.049 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:26:30.049 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:26:30.049 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:26:30.049 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:30.049 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:30.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:30.050 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:26:30.050 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:26:30.050 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:30.050 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:30.050 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:30.050 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:26:30.050 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:30.050 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:30.050 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:26:30.050 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:30.050 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:30.050 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:30.050 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:26:30.051 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:30.051 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:30.051 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:26:30.051 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:30.051 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:30.051 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:30.051 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:26:30.051 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:30.051 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:30.051 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:26:30.051 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:30.051 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:30.051 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:30.052 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:26:30.052 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:30.052 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:30.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:30.052 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:26:30.052 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:26:30.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.059+0000 7f0d762cd640 1 -- 172.21.15.112:0/1691947284 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0d5c06be30 msgr2=0x7f0d5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:30.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.059+0000 7f0d762cd640 1 --2- 172.21.15.112:0/1691947284 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0d5c06be30 0x7f0d5c06e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0d701511c0 tx=0x7f0d60007d70 comp rx=0 tx=0).stop 2024-04-25T01:26:30.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.059+0000 7f0d762cd640 1 -- 172.21.15.112:0/1691947284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d701536c0 msgr2=0x7f0d70142e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:30.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.059+0000 7f0d762cd640 1 --2- 172.21.15.112:0/1691947284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d701536c0 0x7f0d70142e00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0d6802acc0 tx=0x7f0d68004a40 comp rx=0 tx=0).stop 2024-04-25T01:26:30.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.059+0000 7f0d7486a640 1 -- 172.21.15.112:0/1691947284 reap_dead start 2024-04-25T01:26:30.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.059+0000 7f0d7486a640 1 -- 172.21.15.112:0/1691947284 reap_dead start 2024-04-25T01:26:30.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.059+0000 7f0d762cd640 1 -- 172.21.15.112:0/1691947284 shutdown_connections 2024-04-25T01:26:30.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.059+0000 7f0d762cd640 1 -- 172.21.15.112:0/1691947284 >> 172.21.15.112:0/1691947284 conn(0x7f0d7006b160 msgr2=0x7f0d7014d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:30.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.059+0000 7f0d762cd640 1 -- 172.21.15.112:0/1691947284 shutdown_connections 2024-04-25T01:26:30.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:30.059+0000 7f0d762cd640 1 -- 172.21.15.112:0/1691947284 wait complete. 2024-04-25T01:26:30.054 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:26:30.054 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-04-25T01:26:30.055 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['c'] 2024-04-25T01:26:30.055 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-04-25T01:26:30.055 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-04-25T01:26:30.055 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T01:26:30.055 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:26:30.067 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T01:26:30.067 INFO:tasks.mon_thrash:Sending STOP to mon c 2024-04-25T01:26:30.067 INFO:tasks.ceph.mon.c:Sent signal 19 2024-04-25T01:26:30.067 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-04-25T01:26:45.068 INFO:tasks.mon_thrash:Sending CONT to mon c 2024-04-25T01:26:45.069 INFO:tasks.ceph.mon.c:Sent signal 18 2024-04-25T01:26:45.069 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T01:26:45.069 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:26:45.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.198+0000 7f402e7ce640 1 Processor -- start 2024-04-25T01:26:45.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.198+0000 7f402e7ce640 1 -- start start 2024-04-25T01:26:45.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.199+0000 7f402e7ce640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4028153610 0x7f402806df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:45.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.199+0000 7f402e7ce640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 0x7f402806e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:45.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.199+0000 7f402e7ce640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f402806cd90 con 0x7f4028150ac0 2024-04-25T01:26:45.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.199+0000 7f402e7ce640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f402806e9d0 con 0x7f4028153610 2024-04-25T01:26:45.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.199+0000 7f402e7ce640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f40280ac4b0 con 0x7f402806d1a0 2024-04-25T01:26:45.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.199+0000 7f4027fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4028150ac0 0x7f40280a9490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59120/0 (socket says 172.21.15.112:59120) 2024-04-25T01:26:45.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.199+0000 7f4027fff640 1 -- 172.21.15.112:0/3928204385 learned_addr learned my addr 172.21.15.112:0/3928204385 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:45.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.199+0000 7f40277fe640 1 -- 172.21.15.112:0/3928204385 >> v2:172.21.15.187:3300/0 conn(0x7f4028153610 msgr2=0x7f402806df50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:26:45.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.199+0000 7f40277fe640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.187:3300/0 conn(0x7f4028153610 0x7f402806df50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:26:45.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.199+0000 7f402cd6b640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 0x7f402806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:45.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.199+0000 7f402cd6b640 1 -- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 msgr2=0x7f402806e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:26:45.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.199+0000 7f402cd6b640 1 -- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 msgr2=0x7f402806e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:26:45.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.199+0000 7f402cd6b640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 0x7f402806e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:26:45.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.199+0000 7f402cd6b640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 0x7f402806e490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:26:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.400+0000 7f40277fe640 1 -- 172.21.15.112:0/3928204385 >> v2:172.21.15.187:3300/0 conn(0x7f4028153610 msgr2=0x7f402806df50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:26:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.400+0000 7f40277fe640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.187:3300/0 conn(0x7f4028153610 0x7f402806df50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:26:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.400+0000 7f402cd6b640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 0x7f402806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.400+0000 7f402cd6b640 1 -- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 msgr2=0x7f402806e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:26:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.400+0000 7f402cd6b640 1 -- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 msgr2=0x7f402806e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:26:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.400+0000 7f402cd6b640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 0x7f402806e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:26:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.400+0000 7f402cd6b640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 0x7f402806e490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:26:45.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.801+0000 7f40277fe640 1 -- 172.21.15.112:0/3928204385 >> v2:172.21.15.187:3300/0 conn(0x7f4028153610 msgr2=0x7f402806df50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:26:45.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.801+0000 7f40277fe640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.187:3300/0 conn(0x7f4028153610 0x7f402806df50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:26:45.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.801+0000 7f402cd6b640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 0x7f402806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:45.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.801+0000 7f402cd6b640 1 -- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 msgr2=0x7f402806e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:26:45.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.801+0000 7f402cd6b640 1 -- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 msgr2=0x7f402806e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:26:45.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.801+0000 7f402cd6b640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 0x7f402806e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:26:45.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:45.801+0000 7f402cd6b640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 0x7f402806e490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:26:46.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:46.602+0000 7f40277fe640 1 -- 172.21.15.112:0/3928204385 >> v2:172.21.15.187:3300/0 conn(0x7f4028153610 msgr2=0x7f402806df50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:26:46.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:46.602+0000 7f40277fe640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.187:3300/0 conn(0x7f4028153610 0x7f402806df50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:26:46.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:46.602+0000 7f402cd6b640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 0x7f402806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:46.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:46.602+0000 7f402cd6b640 1 -- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 msgr2=0x7f402806e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:26:46.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:46.603+0000 7f402cd6b640 1 -- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 msgr2=0x7f402806e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:26:46.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:46.603+0000 7f402cd6b640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 0x7f402806e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:26:46.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:46.603+0000 7f402cd6b640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 0x7f402806e490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:26:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.199+0000 7f4025ffb640 1 -- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 msgr2=0x7f402806e490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:26:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.199+0000 7f4025ffb640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f402806d1a0 0x7f402806e490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.199+0000 7f4025ffb640 1 -- 172.21.15.112:0/3928204385 >> v2:172.21.15.187:3300/0 conn(0x7f4028153610 msgr2=0x7f402806df50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:26:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.199+0000 7f4025ffb640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.187:3300/0 conn(0x7f4028153610 0x7f402806df50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.199+0000 7f4025ffb640 1 -- 172.21.15.112:0/3928204385 >> v1:172.21.15.112:6789/0 conn(0x7f4028150ac0 legacy=0x7f40280a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.199+0000 7f4025ffb640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 0x7f4014009830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.199+0000 7f4025ffb640 1 -- 172.21.15.112:0/3928204385 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40280ab410 con 0x7f40140036c0 2024-04-25T01:26:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.199+0000 7f4025ffb640 1 -- 172.21.15.112:0/3928204385 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f402806e9d0 con 0x7f4014007370 2024-04-25T01:26:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.200+0000 7f40277fe640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 0x7f4014009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.200+0000 7f4027fff640 1 -- 172.21.15.112:0/3928204385 >> v1:172.21.15.187:6789/0 conn(0x7f40140036c0 legacy=0x7f4014005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.187:6789/0 2024-04-25T01:26:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.200+0000 7f40277fe640 1 -- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 msgr2=0x7f4014009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:26:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.200+0000 7f40277fe640 1 -- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 msgr2=0x7f4014009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:26:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.200+0000 7f40277fe640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 0x7f4014009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:26:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.200+0000 7f40277fe640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 0x7f4014009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:26:48.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.401+0000 7f40277fe640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 0x7f4014009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:48.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.401+0000 7f4027fff640 1 -- 172.21.15.112:0/3928204385 >> v1:172.21.15.187:6789/0 conn(0x7f40140036c0 legacy=0x7f4014005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.187:6789/0 2024-04-25T01:26:48.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.401+0000 7f40277fe640 1 -- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 msgr2=0x7f4014009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:26:48.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.401+0000 7f40277fe640 1 -- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 msgr2=0x7f4014009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:26:48.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.401+0000 7f40277fe640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 0x7f4014009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:26:48.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.401+0000 7f40277fe640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 0x7f4014009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:26:48.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.802+0000 7f40277fe640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 0x7f4014009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:48.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.802+0000 7f4027fff640 1 -- 172.21.15.112:0/3928204385 >> v1:172.21.15.187:6789/0 conn(0x7f40140036c0 legacy=0x7f4014005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.187:6789/0 2024-04-25T01:26:48.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.802+0000 7f40277fe640 1 -- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 msgr2=0x7f4014009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:26:48.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.802+0000 7f40277fe640 1 -- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 msgr2=0x7f4014009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:26:48.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.802+0000 7f40277fe640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 0x7f4014009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:26:48.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:48.802+0000 7f40277fe640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 0x7f4014009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:26:49.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:49.603+0000 7f4027fff640 1 -- 172.21.15.112:0/3928204385 >> v1:172.21.15.187:6789/0 conn(0x7f40140036c0 legacy=0x7f4014005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.187:6789/0 2024-04-25T01:26:49.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:49.603+0000 7f40277fe640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 0x7f4014009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:49.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:49.603+0000 7f40277fe640 1 -- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 msgr2=0x7f4014009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:26:49.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:49.603+0000 7f40277fe640 1 -- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 msgr2=0x7f4014009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:26:49.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:49.603+0000 7f40277fe640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 0x7f4014009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:26:49.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:49.603+0000 7f40277fe640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 0x7f4014009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:26:51.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f4025ffb640 1 -- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 msgr2=0x7f4014009830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:26:51.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f4025ffb640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.112:3300/0 conn(0x7f4014007370 0x7f4014009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f4025ffb640 1 -- 172.21.15.112:0/3928204385 >> v1:172.21.15.187:6789/0 conn(0x7f40140036c0 legacy=0x7f4014005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:26:51.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f402cd6b640 1 -- 172.21.15.112:0/3928204385 reap_dead start 2024-04-25T01:26:51.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f4025ffb640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.187:3300/0 conn(0x7f4028153610 0x7f4014009ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f4025ffb640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f401400a400 0x7f4014009830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f4025ffb640 1 -- 172.21.15.112:0/3928204385 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f401400afa0 con 0x7f401400a8f0 2024-04-25T01:26:51.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f4025ffb640 1 -- 172.21.15.112:0/3928204385 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f402806e9d0 con 0x7f4028153610 2024-04-25T01:26:51.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f4025ffb640 1 -- 172.21.15.112:0/3928204385 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f40280ac4b0 con 0x7f401400a400 2024-04-25T01:26:51.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f4027fff640 1 -- 172.21.15.112:0/3928204385 >> v2:172.21.15.187:3300/0 conn(0x7f4028153610 msgr2=0x7f4014009ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:26:51.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f4027fff640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.187:3300/0 conn(0x7f4028153610 0x7f4014009ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:26:51.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f40277fe640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f401400a400 0x7f4014009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f4026ffd640 1 -- 172.21.15.112:0/3928204385 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3298411800 0 0) 0x7f401400afa0 con 0x7f401400a8f0 2024-04-25T01:26:51.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f4026ffd640 1 -- 172.21.15.112:0/3928204385 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4010003520 con 0x7f401400a8f0 2024-04-25T01:26:51.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f4026ffd640 1 -- 172.21.15.112:0/3928204385 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3632664295 0 0) 0x7f4010003520 con 0x7f401400a8f0 2024-04-25T01:26:51.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f4026ffd640 1 -- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f401400a400 msgr2=0x7f4014009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:51.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f4026ffd640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f401400a400 0x7f4014009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f4026ffd640 1 -- 172.21.15.112:0/3928204385 >> v2:172.21.15.187:3300/0 conn(0x7f4028153610 msgr2=0x7f4014009ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:26:51.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f4026ffd640 1 --2- 172.21.15.112:0/3928204385 >> v2:172.21.15.187:3300/0 conn(0x7f4028153610 0x7f4014009ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f4026ffd640 1 -- 172.21.15.112:0/3928204385 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40280ab410 con 0x7f401400a8f0 2024-04-25T01:26:51.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.200+0000 7f4026ffd640 1 -- 172.21.15.112:0/3928204385 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f4020007670 con 0x7f401400a8f0 2024-04-25T01:26:51.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.201+0000 7f4026ffd640 1 -- 172.21.15.112:0/3928204385 >> v1:172.21.15.112:6789/0 conn(0x7f401400a8f0 legacy=0x7f4014005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.201+0000 7f4026ffd640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f401400a400 0x7f40100039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.201+0000 7f4026ffd640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4010003f60 0x7f4010004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.201+0000 7f4026ffd640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4010004d40 0x7f40100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.201+0000 7f4026ffd640 1 -- 172.21.15.112:0/3928204385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f402806e9d0 con 0x7f4010004d40 2024-04-25T01:26:51.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.201+0000 7f4026ffd640 1 -- 172.21.15.112:0/3928204385 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f40280ac4b0 con 0x7f4010003f60 2024-04-25T01:26:51.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.201+0000 7f4026ffd640 1 -- 172.21.15.112:0/3928204385 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f401000bb20 con 0x7f401400a400 2024-04-25T01:26:51.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.201+0000 7f40277fe640 1 -- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4010003f60 msgr2=0x7f4010004800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:26:51.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.201+0000 7f40277fe640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4010003f60 0x7f4010004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:26:51.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.201+0000 7f4027fff640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f401400a400 0x7f40100039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.201+0000 7f402cd6b640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4010004d40 0x7f40100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.201+0000 7f402cd6b640 1 -- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f401400a400 msgr2=0x7f40100039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:51.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.201+0000 7f402cd6b640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f401400a400 0x7f40100039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.201+0000 7f402cd6b640 1 -- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4010003f60 msgr2=0x7f4010004800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:26:51.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.201+0000 7f402cd6b640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4010003f60 0x7f4010004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.201+0000 7f402cd6b640 1 -- 172.21.15.112:0/3928204385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4010003520 con 0x7f4010004d40 2024-04-25T01:26:51.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.202+0000 7f402cd6b640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4010004d40 0x7f40100051c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f402806fae0 tx=0x7f4010005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1945eef817805743 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:51.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.202+0000 7f4026ffd640 1 -- 172.21.15.112:0/3928204385 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020031070 con 0x7f4010004d40 2024-04-25T01:26:51.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.202+0000 7f4026ffd640 1 -- 172.21.15.112:0/3928204385 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f402002f070 con 0x7f4010004d40 2024-04-25T01:26:51.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.202+0000 7f4026ffd640 1 -- 172.21.15.112:0/3928204385 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020007cf0 con 0x7f4010004d40 2024-04-25T01:26:51.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.203+0000 7f402e7ce640 1 -- 172.21.15.112:0/3928204385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4010004d40 msgr2=0x7f40100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.203+0000 7f402e7ce640 1 --2- 172.21.15.112:0/3928204385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4010004d40 0x7f40100051c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f402806fae0 tx=0x7f4010005830 comp rx=0 tx=0).stop 2024-04-25T01:26:51.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.203+0000 7f402cd6b640 1 -- 172.21.15.112:0/3928204385 reap_dead start 2024-04-25T01:26:51.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.203+0000 7f402e7ce640 1 -- 172.21.15.112:0/3928204385 shutdown_connections 2024-04-25T01:26:51.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.203+0000 7f402e7ce640 1 -- 172.21.15.112:0/3928204385 >> 172.21.15.112:0/3928204385 conn(0x7f40280c6f80 msgr2=0x7f40280c7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:51.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.203+0000 7f402e7ce640 1 -- 172.21.15.112:0/3928204385 shutdown_connections 2024-04-25T01:26:51.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.203+0000 7f402e7ce640 1 -- 172.21.15.112:0/3928204385 wait complete. 2024-04-25T01:26:51.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.205+0000 7f402e7ce640 1 Processor -- start 2024-04-25T01:26:51.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.205+0000 7f402e7ce640 1 -- start start 2024-04-25T01:26:51.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.206+0000 7f402e7ce640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40281523d0 0x7f40281efae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.206+0000 7f402e7ce640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4028071ed0 0x7f402814b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.206+0000 7f402e7ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f402814b800 0x7f4028147350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.206+0000 7f402e7ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40280abcb0 con 0x7f402814b800 2024-04-25T01:26:51.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.206+0000 7f402e7ce640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f40280ab9d0 con 0x7f4028071ed0 2024-04-25T01:26:51.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.206+0000 7f402e7ce640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f40280abf90 con 0x7f40281523d0 2024-04-25T01:26:51.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.206+0000 7f402cd6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f402814b800 0x7f4028147350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.206+0000 7f40277fe640 1 -- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4028071ed0 msgr2=0x7f402814b2c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:26:51.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.206+0000 7f40277fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4028071ed0 0x7f402814b2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:26:51.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.206+0000 7f402cd6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f402814b800 0x7f4028147350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33790/0 (socket says 172.21.15.112:33790) 2024-04-25T01:26:51.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.206+0000 7f402cd6b640 1 -- 172.21.15.112:0/4050545418 learned_addr learned my addr 172.21.15.112:0/4050545418 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:51.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.207+0000 7f4027fff640 1 --2- 172.21.15.112:0/4050545418 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40281523d0 0x7f40281efae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.207+0000 7f402cd6b640 1 -- 172.21.15.112:0/4050545418 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40281523d0 msgr2=0x7f40281efae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.207+0000 7f402cd6b640 1 --2- 172.21.15.112:0/4050545418 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40281523d0 0x7f40281efae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.207+0000 7f402cd6b640 1 -- 172.21.15.112:0/4050545418 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4028071ed0 msgr2=0x7f402814b2c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:26:51.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.207+0000 7f402cd6b640 1 --2- 172.21.15.112:0/4050545418 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4028071ed0 0x7f402814b2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.207+0000 7f402cd6b640 1 -- 172.21.15.112:0/4050545418 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4028146ac0 con 0x7f402814b800 2024-04-25T01:26:51.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.207+0000 7f4027fff640 1 --2- 172.21.15.112:0/4050545418 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40281523d0 0x7f40281efae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:51.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.207+0000 7f402cd6b640 1 --2- 172.21.15.112:0/4050545418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f402814b800 0x7f4028147350 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4014009e90 tx=0x7f4020004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:51.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.207+0000 7f4024ff9640 1 -- 172.21.15.112:0/4050545418 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020044070 con 0x7f402814b800 2024-04-25T01:26:51.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.207+0000 7f4024ff9640 1 -- 172.21.15.112:0/4050545418 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4020003e10 con 0x7f402814b800 2024-04-25T01:26:51.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.207+0000 7f402e7ce640 1 -- 172.21.15.112:0/4050545418 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4028146e70 con 0x7f402814b800 2024-04-25T01:26:51.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.207+0000 7f4024ff9640 1 -- 172.21.15.112:0/4050545418 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020007cf0 con 0x7f402814b800 2024-04-25T01:26:51.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.207+0000 7f402e7ce640 1 -- 172.21.15.112:0/4050545418 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f402814bc90 con 0x7f402814b800 2024-04-25T01:26:51.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.208+0000 7f402e7ce640 1 -- 172.21.15.112:0/4050545418 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4028151480 con 0x7f402814b800 2024-04-25T01:26:51.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.209+0000 7f4024ff9640 1 -- 172.21.15.112:0/4050545418 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4020047070 con 0x7f402814b800 2024-04-25T01:26:51.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.209+0000 7f4024ff9640 1 --2- 172.21.15.112:0/4050545418 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f401406d250 0x7f401406f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.209+0000 7f4024ff9640 1 -- 172.21.15.112:0/4050545418 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6637+0+0 (secure 0 0 0) 0x7f40200ab170 con 0x7f402814b800 2024-04-25T01:26:51.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.209+0000 7f4027fff640 1 --2- 172.21.15.112:0/4050545418 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f401406d250 0x7f401406f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.210+0000 7f4027fff640 1 --2- 172.21.15.112:0/4050545418 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f401406d250 0x7f401406f710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4018000c00 tx=0x7f4018009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:51.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.214+0000 7f4024ff9640 1 -- 172.21.15.112:0/4050545418 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4028151480 con 0x7f402814b800 2024-04-25T01:26:51.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.458+0000 7f402e7ce640 1 -- 172.21.15.112:0/4050545418 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4028147040 con 0x7f402814b800 2024-04-25T01:26:51.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.459+0000 7f4024ff9640 1 -- 172.21.15.112:0/4050545418 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f402004d060 con 0x7f402814b800 2024-04-25T01:26:51.438 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":6,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:26:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.461+0000 7f402e7ce640 1 -- 172.21.15.112:0/4050545418 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f401406d250 msgr2=0x7f401406f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.461+0000 7f402e7ce640 1 --2- 172.21.15.112:0/4050545418 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f401406d250 0x7f401406f710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4018000c00 tx=0x7f4018009040 comp rx=0 tx=0).stop 2024-04-25T01:26:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.461+0000 7f402e7ce640 1 -- 172.21.15.112:0/4050545418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f402814b800 msgr2=0x7f4028147350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.461+0000 7f402e7ce640 1 --2- 172.21.15.112:0/4050545418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f402814b800 0x7f4028147350 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4014009e90 tx=0x7f4020004a40 comp rx=0 tx=0).stop 2024-04-25T01:26:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.461+0000 7f402e7ce640 1 -- 172.21.15.112:0/4050545418 shutdown_connections 2024-04-25T01:26:51.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.461+0000 7f402e7ce640 1 --2- 172.21.15.112:0/4050545418 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f401406d250 0x7f401406f710 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.461+0000 7f402e7ce640 1 --2- 172.21.15.112:0/4050545418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f402814b800 0x7f4028147350 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.461+0000 7f402e7ce640 1 --2- 172.21.15.112:0/4050545418 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4028071ed0 0x7f402814b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.461+0000 7f402e7ce640 1 --2- 172.21.15.112:0/4050545418 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40281523d0 0x7f40281efae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.461+0000 7f402e7ce640 1 -- 172.21.15.112:0/4050545418 >> 172.21.15.112:0/4050545418 conn(0x7f40280c6f80 msgr2=0x7f40280c1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:51.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.461+0000 7f402e7ce640 1 -- 172.21.15.112:0/4050545418 shutdown_connections 2024-04-25T01:26:51.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.461+0000 7f402e7ce640 1 -- 172.21.15.112:0/4050545418 wait complete. 2024-04-25T01:26:51.451 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T01:26:51.451 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-04-25T01:26:51.451 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:26:51.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.611+0000 7f166cba9640 1 Processor -- start 2024-04-25T01:26:51.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.611+0000 7f166cba9640 1 -- start start 2024-04-25T01:26:51.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.612+0000 7f166cba9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f16680c3050 0x7f16680c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.612+0000 7f166cba9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1668151d60 0x7f16681521f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.612+0000 7f166cba9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16680abd70 con 0x7f1668071770 2024-04-25T01:26:51.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.612+0000 7f166cba9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f16680aa5b0 con 0x7f1668151d60 2024-04-25T01:26:51.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.612+0000 7f166cba9640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f16680ac100 con 0x7f16680c3050 2024-04-25T01:26:51.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.612+0000 7f1666d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1668151d60 0x7f16681521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.612+0000 7f165ffff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1668071770 0x7f16680c1fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51632/0 (socket says 172.21.15.112:51632) 2024-04-25T01:26:51.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.612+0000 7f165ffff640 1 -- 172.21.15.112:0/102641126 learned_addr learned my addr 172.21.15.112:0/102641126 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:51.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.612+0000 7f166659c640 1 -- 172.21.15.112:0/102641126 >> v2:172.21.15.187:3300/0 conn(0x7f16680c3050 msgr2=0x7f16680c1a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:26:51.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.612+0000 7f166659c640 1 --2- 172.21.15.112:0/102641126 >> v2:172.21.15.187:3300/0 conn(0x7f16680c3050 0x7f16680c1a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:26:51.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.612+0000 7f1666d9d640 1 -- 172.21.15.112:0/102641126 >> v2:172.21.15.187:3300/0 conn(0x7f16680c3050 msgr2=0x7f16680c1a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:26:51.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.612+0000 7f1666d9d640 1 --2- 172.21.15.112:0/102641126 >> v2:172.21.15.187:3300/0 conn(0x7f16680c3050 0x7f16680c1a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.612+0000 7f1666d9d640 1 -- 172.21.15.112:0/102641126 >> v1:172.21.15.112:6789/0 conn(0x7f1668071770 legacy=0x7f16680c1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.612+0000 7f1665d9b640 1 -- 172.21.15.112:0/102641126 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4143433123 0 0) 0x7f16680abd70 con 0x7f1668071770 2024-04-25T01:26:51.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.612+0000 7f1666d9d640 1 -- 172.21.15.112:0/102641126 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16680ac3e0 con 0x7f1668151d60 2024-04-25T01:26:51.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.613+0000 7f1666d9d640 1 --2- 172.21.15.112:0/102641126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1668151d60 0x7f16681521f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f16680c1950 tx=0x7f166002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15df257a82aedcff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:51.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.613+0000 7f1665d9b640 1 -- 172.21.15.112:0/102641126 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1660007740 con 0x7f1668151d60 2024-04-25T01:26:51.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.613+0000 7f1665d9b640 1 -- 172.21.15.112:0/102641126 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16600078e0 con 0x7f1668151d60 2024-04-25T01:26:51.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.613+0000 7f1665d9b640 1 -- 172.21.15.112:0/102641126 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1660007cb0 con 0x7f1668151d60 2024-04-25T01:26:51.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.613+0000 7f166cba9640 1 -- 172.21.15.112:0/102641126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1668151d60 msgr2=0x7f16681521f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.613+0000 7f166cba9640 1 --2- 172.21.15.112:0/102641126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1668151d60 0x7f16681521f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f16680c1950 tx=0x7f166002a610 comp rx=0 tx=0).stop 2024-04-25T01:26:51.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.613+0000 7f166cba9640 1 -- 172.21.15.112:0/102641126 shutdown_connections 2024-04-25T01:26:51.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.613+0000 7f166cba9640 1 --2- 172.21.15.112:0/102641126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1668151d60 0x7f16681521f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.613+0000 7f166cba9640 1 --2- 172.21.15.112:0/102641126 >> v2:172.21.15.187:3300/0 conn(0x7f16680c3050 0x7f16680c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.613+0000 7f166cba9640 1 -- 172.21.15.112:0/102641126 >> 172.21.15.112:0/102641126 conn(0x7f166806b160 msgr2=0x7f166806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:51.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.613+0000 7f166cba9640 1 -- 172.21.15.112:0/102641126 shutdown_connections 2024-04-25T01:26:51.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.613+0000 7f166cba9640 1 -- 172.21.15.112:0/102641126 wait complete. 2024-04-25T01:26:51.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.615+0000 7f166cba9640 1 Processor -- start 2024-04-25T01:26:51.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.615+0000 7f166cba9640 1 -- start start 2024-04-25T01:26:51.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.615+0000 7f166cba9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1668071770 0x7f1668143ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.615+0000 7f166cba9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16680c3050 0x7f1668149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.615+0000 7f166cba9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1668151d60 0x7f166814a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.615+0000 7f166cba9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1668152730 con 0x7f1668071770 2024-04-25T01:26:51.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.615+0000 7f166cba9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f16680ab850 con 0x7f16680c3050 2024-04-25T01:26:51.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.615+0000 7f166cba9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f16681528b0 con 0x7f1668151d60 2024-04-25T01:26:51.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.615+0000 7f166659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1668071770 0x7f1668143ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.615+0000 7f166659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1668071770 0x7f1668143ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33814/0 (socket says 172.21.15.112:33814) 2024-04-25T01:26:51.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.615+0000 7f166659c640 1 -- 172.21.15.112:0/2703299261 learned_addr learned my addr 172.21.15.112:0/2703299261 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:51.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.615+0000 7f165ffff640 1 -- 172.21.15.112:0/2703299261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16680c3050 msgr2=0x7f1668149e70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:26:51.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.616+0000 7f165ffff640 1 --2- 172.21.15.112:0/2703299261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16680c3050 0x7f1668149e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:26:51.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.616+0000 7f166659c640 1 -- 172.21.15.112:0/2703299261 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1668151d60 msgr2=0x7f166814a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.616+0000 7f166659c640 1 --2- 172.21.15.112:0/2703299261 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1668151d60 0x7f166814a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.616+0000 7f166659c640 1 -- 172.21.15.112:0/2703299261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16680c3050 msgr2=0x7f1668149e70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:26:51.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.616+0000 7f166659c640 1 --2- 172.21.15.112:0/2703299261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16680c3050 0x7f1668149e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.616+0000 7f166659c640 1 -- 172.21.15.112:0/2703299261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166814ad90 con 0x7f1668071770 2024-04-25T01:26:51.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.616+0000 7f166659c640 1 --2- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1668071770 0x7f1668143ee0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f165800b550 tx=0x7f165800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:51.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.616+0000 7f165effd640 1 -- 172.21.15.112:0/2703299261 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1658013020 con 0x7f1668071770 2024-04-25T01:26:51.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.616+0000 7f165effd640 1 -- 172.21.15.112:0/2703299261 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1658003de0 con 0x7f1668071770 2024-04-25T01:26:51.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.616+0000 7f165effd640 1 -- 172.21.15.112:0/2703299261 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16580052b0 con 0x7f1668071770 2024-04-25T01:26:51.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.616+0000 7f166cba9640 1 -- 172.21.15.112:0/2703299261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1668142c10 con 0x7f1668071770 2024-04-25T01:26:51.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.616+0000 7f166cba9640 1 -- 172.21.15.112:0/2703299261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f166814b030 con 0x7f1668071770 2024-04-25T01:26:51.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.617+0000 7f166cba9640 1 --2- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16680c7d80 0x7f166806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.617+0000 7f166cba9640 1 -- 172.21.15.112:0/2703299261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1668075980 con 0x7f16680c7d80 2024-04-25T01:26:51.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.617+0000 7f165ffff640 1 --2- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16680c7d80 0x7f166806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.617+0000 7f165ffff640 1 -- 172.21.15.112:0/2703299261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1668142540 con 0x7f16680c7d80 2024-04-25T01:26:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.617+0000 7f165ffff640 1 --2- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16680c7d80 0x7f166806d360 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f16681431e0 tx=0x7f165000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.618+0000 7f165effd640 1 -- 172.21.15.112:0/2703299261 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1650017070 con 0x7f16680c7d80 2024-04-25T01:26:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.618+0000 7f165effd640 1 -- 172.21.15.112:0/2703299261 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1658016020 con 0x7f1668071770 2024-04-25T01:26:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.618+0000 7f165effd640 1 --2- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f164806bd30 0x7f164806e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.618+0000 7f1666d9d640 1 --2- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f164806bd30 0x7f164806e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.618+0000 7f165effd640 1 -- 172.21.15.112:0/2703299261 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6637+0+0 (secure 0 0 0) 0x7f165808aea0 con 0x7f1668071770 2024-04-25T01:26:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.618+0000 7f165effd640 1 -- 172.21.15.112:0/2703299261 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f16500074e0 con 0x7f16680c7d80 2024-04-25T01:26:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.618+0000 7f165effd640 1 -- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16680c7d80 msgr2=0x7f166806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.618+0000 7f165effd640 1 --2- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16680c7d80 0x7f166806d360 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f16681431e0 tx=0x7f165000c8e0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.618+0000 7f165effd640 1 -- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16680c7d80 msgr2=0x7f166806d360 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:26:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.618+0000 7f165effd640 1 --2- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16680c7d80 0x7f166806d360 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.618+0000 7f1666d9d640 1 --2- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f164806bd30 0x7f164806e1f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1660034000 tx=0x7f166002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:51.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.625+0000 7f166cba9640 1 --2- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16680c0f80 0x7f166806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.625+0000 7f166cba9640 1 -- 172.21.15.112:0/2703299261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16680aa7a0 con 0x7f16680c0f80 2024-04-25T01:26:51.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.625+0000 7f165ffff640 1 --2- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16680c0f80 0x7f166806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.625+0000 7f165ffff640 1 -- 172.21.15.112:0/2703299261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f166806b6c0 con 0x7f16680c0f80 2024-04-25T01:26:51.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.625+0000 7f165ffff640 1 --2- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16680c0f80 0x7f166806d8a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1650007fd0 tx=0x7f165000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:51.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.625+0000 7f165effd640 1 -- 172.21.15.112:0/2703299261 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165001be60 con 0x7f16680c0f80 2024-04-25T01:26:51.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.625+0000 7f165effd640 1 -- 172.21.15.112:0/2703299261 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5058 (secure 0 0 0) 0x7f1650012400 con 0x7f16680c0f80 2024-04-25T01:26:51.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.625+0000 7f165effd640 1 -- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16680c0f80 msgr2=0x7f166806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.625+0000 7f165effd640 1 --2- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16680c0f80 0x7f166806d8a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1650007fd0 tx=0x7f165000f000 comp rx=0 tx=0).stop 2024-04-25T01:26:51.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.625+0000 7f165effd640 1 -- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16680c0f80 msgr2=0x7f166806d8a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:26:51.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.625+0000 7f165effd640 1 --2- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16680c0f80 0x7f166806d8a0 secure :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1650007fd0 tx=0x7f165000f000 comp rx=0 tx=0).stop 2024-04-25T01:26:51.606 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:26:51.606 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:26:51.606 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:26:51.606 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:26:51.606 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 6, 2024-04-25T01:26:51.606 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 83253, 2024-04-25T01:26:51.606 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:26:51.606 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:26:51.606 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:26:51.606 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:51.606 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:26:51.607 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:26:51.607 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:26:51.607 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:26:51.607 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:26:51.607 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:26:51.607 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:26:51.607 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:26:51.607 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:26:51.607 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:26:51.607 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:26:51.607 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:26:51.607 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:26:51.607 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:26:51.607 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:26:51.608 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:51.608 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:26:51.608 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:26:51.608 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:26:51.608 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:26:51.608 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:26:51.608 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:26:51.608 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:26:51.608 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:26:51.608 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:26:51.608 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:26:51.608 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:26:51.608 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:26:51.608 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:26:51.608 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:51.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.609 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:26:51.609 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:26:51.609 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:26:51.609 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:26:51.609 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:26:51.609 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:26:51.609 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:26:51.609 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:26:51.609 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:26:51.609 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:26:51.609 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:26:51.609 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:26:51.609 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:26:51.609 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:26:51.610 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:26:51.610 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:26:51.610 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:26:51.610 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:26:51.610 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:26:51.610 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:26:51.610 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:26:51.610 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:26:51.610 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:26:51.610 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:26:51.610 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:26:51.610 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:26:51.610 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:26:51.610 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:26:51.610 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:51.611 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:26:51.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.611 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:26:51.611 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.611 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:26:51.611 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:26:51.611 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:26:51.611 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:26:51.611 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.611 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:26:51.611 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:26:51.611 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:51.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.611 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.611 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:26:51.612 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:26:51.612 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:51.612 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:51.612 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:51.612 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.612 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:26:51.612 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:26:51.612 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:26:51.612 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:26:51.612 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:26:51.612 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.612 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.612 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:26:51.612 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:26:51.612 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:26:51.613 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:26:51.613 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.613 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:26:51.613 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:26:51.613 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:51.613 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.613 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.613 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:26:51.613 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:26:51.613 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:51.613 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:51.613 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:51.613 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.613 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:26:51.613 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:26:51.614 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:26:51.614 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:26:51.614 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:26:51.614 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.614 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.614 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:26:51.614 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:26:51.614 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:26:51.614 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:26:51.614 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.614 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:26:51.614 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:26:51.614 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:51.614 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.615 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.615 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:26:51.615 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:26:51.615 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:51.616 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:51.616 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:51.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.616 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:26:51.616 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:26:51.616 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:26:51.617 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:26:51.617 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:26:51.617 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:51.617 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:51.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.617 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:26:51.617 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:26:51.617 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.617 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:51.617 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:51.617 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:26:51.617 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:51.618 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:51.618 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:26:51.618 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.618 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:51.618 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:51.618 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:26:51.618 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:51.618 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:51.618 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:26:51.618 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.618 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:51.618 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:51.618 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:26:51.618 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:51.619 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:51.619 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:26:51.619 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.619 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:51.619 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:51.619 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 7 2024-04-25T01:26:51.619 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:51.619 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:51.619 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:26:51.619 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.619 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:51.619 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:51.619 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:26:51.619 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:51.619 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:51.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.620 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:26:51.620 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:26:51.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.626+0000 7f166cba9640 1 -- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f164806bd30 msgr2=0x7f164806e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.626+0000 7f166cba9640 1 --2- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f164806bd30 0x7f164806e1f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1660034000 tx=0x7f166002e000 comp rx=0 tx=0).stop 2024-04-25T01:26:51.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.626+0000 7f166cba9640 1 -- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1668071770 msgr2=0x7f1668143ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.626+0000 7f166cba9640 1 --2- 172.21.15.112:0/2703299261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1668071770 0x7f1668143ee0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f165800b550 tx=0x7f165800ba10 comp rx=0 tx=0).stop 2024-04-25T01:26:51.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.626+0000 7f1666d9d640 1 -- 172.21.15.112:0/2703299261 reap_dead start 2024-04-25T01:26:51.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.626+0000 7f1666d9d640 1 -- 172.21.15.112:0/2703299261 reap_dead start 2024-04-25T01:26:51.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.626+0000 7f166cba9640 1 -- 172.21.15.112:0/2703299261 shutdown_connections 2024-04-25T01:26:51.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.626+0000 7f166cba9640 1 -- 172.21.15.112:0/2703299261 >> 172.21.15.112:0/2703299261 conn(0x7f166806b160 msgr2=0x7f1668150b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:51.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.626+0000 7f166cba9640 1 -- 172.21.15.112:0/2703299261 shutdown_connections 2024-04-25T01:26:51.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.626+0000 7f166cba9640 1 -- 172.21.15.112:0/2703299261 wait complete. 2024-04-25T01:26:51.622 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:26:51.622 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-04-25T01:26:51.622 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:26:51.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.741+0000 7fb5ee027640 1 Processor -- start 2024-04-25T01:26:51.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.741+0000 7fb5ee027640 1 -- start start 2024-04-25T01:26:51.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.741+0000 7fb5ee027640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb5e80ac3b0 0x7fb5e8155f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.741+0000 7fb5ee027640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e81524a0 0x7fb5e8156460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.741+0000 7fb5ee027640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5e81529b0 con 0x7fb5e814c3f0 2024-04-25T01:26:51.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.741+0000 7fb5ee027640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fb5e8155a90 con 0x7fb5e80ac3b0 2024-04-25T01:26:51.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.741+0000 7fb5ee027640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e81569a0 con 0x7fb5e81524a0 2024-04-25T01:26:51.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.742+0000 7fb5e6ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb5e80ac3b0 0x7fb5e8155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.742+0000 7fb5e77fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb5e814c3f0 0x7fb5e80c2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:51648/0 (socket says 172.21.15.112:51648) 2024-04-25T01:26:51.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.742+0000 7fb5e77fe640 1 -- 172.21.15.112:0/3822981635 learned_addr learned my addr 172.21.15.112:0/3822981635 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:26:51.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.742+0000 7fb5e7fff640 1 --2- 172.21.15.112:0/3822981635 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e81524a0 0x7fb5e8156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.742+0000 7fb5e6ffd640 1 -- 172.21.15.112:0/3822981635 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e81524a0 msgr2=0x7fb5e8156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:26:51.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.742+0000 7fb5e6ffd640 1 --2- 172.21.15.112:0/3822981635 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e81524a0 0x7fb5e8156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.742+0000 7fb5e6ffd640 1 -- 172.21.15.112:0/3822981635 >> v1:172.21.15.112:6789/0 conn(0x7fb5e814c3f0 legacy=0x7fb5e80c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.742+0000 7fb5e67fc640 1 -- 172.21.15.112:0/3822981635 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 722301127 0 0) 0x7fb5e81529b0 con 0x7fb5e814c3f0 2024-04-25T01:26:51.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.742+0000 7fb5e6ffd640 1 -- 172.21.15.112:0/3822981635 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e8152c70 con 0x7fb5e80ac3b0 2024-04-25T01:26:51.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.742+0000 7fb5e7fff640 1 --2- 172.21.15.112:0/3822981635 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e81524a0 0x7fb5e8156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:26:51.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.743+0000 7fb5e6ffd640 1 --2- 172.21.15.112:0/3822981635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5e80ac3b0 0x7fb5e8155f20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0097a0 tx=0x7fb5dc02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed80eee1a1be1ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:51.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.743+0000 7fb5e67fc640 1 -- 172.21.15.112:0/3822981635 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc037070 con 0x7fb5e80ac3b0 2024-04-25T01:26:51.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.743+0000 7fb5e67fc640 1 -- 172.21.15.112:0/3822981635 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5dc02ae50 con 0x7fb5e80ac3b0 2024-04-25T01:26:51.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.743+0000 7fb5e67fc640 1 -- 172.21.15.112:0/3822981635 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc032660 con 0x7fb5e80ac3b0 2024-04-25T01:26:51.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.743+0000 7fb5ee027640 1 -- 172.21.15.112:0/3822981635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5e80ac3b0 msgr2=0x7fb5e8155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.743+0000 7fb5ee027640 1 --2- 172.21.15.112:0/3822981635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5e80ac3b0 0x7fb5e8155f20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0097a0 tx=0x7fb5dc02a100 comp rx=0 tx=0).stop 2024-04-25T01:26:51.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.743+0000 7fb5ee027640 1 -- 172.21.15.112:0/3822981635 shutdown_connections 2024-04-25T01:26:51.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.743+0000 7fb5ee027640 1 --2- 172.21.15.112:0/3822981635 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e81524a0 0x7fb5e8156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.743+0000 7fb5ee027640 1 --2- 172.21.15.112:0/3822981635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5e80ac3b0 0x7fb5e8155f20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.743+0000 7fb5ee027640 1 -- 172.21.15.112:0/3822981635 >> 172.21.15.112:0/3822981635 conn(0x7fb5e806b160 msgr2=0x7fb5e806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:51.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.743+0000 7fb5ee027640 1 -- 172.21.15.112:0/3822981635 shutdown_connections 2024-04-25T01:26:51.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.743+0000 7fb5ee027640 1 -- 172.21.15.112:0/3822981635 wait complete. 2024-04-25T01:26:51.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.744+0000 7fb5ee027640 1 Processor -- start 2024-04-25T01:26:51.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.745+0000 7fb5ee027640 1 -- start start 2024-04-25T01:26:51.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.745+0000 7fb5ee027640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5e80ac3b0 0x7fb5e8141d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.745+0000 7fb5ee027640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5e814c3f0 0x7fb5e80c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.745+0000 7fb5ee027640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e81524a0 0x7fb5e81eb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.745+0000 7fb5ee027640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e8070d60 con 0x7fb5e814c3f0 2024-04-25T01:26:51.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.745+0000 7fb5ee027640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e8153230 con 0x7fb5e80ac3b0 2024-04-25T01:26:51.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.745+0000 7fb5ee027640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e80c32a0 con 0x7fb5e81524a0 2024-04-25T01:26:51.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.745+0000 7fb5e6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5e814c3f0 0x7fb5e80c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.745+0000 7fb5e6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5e814c3f0 0x7fb5e80c1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33862/0 (socket says 172.21.15.112:33862) 2024-04-25T01:26:51.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.745+0000 7fb5e6ffd640 1 -- 172.21.15.112:0/3802404689 learned_addr learned my addr 172.21.15.112:0/3802404689 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:26:51.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.745+0000 7fb5e77fe640 1 -- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5e80ac3b0 msgr2=0x7fb5e8141d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:26:51.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.745+0000 7fb5e77fe640 1 --2- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5e80ac3b0 0x7fb5e8141d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:26:51.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.746+0000 7fb5e7fff640 1 --2- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e81524a0 0x7fb5e81eb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.746+0000 7fb5e6ffd640 1 -- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e81524a0 msgr2=0x7fb5e81eb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.746+0000 7fb5e6ffd640 1 --2- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e81524a0 0x7fb5e81eb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.746+0000 7fb5e6ffd640 1 -- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5e80ac3b0 msgr2=0x7fb5e8141d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:26:51.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.746+0000 7fb5e6ffd640 1 --2- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5e80ac3b0 0x7fb5e8141d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.746+0000 7fb5e6ffd640 1 -- 172.21.15.112:0/3802404689 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e81ee860 con 0x7fb5e814c3f0 2024-04-25T01:26:51.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.746+0000 7fb5e6ffd640 1 --2- 172.21.15.112:0/3802404689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5e814c3f0 0x7fb5e80c1a80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0098c0 tx=0x7fb5dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:51.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.746+0000 7fb5cffff640 1 -- 172.21.15.112:0/3802404689 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc041030 con 0x7fb5e814c3f0 2024-04-25T01:26:51.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.746+0000 7fb5cffff640 1 -- 172.21.15.112:0/3802404689 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5dc02ac50 con 0x7fb5e814c3f0 2024-04-25T01:26:51.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.746+0000 7fb5ee027640 1 -- 172.21.15.112:0/3802404689 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e81eeb40 con 0x7fb5e814c3f0 2024-04-25T01:26:51.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.746+0000 7fb5cffff640 1 -- 172.21.15.112:0/3802404689 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc037040 con 0x7fb5e814c3f0 2024-04-25T01:26:51.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.746+0000 7fb5ee027640 1 -- 172.21.15.112:0/3802404689 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e81ef100 con 0x7fb5e814c3f0 2024-04-25T01:26:51.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.747+0000 7fb5ee027640 1 --2- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e8146bc0 0x7fb5e80aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.747+0000 7fb5ee027640 1 -- 172.21.15.112:0/3802404689 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e81ec010 con 0x7fb5e8146bc0 2024-04-25T01:26:51.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.747+0000 7fb5e77fe640 1 --2- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e8146bc0 0x7fb5e80aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.747+0000 7fb5cffff640 1 -- 172.21.15.112:0/3802404689 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb5dc044070 con 0x7fb5e814c3f0 2024-04-25T01:26:51.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.747+0000 7fb5e77fe640 1 -- 172.21.15.112:0/3802404689 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb5e8155040 con 0x7fb5e8146bc0 2024-04-25T01:26:51.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.748+0000 7fb5e77fe640 1 --2- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e8146bc0 0x7fb5e80aaf90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8004870 tx=0x7fb5d800c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:51.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.748+0000 7fb5cffff640 1 --2- 172.21.15.112:0/3802404689 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb5d406be10 0x7fb5d406e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.748+0000 7fb5cffff640 1 -- 172.21.15.112:0/3802404689 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6637+0+0 (secure 0 0 0) 0x7fb5dc0ab7b0 con 0x7fb5e814c3f0 2024-04-25T01:26:51.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.748+0000 7fb5e7fff640 1 --2- 172.21.15.112:0/3802404689 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb5d406be10 0x7fb5d406e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.748+0000 7fb5cffff640 1 -- 172.21.15.112:0/3802404689 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8017070 con 0x7fb5e8146bc0 2024-04-25T01:26:51.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.748+0000 7fb5cffff640 1 -- 172.21.15.112:0/3802404689 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb5d8004cb0 con 0x7fb5e8146bc0 2024-04-25T01:26:51.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.748+0000 7fb5cffff640 1 -- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e8146bc0 msgr2=0x7fb5e80aaf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.748+0000 7fb5cffff640 1 --2- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e8146bc0 0x7fb5e80aaf90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8004870 tx=0x7fb5d800c8c0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.748+0000 7fb5e7fff640 1 --2- 172.21.15.112:0/3802404689 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb5d406be10 0x7fb5d406e2d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0008e60 tx=0x7fb5e0008390 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:51.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.748+0000 7fb5cffff640 1 -- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e8146bc0 msgr2=0x7fb5e80aaf90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:26:51.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.748+0000 7fb5cffff640 1 --2- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e8146bc0 0x7fb5e80aaf90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.754+0000 7fb5ee027640 1 --2- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e80ab4d0 0x7fb5e80c04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.754+0000 7fb5ee027640 1 -- 172.21.15.112:0/3802404689 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e8075980 con 0x7fb5e80ab4d0 2024-04-25T01:26:51.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.755+0000 7fb5e77fe640 1 --2- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e80ab4d0 0x7fb5e80c04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.755+0000 7fb5e77fe640 1 -- 172.21.15.112:0/3802404689 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb5e80ad420 con 0x7fb5e80ab4d0 2024-04-25T01:26:51.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.755+0000 7fb5e77fe640 1 --2- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e80ab4d0 0x7fb5e80c04b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8014000 tx=0x7fb5d800f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:51.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.755+0000 7fb5cffff640 1 -- 172.21.15.112:0/3802404689 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d801b9c0 con 0x7fb5e80ab4d0 2024-04-25T01:26:51.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.756+0000 7fb5cffff640 1 -- 172.21.15.112:0/3802404689 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5056 (secure 0 0 0) 0x7fb5d801b9c0 con 0x7fb5e80ab4d0 2024-04-25T01:26:51.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.756+0000 7fb5cffff640 1 -- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e80ab4d0 msgr2=0x7fb5e80c04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.756+0000 7fb5cffff640 1 --2- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e80ab4d0 0x7fb5e80c04b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8014000 tx=0x7fb5d800f000 comp rx=0 tx=0).stop 2024-04-25T01:26:51.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.756+0000 7fb5cffff640 1 -- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e80ab4d0 msgr2=0x7fb5e80c04b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:26:51.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.756+0000 7fb5cffff640 1 --2- 172.21.15.112:0/3802404689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5e80ab4d0 0x7fb5e80c04b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.736 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:26:51.736 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:26:51.736 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:26:51.736 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:26:51.736 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 6, 2024-04-25T01:26:51.736 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 83343, 2024-04-25T01:26:51.736 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:26:51.736 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:26:51.736 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:26:51.736 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:51.736 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:26:51.736 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:26:51.736 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:26:51.738 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:26:51.738 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:26:51.738 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:26:51.738 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:26:51.738 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:26:51.738 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:26:51.738 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:26:51.738 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:26:51.738 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:26:51.738 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:26:51.738 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:51.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.738 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:26:51.738 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:26:51.740 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:26:51.740 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:26:51.740 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:26:51.740 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:26:51.740 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:26:51.740 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:26:51.740 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:26:51.740 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:26:51.740 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:26:51.740 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:26:51.740 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:51.740 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:26:51.740 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.740 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:51.742 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:51.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.742 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:26:51.742 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:26:51.742 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:26:51.742 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:26:51.742 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:26:51.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.742 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.742 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:26:51.742 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:26:51.742 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:26:51.742 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:26:51.742 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.742 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:26:51.743 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:26:51.743 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:51.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.743 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.743 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:26:51.743 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:26:51.743 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:51.743 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:51.743 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:51.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.743 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:26:51.743 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:26:51.743 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:26:51.743 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:26:51.743 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:26:51.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.744 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.744 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:26:51.744 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:26:51.744 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:26:51.744 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:26:51.744 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.744 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:26:51.744 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:26:51.744 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:51.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.744 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.744 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:26:51.744 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:26:51.744 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:51.746 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:51.746 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:26:51.746 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:51.746 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:51.746 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:26:51.746 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.746 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:51.746 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:51.746 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:26:51.746 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:51.746 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:51.746 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:26:51.746 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.746 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:51.746 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:51.747 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:26:51.747 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:51.747 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:51.747 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:26:51.747 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.747 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:51.747 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:51.747 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:26:51.747 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:51.747 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:26:51.747 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:26:51.747 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:26:51.747 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:26:51.747 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:26:51.748 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:26:51.748 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:26:51.748 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:26:51.748 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:26:51.748 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:26:51.748 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:26:51.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.756+0000 7fb5ee027640 1 -- 172.21.15.112:0/3802404689 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb5d406be10 msgr2=0x7fb5d406e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.756+0000 7fb5ee027640 1 --2- 172.21.15.112:0/3802404689 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb5d406be10 0x7fb5d406e2d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0008e60 tx=0x7fb5e0008390 comp rx=0 tx=0).stop 2024-04-25T01:26:51.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.756+0000 7fb5ee027640 1 -- 172.21.15.112:0/3802404689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5e814c3f0 msgr2=0x7fb5e80c1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.756+0000 7fb5ee027640 1 --2- 172.21.15.112:0/3802404689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5e814c3f0 0x7fb5e80c1a80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0098c0 tx=0x7fb5dc0046c0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.756+0000 7fb5e7fff640 1 -- 172.21.15.112:0/3802404689 reap_dead start 2024-04-25T01:26:51.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.756+0000 7fb5e7fff640 1 -- 172.21.15.112:0/3802404689 reap_dead start 2024-04-25T01:26:51.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.756+0000 7fb5ee027640 1 -- 172.21.15.112:0/3802404689 shutdown_connections 2024-04-25T01:26:51.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.756+0000 7fb5ee027640 1 -- 172.21.15.112:0/3802404689 >> 172.21.15.112:0/3802404689 conn(0x7fb5e806b160 msgr2=0x7fb5e80c2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:51.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.756+0000 7fb5ee027640 1 -- 172.21.15.112:0/3802404689 shutdown_connections 2024-04-25T01:26:51.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:26:51.756+0000 7fb5ee027640 1 -- 172.21.15.112:0/3802404689 wait complete. 2024-04-25T01:26:51.750 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:26:51.750 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T01:26:53.233 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:26:53.236 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:27:11.750 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-04-25T01:27:11.751 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-04-25T01:27:11.751 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T01:27:11.751 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T01:27:11.754 INFO:tasks.ceph.mon.b:Started 2024-04-25T01:27:11.754 INFO:tasks.mon_thrash:Sending STOP to mon c 2024-04-25T01:27:11.754 INFO:tasks.ceph.mon.c:Sent signal 19 2024-04-25T01:27:11.755 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-04-25T01:27:23.375 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:27:23.377 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:27:26.755 INFO:tasks.mon_thrash:Sending CONT to mon c 2024-04-25T01:27:26.755 INFO:tasks.ceph.mon.c:Sent signal 18 2024-04-25T01:27:26.756 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:27:26.756 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:27:26.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.921+0000 7f2794590640 1 Processor -- start 2024-04-25T01:27:26.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.921+0000 7f2794590640 1 -- start start 2024-04-25T01:27:26.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.922+0000 7f2794590640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c06dab0 0x7f278c14c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.922+0000 7f2794590640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f278c14c5c0 0x7f278c153a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.922+0000 7f2794590640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f278c06dfb0 con 0x7f278c155d80 2024-04-25T01:27:26.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.922+0000 7f2794590640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f278c156870 con 0x7f278c14c5c0 2024-04-25T01:27:26.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.922+0000 7f2794590640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f278c06ebc0 con 0x7f278c06dab0 2024-04-25T01:27:26.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.922+0000 7f2792b2d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f278c14c5c0 0x7f278c153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.922+0000 7f2792b2d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f278c14c5c0 0x7f278c153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50634/0 (socket says 172.21.15.112:50634) 2024-04-25T01:27:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.922+0000 7f2792b2d640 1 -- 172.21.15.112:0/2095595664 learned_addr learned my addr 172.21.15.112:0/2095595664 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:27:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.922+0000 7f2791b2b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c06dab0 0x7f278c14c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.922+0000 7f2792b2d640 1 -- 172.21.15.112:0/2095595664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c06dab0 msgr2=0x7f278c14c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.922+0000 7f2792b2d640 1 --2- 172.21.15.112:0/2095595664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c06dab0 0x7f278c14c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.922+0000 7f2792b2d640 1 -- 172.21.15.112:0/2095595664 >> v1:172.21.15.112:6789/0 conn(0x7f278c155d80 legacy=0x7f278c156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.922+0000 7f2792b2d640 1 -- 172.21.15.112:0/2095595664 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278c06e290 con 0x7f278c14c5c0 2024-04-25T01:27:26.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.923+0000 7f2792b2d640 1 --2- 172.21.15.112:0/2095595664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c14c5c0 0x7f278c153a80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2784009ed0 tx=0x7f2784033160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6764a66f6444dd08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:26.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.923+0000 7f279132a640 1 -- 172.21.15.112:0/2095595664 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2784033c60 con 0x7f278c14c5c0 2024-04-25T01:27:26.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.923+0000 7f279132a640 1 -- 172.21.15.112:0/2095595664 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2784033e00 con 0x7f278c14c5c0 2024-04-25T01:27:26.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.923+0000 7f279132a640 1 -- 172.21.15.112:0/2095595664 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278403b610 con 0x7f278c14c5c0 2024-04-25T01:27:26.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.924+0000 7f2794590640 1 -- 172.21.15.112:0/2095595664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c14c5c0 msgr2=0x7f278c153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.924+0000 7f2794590640 1 --2- 172.21.15.112:0/2095595664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c14c5c0 0x7f278c153a80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2784009ed0 tx=0x7f2784033160 comp rx=0 tx=0).stop 2024-04-25T01:27:26.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.924+0000 7f2794590640 1 -- 172.21.15.112:0/2095595664 shutdown_connections 2024-04-25T01:27:26.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.924+0000 7f2794590640 1 --2- 172.21.15.112:0/2095595664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c14c5c0 0x7f278c153a80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.924+0000 7f2794590640 1 --2- 172.21.15.112:0/2095595664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c06dab0 0x7f278c14c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.924+0000 7f2794590640 1 -- 172.21.15.112:0/2095595664 >> 172.21.15.112:0/2095595664 conn(0x7f278c0c6f80 msgr2=0x7f278c0c7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:26.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.924+0000 7f2794590640 1 -- 172.21.15.112:0/2095595664 shutdown_connections 2024-04-25T01:27:26.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.924+0000 7f2794590640 1 -- 172.21.15.112:0/2095595664 wait complete. 2024-04-25T01:27:26.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.925+0000 7f2794590640 1 Processor -- start 2024-04-25T01:27:26.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.925+0000 7f2794590640 1 -- start start 2024-04-25T01:27:26.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.926+0000 7f2794590640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c06dab0 0x7f278c0c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.926+0000 7f2794590640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f278c14c5c0 0x7f278c141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.926+0000 7f2794590640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c155d80 0x7f278c142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.926+0000 7f2794590640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f278c06c910 con 0x7f278c155d80 2024-04-25T01:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.926+0000 7f2794590640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f278c06e850 con 0x7f278c14c5c0 2024-04-25T01:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.926+0000 7f2794590640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f278c06ca90 con 0x7f278c06dab0 2024-04-25T01:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.926+0000 7f2792b2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c155d80 0x7f278c142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.926+0000 7f2792b2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c155d80 0x7f278c142060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50640/0 (socket says 172.21.15.112:50640) 2024-04-25T01:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.926+0000 7f2792b2d640 1 -- 172.21.15.112:0/235005933 learned_addr learned my addr 172.21.15.112:0/235005933 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.926+0000 7f2791b2b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f278c14c5c0 0x7f278c141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.926+0000 7f279232c640 1 --2- 172.21.15.112:0/235005933 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c06dab0 0x7f278c0c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.926+0000 7f2792b2d640 1 -- 172.21.15.112:0/235005933 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c06dab0 msgr2=0x7f278c0c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.926+0000 7f2792b2d640 1 --2- 172.21.15.112:0/235005933 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c06dab0 0x7f278c0c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.926+0000 7f2792b2d640 1 -- 172.21.15.112:0/235005933 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f278c14c5c0 msgr2=0x7f278c141b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.926+0000 7f2792b2d640 1 --2- 172.21.15.112:0/235005933 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f278c14c5c0 0x7f278c141b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.926+0000 7f2792b2d640 1 -- 172.21.15.112:0/235005933 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278c153760 con 0x7f278c155d80 2024-04-25T01:27:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.927+0000 7f2792b2d640 1 --2- 172.21.15.112:0/235005933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c155d80 0x7f278c142060 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2784032d40 tx=0x7f2784004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.927+0000 7f2782ffd640 1 -- 172.21.15.112:0/235005933 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278404a070 con 0x7f278c155d80 2024-04-25T01:27:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.927+0000 7f2782ffd640 1 -- 172.21.15.112:0/235005933 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2784046660 con 0x7f278c155d80 2024-04-25T01:27:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.927+0000 7f2794590640 1 -- 172.21.15.112:0/235005933 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278c1539d0 con 0x7f278c155d80 2024-04-25T01:27:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.928+0000 7f2782ffd640 1 -- 172.21.15.112:0/235005933 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2784009510 con 0x7f278c155d80 2024-04-25T01:27:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.928+0000 7f2794590640 1 -- 172.21.15.112:0/235005933 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f278c0ac290 con 0x7f278c155d80 2024-04-25T01:27:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.928+0000 7f2782ffd640 1 -- 172.21.15.112:0/235005933 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2784046ad0 con 0x7f278c155d80 2024-04-25T01:27:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.929+0000 7f2782ffd640 1 --2- 172.21.15.112:0/235005933 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f277006be60 0x7f277006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.929+0000 7f2782ffd640 1 -- 172.21.15.112:0/235005933 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f27840b3ce0 con 0x7f278c155d80 2024-04-25T01:27:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.929+0000 7f279232c640 1 --2- 172.21.15.112:0/235005933 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f277006be60 0x7f277006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.929+0000 7f2794590640 1 -- 172.21.15.112:0/235005933 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f278c0682a0 con 0x7f278c155d80 2024-04-25T01:27:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.931+0000 7f279232c640 1 --2- 172.21.15.112:0/235005933 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f277006be60 0x7f277006e320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f278c0abe90 tx=0x7f2788009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:26.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:26.938+0000 7f2782ffd640 1 -- 172.21.15.112:0/235005933 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2784081ca0 con 0x7f278c155d80 2024-04-25T01:27:27.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.359+0000 7f2794590640 1 -- 172.21.15.112:0/235005933 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f278c0ac530 con 0x7f278c155d80 2024-04-25T01:27:27.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.360+0000 7f2782ffd640 1 -- 172.21.15.112:0/235005933 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1568 (secure 0 0 0) 0x7f2784085fa0 con 0x7f278c155d80 2024-04-25T01:27:27.339 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:27:27.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.362+0000 7f2794590640 1 -- 172.21.15.112:0/235005933 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f277006be60 msgr2=0x7f277006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.362+0000 7f2794590640 1 --2- 172.21.15.112:0/235005933 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f277006be60 0x7f277006e320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f278c0abe90 tx=0x7f2788009040 comp rx=0 tx=0).stop 2024-04-25T01:27:27.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.362+0000 7f2794590640 1 -- 172.21.15.112:0/235005933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c155d80 msgr2=0x7f278c142060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.362+0000 7f2794590640 1 --2- 172.21.15.112:0/235005933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c155d80 0x7f278c142060 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2784032d40 tx=0x7f2784004a40 comp rx=0 tx=0).stop 2024-04-25T01:27:27.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.362+0000 7f2794590640 1 -- 172.21.15.112:0/235005933 shutdown_connections 2024-04-25T01:27:27.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.362+0000 7f2794590640 1 --2- 172.21.15.112:0/235005933 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f277006be60 0x7f277006e320 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.362+0000 7f2794590640 1 --2- 172.21.15.112:0/235005933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c155d80 0x7f278c142060 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.362+0000 7f2794590640 1 --2- 172.21.15.112:0/235005933 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f278c14c5c0 0x7f278c141b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.362+0000 7f2794590640 1 --2- 172.21.15.112:0/235005933 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f278c06dab0 0x7f278c0c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.362+0000 7f2794590640 1 -- 172.21.15.112:0/235005933 >> 172.21.15.112:0/235005933 conn(0x7f278c0c6f80 msgr2=0x7f278c0c7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:27.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.362+0000 7f2794590640 1 -- 172.21.15.112:0/235005933 shutdown_connections 2024-04-25T01:27:27.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.362+0000 7f2794590640 1 -- 172.21.15.112:0/235005933 wait complete. 2024-04-25T01:27:27.356 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:27:27.356 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:27:27.356 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:27:27.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.525+0000 7f06f4b6a640 1 Processor -- start 2024-04-25T01:27:27.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.525+0000 7f06f4b6a640 1 -- start start 2024-04-25T01:27:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.527+0000 7f06f4b6a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06f01596a0 0x7f06f0152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.527+0000 7f06f4b6a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06f0159b90 con 0x7f06f01592c0 2024-04-25T01:27:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.527+0000 7f06f4b6a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06f006c970 con 0x7f06f00c72b0 2024-04-25T01:27:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.527+0000 7f06f4b6a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f06f00c6e20 con 0x7f06f01596a0 2024-04-25T01:27:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.527+0000 7f06ee59c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f06f01592c0 0x7f06f00c9070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52666/0 (socket says 172.21.15.112:52666) 2024-04-25T01:27:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.527+0000 7f06ee59c640 1 -- 172.21.15.112:0/1316834110 learned_addr learned my addr 172.21.15.112:0/1316834110 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:27:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.527+0000 7f06e7fff640 1 --2- 172.21.15.112:0/1316834110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06f01596a0 0x7f06f0152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.527+0000 7f06edd9b640 1 -- 172.21.15.112:0/1316834110 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3944174231 0 0) 0x7f06f0159b90 con 0x7f06f01592c0 2024-04-25T01:27:27.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.527+0000 7f06e7fff640 1 -- 172.21.15.112:0/1316834110 >> v1:172.21.15.187:6789/0 conn(0x7f06f00c72b0 legacy=0x7f06f0160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.527+0000 7f06e7fff640 1 -- 172.21.15.112:0/1316834110 >> v1:172.21.15.112:6789/0 conn(0x7f06f01592c0 legacy=0x7f06f00c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.527+0000 7f06e7fff640 1 -- 172.21.15.112:0/1316834110 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f006cc50 con 0x7f06f01596a0 2024-04-25T01:27:27.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.528+0000 7f06e7fff640 1 --2- 172.21.15.112:0/1316834110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06f01596a0 0x7f06f0152d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009750 tx=0x7f06d8029f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=861f48afc73f2659 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:27.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.529+0000 7f06edd9b640 1 -- 172.21.15.112:0/1316834110 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8037070 con 0x7f06f01596a0 2024-04-25T01:27:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.529+0000 7f06edd9b640 1 -- 172.21.15.112:0/1316834110 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06d802ac90 con 0x7f06f01596a0 2024-04-25T01:27:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.529+0000 7f06edd9b640 1 -- 172.21.15.112:0/1316834110 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8032660 con 0x7f06f01596a0 2024-04-25T01:27:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.529+0000 7f06f4b6a640 1 -- 172.21.15.112:0/1316834110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06f01596a0 msgr2=0x7f06f0152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.529+0000 7f06f4b6a640 1 --2- 172.21.15.112:0/1316834110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06f01596a0 0x7f06f0152d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009750 tx=0x7f06d8029f80 comp rx=0 tx=0).stop 2024-04-25T01:27:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.530+0000 7f06f4b6a640 1 -- 172.21.15.112:0/1316834110 shutdown_connections 2024-04-25T01:27:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.530+0000 7f06f4b6a640 1 --2- 172.21.15.112:0/1316834110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06f01596a0 0x7f06f0152d00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.530+0000 7f06f4b6a640 1 -- 172.21.15.112:0/1316834110 >> 172.21.15.112:0/1316834110 conn(0x7f06f00cb170 msgr2=0x7f06f00cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.530+0000 7f06f4b6a640 1 -- 172.21.15.112:0/1316834110 shutdown_connections 2024-04-25T01:27:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.530+0000 7f06f4b6a640 1 -- 172.21.15.112:0/1316834110 wait complete. 2024-04-25T01:27:27.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.531+0000 7f06f4b6a640 1 Processor -- start 2024-04-25T01:27:27.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.532+0000 7f06f4b6a640 1 -- start start 2024-04-25T01:27:27.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.533+0000 7f06f4b6a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06f00c72b0 0x7f06f00af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.533+0000 7f06f4b6a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06f01592c0 0x7f06f01584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.533+0000 7f06f4b6a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f01596a0 0x7f06f01ed7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.533+0000 7f06f4b6a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06f006c510 con 0x7f06f01596a0 2024-04-25T01:27:27.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.533+0000 7f06f4b6a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f06f006c390 con 0x7f06f00c72b0 2024-04-25T01:27:27.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.533+0000 7f06f4b6a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f06f006db20 con 0x7f06f01592c0 2024-04-25T01:27:27.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.533+0000 7f06e7fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06f01592c0 0x7f06f01584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.533+0000 7f06ee59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06f00c72b0 0x7f06f00af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.533+0000 7f06e7fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06f01592c0 0x7f06f01584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:59668/0 (socket says 172.21.15.112:59668) 2024-04-25T01:27:27.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.533+0000 7f06e7fff640 1 -- 172.21.15.112:0/1400806339 learned_addr learned my addr 172.21.15.112:0/1400806339 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:27:27.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.533+0000 7f06eed9d640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f01596a0 0x7f06f01ed7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.533+0000 7f06ee59c640 1 -- 172.21.15.112:0/1400806339 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06f01592c0 msgr2=0x7f06f01584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.533+0000 7f06ee59c640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06f01592c0 0x7f06f01584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.533+0000 7f06ee59c640 1 -- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f01596a0 msgr2=0x7f06f01ed7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.533+0000 7f06ee59c640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f01596a0 0x7f06f01ed7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.533+0000 7f06ee59c640 1 -- 172.21.15.112:0/1400806339 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f00b0f40 con 0x7f06f00c72b0 2024-04-25T01:27:27.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.533+0000 7f06e7fff640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06f01592c0 0x7f06f01584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:27.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.533+0000 7f06eed9d640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f01596a0 0x7f06f01ed7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:27.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.534+0000 7f06ee59c640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06f00c72b0 0x7f06f00af850 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f06f0159d60 tx=0x7f06e000c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:27.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.534+0000 7f06e6ffd640 1 -- 172.21.15.112:0/1400806339 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e0017070 con 0x7f06f00c72b0 2024-04-25T01:27:27.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.534+0000 7f06e6ffd640 1 -- 172.21.15.112:0/1400806339 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06e0005d00 con 0x7f06f00c72b0 2024-04-25T01:27:27.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.534+0000 7f06f4b6a640 1 -- 172.21.15.112:0/1400806339 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f01ec9f0 con 0x7f06f00c72b0 2024-04-25T01:27:27.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.534+0000 7f06e6ffd640 1 -- 172.21.15.112:0/1400806339 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e0007310 con 0x7f06f00c72b0 2024-04-25T01:27:27.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.534+0000 7f06f4b6a640 1 -- 172.21.15.112:0/1400806339 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f01ecfb0 con 0x7f06f00c72b0 2024-04-25T01:27:27.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.535+0000 7f06f4b6a640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f01ed2f0 0x7f06f01edcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.535+0000 7f06f4b6a640 1 -- 172.21.15.112:0/1400806339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06f00679a0 con 0x7f06f01ed2f0 2024-04-25T01:27:27.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.535+0000 7f06e7fff640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f01ed2f0 0x7f06f01edcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.535+0000 7f06e7fff640 1 -- 172.21.15.112:0/1400806339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f06f00cacb0 con 0x7f06f01ed2f0 2024-04-25T01:27:27.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.536+0000 7f06e7fff640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f01ed2f0 0x7f06f01edcf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f06d8029fc0 tx=0x7f06d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:27.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.536+0000 7f06e6ffd640 1 -- 172.21.15.112:0/1400806339 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8042030 con 0x7f06f01ed2f0 2024-04-25T01:27:27.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.536+0000 7f06e6ffd640 1 -- 172.21.15.112:0/1400806339 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f06e001a070 con 0x7f06f00c72b0 2024-04-25T01:27:27.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.536+0000 7f06e6ffd640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f06d006bd30 0x7f06d006e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.536+0000 7f06eed9d640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f06d006bd30 0x7f06d006e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.536+0000 7f06e6ffd640 1 -- 172.21.15.112:0/1400806339 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f06e008a570 con 0x7f06f00c72b0 2024-04-25T01:27:27.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.537+0000 7f06e6ffd640 1 -- 172.21.15.112:0/1400806339 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f06d8032930 con 0x7f06f01ed2f0 2024-04-25T01:27:27.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.537+0000 7f06e6ffd640 1 -- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f01ed2f0 msgr2=0x7f06f01edcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.537+0000 7f06e6ffd640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f01ed2f0 0x7f06f01edcf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f06d8029fc0 tx=0x7f06d80046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.537+0000 7f06e6ffd640 1 -- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f01ed2f0 msgr2=0x7f06f01edcf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:27:27.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.537+0000 7f06e6ffd640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f01ed2f0 0x7f06f01edcf0 secure :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f06d8029fc0 tx=0x7f06d80046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.537+0000 7f06eed9d640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f06d006bd30 0x7f06d006e1f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f06f00c2eb0 tx=0x7f06e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:27.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.543+0000 7f06f4b6a640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f01ee230 0x7f06f01eeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.543+0000 7f06f4b6a640 1 -- 172.21.15.112:0/1400806339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06f00c79d0 con 0x7f06f01ee230 2024-04-25T01:27:27.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.543+0000 7f06e7fff640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f01ee230 0x7f06f01eeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.544+0000 7f06e7fff640 1 -- 172.21.15.112:0/1400806339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f06f00c5ca0 con 0x7f06f01ee230 2024-04-25T01:27:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.544+0000 7f06e7fff640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f01ee230 0x7f06f01eeaf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f06f0067b20 tx=0x7f06d8008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.545+0000 7f06e6ffd640 1 -- 172.21.15.112:0/1400806339 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8055020 con 0x7f06f01ee230 2024-04-25T01:27:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.545+0000 7f06e6ffd640 1 -- 172.21.15.112:0/1400806339 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5071 (secure 0 0 0) 0x7f06d803e660 con 0x7f06f01ee230 2024-04-25T01:27:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.545+0000 7f06e6ffd640 1 -- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f01ee230 msgr2=0x7f06f01eeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.545+0000 7f06e6ffd640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f01ee230 0x7f06f01eeaf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f06f0067b20 tx=0x7f06d8008030 comp rx=0 tx=0).stop 2024-04-25T01:27:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.545+0000 7f06e6ffd640 1 -- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f01ee230 msgr2=0x7f06f01eeaf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:27:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.545+0000 7f06e6ffd640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f01ee230 0x7f06f01eeaf0 secure :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f06f0067b20 tx=0x7f06d8008030 comp rx=0 tx=0).stop 2024-04-25T01:27:27.524 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:27:27.524 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:27:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:27:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:27:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 10, 2024-04-25T01:27:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 119172, 2024-04-25T01:27:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:27:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:27:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:27:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:27:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:27:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:27:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:27:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:27:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:27:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:27:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:27:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:27:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:27:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:27:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:27:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:27:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:27:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:27:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:27:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:27:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:27:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:27:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:27:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:27:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:27:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:27:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:27:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:27:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:27:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:27:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:27:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:27:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:27:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:27:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:27:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:27:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:27:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:27:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:27:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:27:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:27:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:27:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:27:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:27:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:27:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:27:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:27:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:27:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:27:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:27:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:27:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:27:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:27:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:27:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:27:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 6 2024-04-25T01:27:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:27:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:27.537 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:27.537 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:27:27.537 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.537 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:27.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.537 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:27:27.537 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:27:27.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.545+0000 7f06f4b6a640 1 -- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f06d006bd30 msgr2=0x7f06d006e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.545+0000 7f06f4b6a640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f06d006bd30 0x7f06d006e1f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f06f00c2eb0 tx=0x7f06e8009040 comp rx=0 tx=0).stop 2024-04-25T01:27:27.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.545+0000 7f06f4b6a640 1 -- 172.21.15.112:0/1400806339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06f00c72b0 msgr2=0x7f06f00af850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.545+0000 7f06f4b6a640 1 --2- 172.21.15.112:0/1400806339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06f00c72b0 0x7f06f00af850 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f06f0159d60 tx=0x7f06e000c870 comp rx=0 tx=0).stop 2024-04-25T01:27:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.545+0000 7f06eed9d640 1 -- 172.21.15.112:0/1400806339 reap_dead start 2024-04-25T01:27:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.545+0000 7f06eed9d640 1 -- 172.21.15.112:0/1400806339 reap_dead start 2024-04-25T01:27:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.545+0000 7f06f4b6a640 1 -- 172.21.15.112:0/1400806339 shutdown_connections 2024-04-25T01:27:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.545+0000 7f06f4b6a640 1 -- 172.21.15.112:0/1400806339 >> 172.21.15.112:0/1400806339 conn(0x7f06f00cb170 msgr2=0x7f06f0152ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.545+0000 7f06f4b6a640 1 -- 172.21.15.112:0/1400806339 shutdown_connections 2024-04-25T01:27:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.545+0000 7f06f4b6a640 1 -- 172.21.15.112:0/1400806339 wait complete. 2024-04-25T01:27:27.539 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:27:27.539 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:27:27.540 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:27:27.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.662+0000 7fd3d9458640 1 Processor -- start 2024-04-25T01:27:27.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.662+0000 7fd3d9458640 1 -- start start 2024-04-25T01:27:27.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.663+0000 7fd3d9458640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd3d40c72b0 0x7fd3d4161390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.663+0000 7fd3d9458640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3d4159b90 con 0x7fd3d41596a0 2024-04-25T01:27:27.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.663+0000 7fd3d9458640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3d406c970 con 0x7fd3d41592c0 2024-04-25T01:27:27.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.663+0000 7fd3d9458640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fd3d40c6e20 con 0x7fd3d40c72b0 2024-04-25T01:27:27.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.663+0000 7fd3d27fc640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd3d41596a0 0x7fd3d4152d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52680/0 (socket says 172.21.15.112:52680) 2024-04-25T01:27:27.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.663+0000 7fd3d27fc640 1 -- 172.21.15.112:0/2420048332 learned_addr learned my addr 172.21.15.112:0/2420048332 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:27:27.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.663+0000 7fd3d37fe640 1 --2- 172.21.15.112:0/2420048332 >> v2:172.21.15.112:3300/0 conn(0x7fd3d40c72b0 0x7fd3d4161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.664+0000 7fd3d37fe640 1 -- 172.21.15.112:0/2420048332 >> v1:172.21.15.187:6789/0 conn(0x7fd3d41592c0 legacy=0x7fd3d40c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.664+0000 7fd3d37fe640 1 -- 172.21.15.112:0/2420048332 >> v1:172.21.15.112:6789/0 conn(0x7fd3d41596a0 legacy=0x7fd3d4152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.664+0000 7fd3d37fe640 1 -- 172.21.15.112:0/2420048332 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d406cc50 con 0x7fd3d40c72b0 2024-04-25T01:27:27.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.664+0000 7fd3d37fe640 1 --2- 172.21.15.112:0/2420048332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d40c72b0 0x7fd3d4161390 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009d90 tx=0x7fd3cc02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a21b5f94ddea9ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:27.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.665+0000 7fd3d1ffb640 1 -- 172.21.15.112:0/2420048332 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc031600 con 0x7fd3d40c72b0 2024-04-25T01:27:27.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.665+0000 7fd3d1ffb640 1 -- 172.21.15.112:0/2420048332 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3cc0317a0 con 0x7fd3d40c72b0 2024-04-25T01:27:27.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.665+0000 7fd3d1ffb640 1 -- 172.21.15.112:0/2420048332 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc031b70 con 0x7fd3d40c72b0 2024-04-25T01:27:27.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.665+0000 7fd3d9458640 1 -- 172.21.15.112:0/2420048332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d40c72b0 msgr2=0x7fd3d4161390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.665+0000 7fd3d9458640 1 --2- 172.21.15.112:0/2420048332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d40c72b0 0x7fd3d4161390 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009d90 tx=0x7fd3cc02a5d0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.665+0000 7fd3d9458640 1 -- 172.21.15.112:0/2420048332 shutdown_connections 2024-04-25T01:27:27.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.665+0000 7fd3d9458640 1 --2- 172.21.15.112:0/2420048332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d40c72b0 0x7fd3d4161390 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.665+0000 7fd3d9458640 1 -- 172.21.15.112:0/2420048332 >> 172.21.15.112:0/2420048332 conn(0x7fd3d40cb170 msgr2=0x7fd3d40cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:27.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.666+0000 7fd3d9458640 1 -- 172.21.15.112:0/2420048332 shutdown_connections 2024-04-25T01:27:27.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.666+0000 7fd3d9458640 1 -- 172.21.15.112:0/2420048332 wait complete. 2024-04-25T01:27:27.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.667+0000 7fd3d9458640 1 Processor -- start 2024-04-25T01:27:27.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.667+0000 7fd3d9458640 1 -- start start 2024-04-25T01:27:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.668+0000 7fd3d9458640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d40c72b0 0x7fd3d41eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.668+0000 7fd3d9458640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3d41592c0 0x7fd3d41ef420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.668+0000 7fd3d9458640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d41596a0 0x7fd3d40c2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.668+0000 7fd3d9458640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d406c510 con 0x7fd3d41596a0 2024-04-25T01:27:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.668+0000 7fd3d9458640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d406c390 con 0x7fd3d40c72b0 2024-04-25T01:27:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.668+0000 7fd3d9458640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd3d4153410 con 0x7fd3d41592c0 2024-04-25T01:27:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.668+0000 7fd3d37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d41596a0 0x7fd3d40c2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.668+0000 7fd3d2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d40c72b0 0x7fd3d41eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.668+0000 7fd3d37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d41596a0 0x7fd3d40c2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53380/0 (socket says 172.21.15.112:53380) 2024-04-25T01:27:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.668+0000 7fd3d37fe640 1 -- 172.21.15.112:0/155390892 learned_addr learned my addr 172.21.15.112:0/155390892 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:27:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.668+0000 7fd3d27fc640 1 --2- 172.21.15.112:0/155390892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3d41592c0 0x7fd3d41ef420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.668+0000 7fd3d37fe640 1 -- 172.21.15.112:0/155390892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3d41592c0 msgr2=0x7fd3d41ef420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.668+0000 7fd3d37fe640 1 --2- 172.21.15.112:0/155390892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3d41592c0 0x7fd3d41ef420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.668+0000 7fd3d37fe640 1 -- 172.21.15.112:0/155390892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d40c72b0 msgr2=0x7fd3d41eeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.668+0000 7fd3d37fe640 1 --2- 172.21.15.112:0/155390892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d40c72b0 0x7fd3d41eeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.668+0000 7fd3d37fe640 1 -- 172.21.15.112:0/155390892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d41589d0 con 0x7fd3d41596a0 2024-04-25T01:27:27.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.668+0000 7fd3d2ffd640 1 --2- 172.21.15.112:0/155390892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d40c72b0 0x7fd3d41eeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:27.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.669+0000 7fd3d37fe640 1 --2- 172.21.15.112:0/155390892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d41596a0 0x7fd3d40c2340 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009eb0 tx=0x7fd3cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:27.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.669+0000 7fd3bb7fe640 1 -- 172.21.15.112:0/155390892 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc03f070 con 0x7fd3d41596a0 2024-04-25T01:27:27.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.669+0000 7fd3bb7fe640 1 -- 172.21.15.112:0/155390892 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3cc030b60 con 0x7fd3d41596a0 2024-04-25T01:27:27.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.669+0000 7fd3bb7fe640 1 -- 172.21.15.112:0/155390892 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc044460 con 0x7fd3d41596a0 2024-04-25T01:27:27.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.669+0000 7fd3d9458640 1 -- 172.21.15.112:0/155390892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d41ef960 con 0x7fd3d41596a0 2024-04-25T01:27:27.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.670+0000 7fd3d9458640 1 -- 172.21.15.112:0/155390892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d40c4f80 con 0x7fd3d41596a0 2024-04-25T01:27:27.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.671+0000 7fd3d9458640 1 --2- 172.21.15.112:0/155390892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d40c4af0 0x7fd3d414e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.671+0000 7fd3d9458640 1 -- 172.21.15.112:0/155390892 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d4075980 con 0x7fd3d40c4af0 2024-04-25T01:27:27.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.671+0000 7fd3d2ffd640 1 --2- 172.21.15.112:0/155390892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d40c4af0 0x7fd3d414e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.671+0000 7fd3d2ffd640 1 -- 172.21.15.112:0/155390892 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd3d414e9e0 con 0x7fd3d40c4af0 2024-04-25T01:27:27.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.671+0000 7fd3bb7fe640 1 -- 172.21.15.112:0/155390892 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd3cc03b410 con 0x7fd3d41596a0 2024-04-25T01:27:27.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.671+0000 7fd3d2ffd640 1 --2- 172.21.15.112:0/155390892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d40c4af0 0x7fd3d414e330 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400fae0 tx=0x7fd3c4001d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:27.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.671+0000 7fd3bb7fe640 1 --2- 172.21.15.112:0/155390892 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd3c006be60 0x7fd3c006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.672+0000 7fd3bb7fe640 1 -- 172.21.15.112:0/155390892 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fd3cc0ac6a0 con 0x7fd3d41596a0 2024-04-25T01:27:27.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.672+0000 7fd3d27fc640 1 --2- 172.21.15.112:0/155390892 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd3c006be60 0x7fd3c006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.672+0000 7fd3bb7fe640 1 -- 172.21.15.112:0/155390892 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c40052e0 con 0x7fd3d40c4af0 2024-04-25T01:27:27.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.672+0000 7fd3bb7fe640 1 -- 172.21.15.112:0/155390892 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd3c40052e0 con 0x7fd3d40c4af0 2024-04-25T01:27:27.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.672+0000 7fd3bb7fe640 1 -- 172.21.15.112:0/155390892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d40c4af0 msgr2=0x7fd3d414e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.672+0000 7fd3bb7fe640 1 --2- 172.21.15.112:0/155390892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d40c4af0 0x7fd3d414e330 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400fae0 tx=0x7fd3c4001d70 comp rx=0 tx=0).stop 2024-04-25T01:27:27.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.672+0000 7fd3bb7fe640 1 -- 172.21.15.112:0/155390892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d40c4af0 msgr2=0x7fd3d414e330 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:27:27.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.672+0000 7fd3bb7fe640 1 --2- 172.21.15.112:0/155390892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d40c4af0 0x7fd3d414e330 secure :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400fae0 tx=0x7fd3c4001d70 comp rx=0 tx=0).stop 2024-04-25T01:27:27.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.672+0000 7fd3d27fc640 1 --2- 172.21.15.112:0/155390892 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd3c006be60 0x7fd3c006e320 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40676c0 tx=0x7fd3c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:27.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.684+0000 7fd3d9458640 1 --2- 172.21.15.112:0/155390892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d40c2880 0x7fd3d414bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.684+0000 7fd3d9458640 1 -- 172.21.15.112:0/155390892 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d40682a0 con 0x7fd3d40c2880 2024-04-25T01:27:27.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.684+0000 7fd3d2ffd640 1 --2- 172.21.15.112:0/155390892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d40c2880 0x7fd3d414bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.685+0000 7fd3d2ffd640 1 -- 172.21.15.112:0/155390892 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd3d40c79d0 con 0x7fd3d40c2880 2024-04-25T01:27:27.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.685+0000 7fd3d2ffd640 1 --2- 172.21.15.112:0/155390892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d40c2880 0x7fd3d414bf40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400fae0 tx=0x7fd3c4001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:27.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.685+0000 7fd3bb7fe640 1 -- 172.21.15.112:0/155390892 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c400dcf0 con 0x7fd3d40c2880 2024-04-25T01:27:27.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.685+0000 7fd3bb7fe640 1 -- 172.21.15.112:0/155390892 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4558 (secure 0 0 0) 0x7fd3c400dcf0 con 0x7fd3d40c2880 2024-04-25T01:27:27.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.685+0000 7fd3bb7fe640 1 -- 172.21.15.112:0/155390892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d40c2880 msgr2=0x7fd3d414bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.685+0000 7fd3bb7fe640 1 --2- 172.21.15.112:0/155390892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d40c2880 0x7fd3d414bf40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400fae0 tx=0x7fd3c4001120 comp rx=0 tx=0).stop 2024-04-25T01:27:27.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.685+0000 7fd3bb7fe640 1 -- 172.21.15.112:0/155390892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d40c2880 msgr2=0x7fd3d414bf40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:27:27.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.686+0000 7fd3bb7fe640 1 --2- 172.21.15.112:0/155390892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3d40c2880 0x7fd3d414bf40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.665 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:27:27.665 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:27:27.665 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:27:27.665 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:27:27.665 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 10, 2024-04-25T01:27:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 15824, 2024-04-25T01:27:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:27:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:27:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:27:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:27:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:27:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:27:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:27:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:27:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:27:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:27:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:27:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:27:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:27:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:27:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:27:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:27:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:27:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:27:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:27:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:27:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:27:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:27:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:27:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:27:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:27:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:27:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:27:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:27:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:27:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:27:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:27:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:27:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:27:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:27:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:27:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:27:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:27:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:27:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:27:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:27:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:27:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:27:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:27:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:27:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:27:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:27:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:27:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:27:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:27:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:27:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:27:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:27:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:27:27.677 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:27:27.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.686+0000 7fd3d9458640 1 -- 172.21.15.112:0/155390892 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd3c006be60 msgr2=0x7fd3c006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.686+0000 7fd3d9458640 1 --2- 172.21.15.112:0/155390892 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd3c006be60 0x7fd3c006e320 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40676c0 tx=0x7fd3c8009040 comp rx=0 tx=0).stop 2024-04-25T01:27:27.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.686+0000 7fd3d9458640 1 -- 172.21.15.112:0/155390892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d41596a0 msgr2=0x7fd3d40c2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.686+0000 7fd3d9458640 1 --2- 172.21.15.112:0/155390892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d41596a0 0x7fd3d40c2340 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009eb0 tx=0x7fd3cc004a40 comp rx=0 tx=0).stop 2024-04-25T01:27:27.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.686+0000 7fd3d37fe640 1 -- 172.21.15.112:0/155390892 reap_dead start 2024-04-25T01:27:27.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.686+0000 7fd3d37fe640 1 -- 172.21.15.112:0/155390892 reap_dead start 2024-04-25T01:27:27.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.686+0000 7fd3d9458640 1 -- 172.21.15.112:0/155390892 shutdown_connections 2024-04-25T01:27:27.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.686+0000 7fd3d9458640 1 -- 172.21.15.112:0/155390892 >> 172.21.15.112:0/155390892 conn(0x7fd3d40cb170 msgr2=0x7fd3d40ac4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:27.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.686+0000 7fd3d9458640 1 -- 172.21.15.112:0/155390892 shutdown_connections 2024-04-25T01:27:27.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.686+0000 7fd3d9458640 1 -- 172.21.15.112:0/155390892 wait complete. 2024-04-25T01:27:27.679 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:27:27.680 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:27:27.680 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:27:27.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.799+0000 7f8cbc570640 1 Processor -- start 2024-04-25T01:27:27.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.799+0000 7f8cbc570640 1 -- start start 2024-04-25T01:27:27.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.800+0000 7f8cbc570640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8cb41592c0 0x7f8cb40c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.800+0000 7f8cbc570640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8cb41596a0 0x7f8cb4152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.801+0000 7f8cbc570640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cb4159b90 con 0x7f8cb40c72b0 2024-04-25T01:27:27.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.801+0000 7f8cb9b0b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8cb41596a0 0x7f8cb4152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.801+0000 7f8cbab0d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8cb40c72b0 0x7f8cb4155ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52686/0 (socket says 172.21.15.112:52686) 2024-04-25T01:27:27.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.801+0000 7f8cbab0d640 1 -- 172.21.15.112:0/415063716 learned_addr learned my addr 172.21.15.112:0/415063716 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:27:27.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.801+0000 7f8cba30c640 1 --2- 172.21.15.112:0/415063716 >> v2:172.21.15.187:3300/0 conn(0x7f8cb41592c0 0x7f8cb40c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.801+0000 7f8cbc570640 1 -- 172.21.15.112:0/415063716 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f8cb40c6e20 con 0x7f8cb41596a0 2024-04-25T01:27:27.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.801+0000 7f8cbc570640 1 -- 172.21.15.112:0/415063716 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8cb406c970 con 0x7f8cb41592c0 2024-04-25T01:27:27.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.801+0000 7f8cb930a640 1 -- 172.21.15.112:0/415063716 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4062189167 0 0) 0x7f8cb4159b90 con 0x7f8cb40c72b0 2024-04-25T01:27:27.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.801+0000 7f8cb930a640 1 -- 172.21.15.112:0/415063716 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ca0003520 con 0x7f8cb40c72b0 2024-04-25T01:27:27.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.801+0000 7f8cba30c640 1 -- 172.21.15.112:0/415063716 >> v2:172.21.15.112:3300/0 conn(0x7f8cb41596a0 msgr2=0x7f8cb4152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:27.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.801+0000 7f8cba30c640 1 --2- 172.21.15.112:0/415063716 >> v2:172.21.15.112:3300/0 conn(0x7f8cb41596a0 0x7f8cb4152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.801+0000 7f8cba30c640 1 -- 172.21.15.112:0/415063716 >> v1:172.21.15.112:6789/0 conn(0x7f8cb40c72b0 legacy=0x7f8cb4155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.801+0000 7f8cba30c640 1 -- 172.21.15.112:0/415063716 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb406cc50 con 0x7f8cb41592c0 2024-04-25T01:27:27.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.802+0000 7f8cba30c640 1 --2- 172.21.15.112:0/415063716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cb41592c0 0x7f8cb40c9070 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8cb408cb00 tx=0x7f8cb002a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f22eecd984c78ded server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:27.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.802+0000 7f8cb930a640 1 -- 172.21.15.112:0/415063716 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0037070 con 0x7f8cb41592c0 2024-04-25T01:27:27.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.802+0000 7f8cb930a640 1 -- 172.21.15.112:0/415063716 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cb002add0 con 0x7f8cb41592c0 2024-04-25T01:27:27.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.802+0000 7f8cb930a640 1 -- 172.21.15.112:0/415063716 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0032660 con 0x7f8cb41592c0 2024-04-25T01:27:27.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.803+0000 7f8cbc570640 1 -- 172.21.15.112:0/415063716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cb41592c0 msgr2=0x7f8cb40c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.803+0000 7f8cbc570640 1 --2- 172.21.15.112:0/415063716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cb41592c0 0x7f8cb40c9070 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8cb408cb00 tx=0x7f8cb002a080 comp rx=0 tx=0).stop 2024-04-25T01:27:27.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.803+0000 7f8cbc570640 1 -- 172.21.15.112:0/415063716 shutdown_connections 2024-04-25T01:27:27.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.803+0000 7f8cbc570640 1 --2- 172.21.15.112:0/415063716 >> v2:172.21.15.112:3300/0 conn(0x7f8cb41596a0 0x7f8cb4152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.803+0000 7f8cbc570640 1 --2- 172.21.15.112:0/415063716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cb41592c0 0x7f8cb40c9070 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.803+0000 7f8cbc570640 1 -- 172.21.15.112:0/415063716 >> 172.21.15.112:0/415063716 conn(0x7f8cb40cb170 msgr2=0x7f8cb40cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:27.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.803+0000 7f8cbc570640 1 -- 172.21.15.112:0/415063716 shutdown_connections 2024-04-25T01:27:27.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.803+0000 7f8cbc570640 1 -- 172.21.15.112:0/415063716 wait complete. 2024-04-25T01:27:27.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.805+0000 7f8cbc570640 1 Processor -- start 2024-04-25T01:27:27.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.805+0000 7f8cbc570640 1 -- start start 2024-04-25T01:27:27.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.806+0000 7f8cbc570640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb40c72b0 0x7f8cb4148f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.806+0000 7f8cbc570640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cb41592c0 0x7f8cb4157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.806+0000 7f8cbc570640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb41596a0 0x7f8cb41eb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.806+0000 7f8cba30c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb40c72b0 0x7f8cb4148f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.806+0000 7f8cba30c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb40c72b0 0x7f8cb4148f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:59678/0 (socket says 172.21.15.112:59678) 2024-04-25T01:27:27.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.806+0000 7f8cba30c640 1 -- 172.21.15.112:0/272184452 learned_addr learned my addr 172.21.15.112:0/272184452 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:27:27.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.806+0000 7f8cbc570640 1 -- 172.21.15.112:0/272184452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb406dca0 con 0x7f8cb41596a0 2024-04-25T01:27:27.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.806+0000 7f8cbc570640 1 -- 172.21.15.112:0/272184452 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb406db20 con 0x7f8cb41592c0 2024-04-25T01:27:27.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.807+0000 7f8cbc570640 1 -- 172.21.15.112:0/272184452 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8cb40c95b0 con 0x7f8cb40c72b0 2024-04-25T01:27:27.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.807+0000 7f8cbab0d640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb41596a0 0x7f8cb41eb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.807+0000 7f8cb9b0b640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cb41592c0 0x7f8cb4157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.807+0000 7f8cba30c640 1 -- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cb41592c0 msgr2=0x7f8cb4157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.807+0000 7f8cba30c640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cb41592c0 0x7f8cb4157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.807+0000 7f8cba30c640 1 -- 172.21.15.112:0/272184452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb41596a0 msgr2=0x7f8cb41eb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.807+0000 7f8cba30c640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb41596a0 0x7f8cb41eb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.807+0000 7f8cba30c640 1 -- 172.21.15.112:0/272184452 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb41498e0 con 0x7f8cb40c72b0 2024-04-25T01:27:27.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.807+0000 7f8cbab0d640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb41596a0 0x7f8cb41eb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:27.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.807+0000 7f8cba30c640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb40c72b0 0x7f8cb4148f10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009890 tx=0x7f8cb00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:27.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.807+0000 7f8caaffd640 1 -- 172.21.15.112:0/272184452 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0037040 con 0x7f8cb40c72b0 2024-04-25T01:27:27.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.808+0000 7f8caaffd640 1 -- 172.21.15.112:0/272184452 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cb0042070 con 0x7f8cb40c72b0 2024-04-25T01:27:27.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.808+0000 7f8cbc570640 1 -- 172.21.15.112:0/272184452 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb41ef3e0 con 0x7f8cb40c72b0 2024-04-25T01:27:27.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.808+0000 7f8caaffd640 1 -- 172.21.15.112:0/272184452 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb003d660 con 0x7f8cb40c72b0 2024-04-25T01:27:27.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.808+0000 7f8cbc570640 1 -- 172.21.15.112:0/272184452 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb41ef6c0 con 0x7f8cb40c72b0 2024-04-25T01:27:27.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.809+0000 7f8cbc570640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb4158510 0x7f8cb41eb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.809+0000 7f8cbc570640 1 -- 172.21.15.112:0/272184452 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8cb4075980 con 0x7f8cb4158510 2024-04-25T01:27:27.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.809+0000 7f8cb9b0b640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb4158510 0x7f8cb41eb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.809+0000 7f8caaffd640 1 -- 172.21.15.112:0/272184452 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8cb003d800 con 0x7f8cb40c72b0 2024-04-25T01:27:27.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.809+0000 7f8cb9b0b640 1 -- 172.21.15.112:0/272184452 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8cb40679a0 con 0x7f8cb4158510 2024-04-25T01:27:27.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.809+0000 7f8cb9b0b640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb4158510 0x7f8cb41eb870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8cb40c9330 tx=0x7f8ca4011cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:27.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.810+0000 7f8caaffd640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8c9806be30 0x7f8c9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.810+0000 7f8caaffd640 1 -- 172.21.15.112:0/272184452 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f8cb00ab720 con 0x7f8cb40c72b0 2024-04-25T01:27:27.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.810+0000 7f8cbab0d640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8c9806be30 0x7f8c9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.810+0000 7f8caaffd640 1 -- 172.21.15.112:0/272184452 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4019070 con 0x7f8cb4158510 2024-04-25T01:27:27.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.810+0000 7f8caaffd640 1 -- 172.21.15.112:0/272184452 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8ca4004240 con 0x7f8cb4158510 2024-04-25T01:27:27.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.810+0000 7f8caaffd640 1 -- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb4158510 msgr2=0x7f8cb41eb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.810+0000 7f8caaffd640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb4158510 0x7f8cb41eb870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8cb40c9330 tx=0x7f8ca4011cf0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.810+0000 7f8caaffd640 1 -- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb4158510 msgr2=0x7f8cb41eb870 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:27:27.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.810+0000 7f8caaffd640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb4158510 0x7f8cb41eb870 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.810+0000 7f8cbab0d640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8c9806be30 0x7f8c9806e2f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8cac005c00 tx=0x7f8cac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:27.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.822+0000 7f8cbc570640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb4149d90 0x7f8cb41549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.822+0000 7f8cbc570640 1 -- 172.21.15.112:0/272184452 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8cb41534e0 con 0x7f8cb4149d90 2024-04-25T01:27:27.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.823+0000 7f8cb9b0b640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb4149d90 0x7f8cb41549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.823+0000 7f8cb9b0b640 1 -- 172.21.15.112:0/272184452 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8cb40c79d0 con 0x7f8cb4149d90 2024-04-25T01:27:27.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.823+0000 7f8cb9b0b640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb4149d90 0x7f8cb41549a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400f680 tx=0x7f8ca40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:27.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.823+0000 7f8caaffd640 1 -- 172.21.15.112:0/272184452 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4019040 con 0x7f8cb4149d90 2024-04-25T01:27:27.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.824+0000 7f8caaffd640 1 -- 172.21.15.112:0/272184452 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5069 (secure 0 0 0) 0x7f8ca4003df0 con 0x7f8cb4149d90 2024-04-25T01:27:27.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.824+0000 7f8caaffd640 1 -- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb4149d90 msgr2=0x7f8cb41549a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.824+0000 7f8caaffd640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb4149d90 0x7f8cb41549a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400f680 tx=0x7f8ca40046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.824+0000 7f8caaffd640 1 -- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb4149d90 msgr2=0x7f8cb41549a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:27:27.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.824+0000 7f8caaffd640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb4149d90 0x7f8cb41549a0 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400f680 tx=0x7f8ca40046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.803 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:27:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:27:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:27:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:27:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 10, 2024-04-25T01:27:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 119413, 2024-04-25T01:27:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:27:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:27:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:27:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:27:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:27:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:27:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:27:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:27:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:27:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:27:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:27:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:27:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:27:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:27:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:27:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:27:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:27:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:27:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:27:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:27:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:27:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:27:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:27:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:27:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:27:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:27:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:27:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:27:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:27:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:27:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:27:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:27:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:27:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:27:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:27:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:27:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:27:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:27:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:27:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:27:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:27:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:27:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:27:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:27:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:27:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:27:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:27:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:27:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:27:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:27:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:27:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:27:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:27:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:27:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:27:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:27:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:27:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:27:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:27:27.816 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:27:27.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.824+0000 7f8cbc570640 1 -- 172.21.15.112:0/272184452 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8c9806be30 msgr2=0x7f8c9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.824+0000 7f8cbc570640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8c9806be30 0x7f8c9806e2f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8cac005c00 tx=0x7f8cac009040 comp rx=0 tx=0).stop 2024-04-25T01:27:27.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.824+0000 7f8cbc570640 1 -- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb40c72b0 msgr2=0x7f8cb4148f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.824+0000 7f8cbc570640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb40c72b0 0x7f8cb4148f10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009890 tx=0x7f8cb00046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.824+0000 7f8cbab0d640 1 -- 172.21.15.112:0/272184452 reap_dead start 2024-04-25T01:27:27.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.824+0000 7f8cbc570640 1 -- 172.21.15.112:0/272184452 shutdown_connections 2024-04-25T01:27:27.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.824+0000 7f8cbc570640 1 --2- 172.21.15.112:0/272184452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb40c72b0 0x7f8cb4148f10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.824+0000 7f8cbc570640 1 -- 172.21.15.112:0/272184452 >> 172.21.15.112:0/272184452 conn(0x7f8cb40cb170 msgr2=0x7f8cb40ca6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:27.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.824+0000 7f8cbc570640 1 -- 172.21.15.112:0/272184452 shutdown_connections 2024-04-25T01:27:27.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.825+0000 7f8cbc570640 1 -- 172.21.15.112:0/272184452 wait complete. 2024-04-25T01:27:27.818 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:27:27.818 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T01:27:27.818 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T01:27:27.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.936+0000 7fa32618b640 1 Processor -- start 2024-04-25T01:27:27.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.937+0000 7fa32618b640 1 -- start start 2024-04-25T01:27:27.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.937+0000 7fa32618b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3201596a0 0x7fa320152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.937+0000 7fa32618b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa3200c72b0 0x7fa320160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.937+0000 7fa32618b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa320159b90 con 0x7fa3201592c0 2024-04-25T01:27:27.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.937+0000 7fa32618b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa3200c6e20 con 0x7fa3200c72b0 2024-04-25T01:27:27.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.937+0000 7fa32618b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa32006c970 con 0x7fa3201596a0 2024-04-25T01:27:27.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.937+0000 7fa31effd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3201596a0 0x7fa320152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.937+0000 7fa31effd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3201596a0 0x7fa320152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:59710/0 (socket says 172.21.15.112:59710) 2024-04-25T01:27:27.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.937+0000 7fa31ffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa3200c72b0 0x7fa320160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.937+0000 7fa31effd640 1 -- 172.21.15.112:0/4158842870 learned_addr learned my addr 172.21.15.112:0/4158842870 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:27:27.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.938+0000 7fa31effd640 1 -- 172.21.15.112:0/4158842870 >> v2:172.21.15.187:3300/0 conn(0x7fa3200c72b0 msgr2=0x7fa320160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:27.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.938+0000 7fa31effd640 1 --2- 172.21.15.112:0/4158842870 >> v2:172.21.15.187:3300/0 conn(0x7fa3200c72b0 0x7fa320160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.938+0000 7fa31effd640 1 -- 172.21.15.112:0/4158842870 >> v1:172.21.15.187:6789/0 conn(0x7fa3201592c0 legacy=0x7fa3200c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.938+0000 7fa31effd640 1 -- 172.21.15.112:0/4158842870 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32006cc50 con 0x7fa3201596a0 2024-04-25T01:27:27.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.938+0000 7fa31ffff640 1 --2- 172.21.15.112:0/4158842870 >> v2:172.21.15.187:3300/0 conn(0x7fa3200c72b0 0x7fa320160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:27.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.939+0000 7fa31effd640 1 --2- 172.21.15.112:0/4158842870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3201596a0 0x7fa320152d00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa314009870 tx=0x7fa31402a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32e92e3a8a1003ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:27.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.939+0000 7fa31e7fc640 1 -- 172.21.15.112:0/4158842870 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314037070 con 0x7fa3201596a0 2024-04-25T01:27:27.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.939+0000 7fa31e7fc640 1 -- 172.21.15.112:0/4158842870 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa314032440 con 0x7fa3201596a0 2024-04-25T01:27:27.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.939+0000 7fa31e7fc640 1 -- 172.21.15.112:0/4158842870 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314032810 con 0x7fa3201596a0 2024-04-25T01:27:27.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.939+0000 7fa32618b640 1 -- 172.21.15.112:0/4158842870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3201596a0 msgr2=0x7fa320152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.939+0000 7fa32618b640 1 --2- 172.21.15.112:0/4158842870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3201596a0 0x7fa320152d00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa314009870 tx=0x7fa31402a130 comp rx=0 tx=0).stop 2024-04-25T01:27:27.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.939+0000 7fa32618b640 1 -- 172.21.15.112:0/4158842870 shutdown_connections 2024-04-25T01:27:27.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.939+0000 7fa32618b640 1 --2- 172.21.15.112:0/4158842870 >> v2:172.21.15.187:3300/0 conn(0x7fa3200c72b0 0x7fa320160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.939+0000 7fa32618b640 1 --2- 172.21.15.112:0/4158842870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3201596a0 0x7fa320152d00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.939+0000 7fa32618b640 1 -- 172.21.15.112:0/4158842870 >> 172.21.15.112:0/4158842870 conn(0x7fa3200cb170 msgr2=0x7fa3200cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:27.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.939+0000 7fa32618b640 1 -- 172.21.15.112:0/4158842870 shutdown_connections 2024-04-25T01:27:27.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.939+0000 7fa32618b640 1 -- 172.21.15.112:0/4158842870 wait complete. 2024-04-25T01:27:27.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.941+0000 7fa32618b640 1 Processor -- start 2024-04-25T01:27:27.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.941+0000 7fa32618b640 1 -- start start 2024-04-25T01:27:27.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.942+0000 7fa32618b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3200c72b0 0x7fa320153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.942+0000 7fa32618b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3201592c0 0x7fa320154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.942+0000 7fa32618b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3201596a0 0x7fa320148510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.942+0000 7fa32618b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa32006c510 con 0x7fa3201596a0 2024-04-25T01:27:27.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.942+0000 7fa32618b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa32006c390 con 0x7fa3201592c0 2024-04-25T01:27:27.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.942+0000 7fa32618b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa32006db20 con 0x7fa3200c72b0 2024-04-25T01:27:27.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.942+0000 7fa31ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3201596a0 0x7fa320148510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.942+0000 7fa31effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3201592c0 0x7fa320154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.943+0000 7fa31effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3201592c0 0x7fa320154100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:59218/0 (socket says 172.21.15.112:59218) 2024-04-25T01:27:27.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.943+0000 7fa31effd640 1 -- 172.21.15.112:0/1551439690 learned_addr learned my addr 172.21.15.112:0/1551439690 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:27:27.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.943+0000 7fa31f7fe640 1 --2- 172.21.15.112:0/1551439690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3200c72b0 0x7fa320153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.943+0000 7fa31effd640 1 -- 172.21.15.112:0/1551439690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3200c72b0 msgr2=0x7fa320153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.943+0000 7fa31effd640 1 --2- 172.21.15.112:0/1551439690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3200c72b0 0x7fa320153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.943+0000 7fa31effd640 1 -- 172.21.15.112:0/1551439690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3201596a0 msgr2=0x7fa320148510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.943+0000 7fa31effd640 1 --2- 172.21.15.112:0/1551439690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3201596a0 0x7fa320148510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.943+0000 7fa31effd640 1 -- 172.21.15.112:0/1551439690 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa320150510 con 0x7fa3201592c0 2024-04-25T01:27:27.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.943+0000 7fa31effd640 1 --2- 172.21.15.112:0/1551439690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3201592c0 0x7fa320154100 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa31402a170 tx=0x7fa3140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:27.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.943+0000 7fa307fff640 1 -- 172.21.15.112:0/1551439690 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314041030 con 0x7fa3201592c0 2024-04-25T01:27:27.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.943+0000 7fa307fff640 1 -- 172.21.15.112:0/1551439690 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa314030440 con 0x7fa3201592c0 2024-04-25T01:27:27.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.943+0000 7fa32618b640 1 -- 172.21.15.112:0/1551439690 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3201eef10 con 0x7fa3201592c0 2024-04-25T01:27:27.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.943+0000 7fa307fff640 1 -- 172.21.15.112:0/1551439690 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314037040 con 0x7fa3201592c0 2024-04-25T01:27:27.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.944+0000 7fa32618b640 1 -- 172.21.15.112:0/1551439690 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3201ef4d0 con 0x7fa3201592c0 2024-04-25T01:27:27.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.946+0000 7fa307fff640 1 -- 172.21.15.112:0/1551439690 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa314044070 con 0x7fa3201592c0 2024-04-25T01:27:27.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.946+0000 7fa32618b640 1 -- 172.21.15.112:0/1551439690 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3200682a0 con 0x7fa3201592c0 2024-04-25T01:27:27.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.946+0000 7fa307fff640 1 --2- 172.21.15.112:0/1551439690 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa30c06be30 0x7fa30c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.946+0000 7fa307fff640 1 -- 172.21.15.112:0/1551439690 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fa3140ab1b0 con 0x7fa3201592c0 2024-04-25T01:27:27.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.946+0000 7fa31f7fe640 1 --2- 172.21.15.112:0/1551439690 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa30c06be30 0x7fa30c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.947+0000 7fa31f7fe640 1 --2- 172.21.15.112:0/1551439690 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa30c06be30 0x7fa30c06e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa310000f60 tx=0x7fa310009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:27.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:27.955+0000 7fa307fff640 1 -- 172.21.15.112:0/1551439690 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa314079c50 con 0x7fa3201592c0 2024-04-25T01:27:28.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:28.454+0000 7fa32618b640 1 -- 172.21.15.112:0/1551439690 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fa3200c79d0 con 0x7fa3201592c0 2024-04-25T01:27:28.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:28.456+0000 7fa307fff640 1 -- 172.21.15.112:0/1551439690 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fa314079550 con 0x7fa3201592c0 2024-04-25T01:27:28.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:28.459+0000 7fa32618b640 1 -- 172.21.15.112:0/1551439690 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa30c06be30 msgr2=0x7fa30c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:28.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:28.459+0000 7fa32618b640 1 --2- 172.21.15.112:0/1551439690 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa30c06be30 0x7fa30c06e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa310000f60 tx=0x7fa310009040 comp rx=0 tx=0).stop 2024-04-25T01:27:28.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:28.459+0000 7fa32618b640 1 -- 172.21.15.112:0/1551439690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3201592c0 msgr2=0x7fa320154100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:28.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:28.459+0000 7fa32618b640 1 --2- 172.21.15.112:0/1551439690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3201592c0 0x7fa320154100 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa31402a170 tx=0x7fa3140046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:28.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:28.459+0000 7fa32618b640 1 -- 172.21.15.112:0/1551439690 shutdown_connections 2024-04-25T01:27:28.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:28.459+0000 7fa32618b640 1 --2- 172.21.15.112:0/1551439690 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa30c06be30 0x7fa30c06e2f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:28.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:28.459+0000 7fa32618b640 1 --2- 172.21.15.112:0/1551439690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3201596a0 0x7fa320148510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:28.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:28.459+0000 7fa32618b640 1 --2- 172.21.15.112:0/1551439690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3201592c0 0x7fa320154100 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:28.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:28.459+0000 7fa32618b640 1 --2- 172.21.15.112:0/1551439690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3200c72b0 0x7fa320153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:28.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:28.459+0000 7fa32618b640 1 -- 172.21.15.112:0/1551439690 >> 172.21.15.112:0/1551439690 conn(0x7fa3200cb170 msgr2=0x7fa3200aac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:28.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:28.459+0000 7fa32618b640 1 -- 172.21.15.112:0/1551439690 shutdown_connections 2024-04-25T01:27:28.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:28.459+0000 7fa32618b640 1 -- 172.21.15.112:0/1551439690 wait complete. 2024-04-25T01:27:28.454 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T01:27:38.454 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:27:38.455 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:27:38.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.575+0000 7f827f16a640 1 Processor -- start 2024-04-25T01:27:38.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.575+0000 7f827f16a640 1 -- start start 2024-04-25T01:27:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.575+0000 7f827f16a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82780c3370 0x7f827806eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.575+0000 7f827f16a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f82780c7070 0x7f82780c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.575+0000 7f827f16a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8278070250 con 0x7f8278153030 2024-04-25T01:27:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.575+0000 7f827f16a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8278156980 con 0x7f82780c7070 2024-04-25T01:27:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.575+0000 7f827f16a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f82780c56f0 con 0x7f82780c3370 2024-04-25T01:27:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.576+0000 7f8277fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82780c3370 0x7f827806eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.576+0000 7f827d707640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f82780c7070 0x7f82780c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.576+0000 7f827cf06640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f8278153030 0x7f8278156270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:56270/0 (socket says 172.21.15.112:56270) 2024-04-25T01:27:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.576+0000 7f827cf06640 1 -- 172.21.15.112:0/1996380784 learned_addr learned my addr 172.21.15.112:0/1996380784 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:27:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.576+0000 7f82777fe640 1 -- 172.21.15.112:0/1996380784 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4120805703 0 0) 0x7f8278070250 con 0x7f8278153030 2024-04-25T01:27:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.576+0000 7f82777fe640 1 -- 172.21.15.112:0/1996380784 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8268003520 con 0x7f8278153030 2024-04-25T01:27:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.576+0000 7f8277fff640 1 -- 172.21.15.112:0/1996380784 >> v2:172.21.15.187:3300/0 conn(0x7f82780c7070 msgr2=0x7f82780c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:38.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.576+0000 7f8277fff640 1 --2- 172.21.15.112:0/1996380784 >> v2:172.21.15.187:3300/0 conn(0x7f82780c7070 0x7f82780c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:38.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.576+0000 7f8277fff640 1 -- 172.21.15.112:0/1996380784 >> v1:172.21.15.187:6789/0 conn(0x7f8278153030 legacy=0x7f8278156270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:38.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.576+0000 7f8277fff640 1 -- 172.21.15.112:0/1996380784 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8278070530 con 0x7f82780c3370 2024-04-25T01:27:38.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.576+0000 7f827d707640 1 --2- 172.21.15.112:0/1996380784 >> v2:172.21.15.187:3300/0 conn(0x7f82780c7070 0x7f82780c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:38.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.577+0000 7f8277fff640 1 --2- 172.21.15.112:0/1996380784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82780c3370 0x7f827806eb90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f82640097a0 tx=0x7f826402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b07bf7c773a37863 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:38.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.577+0000 7f82777fe640 1 -- 172.21.15.112:0/1996380784 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264037070 con 0x7f82780c3370 2024-04-25T01:27:38.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.577+0000 7f82777fe640 1 -- 172.21.15.112:0/1996380784 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f826402ae50 con 0x7f82780c3370 2024-04-25T01:27:38.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.577+0000 7f82777fe640 1 -- 172.21.15.112:0/1996380784 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264032640 con 0x7f82780c3370 2024-04-25T01:27:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.578+0000 7f827f16a640 1 -- 172.21.15.112:0/1996380784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82780c3370 msgr2=0x7f827806eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.578+0000 7f827f16a640 1 --2- 172.21.15.112:0/1996380784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82780c3370 0x7f827806eb90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f82640097a0 tx=0x7f826402a100 comp rx=0 tx=0).stop 2024-04-25T01:27:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.578+0000 7f827f16a640 1 -- 172.21.15.112:0/1996380784 shutdown_connections 2024-04-25T01:27:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.578+0000 7f827f16a640 1 --2- 172.21.15.112:0/1996380784 >> v2:172.21.15.187:3300/0 conn(0x7f82780c7070 0x7f82780c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.578+0000 7f827f16a640 1 --2- 172.21.15.112:0/1996380784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82780c3370 0x7f827806eb90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.578+0000 7f827f16a640 1 -- 172.21.15.112:0/1996380784 >> 172.21.15.112:0/1996380784 conn(0x7f82780c9140 msgr2=0x7f82780c9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.578+0000 7f827f16a640 1 -- 172.21.15.112:0/1996380784 shutdown_connections 2024-04-25T01:27:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.578+0000 7f827f16a640 1 -- 172.21.15.112:0/1996380784 wait complete. 2024-04-25T01:27:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.579+0000 7f827f16a640 1 Processor -- start 2024-04-25T01:27:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.579+0000 7f827f16a640 1 -- start start 2024-04-25T01:27:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.579+0000 7f827f16a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82780c3370 0x7f8278158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.579+0000 7f827f16a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82780c7070 0x7f82780acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.579+0000 7f827f16a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8278153030 0x7f8278143110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.580+0000 7f827f16a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8278153410 con 0x7f82780c3370 2024-04-25T01:27:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.580+0000 7f827f16a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8278155d50 con 0x7f82780c7070 2024-04-25T01:27:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.580+0000 7f827f16a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8278155a00 con 0x7f8278153030 2024-04-25T01:27:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.580+0000 7f827cf06640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82780c3370 0x7f8278158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.580+0000 7f8277fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82780c7070 0x7f82780acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.580+0000 7f827cf06640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82780c3370 0x7f8278158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56286/0 (socket says 172.21.15.112:56286) 2024-04-25T01:27:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.580+0000 7f827d707640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8278153030 0x7f8278143110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.580+0000 7f827cf06640 1 -- 172.21.15.112:0/443744199 learned_addr learned my addr 172.21.15.112:0/443744199 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:27:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.580+0000 7f827cf06640 1 -- 172.21.15.112:0/443744199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8278153030 msgr2=0x7f8278143110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.580+0000 7f827cf06640 1 --2- 172.21.15.112:0/443744199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8278153030 0x7f8278143110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.580+0000 7f827cf06640 1 -- 172.21.15.112:0/443744199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82780c7070 msgr2=0x7f82780acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.580+0000 7f827cf06640 1 --2- 172.21.15.112:0/443744199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82780c7070 0x7f82780acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.580+0000 7f827cf06640 1 -- 172.21.15.112:0/443744199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82780ac0d0 con 0x7f82780c3370 2024-04-25T01:27:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.580+0000 7f8277fff640 1 --2- 172.21.15.112:0/443744199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82780c7070 0x7f82780acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.580+0000 7f827cf06640 1 --2- 172.21.15.112:0/443744199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82780c3370 0x7f8278158120 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f826c00c390 tx=0x7f826c00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.580+0000 7f82757fa640 1 -- 172.21.15.112:0/443744199 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c017070 con 0x7f82780c3370 2024-04-25T01:27:38.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.580+0000 7f82757fa640 1 -- 172.21.15.112:0/443744199 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f826c005ce0 con 0x7f82780c3370 2024-04-25T01:27:38.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.580+0000 7f82757fa640 1 -- 172.21.15.112:0/443744199 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c0072f0 con 0x7f82780c3370 2024-04-25T01:27:38.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.580+0000 7f827f16a640 1 -- 172.21.15.112:0/443744199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82780ac370 con 0x7f82780c3370 2024-04-25T01:27:38.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.580+0000 7f827f16a640 1 -- 172.21.15.112:0/443744199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8278157c00 con 0x7f82780c3370 2024-04-25T01:27:38.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.581+0000 7f827f16a640 1 -- 172.21.15.112:0/443744199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82780c8320 con 0x7f82780c3370 2024-04-25T01:27:38.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.582+0000 7f82757fa640 1 -- 172.21.15.112:0/443744199 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f826c01a070 con 0x7f82780c3370 2024-04-25T01:27:38.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.582+0000 7f82757fa640 1 --2- 172.21.15.112:0/443744199 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f825c06be60 0x7f825c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:38.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.582+0000 7f82757fa640 1 -- 172.21.15.112:0/443744199 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f826c08a3d0 con 0x7f82780c3370 2024-04-25T01:27:38.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.582+0000 7f8277fff640 1 --2- 172.21.15.112:0/443744199 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f825c06be60 0x7f825c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:38.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.583+0000 7f8277fff640 1 --2- 172.21.15.112:0/443744199 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f825c06be60 0x7f825c06e320 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8264034000 tx=0x7f826402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:38.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.591+0000 7f82757fa640 1 -- 172.21.15.112:0/443744199 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f82780c8320 con 0x7f82780c3370 2024-04-25T01:27:38.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.845+0000 7f827f16a640 1 -- 172.21.15.112:0/443744199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f82780689b0 con 0x7f82780c3370 2024-04-25T01:27:38.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.845+0000 7f82757fa640 1 -- 172.21.15.112:0/443744199 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f826c058420 con 0x7f82780c3370 2024-04-25T01:27:38.824 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:27:38.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.847+0000 7f827f16a640 1 -- 172.21.15.112:0/443744199 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f825c06be60 msgr2=0x7f825c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:38.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.847+0000 7f827f16a640 1 --2- 172.21.15.112:0/443744199 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f825c06be60 0x7f825c06e320 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8264034000 tx=0x7f826402e000 comp rx=0 tx=0).stop 2024-04-25T01:27:38.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.847+0000 7f827f16a640 1 -- 172.21.15.112:0/443744199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82780c3370 msgr2=0x7f8278158120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:38.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.847+0000 7f827f16a640 1 --2- 172.21.15.112:0/443744199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82780c3370 0x7f8278158120 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f826c00c390 tx=0x7f826c00c790 comp rx=0 tx=0).stop 2024-04-25T01:27:38.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.847+0000 7f827f16a640 1 -- 172.21.15.112:0/443744199 shutdown_connections 2024-04-25T01:27:38.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.847+0000 7f827f16a640 1 --2- 172.21.15.112:0/443744199 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f825c06be60 0x7f825c06e320 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:38.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.847+0000 7f827f16a640 1 --2- 172.21.15.112:0/443744199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8278153030 0x7f8278143110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:38.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.847+0000 7f827f16a640 1 --2- 172.21.15.112:0/443744199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82780c7070 0x7f82780acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:38.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.847+0000 7f827f16a640 1 --2- 172.21.15.112:0/443744199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82780c3370 0x7f8278158120 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:38.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.847+0000 7f827f16a640 1 -- 172.21.15.112:0/443744199 >> 172.21.15.112:0/443744199 conn(0x7f82780c9140 msgr2=0x7f82780c5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:38.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.847+0000 7f827f16a640 1 -- 172.21.15.112:0/443744199 shutdown_connections 2024-04-25T01:27:38.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.847+0000 7f827f16a640 1 -- 172.21.15.112:0/443744199 wait complete. 2024-04-25T01:27:38.837 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:27:38.837 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T01:27:38.837 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:27:38.837 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:27:38.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.956+0000 7f5dd49c4640 1 Processor -- start 2024-04-25T01:27:38.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.956+0000 7f5dd49c4640 1 -- start start 2024-04-25T01:27:38.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.957+0000 7f5dd49c4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5dd01592c0 0x7f5dd00c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:38.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.957+0000 7f5dd49c4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5dd01596a0 0x7f5dd0152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:38.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.957+0000 7f5dd49c4640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dd0159b90 con 0x7f5dd00c72b0 2024-04-25T01:27:38.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.957+0000 7f5dd49c4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f5dd00c6e20 con 0x7f5dd01592c0 2024-04-25T01:27:38.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.957+0000 7f5dd49c4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5dd006c970 con 0x7f5dd01596a0 2024-04-25T01:27:38.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.957+0000 7f5dce59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5dd01592c0 0x7f5dd00c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:38.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.957+0000 7f5dce59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5dd01592c0 0x7f5dd00c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56294/0 (socket says 172.21.15.112:56294) 2024-04-25T01:27:38.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.957+0000 7f5dce59c640 1 -- 172.21.15.112:0/844086862 learned_addr learned my addr 172.21.15.112:0/844086862 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:27:38.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.957+0000 7f5dc7fff640 1 --2- 172.21.15.112:0/844086862 >> v2:172.21.15.187:3300/0 conn(0x7f5dd01596a0 0x7f5dd0152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:38.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.957+0000 7f5dce59c640 1 -- 172.21.15.112:0/844086862 >> v2:172.21.15.187:3300/0 conn(0x7f5dd01596a0 msgr2=0x7f5dd0152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:38.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.957+0000 7f5dce59c640 1 --2- 172.21.15.112:0/844086862 >> v2:172.21.15.187:3300/0 conn(0x7f5dd01596a0 0x7f5dd0152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:38.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.957+0000 7f5dce59c640 1 -- 172.21.15.112:0/844086862 >> v1:172.21.15.187:6789/0 conn(0x7f5dd00c72b0 legacy=0x7f5dd0155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:38.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.957+0000 7f5dce59c640 1 -- 172.21.15.112:0/844086862 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd006cc50 con 0x7f5dd01592c0 2024-04-25T01:27:38.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.958+0000 7f5dce59c640 1 --2- 172.21.15.112:0/844086862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01592c0 0x7f5dd00c9070 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00bea40 tx=0x7f5dc002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0614c1726137bf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:38.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.958+0000 7f5dcdd9b640 1 -- 172.21.15.112:0/844086862 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0037070 con 0x7f5dd01592c0 2024-04-25T01:27:38.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.958+0000 7f5dcdd9b640 1 -- 172.21.15.112:0/844086862 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dc0032440 con 0x7f5dd01592c0 2024-04-25T01:27:38.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.958+0000 7f5dcdd9b640 1 -- 172.21.15.112:0/844086862 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0032810 con 0x7f5dd01592c0 2024-04-25T01:27:38.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.959+0000 7f5dd49c4640 1 -- 172.21.15.112:0/844086862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01592c0 msgr2=0x7f5dd00c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:38.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.959+0000 7f5dd49c4640 1 --2- 172.21.15.112:0/844086862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01592c0 0x7f5dd00c9070 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00bea40 tx=0x7f5dc002a150 comp rx=0 tx=0).stop 2024-04-25T01:27:38.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.959+0000 7f5dd49c4640 1 -- 172.21.15.112:0/844086862 shutdown_connections 2024-04-25T01:27:38.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.959+0000 7f5dd49c4640 1 --2- 172.21.15.112:0/844086862 >> v2:172.21.15.187:3300/0 conn(0x7f5dd01596a0 0x7f5dd0152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:38.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.959+0000 7f5dd49c4640 1 --2- 172.21.15.112:0/844086862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01592c0 0x7f5dd00c9070 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:38.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.959+0000 7f5dd49c4640 1 -- 172.21.15.112:0/844086862 >> 172.21.15.112:0/844086862 conn(0x7f5dd00cb170 msgr2=0x7f5dd00cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:38.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.959+0000 7f5dd49c4640 1 -- 172.21.15.112:0/844086862 shutdown_connections 2024-04-25T01:27:38.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.959+0000 7f5dd49c4640 1 -- 172.21.15.112:0/844086862 wait complete. 2024-04-25T01:27:38.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.960+0000 7f5dd49c4640 1 Processor -- start 2024-04-25T01:27:38.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.960+0000 7f5dd49c4640 1 -- start start 2024-04-25T01:27:38.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.961+0000 7f5dd49c4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dd00c72b0 0x7f5dd01e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:38.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.961+0000 7f5dd49c4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd01592c0 0x7f5dd01ebed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:38.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.961+0000 7f5dd49c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01596a0 0x7f5dd01ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:38.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.961+0000 7f5dd49c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd006dca0 con 0x7f5dd01596a0 2024-04-25T01:27:38.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.961+0000 7f5dd49c4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd006db20 con 0x7f5dd01592c0 2024-04-25T01:27:38.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.961+0000 7f5dd49c4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5dd00c95b0 con 0x7f5dd00c72b0 2024-04-25T01:27:38.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.961+0000 7f5dced9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01596a0 0x7f5dd01ef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:38.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.961+0000 7f5dced9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01596a0 0x7f5dd01ef2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56296/0 (socket says 172.21.15.112:56296) 2024-04-25T01:27:38.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.961+0000 7f5dced9d640 1 -- 172.21.15.112:0/1741381404 learned_addr learned my addr 172.21.15.112:0/1741381404 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:27:38.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.961+0000 7f5dce59c640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dd00c72b0 0x7f5dd01e2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:38.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.961+0000 7f5dc7fff640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd01592c0 0x7f5dd01ebed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:38.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.961+0000 7f5dced9d640 1 -- 172.21.15.112:0/1741381404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dd00c72b0 msgr2=0x7f5dd01e2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.961+0000 7f5dced9d640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dd00c72b0 0x7f5dd01e2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.961+0000 7f5dced9d640 1 -- 172.21.15.112:0/1741381404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd01592c0 msgr2=0x7f5dd01ebed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.961+0000 7f5dced9d640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd01592c0 0x7f5dd01ebed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.961+0000 7f5dced9d640 1 -- 172.21.15.112:0/1741381404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd0149c00 con 0x7f5dd01596a0 2024-04-25T01:27:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.961+0000 7f5dced9d640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01596a0 0x7f5dd01ef2c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5dc800c7a0 tx=0x7f5dc800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.962+0000 7f5dc6ffd640 1 -- 172.21.15.112:0/1741381404 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8003b70 con 0x7f5dd01596a0 2024-04-25T01:27:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.962+0000 7f5dc6ffd640 1 -- 172.21.15.112:0/1741381404 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dc8003d30 con 0x7f5dd01596a0 2024-04-25T01:27:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.962+0000 7f5dc6ffd640 1 -- 172.21.15.112:0/1741381404 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc800aa90 con 0x7f5dd01596a0 2024-04-25T01:27:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.962+0000 7f5dd49c4640 1 -- 172.21.15.112:0/1741381404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd0149ee0 con 0x7f5dd01596a0 2024-04-25T01:27:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.962+0000 7f5dd49c4640 1 -- 172.21.15.112:0/1741381404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd014a4a0 con 0x7f5dd01596a0 2024-04-25T01:27:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.962+0000 7f5dd49c4640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01eb790 0x7f5dd01ef800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.962+0000 7f5dd49c4640 1 -- 172.21.15.112:0/1741381404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd0075980 con 0x7f5dd01eb790 2024-04-25T01:27:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.963+0000 7f5dce59c640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01eb790 0x7f5dd01ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.963+0000 7f5dce59c640 1 -- 172.21.15.112:0/1741381404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5dd00679a0 con 0x7f5dd01eb790 2024-04-25T01:27:38.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.963+0000 7f5dce59c640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01eb790 0x7f5dd01ef800 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00c21d0 tx=0x7f5dc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:38.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.963+0000 7f5dc6ffd640 1 -- 172.21.15.112:0/1741381404 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0043070 con 0x7f5dd01eb790 2024-04-25T01:27:38.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.963+0000 7f5dc6ffd640 1 -- 172.21.15.112:0/1741381404 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5dc800ac30 con 0x7f5dd01596a0 2024-04-25T01:27:38.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.964+0000 7f5dc6ffd640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5db006bd60 0x7f5db006e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:38.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.964+0000 7f5dc7fff640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5db006bd60 0x7f5db006e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:38.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.964+0000 7f5dc6ffd640 1 -- 172.21.15.112:0/1741381404 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f5dc808afd0 con 0x7f5dd01596a0 2024-04-25T01:27:38.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.964+0000 7f5dc6ffd640 1 -- 172.21.15.112:0/1741381404 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5dc003c640 con 0x7f5dd01eb790 2024-04-25T01:27:38.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.964+0000 7f5dc6ffd640 1 -- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01eb790 msgr2=0x7f5dd01ef800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:38.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.964+0000 7f5dc6ffd640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01eb790 0x7f5dd01ef800 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00c21d0 tx=0x7f5dc00046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:38.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.964+0000 7f5dc6ffd640 1 -- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01eb790 msgr2=0x7f5dd01ef800 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:27:38.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.964+0000 7f5dc6ffd640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01eb790 0x7f5dd01ef800 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:38.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.964+0000 7f5dc7fff640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5db006bd60 0x7f5db006e220 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5dd014a710 tx=0x7f5db8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:38.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.970+0000 7f5dd49c4640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01eed00 0x7f5dd0155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:38.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.970+0000 7f5dd49c4640 1 -- 172.21.15.112:0/1741381404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd006d820 con 0x7f5dd01eed00 2024-04-25T01:27:38.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.970+0000 7f5dce59c640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01eed00 0x7f5dd0155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:38.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.971+0000 7f5dce59c640 1 -- 172.21.15.112:0/1741381404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5dd00c79d0 con 0x7f5dd01eed00 2024-04-25T01:27:38.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.971+0000 7f5dce59c640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01eed00 0x7f5dd0155ae0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5dc002a190 tx=0x7f5dc0008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.971+0000 7f5dc6ffd640 1 -- 172.21.15.112:0/1741381404 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0037040 con 0x7f5dd01eed00 2024-04-25T01:27:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.971+0000 7f5dc6ffd640 1 -- 172.21.15.112:0/1741381404 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5072 (secure 0 0 0) 0x7f5dc0003610 con 0x7f5dd01eed00 2024-04-25T01:27:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.971+0000 7f5dc6ffd640 1 -- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01eed00 msgr2=0x7f5dd0155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.971+0000 7f5dc6ffd640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01eed00 0x7f5dd0155ae0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5dc002a190 tx=0x7f5dc0008030 comp rx=0 tx=0).stop 2024-04-25T01:27:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.971+0000 7f5dc6ffd640 1 -- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01eed00 msgr2=0x7f5dd0155ae0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:27:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.971+0000 7f5dc6ffd640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01eed00 0x7f5dd0155ae0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:38.952 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:27:38.952 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:27:38.952 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:27:38.952 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:27:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 10, 2024-04-25T01:27:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 130599, 2024-04-25T01:27:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:27:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:27:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:27:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:27:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 12, 2024-04-25T01:27:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:27:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:27:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:27:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:27:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:27:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:38.955 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:38.955 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:38.955 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:38.955 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:38.955 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:38.955 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:38.955 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:38.955 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:38.955 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:27:38.955 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:27:38.955 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:27:38.955 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:27:38.955 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:27:38.955 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:27:38.955 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:27:38.956 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:27:38.956 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:27:38.956 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:27:38.956 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:27:38.956 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:27:38.956 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:27:38.956 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:27:38.956 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:27:38.956 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:27:38.956 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:27:38.956 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:38.956 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:38.956 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:38.956 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:38.956 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:38.957 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:38.957 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:38.957 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:38.957 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:38.957 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:38.957 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:38.957 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:38.957 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:27:38.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:38.957 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:27:38.957 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:38.957 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:27:38.957 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:27:38.957 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:38.958 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:38.958 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:38.958 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:38.958 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:27:38.958 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:38.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:38.958 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:38.958 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:38.958 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:27:38.958 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:38.958 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:38.958 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:38.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:38.958 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:27:38.959 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:27:38.959 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:38.959 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:38.959 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:38.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:38.959 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:38.959 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:27:38.959 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:27:38.959 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:38.959 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:38.959 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:38.959 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:38.959 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:27:38.959 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:38.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:38.960 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:38.960 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:38.960 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:27:38.960 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:38.960 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:38.960 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:38.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:38.960 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:27:38.960 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:27:38.960 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:38.960 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:38.960 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:38.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:38.960 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:38.960 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:27:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:27:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:27:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:27:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:27:38.962 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:27:38.962 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:38.962 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:38.962 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:38.962 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:38.962 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:38.962 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:38.962 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:27:38.962 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:27:38.962 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:38.962 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:38.962 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:38.962 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:27:38.962 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:38.962 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:38.963 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:27:38.963 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:38.963 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:38.963 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:38.963 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:27:38.963 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:38.963 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:38.963 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:27:38.963 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:38.963 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:38.963 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:38.963 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:27:38.963 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:38.963 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:38.963 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:27:38.964 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:38.964 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:38.964 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:38.964 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 7 2024-04-25T01:27:38.964 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:38.964 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:38.964 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:27:38.964 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:38.964 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:38.964 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:38.964 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:27:38.964 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:38.964 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:38.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:38.964 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:27:38.965 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:27:38.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.972+0000 7f5dd49c4640 1 -- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5db006bd60 msgr2=0x7f5db006e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:38.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.972+0000 7f5dd49c4640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5db006bd60 0x7f5db006e220 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5dd014a710 tx=0x7f5db8009190 comp rx=0 tx=0).stop 2024-04-25T01:27:38.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.972+0000 7f5dd49c4640 1 -- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01596a0 msgr2=0x7f5dd01ef2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:38.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.972+0000 7f5dced9d640 1 -- 172.21.15.112:0/1741381404 reap_dead start 2024-04-25T01:27:38.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.972+0000 7f5dd49c4640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01596a0 0x7f5dd01ef2c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5dc800c7a0 tx=0x7f5dc800cc60 comp rx=0 tx=0).stop 2024-04-25T01:27:38.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.972+0000 7f5dd49c4640 1 -- 172.21.15.112:0/1741381404 shutdown_connections 2024-04-25T01:27:38.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.972+0000 7f5dd49c4640 1 --2- 172.21.15.112:0/1741381404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd01596a0 0x7f5dd01ef2c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:38.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.972+0000 7f5dd49c4640 1 -- 172.21.15.112:0/1741381404 >> 172.21.15.112:0/1741381404 conn(0x7f5dd00cb170 msgr2=0x7f5dd00aac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:38.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.972+0000 7f5dd49c4640 1 -- 172.21.15.112:0/1741381404 shutdown_connections 2024-04-25T01:27:38.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:38.972+0000 7f5dd49c4640 1 -- 172.21.15.112:0/1741381404 wait complete. 2024-04-25T01:27:38.967 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:27:38.967 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:27:38.967 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:27:39.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.086+0000 7f094a529640 1 Processor -- start 2024-04-25T01:27:39.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.086+0000 7f094a529640 1 -- start start 2024-04-25T01:27:39.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.086+0000 7f094a529640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f09441596a0 0x7f0944152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:39.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.086+0000 7f094a529640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09440c72b0 0x7f0944160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:39.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.086+0000 7f094a529640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0944159b90 con 0x7f09441592c0 2024-04-25T01:27:39.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.086+0000 7f094a529640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f09440c6e20 con 0x7f09441596a0 2024-04-25T01:27:39.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.086+0000 7f094a529640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f094406c970 con 0x7f09440c72b0 2024-04-25T01:27:39.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.087+0000 7f09437fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f09441596a0 0x7f0944152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:39.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.087+0000 7f0943fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f09441592c0 0x7f09440c9070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:48286/0 (socket says 172.21.15.112:48286) 2024-04-25T01:27:39.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.087+0000 7f0943fff640 1 -- 172.21.15.112:0/2450736951 learned_addr learned my addr 172.21.15.112:0/2450736951 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:27:39.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.087+0000 7f0948ac6640 1 --2- 172.21.15.112:0/2450736951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09440c72b0 0x7f0944160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:39.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.087+0000 7f09437fe640 1 -- 172.21.15.112:0/2450736951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09440c72b0 msgr2=0x7f0944160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:39.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.087+0000 7f09437fe640 1 --2- 172.21.15.112:0/2450736951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09440c72b0 0x7f0944160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:39.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.087+0000 7f09437fe640 1 -- 172.21.15.112:0/2450736951 >> v1:172.21.15.112:6789/0 conn(0x7f09441592c0 legacy=0x7f09440c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:39.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.087+0000 7f09437fe640 1 -- 172.21.15.112:0/2450736951 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094406cc50 con 0x7f09441596a0 2024-04-25T01:27:39.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.087+0000 7f0942ffd640 1 -- 172.21.15.112:0/2450736951 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1842639490 0 0) 0x7f0944159b90 con 0x7f09441592c0 2024-04-25T01:27:39.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.088+0000 7f09437fe640 1 --2- 172.21.15.112:0/2450736951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09441596a0 0x7f0944152d00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f09340097a0 tx=0x7f093402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ce0c1ac9c1b72f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:39.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.088+0000 7f0942ffd640 1 -- 172.21.15.112:0/2450736951 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934037070 con 0x7f09441596a0 2024-04-25T01:27:39.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.088+0000 7f0942ffd640 1 -- 172.21.15.112:0/2450736951 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f093402adb0 con 0x7f09441596a0 2024-04-25T01:27:39.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.088+0000 7f0942ffd640 1 -- 172.21.15.112:0/2450736951 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934032660 con 0x7f09441596a0 2024-04-25T01:27:39.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.088+0000 7f094a529640 1 -- 172.21.15.112:0/2450736951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09441596a0 msgr2=0x7f0944152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:39.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.088+0000 7f094a529640 1 --2- 172.21.15.112:0/2450736951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09441596a0 0x7f0944152d00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f09340097a0 tx=0x7f093402a060 comp rx=0 tx=0).stop 2024-04-25T01:27:39.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.088+0000 7f094a529640 1 -- 172.21.15.112:0/2450736951 shutdown_connections 2024-04-25T01:27:39.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.088+0000 7f094a529640 1 --2- 172.21.15.112:0/2450736951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09440c72b0 0x7f0944160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:39.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.088+0000 7f094a529640 1 --2- 172.21.15.112:0/2450736951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09441596a0 0x7f0944152d00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:39.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.088+0000 7f094a529640 1 -- 172.21.15.112:0/2450736951 >> 172.21.15.112:0/2450736951 conn(0x7f09440cb170 msgr2=0x7f09440cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:39.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.088+0000 7f094a529640 1 -- 172.21.15.112:0/2450736951 shutdown_connections 2024-04-25T01:27:39.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.088+0000 7f094a529640 1 -- 172.21.15.112:0/2450736951 wait complete. 2024-04-25T01:27:39.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.089+0000 7f094a529640 1 Processor -- start 2024-04-25T01:27:39.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.090+0000 7f094a529640 1 -- start start 2024-04-25T01:27:39.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.090+0000 7f094a529640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09440c72b0 0x7f0944149140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:39.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.090+0000 7f094a529640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09441592c0 0x7f0944149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:39.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.090+0000 7f094a529640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09441596a0 0x7f09441eec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:39.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.090+0000 7f094a529640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f094406c510 con 0x7f09440c72b0 2024-04-25T01:27:39.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.090+0000 7f094a529640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f094406c390 con 0x7f09441596a0 2024-04-25T01:27:39.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.090+0000 7f094a529640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f094406db20 con 0x7f09441592c0 2024-04-25T01:27:39.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.090+0000 7f0943fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09440c72b0 0x7f0944149140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:39.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.090+0000 7f0943fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09440c72b0 0x7f0944149140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56334/0 (socket says 172.21.15.112:56334) 2024-04-25T01:27:39.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.090+0000 7f0943fff640 1 -- 172.21.15.112:0/2451456797 learned_addr learned my addr 172.21.15.112:0/2451456797 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:27:39.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.091+0000 7f0948ac6640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09441596a0 0x7f09441eec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:39.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.091+0000 7f09437fe640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09441592c0 0x7f0944149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:39.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.091+0000 7f0943fff640 1 -- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09441592c0 msgr2=0x7f0944149680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:39.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.091+0000 7f0943fff640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09441592c0 0x7f0944149680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:39.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.091+0000 7f0943fff640 1 -- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09441596a0 msgr2=0x7f09441eec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:39.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.091+0000 7f0943fff640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09441596a0 0x7f09441eec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:39.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.091+0000 7f0943fff640 1 -- 172.21.15.112:0/2451456797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09440bf830 con 0x7f09440c72b0 2024-04-25T01:27:39.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.091+0000 7f09437fe640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09441592c0 0x7f0944149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:27:39.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.091+0000 7f0943fff640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09440c72b0 0x7f0944149140 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f093800c4a0 tx=0x7f093800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:39.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.091+0000 7f0940ff9640 1 -- 172.21.15.112:0/2451456797 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938017070 con 0x7f09440c72b0 2024-04-25T01:27:39.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.091+0000 7f0940ff9640 1 -- 172.21.15.112:0/2451456797 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0938005ce0 con 0x7f09440c72b0 2024-04-25T01:27:39.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.091+0000 7f0940ff9640 1 -- 172.21.15.112:0/2451456797 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09380072b0 con 0x7f09440c72b0 2024-04-25T01:27:39.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.091+0000 7f094a529640 1 -- 172.21.15.112:0/2451456797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09440c22e0 con 0x7f09440c72b0 2024-04-25T01:27:39.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.091+0000 7f094a529640 1 -- 172.21.15.112:0/2451456797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0944150280 con 0x7f09440c72b0 2024-04-25T01:27:39.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.092+0000 7f094a529640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0944149cd0 0x7f09440c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:39.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.092+0000 7f094a529640 1 -- 172.21.15.112:0/2451456797 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f09440c5560 con 0x7f0944149cd0 2024-04-25T01:27:39.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.092+0000 7f09437fe640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0944149cd0 0x7f09440c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:39.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.092+0000 7f09437fe640 1 -- 172.21.15.112:0/2451456797 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0944075980 con 0x7f0944149cd0 2024-04-25T01:27:39.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.093+0000 7f09437fe640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0944149cd0 0x7f09440c5020 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f094408cb00 tx=0x7f09340046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:39.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.093+0000 7f0940ff9640 1 -- 172.21.15.112:0/2451456797 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f093801a070 con 0x7f09440c72b0 2024-04-25T01:27:39.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.093+0000 7f0940ff9640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f093006be30 0x7f093006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:39.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.093+0000 7f0940ff9640 1 -- 172.21.15.112:0/2451456797 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f093808a490 con 0x7f09440c72b0 2024-04-25T01:27:39.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.093+0000 7f0948ac6640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f093006be30 0x7f093006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:39.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.093+0000 7f0940ff9640 1 -- 172.21.15.112:0/2451456797 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934041070 con 0x7f0944149cd0 2024-04-25T01:27:39.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.093+0000 7f0940ff9640 1 -- 172.21.15.112:0/2451456797 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f09340039d0 con 0x7f0944149cd0 2024-04-25T01:27:39.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.093+0000 7f0940ff9640 1 -- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0944149cd0 msgr2=0x7f09440c5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:39.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.093+0000 7f0940ff9640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0944149cd0 0x7f09440c5020 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f094408cb00 tx=0x7f09340046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:39.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.093+0000 7f0940ff9640 1 -- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0944149cd0 msgr2=0x7f09440c5020 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:27:39.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.093+0000 7f0940ff9640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0944149cd0 0x7f09440c5020 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:39.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.094+0000 7f0948ac6640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f093006be30 0x7f093006e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0944151cd0 tx=0x7f093c009780 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:39.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.100+0000 7f094a529640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09441ef1c0 0x7f09441eb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:39.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.100+0000 7f094a529640 1 -- 172.21.15.112:0/2451456797 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f09440682a0 con 0x7f09441ef1c0 2024-04-25T01:27:39.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.100+0000 7f09437fe640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09441ef1c0 0x7f09441eb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:39.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.100+0000 7f09437fe640 1 -- 172.21.15.112:0/2451456797 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f09440c79d0 con 0x7f09441ef1c0 2024-04-25T01:27:39.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.100+0000 7f09437fe640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09441ef1c0 0x7f09441eb4a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f09340097a0 tx=0x7f0934008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:39.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.101+0000 7f0940ff9640 1 -- 172.21.15.112:0/2451456797 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934037040 con 0x7f09441ef1c0 2024-04-25T01:27:39.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.101+0000 7f0940ff9640 1 -- 172.21.15.112:0/2451456797 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4559 (secure 0 0 0) 0x7f0934051420 con 0x7f09441ef1c0 2024-04-25T01:27:39.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.101+0000 7f0940ff9640 1 -- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09441ef1c0 msgr2=0x7f09441eb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:39.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.101+0000 7f0940ff9640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09441ef1c0 0x7f09441eb4a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f09340097a0 tx=0x7f0934008030 comp rx=0 tx=0).stop 2024-04-25T01:27:39.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.101+0000 7f0940ff9640 1 -- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09441ef1c0 msgr2=0x7f09441eb4a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:27:39.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.101+0000 7f0940ff9640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09441ef1c0 0x7f09441eb4a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:39.080 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:27:39.081 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:27:39.081 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:27:39.081 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:27:39.081 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 10, 2024-04-25T01:27:39.081 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 27240, 2024-04-25T01:27:39.081 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:27:39.081 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:27:39.081 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:27:39.081 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:27:39.081 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:39.081 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 12, 2024-04-25T01:27:39.081 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:27:39.081 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:27:39.082 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:27:39.082 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:39.082 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:39.082 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:39.082 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:39.082 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:39.082 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:39.082 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:39.082 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:39.082 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:39.082 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:39.082 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:39.082 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:39.082 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:27:39.083 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:27:39.083 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:39.083 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:39.083 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:39.083 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:39.083 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:39.083 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:39.083 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:39.083 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:39.083 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:39.083 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:39.083 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:39.083 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:39.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:27:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:27:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:27:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:27:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:27:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:27:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:27:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:27:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:27:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:27:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:27:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:27:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:27:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:27:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:27:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:27:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:27:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:27:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:27:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:27:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:27:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:27:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:27:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:27:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:27:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:27:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:27:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.088 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:39.088 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:27:39.088 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:39.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.088 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.088 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:39.088 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:27:39.088 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:39.088 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:39.088 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:39.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.088 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:27:39.088 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:27:39.088 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:39.088 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:27:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:27:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:27:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:27:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:39.090 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:39.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.090 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:27:39.090 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:27:39.090 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:39.090 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:39.090 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:39.090 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:39.090 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:39.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.090 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:27:39.090 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:27:39.090 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.090 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:39.091 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:39.091 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:27:39.091 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:39.091 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:39.091 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:27:39.091 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.091 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:39.091 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:39.091 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:27:39.091 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:39.091 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:39.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.091 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:27:39.091 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:27:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.101+0000 7f094a529640 1 -- 172.21.15.112:0/2451456797 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f093006be30 msgr2=0x7f093006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.101+0000 7f094a529640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f093006be30 0x7f093006e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0944151cd0 tx=0x7f093c009780 comp rx=0 tx=0).stop 2024-04-25T01:27:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.101+0000 7f094a529640 1 -- 172.21.15.112:0/2451456797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09440c72b0 msgr2=0x7f0944149140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.101+0000 7f094a529640 1 --2- 172.21.15.112:0/2451456797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09440c72b0 0x7f0944149140 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f093800c4a0 tx=0x7f093800c960 comp rx=0 tx=0).stop 2024-04-25T01:27:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.101+0000 7f0948ac6640 1 -- 172.21.15.112:0/2451456797 reap_dead start 2024-04-25T01:27:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.101+0000 7f0948ac6640 1 -- 172.21.15.112:0/2451456797 reap_dead start 2024-04-25T01:27:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.101+0000 7f094a529640 1 -- 172.21.15.112:0/2451456797 shutdown_connections 2024-04-25T01:27:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.101+0000 7f094a529640 1 -- 172.21.15.112:0/2451456797 >> 172.21.15.112:0/2451456797 conn(0x7f09440cb170 msgr2=0x7f09440ac220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.101+0000 7f094a529640 1 -- 172.21.15.112:0/2451456797 shutdown_connections 2024-04-25T01:27:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.101+0000 7f094a529640 1 -- 172.21.15.112:0/2451456797 wait complete. 2024-04-25T01:27:39.094 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:27:39.094 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:27:39.094 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:27:39.095 INFO:tasks.workunit.client.1.smithi187.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-25T01:27:39.095 INFO:tasks.workunit.client.1.smithi187.stderr: 2024-04-25T01:27:39.095 INFO:tasks.workunit.client.1.smithi187.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T01:27:39.095 INFO:tasks.workunit.client.1.smithi187.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T01:27:39.095 INFO:tasks.workunit.client.1.smithi187.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T01:27:39.096 INFO:tasks.workunit.client.1.smithi187.stderr: 2024-04-25T01:27:39.096 INFO:tasks.workunit.client.1.smithi187.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T01:27:39.096 INFO:tasks.workunit.client.1.smithi187.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T01:27:39.096 INFO:tasks.workunit.client.1.smithi187.stderr: 2024-04-25T01:27:39.096 INFO:tasks.workunit.client.1.smithi187.stderr: git switch -c 2024-04-25T01:27:39.096 INFO:tasks.workunit.client.1.smithi187.stderr: 2024-04-25T01:27:39.096 INFO:tasks.workunit.client.1.smithi187.stderr:Or undo this operation with: 2024-04-25T01:27:39.096 INFO:tasks.workunit.client.1.smithi187.stderr: 2024-04-25T01:27:39.096 INFO:tasks.workunit.client.1.smithi187.stderr: git switch - 2024-04-25T01:27:39.096 INFO:tasks.workunit.client.1.smithi187.stderr: 2024-04-25T01:27:39.096 INFO:tasks.workunit.client.1.smithi187.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T01:27:39.096 INFO:tasks.workunit.client.1.smithi187.stderr: 2024-04-25T01:27:39.096 INFO:tasks.workunit.client.1.smithi187.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-25T01:27:39.097 DEBUG:teuthology.orchestra.run.smithi187:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-25T01:27:39.122 INFO:tasks.workunit.client.1.smithi187.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T01:27:39.124 INFO:tasks.workunit.client.1.smithi187.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-25T01:27:39.124 INFO:tasks.workunit.client.1.smithi187.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T01:27:39.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.215+0000 7fb60e4bd640 1 Processor -- start 2024-04-25T01:27:39.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.215+0000 7fb60e4bd640 1 -- start start 2024-04-25T01:27:39.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.216+0000 7fb60e4bd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb608154b50 0x7fb6080c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:39.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.216+0000 7fb60e4bd640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb608154fd0 con 0x7fb6080c8810 2024-04-25T01:27:39.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.216+0000 7fb60e4bd640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb608154120 con 0x7fb608071520 2024-04-25T01:27:39.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.216+0000 7fb60e4bd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb6080c7f10 con 0x7fb608154b50 2024-04-25T01:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.216+0000 7fb60ca5a640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb6080c8810 0x7fb608153660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:48294/0 (socket says 172.21.15.112:48294) 2024-04-25T01:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.216+0000 7fb60ca5a640 1 -- 172.21.15.112:0/115834101 learned_addr learned my addr 172.21.15.112:0/115834101 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.216+0000 7fb6077fe640 1 --2- 172.21.15.112:0/115834101 >> v2:172.21.15.187:3300/0 conn(0x7fb608154b50 0x7fb6080c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.216+0000 7fb606ffd640 1 -- 172.21.15.112:0/115834101 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 509361316 0 0) 0x7fb600001490 con 0x7fb6080c8810 2024-04-25T01:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.216+0000 7fb606ffd640 1 -- 172.21.15.112:0/115834101 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5f0003520 con 0x7fb6080c8810 2024-04-25T01:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.216+0000 7fb606ffd640 1 -- 172.21.15.112:0/115834101 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1254443044 0 0) 0x7fb5f0003520 con 0x7fb6080c8810 2024-04-25T01:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.217+0000 7fb6077fe640 1 -- 172.21.15.112:0/115834101 >> v1:172.21.15.187:6789/0 conn(0x7fb608071520 legacy=0x7fb6080c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.217+0000 7fb6077fe640 1 -- 172.21.15.112:0/115834101 >> v1:172.21.15.112:6789/0 conn(0x7fb6080c8810 legacy=0x7fb608153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.217+0000 7fb6077fe640 1 -- 172.21.15.112:0/115834101 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb608154400 con 0x7fb608154b50 2024-04-25T01:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.217+0000 7fb6077fe640 1 --2- 172.21.15.112:0/115834101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb608154b50 0x7fb6080c82d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80097a0 tx=0x7fb5f802a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef4c673144e7f068 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.217+0000 7fb606ffd640 1 -- 172.21.15.112:0/115834101 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8037070 con 0x7fb608154b50 2024-04-25T01:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.217+0000 7fb606ffd640 1 -- 172.21.15.112:0/115834101 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f802ab30 con 0x7fb608154b50 2024-04-25T01:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.217+0000 7fb606ffd640 1 -- 172.21.15.112:0/115834101 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb600001bc0 con 0x7fb608154b50 2024-04-25T01:27:39.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.218+0000 7fb60e4bd640 1 -- 172.21.15.112:0/115834101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb608154b50 msgr2=0x7fb6080c82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:39.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.218+0000 7fb60e4bd640 1 --2- 172.21.15.112:0/115834101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb608154b50 0x7fb6080c82d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80097a0 tx=0x7fb5f802a0a0 comp rx=0 tx=0).stop 2024-04-25T01:27:39.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.218+0000 7fb60e4bd640 1 -- 172.21.15.112:0/115834101 shutdown_connections 2024-04-25T01:27:39.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.218+0000 7fb60e4bd640 1 --2- 172.21.15.112:0/115834101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb608154b50 0x7fb6080c82d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:39.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.218+0000 7fb60e4bd640 1 -- 172.21.15.112:0/115834101 >> 172.21.15.112:0/115834101 conn(0x7fb608074a00 msgr2=0x7fb6080c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:39.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.218+0000 7fb60e4bd640 1 -- 172.21.15.112:0/115834101 shutdown_connections 2024-04-25T01:27:39.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.218+0000 7fb60e4bd640 1 -- 172.21.15.112:0/115834101 wait complete. 2024-04-25T01:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.219+0000 7fb60e4bd640 1 Processor -- start 2024-04-25T01:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.219+0000 7fb60e4bd640 1 -- start start 2024-04-25T01:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.220+0000 7fb60e4bd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb608071520 0x7fb6080be0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.220+0000 7fb60e4bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6080c8810 0x7fb60806fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.220+0000 7fb60e4bd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb608154b50 0x7fb6081e9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.220+0000 7fb60e4bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb608153d70 con 0x7fb6080c8810 2024-04-25T01:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.220+0000 7fb60e4bd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb6080c72e0 con 0x7fb608154b50 2024-04-25T01:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.220+0000 7fb60e4bd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb6080be9b0 con 0x7fb608071520 2024-04-25T01:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.220+0000 7fb6077fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6080c8810 0x7fb60806fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.220+0000 7fb6077fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6080c8810 0x7fb60806fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56342/0 (socket says 172.21.15.112:56342) 2024-04-25T01:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.220+0000 7fb6077fe640 1 -- 172.21.15.112:0/825509543 learned_addr learned my addr 172.21.15.112:0/825509543 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.220+0000 7fb607fff640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb608071520 0x7fb6080be0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.220+0000 7fb60ca5a640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb608154b50 0x7fb6081e9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.220+0000 7fb6077fe640 1 -- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb608071520 msgr2=0x7fb6080be0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.220+0000 7fb6077fe640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb608071520 0x7fb6080be0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.220+0000 7fb6077fe640 1 -- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb608154b50 msgr2=0x7fb6081e9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.220+0000 7fb6077fe640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb608154b50 0x7fb6081e9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.220+0000 7fb6077fe640 1 -- 172.21.15.112:0/825509543 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6080be6d0 con 0x7fb6080c8810 2024-04-25T01:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.220+0000 7fb607fff640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb608071520 0x7fb6080be0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.220+0000 7fb60ca5a640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb608154b50 0x7fb6081e9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.221+0000 7fb6077fe640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6080c8810 0x7fb60806fd90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8000c00 tx=0x7fb5f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.221+0000 7fb604ff9640 1 -- 172.21.15.112:0/825509543 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8041070 con 0x7fb6080c8810 2024-04-25T01:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.221+0000 7fb604ff9640 1 -- 172.21.15.112:0/825509543 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f803c660 con 0x7fb6080c8810 2024-04-25T01:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.221+0000 7fb60e4bd640 1 -- 172.21.15.112:0/825509543 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6081ec900 con 0x7fb6080c8810 2024-04-25T01:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.221+0000 7fb604ff9640 1 -- 172.21.15.112:0/825509543 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8037040 con 0x7fb6080c8810 2024-04-25T01:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.221+0000 7fb60e4bd640 1 -- 172.21.15.112:0/825509543 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6080702d0 con 0x7fb6080c8810 2024-04-25T01:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.222+0000 7fb60e4bd640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6081ed3c0 0x7fb6081e9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.222+0000 7fb60e4bd640 1 -- 172.21.15.112:0/825509543 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb6081ec780 con 0x7fb6081ed3c0 2024-04-25T01:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.222+0000 7fb607fff640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6081ed3c0 0x7fb6081e9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.222+0000 7fb607fff640 1 -- 172.21.15.112:0/825509543 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb608075980 con 0x7fb6081ed3c0 2024-04-25T01:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.222+0000 7fb604ff9640 1 -- 172.21.15.112:0/825509543 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb5f8044070 con 0x7fb6080c8810 2024-04-25T01:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.222+0000 7fb607fff640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6081ed3c0 0x7fb6081e9750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc001eb0 tx=0x7fb5fc00d950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.223+0000 7fb604ff9640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb5f406be30 0x7fb5f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.223+0000 7fb60ca5a640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb5f406be30 0x7fb5f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.223+0000 7fb604ff9640 1 -- 172.21.15.112:0/825509543 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fb5f80abb00 con 0x7fb6080c8810 2024-04-25T01:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.223+0000 7fb604ff9640 1 -- 172.21.15.112:0/825509543 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc005280 con 0x7fb6081ed3c0 2024-04-25T01:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.223+0000 7fb604ff9640 1 -- 172.21.15.112:0/825509543 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb5fc005420 con 0x7fb6081ed3c0 2024-04-25T01:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.223+0000 7fb604ff9640 1 -- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6081ed3c0 msgr2=0x7fb6081e9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.223+0000 7fb604ff9640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6081ed3c0 0x7fb6081e9750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc001eb0 tx=0x7fb5fc00d950 comp rx=0 tx=0).stop 2024-04-25T01:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.223+0000 7fb604ff9640 1 -- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6081ed3c0 msgr2=0x7fb6081e9750 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.223+0000 7fb604ff9640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6081ed3c0 0x7fb6081e9750 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc001eb0 tx=0x7fb5fc00d950 comp rx=0 tx=0).stop 2024-04-25T01:27:39.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.223+0000 7fb60ca5a640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb5f406be30 0x7fb5f406e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb608156a00 tx=0x7fb600009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:39.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.230+0000 7fb60e4bd640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb608150750 0x7fb60814fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:39.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.230+0000 7fb60e4bd640 1 -- 172.21.15.112:0/825509543 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb6080682a0 con 0x7fb608150750 2024-04-25T01:27:39.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.230+0000 7fb607fff640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb608150750 0x7fb60814fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:39.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.230+0000 7fb607fff640 1 -- 172.21.15.112:0/825509543 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb6080c80b0 con 0x7fb608150750 2024-04-25T01:27:39.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.230+0000 7fb607fff640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb608150750 0x7fb60814fa50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc001eb0 tx=0x7fb5fc001120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:39.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.230+0000 7fb604ff9640 1 -- 172.21.15.112:0/825509543 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc006c60 con 0x7fb608150750 2024-04-25T01:27:39.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.231+0000 7fb604ff9640 1 -- 172.21.15.112:0/825509543 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5070 (secure 0 0 0) 0x7fb5fc006c60 con 0x7fb608150750 2024-04-25T01:27:39.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.231+0000 7fb604ff9640 1 -- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb608150750 msgr2=0x7fb60814fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:39.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.231+0000 7fb604ff9640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb608150750 0x7fb60814fa50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc001eb0 tx=0x7fb5fc001120 comp rx=0 tx=0).stop 2024-04-25T01:27:39.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.231+0000 7fb604ff9640 1 -- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb608150750 msgr2=0x7fb60814fa50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:27:39.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.231+0000 7fb604ff9640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb608150750 0x7fb60814fa50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:39.210 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:27:39.210 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:27:39.210 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:27:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:27:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 10, 2024-04-25T01:27:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 130818, 2024-04-25T01:27:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:27:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:27:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:27:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:27:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 12, 2024-04-25T01:27:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:27:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:27:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:27:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:39.212 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:39.212 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:39.212 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:39.212 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:39.212 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:39.212 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:39.212 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:39.212 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:39.212 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:39.212 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:39.212 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:27:39.212 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:27:39.212 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:39.213 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:39.213 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:39.213 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:39.213 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:39.213 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:39.213 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:39.213 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:39.213 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:39.213 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:39.213 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:39.213 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:39.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.213 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:27:39.213 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:27:39.214 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:27:39.214 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:27:39.214 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:27:39.214 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:27:39.214 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:27:39.214 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:27:39.214 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:27:39.214 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:27:39.214 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:27:39.214 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:27:39.214 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:27:39.214 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:27:39.214 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:27:39.215 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:27:39.215 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:27:39.215 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:39.215 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:39.215 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:39.215 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:39.215 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:39.215 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:39.215 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:39.215 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:39.215 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:39.215 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:39.215 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:39.215 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:39.216 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:27:39.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.216 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:27:39.216 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.216 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:27:39.216 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:27:39.216 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:39.216 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:39.216 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.216 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:39.216 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:27:39.216 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:39.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.216 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.217 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:39.217 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:27:39.217 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:39.217 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:39.217 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:39.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.217 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:27:39.217 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:27:39.217 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:39.217 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:39.217 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:39.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.217 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.218 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:27:39.218 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:27:39.218 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:39.218 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:39.218 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.218 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:39.218 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:27:39.218 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:39.218 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.218 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.218 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:39.218 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:27:39.218 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:39.218 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:39.219 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:39.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.219 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:27:39.219 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:27:39.219 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:39.219 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:39.219 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:39.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.219 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.219 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:27:39.219 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:27:39.219 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:39.219 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:39.220 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.220 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:39.220 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:27:39.220 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:39.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.220 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.220 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:39.220 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:27:39.220 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:39.220 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:39.220 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:39.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.220 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:27:39.220 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:27:39.220 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:39.221 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:39.221 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:39.221 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:39.221 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:39.221 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.221 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:27:39.221 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:27:39.221 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.221 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:39.221 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:39.221 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:27:39.221 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:39.221 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:39.221 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:27:39.222 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.222 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:39.222 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:39.222 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:27:39.222 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:39.222 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:39.222 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:27:39.222 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.222 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:39.222 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:39.222 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:27:39.222 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:39.222 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:39.222 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:27:39.223 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.223 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:39.223 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:39.223 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:27:39.223 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:39.223 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:39.223 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:27:39.223 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:39.223 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:39.223 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:39.223 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:27:39.223 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:39.223 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:39.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:39.223 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:27:39.224 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:27:39.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.231+0000 7fb60e4bd640 1 -- 172.21.15.112:0/825509543 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb5f406be30 msgr2=0x7fb5f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:39.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.231+0000 7fb60e4bd640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb5f406be30 0x7fb5f406e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb608156a00 tx=0x7fb600009040 comp rx=0 tx=0).stop 2024-04-25T01:27:39.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.231+0000 7fb60e4bd640 1 -- 172.21.15.112:0/825509543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6080c8810 msgr2=0x7fb60806fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:39.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.231+0000 7fb60e4bd640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6080c8810 0x7fb60806fd90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8000c00 tx=0x7fb5f80046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:39.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.231+0000 7fb60ca5a640 1 -- 172.21.15.112:0/825509543 reap_dead start 2024-04-25T01:27:39.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.231+0000 7fb60e4bd640 1 -- 172.21.15.112:0/825509543 shutdown_connections 2024-04-25T01:27:39.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.231+0000 7fb60e4bd640 1 --2- 172.21.15.112:0/825509543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6080c8810 0x7fb60806fd90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:39.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.231+0000 7fb60e4bd640 1 -- 172.21.15.112:0/825509543 >> 172.21.15.112:0/825509543 conn(0x7fb608074a00 msgr2=0x7fb6081551b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:39.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.231+0000 7fb60e4bd640 1 -- 172.21.15.112:0/825509543 shutdown_connections 2024-04-25T01:27:39.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:39.231+0000 7fb60e4bd640 1 -- 172.21.15.112:0/825509543 wait complete. 2024-04-25T01:27:39.226 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:27:39.226 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-04-25T01:27:39.226 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['b'] 2024-04-25T01:27:39.226 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-04-25T01:27:39.226 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-25T01:27:39.227 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T01:27:39.227 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:27:39.240 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T01:27:39.240 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-04-25T01:27:39.240 INFO:tasks.ceph.mon.b:Sent signal 19 2024-04-25T01:27:39.240 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-04-25T01:27:39.383 INFO:tasks.workunit.client.0.smithi112.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-25T01:27:39.383 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T01:27:39.383 INFO:tasks.workunit.client.0.smithi112.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T01:27:39.383 INFO:tasks.workunit.client.0.smithi112.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T01:27:39.383 INFO:tasks.workunit.client.0.smithi112.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T01:27:39.383 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T01:27:39.384 INFO:tasks.workunit.client.0.smithi112.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T01:27:39.384 INFO:tasks.workunit.client.0.smithi112.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T01:27:39.384 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T01:27:39.384 INFO:tasks.workunit.client.0.smithi112.stderr: git switch -c 2024-04-25T01:27:39.384 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T01:27:39.384 INFO:tasks.workunit.client.0.smithi112.stderr:Or undo this operation with: 2024-04-25T01:27:39.384 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T01:27:39.384 INFO:tasks.workunit.client.0.smithi112.stderr: git switch - 2024-04-25T01:27:39.384 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T01:27:39.384 INFO:tasks.workunit.client.0.smithi112.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T01:27:39.384 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T01:27:39.384 INFO:tasks.workunit.client.0.smithi112.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-25T01:27:39.387 DEBUG:teuthology.orchestra.run.smithi112:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T01:27:39.439 INFO:tasks.workunit.client.0.smithi112.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T01:27:39.441 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T01:27:39.442 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T01:27:39.624 INFO:tasks.workunit.client.1.smithi187.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T01:27:39.676 INFO:tasks.workunit.client.1.smithi187.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T01:27:39.715 INFO:tasks.workunit.client.1.smithi187.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-25T01:27:39.717 INFO:tasks.workunit.client.1.smithi187.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-25T01:27:39.717 INFO:tasks.workunit.client.1.smithi187.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T01:27:39.755 INFO:tasks.workunit.client.1.smithi187.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-25T01:27:39.758 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:27:39.758 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-25T01:27:39.773 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.1... 2024-04-25T01:27:39.774 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-04-25T01:27:39.775 DEBUG:teuthology.orchestra.run.smithi187:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/ffsb.sh 2024-04-25T01:27:39.838 INFO:tasks.workunit.client.1.smithi187.stderr:++ dirname /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/ffsb.sh 2024-04-25T01:27:39.838 INFO:tasks.workunit.client.1.smithi187.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.1/qa/workunits/suites 2024-04-25T01:27:39.839 INFO:tasks.workunit.client.1.smithi187.stderr:+ trap -- retry EXIT 2024-04-25T01:27:39.839 INFO:tasks.workunit.client.1.smithi187.stderr:+ rm -rf ffsb 2024-04-25T01:27:39.839 INFO:tasks.workunit.client.1.smithi187.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-04-25T01:27:39.843 INFO:tasks.workunit.client.1.smithi187.stderr:Cloning into 'ffsb'... 2024-04-25T01:27:40.004 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T01:27:40.059 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T01:27:40.098 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T01:27:40.100 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T01:27:40.100 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T01:27:40.139 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T01:27:40.142 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:27:40.142 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T01:27:40.158 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-04-25T01:27:40.160 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-04-25T01:27:40.161 DEBUG:teuthology.orchestra.run.smithi112:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-04-25T01:27:40.224 INFO:tasks.workunit.client.0.smithi112.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-04-25T01:27:40.225 INFO:tasks.workunit.client.0.smithi112.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-04-25T01:27:40.225 INFO:tasks.workunit.client.0.smithi112.stderr:+ trap -- retry EXIT 2024-04-25T01:27:40.225 INFO:tasks.workunit.client.0.smithi112.stderr:+ rm -rf ffsb 2024-04-25T01:27:40.226 INFO:tasks.workunit.client.0.smithi112.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-04-25T01:27:40.230 INFO:tasks.workunit.client.0.smithi112.stderr:Cloning into 'ffsb'... 2024-04-25T01:27:41.481 INFO:tasks.workunit.client.1.smithi187.stderr:+ trap - EXIT 2024-04-25T01:27:41.481 INFO:tasks.workunit.client.1.smithi187.stderr:+ cd ffsb 2024-04-25T01:27:41.481 INFO:tasks.workunit.client.1.smithi187.stderr:+ ./configure 2024-04-25T01:27:41.669 INFO:tasks.workunit.client.1.smithi187.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-25T01:27:41.957 INFO:tasks.workunit.client.0.smithi112.stderr:+ trap - EXIT 2024-04-25T01:27:41.958 INFO:tasks.workunit.client.0.smithi112.stderr:+ cd ffsb 2024-04-25T01:27:41.958 INFO:tasks.workunit.client.0.smithi112.stderr:+ ./configure 2024-04-25T01:27:42.174 INFO:tasks.workunit.client.0.smithi112.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-25T01:27:42.678 INFO:tasks.workunit.client.1.smithi187.stdout:checking whether build environment is sane... yes 2024-04-25T01:27:42.691 INFO:tasks.workunit.client.1.smithi187.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-04-25T01:27:42.692 INFO:tasks.workunit.client.1.smithi187.stdout:checking for gawk... gawk 2024-04-25T01:27:42.714 INFO:tasks.workunit.client.1.smithi187.stdout:checking whether make sets $(MAKE)... yes 2024-04-25T01:27:42.754 INFO:tasks.workunit.client.1.smithi187.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-04-25T01:27:42.754 INFO:tasks.workunit.client.1.smithi187.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-04-25T01:27:42.773 INFO:tasks.workunit.client.1.smithi187.stdout:checking for style of include used by make... GNU 2024-04-25T01:27:42.779 INFO:tasks.workunit.client.1.smithi187.stdout:checking for gcc... gcc 2024-04-25T01:27:42.857 INFO:tasks.workunit.client.1.smithi187.stdout:checking for C compiler default output file name... a.out 2024-04-25T01:27:42.860 INFO:tasks.workunit.client.1.smithi187.stdout:checking whether the C compiler works... yes 2024-04-25T01:27:42.863 INFO:tasks.workunit.client.1.smithi187.stdout:checking whether we are cross compiling... no 2024-04-25T01:27:42.910 INFO:tasks.workunit.client.1.smithi187.stdout:checking for suffix of executables... 2024-04-25T01:27:42.949 INFO:tasks.workunit.client.1.smithi187.stdout:checking for suffix of object files... o 2024-04-25T01:27:42.991 INFO:tasks.workunit.client.1.smithi187.stdout:checking whether we are using the GNU C compiler... yes 2024-04-25T01:27:43.035 INFO:tasks.workunit.client.1.smithi187.stdout:checking whether gcc accepts -g... yes 2024-04-25T01:27:43.116 INFO:tasks.workunit.client.1.smithi187.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-25T01:27:43.181 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether build environment is sane... yes 2024-04-25T01:27:43.197 INFO:tasks.workunit.client.0.smithi112.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-04-25T01:27:43.198 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gawk... gawk 2024-04-25T01:27:43.220 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether make sets $(MAKE)... yes 2024-04-25T01:27:43.258 INFO:tasks.workunit.client.1.smithi187.stdout:checking dependency style of gcc... gcc3 2024-04-25T01:27:43.264 INFO:tasks.workunit.client.0.smithi112.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-04-25T01:27:43.265 INFO:tasks.workunit.client.0.smithi112.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-04-25T01:27:43.282 INFO:tasks.workunit.client.0.smithi112.stdout:checking for style of include used by make... GNU 2024-04-25T01:27:43.287 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc... gcc 2024-04-25T01:27:43.332 INFO:tasks.workunit.client.1.smithi187.stdout:checking how to run the C preprocessor... gcc -E 2024-04-25T01:27:43.386 INFO:tasks.workunit.client.1.smithi187.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-25T01:27:43.387 INFO:tasks.workunit.client.1.smithi187.stdout:checking for egrep... /usr/bin/grep -E 2024-04-25T01:27:43.401 INFO:tasks.workunit.client.1.smithi187.stdout:checking for AIX... no 2024-04-25T01:27:43.428 INFO:tasks.workunit.client.1.smithi187.stdout:checking for gcc... (cached) gcc 2024-04-25T01:27:43.441 INFO:tasks.workunit.client.1.smithi187.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-25T01:27:43.442 INFO:tasks.workunit.client.1.smithi187.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-25T01:27:43.442 INFO:tasks.workunit.client.1.smithi187.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-25T01:27:43.443 INFO:tasks.workunit.client.1.smithi187.stdout:checking dependency style of gcc... (cached) gcc3 2024-04-25T01:27:43.476 INFO:tasks.workunit.client.0.smithi112.stdout:checking for C compiler default output file name... a.out 2024-04-25T01:27:43.478 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether the C compiler works... yes 2024-04-25T01:27:43.481 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether we are cross compiling... no 2024-04-25T01:27:43.512 INFO:tasks.workunit.client.1.smithi187.stdout:checking for main in -lm... yes 2024-04-25T01:27:43.584 INFO:tasks.workunit.client.1.smithi187.stdout:checking for main in -lpthread... yes 2024-04-25T01:27:43.619 INFO:tasks.workunit.client.0.smithi112.stdout:checking for suffix of executables... 2024-04-25T01:27:43.754 INFO:tasks.workunit.client.0.smithi112.stdout:checking for suffix of object files... o 2024-04-25T01:27:43.775 INFO:tasks.workunit.client.1.smithi187.stdout:checking for ANSI C header files... yes 2024-04-25T01:27:43.834 INFO:tasks.workunit.client.1.smithi187.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-04-25T01:27:43.861 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether we are using the GNU C compiler... yes 2024-04-25T01:27:43.892 INFO:tasks.workunit.client.1.smithi187.stdout:checking for sys/types.h... yes 2024-04-25T01:27:43.950 INFO:tasks.workunit.client.1.smithi187.stdout:checking for sys/stat.h... yes 2024-04-25T01:27:43.996 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether gcc accepts -g... yes 2024-04-25T01:27:44.008 INFO:tasks.workunit.client.1.smithi187.stdout:checking for stdlib.h... yes 2024-04-25T01:27:44.070 INFO:tasks.workunit.client.1.smithi187.stdout:checking for string.h... yes 2024-04-25T01:27:44.132 INFO:tasks.workunit.client.1.smithi187.stdout:checking for memory.h... yes 2024-04-25T01:27:44.190 INFO:tasks.workunit.client.1.smithi187.stdout:checking for strings.h... yes 2024-04-25T01:27:44.249 INFO:tasks.workunit.client.1.smithi187.stdout:checking for inttypes.h... yes 2024-04-25T01:27:44.309 INFO:tasks.workunit.client.1.smithi187.stdout:checking for stdint.h... yes 2024-04-25T01:27:44.371 INFO:tasks.workunit.client.1.smithi187.stdout:checking for unistd.h... yes 2024-04-25T01:27:44.382 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-25T01:27:44.445 INFO:tasks.workunit.client.1.smithi187.stdout:checking pthread.h usability... yes 2024-04-25T01:27:44.472 INFO:tasks.workunit.client.1.smithi187.stdout:checking pthread.h presence... yes 2024-04-25T01:27:44.473 INFO:tasks.workunit.client.1.smithi187.stdout:checking for pthread.h... yes 2024-04-25T01:27:44.539 INFO:tasks.workunit.client.1.smithi187.stdout:checking fcntl.h usability... yes 2024-04-25T01:27:44.564 INFO:tasks.workunit.client.1.smithi187.stdout:checking fcntl.h presence... yes 2024-04-25T01:27:44.564 INFO:tasks.workunit.client.1.smithi187.stdout:checking for fcntl.h... yes 2024-04-25T01:27:44.632 INFO:tasks.workunit.client.1.smithi187.stdout:checking limits.h usability... yes 2024-04-25T01:27:44.658 INFO:tasks.workunit.client.1.smithi187.stdout:checking limits.h presence... yes 2024-04-25T01:27:44.659 INFO:tasks.workunit.client.1.smithi187.stdout:checking for limits.h... yes 2024-04-25T01:27:44.679 INFO:tasks.workunit.client.1.smithi187.stdout:checking for stdint.h... (cached) yes 2024-04-25T01:27:44.745 INFO:tasks.workunit.client.1.smithi187.stdout:checking sys/time.h usability... yes 2024-04-25T01:27:44.771 INFO:tasks.workunit.client.1.smithi187.stdout:checking sys/time.h presence... yes 2024-04-25T01:27:44.772 INFO:tasks.workunit.client.1.smithi187.stdout:checking for sys/time.h... yes 2024-04-25T01:27:44.792 INFO:tasks.workunit.client.1.smithi187.stdout:checking for unistd.h... (cached) yes 2024-04-25T01:27:44.847 INFO:tasks.workunit.client.1.smithi187.stdout:checking sys/vfs.h usability... yes 2024-04-25T01:27:44.872 INFO:tasks.workunit.client.1.smithi187.stdout:checking sys/vfs.h presence... yes 2024-04-25T01:27:44.873 INFO:tasks.workunit.client.1.smithi187.stdout:checking for sys/vfs.h... yes 2024-04-25T01:27:44.934 INFO:tasks.workunit.client.1.smithi187.stdout:checking sys/limits.h usability... no 2024-04-25T01:27:44.962 INFO:tasks.workunit.client.1.smithi187.stdout:checking sys/limits.h presence... no 2024-04-25T01:27:44.962 INFO:tasks.workunit.client.1.smithi187.stdout:checking for sys/limits.h... no 2024-04-25T01:27:45.004 INFO:tasks.workunit.client.1.smithi187.stdout:checking for an ANSI C-conforming const... yes 2024-04-25T01:27:45.048 INFO:tasks.workunit.client.1.smithi187.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-04-25T01:27:45.050 INFO:tasks.workunit.client.1.smithi187.stdout:checking for function prototypes... yes 2024-04-25T01:27:45.108 INFO:tasks.workunit.client.1.smithi187.stdout:checking whether setvbuf arguments are reversed... no 2024-04-25T01:27:45.189 INFO:tasks.workunit.client.1.smithi187.stdout:checking for system... yes 2024-04-25T01:27:45.269 INFO:tasks.workunit.client.1.smithi187.stdout:checking for gettimeofday... yes 2024-04-25T01:27:45.351 INFO:tasks.workunit.client.1.smithi187.stdout:checking for mkdir... yes 2024-04-25T01:27:45.436 INFO:tasks.workunit.client.1.smithi187.stdout:checking for strerror... yes 2024-04-25T01:27:45.527 INFO:tasks.workunit.client.1.smithi187.stdout:checking for open64... yes 2024-04-25T01:27:45.608 INFO:tasks.workunit.client.1.smithi187.stdout:checking for stat64... yes 2024-04-25T01:27:45.689 INFO:tasks.workunit.client.1.smithi187.stdout:checking for fseeko64... yes 2024-04-25T01:27:45.770 INFO:tasks.workunit.client.1.smithi187.stdout:checking for lrand48_r... yes 2024-04-25T01:27:45.850 INFO:tasks.workunit.client.1.smithi187.stdout:checking for srand48_r... yes 2024-04-25T01:27:45.876 INFO:tasks.workunit.client.1.smithi187.stdout:configure: creating ./config.status 2024-04-25T01:27:45.996 INFO:tasks.workunit.client.1.smithi187.stdout:config.status: creating Makefile 2024-04-25T01:27:46.012 INFO:tasks.workunit.client.1.smithi187.stdout:config.status: creating config.h 2024-04-25T01:27:46.036 INFO:tasks.workunit.client.1.smithi187.stdout:config.status: executing depfiles commands 2024-04-25T01:27:46.103 INFO:tasks.workunit.client.1.smithi187.stdout:config.status: executing default commands 2024-04-25T01:27:53.851 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:27:53.855 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:27:54.240 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-04-25T01:27:54.241 INFO:tasks.ceph.mon.b:Sent signal 18 2024-04-25T01:27:54.241 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T01:27:54.241 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:27:54.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.368+0000 7f3a6febd640 1 Processor -- start 2024-04-25T01:27:54.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.369+0000 7f3a6febd640 1 -- start start 2024-04-25T01:27:54.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.369+0000 7f3a6febd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 0x7f3a680c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:54.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.369+0000 7f3a6febd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a68156cf0 0x7f3a6806bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:54.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.369+0000 7f3a6febd640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a680c90e0 con 0x7f3a680c5d70 2024-04-25T01:27:54.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.369+0000 7f3a6febd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f3a680ca9b0 con 0x7f3a68152fd0 2024-04-25T01:27:54.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.369+0000 7f3a6febd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3a680cac00 con 0x7f3a68156cf0 2024-04-25T01:27:54.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.369+0000 7f3a6dc59640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 0x7f3a680c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:54.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.369+0000 7f3a6dc59640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 0x7f3a680c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37216/0 (socket says 172.21.15.112:37216) 2024-04-25T01:27:54.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.369+0000 7f3a6dc59640 1 -- 172.21.15.112:0/2664978267 learned_addr learned my addr 172.21.15.112:0/2664978267 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:27:54.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.369+0000 7f3a6e45a640 1 -- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a68156cf0 msgr2=0x7f3a6806bda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:27:54.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.370+0000 7f3a6e45a640 1 --2- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a68156cf0 0x7f3a6806bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:27:54.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.370+0000 7f3a6dc59640 1 -- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 msgr2=0x7f3a680c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:27:54.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.370+0000 7f3a6dc59640 1 -- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 msgr2=0x7f3a680c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:27:54.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.370+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 0x7f3a680c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:27:54.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.370+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 0x7f3a680c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:27:54.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.571+0000 7f3a6e45a640 1 -- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a68156cf0 msgr2=0x7f3a6806bda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:27:54.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.571+0000 7f3a6e45a640 1 --2- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a68156cf0 0x7f3a6806bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:27:54.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.571+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 0x7f3a680c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:54.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.571+0000 7f3a6dc59640 1 -- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 msgr2=0x7f3a680c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:27:54.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.571+0000 7f3a6dc59640 1 -- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 msgr2=0x7f3a680c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:27:54.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.571+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 0x7f3a680c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:27:54.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.571+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 0x7f3a680c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:27:54.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.972+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 0x7f3a680c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:54.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.972+0000 7f3a6e45a640 1 -- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a68156cf0 msgr2=0x7f3a6806bda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:27:54.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.972+0000 7f3a6e45a640 1 --2- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a68156cf0 0x7f3a6806bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:27:54.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.972+0000 7f3a6dc59640 1 -- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 msgr2=0x7f3a680c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:27:54.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.972+0000 7f3a6dc59640 1 -- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 msgr2=0x7f3a680c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:27:54.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.972+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 0x7f3a680c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:27:54.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:54.972+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 0x7f3a680c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:27:55.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:55.773+0000 7f3a6e45a640 1 -- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a68156cf0 msgr2=0x7f3a6806bda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:27:55.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:55.773+0000 7f3a6e45a640 1 --2- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a68156cf0 0x7f3a6806bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:27:55.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:55.773+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 0x7f3a680c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:55.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:55.773+0000 7f3a6dc59640 1 -- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 msgr2=0x7f3a680c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:27:55.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:55.773+0000 7f3a6dc59640 1 -- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 msgr2=0x7f3a680c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:27:55.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:55.773+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 0x7f3a680c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:27:55.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:55.773+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 0x7f3a680c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:27:57.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.370+0000 7f3a5f7fe640 1 -- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a68156cf0 msgr2=0x7f3a6806bda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:27:57.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.370+0000 7f3a5f7fe640 1 --2- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a68156cf0 0x7f3a6806bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.370+0000 7f3a5f7fe640 1 -- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 msgr2=0x7f3a680c9b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:27:57.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.370+0000 7f3a5f7fe640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.112:3300/0 conn(0x7f3a68152fd0 0x7f3a680c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.370+0000 7f3a5f7fe640 1 -- 172.21.15.112:0/2664978267 >> v1:172.21.15.112:6789/0 conn(0x7f3a680c5d70 legacy=0x7f3a68156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:57.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.370+0000 7f3a5f7fe640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 0x7f3a4c005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:57.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.370+0000 7f3a5f7fe640 1 -- 172.21.15.112:0/2664978267 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a680c9380 con 0x7f3a4c006190 2024-04-25T01:27:57.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.370+0000 7f3a5f7fe640 1 -- 172.21.15.112:0/2664978267 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3a680ca9b0 con 0x7f3a4c0036c0 2024-04-25T01:27:57.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.370+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 0x7f3a4c005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:57.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.370+0000 7f3a6dc59640 1 -- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 msgr2=0x7f3a4c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:27:57.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.370+0000 7f3a6dc59640 1 -- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 msgr2=0x7f3a4c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:27:57.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.370+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 0x7f3a4c005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:27:57.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.370+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 0x7f3a4c005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:27:57.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.571+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 0x7f3a4c005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:57.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.571+0000 7f3a6dc59640 1 -- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 msgr2=0x7f3a4c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:27:57.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.571+0000 7f3a6dc59640 1 -- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 msgr2=0x7f3a4c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:27:57.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.571+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 0x7f3a4c005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:27:57.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.571+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 0x7f3a4c005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:27:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.972+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 0x7f3a4c005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.972+0000 7f3a6dc59640 1 -- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 msgr2=0x7f3a4c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:27:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.972+0000 7f3a6dc59640 1 -- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 msgr2=0x7f3a4c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:27:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.972+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 0x7f3a4c005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:27:57.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:57.972+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 0x7f3a4c005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:27:58.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:58.774+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 0x7f3a4c005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:58.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:58.774+0000 7f3a6dc59640 1 -- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 msgr2=0x7f3a4c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:27:58.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:58.774+0000 7f3a6dc59640 1 -- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 msgr2=0x7f3a4c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:27:58.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:58.774+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 0x7f3a4c005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:27:58.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:58.774+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 0x7f3a4c005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:27:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.271+0000 7f3a6cc57640 1 -- 172.21.15.112:0/2664978267 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 826989208 0 0) 0x7f3a680c9380 con 0x7f3a4c006190 2024-04-25T01:27:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.271+0000 7f3a6cc57640 1 -- 172.21.15.112:0/2664978267 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a54003520 con 0x7f3a4c006190 2024-04-25T01:27:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.271+0000 7f3a6cc57640 1 -- 172.21.15.112:0/2664978267 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 365033786 0 0) 0x7f3a54003520 con 0x7f3a4c006190 2024-04-25T01:27:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.271+0000 7f3a6cc57640 1 -- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 msgr2=0x7f3a4c005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:27:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.271+0000 7f3a6cc57640 1 --2- 172.21.15.112:0/2664978267 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c0036c0 0x7f3a4c005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.271+0000 7f3a6cc57640 1 -- 172.21.15.112:0/2664978267 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4c00af40 con 0x7f3a4c006190 2024-04-25T01:27:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.271+0000 7f3a6cc57640 1 -- 172.21.15.112:0/2664978267 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f3a580016b0 con 0x7f3a4c006190 2024-04-25T01:27:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.271+0000 7f3a6cc57640 1 -- 172.21.15.112:0/2664978267 >> v1:172.21.15.187:6789/0 conn(0x7f3a4c006190 legacy=0x7f3a4c008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.271+0000 7f3a6e45a640 1 -- 172.21.15.112:0/2664978267 reap_dead start 2024-04-25T01:27:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.271+0000 7f3a6cc57640 1 --2- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a540039b0 0x7f3a54003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.271+0000 7f3a6cc57640 1 --2- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a54004800 0x7f3a54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.271+0000 7f3a6cc57640 1 --2- 172.21.15.112:0/2664978267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a54005210 0x7f3a5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.271+0000 7f3a6cc57640 1 -- 172.21.15.112:0/2664978267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3a680ca9b0 con 0x7f3a54005210 2024-04-25T01:27:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.271+0000 7f3a6cc57640 1 -- 172.21.15.112:0/2664978267 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5400c140 con 0x7f3a540039b0 2024-04-25T01:27:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.271+0000 7f3a6cc57640 1 -- 172.21.15.112:0/2664978267 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3a5400c340 con 0x7f3a54004800 2024-04-25T01:27:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.272+0000 7f3a6d458640 1 -- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a54004800 msgr2=0x7f3a54004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:27:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.272+0000 7f3a6d458640 1 --2- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a54004800 0x7f3a54004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:27:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.272+0000 7f3a6e45a640 1 --2- 172.21.15.112:0/2664978267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a54005210 0x7f3a5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.272+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a540039b0 0x7f3a54003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.272+0000 7f3a6e45a640 1 -- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a54004800 msgr2=0x7f3a54004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:27:59.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.272+0000 7f3a6e45a640 1 --2- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a54004800 0x7f3a54004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.272+0000 7f3a6e45a640 1 -- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a540039b0 msgr2=0x7f3a54003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:59.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.272+0000 7f3a6e45a640 1 --2- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a540039b0 0x7f3a54003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.272+0000 7f3a6e45a640 1 -- 172.21.15.112:0/2664978267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a54003520 con 0x7f3a54005210 2024-04-25T01:27:59.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.273+0000 7f3a6e45a640 1 --2- 172.21.15.112:0/2664978267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a54005210 0x7f3a5400b9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3a68158d50 tx=0x7f3a54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202b40a6ee9687ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:59.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.278+0000 7f3a6cc57640 1 -- 172.21.15.112:0/2664978267 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64007a50 con 0x7f3a54005210 2024-04-25T01:27:59.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.278+0000 7f3a6cc57640 1 -- 172.21.15.112:0/2664978267 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a64035070 con 0x7f3a54005210 2024-04-25T01:27:59.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.279+0000 7f3a6cc57640 1 -- 172.21.15.112:0/2664978267 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6402c040 con 0x7f3a54005210 2024-04-25T01:27:59.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.279+0000 7f3a6febd640 1 -- 172.21.15.112:0/2664978267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a54005210 msgr2=0x7f3a5400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.279+0000 7f3a6febd640 1 --2- 172.21.15.112:0/2664978267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a54005210 0x7f3a5400b9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3a68158d50 tx=0x7f3a54005830 comp rx=0 tx=0).stop 2024-04-25T01:27:59.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.279+0000 7f3a6febd640 1 -- 172.21.15.112:0/2664978267 shutdown_connections 2024-04-25T01:27:59.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.279+0000 7f3a6febd640 1 --2- 172.21.15.112:0/2664978267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a54005210 0x7f3a5400b9c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.279+0000 7f3a6febd640 1 --2- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a54004800 0x7f3a54004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.279+0000 7f3a6febd640 1 --2- 172.21.15.112:0/2664978267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a540039b0 0x7f3a54003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.279+0000 7f3a6febd640 1 -- 172.21.15.112:0/2664978267 >> 172.21.15.112:0/2664978267 conn(0x7f3a680acc70 msgr2=0x7f3a680ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:59.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.280+0000 7f3a6febd640 1 -- 172.21.15.112:0/2664978267 shutdown_connections 2024-04-25T01:27:59.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.280+0000 7f3a6febd640 1 -- 172.21.15.112:0/2664978267 wait complete. 2024-04-25T01:27:59.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.282+0000 7f3a6febd640 1 Processor -- start 2024-04-25T01:27:59.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.282+0000 7f3a6febd640 1 -- start start 2024-04-25T01:27:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.283+0000 7f3a6febd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a540039b0 0x7f3a681eb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.283+0000 7f3a6febd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a54004800 0x7f3a68154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.283+0000 7f3a6febd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a54005210 0x7f3a681ee910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.283+0000 7f3a6febd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3a68157140 con 0x7f3a54005210 2024-04-25T01:27:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.283+0000 7f3a6febd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3a680caee0 con 0x7f3a540039b0 2024-04-25T01:27:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.283+0000 7f3a6febd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3a681572c0 con 0x7f3a54004800 2024-04-25T01:27:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.283+0000 7f3a6d458640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a54004800 msgr2=0x7f3a68154ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:27:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.283+0000 7f3a6d458640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a54004800 0x7f3a68154ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:27:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.283+0000 7f3a6e45a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a54005210 0x7f3a681ee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.283+0000 7f3a6dc59640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a540039b0 0x7f3a681eb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.283+0000 7f3a6e45a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a54005210 0x7f3a681ee910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42390/0 (socket says 172.21.15.112:42390) 2024-04-25T01:27:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.283+0000 7f3a6dc59640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a540039b0 0x7f3a681eb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:58516/0 (socket says 172.21.15.112:58516) 2024-04-25T01:27:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.283+0000 7f3a6e45a640 1 -- 172.21.15.112:0/1171805288 learned_addr learned my addr 172.21.15.112:0/1171805288 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:27:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.283+0000 7f3a6dc59640 1 -- 172.21.15.112:0/1171805288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a54004800 msgr2=0x7f3a68154ed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:27:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.284+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/1171805288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a54004800 0x7f3a68154ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.284+0000 7f3a6dc59640 1 -- 172.21.15.112:0/1171805288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a54005210 msgr2=0x7f3a681ee910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.284+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/1171805288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a54005210 0x7f3a681ee910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.284+0000 7f3a6dc59640 1 -- 172.21.15.112:0/1171805288 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a68159140 con 0x7f3a540039b0 2024-04-25T01:27:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.284+0000 7f3a6e45a640 1 --2- 172.21.15.112:0/1171805288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a54005210 0x7f3a681ee910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.284+0000 7f3a6dc59640 1 --2- 172.21.15.112:0/1171805288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a540039b0 0x7f3a681eb740 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3a600169c0 tx=0x7f3a60016e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.284+0000 7f3a5e7fc640 1 -- 172.21.15.112:0/1171805288 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60027020 con 0x7f3a540039b0 2024-04-25T01:27:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.284+0000 7f3a5e7fc640 1 -- 172.21.15.112:0/1171805288 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a60015d70 con 0x7f3a540039b0 2024-04-25T01:27:59.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.284+0000 7f3a5e7fc640 1 -- 172.21.15.112:0/1171805288 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6000f040 con 0x7f3a540039b0 2024-04-25T01:27:59.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.284+0000 7f3a6febd640 1 -- 172.21.15.112:0/1171805288 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a68159420 con 0x7f3a540039b0 2024-04-25T01:27:59.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.285+0000 7f3a6febd640 1 -- 172.21.15.112:0/1171805288 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a68155410 con 0x7f3a540039b0 2024-04-25T01:27:59.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.286+0000 7f3a6febd640 1 -- 172.21.15.112:0/1171805288 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a680682a0 con 0x7f3a540039b0 2024-04-25T01:27:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.286+0000 7f3a5e7fc640 1 -- 172.21.15.112:0/1171805288 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a6002a020 con 0x7f3a540039b0 2024-04-25T01:27:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.287+0000 7f3a5e7fc640 1 --2- 172.21.15.112:0/1171805288 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a4c06c700 0x7f3a4c06ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.287+0000 7f3a5e7fc640 1 -- 172.21.15.112:0/1171805288 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f3a6008cba0 con 0x7f3a540039b0 2024-04-25T01:27:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.287+0000 7f3a6d458640 1 --2- 172.21.15.112:0/1171805288 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a4c06c700 0x7f3a4c06ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.287+0000 7f3a6d458640 1 --2- 172.21.15.112:0/1171805288 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a4c06c700 0x7f3a4c06ebc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3a68152330 tx=0x7f3a58001640 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.292+0000 7f3a5e7fc640 1 -- 172.21.15.112:0/1171805288 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3a60031030 con 0x7f3a540039b0 2024-04-25T01:27:59.293 INFO:tasks.workunit.client.0.smithi112.stdout:checking dependency style of gcc... gcc3 2024-04-25T01:27:59.315 INFO:tasks.workunit.client.1.smithi187.stderr:+ make 2024-04-25T01:27:59.355 INFO:tasks.workunit.client.1.smithi187.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb/missing --run aclocal-1.10 2024-04-25T01:27:59.361 INFO:tasks.workunit.client.1.smithi187.stderr:/home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-04-25T01:27:59.362 INFO:tasks.workunit.client.1.smithi187.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-04-25T01:27:59.362 INFO:tasks.workunit.client.1.smithi187.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-04-25T01:27:59.362 INFO:tasks.workunit.client.1.smithi187.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-04-25T01:27:59.363 INFO:tasks.workunit.client.1.smithi187.stderr: any GNU archive site. 2024-04-25T01:27:59.367 INFO:tasks.workunit.client.1.smithi187.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb/missing --run automake-1.10 --gnu 2024-04-25T01:27:59.373 INFO:tasks.workunit.client.1.smithi187.stderr:/home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-04-25T01:27:59.373 INFO:tasks.workunit.client.1.smithi187.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-04-25T01:27:59.373 INFO:tasks.workunit.client.1.smithi187.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-04-25T01:27:59.373 INFO:tasks.workunit.client.1.smithi187.stderr: You might want to install the `Automake' and `Perl' packages. 2024-04-25T01:27:59.373 INFO:tasks.workunit.client.1.smithi187.stderr: Grab them from any GNU archive site. 2024-04-25T01:27:59.379 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to run the C preprocessor... gcc -E 2024-04-25T01:27:59.441 INFO:tasks.workunit.client.0.smithi112.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-25T01:27:59.443 INFO:tasks.workunit.client.0.smithi112.stdout:checking for egrep... /usr/bin/grep -E 2024-04-25T01:27:59.458 INFO:tasks.workunit.client.0.smithi112.stdout:checking for AIX... no 2024-04-25T01:27:59.485 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc... (cached) gcc 2024-04-25T01:27:59.498 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-25T01:27:59.499 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-25T01:27:59.499 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-25T01:27:59.500 INFO:tasks.workunit.client.0.smithi112.stdout:checking dependency style of gcc... (cached) gcc3 2024-04-25T01:27:59.553 INFO:tasks.workunit.client.1.smithi187.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb/missing --run autoconf 2024-04-25T01:27:59.573 INFO:tasks.workunit.client.0.smithi112.stdout:checking for main in -lm... yes 2024-04-25T01:27:59.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.618+0000 7f3a6febd640 1 -- 172.21.15.112:0/1171805288 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3a680c5ac0 con 0x7f3a540039b0 2024-04-25T01:27:59.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.619+0000 7f3a5e7fc640 1 -- 172.21.15.112:0/1171805288 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7f3a60020040 con 0x7f3a540039b0 2024-04-25T01:27:59.598 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":12,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:27:59.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.621+0000 7f3a6febd640 1 -- 172.21.15.112:0/1171805288 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a4c06c700 msgr2=0x7f3a4c06ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.621+0000 7f3a6febd640 1 --2- 172.21.15.112:0/1171805288 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a4c06c700 0x7f3a4c06ebc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3a68152330 tx=0x7f3a58001640 comp rx=0 tx=0).stop 2024-04-25T01:27:59.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.621+0000 7f3a6febd640 1 -- 172.21.15.112:0/1171805288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a540039b0 msgr2=0x7f3a681eb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.621+0000 7f3a6febd640 1 --2- 172.21.15.112:0/1171805288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a540039b0 0x7f3a681eb740 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3a600169c0 tx=0x7f3a60016e80 comp rx=0 tx=0).stop 2024-04-25T01:27:59.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.621+0000 7f3a6febd640 1 -- 172.21.15.112:0/1171805288 shutdown_connections 2024-04-25T01:27:59.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.621+0000 7f3a6febd640 1 --2- 172.21.15.112:0/1171805288 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a4c06c700 0x7f3a4c06ebc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.621+0000 7f3a6febd640 1 --2- 172.21.15.112:0/1171805288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a54005210 0x7f3a681ee910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.621+0000 7f3a6febd640 1 --2- 172.21.15.112:0/1171805288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a54004800 0x7f3a68154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.621+0000 7f3a6febd640 1 --2- 172.21.15.112:0/1171805288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a540039b0 0x7f3a681eb740 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.621+0000 7f3a6febd640 1 -- 172.21.15.112:0/1171805288 >> 172.21.15.112:0/1171805288 conn(0x7f3a680acc70 msgr2=0x7f3a680c9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:59.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.621+0000 7f3a6febd640 1 -- 172.21.15.112:0/1171805288 shutdown_connections 2024-04-25T01:27:59.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.621+0000 7f3a6febd640 1 -- 172.21.15.112:0/1171805288 wait complete. 2024-04-25T01:27:59.611 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T01:27:59.611 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-04-25T01:27:59.611 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:27:59.651 INFO:tasks.workunit.client.0.smithi112.stdout:checking for main in -lpthread... yes 2024-04-25T01:27:59.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.741+0000 7ff2dade2640 1 Processor -- start 2024-04-25T01:27:59.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.741+0000 7ff2dade2640 1 -- start start 2024-04-25T01:27:59.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.741+0000 7ff2dade2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff2d41592c0 0x7ff2d40c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.741+0000 7ff2dade2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff2d41596a0 0x7ff2d4152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.741+0000 7ff2dade2640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2d4159b90 con 0x7ff2d40c72b0 2024-04-25T01:27:59.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.741+0000 7ff2dade2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7ff2d40c6e20 con 0x7ff2d41592c0 2024-04-25T01:27:59.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.741+0000 7ff2dade2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff2d406c970 con 0x7ff2d41596a0 2024-04-25T01:27:59.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.742+0000 7ff2d8b7e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff2d41592c0 0x7ff2d40c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.742+0000 7ff2d8b7e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff2d41592c0 0x7ff2d40c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42398/0 (socket says 172.21.15.112:42398) 2024-04-25T01:27:59.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.742+0000 7ff2d8b7e640 1 -- 172.21.15.112:0/1545693328 learned_addr learned my addr 172.21.15.112:0/1545693328 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:27:59.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.742+0000 7ff2d937f640 1 --1- 172.21.15.112:0/1545693328 >> v1:172.21.15.112:6789/0 conn(0x7ff2d40c72b0 0x7ff2d4155ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60922/0 (socket says 172.21.15.112:60922) 2024-04-25T01:27:59.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.742+0000 7ff2d3fff640 1 --2- 172.21.15.112:0/1545693328 >> v2:172.21.15.187:3300/0 conn(0x7ff2d41596a0 0x7ff2d4152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.742+0000 7ff2d8b7e640 1 -- 172.21.15.112:0/1545693328 >> v2:172.21.15.187:3300/0 conn(0x7ff2d41596a0 msgr2=0x7ff2d4152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:59.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.742+0000 7ff2d8b7e640 1 --2- 172.21.15.112:0/1545693328 >> v2:172.21.15.187:3300/0 conn(0x7ff2d41596a0 0x7ff2d4152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.742+0000 7ff2d8b7e640 1 -- 172.21.15.112:0/1545693328 >> v1:172.21.15.112:6789/0 conn(0x7ff2d40c72b0 legacy=0x7ff2d4155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.742+0000 7ff2d8b7e640 1 -- 172.21.15.112:0/1545693328 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d406cc50 con 0x7ff2d41592c0 2024-04-25T01:27:59.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.743+0000 7ff2d8b7e640 1 --2- 172.21.15.112:0/1545693328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d41592c0 0x7ff2d40c9070 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40bea40 tx=0x7ff2c802a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd43615230ae6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:59.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.743+0000 7ff2d37fe640 1 -- 172.21.15.112:0/1545693328 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8037070 con 0x7ff2d41592c0 2024-04-25T01:27:59.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.743+0000 7ff2d37fe640 1 -- 172.21.15.112:0/1545693328 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2c8032440 con 0x7ff2d41592c0 2024-04-25T01:27:59.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.743+0000 7ff2d37fe640 1 -- 172.21.15.112:0/1545693328 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8032810 con 0x7ff2d41592c0 2024-04-25T01:27:59.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.743+0000 7ff2dade2640 1 -- 172.21.15.112:0/1545693328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d41592c0 msgr2=0x7ff2d40c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.743+0000 7ff2dade2640 1 --2- 172.21.15.112:0/1545693328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d41592c0 0x7ff2d40c9070 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40bea40 tx=0x7ff2c802a150 comp rx=0 tx=0).stop 2024-04-25T01:27:59.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.743+0000 7ff2dade2640 1 -- 172.21.15.112:0/1545693328 shutdown_connections 2024-04-25T01:27:59.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.743+0000 7ff2dade2640 1 --2- 172.21.15.112:0/1545693328 >> v2:172.21.15.187:3300/0 conn(0x7ff2d41596a0 0x7ff2d4152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.743+0000 7ff2dade2640 1 --2- 172.21.15.112:0/1545693328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d41592c0 0x7ff2d40c9070 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.743+0000 7ff2dade2640 1 -- 172.21.15.112:0/1545693328 >> 172.21.15.112:0/1545693328 conn(0x7ff2d40cb170 msgr2=0x7ff2d40cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:59.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.744+0000 7ff2dade2640 1 -- 172.21.15.112:0/1545693328 shutdown_connections 2024-04-25T01:27:59.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.744+0000 7ff2dade2640 1 -- 172.21.15.112:0/1545693328 wait complete. 2024-04-25T01:27:59.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.745+0000 7ff2dade2640 1 Processor -- start 2024-04-25T01:27:59.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.745+0000 7ff2dade2640 1 -- start start 2024-04-25T01:27:59.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.746+0000 7ff2dade2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d40c72b0 0x7ff2d41e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.746+0000 7ff2dade2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2d41592c0 0x7ff2d41ebed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.746+0000 7ff2dade2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d41596a0 0x7ff2d41ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.746+0000 7ff2dade2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d406dca0 con 0x7ff2d40c72b0 2024-04-25T01:27:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.746+0000 7ff2dade2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d406db20 con 0x7ff2d41592c0 2024-04-25T01:27:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.746+0000 7ff2dade2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff2d40c95b0 con 0x7ff2d41596a0 2024-04-25T01:27:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.746+0000 7ff2d937f640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d41596a0 msgr2=0x7ff2d41ef2c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:27:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.746+0000 7ff2d8b7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d40c72b0 0x7ff2d41e2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.746+0000 7ff2d937f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d41596a0 0x7ff2d41ef2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:27:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.746+0000 7ff2d8b7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d40c72b0 0x7ff2d41e2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42400/0 (socket says 172.21.15.112:42400) 2024-04-25T01:27:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.746+0000 7ff2d8b7e640 1 -- 172.21.15.112:0/3831268339 learned_addr learned my addr 172.21.15.112:0/3831268339 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:27:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.746+0000 7ff2d3fff640 1 --2- 172.21.15.112:0/3831268339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2d41592c0 0x7ff2d41ebed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.746+0000 7ff2d8b7e640 1 -- 172.21.15.112:0/3831268339 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d41596a0 msgr2=0x7ff2d41ef2c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:27:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.746+0000 7ff2d8b7e640 1 --2- 172.21.15.112:0/3831268339 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d41596a0 0x7ff2d41ef2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.746+0000 7ff2d8b7e640 1 -- 172.21.15.112:0/3831268339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2d41592c0 msgr2=0x7ff2d41ebed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.746+0000 7ff2d8b7e640 1 --2- 172.21.15.112:0/3831268339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2d41592c0 0x7ff2d41ebed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.746+0000 7ff2d8b7e640 1 -- 172.21.15.112:0/3831268339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d4149c00 con 0x7ff2d40c72b0 2024-04-25T01:27:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.746+0000 7ff2d3fff640 1 --2- 172.21.15.112:0/3831268339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2d41592c0 0x7ff2d41ebed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.746+0000 7ff2d8b7e640 1 --2- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d40c72b0 0x7ff2d41e2a10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff2c802a190 tx=0x7ff2c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.747+0000 7ff2d17fa640 1 -- 172.21.15.112:0/3831268339 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8037040 con 0x7ff2d40c72b0 2024-04-25T01:27:59.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.747+0000 7ff2d17fa640 1 -- 172.21.15.112:0/3831268339 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2c8042070 con 0x7ff2d40c72b0 2024-04-25T01:27:59.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.747+0000 7ff2d17fa640 1 -- 172.21.15.112:0/3831268339 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8030920 con 0x7ff2d40c72b0 2024-04-25T01:27:59.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.747+0000 7ff2dade2640 1 -- 172.21.15.112:0/3831268339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d4149ee0 con 0x7ff2d40c72b0 2024-04-25T01:27:59.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.747+0000 7ff2dade2640 1 -- 172.21.15.112:0/3831268339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d414a4a0 con 0x7ff2d40c72b0 2024-04-25T01:27:59.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.748+0000 7ff2dade2640 1 --2- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d41eb790 0x7ff2d41ef800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.748+0000 7ff2dade2640 1 -- 172.21.15.112:0/3831268339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d4075980 con 0x7ff2d41eb790 2024-04-25T01:27:59.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.748+0000 7ff2d3fff640 1 --2- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d41eb790 0x7ff2d41ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.748+0000 7ff2d3fff640 1 -- 172.21.15.112:0/3831268339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff2d40679a0 con 0x7ff2d41eb790 2024-04-25T01:27:59.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.748+0000 7ff2d17fa640 1 -- 172.21.15.112:0/3831268339 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff2c8030ac0 con 0x7ff2d40c72b0 2024-04-25T01:27:59.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.748+0000 7ff2d3fff640 1 --2- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d41eb790 0x7ff2d41ef800 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40be1b0 tx=0x7ff2c000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:59.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.749+0000 7ff2d17fa640 1 --2- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff2b806be10 0x7ff2b806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.749+0000 7ff2d937f640 1 --2- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff2b806be10 0x7ff2b806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.749+0000 7ff2d17fa640 1 -- 172.21.15.112:0/3831268339 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7ff2c80aba80 con 0x7ff2d40c72b0 2024-04-25T01:27:59.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.749+0000 7ff2d17fa640 1 -- 172.21.15.112:0/3831268339 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0019070 con 0x7ff2d41eb790 2024-04-25T01:27:59.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.749+0000 7ff2d17fa640 1 -- 172.21.15.112:0/3831268339 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff2c0004030 con 0x7ff2d41eb790 2024-04-25T01:27:59.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.749+0000 7ff2d17fa640 1 -- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d41eb790 msgr2=0x7ff2d41ef800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.749+0000 7ff2d17fa640 1 --2- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d41eb790 0x7ff2d41ef800 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40be1b0 tx=0x7ff2c000f610 comp rx=0 tx=0).stop 2024-04-25T01:27:59.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.749+0000 7ff2d17fa640 1 -- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d41eb790 msgr2=0x7ff2d41ef800 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:27:59.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.749+0000 7ff2d17fa640 1 --2- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d41eb790 0x7ff2d41ef800 secure :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40be1b0 tx=0x7ff2c000f610 comp rx=0 tx=0).stop 2024-04-25T01:27:59.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.749+0000 7ff2d937f640 1 --2- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff2b806be10 0x7ff2b806e2d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0014a0 tx=0x7ff2cc007d20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:59.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.756+0000 7ff2dade2640 1 --2- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d41eed00 0x7ff2d4155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.756+0000 7ff2dade2640 1 -- 172.21.15.112:0/3831268339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d406d820 con 0x7ff2d41eed00 2024-04-25T01:27:59.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.756+0000 7ff2d3fff640 1 --2- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d41eed00 0x7ff2d4155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.756+0000 7ff2d3fff640 1 -- 172.21.15.112:0/3831268339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff2d40c79d0 con 0x7ff2d41eed00 2024-04-25T01:27:59.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.756+0000 7ff2d3fff640 1 --2- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d41eed00 0x7ff2d4155ae0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff2c001b000 tx=0x7ff2c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:59.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.756+0000 7ff2d17fa640 1 -- 172.21.15.112:0/3831268339 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0019040 con 0x7ff2d41eed00 2024-04-25T01:27:59.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.757+0000 7ff2d17fa640 1 -- 172.21.15.112:0/3831268339 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5060 (secure 0 0 0) 0x7ff2c001e450 con 0x7ff2d41eed00 2024-04-25T01:27:59.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.757+0000 7ff2d17fa640 1 -- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d41eed00 msgr2=0x7ff2d4155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.757+0000 7ff2d17fa640 1 --2- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d41eed00 0x7ff2d4155ae0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff2c001b000 tx=0x7ff2c00046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.757+0000 7ff2d17fa640 1 -- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d41eed00 msgr2=0x7ff2d4155ae0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:27:59.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.757+0000 7ff2d17fa640 1 --2- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d41eed00 0x7ff2d4155ae0 secure :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff2c001b000 tx=0x7ff2c00046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.736 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:27:59.736 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:27:59.736 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:27:59.736 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:27:59.737 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 12, 2024-04-25T01:27:59.737 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 151385, 2024-04-25T01:27:59.737 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:27:59.737 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:27:59.737 INFO:teuthology.orchestra.run.smithi112.stdout: 1 2024-04-25T01:27:59.737 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:59.737 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:27:59.737 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:27:59.737 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:27:59.737 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:27:59.737 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:59.737 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:59.737 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:59.737 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:59.737 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:59.738 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:59.738 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:59.738 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:59.738 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:59.738 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:59.738 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:59.738 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:59.738 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:27:59.738 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:27:59.738 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:59.738 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:59.738 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:59.738 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:59.738 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:59.738 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:59.739 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:59.739 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:59.739 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:59.739 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:59.739 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:59.739 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:59.739 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.739 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:27:59.739 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:27:59.739 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:27:59.739 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:27:59.739 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:27:59.739 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:27:59.739 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:27:59.739 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:27:59.739 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:27:59.740 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:27:59.740 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:27:59.740 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:27:59.740 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:27:59.740 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:27:59.740 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:27:59.740 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:27:59.740 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:27:59.740 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:59.740 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:59.740 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:59.740 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:59.740 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:59.740 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:59.740 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:59.740 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:59.741 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:59.741 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:59.741 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:59.741 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:59.741 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:27:59.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.741 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:27:59.741 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.741 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:27:59.741 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:27:59.741 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:59.741 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:59.741 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.741 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:59.742 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:27:59.742 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:59.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.742 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.742 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:59.742 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:27:59.742 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:59.742 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:59.742 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:59.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.742 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:27:59.742 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:27:59.742 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:59.742 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:59.743 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:59.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.743 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.743 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:27:59.743 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:27:59.743 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:59.743 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:59.743 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.743 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:59.743 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:27:59.743 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:59.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.743 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.743 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:59.743 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:27:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:27:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:27:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:27:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:27:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:27:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:27:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:27:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:27:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:27:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:27:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:27:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:27:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:27:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:27:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 7 2024-04-25T01:27:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:27:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 8 2024-04-25T01:27:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:27:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:27:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:27:59.748 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:27:59.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.757+0000 7ff2dade2640 1 -- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff2b806be10 msgr2=0x7ff2b806e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.757+0000 7ff2dade2640 1 --2- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff2b806be10 0x7ff2b806e2d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0014a0 tx=0x7ff2cc007d20 comp rx=0 tx=0).stop 2024-04-25T01:27:59.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.757+0000 7ff2dade2640 1 -- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d40c72b0 msgr2=0x7ff2d41e2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.757+0000 7ff2dade2640 1 --2- 172.21.15.112:0/3831268339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d40c72b0 0x7ff2d41e2a10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff2c802a190 tx=0x7ff2c80046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.757+0000 7ff2d937f640 1 -- 172.21.15.112:0/3831268339 reap_dead start 2024-04-25T01:27:59.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.757+0000 7ff2d937f640 1 -- 172.21.15.112:0/3831268339 reap_dead start 2024-04-25T01:27:59.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.757+0000 7ff2dade2640 1 -- 172.21.15.112:0/3831268339 shutdown_connections 2024-04-25T01:27:59.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.757+0000 7ff2dade2640 1 -- 172.21.15.112:0/3831268339 >> 172.21.15.112:0/3831268339 conn(0x7ff2d40cb170 msgr2=0x7ff2d40aac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:59.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.757+0000 7ff2dade2640 1 -- 172.21.15.112:0/3831268339 shutdown_connections 2024-04-25T01:27:59.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.757+0000 7ff2dade2640 1 -- 172.21.15.112:0/3831268339 wait complete. 2024-04-25T01:27:59.751 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:27:59.751 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-04-25T01:27:59.751 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:27:59.847 INFO:tasks.workunit.client.0.smithi112.stdout:checking for ANSI C header files... yes 2024-04-25T01:27:59.861 INFO:tasks.workunit.client.1.smithi187.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-04-25T01:27:59.861 INFO:tasks.workunit.client.1.smithi187.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-04-25T01:27:59.861 INFO:tasks.workunit.client.1.smithi187.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-04-25T01:27:59.862 INFO:tasks.workunit.client.1.smithi187.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-04-25T01:27:59.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.920+0000 7fc85acda640 1 Processor -- start 2024-04-25T01:27:59.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.920+0000 7fc85acda640 1 -- start start 2024-04-25T01:27:59.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.920+0000 7fc85acda640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc8540ae9b0 0x7fc8540c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.920+0000 7fc85acda640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8540aef00 0x7fc85415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.920+0000 7fc85acda640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8540c53c0 con 0x7fc854151040 2024-04-25T01:27:59.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.920+0000 7fc85acda640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fc854157be0 con 0x7fc8540ae9b0 2024-04-25T01:27:59.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.920+0000 7fc85acda640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc8540c4a10 con 0x7fc8540aef00 2024-04-25T01:27:59.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.921+0000 7fc859277640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8540aef00 msgr2=0x7fc85415dfc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:27:59.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.921+0000 7fc853fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc8540ae9b0 0x7fc8540c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.921+0000 7fc859277640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8540aef00 0x7fc85415dfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:27:59.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.921+0000 7fc853fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc8540ae9b0 0x7fc8540c5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42426/0 (socket says 172.21.15.112:42426) 2024-04-25T01:27:59.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.921+0000 7fc853fff640 1 -- 172.21.15.112:0/1047867349 learned_addr learned my addr 172.21.15.112:0/1047867349 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:27:59.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.921+0000 7fc853fff640 1 -- 172.21.15.112:0/1047867349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8540aef00 msgr2=0x7fc85415dfc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:27:59.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.921+0000 7fc853fff640 1 --2- 172.21.15.112:0/1047867349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8540aef00 0x7fc85415dfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.921+0000 7fc853fff640 1 -- 172.21.15.112:0/1047867349 >> v1:172.21.15.187:6789/0 conn(0x7fc854151040 legacy=0x7fc8540710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.921+0000 7fc853fff640 1 -- 172.21.15.112:0/1047867349 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8540c4c60 con 0x7fc8540ae9b0 2024-04-25T01:27:59.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.922+0000 7fc853fff640 1 --2- 172.21.15.112:0/1047867349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8540ae9b0 0x7fc8540c5b60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc840009870 tx=0x7fc84002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b40d0f78bad78d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:59.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.922+0000 7fc8537fe640 1 -- 172.21.15.112:0/1047867349 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840037070 con 0x7fc8540ae9b0 2024-04-25T01:27:59.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.922+0000 7fc8537fe640 1 -- 172.21.15.112:0/1047867349 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc840032440 con 0x7fc8540ae9b0 2024-04-25T01:27:59.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.922+0000 7fc8537fe640 1 -- 172.21.15.112:0/1047867349 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840032810 con 0x7fc8540ae9b0 2024-04-25T01:27:59.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.922+0000 7fc85acda640 1 -- 172.21.15.112:0/1047867349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8540ae9b0 msgr2=0x7fc8540c5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.922+0000 7fc85acda640 1 --2- 172.21.15.112:0/1047867349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8540ae9b0 0x7fc8540c5b60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc840009870 tx=0x7fc84002a270 comp rx=0 tx=0).stop 2024-04-25T01:27:59.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.923+0000 7fc85acda640 1 -- 172.21.15.112:0/1047867349 shutdown_connections 2024-04-25T01:27:59.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.923+0000 7fc85acda640 1 --2- 172.21.15.112:0/1047867349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8540aef00 0x7fc85415dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.923+0000 7fc85acda640 1 --2- 172.21.15.112:0/1047867349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8540ae9b0 0x7fc8540c5b60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.923+0000 7fc85acda640 1 -- 172.21.15.112:0/1047867349 >> 172.21.15.112:0/1047867349 conn(0x7fc8540c91d0 msgr2=0x7fc8540c95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:59.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.923+0000 7fc85acda640 1 -- 172.21.15.112:0/1047867349 shutdown_connections 2024-04-25T01:27:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.923+0000 7fc85acda640 1 -- 172.21.15.112:0/1047867349 wait complete. 2024-04-25T01:27:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.924+0000 7fc85acda640 1 Processor -- start 2024-04-25T01:27:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.924+0000 7fc85acda640 1 -- start start 2024-04-25T01:27:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc85acda640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8540ae9b0 0x7fc85414eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc85acda640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8540aef00 0x7fc854153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc85acda640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc854151040 0x7fc854149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc85acda640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8540c46e0 con 0x7fc8540aef00 2024-04-25T01:27:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc85acda640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc8540c5870 con 0x7fc854151040 2024-04-25T01:27:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc85acda640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc8540717d0 con 0x7fc8540ae9b0 2024-04-25T01:27:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc858a76640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8540ae9b0 msgr2=0x7fc85414eb00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:27:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc858a76640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8540ae9b0 0x7fc85414eb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:27:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc853fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8540aef00 0x7fc854153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc853fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8540aef00 0x7fc854153c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42436/0 (socket says 172.21.15.112:42436) 2024-04-25T01:27:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc853fff640 1 -- 172.21.15.112:0/520554469 learned_addr learned my addr 172.21.15.112:0/520554469 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:27:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc859277640 1 --2- 172.21.15.112:0/520554469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc854151040 0x7fc854149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc853fff640 1 -- 172.21.15.112:0/520554469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8540ae9b0 msgr2=0x7fc85414eb00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:27:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc853fff640 1 --2- 172.21.15.112:0/520554469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8540ae9b0 0x7fc85414eb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc853fff640 1 -- 172.21.15.112:0/520554469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc854151040 msgr2=0x7fc854149d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc853fff640 1 --2- 172.21.15.112:0/520554469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc854151040 0x7fc854149d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc853fff640 1 -- 172.21.15.112:0/520554469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85414dcf0 con 0x7fc8540aef00 2024-04-25T01:27:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc853fff640 1 --2- 172.21.15.112:0/520554469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8540aef00 0x7fc854153c30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc840000c00 tx=0x7fc8400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc8517fa640 1 -- 172.21.15.112:0/520554469 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840041070 con 0x7fc8540aef00 2024-04-25T01:27:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc8517fa640 1 -- 172.21.15.112:0/520554469 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc84003c870 con 0x7fc8540aef00 2024-04-25T01:27:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.925+0000 7fc85acda640 1 -- 172.21.15.112:0/520554469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc85414aca0 con 0x7fc8540aef00 2024-04-25T01:27:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.926+0000 7fc8517fa640 1 -- 172.21.15.112:0/520554469 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840037040 con 0x7fc8540aef00 2024-04-25T01:27:59.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.926+0000 7fc85acda640 1 -- 172.21.15.112:0/520554469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8541ed5d0 con 0x7fc8540aef00 2024-04-25T01:27:59.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.926+0000 7fc85acda640 1 --2- 172.21.15.112:0/520554469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc85414e390 0x7fc8540ad690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.926+0000 7fc85acda640 1 -- 172.21.15.112:0/520554469 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541ed810 con 0x7fc85414e390 2024-04-25T01:27:59.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.927+0000 7fc858a76640 1 --2- 172.21.15.112:0/520554469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc85414e390 0x7fc8540ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.927+0000 7fc858a76640 1 -- 172.21.15.112:0/520554469 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc854075980 con 0x7fc85414e390 2024-04-25T01:27:59.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.927+0000 7fc8517fa640 1 -- 172.21.15.112:0/520554469 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc840044070 con 0x7fc8540aef00 2024-04-25T01:27:59.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.927+0000 7fc858a76640 1 --2- 172.21.15.112:0/520554469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc85414e390 0x7fc8540ad690 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc848001eb0 tx=0x7fc84800c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:59.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.928+0000 7fc8517fa640 1 --2- 172.21.15.112:0/520554469 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc83806bde0 0x7fc83806e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.928+0000 7fc859277640 1 --2- 172.21.15.112:0/520554469 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc83806bde0 0x7fc83806e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.928+0000 7fc8517fa640 1 -- 172.21.15.112:0/520554469 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fc8400ac080 con 0x7fc8540aef00 2024-04-25T01:27:59.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.928+0000 7fc8517fa640 1 -- 172.21.15.112:0/520554469 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848017070 con 0x7fc85414e390 2024-04-25T01:27:59.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.928+0000 7fc8517fa640 1 -- 172.21.15.112:0/520554469 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc8480074e0 con 0x7fc85414e390 2024-04-25T01:27:59.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.928+0000 7fc8517fa640 1 -- 172.21.15.112:0/520554469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc85414e390 msgr2=0x7fc8540ad690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.928+0000 7fc8517fa640 1 --2- 172.21.15.112:0/520554469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc85414e390 0x7fc8540ad690 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc848001eb0 tx=0x7fc84800c7a0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.928+0000 7fc8517fa640 1 -- 172.21.15.112:0/520554469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc85414e390 msgr2=0x7fc8540ad690 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:27:59.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.928+0000 7fc8517fa640 1 --2- 172.21.15.112:0/520554469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc85414e390 0x7fc8540ad690 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.928+0000 7fc859277640 1 --2- 172.21.15.112:0/520554469 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc83806bde0 0x7fc83806e2a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc8541504f0 tx=0x7fc84c00d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:59.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.936+0000 7fc85acda640 1 --2- 172.21.15.112:0/520554469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8540adbd0 0x7fc8541ec920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.936+0000 7fc85acda640 1 -- 172.21.15.112:0/520554469 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc854153900 con 0x7fc8540adbd0 2024-04-25T01:27:59.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.936+0000 7fc858a76640 1 --2- 172.21.15.112:0/520554469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8540adbd0 0x7fc8541ec920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.936+0000 7fc858a76640 1 -- 172.21.15.112:0/520554469 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc85414aa20 con 0x7fc8540adbd0 2024-04-25T01:27:59.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.936+0000 7fc858a76640 1 --2- 172.21.15.112:0/520554469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8540adbd0 0x7fc8541ec920 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc848014000 tx=0x7fc848000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:59.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.936+0000 7fc8517fa640 1 -- 172.21.15.112:0/520554469 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848017070 con 0x7fc8540adbd0 2024-04-25T01:27:59.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.937+0000 7fc8517fa640 1 -- 172.21.15.112:0/520554469 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4717 (secure 0 0 0) 0x7fc84801ab00 con 0x7fc8540adbd0 2024-04-25T01:27:59.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.937+0000 7fc8517fa640 1 -- 172.21.15.112:0/520554469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8540adbd0 msgr2=0x7fc8541ec920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.937+0000 7fc8517fa640 1 --2- 172.21.15.112:0/520554469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8540adbd0 0x7fc8541ec920 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc848014000 tx=0x7fc848000bc0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.937+0000 7fc8517fa640 1 -- 172.21.15.112:0/520554469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8540adbd0 msgr2=0x7fc8541ec920 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:27:59.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.937+0000 7fc8517fa640 1 --2- 172.21.15.112:0/520554469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8540adbd0 0x7fc8541ec920 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc848014000 tx=0x7fc848000bc0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.916 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:27:59.916 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:27:59.916 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:27:59.916 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:27:59.917 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 12, 2024-04-25T01:27:59.917 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 48075, 2024-04-25T01:27:59.917 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:27:59.917 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:27:59.917 INFO:teuthology.orchestra.run.smithi112.stdout: 1 2024-04-25T01:27:59.917 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:59.917 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:27:59.917 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:27:59.917 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:27:59.917 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:27:59.917 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:59.917 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:59.917 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:59.917 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:59.918 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:59.918 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:59.918 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:59.918 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:59.918 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:59.918 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:59.918 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:59.918 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:59.918 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:27:59.918 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:27:59.918 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:59.918 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:59.918 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:59.918 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:59.918 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:59.919 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:59.919 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:59.919 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:59.919 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:59.919 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:59.919 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:59.919 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:59.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.919 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:27:59.919 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:27:59.919 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:27:59.919 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:27:59.919 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:27:59.919 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:27:59.919 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:27:59.919 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:27:59.920 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:27:59.920 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:27:59.920 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:27:59.920 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:27:59.920 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:27:59.920 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:27:59.920 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:27:59.920 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:27:59.920 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:27:59.920 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:27:59.920 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:27:59.920 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:27:59.920 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:27:59.920 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:27:59.920 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:27:59.920 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:27:59.921 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:27:59.921 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:27:59.921 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:27:59.921 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:27:59.921 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:59.921 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:27:59.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.921 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:27:59.921 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.921 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:27:59.921 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:27:59.921 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:59.921 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:59.921 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.921 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:59.921 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:27:59.922 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:59.922 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.922 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.922 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:59.922 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:27:59.922 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:59.922 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:59.922 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:59.922 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.922 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:27:59.922 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:27:59.922 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:59.922 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:59.922 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:59.922 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.923 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.923 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:27:59.923 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:27:59.923 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:59.923 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:59.923 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.923 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:59.923 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:27:59.923 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:59.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.923 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.923 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:59.923 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:27:59.923 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:59.923 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:59.924 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:59.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.924 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:27:59.924 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:27:59.924 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:59.924 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:59.924 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:59.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.924 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.924 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:27:59.924 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:27:59.924 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:27:59.924 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:27:59.924 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.924 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:27:59.925 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:27:59.925 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:59.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.925 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.925 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:27:59.925 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:27:59.925 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:27:59.925 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:59.925 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:59.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.925 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:27:59.925 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:27:59.925 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:27:59.925 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:27:59.925 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:27:59.926 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:59.926 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:59.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.926 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:27:59.926 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:27:59.926 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.926 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:59.926 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:59.926 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:27:59.926 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:59.926 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:59.926 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:27:59.926 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.926 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:59.927 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:59.927 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:27:59.927 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:59.927 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:27:59.927 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:27:59.927 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:27:59.927 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:27:59.927 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:27:59.927 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:27:59.927 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:27:59.927 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:27:59.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:27:59.927 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:27:59.927 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:27:59.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.937+0000 7fc85acda640 1 -- 172.21.15.112:0/520554469 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc83806bde0 msgr2=0x7fc83806e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.937+0000 7fc85acda640 1 --2- 172.21.15.112:0/520554469 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc83806bde0 0x7fc83806e2a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc8541504f0 tx=0x7fc84c00d040 comp rx=0 tx=0).stop 2024-04-25T01:27:59.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.937+0000 7fc85acda640 1 -- 172.21.15.112:0/520554469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8540aef00 msgr2=0x7fc854153c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.937+0000 7fc85acda640 1 --2- 172.21.15.112:0/520554469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8540aef00 0x7fc854153c30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc840000c00 tx=0x7fc8400046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.937+0000 7fc859277640 1 -- 172.21.15.112:0/520554469 reap_dead start 2024-04-25T01:27:59.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.937+0000 7fc859277640 1 -- 172.21.15.112:0/520554469 reap_dead start 2024-04-25T01:27:59.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.937+0000 7fc85acda640 1 -- 172.21.15.112:0/520554469 shutdown_connections 2024-04-25T01:27:59.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.937+0000 7fc85acda640 1 -- 172.21.15.112:0/520554469 >> 172.21.15.112:0/520554469 conn(0x7fc8540c91d0 msgr2=0x7fc8540a8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:59.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.937+0000 7fc85acda640 1 -- 172.21.15.112:0/520554469 shutdown_connections 2024-04-25T01:27:59.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:27:59.937+0000 7fc85acda640 1 -- 172.21.15.112:0/520554469 wait complete. 2024-04-25T01:27:59.929 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-04-25T01:27:59.930 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:27:59.930 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T01:27:59.976 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/types.h... yes 2024-04-25T01:27:59.994 INFO:tasks.workunit.client.1.smithi187.stdout:/bin/sh ./config.status --recheck 2024-04-25T01:28:00.036 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/stat.h... yes 2024-04-25T01:28:00.053 INFO:tasks.workunit.client.1.smithi187.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-04-25T01:28:00.101 INFO:tasks.workunit.client.0.smithi112.stdout:checking for stdlib.h... yes 2024-04-25T01:28:00.168 INFO:tasks.workunit.client.0.smithi112.stdout:checking for string.h... yes 2024-04-25T01:28:00.196 INFO:tasks.workunit.client.1.smithi187.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-25T01:28:00.230 INFO:tasks.workunit.client.0.smithi112.stdout:checking for memory.h... yes 2024-04-25T01:28:00.290 INFO:tasks.workunit.client.0.smithi112.stdout:checking for strings.h... yes 2024-04-25T01:28:00.350 INFO:tasks.workunit.client.0.smithi112.stdout:checking for inttypes.h... yes 2024-04-25T01:28:00.411 INFO:tasks.workunit.client.0.smithi112.stdout:checking for stdint.h... yes 2024-04-25T01:28:00.471 INFO:tasks.workunit.client.0.smithi112.stdout:checking for unistd.h... yes 2024-04-25T01:28:00.544 INFO:tasks.workunit.client.0.smithi112.stdout:checking pthread.h usability... yes 2024-04-25T01:28:00.572 INFO:tasks.workunit.client.0.smithi112.stdout:checking pthread.h presence... yes 2024-04-25T01:28:00.573 INFO:tasks.workunit.client.0.smithi112.stdout:checking for pthread.h... yes 2024-04-25T01:28:00.637 INFO:tasks.workunit.client.0.smithi112.stdout:checking fcntl.h usability... yes 2024-04-25T01:28:00.663 INFO:tasks.workunit.client.0.smithi112.stdout:checking fcntl.h presence... yes 2024-04-25T01:28:00.663 INFO:tasks.workunit.client.0.smithi112.stdout:checking for fcntl.h... yes 2024-04-25T01:28:00.727 INFO:tasks.workunit.client.0.smithi112.stdout:checking limits.h usability... yes 2024-04-25T01:28:00.753 INFO:tasks.workunit.client.0.smithi112.stdout:checking limits.h presence... yes 2024-04-25T01:28:00.753 INFO:tasks.workunit.client.0.smithi112.stdout:checking for limits.h... yes 2024-04-25T01:28:00.773 INFO:tasks.workunit.client.0.smithi112.stdout:checking for stdint.h... (cached) yes 2024-04-25T01:28:00.831 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/time.h usability... yes 2024-04-25T01:28:00.856 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/time.h presence... yes 2024-04-25T01:28:00.856 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/time.h... yes 2024-04-25T01:28:00.876 INFO:tasks.workunit.client.0.smithi112.stdout:checking for unistd.h... (cached) yes 2024-04-25T01:28:00.931 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/vfs.h usability... yes 2024-04-25T01:28:00.956 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/vfs.h presence... yes 2024-04-25T01:28:00.956 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/vfs.h... yes 2024-04-25T01:28:01.023 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/limits.h usability... no 2024-04-25T01:28:01.052 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/limits.h presence... no 2024-04-25T01:28:01.052 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/limits.h... no 2024-04-25T01:28:01.093 INFO:tasks.workunit.client.0.smithi112.stdout:checking for an ANSI C-conforming const... yes 2024-04-25T01:28:01.138 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-04-25T01:28:01.140 INFO:tasks.workunit.client.0.smithi112.stdout:checking for function prototypes... yes 2024-04-25T01:28:01.184 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether setvbuf arguments are reversed... no 2024-04-25T01:28:01.204 INFO:tasks.workunit.client.1.smithi187.stdout:checking whether build environment is sane... yes 2024-04-25T01:28:01.212 INFO:tasks.workunit.client.1.smithi187.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-04-25T01:28:01.213 INFO:tasks.workunit.client.1.smithi187.stdout:checking for gawk... gawk 2024-04-25T01:28:01.230 INFO:tasks.workunit.client.1.smithi187.stdout:checking whether make sets $(MAKE)... yes 2024-04-25T01:28:01.265 INFO:tasks.workunit.client.0.smithi112.stdout:checking for system... yes 2024-04-25T01:28:01.269 INFO:tasks.workunit.client.1.smithi187.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-04-25T01:28:01.269 INFO:tasks.workunit.client.1.smithi187.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-04-25T01:28:01.286 INFO:tasks.workunit.client.1.smithi187.stdout:checking for style of include used by make... GNU 2024-04-25T01:28:01.292 INFO:tasks.workunit.client.1.smithi187.stdout:checking for gcc... gcc 2024-04-25T01:28:01.354 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gettimeofday... yes 2024-04-25T01:28:01.418 INFO:tasks.workunit.client.1.smithi187.stdout:checking whether the C compiler works... yes 2024-04-25T01:28:01.418 INFO:tasks.workunit.client.1.smithi187.stdout:checking for C compiler default output file name... a.out 2024-04-25T01:28:01.446 INFO:tasks.workunit.client.0.smithi112.stdout:checking for mkdir... yes 2024-04-25T01:28:01.471 INFO:tasks.workunit.client.1.smithi187.stdout:checking for suffix of executables... 2024-04-25T01:28:01.522 INFO:tasks.workunit.client.1.smithi187.stdout:checking whether we are cross compiling... no 2024-04-25T01:28:01.534 INFO:tasks.workunit.client.0.smithi112.stdout:checking for strerror... yes 2024-04-25T01:28:01.568 INFO:tasks.workunit.client.1.smithi187.stdout:checking for suffix of object files... o 2024-04-25T01:28:01.598 INFO:tasks.workunit.client.1.smithi187.stdout:checking whether we are using the GNU C compiler... yes 2024-04-25T01:28:01.621 INFO:tasks.workunit.client.0.smithi112.stdout:checking for open64... yes 2024-04-25T01:28:01.630 INFO:tasks.workunit.client.1.smithi187.stdout:checking whether gcc accepts -g... yes 2024-04-25T01:28:01.685 INFO:tasks.workunit.client.1.smithi187.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-25T01:28:01.705 INFO:tasks.workunit.client.0.smithi112.stdout:checking for stat64... yes 2024-04-25T01:28:01.795 INFO:tasks.workunit.client.0.smithi112.stdout:checking for fseeko64... yes 2024-04-25T01:28:01.828 INFO:tasks.workunit.client.1.smithi187.stdout:checking dependency style of gcc... gcc3 2024-04-25T01:28:01.887 INFO:tasks.workunit.client.0.smithi112.stdout:checking for lrand48_r... yes 2024-04-25T01:28:01.887 INFO:tasks.workunit.client.1.smithi187.stdout:checking how to run the C preprocessor... gcc -E 2024-04-25T01:28:01.949 INFO:tasks.workunit.client.1.smithi187.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-25T01:28:01.950 INFO:tasks.workunit.client.1.smithi187.stdout:checking for egrep... /usr/bin/grep -E 2024-04-25T01:28:01.973 INFO:tasks.workunit.client.0.smithi112.stdout:checking for srand48_r... yes 2024-04-25T01:28:02.000 INFO:tasks.workunit.client.0.smithi112.stdout:configure: creating ./config.status 2024-04-25T01:28:02.118 INFO:tasks.workunit.client.1.smithi187.stdout:checking for ANSI C header files... yes 2024-04-25T01:28:02.133 INFO:tasks.workunit.client.0.smithi112.stdout:config.status: creating Makefile 2024-04-25T01:28:02.150 INFO:tasks.workunit.client.0.smithi112.stdout:config.status: creating config.h 2024-04-25T01:28:02.164 INFO:tasks.workunit.client.1.smithi187.stdout:checking for sys/types.h... yes 2024-04-25T01:28:02.177 INFO:tasks.workunit.client.0.smithi112.stdout:config.status: executing depfiles commands 2024-04-25T01:28:02.218 INFO:tasks.workunit.client.1.smithi187.stdout:checking for sys/stat.h... yes 2024-04-25T01:28:02.246 INFO:tasks.workunit.client.0.smithi112.stdout:config.status: executing default commands 2024-04-25T01:28:02.270 INFO:tasks.workunit.client.1.smithi187.stdout:checking for stdlib.h... yes 2024-04-25T01:28:02.313 INFO:tasks.workunit.client.0.smithi112.stderr:+ make 2024-04-25T01:28:02.323 INFO:tasks.workunit.client.1.smithi187.stdout:checking for string.h... yes 2024-04-25T01:28:02.346 INFO:tasks.workunit.client.0.smithi112.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-04-25T01:28:02.352 INFO:tasks.workunit.client.0.smithi112.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-04-25T01:28:02.353 INFO:tasks.workunit.client.0.smithi112.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-04-25T01:28:02.354 INFO:tasks.workunit.client.0.smithi112.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-04-25T01:28:02.354 INFO:tasks.workunit.client.0.smithi112.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-04-25T01:28:02.354 INFO:tasks.workunit.client.0.smithi112.stderr: any GNU archive site. 2024-04-25T01:28:02.367 INFO:tasks.workunit.client.0.smithi112.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-04-25T01:28:02.369 INFO:tasks.workunit.client.0.smithi112.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-04-25T01:28:02.369 INFO:tasks.workunit.client.0.smithi112.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-04-25T01:28:02.369 INFO:tasks.workunit.client.0.smithi112.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-04-25T01:28:02.369 INFO:tasks.workunit.client.0.smithi112.stderr: You might want to install the `Automake' and `Perl' packages. 2024-04-25T01:28:02.369 INFO:tasks.workunit.client.0.smithi112.stderr: Grab them from any GNU archive site. 2024-04-25T01:28:02.379 INFO:tasks.workunit.client.1.smithi187.stdout:checking for memory.h... yes 2024-04-25T01:28:02.438 INFO:tasks.workunit.client.1.smithi187.stdout:checking for strings.h... yes 2024-04-25T01:28:02.492 INFO:tasks.workunit.client.1.smithi187.stdout:checking for inttypes.h... yes 2024-04-25T01:28:02.547 INFO:tasks.workunit.client.1.smithi187.stdout:checking for stdint.h... yes 2024-04-25T01:28:02.548 INFO:tasks.workunit.client.0.smithi112.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-04-25T01:28:02.602 INFO:tasks.workunit.client.1.smithi187.stdout:checking for unistd.h... yes 2024-04-25T01:28:02.656 INFO:tasks.workunit.client.1.smithi187.stdout:checking minix/config.h usability... no 2024-04-25T01:28:02.691 INFO:tasks.workunit.client.1.smithi187.stdout:checking minix/config.h presence... no 2024-04-25T01:28:02.691 INFO:tasks.workunit.client.1.smithi187.stdout:checking for minix/config.h... no 2024-04-25T01:28:02.733 INFO:tasks.workunit.client.1.smithi187.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-04-25T01:28:02.743 INFO:tasks.workunit.client.1.smithi187.stdout:checking for gcc... (cached) gcc 2024-04-25T01:28:02.791 INFO:tasks.workunit.client.1.smithi187.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-25T01:28:02.791 INFO:tasks.workunit.client.1.smithi187.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-25T01:28:02.791 INFO:tasks.workunit.client.1.smithi187.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-25T01:28:02.791 INFO:tasks.workunit.client.1.smithi187.stdout:checking dependency style of gcc... (cached) gcc3 2024-04-25T01:28:02.849 INFO:tasks.workunit.client.1.smithi187.stdout:checking for main in -lm... yes 2024-04-25T01:28:02.868 INFO:tasks.workunit.client.0.smithi112.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-04-25T01:28:02.869 INFO:tasks.workunit.client.0.smithi112.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-04-25T01:28:02.869 INFO:tasks.workunit.client.0.smithi112.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-04-25T01:28:02.869 INFO:tasks.workunit.client.0.smithi112.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-04-25T01:28:02.909 INFO:tasks.workunit.client.1.smithi187.stdout:checking for main in -lpthread... yes 2024-04-25T01:28:02.913 INFO:tasks.workunit.client.1.smithi187.stdout:checking for ANSI C header files... (cached) yes 2024-04-25T01:28:02.954 INFO:tasks.workunit.client.1.smithi187.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-04-25T01:28:03.005 INFO:tasks.workunit.client.0.smithi112.stdout:/bin/sh ./config.status --recheck 2024-04-25T01:28:03.009 INFO:tasks.workunit.client.1.smithi187.stdout:checking pthread.h usability... yes 2024-04-25T01:28:03.034 INFO:tasks.workunit.client.1.smithi187.stdout:checking pthread.h presence... yes 2024-04-25T01:28:03.035 INFO:tasks.workunit.client.1.smithi187.stdout:checking for pthread.h... yes 2024-04-25T01:28:03.065 INFO:tasks.workunit.client.0.smithi112.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-04-25T01:28:03.097 INFO:tasks.workunit.client.1.smithi187.stdout:checking fcntl.h usability... yes 2024-04-25T01:28:03.119 INFO:tasks.workunit.client.1.smithi187.stdout:checking fcntl.h presence... yes 2024-04-25T01:28:03.119 INFO:tasks.workunit.client.1.smithi187.stdout:checking for fcntl.h... yes 2024-04-25T01:28:03.181 INFO:tasks.workunit.client.1.smithi187.stdout:checking limits.h usability... yes 2024-04-25T01:28:03.205 INFO:tasks.workunit.client.1.smithi187.stdout:checking limits.h presence... yes 2024-04-25T01:28:03.205 INFO:tasks.workunit.client.1.smithi187.stdout:checking for limits.h... yes 2024-04-25T01:28:03.211 INFO:tasks.workunit.client.0.smithi112.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-25T01:28:03.228 INFO:tasks.workunit.client.1.smithi187.stdout:checking for stdint.h... (cached) yes 2024-04-25T01:28:03.282 INFO:tasks.workunit.client.1.smithi187.stdout:checking sys/time.h usability... yes 2024-04-25T01:28:03.305 INFO:tasks.workunit.client.1.smithi187.stdout:checking sys/time.h presence... yes 2024-04-25T01:28:03.305 INFO:tasks.workunit.client.1.smithi187.stdout:checking for sys/time.h... yes 2024-04-25T01:28:03.324 INFO:tasks.workunit.client.1.smithi187.stdout:checking for unistd.h... (cached) yes 2024-04-25T01:28:03.372 INFO:tasks.workunit.client.1.smithi187.stdout:checking sys/vfs.h usability... yes 2024-04-25T01:28:03.393 INFO:tasks.workunit.client.1.smithi187.stdout:checking sys/vfs.h presence... yes 2024-04-25T01:28:03.393 INFO:tasks.workunit.client.1.smithi187.stdout:checking for sys/vfs.h... yes 2024-04-25T01:28:03.475 INFO:tasks.workunit.client.1.smithi187.stdout:checking sys/limits.h usability... no 2024-04-25T01:28:03.511 INFO:tasks.workunit.client.1.smithi187.stdout:checking sys/limits.h presence... no 2024-04-25T01:28:03.512 INFO:tasks.workunit.client.1.smithi187.stdout:checking for sys/limits.h... no 2024-04-25T01:28:03.544 INFO:tasks.workunit.client.1.smithi187.stdout:checking for an ANSI C-conforming const... yes 2024-04-25T01:28:03.578 INFO:tasks.workunit.client.1.smithi187.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-04-25T01:28:03.654 INFO:tasks.workunit.client.1.smithi187.stdout:checking for system... yes 2024-04-25T01:28:03.728 INFO:tasks.workunit.client.1.smithi187.stdout:checking for gettimeofday... yes 2024-04-25T01:28:03.801 INFO:tasks.workunit.client.1.smithi187.stdout:checking for mkdir... yes 2024-04-25T01:28:03.876 INFO:tasks.workunit.client.1.smithi187.stdout:checking for strerror... yes 2024-04-25T01:28:03.950 INFO:tasks.workunit.client.1.smithi187.stdout:checking for open64... yes 2024-04-25T01:28:04.024 INFO:tasks.workunit.client.1.smithi187.stdout:checking for stat64... yes 2024-04-25T01:28:04.098 INFO:tasks.workunit.client.1.smithi187.stdout:checking for fseeko64... yes 2024-04-25T01:28:04.172 INFO:tasks.workunit.client.1.smithi187.stdout:checking for lrand48_r... yes 2024-04-25T01:28:04.219 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether build environment is sane... yes 2024-04-25T01:28:04.227 INFO:tasks.workunit.client.0.smithi112.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-04-25T01:28:04.227 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gawk... gawk 2024-04-25T01:28:04.246 INFO:tasks.workunit.client.1.smithi187.stdout:checking for srand48_r... yes 2024-04-25T01:28:04.249 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether make sets $(MAKE)... yes 2024-04-25T01:28:04.272 INFO:tasks.workunit.client.1.smithi187.stdout:configure: creating ./config.status 2024-04-25T01:28:04.292 INFO:tasks.workunit.client.0.smithi112.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-04-25T01:28:04.293 INFO:tasks.workunit.client.0.smithi112.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-04-25T01:28:04.311 INFO:tasks.workunit.client.0.smithi112.stdout:checking for style of include used by make... GNU 2024-04-25T01:28:04.316 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc... gcc 2024-04-25T01:28:04.372 INFO:tasks.workunit.client.1.smithi187.stdout: /bin/sh ./config.status 2024-04-25T01:28:04.432 INFO:tasks.workunit.client.1.smithi187.stdout:config.status: creating Makefile 2024-04-25T01:28:04.445 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether the C compiler works... yes 2024-04-25T01:28:04.445 INFO:tasks.workunit.client.0.smithi112.stdout:checking for C compiler default output file name... a.out 2024-04-25T01:28:04.450 INFO:tasks.workunit.client.1.smithi187.stdout:config.status: creating config.h 2024-04-25T01:28:04.486 INFO:tasks.workunit.client.1.smithi187.stdout:config.status: executing depfiles commands 2024-04-25T01:28:04.501 INFO:tasks.workunit.client.0.smithi112.stdout:checking for suffix of executables... 2024-04-25T01:28:04.505 INFO:tasks.workunit.client.1.smithi187.stdout:config.status: executing default commands 2024-04-25T01:28:04.547 INFO:tasks.workunit.client.1.smithi187.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb/missing --run autoheader 2024-04-25T01:28:04.557 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether we are cross compiling... no 2024-04-25T01:28:04.600 INFO:tasks.workunit.client.0.smithi112.stdout:checking for suffix of object files... o 2024-04-25T01:28:04.630 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether we are using the GNU C compiler... yes 2024-04-25T01:28:04.661 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether gcc accepts -g... yes 2024-04-25T01:28:04.703 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-25T01:28:04.715 INFO:tasks.workunit.client.1.smithi187.stdout:rm -f stamp-h1 2024-04-25T01:28:04.718 INFO:tasks.workunit.client.1.smithi187.stdout:touch config.h.in 2024-04-25T01:28:04.720 INFO:tasks.workunit.client.1.smithi187.stdout:cd . && /bin/sh ./config.status config.h 2024-04-25T01:28:04.772 INFO:tasks.workunit.client.1.smithi187.stdout:config.status: creating config.h 2024-04-25T01:28:04.806 INFO:tasks.workunit.client.1.smithi187.stdout:make all-am 2024-04-25T01:28:04.831 INFO:tasks.workunit.client.1.smithi187.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb' 2024-04-25T01:28:04.831 INFO:tasks.workunit.client.1.smithi187.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-04-25T01:28:04.840 INFO:tasks.workunit.client.0.smithi112.stdout:checking dependency style of gcc... gcc3 2024-04-25T01:28:04.903 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to run the C preprocessor... gcc -E 2024-04-25T01:28:04.961 INFO:tasks.workunit.client.0.smithi112.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-25T01:28:04.962 INFO:tasks.workunit.client.0.smithi112.stdout:checking for egrep... /usr/bin/grep -E 2024-04-25T01:28:05.012 INFO:tasks.workunit.client.1.smithi187.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-04-25T01:28:05.018 INFO:tasks.workunit.client.1.smithi187.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-04-25T01:28:05.121 INFO:tasks.workunit.client.0.smithi112.stdout:checking for ANSI C header files... yes 2024-04-25T01:28:05.126 INFO:tasks.workunit.client.1.smithi187.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-04-25T01:28:05.131 INFO:tasks.workunit.client.1.smithi187.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-04-25T01:28:05.166 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/types.h... yes 2024-04-25T01:28:05.216 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/stat.h... yes 2024-04-25T01:28:05.234 INFO:tasks.workunit.client.1.smithi187.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-04-25T01:28:05.240 INFO:tasks.workunit.client.1.smithi187.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-04-25T01:28:05.267 INFO:tasks.workunit.client.0.smithi112.stdout:checking for stdlib.h... yes 2024-04-25T01:28:05.318 INFO:tasks.workunit.client.0.smithi112.stdout:checking for string.h... yes 2024-04-25T01:28:05.348 INFO:tasks.workunit.client.1.smithi187.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-04-25T01:28:05.353 INFO:tasks.workunit.client.1.smithi187.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-04-25T01:28:05.369 INFO:tasks.workunit.client.0.smithi112.stdout:checking for memory.h... yes 2024-04-25T01:28:05.421 INFO:tasks.workunit.client.0.smithi112.stdout:checking for strings.h... yes 2024-04-25T01:28:05.476 INFO:tasks.workunit.client.0.smithi112.stdout:checking for inttypes.h... yes 2024-04-25T01:28:05.498 INFO:tasks.workunit.client.1.smithi187.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-04-25T01:28:05.503 INFO:tasks.workunit.client.1.smithi187.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-04-25T01:28:05.528 INFO:tasks.workunit.client.0.smithi112.stdout:checking for stdint.h... yes 2024-04-25T01:28:05.571 INFO:tasks.workunit.client.1.smithi187.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-04-25T01:28:05.576 INFO:tasks.workunit.client.1.smithi187.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-04-25T01:28:05.580 INFO:tasks.workunit.client.0.smithi112.stdout:checking for unistd.h... yes 2024-04-25T01:28:05.629 INFO:tasks.workunit.client.1.smithi187.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-04-25T01:28:05.634 INFO:tasks.workunit.client.1.smithi187.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-04-25T01:28:05.638 INFO:tasks.workunit.client.0.smithi112.stdout:checking minix/config.h usability... no 2024-04-25T01:28:05.674 INFO:tasks.workunit.client.0.smithi112.stdout:checking minix/config.h presence... no 2024-04-25T01:28:05.675 INFO:tasks.workunit.client.0.smithi112.stdout:checking for minix/config.h... no 2024-04-25T01:28:05.694 INFO:tasks.workunit.client.1.smithi187.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-04-25T01:28:05.698 INFO:tasks.workunit.client.1.smithi187.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-04-25T01:28:05.716 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-04-25T01:28:05.726 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc... (cached) gcc 2024-04-25T01:28:05.773 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-25T01:28:05.774 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-25T01:28:05.774 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-25T01:28:05.775 INFO:tasks.workunit.client.0.smithi112.stdout:checking dependency style of gcc... (cached) gcc3 2024-04-25T01:28:05.834 INFO:tasks.workunit.client.0.smithi112.stdout:checking for main in -lm... yes 2024-04-25T01:28:05.897 INFO:tasks.workunit.client.0.smithi112.stdout:checking for main in -lpthread... yes 2024-04-25T01:28:05.901 INFO:tasks.workunit.client.0.smithi112.stdout:checking for ANSI C header files... (cached) yes 2024-04-25T01:28:05.915 INFO:tasks.workunit.client.1.smithi187.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-04-25T01:28:05.919 INFO:tasks.workunit.client.1.smithi187.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-04-25T01:28:05.942 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-04-25T01:28:05.996 INFO:tasks.workunit.client.0.smithi112.stdout:checking pthread.h usability... yes 2024-04-25T01:28:06.019 INFO:tasks.workunit.client.0.smithi112.stdout:checking pthread.h presence... yes 2024-04-25T01:28:06.019 INFO:tasks.workunit.client.0.smithi112.stdout:checking for pthread.h... yes 2024-04-25T01:28:06.062 INFO:tasks.workunit.client.1.smithi187.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-04-25T01:28:06.067 INFO:tasks.workunit.client.1.smithi187.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-04-25T01:28:06.077 INFO:tasks.workunit.client.0.smithi112.stdout:checking fcntl.h usability... yes 2024-04-25T01:28:06.099 INFO:tasks.workunit.client.0.smithi112.stdout:checking fcntl.h presence... yes 2024-04-25T01:28:06.099 INFO:tasks.workunit.client.0.smithi112.stdout:checking for fcntl.h... yes 2024-04-25T01:28:06.156 INFO:tasks.workunit.client.0.smithi112.stdout:checking limits.h usability... yes 2024-04-25T01:28:06.177 INFO:tasks.workunit.client.0.smithi112.stdout:checking limits.h presence... yes 2024-04-25T01:28:06.177 INFO:tasks.workunit.client.0.smithi112.stdout:checking for limits.h... yes 2024-04-25T01:28:06.201 INFO:tasks.workunit.client.0.smithi112.stdout:checking for stdint.h... (cached) yes 2024-04-25T01:28:06.242 INFO:tasks.workunit.client.1.smithi187.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-04-25T01:28:06.248 INFO:tasks.workunit.client.1.smithi187.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-04-25T01:28:06.254 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/time.h usability... yes 2024-04-25T01:28:06.275 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/time.h presence... yes 2024-04-25T01:28:06.275 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/time.h... yes 2024-04-25T01:28:06.294 INFO:tasks.workunit.client.0.smithi112.stdout:checking for unistd.h... (cached) yes 2024-04-25T01:28:06.334 INFO:tasks.workunit.client.1.smithi187.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-04-25T01:28:06.338 INFO:tasks.workunit.client.1.smithi187.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-04-25T01:28:06.348 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/vfs.h usability... yes 2024-04-25T01:28:06.370 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/vfs.h presence... yes 2024-04-25T01:28:06.370 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/vfs.h... yes 2024-04-25T01:28:06.435 INFO:tasks.workunit.client.1.smithi187.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-04-25T01:28:06.440 INFO:tasks.workunit.client.1.smithi187.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-04-25T01:28:06.444 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/limits.h usability... no 2024-04-25T01:28:06.486 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/limits.h presence... no 2024-04-25T01:28:06.486 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/limits.h... no 2024-04-25T01:28:06.519 INFO:tasks.workunit.client.0.smithi112.stdout:checking for an ANSI C-conforming const... yes 2024-04-25T01:28:06.553 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-04-25T01:28:06.568 INFO:tasks.workunit.client.1.smithi187.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-04-25T01:28:06.573 INFO:tasks.workunit.client.1.smithi187.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-04-25T01:28:06.615 INFO:tasks.workunit.client.1.smithi187.stderr:In file included from parser.h:22, 2024-04-25T01:28:06.615 INFO:tasks.workunit.client.1.smithi187.stderr: from parser.c:25: 2024-04-25T01:28:06.615 INFO:tasks.workunit.client.1.smithi187.stderr:parser.c: In function ‘init_filesys’: 2024-04-25T01:28:06.617 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:28:06.617 INFO:tasks.workunit.client.1.smithi187.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:28:06.617 INFO:tasks.workunit.client.1.smithi187.stderr: | ^ 2024-04-25T01:28:06.617 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:28:06.617 INFO:tasks.workunit.client.1.smithi187.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:28:06.617 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~ 2024-04-25T01:28:06.617 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:28:06.617 INFO:tasks.workunit.client.1.smithi187.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:28:06.617 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:28:06.617 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:28:06.617 INFO:tasks.workunit.client.1.smithi187.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-25T01:28:06.618 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~ 2024-04-25T01:28:06.618 INFO:tasks.workunit.client.1.smithi187.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:28:06.618 INFO:tasks.workunit.client.1.smithi187.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-04-25T01:28:06.618 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:28:06.618 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:28:06.618 INFO:tasks.workunit.client.1.smithi187.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:28:06.618 INFO:tasks.workunit.client.1.smithi187.stderr: | ^ 2024-04-25T01:28:06.618 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:28:06.618 INFO:tasks.workunit.client.1.smithi187.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:28:06.618 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~ 2024-04-25T01:28:06.618 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:28:06.618 INFO:tasks.workunit.client.1.smithi187.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:28:06.618 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:28:06.618 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:28:06.618 INFO:tasks.workunit.client.1.smithi187.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-25T01:28:06.619 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~ 2024-04-25T01:28:06.619 INFO:tasks.workunit.client.1.smithi187.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:28:06.619 INFO:tasks.workunit.client.1.smithi187.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-04-25T01:28:06.619 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:28:06.619 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:28:06.619 INFO:tasks.workunit.client.1.smithi187.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:28:06.619 INFO:tasks.workunit.client.1.smithi187.stderr: | ^ 2024-04-25T01:28:06.619 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:28:06.619 INFO:tasks.workunit.client.1.smithi187.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:28:06.619 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~ 2024-04-25T01:28:06.619 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:28:06.619 INFO:tasks.workunit.client.1.smithi187.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:28:06.619 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:28:06.619 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:28:06.619 INFO:tasks.workunit.client.1.smithi187.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-25T01:28:06.619 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~ 2024-04-25T01:28:06.619 INFO:tasks.workunit.client.1.smithi187.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:28:06.620 INFO:tasks.workunit.client.1.smithi187.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-04-25T01:28:06.620 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:28:06.620 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:28:06.620 INFO:tasks.workunit.client.1.smithi187.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:28:06.620 INFO:tasks.workunit.client.1.smithi187.stderr: | ^ 2024-04-25T01:28:06.620 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:28:06.620 INFO:tasks.workunit.client.1.smithi187.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:28:06.620 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~ 2024-04-25T01:28:06.620 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:28:06.620 INFO:tasks.workunit.client.1.smithi187.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:28:06.620 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:28:06.620 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:28:06.620 INFO:tasks.workunit.client.1.smithi187.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-25T01:28:06.620 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~ 2024-04-25T01:28:06.620 INFO:tasks.workunit.client.1.smithi187.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:28:06.620 INFO:tasks.workunit.client.1.smithi187.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-04-25T01:28:06.620 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:28:06.621 INFO:tasks.workunit.client.1.smithi187.stderr:parser.c: In function ‘init_tg_stats’: 2024-04-25T01:28:06.621 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:28:06.621 INFO:tasks.workunit.client.1.smithi187.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:28:06.621 INFO:tasks.workunit.client.1.smithi187.stderr: | ^ 2024-04-25T01:28:06.621 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:28:06.621 INFO:tasks.workunit.client.1.smithi187.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:28:06.621 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~ 2024-04-25T01:28:06.621 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:28:06.621 INFO:tasks.workunit.client.1.smithi187.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:28:06.621 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:28:06.621 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:28:06.621 INFO:tasks.workunit.client.1.smithi187.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-25T01:28:06.621 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~ 2024-04-25T01:28:06.621 INFO:tasks.workunit.client.1.smithi187.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:28:06.621 INFO:tasks.workunit.client.1.smithi187.stderr: 971 | list_for_each_entry(tmp_list, 2024-04-25T01:28:06.621 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:28:06.621 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:28:06.622 INFO:tasks.workunit.client.1.smithi187.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:28:06.622 INFO:tasks.workunit.client.1.smithi187.stderr: | ^ 2024-04-25T01:28:06.622 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:28:06.622 INFO:tasks.workunit.client.1.smithi187.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:28:06.622 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~ 2024-04-25T01:28:06.622 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:28:06.622 INFO:tasks.workunit.client.1.smithi187.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:28:06.622 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:28:06.622 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:28:06.622 INFO:tasks.workunit.client.1.smithi187.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-25T01:28:06.622 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~ 2024-04-25T01:28:06.622 INFO:tasks.workunit.client.1.smithi187.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:28:06.622 INFO:tasks.workunit.client.1.smithi187.stderr: 971 | list_for_each_entry(tmp_list, 2024-04-25T01:28:06.622 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:28:06.623 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:28:06.623 INFO:tasks.workunit.client.1.smithi187.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:28:06.623 INFO:tasks.workunit.client.1.smithi187.stderr: | ^ 2024-04-25T01:28:06.623 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:28:06.623 INFO:tasks.workunit.client.1.smithi187.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:28:06.623 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~ 2024-04-25T01:28:06.623 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:28:06.623 INFO:tasks.workunit.client.1.smithi187.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:28:06.623 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:28:06.623 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:28:06.623 INFO:tasks.workunit.client.1.smithi187.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-25T01:28:06.623 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~ 2024-04-25T01:28:06.623 INFO:tasks.workunit.client.1.smithi187.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:28:06.623 INFO:tasks.workunit.client.1.smithi187.stderr: 980 | list_for_each_entry(tmp_list, 2024-04-25T01:28:06.623 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:28:06.623 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:28:06.624 INFO:tasks.workunit.client.1.smithi187.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:28:06.624 INFO:tasks.workunit.client.1.smithi187.stderr: | ^ 2024-04-25T01:28:06.624 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:28:06.624 INFO:tasks.workunit.client.1.smithi187.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:28:06.624 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~ 2024-04-25T01:28:06.624 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:28:06.624 INFO:tasks.workunit.client.1.smithi187.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:28:06.624 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:28:06.624 INFO:tasks.workunit.client.1.smithi187.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:28:06.624 INFO:tasks.workunit.client.1.smithi187.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-25T01:28:06.624 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~ 2024-04-25T01:28:06.624 INFO:tasks.workunit.client.1.smithi187.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:28:06.624 INFO:tasks.workunit.client.1.smithi187.stderr: 980 | list_for_each_entry(tmp_list, 2024-04-25T01:28:06.624 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:28:06.628 INFO:tasks.workunit.client.0.smithi112.stdout:checking for system... yes 2024-04-25T01:28:06.662 INFO:tasks.workunit.client.1.smithi187.stderr:parser.c: In function ‘get_optstr’: 2024-04-25T01:28:06.662 INFO:tasks.workunit.client.1.smithi187.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-04-25T01:28:06.662 INFO:tasks.workunit.client.1.smithi187.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-04-25T01:28:06.663 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:28:06.663 INFO:tasks.workunit.client.1.smithi187.stderr:parser.c:200:14: note: source object declared here 2024-04-25T01:28:06.663 INFO:tasks.workunit.client.1.smithi187.stderr: 200 | char temp[BUFSIZE]; 2024-04-25T01:28:06.663 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~ 2024-04-25T01:28:06.668 INFO:tasks.workunit.client.1.smithi187.stderr:parser.c: In function ‘size64_convert’: 2024-04-25T01:28:06.668 INFO:tasks.workunit.client.1.smithi187.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-04-25T01:28:06.668 INFO:tasks.workunit.client.1.smithi187.stderr: 119 | memcpy(unit, "\0", 3); 2024-04-25T01:28:06.669 INFO:tasks.workunit.client.1.smithi187.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:28:06.703 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gettimeofday... yes 2024-04-25T01:28:06.776 INFO:tasks.workunit.client.0.smithi112.stdout:checking for mkdir... yes 2024-04-25T01:28:06.848 INFO:tasks.workunit.client.0.smithi112.stdout:checking for strerror... yes 2024-04-25T01:28:06.925 INFO:tasks.workunit.client.0.smithi112.stdout:checking for open64... yes 2024-04-25T01:28:06.963 INFO:tasks.workunit.client.1.smithi187.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-04-25T01:28:06.969 INFO:tasks.workunit.client.1.smithi187.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-04-25T01:28:07.000 INFO:tasks.workunit.client.0.smithi112.stdout:checking for stat64... yes 2024-04-25T01:28:07.044 INFO:tasks.workunit.client.1.smithi187.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-04-25T01:28:07.048 INFO:tasks.workunit.client.1.smithi187.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-04-25T01:28:07.074 INFO:tasks.workunit.client.0.smithi112.stdout:checking for fseeko64... yes 2024-04-25T01:28:07.151 INFO:tasks.workunit.client.0.smithi112.stdout:checking for lrand48_r... yes 2024-04-25T01:28:07.151 INFO:tasks.workunit.client.1.smithi187.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-04-25T01:28:07.157 INFO:tasks.workunit.client.1.smithi187.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-04-25T01:28:07.196 INFO:tasks.workunit.client.1.smithi187.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-04-25T01:28:07.201 INFO:tasks.workunit.client.1.smithi187.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-04-25T01:28:07.229 INFO:tasks.workunit.client.0.smithi112.stdout:checking for srand48_r... yes 2024-04-25T01:28:07.254 INFO:tasks.workunit.client.0.smithi112.stdout:configure: creating ./config.status 2024-04-25T01:28:07.300 INFO:tasks.workunit.client.1.smithi187.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb' 2024-04-25T01:28:07.300 INFO:tasks.workunit.client.1.smithi187.stderr:+ cd .. 2024-04-25T01:28:07.301 INFO:tasks.workunit.client.1.smithi187.stderr:+ mkdir tmp 2024-04-25T01:28:07.304 INFO:tasks.workunit.client.1.smithi187.stderr:+ cd tmp 2024-04-25T01:28:07.305 INFO:tasks.workunit.client.1.smithi187.stderr:+ for f in $mydir/*.ffsb 2024-04-25T01:28:07.305 INFO:tasks.workunit.client.1.smithi187.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/random_write.32.ffsb 2024-04-25T01:28:07.306 INFO:tasks.workunit.client.1.smithi187.stdout:FFSB version 6.0-RC2 started 2024-04-25T01:28:07.306 INFO:tasks.workunit.client.1.smithi187.stdout: 2024-04-25T01:28:07.309 INFO:tasks.workunit.client.1.smithi187.stdout:benchmark time = 300 2024-04-25T01:28:07.309 INFO:tasks.workunit.client.1.smithi187.stdout:ThreadGroup 0 2024-04-25T01:28:07.309 INFO:tasks.workunit.client.1.smithi187.stdout:================ 2024-04-25T01:28:07.309 INFO:tasks.workunit.client.1.smithi187.stdout: num_threads = 32 2024-04-25T01:28:07.309 INFO:tasks.workunit.client.1.smithi187.stdout: 2024-04-25T01:28:07.309 INFO:tasks.workunit.client.1.smithi187.stdout: read_random = off 2024-04-25T01:28:07.309 INFO:tasks.workunit.client.1.smithi187.stdout: read_size = 0 (0B) 2024-04-25T01:28:07.309 INFO:tasks.workunit.client.1.smithi187.stdout: read_blocksize = 0 (0B) 2024-04-25T01:28:07.309 INFO:tasks.workunit.client.1.smithi187.stdout: read_skip = off 2024-04-25T01:28:07.309 INFO:tasks.workunit.client.1.smithi187.stdout: read_skipsize = 0 (0B) 2024-04-25T01:28:07.309 INFO:tasks.workunit.client.1.smithi187.stdout: 2024-04-25T01:28:07.310 INFO:tasks.workunit.client.1.smithi187.stdout: write_random = on 2024-04-25T01:28:07.310 INFO:tasks.workunit.client.1.smithi187.stdout: write_size = 5242880 (5MB) 2024-04-25T01:28:07.310 INFO:tasks.workunit.client.1.smithi187.stdout: fsync_file = 0 2024-04-25T01:28:07.310 INFO:tasks.workunit.client.1.smithi187.stdout: write_blocksize = 4096 (4KB) 2024-04-25T01:28:07.310 INFO:tasks.workunit.client.1.smithi187.stdout: wait time = 0 2024-04-25T01:28:07.310 INFO:tasks.workunit.client.1.smithi187.stdout: 2024-04-25T01:28:07.310 INFO:tasks.workunit.client.1.smithi187.stdout: op weights 2024-04-25T01:28:07.310 INFO:tasks.workunit.client.1.smithi187.stdout: read = 0 (0.00%) 2024-04-25T01:28:07.310 INFO:tasks.workunit.client.1.smithi187.stdout: readall = 0 (0.00%) 2024-04-25T01:28:07.310 INFO:tasks.workunit.client.1.smithi187.stdout: write = 1 (100.00%) 2024-04-25T01:28:07.310 INFO:tasks.workunit.client.1.smithi187.stdout: create = 0 (0.00%) 2024-04-25T01:28:07.310 INFO:tasks.workunit.client.1.smithi187.stdout: append = 0 (0.00%) 2024-04-25T01:28:07.310 INFO:tasks.workunit.client.1.smithi187.stdout: delete = 0 (0.00%) 2024-04-25T01:28:07.311 INFO:tasks.workunit.client.1.smithi187.stdout: metaop = 0 (0.00%) 2024-04-25T01:28:07.311 INFO:tasks.workunit.client.1.smithi187.stdout: createdir = 0 (0.00%) 2024-04-25T01:28:07.311 INFO:tasks.workunit.client.1.smithi187.stdout: stat = 0 (0.00%) 2024-04-25T01:28:07.311 INFO:tasks.workunit.client.1.smithi187.stdout: writeall = 0 (0.00%) 2024-04-25T01:28:07.311 INFO:tasks.workunit.client.1.smithi187.stdout: writeall_fsync = 0 (0.00%) 2024-04-25T01:28:07.311 INFO:tasks.workunit.client.1.smithi187.stdout: open_close = 0 (0.00%) 2024-04-25T01:28:07.311 INFO:tasks.workunit.client.1.smithi187.stdout: write_fsync = 0 (0.00%) 2024-04-25T01:28:07.311 INFO:tasks.workunit.client.1.smithi187.stdout: create_fsync = 0 (0.00%) 2024-04-25T01:28:07.311 INFO:tasks.workunit.client.1.smithi187.stdout: append_fsync = 0 (0.00%) 2024-04-25T01:28:07.311 INFO:tasks.workunit.client.1.smithi187.stdout: 2024-04-25T01:28:07.311 INFO:tasks.workunit.client.1.smithi187.stdout:FileSystem . 2024-04-25T01:28:07.311 INFO:tasks.workunit.client.1.smithi187.stdout:========== 2024-04-25T01:28:07.312 INFO:tasks.workunit.client.1.smithi187.stdout: num_dirs = 0 2024-04-25T01:28:07.312 INFO:tasks.workunit.client.1.smithi187.stdout: starting files = 128 2024-04-25T01:28:07.312 INFO:tasks.workunit.client.1.smithi187.stdout: 2024-04-25T01:28:07.312 INFO:tasks.workunit.client.1.smithi187.stdout: min file size = 104857600 (100MB) 2024-04-25T01:28:07.312 INFO:tasks.workunit.client.1.smithi187.stdout: max file size = 104857600 (100MB) 2024-04-25T01:28:07.312 INFO:tasks.workunit.client.1.smithi187.stdout: directio = off 2024-04-25T01:28:07.312 INFO:tasks.workunit.client.1.smithi187.stdout: alignedio = on 2024-04-25T01:28:07.312 INFO:tasks.workunit.client.1.smithi187.stdout: bufferedio = off 2024-04-25T01:28:07.312 INFO:tasks.workunit.client.1.smithi187.stdout: 2024-04-25T01:28:07.312 INFO:tasks.workunit.client.1.smithi187.stdout: aging is off 2024-04-25T01:28:07.313 INFO:tasks.workunit.client.1.smithi187.stdout: current utilization = 0.00% 2024-04-25T01:28:07.313 INFO:tasks.workunit.client.1.smithi187.stdout: 2024-04-25T01:28:07.313 INFO:tasks.workunit.client.1.smithi187.stdout:checking existing fs: . 2024-04-25T01:28:07.313 INFO:tasks.workunit.client.1.smithi187.stdout:recreating new fileset 2024-04-25T01:28:07.314 INFO:tasks.workunit.client.1.smithi187.stderr:opendir: No such file or directory 2024-04-25T01:28:07.357 INFO:tasks.workunit.client.0.smithi112.stdout: /bin/sh ./config.status 2024-04-25T01:28:07.429 INFO:tasks.workunit.client.0.smithi112.stdout:config.status: creating Makefile 2024-04-25T01:28:07.445 INFO:tasks.workunit.client.0.smithi112.stdout:config.status: creating config.h 2024-04-25T01:28:07.475 INFO:tasks.workunit.client.0.smithi112.stdout:config.status: executing depfiles commands 2024-04-25T01:28:07.493 INFO:tasks.workunit.client.0.smithi112.stdout:config.status: executing default commands 2024-04-25T01:28:07.545 INFO:tasks.workunit.client.0.smithi112.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-04-25T01:28:07.706 INFO:tasks.workunit.client.0.smithi112.stdout:rm -f stamp-h1 2024-04-25T01:28:07.708 INFO:tasks.workunit.client.0.smithi112.stdout:touch config.h.in 2024-04-25T01:28:07.710 INFO:tasks.workunit.client.0.smithi112.stdout:cd . && /bin/sh ./config.status config.h 2024-04-25T01:28:07.762 INFO:tasks.workunit.client.0.smithi112.stdout:config.status: creating config.h 2024-04-25T01:28:07.795 INFO:tasks.workunit.client.0.smithi112.stdout:make all-am 2024-04-25T01:28:07.819 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-04-25T01:28:07.819 INFO:tasks.workunit.client.0.smithi112.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-04-25T01:28:08.007 INFO:tasks.workunit.client.0.smithi112.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-04-25T01:28:08.011 INFO:tasks.workunit.client.0.smithi112.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-04-25T01:28:08.117 INFO:tasks.workunit.client.0.smithi112.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-04-25T01:28:08.121 INFO:tasks.workunit.client.0.smithi112.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-04-25T01:28:08.224 INFO:tasks.workunit.client.0.smithi112.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-04-25T01:28:08.229 INFO:tasks.workunit.client.0.smithi112.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-04-25T01:28:08.338 INFO:tasks.workunit.client.0.smithi112.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-04-25T01:28:08.343 INFO:tasks.workunit.client.0.smithi112.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-04-25T01:28:08.479 INFO:tasks.workunit.client.0.smithi112.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-04-25T01:28:08.484 INFO:tasks.workunit.client.0.smithi112.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-04-25T01:28:08.555 INFO:tasks.workunit.client.0.smithi112.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-04-25T01:28:08.560 INFO:tasks.workunit.client.0.smithi112.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-04-25T01:28:08.611 INFO:tasks.workunit.client.0.smithi112.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-04-25T01:28:08.615 INFO:tasks.workunit.client.0.smithi112.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-04-25T01:28:08.674 INFO:tasks.workunit.client.0.smithi112.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-04-25T01:28:08.679 INFO:tasks.workunit.client.0.smithi112.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-04-25T01:28:08.888 INFO:tasks.workunit.client.0.smithi112.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-04-25T01:28:08.892 INFO:tasks.workunit.client.0.smithi112.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-04-25T01:28:09.059 INFO:tasks.workunit.client.0.smithi112.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-04-25T01:28:09.064 INFO:tasks.workunit.client.0.smithi112.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-04-25T01:28:09.236 INFO:tasks.workunit.client.0.smithi112.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-04-25T01:28:09.240 INFO:tasks.workunit.client.0.smithi112.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-04-25T01:28:09.326 INFO:tasks.workunit.client.0.smithi112.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-04-25T01:28:09.331 INFO:tasks.workunit.client.0.smithi112.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-04-25T01:28:09.428 INFO:tasks.workunit.client.0.smithi112.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-04-25T01:28:09.433 INFO:tasks.workunit.client.0.smithi112.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-04-25T01:28:09.559 INFO:tasks.workunit.client.0.smithi112.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-04-25T01:28:09.564 INFO:tasks.workunit.client.0.smithi112.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-04-25T01:28:09.607 INFO:tasks.workunit.client.0.smithi112.stderr:In file included from parser.h:22, 2024-04-25T01:28:09.607 INFO:tasks.workunit.client.0.smithi112.stderr: from parser.c:25: 2024-04-25T01:28:09.607 INFO:tasks.workunit.client.0.smithi112.stderr:parser.c: In function ‘init_filesys’: 2024-04-25T01:28:09.608 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:28:09.608 INFO:tasks.workunit.client.0.smithi112.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:28:09.609 INFO:tasks.workunit.client.0.smithi112.stderr: | ^ 2024-04-25T01:28:09.609 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:28:09.609 INFO:tasks.workunit.client.0.smithi112.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:28:09.609 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~ 2024-04-25T01:28:09.609 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:28:09.609 INFO:tasks.workunit.client.0.smithi112.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:28:09.609 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:28:09.609 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:28:09.609 INFO:tasks.workunit.client.0.smithi112.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-25T01:28:09.609 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~ 2024-04-25T01:28:09.609 INFO:tasks.workunit.client.0.smithi112.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:28:09.609 INFO:tasks.workunit.client.0.smithi112.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-04-25T01:28:09.610 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:28:09.610 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:28:09.610 INFO:tasks.workunit.client.0.smithi112.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:28:09.610 INFO:tasks.workunit.client.0.smithi112.stderr: | ^ 2024-04-25T01:28:09.610 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:28:09.610 INFO:tasks.workunit.client.0.smithi112.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:28:09.610 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~ 2024-04-25T01:28:09.610 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:28:09.610 INFO:tasks.workunit.client.0.smithi112.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:28:09.610 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:28:09.610 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:28:09.610 INFO:tasks.workunit.client.0.smithi112.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-25T01:28:09.610 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~ 2024-04-25T01:28:09.610 INFO:tasks.workunit.client.0.smithi112.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:28:09.610 INFO:tasks.workunit.client.0.smithi112.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-04-25T01:28:09.610 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:28:09.611 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:28:09.611 INFO:tasks.workunit.client.0.smithi112.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:28:09.611 INFO:tasks.workunit.client.0.smithi112.stderr: | ^ 2024-04-25T01:28:09.611 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:28:09.611 INFO:tasks.workunit.client.0.smithi112.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:28:09.611 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~ 2024-04-25T01:28:09.611 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:28:09.611 INFO:tasks.workunit.client.0.smithi112.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:28:09.611 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:28:09.611 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:28:09.611 INFO:tasks.workunit.client.0.smithi112.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-25T01:28:09.611 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~ 2024-04-25T01:28:09.611 INFO:tasks.workunit.client.0.smithi112.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:28:09.611 INFO:tasks.workunit.client.0.smithi112.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-04-25T01:28:09.611 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:28:09.612 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:28:09.612 INFO:tasks.workunit.client.0.smithi112.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:28:09.612 INFO:tasks.workunit.client.0.smithi112.stderr: | ^ 2024-04-25T01:28:09.612 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:28:09.612 INFO:tasks.workunit.client.0.smithi112.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:28:09.612 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~ 2024-04-25T01:28:09.612 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:28:09.612 INFO:tasks.workunit.client.0.smithi112.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:28:09.612 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:28:09.612 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:28:09.612 INFO:tasks.workunit.client.0.smithi112.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-25T01:28:09.612 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~ 2024-04-25T01:28:09.612 INFO:tasks.workunit.client.0.smithi112.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:28:09.612 INFO:tasks.workunit.client.0.smithi112.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-04-25T01:28:09.613 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:28:09.613 INFO:tasks.workunit.client.0.smithi112.stderr:parser.c: In function ‘init_tg_stats’: 2024-04-25T01:28:09.613 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:28:09.613 INFO:tasks.workunit.client.0.smithi112.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:28:09.613 INFO:tasks.workunit.client.0.smithi112.stderr: | ^ 2024-04-25T01:28:09.613 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:28:09.613 INFO:tasks.workunit.client.0.smithi112.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:28:09.613 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~ 2024-04-25T01:28:09.613 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:28:09.613 INFO:tasks.workunit.client.0.smithi112.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:28:09.613 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:28:09.613 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:28:09.613 INFO:tasks.workunit.client.0.smithi112.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-25T01:28:09.613 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~ 2024-04-25T01:28:09.614 INFO:tasks.workunit.client.0.smithi112.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:28:09.614 INFO:tasks.workunit.client.0.smithi112.stderr: 971 | list_for_each_entry(tmp_list, 2024-04-25T01:28:09.614 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:28:09.614 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:28:09.614 INFO:tasks.workunit.client.0.smithi112.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:28:09.614 INFO:tasks.workunit.client.0.smithi112.stderr: | ^ 2024-04-25T01:28:09.614 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:28:09.614 INFO:tasks.workunit.client.0.smithi112.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:28:09.614 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~ 2024-04-25T01:28:09.614 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:28:09.614 INFO:tasks.workunit.client.0.smithi112.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:28:09.614 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:28:09.614 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:28:09.614 INFO:tasks.workunit.client.0.smithi112.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-25T01:28:09.615 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~ 2024-04-25T01:28:09.615 INFO:tasks.workunit.client.0.smithi112.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:28:09.615 INFO:tasks.workunit.client.0.smithi112.stderr: 971 | list_for_each_entry(tmp_list, 2024-04-25T01:28:09.615 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:28:09.615 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:28:09.615 INFO:tasks.workunit.client.0.smithi112.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:28:09.615 INFO:tasks.workunit.client.0.smithi112.stderr: | ^ 2024-04-25T01:28:09.615 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:28:09.615 INFO:tasks.workunit.client.0.smithi112.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:28:09.615 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~ 2024-04-25T01:28:09.615 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:28:09.615 INFO:tasks.workunit.client.0.smithi112.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:28:09.615 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:28:09.615 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:28:09.615 INFO:tasks.workunit.client.0.smithi112.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-25T01:28:09.616 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~ 2024-04-25T01:28:09.616 INFO:tasks.workunit.client.0.smithi112.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:28:09.616 INFO:tasks.workunit.client.0.smithi112.stderr: 980 | list_for_each_entry(tmp_list, 2024-04-25T01:28:09.616 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:28:09.616 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:28:09.616 INFO:tasks.workunit.client.0.smithi112.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:28:09.616 INFO:tasks.workunit.client.0.smithi112.stderr: | ^ 2024-04-25T01:28:09.616 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:28:09.616 INFO:tasks.workunit.client.0.smithi112.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:28:09.616 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~ 2024-04-25T01:28:09.616 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:28:09.616 INFO:tasks.workunit.client.0.smithi112.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:28:09.616 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:28:09.616 INFO:tasks.workunit.client.0.smithi112.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:28:09.617 INFO:tasks.workunit.client.0.smithi112.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-25T01:28:09.617 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~ 2024-04-25T01:28:09.617 INFO:tasks.workunit.client.0.smithi112.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:28:09.617 INFO:tasks.workunit.client.0.smithi112.stderr: 980 | list_for_each_entry(tmp_list, 2024-04-25T01:28:09.617 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:28:09.654 INFO:tasks.workunit.client.0.smithi112.stderr:parser.c: In function ‘get_optstr’: 2024-04-25T01:28:09.655 INFO:tasks.workunit.client.0.smithi112.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-04-25T01:28:09.655 INFO:tasks.workunit.client.0.smithi112.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-04-25T01:28:09.655 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:28:09.655 INFO:tasks.workunit.client.0.smithi112.stderr:parser.c:200:14: note: source object declared here 2024-04-25T01:28:09.655 INFO:tasks.workunit.client.0.smithi112.stderr: 200 | char temp[BUFSIZE]; 2024-04-25T01:28:09.655 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~ 2024-04-25T01:28:09.661 INFO:tasks.workunit.client.0.smithi112.stderr:parser.c: In function ‘size64_convert’: 2024-04-25T01:28:09.661 INFO:tasks.workunit.client.0.smithi112.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-04-25T01:28:09.661 INFO:tasks.workunit.client.0.smithi112.stderr: 119 | memcpy(unit, "\0", 3); 2024-04-25T01:28:09.661 INFO:tasks.workunit.client.0.smithi112.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:28:09.950 INFO:tasks.workunit.client.0.smithi112.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-04-25T01:28:09.955 INFO:tasks.workunit.client.0.smithi112.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-04-25T01:28:10.031 INFO:tasks.workunit.client.0.smithi112.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-04-25T01:28:10.036 INFO:tasks.workunit.client.0.smithi112.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-04-25T01:28:10.143 INFO:tasks.workunit.client.0.smithi112.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-04-25T01:28:10.148 INFO:tasks.workunit.client.0.smithi112.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-04-25T01:28:10.196 INFO:tasks.workunit.client.0.smithi112.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-04-25T01:28:10.203 INFO:tasks.workunit.client.0.smithi112.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-04-25T01:28:10.306 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-04-25T01:28:10.306 INFO:tasks.workunit.client.0.smithi112.stderr:+ cd .. 2024-04-25T01:28:10.308 INFO:tasks.workunit.client.0.smithi112.stderr:+ mkdir tmp 2024-04-25T01:28:10.310 INFO:tasks.workunit.client.0.smithi112.stderr:+ cd tmp 2024-04-25T01:28:10.311 INFO:tasks.workunit.client.0.smithi112.stderr:+ for f in $mydir/*.ffsb 2024-04-25T01:28:10.311 INFO:tasks.workunit.client.0.smithi112.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-04-25T01:28:10.312 INFO:tasks.workunit.client.0.smithi112.stdout:FFSB version 6.0-RC2 started 2024-04-25T01:28:10.312 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T01:28:10.315 INFO:tasks.workunit.client.0.smithi112.stdout:benchmark time = 300 2024-04-25T01:28:10.315 INFO:tasks.workunit.client.0.smithi112.stdout:ThreadGroup 0 2024-04-25T01:28:10.315 INFO:tasks.workunit.client.0.smithi112.stdout:================ 2024-04-25T01:28:10.315 INFO:tasks.workunit.client.0.smithi112.stdout: num_threads = 32 2024-04-25T01:28:10.315 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T01:28:10.315 INFO:tasks.workunit.client.0.smithi112.stdout: read_random = off 2024-04-25T01:28:10.316 INFO:tasks.workunit.client.0.smithi112.stdout: read_size = 0 (0B) 2024-04-25T01:28:10.316 INFO:tasks.workunit.client.0.smithi112.stdout: read_blocksize = 0 (0B) 2024-04-25T01:28:10.316 INFO:tasks.workunit.client.0.smithi112.stdout: read_skip = off 2024-04-25T01:28:10.316 INFO:tasks.workunit.client.0.smithi112.stdout: read_skipsize = 0 (0B) 2024-04-25T01:28:10.316 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T01:28:10.316 INFO:tasks.workunit.client.0.smithi112.stdout: write_random = on 2024-04-25T01:28:10.316 INFO:tasks.workunit.client.0.smithi112.stdout: write_size = 5242880 (5MB) 2024-04-25T01:28:10.316 INFO:tasks.workunit.client.0.smithi112.stdout: fsync_file = 0 2024-04-25T01:28:10.316 INFO:tasks.workunit.client.0.smithi112.stdout: write_blocksize = 4096 (4KB) 2024-04-25T01:28:10.316 INFO:tasks.workunit.client.0.smithi112.stdout: wait time = 0 2024-04-25T01:28:10.316 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T01:28:10.316 INFO:tasks.workunit.client.0.smithi112.stdout: op weights 2024-04-25T01:28:10.316 INFO:tasks.workunit.client.0.smithi112.stdout: read = 0 (0.00%) 2024-04-25T01:28:10.316 INFO:tasks.workunit.client.0.smithi112.stdout: readall = 0 (0.00%) 2024-04-25T01:28:10.316 INFO:tasks.workunit.client.0.smithi112.stdout: write = 1 (100.00%) 2024-04-25T01:28:10.317 INFO:tasks.workunit.client.0.smithi112.stdout: create = 0 (0.00%) 2024-04-25T01:28:10.317 INFO:tasks.workunit.client.0.smithi112.stdout: append = 0 (0.00%) 2024-04-25T01:28:10.317 INFO:tasks.workunit.client.0.smithi112.stdout: delete = 0 (0.00%) 2024-04-25T01:28:10.317 INFO:tasks.workunit.client.0.smithi112.stdout: metaop = 0 (0.00%) 2024-04-25T01:28:10.317 INFO:tasks.workunit.client.0.smithi112.stdout: createdir = 0 (0.00%) 2024-04-25T01:28:10.317 INFO:tasks.workunit.client.0.smithi112.stdout: stat = 0 (0.00%) 2024-04-25T01:28:10.317 INFO:tasks.workunit.client.0.smithi112.stdout: writeall = 0 (0.00%) 2024-04-25T01:28:10.317 INFO:tasks.workunit.client.0.smithi112.stdout: writeall_fsync = 0 (0.00%) 2024-04-25T01:28:10.317 INFO:tasks.workunit.client.0.smithi112.stdout: open_close = 0 (0.00%) 2024-04-25T01:28:10.317 INFO:tasks.workunit.client.0.smithi112.stdout: write_fsync = 0 (0.00%) 2024-04-25T01:28:10.317 INFO:tasks.workunit.client.0.smithi112.stdout: create_fsync = 0 (0.00%) 2024-04-25T01:28:10.317 INFO:tasks.workunit.client.0.smithi112.stdout: append_fsync = 0 (0.00%) 2024-04-25T01:28:10.317 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T01:28:10.317 INFO:tasks.workunit.client.0.smithi112.stdout:FileSystem . 2024-04-25T01:28:10.317 INFO:tasks.workunit.client.0.smithi112.stdout:========== 2024-04-25T01:28:10.317 INFO:tasks.workunit.client.0.smithi112.stdout: num_dirs = 0 2024-04-25T01:28:10.317 INFO:tasks.workunit.client.0.smithi112.stdout: starting files = 128 2024-04-25T01:28:10.318 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T01:28:10.318 INFO:tasks.workunit.client.0.smithi112.stdout: min file size = 104857600 (100MB) 2024-04-25T01:28:10.318 INFO:tasks.workunit.client.0.smithi112.stdout: max file size = 104857600 (100MB) 2024-04-25T01:28:10.318 INFO:tasks.workunit.client.0.smithi112.stdout: directio = off 2024-04-25T01:28:10.318 INFO:tasks.workunit.client.0.smithi112.stdout: alignedio = on 2024-04-25T01:28:10.318 INFO:tasks.workunit.client.0.smithi112.stdout: bufferedio = off 2024-04-25T01:28:10.318 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T01:28:10.318 INFO:tasks.workunit.client.0.smithi112.stdout: aging is off 2024-04-25T01:28:10.318 INFO:tasks.workunit.client.0.smithi112.stdout: current utilization = 0.00% 2024-04-25T01:28:10.318 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T01:28:10.318 INFO:tasks.workunit.client.0.smithi112.stdout:checking existing fs: . 2024-04-25T01:28:10.318 INFO:tasks.workunit.client.0.smithi112.stdout:recreating new fileset 2024-04-25T01:28:10.318 INFO:tasks.workunit.client.0.smithi112.stderr:opendir: No such file or directory 2024-04-25T01:28:19.931 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-25T01:28:19.931 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-04-25T01:28:19.931 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T01:28:19.932 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T01:28:19.934 INFO:tasks.ceph.mon.c:Started 2024-04-25T01:28:19.934 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-04-25T01:28:19.934 INFO:tasks.ceph.mon.b:Sent signal 19 2024-04-25T01:28:19.935 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-04-25T01:28:24.171 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:28:24.174 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:28:34.936 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-04-25T01:28:34.937 INFO:tasks.ceph.mon.b:Sent signal 18 2024-04-25T01:28:34.937 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:28:34.937 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:28:35.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.066+0000 7f7bba590640 1 Processor -- start 2024-04-25T01:28:35.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.066+0000 7f7bba590640 1 -- start start 2024-04-25T01:28:35.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.066+0000 7f7bba590640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7bb40c5d70 0x7f7bb40c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.066+0000 7f7bba590640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bb4156cf0 0x7f7bb406bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.066+0000 7f7bba590640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bb40c90e0 con 0x7f7bb4152fd0 2024-04-25T01:28:35.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.066+0000 7f7bba590640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7bb40ca9b0 con 0x7f7bb40c5d70 2024-04-25T01:28:35.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.066+0000 7f7bba590640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7bb40cac00 con 0x7f7bb4156cf0 2024-04-25T01:28:35.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.067+0000 7f7bb3fff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f7bb4152fd0 0x7f7bb4156280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:35526/0 (socket says 172.21.15.112:35526) 2024-04-25T01:28:35.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.067+0000 7f7bb3fff640 1 -- 172.21.15.112:0/1968190305 learned_addr learned my addr 172.21.15.112:0/1968190305 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:28:35.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.067+0000 7f7bb8b2d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bb4156cf0 0x7f7bb406bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.067+0000 7f7bb37fe640 1 --2- 172.21.15.112:0/1968190305 >> v2:172.21.15.187:3300/0 conn(0x7f7bb40c5d70 0x7f7bb40c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.067+0000 7f7bb2ffd640 1 -- 172.21.15.112:0/1968190305 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3139663452 0 0) 0x7f7bb40c90e0 con 0x7f7bb4152fd0 2024-04-25T01:28:35.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.067+0000 7f7bb2ffd640 1 -- 172.21.15.112:0/1968190305 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b9c003520 con 0x7f7bb4152fd0 2024-04-25T01:28:35.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.067+0000 7f7bb37fe640 1 -- 172.21.15.112:0/1968190305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bb4156cf0 msgr2=0x7f7bb406bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:28:35.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.067+0000 7f7bb37fe640 1 --2- 172.21.15.112:0/1968190305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bb4156cf0 0x7f7bb406bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.067+0000 7f7bb37fe640 1 -- 172.21.15.112:0/1968190305 >> v1:172.21.15.187:6789/0 conn(0x7f7bb4152fd0 legacy=0x7f7bb4156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.067+0000 7f7bb37fe640 1 -- 172.21.15.112:0/1968190305 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb40c9380 con 0x7f7bb40c5d70 2024-04-25T01:28:35.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.067+0000 7f7bb8b2d640 1 --2- 172.21.15.112:0/1968190305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bb4156cf0 0x7f7bb406bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:28:35.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.068+0000 7f7bb37fe640 1 --2- 172.21.15.112:0/1968190305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bb40c5d70 0x7f7bb40c9b80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80097a0 tx=0x7f7ba802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25839f7532a6a74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.068+0000 7f7bb2ffd640 1 -- 172.21.15.112:0/1968190305 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8037070 con 0x7f7bb40c5d70 2024-04-25T01:28:35.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.068+0000 7f7bb2ffd640 1 -- 172.21.15.112:0/1968190305 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba802ae50 con 0x7f7bb40c5d70 2024-04-25T01:28:35.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.068+0000 7f7bb2ffd640 1 -- 172.21.15.112:0/1968190305 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8032660 con 0x7f7bb40c5d70 2024-04-25T01:28:35.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.068+0000 7f7bba590640 1 -- 172.21.15.112:0/1968190305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bb40c5d70 msgr2=0x7f7bb40c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.068+0000 7f7bba590640 1 --2- 172.21.15.112:0/1968190305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bb40c5d70 0x7f7bb40c9b80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80097a0 tx=0x7f7ba802a100 comp rx=0 tx=0).stop 2024-04-25T01:28:35.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.068+0000 7f7bba590640 1 -- 172.21.15.112:0/1968190305 shutdown_connections 2024-04-25T01:28:35.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.068+0000 7f7bba590640 1 --2- 172.21.15.112:0/1968190305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bb4156cf0 0x7f7bb406bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.068+0000 7f7bba590640 1 --2- 172.21.15.112:0/1968190305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bb40c5d70 0x7f7bb40c9b80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.068+0000 7f7bba590640 1 -- 172.21.15.112:0/1968190305 >> 172.21.15.112:0/1968190305 conn(0x7f7bb40acc70 msgr2=0x7f7bb40ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:35.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.068+0000 7f7bba590640 1 -- 172.21.15.112:0/1968190305 shutdown_connections 2024-04-25T01:28:35.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.068+0000 7f7bba590640 1 -- 172.21.15.112:0/1968190305 wait complete. 2024-04-25T01:28:35.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.070+0000 7f7bba590640 1 Processor -- start 2024-04-25T01:28:35.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.070+0000 7f7bba590640 1 -- start start 2024-04-25T01:28:35.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.070+0000 7f7bba590640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb40c5d70 0x7f7bb41548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.070+0000 7f7bba590640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bb4152fd0 0x7f7bb414d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.070+0000 7f7bba590640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bb4156cf0 0x7f7bb414db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.070+0000 7f7bba590640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb4157140 con 0x7f7bb40c5d70 2024-04-25T01:28:35.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.070+0000 7f7bba590640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb40caee0 con 0x7f7bb4152fd0 2024-04-25T01:28:35.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.070+0000 7f7bba590640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7bb41572c0 con 0x7f7bb4156cf0 2024-04-25T01:28:35.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.071+0000 7f7bb3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb40c5d70 0x7f7bb41548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.071+0000 7f7bb3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb40c5d70 0x7f7bb41548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54188/0 (socket says 172.21.15.112:54188) 2024-04-25T01:28:35.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.071+0000 7f7bb3fff640 1 -- 172.21.15.112:0/80580951 learned_addr learned my addr 172.21.15.112:0/80580951 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:28:35.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.071+0000 7f7bb37fe640 1 --2- 172.21.15.112:0/80580951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bb4152fd0 0x7f7bb414d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.071+0000 7f7bb8b2d640 1 --2- 172.21.15.112:0/80580951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bb4156cf0 0x7f7bb414db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.071+0000 7f7bb3fff640 1 -- 172.21.15.112:0/80580951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bb4156cf0 msgr2=0x7f7bb414db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.071+0000 7f7bb3fff640 1 --2- 172.21.15.112:0/80580951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bb4156cf0 0x7f7bb414db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.071+0000 7f7bb3fff640 1 -- 172.21.15.112:0/80580951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bb4152fd0 msgr2=0x7f7bb414d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.071+0000 7f7bb3fff640 1 --2- 172.21.15.112:0/80580951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bb4152fd0 0x7f7bb414d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.071+0000 7f7bb3fff640 1 -- 172.21.15.112:0/80580951 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb40c2650 con 0x7f7bb40c5d70 2024-04-25T01:28:35.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.071+0000 7f7bb37fe640 1 --2- 172.21.15.112:0/80580951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bb4152fd0 0x7f7bb414d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:28:35.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.071+0000 7f7bb3fff640 1 --2- 172.21.15.112:0/80580951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb40c5d70 0x7f7bb41548f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400c500 tx=0x7f7ba400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.071+0000 7f7bb0ff9640 1 -- 172.21.15.112:0/80580951 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4017070 con 0x7f7bb40c5d70 2024-04-25T01:28:35.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.071+0000 7f7bb0ff9640 1 -- 172.21.15.112:0/80580951 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba4005ce0 con 0x7f7bb40c5d70 2024-04-25T01:28:35.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.071+0000 7f7bba590640 1 -- 172.21.15.112:0/80580951 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb41eb6b0 con 0x7f7bb40c5d70 2024-04-25T01:28:35.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.071+0000 7f7bb0ff9640 1 -- 172.21.15.112:0/80580951 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4007310 con 0x7f7bb40c5d70 2024-04-25T01:28:35.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.071+0000 7f7bba590640 1 -- 172.21.15.112:0/80580951 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb41ebc70 con 0x7f7bb40c5d70 2024-04-25T01:28:35.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.072+0000 7f7bba590640 1 -- 172.21.15.112:0/80580951 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb4157940 con 0x7f7bb40c5d70 2024-04-25T01:28:35.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.073+0000 7f7bb0ff9640 1 -- 172.21.15.112:0/80580951 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ba401a070 con 0x7f7bb40c5d70 2024-04-25T01:28:35.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.073+0000 7f7bb0ff9640 1 --2- 172.21.15.112:0/80580951 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7ba006be30 0x7f7ba006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.074+0000 7f7bb0ff9640 1 -- 172.21.15.112:0/80580951 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f7ba408a4b0 con 0x7f7bb40c5d70 2024-04-25T01:28:35.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.074+0000 7f7bb37fe640 1 --2- 172.21.15.112:0/80580951 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7ba006be30 0x7f7ba006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.074+0000 7f7bb37fe640 1 --2- 172.21.15.112:0/80580951 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7ba006be30 0x7f7ba006e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8034000 tx=0x7f7ba802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.079+0000 7f7bb0ff9640 1 -- 172.21.15.112:0/80580951 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7bb4157940 con 0x7f7bb40c5d70 2024-04-25T01:28:35.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.378+0000 7f7bba590640 1 -- 172.21.15.112:0/80580951 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f7bb40682a0 con 0x7f7bb40c5d70 2024-04-25T01:28:35.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.379+0000 7f7bb0ff9640 1 -- 172.21.15.112:0/80580951 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1568 (secure 0 0 0) 0x7f7ba40585c0 con 0x7f7bb40c5d70 2024-04-25T01:28:35.359 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:28:35.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.381+0000 7f7bba590640 1 -- 172.21.15.112:0/80580951 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7ba006be30 msgr2=0x7f7ba006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.381+0000 7f7bba590640 1 --2- 172.21.15.112:0/80580951 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7ba006be30 0x7f7ba006e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8034000 tx=0x7f7ba802e000 comp rx=0 tx=0).stop 2024-04-25T01:28:35.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.381+0000 7f7bba590640 1 -- 172.21.15.112:0/80580951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb40c5d70 msgr2=0x7f7bb41548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.381+0000 7f7bba590640 1 --2- 172.21.15.112:0/80580951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb40c5d70 0x7f7bb41548f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400c500 tx=0x7f7ba400c900 comp rx=0 tx=0).stop 2024-04-25T01:28:35.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.381+0000 7f7bba590640 1 -- 172.21.15.112:0/80580951 shutdown_connections 2024-04-25T01:28:35.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.381+0000 7f7bba590640 1 --2- 172.21.15.112:0/80580951 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7ba006be30 0x7f7ba006e2f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.381+0000 7f7bba590640 1 --2- 172.21.15.112:0/80580951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bb4156cf0 0x7f7bb414db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.381+0000 7f7bba590640 1 --2- 172.21.15.112:0/80580951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bb4152fd0 0x7f7bb414d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.381+0000 7f7bba590640 1 --2- 172.21.15.112:0/80580951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb40c5d70 0x7f7bb41548f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.381+0000 7f7bba590640 1 -- 172.21.15.112:0/80580951 >> 172.21.15.112:0/80580951 conn(0x7f7bb40acc70 msgr2=0x7f7bb406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:35.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.381+0000 7f7bba590640 1 -- 172.21.15.112:0/80580951 shutdown_connections 2024-04-25T01:28:35.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.381+0000 7f7bba590640 1 -- 172.21.15.112:0/80580951 wait complete. 2024-04-25T01:28:35.373 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:28:35.373 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:28:35.374 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:28:35.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.502+0000 7f608adfb640 1 Processor -- start 2024-04-25T01:28:35.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.503+0000 7f608adfb640 1 -- start start 2024-04-25T01:28:35.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.503+0000 7f608adfb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6084071520 0x7f6084154b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.503+0000 7f608adfb640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6084154120 con 0x7f60840c7800 2024-04-25T01:28:35.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.503+0000 7f608adfb640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6084154400 con 0x7f60840c7c80 2024-04-25T01:28:35.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.503+0000 7f608adfb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f60840c7000 con 0x7f6084071520 2024-04-25T01:28:35.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.503+0000 7f6083fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f60840c7800 0x7f60840c82d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32806/0 (socket says 172.21.15.112:32806) 2024-04-25T01:28:35.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.503+0000 7f6083fff640 1 -- 172.21.15.112:0/3214923291 learned_addr learned my addr 172.21.15.112:0/3214923291 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:28:35.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.504+0000 7f6088b97640 1 --2- 172.21.15.112:0/3214923291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6084071520 0x7f6084154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.504+0000 7f60837fe640 1 -- 172.21.15.112:0/3214923291 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 142299919 0 0) 0x7f6084154120 con 0x7f60840c7800 2024-04-25T01:28:35.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.504+0000 7f60837fe640 1 -- 172.21.15.112:0/3214923291 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6074003520 con 0x7f60840c7800 2024-04-25T01:28:35.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.504+0000 7f60837fe640 1 -- 172.21.15.112:0/3214923291 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3239324354 0 0) 0x7f6084154400 con 0x7f60840c7c80 2024-04-25T01:28:35.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.504+0000 7f60837fe640 1 -- 172.21.15.112:0/3214923291 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6084154120 con 0x7f60840c7c80 2024-04-25T01:28:35.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.504+0000 7f6088b97640 1 -- 172.21.15.112:0/3214923291 >> v1:172.21.15.187:6789/0 conn(0x7f60840c7c80 legacy=0x7f6084153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.504+0000 7f6088b97640 1 -- 172.21.15.112:0/3214923291 >> v1:172.21.15.112:6789/0 conn(0x7f60840c7800 legacy=0x7f60840c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.504+0000 7f6088b97640 1 -- 172.21.15.112:0/3214923291 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60840c72e0 con 0x7f6084071520 2024-04-25T01:28:35.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.505+0000 7f6088b97640 1 --2- 172.21.15.112:0/3214923291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6084071520 0x7f6084154b50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f60780097d0 tx=0x7f607802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=364eee05169e1773 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.505+0000 7f60837fe640 1 -- 172.21.15.112:0/3214923291 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078037070 con 0x7f6084071520 2024-04-25T01:28:35.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.505+0000 7f60837fe640 1 -- 172.21.15.112:0/3214923291 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f607802ae50 con 0x7f6084071520 2024-04-25T01:28:35.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.505+0000 7f60837fe640 1 -- 172.21.15.112:0/3214923291 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078032660 con 0x7f6084071520 2024-04-25T01:28:35.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.506+0000 7f608adfb640 1 -- 172.21.15.112:0/3214923291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6084071520 msgr2=0x7f6084154b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.506+0000 7f608adfb640 1 --2- 172.21.15.112:0/3214923291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6084071520 0x7f6084154b50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f60780097d0 tx=0x7f607802a100 comp rx=0 tx=0).stop 2024-04-25T01:28:35.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.506+0000 7f608adfb640 1 -- 172.21.15.112:0/3214923291 shutdown_connections 2024-04-25T01:28:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.506+0000 7f608adfb640 1 --2- 172.21.15.112:0/3214923291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6084071520 0x7f6084154b50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.506+0000 7f608adfb640 1 -- 172.21.15.112:0/3214923291 >> 172.21.15.112:0/3214923291 conn(0x7f6084074a00 msgr2=0x7f60840c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.506+0000 7f608adfb640 1 -- 172.21.15.112:0/3214923291 shutdown_connections 2024-04-25T01:28:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.506+0000 7f608adfb640 1 -- 172.21.15.112:0/3214923291 wait complete. 2024-04-25T01:28:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.507+0000 7f608adfb640 1 Processor -- start 2024-04-25T01:28:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.507+0000 7f608adfb640 1 -- start start 2024-04-25T01:28:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.508+0000 7f608adfb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6084071520 0x7f60841eef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.508+0000 7f608adfb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60840c7800 0x7f60840c2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.508+0000 7f608adfb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60840c7c80 0x7f60841eaf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.508+0000 7f608adfb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6084153d70 con 0x7f60840c7800 2024-04-25T01:28:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.508+0000 7f608adfb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f60840ad930 con 0x7f6084071520 2024-04-25T01:28:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.508+0000 7f608adfb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f60841ee4f0 con 0x7f60840c7c80 2024-04-25T01:28:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.508+0000 7f6083fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60840c7800 0x7f60840c2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.508+0000 7f6088b97640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6084071520 0x7f60841eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.508+0000 7f6088b97640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6084071520 0x7f60841eef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:51156/0 (socket says 172.21.15.112:51156) 2024-04-25T01:28:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.508+0000 7f6083fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60840c7800 0x7f60840c2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54192/0 (socket says 172.21.15.112:54192) 2024-04-25T01:28:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.508+0000 7f6088b97640 1 -- 172.21.15.112:0/1325342998 learned_addr learned my addr 172.21.15.112:0/1325342998 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:28:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.508+0000 7f6089398640 1 --2- 172.21.15.112:0/1325342998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60840c7c80 0x7f60841eaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.508+0000 7f6088b97640 1 -- 172.21.15.112:0/1325342998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60840c7c80 msgr2=0x7f60841eaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.508+0000 7f6088b97640 1 --2- 172.21.15.112:0/1325342998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60840c7c80 0x7f60841eaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.508+0000 7f6088b97640 1 -- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60840c7800 msgr2=0x7f60840c2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.508+0000 7f6088b97640 1 --2- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60840c7800 0x7f60840c2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.508+0000 7f6088b97640 1 -- 172.21.15.112:0/1325342998 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f608406d990 con 0x7f6084071520 2024-04-25T01:28:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.508+0000 7f6088b97640 1 --2- 172.21.15.112:0/1325342998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6084071520 0x7f60841eef40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f60780097d0 tx=0x7f60780046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.509+0000 7f60817fa640 1 -- 172.21.15.112:0/1325342998 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078041070 con 0x7f6084071520 2024-04-25T01:28:35.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.509+0000 7f60817fa640 1 -- 172.21.15.112:0/1325342998 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f607803c680 con 0x7f6084071520 2024-04-25T01:28:35.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.509+0000 7f608adfb640 1 -- 172.21.15.112:0/1325342998 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f608406dc70 con 0x7f6084071520 2024-04-25T01:28:35.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.509+0000 7f60817fa640 1 -- 172.21.15.112:0/1325342998 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078037040 con 0x7f6084071520 2024-04-25T01:28:35.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.509+0000 7f608adfb640 1 -- 172.21.15.112:0/1325342998 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60841eea20 con 0x7f6084071520 2024-04-25T01:28:35.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.510+0000 7f608adfb640 1 --2- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f608406d490 0x7f608414c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.510+0000 7f608adfb640 1 -- 172.21.15.112:0/1325342998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6084153ef0 con 0x7f608406d490 2024-04-25T01:28:35.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.510+0000 7f6083fff640 1 --2- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f608406d490 0x7f608414c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.510+0000 7f6083fff640 1 -- 172.21.15.112:0/1325342998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6084075980 con 0x7f608406d490 2024-04-25T01:28:35.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.510+0000 7f6083fff640 1 --2- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f608406d490 0x7f608414c130 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6070000f60 tx=0x7f60700019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.510+0000 7f60817fa640 1 -- 172.21.15.112:0/1325342998 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6078044070 con 0x7f6084071520 2024-04-25T01:28:35.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.511+0000 7f60817fa640 1 --2- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f606806be30 0x7f606806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.511+0000 7f60817fa640 1 -- 172.21.15.112:0/1325342998 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f60780ab990 con 0x7f6084071520 2024-04-25T01:28:35.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.511+0000 7f6089398640 1 --2- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f606806be30 0x7f606806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.511+0000 7f60817fa640 1 -- 172.21.15.112:0/1325342998 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6070005a60 con 0x7f608406d490 2024-04-25T01:28:35.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.511+0000 7f60817fa640 1 -- 172.21.15.112:0/1325342998 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6070018070 con 0x7f608406d490 2024-04-25T01:28:35.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.511+0000 7f60817fa640 1 -- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f608406d490 msgr2=0x7f608414c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.511+0000 7f60817fa640 1 --2- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f608406d490 0x7f608414c130 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6070000f60 tx=0x7f60700019b0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.511+0000 7f60817fa640 1 -- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f608406d490 msgr2=0x7f608414c130 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:28:35.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.511+0000 7f60817fa640 1 --2- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f608406d490 0x7f608414c130 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.511+0000 7f6089398640 1 --2- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f606806be30 0x7f606806e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f60840a9830 tx=0x7f607c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.518+0000 7f608adfb640 1 --2- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60840c5420 0x7f608414c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.518+0000 7f608adfb640 1 -- 172.21.15.112:0/1325342998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6084067da0 con 0x7f60840c5420 2024-04-25T01:28:35.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.518+0000 7f6083fff640 1 --2- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60840c5420 0x7f608414c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.518+0000 7f6083fff640 1 -- 172.21.15.112:0/1325342998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f60840682a0 con 0x7f60840c5420 2024-04-25T01:28:35.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.518+0000 7f6083fff640 1 --2- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60840c5420 0x7f608414c670 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6070001eb0 tx=0x7f6070001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.518+0000 7f60817fa640 1 -- 172.21.15.112:0/1325342998 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60700064e0 con 0x7f60840c5420 2024-04-25T01:28:35.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.519+0000 7f60817fa640 1 -- 172.21.15.112:0/1325342998 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5071 (secure 0 0 0) 0x7f60700064e0 con 0x7f60840c5420 2024-04-25T01:28:35.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.519+0000 7f60817fa640 1 -- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60840c5420 msgr2=0x7f608414c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.519+0000 7f60817fa640 1 --2- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60840c5420 0x7f608414c670 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6070001eb0 tx=0x7f6070001150 comp rx=0 tx=0).stop 2024-04-25T01:28:35.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.519+0000 7f60817fa640 1 -- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60840c5420 msgr2=0x7f608414c670 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:28:35.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.519+0000 7f60817fa640 1 --2- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60840c5420 0x7f608414c670 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.499 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:28:35.499 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:28:35.499 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:28:35.499 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:28:35.499 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 20, 2024-04-25T01:28:35.500 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 187147, 2024-04-25T01:28:35.500 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:28:35.500 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:28:35.500 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:28:35.500 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:28:35.500 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:35.500 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:28:35.500 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:28:35.500 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:28:35.500 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:28:35.500 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:28:35.500 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:28:35.500 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:28:35.500 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:28:35.500 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:28:35.501 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:28:35.501 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:28:35.501 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:28:35.501 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:28:35.501 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:28:35.501 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:28:35.501 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:35.501 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:28:35.501 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:28:35.501 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:28:35.501 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:28:35.501 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:28:35.501 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:28:35.502 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:28:35.502 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:28:35.502 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:28:35.502 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:28:35.502 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:28:35.502 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:28:35.502 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:28:35.502 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:35.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.502 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:28:35.502 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:28:35.502 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:28:35.502 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:28:35.503 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:28:35.503 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:28:35.503 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:28:35.503 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:28:35.503 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:28:35.503 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:28:35.503 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:28:35.503 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:28:35.503 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:28:35.503 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:28:35.503 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:28:35.503 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:28:35.503 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:28:35.503 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:28:35.503 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:28:35.504 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:28:35.504 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:28:35.504 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:28:35.504 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:28:35.504 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:28:35.504 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:28:35.504 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:28:35.504 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:28:35.504 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:28:35.504 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:35.504 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:28:35.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.504 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:28:35.504 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.505 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:28:35.505 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:28:35.505 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:28:35.505 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:28:35.505 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.505 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:28:35.505 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:28:35.505 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:35.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.505 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.505 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:28:35.505 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:28:35.505 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:35.505 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.506 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:35.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.506 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:28:35.506 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:28:35.506 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:28:35.506 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:28:35.506 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:28:35.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.506 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:28:35.506 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:28:35.506 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:28:35.506 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:28:35.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.507 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:28:35.507 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:28:35.507 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:35.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.507 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.507 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:28:35.507 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:28:35.507 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:35.507 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.507 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:35.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.508 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:28:35.508 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:28:35.508 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:28:35.508 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:28:35.508 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:28:35.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.508 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.508 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:28:35.508 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:28:35.508 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:28:35.509 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:28:35.509 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.509 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:28:35.509 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:28:35.509 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:35.509 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.509 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.509 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:28:35.509 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:28:35.509 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:35.509 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.510 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:35.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.510 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:28:35.510 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:28:35.510 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:28:35.510 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:28:35.510 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:28:35.510 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.510 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:35.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.510 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:28:35.511 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:28:35.511 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.511 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:28:35.511 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:28:35.511 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:28:35.511 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.511 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:35.511 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:28:35.511 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.511 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:28:35.511 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:28:35.512 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:28:35.512 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.512 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:35.512 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:28:35.512 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.512 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:28:35.512 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:28:35.512 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 7 2024-04-25T01:28:35.512 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.512 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:35.513 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:28:35.513 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.513 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:28:35.513 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:28:35.513 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 7 2024-04-25T01:28:35.513 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.513 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:35.513 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:28:35.513 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.513 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:28:35.513 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:28:35.514 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:28:35.514 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.514 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:35.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.514 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:28:35.514 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:28:35.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.519+0000 7f608adfb640 1 -- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f606806be30 msgr2=0x7f606806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.519+0000 7f608adfb640 1 --2- 172.21.15.112:0/1325342998 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f606806be30 0x7f606806e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f60840a9830 tx=0x7f607c009040 comp rx=0 tx=0).stop 2024-04-25T01:28:35.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.519+0000 7f608adfb640 1 -- 172.21.15.112:0/1325342998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6084071520 msgr2=0x7f60841eef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.519+0000 7f608adfb640 1 --2- 172.21.15.112:0/1325342998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6084071520 0x7f60841eef40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f60780097d0 tx=0x7f60780046c0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.519+0000 7f6089398640 1 -- 172.21.15.112:0/1325342998 reap_dead start 2024-04-25T01:28:35.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.519+0000 7f608adfb640 1 -- 172.21.15.112:0/1325342998 shutdown_connections 2024-04-25T01:28:35.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.519+0000 7f608adfb640 1 --2- 172.21.15.112:0/1325342998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6084071520 0x7f60841eef40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.519+0000 7f608adfb640 1 -- 172.21.15.112:0/1325342998 >> 172.21.15.112:0/1325342998 conn(0x7f6084074a00 msgr2=0x7f60840c82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:35.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.519+0000 7f608adfb640 1 -- 172.21.15.112:0/1325342998 shutdown_connections 2024-04-25T01:28:35.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.519+0000 7f608adfb640 1 -- 172.21.15.112:0/1325342998 wait complete. 2024-04-25T01:28:35.518 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:28:35.518 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:28:35.519 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:28:35.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.656+0000 7f08b1d90640 1 Processor -- start 2024-04-25T01:28:35.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.656+0000 7f08b1d90640 1 -- start start 2024-04-25T01:28:35.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.656+0000 7f08b1d90640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08ac1596a0 0x7f08ac152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.656+0000 7f08b1d90640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f08ac0c72b0 0x7f08ac160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.656+0000 7f08b1d90640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08ac159b90 con 0x7f08ac1592c0 2024-04-25T01:28:35.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.656+0000 7f08b1d90640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f08ac0c6e20 con 0x7f08ac0c72b0 2024-04-25T01:28:35.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.656+0000 7f08b1d90640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f08ac06c970 con 0x7f08ac1596a0 2024-04-25T01:28:35.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.657+0000 7f08abfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f08ac0c72b0 0x7f08ac160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.657+0000 7f08abfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f08ac0c72b0 0x7f08ac160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54206/0 (socket says 172.21.15.112:54206) 2024-04-25T01:28:35.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.657+0000 7f08abfff640 1 -- 172.21.15.112:0/2481352470 learned_addr learned my addr 172.21.15.112:0/2481352470 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:28:35.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.657+0000 7f08aaffd640 1 --2- 172.21.15.112:0/2481352470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08ac1596a0 0x7f08ac152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.657+0000 7f08abfff640 1 -- 172.21.15.112:0/2481352470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08ac1596a0 msgr2=0x7f08ac152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:28:35.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.657+0000 7f08abfff640 1 --2- 172.21.15.112:0/2481352470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08ac1596a0 0x7f08ac152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.657+0000 7f08abfff640 1 -- 172.21.15.112:0/2481352470 >> v1:172.21.15.187:6789/0 conn(0x7f08ac1592c0 legacy=0x7f08ac0c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.657+0000 7f08abfff640 1 -- 172.21.15.112:0/2481352470 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ac06cc50 con 0x7f08ac0c72b0 2024-04-25T01:28:35.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.658+0000 7f08abfff640 1 --2- 172.21.15.112:0/2481352470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08ac0c72b0 0x7f08ac160380 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f08a4009e50 tx=0x7f08a402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9386393ed41a2c14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.658+0000 7f08aa7fc640 1 -- 172.21.15.112:0/2481352470 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a4036070 con 0x7f08ac0c72b0 2024-04-25T01:28:35.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.658+0000 7f08aa7fc640 1 -- 172.21.15.112:0/2481352470 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a4031600 con 0x7f08ac0c72b0 2024-04-25T01:28:35.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.658+0000 7f08aa7fc640 1 -- 172.21.15.112:0/2481352470 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a40319d0 con 0x7f08ac0c72b0 2024-04-25T01:28:35.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.658+0000 7f08b1d90640 1 -- 172.21.15.112:0/2481352470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08ac0c72b0 msgr2=0x7f08ac160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.658+0000 7f08b1d90640 1 --2- 172.21.15.112:0/2481352470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08ac0c72b0 0x7f08ac160380 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f08a4009e50 tx=0x7f08a402a5f0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.658+0000 7f08b1d90640 1 -- 172.21.15.112:0/2481352470 shutdown_connections 2024-04-25T01:28:35.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.658+0000 7f08b1d90640 1 --2- 172.21.15.112:0/2481352470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08ac0c72b0 0x7f08ac160380 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.658+0000 7f08b1d90640 1 --2- 172.21.15.112:0/2481352470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08ac1596a0 0x7f08ac152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.658+0000 7f08b1d90640 1 -- 172.21.15.112:0/2481352470 >> 172.21.15.112:0/2481352470 conn(0x7f08ac0cb170 msgr2=0x7f08ac0cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:35.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.658+0000 7f08b1d90640 1 -- 172.21.15.112:0/2481352470 shutdown_connections 2024-04-25T01:28:35.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.658+0000 7f08b1d90640 1 -- 172.21.15.112:0/2481352470 wait complete. 2024-04-25T01:28:35.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.660+0000 7f08b1d90640 1 Processor -- start 2024-04-25T01:28:35.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.660+0000 7f08b1d90640 1 -- start start 2024-04-25T01:28:35.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.660+0000 7f08b1d90640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08ac0c72b0 0x7f08ac1e2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.660+0000 7f08b1d90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08ac1592c0 0x7f08ac1462c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.660+0000 7f08b1d90640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08ac1596a0 0x7f08ac146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.660+0000 7f08b1d90640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08ac06c510 con 0x7f08ac1592c0 2024-04-25T01:28:35.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.660+0000 7f08b1d90640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f08ac06c390 con 0x7f08ac1596a0 2024-04-25T01:28:35.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.660+0000 7f08b1d90640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f08ac06db20 con 0x7f08ac0c72b0 2024-04-25T01:28:35.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.661+0000 7f08aaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08ac1592c0 0x7f08ac1462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.661+0000 7f08ab7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08ac0c72b0 0x7f08ac1e2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.661+0000 7f08aaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08ac1592c0 0x7f08ac1462c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54212/0 (socket says 172.21.15.112:54212) 2024-04-25T01:28:35.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.661+0000 7f08aaffd640 1 -- 172.21.15.112:0/562926293 learned_addr learned my addr 172.21.15.112:0/562926293 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:28:35.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.661+0000 7f08abfff640 1 --2- 172.21.15.112:0/562926293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08ac1596a0 0x7f08ac146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.661+0000 7f08aaffd640 1 -- 172.21.15.112:0/562926293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08ac0c72b0 msgr2=0x7f08ac1e2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.661+0000 7f08aaffd640 1 --2- 172.21.15.112:0/562926293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08ac0c72b0 0x7f08ac1e2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.661+0000 7f08aaffd640 1 -- 172.21.15.112:0/562926293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08ac1596a0 msgr2=0x7f08ac146800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.661+0000 7f08aaffd640 1 --2- 172.21.15.112:0/562926293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08ac1596a0 0x7f08ac146800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.661+0000 7f08aaffd640 1 -- 172.21.15.112:0/562926293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ac157ad0 con 0x7f08ac1592c0 2024-04-25T01:28:35.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.661+0000 7f08aaffd640 1 --2- 172.21.15.112:0/562926293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08ac1592c0 0x7f08ac1462c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f08a000e840 tx=0x7f08a000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.661+0000 7f0893fff640 1 -- 172.21.15.112:0/562926293 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0017070 con 0x7f08ac1592c0 2024-04-25T01:28:35.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.661+0000 7f0893fff640 1 -- 172.21.15.112:0/562926293 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a000c8e0 con 0x7f08ac1592c0 2024-04-25T01:28:35.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.661+0000 7f08b1d90640 1 -- 172.21.15.112:0/562926293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08ac146d40 con 0x7f08ac1592c0 2024-04-25T01:28:35.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.661+0000 7f0893fff640 1 -- 172.21.15.112:0/562926293 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a000ccb0 con 0x7f08ac1592c0 2024-04-25T01:28:35.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.661+0000 7f08b1d90640 1 -- 172.21.15.112:0/562926293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08ac157130 con 0x7f08ac1592c0 2024-04-25T01:28:35.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.662+0000 7f08b1d90640 1 --2- 172.21.15.112:0/562926293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08ac158940 0x7f08ac157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.662+0000 7f08b1d90640 1 -- 172.21.15.112:0/562926293 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f08ac0682a0 con 0x7f08ac158940 2024-04-25T01:28:35.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.662+0000 7f08ab7fe640 1 --2- 172.21.15.112:0/562926293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08ac158940 0x7f08ac157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.663+0000 7f08ab7fe640 1 -- 172.21.15.112:0/562926293 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f08ac0c79d0 con 0x7f08ac158940 2024-04-25T01:28:35.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.663+0000 7f0893fff640 1 -- 172.21.15.112:0/562926293 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f08a001a070 con 0x7f08ac1592c0 2024-04-25T01:28:35.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.663+0000 7f08ab7fe640 1 --2- 172.21.15.112:0/562926293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08ac158940 0x7f08ac157330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f08ac159d60 tx=0x7f089c00f9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.663+0000 7f0893fff640 1 --2- 172.21.15.112:0/562926293 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f089806be30 0x7f089806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.663+0000 7f0893fff640 1 -- 172.21.15.112:0/562926293 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f08a008af80 con 0x7f08ac1592c0 2024-04-25T01:28:35.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.663+0000 7f08abfff640 1 --2- 172.21.15.112:0/562926293 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f089806be30 0x7f089806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.663+0000 7f0893fff640 1 -- 172.21.15.112:0/562926293 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c0052e0 con 0x7f08ac158940 2024-04-25T01:28:35.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.663+0000 7f0893fff640 1 -- 172.21.15.112:0/562926293 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f089c018070 con 0x7f08ac158940 2024-04-25T01:28:35.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.663+0000 7f0893fff640 1 -- 172.21.15.112:0/562926293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08ac158940 msgr2=0x7f08ac157330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.663+0000 7f0893fff640 1 --2- 172.21.15.112:0/562926293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08ac158940 0x7f08ac157330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f08ac159d60 tx=0x7f089c00f9d0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.664+0000 7f0893fff640 1 -- 172.21.15.112:0/562926293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08ac158940 msgr2=0x7f08ac157330 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:28:35.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.664+0000 7f0893fff640 1 --2- 172.21.15.112:0/562926293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08ac158940 0x7f08ac157330 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.664+0000 7f08abfff640 1 --2- 172.21.15.112:0/562926293 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f089806be30 0x7f089806e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f08a4033000 tx=0x7f08a4009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.671+0000 7f08b1d90640 1 --2- 172.21.15.112:0/562926293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08ac0abf30 0x7f08ac0ac340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.671+0000 7f08b1d90640 1 -- 172.21.15.112:0/562926293 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f08ac0c5ca0 con 0x7f08ac0abf30 2024-04-25T01:28:35.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.672+0000 7f08ab7fe640 1 --2- 172.21.15.112:0/562926293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08ac0abf30 0x7f08ac0ac340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.672+0000 7f08ab7fe640 1 -- 172.21.15.112:0/562926293 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f08ac06c0c0 con 0x7f08ac0abf30 2024-04-25T01:28:35.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.672+0000 7f08ab7fe640 1 --2- 172.21.15.112:0/562926293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08ac0abf30 0x7f08ac0ac340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f08ac159d60 tx=0x7f089c000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.672+0000 7f0893fff640 1 -- 172.21.15.112:0/562926293 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c003ad0 con 0x7f08ac0abf30 2024-04-25T01:28:35.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.672+0000 7f0893fff640 1 -- 172.21.15.112:0/562926293 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4728 (secure 0 0 0) 0x7f089c003ad0 con 0x7f08ac0abf30 2024-04-25T01:28:35.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.672+0000 7f0893fff640 1 -- 172.21.15.112:0/562926293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08ac0abf30 msgr2=0x7f08ac0ac340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.672+0000 7f0893fff640 1 --2- 172.21.15.112:0/562926293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08ac0abf30 0x7f08ac0ac340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f08ac159d60 tx=0x7f089c000ed0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.672+0000 7f0893fff640 1 -- 172.21.15.112:0/562926293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08ac0abf30 msgr2=0x7f08ac0ac340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:28:35.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.672+0000 7f0893fff640 1 --2- 172.21.15.112:0/562926293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08ac0abf30 0x7f08ac0ac340 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.653 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:28:35.653 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:28:35.653 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:28:35.654 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:28:35.654 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 20, 2024-04-25T01:28:35.654 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 83812, 2024-04-25T01:28:35.654 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:28:35.654 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:28:35.654 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:28:35.654 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:28:35.654 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:35.654 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:28:35.654 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:28:35.654 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:28:35.655 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:28:35.655 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:28:35.655 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:28:35.655 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:28:35.655 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:28:35.655 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:28:35.655 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:28:35.655 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:28:35.655 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:28:35.655 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:28:35.655 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:28:35.656 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:28:35.656 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:35.656 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:28:35.656 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:28:35.656 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:28:35.656 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:28:35.657 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:28:35.657 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:28:35.657 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:28:35.657 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:28:35.657 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:28:35.657 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:28:35.657 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:28:35.657 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:28:35.657 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:28:35.657 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:35.658 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.658 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:28:35.658 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:28:35.658 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:28:35.658 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:28:35.658 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:28:35.658 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:28:35.659 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:28:35.659 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:28:35.659 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:28:35.659 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:28:35.659 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:28:35.659 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:28:35.659 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:28:35.659 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:28:35.659 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:28:35.659 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:28:35.659 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:28:35.659 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:28:35.660 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:28:35.660 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:28:35.660 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:28:35.660 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:28:35.660 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:28:35.660 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:28:35.660 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:28:35.660 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:28:35.660 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:28:35.660 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:28:35.661 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:35.661 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:28:35.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.661 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:28:35.661 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.661 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:28:35.661 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:28:35.661 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:28:35.662 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:28:35.662 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.662 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:28:35.662 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:28:35.662 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:35.662 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.662 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.662 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:28:35.662 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:28:35.662 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:35.662 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.663 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:35.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.663 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:28:35.663 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:28:35.663 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:28:35.663 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:28:35.663 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:28:35.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.663 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.663 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:28:35.663 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:28:35.663 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:28:35.664 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:28:35.664 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.664 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:28:35.664 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:28:35.664 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:35.664 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.664 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.664 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:28:35.664 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:28:35.664 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:35.664 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.665 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:35.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.665 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:28:35.665 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:28:35.665 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:28:35.665 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:28:35.665 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:28:35.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.665 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.665 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:28:35.665 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:28:35.666 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:28:35.666 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:28:35.666 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.666 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:28:35.666 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:28:35.666 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:35.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.666 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.666 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:28:35.666 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:28:35.666 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:35.667 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.667 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:35.667 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.667 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:28:35.667 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:28:35.667 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:28:35.667 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:28:35.667 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:28:35.667 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.667 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:35.667 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.667 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:28:35.667 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:28:35.668 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.668 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:28:35.668 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:28:35.668 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:28:35.668 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.668 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:35.668 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:28:35.668 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.668 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:28:35.668 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:28:35.668 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:28:35.668 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.668 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:35.669 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:28:35.669 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.669 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:28:35.669 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:28:35.669 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:28:35.669 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.669 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:35.669 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.669 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:28:35.669 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:28:35.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.673+0000 7f08b1d90640 1 -- 172.21.15.112:0/562926293 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f089806be30 msgr2=0x7f089806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.673+0000 7f08b1d90640 1 --2- 172.21.15.112:0/562926293 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f089806be30 0x7f089806e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f08a4033000 tx=0x7f08a4009000 comp rx=0 tx=0).stop 2024-04-25T01:28:35.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.673+0000 7f08b1d90640 1 -- 172.21.15.112:0/562926293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08ac1592c0 msgr2=0x7f08ac1462c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.673+0000 7f08b1d90640 1 --2- 172.21.15.112:0/562926293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08ac1592c0 0x7f08ac1462c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f08a000e840 tx=0x7f08a000ed00 comp rx=0 tx=0).stop 2024-04-25T01:28:35.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.673+0000 7f08abfff640 1 -- 172.21.15.112:0/562926293 reap_dead start 2024-04-25T01:28:35.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.673+0000 7f08abfff640 1 -- 172.21.15.112:0/562926293 reap_dead start 2024-04-25T01:28:35.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.673+0000 7f08b1d90640 1 -- 172.21.15.112:0/562926293 shutdown_connections 2024-04-25T01:28:35.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.673+0000 7f08b1d90640 1 -- 172.21.15.112:0/562926293 >> 172.21.15.112:0/562926293 conn(0x7f08ac0cb170 msgr2=0x7f08ac1514b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:35.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.673+0000 7f08b1d90640 1 -- 172.21.15.112:0/562926293 shutdown_connections 2024-04-25T01:28:35.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.673+0000 7f08b1d90640 1 -- 172.21.15.112:0/562926293 wait complete. 2024-04-25T01:28:35.672 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:28:35.672 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:28:35.672 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:28:35.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.798+0000 7ff78fb90640 1 Processor -- start 2024-04-25T01:28:35.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.798+0000 7ff78fb90640 1 -- start start 2024-04-25T01:28:35.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.799+0000 7ff78fb90640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7881596a0 0x7ff788152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.799+0000 7ff78fb90640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff7880c72b0 0x7ff788160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.799+0000 7ff78fb90640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff788159b90 con 0x7ff7881592c0 2024-04-25T01:28:35.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.799+0000 7ff78fb90640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff7880c6e20 con 0x7ff7880c72b0 2024-04-25T01:28:35.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.799+0000 7ff78fb90640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff78806c970 con 0x7ff7881596a0 2024-04-25T01:28:35.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.799+0000 7ff78d92c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff7881592c0 0x7ff7880c9070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32818/0 (socket says 172.21.15.112:32818) 2024-04-25T01:28:35.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.799+0000 7ff78d92c640 1 -- 172.21.15.112:0/3236584557 learned_addr learned my addr 172.21.15.112:0/3236584557 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:28:35.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.799+0000 7ff78e12d640 1 --2- 172.21.15.112:0/3236584557 >> v2:172.21.15.187:3300/0 conn(0x7ff7880c72b0 0x7ff788160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.799+0000 7ff78d12b640 1 --2- 172.21.15.112:0/3236584557 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7881596a0 0x7ff788152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.799+0000 7ff78c92a640 1 -- 172.21.15.112:0/3236584557 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1898785653 0 0) 0x7ff788159b90 con 0x7ff7881592c0 2024-04-25T01:28:35.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.799+0000 7ff78c92a640 1 -- 172.21.15.112:0/3236584557 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff774003520 con 0x7ff7881592c0 2024-04-25T01:28:35.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.799+0000 7ff78e12d640 1 -- 172.21.15.112:0/3236584557 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7881596a0 msgr2=0x7ff788152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:28:35.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.799+0000 7ff78e12d640 1 --2- 172.21.15.112:0/3236584557 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7881596a0 0x7ff788152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.799+0000 7ff78e12d640 1 -- 172.21.15.112:0/3236584557 >> v1:172.21.15.112:6789/0 conn(0x7ff7881592c0 legacy=0x7ff7880c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.799+0000 7ff78e12d640 1 -- 172.21.15.112:0/3236584557 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78806cc50 con 0x7ff7880c72b0 2024-04-25T01:28:35.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.799+0000 7ff78d12b640 1 --2- 172.21.15.112:0/3236584557 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7881596a0 0x7ff788152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:28:35.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.800+0000 7ff78e12d640 1 --2- 172.21.15.112:0/3236584557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7880c72b0 0x7ff788160380 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff780009d90 tx=0x7ff78002a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=610886ae6963a412 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.800+0000 7ff78c92a640 1 -- 172.21.15.112:0/3236584557 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff780007740 con 0x7ff7880c72b0 2024-04-25T01:28:35.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.800+0000 7ff78c92a640 1 -- 172.21.15.112:0/3236584557 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7800078e0 con 0x7ff7880c72b0 2024-04-25T01:28:35.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.800+0000 7ff78c92a640 1 -- 172.21.15.112:0/3236584557 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff780007cb0 con 0x7ff7880c72b0 2024-04-25T01:28:35.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.801+0000 7ff78fb90640 1 -- 172.21.15.112:0/3236584557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7880c72b0 msgr2=0x7ff788160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.801+0000 7ff78fb90640 1 --2- 172.21.15.112:0/3236584557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7880c72b0 0x7ff788160380 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff780009d90 tx=0x7ff78002a530 comp rx=0 tx=0).stop 2024-04-25T01:28:35.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.801+0000 7ff78fb90640 1 -- 172.21.15.112:0/3236584557 shutdown_connections 2024-04-25T01:28:35.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.801+0000 7ff78fb90640 1 --2- 172.21.15.112:0/3236584557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7880c72b0 0x7ff788160380 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.801+0000 7ff78fb90640 1 --2- 172.21.15.112:0/3236584557 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7881596a0 0x7ff788152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.801+0000 7ff78fb90640 1 -- 172.21.15.112:0/3236584557 >> 172.21.15.112:0/3236584557 conn(0x7ff7880cb170 msgr2=0x7ff7880cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:35.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.801+0000 7ff78fb90640 1 -- 172.21.15.112:0/3236584557 shutdown_connections 2024-04-25T01:28:35.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.801+0000 7ff78fb90640 1 -- 172.21.15.112:0/3236584557 wait complete. 2024-04-25T01:28:35.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.802+0000 7ff78fb90640 1 Processor -- start 2024-04-25T01:28:35.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.802+0000 7ff78fb90640 1 -- start start 2024-04-25T01:28:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.803+0000 7ff78fb90640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7880c72b0 0x7ff7881ebef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.803+0000 7ff78fb90640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7881592c0 0x7ff78814bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.803+0000 7ff78fb90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7881596a0 0x7ff7880bfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.803+0000 7ff78fb90640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff78806c510 con 0x7ff7881596a0 2024-04-25T01:28:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.803+0000 7ff78fb90640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff78806c390 con 0x7ff7881592c0 2024-04-25T01:28:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.803+0000 7ff78fb90640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff78806db20 con 0x7ff7880c72b0 2024-04-25T01:28:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.803+0000 7ff78e12d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7881596a0 0x7ff7880bfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.803+0000 7ff78e12d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7881596a0 0x7ff7880bfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54218/0 (socket says 172.21.15.112:54218) 2024-04-25T01:28:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.803+0000 7ff78e12d640 1 -- 172.21.15.112:0/4157936406 learned_addr learned my addr 172.21.15.112:0/4157936406 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:28:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.803+0000 7ff78d92c640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7880c72b0 0x7ff7881ebef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.803+0000 7ff78d12b640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7881592c0 0x7ff78814bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.803+0000 7ff78e12d640 1 -- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7880c72b0 msgr2=0x7ff7881ebef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.803+0000 7ff78e12d640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7880c72b0 0x7ff7881ebef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.803+0000 7ff78e12d640 1 -- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7881592c0 msgr2=0x7ff78814bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.803+0000 7ff78e12d640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7881592c0 0x7ff78814bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.803+0000 7ff78e12d640 1 -- 172.21.15.112:0/4157936406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7881eb430 con 0x7ff7881596a0 2024-04-25T01:28:35.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.803+0000 7ff78d92c640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7880c72b0 0x7ff7881ebef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:28:35.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.803+0000 7ff78e12d640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7881596a0 0x7ff7880bfcf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff780009eb0 tx=0x7ff780004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.804+0000 7ff77e7fc640 1 -- 172.21.15.112:0/4157936406 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff780040070 con 0x7ff7881596a0 2024-04-25T01:28:35.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.804+0000 7ff77e7fc640 1 -- 172.21.15.112:0/4157936406 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff780031bc0 con 0x7ff7881596a0 2024-04-25T01:28:35.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.804+0000 7ff77e7fc640 1 -- 172.21.15.112:0/4157936406 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff780009450 con 0x7ff7881596a0 2024-04-25T01:28:35.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.804+0000 7ff78fb90640 1 -- 172.21.15.112:0/4157936406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7880c1650 con 0x7ff7881596a0 2024-04-25T01:28:35.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.804+0000 7ff78fb90640 1 -- 172.21.15.112:0/4157936406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7881eb6f0 con 0x7ff7881596a0 2024-04-25T01:28:35.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.805+0000 7ff78fb90640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff788150130 0x7ff78814ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.805+0000 7ff78fb90640 1 -- 172.21.15.112:0/4157936406 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff788075980 con 0x7ff788150130 2024-04-25T01:28:35.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.805+0000 7ff78d92c640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff788150130 0x7ff78814ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.805+0000 7ff78d92c640 1 -- 172.21.15.112:0/4157936406 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff7880679a0 con 0x7ff788150130 2024-04-25T01:28:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.805+0000 7ff77e7fc640 1 -- 172.21.15.112:0/4157936406 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff780039040 con 0x7ff7881596a0 2024-04-25T01:28:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.805+0000 7ff78d92c640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff788150130 0x7ff78814ef50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff784001200 tx=0x7ff784007700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.806+0000 7ff77e7fc640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff76c06be30 0x7ff76c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.806+0000 7ff77e7fc640 1 -- 172.21.15.112:0/4157936406 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7ff7800ac7a0 con 0x7ff7881596a0 2024-04-25T01:28:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.806+0000 7ff78d12b640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff76c06be30 0x7ff76c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.806+0000 7ff77e7fc640 1 -- 172.21.15.112:0/4157936406 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784005a60 con 0x7ff788150130 2024-04-25T01:28:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.806+0000 7ff77e7fc640 1 -- 172.21.15.112:0/4157936406 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff784018070 con 0x7ff788150130 2024-04-25T01:28:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.806+0000 7ff77e7fc640 1 -- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff788150130 msgr2=0x7ff78814ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.806+0000 7ff77e7fc640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff788150130 0x7ff78814ef50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff784001200 tx=0x7ff784007700 comp rx=0 tx=0).stop 2024-04-25T01:28:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.806+0000 7ff77e7fc640 1 -- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff788150130 msgr2=0x7ff78814ef50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:28:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.806+0000 7ff77e7fc640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff788150130 0x7ff78814ef50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.806+0000 7ff78d12b640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff76c06be30 0x7ff76c06e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff778009560 tx=0x7ff778009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.812+0000 7ff78fb90640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78814f490 0x7ff78814f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.812+0000 7ff78fb90640 1 -- 172.21.15.112:0/4157936406 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff7880682a0 con 0x7ff78814f490 2024-04-25T01:28:35.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.813+0000 7ff78d92c640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78814f490 0x7ff78814f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.813+0000 7ff78d92c640 1 -- 172.21.15.112:0/4157936406 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff7880c79d0 con 0x7ff78814f490 2024-04-25T01:28:35.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.813+0000 7ff78d92c640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78814f490 0x7ff78814f8a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff784001200 tx=0x7ff7840010a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.813+0000 7ff77e7fc640 1 -- 172.21.15.112:0/4157936406 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78400fc30 con 0x7ff78814f490 2024-04-25T01:28:35.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.814+0000 7ff77e7fc640 1 -- 172.21.15.112:0/4157936406 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4558 (secure 0 0 0) 0x7ff78400fc30 con 0x7ff78814f490 2024-04-25T01:28:35.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.814+0000 7ff77e7fc640 1 -- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78814f490 msgr2=0x7ff78814f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.814+0000 7ff77e7fc640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78814f490 0x7ff78814f8a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff784001200 tx=0x7ff7840010a0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.814+0000 7ff77e7fc640 1 -- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78814f490 msgr2=0x7ff78814f8a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:28:35.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.814+0000 7ff77e7fc640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff78814f490 0x7ff78814f8a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.794 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:28:35.794 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:28:35.794 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:28:35.794 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:28:35.794 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 20, 2024-04-25T01:28:35.794 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 15757, 2024-04-25T01:28:35.794 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:28:35.795 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:28:35.795 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:28:35.795 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:28:35.795 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:35.795 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:28:35.795 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:28:35.795 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:28:35.795 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:28:35.795 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:28:35.795 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:28:35.795 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:28:35.795 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:28:35.795 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:28:35.795 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:28:35.796 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:28:35.796 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:28:35.796 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:28:35.796 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:28:35.796 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:28:35.796 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:35.796 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:28:35.796 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:28:35.796 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:28:35.796 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:28:35.796 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:28:35.796 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:28:35.796 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:28:35.796 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:28:35.797 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:28:35.797 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:28:35.797 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:28:35.797 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:28:35.797 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:28:35.797 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:35.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.797 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:28:35.797 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:28:35.797 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:28:35.797 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:28:35.797 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:28:35.797 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:28:35.797 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:28:35.798 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:28:35.798 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:28:35.798 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:28:35.798 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:28:35.798 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:28:35.798 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:28:35.798 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:28:35.798 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:28:35.798 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:28:35.798 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:28:35.798 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:28:35.798 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:28:35.798 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:28:35.799 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:28:35.799 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:28:35.799 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:28:35.799 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:28:35.799 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:28:35.799 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:28:35.799 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:28:35.799 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:28:35.799 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:35.799 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:28:35.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.799 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:28:35.799 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.799 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:28:35.800 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:28:35.800 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:28:35.800 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:28:35.800 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.800 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:28:35.800 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:28:35.800 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:35.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.800 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.800 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:28:35.800 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:28:35.800 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:35.800 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.801 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:35.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.801 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:28:35.801 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:28:35.801 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:28:35.801 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:28:35.801 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:28:35.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.801 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.801 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:28:35.801 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:28:35.801 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:28:35.801 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:28:35.801 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.801 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:28:35.802 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:28:35.802 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:35.802 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.802 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.802 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:28:35.802 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:28:35.802 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:35.802 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.802 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:35.802 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.802 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:28:35.802 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:28:35.802 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:28:35.803 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:28:35.803 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:28:35.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.803 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.803 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:28:35.803 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:28:35.803 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:28:35.803 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:28:35.803 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.803 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:28:35.803 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:28:35.803 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:35.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.803 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.804 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:28:35.804 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:28:35.804 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:35.804 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.804 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:35.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.804 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:28:35.804 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:28:35.804 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:28:35.804 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:28:35.804 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:28:35.804 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.804 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:35.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.805 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:28:35.805 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:28:35.805 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.805 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:28:35.805 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:28:35.805 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:28:35.805 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.805 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:35.805 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:28:35.805 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:35.805 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:28:35.805 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:28:35.805 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:28:35.805 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:35.805 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:35.806 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:35.806 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:28:35.806 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:28:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.814+0000 7ff78fb90640 1 -- 172.21.15.112:0/4157936406 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff76c06be30 msgr2=0x7ff76c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.814+0000 7ff78fb90640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff76c06be30 0x7ff76c06e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff778009560 tx=0x7ff778009190 comp rx=0 tx=0).stop 2024-04-25T01:28:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.814+0000 7ff78fb90640 1 -- 172.21.15.112:0/4157936406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7881596a0 msgr2=0x7ff7880bfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.814+0000 7ff78fb90640 1 --2- 172.21.15.112:0/4157936406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7881596a0 0x7ff7880bfcf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff780009eb0 tx=0x7ff780004a40 comp rx=0 tx=0).stop 2024-04-25T01:28:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.814+0000 7ff78e12d640 1 -- 172.21.15.112:0/4157936406 reap_dead start 2024-04-25T01:28:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.814+0000 7ff78e12d640 1 -- 172.21.15.112:0/4157936406 reap_dead start 2024-04-25T01:28:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.814+0000 7ff78fb90640 1 -- 172.21.15.112:0/4157936406 shutdown_connections 2024-04-25T01:28:35.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.814+0000 7ff78fb90640 1 -- 172.21.15.112:0/4157936406 >> 172.21.15.112:0/4157936406 conn(0x7ff7880cb170 msgr2=0x7ff788067110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:35.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.814+0000 7ff78fb90640 1 -- 172.21.15.112:0/4157936406 shutdown_connections 2024-04-25T01:28:35.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.814+0000 7ff78fb90640 1 -- 172.21.15.112:0/4157936406 wait complete. 2024-04-25T01:28:35.808 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:28:35.808 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T01:28:35.808 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T01:28:35.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.934+0000 7fc35cd14640 1 Processor -- start 2024-04-25T01:28:35.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.934+0000 7fc35cd14640 1 -- start start 2024-04-25T01:28:35.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.935+0000 7fc35cd14640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc35806c170 0x7fc35806c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.935+0000 7fc35cd14640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc3580c1c00 0x7fc358072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.935+0000 7fc35cd14640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3580c2920 con 0x7fc3580c2160 2024-04-25T01:28:35.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.935+0000 7fc35cd14640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fc3580c74b0 con 0x7fc3580c1c00 2024-04-25T01:28:35.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.935+0000 7fc35cd14640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc358152890 con 0x7fc35806c170 2024-04-25T01:28:35.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.935+0000 7fc34ffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc3580c1c00 0x7fc358072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.935+0000 7fc34ffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc3580c1c00 0x7fc358072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54234/0 (socket says 172.21.15.112:54234) 2024-04-25T01:28:35.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.935+0000 7fc34ffff640 1 -- 172.21.15.112:0/2324207676 learned_addr learned my addr 172.21.15.112:0/2324207676 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:28:35.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.935+0000 7fc35659c640 1 --2- 172.21.15.112:0/2324207676 >> v2:172.21.15.187:3300/0 conn(0x7fc35806c170 0x7fc35806c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.935+0000 7fc34ffff640 1 -- 172.21.15.112:0/2324207676 >> v2:172.21.15.187:3300/0 conn(0x7fc35806c170 msgr2=0x7fc35806c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:28:35.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.935+0000 7fc34ffff640 1 --2- 172.21.15.112:0/2324207676 >> v2:172.21.15.187:3300/0 conn(0x7fc35806c170 0x7fc35806c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.935+0000 7fc34ffff640 1 -- 172.21.15.112:0/2324207676 >> v1:172.21.15.187:6789/0 conn(0x7fc3580c2160 legacy=0x7fc358072570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.935+0000 7fc34ffff640 1 -- 172.21.15.112:0/2324207676 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc358152b70 con 0x7fc3580c1c00 2024-04-25T01:28:35.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.936+0000 7fc34ffff640 1 --2- 172.21.15.112:0/2324207676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3580c1c00 0x7fc358072030 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc3580c5080 tx=0x7fc34002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbac78f621c43a84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.936+0000 7fc355d9b640 1 -- 172.21.15.112:0/2324207676 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340037070 con 0x7fc3580c1c00 2024-04-25T01:28:35.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.936+0000 7fc355d9b640 1 -- 172.21.15.112:0/2324207676 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc340032440 con 0x7fc3580c1c00 2024-04-25T01:28:35.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.936+0000 7fc355d9b640 1 -- 172.21.15.112:0/2324207676 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340032810 con 0x7fc3580c1c00 2024-04-25T01:28:35.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.937+0000 7fc35cd14640 1 -- 172.21.15.112:0/2324207676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3580c1c00 msgr2=0x7fc358072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.937+0000 7fc35cd14640 1 --2- 172.21.15.112:0/2324207676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3580c1c00 0x7fc358072030 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc3580c5080 tx=0x7fc34002a270 comp rx=0 tx=0).stop 2024-04-25T01:28:35.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.937+0000 7fc35cd14640 1 -- 172.21.15.112:0/2324207676 shutdown_connections 2024-04-25T01:28:35.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.937+0000 7fc35cd14640 1 --2- 172.21.15.112:0/2324207676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3580c1c00 0x7fc358072030 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.937+0000 7fc35cd14640 1 --2- 172.21.15.112:0/2324207676 >> v2:172.21.15.187:3300/0 conn(0x7fc35806c170 0x7fc35806c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.937+0000 7fc35cd14640 1 -- 172.21.15.112:0/2324207676 >> 172.21.15.112:0/2324207676 conn(0x7fc3580c4260 msgr2=0x7fc3580c4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:35.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.937+0000 7fc35cd14640 1 -- 172.21.15.112:0/2324207676 shutdown_connections 2024-04-25T01:28:35.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.937+0000 7fc35cd14640 1 -- 172.21.15.112:0/2324207676 wait complete. 2024-04-25T01:28:35.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.938+0000 7fc35cd14640 1 Processor -- start 2024-04-25T01:28:35.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.938+0000 7fc35cd14640 1 -- start start 2024-04-25T01:28:35.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.939+0000 7fc35cd14640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc35806c170 0x7fc3580acbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.939+0000 7fc35cd14640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3580c1c00 0x7fc3580aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.939+0000 7fc35cd14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3580c2160 0x7fc3580ab330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.939+0000 7fc35cd14640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3580c2620 con 0x7fc3580c2160 2024-04-25T01:28:35.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.939+0000 7fc35cd14640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc358153130 con 0x7fc35806c170 2024-04-25T01:28:35.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.939+0000 7fc35cd14640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc358153650 con 0x7fc3580c1c00 2024-04-25T01:28:35.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.939+0000 7fc356d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3580c2160 0x7fc3580ab330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.939+0000 7fc356d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3580c2160 0x7fc3580ab330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54238/0 (socket says 172.21.15.112:54238) 2024-04-25T01:28:35.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.939+0000 7fc356d9d640 1 -- 172.21.15.112:0/3351128596 learned_addr learned my addr 172.21.15.112:0/3351128596 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:28:35.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.939+0000 7fc35659c640 1 --2- 172.21.15.112:0/3351128596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc35806c170 0x7fc3580acbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.939+0000 7fc34ffff640 1 --2- 172.21.15.112:0/3351128596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3580c1c00 0x7fc3580aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.939+0000 7fc356d9d640 1 -- 172.21.15.112:0/3351128596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3580c1c00 msgr2=0x7fc3580aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.939+0000 7fc356d9d640 1 --2- 172.21.15.112:0/3351128596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3580c1c00 0x7fc3580aadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.939+0000 7fc356d9d640 1 -- 172.21.15.112:0/3351128596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc35806c170 msgr2=0x7fc3580acbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:35.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.939+0000 7fc356d9d640 1 --2- 172.21.15.112:0/3351128596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc35806c170 0x7fc3580acbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:35.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.939+0000 7fc356d9d640 1 -- 172.21.15.112:0/3351128596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3580bc7f0 con 0x7fc3580c2160 2024-04-25T01:28:35.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.939+0000 7fc356d9d640 1 --2- 172.21.15.112:0/3351128596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3580c2160 0x7fc3580ab330 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc35000c680 tx=0x7fc35000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.940+0000 7fc34effd640 1 -- 172.21.15.112:0/3351128596 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc350003b70 con 0x7fc3580c2160 2024-04-25T01:28:35.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.940+0000 7fc35cd14640 1 -- 172.21.15.112:0/3351128596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3580aba80 con 0x7fc3580c2160 2024-04-25T01:28:35.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.940+0000 7fc34effd640 1 -- 172.21.15.112:0/3351128596 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc350003d10 con 0x7fc3580c2160 2024-04-25T01:28:35.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.940+0000 7fc34effd640 1 -- 172.21.15.112:0/3351128596 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc350005310 con 0x7fc3580c2160 2024-04-25T01:28:35.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.940+0000 7fc35cd14640 1 -- 172.21.15.112:0/3351128596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3581e93c0 con 0x7fc3580c2160 2024-04-25T01:28:35.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.941+0000 7fc35cd14640 1 -- 172.21.15.112:0/3351128596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3580689b0 con 0x7fc3580c2160 2024-04-25T01:28:35.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.941+0000 7fc34effd640 1 -- 172.21.15.112:0/3351128596 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc3500054b0 con 0x7fc3580c2160 2024-04-25T01:28:35.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.941+0000 7fc34effd640 1 --2- 172.21.15.112:0/3351128596 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc338070240 0x7fc338072700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:35.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.942+0000 7fc34effd640 1 -- 172.21.15.112:0/3351128596 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fc35008af00 con 0x7fc3580c2160 2024-04-25T01:28:35.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.942+0000 7fc35659c640 1 --2- 172.21.15.112:0/3351128596 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc338070240 0x7fc338072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:35.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.942+0000 7fc35659c640 1 --2- 172.21.15.112:0/3351128596 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc338070240 0x7fc338072700 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc358155e40 tx=0x7fc348009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:35.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:35.947+0000 7fc34effd640 1 -- 172.21.15.112:0/3351128596 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc350058ed0 con 0x7fc3580c2160 2024-04-25T01:28:36.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:36.257+0000 7fc35cd14640 1 -- 172.21.15.112:0/3351128596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fc3580c39e0 con 0x7fc3580c2160 2024-04-25T01:28:36.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:36.258+0000 7fc34effd640 1 -- 172.21.15.112:0/3351128596 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fc35005d1d0 con 0x7fc3580c2160 2024-04-25T01:28:36.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:36.261+0000 7fc35cd14640 1 -- 172.21.15.112:0/3351128596 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc338070240 msgr2=0x7fc338072700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:36.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:36.261+0000 7fc35cd14640 1 --2- 172.21.15.112:0/3351128596 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc338070240 0x7fc338072700 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc358155e40 tx=0x7fc348009190 comp rx=0 tx=0).stop 2024-04-25T01:28:36.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:36.261+0000 7fc35cd14640 1 -- 172.21.15.112:0/3351128596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3580c2160 msgr2=0x7fc3580ab330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:36.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:36.261+0000 7fc35cd14640 1 --2- 172.21.15.112:0/3351128596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3580c2160 0x7fc3580ab330 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc35000c680 tx=0x7fc35000cb40 comp rx=0 tx=0).stop 2024-04-25T01:28:36.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:36.261+0000 7fc35cd14640 1 -- 172.21.15.112:0/3351128596 shutdown_connections 2024-04-25T01:28:36.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:36.261+0000 7fc35cd14640 1 --2- 172.21.15.112:0/3351128596 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc338070240 0x7fc338072700 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:36.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:36.261+0000 7fc35cd14640 1 --2- 172.21.15.112:0/3351128596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3580c2160 0x7fc3580ab330 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:36.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:36.261+0000 7fc35cd14640 1 --2- 172.21.15.112:0/3351128596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3580c1c00 0x7fc3580aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:36.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:36.261+0000 7fc35cd14640 1 --2- 172.21.15.112:0/3351128596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc35806c170 0x7fc3580acbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:36.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:36.261+0000 7fc35cd14640 1 -- 172.21.15.112:0/3351128596 >> 172.21.15.112:0/3351128596 conn(0x7fc3580c4260 msgr2=0x7fc3581dbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:36.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:36.261+0000 7fc35cd14640 1 -- 172.21.15.112:0/3351128596 shutdown_connections 2024-04-25T01:28:36.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:36.261+0000 7fc35cd14640 1 -- 172.21.15.112:0/3351128596 wait complete. 2024-04-25T01:28:36.251 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T01:28:46.252 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:28:46.252 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:28:46.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.401+0000 7f0bdc7c1640 1 Processor -- start 2024-04-25T01:28:46.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.401+0000 7f0bdc7c1640 1 -- start start 2024-04-25T01:28:46.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.402+0000 7f0bdc7c1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bd4152f50 0x7f0bd40c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.402+0000 7f0bdc7c1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0bd40caa70 0x7f0bd40c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.402+0000 7f0bdc7c1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0bd40c80a0 0x7f0bd40c9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.402+0000 7f0bdc7c1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f0bd4153330 con 0x7f0bd40c80a0 2024-04-25T01:28:46.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.402+0000 7f0bdc7c1640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0bd4159dc0 con 0x7f0bd40caa70 2024-04-25T01:28:46.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.402+0000 7f0bdc7c1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0bd4159000 con 0x7f0bd4152f50 2024-04-25T01:28:46.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.402+0000 7f0bdad5e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0bd40c80a0 0x7f0bd40c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.402+0000 7f0bdad5e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0bd40c80a0 0x7f0bd40c9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54612/0 (socket says 172.21.15.112:54612) 2024-04-25T01:28:46.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.402+0000 7f0bdad5e640 1 -- 172.21.15.112:0/1666884252 learned_addr learned my addr 172.21.15.112:0/1666884252 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:28:46.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.402+0000 7f0bda55d640 1 --2- 172.21.15.112:0/1666884252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bd4152f50 0x7f0bd40c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.402+0000 7f0bd9d5c640 1 --2- 172.21.15.112:0/1666884252 >> v2:172.21.15.187:3300/0 conn(0x7f0bd40caa70 0x7f0bd40c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.402+0000 7f0bdad5e640 1 -- 172.21.15.112:0/1666884252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bd4152f50 msgr2=0x7f0bd40c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:28:46.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.402+0000 7f0bdad5e640 1 --2- 172.21.15.112:0/1666884252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bd4152f50 0x7f0bd40c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.402+0000 7f0bdad5e640 1 -- 172.21.15.112:0/1666884252 >> v2:172.21.15.187:3300/0 conn(0x7f0bd40caa70 msgr2=0x7f0bd40c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:28:46.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.402+0000 7f0bdad5e640 1 --2- 172.21.15.112:0/1666884252 >> v2:172.21.15.187:3300/0 conn(0x7f0bd40caa70 0x7f0bd40c7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.402+0000 7f0bdad5e640 1 -- 172.21.15.112:0/1666884252 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd41592e0 con 0x7f0bd40c80a0 2024-04-25T01:28:46.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.403+0000 7f0bdad5e640 1 --2- 172.21.15.112:0/1666884252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd40c80a0 0x7f0bd40c9050 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40c86f0 tx=0x7f0bcc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55aba7b41a70f481 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:46.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.403+0000 7f0bd955b640 1 -- 172.21.15.112:0/1666884252 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc036070 con 0x7f0bd40c80a0 2024-04-25T01:28:46.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.403+0000 7f0bd955b640 1 -- 172.21.15.112:0/1666884252 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bcc031600 con 0x7f0bd40c80a0 2024-04-25T01:28:46.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.403+0000 7f0bd955b640 1 -- 172.21.15.112:0/1666884252 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc0319d0 con 0x7f0bd40c80a0 2024-04-25T01:28:46.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.404+0000 7f0bdc7c1640 1 -- 172.21.15.112:0/1666884252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd40c80a0 msgr2=0x7f0bd40c9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:46.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.404+0000 7f0bdc7c1640 1 --2- 172.21.15.112:0/1666884252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd40c80a0 0x7f0bd40c9050 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40c86f0 tx=0x7f0bcc02a610 comp rx=0 tx=0).stop 2024-04-25T01:28:46.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.404+0000 7f0bdc7c1640 1 -- 172.21.15.112:0/1666884252 shutdown_connections 2024-04-25T01:28:46.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.404+0000 7f0bdc7c1640 1 --2- 172.21.15.112:0/1666884252 >> v2:172.21.15.187:3300/0 conn(0x7f0bd40caa70 0x7f0bd40c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.404+0000 7f0bdc7c1640 1 --2- 172.21.15.112:0/1666884252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd40c80a0 0x7f0bd40c9050 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.404+0000 7f0bdc7c1640 1 --2- 172.21.15.112:0/1666884252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bd4152f50 0x7f0bd40c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.404+0000 7f0bdc7c1640 1 -- 172.21.15.112:0/1666884252 >> 172.21.15.112:0/1666884252 conn(0x7f0bd40ace40 msgr2=0x7f0bd40ad250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:46.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.404+0000 7f0bdc7c1640 1 -- 172.21.15.112:0/1666884252 shutdown_connections 2024-04-25T01:28:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.404+0000 7f0bdc7c1640 1 -- 172.21.15.112:0/1666884252 wait complete. 2024-04-25T01:28:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.405+0000 7f0bdc7c1640 1 Processor -- start 2024-04-25T01:28:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.405+0000 7f0bdc7c1640 1 -- start start 2024-04-25T01:28:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.406+0000 7f0bdc7c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd40c80a0 0x7f0bd40c2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.406+0000 7f0bdc7c1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd40caa70 0x7f0bd41ef6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.406+0000 7f0bdc7c1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bd4152f50 0x7f0bd414b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.406+0000 7f0bdc7c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd40caed0 con 0x7f0bd40c80a0 2024-04-25T01:28:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.406+0000 7f0bdc7c1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41598a0 con 0x7f0bd40caa70 2024-04-25T01:28:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.406+0000 7f0bdc7c1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0bd40c9590 con 0x7f0bd4152f50 2024-04-25T01:28:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.406+0000 7f0bda55d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd40c80a0 0x7f0bd40c2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.406+0000 7f0bda55d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd40c80a0 0x7f0bd40c2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54620/0 (socket says 172.21.15.112:54620) 2024-04-25T01:28:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.406+0000 7f0bda55d640 1 -- 172.21.15.112:0/10844169 learned_addr learned my addr 172.21.15.112:0/10844169 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:28:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.406+0000 7f0bd9d5c640 1 --2- 172.21.15.112:0/10844169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd40caa70 0x7f0bd41ef6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.406+0000 7f0bdad5e640 1 --2- 172.21.15.112:0/10844169 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bd4152f50 0x7f0bd414b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.406+0000 7f0bda55d640 1 -- 172.21.15.112:0/10844169 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bd4152f50 msgr2=0x7f0bd414b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:46.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.406+0000 7f0bda55d640 1 --2- 172.21.15.112:0/10844169 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bd4152f50 0x7f0bd414b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.406+0000 7f0bda55d640 1 -- 172.21.15.112:0/10844169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd40caa70 msgr2=0x7f0bd41ef6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:46.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.406+0000 7f0bda55d640 1 --2- 172.21.15.112:0/10844169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd40caa70 0x7f0bd41ef6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.406+0000 7f0bda55d640 1 -- 172.21.15.112:0/10844169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd41efc30 con 0x7f0bd40c80a0 2024-04-25T01:28:46.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.406+0000 7f0bda55d640 1 --2- 172.21.15.112:0/10844169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd40c80a0 0x7f0bd40c2430 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000e8a0 tx=0x7f0bd000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:46.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.407+0000 7f0bcaffd640 1 -- 172.21.15.112:0/10844169 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0017070 con 0x7f0bd40c80a0 2024-04-25T01:28:46.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.407+0000 7f0bcaffd640 1 -- 172.21.15.112:0/10844169 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd000c940 con 0x7f0bd40c80a0 2024-04-25T01:28:46.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.407+0000 7f0bcaffd640 1 -- 172.21.15.112:0/10844169 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd000cd30 con 0x7f0bd40c80a0 2024-04-25T01:28:46.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.407+0000 7f0bdc7c1640 1 -- 172.21.15.112:0/10844169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd40abcb0 con 0x7f0bd40c80a0 2024-04-25T01:28:46.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.407+0000 7f0bdc7c1640 1 -- 172.21.15.112:0/10844169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd40ac270 con 0x7f0bd40c80a0 2024-04-25T01:28:46.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.408+0000 7f0bdc7c1640 1 -- 172.21.15.112:0/10844169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bd41eb7e0 con 0x7f0bd40c80a0 2024-04-25T01:28:46.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.408+0000 7f0bcaffd640 1 -- 172.21.15.112:0/10844169 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0bd001a070 con 0x7f0bd40c80a0 2024-04-25T01:28:46.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.409+0000 7f0bcaffd640 1 --2- 172.21.15.112:0/10844169 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0bb806be30 0x7f0bb806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.409+0000 7f0bd9d5c640 1 --2- 172.21.15.112:0/10844169 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0bb806be30 0x7f0bb806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.409+0000 7f0bcaffd640 1 -- 172.21.15.112:0/10844169 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f0bd008ae90 con 0x7f0bd40c80a0 2024-04-25T01:28:46.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.409+0000 7f0bd9d5c640 1 --2- 172.21.15.112:0/10844169 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0bb806be30 0x7f0bb806e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40ab7f0 tx=0x7f0bc4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:46.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.414+0000 7f0bcaffd640 1 -- 172.21.15.112:0/10844169 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0bd0058ee0 con 0x7f0bd40c80a0 2024-04-25T01:28:46.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.730+0000 7f0bdc7c1640 1 -- 172.21.15.112:0/10844169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0bd40682a0 con 0x7f0bd40c80a0 2024-04-25T01:28:46.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.731+0000 7f0bcaffd640 1 -- 172.21.15.112:0/10844169 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f0bd005d1e0 con 0x7f0bd40c80a0 2024-04-25T01:28:46.710 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:28:46.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.733+0000 7f0bdc7c1640 1 -- 172.21.15.112:0/10844169 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0bb806be30 msgr2=0x7f0bb806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:46.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.733+0000 7f0bdc7c1640 1 --2- 172.21.15.112:0/10844169 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0bb806be30 0x7f0bb806e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40ab7f0 tx=0x7f0bc4009190 comp rx=0 tx=0).stop 2024-04-25T01:28:46.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.733+0000 7f0bdc7c1640 1 -- 172.21.15.112:0/10844169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd40c80a0 msgr2=0x7f0bd40c2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:46.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.733+0000 7f0bdc7c1640 1 --2- 172.21.15.112:0/10844169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd40c80a0 0x7f0bd40c2430 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000e8a0 tx=0x7f0bd000ed60 comp rx=0 tx=0).stop 2024-04-25T01:28:46.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.733+0000 7f0bdc7c1640 1 -- 172.21.15.112:0/10844169 shutdown_connections 2024-04-25T01:28:46.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.733+0000 7f0bdc7c1640 1 --2- 172.21.15.112:0/10844169 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0bb806be30 0x7f0bb806e2f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.733+0000 7f0bdc7c1640 1 --2- 172.21.15.112:0/10844169 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bd4152f50 0x7f0bd414b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.733+0000 7f0bdc7c1640 1 --2- 172.21.15.112:0/10844169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd40caa70 0x7f0bd41ef6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.733+0000 7f0bdc7c1640 1 --2- 172.21.15.112:0/10844169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd40c80a0 0x7f0bd40c2430 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.733+0000 7f0bdc7c1640 1 -- 172.21.15.112:0/10844169 >> 172.21.15.112:0/10844169 conn(0x7f0bd40ace40 msgr2=0x7f0bd414e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:46.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.733+0000 7f0bdc7c1640 1 -- 172.21.15.112:0/10844169 shutdown_connections 2024-04-25T01:28:46.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.733+0000 7f0bdc7c1640 1 -- 172.21.15.112:0/10844169 wait complete. 2024-04-25T01:28:46.724 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:28:46.724 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T01:28:46.724 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:28:46.724 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:28:46.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.856+0000 7f1aa599e640 1 Processor -- start 2024-04-25T01:28:46.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.856+0000 7f1aa599e640 1 -- start start 2024-04-25T01:28:46.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.856+0000 7f1aa599e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1aa01594c0 0x7f1aa0161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.856+0000 7f1aa599e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1aa00c9510 con 0x7f1aa00c9080 2024-04-25T01:28:46.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.856+0000 7f1aa599e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1aa00c9780 con 0x7f1aa0152cd0 2024-04-25T01:28:46.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.856+0000 7f1aa599e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1aa00c9bc0 con 0x7f1aa01594c0 2024-04-25T01:28:46.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.856+0000 7f1a9ffff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1aa00c9080 0x7f1aa0158db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60608/0 (socket says 172.21.15.112:60608) 2024-04-25T01:28:46.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.856+0000 7f1a9ffff640 1 -- 172.21.15.112:0/4142100374 learned_addr learned my addr 172.21.15.112:0/4142100374 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:28:46.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.857+0000 7f1aa519d640 1 --2- 172.21.15.112:0/4142100374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1aa01594c0 0x7f1aa0161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.857+0000 7f1a9f7fe640 1 -- 172.21.15.112:0/4142100374 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2046469381 0 0) 0x7f1aa00c9510 con 0x7f1aa00c9080 2024-04-25T01:28:46.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.857+0000 7f1a9f7fe640 1 -- 172.21.15.112:0/4142100374 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a90003520 con 0x7f1aa00c9080 2024-04-25T01:28:46.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.857+0000 7f1a9f7fe640 1 -- 172.21.15.112:0/4142100374 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4239229748 0 0) 0x7f1aa00c9780 con 0x7f1aa0152cd0 2024-04-25T01:28:46.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.857+0000 7f1a9f7fe640 1 -- 172.21.15.112:0/4142100374 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1aa00c9510 con 0x7f1aa0152cd0 2024-04-25T01:28:46.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.857+0000 7f1aa519d640 1 -- 172.21.15.112:0/4142100374 >> v1:172.21.15.187:6789/0 conn(0x7f1aa0152cd0 legacy=0x7f1aa00c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:46.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.857+0000 7f1aa519d640 1 -- 172.21.15.112:0/4142100374 >> v1:172.21.15.112:6789/0 conn(0x7f1aa00c9080 legacy=0x7f1aa0158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:46.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.857+0000 7f1aa519d640 1 -- 172.21.15.112:0/4142100374 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa00c9e10 con 0x7f1aa01594c0 2024-04-25T01:28:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.858+0000 7f1aa519d640 1 --2- 172.21.15.112:0/4142100374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1aa01594c0 0x7f1aa0161150 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00c9d40 tx=0x7f1a9802a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba04b2aaaccbf26e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.858+0000 7f1a9f7fe640 1 -- 172.21.15.112:0/4142100374 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a98007740 con 0x7f1aa01594c0 2024-04-25T01:28:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.858+0000 7f1a9f7fe640 1 -- 172.21.15.112:0/4142100374 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a980078e0 con 0x7f1aa01594c0 2024-04-25T01:28:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.858+0000 7f1a9f7fe640 1 -- 172.21.15.112:0/4142100374 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a98007cb0 con 0x7f1aa01594c0 2024-04-25T01:28:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.859+0000 7f1aa599e640 1 -- 172.21.15.112:0/4142100374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1aa01594c0 msgr2=0x7f1aa0161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.859+0000 7f1aa599e640 1 --2- 172.21.15.112:0/4142100374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1aa01594c0 0x7f1aa0161150 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00c9d40 tx=0x7f1a9802a5b0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.859+0000 7f1aa599e640 1 -- 172.21.15.112:0/4142100374 shutdown_connections 2024-04-25T01:28:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.859+0000 7f1aa599e640 1 --2- 172.21.15.112:0/4142100374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1aa01594c0 0x7f1aa0161150 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.859+0000 7f1aa599e640 1 -- 172.21.15.112:0/4142100374 >> 172.21.15.112:0/4142100374 conn(0x7f1aa0074de0 msgr2=0x7f1aa00caf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.859+0000 7f1aa599e640 1 -- 172.21.15.112:0/4142100374 shutdown_connections 2024-04-25T01:28:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.859+0000 7f1aa599e640 1 -- 172.21.15.112:0/4142100374 wait complete. 2024-04-25T01:28:46.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.860+0000 7f1aa599e640 1 Processor -- start 2024-04-25T01:28:46.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.860+0000 7f1aa599e640 1 -- start start 2024-04-25T01:28:46.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.861+0000 7f1aa599e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1aa00c9080 0x7f1aa0154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.861+0000 7f1aa599e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1aa0152cd0 0x7f1aa01e80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.861+0000 7f1aa599e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa01eae60 0x7f1aa01e8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.861+0000 7f1aa599e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa006db20 con 0x7f1aa01eae60 2024-04-25T01:28:46.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.861+0000 7f1aa599e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa00ca3d0 con 0x7f1aa0152cd0 2024-04-25T01:28:46.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.861+0000 7f1aa599e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1aa00c8080 con 0x7f1aa00c9080 2024-04-25T01:28:46.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.861+0000 7f1aa519d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa01eae60 0x7f1aa01e8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.861+0000 7f1a9ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1aa0152cd0 0x7f1aa01e80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.861+0000 7f1aa519d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa01eae60 0x7f1aa01e8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54632/0 (socket says 172.21.15.112:54632) 2024-04-25T01:28:46.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.861+0000 7f1aa499c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1aa00c9080 0x7f1aa0154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.861+0000 7f1aa519d640 1 -- 172.21.15.112:0/187683924 learned_addr learned my addr 172.21.15.112:0/187683924 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:28:46.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.861+0000 7f1aa519d640 1 -- 172.21.15.112:0/187683924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1aa00c9080 msgr2=0x7f1aa0154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:46.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.861+0000 7f1aa519d640 1 --2- 172.21.15.112:0/187683924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1aa00c9080 0x7f1aa0154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.861+0000 7f1aa519d640 1 -- 172.21.15.112:0/187683924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1aa0152cd0 msgr2=0x7f1aa01e80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:46.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.861+0000 7f1aa519d640 1 --2- 172.21.15.112:0/187683924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1aa0152cd0 0x7f1aa01e80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.861+0000 7f1aa519d640 1 -- 172.21.15.112:0/187683924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa00ad430 con 0x7f1aa01eae60 2024-04-25T01:28:46.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.861+0000 7f1aa499c640 1 --2- 172.21.15.112:0/187683924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1aa00c9080 0x7f1aa0154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:28:46.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.862+0000 7f1aa519d640 1 --2- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa01eae60 0x7f1aa01e8620 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1a9802aba0 tx=0x7f1a98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:46.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.862+0000 7f1a9d7fa640 1 -- 172.21.15.112:0/187683924 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9803f070 con 0x7f1aa01eae60 2024-04-25T01:28:46.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.862+0000 7f1a9d7fa640 1 -- 172.21.15.112:0/187683924 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a98031bc0 con 0x7f1aa01eae60 2024-04-25T01:28:46.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.862+0000 7f1a9d7fa640 1 -- 172.21.15.112:0/187683924 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a98009450 con 0x7f1aa01eae60 2024-04-25T01:28:46.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.862+0000 7f1aa599e640 1 -- 172.21.15.112:0/187683924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa01eb360 con 0x7f1aa01eae60 2024-04-25T01:28:46.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.862+0000 7f1aa599e640 1 -- 172.21.15.112:0/187683924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa01eb920 con 0x7f1aa01eae60 2024-04-25T01:28:46.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.863+0000 7f1aa599e640 1 --2- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa01594c0 0x7f1aa0158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.863+0000 7f1aa599e640 1 -- 172.21.15.112:0/187683924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa00c6250 con 0x7f1aa01594c0 2024-04-25T01:28:46.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.863+0000 7f1aa499c640 1 --2- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa01594c0 0x7f1aa0158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.863+0000 7f1aa499c640 1 -- 172.21.15.112:0/187683924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1aa004f500 con 0x7f1aa01594c0 2024-04-25T01:28:46.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.863+0000 7f1aa499c640 1 --2- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa01594c0 0x7f1aa0158030 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1a940011e0 tx=0x7f1a9400ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:46.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.863+0000 7f1a9d7fa640 1 -- 172.21.15.112:0/187683924 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a940052e0 con 0x7f1aa01594c0 2024-04-25T01:28:46.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.864+0000 7f1a9d7fa640 1 -- 172.21.15.112:0/187683924 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a9803b410 con 0x7f1aa01eae60 2024-04-25T01:28:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.864+0000 7f1a9d7fa640 1 --2- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1a8406bd30 0x7f1a8406e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.864+0000 7f1a9d7fa640 1 -- 172.21.15.112:0/187683924 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f1a980ac6a0 con 0x7f1aa01eae60 2024-04-25T01:28:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.864+0000 7f1a9ffff640 1 --2- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1a8406bd30 0x7f1a8406e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.864+0000 7f1a9d7fa640 1 -- 172.21.15.112:0/187683924 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1a94018070 con 0x7f1aa01594c0 2024-04-25T01:28:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.864+0000 7f1a9d7fa640 1 -- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa01594c0 msgr2=0x7f1aa0158030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.864+0000 7f1a9d7fa640 1 --2- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa01594c0 0x7f1aa0158030 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1a940011e0 tx=0x7f1a9400ff70 comp rx=0 tx=0).stop 2024-04-25T01:28:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.864+0000 7f1a9d7fa640 1 -- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa01594c0 msgr2=0x7f1aa0158030 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:28:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.864+0000 7f1a9d7fa640 1 --2- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa01594c0 0x7f1aa0158030 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.865+0000 7f1a9ffff640 1 --2- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1a8406bd30 0x7f1a8406e1f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1aa01551b0 tx=0x7f1a8c007d70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.873+0000 7f1aa599e640 1 --2- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa00ad6a0 0x7f1aa0158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.873+0000 7f1aa599e640 1 -- 172.21.15.112:0/187683924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa0151000 con 0x7f1aa00ad6a0 2024-04-25T01:28:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.873+0000 7f1aa499c640 1 --2- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa00ad6a0 0x7f1aa0158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.873+0000 7f1aa499c640 1 -- 172.21.15.112:0/187683924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1aa006d890 con 0x7f1aa00ad6a0 2024-04-25T01:28:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.873+0000 7f1aa499c640 1 --2- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa00ad6a0 0x7f1aa0158570 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1a940011e0 tx=0x7f1a94001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.873+0000 7f1a9d7fa640 1 -- 172.21.15.112:0/187683924 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a940035f0 con 0x7f1aa00ad6a0 2024-04-25T01:28:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.874+0000 7f1a9d7fa640 1 -- 172.21.15.112:0/187683924 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5072 (secure 0 0 0) 0x7f1a940035f0 con 0x7f1aa00ad6a0 2024-04-25T01:28:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.874+0000 7f1a9d7fa640 1 -- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa00ad6a0 msgr2=0x7f1aa0158570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.874+0000 7f1a9d7fa640 1 --2- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa00ad6a0 0x7f1aa0158570 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1a940011e0 tx=0x7f1a94001150 comp rx=0 tx=0).stop 2024-04-25T01:28:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.874+0000 7f1a9d7fa640 1 -- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa00ad6a0 msgr2=0x7f1aa0158570 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:28:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.874+0000 7f1a9d7fa640 1 --2- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa00ad6a0 0x7f1aa0158570 secure :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1a940011e0 tx=0x7f1a94001150 comp rx=0 tx=0).stop 2024-04-25T01:28:46.853 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:28:46.853 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:28:46.854 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:28:46.854 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:28:46.854 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 20, 2024-04-25T01:28:46.854 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 198501, 2024-04-25T01:28:46.854 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:28:46.854 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:28:46.854 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:28:46.854 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:28:46.854 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:46.854 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 11, 2024-04-25T01:28:46.854 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:28:46.854 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:28:46.855 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:28:46.855 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:28:46.855 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:28:46.855 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:28:46.855 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:28:46.855 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:28:46.855 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:28:46.855 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:28:46.855 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:28:46.855 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:28:46.855 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:28:46.855 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:28:46.855 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:46.856 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:28:46.856 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:28:46.856 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:28:46.856 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:28:46.856 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:28:46.856 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:28:46.856 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:28:46.856 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:28:46.856 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:28:46.856 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:28:46.856 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:28:46.856 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:28:46.856 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:28:46.856 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:46.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:46.857 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:28:46.857 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:28:46.857 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:28:46.857 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:28:46.857 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:28:46.857 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:28:46.857 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:28:46.857 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:28:46.857 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:28:46.857 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:28:46.857 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:28:46.857 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:28:46.857 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:28:46.857 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:28:46.857 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:28:46.858 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:28:46.858 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:28:46.858 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:28:46.858 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:28:46.858 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:28:46.858 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:28:46.858 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:28:46.858 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:28:46.858 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:28:46.858 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:28:46.858 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:28:46.858 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:28:46.858 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:28:46.858 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:46.859 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:28:46.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:46.859 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:28:46.859 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:46.859 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:28:46.859 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:28:46.859 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:28:46.859 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:28:46.859 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:46.859 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:28:46.859 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:28:46.859 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:46.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:46.859 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:46.859 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:28:46.860 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:28:46.860 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:46.860 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:46.860 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:46.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:46.860 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:28:46.860 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:28:46.860 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:28:46.860 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:28:46.860 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:28:46.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:46.860 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:46.861 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:28:46.861 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:28:46.861 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:28:46.861 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:28:46.861 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:46.861 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:28:46.861 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:28:46.861 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:46.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:46.861 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:46.861 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:28:46.861 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:28:46.861 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:46.862 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:46.862 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:46.862 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:46.862 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:28:46.862 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:28:46.862 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:28:46.862 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:28:46.862 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:28:46.862 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:46.862 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:46.862 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:28:46.862 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:28:46.862 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:28:46.862 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:28:46.862 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:46.863 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:28:46.863 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:28:46.863 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:46.863 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:46.863 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:46.863 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:28:46.863 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:28:46.863 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:46.863 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:46.863 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:46.864 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:46.864 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:28:46.864 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:28:46.864 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:28:46.864 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:28:46.864 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:28:46.864 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:46.864 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:46.864 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:46.864 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:28:46.864 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:28:46.864 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:46.864 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:28:46.865 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:28:46.865 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:28:46.865 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:46.865 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:46.865 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:28:46.865 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:46.865 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:28:46.865 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:28:46.865 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:28:46.865 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:46.865 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:46.865 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:28:46.865 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:46.865 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:28:46.865 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:28:46.866 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 7 2024-04-25T01:28:46.866 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:46.866 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:46.866 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:28:46.866 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:46.866 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:28:46.867 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:28:46.867 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 8 2024-04-25T01:28:46.867 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:46.867 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:46.867 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:28:46.867 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:46.867 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:28:46.867 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:28:46.867 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:28:46.867 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:46.867 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:46.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:46.867 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:28:46.867 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:28:46.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.874+0000 7f1aa599e640 1 -- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1a8406bd30 msgr2=0x7f1a8406e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:46.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.874+0000 7f1aa599e640 1 --2- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1a8406bd30 0x7f1a8406e1f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1aa01551b0 tx=0x7f1a8c007d70 comp rx=0 tx=0).stop 2024-04-25T01:28:46.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.874+0000 7f1aa599e640 1 -- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa01eae60 msgr2=0x7f1aa01e8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:46.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.874+0000 7f1aa599e640 1 --2- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa01eae60 0x7f1aa01e8620 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1a9802aba0 tx=0x7f1a98004a40 comp rx=0 tx=0).stop 2024-04-25T01:28:46.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.874+0000 7f1aa519d640 1 -- 172.21.15.112:0/187683924 reap_dead start 2024-04-25T01:28:46.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.874+0000 7f1aa599e640 1 -- 172.21.15.112:0/187683924 shutdown_connections 2024-04-25T01:28:46.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.874+0000 7f1aa599e640 1 --2- 172.21.15.112:0/187683924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa01eae60 0x7f1aa01e8620 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.874+0000 7f1aa599e640 1 -- 172.21.15.112:0/187683924 >> 172.21.15.112:0/187683924 conn(0x7f1aa0074de0 msgr2=0x7f1aa00c7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:46.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.874+0000 7f1aa599e640 1 -- 172.21.15.112:0/187683924 shutdown_connections 2024-04-25T01:28:46.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:46.874+0000 7f1aa599e640 1 -- 172.21.15.112:0/187683924 wait complete. 2024-04-25T01:28:46.870 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:28:46.870 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:28:46.871 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:28:46.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.003+0000 7fac1ba6b640 1 Processor -- start 2024-04-25T01:28:46.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.003+0000 7fac1ba6b640 1 -- start start 2024-04-25T01:28:46.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.004+0000 7fac1ba6b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac14071520 0x7fac14154b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.004+0000 7fac1ba6b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fac140c7800 0x7fac140c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.004+0000 7fac1ba6b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac14154120 con 0x7fac140c7c80 2024-04-25T01:28:46.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.004+0000 7fac1ba6b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fac14154400 con 0x7fac140c7800 2024-04-25T01:28:46.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.004+0000 7fac1ba6b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fac140c7000 con 0x7fac14071520 2024-04-25T01:28:46.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.004+0000 7fac1a008640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fac140c7c80 0x7fac14153660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60620/0 (socket says 172.21.15.112:60620) 2024-04-25T01:28:46.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.004+0000 7fac1a008640 1 -- 172.21.15.112:0/1256398143 learned_addr learned my addr 172.21.15.112:0/1256398143 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:28:46.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.004+0000 7fac19006640 1 --2- 172.21.15.112:0/1256398143 >> v2:172.21.15.187:3300/0 conn(0x7fac140c7800 0x7fac140c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.004+0000 7fac19807640 1 --2- 172.21.15.112:0/1256398143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac14071520 0x7fac14154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.004+0000 7fac18805640 1 -- 172.21.15.112:0/1256398143 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 615943429 0 0) 0x7fac14154120 con 0x7fac140c7c80 2024-04-25T01:28:46.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.005+0000 7fac18805640 1 -- 172.21.15.112:0/1256398143 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac00003520 con 0x7fac140c7c80 2024-04-25T01:28:46.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.005+0000 7fac19807640 1 -- 172.21.15.112:0/1256398143 >> v2:172.21.15.187:3300/0 conn(0x7fac140c7800 msgr2=0x7fac140c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:28:46.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.005+0000 7fac19807640 1 --2- 172.21.15.112:0/1256398143 >> v2:172.21.15.187:3300/0 conn(0x7fac140c7800 0x7fac140c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.005+0000 7fac19807640 1 -- 172.21.15.112:0/1256398143 >> v1:172.21.15.112:6789/0 conn(0x7fac140c7c80 legacy=0x7fac14153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:46.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.005+0000 7fac19807640 1 -- 172.21.15.112:0/1256398143 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac140c72e0 con 0x7fac14071520 2024-04-25T01:28:46.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.005+0000 7fac19006640 1 --2- 172.21.15.112:0/1256398143 >> v2:172.21.15.187:3300/0 conn(0x7fac140c7800 0x7fac140c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:28:46.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.005+0000 7fac19807640 1 --2- 172.21.15.112:0/1256398143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac14071520 0x7fac14154b50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fac100097a0 tx=0x7fac1002a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a85d19e84f2deef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:46.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.006+0000 7fac18805640 1 -- 172.21.15.112:0/1256398143 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10037070 con 0x7fac14071520 2024-04-25T01:28:46.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.006+0000 7fac18805640 1 -- 172.21.15.112:0/1256398143 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac1002adb0 con 0x7fac14071520 2024-04-25T01:28:46.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.006+0000 7fac18805640 1 -- 172.21.15.112:0/1256398143 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10032660 con 0x7fac14071520 2024-04-25T01:28:46.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.006+0000 7fac1ba6b640 1 -- 172.21.15.112:0/1256398143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac14071520 msgr2=0x7fac14154b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:46.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.006+0000 7fac1ba6b640 1 --2- 172.21.15.112:0/1256398143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac14071520 0x7fac14154b50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fac100097a0 tx=0x7fac1002a060 comp rx=0 tx=0).stop 2024-04-25T01:28:46.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.006+0000 7fac1ba6b640 1 -- 172.21.15.112:0/1256398143 shutdown_connections 2024-04-25T01:28:46.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.006+0000 7fac1ba6b640 1 --2- 172.21.15.112:0/1256398143 >> v2:172.21.15.187:3300/0 conn(0x7fac140c7800 0x7fac140c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.006+0000 7fac1ba6b640 1 --2- 172.21.15.112:0/1256398143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac14071520 0x7fac14154b50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.006+0000 7fac1ba6b640 1 -- 172.21.15.112:0/1256398143 >> 172.21.15.112:0/1256398143 conn(0x7fac14074a00 msgr2=0x7fac140c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:46.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.006+0000 7fac1ba6b640 1 -- 172.21.15.112:0/1256398143 shutdown_connections 2024-04-25T01:28:46.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.006+0000 7fac1ba6b640 1 -- 172.21.15.112:0/1256398143 wait complete. 2024-04-25T01:28:46.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.007+0000 7fac1ba6b640 1 Processor -- start 2024-04-25T01:28:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.007+0000 7fac1ba6b640 1 -- start start 2024-04-25T01:28:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.008+0000 7fac1ba6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac14071520 0x7fac140c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.008+0000 7fac1ba6b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac140c7800 0x7fac1414c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.008+0000 7fac1ba6b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac140c7c80 0x7fac1414c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.008+0000 7fac1ba6b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac140adab0 con 0x7fac14071520 2024-04-25T01:28:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.008+0000 7fac1ba6b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fac140ad930 con 0x7fac140c7800 2024-04-25T01:28:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.008+0000 7fac1ba6b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fac140c8810 con 0x7fac140c7c80 2024-04-25T01:28:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.008+0000 7fac19807640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac14071520 0x7fac140c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.008+0000 7fac19807640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac14071520 0x7fac140c4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54660/0 (socket says 172.21.15.112:54660) 2024-04-25T01:28:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.008+0000 7fac19807640 1 -- 172.21.15.112:0/146634312 learned_addr learned my addr 172.21.15.112:0/146634312 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:28:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.008+0000 7fac19006640 1 --2- 172.21.15.112:0/146634312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac140c7800 0x7fac1414c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.008+0000 7fac1a008640 1 --2- 172.21.15.112:0/146634312 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac140c7c80 0x7fac1414c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.008+0000 7fac19807640 1 -- 172.21.15.112:0/146634312 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac140c7c80 msgr2=0x7fac1414c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.008+0000 7fac19807640 1 --2- 172.21.15.112:0/146634312 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac140c7c80 0x7fac1414c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.008+0000 7fac19807640 1 -- 172.21.15.112:0/146634312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac140c7800 msgr2=0x7fac1414c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.008+0000 7fac19807640 1 --2- 172.21.15.112:0/146634312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac140c7800 0x7fac1414c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.008+0000 7fac19807640 1 -- 172.21.15.112:0/146634312 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac141ed1e0 con 0x7fac14071520 2024-04-25T01:28:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.009+0000 7fac19807640 1 --2- 172.21.15.112:0/146634312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac14071520 0x7fac140c4b40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fac1002a0a0 tx=0x7fac100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.009+0000 7fac0a7fc640 1 -- 172.21.15.112:0/146634312 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10041070 con 0x7fac14071520 2024-04-25T01:28:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.009+0000 7fac0a7fc640 1 -- 172.21.15.112:0/146634312 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac1003c660 con 0x7fac14071520 2024-04-25T01:28:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.009+0000 7fac1ba6b640 1 -- 172.21.15.112:0/146634312 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac14152ab0 con 0x7fac14071520 2024-04-25T01:28:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.009+0000 7fac0a7fc640 1 -- 172.21.15.112:0/146634312 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10037040 con 0x7fac14071520 2024-04-25T01:28:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.009+0000 7fac1ba6b640 1 -- 172.21.15.112:0/146634312 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac140c5080 con 0x7fac14071520 2024-04-25T01:28:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.010+0000 7fac1ba6b640 1 --2- 172.21.15.112:0/146634312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac140682a0 0x7fac141e9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.010+0000 7fac1ba6b640 1 -- 172.21.15.112:0/146634312 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fac14075980 con 0x7fac140682a0 2024-04-25T01:28:46.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.010+0000 7fac19006640 1 --2- 172.21.15.112:0/146634312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac140682a0 0x7fac141e9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.010+0000 7fac19006640 1 -- 172.21.15.112:0/146634312 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac14152fd0 con 0x7fac140682a0 2024-04-25T01:28:46.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.011+0000 7fac0a7fc640 1 -- 172.21.15.112:0/146634312 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fac10044070 con 0x7fac14071520 2024-04-25T01:28:46.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.011+0000 7fac19006640 1 --2- 172.21.15.112:0/146634312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac140682a0 0x7fac141e9040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fac140c5f30 tx=0x7fac04011d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:46.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.011+0000 7fac0a7fc640 1 --2- 172.21.15.112:0/146634312 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fabf806be30 0x7fabf806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:46.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.011+0000 7fac0a7fc640 1 -- 172.21.15.112:0/146634312 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fac100ab7d0 con 0x7fac14071520 2024-04-25T01:28:46.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.011+0000 7fac1a008640 1 --2- 172.21.15.112:0/146634312 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fabf806be30 0x7fabf806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:46.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.011+0000 7fac0a7fc640 1 -- 172.21.15.112:0/146634312 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0400fb60 con 0x7fac140682a0 2024-04-25T01:28:46.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.011+0000 7fac0a7fc640 1 -- 172.21.15.112:0/146634312 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fac0400fd00 con 0x7fac140682a0 2024-04-25T01:28:46.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.011+0000 7fac0a7fc640 1 -- 172.21.15.112:0/146634312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac140682a0 msgr2=0x7fac141e9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:46.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.011+0000 7fac0a7fc640 1 --2- 172.21.15.112:0/146634312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac140682a0 0x7fac141e9040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fac140c5f30 tx=0x7fac04011d40 comp rx=0 tx=0).stop 2024-04-25T01:28:46.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.011+0000 7fac0a7fc640 1 -- 172.21.15.112:0/146634312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac140682a0 msgr2=0x7fac141e9040 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:28:46.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.011+0000 7fac0a7fc640 1 --2- 172.21.15.112:0/146634312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac140682a0 0x7fac141e9040 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:46.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.011+0000 7fac1a008640 1 --2- 172.21.15.112:0/146634312 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fabf806be30 0x7fabf806e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fac140c5b40 tx=0x7fac0c001970 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:47.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.021+0000 7fac1ba6b640 1 --2- 172.21.15.112:0/146634312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac140c2560 0x7fac141e9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:47.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.021+0000 7fac1ba6b640 1 -- 172.21.15.112:0/146634312 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fac14157a70 con 0x7fac140c2560 2024-04-25T01:28:47.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.021+0000 7fac19006640 1 --2- 172.21.15.112:0/146634312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac140c2560 0x7fac141e9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:47.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.022+0000 7fac19006640 1 -- 172.21.15.112:0/146634312 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fac14154c60 con 0x7fac140c2560 2024-04-25T01:28:47.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.022+0000 7fac19006640 1 --2- 172.21.15.112:0/146634312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac140c2560 0x7fac141e9580 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fac140c5f30 tx=0x7fac040046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:47.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.022+0000 7fac0a7fc640 1 -- 172.21.15.112:0/146634312 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0400fb60 con 0x7fac140c2560 2024-04-25T01:28:47.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.022+0000 7fac0a7fc640 1 -- 172.21.15.112:0/146634312 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4729 (secure 0 0 0) 0x7fac0402c420 con 0x7fac140c2560 2024-04-25T01:28:47.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.022+0000 7fac0a7fc640 1 -- 172.21.15.112:0/146634312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac140c2560 msgr2=0x7fac141e9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:47.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.022+0000 7fac0a7fc640 1 --2- 172.21.15.112:0/146634312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac140c2560 0x7fac141e9580 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fac140c5f30 tx=0x7fac040046c0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.022+0000 7fac0a7fc640 1 -- 172.21.15.112:0/146634312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac140c2560 msgr2=0x7fac141e9580 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:28:47.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.022+0000 7fac0a7fc640 1 --2- 172.21.15.112:0/146634312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac140c2560 0x7fac141e9580 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.002 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:28:47.002 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:28:47.002 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:28:47.002 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:28:47.002 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 20, 2024-04-25T01:28:47.002 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 95161, 2024-04-25T01:28:47.002 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:28:47.002 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:28:47.003 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:28:47.003 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:28:47.003 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:47.003 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 12, 2024-04-25T01:28:47.003 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:28:47.003 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:28:47.003 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:28:47.003 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:28:47.003 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:28:47.004 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:28:47.004 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:28:47.004 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:28:47.004 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:28:47.004 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:28:47.004 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:28:47.004 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:28:47.004 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:28:47.004 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:28:47.004 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:47.004 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:28:47.004 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:28:47.004 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:28:47.004 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:28:47.005 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:28:47.005 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:28:47.005 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:28:47.005 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:28:47.005 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:28:47.005 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:28:47.005 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:28:47.005 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:28:47.005 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:28:47.005 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:47.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.005 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:28:47.005 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:28:47.005 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:28:47.005 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:28:47.006 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:28:47.006 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:28:47.006 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:28:47.006 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:28:47.006 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:28:47.006 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:28:47.006 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:28:47.006 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:28:47.006 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:28:47.006 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:28:47.006 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:28:47.006 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:28:47.006 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:28:47.006 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:28:47.007 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:28:47.007 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:28:47.007 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:28:47.007 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:28:47.007 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:28:47.007 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:28:47.007 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:28:47.007 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:28:47.007 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:28:47.007 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:28:47.007 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:47.007 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:28:47.007 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.007 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:28:47.008 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.008 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:28:47.008 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:28:47.008 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:28:47.008 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:28:47.008 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.008 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:28:47.008 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:28:47.008 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:47.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.008 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.008 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:28:47.008 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:28:47.008 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:47.008 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:47.009 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:47.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.009 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:28:47.009 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:28:47.009 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:28:47.009 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:28:47.009 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:28:47.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.009 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.009 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:28:47.009 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:28:47.009 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:28:47.009 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:28:47.009 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.009 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:28:47.010 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:28:47.010 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:47.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.010 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.010 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:28:47.010 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:28:47.010 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:47.010 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:47.010 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:47.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.010 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:28:47.010 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:28:47.010 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:28:47.010 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:28:47.011 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:28:47.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.011 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.011 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:28:47.011 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:28:47.011 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:28:47.011 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:28:47.011 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.011 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:28:47.011 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:28:47.011 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:47.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.011 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.011 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:28:47.012 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:28:47.012 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:47.012 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:47.012 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:47.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.012 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:28:47.012 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:28:47.012 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:28:47.012 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:28:47.012 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:28:47.012 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:47.012 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:47.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.012 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:28:47.012 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:28:47.013 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.013 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:28:47.013 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:28:47.013 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:28:47.013 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:47.013 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:47.013 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:28:47.013 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.013 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:28:47.013 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:28:47.013 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:28:47.013 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:47.013 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:47.013 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:28:47.013 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.014 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:28:47.014 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:28:47.014 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:28:47.014 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:47.014 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:47.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.014 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:28:47.014 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:28:47.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.023+0000 7fac1ba6b640 1 -- 172.21.15.112:0/146634312 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fabf806be30 msgr2=0x7fabf806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:47.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.023+0000 7fac1ba6b640 1 --2- 172.21.15.112:0/146634312 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fabf806be30 0x7fabf806e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fac140c5b40 tx=0x7fac0c001970 comp rx=0 tx=0).stop 2024-04-25T01:28:47.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.023+0000 7fac1ba6b640 1 -- 172.21.15.112:0/146634312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac14071520 msgr2=0x7fac140c4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:47.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.023+0000 7fac1ba6b640 1 --2- 172.21.15.112:0/146634312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac14071520 0x7fac140c4b40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fac1002a0a0 tx=0x7fac100046c0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.023+0000 7fac1a008640 1 -- 172.21.15.112:0/146634312 reap_dead start 2024-04-25T01:28:47.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.023+0000 7fac1a008640 1 -- 172.21.15.112:0/146634312 reap_dead start 2024-04-25T01:28:47.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.023+0000 7fac1ba6b640 1 -- 172.21.15.112:0/146634312 shutdown_connections 2024-04-25T01:28:47.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.023+0000 7fac1ba6b640 1 -- 172.21.15.112:0/146634312 >> 172.21.15.112:0/146634312 conn(0x7fac14074a00 msgr2=0x7fac140ab820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:47.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.023+0000 7fac1ba6b640 1 -- 172.21.15.112:0/146634312 shutdown_connections 2024-04-25T01:28:47.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.023+0000 7fac1ba6b640 1 -- 172.21.15.112:0/146634312 wait complete. 2024-04-25T01:28:47.016 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:28:47.016 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:28:47.017 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:28:47.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.146+0000 7f3976e37640 1 Processor -- start 2024-04-25T01:28:47.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.146+0000 7f3976e37640 1 -- start start 2024-04-25T01:28:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.146+0000 7f3976e37640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f397014f7e0 0x7f397006bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.146+0000 7f3976e37640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39701558a0 0x7f39701574a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.146+0000 7f3976e37640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39701579e0 con 0x7f39700caa40 2024-04-25T01:28:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.147+0000 7f3976e37640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3970159aa0 con 0x7f397014f7e0 2024-04-25T01:28:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.147+0000 7f3976e37640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3970158e90 con 0x7f39701558a0 2024-04-25T01:28:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.147+0000 7f3974bd3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f397014f7e0 0x7f397006bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.147+0000 7f3974bd3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f397014f7e0 0x7f397006bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:55602/0 (socket says 172.21.15.112:55602) 2024-04-25T01:28:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.147+0000 7f3974bd3640 1 -- 172.21.15.112:0/287338633 learned_addr learned my addr 172.21.15.112:0/287338633 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:28:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.147+0000 7f396ffff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f39700caa40 0x7f3970156d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:52890/0 (socket says 172.21.15.112:52890) 2024-04-25T01:28:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.147+0000 7f39753d4640 1 --2- 172.21.15.112:0/287338633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39701558a0 0x7f39701574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.147+0000 7f3974bd3640 1 -- 172.21.15.112:0/287338633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39701558a0 msgr2=0x7f39701574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:28:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.147+0000 7f3974bd3640 1 --2- 172.21.15.112:0/287338633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39701558a0 0x7f39701574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.147+0000 7f3974bd3640 1 -- 172.21.15.112:0/287338633 >> v1:172.21.15.187:6789/0 conn(0x7f39700caa40 legacy=0x7f3970156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:47.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.147+0000 7f3974bd3640 1 -- 172.21.15.112:0/287338633 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3970159170 con 0x7f397014f7e0 2024-04-25T01:28:47.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.148+0000 7f3974bd3640 1 --2- 172.21.15.112:0/287338633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f397014f7e0 0x7f397006bd50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3964009870 tx=0x7f396402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59a859eb10b02b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:47.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.148+0000 7f396f7fe640 1 -- 172.21.15.112:0/287338633 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3964037070 con 0x7f397014f7e0 2024-04-25T01:28:47.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.148+0000 7f396f7fe640 1 -- 172.21.15.112:0/287338633 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3964032440 con 0x7f397014f7e0 2024-04-25T01:28:47.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.148+0000 7f396f7fe640 1 -- 172.21.15.112:0/287338633 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3964032810 con 0x7f397014f7e0 2024-04-25T01:28:47.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.148+0000 7f3976e37640 1 -- 172.21.15.112:0/287338633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f397014f7e0 msgr2=0x7f397006bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:47.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.148+0000 7f3976e37640 1 --2- 172.21.15.112:0/287338633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f397014f7e0 0x7f397006bd50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3964009870 tx=0x7f396402a270 comp rx=0 tx=0).stop 2024-04-25T01:28:47.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.148+0000 7f3976e37640 1 -- 172.21.15.112:0/287338633 shutdown_connections 2024-04-25T01:28:47.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.148+0000 7f3976e37640 1 --2- 172.21.15.112:0/287338633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39701558a0 0x7f39701574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.148+0000 7f3976e37640 1 --2- 172.21.15.112:0/287338633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f397014f7e0 0x7f397006bd50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.148+0000 7f3976e37640 1 -- 172.21.15.112:0/287338633 >> 172.21.15.112:0/287338633 conn(0x7f39700acc90 msgr2=0x7f39700ad0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:47.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.149+0000 7f3976e37640 1 -- 172.21.15.112:0/287338633 shutdown_connections 2024-04-25T01:28:47.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.149+0000 7f3976e37640 1 -- 172.21.15.112:0/287338633 wait complete. 2024-04-25T01:28:47.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.150+0000 7f3976e37640 1 Processor -- start 2024-04-25T01:28:47.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.150+0000 7f3976e37640 1 -- start start 2024-04-25T01:28:47.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.151+0000 7f3976e37640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39700caa40 0x7f39701ec040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:47.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.151+0000 7f3976e37640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f397014f7e0 0x7f39700c10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:47.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.151+0000 7f3976e37640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39701558a0 0x7f39700c2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:47.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.151+0000 7f3976e37640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3970153450 con 0x7f39700caa40 2024-04-25T01:28:47.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.151+0000 7f3976e37640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3970159730 con 0x7f397014f7e0 2024-04-25T01:28:47.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.151+0000 7f3976e37640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f397014fbc0 con 0x7f39701558a0 2024-04-25T01:28:47.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.151+0000 7f3974bd3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39700caa40 0x7f39701ec040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:47.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.151+0000 7f3974bd3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39700caa40 0x7f39701ec040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54670/0 (socket says 172.21.15.112:54670) 2024-04-25T01:28:47.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.151+0000 7f3974bd3640 1 -- 172.21.15.112:0/3316093185 learned_addr learned my addr 172.21.15.112:0/3316093185 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:28:47.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.151+0000 7f396ffff640 1 --2- 172.21.15.112:0/3316093185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f397014f7e0 0x7f39700c10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:47.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.151+0000 7f39753d4640 1 --2- 172.21.15.112:0/3316093185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39701558a0 0x7f39700c2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:47.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.151+0000 7f3974bd3640 1 -- 172.21.15.112:0/3316093185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39701558a0 msgr2=0x7f39700c2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:47.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.151+0000 7f3974bd3640 1 --2- 172.21.15.112:0/3316093185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39701558a0 0x7f39700c2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.151+0000 7f3974bd3640 1 -- 172.21.15.112:0/3316093185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f397014f7e0 msgr2=0x7f39700c10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:47.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.151+0000 7f3974bd3640 1 --2- 172.21.15.112:0/3316093185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f397014f7e0 0x7f39700c10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.151+0000 7f3974bd3640 1 -- 172.21.15.112:0/3316093185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39700aecf0 con 0x7f39700caa40 2024-04-25T01:28:47.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.151+0000 7f3974bd3640 1 --2- 172.21.15.112:0/3316093185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39700caa40 0x7f39701ec040 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f396402a2b0 tx=0x7f39640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:47.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.152+0000 7f396d7fa640 1 -- 172.21.15.112:0/3316093185 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3964037040 con 0x7f39700caa40 2024-04-25T01:28:47.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.152+0000 7f396d7fa640 1 -- 172.21.15.112:0/3316093185 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3964042070 con 0x7f39700caa40 2024-04-25T01:28:47.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.152+0000 7f3976e37640 1 -- 172.21.15.112:0/3316093185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39701eb560 con 0x7f39700caa40 2024-04-25T01:28:47.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.152+0000 7f396d7fa640 1 -- 172.21.15.112:0/3316093185 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3964030880 con 0x7f39700caa40 2024-04-25T01:28:47.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.152+0000 7f3976e37640 1 -- 172.21.15.112:0/3316093185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39701ebb20 con 0x7f39700caa40 2024-04-25T01:28:47.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.153+0000 7f3976e37640 1 --2- 172.21.15.112:0/3316093185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39700c2770 0x7f39700c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:47.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.153+0000 7f3976e37640 1 -- 172.21.15.112:0/3316093185 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f39701ebd60 con 0x7f39700c2770 2024-04-25T01:28:47.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.153+0000 7f396ffff640 1 --2- 172.21.15.112:0/3316093185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39700c2770 0x7f39700c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:47.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.153+0000 7f396ffff640 1 -- 172.21.15.112:0/3316093185 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3970075980 con 0x7f39700c2770 2024-04-25T01:28:47.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.153+0000 7f396d7fa640 1 -- 172.21.15.112:0/3316093185 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3964030a20 con 0x7f39700caa40 2024-04-25T01:28:47.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.153+0000 7f396ffff640 1 --2- 172.21.15.112:0/3316093185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39700c2770 0x7f39700c47c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f39700bec20 tx=0x7f395c001920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:47.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.154+0000 7f396d7fa640 1 --2- 172.21.15.112:0/3316093185 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f395406be60 0x7f395406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:47.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.154+0000 7f396d7fa640 1 -- 172.21.15.112:0/3316093185 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f39640ab890 con 0x7f39700caa40 2024-04-25T01:28:47.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.154+0000 7f39753d4640 1 --2- 172.21.15.112:0/3316093185 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f395406be60 0x7f395406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:47.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.154+0000 7f396d7fa640 1 -- 172.21.15.112:0/3316093185 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c011580 con 0x7f39700c2770 2024-04-25T01:28:47.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.154+0000 7f396d7fa640 1 -- 172.21.15.112:0/3316093185 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f395c018070 con 0x7f39700c2770 2024-04-25T01:28:47.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.154+0000 7f396d7fa640 1 -- 172.21.15.112:0/3316093185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39700c2770 msgr2=0x7f39700c47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:47.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.154+0000 7f396d7fa640 1 --2- 172.21.15.112:0/3316093185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39700c2770 0x7f39700c47c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f39700bec20 tx=0x7f395c001920 comp rx=0 tx=0).stop 2024-04-25T01:28:47.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.154+0000 7f396d7fa640 1 -- 172.21.15.112:0/3316093185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39700c2770 msgr2=0x7f39700c47c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:28:47.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.154+0000 7f396d7fa640 1 --2- 172.21.15.112:0/3316093185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39700c2770 0x7f39700c47c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.154+0000 7f39753d4640 1 --2- 172.21.15.112:0/3316093185 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f395406be60 0x7f395406e320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3968011340 tx=0x7f396800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:47.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.164+0000 7f3976e37640 1 --2- 172.21.15.112:0/3316093185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3970152b50 0x7f39700c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:47.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.164+0000 7f3976e37640 1 -- 172.21.15.112:0/3316093185 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f39700c8d30 con 0x7f3970152b50 2024-04-25T01:28:47.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.164+0000 7f396ffff640 1 --2- 172.21.15.112:0/3316093185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3970152b50 0x7f39700c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:47.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.164+0000 7f396ffff640 1 -- 172.21.15.112:0/3316093185 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f39700cbe10 con 0x7f3970152b50 2024-04-25T01:28:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.165+0000 7f396ffff640 1 --2- 172.21.15.112:0/3316093185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3970152b50 0x7f39700c4d00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f395c014000 tx=0x7f395c000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.165+0000 7f396d7fa640 1 -- 172.21.15.112:0/3316093185 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c028470 con 0x7f3970152b50 2024-04-25T01:28:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.165+0000 7f396d7fa640 1 -- 172.21.15.112:0/3316093185 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4559 (secure 0 0 0) 0x7f395c0133f0 con 0x7f3970152b50 2024-04-25T01:28:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.165+0000 7f396d7fa640 1 -- 172.21.15.112:0/3316093185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3970152b50 msgr2=0x7f39700c4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.165+0000 7f396d7fa640 1 --2- 172.21.15.112:0/3316093185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3970152b50 0x7f39700c4d00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f395c014000 tx=0x7f395c000ed0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.165+0000 7f396d7fa640 1 -- 172.21.15.112:0/3316093185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3970152b50 msgr2=0x7f39700c4d00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:28:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.165+0000 7f396d7fa640 1 --2- 172.21.15.112:0/3316093185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3970152b50 0x7f39700c4d00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.144 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:28:47.145 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:28:47.145 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:28:47.145 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:28:47.145 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 20, 2024-04-25T01:28:47.145 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 27108, 2024-04-25T01:28:47.145 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:28:47.145 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:28:47.145 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:28:47.145 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:28:47.145 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:47.145 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 12, 2024-04-25T01:28:47.145 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:28:47.145 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:28:47.145 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:28:47.145 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:28:47.146 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:28:47.146 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:28:47.146 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:28:47.146 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:28:47.146 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:28:47.146 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:28:47.146 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:28:47.146 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:28:47.146 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:28:47.146 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:28:47.146 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:47.146 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:28:47.146 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:28:47.146 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:28:47.147 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:28:47.147 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:28:47.147 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:28:47.147 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:28:47.147 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:28:47.147 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:28:47.147 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:28:47.147 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:28:47.147 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:28:47.147 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:28:47.147 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:47.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.147 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:28:47.147 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:28:47.148 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:28:47.148 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:28:47.148 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:28:47.148 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:28:47.148 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:28:47.148 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:28:47.148 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:28:47.148 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:28:47.148 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:28:47.148 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:28:47.148 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:28:47.148 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:28:47.148 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:28:47.148 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:28:47.148 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:28:47.149 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:28:47.149 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:28:47.149 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:28:47.149 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:28:47.149 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:28:47.149 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:28:47.149 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:28:47.149 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:28:47.149 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:28:47.149 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:28:47.149 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:28:47.149 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:47.149 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:28:47.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.150 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:28:47.150 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.150 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:28:47.150 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:28:47.150 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:28:47.150 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:28:47.150 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.150 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:28:47.150 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:28:47.150 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:47.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.150 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.150 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:28:47.150 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:28:47.150 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:47.151 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:47.151 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:47.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.151 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:28:47.151 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:28:47.151 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:28:47.151 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:28:47.151 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:28:47.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.151 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.151 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:28:47.151 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:28:47.151 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:28:47.151 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:28:47.151 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.152 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:28:47.152 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:28:47.152 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:47.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.152 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.152 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:28:47.152 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:28:47.152 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:47.152 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:47.152 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:47.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.152 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:28:47.152 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:28:47.152 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:28:47.152 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:28:47.153 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:28:47.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.153 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.153 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:28:47.153 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:28:47.153 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:28:47.153 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:28:47.153 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.153 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:28:47.153 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:28:47.153 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:47.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.153 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.153 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:28:47.153 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:28:47.153 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:28:47.154 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:47.154 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:47.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.154 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:28:47.154 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:28:47.154 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:28:47.154 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:28:47.154 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:28:47.154 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:47.154 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:47.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.154 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:28:47.154 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:28:47.154 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.154 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:28:47.155 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:28:47.155 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:28:47.155 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:47.155 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:28:47.155 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:28:47.155 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:28:47.155 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:28:47.155 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:28:47.155 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:28:47.155 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:28:47.155 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:28:47.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:28:47.156 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:28:47.156 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:28:47.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.165+0000 7f3976e37640 1 -- 172.21.15.112:0/3316093185 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f395406be60 msgr2=0x7f395406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:47.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.165+0000 7f3976e37640 1 --2- 172.21.15.112:0/3316093185 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f395406be60 0x7f395406e320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3968011340 tx=0x7f396800b040 comp rx=0 tx=0).stop 2024-04-25T01:28:47.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.165+0000 7f3976e37640 1 -- 172.21.15.112:0/3316093185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39700caa40 msgr2=0x7f39701ec040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:47.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.165+0000 7f3976e37640 1 --2- 172.21.15.112:0/3316093185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39700caa40 0x7f39701ec040 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f396402a2b0 tx=0x7f39640046c0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.165+0000 7f39753d4640 1 -- 172.21.15.112:0/3316093185 reap_dead start 2024-04-25T01:28:47.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.165+0000 7f39753d4640 1 -- 172.21.15.112:0/3316093185 reap_dead start 2024-04-25T01:28:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.166+0000 7f3976e37640 1 -- 172.21.15.112:0/3316093185 shutdown_connections 2024-04-25T01:28:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.166+0000 7f3976e37640 1 -- 172.21.15.112:0/3316093185 >> 172.21.15.112:0/3316093185 conn(0x7f39700acc90 msgr2=0x7f39700ad070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.166+0000 7f3976e37640 1 -- 172.21.15.112:0/3316093185 shutdown_connections 2024-04-25T01:28:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.166+0000 7f3976e37640 1 -- 172.21.15.112:0/3316093185 wait complete. 2024-04-25T01:28:47.158 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:28:47.158 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-04-25T01:28:47.158 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-25T01:28:47.158 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-04-25T01:28:47.158 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-04-25T01:28:47.159 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T01:28:47.159 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:28:47.170 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T01:28:47.170 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T01:28:47.170 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:28:47.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.306+0000 7f116b21b640 1 Processor -- start 2024-04-25T01:28:47.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.306+0000 7f116b21b640 1 -- start start 2024-04-25T01:28:47.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.307+0000 7f116b21b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f11640c70e0 0x7f1164156530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:47.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.307+0000 7f116b21b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1164155a90 con 0x7f11640c6560 2024-04-25T01:28:47.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.307+0000 7f116b21b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1164070420 con 0x7f11640c3140 2024-04-25T01:28:47.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.307+0000 7f116b21b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1164157580 con 0x7f11640c70e0 2024-04-25T01:28:47.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.307+0000 7f11697b8640 1 -- >> v2:172.21.15.187:3300/0 conn(0x7f11640c70e0 msgr2=0x7f1164156530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:28:47.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.307+0000 7f115bfff640 1 -- >> v1:172.21.15.187:6789/0 conn(0x7f11640c3140 legacy=0x7f11640aac50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.187:6789/0 2024-04-25T01:28:47.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.307+0000 7f11697b8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f11640c70e0 0x7f1164156530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:28:47.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.307+0000 7f1168fb7640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f11640c6560 0x7f1164156e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60632/0 (socket says 172.21.15.112:60632) 2024-04-25T01:28:47.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.307+0000 7f1168fb7640 1 -- 172.21.15.112:0/3896633762 learned_addr learned my addr 172.21.15.112:0/3896633762 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:28:47.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.307+0000 7f115b7fe640 1 -- 172.21.15.112:0/3896633762 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1382021182 0 0) 0x7f1164155a90 con 0x7f11640c6560 2024-04-25T01:28:47.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.307+0000 7f115b7fe640 1 -- 172.21.15.112:0/3896633762 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f114c003520 con 0x7f11640c6560 2024-04-25T01:28:47.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.307+0000 7f115b7fe640 1 -- 172.21.15.112:0/3896633762 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2115190866 0 0) 0x7f114c003520 con 0x7f11640c6560 2024-04-25T01:28:47.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f115b7fe640 1 -- 172.21.15.112:0/3896633762 >> v2:172.21.15.187:3300/0 conn(0x7f11640c70e0 msgr2=0x7f1164156530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:28:47.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f115b7fe640 1 --2- 172.21.15.112:0/3896633762 >> v2:172.21.15.187:3300/0 conn(0x7f11640c70e0 0x7f1164156530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f115b7fe640 1 -- 172.21.15.112:0/3896633762 >> v1:172.21.15.187:6789/0 conn(0x7f11640c3140 legacy=0x7f11640aac50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:28:47.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f115b7fe640 1 -- 172.21.15.112:0/3896633762 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1164155d30 con 0x7f11640c6560 2024-04-25T01:28:47.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f115b7fe640 1 -- 172.21.15.112:0/3896633762 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f1160001690 con 0x7f11640c6560 2024-04-25T01:28:47.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f115b7fe640 1 -- 172.21.15.112:0/3896633762 >> v1:172.21.15.112:6789/0 conn(0x7f11640c6560 legacy=0x7f1164156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:47.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f115b7fe640 1 --2- 172.21.15.112:0/3896633762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f114c0039b0 0x7f114c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:47.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f115b7fe640 1 --2- 172.21.15.112:0/3896633762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f114c004800 0x7f114c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:47.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f115b7fe640 1 --2- 172.21.15.112:0/3896633762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f114c0053c0 0x7f114c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:47.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f115b7fe640 1 -- 172.21.15.112:0/3896633762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1164157580 con 0x7f114c0053c0 2024-04-25T01:28:47.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f115b7fe640 1 -- 172.21.15.112:0/3896633762 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f114c00c260 con 0x7f114c0039b0 2024-04-25T01:28:47.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f115b7fe640 1 -- 172.21.15.112:0/3896633762 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f114c00c460 con 0x7f114c004800 2024-04-25T01:28:47.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f1168fb7640 1 -- 172.21.15.112:0/3896633762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f114c0039b0 msgr2=0x7f114c003db0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:28:47.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f1168fb7640 1 --2- 172.21.15.112:0/3896633762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f114c0039b0 0x7f114c003db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:28:47.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f11697b8640 1 --2- 172.21.15.112:0/3896633762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f114c0053c0 0x7f114c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:47.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f115bfff640 1 --2- 172.21.15.112:0/3896633762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f114c004800 0x7f114c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:47.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f11697b8640 1 -- 172.21.15.112:0/3896633762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f114c004800 msgr2=0x7f114c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:28:47.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f11697b8640 1 --2- 172.21.15.112:0/3896633762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f114c004800 0x7f114c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f11697b8640 1 -- 172.21.15.112:0/3896633762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f114c0039b0 msgr2=0x7f114c003db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:28:47.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f11697b8640 1 --2- 172.21.15.112:0/3896633762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f114c0039b0 0x7f114c003db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.308+0000 7f11697b8640 1 -- 172.21.15.112:0/3896633762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c003520 con 0x7f114c0053c0 2024-04-25T01:28:47.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.309+0000 7f11697b8640 1 -- 172.21.15.112:0/3896633762 reap_dead start 2024-04-25T01:28:47.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.309+0000 7f11697b8640 1 --2- 172.21.15.112:0/3896633762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f114c0053c0 0x7f114c00b9c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f115c009e30 tx=0x7f114c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac29c475a39da840 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:47.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.309+0000 7f115b7fe640 1 -- 172.21.15.112:0/3896633762 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c009a30 con 0x7f114c0053c0 2024-04-25T01:28:47.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.309+0000 7f115b7fe640 1 -- 172.21.15.112:0/3896633762 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f115c02c040 con 0x7f114c0053c0 2024-04-25T01:28:47.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.309+0000 7f115b7fe640 1 -- 172.21.15.112:0/3896633762 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c0308b0 con 0x7f114c0053c0 2024-04-25T01:28:47.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.309+0000 7f116b21b640 1 -- 172.21.15.112:0/3896633762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f114c0053c0 msgr2=0x7f114c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:47.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.309+0000 7f116b21b640 1 --2- 172.21.15.112:0/3896633762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f114c0053c0 0x7f114c00b9c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f115c009e30 tx=0x7f114c005830 comp rx=0 tx=0).stop 2024-04-25T01:28:47.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.310+0000 7f116b21b640 1 -- 172.21.15.112:0/3896633762 shutdown_connections 2024-04-25T01:28:47.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.310+0000 7f116b21b640 1 --2- 172.21.15.112:0/3896633762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f114c0053c0 0x7f114c00b9c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.310+0000 7f116b21b640 1 -- 172.21.15.112:0/3896633762 >> 172.21.15.112:0/3896633762 conn(0x7f11640743d0 msgr2=0x7f11640747b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:47.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.310+0000 7f116b21b640 1 -- 172.21.15.112:0/3896633762 shutdown_connections 2024-04-25T01:28:47.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.310+0000 7f116b21b640 1 -- 172.21.15.112:0/3896633762 wait complete. 2024-04-25T01:28:47.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.311+0000 7f116b21b640 1 Processor -- start 2024-04-25T01:28:47.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.311+0000 7f116b21b640 1 -- start start 2024-04-25T01:28:47.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.311+0000 7f116b21b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f114c0053c0 0x7f1164151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:47.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.311+0000 7f116b21b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f116414f920 0x7f116414fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:47.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.311+0000 7f116b21b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f116414ec40 0x7f116414f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:47.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.312+0000 7f116b21b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1164156530 con 0x7f116414f920 2024-04-25T01:28:47.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.312+0000 7f116b21b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f11640700f0 con 0x7f116414ec40 2024-04-25T01:28:47.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.312+0000 7f116b21b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f11641566b0 con 0x7f114c0053c0 2024-04-25T01:28:47.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.312+0000 7f11697b8640 1 -- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f116414ec40 msgr2=0x7f116414f0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:28:47.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.312+0000 7f11697b8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f116414ec40 0x7f116414f0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:28:47.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.312+0000 7f115bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f116414f920 0x7f116414fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:47.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.312+0000 7f115bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f116414f920 0x7f116414fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54696/0 (socket says 172.21.15.112:54696) 2024-04-25T01:28:47.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.312+0000 7f115bfff640 1 -- 172.21.15.112:0/2554717705 learned_addr learned my addr 172.21.15.112:0/2554717705 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:28:47.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.312+0000 7f1168fb7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f114c0053c0 0x7f1164151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:47.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.312+0000 7f115bfff640 1 -- 172.21.15.112:0/2554717705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f114c0053c0 msgr2=0x7f1164151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:47.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.312+0000 7f115bfff640 1 --2- 172.21.15.112:0/2554717705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f114c0053c0 0x7f1164151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.312+0000 7f115bfff640 1 -- 172.21.15.112:0/2554717705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f116414ec40 msgr2=0x7f116414f0f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:28:47.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.312+0000 7f115bfff640 1 --2- 172.21.15.112:0/2554717705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f116414ec40 0x7f116414f0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.312+0000 7f115bfff640 1 -- 172.21.15.112:0/2554717705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1164159d00 con 0x7f116414f920 2024-04-25T01:28:47.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.312+0000 7f115bfff640 1 --2- 172.21.15.112:0/2554717705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f116414f920 0x7f116414fda0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f115400e840 tx=0x7f115400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:47.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.312+0000 7f11597fa640 1 -- 172.21.15.112:0/2554717705 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154016020 con 0x7f116414f920 2024-04-25T01:28:47.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.312+0000 7f11597fa640 1 -- 172.21.15.112:0/2554717705 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11540039b0 con 0x7f116414f920 2024-04-25T01:28:47.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.312+0000 7f116b21b640 1 -- 172.21.15.112:0/2554717705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1164159180 con 0x7f116414f920 2024-04-25T01:28:47.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.312+0000 7f11597fa640 1 -- 172.21.15.112:0/2554717705 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154003da0 con 0x7f116414f920 2024-04-25T01:28:47.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.312+0000 7f116b21b640 1 -- 172.21.15.112:0/2554717705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1164159740 con 0x7f116414f920 2024-04-25T01:28:47.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.313+0000 7f116b21b640 1 -- 172.21.15.112:0/2554717705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11640c60a0 con 0x7f116414f920 2024-04-25T01:28:47.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.314+0000 7f11597fa640 1 -- 172.21.15.112:0/2554717705 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1154019020 con 0x7f116414f920 2024-04-25T01:28:47.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.314+0000 7f11597fa640 1 --2- 172.21.15.112:0/2554717705 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f115006be10 0x7f115006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:47.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.314+0000 7f11597fa640 1 -- 172.21.15.112:0/2554717705 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f115408aa70 con 0x7f116414f920 2024-04-25T01:28:47.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.314+0000 7f1168fb7640 1 --2- 172.21.15.112:0/2554717705 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f115006be10 0x7f115006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:47.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.315+0000 7f1168fb7640 1 --2- 172.21.15.112:0/2554717705 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f115006be10 0x7f115006e2d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f11640af200 tx=0x7f1160001e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:47.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.319+0000 7f11597fa640 1 -- 172.21.15.112:0/2554717705 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1154020070 con 0x7f116414f920 2024-04-25T01:28:47.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.596+0000 7f116b21b640 1 -- 172.21.15.112:0/2554717705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f11640c2f20 con 0x7f116414f920 2024-04-25T01:28:47.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.597+0000 7f11597fa640 1 -- 172.21.15.112:0/2554717705 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f1154059110 con 0x7f116414f920 2024-04-25T01:28:47.576 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:28:47.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.599+0000 7f116b21b640 1 -- 172.21.15.112:0/2554717705 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f115006be10 msgr2=0x7f115006e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:47.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.599+0000 7f116b21b640 1 --2- 172.21.15.112:0/2554717705 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f115006be10 0x7f115006e2d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f11640af200 tx=0x7f1160001e30 comp rx=0 tx=0).stop 2024-04-25T01:28:47.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.599+0000 7f116b21b640 1 -- 172.21.15.112:0/2554717705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f116414f920 msgr2=0x7f116414fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:47.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.599+0000 7f116b21b640 1 --2- 172.21.15.112:0/2554717705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f116414f920 0x7f116414fda0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f115400e840 tx=0x7f115400ed00 comp rx=0 tx=0).stop 2024-04-25T01:28:47.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.599+0000 7f116b21b640 1 -- 172.21.15.112:0/2554717705 shutdown_connections 2024-04-25T01:28:47.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.599+0000 7f116b21b640 1 --2- 172.21.15.112:0/2554717705 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f115006be10 0x7f115006e2d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.599+0000 7f116b21b640 1 --2- 172.21.15.112:0/2554717705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f116414ec40 0x7f116414f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.599+0000 7f116b21b640 1 --2- 172.21.15.112:0/2554717705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f116414f920 0x7f116414fda0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.599+0000 7f116b21b640 1 --2- 172.21.15.112:0/2554717705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f114c0053c0 0x7f1164151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.599+0000 7f116b21b640 1 -- 172.21.15.112:0/2554717705 >> 172.21.15.112:0/2554717705 conn(0x7f11640743d0 msgr2=0x7f1164154fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:47.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.599+0000 7f116b21b640 1 -- 172.21.15.112:0/2554717705 shutdown_connections 2024-04-25T01:28:47.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:47.599+0000 7f116b21b640 1 -- 172.21.15.112:0/2554717705 wait complete. 2024-04-25T01:28:50.589 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:28:50.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.723+0000 7f5465212640 1 Processor -- start 2024-04-25T01:28:50.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.723+0000 7f5465212640 1 -- start start 2024-04-25T01:28:50.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.724+0000 7f5465212640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5460152fd0 0x7f54600c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:50.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.724+0000 7f5465212640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5460156cf0 0x7f546006bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:50.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.724+0000 7f5465212640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54600c90e0 con 0x7f54600c5d70 2024-04-25T01:28:50.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.724+0000 7f5465212640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f54600ca9b0 con 0x7f5460156cf0 2024-04-25T01:28:50.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.724+0000 7f5465212640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f54600cac00 con 0x7f5460152fd0 2024-04-25T01:28:50.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.724+0000 7f545ed9d640 1 -- >> v2:172.21.15.187:3300/0 conn(0x7f5460152fd0 msgr2=0x7f54600c9b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:28:50.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.724+0000 7f5457fff640 1 -- >> v1:172.21.15.187:6789/0 conn(0x7f54600c5d70 legacy=0x7f5460156280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.187:6789/0 2024-04-25T01:28:50.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.724+0000 7f545ed9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5460152fd0 0x7f54600c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:28:50.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.724+0000 7f545f59e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5460156cf0 0x7f546006bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:50.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.724+0000 7f545f59e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5460156cf0 0x7f546006bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46552/0 (socket says 172.21.15.112:46552) 2024-04-25T01:28:50.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.724+0000 7f545f59e640 1 -- 172.21.15.112:0/3118374972 learned_addr learned my addr 172.21.15.112:0/3118374972 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:28:50.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.724+0000 7f545f59e640 1 -- 172.21.15.112:0/3118374972 >> v2:172.21.15.187:3300/0 conn(0x7f5460152fd0 msgr2=0x7f54600c9b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:28:50.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.724+0000 7f545f59e640 1 --2- 172.21.15.112:0/3118374972 >> v2:172.21.15.187:3300/0 conn(0x7f5460152fd0 0x7f54600c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:50.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.724+0000 7f545f59e640 1 -- 172.21.15.112:0/3118374972 >> v1:172.21.15.187:6789/0 conn(0x7f54600c5d70 legacy=0x7f5460156280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:28:50.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.724+0000 7f545f59e640 1 -- 172.21.15.112:0/3118374972 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54600c9380 con 0x7f5460156cf0 2024-04-25T01:28:50.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.725+0000 7f545f59e640 1 --2- 172.21.15.112:0/3118374972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5460156cf0 0x7f546006bda0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f546006ba40 tx=0x7f545802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec928d96e0d4f773 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:50.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.725+0000 7f545e59c640 1 -- 172.21.15.112:0/3118374972 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5458036070 con 0x7f5460156cf0 2024-04-25T01:28:50.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.725+0000 7f545e59c640 1 -- 172.21.15.112:0/3118374972 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5458031600 con 0x7f5460156cf0 2024-04-25T01:28:50.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.725+0000 7f545e59c640 1 -- 172.21.15.112:0/3118374972 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54580319d0 con 0x7f5460156cf0 2024-04-25T01:28:50.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.726+0000 7f5465212640 1 -- 172.21.15.112:0/3118374972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5460156cf0 msgr2=0x7f546006bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:50.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.726+0000 7f5465212640 1 --2- 172.21.15.112:0/3118374972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5460156cf0 0x7f546006bda0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f546006ba40 tx=0x7f545802a660 comp rx=0 tx=0).stop 2024-04-25T01:28:50.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.726+0000 7f5465212640 1 -- 172.21.15.112:0/3118374972 shutdown_connections 2024-04-25T01:28:50.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.726+0000 7f5465212640 1 --2- 172.21.15.112:0/3118374972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5460156cf0 0x7f546006bda0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:50.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.726+0000 7f5465212640 1 --2- 172.21.15.112:0/3118374972 >> v2:172.21.15.187:3300/0 conn(0x7f5460152fd0 0x7f54600c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:50.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.726+0000 7f5465212640 1 -- 172.21.15.112:0/3118374972 >> 172.21.15.112:0/3118374972 conn(0x7f54600acc70 msgr2=0x7f54600ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:50.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.726+0000 7f5465212640 1 -- 172.21.15.112:0/3118374972 shutdown_connections 2024-04-25T01:28:50.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.726+0000 7f5465212640 1 -- 172.21.15.112:0/3118374972 wait complete. 2024-04-25T01:28:50.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.727+0000 7f5465212640 1 Processor -- start 2024-04-25T01:28:50.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.728+0000 7f5465212640 1 -- start start 2024-04-25T01:28:50.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.728+0000 7f5465212640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54600c5d70 0x7f54600c1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:50.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.728+0000 7f5465212640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5460152fd0 0x7f5460155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:50.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.728+0000 7f5465212640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5460156cf0 0x7f546014eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:50.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.728+0000 7f5465212640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5460157100 con 0x7f5460152fd0 2024-04-25T01:28:50.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.728+0000 7f5465212640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f54600caee0 con 0x7f54600c5d70 2024-04-25T01:28:50.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.728+0000 7f5465212640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5460157280 con 0x7f5460156cf0 2024-04-25T01:28:50.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.728+0000 7f545ed9d640 1 -- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54600c5d70 msgr2=0x7f54600c1790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:28:50.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.728+0000 7f545ed9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54600c5d70 0x7f54600c1790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:28:50.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.729+0000 7f5457fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5460152fd0 0x7f5460155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:50.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.729+0000 7f5457fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5460152fd0 0x7f5460155000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46562/0 (socket says 172.21.15.112:46562) 2024-04-25T01:28:50.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.729+0000 7f5457fff640 1 -- 172.21.15.112:0/4004011630 learned_addr learned my addr 172.21.15.112:0/4004011630 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:28:50.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.729+0000 7f545f59e640 1 --2- 172.21.15.112:0/4004011630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5460156cf0 0x7f546014eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:50.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.729+0000 7f5457fff640 1 -- 172.21.15.112:0/4004011630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5460156cf0 msgr2=0x7f546014eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:50.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.729+0000 7f5457fff640 1 --2- 172.21.15.112:0/4004011630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5460156cf0 0x7f546014eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:50.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.729+0000 7f5457fff640 1 -- 172.21.15.112:0/4004011630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54600c5d70 msgr2=0x7f54600c1790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:28:50.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.729+0000 7f5457fff640 1 --2- 172.21.15.112:0/4004011630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54600c5d70 0x7f54600c1790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:50.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.729+0000 7f5457fff640 1 -- 172.21.15.112:0/4004011630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546014c600 con 0x7f5460152fd0 2024-04-25T01:28:50.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.729+0000 7f5457fff640 1 --2- 172.21.15.112:0/4004011630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5460152fd0 0x7f5460155000 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f544800b550 tx=0x7f544800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:50.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.729+0000 7f54577fe640 1 -- 172.21.15.112:0/4004011630 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448003cf0 con 0x7f5460152fd0 2024-04-25T01:28:50.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.729+0000 7f54577fe640 1 -- 172.21.15.112:0/4004011630 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5448015070 con 0x7f5460152fd0 2024-04-25T01:28:50.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.729+0000 7f5465212640 1 -- 172.21.15.112:0/4004011630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546014c120 con 0x7f5460152fd0 2024-04-25T01:28:50.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.729+0000 7f54577fe640 1 -- 172.21.15.112:0/4004011630 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448005290 con 0x7f5460152fd0 2024-04-25T01:28:50.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.729+0000 7f5465212640 1 -- 172.21.15.112:0/4004011630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54601587e0 con 0x7f5460152fd0 2024-04-25T01:28:50.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.730+0000 7f5465212640 1 -- 172.21.15.112:0/4004011630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54600c1cd0 con 0x7f5460152fd0 2024-04-25T01:28:50.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.731+0000 7f54577fe640 1 -- 172.21.15.112:0/4004011630 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5448005430 con 0x7f5460152fd0 2024-04-25T01:28:50.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.731+0000 7f54577fe640 1 --2- 172.21.15.112:0/4004011630 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f544006bde0 0x7f544006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:50.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.731+0000 7f545ed9d640 1 --2- 172.21.15.112:0/4004011630 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f544006bde0 0x7f544006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:50.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.731+0000 7f54577fe640 1 -- 172.21.15.112:0/4004011630 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f544808ae50 con 0x7f5460152fd0 2024-04-25T01:28:50.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.732+0000 7f545ed9d640 1 --2- 172.21.15.112:0/4004011630 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f544006bde0 0x7f544006e2a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5450006fd0 tx=0x7f5450006d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:50.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:50.737+0000 7f54577fe640 1 -- 172.21.15.112:0/4004011630 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f54600c1cd0 con 0x7f5460152fd0 2024-04-25T01:28:51.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:51.033+0000 7f5465212640 1 -- 172.21.15.112:0/4004011630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f546014e3c0 con 0x7f5460152fd0 2024-04-25T01:28:51.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:51.033+0000 7f54577fe640 1 -- 172.21.15.112:0/4004011630 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f54480594f0 con 0x7f5460152fd0 2024-04-25T01:28:51.012 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:28:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:51.035+0000 7f5465212640 1 -- 172.21.15.112:0/4004011630 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f544006bde0 msgr2=0x7f544006e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:51.035+0000 7f5465212640 1 --2- 172.21.15.112:0/4004011630 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f544006bde0 0x7f544006e2a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5450006fd0 tx=0x7f5450006d80 comp rx=0 tx=0).stop 2024-04-25T01:28:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:51.035+0000 7f5465212640 1 -- 172.21.15.112:0/4004011630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5460152fd0 msgr2=0x7f5460155000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:51.035+0000 7f5465212640 1 --2- 172.21.15.112:0/4004011630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5460152fd0 0x7f5460155000 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f544800b550 tx=0x7f544800ba10 comp rx=0 tx=0).stop 2024-04-25T01:28:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:51.035+0000 7f5465212640 1 -- 172.21.15.112:0/4004011630 shutdown_connections 2024-04-25T01:28:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:51.035+0000 7f5465212640 1 --2- 172.21.15.112:0/4004011630 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f544006bde0 0x7f544006e2a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:51.035+0000 7f5465212640 1 --2- 172.21.15.112:0/4004011630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5460156cf0 0x7f546014eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:51.035+0000 7f5465212640 1 --2- 172.21.15.112:0/4004011630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5460152fd0 0x7f5460155000 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:51.035+0000 7f5465212640 1 --2- 172.21.15.112:0/4004011630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54600c5d70 0x7f54600c1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:51.035+0000 7f5465212640 1 -- 172.21.15.112:0/4004011630 >> 172.21.15.112:0/4004011630 conn(0x7f54600acc70 msgr2=0x7f54600cb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:51.035+0000 7f5465212640 1 -- 172.21.15.112:0/4004011630 shutdown_connections 2024-04-25T01:28:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:51.035+0000 7f5465212640 1 -- 172.21.15.112:0/4004011630 wait complete. 2024-04-25T01:28:54.025 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:28:54.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.148+0000 7fa3ff42a640 1 Processor -- start 2024-04-25T01:28:54.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.148+0000 7fa3ff42a640 1 -- start start 2024-04-25T01:28:54.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.149+0000 7fa3ff42a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3f8152fd0 0x7fa3f80c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:54.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.149+0000 7fa3ff42a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa3f80c5d70 0x7fa3f8156280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:54.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.149+0000 7fa3ff42a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3f81567c0 con 0x7fa3f8156cf0 2024-04-25T01:28:54.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.149+0000 7fa3ff42a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa3f80ca9b0 con 0x7fa3f80c5d70 2024-04-25T01:28:54.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.149+0000 7fa3ff42a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f80c90e0 con 0x7fa3f8152fd0 2024-04-25T01:28:54.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.149+0000 7fa3fc9c5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa3f80c5d70 0x7fa3f8156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:54.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.149+0000 7fa3fd9c7640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa3f8156cf0 0x7fa3f806bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49402/0 (socket says 172.21.15.112:49402) 2024-04-25T01:28:54.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.149+0000 7fa3fd9c7640 1 -- 172.21.15.112:0/1489232981 learned_addr learned my addr 172.21.15.112:0/1489232981 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:28:54.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.149+0000 7fa3fd1c6640 1 --2- 172.21.15.112:0/1489232981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3f8152fd0 0x7fa3f80c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:54.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.150+0000 7fa3fc9c5640 1 -- 172.21.15.112:0/1489232981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3f8152fd0 msgr2=0x7fa3f80c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:28:54.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.150+0000 7fa3fc9c5640 1 --2- 172.21.15.112:0/1489232981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3f8152fd0 0x7fa3f80c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:54.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.150+0000 7fa3fc9c5640 1 -- 172.21.15.112:0/1489232981 >> v1:172.21.15.112:6789/0 conn(0x7fa3f8156cf0 legacy=0x7fa3f806bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:54.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.150+0000 7fa3fc9c5640 1 -- 172.21.15.112:0/1489232981 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f80cac00 con 0x7fa3f80c5d70 2024-04-25T01:28:54.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.150+0000 7fa3fc9c5640 1 --2- 172.21.15.112:0/1489232981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f80c5d70 0x7fa3f8156280 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80097a0 tx=0x7fa3e802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6fccdb3827f6fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:54.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.150+0000 7fa3effff640 1 -- 172.21.15.112:0/1489232981 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8037070 con 0x7fa3f80c5d70 2024-04-25T01:28:54.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.150+0000 7fa3effff640 1 -- 172.21.15.112:0/1489232981 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e802ae50 con 0x7fa3f80c5d70 2024-04-25T01:28:54.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.150+0000 7fa3effff640 1 -- 172.21.15.112:0/1489232981 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8032660 con 0x7fa3f80c5d70 2024-04-25T01:28:54.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.151+0000 7fa3ff42a640 1 -- 172.21.15.112:0/1489232981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f80c5d70 msgr2=0x7fa3f8156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:54.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.151+0000 7fa3ff42a640 1 --2- 172.21.15.112:0/1489232981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f80c5d70 0x7fa3f8156280 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80097a0 tx=0x7fa3e802a100 comp rx=0 tx=0).stop 2024-04-25T01:28:54.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.151+0000 7fa3ff42a640 1 -- 172.21.15.112:0/1489232981 shutdown_connections 2024-04-25T01:28:54.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.151+0000 7fa3ff42a640 1 --2- 172.21.15.112:0/1489232981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f80c5d70 0x7fa3f8156280 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:54.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.151+0000 7fa3ff42a640 1 --2- 172.21.15.112:0/1489232981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3f8152fd0 0x7fa3f80c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:54.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.151+0000 7fa3ff42a640 1 -- 172.21.15.112:0/1489232981 >> 172.21.15.112:0/1489232981 conn(0x7fa3f80acc70 msgr2=0x7fa3f80ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:54.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.151+0000 7fa3ff42a640 1 -- 172.21.15.112:0/1489232981 shutdown_connections 2024-04-25T01:28:54.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.151+0000 7fa3ff42a640 1 -- 172.21.15.112:0/1489232981 wait complete. 2024-04-25T01:28:54.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.152+0000 7fa3ff42a640 1 Processor -- start 2024-04-25T01:28:54.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.152+0000 7fa3ff42a640 1 -- start start 2024-04-25T01:28:54.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.153+0000 7fa3ff42a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3f80c5d70 0x7fa3f81513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:54.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.153+0000 7fa3ff42a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f8152fd0 0x7fa3f8158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:54.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.153+0000 7fa3ff42a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3f8156cf0 0x7fa3f8158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:54.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.153+0000 7fa3ff42a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f80caee0 con 0x7fa3f8152fd0 2024-04-25T01:28:54.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.153+0000 7fa3ff42a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f80c9660 con 0x7fa3f8156cf0 2024-04-25T01:28:54.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.153+0000 7fa3ff42a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f806c4b0 con 0x7fa3f80c5d70 2024-04-25T01:28:54.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.153+0000 7fa3fd9c7640 1 -- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3f8156cf0 msgr2=0x7fa3f8158cd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:28:54.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.153+0000 7fa3fc9c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f8152fd0 0x7fa3f8158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:54.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.153+0000 7fa3fd9c7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3f8156cf0 0x7fa3f8158cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:28:54.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.153+0000 7fa3fc9c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f8152fd0 0x7fa3f8158790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46572/0 (socket says 172.21.15.112:46572) 2024-04-25T01:28:54.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.153+0000 7fa3fc9c5640 1 -- 172.21.15.112:0/1766921899 learned_addr learned my addr 172.21.15.112:0/1766921899 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:28:54.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.153+0000 7fa3fd1c6640 1 --2- 172.21.15.112:0/1766921899 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3f80c5d70 0x7fa3f81513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:54.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.154+0000 7fa3fc9c5640 1 -- 172.21.15.112:0/1766921899 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3f80c5d70 msgr2=0x7fa3f81513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:54.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.154+0000 7fa3fc9c5640 1 --2- 172.21.15.112:0/1766921899 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3f80c5d70 0x7fa3f81513a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:54.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.154+0000 7fa3fc9c5640 1 -- 172.21.15.112:0/1766921899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3f8156cf0 msgr2=0x7fa3f8158cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:28:54.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.154+0000 7fa3fc9c5640 1 --2- 172.21.15.112:0/1766921899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3f8156cf0 0x7fa3f8158cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:54.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.154+0000 7fa3fc9c5640 1 -- 172.21.15.112:0/1766921899 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f81538e0 con 0x7fa3f8152fd0 2024-04-25T01:28:54.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.154+0000 7fa3fc9c5640 1 --2- 172.21.15.112:0/1766921899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f8152fd0 0x7fa3f8158790 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80098c0 tx=0x7fa3e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:54.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.154+0000 7fa3edffb640 1 -- 172.21.15.112:0/1766921899 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8041030 con 0x7fa3f8152fd0 2024-04-25T01:28:54.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.154+0000 7fa3edffb640 1 -- 172.21.15.112:0/1766921899 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e802ac50 con 0x7fa3f8152fd0 2024-04-25T01:28:54.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.154+0000 7fa3ff42a640 1 -- 172.21.15.112:0/1766921899 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f8155bb0 con 0x7fa3f8152fd0 2024-04-25T01:28:54.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.154+0000 7fa3edffb640 1 -- 172.21.15.112:0/1766921899 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8037040 con 0x7fa3f8152fd0 2024-04-25T01:28:54.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.154+0000 7fa3ff42a640 1 -- 172.21.15.112:0/1766921899 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f8153600 con 0x7fa3f8152fd0 2024-04-25T01:28:54.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.155+0000 7fa3ff42a640 1 -- 172.21.15.112:0/1766921899 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f80682a0 con 0x7fa3f8152fd0 2024-04-25T01:28:54.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.156+0000 7fa3edffb640 1 -- 172.21.15.112:0/1766921899 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa3e8044070 con 0x7fa3f8152fd0 2024-04-25T01:28:54.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.156+0000 7fa3edffb640 1 --2- 172.21.15.112:0/1766921899 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa3dc06bde0 0x7fa3dc06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:54.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.156+0000 7fa3edffb640 1 -- 172.21.15.112:0/1766921899 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fa3e80ab690 con 0x7fa3f8152fd0 2024-04-25T01:28:54.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.156+0000 7fa3fd1c6640 1 --2- 172.21.15.112:0/1766921899 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa3dc06bde0 0x7fa3dc06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:54.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.157+0000 7fa3fd1c6640 1 --2- 172.21.15.112:0/1766921899 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa3dc06bde0 0x7fa3dc06e2a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4009b40 tx=0x7fa3f4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:54.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.162+0000 7fa3edffb640 1 -- 172.21.15.112:0/1766921899 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa3e8079e50 con 0x7fa3f8152fd0 2024-04-25T01:28:54.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.468+0000 7fa3ff42a640 1 -- 172.21.15.112:0/1766921899 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa3f80c5ac0 con 0x7fa3f8152fd0 2024-04-25T01:28:54.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.469+0000 7fa3edffb640 1 -- 172.21.15.112:0/1766921899 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7fa3e8079750 con 0x7fa3f8152fd0 2024-04-25T01:28:54.447 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:28:54.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.471+0000 7fa3ff42a640 1 -- 172.21.15.112:0/1766921899 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa3dc06bde0 msgr2=0x7fa3dc06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:54.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.471+0000 7fa3ff42a640 1 --2- 172.21.15.112:0/1766921899 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa3dc06bde0 0x7fa3dc06e2a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4009b40 tx=0x7fa3f4009190 comp rx=0 tx=0).stop 2024-04-25T01:28:54.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.471+0000 7fa3ff42a640 1 -- 172.21.15.112:0/1766921899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f8152fd0 msgr2=0x7fa3f8158790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:54.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.471+0000 7fa3ff42a640 1 --2- 172.21.15.112:0/1766921899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f8152fd0 0x7fa3f8158790 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80098c0 tx=0x7fa3e80046c0 comp rx=0 tx=0).stop 2024-04-25T01:28:54.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.471+0000 7fa3ff42a640 1 -- 172.21.15.112:0/1766921899 shutdown_connections 2024-04-25T01:28:54.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.471+0000 7fa3ff42a640 1 --2- 172.21.15.112:0/1766921899 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa3dc06bde0 0x7fa3dc06e2a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:54.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.471+0000 7fa3ff42a640 1 --2- 172.21.15.112:0/1766921899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3f8156cf0 0x7fa3f8158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:54.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.471+0000 7fa3ff42a640 1 --2- 172.21.15.112:0/1766921899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f8152fd0 0x7fa3f8158790 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:54.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.471+0000 7fa3ff42a640 1 --2- 172.21.15.112:0/1766921899 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3f80c5d70 0x7fa3f81513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:54.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.471+0000 7fa3ff42a640 1 -- 172.21.15.112:0/1766921899 >> 172.21.15.112:0/1766921899 conn(0x7fa3f80acc70 msgr2=0x7fa3f80ad050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:54.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.471+0000 7fa3ff42a640 1 -- 172.21.15.112:0/1766921899 shutdown_connections 2024-04-25T01:28:54.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:54.471+0000 7fa3ff42a640 1 -- 172.21.15.112:0/1766921899 wait complete. 2024-04-25T01:28:54.718 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:28:54.721 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:28:57.461 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:28:57.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.594+0000 7fcc822b2640 1 Processor -- start 2024-04-25T01:28:57.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.595+0000 7fcc822b2640 1 -- start start 2024-04-25T01:28:57.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.595+0000 7fcc822b2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 0x7fcc7c0c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:57.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.595+0000 7fcc822b2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcc7c156cf0 0x7fcc7c06bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:57.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.595+0000 7fcc822b2640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc7c0c90e0 con 0x7fcc7c0c5d70 2024-04-25T01:28:57.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.595+0000 7fcc822b2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fcc7c0ca9b0 con 0x7fcc7c152fd0 2024-04-25T01:28:57.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.595+0000 7fcc822b2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcc7c0cac00 con 0x7fcc7c156cf0 2024-04-25T01:28:57.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.595+0000 7fcc8084f640 1 -- >> v2:172.21.15.187:3300/0 conn(0x7fcc7c156cf0 msgr2=0x7fcc7c06bda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:28:57.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.595+0000 7fcc8084f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcc7c156cf0 0x7fcc7c06bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:28:57.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.595+0000 7fcc7b7fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fcc7c0c5d70 0x7fcc7c156280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:43400/0 (socket says 172.21.15.112:43400) 2024-04-25T01:28:57.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.596+0000 7fcc7b7fe640 1 -- 172.21.15.112:0/1822631976 learned_addr learned my addr 172.21.15.112:0/1822631976 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:28:57.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.595+0000 7fcc7bfff640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 0x7fcc7c0c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:57.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.596+0000 7fcc7bfff640 1 -- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 msgr2=0x7fcc7c0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:28:57.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.596+0000 7fcc7bfff640 1 -- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 msgr2=0x7fcc7c0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:28:57.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.596+0000 7fcc7bfff640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 0x7fcc7c0c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:28:57.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.596+0000 7fcc7bfff640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 0x7fcc7c0c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:28:57.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.796+0000 7fcc8084f640 1 -- 172.21.15.112:0/1822631976 >> v2:172.21.15.187:3300/0 conn(0x7fcc7c156cf0 msgr2=0x7fcc7c06bda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:28:57.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.796+0000 7fcc8084f640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.187:3300/0 conn(0x7fcc7c156cf0 0x7fcc7c06bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:28:57.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.797+0000 7fcc7bfff640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 0x7fcc7c0c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:57.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.797+0000 7fcc7bfff640 1 -- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 msgr2=0x7fcc7c0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:28:57.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.797+0000 7fcc7bfff640 1 -- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 msgr2=0x7fcc7c0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:28:57.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.797+0000 7fcc7bfff640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 0x7fcc7c0c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:28:57.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:57.797+0000 7fcc7bfff640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 0x7fcc7c0c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:28:58.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:58.197+0000 7fcc8084f640 1 -- 172.21.15.112:0/1822631976 >> v2:172.21.15.187:3300/0 conn(0x7fcc7c156cf0 msgr2=0x7fcc7c06bda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:28:58.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:58.197+0000 7fcc8084f640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.187:3300/0 conn(0x7fcc7c156cf0 0x7fcc7c06bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:28:58.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:58.198+0000 7fcc7bfff640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 0x7fcc7c0c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:58.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:58.198+0000 7fcc7bfff640 1 -- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 msgr2=0x7fcc7c0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:28:58.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:58.198+0000 7fcc7bfff640 1 -- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 msgr2=0x7fcc7c0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:28:58.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:58.198+0000 7fcc7bfff640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 0x7fcc7c0c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:28:58.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:58.198+0000 7fcc7bfff640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 0x7fcc7c0c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:28:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:58.998+0000 7fcc8084f640 1 -- 172.21.15.112:0/1822631976 >> v2:172.21.15.187:3300/0 conn(0x7fcc7c156cf0 msgr2=0x7fcc7c06bda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:28:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:58.998+0000 7fcc8084f640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.187:3300/0 conn(0x7fcc7c156cf0 0x7fcc7c06bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:28:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:58.999+0000 7fcc7bfff640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 0x7fcc7c0c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:58.999+0000 7fcc7bfff640 1 -- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 msgr2=0x7fcc7c0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:28:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:58.999+0000 7fcc7bfff640 1 -- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 msgr2=0x7fcc7c0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:28:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:58.999+0000 7fcc7bfff640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 0x7fcc7c0c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:28:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:28:58.999+0000 7fcc7bfff640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 0x7fcc7c0c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:29:00.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:00.595+0000 7fcc79ffb640 1 -- 172.21.15.112:0/1822631976 >> v2:172.21.15.187:3300/0 conn(0x7fcc7c156cf0 msgr2=0x7fcc7c06bda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:29:00.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:00.595+0000 7fcc79ffb640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.187:3300/0 conn(0x7fcc7c156cf0 0x7fcc7c06bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:00.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:00.595+0000 7fcc79ffb640 1 -- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 msgr2=0x7fcc7c0c9b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:29:00.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:00.595+0000 7fcc79ffb640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc7c152fd0 0x7fcc7c0c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:00.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:00.595+0000 7fcc79ffb640 1 -- 172.21.15.112:0/1822631976 >> v1:172.21.15.112:6789/0 conn(0x7fcc7c0c5d70 legacy=0x7fcc7c156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:00.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:00.595+0000 7fcc79ffb640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 0x7fcc680097a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:00.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:00.595+0000 7fcc79ffb640 1 -- 172.21.15.112:0/1822631976 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc7c0c9380 con 0x7fcc68003690 2024-04-25T01:29:00.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:00.595+0000 7fcc79ffb640 1 -- 172.21.15.112:0/1822631976 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcc7c0ca9b0 con 0x7fcc680072e0 2024-04-25T01:29:00.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:00.596+0000 7fcc7bfff640 1 -- 172.21.15.112:0/1822631976 >> v1:172.21.15.187:6789/0 conn(0x7fcc68003690 legacy=0x7fcc68005b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.187:6789/0 2024-04-25T01:29:00.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:00.596+0000 7fcc7b7fe640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 0x7fcc680097a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:00.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:00.596+0000 7fcc7b7fe640 1 -- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 msgr2=0x7fcc680097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:29:00.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:00.596+0000 7fcc7b7fe640 1 -- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 msgr2=0x7fcc680097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:29:00.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:00.596+0000 7fcc7b7fe640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 0x7fcc680097a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:29:00.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:00.596+0000 7fcc7b7fe640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 0x7fcc680097a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:00.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:00.797+0000 7fcc7bfff640 1 -- 172.21.15.112:0/1822631976 >> v1:172.21.15.187:6789/0 conn(0x7fcc68003690 legacy=0x7fcc68005b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.187:6789/0 2024-04-25T01:29:00.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:00.797+0000 7fcc7b7fe640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 0x7fcc680097a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:00.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:00.797+0000 7fcc7b7fe640 1 -- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 msgr2=0x7fcc680097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:29:00.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:00.797+0000 7fcc7b7fe640 1 -- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 msgr2=0x7fcc680097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:29:00.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:00.797+0000 7fcc7b7fe640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 0x7fcc680097a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:29:00.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:00.797+0000 7fcc7b7fe640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 0x7fcc680097a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:29:01.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:01.198+0000 7fcc7bfff640 1 -- 172.21.15.112:0/1822631976 >> v1:172.21.15.187:6789/0 conn(0x7fcc68003690 legacy=0x7fcc68005b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.187:6789/0 2024-04-25T01:29:01.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:01.198+0000 7fcc7b7fe640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 0x7fcc680097a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:01.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:01.198+0000 7fcc7b7fe640 1 -- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 msgr2=0x7fcc680097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:29:01.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:01.198+0000 7fcc7b7fe640 1 -- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 msgr2=0x7fcc680097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:29:01.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:01.198+0000 7fcc7b7fe640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 0x7fcc680097a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:29:01.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:01.198+0000 7fcc7b7fe640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 0x7fcc680097a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:29:01.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:01.999+0000 7fcc7bfff640 1 -- 172.21.15.112:0/1822631976 >> v1:172.21.15.187:6789/0 conn(0x7fcc68003690 legacy=0x7fcc68005b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.187:6789/0 2024-04-25T01:29:01.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:01.999+0000 7fcc7b7fe640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 0x7fcc680097a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:01.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:01.999+0000 7fcc7b7fe640 1 -- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 msgr2=0x7fcc680097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:29:01.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:01.999+0000 7fcc7b7fe640 1 -- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 msgr2=0x7fcc680097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:29:01.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:01.999+0000 7fcc7b7fe640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 0x7fcc680097a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:29:01.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:01.999+0000 7fcc7b7fe640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 0x7fcc680097a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:29:03.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.596+0000 7fcc79ffb640 1 -- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 msgr2=0x7fcc680097a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:29:03.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.596+0000 7fcc79ffb640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc680072e0 0x7fcc680097a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:03.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.596+0000 7fcc79ffb640 1 -- 172.21.15.112:0/1822631976 >> v1:172.21.15.187:6789/0 conn(0x7fcc68003690 legacy=0x7fcc68005b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:29:03.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.596+0000 7fcc8084f640 1 -- 172.21.15.112:0/1822631976 reap_dead start 2024-04-25T01:29:03.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.596+0000 7fcc79ffb640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc68009ed0 0x7fcc680097a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:03.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.596+0000 7fcc79ffb640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc6800a3a0 0x7fcc6800a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:03.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.596+0000 7fcc79ffb640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.187:3300/0 conn(0x7fcc68005a80 0x7fcc68006240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:03.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.596+0000 7fcc79ffb640 1 -- 172.21.15.112:0/1822631976 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fcc7c0ca9b0 con 0x7fcc6800a3a0 2024-04-25T01:29:03.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.596+0000 7fcc79ffb640 1 -- 172.21.15.112:0/1822631976 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcc7c0cac00 con 0x7fcc68005a80 2024-04-25T01:29:03.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.596+0000 7fcc79ffb640 1 -- 172.21.15.112:0/1822631976 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcc68006780 con 0x7fcc68009ed0 2024-04-25T01:29:03.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.596+0000 7fcc8084f640 1 -- 172.21.15.112:0/1822631976 >> v2:172.21.15.187:3300/0 conn(0x7fcc68005a80 msgr2=0x7fcc68006240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:29:03.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.596+0000 7fcc8084f640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.187:3300/0 conn(0x7fcc68005a80 0x7fcc68006240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:03.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.596+0000 7fcc7b7fe640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.112:3300/0 conn(0x7fcc6800a3a0 0x7fcc6800a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:03.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.596+0000 7fcc7bfff640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc68009ed0 0x7fcc680097a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:03.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.596+0000 7fcc7b7fe640 1 -- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc68009ed0 msgr2=0x7fcc680097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:29:03.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.596+0000 7fcc7b7fe640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc68009ed0 0x7fcc680097a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:03.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.596+0000 7fcc7b7fe640 1 -- 172.21.15.112:0/1822631976 >> v2:172.21.15.187:3300/0 conn(0x7fcc68005a80 msgr2=0x7fcc68006240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:29:03.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.596+0000 7fcc7b7fe640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.187:3300/0 conn(0x7fcc68005a80 0x7fcc68006240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:03.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.596+0000 7fcc7b7fe640 1 -- 172.21.15.112:0/1822631976 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6800afd0 con 0x7fcc6800a3a0 2024-04-25T01:29:03.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.597+0000 7fcc7b7fe640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc6800a3a0 0x7fcc6800a800 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800ae30 tx=0x7fcc7002ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0f63c9d283a436 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:03.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.597+0000 7fcc7affd640 1 -- 172.21.15.112:0/1822631976 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc70034070 con 0x7fcc6800a3a0 2024-04-25T01:29:03.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.597+0000 7fcc7affd640 1 -- 172.21.15.112:0/1822631976 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc7002c6b0 con 0x7fcc6800a3a0 2024-04-25T01:29:03.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.597+0000 7fcc7affd640 1 -- 172.21.15.112:0/1822631976 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7002caa0 con 0x7fcc6800a3a0 2024-04-25T01:29:03.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.598+0000 7fcc822b2640 1 -- 172.21.15.112:0/1822631976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc6800a3a0 msgr2=0x7fcc6800a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:03.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.598+0000 7fcc822b2640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc6800a3a0 0x7fcc6800a800 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800ae30 tx=0x7fcc7002ab80 comp rx=0 tx=0).stop 2024-04-25T01:29:03.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.598+0000 7fcc822b2640 1 -- 172.21.15.112:0/1822631976 shutdown_connections 2024-04-25T01:29:03.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.598+0000 7fcc822b2640 1 --2- 172.21.15.112:0/1822631976 >> v2:172.21.15.187:3300/0 conn(0x7fcc68005a80 0x7fcc68006240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:03.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.598+0000 7fcc822b2640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc6800a3a0 0x7fcc6800a800 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:03.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.598+0000 7fcc822b2640 1 --2- 172.21.15.112:0/1822631976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc68009ed0 0x7fcc680097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:03.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.598+0000 7fcc822b2640 1 -- 172.21.15.112:0/1822631976 >> 172.21.15.112:0/1822631976 conn(0x7fcc7c0acc70 msgr2=0x7fcc7c0ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:03.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.598+0000 7fcc822b2640 1 -- 172.21.15.112:0/1822631976 shutdown_connections 2024-04-25T01:29:03.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.598+0000 7fcc822b2640 1 -- 172.21.15.112:0/1822631976 wait complete. 2024-04-25T01:29:03.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.599+0000 7fcc822b2640 1 Processor -- start 2024-04-25T01:29:03.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.599+0000 7fcc822b2640 1 -- start start 2024-04-25T01:29:03.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.600+0000 7fcc822b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc68005a80 0x7fcc7c158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:03.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.600+0000 7fcc822b2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc68009ed0 0x7fcc7c152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:03.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.600+0000 7fcc822b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc6800a3a0 0x7fcc7c14ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:03.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.600+0000 7fcc822b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c157100 con 0x7fcc68005a80 2024-04-25T01:29:03.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.600+0000 7fcc822b2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c0caee0 con 0x7fcc68009ed0 2024-04-25T01:29:03.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.600+0000 7fcc822b2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcc7c157280 con 0x7fcc6800a3a0 2024-04-25T01:29:03.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.600+0000 7fcc7b7fe640 1 -- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc68009ed0 msgr2=0x7fcc7c152000 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:29:03.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.600+0000 7fcc7b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc68009ed0 0x7fcc7c152000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:03.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.600+0000 7fcc7bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc68005a80 0x7fcc7c158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:03.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.600+0000 7fcc7bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc68005a80 0x7fcc7c158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41882/0 (socket says 172.21.15.112:41882) 2024-04-25T01:29:03.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.600+0000 7fcc7bfff640 1 -- 172.21.15.112:0/1530822445 learned_addr learned my addr 172.21.15.112:0/1530822445 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:03.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.600+0000 7fcc8084f640 1 --2- 172.21.15.112:0/1530822445 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc6800a3a0 0x7fcc7c14ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:03.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.600+0000 7fcc7bfff640 1 -- 172.21.15.112:0/1530822445 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc6800a3a0 msgr2=0x7fcc7c14ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:03.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.600+0000 7fcc7bfff640 1 --2- 172.21.15.112:0/1530822445 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc6800a3a0 0x7fcc7c14ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:03.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.600+0000 7fcc7bfff640 1 -- 172.21.15.112:0/1530822445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc68009ed0 msgr2=0x7fcc7c152000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:29:03.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.600+0000 7fcc7bfff640 1 --2- 172.21.15.112:0/1530822445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc68009ed0 0x7fcc7c152000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:03.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.600+0000 7fcc7bfff640 1 -- 172.21.15.112:0/1530822445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7c158550 con 0x7fcc68005a80 2024-04-25T01:29:03.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.601+0000 7fcc7bfff640 1 --2- 172.21.15.112:0/1530822445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc68005a80 0x7fcc7c158010 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0166d0 tx=0x7fcc6c016ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:03.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.601+0000 7fcc78ff9640 1 -- 172.21.15.112:0/1530822445 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c014850 con 0x7fcc68005a80 2024-04-25T01:29:03.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.601+0000 7fcc78ff9640 1 -- 172.21.15.112:0/1530822445 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc6c0149f0 con 0x7fcc68005a80 2024-04-25T01:29:03.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.601+0000 7fcc822b2640 1 -- 172.21.15.112:0/1530822445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc7c14e290 con 0x7fcc68005a80 2024-04-25T01:29:03.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.601+0000 7fcc78ff9640 1 -- 172.21.15.112:0/1530822445 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c014dc0 con 0x7fcc68005a80 2024-04-25T01:29:03.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.601+0000 7fcc822b2640 1 -- 172.21.15.112:0/1530822445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc7c14e850 con 0x7fcc68005a80 2024-04-25T01:29:03.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.602+0000 7fcc822b2640 1 -- 172.21.15.112:0/1530822445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc7c0c8d10 con 0x7fcc68005a80 2024-04-25T01:29:03.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.602+0000 7fcc78ff9640 1 -- 172.21.15.112:0/1530822445 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcc6c029440 con 0x7fcc68005a80 2024-04-25T01:29:03.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.603+0000 7fcc78ff9640 1 --2- 172.21.15.112:0/1530822445 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcc6806db80 0x7fcc68070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:03.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.603+0000 7fcc78ff9640 1 -- 172.21.15.112:0/1530822445 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fcc6c08c370 con 0x7fcc68005a80 2024-04-25T01:29:03.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.603+0000 7fcc7b7fe640 1 --2- 172.21.15.112:0/1530822445 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcc6806db80 0x7fcc68070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:03.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.603+0000 7fcc7b7fe640 1 --2- 172.21.15.112:0/1530822445 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcc6806db80 0x7fcc68070040 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcc70036000 tx=0x7fcc7002f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:03.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.607+0000 7fcc78ff9640 1 -- 172.21.15.112:0/1530822445 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcc7c0c8d10 con 0x7fcc68005a80 2024-04-25T01:29:03.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.908+0000 7fcc822b2640 1 -- 172.21.15.112:0/1530822445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fcc7c067da0 con 0x7fcc68005a80 2024-04-25T01:29:03.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.908+0000 7fcc78ff9640 1 -- 172.21.15.112:0/1530822445 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7fcc7c067da0 con 0x7fcc68005a80 2024-04-25T01:29:03.890 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":24,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:29:03.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.910+0000 7fcc822b2640 1 -- 172.21.15.112:0/1530822445 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcc6806db80 msgr2=0x7fcc68070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:03.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.910+0000 7fcc822b2640 1 --2- 172.21.15.112:0/1530822445 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcc6806db80 0x7fcc68070040 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcc70036000 tx=0x7fcc7002f000 comp rx=0 tx=0).stop 2024-04-25T01:29:03.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.910+0000 7fcc822b2640 1 -- 172.21.15.112:0/1530822445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc68005a80 msgr2=0x7fcc7c158010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:03.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.910+0000 7fcc822b2640 1 --2- 172.21.15.112:0/1530822445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc68005a80 0x7fcc7c158010 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0166d0 tx=0x7fcc6c016ad0 comp rx=0 tx=0).stop 2024-04-25T01:29:03.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.911+0000 7fcc822b2640 1 -- 172.21.15.112:0/1530822445 shutdown_connections 2024-04-25T01:29:03.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.911+0000 7fcc822b2640 1 --2- 172.21.15.112:0/1530822445 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcc6806db80 0x7fcc68070040 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:03.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.911+0000 7fcc822b2640 1 --2- 172.21.15.112:0/1530822445 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc6800a3a0 0x7fcc7c14ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:03.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.911+0000 7fcc822b2640 1 --2- 172.21.15.112:0/1530822445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc68009ed0 0x7fcc7c152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:03.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.911+0000 7fcc822b2640 1 --2- 172.21.15.112:0/1530822445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc68005a80 0x7fcc7c158010 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:03.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.911+0000 7fcc822b2640 1 -- 172.21.15.112:0/1530822445 >> 172.21.15.112:0/1530822445 conn(0x7fcc7c0acc70 msgr2=0x7fcc7c158d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:03.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.911+0000 7fcc822b2640 1 -- 172.21.15.112:0/1530822445 shutdown_connections 2024-04-25T01:29:03.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:03.911+0000 7fcc822b2640 1 -- 172.21.15.112:0/1530822445 wait complete. 2024-04-25T01:29:03.901 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T01:29:03.901 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-04-25T01:29:03.901 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:29:04.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.028+0000 7f0a28cbd640 1 Processor -- start 2024-04-25T01:29:04.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.028+0000 7f0a28cbd640 1 -- start start 2024-04-25T01:29:04.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.029+0000 7f0a28cbd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a241596a0 0x7f0a24152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:04.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.029+0000 7f0a28cbd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0a240c72b0 0x7f0a24160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:04.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.029+0000 7f0a28cbd640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a24159b90 con 0x7f0a241592c0 2024-04-25T01:29:04.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.029+0000 7f0a28cbd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f0a240c6e20 con 0x7f0a240c72b0 2024-04-25T01:29:04.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.029+0000 7f0a28cbd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0a2406c970 con 0x7f0a241596a0 2024-04-25T01:29:04.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.029+0000 7f0a2259c640 1 -- >> v1:172.21.15.187:6789/0 conn(0x7f0a241592c0 legacy=0x7f0a240c9070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.187:6789/0 2024-04-25T01:29:04.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.030+0000 7f0a22d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0a240c72b0 0x7f0a24160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:04.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.030+0000 7f0a22d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0a240c72b0 0x7f0a24160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41896/0 (socket says 172.21.15.112:41896) 2024-04-25T01:29:04.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.030+0000 7f0a22d9d640 1 -- 172.21.15.112:0/223344280 learned_addr learned my addr 172.21.15.112:0/223344280 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:04.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.030+0000 7f0a21d9b640 1 --2- 172.21.15.112:0/223344280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a241596a0 0x7f0a24152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:04.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.030+0000 7f0a22d9d640 1 -- 172.21.15.112:0/223344280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a241596a0 msgr2=0x7f0a24152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:29:04.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.030+0000 7f0a22d9d640 1 --2- 172.21.15.112:0/223344280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a241596a0 0x7f0a24152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:04.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.030+0000 7f0a22d9d640 1 -- 172.21.15.112:0/223344280 >> v1:172.21.15.187:6789/0 conn(0x7f0a241592c0 legacy=0x7f0a240c9070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:29:04.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.030+0000 7f0a22d9d640 1 -- 172.21.15.112:0/223344280 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2406cc50 con 0x7f0a240c72b0 2024-04-25T01:29:04.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.031+0000 7f0a22d9d640 1 --2- 172.21.15.112:0/223344280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a240c72b0 0x7f0a24160380 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0a240aac40 tx=0x7f0a1c02a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9b5843c112c224 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:04.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.031+0000 7f0a2159a640 1 -- 172.21.15.112:0/223344280 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c036070 con 0x7f0a240c72b0 2024-04-25T01:29:04.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.031+0000 7f0a2159a640 1 -- 172.21.15.112:0/223344280 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a1c031600 con 0x7f0a240c72b0 2024-04-25T01:29:04.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.031+0000 7f0a2159a640 1 -- 172.21.15.112:0/223344280 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c0319d0 con 0x7f0a240c72b0 2024-04-25T01:29:04.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.031+0000 7f0a28cbd640 1 -- 172.21.15.112:0/223344280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a240c72b0 msgr2=0x7f0a24160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:04.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.031+0000 7f0a28cbd640 1 --2- 172.21.15.112:0/223344280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a240c72b0 0x7f0a24160380 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0a240aac40 tx=0x7f0a1c02a590 comp rx=0 tx=0).stop 2024-04-25T01:29:04.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.032+0000 7f0a28cbd640 1 -- 172.21.15.112:0/223344280 shutdown_connections 2024-04-25T01:29:04.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.032+0000 7f0a28cbd640 1 --2- 172.21.15.112:0/223344280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a240c72b0 0x7f0a24160380 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:04.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.032+0000 7f0a28cbd640 1 --2- 172.21.15.112:0/223344280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a241596a0 0x7f0a24152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:04.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.032+0000 7f0a28cbd640 1 -- 172.21.15.112:0/223344280 >> 172.21.15.112:0/223344280 conn(0x7f0a240cb170 msgr2=0x7f0a240cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:04.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.032+0000 7f0a28cbd640 1 -- 172.21.15.112:0/223344280 shutdown_connections 2024-04-25T01:29:04.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.032+0000 7f0a28cbd640 1 -- 172.21.15.112:0/223344280 wait complete. 2024-04-25T01:29:04.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.033+0000 7f0a28cbd640 1 Processor -- start 2024-04-25T01:29:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.033+0000 7f0a28cbd640 1 -- start start 2024-04-25T01:29:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.034+0000 7f0a28cbd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a240c72b0 0x7f0a2414bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.034+0000 7f0a28cbd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a241592c0 0x7f0a24156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.034+0000 7f0a28cbd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a241596a0 0x7f0a24157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.034+0000 7f0a28cbd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2406c510 con 0x7f0a240c72b0 2024-04-25T01:29:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.034+0000 7f0a28cbd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2406c390 con 0x7f0a241596a0 2024-04-25T01:29:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.034+0000 7f0a28cbd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0a2406db20 con 0x7f0a241592c0 2024-04-25T01:29:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.034+0000 7f0a22d9d640 1 -- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a241596a0 msgr2=0x7f0a24157390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:29:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.034+0000 7f0a2259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a240c72b0 0x7f0a2414bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:04.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.034+0000 7f0a22d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a241596a0 0x7f0a24157390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:04.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.034+0000 7f0a2259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a240c72b0 0x7f0a2414bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41910/0 (socket says 172.21.15.112:41910) 2024-04-25T01:29:04.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.034+0000 7f0a2259c640 1 -- 172.21.15.112:0/1824007805 learned_addr learned my addr 172.21.15.112:0/1824007805 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:04.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.034+0000 7f0a21d9b640 1 --2- 172.21.15.112:0/1824007805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a241592c0 0x7f0a24156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:04.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.034+0000 7f0a2259c640 1 -- 172.21.15.112:0/1824007805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a241592c0 msgr2=0x7f0a24156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:04.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.034+0000 7f0a2259c640 1 --2- 172.21.15.112:0/1824007805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a241592c0 0x7f0a24156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:04.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.034+0000 7f0a2259c640 1 -- 172.21.15.112:0/1824007805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a241596a0 msgr2=0x7f0a24157390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:29:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.034+0000 7f0a2259c640 1 --2- 172.21.15.112:0/1824007805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a241596a0 0x7f0a24157390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.034+0000 7f0a2259c640 1 -- 172.21.15.112:0/1824007805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2414a6a0 con 0x7f0a240c72b0 2024-04-25T01:29:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.035+0000 7f0a2259c640 1 --2- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a240c72b0 0x7f0a2414bee0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400b550 tx=0x7f0a1400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.035+0000 7f0a12ffd640 1 -- 172.21.15.112:0/1824007805 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14003d70 con 0x7f0a240c72b0 2024-04-25T01:29:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.035+0000 7f0a12ffd640 1 -- 172.21.15.112:0/1824007805 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a14015070 con 0x7f0a240c72b0 2024-04-25T01:29:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.035+0000 7f0a12ffd640 1 -- 172.21.15.112:0/1824007805 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a140052b0 con 0x7f0a240c72b0 2024-04-25T01:29:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.035+0000 7f0a28cbd640 1 -- 172.21.15.112:0/1824007805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2414a9b0 con 0x7f0a240c72b0 2024-04-25T01:29:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.035+0000 7f0a28cbd640 1 -- 172.21.15.112:0/1824007805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a2414af70 con 0x7f0a240c72b0 2024-04-25T01:29:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.036+0000 7f0a28cbd640 1 --2- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a24156700 0x7f0a24146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.036+0000 7f0a28cbd640 1 -- 172.21.15.112:0/1824007805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241578d0 con 0x7f0a24156700 2024-04-25T01:29:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.036+0000 7f0a21d9b640 1 --2- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a24156700 0x7f0a24146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.036+0000 7f0a21d9b640 1 -- 172.21.15.112:0/1824007805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a24075980 con 0x7f0a24156700 2024-04-25T01:29:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.036+0000 7f0a21d9b640 1 --2- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a24156700 0x7f0a24146ff0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0a24155c50 tx=0x7f0a18011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.036+0000 7f0a12ffd640 1 -- 172.21.15.112:0/1824007805 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18019070 con 0x7f0a24156700 2024-04-25T01:29:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.036+0000 7f0a12ffd640 1 -- 172.21.15.112:0/1824007805 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0a14005450 con 0x7f0a240c72b0 2024-04-25T01:29:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.037+0000 7f0a12ffd640 1 --2- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0a0406bd30 0x7f0a0406e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.037+0000 7f0a12ffd640 1 -- 172.21.15.112:0/1824007805 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f0a1408aed0 con 0x7f0a240c72b0 2024-04-25T01:29:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.037+0000 7f0a22d9d640 1 --2- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0a0406bd30 0x7f0a0406e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.037+0000 7f0a12ffd640 1 -- 172.21.15.112:0/1824007805 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0a18004240 con 0x7f0a24156700 2024-04-25T01:29:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.037+0000 7f0a12ffd640 1 -- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a24156700 msgr2=0x7f0a24146ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.037+0000 7f0a12ffd640 1 --2- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a24156700 0x7f0a24146ff0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0a24155c50 tx=0x7f0a18011cf0 comp rx=0 tx=0).stop 2024-04-25T01:29:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.037+0000 7f0a12ffd640 1 -- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a24156700 msgr2=0x7f0a24146ff0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:29:04.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.037+0000 7f0a12ffd640 1 --2- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a24156700 0x7f0a24146ff0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:04.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.037+0000 7f0a22d9d640 1 --2- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0a0406bd30 0x7f0a0406e1f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c033000 tx=0x7f0a1c009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:04.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.043+0000 7f0a28cbd640 1 --2- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a24147530 0x7f0a24147910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:04.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.043+0000 7f0a28cbd640 1 -- 172.21.15.112:0/1824007805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a24147e50 con 0x7f0a24147530 2024-04-25T01:29:04.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.044+0000 7f0a21d9b640 1 --2- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a24147530 0x7f0a24147910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:04.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.044+0000 7f0a21d9b640 1 -- 172.21.15.112:0/1824007805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0a2406bdc0 con 0x7f0a24147530 2024-04-25T01:29:04.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.044+0000 7f0a21d9b640 1 --2- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a24147530 0x7f0a24147910 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0a24155c50 tx=0x7f0a180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:04.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.044+0000 7f0a12ffd640 1 -- 172.21.15.112:0/1824007805 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18003df0 con 0x7f0a24147530 2024-04-25T01:29:04.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.044+0000 7f0a12ffd640 1 -- 172.21.15.112:0/1824007805 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5060 (secure 0 0 0) 0x7f0a18003df0 con 0x7f0a24147530 2024-04-25T01:29:04.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.044+0000 7f0a12ffd640 1 -- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a24147530 msgr2=0x7f0a24147910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:04.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.044+0000 7f0a12ffd640 1 --2- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a24147530 0x7f0a24147910 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0a24155c50 tx=0x7f0a180046c0 comp rx=0 tx=0).stop 2024-04-25T01:29:04.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.044+0000 7f0a12ffd640 1 -- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a24147530 msgr2=0x7f0a24147910 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:29:04.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.044+0000 7f0a12ffd640 1 --2- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a24147530 0x7f0a24147910 secure :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0a24155c50 tx=0x7f0a180046c0 comp rx=0 tx=0).stop 2024-04-25T01:29:04.024 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:29:04.024 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:29:04.024 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:29:04.025 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:29:04.025 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 24, 2024-04-25T01:29:04.025 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 215672, 2024-04-25T01:29:04.025 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:29:04.025 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:29:04.025 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:29:04.025 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:04.025 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:29:04.025 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:29:04.025 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:29:04.025 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:29:04.025 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:04.025 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:04.025 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:04.026 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:04.026 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:04.026 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:04.026 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:04.026 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:04.026 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:04.026 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:04.026 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:04.026 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:04.026 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:29:04.026 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:29:04.026 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:04.026 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:04.026 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:04.027 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:04.027 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:04.027 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:04.027 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:04.027 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:04.027 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:04.027 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:04.027 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:04.027 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:04.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.027 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:29:04.027 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:29:04.027 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:29:04.027 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:29:04.028 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:29:04.028 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:29:04.028 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:29:04.028 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:29:04.028 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:29:04.028 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:29:04.028 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:29:04.028 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:29:04.028 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:29:04.028 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:29:04.028 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:29:04.028 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:29:04.028 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:29:04.028 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:04.028 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:04.029 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:04.029 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:04.029 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:04.029 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:04.029 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:04.029 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:04.029 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:04.029 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:04.029 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:04.029 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:04.029 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:29:04.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.029 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:29:04.029 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.029 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:29:04.030 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:29:04.030 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:04.030 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:04.030 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.030 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:04.030 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:29:04.030 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:04.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.030 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.030 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:04.030 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:29:04.030 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:04.030 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:04.031 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:04.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.031 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:29:04.031 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:29:04.031 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:04.031 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:04.031 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:04.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.031 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.031 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:29:04.031 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:29:04.031 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:04.031 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:04.031 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.031 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:04.032 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:29:04.032 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:04.032 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.032 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.032 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:04.032 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:29:04.032 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:04.032 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:04.032 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:04.032 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.032 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:29:04.032 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:29:04.032 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:04.032 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:04.032 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:04.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.033 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.033 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:29:04.033 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:29:04.033 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:04.033 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:04.033 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.033 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:04.033 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:29:04.033 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:04.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.033 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.033 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:04.033 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:29:04.034 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:04.034 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:04.034 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:04.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.034 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:29:04.034 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:29:04.034 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:04.034 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:04.034 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:04.034 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:04.034 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:04.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.034 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:29:04.034 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:29:04.035 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.035 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:04.035 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:04.035 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:29:04.035 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:04.035 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:04.035 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:29:04.035 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.035 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:04.035 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:04.035 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:29:04.035 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:04.035 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:04.035 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:29:04.035 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.036 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:04.036 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:04.036 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 7 2024-04-25T01:29:04.036 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:04.036 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:04.036 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:29:04.036 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.036 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:04.036 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:04.036 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 8 2024-04-25T01:29:04.036 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:04.036 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:04.036 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:29:04.036 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.036 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:04.037 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:04.037 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:29:04.037 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:04.037 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:04.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.037 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:29:04.037 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:29:04.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.045+0000 7f0a28cbd640 1 -- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0a0406bd30 msgr2=0x7f0a0406e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:04.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.045+0000 7f0a28cbd640 1 --2- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0a0406bd30 0x7f0a0406e1f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c033000 tx=0x7f0a1c009000 comp rx=0 tx=0).stop 2024-04-25T01:29:04.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.045+0000 7f0a28cbd640 1 -- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a240c72b0 msgr2=0x7f0a2414bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:04.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.045+0000 7f0a28cbd640 1 --2- 172.21.15.112:0/1824007805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a240c72b0 0x7f0a2414bee0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400b550 tx=0x7f0a1400ba10 comp rx=0 tx=0).stop 2024-04-25T01:29:04.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.045+0000 7f0a22d9d640 1 -- 172.21.15.112:0/1824007805 reap_dead start 2024-04-25T01:29:04.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.045+0000 7f0a22d9d640 1 -- 172.21.15.112:0/1824007805 reap_dead start 2024-04-25T01:29:04.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.045+0000 7f0a28cbd640 1 -- 172.21.15.112:0/1824007805 shutdown_connections 2024-04-25T01:29:04.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.045+0000 7f0a28cbd640 1 -- 172.21.15.112:0/1824007805 >> 172.21.15.112:0/1824007805 conn(0x7f0a240cb170 msgr2=0x7f0a240aac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:04.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.045+0000 7f0a28cbd640 1 -- 172.21.15.112:0/1824007805 shutdown_connections 2024-04-25T01:29:04.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.045+0000 7f0a28cbd640 1 -- 172.21.15.112:0/1824007805 wait complete. 2024-04-25T01:29:04.040 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:29:04.040 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-04-25T01:29:04.040 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:29:04.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.165+0000 7f3f66a2a640 1 Processor -- start 2024-04-25T01:29:04.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.165+0000 7f3f66a2a640 1 -- start start 2024-04-25T01:29:04.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.166+0000 7f3f66a2a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3f600c1c00 0x7f3f60072740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:04.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.166+0000 7f3f66a2a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3f600c2160 0x7f3f6006c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:04.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.166+0000 7f3f66a2a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f600c2920 con 0x7f3f6006c170 2024-04-25T01:29:04.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.166+0000 7f3f66a2a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f3f60152890 con 0x7f3f600c2160 2024-04-25T01:29:04.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.166+0000 7f3f66a2a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3f60152b70 con 0x7f3f600c1c00 2024-04-25T01:29:04.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.166+0000 7f3f5f7fe640 1 -- >> v2:172.21.15.187:3300/0 conn(0x7f3f600c1c00 msgr2=0x7f3f60072740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:29:04.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.166+0000 7f3f5f7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3f600c1c00 0x7f3f60072740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:04.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.166+0000 7f3f64fc7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3f600c2160 0x7f3f6006c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:04.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.166+0000 7f3f5ffff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3f6006c170 0x7f3f60072030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:43404/0 (socket says 172.21.15.112:43404) 2024-04-25T01:29:04.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.166+0000 7f3f5ffff640 1 -- 172.21.15.112:0/3290468739 learned_addr learned my addr 172.21.15.112:0/3290468739 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:29:04.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.167+0000 7f3f64fc7640 1 -- 172.21.15.112:0/3290468739 >> v2:172.21.15.187:3300/0 conn(0x7f3f600c1c00 msgr2=0x7f3f60072740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:29:04.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.167+0000 7f3f64fc7640 1 --2- 172.21.15.112:0/3290468739 >> v2:172.21.15.187:3300/0 conn(0x7f3f600c1c00 0x7f3f60072740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:04.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.167+0000 7f3f64fc7640 1 -- 172.21.15.112:0/3290468739 >> v1:172.21.15.112:6789/0 conn(0x7f3f6006c170 legacy=0x7f3f60072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:04.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.167+0000 7f3f64fc7640 1 -- 172.21.15.112:0/3290468739 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f60152e50 con 0x7f3f600c2160 2024-04-25T01:29:04.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.167+0000 7f3f5effd640 1 -- 172.21.15.112:0/3290468739 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2933902990 0 0) 0x7f3f600c2920 con 0x7f3f6006c170 2024-04-25T01:29:04.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.167+0000 7f3f64fc7640 1 --2- 172.21.15.112:0/3290468739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f600c2160 0x7f3f6006c550 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3f600710b0 tx=0x7f3f5802c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea35f7bf51982b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:04.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.167+0000 7f3f5effd640 1 -- 172.21.15.112:0/3290468739 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f58033640 con 0x7f3f600c2160 2024-04-25T01:29:04.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.167+0000 7f3f5effd640 1 -- 172.21.15.112:0/3290468739 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f58038070 con 0x7f3f600c2160 2024-04-25T01:29:04.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.168+0000 7f3f5effd640 1 -- 172.21.15.112:0/3290468739 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f58033a00 con 0x7f3f600c2160 2024-04-25T01:29:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.168+0000 7f3f66a2a640 1 -- 172.21.15.112:0/3290468739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f600c2160 msgr2=0x7f3f6006c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.168+0000 7f3f66a2a640 1 --2- 172.21.15.112:0/3290468739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f600c2160 0x7f3f6006c550 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3f600710b0 tx=0x7f3f5802c8b0 comp rx=0 tx=0).stop 2024-04-25T01:29:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.168+0000 7f3f66a2a640 1 -- 172.21.15.112:0/3290468739 shutdown_connections 2024-04-25T01:29:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.168+0000 7f3f66a2a640 1 --2- 172.21.15.112:0/3290468739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f600c2160 0x7f3f6006c550 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.168+0000 7f3f66a2a640 1 --2- 172.21.15.112:0/3290468739 >> v2:172.21.15.187:3300/0 conn(0x7f3f600c1c00 0x7f3f60072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.168+0000 7f3f66a2a640 1 -- 172.21.15.112:0/3290468739 >> 172.21.15.112:0/3290468739 conn(0x7f3f600c4260 msgr2=0x7f3f600c4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.168+0000 7f3f66a2a640 1 -- 172.21.15.112:0/3290468739 shutdown_connections 2024-04-25T01:29:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.168+0000 7f3f66a2a640 1 -- 172.21.15.112:0/3290468739 wait complete. 2024-04-25T01:29:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.169+0000 7f3f66a2a640 1 Processor -- start 2024-04-25T01:29:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.170+0000 7f3f66a2a640 1 -- start start 2024-04-25T01:29:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.170+0000 7f3f66a2a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f6006c170 0x7f3f6014e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.170+0000 7f3f66a2a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f600c1c00 0x7f3f601de280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.170+0000 7f3f66a2a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f600c2160 0x7f3f601eb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.170+0000 7f3f66a2a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f600c2620 con 0x7f3f600c2160 2024-04-25T01:29:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.170+0000 7f3f66a2a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3f600c4000 con 0x7f3f600c1c00 2024-04-25T01:29:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.170+0000 7f3f66a2a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f60153650 con 0x7f3f6006c170 2024-04-25T01:29:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.170+0000 7f3f5f7fe640 1 -- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f600c1c00 msgr2=0x7f3f601de280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:29:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.170+0000 7f3f5f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f600c1c00 0x7f3f601de280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.170+0000 7f3f64fc7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f600c2160 0x7f3f601eb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:04.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.170+0000 7f3f64fc7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f600c2160 0x7f3f601eb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41966/0 (socket says 172.21.15.112:41966) 2024-04-25T01:29:04.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.170+0000 7f3f64fc7640 1 -- 172.21.15.112:0/2164007391 learned_addr learned my addr 172.21.15.112:0/2164007391 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:04.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.170+0000 7f3f5ffff640 1 --2- 172.21.15.112:0/2164007391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f6006c170 0x7f3f6014e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:04.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.171+0000 7f3f64fc7640 1 -- 172.21.15.112:0/2164007391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f6006c170 msgr2=0x7f3f6014e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:04.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.171+0000 7f3f64fc7640 1 --2- 172.21.15.112:0/2164007391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f6006c170 0x7f3f6014e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:04.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.171+0000 7f3f64fc7640 1 -- 172.21.15.112:0/2164007391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f600c1c00 msgr2=0x7f3f601de280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:29:04.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.171+0000 7f3f64fc7640 1 --2- 172.21.15.112:0/2164007391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f600c1c00 0x7f3f601de280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:04.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.171+0000 7f3f64fc7640 1 -- 172.21.15.112:0/2164007391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f60153e90 con 0x7f3f600c2160 2024-04-25T01:29:04.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.171+0000 7f3f64fc7640 1 --2- 172.21.15.112:0/2164007391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f600c2160 0x7f3f601eb340 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009e80 tx=0x7f3f58004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:04.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.171+0000 7f3f5cff9640 1 -- 172.21.15.112:0/2164007391 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5803c800 con 0x7f3f600c2160 2024-04-25T01:29:04.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.171+0000 7f3f5cff9640 1 -- 172.21.15.112:0/2164007391 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f5803c9a0 con 0x7f3f600c2160 2024-04-25T01:29:04.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.171+0000 7f3f66a2a640 1 -- 172.21.15.112:0/2164007391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f60151160 con 0x7f3f600c2160 2024-04-25T01:29:04.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.171+0000 7f3f5cff9640 1 -- 172.21.15.112:0/2164007391 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5803cd70 con 0x7f3f600c2160 2024-04-25T01:29:04.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.171+0000 7f3f66a2a640 1 -- 172.21.15.112:0/2164007391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f60151720 con 0x7f3f600c2160 2024-04-25T01:29:04.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.172+0000 7f3f66a2a640 1 --2- 172.21.15.112:0/2164007391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f601de7c0 0x7f3f601eb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:04.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.172+0000 7f3f66a2a640 1 -- 172.21.15.112:0/2164007391 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f60075980 con 0x7f3f601de7c0 2024-04-25T01:29:04.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.172+0000 7f3f5ffff640 1 --2- 172.21.15.112:0/2164007391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f601de7c0 0x7f3f601eb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:04.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.173+0000 7f3f5ffff640 1 -- 172.21.15.112:0/2164007391 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f60151ef0 con 0x7f3f601de7c0 2024-04-25T01:29:04.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.173+0000 7f3f5cff9640 1 -- 172.21.15.112:0/2164007391 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f5804e3d0 con 0x7f3f600c2160 2024-04-25T01:29:04.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.173+0000 7f3f5ffff640 1 --2- 172.21.15.112:0/2164007391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f601de7c0 0x7f3f601eb880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3f500011e0 tx=0x7f3f50001990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:04.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.173+0000 7f3f5cff9640 1 --2- 172.21.15.112:0/2164007391 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3f4c06bde0 0x7f3f4c06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:04.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.173+0000 7f3f5cff9640 1 -- 172.21.15.112:0/2164007391 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f3f580ac440 con 0x7f3f600c2160 2024-04-25T01:29:04.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.173+0000 7f3f5f7fe640 1 --2- 172.21.15.112:0/2164007391 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3f4c06bde0 0x7f3f4c06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:04.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.173+0000 7f3f5cff9640 1 -- 172.21.15.112:0/2164007391 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5000f600 con 0x7f3f601de7c0 2024-04-25T01:29:04.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.173+0000 7f3f5cff9640 1 -- 172.21.15.112:0/2164007391 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3f50018070 con 0x7f3f601de7c0 2024-04-25T01:29:04.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.173+0000 7f3f5cff9640 1 -- 172.21.15.112:0/2164007391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f601de7c0 msgr2=0x7f3f601eb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:04.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.173+0000 7f3f5cff9640 1 --2- 172.21.15.112:0/2164007391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f601de7c0 0x7f3f601eb880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3f500011e0 tx=0x7f3f50001990 comp rx=0 tx=0).stop 2024-04-25T01:29:04.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.173+0000 7f3f5cff9640 1 -- 172.21.15.112:0/2164007391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f601de7c0 msgr2=0x7f3f601eb880 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:29:04.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.173+0000 7f3f5cff9640 1 --2- 172.21.15.112:0/2164007391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f601de7c0 0x7f3f601eb880 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:04.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.174+0000 7f3f5f7fe640 1 --2- 172.21.15.112:0/2164007391 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3f4c06bde0 0x7f3f4c06e2a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3f54006fd0 tx=0x7f3f54006d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:04.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.180+0000 7f3f66a2a640 1 --2- 172.21.15.112:0/2164007391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f601ebdc0 0x7f3f601ee930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:04.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.180+0000 7f3f66a2a640 1 -- 172.21.15.112:0/2164007391 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f600689b0 con 0x7f3f601ebdc0 2024-04-25T01:29:04.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.180+0000 7f3f5ffff640 1 --2- 172.21.15.112:0/2164007391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f601ebdc0 0x7f3f601ee930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:04.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.180+0000 7f3f5ffff640 1 -- 172.21.15.112:0/2164007391 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3f60072a60 con 0x7f3f601ebdc0 2024-04-25T01:29:04.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.181+0000 7f3f5ffff640 1 --2- 172.21.15.112:0/2164007391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f601ebdc0 0x7f3f601ee930 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3f50015040 tx=0x7f3f50001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:04.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.181+0000 7f3f5cff9640 1 -- 172.21.15.112:0/2164007391 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50026450 con 0x7f3f601ebdc0 2024-04-25T01:29:04.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.181+0000 7f3f5cff9640 1 -- 172.21.15.112:0/2164007391 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4717 (secure 0 0 0) 0x7f3f500133f0 con 0x7f3f601ebdc0 2024-04-25T01:29:04.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.181+0000 7f3f5cff9640 1 -- 172.21.15.112:0/2164007391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f601ebdc0 msgr2=0x7f3f601ee930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:04.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.181+0000 7f3f5cff9640 1 --2- 172.21.15.112:0/2164007391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f601ebdc0 0x7f3f601ee930 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3f50015040 tx=0x7f3f50001080 comp rx=0 tx=0).stop 2024-04-25T01:29:04.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.181+0000 7f3f5cff9640 1 -- 172.21.15.112:0/2164007391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f601ebdc0 msgr2=0x7f3f601ee930 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:29:04.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.181+0000 7f3f5cff9640 1 --2- 172.21.15.112:0/2164007391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f601ebdc0 0x7f3f601ee930 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:04.172 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:29:04.172 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:29:04.173 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:29:04.173 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:29:04.173 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 24, 2024-04-25T01:29:04.173 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 44124, 2024-04-25T01:29:04.173 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:29:04.173 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:29:04.173 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:29:04.174 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:04.174 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 2, 2024-04-25T01:29:04.174 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:29:04.174 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:29:04.175 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:29:04.176 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:04.176 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:04.176 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:04.176 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:04.176 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:04.176 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:04.176 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:04.176 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:04.177 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:04.177 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:04.177 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:04.177 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:04.177 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:29:04.177 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:29:04.177 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:04.177 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:04.177 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:04.177 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:04.177 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:04.177 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:04.177 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:04.177 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:04.178 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:04.178 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:04.178 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:04.178 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:04.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.178 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:29:04.178 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:29:04.178 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:29:04.178 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:29:04.178 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:29:04.178 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:29:04.179 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:29:04.179 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:29:04.179 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:29:04.179 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:29:04.179 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:29:04.179 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:29:04.179 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:29:04.179 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:29:04.179 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:29:04.179 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:29:04.179 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:29:04.179 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:04.179 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:04.179 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:04.179 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:04.180 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:04.180 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:04.180 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:04.180 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:04.180 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:04.180 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:04.180 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:04.180 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:04.180 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:29:04.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.180 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:29:04.180 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.180 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:29:04.180 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:29:04.181 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:04.181 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:04.181 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.181 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:04.181 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:29:04.181 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:04.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.181 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.181 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:04.181 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:29:04.181 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:04.181 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:04.181 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:04.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.181 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:29:04.182 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:29:04.182 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:04.182 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:04.182 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:04.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.182 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.182 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:29:04.182 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:29:04.182 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:04.182 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:04.182 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.182 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:04.182 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:29:04.182 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:04.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.183 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.183 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:04.183 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:29:04.183 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:04.183 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:04.183 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:04.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.183 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:29:04.183 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:29:04.183 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:04.183 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:04.183 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:04.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.183 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.183 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:29:04.184 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:29:04.184 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:04.184 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:04.184 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.184 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:04.184 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:29:04.184 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:04.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.184 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.184 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:04.184 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:29:04.184 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:04.184 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:04.184 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:04.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.185 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:29:04.185 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:29:04.185 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:04.185 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:04.185 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:04.185 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:04.185 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:04.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.185 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:29:04.185 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:29:04.185 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.185 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:04.185 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:04.185 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:29:04.185 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:29:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:29:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:29:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:29:04.187 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:04.187 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:04.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:04.187 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:29:04.187 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:29:04.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.181+0000 7f3f66a2a640 1 -- 172.21.15.112:0/2164007391 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3f4c06bde0 msgr2=0x7f3f4c06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:04.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.181+0000 7f3f66a2a640 1 --2- 172.21.15.112:0/2164007391 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3f4c06bde0 0x7f3f4c06e2a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3f54006fd0 tx=0x7f3f54006d80 comp rx=0 tx=0).stop 2024-04-25T01:29:04.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.181+0000 7f3f66a2a640 1 -- 172.21.15.112:0/2164007391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f600c2160 msgr2=0x7f3f601eb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:04.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.181+0000 7f3f66a2a640 1 --2- 172.21.15.112:0/2164007391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f600c2160 0x7f3f601eb340 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009e80 tx=0x7f3f58004ab0 comp rx=0 tx=0).stop 2024-04-25T01:29:04.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.181+0000 7f3f64fc7640 1 -- 172.21.15.112:0/2164007391 reap_dead start 2024-04-25T01:29:04.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.182+0000 7f3f66a2a640 1 -- 172.21.15.112:0/2164007391 shutdown_connections 2024-04-25T01:29:04.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.182+0000 7f3f66a2a640 1 --2- 172.21.15.112:0/2164007391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f600c2160 0x7f3f601eb340 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:04.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.182+0000 7f3f66a2a640 1 -- 172.21.15.112:0/2164007391 >> 172.21.15.112:0/2164007391 conn(0x7f3f600c4260 msgr2=0x7f3f60071f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:04.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.182+0000 7f3f66a2a640 1 -- 172.21.15.112:0/2164007391 shutdown_connections 2024-04-25T01:29:04.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:04.182+0000 7f3f66a2a640 1 -- 172.21.15.112:0/2164007391 wait complete. 2024-04-25T01:29:04.189 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:29:04.189 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T01:29:24.190 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-04-25T01:29:24.190 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-04-25T01:29:24.190 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T01:29:24.191 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T01:29:24.193 INFO:tasks.ceph.mon.b:Started 2024-04-25T01:29:24.193 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:29:24.194 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:29:24.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.321+0000 7f8a3e0b3640 1 Processor -- start 2024-04-25T01:29:24.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.321+0000 7f8a3e0b3640 1 -- start start 2024-04-25T01:29:24.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.322+0000 7f8a3e0b3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8a3806bd90 0x7f8a380c1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.322+0000 7f8a3e0b3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a381560a0 con 0x7f8a381531d0 2024-04-25T01:29:24.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.322+0000 7f8a3e0b3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a380aaa50 con 0x7f8a380c1560 2024-04-25T01:29:24.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.322+0000 7f8a3e0b3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8a380aafd0 con 0x7f8a3806bd90 2024-04-25T01:29:24.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.322+0000 7f8a377fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8a381531d0 0x7f8a38155990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:40952/0 (socket says 172.21.15.112:40952) 2024-04-25T01:29:24.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.322+0000 7f8a377fe640 1 -- 172.21.15.112:0/3413842937 learned_addr learned my addr 172.21.15.112:0/3413842937 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:29:24.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.322+0000 7f8a37fff640 1 --2- 172.21.15.112:0/3413842937 >> v2:172.21.15.187:3300/0 conn(0x7f8a3806bd90 0x7f8a380c1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a37fff640 1 -- 172.21.15.112:0/3413842937 >> v2:172.21.15.187:3300/0 conn(0x7f8a3806bd90 msgr2=0x7f8a380c1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:29:24.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a37fff640 1 -- 172.21.15.112:0/3413842937 >> v2:172.21.15.187:3300/0 conn(0x7f8a3806bd90 msgr2=0x7f8a380c1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:29:24.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a37fff640 1 --2- 172.21.15.112:0/3413842937 >> v2:172.21.15.187:3300/0 conn(0x7f8a3806bd90 0x7f8a380c1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:29:24.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a367fc640 1 -- 172.21.15.112:0/3413842937 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3325198867 0 0) 0x7f8a381560a0 con 0x7f8a381531d0 2024-04-25T01:29:24.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a367fc640 1 -- 172.21.15.112:0/3413842937 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a20003520 con 0x7f8a381531d0 2024-04-25T01:29:24.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a37fff640 1 --2- 172.21.15.112:0/3413842937 >> v2:172.21.15.187:3300/0 conn(0x7f8a3806bd90 0x7f8a380c1de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:24.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a367fc640 1 -- 172.21.15.112:0/3413842937 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 486202179 0 0) 0x7f8a20003520 con 0x7f8a381531d0 2024-04-25T01:29:24.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a367fc640 1 -- 172.21.15.112:0/3413842937 >> v2:172.21.15.187:3300/0 conn(0x7f8a3806bd90 msgr2=0x7f8a380c1de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:29:24.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a367fc640 1 --2- 172.21.15.112:0/3413842937 >> v2:172.21.15.187:3300/0 conn(0x7f8a3806bd90 0x7f8a380c1de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a367fc640 1 -- 172.21.15.112:0/3413842937 >> v1:172.21.15.187:6789/0 conn(0x7f8a380c1560 legacy=0x7f8a3806b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a367fc640 1 -- 172.21.15.112:0/3413842937 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a38159ab0 con 0x7f8a381531d0 2024-04-25T01:29:24.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a367fc640 1 -- 172.21.15.112:0/3413842937 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f8a28001670 con 0x7f8a381531d0 2024-04-25T01:29:24.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a367fc640 1 -- 172.21.15.112:0/3413842937 >> v1:172.21.15.112:6789/0 conn(0x7f8a381531d0 legacy=0x7f8a38155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a367fc640 1 --2- 172.21.15.112:0/3413842937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a200039b0 0x7f8a20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a367fc640 1 --2- 172.21.15.112:0/3413842937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a20004800 0x7f8a20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a367fc640 1 --2- 172.21.15.112:0/3413842937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a200053c0 0x7f8a2000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a367fc640 1 -- 172.21.15.112:0/3413842937 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8a380aafd0 con 0x7f8a200053c0 2024-04-25T01:29:24.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a367fc640 1 -- 172.21.15.112:0/3413842937 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2000c250 con 0x7f8a20004800 2024-04-25T01:29:24.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a367fc640 1 -- 172.21.15.112:0/3413842937 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8a2000c450 con 0x7f8a200039b0 2024-04-25T01:29:24.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a37fff640 1 --2- 172.21.15.112:0/3413842937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a200053c0 0x7f8a2000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a377fe640 1 --2- 172.21.15.112:0/3413842937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a200039b0 0x7f8a20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a36ffd640 1 --2- 172.21.15.112:0/3413842937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a20004800 0x7f8a20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a37fff640 1 -- 172.21.15.112:0/3413842937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a200039b0 msgr2=0x7f8a20003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:29:24.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a37fff640 1 --2- 172.21.15.112:0/3413842937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a200039b0 0x7f8a20003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a37fff640 1 -- 172.21.15.112:0/3413842937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a20004800 msgr2=0x7f8a20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:29:24.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.323+0000 7f8a37fff640 1 --2- 172.21.15.112:0/3413842937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a20004800 0x7f8a20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.324+0000 7f8a37fff640 1 -- 172.21.15.112:0/3413842937 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a20003520 con 0x7f8a200053c0 2024-04-25T01:29:24.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.324+0000 7f8a37fff640 1 -- 172.21.15.112:0/3413842937 reap_dead start 2024-04-25T01:29:24.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.324+0000 7f8a37fff640 1 --2- 172.21.15.112:0/3413842937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a200053c0 0x7f8a2000b9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009610 tx=0x7f8a20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d17dbd21e63a870 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:24.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.324+0000 7f8a367fc640 1 -- 172.21.15.112:0/3413842937 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30033070 con 0x7f8a200053c0 2024-04-25T01:29:24.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.324+0000 7f8a367fc640 1 -- 172.21.15.112:0/3413842937 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a3002f670 con 0x7f8a200053c0 2024-04-25T01:29:24.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.324+0000 7f8a367fc640 1 -- 172.21.15.112:0/3413842937 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30009bd0 con 0x7f8a200053c0 2024-04-25T01:29:24.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.325+0000 7f8a3e0b3640 1 -- 172.21.15.112:0/3413842937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a200053c0 msgr2=0x7f8a2000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.325+0000 7f8a3e0b3640 1 --2- 172.21.15.112:0/3413842937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a200053c0 0x7f8a2000b9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009610 tx=0x7f8a20005830 comp rx=0 tx=0).stop 2024-04-25T01:29:24.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.325+0000 7f8a3e0b3640 1 -- 172.21.15.112:0/3413842937 shutdown_connections 2024-04-25T01:29:24.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.325+0000 7f8a3e0b3640 1 --2- 172.21.15.112:0/3413842937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a200053c0 0x7f8a2000b9c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.325+0000 7f8a3e0b3640 1 -- 172.21.15.112:0/3413842937 >> 172.21.15.112:0/3413842937 conn(0x7f8a380c8130 msgr2=0x7f8a380c8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:24.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.325+0000 7f8a3e0b3640 1 -- 172.21.15.112:0/3413842937 shutdown_connections 2024-04-25T01:29:24.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.325+0000 7f8a3e0b3640 1 -- 172.21.15.112:0/3413842937 wait complete. 2024-04-25T01:29:24.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.326+0000 7f8a3e0b3640 1 Processor -- start 2024-04-25T01:29:24.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.327+0000 7f8a3e0b3640 1 -- start start 2024-04-25T01:29:24.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.327+0000 7f8a3e0b3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a200053c0 0x7f8a3814b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.327+0000 7f8a3e0b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3814cf10 0x7f8a3813c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.327+0000 7f8a3e0b3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a3813ccf0 0x7f8a380ab390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.327+0000 7f8a3e0b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8a380c4900 con 0x7f8a3814cf10 2024-04-25T01:29:24.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.327+0000 7f8a3e0b3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8a38159d90 con 0x7f8a3813ccf0 2024-04-25T01:29:24.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.327+0000 7f8a3e0b3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8a380c1de0 con 0x7f8a200053c0 2024-04-25T01:29:24.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.327+0000 7f8a36ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3814cf10 0x7f8a3813c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.328+0000 7f8a36ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3814cf10 0x7f8a3813c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38630/0 (socket says 172.21.15.112:38630) 2024-04-25T01:29:24.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.328+0000 7f8a36ffd640 1 -- 172.21.15.112:0/3916823087 learned_addr learned my addr 172.21.15.112:0/3916823087 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:24.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.328+0000 7f8a377fe640 1 --2- 172.21.15.112:0/3916823087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a200053c0 0x7f8a3814b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.328+0000 7f8a37fff640 1 --2- 172.21.15.112:0/3916823087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a3813ccf0 0x7f8a380ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.328+0000 7f8a37fff640 1 -- 172.21.15.112:0/3916823087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a3813ccf0 msgr2=0x7f8a380ab390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-25T01:29:24.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.328+0000 7f8a37fff640 1 -- 172.21.15.112:0/3916823087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a3813ccf0 msgr2=0x7f8a380ab390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:29:24.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.328+0000 7f8a37fff640 1 --2- 172.21.15.112:0/3916823087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a3813ccf0 0x7f8a380ab390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:29:24.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.328+0000 7f8a37fff640 1 --2- 172.21.15.112:0/3916823087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a3813ccf0 0x7f8a380ab390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:24.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.328+0000 7f8a36ffd640 1 -- 172.21.15.112:0/3916823087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a200053c0 msgr2=0x7f8a3814b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.328+0000 7f8a36ffd640 1 --2- 172.21.15.112:0/3916823087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a200053c0 0x7f8a3814b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.328+0000 7f8a36ffd640 1 -- 172.21.15.112:0/3916823087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a3813ccf0 msgr2=0x7f8a380ab390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:29:24.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.328+0000 7f8a36ffd640 1 --2- 172.21.15.112:0/3916823087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a3813ccf0 0x7f8a380ab390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.328+0000 7f8a36ffd640 1 -- 172.21.15.112:0/3916823087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a381ee690 con 0x7f8a3814cf10 2024-04-25T01:29:24.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.328+0000 7f8a36ffd640 1 --2- 172.21.15.112:0/3916823087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3814cf10 0x7f8a3813c7b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00f4f0 tx=0x7f8a2c00f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:24.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.328+0000 7f8a1ffff640 1 -- 172.21.15.112:0/3916823087 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0052e0 con 0x7f8a3814cf10 2024-04-25T01:29:24.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.328+0000 7f8a1ffff640 1 -- 172.21.15.112:0/3916823087 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a2c018070 con 0x7f8a3814cf10 2024-04-25T01:29:24.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.328+0000 7f8a3e0b3640 1 -- 172.21.15.112:0/3916823087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a381ee950 con 0x7f8a3814cf10 2024-04-25T01:29:24.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.328+0000 7f8a1ffff640 1 -- 172.21.15.112:0/3916823087 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c003830 con 0x7f8a3814cf10 2024-04-25T01:29:24.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.328+0000 7f8a3e0b3640 1 -- 172.21.15.112:0/3916823087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a381eef10 con 0x7f8a3814cf10 2024-04-25T01:29:24.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.329+0000 7f8a3e0b3640 1 -- 172.21.15.112:0/3916823087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a380689b0 con 0x7f8a3814cf10 2024-04-25T01:29:24.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.330+0000 7f8a1ffff640 1 -- 172.21.15.112:0/3916823087 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a2c0039d0 con 0x7f8a3814cf10 2024-04-25T01:29:24.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.330+0000 7f8a1ffff640 1 --2- 172.21.15.112:0/3916823087 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8a2406be30 0x7f8a2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.330+0000 7f8a377fe640 1 --2- 172.21.15.112:0/3916823087 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8a2406be30 0x7f8a2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.330+0000 7f8a1ffff640 1 -- 172.21.15.112:0/3916823087 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f8a2c08ae60 con 0x7f8a3814cf10 2024-04-25T01:29:24.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.331+0000 7f8a377fe640 1 --2- 172.21.15.112:0/3916823087 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8a2406be30 0x7f8a2406e2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8a28000fb0 tx=0x7f8a280078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:24.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.336+0000 7f8a1ffff640 1 -- 172.21.15.112:0/3916823087 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8a2c0594e0 con 0x7f8a3814cf10 2024-04-25T01:29:24.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.631+0000 7f8a3e0b3640 1 -- 172.21.15.112:0/3916823087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8a380c45c0 con 0x7f8a3814cf10 2024-04-25T01:29:24.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.631+0000 7f8a1ffff640 1 -- 172.21.15.112:0/3916823087 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1568 (secure 0 0 0) 0x7f8a2c058de0 con 0x7f8a3814cf10 2024-04-25T01:29:24.610 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":26,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:29:24.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.634+0000 7f8a3e0b3640 1 -- 172.21.15.112:0/3916823087 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8a2406be30 msgr2=0x7f8a2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.634+0000 7f8a3e0b3640 1 --2- 172.21.15.112:0/3916823087 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8a2406be30 0x7f8a2406e2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8a28000fb0 tx=0x7f8a280078f0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.634+0000 7f8a3e0b3640 1 -- 172.21.15.112:0/3916823087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3814cf10 msgr2=0x7f8a3813c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.634+0000 7f8a3e0b3640 1 --2- 172.21.15.112:0/3916823087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3814cf10 0x7f8a3813c7b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00f4f0 tx=0x7f8a2c00f9b0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.634+0000 7f8a3e0b3640 1 -- 172.21.15.112:0/3916823087 shutdown_connections 2024-04-25T01:29:24.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.634+0000 7f8a3e0b3640 1 --2- 172.21.15.112:0/3916823087 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8a2406be30 0x7f8a2406e2f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.634+0000 7f8a3e0b3640 1 --2- 172.21.15.112:0/3916823087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a3813ccf0 0x7f8a380ab390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.634+0000 7f8a3e0b3640 1 --2- 172.21.15.112:0/3916823087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3814cf10 0x7f8a3813c7b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.634+0000 7f8a3e0b3640 1 --2- 172.21.15.112:0/3916823087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a200053c0 0x7f8a3814b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.634+0000 7f8a3e0b3640 1 -- 172.21.15.112:0/3916823087 >> 172.21.15.112:0/3916823087 conn(0x7f8a380c8130 msgr2=0x7f8a38067110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:24.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.634+0000 7f8a3e0b3640 1 -- 172.21.15.112:0/3916823087 shutdown_connections 2024-04-25T01:29:24.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.634+0000 7f8a3e0b3640 1 -- 172.21.15.112:0/3916823087 wait complete. 2024-04-25T01:29:24.623 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:29:24.623 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:29:24.623 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:29:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.757+0000 7f5b9a5c1640 1 Processor -- start 2024-04-25T01:29:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.757+0000 7f5b9a5c1640 1 -- start start 2024-04-25T01:29:24.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.758+0000 7f5b9a5c1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5b940c42e0 0x7f5b9406dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.758+0000 7f5b9a5c1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b9406e130 con 0x7f5b94152ca0 2024-04-25T01:29:24.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.758+0000 7f5b9a5c1640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b941598c0 con 0x7f5b94156290 2024-04-25T01:29:24.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.758+0000 7f5b9a5c1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f5b94159ba0 con 0x7f5b940c42e0 2024-04-25T01:29:24.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.758+0000 7f5b93fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5b940c42e0 0x7f5b9406dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.758+0000 7f5b93fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5b940c42e0 0x7f5b9406dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38636/0 (socket says 172.21.15.112:38636) 2024-04-25T01:29:24.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.758+0000 7f5b93fff640 1 -- 172.21.15.112:0/2384030039 learned_addr learned my addr 172.21.15.112:0/2384030039 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:24.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.759+0000 7f5b93fff640 1 -- 172.21.15.112:0/2384030039 >> v1:172.21.15.187:6789/0 conn(0x7f5b94156290 legacy=0x7f5b94158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.759+0000 7f5b93fff640 1 -- 172.21.15.112:0/2384030039 >> v1:172.21.15.112:6789/0 conn(0x7f5b94152ca0 legacy=0x7f5b94160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.759+0000 7f5b93fff640 1 -- 172.21.15.112:0/2384030039 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b94155a90 con 0x7f5b940c42e0 2024-04-25T01:29:24.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.759+0000 7f5b93fff640 1 --2- 172.21.15.112:0/2384030039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b940c42e0 0x7f5b9406dbf0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009870 tx=0x7f5b8402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36812597d7926125 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:24.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.759+0000 7f5b92ffd640 1 -- 172.21.15.112:0/2384030039 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84037070 con 0x7f5b940c42e0 2024-04-25T01:29:24.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.760+0000 7f5b92ffd640 1 -- 172.21.15.112:0/2384030039 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b8402ae50 con 0x7f5b940c42e0 2024-04-25T01:29:24.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.760+0000 7f5b92ffd640 1 -- 172.21.15.112:0/2384030039 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84032860 con 0x7f5b940c42e0 2024-04-25T01:29:24.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.760+0000 7f5b9a5c1640 1 -- 172.21.15.112:0/2384030039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b940c42e0 msgr2=0x7f5b9406dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.760+0000 7f5b9a5c1640 1 --2- 172.21.15.112:0/2384030039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b940c42e0 0x7f5b9406dbf0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009870 tx=0x7f5b8402a300 comp rx=0 tx=0).stop 2024-04-25T01:29:24.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.760+0000 7f5b9a5c1640 1 -- 172.21.15.112:0/2384030039 shutdown_connections 2024-04-25T01:29:24.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.760+0000 7f5b9a5c1640 1 --2- 172.21.15.112:0/2384030039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b940c42e0 0x7f5b9406dbf0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.760+0000 7f5b9a5c1640 1 -- 172.21.15.112:0/2384030039 >> 172.21.15.112:0/2384030039 conn(0x7f5b940c4a90 msgr2=0x7f5b940c4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:24.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.760+0000 7f5b9a5c1640 1 -- 172.21.15.112:0/2384030039 shutdown_connections 2024-04-25T01:29:24.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.760+0000 7f5b9a5c1640 1 -- 172.21.15.112:0/2384030039 wait complete. 2024-04-25T01:29:24.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.762+0000 7f5b9a5c1640 1 Processor -- start 2024-04-25T01:29:24.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.762+0000 7f5b9a5c1640 1 -- start start 2024-04-25T01:29:24.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b9a5c1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b940c42e0 0x7f5b94147f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b9a5c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b94152ca0 0x7f5b9414e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b9a5c1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b94156290 0x7f5b940c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b9a5c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b94156700 con 0x7f5b94152ca0 2024-04-25T01:29:24.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b9a5c1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5b941595a0 con 0x7f5b94156290 2024-04-25T01:29:24.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b9a5c1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5b94156880 con 0x7f5b940c42e0 2024-04-25T01:29:24.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b937fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b94152ca0 0x7f5b9414e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b937fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b94152ca0 0x7f5b9414e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38644/0 (socket says 172.21.15.112:38644) 2024-04-25T01:29:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b937fe640 1 -- 172.21.15.112:0/3021837636 learned_addr learned my addr 172.21.15.112:0/3021837636 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b93fff640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b940c42e0 0x7f5b94147f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b98b5e640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b94156290 0x7f5b940c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b937fe640 1 -- 172.21.15.112:0/3021837636 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b940c42e0 msgr2=0x7f5b94147f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b937fe640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b940c42e0 0x7f5b94147f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b937fe640 1 -- 172.21.15.112:0/3021837636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b94156290 msgr2=0x7f5b940c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b937fe640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b94156290 0x7f5b940c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b937fe640 1 -- 172.21.15.112:0/3021837636 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b94153b20 con 0x7f5b94152ca0 2024-04-25T01:29:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b937fe640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b94152ca0 0x7f5b9414e930 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800c3c0 tx=0x7f5b8800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b90ff9640 1 -- 172.21.15.112:0/3021837636 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b880052d0 con 0x7f5b94152ca0 2024-04-25T01:29:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b90ff9640 1 -- 172.21.15.112:0/3021837636 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b88005490 con 0x7f5b94152ca0 2024-04-25T01:29:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b9a5c1640 1 -- 172.21.15.112:0/3021837636 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b94148450 con 0x7f5b94152ca0 2024-04-25T01:29:24.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.763+0000 7f5b90ff9640 1 -- 172.21.15.112:0/3021837636 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88007260 con 0x7f5b94152ca0 2024-04-25T01:29:24.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.764+0000 7f5b9a5c1640 1 -- 172.21.15.112:0/3021837636 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b94153de0 con 0x7f5b94152ca0 2024-04-25T01:29:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.764+0000 7f5b9a5c1640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b94154020 0x7f5b9414d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.764+0000 7f5b9a5c1640 1 -- 172.21.15.112:0/3021837636 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b94075980 con 0x7f5b94154020 2024-04-25T01:29:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.765+0000 7f5b93fff640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b94154020 0x7f5b9414d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.765+0000 7f5b93fff640 1 -- 172.21.15.112:0/3021837636 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b940682a0 con 0x7f5b94154020 2024-04-25T01:29:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.765+0000 7f5b93fff640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b94154020 0x7f5b9414d050 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5b94070640 tx=0x7f5b840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.765+0000 7f5b90ff9640 1 -- 172.21.15.112:0/3021837636 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b88007400 con 0x7f5b94152ca0 2024-04-25T01:29:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.765+0000 7f5b90ff9640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b8006be30 0x7f5b8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.766+0000 7f5b98b5e640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b8006be30 0x7f5b8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.766+0000 7f5b90ff9640 1 -- 172.21.15.112:0/3021837636 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84037040 con 0x7f5b94154020 2024-04-25T01:29:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.766+0000 7f5b90ff9640 1 -- 172.21.15.112:0/3021837636 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f5b8808a560 con 0x7f5b94152ca0 2024-04-25T01:29:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.766+0000 7f5b90ff9640 1 -- 172.21.15.112:0/3021837636 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5b8402ae50 con 0x7f5b94154020 2024-04-25T01:29:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.766+0000 7f5b90ff9640 1 -- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b94154020 msgr2=0x7f5b9414d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.766+0000 7f5b90ff9640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b94154020 0x7f5b9414d050 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5b94070640 tx=0x7f5b840046c0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.766+0000 7f5b90ff9640 1 -- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b94154020 msgr2=0x7f5b9414d050 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:29:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.766+0000 7f5b90ff9640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b94154020 0x7f5b9414d050 secure :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5b94070640 tx=0x7f5b840046c0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.766+0000 7f5b98b5e640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b8006be30 0x7f5b8006e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0012e0 tx=0x7f5b8c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:24.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.773+0000 7f5b9a5c1640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b941486f0 0x7f5b9414d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.773+0000 7f5b9a5c1640 1 -- 172.21.15.112:0/3021837636 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9414cce0 con 0x7f5b941486f0 2024-04-25T01:29:24.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.773+0000 7f5b93fff640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b941486f0 0x7f5b9414d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.774+0000 7f5b93fff640 1 -- 172.21.15.112:0/3021837636 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5b9406d280 con 0x7f5b941486f0 2024-04-25T01:29:24.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.774+0000 7f5b93fff640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b941486f0 0x7f5b9414d590 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5b84000c00 tx=0x7f5b84008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:24.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.774+0000 7f5b90ff9640 1 -- 172.21.15.112:0/3021837636 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8404e070 con 0x7f5b941486f0 2024-04-25T01:29:24.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.774+0000 7f5b90ff9640 1 -- 172.21.15.112:0/3021837636 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5071 (secure 0 0 0) 0x7f5b8404c070 con 0x7f5b941486f0 2024-04-25T01:29:24.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.774+0000 7f5b90ff9640 1 -- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b941486f0 msgr2=0x7f5b9414d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.774+0000 7f5b90ff9640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b941486f0 0x7f5b9414d590 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5b84000c00 tx=0x7f5b84008030 comp rx=0 tx=0).stop 2024-04-25T01:29:24.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.774+0000 7f5b90ff9640 1 -- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b941486f0 msgr2=0x7f5b9414d590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:29:24.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.774+0000 7f5b90ff9640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b941486f0 0x7f5b9414d590 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.753 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:29:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:29:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:29:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:29:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 26, 2024-04-25T01:29:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 236402, 2024-04-25T01:29:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:29:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:29:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:29:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:29:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:29:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:29:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:29:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:29:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:24.755 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:24.755 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:24.755 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:24.755 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:24.755 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:24.755 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:24.755 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:24.757 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:24.757 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:24.757 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:24.757 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:24.757 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:29:24.757 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:29:24.757 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:29:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:29:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:29:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:29:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:29:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:29:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:29:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:29:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:29:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:29:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:29:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:29:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:29:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:29:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:29:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:29:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:29:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:29:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:29:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:29:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:29:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:29:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:29:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:29:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:29:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:29:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:29:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:29:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:29:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:29:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:29:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:29:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:29:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:29:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:29:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:29:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:29:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:29:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:29:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:29:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:29:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:29:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:29:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 7 2024-04-25T01:29:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:29:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 8 2024-04-25T01:29:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:24.768 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:29:24.768 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.768 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:24.768 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:24.768 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:29:24.768 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:24.768 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:24.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.768 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:29:24.768 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:29:24.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.775+0000 7f5b9a5c1640 1 -- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b8006be30 msgr2=0x7f5b8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.775+0000 7f5b9a5c1640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b8006be30 0x7f5b8006e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0012e0 tx=0x7f5b8c009040 comp rx=0 tx=0).stop 2024-04-25T01:29:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.775+0000 7f5b9a5c1640 1 -- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b94152ca0 msgr2=0x7f5b9414e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.775+0000 7f5b9a5c1640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b94152ca0 0x7f5b9414e930 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800c3c0 tx=0x7f5b8800c880 comp rx=0 tx=0).stop 2024-04-25T01:29:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.775+0000 7f5b98b5e640 1 -- 172.21.15.112:0/3021837636 reap_dead start 2024-04-25T01:29:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.775+0000 7f5b9a5c1640 1 -- 172.21.15.112:0/3021837636 shutdown_connections 2024-04-25T01:29:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.775+0000 7f5b9a5c1640 1 --2- 172.21.15.112:0/3021837636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b94152ca0 0x7f5b9414e930 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.775+0000 7f5b9a5c1640 1 -- 172.21.15.112:0/3021837636 >> 172.21.15.112:0/3021837636 conn(0x7f5b940c4a90 msgr2=0x7f5b9406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.775+0000 7f5b9a5c1640 1 -- 172.21.15.112:0/3021837636 shutdown_connections 2024-04-25T01:29:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.775+0000 7f5b9a5c1640 1 -- 172.21.15.112:0/3021837636 wait complete. 2024-04-25T01:29:24.770 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:29:24.771 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:29:24.771 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:29:24.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.903+0000 7ff019b41640 1 Processor -- start 2024-04-25T01:29:24.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.903+0000 7ff019b41640 1 -- start start 2024-04-25T01:29:24.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.904+0000 7ff019b41640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0140c9230 0x7ff0140c9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.904+0000 7ff019b41640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff0140c34b0 0x7ff0140c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.904+0000 7ff019b41640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff014159760 con 0x7ff0140713c0 2024-04-25T01:29:24.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.904+0000 7ff019b41640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff014159cc0 con 0x7ff0140c34b0 2024-04-25T01:29:24.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.904+0000 7ff019b41640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff0141599e0 con 0x7ff0140c9230 2024-04-25T01:29:24.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.904+0000 7ff013fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff0140c34b0 0x7ff0140c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.904+0000 7ff0137fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0140c9230 0x7ff0140c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.904+0000 7ff013fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff0140c34b0 0x7ff0140c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43442/0 (socket says 172.21.15.112:43442) 2024-04-25T01:29:24.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.904+0000 7ff012ffd640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7ff0140713c0 0x7ff0140c70a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:35336/0 (socket says 172.21.15.112:35336) 2024-04-25T01:29:24.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.904+0000 7ff0137fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0140c9230 0x7ff0140c9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:55036/0 (socket says 172.21.15.112:55036) 2024-04-25T01:29:24.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.904+0000 7ff013fff640 1 -- 172.21.15.112:0/474223142 learned_addr learned my addr 172.21.15.112:0/474223142 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:24.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.904+0000 7ff0137fe640 1 -- 172.21.15.112:0/474223142 >> v2:172.21.15.187:3300/0 conn(0x7ff0140c34b0 msgr2=0x7ff0140c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:29:24.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.904+0000 7ff0137fe640 1 --2- 172.21.15.112:0/474223142 >> v2:172.21.15.187:3300/0 conn(0x7ff0140c34b0 0x7ff0140c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.904+0000 7ff0137fe640 1 -- 172.21.15.112:0/474223142 >> v1:172.21.15.187:6789/0 conn(0x7ff0140713c0 legacy=0x7ff0140c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.904+0000 7ff0137fe640 1 -- 172.21.15.112:0/474223142 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff014070230 con 0x7ff0140c9230 2024-04-25T01:29:24.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.904+0000 7ff013fff640 1 --2- 172.21.15.112:0/474223142 >> v2:172.21.15.187:3300/0 conn(0x7ff0140c34b0 0x7ff0140c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:29:24.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.905+0000 7ff0137fe640 1 --2- 172.21.15.112:0/474223142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0140c9230 0x7ff0140c9610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff0040097a0 tx=0x7ff00402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d92d1e3fdd88edc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:24.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.905+0000 7ff0127fc640 1 -- 172.21.15.112:0/474223142 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004037070 con 0x7ff0140c9230 2024-04-25T01:29:24.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.905+0000 7ff0127fc640 1 -- 172.21.15.112:0/474223142 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff00402ae50 con 0x7ff0140c9230 2024-04-25T01:29:24.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.905+0000 7ff0127fc640 1 -- 172.21.15.112:0/474223142 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004032660 con 0x7ff0140c9230 2024-04-25T01:29:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.906+0000 7ff019b41640 1 -- 172.21.15.112:0/474223142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0140c9230 msgr2=0x7ff0140c9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.906+0000 7ff019b41640 1 --2- 172.21.15.112:0/474223142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0140c9230 0x7ff0140c9610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff0040097a0 tx=0x7ff00402a100 comp rx=0 tx=0).stop 2024-04-25T01:29:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.906+0000 7ff019b41640 1 -- 172.21.15.112:0/474223142 shutdown_connections 2024-04-25T01:29:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.906+0000 7ff019b41640 1 --2- 172.21.15.112:0/474223142 >> v2:172.21.15.187:3300/0 conn(0x7ff0140c34b0 0x7ff0140c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.906+0000 7ff019b41640 1 --2- 172.21.15.112:0/474223142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0140c9230 0x7ff0140c9610 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.906+0000 7ff019b41640 1 -- 172.21.15.112:0/474223142 >> 172.21.15.112:0/474223142 conn(0x7ff0140745b0 msgr2=0x7ff014074990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.906+0000 7ff019b41640 1 -- 172.21.15.112:0/474223142 shutdown_connections 2024-04-25T01:29:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.906+0000 7ff019b41640 1 -- 172.21.15.112:0/474223142 wait complete. 2024-04-25T01:29:24.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.907+0000 7ff019b41640 1 Processor -- start 2024-04-25T01:29:24.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.907+0000 7ff019b41640 1 -- start start 2024-04-25T01:29:24.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.908+0000 7ff019b41640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0140c34b0 0x7ff0141491f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.908+0000 7ff019b41640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0140c9230 0x7ff014149730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.908+0000 7ff019b41640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff01414b780 0x7ff01414bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.908+0000 7ff019b41640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff014156e00 con 0x7ff01414b780 2024-04-25T01:29:24.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.908+0000 7ff019b41640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff014156c80 con 0x7ff0140c9230 2024-04-25T01:29:24.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.908+0000 7ff019b41640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff014156f80 con 0x7ff0140c34b0 2024-04-25T01:29:24.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.908+0000 7ff0137fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0140c34b0 0x7ff0141491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.908+0000 7ff013fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff01414b780 0x7ff01414bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.908+0000 7ff0137fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0140c34b0 0x7ff0141491f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:55044/0 (socket says 172.21.15.112:55044) 2024-04-25T01:29:24.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.908+0000 7ff0137fe640 1 -- 172.21.15.112:0/1042289136 learned_addr learned my addr 172.21.15.112:0/1042289136 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:24.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.908+0000 7ff012ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0140c9230 0x7ff014149730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.909+0000 7ff013fff640 1 -- 172.21.15.112:0/1042289136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0140c34b0 msgr2=0x7ff0141491f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.909+0000 7ff013fff640 1 --2- 172.21.15.112:0/1042289136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0140c34b0 0x7ff0141491f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.909+0000 7ff013fff640 1 -- 172.21.15.112:0/1042289136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0140c9230 msgr2=0x7ff014149730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.909+0000 7ff013fff640 1 --2- 172.21.15.112:0/1042289136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0140c9230 0x7ff014149730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.909+0000 7ff013fff640 1 -- 172.21.15.112:0/1042289136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01414aa10 con 0x7ff01414b780 2024-04-25T01:29:24.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.909+0000 7ff012ffd640 1 --2- 172.21.15.112:0/1042289136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0140c9230 0x7ff014149730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:29:24.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.909+0000 7ff0137fe640 1 --2- 172.21.15.112:0/1042289136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0140c34b0 0x7ff0141491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:29:24.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.909+0000 7ff013fff640 1 --2- 172.21.15.112:0/1042289136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff01414b780 0x7ff01414bc30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00ef90 tx=0x7ff00c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:24.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.909+0000 7feffbfff640 1 -- 172.21.15.112:0/1042289136 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c00cb20 con 0x7ff01414b780 2024-04-25T01:29:24.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.909+0000 7ff019b41640 1 -- 172.21.15.112:0/1042289136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff01414acb0 con 0x7ff01414b780 2024-04-25T01:29:24.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.909+0000 7feffbfff640 1 -- 172.21.15.112:0/1042289136 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff00c00cce0 con 0x7ff01414b780 2024-04-25T01:29:24.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.909+0000 7feffbfff640 1 -- 172.21.15.112:0/1042289136 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c003bd0 con 0x7ff01414b780 2024-04-25T01:29:24.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.909+0000 7ff019b41640 1 -- 172.21.15.112:0/1042289136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0140c5bb0 con 0x7ff01414b780 2024-04-25T01:29:24.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.910+0000 7ff019b41640 1 --2- 172.21.15.112:0/1042289136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0140c5df0 0x7ff0141eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.910+0000 7ff019b41640 1 -- 172.21.15.112:0/1042289136 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff014075980 con 0x7ff0140c5df0 2024-04-25T01:29:24.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.910+0000 7ff0137fe640 1 --2- 172.21.15.112:0/1042289136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0140c5df0 0x7ff0141eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.910+0000 7ff0137fe640 1 -- 172.21.15.112:0/1042289136 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff0140682a0 con 0x7ff0140c5df0 2024-04-25T01:29:24.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.911+0000 7ff0137fe640 1 --2- 172.21.15.112:0/1042289136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0140c5df0 0x7ff0141eeb00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff0140af0e0 tx=0x7ff0040046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:24.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.911+0000 7feffbfff640 1 -- 172.21.15.112:0/1042289136 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff00c01b030 con 0x7ff01414b780 2024-04-25T01:29:24.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.911+0000 7feffbfff640 1 --2- 172.21.15.112:0/1042289136 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff00006be30 0x7ff00006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.911+0000 7feffbfff640 1 -- 172.21.15.112:0/1042289136 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7ff00c08b2a0 con 0x7ff01414b780 2024-04-25T01:29:24.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.911+0000 7ff012ffd640 1 --2- 172.21.15.112:0/1042289136 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff00006be30 0x7ff00006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.911+0000 7feffbfff640 1 -- 172.21.15.112:0/1042289136 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004037040 con 0x7ff0140c5df0 2024-04-25T01:29:24.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.911+0000 7feffbfff640 1 -- 172.21.15.112:0/1042289136 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff00403c640 con 0x7ff0140c5df0 2024-04-25T01:29:24.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.911+0000 7feffbfff640 1 -- 172.21.15.112:0/1042289136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0140c5df0 msgr2=0x7ff0141eeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.911+0000 7feffbfff640 1 --2- 172.21.15.112:0/1042289136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0140c5df0 0x7ff0141eeb00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff0140af0e0 tx=0x7ff0040046c0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.911+0000 7feffbfff640 1 -- 172.21.15.112:0/1042289136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0140c5df0 msgr2=0x7ff0141eeb00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:29:24.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.911+0000 7feffbfff640 1 --2- 172.21.15.112:0/1042289136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0140c5df0 0x7ff0141eeb00 secure :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff0140af0e0 tx=0x7ff0040046c0 comp rx=0 tx=0).stop 2024-04-25T01:29:24.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.912+0000 7ff012ffd640 1 --2- 172.21.15.112:0/1042289136 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff00006be30 0x7ff00006e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff01414c960 tx=0x7ff008007e80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:24.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.918+0000 7ff019b41640 1 --2- 172.21.15.112:0/1042289136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff014071340 0x7ff0141ef040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:24.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.918+0000 7ff019b41640 1 -- 172.21.15.112:0/1042289136 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0140c3290 con 0x7ff014071340 2024-04-25T01:29:24.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.919+0000 7ff0137fe640 1 --2- 172.21.15.112:0/1042289136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff014071340 0x7ff0141ef040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:24.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.919+0000 7ff0137fe640 1 -- 172.21.15.112:0/1042289136 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff01406fee0 con 0x7ff014071340 2024-04-25T01:29:24.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.919+0000 7ff0137fe640 1 --2- 172.21.15.112:0/1042289136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff014071340 0x7ff0141ef040 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff01406bac0 tx=0x7ff004008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:24.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.919+0000 7feffbfff640 1 -- 172.21.15.112:0/1042289136 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004037030 con 0x7ff014071340 2024-04-25T01:29:24.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.919+0000 7feffbfff640 1 -- 172.21.15.112:0/1042289136 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4556 (secure 0 0 0) 0x7ff004003610 con 0x7ff014071340 2024-04-25T01:29:24.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.919+0000 7feffbfff640 1 -- 172.21.15.112:0/1042289136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff014071340 msgr2=0x7ff0141ef040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.919+0000 7feffbfff640 1 --2- 172.21.15.112:0/1042289136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff014071340 0x7ff0141ef040 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff01406bac0 tx=0x7ff004008030 comp rx=0 tx=0).stop 2024-04-25T01:29:24.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.919+0000 7feffbfff640 1 -- 172.21.15.112:0/1042289136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff014071340 msgr2=0x7ff0141ef040 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:29:24.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.919+0000 7feffbfff640 1 --2- 172.21.15.112:0/1042289136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff014071340 0x7ff0141ef040 secure :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff01406bac0 tx=0x7ff004008030 comp rx=0 tx=0).stop 2024-04-25T01:29:24.899 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:29:24.899 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:29:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:29:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:29:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 26, 2024-04-25T01:29:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 608, 2024-04-25T01:29:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:29:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:29:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:29:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:29:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:29:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:29:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:29:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:29:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:29:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:29:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:29:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:29:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:29:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:29:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:29:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:29:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:29:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:29:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:29:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:29:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:29:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:29:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:29:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:29:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:29:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:29:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:29:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:29:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:29:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:29:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:29:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:29:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:29:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:29:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:29:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:29:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:29:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:29:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:29:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:29:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:29:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:29:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:29:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:29:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:29:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:29:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:29:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:29:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:29:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:29:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:29:24.910 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:24.910 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:24.910 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:24.910 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:29:24.910 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:24.910 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:24.910 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:24.910 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:29:24.910 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:29:24.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.920+0000 7ff019b41640 1 -- 172.21.15.112:0/1042289136 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff00006be30 msgr2=0x7ff00006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.920+0000 7ff019b41640 1 --2- 172.21.15.112:0/1042289136 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff00006be30 0x7ff00006e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff01414c960 tx=0x7ff008007e80 comp rx=0 tx=0).stop 2024-04-25T01:29:24.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.920+0000 7ff019b41640 1 -- 172.21.15.112:0/1042289136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff01414b780 msgr2=0x7ff01414bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:24.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.920+0000 7ff019b41640 1 --2- 172.21.15.112:0/1042289136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff01414b780 0x7ff01414bc30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00ef90 tx=0x7ff00c00c490 comp rx=0 tx=0).stop 2024-04-25T01:29:24.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.920+0000 7ff013fff640 1 -- 172.21.15.112:0/1042289136 reap_dead start 2024-04-25T01:29:24.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.920+0000 7ff013fff640 1 -- 172.21.15.112:0/1042289136 reap_dead start 2024-04-25T01:29:24.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.920+0000 7ff019b41640 1 -- 172.21.15.112:0/1042289136 shutdown_connections 2024-04-25T01:29:24.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.920+0000 7ff019b41640 1 -- 172.21.15.112:0/1042289136 >> 172.21.15.112:0/1042289136 conn(0x7ff0140745b0 msgr2=0x7ff01406ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:24.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.920+0000 7ff019b41640 1 -- 172.21.15.112:0/1042289136 shutdown_connections 2024-04-25T01:29:24.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:24.920+0000 7ff019b41640 1 -- 172.21.15.112:0/1042289136 wait complete. 2024-04-25T01:29:24.913 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:29:24.913 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:29:24.913 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:29:25.019 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:29:25.022 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:29:25.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.059+0000 7f1f0c2b2640 1 Processor -- start 2024-04-25T01:29:25.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.059+0000 7f1f0c2b2640 1 -- start start 2024-04-25T01:29:25.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.060+0000 7f1f0c2b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f04071ce0 0x7f1f0406f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:25.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.060+0000 7f1f0c2b2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1f0406b160 0x7f1f0406b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:25.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.060+0000 7f1f0c2b2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f040c20e0 con 0x7f1f04072d50 2024-04-25T01:29:25.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.060+0000 7f1f0c2b2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1f0406f3b0 con 0x7f1f0406b160 2024-04-25T01:29:25.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.060+0000 7f1f0c2b2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1f04073130 con 0x7f1f04071ce0 2024-04-25T01:29:25.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.060+0000 7f1f0a84f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1f0406b160 0x7f1f0406b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:25.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.060+0000 7f1f0984d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f04071ce0 0x7f1f0406f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:25.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.060+0000 7f1f0a04e640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f1f04072d50 0x7f1f041563e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:35344/0 (socket says 172.21.15.112:35344) 2024-04-25T01:29:25.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.060+0000 7f1f0a84f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1f0406b160 0x7f1f0406b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43484/0 (socket says 172.21.15.112:43484) 2024-04-25T01:29:25.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.060+0000 7f1f0984d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f04071ce0 0x7f1f0406f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:55046/0 (socket says 172.21.15.112:55046) 2024-04-25T01:29:25.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.060+0000 7f1f0a04e640 1 -- 172.21.15.112:0/925354149 learned_addr learned my addr 172.21.15.112:0/925354149 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:29:25.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.061+0000 7f1f0a84f640 1 -- 172.21.15.112:0/925354149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f04071ce0 msgr2=0x7f1f0406f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:29:25.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.061+0000 7f1f0a84f640 1 --2- 172.21.15.112:0/925354149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f04071ce0 0x7f1f0406f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:25.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.061+0000 7f1f0a84f640 1 -- 172.21.15.112:0/925354149 >> v1:172.21.15.187:6789/0 conn(0x7f1f04072d50 legacy=0x7f1f041563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:25.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.061+0000 7f1f0a84f640 1 -- 172.21.15.112:0/925354149 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f040c2380 con 0x7f1f0406b160 2024-04-25T01:29:25.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.061+0000 7f1f0a84f640 1 --2- 172.21.15.112:0/925354149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f0406b160 0x7f1f0406b5f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1f0406f530 tx=0x7f1efc02a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfc990bf3558f9d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:25.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.062+0000 7f1f0904c640 1 -- 172.21.15.112:0/925354149 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc007740 con 0x7f1f0406b160 2024-04-25T01:29:25.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.062+0000 7f1f0904c640 1 -- 172.21.15.112:0/925354149 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1efc0078e0 con 0x7f1f0406b160 2024-04-25T01:29:25.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.062+0000 7f1f0904c640 1 -- 172.21.15.112:0/925354149 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc007cb0 con 0x7f1f0406b160 2024-04-25T01:29:25.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.062+0000 7f1f0c2b2640 1 -- 172.21.15.112:0/925354149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f0406b160 msgr2=0x7f1f0406b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:25.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.062+0000 7f1f0c2b2640 1 --2- 172.21.15.112:0/925354149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f0406b160 0x7f1f0406b5f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1f0406f530 tx=0x7f1efc02a560 comp rx=0 tx=0).stop 2024-04-25T01:29:25.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.062+0000 7f1f0c2b2640 1 -- 172.21.15.112:0/925354149 shutdown_connections 2024-04-25T01:29:25.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.062+0000 7f1f0c2b2640 1 --2- 172.21.15.112:0/925354149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f0406b160 0x7f1f0406b5f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:25.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.062+0000 7f1f0c2b2640 1 --2- 172.21.15.112:0/925354149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f04071ce0 0x7f1f0406f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:25.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.062+0000 7f1f0c2b2640 1 -- 172.21.15.112:0/925354149 >> 172.21.15.112:0/925354149 conn(0x7f1f040c8090 msgr2=0x7f1f040c8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:25.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.062+0000 7f1f0c2b2640 1 -- 172.21.15.112:0/925354149 shutdown_connections 2024-04-25T01:29:25.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.063+0000 7f1f0c2b2640 1 -- 172.21.15.112:0/925354149 wait complete. 2024-04-25T01:29:25.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.064+0000 7f1f0c2b2640 1 Processor -- start 2024-04-25T01:29:25.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.064+0000 7f1f0c2b2640 1 -- start start 2024-04-25T01:29:25.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.065+0000 7f1f0c2b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f0406b160 0x7f1f041541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:25.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.065+0000 7f1f0c2b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f04071ce0 0x7f1f04154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:25.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.065+0000 7f1f0c2b2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f04072d50 0x7f1f0414a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:25.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.065+0000 7f1f0c2b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0406f040 con 0x7f1f04071ce0 2024-04-25T01:29:25.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.065+0000 7f1f0c2b2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0406ed60 con 0x7f1f04072d50 2024-04-25T01:29:25.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.065+0000 7f1f0c2b2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1f04072160 con 0x7f1f0406b160 2024-04-25T01:29:25.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.065+0000 7f1f0984d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f04071ce0 0x7f1f04154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:25.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.065+0000 7f1f0a04e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f0406b160 0x7f1f041541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:25.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.065+0000 7f1f0984d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f04071ce0 0x7f1f04154720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38688/0 (socket says 172.21.15.112:38688) 2024-04-25T01:29:25.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.065+0000 7f1f0984d640 1 -- 172.21.15.112:0/309346024 learned_addr learned my addr 172.21.15.112:0/309346024 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:25.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.065+0000 7f1f0a84f640 1 --2- 172.21.15.112:0/309346024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f04072d50 0x7f1f0414a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:25.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.065+0000 7f1f0984d640 1 -- 172.21.15.112:0/309346024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f0406b160 msgr2=0x7f1f041541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:25.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.065+0000 7f1f0984d640 1 --2- 172.21.15.112:0/309346024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f0406b160 0x7f1f041541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:25.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.065+0000 7f1f0984d640 1 -- 172.21.15.112:0/309346024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f04072d50 msgr2=0x7f1f0414a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:25.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.065+0000 7f1f0984d640 1 --2- 172.21.15.112:0/309346024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f04072d50 0x7f1f0414a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:25.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.065+0000 7f1f0984d640 1 -- 172.21.15.112:0/309346024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f04156720 con 0x7f1f04071ce0 2024-04-25T01:29:25.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.065+0000 7f1f0984d640 1 --2- 172.21.15.112:0/309346024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f04071ce0 0x7f1f04154720 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400e9b0 tx=0x7f1ef400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:25.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.066+0000 7f1efaffd640 1 -- 172.21.15.112:0/309346024 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4017070 con 0x7f1f04071ce0 2024-04-25T01:29:25.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.066+0000 7f1f0c2b2640 1 -- 172.21.15.112:0/309346024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f0414e9d0 con 0x7f1f04071ce0 2024-04-25T01:29:25.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.066+0000 7f1efaffd640 1 -- 172.21.15.112:0/309346024 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ef400c9a0 con 0x7f1f04071ce0 2024-04-25T01:29:25.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.066+0000 7f1f0c2b2640 1 -- 172.21.15.112:0/309346024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f04148510 con 0x7f1f04071ce0 2024-04-25T01:29:25.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.066+0000 7f1efaffd640 1 -- 172.21.15.112:0/309346024 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef400cd90 con 0x7f1f04071ce0 2024-04-25T01:29:25.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.067+0000 7f1f0c2b2640 1 --2- 172.21.15.112:0/309346024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f04148a30 0x7f1f0414ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:25.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.067+0000 7f1f0c2b2640 1 -- 172.21.15.112:0/309346024 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1f0414e850 con 0x7f1f04148a30 2024-04-25T01:29:25.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.067+0000 7f1f0a04e640 1 --2- 172.21.15.112:0/309346024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f04148a30 0x7f1f0414ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:25.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.067+0000 7f1f0a04e640 1 -- 172.21.15.112:0/309346024 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f04075980 con 0x7f1f04148a30 2024-04-25T01:29:25.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.067+0000 7f1f0a04e640 1 --2- 172.21.15.112:0/309346024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f04148a30 0x7f1f0414ab70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1f00000f60 tx=0x7f1f0000f9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:25.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.068+0000 7f1efaffd640 1 -- 172.21.15.112:0/309346024 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ef401a070 con 0x7f1f04071ce0 2024-04-25T01:29:25.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.068+0000 7f1efaffd640 1 --2- 172.21.15.112:0/309346024 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1ee806be30 0x7f1ee806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:25.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.068+0000 7f1f0a84f640 1 --2- 172.21.15.112:0/309346024 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1ee806be30 0x7f1ee806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:25.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.068+0000 7f1efaffd640 1 -- 172.21.15.112:0/309346024 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f1ef408b470 con 0x7f1f04071ce0 2024-04-25T01:29:25.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.069+0000 7f1efaffd640 1 -- 172.21.15.112:0/309346024 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00005ac0 con 0x7f1f04148a30 2024-04-25T01:29:25.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.069+0000 7f1efaffd640 1 -- 172.21.15.112:0/309346024 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1f00018070 con 0x7f1f04148a30 2024-04-25T01:29:25.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.069+0000 7f1efaffd640 1 -- 172.21.15.112:0/309346024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f04148a30 msgr2=0x7f1f0414ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:25.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.069+0000 7f1efaffd640 1 --2- 172.21.15.112:0/309346024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f04148a30 0x7f1f0414ab70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1f00000f60 tx=0x7f1f0000f9d0 comp rx=0 tx=0).stop 2024-04-25T01:29:25.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.069+0000 7f1efaffd640 1 -- 172.21.15.112:0/309346024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f04148a30 msgr2=0x7f1f0414ab70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:29:25.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.069+0000 7f1efaffd640 1 --2- 172.21.15.112:0/309346024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f04148a30 0x7f1f0414ab70 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1f00000f60 tx=0x7f1f0000f9d0 comp rx=0 tx=0).stop 2024-04-25T01:29:25.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.069+0000 7f1f0a84f640 1 --2- 172.21.15.112:0/309346024 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1ee806be30 0x7f1ee806e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1efc034000 tx=0x7f1efc02f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:25.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.076+0000 7f1f0c2b2640 1 --2- 172.21.15.112:0/309346024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f041f3520 0x7f1f041f3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:25.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.076+0000 7f1f0c2b2640 1 -- 172.21.15.112:0/309346024 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1f0414b0b0 con 0x7f1f041f3520 2024-04-25T01:29:25.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.076+0000 7f1f0a04e640 1 --2- 172.21.15.112:0/309346024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f041f3520 0x7f1f041f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:25.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.076+0000 7f1f0a04e640 1 -- 172.21.15.112:0/309346024 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1f040752c0 con 0x7f1f041f3520 2024-04-25T01:29:25.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.076+0000 7f1f0a04e640 1 --2- 172.21.15.112:0/309346024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f041f3520 0x7f1f041f3930 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1f00000f60 tx=0x7f1f000057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:25.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.077+0000 7f1efaffd640 1 -- 172.21.15.112:0/309346024 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0001d7c0 con 0x7f1f041f3520 2024-04-25T01:29:25.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.077+0000 7f1efaffd640 1 -- 172.21.15.112:0/309346024 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4728 (secure 0 0 0) 0x7f1f0001d7c0 con 0x7f1f041f3520 2024-04-25T01:29:25.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.077+0000 7f1efaffd640 1 -- 172.21.15.112:0/309346024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f041f3520 msgr2=0x7f1f041f3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:25.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.077+0000 7f1efaffd640 1 --2- 172.21.15.112:0/309346024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f041f3520 0x7f1f041f3930 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1f00000f60 tx=0x7f1f000057f0 comp rx=0 tx=0).stop 2024-04-25T01:29:25.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.077+0000 7f1efaffd640 1 -- 172.21.15.112:0/309346024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f041f3520 msgr2=0x7f1f041f3930 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:29:25.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.077+0000 7f1efaffd640 1 --2- 172.21.15.112:0/309346024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f041f3520 0x7f1f041f3930 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1f00000f60 tx=0x7f1f000057f0 comp rx=0 tx=0).stop 2024-04-25T01:29:25.056 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:29:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:29:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:29:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:29:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 26, 2024-04-25T01:29:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 65021, 2024-04-25T01:29:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:29:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:29:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:29:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:29:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:29:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:29:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:29:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:29:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:29:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:29:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:29:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:29:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:29:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:29:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:29:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:29:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:29:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:29:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:29:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:29:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:29:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:29:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:29:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:29:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:29:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:29:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:29:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:29:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:29:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:29:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:29:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:29:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:29:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:29:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:29:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:29:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:29:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:29:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:29:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:29:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:29:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:29:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:29:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:29:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:29:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:29:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:29:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:29:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:29:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:29:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:29:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:29:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:29:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:29:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:25.068 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:29:25.068 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:29:25.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.077+0000 7f1f0c2b2640 1 -- 172.21.15.112:0/309346024 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1ee806be30 msgr2=0x7f1ee806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:25.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.077+0000 7f1f0c2b2640 1 --2- 172.21.15.112:0/309346024 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1ee806be30 0x7f1ee806e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1efc034000 tx=0x7f1efc02f000 comp rx=0 tx=0).stop 2024-04-25T01:29:25.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.077+0000 7f1f0c2b2640 1 -- 172.21.15.112:0/309346024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f04071ce0 msgr2=0x7f1f04154720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:25.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.077+0000 7f1f0c2b2640 1 --2- 172.21.15.112:0/309346024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f04071ce0 0x7f1f04154720 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400e9b0 tx=0x7f1ef400ee70 comp rx=0 tx=0).stop 2024-04-25T01:29:25.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.077+0000 7f1f0a84f640 1 -- 172.21.15.112:0/309346024 reap_dead start 2024-04-25T01:29:25.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.077+0000 7f1f0a84f640 1 -- 172.21.15.112:0/309346024 reap_dead start 2024-04-25T01:29:25.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.077+0000 7f1f0c2b2640 1 -- 172.21.15.112:0/309346024 shutdown_connections 2024-04-25T01:29:25.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.077+0000 7f1f0c2b2640 1 -- 172.21.15.112:0/309346024 >> 172.21.15.112:0/309346024 conn(0x7f1f040c8090 msgr2=0x7f1f040c3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:25.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.077+0000 7f1f0c2b2640 1 -- 172.21.15.112:0/309346024 shutdown_connections 2024-04-25T01:29:25.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.077+0000 7f1f0c2b2640 1 -- 172.21.15.112:0/309346024 wait complete. 2024-04-25T01:29:25.070 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:29:25.070 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T01:29:25.070 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T01:29:25.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.268+0000 7f7e53f89640 1 Processor -- start 2024-04-25T01:29:25.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.269+0000 7f7e53f89640 1 -- start start 2024-04-25T01:29:25.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.269+0000 7f7e53f89640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e4c1596a0 0x7f7e4c152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:25.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.270+0000 7f7e53f89640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e4c159b90 con 0x7f7e4c0c72b0 2024-04-25T01:29:25.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.270+0000 7f7e53f89640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e4c06c970 con 0x7f7e4c1592c0 2024-04-25T01:29:25.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.270+0000 7f7e53f89640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7e4c0c6e20 con 0x7f7e4c1596a0 2024-04-25T01:29:25.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.270+0000 7f7e52526640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7e4c0c72b0 0x7f7e4c160380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:40972/0 (socket says 172.21.15.112:40972) 2024-04-25T01:29:25.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.270+0000 7f7e52526640 1 -- 172.21.15.112:0/4089422474 learned_addr learned my addr 172.21.15.112:0/4089422474 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:29:25.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.270+0000 7f7e51524640 1 --2- 172.21.15.112:0/4089422474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e4c1596a0 0x7f7e4c152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:25.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.270+0000 7f7e50d23640 1 -- 172.21.15.112:0/4089422474 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1590575772 0 0) 0x7f7e4c159b90 con 0x7f7e4c0c72b0 2024-04-25T01:29:25.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.270+0000 7f7e50d23640 1 -- 172.21.15.112:0/4089422474 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e38003520 con 0x7f7e4c0c72b0 2024-04-25T01:29:25.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.270+0000 7f7e50d23640 1 -- 172.21.15.112:0/4089422474 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1738739986 0 0) 0x7f7e4c06c970 con 0x7f7e4c1592c0 2024-04-25T01:29:25.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.270+0000 7f7e50d23640 1 -- 172.21.15.112:0/4089422474 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e4c159b90 con 0x7f7e4c1592c0 2024-04-25T01:29:25.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.270+0000 7f7e51524640 1 -- 172.21.15.112:0/4089422474 >> v1:172.21.15.187:6789/0 conn(0x7f7e4c1592c0 legacy=0x7f7e4c0c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:25.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.270+0000 7f7e51524640 1 -- 172.21.15.112:0/4089422474 >> v1:172.21.15.112:6789/0 conn(0x7f7e4c0c72b0 legacy=0x7f7e4c160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:25.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.270+0000 7f7e50d23640 1 -- 172.21.15.112:0/4089422474 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3297915317 0 0) 0x7f7e38003520 con 0x7f7e4c0c72b0 2024-04-25T01:29:25.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.270+0000 7f7e51524640 1 -- 172.21.15.112:0/4089422474 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e4c06cc50 con 0x7f7e4c1596a0 2024-04-25T01:29:25.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.272+0000 7f7e51524640 1 --2- 172.21.15.112:0/4089422474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e4c1596a0 0x7f7e4c152d00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c0097d0 tx=0x7f7e3c02a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7459c62e125e92e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:25.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.272+0000 7f7e50d23640 1 -- 172.21.15.112:0/4089422474 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c037070 con 0x7f7e4c1596a0 2024-04-25T01:29:25.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.272+0000 7f7e50d23640 1 -- 172.21.15.112:0/4089422474 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e3c02ab10 con 0x7f7e4c1596a0 2024-04-25T01:29:25.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.272+0000 7f7e50d23640 1 -- 172.21.15.112:0/4089422474 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e44001a10 con 0x7f7e4c1596a0 2024-04-25T01:29:25.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.273+0000 7f7e53f89640 1 -- 172.21.15.112:0/4089422474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e4c1596a0 msgr2=0x7f7e4c152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:25.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.273+0000 7f7e53f89640 1 --2- 172.21.15.112:0/4089422474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e4c1596a0 0x7f7e4c152d00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c0097d0 tx=0x7f7e3c02a080 comp rx=0 tx=0).stop 2024-04-25T01:29:25.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.273+0000 7f7e53f89640 1 -- 172.21.15.112:0/4089422474 shutdown_connections 2024-04-25T01:29:25.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.273+0000 7f7e53f89640 1 --2- 172.21.15.112:0/4089422474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e4c1596a0 0x7f7e4c152d00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:25.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.273+0000 7f7e53f89640 1 -- 172.21.15.112:0/4089422474 >> 172.21.15.112:0/4089422474 conn(0x7f7e4c0cb170 msgr2=0x7f7e4c0cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:25.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.273+0000 7f7e53f89640 1 -- 172.21.15.112:0/4089422474 shutdown_connections 2024-04-25T01:29:25.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.274+0000 7f7e53f89640 1 -- 172.21.15.112:0/4089422474 wait complete. 2024-04-25T01:29:25.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.275+0000 7f7e53f89640 1 Processor -- start 2024-04-25T01:29:25.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.275+0000 7f7e53f89640 1 -- start start 2024-04-25T01:29:25.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.276+0000 7f7e53f89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e4c0c72b0 0x7f7e4c1f32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:25.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.276+0000 7f7e53f89640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e4c1592c0 0x7f7e4c0b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:25.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.276+0000 7f7e53f89640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e4c1596a0 0x7f7e4c0b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:25.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.276+0000 7f7e53f89640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4c06c510 con 0x7f7e4c0c72b0 2024-04-25T01:29:25.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.276+0000 7f7e53f89640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4c06c390 con 0x7f7e4c1592c0 2024-04-25T01:29:25.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.276+0000 7f7e53f89640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7e4c06db20 con 0x7f7e4c1596a0 2024-04-25T01:29:25.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.276+0000 7f7e51d25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e4c0c72b0 0x7f7e4c1f32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:25.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.276+0000 7f7e51d25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e4c0c72b0 0x7f7e4c1f32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38692/0 (socket says 172.21.15.112:38692) 2024-04-25T01:29:25.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.276+0000 7f7e51d25640 1 -- 172.21.15.112:0/199092890 learned_addr learned my addr 172.21.15.112:0/199092890 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:25.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.276+0000 7f7e52526640 1 --2- 172.21.15.112:0/199092890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e4c1596a0 0x7f7e4c0b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:25.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.277+0000 7f7e51524640 1 --2- 172.21.15.112:0/199092890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e4c1592c0 0x7f7e4c0b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:25.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.277+0000 7f7e52526640 1 -- 172.21.15.112:0/199092890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e4c1592c0 msgr2=0x7f7e4c0b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:25.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.277+0000 7f7e52526640 1 --2- 172.21.15.112:0/199092890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e4c1592c0 0x7f7e4c0b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:25.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.277+0000 7f7e52526640 1 -- 172.21.15.112:0/199092890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e4c0c72b0 msgr2=0x7f7e4c1f32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:25.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.277+0000 7f7e52526640 1 --2- 172.21.15.112:0/199092890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e4c0c72b0 0x7f7e4c1f32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:25.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.277+0000 7f7e52526640 1 -- 172.21.15.112:0/199092890 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e4c14ee10 con 0x7f7e4c1596a0 2024-04-25T01:29:25.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.277+0000 7f7e51d25640 1 --2- 172.21.15.112:0/199092890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e4c0c72b0 0x7f7e4c1f32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:29:25.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.277+0000 7f7e52526640 1 --2- 172.21.15.112:0/199092890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e4c1596a0 0x7f7e4c0b0b60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7e44001580 tx=0x7f7e4400cf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:25.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.277+0000 7f7e427fc640 1 -- 172.21.15.112:0/199092890 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e44018070 con 0x7f7e4c1596a0 2024-04-25T01:29:25.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.277+0000 7f7e427fc640 1 -- 172.21.15.112:0/199092890 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e4400a610 con 0x7f7e4c1596a0 2024-04-25T01:29:25.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.277+0000 7f7e427fc640 1 -- 172.21.15.112:0/199092890 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4400aa00 con 0x7f7e4c1596a0 2024-04-25T01:29:25.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.277+0000 7f7e53f89640 1 -- 172.21.15.112:0/199092890 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e4c14f0f0 con 0x7f7e4c1596a0 2024-04-25T01:29:25.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.277+0000 7f7e53f89640 1 -- 172.21.15.112:0/199092890 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e4c14f6b0 con 0x7f7e4c1596a0 2024-04-25T01:29:25.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.278+0000 7f7e53f89640 1 -- 172.21.15.112:0/199092890 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e4c0679a0 con 0x7f7e4c1596a0 2024-04-25T01:29:25.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.279+0000 7f7e427fc640 1 -- 172.21.15.112:0/199092890 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e4400aba0 con 0x7f7e4c1596a0 2024-04-25T01:29:25.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.280+0000 7f7e427fc640 1 --2- 172.21.15.112:0/199092890 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7e3006be30 0x7f7e3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:25.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.280+0000 7f7e427fc640 1 -- 172.21.15.112:0/199092890 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f7e4408a7c0 con 0x7f7e4c1596a0 2024-04-25T01:29:25.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.280+0000 7f7e51d25640 1 --2- 172.21.15.112:0/199092890 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7e3006be30 0x7f7e3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:25.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.280+0000 7f7e51d25640 1 --2- 172.21.15.112:0/199092890 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7e3006be30 0x7f7e3006e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7e48007fd0 tx=0x7f7e48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:25.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.286+0000 7f7e427fc640 1 -- 172.21.15.112:0/199092890 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7e44058780 con 0x7f7e4c1596a0 2024-04-25T01:29:25.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.624+0000 7f7e53f89640 1 -- 172.21.15.112:0/199092890 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f7e4c14f880 con 0x7f7e4c1596a0 2024-04-25T01:29:25.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.625+0000 7f7e427fc640 1 -- 172.21.15.112:0/199092890 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f7e4405ca80 con 0x7f7e4c1596a0 2024-04-25T01:29:25.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.627+0000 7f7e53f89640 1 -- 172.21.15.112:0/199092890 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7e3006be30 msgr2=0x7f7e3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:25.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.627+0000 7f7e53f89640 1 --2- 172.21.15.112:0/199092890 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7e3006be30 0x7f7e3006e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7e48007fd0 tx=0x7f7e48009040 comp rx=0 tx=0).stop 2024-04-25T01:29:25.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.627+0000 7f7e53f89640 1 -- 172.21.15.112:0/199092890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e4c1596a0 msgr2=0x7f7e4c0b0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:25.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.627+0000 7f7e53f89640 1 --2- 172.21.15.112:0/199092890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e4c1596a0 0x7f7e4c0b0b60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7e44001580 tx=0x7f7e4400cf50 comp rx=0 tx=0).stop 2024-04-25T01:29:25.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.627+0000 7f7e53f89640 1 -- 172.21.15.112:0/199092890 shutdown_connections 2024-04-25T01:29:25.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.627+0000 7f7e53f89640 1 --2- 172.21.15.112:0/199092890 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7e3006be30 0x7f7e3006e2f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:25.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.627+0000 7f7e53f89640 1 --2- 172.21.15.112:0/199092890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e4c1596a0 0x7f7e4c0b0b60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:25.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.627+0000 7f7e53f89640 1 --2- 172.21.15.112:0/199092890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e4c1592c0 0x7f7e4c0b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:25.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.627+0000 7f7e53f89640 1 --2- 172.21.15.112:0/199092890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e4c0c72b0 0x7f7e4c1f32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:25.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.627+0000 7f7e53f89640 1 -- 172.21.15.112:0/199092890 >> 172.21.15.112:0/199092890 conn(0x7f7e4c0cb170 msgr2=0x7f7e4c158890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:25.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.627+0000 7f7e53f89640 1 -- 172.21.15.112:0/199092890 shutdown_connections 2024-04-25T01:29:25.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:25.627+0000 7f7e53f89640 1 -- 172.21.15.112:0/199092890 wait complete. 2024-04-25T01:29:25.619 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T01:29:35.620 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:29:35.620 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:29:35.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.752+0000 7fd808922640 1 Processor -- start 2024-04-25T01:29:35.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.752+0000 7fd808922640 1 -- start start 2024-04-25T01:29:35.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.753+0000 7fd808922640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd804152fd0 0x7fd8040c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:35.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.753+0000 7fd808922640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd8040c5d70 0x7fd804156280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:35.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.753+0000 7fd808922640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8041567c0 con 0x7fd804156cf0 2024-04-25T01:29:35.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.753+0000 7fd808922640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd8040ca9b0 con 0x7fd8040c5d70 2024-04-25T01:29:35.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.753+0000 7fd808922640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd8040c90e0 con 0x7fd804152fd0 2024-04-25T01:29:35.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.753+0000 7fd801d9b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd8040c5d70 0x7fd804156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:35.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.753+0000 7fd80259c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd804152fd0 0x7fd8040c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:35.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.753+0000 7fd801d9b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd8040c5d70 0x7fd804156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:57234/0 (socket says 172.21.15.112:57234) 2024-04-25T01:29:35.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.753+0000 7fd80259c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd804152fd0 0x7fd8040c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:34274/0 (socket says 172.21.15.112:34274) 2024-04-25T01:29:35.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.753+0000 7fd80259c640 1 -- 172.21.15.112:0/2536899670 learned_addr learned my addr 172.21.15.112:0/2536899670 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:35.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.753+0000 7fd802d9d640 1 --1- 172.21.15.112:0/2536899670 >> v1:172.21.15.187:6789/0 conn(0x7fd804156cf0 0x7fd80406bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:45266/0 (socket says 172.21.15.112:45266) 2024-04-25T01:29:35.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.753+0000 7fd80259c640 1 -- 172.21.15.112:0/2536899670 >> v2:172.21.15.187:3300/0 conn(0x7fd8040c5d70 msgr2=0x7fd804156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:29:35.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.753+0000 7fd80259c640 1 --2- 172.21.15.112:0/2536899670 >> v2:172.21.15.187:3300/0 conn(0x7fd8040c5d70 0x7fd804156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:35.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.753+0000 7fd80259c640 1 -- 172.21.15.112:0/2536899670 >> v1:172.21.15.187:6789/0 conn(0x7fd804156cf0 legacy=0x7fd80406bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:35.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.753+0000 7fd80259c640 1 -- 172.21.15.112:0/2536899670 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8040cac00 con 0x7fd804152fd0 2024-04-25T01:29:35.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.753+0000 7fd801d9b640 1 --2- 172.21.15.112:0/2536899670 >> v2:172.21.15.187:3300/0 conn(0x7fd8040c5d70 0x7fd804156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:29:35.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.754+0000 7fd80259c640 1 --2- 172.21.15.112:0/2536899670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd804152fd0 0x7fd8040c9b80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4009870 tx=0x7fd7f402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b32493728008fb8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:35.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.755+0000 7fd80159a640 1 -- 172.21.15.112:0/2536899670 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4037070 con 0x7fd804152fd0 2024-04-25T01:29:35.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.755+0000 7fd80159a640 1 -- 172.21.15.112:0/2536899670 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f4032440 con 0x7fd804152fd0 2024-04-25T01:29:35.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.755+0000 7fd80159a640 1 -- 172.21.15.112:0/2536899670 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4032810 con 0x7fd804152fd0 2024-04-25T01:29:35.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.755+0000 7fd808922640 1 -- 172.21.15.112:0/2536899670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd804152fd0 msgr2=0x7fd8040c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:35.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.755+0000 7fd808922640 1 --2- 172.21.15.112:0/2536899670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd804152fd0 0x7fd8040c9b80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4009870 tx=0x7fd7f402a1c0 comp rx=0 tx=0).stop 2024-04-25T01:29:35.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.755+0000 7fd808922640 1 -- 172.21.15.112:0/2536899670 shutdown_connections 2024-04-25T01:29:35.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.755+0000 7fd808922640 1 --2- 172.21.15.112:0/2536899670 >> v2:172.21.15.187:3300/0 conn(0x7fd8040c5d70 0x7fd804156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:35.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.755+0000 7fd808922640 1 --2- 172.21.15.112:0/2536899670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd804152fd0 0x7fd8040c9b80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:35.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.755+0000 7fd808922640 1 -- 172.21.15.112:0/2536899670 >> 172.21.15.112:0/2536899670 conn(0x7fd8040acc70 msgr2=0x7fd8040ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:35.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.755+0000 7fd808922640 1 -- 172.21.15.112:0/2536899670 shutdown_connections 2024-04-25T01:29:35.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.755+0000 7fd808922640 1 -- 172.21.15.112:0/2536899670 wait complete. 2024-04-25T01:29:35.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.757+0000 7fd808922640 1 Processor -- start 2024-04-25T01:29:35.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.757+0000 7fd808922640 1 -- start start 2024-04-25T01:29:35.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd808922640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8040c5d70 0x7fd80414c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:35.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd808922640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd804152fd0 0x7fd8040af590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:35.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd808922640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd804156cf0 0x7fd804155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:35.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd808922640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd8040caee0 con 0x7fd8040c5d70 2024-04-25T01:29:35.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd808922640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd8040c9660 con 0x7fd804152fd0 2024-04-25T01:29:35.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd808922640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd80406c4b0 con 0x7fd804156cf0 2024-04-25T01:29:35.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd80259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8040c5d70 0x7fd80414c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:35.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd80259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8040c5d70 0x7fd80414c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34684/0 (socket says 172.21.15.112:34684) 2024-04-25T01:29:35.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd80259c640 1 -- 172.21.15.112:0/2657696164 learned_addr learned my addr 172.21.15.112:0/2657696164 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:35.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd801d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd804152fd0 0x7fd8040af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:35.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd802d9d640 1 --2- 172.21.15.112:0/2657696164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd804156cf0 0x7fd804155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:35.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd80259c640 1 -- 172.21.15.112:0/2657696164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd804156cf0 msgr2=0x7fd804155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:35.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd80259c640 1 --2- 172.21.15.112:0/2657696164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd804156cf0 0x7fd804155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:35.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd80259c640 1 -- 172.21.15.112:0/2657696164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd804152fd0 msgr2=0x7fd8040af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:35.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd80259c640 1 --2- 172.21.15.112:0/2657696164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd804152fd0 0x7fd8040af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:35.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd80259c640 1 -- 172.21.15.112:0/2657696164 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8040c50e0 con 0x7fd8040c5d70 2024-04-25T01:29:35.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd801d9b640 1 --2- 172.21.15.112:0/2657696164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd804152fd0 0x7fd8040af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:29:35.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd80259c640 1 --2- 172.21.15.112:0/2657696164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8040c5d70 0x7fd80414c540 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4000c00 tx=0x7fd7f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:35.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd7f2ffd640 1 -- 172.21.15.112:0/2657696164 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4037040 con 0x7fd8040c5d70 2024-04-25T01:29:35.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd7f2ffd640 1 -- 172.21.15.112:0/2657696164 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f4042070 con 0x7fd8040c5d70 2024-04-25T01:29:35.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd7f2ffd640 1 -- 172.21.15.112:0/2657696164 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4030880 con 0x7fd8040c5d70 2024-04-25T01:29:35.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.758+0000 7fd808922640 1 -- 172.21.15.112:0/2657696164 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80414d2e0 con 0x7fd8040c5d70 2024-04-25T01:29:35.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.759+0000 7fd808922640 1 -- 172.21.15.112:0/2657696164 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8041ef580 con 0x7fd8040c5d70 2024-04-25T01:29:35.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.760+0000 7fd808922640 1 -- 172.21.15.112:0/2657696164 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8040682a0 con 0x7fd8040c5d70 2024-04-25T01:29:35.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.760+0000 7fd7f2ffd640 1 -- 172.21.15.112:0/2657696164 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7f402ad10 con 0x7fd8040c5d70 2024-04-25T01:29:35.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.761+0000 7fd7f2ffd640 1 --2- 172.21.15.112:0/2657696164 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd7e406be30 0x7fd7e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:35.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.761+0000 7fd7f2ffd640 1 -- 172.21.15.112:0/2657696164 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fd7f40ab530 con 0x7fd8040c5d70 2024-04-25T01:29:35.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.761+0000 7fd801d9b640 1 --2- 172.21.15.112:0/2657696164 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd7e406be30 0x7fd7e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:35.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.761+0000 7fd801d9b640 1 --2- 172.21.15.112:0/2657696164 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd7e406be30 0x7fd7e406e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd80406d150 tx=0x7fd7f8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:35.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:35.765+0000 7fd7f2ffd640 1 -- 172.21.15.112:0/2657696164 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd7f4079580 con 0x7fd8040c5d70 2024-04-25T01:29:36.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.049+0000 7fd808922640 1 -- 172.21.15.112:0/2657696164 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd8040c5ac0 con 0x7fd8040c5d70 2024-04-25T01:29:36.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.049+0000 7fd7f2ffd640 1 -- 172.21.15.112:0/2657696164 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7fd7f407d880 con 0x7fd8040c5d70 2024-04-25T01:29:36.028 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":26,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:29:36.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.052+0000 7fd808922640 1 -- 172.21.15.112:0/2657696164 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd7e406be30 msgr2=0x7fd7e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.052+0000 7fd808922640 1 --2- 172.21.15.112:0/2657696164 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd7e406be30 0x7fd7e406e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd80406d150 tx=0x7fd7f8009190 comp rx=0 tx=0).stop 2024-04-25T01:29:36.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.052+0000 7fd808922640 1 -- 172.21.15.112:0/2657696164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8040c5d70 msgr2=0x7fd80414c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.052+0000 7fd808922640 1 --2- 172.21.15.112:0/2657696164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8040c5d70 0x7fd80414c540 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4000c00 tx=0x7fd7f40046c0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.052+0000 7fd808922640 1 -- 172.21.15.112:0/2657696164 shutdown_connections 2024-04-25T01:29:36.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.052+0000 7fd808922640 1 --2- 172.21.15.112:0/2657696164 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd7e406be30 0x7fd7e406e2f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.052+0000 7fd808922640 1 --2- 172.21.15.112:0/2657696164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd804156cf0 0x7fd804155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.052+0000 7fd808922640 1 --2- 172.21.15.112:0/2657696164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd804152fd0 0x7fd8040af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.052+0000 7fd808922640 1 --2- 172.21.15.112:0/2657696164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8040c5d70 0x7fd80414c540 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.052+0000 7fd808922640 1 -- 172.21.15.112:0/2657696164 >> 172.21.15.112:0/2657696164 conn(0x7fd8040acc70 msgr2=0x7fd8040ab930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:36.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.052+0000 7fd808922640 1 -- 172.21.15.112:0/2657696164 shutdown_connections 2024-04-25T01:29:36.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.052+0000 7fd808922640 1 -- 172.21.15.112:0/2657696164 wait complete. 2024-04-25T01:29:36.043 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:29:36.043 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T01:29:36.043 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:29:36.043 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:29:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.172+0000 7f85b9963640 1 Processor -- start 2024-04-25T01:29:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.172+0000 7f85b9963640 1 -- start start 2024-04-25T01:29:36.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.172+0000 7f85b9963640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f85b41596a0 0x7f85b4152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.172+0000 7f85b9963640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f85b40c72b0 0x7f85b4160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.173+0000 7f85b9963640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b4159b90 con 0x7f85b41592c0 2024-04-25T01:29:36.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.173+0000 7f85b9963640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f85b40c6e20 con 0x7f85b41596a0 2024-04-25T01:29:36.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.173+0000 7f85b9963640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f85b406c970 con 0x7f85b40c72b0 2024-04-25T01:29:36.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.173+0000 7f85abfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f85b41596a0 0x7f85b4152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.173+0000 7f85abfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f85b41596a0 0x7f85b4152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34688/0 (socket says 172.21.15.112:34688) 2024-04-25T01:29:36.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.173+0000 7f85abfff640 1 -- 172.21.15.112:0/211986891 learned_addr learned my addr 172.21.15.112:0/211986891 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:36.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.173+0000 7f85b37fe640 1 --2- 172.21.15.112:0/211986891 >> v2:172.21.15.187:3300/0 conn(0x7f85b40c72b0 0x7f85b4160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.173+0000 7f85abfff640 1 -- 172.21.15.112:0/211986891 >> v2:172.21.15.187:3300/0 conn(0x7f85b40c72b0 msgr2=0x7f85b4160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:29:36.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.173+0000 7f85abfff640 1 --2- 172.21.15.112:0/211986891 >> v2:172.21.15.187:3300/0 conn(0x7f85b40c72b0 0x7f85b4160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.173+0000 7f85abfff640 1 -- 172.21.15.112:0/211986891 >> v1:172.21.15.187:6789/0 conn(0x7f85b41592c0 legacy=0x7f85b40c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.173+0000 7f85abfff640 1 -- 172.21.15.112:0/211986891 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b406cc50 con 0x7f85b41596a0 2024-04-25T01:29:36.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.173+0000 7f85b37fe640 1 --2- 172.21.15.112:0/211986891 >> v2:172.21.15.187:3300/0 conn(0x7f85b40c72b0 0x7f85b4160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:29:36.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.174+0000 7f85abfff640 1 --2- 172.21.15.112:0/211986891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41596a0 0x7f85b4152d00 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f85b40c21d0 tx=0x7f859c02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e8d0c0161f552c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:36.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.174+0000 7f85b27fc640 1 -- 172.21.15.112:0/211986891 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c037070 con 0x7f85b41596a0 2024-04-25T01:29:36.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.174+0000 7f85b27fc640 1 -- 172.21.15.112:0/211986891 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f859c032440 con 0x7f85b41596a0 2024-04-25T01:29:36.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.174+0000 7f85b27fc640 1 -- 172.21.15.112:0/211986891 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c032810 con 0x7f85b41596a0 2024-04-25T01:29:36.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.175+0000 7f85b9963640 1 -- 172.21.15.112:0/211986891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41596a0 msgr2=0x7f85b4152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.175+0000 7f85b9963640 1 --2- 172.21.15.112:0/211986891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41596a0 0x7f85b4152d00 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f85b40c21d0 tx=0x7f859c02a150 comp rx=0 tx=0).stop 2024-04-25T01:29:36.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.175+0000 7f85b9963640 1 -- 172.21.15.112:0/211986891 shutdown_connections 2024-04-25T01:29:36.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.175+0000 7f85b9963640 1 --2- 172.21.15.112:0/211986891 >> v2:172.21.15.187:3300/0 conn(0x7f85b40c72b0 0x7f85b4160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.175+0000 7f85b9963640 1 --2- 172.21.15.112:0/211986891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41596a0 0x7f85b4152d00 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.175+0000 7f85b9963640 1 -- 172.21.15.112:0/211986891 >> 172.21.15.112:0/211986891 conn(0x7f85b40cb170 msgr2=0x7f85b40cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:36.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.175+0000 7f85b9963640 1 -- 172.21.15.112:0/211986891 shutdown_connections 2024-04-25T01:29:36.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.175+0000 7f85b9963640 1 -- 172.21.15.112:0/211986891 wait complete. 2024-04-25T01:29:36.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.176+0000 7f85b9963640 1 Processor -- start 2024-04-25T01:29:36.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.176+0000 7f85b9963640 1 -- start start 2024-04-25T01:29:36.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85b9963640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85b40c72b0 0x7f85b414d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85b9963640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41592c0 0x7f85b4158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85b9963640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85b41596a0 0x7f85b41ec110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85b9963640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85b406c510 con 0x7f85b41592c0 2024-04-25T01:29:36.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85b9963640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f85b406c390 con 0x7f85b40c72b0 2024-04-25T01:29:36.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85b9963640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f85b406db20 con 0x7f85b41596a0 2024-04-25T01:29:36.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85abfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41592c0 0x7f85b4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85abfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41592c0 0x7f85b4158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34704/0 (socket says 172.21.15.112:34704) 2024-04-25T01:29:36.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85abfff640 1 -- 172.21.15.112:0/2737462887 learned_addr learned my addr 172.21.15.112:0/2737462887 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:36.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85b2ffd640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85b40c72b0 0x7f85b414d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85b37fe640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85b41596a0 0x7f85b41ec110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85abfff640 1 -- 172.21.15.112:0/2737462887 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85b41596a0 msgr2=0x7f85b41ec110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85abfff640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85b41596a0 0x7f85b41ec110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85abfff640 1 -- 172.21.15.112:0/2737462887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85b40c72b0 msgr2=0x7f85b414d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85abfff640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85b40c72b0 0x7f85b414d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85abfff640 1 -- 172.21.15.112:0/2737462887 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b41eb970 con 0x7f85b41592c0 2024-04-25T01:29:36.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85abfff640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41592c0 0x7f85b4158110 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f859c000c00 tx=0x7f859c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:36.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85b0ff9640 1 -- 172.21.15.112:0/2737462887 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c041020 con 0x7f85b41592c0 2024-04-25T01:29:36.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85b0ff9640 1 -- 172.21.15.112:0/2737462887 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f859c0306c0 con 0x7f85b41592c0 2024-04-25T01:29:36.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85b9963640 1 -- 172.21.15.112:0/2737462887 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b41540d0 con 0x7f85b41592c0 2024-04-25T01:29:36.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85b0ff9640 1 -- 172.21.15.112:0/2737462887 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c037040 con 0x7f85b41592c0 2024-04-25T01:29:36.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.177+0000 7f85b9963640 1 -- 172.21.15.112:0/2737462887 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b40c1e70 con 0x7f85b41592c0 2024-04-25T01:29:36.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.178+0000 7f85b9963640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41ec650 0x7f85b41538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.178+0000 7f85b9963640 1 -- 172.21.15.112:0/2737462887 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85b4153e80 con 0x7f85b41ec650 2024-04-25T01:29:36.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.178+0000 7f85b2ffd640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41ec650 0x7f85b41538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.179+0000 7f85b2ffd640 1 -- 172.21.15.112:0/2737462887 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f85b4075980 con 0x7f85b41ec650 2024-04-25T01:29:36.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.179+0000 7f85b2ffd640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41ec650 0x7f85b41538d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f85a400e420 tx=0x7f85a400e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:36.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.179+0000 7f85b0ff9640 1 -- 172.21.15.112:0/2737462887 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f859c044020 con 0x7f85b41592c0 2024-04-25T01:29:36.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.179+0000 7f85b0ff9640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f859406be30 0x7f859406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.179+0000 7f85b0ff9640 1 -- 172.21.15.112:0/2737462887 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f859c0ab9b0 con 0x7f85b41592c0 2024-04-25T01:29:36.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.179+0000 7f85b37fe640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f859406be30 0x7f859406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.179+0000 7f85b0ff9640 1 -- 172.21.15.112:0/2737462887 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a40052e0 con 0x7f85b41ec650 2024-04-25T01:29:36.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.179+0000 7f85b0ff9640 1 -- 172.21.15.112:0/2737462887 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f85a4018070 con 0x7f85b41ec650 2024-04-25T01:29:36.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.180+0000 7f85b0ff9640 1 -- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41ec650 msgr2=0x7f85b41538d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.180+0000 7f85b0ff9640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41ec650 0x7f85b41538d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f85a400e420 tx=0x7f85a400e8e0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.180+0000 7f85b0ff9640 1 -- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41ec650 msgr2=0x7f85b41538d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:29:36.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.180+0000 7f85b0ff9640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41ec650 0x7f85b41538d0 secure :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f85a400e420 tx=0x7f85a400e8e0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.180+0000 7f85b37fe640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f859406be30 0x7f859406e2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009a00 tx=0x7f85ac003ce0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:36.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.189+0000 7f85b9963640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41543d0 0x7f85b414dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.189+0000 7f85b9963640 1 -- 172.21.15.112:0/2737462887 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85b40679a0 con 0x7f85b41543d0 2024-04-25T01:29:36.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.190+0000 7f85b2ffd640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41543d0 0x7f85b414dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.190+0000 7f85b2ffd640 1 -- 172.21.15.112:0/2737462887 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f85b40cc050 con 0x7f85b41543d0 2024-04-25T01:29:36.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.190+0000 7f85b2ffd640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41543d0 0x7f85b414dae0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f85a400e420 tx=0x7f85a4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:36.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.190+0000 7f85b0ff9640 1 -- 172.21.15.112:0/2737462887 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a40037f0 con 0x7f85b41543d0 2024-04-25T01:29:36.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.190+0000 7f85b0ff9640 1 -- 172.21.15.112:0/2737462887 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5072 (secure 0 0 0) 0x7f85a40037f0 con 0x7f85b41543d0 2024-04-25T01:29:36.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.190+0000 7f85b0ff9640 1 -- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41543d0 msgr2=0x7f85b414dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.190+0000 7f85b0ff9640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41543d0 0x7f85b414dae0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f85a400e420 tx=0x7f85a4000ed0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.190+0000 7f85b0ff9640 1 -- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41543d0 msgr2=0x7f85b414dae0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:29:36.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.190+0000 7f85b0ff9640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41543d0 0x7f85b414dae0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.169 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:29:36.170 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:29:36.170 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:29:36.170 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:29:36.170 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 26, 2024-04-25T01:29:36.170 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 247818, 2024-04-25T01:29:36.170 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:29:36.170 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:29:36.170 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:29:36.170 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:29:36.170 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:36.170 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 11, 2024-04-25T01:29:36.170 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:29:36.170 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:29:36.171 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:29:36.171 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:36.171 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:36.171 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:36.171 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:36.171 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:36.171 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:36.171 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:36.171 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:36.171 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:36.171 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:36.171 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:36.171 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:36.171 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:29:36.171 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:29:36.171 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:36.172 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:36.172 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:36.172 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:36.172 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:36.172 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:36.172 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:36.172 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:36.172 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:36.172 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:36.172 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:36.172 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:36.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.172 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:29:36.172 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:29:36.172 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:29:36.172 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:29:36.172 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:29:36.173 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:29:36.173 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:29:36.173 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:29:36.173 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:29:36.173 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:29:36.173 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:29:36.173 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:29:36.173 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:29:36.173 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:29:36.173 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:29:36.173 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:29:36.173 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:29:36.173 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:36.173 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:36.174 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:36.174 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:36.174 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:36.174 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:36.174 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:36.174 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:36.174 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:36.174 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:36.174 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:36.174 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:36.174 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:29:36.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.174 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:29:36.174 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.174 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:29:36.175 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:29:36.175 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:36.175 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:36.175 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.175 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:36.175 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:29:36.175 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:36.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.175 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.175 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:36.175 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:29:36.175 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:36.175 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.175 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:36.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.176 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:29:36.176 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:29:36.176 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:36.176 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:36.176 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:36.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.176 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.176 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:29:36.176 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:29:36.176 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:36.176 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:36.176 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.176 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:36.176 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:29:36.176 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:36.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.177 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.177 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:36.177 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:29:36.177 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:36.177 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.177 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:36.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.177 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:29:36.177 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:29:36.177 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:36.177 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:36.177 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:36.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.177 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.177 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:29:36.178 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:29:36.178 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:36.178 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:36.178 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.178 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:36.178 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:29:36.178 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:36.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.178 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.178 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:36.178 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:29:36.178 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:36.178 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.178 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:36.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.179 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:29:36.179 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:29:36.179 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:36.179 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:36.179 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:36.179 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.179 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:36.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.179 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:29:36.179 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:29:36.179 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.179 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:36.179 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:36.180 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:29:36.180 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.180 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:36.180 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:29:36.180 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.180 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:36.180 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:36.180 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:29:36.180 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.180 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:36.180 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:29:36.180 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.180 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:36.180 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:36.180 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 7 2024-04-25T01:29:36.181 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.181 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:36.181 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:29:36.181 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.181 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:36.181 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:36.181 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 8 2024-04-25T01:29:36.181 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.181 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:36.181 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:29:36.181 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.181 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:36.181 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:36.181 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:29:36.181 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.182 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:36.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.182 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:29:36.182 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:29:36.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.191+0000 7f85b9963640 1 -- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f859406be30 msgr2=0x7f859406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.191+0000 7f85b9963640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f859406be30 0x7f859406e2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009a00 tx=0x7f85ac003ce0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.191+0000 7f85b9963640 1 -- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41592c0 msgr2=0x7f85b4158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.191+0000 7f85b9963640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41592c0 0x7f85b4158110 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f859c000c00 tx=0x7f859c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.191+0000 7f85b37fe640 1 -- 172.21.15.112:0/2737462887 reap_dead start 2024-04-25T01:29:36.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.191+0000 7f85b9963640 1 -- 172.21.15.112:0/2737462887 shutdown_connections 2024-04-25T01:29:36.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.191+0000 7f85b9963640 1 --2- 172.21.15.112:0/2737462887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b41592c0 0x7f85b4158110 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.191+0000 7f85b9963640 1 -- 172.21.15.112:0/2737462887 >> 172.21.15.112:0/2737462887 conn(0x7f85b40cb170 msgr2=0x7f85b4152ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:36.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.191+0000 7f85b9963640 1 -- 172.21.15.112:0/2737462887 shutdown_connections 2024-04-25T01:29:36.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.191+0000 7f85b9963640 1 -- 172.21.15.112:0/2737462887 wait complete. 2024-04-25T01:29:36.184 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:29:36.184 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:29:36.184 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:29:36.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.316+0000 7f98ec429640 1 Processor -- start 2024-04-25T01:29:36.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.316+0000 7f98ec429640 1 -- start start 2024-04-25T01:29:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.317+0000 7f98ec429640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98e40c72b0 0x7f98e4161390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.317+0000 7f98ec429640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98e4159b90 con 0x7f98e41596a0 2024-04-25T01:29:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.317+0000 7f98ec429640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98e406c970 con 0x7f98e41592c0 2024-04-25T01:29:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.317+0000 7f98ec429640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f98e40c6e20 con 0x7f98e40c72b0 2024-04-25T01:29:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.317+0000 7f98e99c4640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f98e41596a0 0x7f98e4152d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53246/0 (socket says 172.21.15.112:53246) 2024-04-25T01:29:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.317+0000 7f98e99c4640 1 -- 172.21.15.112:0/4145474768 learned_addr learned my addr 172.21.15.112:0/4145474768 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:29:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.317+0000 7f98ea9c6640 1 --2- 172.21.15.112:0/4145474768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98e40c72b0 0x7f98e4161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.317+0000 7f98e91c3640 1 -- 172.21.15.112:0/4145474768 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 370767141 0 0) 0x7f98e4159b90 con 0x7f98e41596a0 2024-04-25T01:29:36.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.317+0000 7f98e91c3640 1 -- 172.21.15.112:0/4145474768 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98d0003520 con 0x7f98e41596a0 2024-04-25T01:29:36.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.317+0000 7f98e91c3640 1 -- 172.21.15.112:0/4145474768 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1425405401 0 0) 0x7f98e406c970 con 0x7f98e41592c0 2024-04-25T01:29:36.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.317+0000 7f98e91c3640 1 -- 172.21.15.112:0/4145474768 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98e4159b90 con 0x7f98e41592c0 2024-04-25T01:29:36.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.318+0000 7f98ea9c6640 1 -- 172.21.15.112:0/4145474768 >> v1:172.21.15.187:6789/0 conn(0x7f98e41592c0 legacy=0x7f98e40c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.318+0000 7f98e91c3640 1 -- 172.21.15.112:0/4145474768 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 575051055 0 0) 0x7f98d0003520 con 0x7f98e41596a0 2024-04-25T01:29:36.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.318+0000 7f98ea9c6640 1 -- 172.21.15.112:0/4145474768 >> v1:172.21.15.112:6789/0 conn(0x7f98e41596a0 legacy=0x7f98e4152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.318+0000 7f98ea9c6640 1 -- 172.21.15.112:0/4145474768 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e406cc50 con 0x7f98e40c72b0 2024-04-25T01:29:36.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.318+0000 7f98ea9c6640 1 --2- 172.21.15.112:0/4145474768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98e40c72b0 0x7f98e4161390 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009dc0 tx=0x7f98dc02a540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54aacf7de18ed1de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:36.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.319+0000 7f98e91c3640 1 -- 172.21.15.112:0/4145474768 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4001690 con 0x7f98e40c72b0 2024-04-25T01:29:36.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.319+0000 7f98e91c3640 1 -- 172.21.15.112:0/4145474768 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98dc007540 con 0x7f98e40c72b0 2024-04-25T01:29:36.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.319+0000 7f98e91c3640 1 -- 172.21.15.112:0/4145474768 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc007910 con 0x7f98e40c72b0 2024-04-25T01:29:36.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.319+0000 7f98ec429640 1 -- 172.21.15.112:0/4145474768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98e40c72b0 msgr2=0x7f98e4161390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.319+0000 7f98ec429640 1 --2- 172.21.15.112:0/4145474768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98e40c72b0 0x7f98e4161390 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009dc0 tx=0x7f98dc02a540 comp rx=0 tx=0).stop 2024-04-25T01:29:36.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.319+0000 7f98ec429640 1 -- 172.21.15.112:0/4145474768 shutdown_connections 2024-04-25T01:29:36.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.319+0000 7f98ec429640 1 --2- 172.21.15.112:0/4145474768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98e40c72b0 0x7f98e4161390 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.319+0000 7f98ec429640 1 -- 172.21.15.112:0/4145474768 >> 172.21.15.112:0/4145474768 conn(0x7f98e40cb170 msgr2=0x7f98e40cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:36.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.319+0000 7f98ec429640 1 -- 172.21.15.112:0/4145474768 shutdown_connections 2024-04-25T01:29:36.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.319+0000 7f98ec429640 1 -- 172.21.15.112:0/4145474768 wait complete. 2024-04-25T01:29:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.320+0000 7f98ec429640 1 Processor -- start 2024-04-25T01:29:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.321+0000 7f98ec429640 1 -- start start 2024-04-25T01:29:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.321+0000 7f98ec429640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e40c72b0 0x7f98e41ef620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.321+0000 7f98ec429640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e41592c0 0x7f98e4158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.321+0000 7f98ec429640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98e41596a0 0x7f98e41eb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.321+0000 7f98ec429640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f98e406c510 con 0x7f98e41592c0 2024-04-25T01:29:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.321+0000 7f98ec429640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f98e406c390 con 0x7f98e40c72b0 2024-04-25T01:29:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.321+0000 7f98ec429640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f98e4153410 con 0x7f98e41596a0 2024-04-25T01:29:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.321+0000 7f98e99c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e41592c0 0x7f98e4158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.321+0000 7f98e99c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e41592c0 0x7f98e4158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34720/0 (socket says 172.21.15.112:34720) 2024-04-25T01:29:36.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.321+0000 7f98e99c4640 1 -- 172.21.15.112:0/3658451704 learned_addr learned my addr 172.21.15.112:0/3658451704 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:36.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.321+0000 7f98ea1c5640 1 --2- 172.21.15.112:0/3658451704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e40c72b0 0x7f98e41ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.321+0000 7f98ea9c6640 1 --2- 172.21.15.112:0/3658451704 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98e41596a0 0x7f98e41eb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.322+0000 7f98e99c4640 1 -- 172.21.15.112:0/3658451704 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98e41596a0 msgr2=0x7f98e41eb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.322+0000 7f98e99c4640 1 --2- 172.21.15.112:0/3658451704 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98e41596a0 0x7f98e41eb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.322+0000 7f98e99c4640 1 -- 172.21.15.112:0/3658451704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e40c72b0 msgr2=0x7f98e41ef620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.322+0000 7f98e99c4640 1 --2- 172.21.15.112:0/3658451704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e40c72b0 0x7f98e41ef620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.322+0000 7f98e99c4640 1 -- 172.21.15.112:0/3658451704 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e41eeb60 con 0x7f98e41592c0 2024-04-25T01:29:36.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.322+0000 7f98e99c4640 1 --2- 172.21.15.112:0/3658451704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e41592c0 0x7f98e4158240 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f98d400c500 tx=0x7f98d400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:36.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.322+0000 7f98daffd640 1 -- 172.21.15.112:0/3658451704 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4017070 con 0x7f98e41592c0 2024-04-25T01:29:36.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.322+0000 7f98daffd640 1 -- 172.21.15.112:0/3658451704 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d4005ce0 con 0x7f98e41592c0 2024-04-25T01:29:36.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.322+0000 7f98ec429640 1 -- 172.21.15.112:0/3658451704 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e4156610 con 0x7f98e41592c0 2024-04-25T01:29:36.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.322+0000 7f98daffd640 1 -- 172.21.15.112:0/3658451704 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4007330 con 0x7f98e41592c0 2024-04-25T01:29:36.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.322+0000 7f98ec429640 1 -- 172.21.15.112:0/3658451704 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e41eee20 con 0x7f98e41592c0 2024-04-25T01:29:36.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.323+0000 7f98ec429640 1 --2- 172.21.15.112:0/3658451704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e40682a0 0x7f98e41538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.323+0000 7f98ec429640 1 -- 172.21.15.112:0/3658451704 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f98e40679a0 con 0x7f98e40682a0 2024-04-25T01:29:36.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.323+0000 7f98ea1c5640 1 --2- 172.21.15.112:0/3658451704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e40682a0 0x7f98e41538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.324+0000 7f98ea1c5640 1 -- 172.21.15.112:0/3658451704 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98e40c79d0 con 0x7f98e40682a0 2024-04-25T01:29:36.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.324+0000 7f98daffd640 1 -- 172.21.15.112:0/3658451704 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f98d401a070 con 0x7f98e41592c0 2024-04-25T01:29:36.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.324+0000 7f98ea1c5640 1 --2- 172.21.15.112:0/3658451704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e40682a0 0x7f98e41538d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f98e000f770 tx=0x7f98e000fc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:36.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.324+0000 7f98daffd640 1 --2- 172.21.15.112:0/3658451704 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f98c806be30 0x7f98c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.324+0000 7f98daffd640 1 -- 172.21.15.112:0/3658451704 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f98d408a570 con 0x7f98e41592c0 2024-04-25T01:29:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.324+0000 7f98ea9c6640 1 --2- 172.21.15.112:0/3658451704 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f98c806be30 0x7f98c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.325+0000 7f98daffd640 1 -- 172.21.15.112:0/3658451704 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e00050d0 con 0x7f98e40682a0 2024-04-25T01:29:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.325+0000 7f98daffd640 1 -- 172.21.15.112:0/3658451704 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f98e0018070 con 0x7f98e40682a0 2024-04-25T01:29:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.325+0000 7f98daffd640 1 -- 172.21.15.112:0/3658451704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e40682a0 msgr2=0x7f98e41538d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.325+0000 7f98daffd640 1 --2- 172.21.15.112:0/3658451704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e40682a0 0x7f98e41538d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f98e000f770 tx=0x7f98e000fc30 comp rx=0 tx=0).stop 2024-04-25T01:29:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.325+0000 7f98daffd640 1 -- 172.21.15.112:0/3658451704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e40682a0 msgr2=0x7f98e41538d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:29:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.325+0000 7f98daffd640 1 --2- 172.21.15.112:0/3658451704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e40682a0 0x7f98e41538d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.325+0000 7f98ea9c6640 1 --2- 172.21.15.112:0/3658451704 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f98c806be30 0x7f98c806e2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f98dc034000 tx=0x7f98dc02f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:36.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.331+0000 7f98ec429640 1 --2- 172.21.15.112:0/3658451704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e41ef190 0x7f98e414c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.331+0000 7f98ec429640 1 -- 172.21.15.112:0/3658451704 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41525e0 con 0x7f98e41ef190 2024-04-25T01:29:36.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.331+0000 7f98ea1c5640 1 --2- 172.21.15.112:0/3658451704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e41ef190 0x7f98e414c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.332+0000 7f98ea1c5640 1 -- 172.21.15.112:0/3658451704 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f98e4158780 con 0x7f98e41ef190 2024-04-25T01:29:36.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.332+0000 7f98ea1c5640 1 --2- 172.21.15.112:0/3658451704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e41ef190 0x7f98e414c320 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f98e0017000 tx=0x7f98e0001150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:36.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.332+0000 7f98daffd640 1 -- 172.21.15.112:0/3658451704 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e000d960 con 0x7f98e41ef190 2024-04-25T01:29:36.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.332+0000 7f98daffd640 1 -- 172.21.15.112:0/3658451704 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4559 (secure 0 0 0) 0x7f98e000d960 con 0x7f98e41ef190 2024-04-25T01:29:36.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.332+0000 7f98daffd640 1 -- 172.21.15.112:0/3658451704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e41ef190 msgr2=0x7f98e414c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.332+0000 7f98daffd640 1 --2- 172.21.15.112:0/3658451704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e41ef190 0x7f98e414c320 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f98e0017000 tx=0x7f98e0001150 comp rx=0 tx=0).stop 2024-04-25T01:29:36.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.332+0000 7f98daffd640 1 -- 172.21.15.112:0/3658451704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e41ef190 msgr2=0x7f98e414c320 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:29:36.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.332+0000 7f98daffd640 1 --2- 172.21.15.112:0/3658451704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e41ef190 0x7f98e414c320 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.311 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:29:36.311 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:29:36.311 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:29:36.311 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:29:36.311 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 26, 2024-04-25T01:29:36.312 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 12021, 2024-04-25T01:29:36.312 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:29:36.312 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:29:36.312 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:29:36.312 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:29:36.312 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:36.312 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 11, 2024-04-25T01:29:36.312 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:29:36.312 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:29:36.312 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:29:36.312 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:36.312 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:36.312 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:36.312 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:36.312 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:36.313 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:36.313 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:36.313 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:36.313 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:36.313 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:36.313 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:36.313 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:36.313 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:29:36.313 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:29:36.313 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:36.313 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:36.313 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:36.313 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:36.313 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:29:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:29:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:29:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:29:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:29:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:29:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:29:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:29:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:29:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:29:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:29:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:29:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:29:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:29:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:29:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:29:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:29:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:36.316 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:36.316 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:36.316 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:36.316 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:36.316 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:36.316 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:36.316 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:36.316 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:36.316 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:36.316 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:29:36.316 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.316 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:29:36.316 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.316 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:29:36.317 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:29:36.317 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:36.317 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:36.317 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.317 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:36.317 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:29:36.317 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:36.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.317 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.317 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:36.317 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:29:36.317 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:36.317 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.317 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:36.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.318 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:29:36.318 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:29:36.318 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:36.318 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:36.318 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:36.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.318 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.318 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:29:36.318 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:29:36.318 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:36.318 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:29:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:29:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:29:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:29:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:36.320 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:36.320 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:36.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.320 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.320 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:29:36.320 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:29:36.320 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:36.320 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:36.320 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.320 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:36.320 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:29:36.320 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:36.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.320 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.321 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:36.321 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:29:36.321 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:36.321 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.321 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:36.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.321 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:29:36.321 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:29:36.321 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:36.321 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:36.321 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:36.321 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.321 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:36.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.322 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:29:36.322 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:29:36.322 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.322 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:36.322 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:36.322 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:29:36.322 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.322 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:36.322 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:29:36.322 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.322 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:36.322 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:36.323 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:29:36.323 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.323 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:36.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.323 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:29:36.323 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:29:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.333+0000 7f98ec429640 1 -- 172.21.15.112:0/3658451704 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f98c806be30 msgr2=0x7f98c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.333+0000 7f98ec429640 1 --2- 172.21.15.112:0/3658451704 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f98c806be30 0x7f98c806e2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f98dc034000 tx=0x7f98dc02f000 comp rx=0 tx=0).stop 2024-04-25T01:29:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.333+0000 7f98ec429640 1 -- 172.21.15.112:0/3658451704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e41592c0 msgr2=0x7f98e4158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.333+0000 7f98ec429640 1 --2- 172.21.15.112:0/3658451704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e41592c0 0x7f98e4158240 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f98d400c500 tx=0x7f98d400c9c0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.333+0000 7f98ea9c6640 1 -- 172.21.15.112:0/3658451704 reap_dead start 2024-04-25T01:29:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.333+0000 7f98ea9c6640 1 -- 172.21.15.112:0/3658451704 reap_dead start 2024-04-25T01:29:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.333+0000 7f98ec429640 1 -- 172.21.15.112:0/3658451704 shutdown_connections 2024-04-25T01:29:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.333+0000 7f98ec429640 1 -- 172.21.15.112:0/3658451704 >> 172.21.15.112:0/3658451704 conn(0x7f98e40cb170 msgr2=0x7f98e40ab3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.333+0000 7f98ec429640 1 -- 172.21.15.112:0/3658451704 shutdown_connections 2024-04-25T01:29:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.333+0000 7f98ec429640 1 -- 172.21.15.112:0/3658451704 wait complete. 2024-04-25T01:29:36.325 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:29:36.325 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:29:36.326 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:29:36.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.453+0000 7f133642c640 1 Processor -- start 2024-04-25T01:29:36.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.453+0000 7f133642c640 1 -- start start 2024-04-25T01:29:36.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.454+0000 7f133642c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f13301592c0 0x7f13300c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.454+0000 7f133642c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13300c72b0 0x7f1330160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.454+0000 7f133642c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1330159b90 con 0x7f13301596a0 2024-04-25T01:29:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.454+0000 7f133642c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f13300c6e20 con 0x7f13301592c0 2024-04-25T01:29:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.454+0000 7f133642c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f133006c970 con 0x7f13300c72b0 2024-04-25T01:29:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.454+0000 7f132ffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f13301592c0 0x7f13300c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.454+0000 7f132ffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f13301592c0 0x7f13300c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34724/0 (socket says 172.21.15.112:34724) 2024-04-25T01:29:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.454+0000 7f132ffff640 1 -- 172.21.15.112:0/2054785513 learned_addr learned my addr 172.21.15.112:0/2054785513 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.454+0000 7f13349c9640 1 --2- 172.21.15.112:0/2054785513 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13300c72b0 0x7f1330160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.454+0000 7f132ffff640 1 -- 172.21.15.112:0/2054785513 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13300c72b0 msgr2=0x7f1330160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:29:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.454+0000 7f132ffff640 1 --2- 172.21.15.112:0/2054785513 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13300c72b0 0x7f1330160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.454+0000 7f132ffff640 1 -- 172.21.15.112:0/2054785513 >> v1:172.21.15.187:6789/0 conn(0x7f13301596a0 legacy=0x7f1330152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.454+0000 7f132effd640 1 -- 172.21.15.112:0/2054785513 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1528999364 0 0) 0x7f1330159b90 con 0x7f13301596a0 2024-04-25T01:29:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.454+0000 7f132ffff640 1 -- 172.21.15.112:0/2054785513 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133006cc50 con 0x7f13301592c0 2024-04-25T01:29:36.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.455+0000 7f132ffff640 1 --2- 172.21.15.112:0/2054785513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13301592c0 0x7f13300c9070 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1320009870 tx=0x7f132002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fab7296633e92a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:36.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.455+0000 7f132effd640 1 -- 172.21.15.112:0/2054785513 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320037070 con 0x7f13301592c0 2024-04-25T01:29:36.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.455+0000 7f132effd640 1 -- 172.21.15.112:0/2054785513 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1320032440 con 0x7f13301592c0 2024-04-25T01:29:36.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.455+0000 7f132effd640 1 -- 172.21.15.112:0/2054785513 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320032810 con 0x7f13301592c0 2024-04-25T01:29:36.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.456+0000 7f133642c640 1 -- 172.21.15.112:0/2054785513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13301592c0 msgr2=0x7f13300c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.456+0000 7f133642c640 1 --2- 172.21.15.112:0/2054785513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13301592c0 0x7f13300c9070 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1320009870 tx=0x7f132002a130 comp rx=0 tx=0).stop 2024-04-25T01:29:36.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.456+0000 7f133642c640 1 -- 172.21.15.112:0/2054785513 shutdown_connections 2024-04-25T01:29:36.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.456+0000 7f133642c640 1 --2- 172.21.15.112:0/2054785513 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13300c72b0 0x7f1330160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.456+0000 7f133642c640 1 --2- 172.21.15.112:0/2054785513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13301592c0 0x7f13300c9070 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.456+0000 7f133642c640 1 -- 172.21.15.112:0/2054785513 >> 172.21.15.112:0/2054785513 conn(0x7f13300cb170 msgr2=0x7f13300cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:36.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.456+0000 7f133642c640 1 -- 172.21.15.112:0/2054785513 shutdown_connections 2024-04-25T01:29:36.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.456+0000 7f133642c640 1 -- 172.21.15.112:0/2054785513 wait complete. 2024-04-25T01:29:36.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.457+0000 7f133642c640 1 Processor -- start 2024-04-25T01:29:36.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.457+0000 7f133642c640 1 -- start start 2024-04-25T01:29:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.458+0000 7f133642c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13300c72b0 0x7f133014ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.458+0000 7f133642c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13301592c0 0x7f133014bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.458+0000 7f133642c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13301596a0 0x7f13300bf940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.458+0000 7f133642c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1330153590 con 0x7f13300c72b0 2024-04-25T01:29:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.458+0000 7f133642c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1330153410 con 0x7f13301596a0 2024-04-25T01:29:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.458+0000 7f133642c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f133006db20 con 0x7f13301592c0 2024-04-25T01:29:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.458+0000 7f132ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13300c72b0 0x7f133014ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.458+0000 7f132ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13300c72b0 0x7f133014ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34726/0 (socket says 172.21.15.112:34726) 2024-04-25T01:29:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.458+0000 7f132ffff640 1 -- 172.21.15.112:0/3847439736 learned_addr learned my addr 172.21.15.112:0/3847439736 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.458+0000 7f13349c9640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13301596a0 0x7f13300bf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.458+0000 7f132f7fe640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13301592c0 0x7f133014bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.458+0000 7f132ffff640 1 -- 172.21.15.112:0/3847439736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13301592c0 msgr2=0x7f133014bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.458+0000 7f132ffff640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13301592c0 0x7f133014bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.458+0000 7f132ffff640 1 -- 172.21.15.112:0/3847439736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13301596a0 msgr2=0x7f13300bf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.458+0000 7f132ffff640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13301596a0 0x7f13300bf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.458+0000 7f132ffff640 1 -- 172.21.15.112:0/3847439736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13301ec000 con 0x7f13300c72b0 2024-04-25T01:29:36.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.459+0000 7f132ffff640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13300c72b0 0x7f133014ee70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1320009870 tx=0x7f13200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:36.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.459+0000 7f132cff9640 1 -- 172.21.15.112:0/3847439736 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320041020 con 0x7f13300c72b0 2024-04-25T01:29:36.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.459+0000 7f132cff9640 1 -- 172.21.15.112:0/3847439736 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13200306c0 con 0x7f13300c72b0 2024-04-25T01:29:36.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.459+0000 7f132cff9640 1 -- 172.21.15.112:0/3847439736 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320037040 con 0x7f13300c72b0 2024-04-25T01:29:36.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.459+0000 7f133642c640 1 -- 172.21.15.112:0/3847439736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13301ec280 con 0x7f13300c72b0 2024-04-25T01:29:36.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.459+0000 7f133642c640 1 -- 172.21.15.112:0/3847439736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f133014e670 con 0x7f13300c72b0 2024-04-25T01:29:36.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.460+0000 7f133642c640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13300bfe80 0x7f13301eeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.460+0000 7f133642c640 1 -- 172.21.15.112:0/3847439736 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f13300bf540 con 0x7f13300bfe80 2024-04-25T01:29:36.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.460+0000 7f132f7fe640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13300bfe80 0x7f13301eeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.460+0000 7f132f7fe640 1 -- 172.21.15.112:0/3847439736 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f13301ec560 con 0x7f13300bfe80 2024-04-25T01:29:36.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.461+0000 7f132cff9640 1 -- 172.21.15.112:0/3847439736 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1320044020 con 0x7f13300c72b0 2024-04-25T01:29:36.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.461+0000 7f132f7fe640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13300bfe80 0x7f13301eeb20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f13300abb50 tx=0x7f132400f9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:36.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.461+0000 7f132cff9640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f131c06be30 0x7f131c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.461+0000 7f132cff9640 1 -- 172.21.15.112:0/3847439736 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f13200ab8e0 con 0x7f13300c72b0 2024-04-25T01:29:36.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.461+0000 7f13349c9640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f131c06be30 0x7f131c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.461+0000 7f132cff9640 1 -- 172.21.15.112:0/3847439736 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13240052e0 con 0x7f13300bfe80 2024-04-25T01:29:36.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.461+0000 7f132cff9640 1 -- 172.21.15.112:0/3847439736 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1324018070 con 0x7f13300bfe80 2024-04-25T01:29:36.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.461+0000 7f132cff9640 1 -- 172.21.15.112:0/3847439736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13300bfe80 msgr2=0x7f13301eeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.461+0000 7f132cff9640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13300bfe80 0x7f13301eeb20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f13300abb50 tx=0x7f132400f9d0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.461+0000 7f132cff9640 1 -- 172.21.15.112:0/3847439736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13300bfe80 msgr2=0x7f13301eeb20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:29:36.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.461+0000 7f132cff9640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13300bfe80 0x7f13301eeb20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.461+0000 7f13349c9640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f131c06be30 0x7f131c06e2f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f13280113e0 tx=0x7f132800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:36.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.468+0000 7f133642c640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f133014fd90 0x7f13301ef060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.468+0000 7f133642c640 1 -- 172.21.15.112:0/3847439736 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1330075980 con 0x7f133014fd90 2024-04-25T01:29:36.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.468+0000 7f132f7fe640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f133014fd90 0x7f13301ef060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.468+0000 7f132f7fe640 1 -- 172.21.15.112:0/3847439736 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f13300679a0 con 0x7f133014fd90 2024-04-25T01:29:36.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.469+0000 7f132f7fe640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f133014fd90 0x7f13301ef060 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f13300c5470 tx=0x7f1324001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:36.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.469+0000 7f132cff9640 1 -- 172.21.15.112:0/3847439736 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324003b40 con 0x7f133014fd90 2024-04-25T01:29:36.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.469+0000 7f132cff9640 1 -- 172.21.15.112:0/3847439736 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4729 (secure 0 0 0) 0x7f1324003b40 con 0x7f133014fd90 2024-04-25T01:29:36.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.469+0000 7f132cff9640 1 -- 172.21.15.112:0/3847439736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f133014fd90 msgr2=0x7f13301ef060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.469+0000 7f132cff9640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f133014fd90 0x7f13301ef060 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f13300c5470 tx=0x7f1324001080 comp rx=0 tx=0).stop 2024-04-25T01:29:36.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.469+0000 7f132cff9640 1 -- 172.21.15.112:0/3847439736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f133014fd90 msgr2=0x7f13301ef060 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:29:36.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.469+0000 7f132cff9640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f133014fd90 0x7f13301ef060 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.448 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:29:36.448 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:29:36.448 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:29:36.448 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:29:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 26, 2024-04-25T01:29:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 76412, 2024-04-25T01:29:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:29:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:29:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:29:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:29:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 11, 2024-04-25T01:29:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:29:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:29:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:29:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:36.450 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:36.450 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:36.450 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:36.450 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:36.450 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:36.450 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:36.450 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:36.450 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:29:36.450 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:29:36.450 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:36.450 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:36.450 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:36.450 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:36.451 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:36.451 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:36.451 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:36.451 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:36.451 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:36.451 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:36.451 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:36.451 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:36.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.451 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:29:36.451 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:29:36.452 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:29:36.452 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:29:36.452 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:29:36.452 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:29:36.452 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:29:36.452 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:29:36.452 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:29:36.452 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:29:36.452 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:29:36.452 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:29:36.452 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:29:36.452 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:29:36.452 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:29:36.452 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:29:36.452 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:29:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:29:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:29:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.454 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:29:36.454 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:29:36.454 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:36.454 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:36.454 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.454 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:36.454 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:29:36.454 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:36.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.454 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.454 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:36.454 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:29:36.454 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:36.454 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.454 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:36.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.455 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:29:36.455 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:29:36.455 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:36.455 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:36.455 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:36.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.455 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.455 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:29:36.455 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:29:36.455 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:36.455 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:36.455 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.455 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:29:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:29:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:29:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:29:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.457 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:29:36.457 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:29:36.457 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:36.457 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:36.457 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.457 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:36.457 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:29:36.457 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:36.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.457 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.457 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:36.457 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:29:36.457 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:36.457 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.457 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:36.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.458 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:29:36.458 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:29:36.458 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:36.458 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:36.458 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:36.458 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.458 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:36.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.458 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:29:36.458 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:29:36.458 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.458 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:36.458 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:36.458 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:29:36.459 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.459 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:36.459 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:29:36.459 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.459 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:36.459 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:36.459 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:29:36.459 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.459 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:36.459 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:29:36.459 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:36.459 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:36.459 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:36.459 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:29:36.459 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:36.460 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:36.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:36.460 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:29:36.460 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:29:36.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.469+0000 7f133642c640 1 -- 172.21.15.112:0/3847439736 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f131c06be30 msgr2=0x7f131c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.469+0000 7f133642c640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f131c06be30 0x7f131c06e2f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f13280113e0 tx=0x7f132800b040 comp rx=0 tx=0).stop 2024-04-25T01:29:36.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.469+0000 7f133642c640 1 -- 172.21.15.112:0/3847439736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13300c72b0 msgr2=0x7f133014ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.469+0000 7f133642c640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13300c72b0 0x7f133014ee70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1320009870 tx=0x7f13200046c0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.469+0000 7f13349c9640 1 -- 172.21.15.112:0/3847439736 reap_dead start 2024-04-25T01:29:36.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.470+0000 7f133642c640 1 -- 172.21.15.112:0/3847439736 shutdown_connections 2024-04-25T01:29:36.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.470+0000 7f133642c640 1 --2- 172.21.15.112:0/3847439736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13300c72b0 0x7f133014ee70 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.470+0000 7f133642c640 1 -- 172.21.15.112:0/3847439736 >> 172.21.15.112:0/3847439736 conn(0x7f13300cb170 msgr2=0x7f13300cb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:36.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.470+0000 7f133642c640 1 -- 172.21.15.112:0/3847439736 shutdown_connections 2024-04-25T01:29:36.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.470+0000 7f133642c640 1 -- 172.21.15.112:0/3847439736 wait complete. 2024-04-25T01:29:36.462 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:29:36.462 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-04-25T01:29:36.462 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-25T01:29:36.462 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-04-25T01:29:36.462 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-25T01:29:36.463 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T01:29:36.463 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:29:36.474 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T01:29:36.474 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T01:29:36.474 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:29:36.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.602+0000 7ff132b3b640 1 Processor -- start 2024-04-25T01:29:36.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.602+0000 7ff132b3b640 1 -- start start 2024-04-25T01:29:36.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.602+0000 7ff132b3b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c5280 0x7ff12c0c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.602+0000 7ff132b3b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff12c0c8b30 0x7ff12c14d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.602+0000 7ff132b3b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff12c0c35d0 con 0x7ff12c070d20 2024-04-25T01:29:36.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.602+0000 7ff132b3b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff12c0705b0 con 0x7ff12c0c8b30 2024-04-25T01:29:36.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.602+0000 7ff132b3b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff12c070890 con 0x7ff12c0c5280 2024-04-25T01:29:36.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.603+0000 7ff1308d7640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c5280 msgr2=0x7ff12c0c85f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:29:36.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.603+0000 7ff1308d7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c5280 0x7ff12c0c85f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:36.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.603+0000 7ff1310d8640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff12c070d20 0x7ff12c153640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53258/0 (socket says 172.21.15.112:53258) 2024-04-25T01:29:36.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.603+0000 7ff1310d8640 1 -- 172.21.15.112:0/1536282147 learned_addr learned my addr 172.21.15.112:0/1536282147 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:29:36.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.603+0000 7ff12bfff640 1 --2- 172.21.15.112:0/1536282147 >> v2:172.21.15.187:3300/0 conn(0x7ff12c0c8b30 0x7ff12c14d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.603+0000 7ff12b7fe640 1 -- 172.21.15.112:0/1536282147 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3804599871 0 0) 0x7ff12c0c35d0 con 0x7ff12c070d20 2024-04-25T01:29:36.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.603+0000 7ff12b7fe640 1 -- 172.21.15.112:0/1536282147 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff11c003520 con 0x7ff12c070d20 2024-04-25T01:29:36.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.603+0000 7ff12bfff640 1 -- 172.21.15.112:0/1536282147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c5280 msgr2=0x7ff12c0c85f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:29:36.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.603+0000 7ff12bfff640 1 --2- 172.21.15.112:0/1536282147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c5280 0x7ff12c0c85f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.603+0000 7ff12bfff640 1 -- 172.21.15.112:0/1536282147 >> v1:172.21.15.112:6789/0 conn(0x7ff12c070d20 legacy=0x7ff12c153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.603+0000 7ff12bfff640 1 -- 172.21.15.112:0/1536282147 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c0c96c0 con 0x7ff12c0c8b30 2024-04-25T01:29:36.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.604+0000 7ff12bfff640 1 --2- 172.21.15.112:0/1536282147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c0c8b30 0x7ff12c14d230 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff1180097d0 tx=0x7ff11802a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d85c335db8f36198 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:36.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.604+0000 7ff12b7fe640 1 -- 172.21.15.112:0/1536282147 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118037070 con 0x7ff12c0c8b30 2024-04-25T01:29:36.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.604+0000 7ff12b7fe640 1 -- 172.21.15.112:0/1536282147 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff118032440 con 0x7ff12c0c8b30 2024-04-25T01:29:36.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.604+0000 7ff12b7fe640 1 -- 172.21.15.112:0/1536282147 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118032810 con 0x7ff12c0c8b30 2024-04-25T01:29:36.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.605+0000 7ff132b3b640 1 -- 172.21.15.112:0/1536282147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c0c8b30 msgr2=0x7ff12c14d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.605+0000 7ff132b3b640 1 --2- 172.21.15.112:0/1536282147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c0c8b30 0x7ff12c14d230 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff1180097d0 tx=0x7ff11802a1e0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.605+0000 7ff132b3b640 1 -- 172.21.15.112:0/1536282147 shutdown_connections 2024-04-25T01:29:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.605+0000 7ff132b3b640 1 --2- 172.21.15.112:0/1536282147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c0c8b30 0x7ff12c14d230 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.605+0000 7ff132b3b640 1 --2- 172.21.15.112:0/1536282147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c5280 0x7ff12c0c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.605+0000 7ff132b3b640 1 -- 172.21.15.112:0/1536282147 >> 172.21.15.112:0/1536282147 conn(0x7ff12c0747f0 msgr2=0x7ff12c074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.605+0000 7ff132b3b640 1 -- 172.21.15.112:0/1536282147 shutdown_connections 2024-04-25T01:29:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.605+0000 7ff132b3b640 1 -- 172.21.15.112:0/1536282147 wait complete. 2024-04-25T01:29:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.606+0000 7ff132b3b640 1 Processor -- start 2024-04-25T01:29:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.606+0000 7ff132b3b640 1 -- start start 2024-04-25T01:29:36.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.607+0000 7ff132b3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c070d20 0x7ff12c0aaa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.607+0000 7ff132b3b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c5280 0x7ff12c1e5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.607+0000 7ff132b3b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c0c8b30 0x7ff12c1e6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.607+0000 7ff132b3b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c0c5660 con 0x7ff12c070d20 2024-04-25T01:29:36.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.607+0000 7ff132b3b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c0c9c80 con 0x7ff12c0c8b30 2024-04-25T01:29:36.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.607+0000 7ff132b3b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff12c0c57e0 con 0x7ff12c0c5280 2024-04-25T01:29:36.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.607+0000 7ff12bfff640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c5280 msgr2=0x7ff12c1e5d30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:29:36.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.607+0000 7ff12bfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c5280 0x7ff12c1e5d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:36.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.607+0000 7ff1308d7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c070d20 0x7ff12c0aaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.607+0000 7ff1308d7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c070d20 0x7ff12c0aaa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34734/0 (socket says 172.21.15.112:34734) 2024-04-25T01:29:36.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.607+0000 7ff1308d7640 1 -- 172.21.15.112:0/2336328835 learned_addr learned my addr 172.21.15.112:0/2336328835 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:36.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.607+0000 7ff1310d8640 1 --2- 172.21.15.112:0/2336328835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c0c8b30 0x7ff12c1e6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.607+0000 7ff1308d7640 1 -- 172.21.15.112:0/2336328835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c5280 msgr2=0x7ff12c1e5d30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:29:36.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.607+0000 7ff1308d7640 1 --2- 172.21.15.112:0/2336328835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c5280 0x7ff12c1e5d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.607+0000 7ff1308d7640 1 -- 172.21.15.112:0/2336328835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c0c8b30 msgr2=0x7ff12c1e6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.607+0000 7ff1308d7640 1 --2- 172.21.15.112:0/2336328835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c0c8b30 0x7ff12c1e6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.607+0000 7ff1308d7640 1 -- 172.21.15.112:0/2336328835 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c1e9890 con 0x7ff12c070d20 2024-04-25T01:29:36.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.607+0000 7ff1308d7640 1 --2- 172.21.15.112:0/2336328835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c070d20 0x7ff12c0aaa40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff12000b580 tx=0x7ff12000ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:36.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.607+0000 7ff1297fa640 1 -- 172.21.15.112:0/2336328835 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120013020 con 0x7ff12c070d20 2024-04-25T01:29:36.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.608+0000 7ff1297fa640 1 -- 172.21.15.112:0/2336328835 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff120003de0 con 0x7ff12c070d20 2024-04-25T01:29:36.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.608+0000 7ff1297fa640 1 -- 172.21.15.112:0/2336328835 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1200052b0 con 0x7ff12c070d20 2024-04-25T01:29:36.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.608+0000 7ff132b3b640 1 -- 172.21.15.112:0/2336328835 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12c153f10 con 0x7ff12c070d20 2024-04-25T01:29:36.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.608+0000 7ff132b3b640 1 -- 172.21.15.112:0/2336328835 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12c0aaf80 con 0x7ff12c070d20 2024-04-25T01:29:36.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.609+0000 7ff132b3b640 1 -- 172.21.15.112:0/2336328835 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff12c0689b0 con 0x7ff12c070d20 2024-04-25T01:29:36.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.609+0000 7ff1297fa640 1 -- 172.21.15.112:0/2336328835 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff120016020 con 0x7ff12c070d20 2024-04-25T01:29:36.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.609+0000 7ff1297fa640 1 --2- 172.21.15.112:0/2336328835 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff11006be10 0x7ff11006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:36.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.610+0000 7ff1297fa640 1 -- 172.21.15.112:0/2336328835 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7ff12008ae80 con 0x7ff12c070d20 2024-04-25T01:29:36.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.610+0000 7ff12bfff640 1 --2- 172.21.15.112:0/2336328835 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff11006be10 0x7ff11006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:36.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.610+0000 7ff12bfff640 1 --2- 172.21.15.112:0/2336328835 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff11006be10 0x7ff11006e2d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff118034000 tx=0x7ff11802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:36.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.616+0000 7ff1297fa640 1 -- 172.21.15.112:0/2336328835 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff12001d020 con 0x7ff12c070d20 2024-04-25T01:29:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.907+0000 7ff132b3b640 1 -- 172.21.15.112:0/2336328835 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff12c070390 con 0x7ff12c070d20 2024-04-25T01:29:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.908+0000 7ff1297fa640 1 -- 172.21.15.112:0/2336328835 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7ff120059520 con 0x7ff12c070d20 2024-04-25T01:29:36.887 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":26,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:29:36.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.910+0000 7ff132b3b640 1 -- 172.21.15.112:0/2336328835 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff11006be10 msgr2=0x7ff11006e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.910+0000 7ff132b3b640 1 --2- 172.21.15.112:0/2336328835 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff11006be10 0x7ff11006e2d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff118034000 tx=0x7ff11802e000 comp rx=0 tx=0).stop 2024-04-25T01:29:36.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.910+0000 7ff132b3b640 1 -- 172.21.15.112:0/2336328835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c070d20 msgr2=0x7ff12c0aaa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:36.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.910+0000 7ff132b3b640 1 --2- 172.21.15.112:0/2336328835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c070d20 0x7ff12c0aaa40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff12000b580 tx=0x7ff12000ba40 comp rx=0 tx=0).stop 2024-04-25T01:29:36.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.910+0000 7ff132b3b640 1 -- 172.21.15.112:0/2336328835 shutdown_connections 2024-04-25T01:29:36.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.910+0000 7ff132b3b640 1 --2- 172.21.15.112:0/2336328835 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff11006be10 0x7ff11006e2d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.910+0000 7ff132b3b640 1 --2- 172.21.15.112:0/2336328835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c0c8b30 0x7ff12c1e6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.910+0000 7ff132b3b640 1 --2- 172.21.15.112:0/2336328835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c5280 0x7ff12c1e5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.910+0000 7ff132b3b640 1 --2- 172.21.15.112:0/2336328835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c070d20 0x7ff12c0aaa40 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:36.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.910+0000 7ff132b3b640 1 -- 172.21.15.112:0/2336328835 >> 172.21.15.112:0/2336328835 conn(0x7ff12c0747f0 msgr2=0x7ff12c0ae460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:36.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.910+0000 7ff132b3b640 1 -- 172.21.15.112:0/2336328835 shutdown_connections 2024-04-25T01:29:36.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:36.910+0000 7ff132b3b640 1 -- 172.21.15.112:0/2336328835 wait complete. 2024-04-25T01:29:39.901 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:29:40.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.031+0000 7fc855da9640 1 Processor -- start 2024-04-25T01:29:40.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.031+0000 7fc855da9640 1 -- start start 2024-04-25T01:29:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.032+0000 7fc855da9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc85006c3e0 0x7fc850153f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.032+0000 7fc855da9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc8500c21d0 0x7fc8501579a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.032+0000 7fc855da9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8500c28e0 con 0x7fc8500c2bc0 2024-04-25T01:29:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.032+0000 7fc855da9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fc8500c3cd0 con 0x7fc8500c21d0 2024-04-25T01:29:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.032+0000 7fc855da9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc8500c2690 con 0x7fc85006c3e0 2024-04-25T01:29:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.032+0000 7fc84f7fe640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc85006c3e0 msgr2=0x7fc850153f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:29:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.032+0000 7fc84f7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc85006c3e0 0x7fc850153f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.032+0000 7fc84ffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc8500c21d0 0x7fc8501579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.032+0000 7fc84effd640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fc8500c2bc0 0x7fc850157290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:48140/0 (socket says 172.21.15.112:48140) 2024-04-25T01:29:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.032+0000 7fc84ffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc8500c21d0 0x7fc8501579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54286/0 (socket says 172.21.15.112:54286) 2024-04-25T01:29:40.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.032+0000 7fc84effd640 1 -- 172.21.15.112:0/4291008389 learned_addr learned my addr 172.21.15.112:0/4291008389 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:29:40.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.033+0000 7fc84ffff640 1 -- 172.21.15.112:0/4291008389 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc85006c3e0 msgr2=0x7fc850153f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:29:40.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.033+0000 7fc84ffff640 1 --2- 172.21.15.112:0/4291008389 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc85006c3e0 0x7fc850153f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:40.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.033+0000 7fc84ffff640 1 -- 172.21.15.112:0/4291008389 >> v1:172.21.15.187:6789/0 conn(0x7fc8500c2bc0 legacy=0x7fc850157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:40.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.033+0000 7fc84ffff640 1 -- 172.21.15.112:0/4291008389 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc850074560 con 0x7fc8500c21d0 2024-04-25T01:29:40.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.033+0000 7fc84ffff640 1 --2- 172.21.15.112:0/4291008389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8500c21d0 0x7fc8501579a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc848009d90 tx=0x7fc84802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c005d11e7bbcda3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:40.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.034+0000 7fc84e7fc640 1 -- 172.21.15.112:0/4291008389 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848007740 con 0x7fc8500c21d0 2024-04-25T01:29:40.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.034+0000 7fc84e7fc640 1 -- 172.21.15.112:0/4291008389 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8480078e0 con 0x7fc8500c21d0 2024-04-25T01:29:40.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.034+0000 7fc84e7fc640 1 -- 172.21.15.112:0/4291008389 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848007cb0 con 0x7fc8500c21d0 2024-04-25T01:29:40.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.034+0000 7fc855da9640 1 -- 172.21.15.112:0/4291008389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8500c21d0 msgr2=0x7fc8501579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:40.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.034+0000 7fc855da9640 1 --2- 172.21.15.112:0/4291008389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8500c21d0 0x7fc8501579a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc848009d90 tx=0x7fc84802a5f0 comp rx=0 tx=0).stop 2024-04-25T01:29:40.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.034+0000 7fc855da9640 1 -- 172.21.15.112:0/4291008389 shutdown_connections 2024-04-25T01:29:40.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.034+0000 7fc855da9640 1 --2- 172.21.15.112:0/4291008389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8500c21d0 0x7fc8501579a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:40.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.034+0000 7fc855da9640 1 --2- 172.21.15.112:0/4291008389 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc85006c3e0 0x7fc850153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:40.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.034+0000 7fc855da9640 1 -- 172.21.15.112:0/4291008389 >> 172.21.15.112:0/4291008389 conn(0x7fc8500c42d0 msgr2=0x7fc8500c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:40.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.034+0000 7fc855da9640 1 -- 172.21.15.112:0/4291008389 shutdown_connections 2024-04-25T01:29:40.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.034+0000 7fc855da9640 1 -- 172.21.15.112:0/4291008389 wait complete. 2024-04-25T01:29:40.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.036+0000 7fc855da9640 1 Processor -- start 2024-04-25T01:29:40.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.036+0000 7fc855da9640 1 -- start start 2024-04-25T01:29:40.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.037+0000 7fc855da9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc85006c3e0 0x7fc85014f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:40.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.037+0000 7fc855da9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8500c21d0 0x7fc8500ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:40.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.037+0000 7fc855da9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8500c2bc0 0x7fc8500bcf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:40.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.037+0000 7fc855da9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8500c7350 con 0x7fc8500c2bc0 2024-04-25T01:29:40.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.037+0000 7fc855da9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc850153900 con 0x7fc85006c3e0 2024-04-25T01:29:40.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.037+0000 7fc855da9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc8500ac920 con 0x7fc8500c21d0 2024-04-25T01:29:40.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.037+0000 7fc84effd640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8500c21d0 msgr2=0x7fc8500ab2e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:29:40.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.037+0000 7fc84effd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8500c21d0 0x7fc8500ab2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:40.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.037+0000 7fc84ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8500c2bc0 0x7fc8500bcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:40.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.037+0000 7fc84ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8500c2bc0 0x7fc8500bcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54296/0 (socket says 172.21.15.112:54296) 2024-04-25T01:29:40.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.037+0000 7fc84ffff640 1 -- 172.21.15.112:0/96893409 learned_addr learned my addr 172.21.15.112:0/96893409 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:40.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.037+0000 7fc84f7fe640 1 --2- 172.21.15.112:0/96893409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc85006c3e0 0x7fc85014f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:40.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.037+0000 7fc84ffff640 1 -- 172.21.15.112:0/96893409 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8500c21d0 msgr2=0x7fc8500ab2e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:29:40.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.037+0000 7fc84ffff640 1 --2- 172.21.15.112:0/96893409 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8500c21d0 0x7fc8500ab2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:40.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.037+0000 7fc84ffff640 1 -- 172.21.15.112:0/96893409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc85006c3e0 msgr2=0x7fc85014f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:40.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.037+0000 7fc84ffff640 1 --2- 172.21.15.112:0/96893409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc85006c3e0 0x7fc85014f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:40.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.037+0000 7fc84ffff640 1 -- 172.21.15.112:0/96893409 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8501ed3e0 con 0x7fc8500c2bc0 2024-04-25T01:29:40.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.037+0000 7fc84ffff640 1 --2- 172.21.15.112:0/96893409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8500c2bc0 0x7fc8500bcf20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc848009eb0 tx=0x7fc848004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:40.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.037+0000 7fc837fff640 1 -- 172.21.15.112:0/96893409 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848040070 con 0x7fc8500c2bc0 2024-04-25T01:29:40.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.038+0000 7fc837fff640 1 -- 172.21.15.112:0/96893409 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc848031c40 con 0x7fc8500c2bc0 2024-04-25T01:29:40.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.038+0000 7fc855da9640 1 -- 172.21.15.112:0/96893409 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8501ed6c0 con 0x7fc8500c2bc0 2024-04-25T01:29:40.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.038+0000 7fc837fff640 1 -- 172.21.15.112:0/96893409 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84803c420 con 0x7fc8500c2bc0 2024-04-25T01:29:40.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.038+0000 7fc855da9640 1 -- 172.21.15.112:0/96893409 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8500c16c0 con 0x7fc8500c2bc0 2024-04-25T01:29:40.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.039+0000 7fc855da9640 1 -- 172.21.15.112:0/96893409 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8500c1aa0 con 0x7fc8500c2bc0 2024-04-25T01:29:40.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.039+0000 7fc837fff640 1 -- 172.21.15.112:0/96893409 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc848039040 con 0x7fc8500c2bc0 2024-04-25T01:29:40.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.040+0000 7fc837fff640 1 --2- 172.21.15.112:0/96893409 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc83c06bde0 0x7fc83c06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:40.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.040+0000 7fc837fff640 1 -- 172.21.15.112:0/96893409 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fc8480ac730 con 0x7fc8500c2bc0 2024-04-25T01:29:40.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.040+0000 7fc84f7fe640 1 --2- 172.21.15.112:0/96893409 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc83c06bde0 0x7fc83c06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:40.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.040+0000 7fc84f7fe640 1 --2- 172.21.15.112:0/96893409 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc83c06bde0 0x7fc83c06e2a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc8400093d0 tx=0x7fc840009360 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:40.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.045+0000 7fc837fff640 1 -- 172.21.15.112:0/96893409 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc84807add0 con 0x7fc8500c2bc0 2024-04-25T01:29:40.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.352+0000 7fc855da9640 1 -- 172.21.15.112:0/96893409 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc8500689b0 con 0x7fc8500c2bc0 2024-04-25T01:29:40.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.353+0000 7fc837fff640 1 -- 172.21.15.112:0/96893409 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7fc84807a6d0 con 0x7fc8500c2bc0 2024-04-25T01:29:40.331 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":26,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:29:40.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.355+0000 7fc855da9640 1 -- 172.21.15.112:0/96893409 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc83c06bde0 msgr2=0x7fc83c06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:40.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.355+0000 7fc855da9640 1 --2- 172.21.15.112:0/96893409 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc83c06bde0 0x7fc83c06e2a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc8400093d0 tx=0x7fc840009360 comp rx=0 tx=0).stop 2024-04-25T01:29:40.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.355+0000 7fc855da9640 1 -- 172.21.15.112:0/96893409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8500c2bc0 msgr2=0x7fc8500bcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:40.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.355+0000 7fc855da9640 1 --2- 172.21.15.112:0/96893409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8500c2bc0 0x7fc8500bcf20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc848009eb0 tx=0x7fc848004a40 comp rx=0 tx=0).stop 2024-04-25T01:29:40.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.355+0000 7fc855da9640 1 -- 172.21.15.112:0/96893409 shutdown_connections 2024-04-25T01:29:40.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.355+0000 7fc855da9640 1 --2- 172.21.15.112:0/96893409 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc83c06bde0 0x7fc83c06e2a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:40.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.355+0000 7fc855da9640 1 --2- 172.21.15.112:0/96893409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8500c2bc0 0x7fc8500bcf20 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:40.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.355+0000 7fc855da9640 1 --2- 172.21.15.112:0/96893409 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8500c21d0 0x7fc8500ab2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:40.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.355+0000 7fc855da9640 1 --2- 172.21.15.112:0/96893409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc85006c3e0 0x7fc85014f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:40.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.355+0000 7fc855da9640 1 -- 172.21.15.112:0/96893409 >> 172.21.15.112:0/96893409 conn(0x7fc8500c42d0 msgr2=0x7fc8500c55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:40.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.355+0000 7fc855da9640 1 -- 172.21.15.112:0/96893409 shutdown_connections 2024-04-25T01:29:40.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:40.355+0000 7fc855da9640 1 -- 172.21.15.112:0/96893409 wait complete. 2024-04-25T01:29:43.345 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:29:43.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.467+0000 7ff0f3f61640 1 Processor -- start 2024-04-25T01:29:43.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.467+0000 7ff0f3f61640 1 -- start start 2024-04-25T01:29:43.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.468+0000 7ff0f3f61640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff0ec0c5d70 0x7ff0ec0c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:43.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.468+0000 7ff0f3f61640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff0ec156cf0 0x7ff0ec06bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:43.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.468+0000 7ff0f3f61640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0ec0c90e0 con 0x7ff0ec152fd0 2024-04-25T01:29:43.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.468+0000 7ff0f3f61640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7ff0ec0ca9b0 con 0x7ff0ec156cf0 2024-04-25T01:29:43.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.468+0000 7ff0f3f61640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff0ec0cac00 con 0x7ff0ec0c5d70 2024-04-25T01:29:43.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.468+0000 7ff0f24fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff0ec156cf0 0x7ff0ec06bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:43.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.468+0000 7ff0f24fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff0ec156cf0 0x7ff0ec06bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54304/0 (socket says 172.21.15.112:54304) 2024-04-25T01:29:43.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.468+0000 7ff0f24fe640 1 -- 172.21.15.112:0/3537019187 learned_addr learned my addr 172.21.15.112:0/3537019187 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:43.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.468+0000 7ff0f14fc640 1 --2- 172.21.15.112:0/3537019187 >> v2:172.21.15.187:3300/0 conn(0x7ff0ec0c5d70 0x7ff0ec0c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:43.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.469+0000 7ff0f24fe640 1 -- 172.21.15.112:0/3537019187 >> v2:172.21.15.187:3300/0 conn(0x7ff0ec0c5d70 msgr2=0x7ff0ec0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:29:43.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.469+0000 7ff0f24fe640 1 --2- 172.21.15.112:0/3537019187 >> v2:172.21.15.187:3300/0 conn(0x7ff0ec0c5d70 0x7ff0ec0c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:43.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.469+0000 7ff0f24fe640 1 -- 172.21.15.112:0/3537019187 >> v1:172.21.15.187:6789/0 conn(0x7ff0ec152fd0 legacy=0x7ff0ec156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:43.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.469+0000 7ff0f24fe640 1 -- 172.21.15.112:0/3537019187 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ec0c9380 con 0x7ff0ec156cf0 2024-04-25T01:29:43.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.469+0000 7ff0f24fe640 1 --2- 172.21.15.112:0/3537019187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0ec156cf0 0x7ff0ec06bda0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec0bec40 tx=0x7ff0e402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ae0f0d440549b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:43.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.469+0000 7ff0f0cfb640 1 -- 172.21.15.112:0/3537019187 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4036070 con 0x7ff0ec156cf0 2024-04-25T01:29:43.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.469+0000 7ff0f0cfb640 1 -- 172.21.15.112:0/3537019187 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0e4031600 con 0x7ff0ec156cf0 2024-04-25T01:29:43.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.470+0000 7ff0f0cfb640 1 -- 172.21.15.112:0/3537019187 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e40319d0 con 0x7ff0ec156cf0 2024-04-25T01:29:43.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.470+0000 7ff0f3f61640 1 -- 172.21.15.112:0/3537019187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0ec156cf0 msgr2=0x7ff0ec06bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:43.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.470+0000 7ff0f3f61640 1 --2- 172.21.15.112:0/3537019187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0ec156cf0 0x7ff0ec06bda0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec0bec40 tx=0x7ff0e402a6a0 comp rx=0 tx=0).stop 2024-04-25T01:29:43.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.470+0000 7ff0f3f61640 1 -- 172.21.15.112:0/3537019187 shutdown_connections 2024-04-25T01:29:43.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.470+0000 7ff0f3f61640 1 --2- 172.21.15.112:0/3537019187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0ec156cf0 0x7ff0ec06bda0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:43.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.470+0000 7ff0f3f61640 1 --2- 172.21.15.112:0/3537019187 >> v2:172.21.15.187:3300/0 conn(0x7ff0ec0c5d70 0x7ff0ec0c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:43.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.470+0000 7ff0f3f61640 1 -- 172.21.15.112:0/3537019187 >> 172.21.15.112:0/3537019187 conn(0x7ff0ec0acc70 msgr2=0x7ff0ec0ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:43.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.470+0000 7ff0f3f61640 1 -- 172.21.15.112:0/3537019187 shutdown_connections 2024-04-25T01:29:43.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.470+0000 7ff0f3f61640 1 -- 172.21.15.112:0/3537019187 wait complete. 2024-04-25T01:29:43.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.471+0000 7ff0f3f61640 1 Processor -- start 2024-04-25T01:29:43.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.471+0000 7ff0f3f61640 1 -- start start 2024-04-25T01:29:43.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.472+0000 7ff0f3f61640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0ec0c5d70 0x7ff0ec154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:43.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.472+0000 7ff0f3f61640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0ec152fd0 0x7ff0ec14ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:43.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.472+0000 7ff0f3f61640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0ec156cf0 0x7ff0ec14e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:43.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.472+0000 7ff0f3f61640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec157100 con 0x7ff0ec0c5d70 2024-04-25T01:29:43.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.472+0000 7ff0f3f61640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec0caee0 con 0x7ff0ec156cf0 2024-04-25T01:29:43.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.472+0000 7ff0f3f61640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff0ec157280 con 0x7ff0ec152fd0 2024-04-25T01:29:43.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.472+0000 7ff0f14fc640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0ec152fd0 msgr2=0x7ff0ec14ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:29:43.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.472+0000 7ff0f1cfd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0ec0c5d70 0x7ff0ec154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:43.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.472+0000 7ff0f14fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0ec152fd0 0x7ff0ec14ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:43.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.472+0000 7ff0f1cfd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0ec0c5d70 0x7ff0ec154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54308/0 (socket says 172.21.15.112:54308) 2024-04-25T01:29:43.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.472+0000 7ff0f1cfd640 1 -- 172.21.15.112:0/2965091220 learned_addr learned my addr 172.21.15.112:0/2965091220 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:43.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.472+0000 7ff0f24fe640 1 --2- 172.21.15.112:0/2965091220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0ec156cf0 0x7ff0ec14e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:43.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.472+0000 7ff0f1cfd640 1 -- 172.21.15.112:0/2965091220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0ec152fd0 msgr2=0x7ff0ec14ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:29:43.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.472+0000 7ff0f1cfd640 1 --2- 172.21.15.112:0/2965091220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0ec152fd0 0x7ff0ec14ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:43.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.472+0000 7ff0f1cfd640 1 -- 172.21.15.112:0/2965091220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0ec156cf0 msgr2=0x7ff0ec14e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:43.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.472+0000 7ff0f1cfd640 1 --2- 172.21.15.112:0/2965091220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0ec156cf0 0x7ff0ec14e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:43.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.472+0000 7ff0f1cfd640 1 -- 172.21.15.112:0/2965091220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ec1545c0 con 0x7ff0ec0c5d70 2024-04-25T01:29:43.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.472+0000 7ff0f1cfd640 1 --2- 172.21.15.112:0/2965091220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0ec0c5d70 0x7ff0ec154080 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff0e800c430 tx=0x7ff0e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:43.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.473+0000 7ff0e27fc640 1 -- 172.21.15.112:0/2965091220 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8017070 con 0x7ff0ec0c5d70 2024-04-25T01:29:43.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.473+0000 7ff0e27fc640 1 -- 172.21.15.112:0/2965091220 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0e80052d0 con 0x7ff0ec0c5d70 2024-04-25T01:29:43.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.473+0000 7ff0f3f61640 1 -- 172.21.15.112:0/2965091220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ec1556c0 con 0x7ff0ec0c5d70 2024-04-25T01:29:43.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.473+0000 7ff0e27fc640 1 -- 172.21.15.112:0/2965091220 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8003850 con 0x7ff0ec0c5d70 2024-04-25T01:29:43.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.473+0000 7ff0f3f61640 1 -- 172.21.15.112:0/2965091220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ec14d940 con 0x7ff0ec0c5d70 2024-04-25T01:29:43.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.474+0000 7ff0f3f61640 1 -- 172.21.15.112:0/2965091220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0ec0682a0 con 0x7ff0ec0c5d70 2024-04-25T01:29:43.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.474+0000 7ff0e27fc640 1 -- 172.21.15.112:0/2965091220 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff0e801a070 con 0x7ff0ec0c5d70 2024-04-25T01:29:43.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.475+0000 7ff0e27fc640 1 --2- 172.21.15.112:0/2965091220 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff0d006be10 0x7ff0d006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:43.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.475+0000 7ff0e27fc640 1 -- 172.21.15.112:0/2965091220 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7ff0e808a4c0 con 0x7ff0ec0c5d70 2024-04-25T01:29:43.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.475+0000 7ff0f14fc640 1 --2- 172.21.15.112:0/2965091220 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff0d006be10 0x7ff0d006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:43.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.475+0000 7ff0f14fc640 1 --2- 172.21.15.112:0/2965091220 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff0d006be10 0x7ff0d006e2d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec0c6190 tx=0x7ff0dc009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:43.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.481+0000 7ff0e27fc640 1 -- 172.21.15.112:0/2965091220 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff0e8058480 con 0x7ff0ec0c5d70 2024-04-25T01:29:43.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.768+0000 7ff0f3f61640 1 -- 172.21.15.112:0/2965091220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff0ec0c5ac0 con 0x7ff0ec0c5d70 2024-04-25T01:29:43.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.768+0000 7ff0e27fc640 1 -- 172.21.15.112:0/2965091220 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7ff0e805c780 con 0x7ff0ec0c5d70 2024-04-25T01:29:43.747 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":26,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:29:43.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.770+0000 7ff0f3f61640 1 -- 172.21.15.112:0/2965091220 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff0d006be10 msgr2=0x7ff0d006e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:43.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.770+0000 7ff0f3f61640 1 --2- 172.21.15.112:0/2965091220 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff0d006be10 0x7ff0d006e2d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec0c6190 tx=0x7ff0dc009190 comp rx=0 tx=0).stop 2024-04-25T01:29:43.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.770+0000 7ff0f3f61640 1 -- 172.21.15.112:0/2965091220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0ec0c5d70 msgr2=0x7ff0ec154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:43.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.770+0000 7ff0f3f61640 1 --2- 172.21.15.112:0/2965091220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0ec0c5d70 0x7ff0ec154080 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff0e800c430 tx=0x7ff0e800c8f0 comp rx=0 tx=0).stop 2024-04-25T01:29:43.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.770+0000 7ff0f3f61640 1 -- 172.21.15.112:0/2965091220 shutdown_connections 2024-04-25T01:29:43.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.770+0000 7ff0f3f61640 1 --2- 172.21.15.112:0/2965091220 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff0d006be10 0x7ff0d006e2d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:43.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.770+0000 7ff0f3f61640 1 --2- 172.21.15.112:0/2965091220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0ec156cf0 0x7ff0ec14e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:43.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.770+0000 7ff0f3f61640 1 --2- 172.21.15.112:0/2965091220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0ec152fd0 0x7ff0ec14ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:43.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.770+0000 7ff0f3f61640 1 --2- 172.21.15.112:0/2965091220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0ec0c5d70 0x7ff0ec154080 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:43.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.770+0000 7ff0f3f61640 1 -- 172.21.15.112:0/2965091220 >> 172.21.15.112:0/2965091220 conn(0x7ff0ec0acc70 msgr2=0x7ff0ec155e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:43.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.770+0000 7ff0f3f61640 1 -- 172.21.15.112:0/2965091220 shutdown_connections 2024-04-25T01:29:43.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:43.770+0000 7ff0f3f61640 1 -- 172.21.15.112:0/2965091220 wait complete. 2024-04-25T01:29:46.760 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:29:46.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.886+0000 7f031a4bd640 1 Processor -- start 2024-04-25T01:29:46.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.886+0000 7f031a4bd640 1 -- start start 2024-04-25T01:29:46.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.887+0000 7f031a4bd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03140b06f0 0x7f03140c9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:46.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.887+0000 7f031a4bd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 0x7f03140c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:46.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.887+0000 7f031a4bd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 0x7f0314155e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:46.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.887+0000 7f031a4bd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f0314159b50 con 0x7f03140c5b20 2024-04-25T01:29:46.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.887+0000 7f031a4bd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0314158f60 con 0x7f0314155970 2024-04-25T01:29:46.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.887+0000 7f031a4bd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0314159220 con 0x7f03140b06f0 2024-04-25T01:29:46.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.887+0000 7f0313fff640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03140b06f0 msgr2=0x7f03140c9150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:29:46.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.887+0000 7f0313fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03140b06f0 0x7f03140c9150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:46.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.887+0000 7f03137fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 0x7f03140c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:46.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.887+0000 7f03137fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 0x7f03140c5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54314/0 (socket says 172.21.15.112:54314) 2024-04-25T01:29:46.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.887+0000 7f03137fe640 1 -- 172.21.15.112:0/3261407723 learned_addr learned my addr 172.21.15.112:0/3261407723 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:46.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.887+0000 7f0318a5a640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 0x7f0314155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:46.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.888+0000 7f03137fe640 1 -- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 msgr2=0x7f03140c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:29:46.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.888+0000 7f03137fe640 1 -- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 msgr2=0x7f03140c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:29:46.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.888+0000 7f03137fe640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 0x7f03140c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:29:46.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.888+0000 7f0318a5a640 1 -- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 msgr2=0x7f0314155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:29:46.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.888+0000 7f0318a5a640 1 -- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 msgr2=0x7f0314155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:29:46.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.888+0000 7f0318a5a640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 0x7f0314155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:29:46.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.888+0000 7f0318a5a640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 0x7f0314155e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:46.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:46.888+0000 7f03137fe640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 0x7f03140c5fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:47.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.088+0000 7f0313fff640 1 -- 172.21.15.112:0/3261407723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03140b06f0 msgr2=0x7f03140c9150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:29:47.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.088+0000 7f0313fff640 1 --2- 172.21.15.112:0/3261407723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03140b06f0 0x7f03140c9150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:29:47.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.089+0000 7f03137fe640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 0x7f03140c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:47.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.089+0000 7f0318a5a640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 0x7f0314155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:47.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.089+0000 7f03137fe640 1 -- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 msgr2=0x7f03140c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:29:47.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.089+0000 7f03137fe640 1 -- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 msgr2=0x7f03140c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:29:47.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.089+0000 7f03137fe640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 0x7f03140c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:29:47.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.089+0000 7f03137fe640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 0x7f03140c5fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:29:47.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.089+0000 7f0318a5a640 1 -- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 msgr2=0x7f0314155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:29:47.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.089+0000 7f0318a5a640 1 -- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 msgr2=0x7f0314155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:29:47.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.089+0000 7f0318a5a640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 0x7f0314155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:29:47.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.089+0000 7f0318a5a640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 0x7f0314155e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:29:47.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.489+0000 7f0313fff640 1 -- 172.21.15.112:0/3261407723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03140b06f0 msgr2=0x7f03140c9150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:29:47.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.489+0000 7f0313fff640 1 --2- 172.21.15.112:0/3261407723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03140b06f0 0x7f03140c9150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:29:47.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.490+0000 7f03137fe640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 0x7f03140c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:47.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.490+0000 7f0318a5a640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 0x7f0314155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:47.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.490+0000 7f03137fe640 1 -- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 msgr2=0x7f03140c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:29:47.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.490+0000 7f03137fe640 1 -- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 msgr2=0x7f03140c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:29:47.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.490+0000 7f03137fe640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 0x7f03140c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:29:47.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.490+0000 7f03137fe640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 0x7f03140c5fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:29:47.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.490+0000 7f0318a5a640 1 -- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 msgr2=0x7f0314155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:29:47.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.490+0000 7f0318a5a640 1 -- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 msgr2=0x7f0314155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:29:47.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.490+0000 7f0318a5a640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 0x7f0314155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:29:47.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:47.490+0000 7f0318a5a640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 0x7f0314155e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:29:48.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:48.289+0000 7f0313fff640 1 -- 172.21.15.112:0/3261407723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03140b06f0 msgr2=0x7f03140c9150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:29:48.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:48.289+0000 7f0313fff640 1 --2- 172.21.15.112:0/3261407723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03140b06f0 0x7f03140c9150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:29:48.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:48.290+0000 7f03137fe640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 0x7f03140c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:48.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:48.290+0000 7f0318a5a640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 0x7f0314155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:48.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:48.290+0000 7f03137fe640 1 -- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 msgr2=0x7f03140c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:29:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:48.290+0000 7f03137fe640 1 -- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 msgr2=0x7f03140c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:29:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:48.290+0000 7f03137fe640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 0x7f03140c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:29:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:48.290+0000 7f03137fe640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 0x7f03140c5fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:29:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:48.290+0000 7f0318a5a640 1 -- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 msgr2=0x7f0314155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:29:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:48.290+0000 7f0318a5a640 1 -- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 msgr2=0x7f0314155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:29:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:48.290+0000 7f0318a5a640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 0x7f0314155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:29:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:48.290+0000 7f0318a5a640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 0x7f0314155e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:29:49.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:49.886+0000 7f0311ffb640 1 -- 172.21.15.112:0/3261407723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03140b06f0 msgr2=0x7f03140c9150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:29:49.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:49.886+0000 7f0311ffb640 1 --2- 172.21.15.112:0/3261407723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03140b06f0 0x7f03140c9150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:49.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:49.886+0000 7f0311ffb640 1 -- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 msgr2=0x7f0314155e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:29:49.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:49.886+0000 7f0311ffb640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.187:3300/0 conn(0x7f0314155970 0x7f0314155e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:49.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:49.886+0000 7f0311ffb640 1 -- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 msgr2=0x7f03140c5fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:29:49.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:49.886+0000 7f0311ffb640 1 --2- 172.21.15.112:0/3261407723 >> v2:172.21.15.112:3300/0 conn(0x7f03140c5b20 0x7f03140c5fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:49.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:49.886+0000 7f0311ffb640 1 -- 172.21.15.112:0/3261407723 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0314159500 con 0x7f0300003670 2024-04-25T01:29:49.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:49.886+0000 7f0311ffb640 1 -- 172.21.15.112:0/3261407723 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f030000c140 con 0x7f03000073c0 2024-04-25T01:29:50.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.565+0000 7f0312ffd640 1 -- 172.21.15.112:0/3261407723 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1929002000 0 0) 0x7f030000c140 con 0x7f03000073c0 2024-04-25T01:29:50.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.565+0000 7f0312ffd640 1 -- 172.21.15.112:0/3261407723 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02fc003520 con 0x7f03000073c0 2024-04-25T01:29:50.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f0312ffd640 1 -- 172.21.15.112:0/3261407723 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4289000144 0 0) 0x7f02fc003520 con 0x7f03000073c0 2024-04-25T01:29:50.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f0312ffd640 1 -- 172.21.15.112:0/3261407723 >> v1:172.21.15.112:6789/0 conn(0x7f0300003670 legacy=0x7f0300005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:50.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f0312ffd640 1 -- 172.21.15.112:0/3261407723 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030000d3e0 con 0x7f03000073c0 2024-04-25T01:29:50.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f0312ffd640 1 -- 172.21.15.112:0/3261407723 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f030800f040 con 0x7f03000073c0 2024-04-25T01:29:50.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f0312ffd640 1 -- 172.21.15.112:0/3261407723 >> v1:172.21.15.187:6789/0 conn(0x7f03000073c0 legacy=0x7f0300009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:50.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f0318a5a640 1 -- 172.21.15.112:0/3261407723 reap_dead start 2024-04-25T01:29:50.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f0312ffd640 1 --2- 172.21.15.112:0/3261407723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02fc0039b0 0x7f02fc003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:50.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f0312ffd640 1 --2- 172.21.15.112:0/3261407723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02fc004800 0x7f02fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:50.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f0312ffd640 1 --2- 172.21.15.112:0/3261407723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02fc0052e0 0x7f02fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:50.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f0312ffd640 1 -- 172.21.15.112:0/3261407723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc00c200 con 0x7f02fc004800 2024-04-25T01:29:50.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f0312ffd640 1 -- 172.21.15.112:0/3261407723 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc00c3b0 con 0x7f02fc0052e0 2024-04-25T01:29:50.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f0312ffd640 1 -- 172.21.15.112:0/3261407723 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f02fc00c5b0 con 0x7f02fc0039b0 2024-04-25T01:29:50.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f0313fff640 1 -- 172.21.15.112:0/3261407723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02fc0039b0 msgr2=0x7f02fc003db0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:29:50.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f0313fff640 1 --2- 172.21.15.112:0/3261407723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02fc0039b0 0x7f02fc003db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:50.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f03137fe640 1 --2- 172.21.15.112:0/3261407723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02fc004800 0x7f02fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:50.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f0318a5a640 1 --2- 172.21.15.112:0/3261407723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02fc0052e0 0x7f02fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:50.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f03137fe640 1 -- 172.21.15.112:0/3261407723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02fc0039b0 msgr2=0x7f02fc003db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:29:50.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f03137fe640 1 --2- 172.21.15.112:0/3261407723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02fc0039b0 0x7f02fc003db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:50.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f03137fe640 1 -- 172.21.15.112:0/3261407723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02fc0052e0 msgr2=0x7f02fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:29:50.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f03137fe640 1 --2- 172.21.15.112:0/3261407723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02fc0052e0 0x7f02fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:50.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.566+0000 7f03137fe640 1 -- 172.21.15.112:0/3261407723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02fc003520 con 0x7f02fc004800 2024-04-25T01:29:50.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.567+0000 7f03137fe640 1 --2- 172.21.15.112:0/3261407723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02fc004800 0x7f02fc004c80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0308009100 tx=0x7f02fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a784a0834aa29f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:50.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.571+0000 7f0312ffd640 1 -- 172.21.15.112:0/3261407723 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308031070 con 0x7f02fc004800 2024-04-25T01:29:50.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.571+0000 7f0312ffd640 1 -- 172.21.15.112:0/3261407723 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f030802be40 con 0x7f02fc004800 2024-04-25T01:29:50.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.571+0000 7f0312ffd640 1 -- 172.21.15.112:0/3261407723 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030800cb50 con 0x7f02fc004800 2024-04-25T01:29:50.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.572+0000 7f031a4bd640 1 -- 172.21.15.112:0/3261407723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02fc004800 msgr2=0x7f02fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:50.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.572+0000 7f031a4bd640 1 --2- 172.21.15.112:0/3261407723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02fc004800 0x7f02fc004c80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0308009100 tx=0x7f02fc005830 comp rx=0 tx=0).stop 2024-04-25T01:29:50.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.572+0000 7f031a4bd640 1 -- 172.21.15.112:0/3261407723 shutdown_connections 2024-04-25T01:29:50.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.572+0000 7f031a4bd640 1 --2- 172.21.15.112:0/3261407723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02fc0052e0 0x7f02fc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:50.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.572+0000 7f031a4bd640 1 --2- 172.21.15.112:0/3261407723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02fc004800 0x7f02fc004c80 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:50.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.572+0000 7f031a4bd640 1 --2- 172.21.15.112:0/3261407723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02fc0039b0 0x7f02fc003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:50.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.572+0000 7f031a4bd640 1 -- 172.21.15.112:0/3261407723 >> 172.21.15.112:0/3261407723 conn(0x7f0314074d80 msgr2=0x7f03140cb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:50.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.572+0000 7f031a4bd640 1 -- 172.21.15.112:0/3261407723 shutdown_connections 2024-04-25T01:29:50.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.572+0000 7f031a4bd640 1 -- 172.21.15.112:0/3261407723 wait complete. 2024-04-25T01:29:50.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.573+0000 7f031a4bd640 1 Processor -- start 2024-04-25T01:29:50.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.574+0000 7f031a4bd640 1 -- start start 2024-04-25T01:29:50.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.574+0000 7f031a4bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02fc0039b0 0x7f03140bdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:50.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.574+0000 7f031a4bd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02fc004800 0x7f03140af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:50.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.574+0000 7f031a4bd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02fc0052e0 0x7f03140c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:50.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.574+0000 7f031a4bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f03140b0ad0 con 0x7f02fc0039b0 2024-04-25T01:29:50.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.574+0000 7f031a4bd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f031406db80 con 0x7f02fc004800 2024-04-25T01:29:50.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.574+0000 7f031a4bd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f03140c5fa0 con 0x7f02fc0052e0 2024-04-25T01:29:50.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.574+0000 7f0318a5a640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02fc0052e0 msgr2=0x7f03140c21e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:29:50.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.574+0000 7f0313fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02fc0039b0 0x7f03140bdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:50.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.574+0000 7f0318a5a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02fc0052e0 0x7f03140c21e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:50.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.574+0000 7f0313fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02fc0039b0 0x7f03140bdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51326/0 (socket says 172.21.15.112:51326) 2024-04-25T01:29:50.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.574+0000 7f03137fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02fc004800 0x7f03140af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:50.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.574+0000 7f0313fff640 1 -- 172.21.15.112:0/3797417186 learned_addr learned my addr 172.21.15.112:0/3797417186 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:50.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.575+0000 7f0313fff640 1 -- 172.21.15.112:0/3797417186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02fc0052e0 msgr2=0x7f03140c21e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:29:50.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.575+0000 7f0313fff640 1 --2- 172.21.15.112:0/3797417186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02fc0052e0 0x7f03140c21e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:50.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.575+0000 7f0313fff640 1 -- 172.21.15.112:0/3797417186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02fc004800 msgr2=0x7f03140af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:50.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.575+0000 7f0313fff640 1 --2- 172.21.15.112:0/3797417186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02fc004800 0x7f03140af620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:50.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.575+0000 7f0313fff640 1 -- 172.21.15.112:0/3797417186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03141ea9b0 con 0x7f02fc0039b0 2024-04-25T01:29:50.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.575+0000 7f0313fff640 1 --2- 172.21.15.112:0/3797417186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02fc0039b0 0x7f03140bdb20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0304003bd0 tx=0x7f030400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:50.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.575+0000 7f0310ff9640 1 -- 172.21.15.112:0/3797417186 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030400cd10 con 0x7f02fc0039b0 2024-04-25T01:29:50.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.575+0000 7f0310ff9640 1 -- 172.21.15.112:0/3797417186 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0304010030 con 0x7f02fc0039b0 2024-04-25T01:29:50.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.575+0000 7f031a4bd640 1 -- 172.21.15.112:0/3797417186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03140be060 con 0x7f02fc0039b0 2024-04-25T01:29:50.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.575+0000 7f0310ff9640 1 -- 172.21.15.112:0/3797417186 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304016670 con 0x7f02fc0039b0 2024-04-25T01:29:50.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.575+0000 7f031a4bd640 1 -- 172.21.15.112:0/3797417186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03141eac70 con 0x7f02fc0039b0 2024-04-25T01:29:50.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.576+0000 7f031a4bd640 1 -- 172.21.15.112:0/3797417186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03140682a0 con 0x7f02fc0039b0 2024-04-25T01:29:50.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.577+0000 7f0310ff9640 1 -- 172.21.15.112:0/3797417186 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f03040153e0 con 0x7f02fc0039b0 2024-04-25T01:29:50.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.577+0000 7f0310ff9640 1 --2- 172.21.15.112:0/3797417186 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f030006c730 0x7f030006ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:50.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.577+0000 7f0310ff9640 1 -- 172.21.15.112:0/3797417186 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f030408b1d0 con 0x7f02fc0039b0 2024-04-25T01:29:50.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.577+0000 7f03137fe640 1 --2- 172.21.15.112:0/3797417186 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f030006c730 0x7f030006ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:50.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.578+0000 7f03137fe640 1 --2- 172.21.15.112:0/3797417186 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f030006c730 0x7f030006ebf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0308036000 tx=0x7f0308031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:50.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.582+0000 7f0310ff9640 1 -- 172.21.15.112:0/3797417186 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0304059220 con 0x7f02fc0039b0 2024-04-25T01:29:50.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.885+0000 7f031a4bd640 1 -- 172.21.15.112:0/3797417186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0314158f60 con 0x7f02fc0039b0 2024-04-25T01:29:50.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.885+0000 7f0310ff9640 1 -- 172.21.15.112:0/3797417186 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7f030405d520 con 0x7f02fc0039b0 2024-04-25T01:29:50.865 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":30,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:29:50.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.887+0000 7f031a4bd640 1 -- 172.21.15.112:0/3797417186 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f030006c730 msgr2=0x7f030006ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:50.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.887+0000 7f031a4bd640 1 --2- 172.21.15.112:0/3797417186 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f030006c730 0x7f030006ebf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0308036000 tx=0x7f0308031000 comp rx=0 tx=0).stop 2024-04-25T01:29:50.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.887+0000 7f031a4bd640 1 -- 172.21.15.112:0/3797417186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02fc0039b0 msgr2=0x7f03140bdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:50.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.887+0000 7f031a4bd640 1 --2- 172.21.15.112:0/3797417186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02fc0039b0 0x7f03140bdb20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0304003bd0 tx=0x7f030400be60 comp rx=0 tx=0).stop 2024-04-25T01:29:50.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.888+0000 7f031a4bd640 1 -- 172.21.15.112:0/3797417186 shutdown_connections 2024-04-25T01:29:50.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.888+0000 7f031a4bd640 1 --2- 172.21.15.112:0/3797417186 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f030006c730 0x7f030006ebf0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:50.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.888+0000 7f031a4bd640 1 --2- 172.21.15.112:0/3797417186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02fc0052e0 0x7f03140c21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:50.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.888+0000 7f031a4bd640 1 --2- 172.21.15.112:0/3797417186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02fc004800 0x7f03140af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:50.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.888+0000 7f031a4bd640 1 --2- 172.21.15.112:0/3797417186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02fc0039b0 0x7f03140bdb20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:50.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.888+0000 7f031a4bd640 1 -- 172.21.15.112:0/3797417186 >> 172.21.15.112:0/3797417186 conn(0x7f0314074d80 msgr2=0x7f0314154880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:50.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.888+0000 7f031a4bd640 1 -- 172.21.15.112:0/3797417186 shutdown_connections 2024-04-25T01:29:50.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:50.888+0000 7f031a4bd640 1 -- 172.21.15.112:0/3797417186 wait complete. 2024-04-25T01:29:50.878 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T01:29:50.878 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-04-25T01:29:50.878 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:29:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.005+0000 7f63c3ae1640 1 Processor -- start 2024-04-25T01:29:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.005+0000 7f63c3ae1640 1 -- start start 2024-04-25T01:29:50.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.006+0000 7f63c3ae1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f63bc1592c0 0x7f63bc0c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:50.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.006+0000 7f63c3ae1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63bc0c72b0 0x7f63bc160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:50.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.006+0000 7f63c3ae1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63bc159b90 con 0x7f63bc1596a0 2024-04-25T01:29:50.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.006+0000 7f63c3ae1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f63bc0c6e20 con 0x7f63bc1592c0 2024-04-25T01:29:50.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.006+0000 7f63c3ae1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f63bc06c970 con 0x7f63bc0c72b0 2024-04-25T01:29:50.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.006+0000 7f63c207e640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63bc0c72b0 msgr2=0x7f63bc160380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:29:50.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.007+0000 7f63c207e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63bc0c72b0 0x7f63bc160380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:50.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.007+0000 7f63c187d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f63bc1592c0 0x7f63bc0c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:50.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.007+0000 7f63c187d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f63bc1592c0 0x7f63bc0c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51338/0 (socket says 172.21.15.112:51338) 2024-04-25T01:29:50.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.007+0000 7f63c187d640 1 -- 172.21.15.112:0/1806388784 learned_addr learned my addr 172.21.15.112:0/1806388784 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:50.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.007+0000 7f63c187d640 1 -- 172.21.15.112:0/1806388784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63bc0c72b0 msgr2=0x7f63bc160380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:29:50.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.007+0000 7f63c187d640 1 --2- 172.21.15.112:0/1806388784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63bc0c72b0 0x7f63bc160380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:50.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.007+0000 7f63c187d640 1 -- 172.21.15.112:0/1806388784 >> v1:172.21.15.112:6789/0 conn(0x7f63bc1596a0 legacy=0x7f63bc152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:50.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.007+0000 7f63c187d640 1 -- 172.21.15.112:0/1806388784 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63bc06cc50 con 0x7f63bc1592c0 2024-04-25T01:29:50.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.008+0000 7f63c187d640 1 --2- 172.21.15.112:0/1806388784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc1592c0 0x7f63bc0c9070 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009870 tx=0x7f63ac02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c74d74b43f29fa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:50.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.008+0000 7f63c087b640 1 -- 172.21.15.112:0/1806388784 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac037070 con 0x7f63bc1592c0 2024-04-25T01:29:50.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.008+0000 7f63c087b640 1 -- 172.21.15.112:0/1806388784 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63ac032440 con 0x7f63bc1592c0 2024-04-25T01:29:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.008+0000 7f63c087b640 1 -- 172.21.15.112:0/1806388784 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac032810 con 0x7f63bc1592c0 2024-04-25T01:29:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.008+0000 7f63c3ae1640 1 -- 172.21.15.112:0/1806388784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc1592c0 msgr2=0x7f63bc0c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.008+0000 7f63c3ae1640 1 --2- 172.21.15.112:0/1806388784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc1592c0 0x7f63bc0c9070 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009870 tx=0x7f63ac02a150 comp rx=0 tx=0).stop 2024-04-25T01:29:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.008+0000 7f63c3ae1640 1 -- 172.21.15.112:0/1806388784 shutdown_connections 2024-04-25T01:29:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.008+0000 7f63c3ae1640 1 --2- 172.21.15.112:0/1806388784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63bc0c72b0 0x7f63bc160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.008+0000 7f63c3ae1640 1 --2- 172.21.15.112:0/1806388784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc1592c0 0x7f63bc0c9070 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.008+0000 7f63c3ae1640 1 -- 172.21.15.112:0/1806388784 >> 172.21.15.112:0/1806388784 conn(0x7f63bc0cb170 msgr2=0x7f63bc0cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.008+0000 7f63c3ae1640 1 -- 172.21.15.112:0/1806388784 shutdown_connections 2024-04-25T01:29:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.008+0000 7f63c3ae1640 1 -- 172.21.15.112:0/1806388784 wait complete. 2024-04-25T01:29:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.009+0000 7f63c3ae1640 1 Processor -- start 2024-04-25T01:29:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.010+0000 7f63c3ae1640 1 -- start start 2024-04-25T01:29:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.010+0000 7f63c3ae1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63bc0c72b0 0x7f63bc0bef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.010+0000 7f63c3ae1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63bc1592c0 0x7f63bc151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.010+0000 7f63c3ae1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc1596a0 0x7f63bc151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.010+0000 7f63c3ae1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63bc153590 con 0x7f63bc1596a0 2024-04-25T01:29:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.010+0000 7f63c3ae1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f63bc153410 con 0x7f63bc1592c0 2024-04-25T01:29:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.010+0000 7f63c3ae1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f63bc06db20 con 0x7f63bc0c72b0 2024-04-25T01:29:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.010+0000 7f63c187d640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63bc0c72b0 msgr2=0x7f63bc0bef10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:29:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.010+0000 7f63c187d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63bc0c72b0 0x7f63bc0bef10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.010+0000 7f63c207e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc1596a0 0x7f63bc151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.010+0000 7f63c207e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc1596a0 0x7f63bc151890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51342/0 (socket says 172.21.15.112:51342) 2024-04-25T01:29:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.010+0000 7f63c207e640 1 -- 172.21.15.112:0/3982673840 learned_addr learned my addr 172.21.15.112:0/3982673840 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.010+0000 7f63c107c640 1 --2- 172.21.15.112:0/3982673840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63bc1592c0 0x7f63bc151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.011+0000 7f63c207e640 1 -- 172.21.15.112:0/3982673840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63bc0c72b0 msgr2=0x7f63bc0bef10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:29:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.011+0000 7f63c207e640 1 --2- 172.21.15.112:0/3982673840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63bc0c72b0 0x7f63bc0bef10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.011+0000 7f63c207e640 1 -- 172.21.15.112:0/3982673840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63bc1592c0 msgr2=0x7f63bc151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.011+0000 7f63c207e640 1 --2- 172.21.15.112:0/3982673840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63bc1592c0 0x7f63bc151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.011+0000 7f63c207e640 1 -- 172.21.15.112:0/3982673840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63bc151dd0 con 0x7f63bc1596a0 2024-04-25T01:29:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.011+0000 7f63c207e640 1 --2- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc1596a0 0x7f63bc151890 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f63b400b9c0 tx=0x7f63b400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.011+0000 7f63b27fc640 1 -- 172.21.15.112:0/3982673840 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b400cb20 con 0x7f63bc1596a0 2024-04-25T01:29:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.011+0000 7f63b27fc640 1 -- 172.21.15.112:0/3982673840 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63b4010070 con 0x7f63bc1596a0 2024-04-25T01:29:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.011+0000 7f63b27fc640 1 -- 172.21.15.112:0/3982673840 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b4015470 con 0x7f63bc1596a0 2024-04-25T01:29:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.011+0000 7f63c3ae1640 1 -- 172.21.15.112:0/3982673840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63bc154490 con 0x7f63bc1596a0 2024-04-25T01:29:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.011+0000 7f63c3ae1640 1 -- 172.21.15.112:0/3982673840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63bc155040 con 0x7f63bc1596a0 2024-04-25T01:29:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.012+0000 7f63c3ae1640 1 --2- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc0c15d0 0x7f63bc1eb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:50.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.012+0000 7f63c3ae1640 1 -- 172.21.15.112:0/3982673840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63bc075980 con 0x7f63bc0c15d0 2024-04-25T01:29:50.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.012+0000 7f63c187d640 1 --2- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc0c15d0 0x7f63bc1eb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:50.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.012+0000 7f63c187d640 1 -- 172.21.15.112:0/3982673840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f63bc0679a0 con 0x7f63bc0c15d0 2024-04-25T01:29:50.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.013+0000 7f63c187d640 1 --2- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc0c15d0 0x7f63bc1eb770 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0bea40 tx=0x7f63ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:50.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.013+0000 7f63b27fc640 1 -- 172.21.15.112:0/3982673840 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac037040 con 0x7f63bc0c15d0 2024-04-25T01:29:50.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.013+0000 7f63b27fc640 1 -- 172.21.15.112:0/3982673840 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f63b401e3f0 con 0x7f63bc1596a0 2024-04-25T01:29:50.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.013+0000 7f63b27fc640 1 --2- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f63a006bd30 0x7f63a006e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:50.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.013+0000 7f63b27fc640 1 -- 172.21.15.112:0/3982673840 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f63b408b2e0 con 0x7f63bc1596a0 2024-04-25T01:29:50.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.013+0000 7f63c107c640 1 --2- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f63a006bd30 0x7f63a006e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:50.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.013+0000 7f63b27fc640 1 -- 172.21.15.112:0/3982673840 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f63ac004020 con 0x7f63bc0c15d0 2024-04-25T01:29:50.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.013+0000 7f63b27fc640 1 -- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc0c15d0 msgr2=0x7f63bc1eb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:50.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.013+0000 7f63b27fc640 1 --2- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc0c15d0 0x7f63bc1eb770 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0bea40 tx=0x7f63ac0046c0 comp rx=0 tx=0).stop 2024-04-25T01:29:50.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.013+0000 7f63b27fc640 1 -- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc0c15d0 msgr2=0x7f63bc1eb770 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:29:50.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.013+0000 7f63b27fc640 1 --2- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc0c15d0 0x7f63bc1eb770 secure :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0bea40 tx=0x7f63ac0046c0 comp rx=0 tx=0).stop 2024-04-25T01:29:50.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.014+0000 7f63c107c640 1 --2- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f63a006bd30 0x7f63a006e1f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f63bc1543c0 tx=0x7f63b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:50.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.020+0000 7f63c3ae1640 1 --2- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc0c1cd0 0x7f63bc1ebcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:50.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.020+0000 7f63c3ae1640 1 -- 172.21.15.112:0/3982673840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63bc152720 con 0x7f63bc0c1cd0 2024-04-25T01:29:50.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.020+0000 7f63c187d640 1 --2- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc0c1cd0 0x7f63bc1ebcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:51.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.021+0000 7f63c187d640 1 -- 172.21.15.112:0/3982673840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f63bc0c79d0 con 0x7f63bc0c1cd0 2024-04-25T01:29:51.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.021+0000 7f63c187d640 1 --2- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc0c1cd0 0x7f63bc1ebcb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f63ac000c00 tx=0x7f63ac009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:51.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.021+0000 7f63b27fc640 1 -- 172.21.15.112:0/3982673840 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac047230 con 0x7f63bc0c1cd0 2024-04-25T01:29:51.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.021+0000 7f63b27fc640 1 -- 172.21.15.112:0/3982673840 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5060 (secure 0 0 0) 0x7f63ac047230 con 0x7f63bc0c1cd0 2024-04-25T01:29:51.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.021+0000 7f63b27fc640 1 -- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc0c1cd0 msgr2=0x7f63bc1ebcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:51.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.021+0000 7f63b27fc640 1 --2- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc0c1cd0 0x7f63bc1ebcb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f63ac000c00 tx=0x7f63ac009240 comp rx=0 tx=0).stop 2024-04-25T01:29:51.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.021+0000 7f63b27fc640 1 -- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc0c1cd0 msgr2=0x7f63bc1ebcb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:29:51.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.021+0000 7f63b27fc640 1 --2- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc0c1cd0 0x7f63bc1ebcb0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:51.001 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:29:51.001 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:29:51.001 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:29:51.001 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:29:51.001 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 30, 2024-04-25T01:29:51.002 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 262650, 2024-04-25T01:29:51.002 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:29:51.002 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:29:51.002 INFO:teuthology.orchestra.run.smithi112.stdout: 1 2024-04-25T01:29:51.002 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:51.002 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:29:51.002 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:29:51.002 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:29:51.002 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:29:51.002 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:51.002 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:51.002 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:51.002 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:51.002 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:51.003 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:51.003 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:51.003 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:51.003 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:51.003 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:51.003 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:51.003 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:51.003 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:29:51.003 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:29:51.003 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:51.003 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:51.003 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:51.003 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:51.003 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:51.003 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:51.003 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:51.004 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:51.004 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:51.004 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:51.004 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:51.004 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:51.004 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.004 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:29:51.004 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:29:51.004 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:29:51.004 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:29:51.004 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:29:51.004 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:29:51.004 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:29:51.004 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:29:51.004 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:29:51.004 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:29:51.004 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:29:51.005 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:29:51.005 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:29:51.005 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:29:51.005 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:29:51.005 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:29:51.005 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:29:51.005 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:51.005 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:51.005 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:51.005 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:51.005 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:51.005 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:51.005 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:51.005 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:51.005 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:51.005 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:51.006 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:51.006 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:51.006 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:29:51.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.006 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:29:51.006 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.006 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:29:51.006 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:29:51.006 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:51.006 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:51.006 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.006 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:51.006 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:29:51.006 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:51.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.007 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.007 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:51.007 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:29:51.007 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:51.007 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:51.007 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:51.007 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.007 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:29:51.007 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:29:51.007 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:51.007 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:51.007 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:51.007 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.007 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.007 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:29:51.007 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:29:51.008 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:51.008 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:51.008 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.008 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:51.008 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:29:51.008 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:51.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.008 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.008 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:51.008 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:29:51.008 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:51.008 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:51.008 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:51.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.008 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:29:51.008 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:29:51.008 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:51.009 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:51.009 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:51.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.009 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.009 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:29:51.009 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:29:51.009 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:51.009 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:51.009 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.009 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:51.009 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:29:51.009 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:51.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.009 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.009 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:51.009 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:29:51.010 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:51.010 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:51.010 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:51.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.010 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:29:51.010 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:29:51.010 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:51.010 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:51.010 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:51.010 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:51.010 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:51.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.010 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:29:51.010 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:29:51.010 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.011 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:51.011 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:51.011 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:29:51.011 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:51.011 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:51.011 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:29:51.011 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.011 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:51.011 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:51.011 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:29:51.011 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:51.011 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:51.011 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:29:51.011 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.011 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:51.011 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:51.012 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 7 2024-04-25T01:29:51.012 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:51.012 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:51.012 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:29:51.012 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.012 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:51.012 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:51.012 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 8 2024-04-25T01:29:51.012 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:51.012 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:51.012 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:29:51.012 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.012 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:51.012 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:51.012 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:29:51.012 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:51.013 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:51.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.013 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:29:51.013 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:29:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.022+0000 7f63c3ae1640 1 -- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f63a006bd30 msgr2=0x7f63a006e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.022+0000 7f63c3ae1640 1 --2- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f63a006bd30 0x7f63a006e1f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f63bc1543c0 tx=0x7f63b8009040 comp rx=0 tx=0).stop 2024-04-25T01:29:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.022+0000 7f63c3ae1640 1 -- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc1596a0 msgr2=0x7f63bc151890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.022+0000 7f63c3ae1640 1 --2- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc1596a0 0x7f63bc151890 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f63b400b9c0 tx=0x7f63b400be80 comp rx=0 tx=0).stop 2024-04-25T01:29:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.022+0000 7f63c207e640 1 -- 172.21.15.112:0/3982673840 reap_dead start 2024-04-25T01:29:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.022+0000 7f63c3ae1640 1 -- 172.21.15.112:0/3982673840 shutdown_connections 2024-04-25T01:29:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.022+0000 7f63c3ae1640 1 --2- 172.21.15.112:0/3982673840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63bc1596a0 0x7f63bc151890 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.022+0000 7f63c3ae1640 1 -- 172.21.15.112:0/3982673840 >> 172.21.15.112:0/3982673840 conn(0x7f63bc0cb170 msgr2=0x7f63bc0ac4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.022+0000 7f63c3ae1640 1 -- 172.21.15.112:0/3982673840 shutdown_connections 2024-04-25T01:29:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.022+0000 7f63c3ae1640 1 -- 172.21.15.112:0/3982673840 wait complete. 2024-04-25T01:29:51.015 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:29:51.015 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-04-25T01:29:51.015 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:29:51.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.143+0000 7f9d36514640 1 Processor -- start 2024-04-25T01:29:51.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.144+0000 7f9d36514640 1 -- start start 2024-04-25T01:29:51.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.144+0000 7f9d36514640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d30072d50 0x7f9d30071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:51.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.144+0000 7f9d36514640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9d301563e0 0x7f9d3006f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:51.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.144+0000 7f9d36514640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9d3006b160 0x7f9d3006b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:51.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.144+0000 7f9d36514640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f9d3006f3b0 con 0x7f9d301563e0 2024-04-25T01:29:51.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.144+0000 7f9d36514640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9d30072220 con 0x7f9d3006b160 2024-04-25T01:29:51.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.144+0000 7f9d36514640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9d300c20e0 con 0x7f9d30072d50 2024-04-25T01:29:51.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.145+0000 7f9d2ffff640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d30072d50 msgr2=0x7f9d30071ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:29:51.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.145+0000 7f9d2ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d30072d50 0x7f9d30071ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:51.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.145+0000 7f9d2f7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9d301563e0 0x7f9d3006f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:51.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.145+0000 7f9d2f7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9d301563e0 0x7f9d3006f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51370/0 (socket says 172.21.15.112:51370) 2024-04-25T01:29:51.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.145+0000 7f9d2f7fe640 1 -- 172.21.15.112:0/2051710731 learned_addr learned my addr 172.21.15.112:0/2051710731 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:51.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.145+0000 7f9d34ab1640 1 --2- 172.21.15.112:0/2051710731 >> v2:172.21.15.187:3300/0 conn(0x7f9d3006b160 0x7f9d3006b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:51.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.145+0000 7f9d2f7fe640 1 -- 172.21.15.112:0/2051710731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d30072d50 msgr2=0x7f9d30071ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:29:51.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.145+0000 7f9d2f7fe640 1 --2- 172.21.15.112:0/2051710731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d30072d50 0x7f9d30071ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:51.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.145+0000 7f9d2f7fe640 1 -- 172.21.15.112:0/2051710731 >> v2:172.21.15.187:3300/0 conn(0x7f9d3006b160 msgr2=0x7f9d3006b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:29:51.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.145+0000 7f9d2f7fe640 1 --2- 172.21.15.112:0/2051710731 >> v2:172.21.15.187:3300/0 conn(0x7f9d3006b160 0x7f9d3006b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:51.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.145+0000 7f9d2f7fe640 1 -- 172.21.15.112:0/2051710731 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d30073130 con 0x7f9d301563e0 2024-04-25T01:29:51.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.145+0000 7f9d34ab1640 1 --2- 172.21.15.112:0/2051710731 >> v2:172.21.15.187:3300/0 conn(0x7f9d3006b160 0x7f9d3006b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:29:51.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.146+0000 7f9d2f7fe640 1 --2- 172.21.15.112:0/2051710731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d301563e0 0x7f9d3006f840 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9d3015a010 tx=0x7f9d2402a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afba40a41aed5cd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:51.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.146+0000 7f9d2effd640 1 -- 172.21.15.112:0/2051710731 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24037070 con 0x7f9d301563e0 2024-04-25T01:29:51.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.146+0000 7f9d2effd640 1 -- 172.21.15.112:0/2051710731 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d2402ae20 con 0x7f9d301563e0 2024-04-25T01:29:51.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.146+0000 7f9d2effd640 1 -- 172.21.15.112:0/2051710731 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24032660 con 0x7f9d301563e0 2024-04-25T01:29:51.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.147+0000 7f9d36514640 1 -- 172.21.15.112:0/2051710731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d301563e0 msgr2=0x7f9d3006f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:51.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.147+0000 7f9d36514640 1 --2- 172.21.15.112:0/2051710731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d301563e0 0x7f9d3006f840 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9d3015a010 tx=0x7f9d2402a0d0 comp rx=0 tx=0).stop 2024-04-25T01:29:51.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.147+0000 7f9d36514640 1 -- 172.21.15.112:0/2051710731 shutdown_connections 2024-04-25T01:29:51.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.147+0000 7f9d36514640 1 --2- 172.21.15.112:0/2051710731 >> v2:172.21.15.187:3300/0 conn(0x7f9d3006b160 0x7f9d3006b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:51.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.147+0000 7f9d36514640 1 --2- 172.21.15.112:0/2051710731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d301563e0 0x7f9d3006f840 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:51.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.147+0000 7f9d36514640 1 --2- 172.21.15.112:0/2051710731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d30072d50 0x7f9d30071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:51.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.147+0000 7f9d36514640 1 -- 172.21.15.112:0/2051710731 >> 172.21.15.112:0/2051710731 conn(0x7f9d300c8090 msgr2=0x7f9d300c8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:51.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.147+0000 7f9d36514640 1 -- 172.21.15.112:0/2051710731 shutdown_connections 2024-04-25T01:29:51.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.147+0000 7f9d36514640 1 -- 172.21.15.112:0/2051710731 wait complete. 2024-04-25T01:29:51.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.148+0000 7f9d36514640 1 Processor -- start 2024-04-25T01:29:51.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.148+0000 7f9d36514640 1 -- start start 2024-04-25T01:29:51.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.149+0000 7f9d36514640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d3006b160 0x7f9d301eb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:51.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.149+0000 7f9d36514640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d30072d50 0x7f9d300adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:51.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.149+0000 7f9d36514640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d301563e0 0x7f9d30151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:51.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.149+0000 7f9d36514640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3006ed60 con 0x7f9d30072d50 2024-04-25T01:29:51.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.149+0000 7f9d36514640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9d300c2660 con 0x7f9d301563e0 2024-04-25T01:29:51.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.149+0000 7f9d36514640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9d3006f040 con 0x7f9d3006b160 2024-04-25T01:29:51.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.149+0000 7f9d2ffff640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d3006b160 msgr2=0x7f9d301eb680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:29:51.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.149+0000 7f9d2ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d3006b160 0x7f9d301eb680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:29:51.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.149+0000 7f9d2f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d30072d50 0x7f9d300adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:51.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.149+0000 7f9d2f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d30072d50 0x7f9d300adc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51376/0 (socket says 172.21.15.112:51376) 2024-04-25T01:29:51.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.149+0000 7f9d2f7fe640 1 -- 172.21.15.112:0/2989745903 learned_addr learned my addr 172.21.15.112:0/2989745903 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:29:51.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.149+0000 7f9d34ab1640 1 --2- 172.21.15.112:0/2989745903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d301563e0 0x7f9d30151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:51.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.149+0000 7f9d2f7fe640 1 -- 172.21.15.112:0/2989745903 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d3006b160 msgr2=0x7f9d301eb680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:29:51.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.149+0000 7f9d2f7fe640 1 --2- 172.21.15.112:0/2989745903 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d3006b160 0x7f9d301eb680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:51.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.149+0000 7f9d2f7fe640 1 -- 172.21.15.112:0/2989745903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d301563e0 msgr2=0x7f9d30151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:51.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.149+0000 7f9d2f7fe640 1 --2- 172.21.15.112:0/2989745903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d301563e0 0x7f9d30151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:51.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.149+0000 7f9d2f7fe640 1 -- 172.21.15.112:0/2989745903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d300ae1b0 con 0x7f9d30072d50 2024-04-25T01:29:51.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.150+0000 7f9d2f7fe640 1 --2- 172.21.15.112:0/2989745903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d30072d50 0x7f9d300adc70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9d2402a110 tx=0x7f9d240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:51.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.150+0000 7f9d2cff9640 1 -- 172.21.15.112:0/2989745903 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24041070 con 0x7f9d30072d50 2024-04-25T01:29:51.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.150+0000 7f9d2cff9640 1 -- 172.21.15.112:0/2989745903 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d2403c670 con 0x7f9d30072d50 2024-04-25T01:29:51.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.150+0000 7f9d36514640 1 -- 172.21.15.112:0/2989745903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d301ebc60 con 0x7f9d30072d50 2024-04-25T01:29:51.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.150+0000 7f9d2cff9640 1 -- 172.21.15.112:0/2989745903 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24037040 con 0x7f9d30072d50 2024-04-25T01:29:51.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.150+0000 7f9d36514640 1 -- 172.21.15.112:0/2989745903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d301ec220 con 0x7f9d30072d50 2024-04-25T01:29:51.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.151+0000 7f9d36514640 1 --2- 172.21.15.112:0/2989745903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d301eebf0 0x7f9d301ef110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:51.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.151+0000 7f9d36514640 1 -- 172.21.15.112:0/2989745903 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9d30075980 con 0x7f9d301eebf0 2024-04-25T01:29:51.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.151+0000 7f9d2ffff640 1 --2- 172.21.15.112:0/2989745903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d301eebf0 0x7f9d301ef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:51.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.151+0000 7f9d2ffff640 1 -- 172.21.15.112:0/2989745903 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9d300689b0 con 0x7f9d301eebf0 2024-04-25T01:29:51.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.151+0000 7f9d2ffff640 1 --2- 172.21.15.112:0/2989745903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d301eebf0 0x7f9d301ef110 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9d20003930 tx=0x7f9d2000b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:51.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.151+0000 7f9d2cff9640 1 -- 172.21.15.112:0/2989745903 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d24044070 con 0x7f9d30072d50 2024-04-25T01:29:51.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.152+0000 7f9d2cff9640 1 --2- 172.21.15.112:0/2989745903 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9d1c06bde0 0x7f9d1c06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:51.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.152+0000 7f9d2cff9640 1 -- 172.21.15.112:0/2989745903 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f9d240ab700 con 0x7f9d30072d50 2024-04-25T01:29:51.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.152+0000 7f9d34ab1640 1 --2- 172.21.15.112:0/2989745903 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9d1c06bde0 0x7f9d1c06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:51.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.152+0000 7f9d2cff9640 1 -- 172.21.15.112:0/2989745903 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20003e10 con 0x7f9d301eebf0 2024-04-25T01:29:51.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.152+0000 7f9d2cff9640 1 -- 172.21.15.112:0/2989745903 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9d20015070 con 0x7f9d301eebf0 2024-04-25T01:29:51.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.152+0000 7f9d2cff9640 1 -- 172.21.15.112:0/2989745903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d301eebf0 msgr2=0x7f9d301ef110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:51.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.152+0000 7f9d2cff9640 1 --2- 172.21.15.112:0/2989745903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d301eebf0 0x7f9d301ef110 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9d20003930 tx=0x7f9d2000b8c0 comp rx=0 tx=0).stop 2024-04-25T01:29:51.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.152+0000 7f9d2cff9640 1 -- 172.21.15.112:0/2989745903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d301eebf0 msgr2=0x7f9d301ef110 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:29:51.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.152+0000 7f9d2cff9640 1 --2- 172.21.15.112:0/2989745903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d301eebf0 0x7f9d301ef110 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:51.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.152+0000 7f9d34ab1640 1 --2- 172.21.15.112:0/2989745903 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9d1c06bde0 0x7f9d1c06e2a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9d30159ff0 tx=0x7f9d280097d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:51.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.160+0000 7f9d36514640 1 --2- 172.21.15.112:0/2989745903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d301ef650 0x7f9d301508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:51.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.160+0000 7f9d36514640 1 -- 172.21.15.112:0/2989745903 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3006bf60 con 0x7f9d301ef650 2024-04-25T01:29:51.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.160+0000 7f9d2ffff640 1 --2- 172.21.15.112:0/2989745903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d301ef650 0x7f9d301508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:51.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.161+0000 7f9d2ffff640 1 -- 172.21.15.112:0/2989745903 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9d300c1a60 con 0x7f9d301ef650 2024-04-25T01:29:51.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.161+0000 7f9d2ffff640 1 --2- 172.21.15.112:0/2989745903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d301ef650 0x7f9d301508c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9d20003930 tx=0x7f9d20005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:51.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.161+0000 7f9d2cff9640 1 -- 172.21.15.112:0/2989745903 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20005820 con 0x7f9d301ef650 2024-04-25T01:29:51.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.161+0000 7f9d2cff9640 1 -- 172.21.15.112:0/2989745903 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4717 (secure 0 0 0) 0x7f9d20005820 con 0x7f9d301ef650 2024-04-25T01:29:51.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.161+0000 7f9d2cff9640 1 -- 172.21.15.112:0/2989745903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d301ef650 msgr2=0x7f9d301508c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:51.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.161+0000 7f9d2cff9640 1 --2- 172.21.15.112:0/2989745903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d301ef650 0x7f9d301508c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9d20003930 tx=0x7f9d20005a20 comp rx=0 tx=0).stop 2024-04-25T01:29:51.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.161+0000 7f9d2cff9640 1 -- 172.21.15.112:0/2989745903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d301ef650 msgr2=0x7f9d301508c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:29:51.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.161+0000 7f9d2cff9640 1 --2- 172.21.15.112:0/2989745903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d301ef650 0x7f9d301508c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:51.141 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:29:51.141 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:29:51.141 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:29:51.141 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:29:51.141 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 30, 2024-04-25T01:29:51.141 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 26851, 2024-04-25T01:29:51.141 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:29:51.141 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:29:51.141 INFO:teuthology.orchestra.run.smithi112.stdout: 1 2024-04-25T01:29:51.141 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:51.141 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:29:51.142 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:29:51.142 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:29:51.142 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:29:51.142 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:51.142 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:51.142 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:51.142 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:51.142 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:51.142 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:51.142 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:51.142 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:51.142 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:51.142 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:51.142 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:51.142 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:51.143 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:29:51.143 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:29:51.143 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:51.143 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:51.143 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:51.143 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:51.143 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:51.143 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:51.143 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:51.143 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:51.143 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:51.143 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:51.143 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:51.143 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:51.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.143 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:29:51.144 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:29:51.144 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:29:51.144 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:29:51.144 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:29:51.144 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:29:51.144 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:29:51.144 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:29:51.144 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:29:51.144 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:29:51.144 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:29:51.144 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:29:51.144 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:29:51.144 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:29:51.144 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:29:51.144 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:29:51.145 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:29:51.145 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:29:51.145 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:29:51.145 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:29:51.145 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:29:51.145 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:29:51.145 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:29:51.145 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:29:51.145 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:29:51.145 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:29:51.145 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:29:51.145 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:29:51.145 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:51.145 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:29:51.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.145 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:29:51.145 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.146 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:29:51.146 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:29:51.146 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:51.146 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:51.146 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.146 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:51.146 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:29:51.146 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:51.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.146 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.146 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:51.146 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:29:51.146 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:51.146 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:51.147 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:51.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.147 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:29:51.147 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:29:51.147 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:51.147 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:51.147 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:51.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.147 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.147 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:29:51.147 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:29:51.147 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:51.147 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:51.147 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.147 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:51.147 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:29:51.147 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:51.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.148 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.148 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:51.148 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:29:51.148 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:51.148 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:51.148 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:51.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.148 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:29:51.148 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:29:51.148 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:51.148 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:51.148 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:51.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.148 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.148 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:29:51.148 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:29:51.149 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:29:51.149 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:29:51.149 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.149 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:29:51.149 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:29:51.149 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:51.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.149 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.149 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:29:51.149 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:29:51.149 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:29:51.149 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:51.149 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:51.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.149 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:29:51.149 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:29:51.150 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:29:51.150 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:29:51.150 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:29:51.150 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:51.150 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:51.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.150 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:29:51.150 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:29:51.150 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.150 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:51.150 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:51.150 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:29:51.150 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:51.150 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:51.150 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:29:51.150 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.151 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:51.151 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:51.151 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:29:51.151 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:51.151 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:29:51.151 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:29:51.151 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:29:51.151 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:29:51.151 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:29:51.151 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:29:51.151 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:29:51.151 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:29:51.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:29:51.151 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:29:51.151 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:29:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.161+0000 7f9d36514640 1 -- 172.21.15.112:0/2989745903 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9d1c06bde0 msgr2=0x7f9d1c06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.161+0000 7f9d36514640 1 --2- 172.21.15.112:0/2989745903 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9d1c06bde0 0x7f9d1c06e2a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9d30159ff0 tx=0x7f9d280097d0 comp rx=0 tx=0).stop 2024-04-25T01:29:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.161+0000 7f9d36514640 1 -- 172.21.15.112:0/2989745903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d30072d50 msgr2=0x7f9d300adc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.161+0000 7f9d36514640 1 --2- 172.21.15.112:0/2989745903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d30072d50 0x7f9d300adc70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9d2402a110 tx=0x7f9d240046c0 comp rx=0 tx=0).stop 2024-04-25T01:29:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.162+0000 7f9d34ab1640 1 -- 172.21.15.112:0/2989745903 reap_dead start 2024-04-25T01:29:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.162+0000 7f9d34ab1640 1 -- 172.21.15.112:0/2989745903 reap_dead start 2024-04-25T01:29:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.162+0000 7f9d36514640 1 -- 172.21.15.112:0/2989745903 shutdown_connections 2024-04-25T01:29:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.162+0000 7f9d36514640 1 -- 172.21.15.112:0/2989745903 >> 172.21.15.112:0/2989745903 conn(0x7f9d300c8090 msgr2=0x7f9d300c68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.162+0000 7f9d36514640 1 -- 172.21.15.112:0/2989745903 shutdown_connections 2024-04-25T01:29:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:29:51.162+0000 7f9d36514640 1 -- 172.21.15.112:0/2989745903 wait complete. 2024-04-25T01:29:51.154 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:29:51.154 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T01:29:55.099 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:29:55.102 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:30:11.155 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-25T01:30:11.155 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-04-25T01:30:11.155 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T01:30:11.155 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T01:30:11.158 INFO:tasks.ceph.mon.c:Started 2024-04-25T01:30:11.158 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:30:11.158 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:30:11.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.290+0000 7f6d2ce10640 1 Processor -- start 2024-04-25T01:30:11.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.290+0000 7f6d2ce10640 1 -- start start 2024-04-25T01:30:11.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.291+0000 7f6d2ce10640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6d28155d80 0x7f6d28156160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.291+0000 7f6d2ce10640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d2806dfb0 con 0x7f6d2806dab0 2024-04-25T01:30:11.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.291+0000 7f6d2ce10640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d2806e290 con 0x7f6d2814c790 2024-04-25T01:30:11.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.291+0000 7f6d2ce10640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f6d2806ebc0 con 0x7f6d28155d80 2024-04-25T01:30:11.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.291+0000 7f6d2659c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6d28155d80 0x7f6d28156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.292+0000 7f6d2659c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6d28155d80 0x7f6d28156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49478/0 (socket says 172.21.15.112:49478) 2024-04-25T01:30:11.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.292+0000 7f6d2659c640 1 -- 172.21.15.112:0/503863886 learned_addr learned my addr 172.21.15.112:0/503863886 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:11.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.292+0000 7f6d2659c640 1 -- 172.21.15.112:0/503863886 >> v1:172.21.15.187:6789/0 conn(0x7f6d2814c790 legacy=0x7f6d28153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.292+0000 7f6d2659c640 1 -- 172.21.15.112:0/503863886 >> v1:172.21.15.112:6789/0 conn(0x7f6d2806dab0 legacy=0x7f6d2814c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.292+0000 7f6d2659c640 1 -- 172.21.15.112:0/503863886 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2806e570 con 0x7f6d28155d80 2024-04-25T01:30:11.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.293+0000 7f6d2659c640 1 --2- 172.21.15.112:0/503863886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d28155d80 0x7f6d28156160 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6d180098a0 tx=0x7f6d1802a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49dfb7c8d5b80bd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:11.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.293+0000 7f6d2559a640 1 -- 172.21.15.112:0/503863886 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18037070 con 0x7f6d28155d80 2024-04-25T01:30:11.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.293+0000 7f6d2559a640 1 -- 172.21.15.112:0/503863886 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d18032440 con 0x7f6d28155d80 2024-04-25T01:30:11.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.293+0000 7f6d2559a640 1 -- 172.21.15.112:0/503863886 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18032810 con 0x7f6d28155d80 2024-04-25T01:30:11.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.294+0000 7f6d2ce10640 1 -- 172.21.15.112:0/503863886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d28155d80 msgr2=0x7f6d28156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.294+0000 7f6d2ce10640 1 --2- 172.21.15.112:0/503863886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d28155d80 0x7f6d28156160 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6d180098a0 tx=0x7f6d1802a290 comp rx=0 tx=0).stop 2024-04-25T01:30:11.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.294+0000 7f6d2ce10640 1 -- 172.21.15.112:0/503863886 shutdown_connections 2024-04-25T01:30:11.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.294+0000 7f6d2ce10640 1 --2- 172.21.15.112:0/503863886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d28155d80 0x7f6d28156160 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.294+0000 7f6d2ce10640 1 -- 172.21.15.112:0/503863886 >> 172.21.15.112:0/503863886 conn(0x7f6d280c6f80 msgr2=0x7f6d280c7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:11.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.294+0000 7f6d2ce10640 1 -- 172.21.15.112:0/503863886 shutdown_connections 2024-04-25T01:30:11.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.294+0000 7f6d2ce10640 1 -- 172.21.15.112:0/503863886 wait complete. 2024-04-25T01:30:11.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.295+0000 7f6d2ce10640 1 Processor -- start 2024-04-25T01:30:11.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.295+0000 7f6d2ce10640 1 -- start start 2024-04-25T01:30:11.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.296+0000 7f6d2ce10640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d2806dab0 0x7f6d28145580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.296+0000 7f6d2ce10640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d2814c790 0x7f6d28145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.296+0000 7f6d2ce10640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d28155d80 0x7f6d28153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.296+0000 7f6d2ce10640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d28156820 con 0x7f6d2814c790 2024-04-25T01:30:11.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.296+0000 7f6d2ce10640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281566a0 con 0x7f6d2806dab0 2024-04-25T01:30:11.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.296+0000 7f6d2ce10640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6d2806c910 con 0x7f6d28155d80 2024-04-25T01:30:11.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.296+0000 7f6d25d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d2814c790 0x7f6d28145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.296+0000 7f6d25d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d2814c790 0x7f6d28145ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49492/0 (socket says 172.21.15.112:49492) 2024-04-25T01:30:11.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.296+0000 7f6d25d9b640 1 -- 172.21.15.112:0/264191153 learned_addr learned my addr 172.21.15.112:0/264191153 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:11.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.296+0000 7f6d2659c640 1 --2- 172.21.15.112:0/264191153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d2806dab0 0x7f6d28145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.296+0000 7f6d26d9d640 1 --2- 172.21.15.112:0/264191153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d28155d80 0x7f6d28153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.296+0000 7f6d26d9d640 1 -- 172.21.15.112:0/264191153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d28155d80 msgr2=0x7f6d28153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-25T01:30:11.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.296+0000 7f6d26d9d640 1 -- 172.21.15.112:0/264191153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d28155d80 msgr2=0x7f6d28153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:30:11.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.296+0000 7f6d26d9d640 1 --2- 172.21.15.112:0/264191153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d28155d80 0x7f6d28153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:11.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.296+0000 7f6d26d9d640 1 --2- 172.21.15.112:0/264191153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d28155d80 0x7f6d28153be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:11.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.296+0000 7f6d25d9b640 1 -- 172.21.15.112:0/264191153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d28155d80 msgr2=0x7f6d28153be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:30:11.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.296+0000 7f6d25d9b640 1 --2- 172.21.15.112:0/264191153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d28155d80 0x7f6d28153be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.296+0000 7f6d25d9b640 1 -- 172.21.15.112:0/264191153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d2806dab0 msgr2=0x7f6d28145580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.296+0000 7f6d25d9b640 1 --2- 172.21.15.112:0/264191153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d2806dab0 0x7f6d28145580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.296+0000 7f6d25d9b640 1 -- 172.21.15.112:0/264191153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d28146620 con 0x7f6d2814c790 2024-04-25T01:30:11.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.297+0000 7f6d25d9b640 1 --2- 172.21.15.112:0/264191153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d2814c790 0x7f6d28145ac0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00c400 tx=0x7f6d1c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:11.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.297+0000 7f6d16ffd640 1 -- 172.21.15.112:0/264191153 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c0052d0 con 0x7f6d2814c790 2024-04-25T01:30:11.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.297+0000 7f6d16ffd640 1 -- 172.21.15.112:0/264191153 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d1c005490 con 0x7f6d2814c790 2024-04-25T01:30:11.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.297+0000 7f6d16ffd640 1 -- 172.21.15.112:0/264191153 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c007260 con 0x7f6d2814c790 2024-04-25T01:30:11.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.297+0000 7f6d2ce10640 1 -- 172.21.15.112:0/264191153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d281ed2b0 con 0x7f6d2814c790 2024-04-25T01:30:11.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.297+0000 7f6d2ce10640 1 -- 172.21.15.112:0/264191153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d281ecf40 con 0x7f6d2814c790 2024-04-25T01:30:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.298+0000 7f6d2ce10640 1 -- 172.21.15.112:0/264191153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d280682a0 con 0x7f6d2814c790 2024-04-25T01:30:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.299+0000 7f6d16ffd640 1 -- 172.21.15.112:0/264191153 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d1c007400 con 0x7f6d2814c790 2024-04-25T01:30:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.299+0000 7f6d16ffd640 1 --2- 172.21.15.112:0/264191153 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6d0806beb0 0x7f6d0806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.299+0000 7f6d16ffd640 1 -- 172.21.15.112:0/264191153 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f6d1c08a520 con 0x7f6d2814c790 2024-04-25T01:30:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.299+0000 7f6d2659c640 1 --2- 172.21.15.112:0/264191153 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6d0806beb0 0x7f6d0806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.300+0000 7f6d2659c640 1 --2- 172.21.15.112:0/264191153 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6d0806beb0 0x7f6d0806e370 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6d18034000 tx=0x7f6d1802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:11.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.305+0000 7f6d16ffd640 1 -- 172.21.15.112:0/264191153 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6d28146620 con 0x7f6d2814c790 2024-04-25T01:30:11.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.611+0000 7f6d2ce10640 1 -- 172.21.15.112:0/264191153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6d281498f0 con 0x7f6d2814c790 2024-04-25T01:30:11.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.611+0000 7f6d16ffd640 1 -- 172.21.15.112:0/264191153 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1568 (secure 0 0 0) 0x7f6d1c058650 con 0x7f6d2814c790 2024-04-25T01:30:11.589 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":32,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:30:11.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.613+0000 7f6d2ce10640 1 -- 172.21.15.112:0/264191153 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6d0806beb0 msgr2=0x7f6d0806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.613+0000 7f6d2ce10640 1 --2- 172.21.15.112:0/264191153 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6d0806beb0 0x7f6d0806e370 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6d18034000 tx=0x7f6d1802e000 comp rx=0 tx=0).stop 2024-04-25T01:30:11.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.613+0000 7f6d2ce10640 1 -- 172.21.15.112:0/264191153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d2814c790 msgr2=0x7f6d28145ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.613+0000 7f6d2ce10640 1 --2- 172.21.15.112:0/264191153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d2814c790 0x7f6d28145ac0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00c400 tx=0x7f6d1c00c8c0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.613+0000 7f6d2ce10640 1 -- 172.21.15.112:0/264191153 shutdown_connections 2024-04-25T01:30:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.613+0000 7f6d2ce10640 1 --2- 172.21.15.112:0/264191153 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6d0806beb0 0x7f6d0806e370 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.613+0000 7f6d2ce10640 1 --2- 172.21.15.112:0/264191153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d28155d80 0x7f6d28153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.613+0000 7f6d2ce10640 1 --2- 172.21.15.112:0/264191153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d2814c790 0x7f6d28145ac0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.613+0000 7f6d2ce10640 1 --2- 172.21.15.112:0/264191153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d2806dab0 0x7f6d28145580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.613+0000 7f6d2ce10640 1 -- 172.21.15.112:0/264191153 >> 172.21.15.112:0/264191153 conn(0x7f6d280c6f80 msgr2=0x7f6d281486e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.613+0000 7f6d2ce10640 1 -- 172.21.15.112:0/264191153 shutdown_connections 2024-04-25T01:30:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.613+0000 7f6d2ce10640 1 -- 172.21.15.112:0/264191153 wait complete. 2024-04-25T01:30:11.605 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:30:11.605 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:30:11.606 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:30:11.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.729+0000 7f6a6eeb3640 1 Processor -- start 2024-04-25T01:30:11.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.729+0000 7f6a6eeb3640 1 -- start start 2024-04-25T01:30:11.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.729+0000 7f6a6eeb3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6a681592c0 0x7f6a680c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.729+0000 7f6a6eeb3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a681596a0 0x7f6a68152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.729+0000 7f6a6eeb3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a68159b90 con 0x7f6a680c72b0 2024-04-25T01:30:11.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.729+0000 7f6a6eeb3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f6a680c6e20 con 0x7f6a681592c0 2024-04-25T01:30:11.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.729+0000 7f6a6eeb3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6a6806c970 con 0x7f6a681596a0 2024-04-25T01:30:11.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.730+0000 7f6a6cc4f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6a681592c0 0x7f6a680c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.730+0000 7f6a6d450640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6a680c72b0 0x7f6a68155ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:44326/0 (socket says 172.21.15.112:44326) 2024-04-25T01:30:11.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.730+0000 7f6a6d450640 1 -- 172.21.15.112:0/436877979 learned_addr learned my addr 172.21.15.112:0/436877979 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:30:11.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.730+0000 7f6a5ffff640 1 --2- 172.21.15.112:0/436877979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a681596a0 0x7f6a68152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.730+0000 7f6a5ffff640 1 -- 172.21.15.112:0/436877979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a681596a0 msgr2=0x7f6a68152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:30:11.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.730+0000 7f6a5ffff640 1 -- 172.21.15.112:0/436877979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a681596a0 msgr2=0x7f6a68152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:30:11.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.730+0000 7f6a5ffff640 1 --2- 172.21.15.112:0/436877979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a681596a0 0x7f6a68152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:11.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.730+0000 7f6a6cc4f640 1 -- 172.21.15.112:0/436877979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a681596a0 msgr2=0x7f6a68152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:30:11.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.730+0000 7f6a5ffff640 1 --2- 172.21.15.112:0/436877979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a681596a0 0x7f6a68152d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:11.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.730+0000 7f6a6cc4f640 1 --2- 172.21.15.112:0/436877979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a681596a0 0x7f6a68152d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.730+0000 7f6a6cc4f640 1 -- 172.21.15.112:0/436877979 >> v1:172.21.15.112:6789/0 conn(0x7f6a680c72b0 legacy=0x7f6a68155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.730+0000 7f6a6cc4f640 1 -- 172.21.15.112:0/436877979 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a6806cc50 con 0x7f6a681592c0 2024-04-25T01:30:11.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.730+0000 7f6a6cc4f640 1 --2- 172.21.15.112:0/436877979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a681592c0 0x7f6a680c9070 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6a640097a0 tx=0x7f6a6402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d43fb6941b61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:11.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.731+0000 7f6a5f7fe640 1 -- 172.21.15.112:0/436877979 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a64037070 con 0x7f6a681592c0 2024-04-25T01:30:11.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.731+0000 7f6a5f7fe640 1 -- 172.21.15.112:0/436877979 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a6402adb0 con 0x7f6a681592c0 2024-04-25T01:30:11.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.731+0000 7f6a5f7fe640 1 -- 172.21.15.112:0/436877979 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a64032660 con 0x7f6a681592c0 2024-04-25T01:30:11.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.731+0000 7f6a6eeb3640 1 -- 172.21.15.112:0/436877979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a681592c0 msgr2=0x7f6a680c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.731+0000 7f6a6eeb3640 1 --2- 172.21.15.112:0/436877979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a681592c0 0x7f6a680c9070 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6a640097a0 tx=0x7f6a6402a060 comp rx=0 tx=0).stop 2024-04-25T01:30:11.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.731+0000 7f6a6eeb3640 1 -- 172.21.15.112:0/436877979 shutdown_connections 2024-04-25T01:30:11.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.731+0000 7f6a6eeb3640 1 --2- 172.21.15.112:0/436877979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a681596a0 0x7f6a68152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.731+0000 7f6a6eeb3640 1 --2- 172.21.15.112:0/436877979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a681592c0 0x7f6a680c9070 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.731+0000 7f6a6eeb3640 1 -- 172.21.15.112:0/436877979 >> 172.21.15.112:0/436877979 conn(0x7f6a680cb170 msgr2=0x7f6a680cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:11.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.731+0000 7f6a6eeb3640 1 -- 172.21.15.112:0/436877979 shutdown_connections 2024-04-25T01:30:11.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.731+0000 7f6a6eeb3640 1 -- 172.21.15.112:0/436877979 wait complete. 2024-04-25T01:30:11.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.733+0000 7f6a6eeb3640 1 Processor -- start 2024-04-25T01:30:11.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.733+0000 7f6a6eeb3640 1 -- start start 2024-04-25T01:30:11.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.734+0000 7f6a6eeb3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a680c72b0 0x7f6a6814bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.734+0000 7f6a6eeb3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a681592c0 0x7f6a681ef4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.734+0000 7f6a6eeb3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a681596a0 0x7f6a681efa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.734+0000 7f6a6eeb3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6806dca0 con 0x7f6a681592c0 2024-04-25T01:30:11.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.734+0000 7f6a6eeb3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6806db20 con 0x7f6a680c72b0 2024-04-25T01:30:11.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.734+0000 7f6a6eeb3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6a680c95b0 con 0x7f6a681596a0 2024-04-25T01:30:11.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.734+0000 7f6a5ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a681592c0 0x7f6a681ef4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.734+0000 7f6a5ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a681592c0 0x7f6a681ef4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49520/0 (socket says 172.21.15.112:49520) 2024-04-25T01:30:11.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.734+0000 7f6a5ffff640 1 -- 172.21.15.112:0/2800712430 learned_addr learned my addr 172.21.15.112:0/2800712430 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:11.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.734+0000 7f6a6cc4f640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a680c72b0 0x7f6a6814bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.734+0000 7f6a6d450640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a681596a0 0x7f6a681efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.734+0000 7f6a6d450640 1 -- 172.21.15.112:0/2800712430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a681596a0 msgr2=0x7f6a681efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-25T01:30:11.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.734+0000 7f6a6d450640 1 -- 172.21.15.112:0/2800712430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a681596a0 msgr2=0x7f6a681efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:30:11.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.734+0000 7f6a6d450640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a681596a0 0x7f6a681efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:11.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.734+0000 7f6a6d450640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a681596a0 0x7f6a681efa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:11.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.734+0000 7f6a5ffff640 1 -- 172.21.15.112:0/2800712430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a681596a0 msgr2=0x7f6a681efa10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:30:11.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.734+0000 7f6a5ffff640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a681596a0 0x7f6a681efa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.734+0000 7f6a5ffff640 1 -- 172.21.15.112:0/2800712430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a680c72b0 msgr2=0x7f6a6814bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.734+0000 7f6a5ffff640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a680c72b0 0x7f6a6814bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.734+0000 7f6a5ffff640 1 -- 172.21.15.112:0/2800712430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a6814a530 con 0x7f6a681592c0 2024-04-25T01:30:11.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.735+0000 7f6a5ffff640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a681592c0 0x7f6a681ef4d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800d800 tx=0x7f6a5800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:11.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.735+0000 7f6a5d7fa640 1 -- 172.21.15.112:0/2800712430 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5800b660 con 0x7f6a681592c0 2024-04-25T01:30:11.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.735+0000 7f6a5d7fa640 1 -- 172.21.15.112:0/2800712430 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a58019070 con 0x7f6a681592c0 2024-04-25T01:30:11.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.735+0000 7f6a6eeb3640 1 -- 172.21.15.112:0/2800712430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a68147120 con 0x7f6a681592c0 2024-04-25T01:30:11.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.735+0000 7f6a5d7fa640 1 -- 172.21.15.112:0/2800712430 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5800ba60 con 0x7f6a681592c0 2024-04-25T01:30:11.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.735+0000 7f6a6eeb3640 1 -- 172.21.15.112:0/2800712430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a681476e0 con 0x7f6a681592c0 2024-04-25T01:30:11.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.736+0000 7f6a6eeb3640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a681eb9a0 0x7f6a681463c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.736+0000 7f6a6eeb3640 1 -- 172.21.15.112:0/2800712430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a680679a0 con 0x7f6a681eb9a0 2024-04-25T01:30:11.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.736+0000 7f6a6cc4f640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a681eb9a0 0x7f6a681463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.736+0000 7f6a6cc4f640 1 -- 172.21.15.112:0/2800712430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6a680682a0 con 0x7f6a681eb9a0 2024-04-25T01:30:11.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.736+0000 7f6a6cc4f640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a681eb9a0 0x7f6a681463c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6a680aac40 tx=0x7f6a640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:11.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.736+0000 7f6a5d7fa640 1 -- 172.21.15.112:0/2800712430 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a64037040 con 0x7f6a681eb9a0 2024-04-25T01:30:11.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.737+0000 7f6a5d7fa640 1 -- 172.21.15.112:0/2800712430 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6a5800bc00 con 0x7f6a681592c0 2024-04-25T01:30:11.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.737+0000 7f6a5d7fa640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6a5406bd30 0x7f6a5406e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.737+0000 7f6a5d7fa640 1 -- 172.21.15.112:0/2800712430 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f6a5808b490 con 0x7f6a681592c0 2024-04-25T01:30:11.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.737+0000 7f6a6d450640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6a5406bd30 0x7f6a5406e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.737+0000 7f6a5d7fa640 1 -- 172.21.15.112:0/2800712430 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6a64032dd0 con 0x7f6a681eb9a0 2024-04-25T01:30:11.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.737+0000 7f6a5d7fa640 1 -- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a681eb9a0 msgr2=0x7f6a681463c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.737+0000 7f6a5d7fa640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a681eb9a0 0x7f6a681463c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6a680aac40 tx=0x7f6a640046c0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.737+0000 7f6a5d7fa640 1 -- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a681eb9a0 msgr2=0x7f6a681463c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:30:11.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.737+0000 7f6a5d7fa640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a681eb9a0 0x7f6a681463c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.738+0000 7f6a6d450640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6a5406bd30 0x7f6a5406e1f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6a600012e0 tx=0x7f6a60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:11.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.746+0000 7f6a6eeb3640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a68146900 0x7f6a6814a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.746+0000 7f6a6eeb3640 1 -- 172.21.15.112:0/2800712430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a68146ce0 con 0x7f6a68146900 2024-04-25T01:30:11.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.746+0000 7f6a6cc4f640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a68146900 0x7f6a6814a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.746+0000 7f6a6cc4f640 1 -- 172.21.15.112:0/2800712430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6a680c79d0 con 0x7f6a68146900 2024-04-25T01:30:11.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.747+0000 7f6a6cc4f640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a68146900 0x7f6a6814a750 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6a640097a0 tx=0x7f6a64008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:11.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.747+0000 7f6a5d7fa640 1 -- 172.21.15.112:0/2800712430 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a64046640 con 0x7f6a68146900 2024-04-25T01:30:11.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.747+0000 7f6a5d7fa640 1 -- 172.21.15.112:0/2800712430 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5071 (secure 0 0 0) 0x7f6a64046640 con 0x7f6a68146900 2024-04-25T01:30:11.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.747+0000 7f6a5d7fa640 1 -- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a68146900 msgr2=0x7f6a6814a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.747+0000 7f6a5d7fa640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a68146900 0x7f6a6814a750 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6a640097a0 tx=0x7f6a64008030 comp rx=0 tx=0).stop 2024-04-25T01:30:11.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.747+0000 7f6a5d7fa640 1 -- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a68146900 msgr2=0x7f6a6814a750 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:30:11.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.747+0000 7f6a5d7fa640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a68146900 0x7f6a6814a750 secure :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6a640097a0 tx=0x7f6a64008030 comp rx=0 tx=0).stop 2024-04-25T01:30:11.726 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:30:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:30:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:30:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:30:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 32, 2024-04-25T01:30:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 283375, 2024-04-25T01:30:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:30:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:30:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:30:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:30:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:30:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:30:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:30:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:30:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:30:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:30:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:30:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:30:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:30:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:30:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:30:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:30:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:30:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:30:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:30:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:30:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:30:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:30:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:30:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:30:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:30:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:30:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:30:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:30:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:30:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:30:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:30:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:30:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 7 2024-04-25T01:30:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:30:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 8 2024-04-25T01:30:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:30:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:30:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:30:11.738 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:30:11.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.747+0000 7f6a6eeb3640 1 -- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6a5406bd30 msgr2=0x7f6a5406e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.747+0000 7f6a6eeb3640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6a5406bd30 0x7f6a5406e1f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6a600012e0 tx=0x7f6a60009040 comp rx=0 tx=0).stop 2024-04-25T01:30:11.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.747+0000 7f6a6eeb3640 1 -- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a681592c0 msgr2=0x7f6a681ef4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.747+0000 7f6a6eeb3640 1 --2- 172.21.15.112:0/2800712430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a681592c0 0x7f6a681ef4d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800d800 tx=0x7f6a5800dcc0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.747+0000 7f6a6d450640 1 -- 172.21.15.112:0/2800712430 reap_dead start 2024-04-25T01:30:11.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.747+0000 7f6a6d450640 1 -- 172.21.15.112:0/2800712430 reap_dead start 2024-04-25T01:30:11.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.748+0000 7f6a6eeb3640 1 -- 172.21.15.112:0/2800712430 shutdown_connections 2024-04-25T01:30:11.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.748+0000 7f6a6eeb3640 1 -- 172.21.15.112:0/2800712430 >> 172.21.15.112:0/2800712430 conn(0x7f6a680cb170 msgr2=0x7f6a68152ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:11.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.748+0000 7f6a6eeb3640 1 -- 172.21.15.112:0/2800712430 shutdown_connections 2024-04-25T01:30:11.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.748+0000 7f6a6eeb3640 1 -- 172.21.15.112:0/2800712430 wait complete. 2024-04-25T01:30:11.740 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:30:11.740 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:30:11.741 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:30:11.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.868+0000 7f6fa709a640 1 Processor -- start 2024-04-25T01:30:11.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.868+0000 7f6fa709a640 1 -- start start 2024-04-25T01:30:11.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6fa709a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa01592c0 0x7f6fa00c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6fa709a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6fa01596a0 0x7f6fa0152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6fa709a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6fa00c72b0 0x7f6fa0155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6fa709a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f6fa00c6e20 con 0x7f6fa00c72b0 2024-04-25T01:30:11.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6fa709a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6fa0159b90 con 0x7f6fa01596a0 2024-04-25T01:30:11.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6fa709a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa006c970 con 0x7f6fa01592c0 2024-04-25T01:30:11.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6fa5637640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6fa00c72b0 0x7f6fa0155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6fa5637640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6fa00c72b0 0x7f6fa0155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49556/0 (socket says 172.21.15.112:49556) 2024-04-25T01:30:11.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6fa5637640 1 -- 172.21.15.112:0/1067639929 learned_addr learned my addr 172.21.15.112:0/1067639929 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:11.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6fa4e36640 1 --2- 172.21.15.112:0/1067639929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa01592c0 0x7f6fa00c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6f9ffff640 1 --2- 172.21.15.112:0/1067639929 >> v2:172.21.15.187:3300/0 conn(0x7f6fa01596a0 0x7f6fa0152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6fa4e36640 1 -- 172.21.15.112:0/1067639929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa01592c0 msgr2=0x7f6fa00c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:30:11.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6fa4e36640 1 -- 172.21.15.112:0/1067639929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa01592c0 msgr2=0x7f6fa00c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:30:11.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6fa4e36640 1 --2- 172.21.15.112:0/1067639929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa01592c0 0x7f6fa00c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:11.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6fa5637640 1 -- 172.21.15.112:0/1067639929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa01592c0 msgr2=0x7f6fa00c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:30:11.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6fa4e36640 1 --2- 172.21.15.112:0/1067639929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa01592c0 0x7f6fa00c9070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:11.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6fa5637640 1 --2- 172.21.15.112:0/1067639929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa01592c0 0x7f6fa00c9070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6fa5637640 1 -- 172.21.15.112:0/1067639929 >> v2:172.21.15.187:3300/0 conn(0x7f6fa01596a0 msgr2=0x7f6fa0152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:30:11.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6fa5637640 1 --2- 172.21.15.112:0/1067639929 >> v2:172.21.15.187:3300/0 conn(0x7f6fa01596a0 0x7f6fa0152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6fa5637640 1 -- 172.21.15.112:0/1067639929 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa006cc50 con 0x7f6fa00c72b0 2024-04-25T01:30:11.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.869+0000 7f6f9ffff640 1 --2- 172.21.15.112:0/1067639929 >> v2:172.21.15.187:3300/0 conn(0x7f6fa01596a0 0x7f6fa0152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:30:11.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.870+0000 7f6fa5637640 1 --2- 172.21.15.112:0/1067639929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa00c72b0 0x7f6fa0155ae0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00753b0 tx=0x7f6f9802a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0a90ab4d15f384 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:11.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.870+0000 7f6f9f7fe640 1 -- 172.21.15.112:0/1067639929 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98036070 con 0x7f6fa00c72b0 2024-04-25T01:30:11.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.870+0000 7f6f9f7fe640 1 -- 172.21.15.112:0/1067639929 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f98031600 con 0x7f6fa00c72b0 2024-04-25T01:30:11.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.870+0000 7f6f9f7fe640 1 -- 172.21.15.112:0/1067639929 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f980319d0 con 0x7f6fa00c72b0 2024-04-25T01:30:11.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.870+0000 7f6fa709a640 1 -- 172.21.15.112:0/1067639929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa00c72b0 msgr2=0x7f6fa0155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.870+0000 7f6fa709a640 1 --2- 172.21.15.112:0/1067639929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa00c72b0 0x7f6fa0155ae0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00753b0 tx=0x7f6f9802a570 comp rx=0 tx=0).stop 2024-04-25T01:30:11.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.871+0000 7f6fa709a640 1 -- 172.21.15.112:0/1067639929 shutdown_connections 2024-04-25T01:30:11.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.871+0000 7f6fa709a640 1 --2- 172.21.15.112:0/1067639929 >> v2:172.21.15.187:3300/0 conn(0x7f6fa01596a0 0x7f6fa0152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.871+0000 7f6fa709a640 1 --2- 172.21.15.112:0/1067639929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa00c72b0 0x7f6fa0155ae0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.871+0000 7f6fa709a640 1 --2- 172.21.15.112:0/1067639929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa01592c0 0x7f6fa00c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.871+0000 7f6fa709a640 1 -- 172.21.15.112:0/1067639929 >> 172.21.15.112:0/1067639929 conn(0x7f6fa00cb170 msgr2=0x7f6fa00cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:11.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.871+0000 7f6fa709a640 1 -- 172.21.15.112:0/1067639929 shutdown_connections 2024-04-25T01:30:11.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.871+0000 7f6fa709a640 1 -- 172.21.15.112:0/1067639929 wait complete. 2024-04-25T01:30:11.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.872+0000 7f6fa709a640 1 Processor -- start 2024-04-25T01:30:11.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.872+0000 7f6fa709a640 1 -- start start 2024-04-25T01:30:11.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.873+0000 7f6fa709a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa00c72b0 0x7f6fa01eef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.873+0000 7f6fa709a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa01592c0 0x7f6fa01ef4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.873+0000 7f6fa709a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa01596a0 0x7f6fa0149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.873+0000 7f6fa709a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa006dca0 con 0x7f6fa00c72b0 2024-04-25T01:30:11.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.873+0000 7f6fa709a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa006db20 con 0x7f6fa01592c0 2024-04-25T01:30:11.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.873+0000 7f6fa709a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa00c95b0 con 0x7f6fa01596a0 2024-04-25T01:30:11.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.873+0000 7f6fa4e36640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa00c72b0 0x7f6fa01eef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.873+0000 7f6fa4e36640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa00c72b0 0x7f6fa01eef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49572/0 (socket says 172.21.15.112:49572) 2024-04-25T01:30:11.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.873+0000 7f6fa4e36640 1 -- 172.21.15.112:0/62527768 learned_addr learned my addr 172.21.15.112:0/62527768 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:11.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.873+0000 7f6f9ffff640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa01592c0 0x7f6fa01ef4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.873+0000 7f6fa5637640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa01596a0 0x7f6fa0149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.873+0000 7f6fa5637640 1 -- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa01596a0 msgr2=0x7f6fa0149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-25T01:30:11.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.873+0000 7f6fa5637640 1 -- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa01596a0 msgr2=0x7f6fa0149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:30:11.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.873+0000 7f6fa5637640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa01596a0 0x7f6fa0149720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:11.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.873+0000 7f6fa5637640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa01596a0 0x7f6fa0149720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:11.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.874+0000 7f6fa4e36640 1 -- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa01596a0 msgr2=0x7f6fa0149720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:30:11.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.874+0000 7f6fa4e36640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa01596a0 0x7f6fa0149720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.874+0000 7f6fa4e36640 1 -- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa01592c0 msgr2=0x7f6fa01ef4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.874+0000 7f6fa4e36640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa01592c0 0x7f6fa01ef4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.874+0000 7f6fa4e36640 1 -- 172.21.15.112:0/62527768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa0155360 con 0x7f6fa00c72b0 2024-04-25T01:30:11.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.874+0000 7f6fa4e36640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa00c72b0 0x7f6fa01eef80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400d7d0 tx=0x7f6f9400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:11.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.874+0000 7f6f9d7fa640 1 -- 172.21.15.112:0/62527768 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9400b6c0 con 0x7f6fa00c72b0 2024-04-25T01:30:11.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.874+0000 7f6f9d7fa640 1 -- 172.21.15.112:0/62527768 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f94019070 con 0x7f6fa00c72b0 2024-04-25T01:30:11.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.874+0000 7f6fa709a640 1 -- 172.21.15.112:0/62527768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa01555e0 con 0x7f6fa00c72b0 2024-04-25T01:30:11.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.874+0000 7f6f9d7fa640 1 -- 172.21.15.112:0/62527768 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9400bac0 con 0x7f6fa00c72b0 2024-04-25T01:30:11.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.874+0000 7f6fa709a640 1 -- 172.21.15.112:0/62527768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa0155ba0 con 0x7f6fa00c72b0 2024-04-25T01:30:11.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.875+0000 7f6fa709a640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa00bf540 0x7f6fa0149c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.875+0000 7f6fa709a640 1 -- 172.21.15.112:0/62527768 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa0075980 con 0x7f6fa00bf540 2024-04-25T01:30:11.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.875+0000 7f6f9ffff640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa00bf540 0x7f6fa0149c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.875+0000 7f6f9ffff640 1 -- 172.21.15.112:0/62527768 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6fa00679a0 con 0x7f6fa00bf540 2024-04-25T01:30:11.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.876+0000 7f6f9ffff640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa00bf540 0x7f6fa0149c60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c006e80 tx=0x7f6f8c010c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:11.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.876+0000 7f6f9d7fa640 1 -- 172.21.15.112:0/62527768 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c018020 con 0x7f6fa00bf540 2024-04-25T01:30:11.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.876+0000 7f6f9d7fa640 1 -- 172.21.15.112:0/62527768 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f9400bc60 con 0x7f6fa00c72b0 2024-04-25T01:30:11.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.876+0000 7f6f9d7fa640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6f8406bd30 0x7f6f8406e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.876+0000 7f6fa5637640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6f8406bd30 0x7f6f8406e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.876+0000 7f6f9d7fa640 1 -- 172.21.15.112:0/62527768 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f6f9408b650 con 0x7f6fa00c72b0 2024-04-25T01:30:11.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.877+0000 7f6f9d7fa640 1 -- 172.21.15.112:0/62527768 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6f8c010e70 con 0x7f6fa00bf540 2024-04-25T01:30:11.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.877+0000 7f6f9d7fa640 1 -- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa00bf540 msgr2=0x7f6fa0149c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.877+0000 7f6f9d7fa640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa00bf540 0x7f6fa0149c60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c006e80 tx=0x7f6f8c010c80 comp rx=0 tx=0).stop 2024-04-25T01:30:11.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.877+0000 7f6f9d7fa640 1 -- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa00bf540 msgr2=0x7f6fa0149c60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:30:11.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.877+0000 7f6f9d7fa640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa00bf540 0x7f6fa0149c60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.877+0000 7f6fa5637640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6f8406bd30 0x7f6f8406e1f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6f98033000 tx=0x7f6f98009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:11.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.883+0000 7f6fa709a640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa01ecc10 0x7f6fa01ed020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.883+0000 7f6fa709a640 1 -- 172.21.15.112:0/62527768 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa00aaf10 con 0x7f6fa01ecc10 2024-04-25T01:30:11.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.884+0000 7f6f9ffff640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa01ecc10 0x7f6fa01ed020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.884+0000 7f6f9ffff640 1 -- 172.21.15.112:0/62527768 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6fa00682a0 con 0x7f6fa01ecc10 2024-04-25T01:30:11.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.884+0000 7f6f9ffff640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa01ecc10 0x7f6fa01ed020 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c019000 tx=0x7f6f8c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:11.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.884+0000 7f6f9d7fa640 1 -- 172.21.15.112:0/62527768 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c00eac0 con 0x7f6fa01ecc10 2024-04-25T01:30:11.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.884+0000 7f6f9d7fa640 1 -- 172.21.15.112:0/62527768 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4728 (secure 0 0 0) 0x7f6f8c00eac0 con 0x7f6fa01ecc10 2024-04-25T01:30:11.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.884+0000 7f6f9d7fa640 1 -- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa01ecc10 msgr2=0x7f6fa01ed020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.884+0000 7f6f9d7fa640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa01ecc10 0x7f6fa01ed020 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c019000 tx=0x7f6f8c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.884+0000 7f6f9d7fa640 1 -- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa01ecc10 msgr2=0x7f6fa01ed020 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:30:11.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.884+0000 7f6f9d7fa640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa01ecc10 0x7f6fa01ed020 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c019000 tx=0x7f6f8c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.863 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:30:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:30:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:30:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:30:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 32, 2024-04-25T01:30:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 47573, 2024-04-25T01:30:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:30:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:30:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:30:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:30:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:30:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:30:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:30:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:30:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:30:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:30:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:30:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:30:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:30:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:30:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:30:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:30:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:30:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:30:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:30:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:30:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:30:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:30:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:30:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:30:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:30:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:30:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:30:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:30:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:30:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:30:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:30:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:30:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:30:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:30:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:30:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:30:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:30:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:30:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:30:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:30:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:30:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:30:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:30:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:30:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:30:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:30:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:30:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:30:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:30:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:30:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:30:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:30:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:30:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:30:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:30:11.876 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:30:11.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.885+0000 7f6fa709a640 1 -- 172.21.15.112:0/62527768 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6f8406bd30 msgr2=0x7f6f8406e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.885+0000 7f6fa709a640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6f8406bd30 0x7f6f8406e1f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6f98033000 tx=0x7f6f98009000 comp rx=0 tx=0).stop 2024-04-25T01:30:11.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.885+0000 7f6fa709a640 1 -- 172.21.15.112:0/62527768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa00c72b0 msgr2=0x7f6fa01eef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.885+0000 7f6fa709a640 1 --2- 172.21.15.112:0/62527768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa00c72b0 0x7f6fa01eef80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400d7d0 tx=0x7f6f9400dc90 comp rx=0 tx=0).stop 2024-04-25T01:30:11.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.885+0000 7f6fa5637640 1 -- 172.21.15.112:0/62527768 reap_dead start 2024-04-25T01:30:11.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.885+0000 7f6fa5637640 1 -- 172.21.15.112:0/62527768 reap_dead start 2024-04-25T01:30:11.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.885+0000 7f6fa709a640 1 -- 172.21.15.112:0/62527768 shutdown_connections 2024-04-25T01:30:11.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.885+0000 7f6fa709a640 1 -- 172.21.15.112:0/62527768 >> 172.21.15.112:0/62527768 conn(0x7f6fa00cb170 msgr2=0x7f6fa00ac190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:11.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.885+0000 7f6fa709a640 1 -- 172.21.15.112:0/62527768 shutdown_connections 2024-04-25T01:30:11.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:11.885+0000 7f6fa709a640 1 -- 172.21.15.112:0/62527768 wait complete. 2024-04-25T01:30:11.879 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:30:11.879 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:30:11.879 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:30:11.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.008+0000 7f0e85b22640 1 Processor -- start 2024-04-25T01:30:11.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.008+0000 7f0e85b22640 1 -- start start 2024-04-25T01:30:11.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e85b22640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e800c3900 0x7f0e80154470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e85b22640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e80071850 con 0x7f0e80071000 2024-04-25T01:30:11.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e85b22640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e80071570 con 0x7f0e80150d80 2024-04-25T01:30:11.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e85b22640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0e80153990 con 0x7f0e800c3900 2024-04-25T01:30:11.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7effd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0e80071000 0x7f0e800c84a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:44342/0 (socket says 172.21.15.112:44342) 2024-04-25T01:30:11.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7effd640 1 -- 172.21.15.112:0/3256778305 learned_addr learned my addr 172.21.15.112:0/3256778305 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:30:11.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7ffff640 1 --2- 172.21.15.112:0/3256778305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e800c3900 0x7f0e80154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7ffff640 1 -- 172.21.15.112:0/3256778305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e800c3900 msgr2=0x7f0e80154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:30:11.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7ffff640 1 -- 172.21.15.112:0/3256778305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e800c3900 msgr2=0x7f0e80154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:30:11.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7ffff640 1 --2- 172.21.15.112:0/3256778305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e800c3900 0x7f0e80154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:11.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7ffff640 1 --2- 172.21.15.112:0/3256778305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e800c3900 0x7f0e80154470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:11.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7e7fc640 1 -- 172.21.15.112:0/3256778305 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1164771130 0 0) 0x7f0e80071850 con 0x7f0e80071000 2024-04-25T01:30:11.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7e7fc640 1 -- 172.21.15.112:0/3256778305 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e68003520 con 0x7f0e80071000 2024-04-25T01:30:11.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7e7fc640 1 -- 172.21.15.112:0/3256778305 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3247974270 0 0) 0x7f0e80071570 con 0x7f0e80150d80 2024-04-25T01:30:11.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7e7fc640 1 -- 172.21.15.112:0/3256778305 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e80071850 con 0x7f0e80150d80 2024-04-25T01:30:11.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7e7fc640 1 -- 172.21.15.112:0/3256778305 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2458599039 0 0) 0x7f0e68003520 con 0x7f0e80071000 2024-04-25T01:30:11.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7e7fc640 1 -- 172.21.15.112:0/3256778305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e800c3900 msgr2=0x7f0e80154470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:30:11.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7e7fc640 1 --2- 172.21.15.112:0/3256778305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e800c3900 0x7f0e80154470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7e7fc640 1 -- 172.21.15.112:0/3256778305 >> v1:172.21.15.187:6789/0 conn(0x7f0e80150d80 legacy=0x7f0e800c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7e7fc640 1 -- 172.21.15.112:0/3256778305 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e80153c70 con 0x7f0e80071000 2024-04-25T01:30:11.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7e7fc640 1 -- 172.21.15.112:0/3256778305 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f0e74001670 con 0x7f0e80071000 2024-04-25T01:30:11.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7e7fc640 1 -- 172.21.15.112:0/3256778305 >> v1:172.21.15.112:6789/0 conn(0x7f0e80071000 legacy=0x7f0e800c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7e7fc640 1 --2- 172.21.15.112:0/3256778305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e800c3900 0x7f0e68003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7e7fc640 1 --2- 172.21.15.112:0/3256778305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e680040d0 0x7f0e68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7e7fc640 1 --2- 172.21.15.112:0/3256778305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e68004d40 0x7f0e680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7e7fc640 1 -- 172.21.15.112:0/3256778305 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0e80153990 con 0x7f0e68004d40 2024-04-25T01:30:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7e7fc640 1 -- 172.21.15.112:0/3256778305 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6800ffe0 con 0x7f0e680040d0 2024-04-25T01:30:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.009+0000 7f0e7e7fc640 1 -- 172.21.15.112:0/3256778305 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0e680101e0 con 0x7f0e800c3900 2024-04-25T01:30:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.010+0000 7f0e7ffff640 1 --2- 172.21.15.112:0/3256778305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e68004d40 0x7f0e680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.010+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/3256778305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e800c3900 0x7f0e68003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.010+0000 7f0e7effd640 1 --2- 172.21.15.112:0/3256778305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e680040d0 0x7f0e68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.010+0000 7f0e7ffff640 1 -- 172.21.15.112:0/3256778305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e800c3900 msgr2=0x7f0e68003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:30:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.010+0000 7f0e7ffff640 1 --2- 172.21.15.112:0/3256778305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e800c3900 0x7f0e68003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.010+0000 7f0e7ffff640 1 -- 172.21.15.112:0/3256778305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e680040d0 msgr2=0x7f0e68004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:30:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.010+0000 7f0e7ffff640 1 --2- 172.21.15.112:0/3256778305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e680040d0 0x7f0e68004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.010+0000 7f0e7ffff640 1 -- 172.21.15.112:0/3256778305 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e68003520 con 0x7f0e68004d40 2024-04-25T01:30:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.011+0000 7f0e7ffff640 1 --2- 172.21.15.112:0/3256778305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e68004d40 0x7f0e680051c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009fd0 tx=0x7f0e68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf129bda6146564 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.011+0000 7f0e7e7fc640 1 -- 172.21.15.112:0/3256778305 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78033070 con 0x7f0e68004d40 2024-04-25T01:30:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.011+0000 7f0e7e7fc640 1 -- 172.21.15.112:0/3256778305 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e7802f650 con 0x7f0e68004d40 2024-04-25T01:30:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.011+0000 7f0e7e7fc640 1 -- 172.21.15.112:0/3256778305 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78009bd0 con 0x7f0e68004d40 2024-04-25T01:30:11.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.012+0000 7f0e85b22640 1 -- 172.21.15.112:0/3256778305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e68004d40 msgr2=0x7f0e680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.012+0000 7f0e85b22640 1 --2- 172.21.15.112:0/3256778305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e68004d40 0x7f0e680051c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009fd0 tx=0x7f0e68005830 comp rx=0 tx=0).stop 2024-04-25T01:30:11.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.012+0000 7f0e7ffff640 1 -- 172.21.15.112:0/3256778305 reap_dead start 2024-04-25T01:30:11.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.012+0000 7f0e85b22640 1 -- 172.21.15.112:0/3256778305 shutdown_connections 2024-04-25T01:30:11.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.012+0000 7f0e85b22640 1 -- 172.21.15.112:0/3256778305 >> 172.21.15.112:0/3256778305 conn(0x7f0e80074a90 msgr2=0x7f0e800c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:11.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.012+0000 7f0e85b22640 1 -- 172.21.15.112:0/3256778305 shutdown_connections 2024-04-25T01:30:11.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.012+0000 7f0e85b22640 1 -- 172.21.15.112:0/3256778305 wait complete. 2024-04-25T01:30:11.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.013+0000 7f0e85b22640 1 Processor -- start 2024-04-25T01:30:11.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.013+0000 7f0e85b22640 1 -- start start 2024-04-25T01:30:11.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.014+0000 7f0e85b22640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e80143d00 0x7f0e8013ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.014+0000 7f0e85b22640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e800ab7f0 0x7f0e80140520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.014+0000 7f0e85b22640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e800ab2a0 0x7f0e8006fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.014+0000 7f0e85b22640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0e800c8bb0 con 0x7f0e800ab7f0 2024-04-25T01:30:11.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.014+0000 7f0e85b22640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0e800c7590 con 0x7f0e80143d00 2024-04-25T01:30:11.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.014+0000 7f0e85b22640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0e800c8d30 con 0x7f0e800ab2a0 2024-04-25T01:30:11.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.014+0000 7f0e7effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e800ab7f0 0x7f0e80140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.014+0000 7f0e7effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e800ab7f0 0x7f0e80140520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49602/0 (socket says 172.21.15.112:49602) 2024-04-25T01:30:11.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.014+0000 7f0e7effd640 1 -- 172.21.15.112:0/2053227476 learned_addr learned my addr 172.21.15.112:0/2053227476 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:11.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.014+0000 7f0e7ffff640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e800ab2a0 0x7f0e8006fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.014+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e80143d00 0x7f0e8013ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.014+0000 7f0e7ffff640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e800ab2a0 msgr2=0x7f0e8006fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-25T01:30:11.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.014+0000 7f0e7ffff640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e800ab2a0 msgr2=0x7f0e8006fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:30:11.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.014+0000 7f0e7ffff640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e800ab2a0 0x7f0e8006fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:11.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.014+0000 7f0e7ffff640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e800ab2a0 0x7f0e8006fbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:11.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.014+0000 7f0e7effd640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e800ab2a0 msgr2=0x7f0e8006fbe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:30:11.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.014+0000 7f0e7effd640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e800ab2a0 0x7f0e8006fbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.014+0000 7f0e7effd640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e80143d00 msgr2=0x7f0e8013ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:11.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.014+0000 7f0e7effd640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e80143d00 0x7f0e8013ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:11.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.014+0000 7f0e7effd640 1 -- 172.21.15.112:0/2053227476 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e801e81d0 con 0x7f0e800ab7f0 2024-04-25T01:30:11.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.015+0000 7f0e7effd640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e800ab7f0 0x7f0e80140520 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0e74007340 tx=0x7f0e74007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:11.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.015+0000 7f0e67fff640 1 -- 172.21.15.112:0/2053227476 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e74001610 con 0x7f0e800ab7f0 2024-04-25T01:30:11.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.015+0000 7f0e67fff640 1 -- 172.21.15.112:0/2053227476 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e74018070 con 0x7f0e800ab7f0 2024-04-25T01:30:11.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.015+0000 7f0e67fff640 1 -- 172.21.15.112:0/2053227476 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7400fb00 con 0x7f0e800ab7f0 2024-04-25T01:30:11.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.015+0000 7f0e85b22640 1 -- 172.21.15.112:0/2053227476 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e801e7410 con 0x7f0e800ab7f0 2024-04-25T01:30:11.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.015+0000 7f0e85b22640 1 -- 172.21.15.112:0/2053227476 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e801e79d0 con 0x7f0e800ab7f0 2024-04-25T01:30:11.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.016+0000 7f0e85b22640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 0x7f0e80070120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.016+0000 7f0e85b22640 1 -- 172.21.15.112:0/2053227476 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0e80140a60 con 0x7f0e801440e0 2024-04-25T01:30:11.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.016+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 0x7f0e80070120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.016+0000 7f0e7f7fe640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 msgr2=0x7f0e80070120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-04-25T01:30:11.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.016+0000 7f0e7f7fe640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 msgr2=0x7f0e80070120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:30:11.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.016+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 0x7f0e80070120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:11.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.016+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 0x7f0e80070120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:11.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.016+0000 7f0e67fff640 1 -- 172.21.15.112:0/2053227476 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0e7400fca0 con 0x7f0e800ab7f0 2024-04-25T01:30:11.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.017+0000 7f0e67fff640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0e6c06be30 0x7f0e6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:11.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.017+0000 7f0e7ffff640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0e6c06be30 0x7f0e6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:11.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.017+0000 7f0e67fff640 1 -- 172.21.15.112:0/2053227476 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f0e7408b120 con 0x7f0e800ab7f0 2024-04-25T01:30:11.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.017+0000 7f0e7ffff640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0e6c06be30 0x7f0e6c06e2f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0e78035000 tx=0x7f0e7800b030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:12.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.217+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 0x7f0e80070120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:12.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.217+0000 7f0e7f7fe640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 msgr2=0x7f0e80070120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-04-25T01:30:12.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.217+0000 7f0e7f7fe640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 msgr2=0x7f0e80070120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:30:12.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.217+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 0x7f0e80070120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:12.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.217+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 0x7f0e80070120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:30:12.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.618+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 0x7f0e80070120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:12.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.618+0000 7f0e7f7fe640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 msgr2=0x7f0e80070120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-04-25T01:30:12.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.618+0000 7f0e7f7fe640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 msgr2=0x7f0e80070120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:30:12.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.618+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 0x7f0e80070120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:12.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:12.618+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 0x7f0e80070120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:30:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:13.419+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 0x7f0e80070120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:13.419+0000 7f0e7f7fe640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 msgr2=0x7f0e80070120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-04-25T01:30:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:13.419+0000 7f0e7f7fe640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 msgr2=0x7f0e80070120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:30:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:13.419+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 0x7f0e80070120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:13.419+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 0x7f0e80070120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:30:14.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:15.021+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 0x7f0e80070120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:14.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:15.021+0000 7f0e7f7fe640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 msgr2=0x7f0e80070120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-04-25T01:30:14.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:15.021+0000 7f0e7f7fe640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 msgr2=0x7f0e80070120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:30:14.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:15.021+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 0x7f0e80070120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:14.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:15.021+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 0x7f0e80070120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T01:30:15.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.014+0000 7f0e66ffd640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 msgr2=0x7f0e80070120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:30:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.014+0000 7f0e66ffd640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 0x7f0e80070120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.014+0000 7f0e66ffd640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e58003540 0x7f0e580059d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.014+0000 7f0e66ffd640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 msgr2=0x7f0e80070120 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:30:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.014+0000 7f0e66ffd640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801440e0 0x7f0e80070120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.014+0000 7f0e66ffd640 1 -- 172.21.15.112:0/2053227476 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0e58006080 con 0x7f0e58003540 2024-04-25T01:30:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.014+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e58003540 0x7f0e580059d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.015+0000 7f0e7f7fe640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e58003540 msgr2=0x7f0e580059d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-04-25T01:30:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.015+0000 7f0e7f7fe640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e58003540 msgr2=0x7f0e580059d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:30:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.015+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e58003540 0x7f0e580059d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.015+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e58003540 0x7f0e580059d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.216+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e58003540 0x7f0e580059d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.216+0000 7f0e7f7fe640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e58003540 msgr2=0x7f0e580059d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-04-25T01:30:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.216+0000 7f0e7f7fe640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e58003540 msgr2=0x7f0e580059d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:30:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.216+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e58003540 0x7f0e580059d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.216+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e58003540 0x7f0e580059d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:30:16.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.617+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e58003540 0x7f0e580059d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:16.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.617+0000 7f0e7f7fe640 1 -- 172.21.15.112:0/2053227476 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e58006230 con 0x7f0e58003540 2024-04-25T01:30:16.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.617+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e58003540 0x7f0e580059d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0e70001430 tx=0x7f0e70003cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:16.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.618+0000 7f0e67fff640 1 -- 172.21.15.112:0/2053227476 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70017b10 con 0x7f0e58003540 2024-04-25T01:30:16.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.618+0000 7f0e67fff640 1 -- 172.21.15.112:0/2053227476 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0e70017b10 con 0x7f0e58003540 2024-04-25T01:30:16.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.618+0000 7f0e67fff640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e58003540 msgr2=0x7f0e580059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:16.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.618+0000 7f0e67fff640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e58003540 0x7f0e580059d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0e70001430 tx=0x7f0e70003cb0 comp rx=0 tx=0).stop 2024-04-25T01:30:16.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.618+0000 7f0e67fff640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e58003540 msgr2=0x7f0e580059d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:30:16.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.618+0000 7f0e67fff640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e58003540 0x7f0e580059d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:16.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.628+0000 7f0e85b22640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801e7c10 0x7f0e801ea870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:16.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.628+0000 7f0e85b22640 1 -- 172.21.15.112:0/2053227476 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0e80075980 con 0x7f0e801e7c10 2024-04-25T01:30:16.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.628+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801e7c10 0x7f0e801ea870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:16.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.629+0000 7f0e7f7fe640 1 -- 172.21.15.112:0/2053227476 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0e80070660 con 0x7f0e801e7c10 2024-04-25T01:30:16.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.629+0000 7f0e7f7fe640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801e7c10 0x7f0e801ea870 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0e70001430 tx=0x7f0e700012d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:16.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.629+0000 7f0e67fff640 1 -- 172.21.15.112:0/2053227476 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70026570 con 0x7f0e801e7c10 2024-04-25T01:30:16.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.629+0000 7f0e67fff640 1 -- 172.21.15.112:0/2053227476 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4557 (secure 0 0 0) 0x7f0e7002a070 con 0x7f0e801e7c10 2024-04-25T01:30:16.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.629+0000 7f0e67fff640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801e7c10 msgr2=0x7f0e801ea870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:16.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.629+0000 7f0e67fff640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801e7c10 0x7f0e801ea870 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0e70001430 tx=0x7f0e700012d0 comp rx=0 tx=0).stop 2024-04-25T01:30:16.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.629+0000 7f0e67fff640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801e7c10 msgr2=0x7f0e801ea870 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:30:16.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.629+0000 7f0e7ffff640 1 -- 172.21.15.112:0/2053227476 reap_dead start 2024-04-25T01:30:16.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.629+0000 7f0e67fff640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e801e7c10 0x7f0e801ea870 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:16.608 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:30:16.608 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:30:16.608 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:30:16.608 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:30:16.608 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 32, 2024-04-25T01:30:16.608 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 5368, 2024-04-25T01:30:16.608 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:30:16.609 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:30:16.609 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:30:16.609 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:30:16.609 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:16.609 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 5, 2024-04-25T01:30:16.609 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:30:16.609 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:30:16.609 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:30:16.609 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:16.609 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:16.609 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:16.609 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:16.609 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:16.609 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:16.609 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:16.609 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:16.610 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:16.610 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:16.610 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:16.610 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:16.610 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:30:16.610 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:30:16.610 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:16.610 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:16.610 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:16.610 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:16.610 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:16.610 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:16.610 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:16.610 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:16.611 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:16.611 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:16.611 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:16.611 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:16.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:16.611 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:30:16.611 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:30:16.611 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:30:16.611 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:30:16.611 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:30:16.611 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:30:16.611 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:30:16.611 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:30:16.611 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:30:16.612 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:30:16.612 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:30:16.612 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:30:16.612 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:30:16.612 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:30:16.612 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:30:16.612 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:30:16.612 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:30:16.612 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:16.612 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:16.612 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:16.612 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:16.612 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:16.612 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:16.612 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:16.613 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:16.613 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:16.613 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:16.613 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:16.613 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:16.613 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:30:16.613 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:16.613 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:30:16.613 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:16.613 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:30:16.613 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:30:16.613 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:16.613 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:16.613 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:16.614 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:16.614 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:30:16.614 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:16.614 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:16.614 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:16.614 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:16.614 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:30:16.614 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:16.614 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:16.614 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:16.614 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:16.614 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:30:16.614 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:30:16.615 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:16.615 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:16.615 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:16.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:16.615 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:16.615 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:30:16.615 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:30:16.615 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:16.615 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:16.615 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:16.615 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:16.615 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:30:16.615 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:16.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:16.615 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:16.616 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:16.616 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:30:16.616 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:16.616 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:16.616 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:16.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:16.616 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:30:16.616 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:30:16.616 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:16.616 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:16.616 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:16.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:16.616 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:16.616 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:30:16.616 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:30:16.617 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:16.617 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:16.617 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:16.617 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:16.617 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:30:16.617 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:16.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:16.617 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:16.617 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:16.617 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:30:16.617 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:16.617 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:16.617 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:16.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:16.617 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:30:16.618 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:30:16.618 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:16.618 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:16.618 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:16.618 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:16.618 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:16.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:16.618 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:30:16.618 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:30:16.618 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:16.618 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:16.618 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:16.618 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:30:16.618 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:16.619 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:16.619 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:30:16.619 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:16.619 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:16.619 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:16.619 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:30:16.619 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:16.619 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:16.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:16.619 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:30:16.619 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:30:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.630+0000 7f0e85b22640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0e6c06be30 msgr2=0x7f0e6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.630+0000 7f0e85b22640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0e6c06be30 0x7f0e6c06e2f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0e78035000 tx=0x7f0e7800b030 comp rx=0 tx=0).stop 2024-04-25T01:30:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.630+0000 7f0e85b22640 1 -- 172.21.15.112:0/2053227476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e800ab7f0 msgr2=0x7f0e80140520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.630+0000 7f0e85b22640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e800ab7f0 0x7f0e80140520 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0e74007340 tx=0x7f0e74007740 comp rx=0 tx=0).stop 2024-04-25T01:30:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.630+0000 7f0e85b22640 1 -- 172.21.15.112:0/2053227476 shutdown_connections 2024-04-25T01:30:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.630+0000 7f0e85b22640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0e6c06be30 0x7f0e6c06e2f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.630+0000 7f0e85b22640 1 --2- 172.21.15.112:0/2053227476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e800ab7f0 0x7f0e80140520 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.630+0000 7f0e85b22640 1 -- 172.21.15.112:0/2053227476 >> 172.21.15.112:0/2053227476 conn(0x7f0e80074a90 msgr2=0x7f0e8014a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.630+0000 7f0e85b22640 1 -- 172.21.15.112:0/2053227476 shutdown_connections 2024-04-25T01:30:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.630+0000 7f0e85b22640 1 -- 172.21.15.112:0/2053227476 wait complete. 2024-04-25T01:30:16.622 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:30:16.622 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T01:30:16.622 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T01:30:16.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.750+0000 7fa54ad96640 1 Processor -- start 2024-04-25T01:30:16.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.750+0000 7fa54ad96640 1 -- start start 2024-04-25T01:30:16.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.751+0000 7fa54ad96640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa544153030 0x7fa5440c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:16.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.751+0000 7fa54ad96640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5440c9af0 0x7fa5441595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:16.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.751+0000 7fa54ad96640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5440c8910 con 0x7fa544159b30 2024-04-25T01:30:16.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.751+0000 7fa54ad96640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa544159160 con 0x7fa544153030 2024-04-25T01:30:16.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.751+0000 7fa54ad96640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa5440c8bf0 con 0x7fa5440c9af0 2024-04-25T01:30:16.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.751+0000 7fa543fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5440c9af0 0x7fa5441595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:16.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.751+0000 7fa543fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5440c9af0 0x7fa5441595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:47082/0 (socket says 172.21.15.112:47082) 2024-04-25T01:30:16.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.751+0000 7fa543fff640 1 -- 172.21.15.112:0/3312053015 learned_addr learned my addr 172.21.15.112:0/3312053015 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:16.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.751+0000 7fa548b32640 1 --2- 172.21.15.112:0/3312053015 >> v2:172.21.15.187:3300/0 conn(0x7fa544153030 0x7fa5440c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:16.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.752+0000 7fa543fff640 1 -- 172.21.15.112:0/3312053015 >> v2:172.21.15.187:3300/0 conn(0x7fa544153030 msgr2=0x7fa5440c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:30:16.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.752+0000 7fa543fff640 1 --2- 172.21.15.112:0/3312053015 >> v2:172.21.15.187:3300/0 conn(0x7fa544153030 0x7fa5440c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:16.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.752+0000 7fa543fff640 1 -- 172.21.15.112:0/3312053015 >> v1:172.21.15.187:6789/0 conn(0x7fa544159b30 legacy=0x7fa5440c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:16.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.752+0000 7fa543fff640 1 -- 172.21.15.112:0/3312053015 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5440c4400 con 0x7fa5440c9af0 2024-04-25T01:30:16.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.752+0000 7fa5437fe640 1 -- 172.21.15.112:0/3312053015 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2118743201 0 0) 0x7fa5440c8910 con 0x7fa544159b30 2024-04-25T01:30:16.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.752+0000 7fa548b32640 1 --2- 172.21.15.112:0/3312053015 >> v2:172.21.15.187:3300/0 conn(0x7fa544153030 0x7fa5440c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:30:16.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.753+0000 7fa543fff640 1 --2- 172.21.15.112:0/3312053015 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5440c9af0 0x7fa5441595f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa530009870 tx=0x7fa53002a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57c3b23a02c69363 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:16.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.753+0000 7fa5437fe640 1 -- 172.21.15.112:0/3312053015 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530037070 con 0x7fa5440c9af0 2024-04-25T01:30:16.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.753+0000 7fa5437fe640 1 -- 172.21.15.112:0/3312053015 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa53002ae50 con 0x7fa5440c9af0 2024-04-25T01:30:16.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.753+0000 7fa5437fe640 1 -- 172.21.15.112:0/3312053015 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530032860 con 0x7fa5440c9af0 2024-04-25T01:30:16.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.754+0000 7fa54ad96640 1 -- 172.21.15.112:0/3312053015 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5440c9af0 msgr2=0x7fa5441595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:16.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.754+0000 7fa54ad96640 1 --2- 172.21.15.112:0/3312053015 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5440c9af0 0x7fa5441595f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa530009870 tx=0x7fa53002a300 comp rx=0 tx=0).stop 2024-04-25T01:30:16.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.754+0000 7fa54ad96640 1 -- 172.21.15.112:0/3312053015 shutdown_connections 2024-04-25T01:30:16.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.754+0000 7fa54ad96640 1 --2- 172.21.15.112:0/3312053015 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5440c9af0 0x7fa5441595f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:16.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.754+0000 7fa54ad96640 1 --2- 172.21.15.112:0/3312053015 >> v2:172.21.15.187:3300/0 conn(0x7fa544153030 0x7fa5440c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:16.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.754+0000 7fa54ad96640 1 -- 172.21.15.112:0/3312053015 >> 172.21.15.112:0/3312053015 conn(0x7fa544074a00 msgr2=0x7fa5440c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:16.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.754+0000 7fa54ad96640 1 -- 172.21.15.112:0/3312053015 shutdown_connections 2024-04-25T01:30:16.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.754+0000 7fa54ad96640 1 -- 172.21.15.112:0/3312053015 wait complete. 2024-04-25T01:30:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.755+0000 7fa54ad96640 1 Processor -- start 2024-04-25T01:30:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.755+0000 7fa54ad96640 1 -- start start 2024-04-25T01:30:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.756+0000 7fa54ad96640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5440c9af0 0x7fa5441eecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.756+0000 7fa54ad96640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa544153030 0x7fa5441eb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.756+0000 7fa54ad96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa544159b30 0x7fa54406da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.756+0000 7fa54ad96640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5441564f0 con 0x7fa544159b30 2024-04-25T01:30:16.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.756+0000 7fa54ad96640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa544156370 con 0x7fa544153030 2024-04-25T01:30:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.756+0000 7fa54ad96640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa544156670 con 0x7fa5440c9af0 2024-04-25T01:30:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.756+0000 7fa549333640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa544159b30 0x7fa54406da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.756+0000 7fa549333640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa544159b30 0x7fa54406da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49616/0 (socket says 172.21.15.112:49616) 2024-04-25T01:30:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.756+0000 7fa549333640 1 -- 172.21.15.112:0/985356461 learned_addr learned my addr 172.21.15.112:0/985356461 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.756+0000 7fa548b32640 1 --2- 172.21.15.112:0/985356461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5440c9af0 0x7fa5441eecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.756+0000 7fa543fff640 1 --2- 172.21.15.112:0/985356461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa544153030 0x7fa5441eb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.756+0000 7fa549333640 1 -- 172.21.15.112:0/985356461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5440c9af0 msgr2=0x7fa5441eecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.756+0000 7fa549333640 1 --2- 172.21.15.112:0/985356461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5440c9af0 0x7fa5441eecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.756+0000 7fa549333640 1 -- 172.21.15.112:0/985356461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa544153030 msgr2=0x7fa5441eb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.756+0000 7fa549333640 1 --2- 172.21.15.112:0/985356461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa544153030 0x7fa5441eb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.756+0000 7fa549333640 1 -- 172.21.15.112:0/985356461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5441ee230 con 0x7fa544159b30 2024-04-25T01:30:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.757+0000 7fa549333640 1 --2- 172.21.15.112:0/985356461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa544159b30 0x7fa54406da40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa53c00c780 tx=0x7fa53c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.757+0000 7fa5417fa640 1 -- 172.21.15.112:0/985356461 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c003da0 con 0x7fa544159b30 2024-04-25T01:30:16.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.757+0000 7fa5417fa640 1 -- 172.21.15.112:0/985356461 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa53c00ce30 con 0x7fa544159b30 2024-04-25T01:30:16.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.757+0000 7fa54ad96640 1 -- 172.21.15.112:0/985356461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5441ebbb0 con 0x7fa544159b30 2024-04-25T01:30:16.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.757+0000 7fa5417fa640 1 -- 172.21.15.112:0/985356461 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c005290 con 0x7fa544159b30 2024-04-25T01:30:16.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.757+0000 7fa54ad96640 1 -- 172.21.15.112:0/985356461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5441dfb20 con 0x7fa544159b30 2024-04-25T01:30:16.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.758+0000 7fa54ad96640 1 -- 172.21.15.112:0/985356461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5440682a0 con 0x7fa544159b30 2024-04-25T01:30:16.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.758+0000 7fa5417fa640 1 -- 172.21.15.112:0/985356461 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa53c005430 con 0x7fa544159b30 2024-04-25T01:30:16.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.759+0000 7fa5417fa640 1 --2- 172.21.15.112:0/985356461 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa52806be30 0x7fa52806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:16.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.759+0000 7fa5417fa640 1 -- 172.21.15.112:0/985356461 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fa53c08ae70 con 0x7fa544159b30 2024-04-25T01:30:16.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.759+0000 7fa548b32640 1 --2- 172.21.15.112:0/985356461 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa52806be30 0x7fa52806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:16.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.759+0000 7fa548b32640 1 --2- 172.21.15.112:0/985356461 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa52806be30 0x7fa52806e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa538009560 tx=0x7fa538009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:16.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:16.764+0000 7fa5417fa640 1 -- 172.21.15.112:0/985356461 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa53c058ec0 con 0x7fa544159b30 2024-04-25T01:30:17.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:17.043+0000 7fa54ad96640 1 -- 172.21.15.112:0/985356461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fa5440c6e80 con 0x7fa544159b30 2024-04-25T01:30:17.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:17.044+0000 7fa5417fa640 1 -- 172.21.15.112:0/985356461 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fa53c05d1c0 con 0x7fa544159b30 2024-04-25T01:30:17.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:17.046+0000 7fa54ad96640 1 -- 172.21.15.112:0/985356461 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa52806be30 msgr2=0x7fa52806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:17.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:17.046+0000 7fa54ad96640 1 --2- 172.21.15.112:0/985356461 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa52806be30 0x7fa52806e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa538009560 tx=0x7fa538009190 comp rx=0 tx=0).stop 2024-04-25T01:30:17.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:17.046+0000 7fa54ad96640 1 -- 172.21.15.112:0/985356461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa544159b30 msgr2=0x7fa54406da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:17.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:17.046+0000 7fa54ad96640 1 --2- 172.21.15.112:0/985356461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa544159b30 0x7fa54406da40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa53c00c780 tx=0x7fa53c00cc40 comp rx=0 tx=0).stop 2024-04-25T01:30:17.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:17.046+0000 7fa54ad96640 1 -- 172.21.15.112:0/985356461 shutdown_connections 2024-04-25T01:30:17.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:17.046+0000 7fa54ad96640 1 --2- 172.21.15.112:0/985356461 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa52806be30 0x7fa52806e2f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:17.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:17.046+0000 7fa54ad96640 1 --2- 172.21.15.112:0/985356461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa544159b30 0x7fa54406da40 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:17.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:17.046+0000 7fa54ad96640 1 --2- 172.21.15.112:0/985356461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa544153030 0x7fa5441eb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:17.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:17.046+0000 7fa54ad96640 1 --2- 172.21.15.112:0/985356461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5440c9af0 0x7fa5441eecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:17.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:17.046+0000 7fa54ad96640 1 -- 172.21.15.112:0/985356461 >> 172.21.15.112:0/985356461 conn(0x7fa544074a00 msgr2=0x7fa544158980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:17.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:17.046+0000 7fa54ad96640 1 -- 172.21.15.112:0/985356461 shutdown_connections 2024-04-25T01:30:17.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:17.046+0000 7fa54ad96640 1 -- 172.21.15.112:0/985356461 wait complete. 2024-04-25T01:30:17.036 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T01:30:25.174 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:30:25.176 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:30:27.037 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:30:27.038 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:30:27.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.172+0000 7fee62ad4640 1 Processor -- start 2024-04-25T01:30:27.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.172+0000 7fee62ad4640 1 -- start start 2024-04-25T01:30:27.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.173+0000 7fee62ad4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee5c0c5d70 0x7fee5c0c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.173+0000 7fee62ad4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee5c0c90e0 con 0x7fee5c152fd0 2024-04-25T01:30:27.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.173+0000 7fee62ad4640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee5c0cac00 con 0x7fee5c156cf0 2024-04-25T01:30:27.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.173+0000 7fee62ad4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fee5c0ca9b0 con 0x7fee5c0c5d70 2024-04-25T01:30:27.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.173+0000 7fee60870640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fee5c152fd0 0x7fee5c156280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:35318/0 (socket says 172.21.15.112:35318) 2024-04-25T01:30:27.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.173+0000 7fee60870640 1 -- 172.21.15.112:0/1833874110 learned_addr learned my addr 172.21.15.112:0/1833874110 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:30:27.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.173+0000 7fee5bfff640 1 --2- 172.21.15.112:0/1833874110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee5c0c5d70 0x7fee5c0c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.173+0000 7fee5b7fe640 1 -- 172.21.15.112:0/1833874110 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 713544778 0 0) 0x7fee5c0c90e0 con 0x7fee5c152fd0 2024-04-25T01:30:27.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.173+0000 7fee5b7fe640 1 -- 172.21.15.112:0/1833874110 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee4c003520 con 0x7fee5c152fd0 2024-04-25T01:30:27.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.173+0000 7fee5b7fe640 1 -- 172.21.15.112:0/1833874110 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3167703512 0 0) 0x7fee5c0cac00 con 0x7fee5c156cf0 2024-04-25T01:30:27.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.173+0000 7fee5b7fe640 1 -- 172.21.15.112:0/1833874110 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee5c0c90e0 con 0x7fee5c156cf0 2024-04-25T01:30:27.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.173+0000 7fee5bfff640 1 -- 172.21.15.112:0/1833874110 >> v1:172.21.15.187:6789/0 conn(0x7fee5c156cf0 legacy=0x7fee5c06bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.173+0000 7fee5bfff640 1 -- 172.21.15.112:0/1833874110 >> v1:172.21.15.112:6789/0 conn(0x7fee5c152fd0 legacy=0x7fee5c156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.173+0000 7fee5bfff640 1 -- 172.21.15.112:0/1833874110 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5c0c9380 con 0x7fee5c0c5d70 2024-04-25T01:30:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.174+0000 7fee5bfff640 1 --2- 172.21.15.112:0/1833874110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee5c0c5d70 0x7fee5c0c9b80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fee480097d0 tx=0x7fee4802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56e408ddce7589e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.174+0000 7fee5b7fe640 1 -- 172.21.15.112:0/1833874110 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48037070 con 0x7fee5c0c5d70 2024-04-25T01:30:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.174+0000 7fee5b7fe640 1 -- 172.21.15.112:0/1833874110 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee48032440 con 0x7fee5c0c5d70 2024-04-25T01:30:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.174+0000 7fee5b7fe640 1 -- 172.21.15.112:0/1833874110 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48032810 con 0x7fee5c0c5d70 2024-04-25T01:30:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.175+0000 7fee62ad4640 1 -- 172.21.15.112:0/1833874110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee5c0c5d70 msgr2=0x7fee5c0c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.175+0000 7fee62ad4640 1 --2- 172.21.15.112:0/1833874110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee5c0c5d70 0x7fee5c0c9b80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fee480097d0 tx=0x7fee4802a180 comp rx=0 tx=0).stop 2024-04-25T01:30:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.175+0000 7fee62ad4640 1 -- 172.21.15.112:0/1833874110 shutdown_connections 2024-04-25T01:30:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.175+0000 7fee62ad4640 1 --2- 172.21.15.112:0/1833874110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee5c0c5d70 0x7fee5c0c9b80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.175+0000 7fee62ad4640 1 -- 172.21.15.112:0/1833874110 >> 172.21.15.112:0/1833874110 conn(0x7fee5c0acc70 msgr2=0x7fee5c0ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:27.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.175+0000 7fee62ad4640 1 -- 172.21.15.112:0/1833874110 shutdown_connections 2024-04-25T01:30:27.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.175+0000 7fee62ad4640 1 -- 172.21.15.112:0/1833874110 wait complete. 2024-04-25T01:30:27.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.176+0000 7fee62ad4640 1 Processor -- start 2024-04-25T01:30:27.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.176+0000 7fee62ad4640 1 -- start start 2024-04-25T01:30:27.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.177+0000 7fee62ad4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee5c0c5d70 0x7fee5c149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.177+0000 7fee62ad4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee5c152fd0 0x7fee5c0abf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.177+0000 7fee62ad4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee5c156cf0 0x7fee5c1542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.177+0000 7fee62ad4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c06c4b0 con 0x7fee5c152fd0 2024-04-25T01:30:27.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.177+0000 7fee62ad4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c0caee0 con 0x7fee5c156cf0 2024-04-25T01:30:27.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.177+0000 7fee62ad4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fee5c06c630 con 0x7fee5c0c5d70 2024-04-25T01:30:27.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.177+0000 7fee5bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee5c152fd0 0x7fee5c0abf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.177+0000 7fee5bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee5c152fd0 0x7fee5c0abf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33716/0 (socket says 172.21.15.112:33716) 2024-04-25T01:30:27.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.177+0000 7fee5bfff640 1 -- 172.21.15.112:0/420104891 learned_addr learned my addr 172.21.15.112:0/420104891 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:27.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.177+0000 7fee60870640 1 --2- 172.21.15.112:0/420104891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee5c0c5d70 0x7fee5c149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.177+0000 7fee61071640 1 --2- 172.21.15.112:0/420104891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee5c156cf0 0x7fee5c1542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.177+0000 7fee5bfff640 1 -- 172.21.15.112:0/420104891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee5c0c5d70 msgr2=0x7fee5c149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.177+0000 7fee5bfff640 1 --2- 172.21.15.112:0/420104891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee5c0c5d70 0x7fee5c149150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.177+0000 7fee5bfff640 1 -- 172.21.15.112:0/420104891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee5c156cf0 msgr2=0x7fee5c1542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.177+0000 7fee5bfff640 1 --2- 172.21.15.112:0/420104891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee5c156cf0 0x7fee5c1542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.177+0000 7fee5bfff640 1 -- 172.21.15.112:0/420104891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5c149690 con 0x7fee5c152fd0 2024-04-25T01:30:27.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.178+0000 7fee5bfff640 1 --2- 172.21.15.112:0/420104891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee5c152fd0 0x7fee5c0abf70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fee48035040 tx=0x7fee480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:27.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.178+0000 7fee597fa640 1 -- 172.21.15.112:0/420104891 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48041020 con 0x7fee5c152fd0 2024-04-25T01:30:27.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.178+0000 7fee62ad4640 1 -- 172.21.15.112:0/420104891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5c1ef5b0 con 0x7fee5c152fd0 2024-04-25T01:30:27.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.178+0000 7fee597fa640 1 -- 172.21.15.112:0/420104891 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee4803d720 con 0x7fee5c152fd0 2024-04-25T01:30:27.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.178+0000 7fee597fa640 1 -- 172.21.15.112:0/420104891 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48037040 con 0x7fee5c152fd0 2024-04-25T01:30:27.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.178+0000 7fee62ad4640 1 -- 172.21.15.112:0/420104891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee5c1eb590 con 0x7fee5c152fd0 2024-04-25T01:30:27.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.179+0000 7fee62ad4640 1 -- 172.21.15.112:0/420104891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee5c0682a0 con 0x7fee5c152fd0 2024-04-25T01:30:27.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.179+0000 7fee597fa640 1 -- 172.21.15.112:0/420104891 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fee48044020 con 0x7fee5c152fd0 2024-04-25T01:30:27.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.180+0000 7fee597fa640 1 --2- 172.21.15.112:0/420104891 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fee4006be30 0x7fee4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.180+0000 7fee597fa640 1 -- 172.21.15.112:0/420104891 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fee480ab820 con 0x7fee5c152fd0 2024-04-25T01:30:27.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.180+0000 7fee60870640 1 --2- 172.21.15.112:0/420104891 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fee4006be30 0x7fee4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.180+0000 7fee60870640 1 --2- 172.21.15.112:0/420104891 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fee4006be30 0x7fee4006e2f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fee50000f60 tx=0x7fee50007d70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:27.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.185+0000 7fee597fa640 1 -- 172.21.15.112:0/420104891 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fee4804b030 con 0x7fee5c152fd0 2024-04-25T01:30:27.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.487+0000 7fee62ad4640 1 -- 172.21.15.112:0/420104891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fee5c0c5090 con 0x7fee5c152fd0 2024-04-25T01:30:27.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.487+0000 7fee597fa640 1 -- 172.21.15.112:0/420104891 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7fee480797f0 con 0x7fee5c152fd0 2024-04-25T01:30:27.466 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":32,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:30:27.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.490+0000 7fee62ad4640 1 -- 172.21.15.112:0/420104891 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fee4006be30 msgr2=0x7fee4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.490+0000 7fee62ad4640 1 --2- 172.21.15.112:0/420104891 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fee4006be30 0x7fee4006e2f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fee50000f60 tx=0x7fee50007d70 comp rx=0 tx=0).stop 2024-04-25T01:30:27.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.490+0000 7fee62ad4640 1 -- 172.21.15.112:0/420104891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee5c152fd0 msgr2=0x7fee5c0abf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.490+0000 7fee62ad4640 1 --2- 172.21.15.112:0/420104891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee5c152fd0 0x7fee5c0abf70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fee48035040 tx=0x7fee480046c0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.490+0000 7fee62ad4640 1 -- 172.21.15.112:0/420104891 shutdown_connections 2024-04-25T01:30:27.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.490+0000 7fee62ad4640 1 --2- 172.21.15.112:0/420104891 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fee4006be30 0x7fee4006e2f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.490+0000 7fee62ad4640 1 --2- 172.21.15.112:0/420104891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee5c156cf0 0x7fee5c1542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.490+0000 7fee62ad4640 1 --2- 172.21.15.112:0/420104891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee5c152fd0 0x7fee5c0abf70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.490+0000 7fee62ad4640 1 --2- 172.21.15.112:0/420104891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee5c0c5d70 0x7fee5c149150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.490+0000 7fee62ad4640 1 -- 172.21.15.112:0/420104891 >> 172.21.15.112:0/420104891 conn(0x7fee5c0acc70 msgr2=0x7fee5c0ad050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:27.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.490+0000 7fee62ad4640 1 -- 172.21.15.112:0/420104891 shutdown_connections 2024-04-25T01:30:27.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.490+0000 7fee62ad4640 1 -- 172.21.15.112:0/420104891 wait complete. 2024-04-25T01:30:27.480 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:30:27.480 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T01:30:27.480 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:30:27.480 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:30:27.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.613+0000 7ff2d7790640 1 Processor -- start 2024-04-25T01:30:27.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.614+0000 7ff2d7790640 1 -- start start 2024-04-25T01:30:27.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.614+0000 7ff2d7790640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d0071520 0x7ff2d0154b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.614+0000 7ff2d7790640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff2d00c7c80 0x7ff2d0153660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.614+0000 7ff2d7790640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2d0154120 con 0x7ff2d00c7800 2024-04-25T01:30:27.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.614+0000 7ff2d7790640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7ff2d0154400 con 0x7ff2d00c7c80 2024-04-25T01:30:27.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.614+0000 7ff2d7790640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff2d00c7000 con 0x7ff2d0071520 2024-04-25T01:30:27.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.615+0000 7ff2d5d2d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff2d00c7c80 0x7ff2d0153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.615+0000 7ff2d4d2b640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff2d00c7800 0x7ff2d00c82d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:35708/0 (socket says 172.21.15.112:35708) 2024-04-25T01:30:27.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.615+0000 7ff2d4d2b640 1 -- 172.21.15.112:0/1322405640 learned_addr learned my addr 172.21.15.112:0/1322405640 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:30:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.615+0000 7ff2d552c640 1 --2- 172.21.15.112:0/1322405640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d0071520 0x7ff2d0154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.615+0000 7ff2d5d2d640 1 -- 172.21.15.112:0/1322405640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d0071520 msgr2=0x7ff2d0154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:30:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.615+0000 7ff2d5d2d640 1 --2- 172.21.15.112:0/1322405640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d0071520 0x7ff2d0154b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.615+0000 7ff2d5d2d640 1 -- 172.21.15.112:0/1322405640 >> v1:172.21.15.112:6789/0 conn(0x7ff2d00c7800 legacy=0x7ff2d00c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.615+0000 7ff2c7fff640 1 -- 172.21.15.112:0/1322405640 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2103233839 0 0) 0x7ff2d0154120 con 0x7ff2d00c7800 2024-04-25T01:30:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.615+0000 7ff2d5d2d640 1 -- 172.21.15.112:0/1322405640 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d00c72e0 con 0x7ff2d00c7c80 2024-04-25T01:30:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.616+0000 7ff2d5d2d640 1 --2- 172.21.15.112:0/1322405640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d00c7c80 0x7ff2d0153660 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8009d90 tx=0x7ff2c802a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6832f37e3191d2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:27.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.618+0000 7ff2c7fff640 1 -- 172.21.15.112:0/1322405640 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8007740 con 0x7ff2d00c7c80 2024-04-25T01:30:27.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.618+0000 7ff2c7fff640 1 -- 172.21.15.112:0/1322405640 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2c80078e0 con 0x7ff2d00c7c80 2024-04-25T01:30:27.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.618+0000 7ff2c7fff640 1 -- 172.21.15.112:0/1322405640 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8007cb0 con 0x7ff2d00c7c80 2024-04-25T01:30:27.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.619+0000 7ff2d7790640 1 -- 172.21.15.112:0/1322405640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d00c7c80 msgr2=0x7ff2d0153660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.619+0000 7ff2d7790640 1 --2- 172.21.15.112:0/1322405640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d00c7c80 0x7ff2d0153660 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8009d90 tx=0x7ff2c802a530 comp rx=0 tx=0).stop 2024-04-25T01:30:27.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.619+0000 7ff2d7790640 1 -- 172.21.15.112:0/1322405640 shutdown_connections 2024-04-25T01:30:27.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.619+0000 7ff2d7790640 1 --2- 172.21.15.112:0/1322405640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d00c7c80 0x7ff2d0153660 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.619+0000 7ff2d7790640 1 --2- 172.21.15.112:0/1322405640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d0071520 0x7ff2d0154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.619+0000 7ff2d7790640 1 -- 172.21.15.112:0/1322405640 >> 172.21.15.112:0/1322405640 conn(0x7ff2d0074a00 msgr2=0x7ff2d00c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:27.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.619+0000 7ff2d7790640 1 -- 172.21.15.112:0/1322405640 shutdown_connections 2024-04-25T01:30:27.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.619+0000 7ff2d7790640 1 -- 172.21.15.112:0/1322405640 wait complete. 2024-04-25T01:30:27.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.620+0000 7ff2d7790640 1 Processor -- start 2024-04-25T01:30:27.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.620+0000 7ff2d7790640 1 -- start start 2024-04-25T01:30:27.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.621+0000 7ff2d7790640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d0071520 0x7ff2d01ef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.621+0000 7ff2d7790640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d00c7800 0x7ff2d0156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.621+0000 7ff2d7790640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2d00c7c80 0x7ff2d00c2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.621+0000 7ff2d7790640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d00adab0 con 0x7ff2d00c7800 2024-04-25T01:30:27.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.621+0000 7ff2d7790640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d00ad930 con 0x7ff2d00c7c80 2024-04-25T01:30:27.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.621+0000 7ff2d7790640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff2d0153ba0 con 0x7ff2d0071520 2024-04-25T01:30:27.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.621+0000 7ff2d4d2b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d00c7800 0x7ff2d0156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.621+0000 7ff2d4d2b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d00c7800 0x7ff2d0156030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50728/0 (socket says 172.21.15.112:50728) 2024-04-25T01:30:27.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.621+0000 7ff2d4d2b640 1 -- 172.21.15.112:0/2996795972 learned_addr learned my addr 172.21.15.112:0/2996795972 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:27.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.621+0000 7ff2d552c640 1 --2- 172.21.15.112:0/2996795972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d0071520 0x7ff2d01ef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.621+0000 7ff2d5d2d640 1 --2- 172.21.15.112:0/2996795972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2d00c7c80 0x7ff2d00c2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.622+0000 7ff2d4d2b640 1 -- 172.21.15.112:0/2996795972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d0071520 msgr2=0x7ff2d01ef1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.622+0000 7ff2d4d2b640 1 --2- 172.21.15.112:0/2996795972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2d0071520 0x7ff2d01ef1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.622+0000 7ff2d4d2b640 1 -- 172.21.15.112:0/2996795972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2d00c7c80 msgr2=0x7ff2d00c2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.622+0000 7ff2d4d2b640 1 --2- 172.21.15.112:0/2996795972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2d00c7c80 0x7ff2d00c2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.622+0000 7ff2d4d2b640 1 -- 172.21.15.112:0/2996795972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d006e250 con 0x7ff2d00c7800 2024-04-25T01:30:27.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.622+0000 7ff2d4d2b640 1 --2- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d00c7800 0x7ff2d0156030 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000c500 tx=0x7ff2c000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:27.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.623+0000 7ff2c5ffb640 1 -- 172.21.15.112:0/2996795972 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0018070 con 0x7ff2d00c7800 2024-04-25T01:30:27.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.623+0000 7ff2c5ffb640 1 -- 172.21.15.112:0/2996795972 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2c0007580 con 0x7ff2d00c7800 2024-04-25T01:30:27.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.623+0000 7ff2d7790640 1 -- 172.21.15.112:0/2996795972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d006dc90 con 0x7ff2d00c7800 2024-04-25T01:30:27.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.623+0000 7ff2c5ffb640 1 -- 172.21.15.112:0/2996795972 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0004a20 con 0x7ff2d00c7800 2024-04-25T01:30:27.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.624+0000 7ff2d7790640 1 -- 172.21.15.112:0/2996795972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d01eb3d0 con 0x7ff2d00c7800 2024-04-25T01:30:27.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.625+0000 7ff2d7790640 1 --2- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d01557c0 0x7ff2d01ebbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.625+0000 7ff2d7790640 1 -- 172.21.15.112:0/2996795972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d01ef730 con 0x7ff2d01557c0 2024-04-25T01:30:27.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.625+0000 7ff2d552c640 1 --2- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d01557c0 0x7ff2d01ebbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.625+0000 7ff2c5ffb640 1 -- 172.21.15.112:0/2996795972 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff2c001b070 con 0x7ff2d00c7800 2024-04-25T01:30:27.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.625+0000 7ff2d552c640 1 -- 172.21.15.112:0/2996795972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff2d0156570 con 0x7ff2d01557c0 2024-04-25T01:30:27.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.625+0000 7ff2d552c640 1 --2- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d01557c0 0x7ff2d01ebbd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0118e0 tx=0x7ff2cc011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:27.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.625+0000 7ff2c5ffb640 1 --2- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff2b406be30 0x7ff2b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.625+0000 7ff2c5ffb640 1 -- 172.21.15.112:0/2996795972 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7ff2c008a3c0 con 0x7ff2d00c7800 2024-04-25T01:30:27.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.626+0000 7ff2d5d2d640 1 --2- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff2b406be30 0x7ff2b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.626+0000 7ff2c5ffb640 1 -- 172.21.15.112:0/2996795972 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc019070 con 0x7ff2d01557c0 2024-04-25T01:30:27.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.626+0000 7ff2c5ffb640 1 -- 172.21.15.112:0/2996795972 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff2cc004300 con 0x7ff2d01557c0 2024-04-25T01:30:27.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.626+0000 7ff2c5ffb640 1 -- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d01557c0 msgr2=0x7ff2d01ebbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.626+0000 7ff2c5ffb640 1 --2- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d01557c0 0x7ff2d01ebbd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0118e0 tx=0x7ff2cc011da0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.626+0000 7ff2c5ffb640 1 -- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d01557c0 msgr2=0x7ff2d01ebbd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:30:27.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.626+0000 7ff2c5ffb640 1 --2- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d01557c0 0x7ff2d01ebbd0 secure :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0118e0 tx=0x7ff2cc011da0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.626+0000 7ff2d5d2d640 1 --2- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff2b406be30 0x7ff2b406e2f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8034000 tx=0x7ff2c802f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:27.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.636+0000 7ff2d7790640 1 --2- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d006c270 0x7ff2d006c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.636+0000 7ff2d7790640 1 -- 172.21.15.112:0/2996795972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d00682a0 con 0x7ff2d006c270 2024-04-25T01:30:27.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.636+0000 7ff2d552c640 1 --2- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d006c270 0x7ff2d006c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.636+0000 7ff2d552c640 1 -- 172.21.15.112:0/2996795972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff2d0070bd0 con 0x7ff2d006c270 2024-04-25T01:30:27.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.636+0000 7ff2d552c640 1 --2- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d006c270 0x7ff2d006c680 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0118e0 tx=0x7ff2cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:27.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.636+0000 7ff2c5ffb640 1 -- 172.21.15.112:0/2996795972 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc019040 con 0x7ff2d006c270 2024-04-25T01:30:27.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.636+0000 7ff2c5ffb640 1 -- 172.21.15.112:0/2996795972 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5072 (secure 0 0 0) 0x7ff2cc003da0 con 0x7ff2d006c270 2024-04-25T01:30:27.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.636+0000 7ff2c5ffb640 1 -- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d006c270 msgr2=0x7ff2d006c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.636+0000 7ff2c5ffb640 1 --2- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d006c270 0x7ff2d006c680 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0118e0 tx=0x7ff2cc0046c0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.636+0000 7ff2c5ffb640 1 -- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d006c270 msgr2=0x7ff2d006c680 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:30:27.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.636+0000 7ff2c5ffb640 1 --2- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d006c270 0x7ff2d006c680 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.617 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:30:27.617 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:30:27.617 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:30:27.617 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:30:27.617 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 32, 2024-04-25T01:30:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 299265, 2024-04-25T01:30:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:30:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:30:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:30:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:30:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 16, 2024-04-25T01:30:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:30:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:30:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:30:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:30:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:30:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:30:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:30:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:30:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:30:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:30:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:30:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:30:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:30:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:30:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:30:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:30:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:30:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:30:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:30:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:30:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:30:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:30:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:30:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:30:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:30:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:30:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:30:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:30:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:30:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:30:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:30:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:30:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:30:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:30:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:30:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:30:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:30:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:30:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:30:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:30:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:30:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:30:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:30:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:30:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:30:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:30:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:30:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:30:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 7 2024-04-25T01:30:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:30:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 8 2024-04-25T01:30:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:30:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:30:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.631 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:27.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.631 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:30:27.631 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:30:27.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.637+0000 7ff2d7790640 1 -- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff2b406be30 msgr2=0x7ff2b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.637+0000 7ff2d7790640 1 --2- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff2b406be30 0x7ff2b406e2f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8034000 tx=0x7ff2c802f000 comp rx=0 tx=0).stop 2024-04-25T01:30:27.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.637+0000 7ff2d7790640 1 -- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d00c7800 msgr2=0x7ff2d0156030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.637+0000 7ff2d7790640 1 --2- 172.21.15.112:0/2996795972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2d00c7800 0x7ff2d0156030 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000c500 tx=0x7ff2c000c9c0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.637+0000 7ff2d5d2d640 1 -- 172.21.15.112:0/2996795972 reap_dead start 2024-04-25T01:30:27.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.637+0000 7ff2d5d2d640 1 -- 172.21.15.112:0/2996795972 reap_dead start 2024-04-25T01:30:27.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.637+0000 7ff2d7790640 1 -- 172.21.15.112:0/2996795972 shutdown_connections 2024-04-25T01:30:27.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.637+0000 7ff2d7790640 1 -- 172.21.15.112:0/2996795972 >> 172.21.15.112:0/2996795972 conn(0x7ff2d0074a00 msgr2=0x7ff2d0151f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:27.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.637+0000 7ff2d7790640 1 -- 172.21.15.112:0/2996795972 shutdown_connections 2024-04-25T01:30:27.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.637+0000 7ff2d7790640 1 -- 172.21.15.112:0/2996795972 wait complete. 2024-04-25T01:30:27.633 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:30:27.633 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:30:27.633 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:30:27.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.764+0000 7f9d58109640 1 Processor -- start 2024-04-25T01:30:27.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.764+0000 7f9d58109640 1 -- start start 2024-04-25T01:30:27.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.765+0000 7f9d58109640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d501592c0 0x7f9d500c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.765+0000 7f9d58109640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9d500c72b0 0x7f9d50160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.765+0000 7f9d58109640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d50159b90 con 0x7f9d501596a0 2024-04-25T01:30:27.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.765+0000 7f9d58109640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f9d500c6e20 con 0x7f9d500c72b0 2024-04-25T01:30:27.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.765+0000 7f9d58109640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9d5006c970 con 0x7f9d501592c0 2024-04-25T01:30:27.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.765+0000 7f9d566a6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9d500c72b0 0x7f9d50160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.765+0000 7f9d556a4640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9d501596a0 0x7f9d50152d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:35722/0 (socket says 172.21.15.112:35722) 2024-04-25T01:30:27.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.765+0000 7f9d556a4640 1 -- 172.21.15.112:0/2169515263 learned_addr learned my addr 172.21.15.112:0/2169515263 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:30:27.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.765+0000 7f9d55ea5640 1 --2- 172.21.15.112:0/2169515263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d501592c0 0x7f9d500c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.765+0000 7f9d566a6640 1 -- 172.21.15.112:0/2169515263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d501592c0 msgr2=0x7f9d500c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:30:27.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.765+0000 7f9d566a6640 1 --2- 172.21.15.112:0/2169515263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d501592c0 0x7f9d500c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.765+0000 7f9d566a6640 1 -- 172.21.15.112:0/2169515263 >> v1:172.21.15.112:6789/0 conn(0x7f9d501596a0 legacy=0x7f9d50152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.765+0000 7f9d566a6640 1 -- 172.21.15.112:0/2169515263 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5006cc50 con 0x7f9d500c72b0 2024-04-25T01:30:27.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.765+0000 7f9d54ea3640 1 -- 172.21.15.112:0/2169515263 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2522818486 0 0) 0x7f9d50159b90 con 0x7f9d501596a0 2024-04-25T01:30:27.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.766+0000 7f9d566a6640 1 --2- 172.21.15.112:0/2169515263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d500c72b0 0x7f9d50160380 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009d90 tx=0x7f9d4802a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a385604504d3d7de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:27.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.766+0000 7f9d54ea3640 1 -- 172.21.15.112:0/2169515263 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48007740 con 0x7f9d500c72b0 2024-04-25T01:30:27.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.766+0000 7f9d54ea3640 1 -- 172.21.15.112:0/2169515263 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d480078e0 con 0x7f9d500c72b0 2024-04-25T01:30:27.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.766+0000 7f9d54ea3640 1 -- 172.21.15.112:0/2169515263 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48007cb0 con 0x7f9d500c72b0 2024-04-25T01:30:27.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.767+0000 7f9d58109640 1 -- 172.21.15.112:0/2169515263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d500c72b0 msgr2=0x7f9d50160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.767+0000 7f9d58109640 1 --2- 172.21.15.112:0/2169515263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d500c72b0 0x7f9d50160380 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009d90 tx=0x7f9d4802a530 comp rx=0 tx=0).stop 2024-04-25T01:30:27.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.767+0000 7f9d58109640 1 -- 172.21.15.112:0/2169515263 shutdown_connections 2024-04-25T01:30:27.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.767+0000 7f9d58109640 1 --2- 172.21.15.112:0/2169515263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d500c72b0 0x7f9d50160380 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.767+0000 7f9d58109640 1 --2- 172.21.15.112:0/2169515263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d501592c0 0x7f9d500c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.767+0000 7f9d58109640 1 -- 172.21.15.112:0/2169515263 >> 172.21.15.112:0/2169515263 conn(0x7f9d500cb170 msgr2=0x7f9d500cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:27.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.767+0000 7f9d58109640 1 -- 172.21.15.112:0/2169515263 shutdown_connections 2024-04-25T01:30:27.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.767+0000 7f9d58109640 1 -- 172.21.15.112:0/2169515263 wait complete. 2024-04-25T01:30:27.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.768+0000 7f9d58109640 1 Processor -- start 2024-04-25T01:30:27.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.768+0000 7f9d58109640 1 -- start start 2024-04-25T01:30:27.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.769+0000 7f9d58109640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d500c72b0 0x7f9d50158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.769+0000 7f9d58109640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d501592c0 0x7f9d50146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.769+0000 7f9d58109640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d501596a0 0x7f9d50146650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.769+0000 7f9d58109640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9d50153590 con 0x7f9d501592c0 2024-04-25T01:30:27.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.769+0000 7f9d58109640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9d50153410 con 0x7f9d500c72b0 2024-04-25T01:30:27.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.769+0000 7f9d58109640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9d5006db20 con 0x7f9d501596a0 2024-04-25T01:30:27.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.769+0000 7f9d556a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d501592c0 0x7f9d50146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.769+0000 7f9d566a6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d501596a0 0x7f9d50146650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.769+0000 7f9d55ea5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d500c72b0 0x7f9d50158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.769+0000 7f9d566a6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d501596a0 0x7f9d50146650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:36922/0 (socket says 172.21.15.112:36922) 2024-04-25T01:30:27.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.769+0000 7f9d556a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d501592c0 0x7f9d50146110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50766/0 (socket says 172.21.15.112:50766) 2024-04-25T01:30:27.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.769+0000 7f9d566a6640 1 -- 172.21.15.112:0/3854618497 learned_addr learned my addr 172.21.15.112:0/3854618497 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:27.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.770+0000 7f9d556a4640 1 -- 172.21.15.112:0/3854618497 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d501596a0 msgr2=0x7f9d50146650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.770+0000 7f9d556a4640 1 --2- 172.21.15.112:0/3854618497 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d501596a0 0x7f9d50146650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.770+0000 7f9d556a4640 1 -- 172.21.15.112:0/3854618497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d500c72b0 msgr2=0x7f9d50158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.770+0000 7f9d556a4640 1 --2- 172.21.15.112:0/3854618497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d500c72b0 0x7f9d50158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.770+0000 7f9d556a4640 1 -- 172.21.15.112:0/3854618497 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d501ecbc0 con 0x7f9d501592c0 2024-04-25T01:30:27.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.770+0000 7f9d556a4640 1 --2- 172.21.15.112:0/3854618497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d501592c0 0x7f9d50146110 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9d50159d60 tx=0x7f9d4000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:27.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.770+0000 7f9d467fc640 1 -- 172.21.15.112:0/3854618497 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d400058b0 con 0x7f9d501592c0 2024-04-25T01:30:27.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.770+0000 7f9d467fc640 1 -- 172.21.15.112:0/3854618497 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d40019030 con 0x7f9d501592c0 2024-04-25T01:30:27.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.770+0000 7f9d58109640 1 -- 172.21.15.112:0/3854618497 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d501ecea0 con 0x7f9d501592c0 2024-04-25T01:30:27.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.770+0000 7f9d467fc640 1 -- 172.21.15.112:0/3854618497 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d40005c90 con 0x7f9d501592c0 2024-04-25T01:30:27.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.770+0000 7f9d58109640 1 -- 172.21.15.112:0/3854618497 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d50158c90 con 0x7f9d501592c0 2024-04-25T01:30:27.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.771+0000 7f9d58109640 1 --2- 172.21.15.112:0/3854618497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d500c0d90 0x7f9d501ed980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.771+0000 7f9d58109640 1 -- 172.21.15.112:0/3854618497 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9d50156a60 con 0x7f9d500c0d90 2024-04-25T01:30:27.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.772+0000 7f9d55ea5640 1 --2- 172.21.15.112:0/3854618497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d500c0d90 0x7f9d501ed980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.772+0000 7f9d467fc640 1 -- 172.21.15.112:0/3854618497 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d400070d0 con 0x7f9d501592c0 2024-04-25T01:30:27.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.772+0000 7f9d55ea5640 1 -- 172.21.15.112:0/3854618497 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9d50075980 con 0x7f9d500c0d90 2024-04-25T01:30:27.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.772+0000 7f9d55ea5640 1 --2- 172.21.15.112:0/3854618497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d500c0d90 0x7f9d501ed980 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0118e0 tx=0x7f9d4c011da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:27.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.772+0000 7f9d467fc640 1 --2- 172.21.15.112:0/3854618497 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9d3406be60 0x7f9d3406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.772+0000 7f9d566a6640 1 --2- 172.21.15.112:0/3854618497 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9d3406be60 0x7f9d3406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.772+0000 7f9d467fc640 1 -- 172.21.15.112:0/3854618497 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f9d40092c40 con 0x7f9d501592c0 2024-04-25T01:30:27.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.772+0000 7f9d467fc640 1 -- 172.21.15.112:0/3854618497 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c019070 con 0x7f9d500c0d90 2024-04-25T01:30:27.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.772+0000 7f9d467fc640 1 -- 172.21.15.112:0/3854618497 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9d4c004300 con 0x7f9d500c0d90 2024-04-25T01:30:27.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.773+0000 7f9d467fc640 1 -- 172.21.15.112:0/3854618497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d500c0d90 msgr2=0x7f9d501ed980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.773+0000 7f9d467fc640 1 --2- 172.21.15.112:0/3854618497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d500c0d90 0x7f9d501ed980 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0118e0 tx=0x7f9d4c011da0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.773+0000 7f9d467fc640 1 -- 172.21.15.112:0/3854618497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d500c0d90 msgr2=0x7f9d501ed980 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:30:27.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.773+0000 7f9d467fc640 1 --2- 172.21.15.112:0/3854618497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d500c0d90 0x7f9d501ed980 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.773+0000 7f9d566a6640 1 --2- 172.21.15.112:0/3854618497 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9d3406be60 0x7f9d3406e320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9d500c2fe0 tx=0x7f9d4802f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:27.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.780+0000 7f9d58109640 1 --2- 172.21.15.112:0/3854618497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d501edec0 0x7f9d501ee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.780+0000 7f9d58109640 1 -- 172.21.15.112:0/3854618497 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9d500682a0 con 0x7f9d501edec0 2024-04-25T01:30:27.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.780+0000 7f9d55ea5640 1 --2- 172.21.15.112:0/3854618497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d501edec0 0x7f9d501ee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.780+0000 7f9d55ea5640 1 -- 172.21.15.112:0/3854618497 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9d500c79d0 con 0x7f9d501edec0 2024-04-25T01:30:27.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.781+0000 7f9d55ea5640 1 --2- 172.21.15.112:0/3854618497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d501edec0 0x7f9d501ee2a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0118e0 tx=0x7f9d4c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:27.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.781+0000 7f9d467fc640 1 -- 172.21.15.112:0/3854618497 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c019040 con 0x7f9d501edec0 2024-04-25T01:30:27.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.781+0000 7f9d467fc640 1 -- 172.21.15.112:0/3854618497 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4729 (secure 0 0 0) 0x7f9d4c00fe40 con 0x7f9d501edec0 2024-04-25T01:30:27.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.781+0000 7f9d467fc640 1 -- 172.21.15.112:0/3854618497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d501edec0 msgr2=0x7f9d501ee2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.781+0000 7f9d467fc640 1 --2- 172.21.15.112:0/3854618497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d501edec0 0x7f9d501ee2a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0118e0 tx=0x7f9d4c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.781+0000 7f9d467fc640 1 -- 172.21.15.112:0/3854618497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d501edec0 msgr2=0x7f9d501ee2a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:30:27.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.781+0000 7f9d467fc640 1 --2- 172.21.15.112:0/3854618497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d501edec0 0x7f9d501ee2a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.763 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:30:27.763 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:30:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:30:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:30:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 32, 2024-04-25T01:30:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 63472, 2024-04-25T01:30:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:30:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:30:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:30:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:30:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 16, 2024-04-25T01:30:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:30:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:30:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:30:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:30:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:30:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:30:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:30:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:30:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:30:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:30:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:30:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:30:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:30:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:30:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:30:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:30:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:30:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:30:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:30:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:30:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:30:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:30:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:30:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:30:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:30:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:30:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:30:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:30:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:30:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:30:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:30:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:30:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:30:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:30:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:30:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:30:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:30:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:30:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:30:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:30:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:30:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:30:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:30:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:30:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:30:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:30:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:30:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:30:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:30:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:30:27.776 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:30:27.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.781+0000 7f9d58109640 1 -- 172.21.15.112:0/3854618497 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9d3406be60 msgr2=0x7f9d3406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.781+0000 7f9d58109640 1 --2- 172.21.15.112:0/3854618497 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9d3406be60 0x7f9d3406e320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9d500c2fe0 tx=0x7f9d4802f000 comp rx=0 tx=0).stop 2024-04-25T01:30:27.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.781+0000 7f9d58109640 1 -- 172.21.15.112:0/3854618497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d501592c0 msgr2=0x7f9d50146110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.781+0000 7f9d58109640 1 --2- 172.21.15.112:0/3854618497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d501592c0 0x7f9d50146110 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9d50159d60 tx=0x7f9d4000c8e0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.781+0000 7f9d566a6640 1 -- 172.21.15.112:0/3854618497 reap_dead start 2024-04-25T01:30:27.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.781+0000 7f9d566a6640 1 -- 172.21.15.112:0/3854618497 reap_dead start 2024-04-25T01:30:27.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.781+0000 7f9d58109640 1 -- 172.21.15.112:0/3854618497 shutdown_connections 2024-04-25T01:30:27.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.781+0000 7f9d58109640 1 -- 172.21.15.112:0/3854618497 >> 172.21.15.112:0/3854618497 conn(0x7f9d500cb170 msgr2=0x7f9d500cb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:27.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.782+0000 7f9d58109640 1 -- 172.21.15.112:0/3854618497 shutdown_connections 2024-04-25T01:30:27.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.782+0000 7f9d58109640 1 -- 172.21.15.112:0/3854618497 wait complete. 2024-04-25T01:30:27.778 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:30:27.778 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:30:27.779 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:30:27.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.911+0000 7fe2c7d90640 1 Processor -- start 2024-04-25T01:30:27.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.911+0000 7fe2c7d90640 1 -- start start 2024-04-25T01:30:27.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.912+0000 7fe2c7d90640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c01596a0 0x7fe2c0152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.912+0000 7fe2c7d90640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2c0159b90 con 0x7fe2c00c72b0 2024-04-25T01:30:27.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.912+0000 7fe2c7d90640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2c006c970 con 0x7fe2c01592c0 2024-04-25T01:30:27.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.912+0000 7fe2c7d90640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c00c6e20 con 0x7fe2c01596a0 2024-04-25T01:30:27.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.912+0000 7fe2c632d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe2c00c72b0 0x7fe2c0160380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:35732/0 (socket says 172.21.15.112:35732) 2024-04-25T01:30:27.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.912+0000 7fe2c632d640 1 -- 172.21.15.112:0/2430059919 learned_addr learned my addr 172.21.15.112:0/2430059919 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:30:27.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.912+0000 7fe2c532b640 1 --2- 172.21.15.112:0/2430059919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c01596a0 0x7fe2c0152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.913+0000 7fe2c4b2a640 1 -- 172.21.15.112:0/2430059919 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 53665790 0 0) 0x7fe2c0159b90 con 0x7fe2c00c72b0 2024-04-25T01:30:27.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.913+0000 7fe2c4b2a640 1 -- 172.21.15.112:0/2430059919 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2ac003520 con 0x7fe2c00c72b0 2024-04-25T01:30:27.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.913+0000 7fe2c4b2a640 1 -- 172.21.15.112:0/2430059919 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1106698651 0 0) 0x7fe2c006c970 con 0x7fe2c01592c0 2024-04-25T01:30:27.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.913+0000 7fe2c4b2a640 1 -- 172.21.15.112:0/2430059919 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2c0159b90 con 0x7fe2c01592c0 2024-04-25T01:30:27.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.913+0000 7fe2c532b640 1 -- 172.21.15.112:0/2430059919 >> v1:172.21.15.187:6789/0 conn(0x7fe2c01592c0 legacy=0x7fe2c00c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.913+0000 7fe2c532b640 1 -- 172.21.15.112:0/2430059919 >> v1:172.21.15.112:6789/0 conn(0x7fe2c00c72b0 legacy=0x7fe2c0160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.913+0000 7fe2c532b640 1 -- 172.21.15.112:0/2430059919 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c006cc50 con 0x7fe2c01596a0 2024-04-25T01:30:27.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.913+0000 7fe2c532b640 1 --2- 172.21.15.112:0/2430059919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c01596a0 0x7fe2c0152d00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00097d0 tx=0x7fe2b002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a302d28390a2a51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:27.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.914+0000 7fe2c4b2a640 1 -- 172.21.15.112:0/2430059919 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0037070 con 0x7fe2c01596a0 2024-04-25T01:30:27.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.914+0000 7fe2c4b2a640 1 -- 172.21.15.112:0/2430059919 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2b002ae50 con 0x7fe2c01596a0 2024-04-25T01:30:27.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.914+0000 7fe2c4b2a640 1 -- 172.21.15.112:0/2430059919 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0032640 con 0x7fe2c01596a0 2024-04-25T01:30:27.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.914+0000 7fe2c7d90640 1 -- 172.21.15.112:0/2430059919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c01596a0 msgr2=0x7fe2c0152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.914+0000 7fe2c7d90640 1 --2- 172.21.15.112:0/2430059919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c01596a0 0x7fe2c0152d00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00097d0 tx=0x7fe2b002a100 comp rx=0 tx=0).stop 2024-04-25T01:30:27.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.914+0000 7fe2c7d90640 1 -- 172.21.15.112:0/2430059919 shutdown_connections 2024-04-25T01:30:27.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.914+0000 7fe2c7d90640 1 --2- 172.21.15.112:0/2430059919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c01596a0 0x7fe2c0152d00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.914+0000 7fe2c7d90640 1 -- 172.21.15.112:0/2430059919 >> 172.21.15.112:0/2430059919 conn(0x7fe2c00cb170 msgr2=0x7fe2c00cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:27.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.914+0000 7fe2c7d90640 1 -- 172.21.15.112:0/2430059919 shutdown_connections 2024-04-25T01:30:27.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.914+0000 7fe2c7d90640 1 -- 172.21.15.112:0/2430059919 wait complete. 2024-04-25T01:30:27.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.916+0000 7fe2c7d90640 1 Processor -- start 2024-04-25T01:30:27.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.916+0000 7fe2c7d90640 1 -- start start 2024-04-25T01:30:27.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.916+0000 7fe2c7d90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c00c72b0 0x7fe2c0153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.916+0000 7fe2c7d90640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2c01592c0 0x7fe2c0157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.916+0000 7fe2c7d90640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c01596a0 0x7fe2c014c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.916+0000 7fe2c7d90640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c006c510 con 0x7fe2c00c72b0 2024-04-25T01:30:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.916+0000 7fe2c7d90640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c006c390 con 0x7fe2c01592c0 2024-04-25T01:30:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.916+0000 7fe2c7d90640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c006db20 con 0x7fe2c01596a0 2024-04-25T01:30:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.916+0000 7fe2c5b2c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c00c72b0 0x7fe2c0153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.917+0000 7fe2c5b2c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c00c72b0 0x7fe2c0153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50780/0 (socket says 172.21.15.112:50780) 2024-04-25T01:30:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.917+0000 7fe2c5b2c640 1 -- 172.21.15.112:0/1107241315 learned_addr learned my addr 172.21.15.112:0/1107241315 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.917+0000 7fe2c532b640 1 --2- 172.21.15.112:0/1107241315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2c01592c0 0x7fe2c0157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.917+0000 7fe2c632d640 1 --2- 172.21.15.112:0/1107241315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c01596a0 0x7fe2c014c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.917+0000 7fe2c5b2c640 1 -- 172.21.15.112:0/1107241315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c01596a0 msgr2=0x7fe2c014c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.917+0000 7fe2c5b2c640 1 --2- 172.21.15.112:0/1107241315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c01596a0 0x7fe2c014c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.917+0000 7fe2c5b2c640 1 -- 172.21.15.112:0/1107241315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2c01592c0 msgr2=0x7fe2c0157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.917+0000 7fe2c5b2c640 1 --2- 172.21.15.112:0/1107241315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2c01592c0 0x7fe2c0157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.917+0000 7fe2c5b2c640 1 -- 172.21.15.112:0/1107241315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c0154380 con 0x7fe2c00c72b0 2024-04-25T01:30:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.917+0000 7fe2c5b2c640 1 --2- 172.21.15.112:0/1107241315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c00c72b0 0x7fe2c0153e40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc00c500 tx=0x7fe2bc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.917+0000 7fe2b67fc640 1 -- 172.21.15.112:0/1107241315 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc0052d0 con 0x7fe2c00c72b0 2024-04-25T01:30:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.917+0000 7fe2b67fc640 1 -- 172.21.15.112:0/1107241315 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2bc005490 con 0x7fe2c00c72b0 2024-04-25T01:30:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.917+0000 7fe2c7d90640 1 -- 172.21.15.112:0/1107241315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c0156560 con 0x7fe2c00c72b0 2024-04-25T01:30:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.917+0000 7fe2b67fc640 1 -- 172.21.15.112:0/1107241315 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc012870 con 0x7fe2c00c72b0 2024-04-25T01:30:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.917+0000 7fe2c7d90640 1 -- 172.21.15.112:0/1107241315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c0153b60 con 0x7fe2c00c72b0 2024-04-25T01:30:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.918+0000 7fe2c7d90640 1 --2- 172.21.15.112:0/1107241315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c00c1750 0x7fe2c014c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.918+0000 7fe2c7d90640 1 -- 172.21.15.112:0/1107241315 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c0075980 con 0x7fe2c00c1750 2024-04-25T01:30:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.918+0000 7fe2c532b640 1 --2- 172.21.15.112:0/1107241315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c00c1750 0x7fe2c014c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.919+0000 7fe2c532b640 1 -- 172.21.15.112:0/1107241315 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe2c00679a0 con 0x7fe2c00c1750 2024-04-25T01:30:27.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.919+0000 7fe2c532b640 1 --2- 172.21.15.112:0/1107241315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c00c1750 0x7fe2c014c920 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00aac40 tx=0x7fe2b00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:27.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.919+0000 7fe2b67fc640 1 -- 172.21.15.112:0/1107241315 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe2bc012a10 con 0x7fe2c00c72b0 2024-04-25T01:30:27.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.919+0000 7fe2b67fc640 1 --2- 172.21.15.112:0/1107241315 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe2a406be30 0x7fe2a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.919+0000 7fe2b67fc640 1 -- 172.21.15.112:0/1107241315 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fe2bc08ad60 con 0x7fe2c00c72b0 2024-04-25T01:30:27.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.919+0000 7fe2c632d640 1 --2- 172.21.15.112:0/1107241315 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe2a406be30 0x7fe2a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.920+0000 7fe2b67fc640 1 -- 172.21.15.112:0/1107241315 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0037040 con 0x7fe2c00c1750 2024-04-25T01:30:27.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.920+0000 7fe2b67fc640 1 -- 172.21.15.112:0/1107241315 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe2b0032c20 con 0x7fe2c00c1750 2024-04-25T01:30:27.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.920+0000 7fe2b67fc640 1 -- 172.21.15.112:0/1107241315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c00c1750 msgr2=0x7fe2c014c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.920+0000 7fe2b67fc640 1 --2- 172.21.15.112:0/1107241315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c00c1750 0x7fe2c014c920 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00aac40 tx=0x7fe2b00046c0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.920+0000 7fe2b67fc640 1 -- 172.21.15.112:0/1107241315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c00c1750 msgr2=0x7fe2c014c920 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:30:27.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.920+0000 7fe2b67fc640 1 --2- 172.21.15.112:0/1107241315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c00c1750 0x7fe2c014c920 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.920+0000 7fe2c632d640 1 --2- 172.21.15.112:0/1107241315 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe2a406be30 0x7fe2a406e2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8005c00 tx=0x7fe2b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:27.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.926+0000 7fe2c7d90640 1 --2- 172.21.15.112:0/1107241315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c01ebb50 0x7fe2c01ebf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:27.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.926+0000 7fe2c7d90640 1 -- 172.21.15.112:0/1107241315 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c0153950 con 0x7fe2c01ebb50 2024-04-25T01:30:27.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.926+0000 7fe2c532b640 1 --2- 172.21.15.112:0/1107241315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c01ebb50 0x7fe2c01ebf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:27.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.926+0000 7fe2c532b640 1 -- 172.21.15.112:0/1107241315 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe2c00682a0 con 0x7fe2c01ebb50 2024-04-25T01:30:27.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.927+0000 7fe2c532b640 1 --2- 172.21.15.112:0/1107241315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c01ebb50 0x7fe2c01ebf30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe2b002a140 tx=0x7fe2b0008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:27.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.927+0000 7fe2b67fc640 1 -- 172.21.15.112:0/1107241315 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b004d070 con 0x7fe2c01ebb50 2024-04-25T01:30:27.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.927+0000 7fe2b67fc640 1 -- 172.21.15.112:0/1107241315 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4559 (secure 0 0 0) 0x7fe2b0051450 con 0x7fe2c01ebb50 2024-04-25T01:30:27.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.927+0000 7fe2b67fc640 1 -- 172.21.15.112:0/1107241315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c01ebb50 msgr2=0x7fe2c01ebf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.927+0000 7fe2b67fc640 1 --2- 172.21.15.112:0/1107241315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c01ebb50 0x7fe2c01ebf30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe2b002a140 tx=0x7fe2b0008030 comp rx=0 tx=0).stop 2024-04-25T01:30:27.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.927+0000 7fe2b67fc640 1 -- 172.21.15.112:0/1107241315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c01ebb50 msgr2=0x7fe2c01ebf30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:30:27.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.927+0000 7fe2b67fc640 1 --2- 172.21.15.112:0/1107241315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2c01ebb50 0x7fe2c01ebf30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.907 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:30:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:30:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:30:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:30:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 32, 2024-04-25T01:30:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 16667, 2024-04-25T01:30:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:30:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:30:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:30:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:30:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 16, 2024-04-25T01:30:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:30:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:30:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:30:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:30:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:30:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:30:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:30:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:30:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:30:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:30:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:30:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:30:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:30:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:30:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:30:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:30:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:30:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:30:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:30:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:30:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:30:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:30:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:30:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:30:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:30:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:30:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:30:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:30:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:30:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:30:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:30:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:30:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:30:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:30:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:30:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:30:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:30:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:30:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:30:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:30:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:30:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:30:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:30:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:30:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:30:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:30:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:30:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:30:27.919 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:30:27.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.927+0000 7fe2c7d90640 1 -- 172.21.15.112:0/1107241315 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe2a406be30 msgr2=0x7fe2a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.927+0000 7fe2c7d90640 1 --2- 172.21.15.112:0/1107241315 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe2a406be30 0x7fe2a406e2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8005c00 tx=0x7fe2b8009040 comp rx=0 tx=0).stop 2024-04-25T01:30:27.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.927+0000 7fe2c7d90640 1 -- 172.21.15.112:0/1107241315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c00c72b0 msgr2=0x7fe2c0153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:27.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.927+0000 7fe2c7d90640 1 --2- 172.21.15.112:0/1107241315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c00c72b0 0x7fe2c0153e40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc00c500 tx=0x7fe2bc00c9c0 comp rx=0 tx=0).stop 2024-04-25T01:30:27.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.927+0000 7fe2c632d640 1 -- 172.21.15.112:0/1107241315 reap_dead start 2024-04-25T01:30:27.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.927+0000 7fe2c632d640 1 -- 172.21.15.112:0/1107241315 reap_dead start 2024-04-25T01:30:27.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.927+0000 7fe2c7d90640 1 -- 172.21.15.112:0/1107241315 shutdown_connections 2024-04-25T01:30:27.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.927+0000 7fe2c7d90640 1 -- 172.21.15.112:0/1107241315 >> 172.21.15.112:0/1107241315 conn(0x7fe2c00cb170 msgr2=0x7fe2c00ab440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:27.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.927+0000 7fe2c7d90640 1 -- 172.21.15.112:0/1107241315 shutdown_connections 2024-04-25T01:30:27.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:27.928+0000 7fe2c7d90640 1 -- 172.21.15.112:0/1107241315 wait complete. 2024-04-25T01:30:27.922 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:30:27.922 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-04-25T01:30:27.922 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-25T01:30:27.922 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-04-25T01:30:27.922 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-04-25T01:30:27.923 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T01:30:27.923 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:30:27.938 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T01:30:27.938 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T01:30:27.938 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:30:28.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.108+0000 7f041a0d4640 1 Processor -- start 2024-04-25T01:30:28.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.108+0000 7f041a0d4640 1 -- start start 2024-04-25T01:30:28.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.109+0000 7f041a0d4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f041414fbe0 0x7f041406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:28.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.109+0000 7f041a0d4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f04140746a0 0x7f0414152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:28.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.109+0000 7f041a0d4640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04140c6a70 con 0x7f04140c7250 2024-04-25T01:30:28.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.109+0000 7f041a0d4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f0414155ce0 con 0x7f04140746a0 2024-04-25T01:30:28.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.109+0000 7f041a0d4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0414155fc0 con 0x7f041414fbe0 2024-04-25T01:30:28.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.109+0000 7f0413fff640 1 -- >> v2:172.21.15.112:3300/0 conn(0x7f04140746a0 msgr2=0x7f0414152cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:30:28.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.109+0000 7f0413fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f04140746a0 0x7f0414152cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:28.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.109+0000 7f04137fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f041414fbe0 0x7f041406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:28.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.110+0000 7f04137fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f041414fbe0 0x7f041406c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:47586/0 (socket says 172.21.15.112:47586) 2024-04-25T01:30:28.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.110+0000 7f04137fe640 1 -- 172.21.15.112:0/3084427309 learned_addr learned my addr 172.21.15.112:0/3084427309 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:28.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.110+0000 7f04137fe640 1 -- 172.21.15.112:0/3084427309 >> v2:172.21.15.112:3300/0 conn(0x7f04140746a0 msgr2=0x7f0414152cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:30:28.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.110+0000 7f04137fe640 1 --2- 172.21.15.112:0/3084427309 >> v2:172.21.15.112:3300/0 conn(0x7f04140746a0 0x7f0414152cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:28.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.110+0000 7f04137fe640 1 -- 172.21.15.112:0/3084427309 >> v1:172.21.15.187:6789/0 conn(0x7f04140c7250 legacy=0x7f04141525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:28.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.110+0000 7f04137fe640 1 -- 172.21.15.112:0/3084427309 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04141562a0 con 0x7f041414fbe0 2024-04-25T01:30:28.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.111+0000 7f04137fe640 1 --2- 172.21.15.112:0/3084427309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f041414fbe0 0x7f041406c170 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f041404fef0 tx=0x7f040402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85f52118a8cd6f88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:28.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.111+0000 7f04127fc640 1 -- 172.21.15.112:0/3084427309 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404037070 con 0x7f041414fbe0 2024-04-25T01:30:28.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.111+0000 7f04127fc640 1 -- 172.21.15.112:0/3084427309 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0404032440 con 0x7f041414fbe0 2024-04-25T01:30:28.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.111+0000 7f04127fc640 1 -- 172.21.15.112:0/3084427309 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404032810 con 0x7f041414fbe0 2024-04-25T01:30:28.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.111+0000 7f041a0d4640 1 -- 172.21.15.112:0/3084427309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f041414fbe0 msgr2=0x7f041406c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:28.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.111+0000 7f041a0d4640 1 --2- 172.21.15.112:0/3084427309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f041414fbe0 0x7f041406c170 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f041404fef0 tx=0x7f040402a180 comp rx=0 tx=0).stop 2024-04-25T01:30:28.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.111+0000 7f041a0d4640 1 -- 172.21.15.112:0/3084427309 shutdown_connections 2024-04-25T01:30:28.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.111+0000 7f041a0d4640 1 --2- 172.21.15.112:0/3084427309 >> v2:172.21.15.112:3300/0 conn(0x7f04140746a0 0x7f0414152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:28.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.111+0000 7f041a0d4640 1 --2- 172.21.15.112:0/3084427309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f041414fbe0 0x7f041406c170 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:28.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.111+0000 7f041a0d4640 1 -- 172.21.15.112:0/3084427309 >> 172.21.15.112:0/3084427309 conn(0x7f04140c42d0 msgr2=0x7f04140c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:28.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.111+0000 7f041a0d4640 1 -- 172.21.15.112:0/3084427309 shutdown_connections 2024-04-25T01:30:28.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.111+0000 7f041a0d4640 1 -- 172.21.15.112:0/3084427309 wait complete. 2024-04-25T01:30:28.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.113+0000 7f041a0d4640 1 Processor -- start 2024-04-25T01:30:28.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.113+0000 7f041a0d4640 1 -- start start 2024-04-25T01:30:28.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.114+0000 7f041a0d4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04140746a0 0x7f04141ea750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:28.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.114+0000 7f041a0d4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04140c7250 0x7f0414154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:28.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.114+0000 7f041a0d4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041414fbe0 0x7f0414154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:28.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.114+0000 7f041a0d4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f041414ffc0 con 0x7f041414fbe0 2024-04-25T01:30:28.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.114+0000 7f041a0d4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0414156860 con 0x7f04140c7250 2024-04-25T01:30:28.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.114+0000 7f041a0d4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0414153200 con 0x7f04140746a0 2024-04-25T01:30:28.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.114+0000 7f0413fff640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041414fbe0 msgr2=0x7f0414154790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:30:28.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.114+0000 7f0413fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041414fbe0 0x7f0414154790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:28.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.114+0000 7f04137fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04140746a0 0x7f04141ea750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:28.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.114+0000 7f0412ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04140c7250 0x7f0414154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:28.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.114+0000 7f04137fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04140746a0 0x7f04141ea750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:37054/0 (socket says 172.21.15.112:37054) 2024-04-25T01:30:28.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.114+0000 7f04137fe640 1 -- 172.21.15.112:0/2823655505 learned_addr learned my addr 172.21.15.112:0/2823655505 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:28.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.114+0000 7f0412ffd640 1 -- 172.21.15.112:0/2823655505 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04140746a0 msgr2=0x7f04141ea750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:28.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.114+0000 7f0412ffd640 1 --2- 172.21.15.112:0/2823655505 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04140746a0 0x7f04141ea750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:28.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.114+0000 7f0412ffd640 1 -- 172.21.15.112:0/2823655505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041414fbe0 msgr2=0x7f0414154790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:30:28.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.114+0000 7f0412ffd640 1 --2- 172.21.15.112:0/2823655505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041414fbe0 0x7f0414154790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:28.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.114+0000 7f0412ffd640 1 -- 172.21.15.112:0/2823655505 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04141eb1d0 con 0x7f04140c7250 2024-04-25T01:30:28.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.114+0000 7f04137fe640 1 --2- 172.21.15.112:0/2823655505 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04140746a0 0x7f04141ea750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:30:28.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.114+0000 7f0412ffd640 1 --2- 172.21.15.112:0/2823655505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04140c7250 0x7f0414154250 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f04140c6c40 tx=0x7f040800c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:28.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.115+0000 7f03fbfff640 1 -- 172.21.15.112:0/2823655505 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0408017070 con 0x7f04140c7250 2024-04-25T01:30:28.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.115+0000 7f03fbfff640 1 -- 172.21.15.112:0/2823655505 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04080052f0 con 0x7f04140c7250 2024-04-25T01:30:28.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.115+0000 7f041a0d4640 1 -- 172.21.15.112:0/2823655505 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04140aaf20 con 0x7f04140c7250 2024-04-25T01:30:28.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.115+0000 7f03fbfff640 1 -- 172.21.15.112:0/2823655505 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0408003850 con 0x7f04140c7250 2024-04-25T01:30:28.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.115+0000 7f041a0d4640 1 -- 172.21.15.112:0/2823655505 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0414154cd0 con 0x7f04140c7250 2024-04-25T01:30:28.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.116+0000 7f041a0d4640 1 -- 172.21.15.112:0/2823655505 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04140689b0 con 0x7f04140c7250 2024-04-25T01:30:28.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.116+0000 7f03fbfff640 1 -- 172.21.15.112:0/2823655505 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f040801a070 con 0x7f04140c7250 2024-04-25T01:30:28.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.117+0000 7f03fbfff640 1 --2- 172.21.15.112:0/2823655505 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f040006be10 0x7f040006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:28.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.117+0000 7f03fbfff640 1 -- 172.21.15.112:0/2823655505 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f040808a3e0 con 0x7f04140c7250 2024-04-25T01:30:28.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.117+0000 7f04137fe640 1 --2- 172.21.15.112:0/2823655505 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f040006be10 0x7f040006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:28.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.117+0000 7f04137fe640 1 --2- 172.21.15.112:0/2823655505 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f040006be10 0x7f040006e2d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f040402a1c0 tx=0x7f040402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:28.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.122+0000 7f03fbfff640 1 -- 172.21.15.112:0/2823655505 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f04080583a0 con 0x7f04140c7250 2024-04-25T01:30:28.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.447+0000 7f041a0d4640 1 -- 172.21.15.112:0/2823655505 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f04140c67e0 con 0x7f04140c7250 2024-04-25T01:30:28.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.447+0000 7f03fbfff640 1 -- 172.21.15.112:0/2823655505 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f040805c6a0 con 0x7f04140c7250 2024-04-25T01:30:28.426 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":32,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:30:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.449+0000 7f041a0d4640 1 -- 172.21.15.112:0/2823655505 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f040006be10 msgr2=0x7f040006e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.449+0000 7f041a0d4640 1 --2- 172.21.15.112:0/2823655505 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f040006be10 0x7f040006e2d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f040402a1c0 tx=0x7f040402e000 comp rx=0 tx=0).stop 2024-04-25T01:30:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.449+0000 7f041a0d4640 1 -- 172.21.15.112:0/2823655505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04140c7250 msgr2=0x7f0414154250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.449+0000 7f041a0d4640 1 --2- 172.21.15.112:0/2823655505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04140c7250 0x7f0414154250 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f04140c6c40 tx=0x7f040800c820 comp rx=0 tx=0).stop 2024-04-25T01:30:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.450+0000 7f041a0d4640 1 -- 172.21.15.112:0/2823655505 shutdown_connections 2024-04-25T01:30:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.450+0000 7f041a0d4640 1 --2- 172.21.15.112:0/2823655505 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f040006be10 0x7f040006e2d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.450+0000 7f041a0d4640 1 --2- 172.21.15.112:0/2823655505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041414fbe0 0x7f0414154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.450+0000 7f041a0d4640 1 --2- 172.21.15.112:0/2823655505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04140c7250 0x7f0414154250 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.450+0000 7f041a0d4640 1 --2- 172.21.15.112:0/2823655505 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04140746a0 0x7f04141ea750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.450+0000 7f041a0d4640 1 -- 172.21.15.112:0/2823655505 >> 172.21.15.112:0/2823655505 conn(0x7f04140c42d0 msgr2=0x7f04140c2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.450+0000 7f041a0d4640 1 -- 172.21.15.112:0/2823655505 shutdown_connections 2024-04-25T01:30:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:28.450+0000 7f041a0d4640 1 -- 172.21.15.112:0/2823655505 wait complete. 2024-04-25T01:30:31.444 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:30:31.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.574+0000 7f408545d640 1 Processor -- start 2024-04-25T01:30:31.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.574+0000 7f408545d640 1 -- start start 2024-04-25T01:30:31.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.575+0000 7f408545d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f40800c6d00 0x7f40800c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:31.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.575+0000 7f408545d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40800c77f0 0x7f40800c9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:31.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.575+0000 7f408545d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f408006c970 con 0x7f40801565e0 2024-04-25T01:30:31.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.575+0000 7f408545d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f40800c6b60 con 0x7f40800c6d00 2024-04-25T01:30:31.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.575+0000 7f408545d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f40800c63f0 con 0x7f40800c77f0 2024-04-25T01:30:31.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.575+0000 7f407f7fe640 1 -- >> v1:172.21.15.112:6789/0 conn(0x7f40801565e0 legacy=0x7f40800c9590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:30:31.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.575+0000 7f4077fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40800c77f0 0x7f40800c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:31.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.575+0000 7f407effd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f40800c6d00 0x7f40800c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:31.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.575+0000 7f4077fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40800c77f0 0x7f40800c9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:37066/0 (socket says 172.21.15.112:37066) 2024-04-25T01:30:31.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.575+0000 7f4077fff640 1 -- 172.21.15.112:0/1156216038 learned_addr learned my addr 172.21.15.112:0/1156216038 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:31.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.575+0000 7f407effd640 1 -- 172.21.15.112:0/1156216038 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40800c77f0 msgr2=0x7f40800c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:30:31.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.575+0000 7f407effd640 1 --2- 172.21.15.112:0/1156216038 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40800c77f0 0x7f40800c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:31.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.575+0000 7f407effd640 1 -- 172.21.15.112:0/1156216038 >> v1:172.21.15.112:6789/0 conn(0x7f40801565e0 legacy=0x7f40800c9590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:30:31.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.575+0000 7f407effd640 1 -- 172.21.15.112:0/1156216038 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408006cc50 con 0x7f40800c6d00 2024-04-25T01:30:31.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.575+0000 7f4077fff640 1 --2- 172.21.15.112:0/1156216038 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40800c77f0 0x7f40800c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:30:31.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.576+0000 7f407effd640 1 --2- 172.21.15.112:0/1156216038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40800c6d00 0x7f40800c70e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f40700097a0 tx=0x7f407002a010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80ef13741c124b54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:31.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.576+0000 7f407e7fc640 1 -- 172.21.15.112:0/1156216038 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070037070 con 0x7f40800c6d00 2024-04-25T01:30:31.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.576+0000 7f407e7fc640 1 -- 172.21.15.112:0/1156216038 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f407002ad20 con 0x7f40800c6d00 2024-04-25T01:30:31.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.576+0000 7f407e7fc640 1 -- 172.21.15.112:0/1156216038 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070032660 con 0x7f40800c6d00 2024-04-25T01:30:31.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.577+0000 7f408545d640 1 -- 172.21.15.112:0/1156216038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40800c6d00 msgr2=0x7f40800c70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:31.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.577+0000 7f408545d640 1 --2- 172.21.15.112:0/1156216038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40800c6d00 0x7f40800c70e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f40700097a0 tx=0x7f407002a010 comp rx=0 tx=0).stop 2024-04-25T01:30:31.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.577+0000 7f408545d640 1 -- 172.21.15.112:0/1156216038 shutdown_connections 2024-04-25T01:30:31.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.577+0000 7f408545d640 1 --2- 172.21.15.112:0/1156216038 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40800c77f0 0x7f40800c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:31.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.577+0000 7f408545d640 1 --2- 172.21.15.112:0/1156216038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40800c6d00 0x7f40800c70e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:31.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.577+0000 7f408545d640 1 -- 172.21.15.112:0/1156216038 >> 172.21.15.112:0/1156216038 conn(0x7f40800cb330 msgr2=0x7f40800cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:31.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.577+0000 7f408545d640 1 -- 172.21.15.112:0/1156216038 shutdown_connections 2024-04-25T01:30:31.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.577+0000 7f408545d640 1 -- 172.21.15.112:0/1156216038 wait complete. 2024-04-25T01:30:31.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.578+0000 7f408545d640 1 Processor -- start 2024-04-25T01:30:31.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.579+0000 7f408545d640 1 -- start start 2024-04-25T01:30:31.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.579+0000 7f408545d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40800c6d00 0x7f40801e0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.579+0000 7f408545d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40800c77f0 0x7f4080156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.579+0000 7f408545d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40801565e0 0x7f4080157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.579+0000 7f408545d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4080153430 con 0x7f40801565e0 2024-04-25T01:30:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.579+0000 7f408545d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f40800ca940 con 0x7f40800c77f0 2024-04-25T01:30:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.579+0000 7f408545d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f40801535b0 con 0x7f40800c6d00 2024-04-25T01:30:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.579+0000 7f407f7fe640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40801565e0 msgr2=0x7f4080157410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:30:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.579+0000 7f407f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40801565e0 0x7f4080157410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.580+0000 7f407effd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40800c6d00 0x7f40801e0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.580+0000 7f4077fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40800c77f0 0x7f4080156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.580+0000 7f407effd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40800c6d00 0x7f40801e0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:37076/0 (socket says 172.21.15.112:37076) 2024-04-25T01:30:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.580+0000 7f407effd640 1 -- 172.21.15.112:0/256853831 learned_addr learned my addr 172.21.15.112:0/256853831 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.580+0000 7f407effd640 1 -- 172.21.15.112:0/256853831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40800c77f0 msgr2=0x7f4080156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.580+0000 7f407effd640 1 --2- 172.21.15.112:0/256853831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40800c77f0 0x7f4080156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.580+0000 7f407effd640 1 -- 172.21.15.112:0/256853831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40801565e0 msgr2=0x7f4080157410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:30:31.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.580+0000 7f407effd640 1 --2- 172.21.15.112:0/256853831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40801565e0 0x7f4080157410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:31.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.580+0000 7f407effd640 1 -- 172.21.15.112:0/256853831 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40800c4b60 con 0x7f40800c6d00 2024-04-25T01:30:31.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.580+0000 7f4077fff640 1 --2- 172.21.15.112:0/256853831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40800c77f0 0x7f4080156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:30:31.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.580+0000 7f407effd640 1 --2- 172.21.15.112:0/256853831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40800c6d00 0x7f40801e0f30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4070000c00 tx=0x7f40700046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:31.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.580+0000 7f40777fe640 1 -- 172.21.15.112:0/256853831 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070041020 con 0x7f40800c6d00 2024-04-25T01:30:31.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.580+0000 7f40777fe640 1 -- 172.21.15.112:0/256853831 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f407003d620 con 0x7f40800c6d00 2024-04-25T01:30:31.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.580+0000 7f408545d640 1 -- 172.21.15.112:0/256853831 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4080159d50 con 0x7f40800c6d00 2024-04-25T01:30:31.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.580+0000 7f40777fe640 1 -- 172.21.15.112:0/256853831 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070037040 con 0x7f40800c6d00 2024-04-25T01:30:31.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.580+0000 7f408545d640 1 -- 172.21.15.112:0/256853831 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4080158850 con 0x7f40800c6d00 2024-04-25T01:30:31.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.582+0000 7f408545d640 1 -- 172.21.15.112:0/256853831 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40800682a0 con 0x7f40800c6d00 2024-04-25T01:30:31.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.582+0000 7f40777fe640 1 -- 172.21.15.112:0/256853831 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4070044020 con 0x7f40800c6d00 2024-04-25T01:30:31.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.582+0000 7f40777fe640 1 --2- 172.21.15.112:0/256853831 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f406006bde0 0x7f406006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:31.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.582+0000 7f4077fff640 1 --2- 172.21.15.112:0/256853831 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f406006bde0 0x7f406006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:31.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.582+0000 7f40777fe640 1 -- 172.21.15.112:0/256853831 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f40700ab550 con 0x7f40800c6d00 2024-04-25T01:30:31.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.583+0000 7f4077fff640 1 --2- 172.21.15.112:0/256853831 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f406006bde0 0x7f406006e2a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4068009630 tx=0x7f4068009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:31.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.588+0000 7f40777fe640 1 -- 172.21.15.112:0/256853831 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f407004b030 con 0x7f40800c6d00 2024-04-25T01:30:31.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.881+0000 7f408545d640 1 -- 172.21.15.112:0/256853831 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f40800bea20 con 0x7f40800c6d00 2024-04-25T01:30:31.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.881+0000 7f40777fe640 1 -- 172.21.15.112:0/256853831 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f4070079bf0 con 0x7f40800c6d00 2024-04-25T01:30:31.859 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":32,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:30:31.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.884+0000 7f408545d640 1 -- 172.21.15.112:0/256853831 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f406006bde0 msgr2=0x7f406006e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:31.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.884+0000 7f408545d640 1 --2- 172.21.15.112:0/256853831 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f406006bde0 0x7f406006e2a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4068009630 tx=0x7f4068009240 comp rx=0 tx=0).stop 2024-04-25T01:30:31.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.884+0000 7f408545d640 1 -- 172.21.15.112:0/256853831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40800c6d00 msgr2=0x7f40801e0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:31.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.884+0000 7f408545d640 1 --2- 172.21.15.112:0/256853831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40800c6d00 0x7f40801e0f30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4070000c00 tx=0x7f40700046c0 comp rx=0 tx=0).stop 2024-04-25T01:30:31.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.884+0000 7f408545d640 1 -- 172.21.15.112:0/256853831 shutdown_connections 2024-04-25T01:30:31.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.884+0000 7f408545d640 1 --2- 172.21.15.112:0/256853831 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f406006bde0 0x7f406006e2a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.884+0000 7f408545d640 1 --2- 172.21.15.112:0/256853831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40801565e0 0x7f4080157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.884+0000 7f408545d640 1 --2- 172.21.15.112:0/256853831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40800c77f0 0x7f4080156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.884+0000 7f408545d640 1 --2- 172.21.15.112:0/256853831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40800c6d00 0x7f40801e0f30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.884+0000 7f408545d640 1 -- 172.21.15.112:0/256853831 >> 172.21.15.112:0/256853831 conn(0x7f40800cb330 msgr2=0x7f40800cb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.884+0000 7f408545d640 1 -- 172.21.15.112:0/256853831 shutdown_connections 2024-04-25T01:30:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:31.884+0000 7f408545d640 1 -- 172.21.15.112:0/256853831 wait complete. 2024-04-25T01:30:34.873 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:30:34.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:34.999+0000 7f1f5d2f5640 1 Processor -- start 2024-04-25T01:30:34.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:34.999+0000 7f1f5d2f5640 1 -- start start 2024-04-25T01:30:34.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:35.000+0000 7f1f5d2f5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1f580c9b80 0x7f1f5806bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:34.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:35.000+0000 7f1f5d2f5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f580c90e0 con 0x7f1f58152fd0 2024-04-25T01:30:34.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:35.000+0000 7f1f5d2f5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f580c9f90 con 0x7f1f580c5d70 2024-04-25T01:30:34.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:35.000+0000 7f1f5d2f5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f1f58157400 con 0x7f1f580c9b80 2024-04-25T01:30:34.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:35.000+0000 7f1f577fe640 1 -- >> v2:172.21.15.112:3300/0 conn(0x7f1f580c9b80 msgr2=0x7f1f5806bda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:30:34.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:35.000+0000 7f1f56ffd640 1 -- >> v1:172.21.15.112:6789/0 conn(0x7f1f58152fd0 legacy=0x7f1f58156280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:30:34.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:35.000+0000 7f1f577fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1f580c9b80 0x7f1f5806bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:34.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:35.000+0000 7f1f4ffff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f1f580c5d70 0x7f1f58156cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:35188/0 (socket says 172.21.15.112:35188) 2024-04-25T01:30:34.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:35.000+0000 7f1f4ffff640 1 -- 172.21.15.112:0/4146916153 learned_addr learned my addr 172.21.15.112:0/4146916153 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:30:35.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:35.201+0000 7f1f56ffd640 1 -- 172.21.15.112:0/4146916153 >> v1:172.21.15.112:6789/0 conn(0x7f1f58152fd0 legacy=0x7f1f58156280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:30:35.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:35.201+0000 7f1f577fe640 1 -- 172.21.15.112:0/4146916153 >> v2:172.21.15.112:3300/0 conn(0x7f1f580c9b80 msgr2=0x7f1f5806bda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:30:35.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:35.201+0000 7f1f577fe640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.112:3300/0 conn(0x7f1f580c9b80 0x7f1f5806bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:30:35.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:35.602+0000 7f1f56ffd640 1 -- 172.21.15.112:0/4146916153 >> v1:172.21.15.112:6789/0 conn(0x7f1f58152fd0 legacy=0x7f1f58156280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:30:35.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:35.602+0000 7f1f577fe640 1 -- 172.21.15.112:0/4146916153 >> v2:172.21.15.112:3300/0 conn(0x7f1f580c9b80 msgr2=0x7f1f5806bda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:30:35.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:35.602+0000 7f1f577fe640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.112:3300/0 conn(0x7f1f580c9b80 0x7f1f5806bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:30:36.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:36.403+0000 7f1f56ffd640 1 -- 172.21.15.112:0/4146916153 >> v1:172.21.15.112:6789/0 conn(0x7f1f58152fd0 legacy=0x7f1f58156280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:30:36.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:36.403+0000 7f1f577fe640 1 -- 172.21.15.112:0/4146916153 >> v2:172.21.15.112:3300/0 conn(0x7f1f580c9b80 msgr2=0x7f1f5806bda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:30:36.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:36.403+0000 7f1f577fe640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.112:3300/0 conn(0x7f1f580c9b80 0x7f1f5806bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:30:37.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.000+0000 7f1f557fa640 1 -- 172.21.15.112:0/4146916153 >> v2:172.21.15.112:3300/0 conn(0x7f1f580c9b80 msgr2=0x7f1f5806bda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:30:37.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.000+0000 7f1f557fa640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.112:3300/0 conn(0x7f1f580c9b80 0x7f1f5806bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:37.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.000+0000 7f1f557fa640 1 -- 172.21.15.112:0/4146916153 >> v1:172.21.15.187:6789/0 conn(0x7f1f580c5d70 legacy=0x7f1f58156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:37.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.000+0000 7f1f557fa640 1 -- 172.21.15.112:0/4146916153 >> v1:172.21.15.112:6789/0 conn(0x7f1f58152fd0 legacy=0x7f1f58156280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:30:37.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.000+0000 7f1f557fa640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 0x7f1f38005b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:37.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.000+0000 7f1f557fa640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 0x7f1f38008630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:37.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.000+0000 7f1f557fa640 1 -- 172.21.15.112:0/4146916153 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1f58157400 con 0x7f1f38006190 2024-04-25T01:30:37.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.000+0000 7f1f557fa640 1 -- 172.21.15.112:0/4146916153 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1f38008cf0 con 0x7f1f380036b0 2024-04-25T01:30:37.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.000+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 0x7f1f38008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:37.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.000+0000 7f1f56ffd640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 0x7f1f38005b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:37.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.000+0000 7f1f4ffff640 1 -- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 msgr2=0x7f1f38008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:30:37.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.000+0000 7f1f4ffff640 1 -- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 msgr2=0x7f1f38008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:30:37.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.000+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 0x7f1f38008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:37.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.000+0000 7f1f56ffd640 1 -- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 msgr2=0x7f1f38005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:30:37.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.000+0000 7f1f56ffd640 1 -- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 msgr2=0x7f1f38005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:30:37.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.000+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 0x7f1f38008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:37.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.000+0000 7f1f56ffd640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 0x7f1f38005b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:37.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.000+0000 7f1f56ffd640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 0x7f1f38005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:38.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.201+0000 7f1f56ffd640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 0x7f1f38005b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:38.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.201+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 0x7f1f38008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:38.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.201+0000 7f1f56ffd640 1 -- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 msgr2=0x7f1f38005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:30:38.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.201+0000 7f1f56ffd640 1 -- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 msgr2=0x7f1f38005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:30:38.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.201+0000 7f1f56ffd640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 0x7f1f38005b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:38.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.201+0000 7f1f4ffff640 1 -- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 msgr2=0x7f1f38008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:30:38.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.201+0000 7f1f4ffff640 1 -- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 msgr2=0x7f1f38008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:30:38.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.201+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 0x7f1f38008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:38.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.201+0000 7f1f56ffd640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 0x7f1f38005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:30:38.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.201+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 0x7f1f38008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:30:38.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.602+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 0x7f1f38008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:38.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.602+0000 7f1f56ffd640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 0x7f1f38005b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:38.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.602+0000 7f1f4ffff640 1 -- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 msgr2=0x7f1f38008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:30:38.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.602+0000 7f1f4ffff640 1 -- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 msgr2=0x7f1f38008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:30:38.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.602+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 0x7f1f38008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:38.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.602+0000 7f1f56ffd640 1 -- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 msgr2=0x7f1f38005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:30:38.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.602+0000 7f1f56ffd640 1 -- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 msgr2=0x7f1f38005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:30:38.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.602+0000 7f1f56ffd640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 0x7f1f38005b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:38.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.602+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 0x7f1f38008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:30:38.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:38.602+0000 7f1f56ffd640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 0x7f1f38005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:30:39.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:39.403+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 0x7f1f38008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:39.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:39.403+0000 7f1f56ffd640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 0x7f1f38005b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:39.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:39.403+0000 7f1f56ffd640 1 -- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 msgr2=0x7f1f38005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:30:39.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:39.403+0000 7f1f4ffff640 1 -- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 msgr2=0x7f1f38008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:30:39.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:39.403+0000 7f1f56ffd640 1 -- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 msgr2=0x7f1f38005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:30:39.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:39.403+0000 7f1f4ffff640 1 -- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 msgr2=0x7f1f38008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:30:39.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:39.403+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 0x7f1f38008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:39.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:39.403+0000 7f1f56ffd640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 0x7f1f38005b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:39.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:39.403+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 0x7f1f38008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:30:39.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:39.403+0000 7f1f56ffd640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 0x7f1f38005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:30:40.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.000+0000 7f1f557fa640 1 -- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 msgr2=0x7f1f38005b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:30:40.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.000+0000 7f1f557fa640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f380036b0 0x7f1f38005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:40.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.000+0000 7f1f557fa640 1 -- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 msgr2=0x7f1f38008630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:30:40.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.000+0000 7f1f557fa640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38006190 0x7f1f38008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:40.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.000+0000 7f1f577fe640 1 -- 172.21.15.112:0/4146916153 reap_dead start 2024-04-25T01:30:40.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.000+0000 7f1f557fa640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 0x7f1f38006620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:40.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.000+0000 7f1f557fa640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.112:3300/0 conn(0x7f1f38006b60 0x7f1f38006fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:40.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.000+0000 7f1f557fa640 1 -- 172.21.15.112:0/4146916153 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f580cac00 con 0x7f1f58152fd0 2024-04-25T01:30:40.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.000+0000 7f1f557fa640 1 -- 172.21.15.112:0/4146916153 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f1f58157400 con 0x7f1f38006b60 2024-04-25T01:30:40.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.000+0000 7f1f56ffd640 1 -- 172.21.15.112:0/4146916153 >> v1:172.21.15.112:6789/0 conn(0x7f1f58152fd0 legacy=0x7f1f3800b160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:30:40.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.000+0000 7f1f557fa640 1 -- 172.21.15.112:0/4146916153 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1f38008cf0 con 0x7f1f38003af0 2024-04-25T01:30:40.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.000+0000 7f1f577fe640 1 -- 172.21.15.112:0/4146916153 >> v2:172.21.15.112:3300/0 conn(0x7f1f38006b60 msgr2=0x7f1f38006fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:30:40.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.000+0000 7f1f577fe640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.112:3300/0 conn(0x7f1f38006b60 0x7f1f38006fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:40.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.001+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 0x7f1f38006620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:40.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.001+0000 7f1f4ffff640 1 -- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 msgr2=0x7f1f38006620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:30:40.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.001+0000 7f1f4ffff640 1 -- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 msgr2=0x7f1f38006620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:30:40.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.001+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 0x7f1f38006620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:40.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.001+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 0x7f1f38006620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:41.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.201+0000 7f1f577fe640 1 -- 172.21.15.112:0/4146916153 >> v2:172.21.15.112:3300/0 conn(0x7f1f38006b60 msgr2=0x7f1f38006fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:30:41.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.201+0000 7f1f577fe640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.112:3300/0 conn(0x7f1f38006b60 0x7f1f38006fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:30:41.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.201+0000 7f1f56ffd640 1 -- 172.21.15.112:0/4146916153 >> v1:172.21.15.112:6789/0 conn(0x7f1f58152fd0 legacy=0x7f1f3800b160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:30:41.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.202+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 0x7f1f38006620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:41.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.202+0000 7f1f4ffff640 1 -- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 msgr2=0x7f1f38006620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:30:41.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.202+0000 7f1f4ffff640 1 -- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 msgr2=0x7f1f38006620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:30:41.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.202+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 0x7f1f38006620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:41.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.202+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 0x7f1f38006620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:30:41.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.602+0000 7f1f577fe640 1 -- 172.21.15.112:0/4146916153 >> v2:172.21.15.112:3300/0 conn(0x7f1f38006b60 msgr2=0x7f1f38006fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:30:41.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.602+0000 7f1f56ffd640 1 -- 172.21.15.112:0/4146916153 >> v1:172.21.15.112:6789/0 conn(0x7f1f58152fd0 legacy=0x7f1f3800b160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:30:41.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.602+0000 7f1f577fe640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.112:3300/0 conn(0x7f1f38006b60 0x7f1f38006fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:30:41.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.603+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 0x7f1f38006620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:41.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.603+0000 7f1f4ffff640 1 -- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 msgr2=0x7f1f38006620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:30:41.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.603+0000 7f1f4ffff640 1 -- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 msgr2=0x7f1f38006620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:30:41.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.603+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 0x7f1f38006620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:41.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:41.603+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 0x7f1f38006620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:30:42.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:42.403+0000 7f1f56ffd640 1 -- 172.21.15.112:0/4146916153 >> v1:172.21.15.112:6789/0 conn(0x7f1f58152fd0 legacy=0x7f1f3800b160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:30:42.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:42.403+0000 7f1f577fe640 1 -- 172.21.15.112:0/4146916153 >> v2:172.21.15.112:3300/0 conn(0x7f1f38006b60 msgr2=0x7f1f38006fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:30:42.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:42.403+0000 7f1f577fe640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.112:3300/0 conn(0x7f1f38006b60 0x7f1f38006fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:30:42.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:42.404+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 0x7f1f38006620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:42.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:42.404+0000 7f1f4ffff640 1 -- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 msgr2=0x7f1f38006620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:30:42.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:42.404+0000 7f1f4ffff640 1 -- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 msgr2=0x7f1f38006620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:30:42.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:42.404+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 0x7f1f38006620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:30:42.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:42.404+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 0x7f1f38006620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:30:43.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.001+0000 7f1f557fa640 1 -- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 msgr2=0x7f1f38006620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:30:43.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.001+0000 7f1f557fa640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f38003af0 0x7f1f38006620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:43.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.001+0000 7f1f557fa640 1 -- 172.21.15.112:0/4146916153 >> v2:172.21.15.112:3300/0 conn(0x7f1f38006b60 msgr2=0x7f1f38006fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:30:43.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.001+0000 7f1f557fa640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.112:3300/0 conn(0x7f1f38006b60 0x7f1f38006fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:43.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.001+0000 7f1f557fa640 1 -- 172.21.15.112:0/4146916153 >> v1:172.21.15.112:6789/0 conn(0x7f1f58152fd0 legacy=0x7f1f3800b160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:30:43.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.001+0000 7f1f557fa640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38008200 0x7f1f380138c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:43.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.001+0000 7f1f557fa640 1 -- 172.21.15.112:0/4146916153 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f580c90e0 con 0x7f1f380076e0 2024-04-25T01:30:43.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.001+0000 7f1f557fa640 1 -- 172.21.15.112:0/4146916153 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1f58157400 con 0x7f1f38008200 2024-04-25T01:30:43.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.001+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> v2:172.21.15.187:3300/0 conn(0x7f1f38008200 0x7f1f380138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:43.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.001+0000 7f1f4ffff640 1 -- 172.21.15.112:0/4146916153 >> v1:172.21.15.187:6789/0 conn(0x7f1f380076e0 legacy=0x7f1f38007af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:43.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.001+0000 7f1f4ffff640 1 -- 172.21.15.112:0/4146916153 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f580cac00 con 0x7f1f38008200 2024-04-25T01:30:43.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.002+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f38008200 0x7f1f380138c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1f580c6190 tx=0x7f1f4002c160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b71cf257be63de75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:43.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.002+0000 7f1f567fc640 1 -- 172.21.15.112:0/4146916153 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40032070 con 0x7f1f38008200 2024-04-25T01:30:43.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.002+0000 7f1f567fc640 1 -- 172.21.15.112:0/4146916153 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f4002c9e0 con 0x7f1f38008200 2024-04-25T01:30:43.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.002+0000 7f1f567fc640 1 -- 172.21.15.112:0/4146916153 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4002cdb0 con 0x7f1f38008200 2024-04-25T01:30:43.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.003+0000 7f1f5d2f5640 1 -- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f38008200 msgr2=0x7f1f380138c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:43.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.003+0000 7f1f5d2f5640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f38008200 0x7f1f380138c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1f580c6190 tx=0x7f1f4002c160 comp rx=0 tx=0).stop 2024-04-25T01:30:43.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.003+0000 7f1f577fe640 1 -- 172.21.15.112:0/4146916153 reap_dead start 2024-04-25T01:30:43.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.003+0000 7f1f5d2f5640 1 -- 172.21.15.112:0/4146916153 shutdown_connections 2024-04-25T01:30:43.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.003+0000 7f1f5d2f5640 1 --2- 172.21.15.112:0/4146916153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f38008200 0x7f1f380138c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:43.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.003+0000 7f1f5d2f5640 1 -- 172.21.15.112:0/4146916153 >> 172.21.15.112:0/4146916153 conn(0x7f1f580acc70 msgr2=0x7f1f580ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:43.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.003+0000 7f1f5d2f5640 1 -- 172.21.15.112:0/4146916153 shutdown_connections 2024-04-25T01:30:43.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.003+0000 7f1f5d2f5640 1 -- 172.21.15.112:0/4146916153 wait complete. 2024-04-25T01:30:43.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.004+0000 7f1f5d2f5640 1 Processor -- start 2024-04-25T01:30:43.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.004+0000 7f1f5d2f5640 1 -- start start 2024-04-25T01:30:43.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.005+0000 7f1f5d2f5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f38008200 0x7f1f580c1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:43.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.005+0000 7f1f5d2f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f581540a0 0x7f1f580bfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:43.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.005+0000 7f1f5d2f5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f580c0110 0x7f1f5814c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:43.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.005+0000 7f1f5d2f5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1f580c9660 con 0x7f1f581540a0 2024-04-25T01:30:43.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.005+0000 7f1f5d2f5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1f580ca9b0 con 0x7f1f38008200 2024-04-25T01:30:43.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.005+0000 7f1f5d2f5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1f580caee0 con 0x7f1f580c0110 2024-04-25T01:30:43.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.005+0000 7f1f4ffff640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f581540a0 msgr2=0x7f1f580bfbd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:30:43.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.005+0000 7f1f4ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f581540a0 0x7f1f580bfbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:43.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.005+0000 7f1f56ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f38008200 0x7f1f580c1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:43.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.005+0000 7f1f56ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f38008200 0x7f1f580c1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40328/0 (socket says 172.21.15.112:40328) 2024-04-25T01:30:43.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.005+0000 7f1f56ffd640 1 -- 172.21.15.112:0/3369987837 learned_addr learned my addr 172.21.15.112:0/3369987837 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:43.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.005+0000 7f1f577fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f580c0110 0x7f1f5814c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:43.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.006+0000 7f1f56ffd640 1 -- 172.21.15.112:0/3369987837 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f580c0110 msgr2=0x7f1f5814c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:43.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.006+0000 7f1f56ffd640 1 --2- 172.21.15.112:0/3369987837 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f580c0110 0x7f1f5814c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:43.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.006+0000 7f1f56ffd640 1 -- 172.21.15.112:0/3369987837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f581540a0 msgr2=0x7f1f580bfbd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:30:43.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.006+0000 7f1f56ffd640 1 --2- 172.21.15.112:0/3369987837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f581540a0 0x7f1f580bfbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:43.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.006+0000 7f1f56ffd640 1 -- 172.21.15.112:0/3369987837 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f580bee10 con 0x7f1f38008200 2024-04-25T01:30:43.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.006+0000 7f1f56ffd640 1 --2- 172.21.15.112:0/3369987837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f38008200 0x7f1f580c1590 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f480056d0 tx=0x7f1f48005b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:43.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.006+0000 7f1f4f7fe640 1 -- 172.21.15.112:0/3369987837 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48009ea0 con 0x7f1f38008200 2024-04-25T01:30:43.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.006+0000 7f1f4f7fe640 1 -- 172.21.15.112:0/3369987837 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f4801c070 con 0x7f1f38008200 2024-04-25T01:30:43.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.006+0000 7f1f5d2f5640 1 -- 172.21.15.112:0/3369987837 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f580bf0f0 con 0x7f1f38008200 2024-04-25T01:30:43.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.006+0000 7f1f4f7fe640 1 -- 172.21.15.112:0/3369987837 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f480176c0 con 0x7f1f38008200 2024-04-25T01:30:43.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.006+0000 7f1f5d2f5640 1 -- 172.21.15.112:0/3369987837 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f580c1ad0 con 0x7f1f38008200 2024-04-25T01:30:43.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.007+0000 7f1f5d2f5640 1 -- 172.21.15.112:0/3369987837 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f580682a0 con 0x7f1f38008200 2024-04-25T01:30:43.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.008+0000 7f1f4f7fe640 1 -- 172.21.15.112:0/3369987837 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f48017860 con 0x7f1f38008200 2024-04-25T01:30:43.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.008+0000 7f1f4f7fe640 1 --2- 172.21.15.112:0/3369987837 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1f3806d9d0 0x7f1f3806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:43.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.008+0000 7f1f4f7fe640 1 -- 172.21.15.112:0/3369987837 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f1f4808c160 con 0x7f1f38008200 2024-04-25T01:30:43.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.008+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/3369987837 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1f3806d9d0 0x7f1f3806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:43.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.008+0000 7f1f4ffff640 1 --2- 172.21.15.112:0/3369987837 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1f3806d9d0 0x7f1f3806fe90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1f40037000 tx=0x7f1f40032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:43.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.013+0000 7f1f4f7fe640 1 -- 172.21.15.112:0/3369987837 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1f48012050 con 0x7f1f38008200 2024-04-25T01:30:44.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.257+0000 7f1f5d2f5640 1 -- 172.21.15.112:0/3369987837 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1f58150360 con 0x7f1f38008200 2024-04-25T01:30:44.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.257+0000 7f1f4f7fe640 1 -- 172.21.15.112:0/3369987837 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7f1f4805a760 con 0x7f1f38008200 2024-04-25T01:30:44.235 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":34,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:30:44.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.260+0000 7f1f5d2f5640 1 -- 172.21.15.112:0/3369987837 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1f3806d9d0 msgr2=0x7f1f3806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:44.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.260+0000 7f1f5d2f5640 1 --2- 172.21.15.112:0/3369987837 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1f3806d9d0 0x7f1f3806fe90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1f40037000 tx=0x7f1f40032000 comp rx=0 tx=0).stop 2024-04-25T01:30:44.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.260+0000 7f1f5d2f5640 1 -- 172.21.15.112:0/3369987837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f38008200 msgr2=0x7f1f580c1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:44.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.260+0000 7f1f5d2f5640 1 --2- 172.21.15.112:0/3369987837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f38008200 0x7f1f580c1590 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f480056d0 tx=0x7f1f48005b90 comp rx=0 tx=0).stop 2024-04-25T01:30:44.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.260+0000 7f1f5d2f5640 1 -- 172.21.15.112:0/3369987837 shutdown_connections 2024-04-25T01:30:44.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.260+0000 7f1f5d2f5640 1 --2- 172.21.15.112:0/3369987837 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1f3806d9d0 0x7f1f3806fe90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.260+0000 7f1f5d2f5640 1 --2- 172.21.15.112:0/3369987837 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f580c0110 0x7f1f5814c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.260+0000 7f1f5d2f5640 1 --2- 172.21.15.112:0/3369987837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f581540a0 0x7f1f580bfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.260+0000 7f1f5d2f5640 1 --2- 172.21.15.112:0/3369987837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f38008200 0x7f1f580c1590 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.260+0000 7f1f5d2f5640 1 -- 172.21.15.112:0/3369987837 >> 172.21.15.112:0/3369987837 conn(0x7f1f580acc70 msgr2=0x7f1f581522a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:44.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.260+0000 7f1f5d2f5640 1 -- 172.21.15.112:0/3369987837 shutdown_connections 2024-04-25T01:30:44.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.260+0000 7f1f5d2f5640 1 -- 172.21.15.112:0/3369987837 wait complete. 2024-04-25T01:30:44.248 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T01:30:44.248 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-04-25T01:30:44.248 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:30:44.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.370+0000 7ff32675c640 1 Processor -- start 2024-04-25T01:30:44.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.370+0000 7ff32675c640 1 -- start start 2024-04-25T01:30:44.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.370+0000 7ff32675c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff320072d80 0x7ff3201536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:44.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.370+0000 7ff32675c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff3200c1a80 0x7ff32014d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:44.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.370+0000 7ff32675c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff32006e0f0 con 0x7ff32014d880 2024-04-25T01:30:44.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.370+0000 7ff32675c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7ff3200c1f30 con 0x7ff320072d80 2024-04-25T01:30:44.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.370+0000 7ff32675c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff3200c2180 con 0x7ff3200c1a80 2024-04-25T01:30:44.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.370+0000 7ff31ffff640 1 -- >> v2:172.21.15.112:3300/0 conn(0x7ff320072d80 msgr2=0x7ff3201536c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:30:44.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.370+0000 7ff31ffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff320072d80 0x7ff3201536c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:44.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.371+0000 7ff31f7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff3200c1a80 0x7ff32014d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:44.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.371+0000 7ff31f7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff3200c1a80 0x7ff32014d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40330/0 (socket says 172.21.15.112:40330) 2024-04-25T01:30:44.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.371+0000 7ff31f7fe640 1 -- 172.21.15.112:0/3279220262 learned_addr learned my addr 172.21.15.112:0/3279220262 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:44.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.371+0000 7ff31f7fe640 1 -- 172.21.15.112:0/3279220262 >> v2:172.21.15.112:3300/0 conn(0x7ff320072d80 msgr2=0x7ff3201536c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:30:44.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.371+0000 7ff31f7fe640 1 --2- 172.21.15.112:0/3279220262 >> v2:172.21.15.112:3300/0 conn(0x7ff320072d80 0x7ff3201536c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.371+0000 7ff31f7fe640 1 -- 172.21.15.112:0/3279220262 >> v1:172.21.15.187:6789/0 conn(0x7ff32014d880 legacy=0x7ff320071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:44.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.371+0000 7ff31f7fe640 1 -- 172.21.15.112:0/3279220262 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3200c2460 con 0x7ff3200c1a80 2024-04-25T01:30:44.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.372+0000 7ff31f7fe640 1 --2- 172.21.15.112:0/3279220262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3200c1a80 0x7ff32014d340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff3200ac5e0 tx=0x7ff31402a290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5db71f424bef7773 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:44.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.372+0000 7ff31effd640 1 -- 172.21.15.112:0/3279220262 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314037070 con 0x7ff3200c1a80 2024-04-25T01:30:44.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.372+0000 7ff31effd640 1 -- 172.21.15.112:0/3279220262 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff314032440 con 0x7ff3200c1a80 2024-04-25T01:30:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.372+0000 7ff31effd640 1 -- 172.21.15.112:0/3279220262 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3140327f0 con 0x7ff3200c1a80 2024-04-25T01:30:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.372+0000 7ff32675c640 1 -- 172.21.15.112:0/3279220262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3200c1a80 msgr2=0x7ff32014d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.372+0000 7ff32675c640 1 --2- 172.21.15.112:0/3279220262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3200c1a80 0x7ff32014d340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff3200ac5e0 tx=0x7ff31402a290 comp rx=0 tx=0).stop 2024-04-25T01:30:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.372+0000 7ff32675c640 1 -- 172.21.15.112:0/3279220262 shutdown_connections 2024-04-25T01:30:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.372+0000 7ff32675c640 1 --2- 172.21.15.112:0/3279220262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3200c1a80 0x7ff32014d340 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.372+0000 7ff32675c640 1 --2- 172.21.15.112:0/3279220262 >> v2:172.21.15.112:3300/0 conn(0x7ff320072d80 0x7ff3201536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.372+0000 7ff32675c640 1 -- 172.21.15.112:0/3279220262 >> 172.21.15.112:0/3279220262 conn(0x7ff32006b160 msgr2=0x7ff32006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.372+0000 7ff32675c640 1 -- 172.21.15.112:0/3279220262 shutdown_connections 2024-04-25T01:30:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.372+0000 7ff32675c640 1 -- 172.21.15.112:0/3279220262 wait complete. 2024-04-25T01:30:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.373+0000 7ff32675c640 1 Processor -- start 2024-04-25T01:30:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.374+0000 7ff32675c640 1 -- start start 2024-04-25T01:30:44.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.374+0000 7ff32675c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff320072d80 0x7ff32014fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:44.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.374+0000 7ff32675c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3200c1a80 0x7ff320157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:44.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.374+0000 7ff32675c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff32014d880 0x7ff3201ed170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:44.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.374+0000 7ff32675c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff320071890 con 0x7ff3200c1a80 2024-04-25T01:30:44.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.374+0000 7ff32675c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff3200c2a20 con 0x7ff32014d880 2024-04-25T01:30:44.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.374+0000 7ff32675c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff320073160 con 0x7ff320072d80 2024-04-25T01:30:44.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.374+0000 7ff31f7fe640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3200c1a80 msgr2=0x7ff320157450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:30:44.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.374+0000 7ff31f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3200c1a80 0x7ff320157450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:44.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.375+0000 7ff324cf9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff32014d880 0x7ff3201ed170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:44.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.375+0000 7ff31ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff320072d80 0x7ff32014fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:44.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.375+0000 7ff324cf9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff32014d880 0x7ff3201ed170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40344/0 (socket says 172.21.15.112:40344) 2024-04-25T01:30:44.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.375+0000 7ff324cf9640 1 -- 172.21.15.112:0/1276829324 learned_addr learned my addr 172.21.15.112:0/1276829324 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:44.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.375+0000 7ff31ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff320072d80 0x7ff32014fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:53398/0 (socket says 172.21.15.112:53398) 2024-04-25T01:30:44.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.375+0000 7ff31ffff640 1 -- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff32014d880 msgr2=0x7ff3201ed170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:44.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.375+0000 7ff31ffff640 1 --2- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff32014d880 0x7ff3201ed170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.375+0000 7ff31ffff640 1 -- 172.21.15.112:0/1276829324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3200c1a80 msgr2=0x7ff320157450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:30:44.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.375+0000 7ff31ffff640 1 --2- 172.21.15.112:0/1276829324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3200c1a80 0x7ff320157450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.375+0000 7ff31ffff640 1 -- 172.21.15.112:0/1276829324 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3200c7480 con 0x7ff320072d80 2024-04-25T01:30:44.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.375+0000 7ff324cf9640 1 --2- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff32014d880 0x7ff3201ed170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:30:44.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.375+0000 7ff31ffff640 1 --2- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff320072d80 0x7ff32014fe30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff310003980 tx=0x7ff31000b9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:44.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.375+0000 7ff31cff9640 1 -- 172.21.15.112:0/1276829324 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff310003dc0 con 0x7ff320072d80 2024-04-25T01:30:44.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.375+0000 7ff31cff9640 1 -- 172.21.15.112:0/1276829324 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff310015070 con 0x7ff320072d80 2024-04-25T01:30:44.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.375+0000 7ff32675c640 1 -- 172.21.15.112:0/1276829324 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3200c7760 con 0x7ff320072d80 2024-04-25T01:30:44.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.375+0000 7ff31cff9640 1 -- 172.21.15.112:0/1276829324 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff310005290 con 0x7ff320072d80 2024-04-25T01:30:44.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.375+0000 7ff32675c640 1 -- 172.21.15.112:0/1276829324 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff320150370 con 0x7ff320072d80 2024-04-25T01:30:44.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.376+0000 7ff32675c640 1 --2- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff320150e30 0x7ff3201468b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:44.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.376+0000 7ff32675c640 1 -- 172.21.15.112:0/1276829324 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff320075980 con 0x7ff320150e30 2024-04-25T01:30:44.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.376+0000 7ff31f7fe640 1 --2- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff320150e30 0x7ff3201468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:44.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.377+0000 7ff31f7fe640 1 -- 172.21.15.112:0/1276829324 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff3200682a0 con 0x7ff320150e30 2024-04-25T01:30:44.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.377+0000 7ff31cff9640 1 -- 172.21.15.112:0/1276829324 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff31000f070 con 0x7ff320072d80 2024-04-25T01:30:44.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.377+0000 7ff31f7fe640 1 --2- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff320150e30 0x7ff3201468b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff320151290 tx=0x7ff3140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:44.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.377+0000 7ff31cff9640 1 --2- 172.21.15.112:0/1276829324 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff30c06bde0 0x7ff30c06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:44.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.377+0000 7ff31cff9640 1 -- 172.21.15.112:0/1276829324 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7ff31008aa00 con 0x7ff320072d80 2024-04-25T01:30:44.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.377+0000 7ff324cf9640 1 --2- 172.21.15.112:0/1276829324 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff30c06bde0 0x7ff30c06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:44.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.377+0000 7ff31cff9640 1 -- 172.21.15.112:0/1276829324 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314037040 con 0x7ff320150e30 2024-04-25T01:30:44.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.377+0000 7ff31cff9640 1 -- 172.21.15.112:0/1276829324 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff314004020 con 0x7ff320150e30 2024-04-25T01:30:44.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.377+0000 7ff31cff9640 1 -- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff320150e30 msgr2=0x7ff3201468b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:44.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.377+0000 7ff31cff9640 1 --2- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff320150e30 0x7ff3201468b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff320151290 tx=0x7ff3140046c0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.377+0000 7ff31cff9640 1 -- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff320150e30 msgr2=0x7ff3201468b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:30:44.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.377+0000 7ff31cff9640 1 --2- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff320150e30 0x7ff3201468b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.378+0000 7ff324cf9640 1 --2- 172.21.15.112:0/1276829324 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff30c06bde0 0x7ff30c06e2a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff31800f1d0 tx=0x7ff318009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:44.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.384+0000 7ff32675c640 1 --2- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff320157990 0x7ff320146df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:44.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.384+0000 7ff32675c640 1 -- 172.21.15.112:0/1276829324 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff3201ecaf0 con 0x7ff320157990 2024-04-25T01:30:44.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.384+0000 7ff31f7fe640 1 --2- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff320157990 0x7ff320146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:44.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.384+0000 7ff31f7fe640 1 -- 172.21.15.112:0/1276829324 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff32014c460 con 0x7ff320157990 2024-04-25T01:30:44.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.384+0000 7ff31f7fe640 1 --2- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff320157990 0x7ff320146df0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff314000c00 tx=0x7ff314009240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:44.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.384+0000 7ff31cff9640 1 -- 172.21.15.112:0/1276829324 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31404d070 con 0x7ff320157990 2024-04-25T01:30:44.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.385+0000 7ff31cff9640 1 -- 172.21.15.112:0/1276829324 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5059 (secure 0 0 0) 0x7ff31404b070 con 0x7ff320157990 2024-04-25T01:30:44.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.385+0000 7ff31cff9640 1 -- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff320157990 msgr2=0x7ff320146df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:44.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.385+0000 7ff31cff9640 1 --2- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff320157990 0x7ff320146df0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff314000c00 tx=0x7ff314009240 comp rx=0 tx=0).stop 2024-04-25T01:30:44.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.385+0000 7ff31cff9640 1 -- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff320157990 msgr2=0x7ff320146df0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:30:44.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.385+0000 7ff31cff9640 1 --2- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff320157990 0x7ff320146df0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.364 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:30:44.364 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:30:44.364 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:30:44.365 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:30:44.365 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 34, 2024-04-25T01:30:44.365 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 80074, 2024-04-25T01:30:44.365 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:30:44.365 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:30:44.365 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:30:44.365 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:44.365 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:30:44.365 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:30:44.365 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:30:44.365 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:30:44.365 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:44.366 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:44.366 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:44.366 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:44.366 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:44.366 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:44.366 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:44.366 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:44.366 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:44.366 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:44.366 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:44.366 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:44.366 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:30:44.367 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:30:44.367 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:44.367 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:44.367 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:44.367 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:44.367 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:44.367 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:44.367 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:44.367 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:44.367 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:44.367 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:44.367 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:44.367 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:44.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.367 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:30:44.368 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:30:44.368 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:30:44.368 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:30:44.368 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:30:44.368 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:30:44.368 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:30:44.368 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:30:44.368 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:30:44.368 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:30:44.368 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:30:44.368 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:30:44.368 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:30:44.368 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:30:44.368 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:30:44.368 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:30:44.369 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:30:44.369 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:44.369 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:44.369 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:44.369 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:44.369 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:44.369 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:44.369 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:44.369 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:44.369 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:44.369 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:44.369 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:44.369 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:44.369 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:30:44.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.370 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:30:44.370 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.370 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:30:44.370 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:30:44.370 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:44.370 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:44.370 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.370 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:44.370 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:30:44.370 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:44.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.370 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.370 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:44.371 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:30:44.371 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:44.371 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:44.371 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:44.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.371 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:30:44.371 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:30:44.371 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:44.371 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:44.371 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:44.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.371 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.371 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:30:44.371 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:30:44.371 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:44.372 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:44.372 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.372 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:44.372 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:30:44.372 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:44.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.372 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.372 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:44.372 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:30:44.372 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:44.372 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:44.372 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:44.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.372 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:30:44.373 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:30:44.373 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:44.373 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:44.373 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:44.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.373 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.373 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:30:44.373 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:30:44.373 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:44.373 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:44.373 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.373 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:44.373 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:30:44.373 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:44.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.374 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.374 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:44.374 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:30:44.374 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:44.374 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:44.374 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:44.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.374 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:30:44.374 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:30:44.374 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:44.374 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:44.374 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:44.374 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:44.375 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:44.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.375 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:30:44.375 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:30:44.375 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.375 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:44.375 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:44.375 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:30:44.375 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:44.375 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:44.375 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:30:44.375 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.375 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:44.375 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:44.375 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:30:44.376 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:44.376 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:44.376 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:30:44.376 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.376 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:44.376 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:44.376 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:30:44.376 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:44.376 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:44.376 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:30:44.376 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.376 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:44.376 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:44.376 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:30:44.377 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:44.377 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:44.377 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:30:44.377 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.377 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:44.377 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:44.377 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:30:44.377 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:44.377 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:44.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.377 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:30:44.377 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:30:44.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.385+0000 7ff32675c640 1 -- 172.21.15.112:0/1276829324 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff30c06bde0 msgr2=0x7ff30c06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:44.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.385+0000 7ff32675c640 1 --2- 172.21.15.112:0/1276829324 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff30c06bde0 0x7ff30c06e2a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff31800f1d0 tx=0x7ff318009040 comp rx=0 tx=0).stop 2024-04-25T01:30:44.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.385+0000 7ff32675c640 1 -- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff320072d80 msgr2=0x7ff32014fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:44.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.385+0000 7ff32675c640 1 --2- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff320072d80 0x7ff32014fe30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff310003980 tx=0x7ff31000b9b0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.385+0000 7ff324cf9640 1 -- 172.21.15.112:0/1276829324 reap_dead start 2024-04-25T01:30:44.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.385+0000 7ff32675c640 1 -- 172.21.15.112:0/1276829324 shutdown_connections 2024-04-25T01:30:44.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.385+0000 7ff32675c640 1 --2- 172.21.15.112:0/1276829324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff320072d80 0x7ff32014fe30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.385+0000 7ff32675c640 1 -- 172.21.15.112:0/1276829324 >> 172.21.15.112:0/1276829324 conn(0x7ff32006b160 msgr2=0x7ff32014bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:44.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.385+0000 7ff32675c640 1 -- 172.21.15.112:0/1276829324 shutdown_connections 2024-04-25T01:30:44.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.385+0000 7ff32675c640 1 -- 172.21.15.112:0/1276829324 wait complete. 2024-04-25T01:30:44.380 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:30:44.380 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-04-25T01:30:44.380 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:30:44.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.501+0000 7f8749c4c640 1 Processor -- start 2024-04-25T01:30:44.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.501+0000 7f8749c4c640 1 -- start start 2024-04-25T01:30:44.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.501+0000 7f8749c4c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f874414c3f0 0x7f87440c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:44.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.502+0000 7f8749c4c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87441529b0 con 0x7f87440ac3b0 2024-04-25T01:30:44.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.502+0000 7f8749c4c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8744152c70 con 0x7f87441524a0 2024-04-25T01:30:44.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.502+0000 7f8749c4c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8744155a90 con 0x7f874414c3f0 2024-04-25T01:30:44.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.502+0000 7f8742ffd640 1 -- >> v1:172.21.15.112:6789/0 conn(0x7f87440ac3b0 legacy=0x7f8744155f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:30:44.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.502+0000 7f87437fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f874414c3f0 0x7f87440c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:44.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.502+0000 7f87437fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f874414c3f0 0x7f87440c2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40360/0 (socket says 172.21.15.112:40360) 2024-04-25T01:30:44.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.502+0000 7f87437fe640 1 -- 172.21.15.112:0/109924355 learned_addr learned my addr 172.21.15.112:0/109924355 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:44.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.502+0000 7f8743fff640 1 --1- 172.21.15.112:0/109924355 >> v1:172.21.15.187:6789/0 conn(0x7f87441524a0 0x7f874415ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:33354/0 (socket says 172.21.15.112:33354) 2024-04-25T01:30:44.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.502+0000 7f87437fe640 1 -- 172.21.15.112:0/109924355 >> v1:172.21.15.187:6789/0 conn(0x7f87441524a0 legacy=0x7f874415ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:44.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.502+0000 7f87437fe640 1 -- 172.21.15.112:0/109924355 >> v1:172.21.15.112:6789/0 conn(0x7f87440ac3b0 legacy=0x7f8744155f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:30:44.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.502+0000 7f87437fe640 1 -- 172.21.15.112:0/109924355 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8744152f50 con 0x7f874414c3f0 2024-04-25T01:30:44.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.503+0000 7f87437fe640 1 --2- 172.21.15.112:0/109924355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f874414c3f0 0x7f87440c2b90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8734009870 tx=0x7f873402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdaa79cd0fa1832 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:44.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.503+0000 7f87427fc640 1 -- 172.21.15.112:0/109924355 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734037070 con 0x7f874414c3f0 2024-04-25T01:30:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.503+0000 7f87427fc640 1 -- 172.21.15.112:0/109924355 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8734032440 con 0x7f874414c3f0 2024-04-25T01:30:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.503+0000 7f87427fc640 1 -- 172.21.15.112:0/109924355 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734032810 con 0x7f874414c3f0 2024-04-25T01:30:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.504+0000 7f8749c4c640 1 -- 172.21.15.112:0/109924355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f874414c3f0 msgr2=0x7f87440c2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.504+0000 7f8749c4c640 1 --2- 172.21.15.112:0/109924355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f874414c3f0 0x7f87440c2b90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8734009870 tx=0x7f873402a210 comp rx=0 tx=0).stop 2024-04-25T01:30:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.504+0000 7f8749c4c640 1 -- 172.21.15.112:0/109924355 shutdown_connections 2024-04-25T01:30:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.504+0000 7f8749c4c640 1 --2- 172.21.15.112:0/109924355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f874414c3f0 0x7f87440c2b90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.504+0000 7f8749c4c640 1 -- 172.21.15.112:0/109924355 >> 172.21.15.112:0/109924355 conn(0x7f874406b160 msgr2=0x7f874406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.504+0000 7f8749c4c640 1 -- 172.21.15.112:0/109924355 shutdown_connections 2024-04-25T01:30:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.504+0000 7f8749c4c640 1 -- 172.21.15.112:0/109924355 wait complete. 2024-04-25T01:30:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.505+0000 7f8749c4c640 1 Processor -- start 2024-04-25T01:30:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.505+0000 7f8749c4c640 1 -- start start 2024-04-25T01:30:44.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f8749c4c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87440ac3b0 0x7f87441ec640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:44.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f8749c4c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f874414c3f0 0x7f87441ecb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:44.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f8749c4c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87441524a0 0x7f8744145450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:44.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f8749c4c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f874414c7d0 con 0x7f874414c3f0 2024-04-25T01:30:44.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f8749c4c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8744070d60 con 0x7f87440ac3b0 2024-04-25T01:30:44.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f8749c4c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f874414c950 con 0x7f87441524a0 2024-04-25T01:30:44.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f8742ffd640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f874414c3f0 msgr2=0x7f87441ecb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:30:44.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f8742ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f874414c3f0 0x7f87441ecb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:30:44.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f87437fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87440ac3b0 0x7f87441ec640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:44.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f87437fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87440ac3b0 0x7f87441ec640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40376/0 (socket says 172.21.15.112:40376) 2024-04-25T01:30:44.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f87437fe640 1 -- 172.21.15.112:0/1872885126 learned_addr learned my addr 172.21.15.112:0/1872885126 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:30:44.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f8743fff640 1 --2- 172.21.15.112:0/1872885126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87441524a0 0x7f8744145450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:44.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f87437fe640 1 -- 172.21.15.112:0/1872885126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87441524a0 msgr2=0x7f8744145450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:44.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f87437fe640 1 --2- 172.21.15.112:0/1872885126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87441524a0 0x7f8744145450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f87437fe640 1 -- 172.21.15.112:0/1872885126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f874414c3f0 msgr2=0x7f87441ecb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:30:44.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f87437fe640 1 --2- 172.21.15.112:0/1872885126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f874414c3f0 0x7f87441ecb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f87437fe640 1 -- 172.21.15.112:0/1872885126 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87441ed2a0 con 0x7f87440ac3b0 2024-04-25T01:30:44.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f87437fe640 1 --2- 172.21.15.112:0/1872885126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87440ac3b0 0x7f87441ec640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8734009870 tx=0x7f87340046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:44.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f872bfff640 1 -- 172.21.15.112:0/1872885126 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734041070 con 0x7f87440ac3b0 2024-04-25T01:30:44.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f872bfff640 1 -- 172.21.15.112:0/1872885126 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f873403c870 con 0x7f87440ac3b0 2024-04-25T01:30:44.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f8749c4c640 1 -- 172.21.15.112:0/1872885126 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87441490b0 con 0x7f87440ac3b0 2024-04-25T01:30:44.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.506+0000 7f872bfff640 1 -- 172.21.15.112:0/1872885126 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734037040 con 0x7f87440ac3b0 2024-04-25T01:30:44.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.507+0000 7f8749c4c640 1 -- 172.21.15.112:0/1872885126 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87441483c0 con 0x7f87440ac3b0 2024-04-25T01:30:44.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.507+0000 7f8749c4c640 1 --2- 172.21.15.112:0/1872885126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87440c1210 0x7f8744145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:44.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.507+0000 7f8749c4c640 1 -- 172.21.15.112:0/1872885126 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8744075980 con 0x7f87440c1210 2024-04-25T01:30:44.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.507+0000 7f8742ffd640 1 --2- 172.21.15.112:0/1872885126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87440c1210 0x7f8744145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:44.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.508+0000 7f8742ffd640 1 -- 172.21.15.112:0/1872885126 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8744145ed0 con 0x7f87440c1210 2024-04-25T01:30:44.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.508+0000 7f8742ffd640 1 --2- 172.21.15.112:0/1872885126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87440c1210 0x7f8744145990 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8744072280 tx=0x7f873800b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:44.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.508+0000 7f872bfff640 1 -- 172.21.15.112:0/1872885126 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8734044070 con 0x7f87440ac3b0 2024-04-25T01:30:44.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.508+0000 7f872bfff640 1 --2- 172.21.15.112:0/1872885126 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f873006bde0 0x7f873006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:44.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.508+0000 7f872bfff640 1 -- 172.21.15.112:0/1872885126 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f87340ac020 con 0x7f87440ac3b0 2024-04-25T01:30:44.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.508+0000 7f8743fff640 1 --2- 172.21.15.112:0/1872885126 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f873006bde0 0x7f873006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:44.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.509+0000 7f872bfff640 1 -- 172.21.15.112:0/1872885126 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8738003e10 con 0x7f87440c1210 2024-04-25T01:30:44.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.509+0000 7f872bfff640 1 -- 172.21.15.112:0/1872885126 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8738015070 con 0x7f87440c1210 2024-04-25T01:30:44.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.509+0000 7f872bfff640 1 -- 172.21.15.112:0/1872885126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87440c1210 msgr2=0x7f8744145990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:44.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.509+0000 7f872bfff640 1 --2- 172.21.15.112:0/1872885126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87440c1210 0x7f8744145990 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8744072280 tx=0x7f873800b8c0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.509+0000 7f872bfff640 1 -- 172.21.15.112:0/1872885126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87440c1210 msgr2=0x7f8744145990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:30:44.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.509+0000 7f872bfff640 1 --2- 172.21.15.112:0/1872885126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87440c1210 0x7f8744145990 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8744072280 tx=0x7f873800b8c0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.509+0000 7f8743fff640 1 --2- 172.21.15.112:0/1872885126 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f873006bde0 0x7f873006e2a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f873c0012e0 tx=0x7f873c0015d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:44.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.515+0000 7f8749c4c640 1 --2- 172.21.15.112:0/1872885126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f874414d540 0x7f874414d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:44.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.515+0000 7f8749c4c640 1 -- 172.21.15.112:0/1872885126 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f87440722b0 con 0x7f874414d540 2024-04-25T01:30:44.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.515+0000 7f8742ffd640 1 --2- 172.21.15.112:0/1872885126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f874414d540 0x7f874414d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:44.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.515+0000 7f8742ffd640 1 -- 172.21.15.112:0/1872885126 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f87440c1eb0 con 0x7f874414d540 2024-04-25T01:30:44.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.516+0000 7f8742ffd640 1 --2- 172.21.15.112:0/1872885126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f874414d540 0x7f874414d920 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8738006fd0 tx=0x7f8738005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:44.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.516+0000 7f872bfff640 1 -- 172.21.15.112:0/1872885126 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87380055c0 con 0x7f874414d540 2024-04-25T01:30:44.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.516+0000 7f872bfff640 1 -- 172.21.15.112:0/1872885126 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4887 (secure 0 0 0) 0x7f87380055c0 con 0x7f874414d540 2024-04-25T01:30:44.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.516+0000 7f872bfff640 1 -- 172.21.15.112:0/1872885126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f874414d540 msgr2=0x7f874414d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:44.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.516+0000 7f872bfff640 1 --2- 172.21.15.112:0/1872885126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f874414d540 0x7f874414d920 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8738006fd0 tx=0x7f8738005a20 comp rx=0 tx=0).stop 2024-04-25T01:30:44.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.516+0000 7f872bfff640 1 -- 172.21.15.112:0/1872885126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f874414d540 msgr2=0x7f874414d920 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:30:44.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.516+0000 7f872bfff640 1 --2- 172.21.15.112:0/1872885126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f874414d540 0x7f874414d920 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8738006fd0 tx=0x7f8738005a20 comp rx=0 tx=0).stop 2024-04-25T01:30:44.495 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:30:44.495 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:30:44.495 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:30:44.495 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:30:44.495 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 34, 2024-04-25T01:30:44.495 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 33255, 2024-04-25T01:30:44.495 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:30:44.495 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:30:44.495 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:30:44.495 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:44.495 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:30:44.496 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:30:44.496 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:30:44.496 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:30:44.496 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:44.496 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:44.496 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:44.496 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:44.496 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:44.496 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:44.496 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:44.496 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:44.496 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:44.496 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:44.496 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:44.497 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:44.497 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:30:44.497 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:30:44.497 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:44.497 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:44.497 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:44.497 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:44.497 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:44.497 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:44.497 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:44.497 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:44.497 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:44.497 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:44.497 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:44.498 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:44.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.498 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:30:44.498 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:30:44.498 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:30:44.498 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:30:44.498 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:30:44.498 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:30:44.498 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:30:44.498 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:30:44.498 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:30:44.498 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:30:44.498 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:30:44.498 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:30:44.499 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:30:44.499 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:30:44.499 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:30:44.499 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:30:44.499 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:30:44.499 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:30:44.499 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:30:44.499 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:30:44.499 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:30:44.499 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:30:44.499 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:30:44.499 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:30:44.499 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:30:44.499 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:30:44.500 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:30:44.500 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:30:44.500 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:44.500 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:30:44.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.500 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:30:44.500 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.500 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:30:44.500 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:30:44.500 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:44.500 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:44.500 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.500 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:44.500 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:30:44.501 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:44.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.501 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.501 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:44.501 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:30:44.501 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:44.501 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:44.501 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:44.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.501 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:30:44.501 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:30:44.501 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:44.501 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:44.502 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:44.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.502 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.502 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:30:44.502 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:30:44.502 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:44.502 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:44.502 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.502 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:44.502 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:30:44.502 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:44.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.502 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.502 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:44.502 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:30:44.503 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:44.503 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:44.503 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:44.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.503 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:30:44.503 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:30:44.503 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:44.503 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:44.503 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:44.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.503 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.503 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:30:44.503 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:30:44.504 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:30:44.504 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:30:44.504 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.504 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:30:44.504 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:30:44.504 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:44.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.504 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.504 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:30:44.504 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:30:44.504 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:30:44.504 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:44.504 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:44.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.505 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:30:44.505 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:30:44.505 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:30:44.505 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:30:44.505 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:30:44.505 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:44.505 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:44.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.505 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:30:44.505 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:30:44.505 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.505 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:44.505 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:44.505 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:30:44.505 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:44.506 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:44.506 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:30:44.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.506 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:44.506 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:44.506 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:30:44.506 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:44.506 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:44.506 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:30:44.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.506 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:44.506 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:44.506 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:30:44.506 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:44.506 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:30:44.507 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:30:44.507 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:30:44.507 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:30:44.507 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:30:44.507 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:30:44.507 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:30:44.507 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:30:44.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:30:44.507 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:30:44.507 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:30:44.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.516+0000 7f8749c4c640 1 -- 172.21.15.112:0/1872885126 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f873006bde0 msgr2=0x7f873006e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:44.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.516+0000 7f8749c4c640 1 --2- 172.21.15.112:0/1872885126 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f873006bde0 0x7f873006e2a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f873c0012e0 tx=0x7f873c0015d0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.516+0000 7f8749c4c640 1 -- 172.21.15.112:0/1872885126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87440ac3b0 msgr2=0x7f87441ec640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:44.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.516+0000 7f8749c4c640 1 --2- 172.21.15.112:0/1872885126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87440ac3b0 0x7f87441ec640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8734009870 tx=0x7f87340046c0 comp rx=0 tx=0).stop 2024-04-25T01:30:44.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.516+0000 7f8743fff640 1 -- 172.21.15.112:0/1872885126 reap_dead start 2024-04-25T01:30:44.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.516+0000 7f8743fff640 1 -- 172.21.15.112:0/1872885126 reap_dead start 2024-04-25T01:30:44.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.516+0000 7f8749c4c640 1 -- 172.21.15.112:0/1872885126 shutdown_connections 2024-04-25T01:30:44.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.516+0000 7f8749c4c640 1 -- 172.21.15.112:0/1872885126 >> 172.21.15.112:0/1872885126 conn(0x7f874406b160 msgr2=0x7f8744155f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:44.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.517+0000 7f8749c4c640 1 -- 172.21.15.112:0/1872885126 shutdown_connections 2024-04-25T01:30:44.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:30:44.517+0000 7f8749c4c640 1 -- 172.21.15.112:0/1872885126 wait complete. 2024-04-25T01:30:44.509 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:30:44.510 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T01:30:55.437 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:30:55.439 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:31:04.511 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-04-25T01:31:04.511 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-04-25T01:31:04.511 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T01:31:04.512 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T01:31:04.515 INFO:tasks.ceph.mon.a:Started 2024-04-25T01:31:04.515 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:31:04.515 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:31:04.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.642+0000 7fa719215640 1 Processor -- start 2024-04-25T01:31:04.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.642+0000 7fa719215640 1 -- start start 2024-04-25T01:31:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.643+0000 7fa719215640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa714152fd0 0x7fa7140c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.643+0000 7fa719215640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa714156cf0 0x7fa71406bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.643+0000 7fa719215640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7140c90e0 con 0x7fa7140c5d70 2024-04-25T01:31:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.643+0000 7fa719215640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa7140ca9b0 con 0x7fa714156cf0 2024-04-25T01:31:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.643+0000 7fa719215640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa7140cac00 con 0x7fa714152fd0 2024-04-25T01:31:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.643+0000 7fa71259c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa7140c5d70 0x7fa714156280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:55430/0 (socket says 172.21.15.112:55430) 2024-04-25T01:31:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.643+0000 7fa71259c640 1 -- 172.21.15.112:0/3692286086 learned_addr learned my addr 172.21.15.112:0/3692286086 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:31:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.643+0000 7fa712d9d640 1 --2- 172.21.15.112:0/3692286086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa714152fd0 0x7fa7140c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.643+0000 7fa71359e640 1 --2- 172.21.15.112:0/3692286086 >> v2:172.21.15.187:3300/0 conn(0x7fa714156cf0 0x7fa71406bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.643+0000 7fa712d9d640 1 -- 172.21.15.112:0/3692286086 >> v2:172.21.15.187:3300/0 conn(0x7fa714156cf0 msgr2=0x7fa71406bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:31:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.643+0000 7fa712d9d640 1 --2- 172.21.15.112:0/3692286086 >> v2:172.21.15.187:3300/0 conn(0x7fa714156cf0 0x7fa71406bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.643+0000 7fa712d9d640 1 -- 172.21.15.112:0/3692286086 >> v1:172.21.15.112:6789/0 conn(0x7fa7140c5d70 legacy=0x7fa714156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.643+0000 7fa712d9d640 1 -- 172.21.15.112:0/3692286086 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7140c9380 con 0x7fa714152fd0 2024-04-25T01:31:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.644+0000 7fa712d9d640 1 --2- 172.21.15.112:0/3692286086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa714152fd0 0x7fa7140c9b80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa7040097a0 tx=0x7fa70402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f48d8fe1a18fc8ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.644+0000 7fa711d9b640 1 -- 172.21.15.112:0/3692286086 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704037070 con 0x7fa714152fd0 2024-04-25T01:31:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.644+0000 7fa711d9b640 1 -- 172.21.15.112:0/3692286086 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa70402ae50 con 0x7fa714152fd0 2024-04-25T01:31:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.644+0000 7fa711d9b640 1 -- 172.21.15.112:0/3692286086 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704032660 con 0x7fa714152fd0 2024-04-25T01:31:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.645+0000 7fa719215640 1 -- 172.21.15.112:0/3692286086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa714152fd0 msgr2=0x7fa7140c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.645+0000 7fa719215640 1 --2- 172.21.15.112:0/3692286086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa714152fd0 0x7fa7140c9b80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa7040097a0 tx=0x7fa70402a100 comp rx=0 tx=0).stop 2024-04-25T01:31:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.645+0000 7fa719215640 1 -- 172.21.15.112:0/3692286086 shutdown_connections 2024-04-25T01:31:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.645+0000 7fa719215640 1 --2- 172.21.15.112:0/3692286086 >> v2:172.21.15.187:3300/0 conn(0x7fa714156cf0 0x7fa71406bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.645+0000 7fa719215640 1 --2- 172.21.15.112:0/3692286086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa714152fd0 0x7fa7140c9b80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.645+0000 7fa719215640 1 -- 172.21.15.112:0/3692286086 >> 172.21.15.112:0/3692286086 conn(0x7fa7140acc70 msgr2=0x7fa7140ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.645+0000 7fa719215640 1 -- 172.21.15.112:0/3692286086 shutdown_connections 2024-04-25T01:31:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.645+0000 7fa719215640 1 -- 172.21.15.112:0/3692286086 wait complete. 2024-04-25T01:31:04.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.647+0000 7fa719215640 1 Processor -- start 2024-04-25T01:31:04.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.647+0000 7fa719215640 1 -- start start 2024-04-25T01:31:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa719215640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7140c5d70 0x7fa71414fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa719215640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa714152fd0 0x7fa71414d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa719215640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa714156cf0 0x7fa714153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa719215640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa714157160 con 0x7fa714152fd0 2024-04-25T01:31:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa719215640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa7140caee0 con 0x7fa7140c5d70 2024-04-25T01:31:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa719215640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa7141572e0 con 0x7fa714156cf0 2024-04-25T01:31:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa712d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7140c5d70 0x7fa71414fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa71359e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa714156cf0 0x7fa714153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa712d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7140c5d70 0x7fa71414fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:51920/0 (socket says 172.21.15.112:51920) 2024-04-25T01:31:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa712d9d640 1 -- 172.21.15.112:0/722863654 learned_addr learned my addr 172.21.15.112:0/722863654 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa71259c640 1 --2- 172.21.15.112:0/722863654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa714152fd0 0x7fa71414d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa71259c640 1 -- 172.21.15.112:0/722863654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa714152fd0 msgr2=0x7fa71414d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T01:31:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa71259c640 1 -- 172.21.15.112:0/722863654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa714152fd0 msgr2=0x7fa71414d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:31:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa71259c640 1 --2- 172.21.15.112:0/722863654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa714152fd0 0x7fa71414d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:31:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa71259c640 1 --2- 172.21.15.112:0/722863654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa714152fd0 0x7fa71414d270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:31:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa71359e640 1 -- 172.21.15.112:0/722863654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7140c5d70 msgr2=0x7fa71414fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa71359e640 1 --2- 172.21.15.112:0/722863654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7140c5d70 0x7fa71414fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa71359e640 1 -- 172.21.15.112:0/722863654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa714152fd0 msgr2=0x7fa71414d270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:31:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa71359e640 1 --2- 172.21.15.112:0/722863654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa714152fd0 0x7fa71414d270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa71359e640 1 -- 172.21.15.112:0/722863654 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7141ec010 con 0x7fa714156cf0 2024-04-25T01:31:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa712d9d640 1 --2- 172.21.15.112:0/722863654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7140c5d70 0x7fa71414fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:31:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.648+0000 7fa71359e640 1 --2- 172.21.15.112:0/722863654 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa714156cf0 0x7fa714153f90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00eba0 tx=0x7fa70c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.649+0000 7fa6fb7fe640 1 -- 172.21.15.112:0/722863654 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c018020 con 0x7fa714156cf0 2024-04-25T01:31:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.649+0000 7fa6fb7fe640 1 -- 172.21.15.112:0/722863654 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa70c00cd20 con 0x7fa714156cf0 2024-04-25T01:31:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.649+0000 7fa6fb7fe640 1 -- 172.21.15.112:0/722863654 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c0146c0 con 0x7fa714156cf0 2024-04-25T01:31:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.649+0000 7fa719215640 1 -- 172.21.15.112:0/722863654 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa714154a40 con 0x7fa714156cf0 2024-04-25T01:31:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.649+0000 7fa719215640 1 -- 172.21.15.112:0/722863654 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7140c7f20 con 0x7fa714156cf0 2024-04-25T01:31:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.650+0000 7fa6f97fa640 1 -- 172.21.15.112:0/722863654 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7140682a0 con 0x7fa714156cf0 2024-04-25T01:31:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.651+0000 7fa6fb7fe640 1 -- 172.21.15.112:0/722863654 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa70c005d10 con 0x7fa714156cf0 2024-04-25T01:31:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.651+0000 7fa6fb7fe640 1 --2- 172.21.15.112:0/722863654 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa70006be30 0x7fa70006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.651+0000 7fa6fb7fe640 1 -- 172.21.15.112:0/722863654 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fa70c08be00 con 0x7fa714156cf0 2024-04-25T01:31:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.651+0000 7fa712d9d640 1 --2- 172.21.15.112:0/722863654 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa70006be30 0x7fa70006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.652+0000 7fa712d9d640 1 --2- 172.21.15.112:0/722863654 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa70006be30 0x7fa70006e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa70402a140 tx=0x7fa70402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:04.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.655+0000 7fa6fb7fe640 1 -- 172.21.15.112:0/722863654 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa70c022030 con 0x7fa714156cf0 2024-04-25T01:31:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.989+0000 7fa6f97fa640 1 -- 172.21.15.112:0/722863654 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa714154fc0 con 0x7fa714156cf0 2024-04-25T01:31:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.990+0000 7fa6fb7fe640 1 -- 172.21.15.112:0/722863654 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1563 (secure 0 0 0) 0x7fa70c010040 con 0x7fa714156cf0 2024-04-25T01:31:04.968 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":34,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:31:04.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.992+0000 7fa6f97fa640 1 -- 172.21.15.112:0/722863654 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa70006be30 msgr2=0x7fa70006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:04.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.992+0000 7fa6f97fa640 1 --2- 172.21.15.112:0/722863654 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa70006be30 0x7fa70006e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa70402a140 tx=0x7fa70402e000 comp rx=0 tx=0).stop 2024-04-25T01:31:04.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.992+0000 7fa6f97fa640 1 -- 172.21.15.112:0/722863654 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa714156cf0 msgr2=0x7fa714153f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:04.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.992+0000 7fa6f97fa640 1 --2- 172.21.15.112:0/722863654 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa714156cf0 0x7fa714153f90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00eba0 tx=0x7fa70c00c490 comp rx=0 tx=0).stop 2024-04-25T01:31:04.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.992+0000 7fa6f97fa640 1 -- 172.21.15.112:0/722863654 shutdown_connections 2024-04-25T01:31:04.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.992+0000 7fa6f97fa640 1 --2- 172.21.15.112:0/722863654 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa70006be30 0x7fa70006e2f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:04.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.992+0000 7fa6f97fa640 1 --2- 172.21.15.112:0/722863654 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa714156cf0 0x7fa714153f90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:04.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.992+0000 7fa6f97fa640 1 --2- 172.21.15.112:0/722863654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa714152fd0 0x7fa71414d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:04.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.992+0000 7fa6f97fa640 1 --2- 172.21.15.112:0/722863654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7140c5d70 0x7fa71414fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:04.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.992+0000 7fa6f97fa640 1 -- 172.21.15.112:0/722863654 >> 172.21.15.112:0/722863654 conn(0x7fa7140acc70 msgr2=0x7fa7141dee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:04.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.992+0000 7fa6f97fa640 1 -- 172.21.15.112:0/722863654 shutdown_connections 2024-04-25T01:31:04.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:04.992+0000 7fa6f97fa640 1 -- 172.21.15.112:0/722863654 wait complete. 2024-04-25T01:31:07.983 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:31:08.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.107+0000 7fbc5f838640 1 Processor -- start 2024-04-25T01:31:08.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.108+0000 7fbc5f838640 1 -- start start 2024-04-25T01:31:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.108+0000 7fbc5f838640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbc580c6d00 0x7fbc580c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.108+0000 7fbc5f838640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbc580c77f0 0x7fbc580c9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.108+0000 7fbc5f838640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc581565e0 0x7fbc580c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.108+0000 7fbc5f838640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fbc580c6b60 con 0x7fbc580c77f0 2024-04-25T01:31:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.108+0000 7fbc5f838640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbc580c63f0 con 0x7fbc580c6d00 2024-04-25T01:31:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.108+0000 7fbc5f838640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbc580ca940 con 0x7fbc581565e0 2024-04-25T01:31:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.109+0000 7fbc5cdd3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbc580c77f0 0x7fbc580c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.109+0000 7fbc5cdd3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbc580c77f0 0x7fbc580c9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50200/0 (socket says 172.21.15.112:50200) 2024-04-25T01:31:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.109+0000 7fbc5cdd3640 1 -- 172.21.15.112:0/3765841761 learned_addr learned my addr 172.21.15.112:0/3765841761 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.109+0000 7fbc5ddd5640 1 --2- 172.21.15.112:0/3765841761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc581565e0 0x7fbc580c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.109+0000 7fbc5d5d4640 1 --2- 172.21.15.112:0/3765841761 >> v2:172.21.15.187:3300/0 conn(0x7fbc580c6d00 0x7fbc580c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.109+0000 7fbc5cdd3640 1 -- 172.21.15.112:0/3765841761 >> v2:172.21.15.112:3300/0 conn(0x7fbc580c77f0 msgr2=0x7fbc580c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:31:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.109+0000 7fbc5cdd3640 1 -- 172.21.15.112:0/3765841761 >> v2:172.21.15.112:3300/0 conn(0x7fbc580c77f0 msgr2=0x7fbc580c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:31:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.109+0000 7fbc5cdd3640 1 --2- 172.21.15.112:0/3765841761 >> v2:172.21.15.112:3300/0 conn(0x7fbc580c77f0 0x7fbc580c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:31:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.109+0000 7fbc5cdd3640 1 --2- 172.21.15.112:0/3765841761 >> v2:172.21.15.112:3300/0 conn(0x7fbc580c77f0 0x7fbc580c9050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:31:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.109+0000 7fbc5d5d4640 1 -- 172.21.15.112:0/3765841761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc581565e0 msgr2=0x7fbc580c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:31:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.109+0000 7fbc5d5d4640 1 --2- 172.21.15.112:0/3765841761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc581565e0 0x7fbc580c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:08.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.109+0000 7fbc5d5d4640 1 -- 172.21.15.112:0/3765841761 >> v2:172.21.15.112:3300/0 conn(0x7fbc580c77f0 msgr2=0x7fbc580c9050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:31:08.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.109+0000 7fbc5d5d4640 1 --2- 172.21.15.112:0/3765841761 >> v2:172.21.15.112:3300/0 conn(0x7fbc580c77f0 0x7fbc580c9050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:08.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.109+0000 7fbc5d5d4640 1 -- 172.21.15.112:0/3765841761 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5806c970 con 0x7fbc580c6d00 2024-04-25T01:31:08.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.109+0000 7fbc5ddd5640 1 --2- 172.21.15.112:0/3765841761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc581565e0 0x7fbc580c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:31:08.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.110+0000 7fbc5d5d4640 1 --2- 172.21.15.112:0/3765841761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc580c6d00 0x7fbc580c70e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbc580cb120 tx=0x7fbc5402a010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6584a7fc3a77f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:08.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.110+0000 7fbc4ffff640 1 -- 172.21.15.112:0/3765841761 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54037070 con 0x7fbc580c6d00 2024-04-25T01:31:08.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.110+0000 7fbc4ffff640 1 -- 172.21.15.112:0/3765841761 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc5402ad20 con 0x7fbc580c6d00 2024-04-25T01:31:08.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.110+0000 7fbc4ffff640 1 -- 172.21.15.112:0/3765841761 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54032660 con 0x7fbc580c6d00 2024-04-25T01:31:08.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.110+0000 7fbc5f838640 1 -- 172.21.15.112:0/3765841761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc580c6d00 msgr2=0x7fbc580c70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:08.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.110+0000 7fbc5f838640 1 --2- 172.21.15.112:0/3765841761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc580c6d00 0x7fbc580c70e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbc580cb120 tx=0x7fbc5402a010 comp rx=0 tx=0).stop 2024-04-25T01:31:08.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.111+0000 7fbc5f838640 1 -- 172.21.15.112:0/3765841761 shutdown_connections 2024-04-25T01:31:08.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.111+0000 7fbc5f838640 1 --2- 172.21.15.112:0/3765841761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc581565e0 0x7fbc580c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:08.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.111+0000 7fbc5f838640 1 --2- 172.21.15.112:0/3765841761 >> v2:172.21.15.112:3300/0 conn(0x7fbc580c77f0 0x7fbc580c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:08.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.111+0000 7fbc5f838640 1 --2- 172.21.15.112:0/3765841761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc580c6d00 0x7fbc580c70e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:08.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.111+0000 7fbc5f838640 1 -- 172.21.15.112:0/3765841761 >> 172.21.15.112:0/3765841761 conn(0x7fbc580cb330 msgr2=0x7fbc580cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:08.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.111+0000 7fbc5f838640 1 -- 172.21.15.112:0/3765841761 shutdown_connections 2024-04-25T01:31:08.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.111+0000 7fbc5f838640 1 -- 172.21.15.112:0/3765841761 wait complete. 2024-04-25T01:31:08.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.112+0000 7fbc5f838640 1 Processor -- start 2024-04-25T01:31:08.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.112+0000 7fbc5f838640 1 -- start start 2024-04-25T01:31:08.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.113+0000 7fbc5f838640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc580c6d00 0x7fbc581592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:08.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.113+0000 7fbc5f838640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc580c77f0 0x7fbc581eee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:08.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.113+0000 7fbc5f838640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc581565e0 0x7fbc581ef340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:08.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.113+0000 7fbc5f838640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbc580c9ad0 con 0x7fbc580c77f0 2024-04-25T01:31:08.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.113+0000 7fbc5f838640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbc580c6640 con 0x7fbc581565e0 2024-04-25T01:31:08.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.113+0000 7fbc5f838640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbc58153430 con 0x7fbc580c6d00 2024-04-25T01:31:08.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.113+0000 7fbc5cdd3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc580c77f0 0x7fbc581eee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:08.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.113+0000 7fbc5cdd3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc580c77f0 0x7fbc581eee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50214/0 (socket says 172.21.15.112:50214) 2024-04-25T01:31:08.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.113+0000 7fbc5d5d4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc580c6d00 0x7fbc581592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:08.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.113+0000 7fbc5cdd3640 1 -- 172.21.15.112:0/3531978540 learned_addr learned my addr 172.21.15.112:0/3531978540 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:08.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.113+0000 7fbc5ddd5640 1 --2- 172.21.15.112:0/3531978540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc581565e0 0x7fbc581ef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:08.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.113+0000 7fbc5cdd3640 1 -- 172.21.15.112:0/3531978540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc580c77f0 msgr2=0x7fbc581eee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T01:31:08.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.113+0000 7fbc5cdd3640 1 -- 172.21.15.112:0/3531978540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc580c77f0 msgr2=0x7fbc581eee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:31:08.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.113+0000 7fbc5cdd3640 1 --2- 172.21.15.112:0/3531978540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc580c77f0 0x7fbc581eee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:31:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.113+0000 7fbc5cdd3640 1 --2- 172.21.15.112:0/3531978540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc580c77f0 0x7fbc581eee00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:31:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.114+0000 7fbc5d5d4640 1 -- 172.21.15.112:0/3531978540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc581565e0 msgr2=0x7fbc581ef340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.114+0000 7fbc5d5d4640 1 --2- 172.21.15.112:0/3531978540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc581565e0 0x7fbc581ef340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.114+0000 7fbc5d5d4640 1 -- 172.21.15.112:0/3531978540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc580c77f0 msgr2=0x7fbc581eee00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:31:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.114+0000 7fbc5d5d4640 1 --2- 172.21.15.112:0/3531978540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc580c77f0 0x7fbc581eee00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.114+0000 7fbc5d5d4640 1 -- 172.21.15.112:0/3531978540 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc580c5470 con 0x7fbc580c6d00 2024-04-25T01:31:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.114+0000 7fbc5d5d4640 1 --2- 172.21.15.112:0/3531978540 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc580c6d00 0x7fbc581592d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbc54031000 tx=0x7fbc540046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.114+0000 7fbc4dffb640 1 -- 172.21.15.112:0/3531978540 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54041030 con 0x7fbc580c6d00 2024-04-25T01:31:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.114+0000 7fbc4dffb640 1 -- 172.21.15.112:0/3531978540 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc5402ab20 con 0x7fbc580c6d00 2024-04-25T01:31:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.114+0000 7fbc5f838640 1 -- 172.21.15.112:0/3531978540 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc580c46c0 con 0x7fbc580c6d00 2024-04-25T01:31:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.114+0000 7fbc4dffb640 1 -- 172.21.15.112:0/3531978540 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54037040 con 0x7fbc580c6d00 2024-04-25T01:31:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.114+0000 7fbc5f838640 1 -- 172.21.15.112:0/3531978540 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc580c4c80 con 0x7fbc580c6d00 2024-04-25T01:31:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.115+0000 7fbc5f838640 1 -- 172.21.15.112:0/3531978540 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc580682a0 con 0x7fbc580c6d00 2024-04-25T01:31:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.116+0000 7fbc4dffb640 1 -- 172.21.15.112:0/3531978540 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbc5403d640 con 0x7fbc580c6d00 2024-04-25T01:31:08.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.116+0000 7fbc4dffb640 1 --2- 172.21.15.112:0/3531978540 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbc3c06be30 0x7fbc3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:08.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.116+0000 7fbc5cdd3640 1 --2- 172.21.15.112:0/3531978540 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbc3c06be30 0x7fbc3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:08.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.116+0000 7fbc4dffb640 1 -- 172.21.15.112:0/3531978540 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fbc540aafa0 con 0x7fbc580c6d00 2024-04-25T01:31:08.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.117+0000 7fbc5cdd3640 1 --2- 172.21.15.112:0/3531978540 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbc3c06be30 0x7fbc3c06e2f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbc5814ccc0 tx=0x7fbc48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:08.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.121+0000 7fbc4dffb640 1 -- 172.21.15.112:0/3531978540 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbc5404b020 con 0x7fbc580c6d00 2024-04-25T01:31:08.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.484+0000 7fbc5f838640 1 -- 172.21.15.112:0/3531978540 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fbc580c5c80 con 0x7fbc580c6d00 2024-04-25T01:31:08.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.485+0000 7fbc4dffb640 1 -- 172.21.15.112:0/3531978540 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1563 (secure 0 0 0) 0x7fbc54079620 con 0x7fbc580c6d00 2024-04-25T01:31:08.462 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":34,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:31:08.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.487+0000 7fbc5f838640 1 -- 172.21.15.112:0/3531978540 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbc3c06be30 msgr2=0x7fbc3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:08.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.487+0000 7fbc5f838640 1 --2- 172.21.15.112:0/3531978540 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbc3c06be30 0x7fbc3c06e2f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbc5814ccc0 tx=0x7fbc48008040 comp rx=0 tx=0).stop 2024-04-25T01:31:08.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.487+0000 7fbc5f838640 1 -- 172.21.15.112:0/3531978540 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc580c6d00 msgr2=0x7fbc581592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:08.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.487+0000 7fbc5f838640 1 --2- 172.21.15.112:0/3531978540 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc580c6d00 0x7fbc581592d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbc54031000 tx=0x7fbc540046c0 comp rx=0 tx=0).stop 2024-04-25T01:31:08.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.487+0000 7fbc5f838640 1 -- 172.21.15.112:0/3531978540 shutdown_connections 2024-04-25T01:31:08.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.487+0000 7fbc5f838640 1 --2- 172.21.15.112:0/3531978540 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbc3c06be30 0x7fbc3c06e2f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:08.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.487+0000 7fbc5f838640 1 --2- 172.21.15.112:0/3531978540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc581565e0 0x7fbc581ef340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:08.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.487+0000 7fbc5f838640 1 --2- 172.21.15.112:0/3531978540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc580c77f0 0x7fbc581eee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:08.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.487+0000 7fbc5f838640 1 --2- 172.21.15.112:0/3531978540 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc580c6d00 0x7fbc581592d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:08.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.487+0000 7fbc5f838640 1 -- 172.21.15.112:0/3531978540 >> 172.21.15.112:0/3531978540 conn(0x7fbc580cb330 msgr2=0x7fbc580cb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:08.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.488+0000 7fbc5f838640 1 -- 172.21.15.112:0/3531978540 shutdown_connections 2024-04-25T01:31:08.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:08.488+0000 7fbc5f838640 1 -- 172.21.15.112:0/3531978540 wait complete. 2024-04-25T01:31:11.479 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:31:11.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.607+0000 7f8d17838640 1 Processor -- start 2024-04-25T01:31:11.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.607+0000 7f8d17838640 1 -- start start 2024-04-25T01:31:11.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.608+0000 7f8d17838640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8d10152fd0 0x7f8d100c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:11.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.608+0000 7f8d17838640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d100c5d70 0x7f8d10156280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:11.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.608+0000 7f8d17838640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d101567c0 con 0x7f8d10156cf0 2024-04-25T01:31:11.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.608+0000 7f8d17838640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8d100ca9b0 con 0x7f8d10152fd0 2024-04-25T01:31:11.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.608+0000 7f8d17838640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8d100c90e0 con 0x7f8d100c5d70 2024-04-25T01:31:11.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.608+0000 7f8d155d4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8d10152fd0 0x7f8d100c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:11.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.608+0000 7f8d14dd3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d100c5d70 0x7f8d10156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:11.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.608+0000 7f8d14dd3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d100c5d70 0x7f8d10156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:52266/0 (socket says 172.21.15.112:52266) 2024-04-25T01:31:11.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.608+0000 7f8d14dd3640 1 -- 172.21.15.112:0/986591812 learned_addr learned my addr 172.21.15.112:0/986591812 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:11.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.608+0000 7f8d155d4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8d10152fd0 0x7f8d100c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:44214/0 (socket says 172.21.15.112:44214) 2024-04-25T01:31:11.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.608+0000 7f8d15dd5640 1 --1- 172.21.15.112:0/986591812 >> v1:172.21.15.187:6789/0 conn(0x7f8d10156cf0 0x7f8d1006bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:58986/0 (socket says 172.21.15.112:58986) 2024-04-25T01:31:11.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.609+0000 7f8d155d4640 1 -- 172.21.15.112:0/986591812 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d100c5d70 msgr2=0x7f8d10156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:31:11.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.609+0000 7f8d155d4640 1 --2- 172.21.15.112:0/986591812 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d100c5d70 0x7f8d10156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:11.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.609+0000 7f8d155d4640 1 -- 172.21.15.112:0/986591812 >> v1:172.21.15.187:6789/0 conn(0x7f8d10156cf0 legacy=0x7f8d1006bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:11.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.609+0000 7f8d155d4640 1 -- 172.21.15.112:0/986591812 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d100cac00 con 0x7f8d10152fd0 2024-04-25T01:31:11.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.609+0000 7f8d155d4640 1 --2- 172.21.15.112:0/986591812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d10152fd0 0x7f8d100c9b80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0097a0 tx=0x7f8d0c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e474ae1bbf0bfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:11.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.609+0000 7f8d07fff640 1 -- 172.21.15.112:0/986591812 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c037070 con 0x7f8d10152fd0 2024-04-25T01:31:11.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.610+0000 7f8d07fff640 1 -- 172.21.15.112:0/986591812 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d0c02ae50 con 0x7f8d10152fd0 2024-04-25T01:31:11.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.610+0000 7f8d07fff640 1 -- 172.21.15.112:0/986591812 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c032660 con 0x7f8d10152fd0 2024-04-25T01:31:11.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.610+0000 7f8d17838640 1 -- 172.21.15.112:0/986591812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d10152fd0 msgr2=0x7f8d100c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:11.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.610+0000 7f8d17838640 1 --2- 172.21.15.112:0/986591812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d10152fd0 0x7f8d100c9b80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0097a0 tx=0x7f8d0c02a100 comp rx=0 tx=0).stop 2024-04-25T01:31:11.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.610+0000 7f8d17838640 1 -- 172.21.15.112:0/986591812 shutdown_connections 2024-04-25T01:31:11.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.610+0000 7f8d17838640 1 --2- 172.21.15.112:0/986591812 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d100c5d70 0x7f8d10156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:11.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.610+0000 7f8d17838640 1 --2- 172.21.15.112:0/986591812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d10152fd0 0x7f8d100c9b80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:11.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.610+0000 7f8d17838640 1 -- 172.21.15.112:0/986591812 >> 172.21.15.112:0/986591812 conn(0x7f8d100acc70 msgr2=0x7f8d100ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:11.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.610+0000 7f8d17838640 1 -- 172.21.15.112:0/986591812 shutdown_connections 2024-04-25T01:31:11.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.610+0000 7f8d17838640 1 -- 172.21.15.112:0/986591812 wait complete. 2024-04-25T01:31:11.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.611+0000 7f8d17838640 1 Processor -- start 2024-04-25T01:31:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.612+0000 7f8d17838640 1 -- start start 2024-04-25T01:31:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.612+0000 7f8d17838640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d100c5d70 0x7f8d10155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.612+0000 7f8d17838640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d10152fd0 0x7f8d101ea8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.612+0000 7f8d17838640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d10156cf0 0x7f8d101eae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.612+0000 7f8d17838640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8d100caee0 con 0x7f8d10156cf0 2024-04-25T01:31:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.612+0000 7f8d17838640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8d100c9660 con 0x7f8d10152fd0 2024-04-25T01:31:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.612+0000 7f8d17838640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8d1006c4b0 con 0x7f8d100c5d70 2024-04-25T01:31:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.612+0000 7f8d15dd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d10156cf0 0x7f8d101eae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.612+0000 7f8d155d4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d100c5d70 0x7f8d10155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.612+0000 7f8d15dd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d10156cf0 0x7f8d101eae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50218/0 (socket says 172.21.15.112:50218) 2024-04-25T01:31:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.612+0000 7f8d155d4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d100c5d70 0x7f8d10155370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:52274/0 (socket says 172.21.15.112:52274) 2024-04-25T01:31:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.612+0000 7f8d15dd5640 1 -- 172.21.15.112:0/388251625 learned_addr learned my addr 172.21.15.112:0/388251625 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.612+0000 7f8d14dd3640 1 --2- 172.21.15.112:0/388251625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d10152fd0 0x7f8d101ea8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:11.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.613+0000 7f8d155d4640 1 -- 172.21.15.112:0/388251625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d10152fd0 msgr2=0x7f8d101ea8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:11.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.613+0000 7f8d155d4640 1 --2- 172.21.15.112:0/388251625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d10152fd0 0x7f8d101ea8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:11.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.613+0000 7f8d155d4640 1 -- 172.21.15.112:0/388251625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d10156cf0 msgr2=0x7f8d101eae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:11.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.613+0000 7f8d155d4640 1 --2- 172.21.15.112:0/388251625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d10156cf0 0x7f8d101eae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:11.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.613+0000 7f8d155d4640 1 -- 172.21.15.112:0/388251625 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d101516c0 con 0x7f8d100c5d70 2024-04-25T01:31:11.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.613+0000 7f8d15dd5640 1 --2- 172.21.15.112:0/388251625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d10156cf0 0x7f8d101eae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:31:11.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.613+0000 7f8d155d4640 1 --2- 172.21.15.112:0/388251625 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d100c5d70 0x7f8d10155370 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8d100c8940 tx=0x7f8d0c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:11.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.613+0000 7f8d05ffb640 1 -- 172.21.15.112:0/388251625 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c041030 con 0x7f8d100c5d70 2024-04-25T01:31:11.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.613+0000 7f8d05ffb640 1 -- 172.21.15.112:0/388251625 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d0c02ac50 con 0x7f8d100c5d70 2024-04-25T01:31:11.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.613+0000 7f8d17838640 1 -- 172.21.15.112:0/388251625 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d101522b0 con 0x7f8d100c5d70 2024-04-25T01:31:11.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.613+0000 7f8d05ffb640 1 -- 172.21.15.112:0/388251625 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c037040 con 0x7f8d100c5d70 2024-04-25T01:31:11.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.613+0000 7f8d17838640 1 -- 172.21.15.112:0/388251625 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d101ef560 con 0x7f8d100c5d70 2024-04-25T01:31:11.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.614+0000 7f8d17838640 1 -- 172.21.15.112:0/388251625 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d101eb550 con 0x7f8d100c5d70 2024-04-25T01:31:11.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.615+0000 7f8d05ffb640 1 -- 172.21.15.112:0/388251625 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d0c044070 con 0x7f8d100c5d70 2024-04-25T01:31:11.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.615+0000 7f8d05ffb640 1 --2- 172.21.15.112:0/388251625 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8cf406be30 0x7f8cf406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:11.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.615+0000 7f8d05ffb640 1 -- 172.21.15.112:0/388251625 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f8d0c0ab5a0 con 0x7f8d100c5d70 2024-04-25T01:31:11.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.615+0000 7f8d14dd3640 1 --2- 172.21.15.112:0/388251625 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8cf406be30 0x7f8cf406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:11.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.616+0000 7f8d14dd3640 1 --2- 172.21.15.112:0/388251625 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8cf406be30 0x7f8cf406e2f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8d10157b00 tx=0x7f8d00009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:11.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.621+0000 7f8d05ffb640 1 -- 172.21.15.112:0/388251625 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8d101eb550 con 0x7f8d100c5d70 2024-04-25T01:31:11.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.918+0000 7f8d17838640 1 -- 172.21.15.112:0/388251625 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8d100682a0 con 0x7f8d100c5d70 2024-04-25T01:31:11.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.919+0000 7f8d05ffb640 1 -- 172.21.15.112:0/388251625 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1568 (secure 0 0 0) 0x7f8d0c032e20 con 0x7f8d100c5d70 2024-04-25T01:31:11.896 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:31:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.921+0000 7f8d17838640 1 -- 172.21.15.112:0/388251625 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8cf406be30 msgr2=0x7f8cf406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.921+0000 7f8d17838640 1 --2- 172.21.15.112:0/388251625 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8cf406be30 0x7f8cf406e2f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8d10157b00 tx=0x7f8d00009240 comp rx=0 tx=0).stop 2024-04-25T01:31:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.921+0000 7f8d17838640 1 -- 172.21.15.112:0/388251625 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d100c5d70 msgr2=0x7f8d10155370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.921+0000 7f8d17838640 1 --2- 172.21.15.112:0/388251625 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d100c5d70 0x7f8d10155370 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8d100c8940 tx=0x7f8d0c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:31:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.921+0000 7f8d17838640 1 -- 172.21.15.112:0/388251625 shutdown_connections 2024-04-25T01:31:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.921+0000 7f8d17838640 1 --2- 172.21.15.112:0/388251625 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8cf406be30 0x7f8cf406e2f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.921+0000 7f8d17838640 1 --2- 172.21.15.112:0/388251625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d10156cf0 0x7f8d101eae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.921+0000 7f8d17838640 1 --2- 172.21.15.112:0/388251625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d10152fd0 0x7f8d101ea8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.921+0000 7f8d17838640 1 --2- 172.21.15.112:0/388251625 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d100c5d70 0x7f8d10155370 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:11.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.921+0000 7f8d17838640 1 -- 172.21.15.112:0/388251625 >> 172.21.15.112:0/388251625 conn(0x7f8d100acc70 msgr2=0x7f8d10157ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:11.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.921+0000 7f8d17838640 1 -- 172.21.15.112:0/388251625 shutdown_connections 2024-04-25T01:31:11.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:11.921+0000 7f8d17838640 1 -- 172.21.15.112:0/388251625 wait complete. 2024-04-25T01:31:11.910 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:31:11.910 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:31:11.910 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:31:12.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.044+0000 7eff3be1b640 1 Processor -- start 2024-04-25T01:31:12.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.045+0000 7eff3be1b640 1 -- start start 2024-04-25T01:31:12.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.045+0000 7eff3be1b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7eff341592c0 0x7eff340c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.045+0000 7eff3be1b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7eff341596a0 0x7eff34152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.045+0000 7eff3be1b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff34159b90 con 0x7eff340c72b0 2024-04-25T01:31:12.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.045+0000 7eff3be1b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7eff340c6e20 con 0x7eff341596a0 2024-04-25T01:31:12.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.045+0000 7eff3be1b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7eff3406c970 con 0x7eff341592c0 2024-04-25T01:31:12.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.045+0000 7eff393b6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7eff341596a0 0x7eff34152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.046+0000 7eff393b6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7eff341596a0 0x7eff34152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50224/0 (socket says 172.21.15.112:50224) 2024-04-25T01:31:12.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.046+0000 7eff393b6640 1 -- 172.21.15.112:0/3859444431 learned_addr learned my addr 172.21.15.112:0/3859444431 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:12.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.046+0000 7eff39bb7640 1 --2- 172.21.15.112:0/3859444431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7eff341592c0 0x7eff340c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.046+0000 7eff393b6640 1 -- 172.21.15.112:0/3859444431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7eff341592c0 msgr2=0x7eff340c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:31:12.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.046+0000 7eff393b6640 1 --2- 172.21.15.112:0/3859444431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7eff341592c0 0x7eff340c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.046+0000 7eff393b6640 1 -- 172.21.15.112:0/3859444431 >> v1:172.21.15.187:6789/0 conn(0x7eff340c72b0 legacy=0x7eff34155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.046+0000 7eff393b6640 1 -- 172.21.15.112:0/3859444431 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3406cc50 con 0x7eff341596a0 2024-04-25T01:31:12.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.047+0000 7eff393b6640 1 --2- 172.21.15.112:0/3859444431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff341596a0 0x7eff34152d00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7eff24009870 tx=0x7eff2402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755c8214434fd80e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:12.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.047+0000 7eff38bb5640 1 -- 172.21.15.112:0/3859444431 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24037070 con 0x7eff341596a0 2024-04-25T01:31:12.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.047+0000 7eff38bb5640 1 -- 172.21.15.112:0/3859444431 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff24032440 con 0x7eff341596a0 2024-04-25T01:31:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.047+0000 7eff38bb5640 1 -- 172.21.15.112:0/3859444431 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24032810 con 0x7eff341596a0 2024-04-25T01:31:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.047+0000 7eff3be1b640 1 -- 172.21.15.112:0/3859444431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff341596a0 msgr2=0x7eff34152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.047+0000 7eff3be1b640 1 --2- 172.21.15.112:0/3859444431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff341596a0 0x7eff34152d00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7eff24009870 tx=0x7eff2402a130 comp rx=0 tx=0).stop 2024-04-25T01:31:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.047+0000 7eff3be1b640 1 -- 172.21.15.112:0/3859444431 shutdown_connections 2024-04-25T01:31:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.047+0000 7eff3be1b640 1 --2- 172.21.15.112:0/3859444431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff341596a0 0x7eff34152d00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.047+0000 7eff3be1b640 1 --2- 172.21.15.112:0/3859444431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7eff341592c0 0x7eff340c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.047+0000 7eff3be1b640 1 -- 172.21.15.112:0/3859444431 >> 172.21.15.112:0/3859444431 conn(0x7eff340cb170 msgr2=0x7eff340cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.048+0000 7eff3be1b640 1 -- 172.21.15.112:0/3859444431 shutdown_connections 2024-04-25T01:31:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.048+0000 7eff3be1b640 1 -- 172.21.15.112:0/3859444431 wait complete. 2024-04-25T01:31:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.049+0000 7eff3be1b640 1 Processor -- start 2024-04-25T01:31:12.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.049+0000 7eff3be1b640 1 -- start start 2024-04-25T01:31:12.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.050+0000 7eff3be1b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7eff340c72b0 0x7eff34146de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.050+0000 7eff3be1b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff341592c0 0x7eff3414c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.050+0000 7eff3be1b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff341596a0 0x7eff341ef260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.050+0000 7eff3be1b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff3406dca0 con 0x7eff341596a0 2024-04-25T01:31:12.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.050+0000 7eff3be1b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7eff3406db20 con 0x7eff341592c0 2024-04-25T01:31:12.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.050+0000 7eff3be1b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7eff340c95b0 con 0x7eff340c72b0 2024-04-25T01:31:12.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.050+0000 7eff3a3b8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff341596a0 0x7eff341ef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.050+0000 7eff3a3b8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff341596a0 0x7eff341ef260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50236/0 (socket says 172.21.15.112:50236) 2024-04-25T01:31:12.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.050+0000 7eff3a3b8640 1 -- 172.21.15.112:0/3248639644 learned_addr learned my addr 172.21.15.112:0/3248639644 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:12.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.050+0000 7eff393b6640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff341592c0 0x7eff3414c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.050+0000 7eff39bb7640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7eff340c72b0 0x7eff34146de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.050+0000 7eff3a3b8640 1 -- 172.21.15.112:0/3248639644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7eff340c72b0 msgr2=0x7eff34146de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.050+0000 7eff3a3b8640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7eff340c72b0 0x7eff34146de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.050+0000 7eff3a3b8640 1 -- 172.21.15.112:0/3248639644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff341592c0 msgr2=0x7eff3414c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.050+0000 7eff3a3b8640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff341592c0 0x7eff3414c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.050+0000 7eff3a3b8640 1 -- 172.21.15.112:0/3248639644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff341efaa0 con 0x7eff341596a0 2024-04-25T01:31:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.050+0000 7eff393b6640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff341592c0 0x7eff3414c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:31:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.050+0000 7eff3a3b8640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff341596a0 0x7eff341ef260 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7eff2c00c730 tx=0x7eff2c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.051+0000 7eff2a7fc640 1 -- 172.21.15.112:0/3248639644 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c003b90 con 0x7eff341596a0 2024-04-25T01:31:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.051+0000 7eff2a7fc640 1 -- 172.21.15.112:0/3248639644 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff2c003d30 con 0x7eff341596a0 2024-04-25T01:31:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.051+0000 7eff3be1b640 1 -- 172.21.15.112:0/3248639644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff341efd80 con 0x7eff341596a0 2024-04-25T01:31:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.051+0000 7eff2a7fc640 1 -- 172.21.15.112:0/3248639644 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c00aa90 con 0x7eff341596a0 2024-04-25T01:31:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.051+0000 7eff3be1b640 1 -- 172.21.15.112:0/3248639644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff341ef7a0 con 0x7eff341596a0 2024-04-25T01:31:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.052+0000 7eff3be1b640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff340aeb60 0x7eff341ecef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.052+0000 7eff3be1b640 1 -- 172.21.15.112:0/3248639644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff34075980 con 0x7eff340aeb60 2024-04-25T01:31:12.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.052+0000 7eff39bb7640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff340aeb60 0x7eff341ecef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.052+0000 7eff39bb7640 1 -- 172.21.15.112:0/3248639644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7eff340679a0 con 0x7eff340aeb60 2024-04-25T01:31:12.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.052+0000 7eff39bb7640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff340aeb60 0x7eff341ecef0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7eff300118b0 tx=0x7eff30011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:12.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.052+0000 7eff2a7fc640 1 -- 172.21.15.112:0/3248639644 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30019070 con 0x7eff340aeb60 2024-04-25T01:31:12.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.052+0000 7eff2a7fc640 1 -- 172.21.15.112:0/3248639644 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7eff2c00ac30 con 0x7eff341596a0 2024-04-25T01:31:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.053+0000 7eff2a7fc640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7eff1806bd30 0x7eff1806e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.053+0000 7eff393b6640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7eff1806bd30 0x7eff1806e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.053+0000 7eff2a7fc640 1 -- 172.21.15.112:0/3248639644 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7eff2c08af10 con 0x7eff341596a0 2024-04-25T01:31:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.053+0000 7eff2a7fc640 1 -- 172.21.15.112:0/3248639644 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7eff30004300 con 0x7eff340aeb60 2024-04-25T01:31:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.053+0000 7eff2a7fc640 1 -- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff340aeb60 msgr2=0x7eff341ecef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.053+0000 7eff2a7fc640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff340aeb60 0x7eff341ecef0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7eff300118b0 tx=0x7eff30011d70 comp rx=0 tx=0).stop 2024-04-25T01:31:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.053+0000 7eff2a7fc640 1 -- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff340aeb60 msgr2=0x7eff341ecef0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:31:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.053+0000 7eff2a7fc640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff340aeb60 0x7eff341ecef0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.053+0000 7eff393b6640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7eff1806bd30 0x7eff1806e1f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7eff24034000 tx=0x7eff2402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:12.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.060+0000 7eff3be1b640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff341468f0 0x7eff34145870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.060+0000 7eff3be1b640 1 -- 172.21.15.112:0/3248639644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff340682a0 con 0x7eff341468f0 2024-04-25T01:31:12.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.060+0000 7eff39bb7640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff341468f0 0x7eff34145870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.060+0000 7eff39bb7640 1 -- 172.21.15.112:0/3248639644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7eff340c79d0 con 0x7eff341468f0 2024-04-25T01:31:12.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.061+0000 7eff39bb7640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff341468f0 0x7eff34145870 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7eff30000c00 tx=0x7eff300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:12.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.061+0000 7eff2a7fc640 1 -- 172.21.15.112:0/3248639644 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3001e650 con 0x7eff341468f0 2024-04-25T01:31:12.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.061+0000 7eff2a7fc640 1 -- 172.21.15.112:0/3248639644 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4559 (secure 0 0 0) 0x7eff300283a0 con 0x7eff341468f0 2024-04-25T01:31:12.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.061+0000 7eff2a7fc640 1 -- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff341468f0 msgr2=0x7eff34145870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.061+0000 7eff2a7fc640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff341468f0 0x7eff34145870 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7eff30000c00 tx=0x7eff300046c0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.061+0000 7eff2a7fc640 1 -- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff341468f0 msgr2=0x7eff34145870 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:31:12.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.061+0000 7eff2a7fc640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff341468f0 0x7eff34145870 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.039 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:31:12.039 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:31:12.040 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:31:12.040 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:31:12.040 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 38, 2024-04-25T01:31:12.040 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 7430, 2024-04-25T01:31:12.040 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:31:12.040 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:31:12.040 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:31:12.040 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:31:12.040 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:12.040 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:31:12.040 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:31:12.040 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:31:12.040 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:31:12.040 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:12.040 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:12.041 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:12.041 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:12.041 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:12.041 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:12.041 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:12.041 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:12.041 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:12.041 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:12.041 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:12.041 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:12.041 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:31:12.041 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:31:12.041 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:12.041 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:12.042 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:12.042 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:12.042 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:12.042 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:12.042 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:12.042 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:12.042 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:12.042 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:12.042 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:12.042 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:12.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.042 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:31:12.042 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:31:12.042 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:31:12.042 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:31:12.043 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:31:12.043 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:31:12.043 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:31:12.043 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:31:12.043 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:31:12.043 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:31:12.043 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:31:12.043 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:31:12.043 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:31:12.043 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:31:12.043 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:31:12.043 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:31:12.043 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:31:12.043 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:12.043 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:12.044 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:12.044 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:12.044 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:12.044 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:12.044 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:12.044 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:12.044 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:12.044 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:12.044 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:12.044 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:12.044 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:31:12.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.044 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:31:12.044 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.044 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:31:12.045 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:31:12.045 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:12.045 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:12.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.045 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:12.045 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:31:12.045 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:12.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.045 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:12.045 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:31:12.045 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:12.045 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.045 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:12.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.046 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:31:12.046 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:31:12.046 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:12.046 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:12.046 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:12.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.046 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.046 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:31:12.046 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:31:12.046 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:12.046 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:12.046 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.046 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:12.046 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:31:12.047 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:12.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.047 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.047 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:12.047 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:31:12.047 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:12.047 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.047 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:12.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.047 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:31:12.047 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:31:12.047 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:12.047 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:12.047 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:12.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.048 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.048 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:31:12.048 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:31:12.048 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:12.048 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:12.048 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.048 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:12.048 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:31:12.048 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:12.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.048 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.048 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:12.048 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:31:12.048 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:12.048 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.049 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:12.049 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.049 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:31:12.049 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:31:12.049 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:12.049 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:12.049 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:12.049 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.049 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:12.049 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.049 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:31:12.049 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:31:12.049 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.049 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:12.049 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:12.050 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:31:12.050 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.050 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:12.050 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:31:12.050 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.050 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:12.050 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:12.050 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:31:12.050 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.050 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:12.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.050 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:31:12.050 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:31:12.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.061+0000 7eff3be1b640 1 -- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7eff1806bd30 msgr2=0x7eff1806e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.061+0000 7eff3be1b640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7eff1806bd30 0x7eff1806e1f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7eff24034000 tx=0x7eff2402e000 comp rx=0 tx=0).stop 2024-04-25T01:31:12.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.061+0000 7eff3be1b640 1 -- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff341596a0 msgr2=0x7eff341ef260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.061+0000 7eff3be1b640 1 --2- 172.21.15.112:0/3248639644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff341596a0 0x7eff341ef260 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7eff2c00c730 tx=0x7eff2c00cbf0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.061+0000 7eff3a3b8640 1 -- 172.21.15.112:0/3248639644 reap_dead start 2024-04-25T01:31:12.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.061+0000 7eff3a3b8640 1 -- 172.21.15.112:0/3248639644 reap_dead start 2024-04-25T01:31:12.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.061+0000 7eff3be1b640 1 -- 172.21.15.112:0/3248639644 shutdown_connections 2024-04-25T01:31:12.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.061+0000 7eff3be1b640 1 -- 172.21.15.112:0/3248639644 >> 172.21.15.112:0/3248639644 conn(0x7eff340cb170 msgr2=0x7eff340ac2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:12.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.062+0000 7eff3be1b640 1 -- 172.21.15.112:0/3248639644 shutdown_connections 2024-04-25T01:31:12.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.062+0000 7eff3be1b640 1 -- 172.21.15.112:0/3248639644 wait complete. 2024-04-25T01:31:12.053 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:31:12.053 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:31:12.053 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:31:12.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.195+0000 7fc50c95c640 1 Processor -- start 2024-04-25T01:31:12.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.195+0000 7fc50c95c640 1 -- start start 2024-04-25T01:31:12.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.196+0000 7fc50c95c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc50806b160 0x7fc50806b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.196+0000 7fc50c95c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc5080a9490 0x7fc50806ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.196+0000 7fc50c95c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc508156460 con 0x7fc508158ff0 2024-04-25T01:31:12.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.196+0000 7fc50c95c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fc5080c4a50 con 0x7fc5080a9490 2024-04-25T01:31:12.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.196+0000 7fc50c95c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc508156740 con 0x7fc50806b160 2024-04-25T01:31:12.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.196+0000 7fc4fffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc5080a9490 0x7fc50806ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.196+0000 7fc4fffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc5080a9490 0x7fc50806ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50266/0 (socket says 172.21.15.112:50266) 2024-04-25T01:31:12.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.196+0000 7fc4fffff640 1 -- 172.21.15.112:0/2017713896 learned_addr learned my addr 172.21.15.112:0/2017713896 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:12.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.196+0000 7fc50659c640 1 --2- 172.21.15.112:0/2017713896 >> v2:172.21.15.187:3300/0 conn(0x7fc50806b160 0x7fc50806b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.197+0000 7fc4fffff640 1 -- 172.21.15.112:0/2017713896 >> v2:172.21.15.187:3300/0 conn(0x7fc50806b160 msgr2=0x7fc50806b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:31:12.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.197+0000 7fc4fffff640 1 --2- 172.21.15.112:0/2017713896 >> v2:172.21.15.187:3300/0 conn(0x7fc50806b160 0x7fc50806b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.197+0000 7fc4fffff640 1 -- 172.21.15.112:0/2017713896 >> v1:172.21.15.187:6789/0 conn(0x7fc508158ff0 legacy=0x7fc508159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.197+0000 7fc4fffff640 1 -- 172.21.15.112:0/2017713896 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5080c2c80 con 0x7fc5080a9490 2024-04-25T01:31:12.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.197+0000 7fc4fffff640 1 --2- 172.21.15.112:0/2017713896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5080a9490 0x7fc50806ba80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc5080c5a80 tx=0x7fc4f002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72045cfd3101519 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:12.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.197+0000 7fc505d9b640 1 -- 172.21.15.112:0/2017713896 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0037070 con 0x7fc5080a9490 2024-04-25T01:31:12.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.198+0000 7fc505d9b640 1 -- 172.21.15.112:0/2017713896 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f0032440 con 0x7fc5080a9490 2024-04-25T01:31:12.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.198+0000 7fc505d9b640 1 -- 172.21.15.112:0/2017713896 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0032810 con 0x7fc5080a9490 2024-04-25T01:31:12.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.198+0000 7fc50c95c640 1 -- 172.21.15.112:0/2017713896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5080a9490 msgr2=0x7fc50806ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.198+0000 7fc50c95c640 1 --2- 172.21.15.112:0/2017713896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5080a9490 0x7fc50806ba80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc5080c5a80 tx=0x7fc4f002a1e0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.198+0000 7fc50c95c640 1 -- 172.21.15.112:0/2017713896 shutdown_connections 2024-04-25T01:31:12.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.198+0000 7fc50c95c640 1 --2- 172.21.15.112:0/2017713896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5080a9490 0x7fc50806ba80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.198+0000 7fc50c95c640 1 --2- 172.21.15.112:0/2017713896 >> v2:172.21.15.187:3300/0 conn(0x7fc50806b160 0x7fc50806b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.198+0000 7fc50c95c640 1 -- 172.21.15.112:0/2017713896 >> 172.21.15.112:0/2017713896 conn(0x7fc5080c4c00 msgr2=0x7fc5080c4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:12.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.198+0000 7fc50c95c640 1 -- 172.21.15.112:0/2017713896 shutdown_connections 2024-04-25T01:31:12.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.198+0000 7fc50c95c640 1 -- 172.21.15.112:0/2017713896 wait complete. 2024-04-25T01:31:12.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.199+0000 7fc50c95c640 1 Processor -- start 2024-04-25T01:31:12.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.199+0000 7fc50c95c640 1 -- start start 2024-04-25T01:31:12.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.200+0000 7fc50c95c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc50806b160 0x7fc50813a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.200+0000 7fc50c95c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5080a9490 0x7fc5081ec0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.200+0000 7fc50c95c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc508158ff0 0x7fc5081581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.200+0000 7fc50c95c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc508159b70 con 0x7fc508158ff0 2024-04-25T01:31:12.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.200+0000 7fc50c95c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc50806bfc0 con 0x7fc50806b160 2024-04-25T01:31:12.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.200+0000 7fc50c95c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc508159cf0 con 0x7fc5080a9490 2024-04-25T01:31:12.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.200+0000 7fc506d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc508158ff0 0x7fc5081581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.200+0000 7fc506d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc508158ff0 0x7fc5081581e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50278/0 (socket says 172.21.15.112:50278) 2024-04-25T01:31:12.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.200+0000 7fc506d9d640 1 -- 172.21.15.112:0/2565234594 learned_addr learned my addr 172.21.15.112:0/2565234594 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:12.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.200+0000 7fc50659c640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc50806b160 0x7fc50813a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.200+0000 7fc4fffff640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5080a9490 0x7fc5081ec0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.201+0000 7fc506d9d640 1 -- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5080a9490 msgr2=0x7fc5081ec0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.201+0000 7fc506d9d640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5080a9490 0x7fc5081ec0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.201+0000 7fc506d9d640 1 -- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc50806b160 msgr2=0x7fc50813a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.201+0000 7fc506d9d640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc50806b160 0x7fc50813a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.201+0000 7fc506d9d640 1 -- 172.21.15.112:0/2565234594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5081537b0 con 0x7fc508158ff0 2024-04-25T01:31:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.201+0000 7fc4fffff640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5080a9490 0x7fc5081ec0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:31:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.201+0000 7fc50659c640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc50806b160 0x7fc50813a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:31:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.201+0000 7fc506d9d640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc508158ff0 0x7fc5081581e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc50000c6e0 tx=0x7fc50000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.201+0000 7fc4feffd640 1 -- 172.21.15.112:0/2565234594 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500003b70 con 0x7fc508158ff0 2024-04-25T01:31:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.201+0000 7fc4feffd640 1 -- 172.21.15.112:0/2565234594 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc500003d30 con 0x7fc508158ff0 2024-04-25T01:31:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.201+0000 7fc50c95c640 1 -- 172.21.15.112:0/2565234594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5081553a0 con 0x7fc508158ff0 2024-04-25T01:31:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.201+0000 7fc4feffd640 1 -- 172.21.15.112:0/2565234594 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500005310 con 0x7fc508158ff0 2024-04-25T01:31:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.201+0000 7fc50c95c640 1 -- 172.21.15.112:0/2565234594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc508155960 con 0x7fc508158ff0 2024-04-25T01:31:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.202+0000 7fc50c95c640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc508136600 0x7fc5081369e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.202+0000 7fc50c95c640 1 -- 172.21.15.112:0/2565234594 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc508153c90 con 0x7fc508136600 2024-04-25T01:31:12.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.202+0000 7fc50659c640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc508136600 0x7fc5081369e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.202+0000 7fc50659c640 1 -- 172.21.15.112:0/2565234594 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc508075980 con 0x7fc508136600 2024-04-25T01:31:12.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.203+0000 7fc4feffd640 1 -- 172.21.15.112:0/2565234594 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc5000054b0 con 0x7fc508158ff0 2024-04-25T01:31:12.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.203+0000 7fc50659c640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc508136600 0x7fc5081369e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8011d20 tx=0x7fc4f800f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:12.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.203+0000 7fc4feffd640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc4e806be60 0x7fc4e806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.203+0000 7fc4fffff640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc4e806be60 0x7fc4e806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.203+0000 7fc4feffd640 1 -- 172.21.15.112:0/2565234594 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fc50008ae70 con 0x7fc508158ff0 2024-04-25T01:31:12.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.203+0000 7fc4feffd640 1 -- 172.21.15.112:0/2565234594 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8019070 con 0x7fc508136600 2024-04-25T01:31:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.203+0000 7fc4feffd640 1 -- 172.21.15.112:0/2565234594 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc4f8004030 con 0x7fc508136600 2024-04-25T01:31:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.203+0000 7fc4feffd640 1 -- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc508136600 msgr2=0x7fc5081369e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.203+0000 7fc4feffd640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc508136600 0x7fc5081369e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8011d20 tx=0x7fc4f800f610 comp rx=0 tx=0).stop 2024-04-25T01:31:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.203+0000 7fc4feffd640 1 -- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc508136600 msgr2=0x7fc5081369e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:31:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.203+0000 7fc4feffd640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc508136600 0x7fc5081369e0 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8011d20 tx=0x7fc4f800f610 comp rx=0 tx=0).stop 2024-04-25T01:31:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.203+0000 7fc4fffff640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc4e806be60 0x7fc4e806e320 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0034000 tx=0x7fc4f002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.213+0000 7fc50c95c640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc508153f50 0x7fc508154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.213+0000 7fc50c95c640 1 -- 172.21.15.112:0/2565234594 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc508154870 con 0x7fc508153f50 2024-04-25T01:31:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.213+0000 7fc50659c640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc508153f50 0x7fc508154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.213+0000 7fc50659c640 1 -- 172.21.15.112:0/2565234594 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc5080682a0 con 0x7fc508153f50 2024-04-25T01:31:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.214+0000 7fc50659c640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc508153f50 0x7fc508154330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8011d20 tx=0x7fc4f80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.214+0000 7fc4feffd640 1 -- 172.21.15.112:0/2565234594 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8019040 con 0x7fc508153f50 2024-04-25T01:31:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.214+0000 7fc4feffd640 1 -- 172.21.15.112:0/2565234594 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5069 (secure 0 0 0) 0x7fc4f801e660 con 0x7fc508153f50 2024-04-25T01:31:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.214+0000 7fc4feffd640 1 -- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc508153f50 msgr2=0x7fc508154330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.214+0000 7fc4feffd640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc508153f50 0x7fc508154330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8011d20 tx=0x7fc4f80046c0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.214+0000 7fc4feffd640 1 -- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc508153f50 msgr2=0x7fc508154330 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:31:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.214+0000 7fc4feffd640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc508153f50 0x7fc508154330 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8011d20 tx=0x7fc4f80046c0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.193 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:31:12.193 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:31:12.193 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:31:12.193 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:31:12.193 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 38, 2024-04-25T01:31:12.193 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 107903, 2024-04-25T01:31:12.193 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:31:12.194 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:31:12.194 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:31:12.194 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:31:12.194 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:12.194 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:31:12.194 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:31:12.194 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:31:12.194 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:31:12.194 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:12.194 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:12.194 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:12.194 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:12.195 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:12.195 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:12.195 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:12.195 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:12.195 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:12.195 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:12.195 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:12.195 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:12.195 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:31:12.195 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:31:12.195 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:12.196 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:12.196 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:12.196 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:12.196 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:12.196 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:12.196 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:12.196 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:12.196 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:12.196 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:12.196 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:12.197 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:12.197 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.197 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:31:12.197 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:31:12.197 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:31:12.197 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:31:12.197 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:31:12.197 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:31:12.197 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:31:12.197 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:31:12.197 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:31:12.197 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:31:12.198 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:31:12.198 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:31:12.198 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:31:12.198 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:31:12.198 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:31:12.198 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:31:12.198 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:31:12.198 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:12.198 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:12.198 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:12.198 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:12.198 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:12.198 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:12.198 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:12.198 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:12.199 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:12.199 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:12.199 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:12.199 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:12.199 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:31:12.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.199 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:31:12.199 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.199 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:31:12.199 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:31:12.199 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:12.199 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:12.199 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.200 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:12.200 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:31:12.200 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:12.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.200 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.200 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:12.200 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:31:12.200 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:12.200 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.200 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:12.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.200 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:31:12.200 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:31:12.200 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:12.200 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:12.201 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:12.201 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.201 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.201 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:31:12.201 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:31:12.201 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:12.201 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:12.201 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.201 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:12.201 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:31:12.201 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:12.201 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.201 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.202 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:12.202 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:31:12.202 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:12.202 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.202 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:12.202 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.202 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:31:12.203 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:31:12.203 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:12.203 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:12.203 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:12.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.203 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.203 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:31:12.203 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:31:12.203 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:12.203 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:12.203 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.203 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:12.203 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:31:12.203 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:12.204 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.204 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.204 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:12.204 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:31:12.204 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:12.204 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.204 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:12.204 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.204 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:31:12.204 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:31:12.204 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:12.204 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:12.205 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:12.205 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.205 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:12.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.205 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:31:12.205 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:31:12.205 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.205 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:12.205 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:12.205 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:31:12.205 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.205 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:12.205 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:31:12.205 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.206 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:12.206 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:12.206 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:31:12.206 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.206 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:12.206 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:31:12.206 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.206 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:12.206 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:12.206 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:31:12.206 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.206 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:12.206 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:31:12.206 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.206 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:12.207 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:12.207 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:31:12.207 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.207 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:12.207 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:31:12.207 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.207 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:12.207 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:12.207 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:31:12.207 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.207 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:12.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.207 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:31:12.208 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:31:12.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.214+0000 7fc50c95c640 1 -- 172.21.15.112:0/2565234594 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc4e806be60 msgr2=0x7fc4e806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.214+0000 7fc50c95c640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc4e806be60 0x7fc4e806e320 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0034000 tx=0x7fc4f002e000 comp rx=0 tx=0).stop 2024-04-25T01:31:12.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.214+0000 7fc50c95c640 1 -- 172.21.15.112:0/2565234594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc508158ff0 msgr2=0x7fc5081581e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.214+0000 7fc50c95c640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc508158ff0 0x7fc5081581e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc50000c6e0 tx=0x7fc50000cba0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.214+0000 7fc506d9d640 1 -- 172.21.15.112:0/2565234594 reap_dead start 2024-04-25T01:31:12.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.214+0000 7fc50c95c640 1 -- 172.21.15.112:0/2565234594 shutdown_connections 2024-04-25T01:31:12.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.215+0000 7fc50c95c640 1 --2- 172.21.15.112:0/2565234594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc508158ff0 0x7fc5081581e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.215+0000 7fc50c95c640 1 -- 172.21.15.112:0/2565234594 >> 172.21.15.112:0/2565234594 conn(0x7fc5080c4c00 msgr2=0x7fc5080c2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:12.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.215+0000 7fc50c95c640 1 -- 172.21.15.112:0/2565234594 shutdown_connections 2024-04-25T01:31:12.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.215+0000 7fc50c95c640 1 -- 172.21.15.112:0/2565234594 wait complete. 2024-04-25T01:31:12.210 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:31:12.210 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:31:12.210 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:31:12.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.342+0000 7f54afe31640 1 Processor -- start 2024-04-25T01:31:12.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.342+0000 7f54afe31640 1 -- start start 2024-04-25T01:31:12.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.343+0000 7f54afe31640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a806c0c0 0x7f54a80a8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.343+0000 7f54afe31640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54a81548e0 con 0x7f54a81567c0 2024-04-25T01:31:12.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.343+0000 7f54afe31640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54a80a9d10 con 0x7f54a806c610 2024-04-25T01:31:12.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.343+0000 7f54afe31640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f54a80aa320 con 0x7f54a806c0c0 2024-04-25T01:31:12.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.343+0000 7f54adbcd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f54a81567c0 0x7f54a80743d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:35918/0 (socket says 172.21.15.112:35918) 2024-04-25T01:31:12.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.343+0000 7f54adbcd640 1 -- 172.21.15.112:0/553399896 learned_addr learned my addr 172.21.15.112:0/553399896 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:31:12.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.344+0000 7f54ae3ce640 1 --2- 172.21.15.112:0/553399896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a806c0c0 0x7f54a80a8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.344+0000 7f54acbcb640 1 -- 172.21.15.112:0/553399896 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1178559664 0 0) 0x7f54a81548e0 con 0x7f54a81567c0 2024-04-25T01:31:12.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.344+0000 7f54acbcb640 1 -- 172.21.15.112:0/553399896 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5494003520 con 0x7f54a81567c0 2024-04-25T01:31:12.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.344+0000 7f54acbcb640 1 -- 172.21.15.112:0/553399896 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2436439902 0 0) 0x7f54a80a9d10 con 0x7f54a806c610 2024-04-25T01:31:12.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.344+0000 7f54acbcb640 1 -- 172.21.15.112:0/553399896 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54a81548e0 con 0x7f54a806c610 2024-04-25T01:31:12.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.344+0000 7f54ae3ce640 1 -- 172.21.15.112:0/553399896 >> v1:172.21.15.187:6789/0 conn(0x7f54a806c610 legacy=0x7f54a80a7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.344+0000 7f54ae3ce640 1 -- 172.21.15.112:0/553399896 >> v1:172.21.15.112:6789/0 conn(0x7f54a81567c0 legacy=0x7f54a80743d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.344+0000 7f54ae3ce640 1 -- 172.21.15.112:0/553399896 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a80a8650 con 0x7f54a806c0c0 2024-04-25T01:31:12.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.345+0000 7f54ae3ce640 1 --2- 172.21.15.112:0/553399896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a806c0c0 0x7f54a80a8110 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009dc0 tx=0x7f54a002a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eac6d6869d449956 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:12.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.345+0000 7f54acbcb640 1 -- 172.21.15.112:0/553399896 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0007740 con 0x7f54a806c0c0 2024-04-25T01:31:12.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.345+0000 7f54acbcb640 1 -- 172.21.15.112:0/553399896 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54a00078e0 con 0x7f54a806c0c0 2024-04-25T01:31:12.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.345+0000 7f54acbcb640 1 -- 172.21.15.112:0/553399896 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0007cb0 con 0x7f54a806c0c0 2024-04-25T01:31:12.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.346+0000 7f54afe31640 1 -- 172.21.15.112:0/553399896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a806c0c0 msgr2=0x7f54a80a8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.346+0000 7f54afe31640 1 --2- 172.21.15.112:0/553399896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a806c0c0 0x7f54a80a8110 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009dc0 tx=0x7f54a002a5b0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.346+0000 7f54afe31640 1 -- 172.21.15.112:0/553399896 shutdown_connections 2024-04-25T01:31:12.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.346+0000 7f54afe31640 1 --2- 172.21.15.112:0/553399896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a806c0c0 0x7f54a80a8110 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.346+0000 7f54afe31640 1 -- 172.21.15.112:0/553399896 >> 172.21.15.112:0/553399896 conn(0x7f54a806b160 msgr2=0x7f54a806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:12.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.346+0000 7f54afe31640 1 -- 172.21.15.112:0/553399896 shutdown_connections 2024-04-25T01:31:12.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.346+0000 7f54afe31640 1 -- 172.21.15.112:0/553399896 wait complete. 2024-04-25T01:31:12.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.347+0000 7f54afe31640 1 Processor -- start 2024-04-25T01:31:12.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.347+0000 7f54afe31640 1 -- start start 2024-04-25T01:31:12.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.348+0000 7f54afe31640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a806c0c0 0x7f54a814fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.348+0000 7f54afe31640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54a806c610 0x7f54a80c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.348+0000 7f54afe31640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54a81567c0 0x7f54a81ee990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.348+0000 7f54afe31640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54a80a9fb0 con 0x7f54a81567c0 2024-04-25T01:31:12.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.348+0000 7f54afe31640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f54a8154600 con 0x7f54a806c610 2024-04-25T01:31:12.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.348+0000 7f54afe31640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f54a8156ba0 con 0x7f54a806c0c0 2024-04-25T01:31:12.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.348+0000 7f54ae3ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54a81567c0 0x7f54a81ee990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.348+0000 7f54adbcd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a806c0c0 0x7f54a814fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.348+0000 7f54adbcd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a806c0c0 0x7f54a814fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:52306/0 (socket says 172.21.15.112:52306) 2024-04-25T01:31:12.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.348+0000 7f54adbcd640 1 -- 172.21.15.112:0/1825635278 learned_addr learned my addr 172.21.15.112:0/1825635278 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:12.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.348+0000 7f54ad3cc640 1 --2- 172.21.15.112:0/1825635278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54a806c610 0x7f54a80c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.349+0000 7f54ae3ce640 1 -- 172.21.15.112:0/1825635278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a806c0c0 msgr2=0x7f54a814fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.349+0000 7f54ae3ce640 1 --2- 172.21.15.112:0/1825635278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a806c0c0 0x7f54a814fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.349+0000 7f54ae3ce640 1 -- 172.21.15.112:0/1825635278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54a806c610 msgr2=0x7f54a80c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.349+0000 7f54ae3ce640 1 --2- 172.21.15.112:0/1825635278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54a806c610 0x7f54a80c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.349+0000 7f54ae3ce640 1 -- 172.21.15.112:0/1825635278 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a81ec250 con 0x7f54a81567c0 2024-04-25T01:31:12.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.349+0000 7f54ae3ce640 1 --2- 172.21.15.112:0/1825635278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54a81567c0 0x7f54a81ee990 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f54a0031000 tx=0x7f54a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:12.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.349+0000 7f549e7fc640 1 -- 172.21.15.112:0/1825635278 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0040070 con 0x7f54a81567c0 2024-04-25T01:31:12.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.349+0000 7f549e7fc640 1 -- 172.21.15.112:0/1825635278 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54a0031bc0 con 0x7f54a81567c0 2024-04-25T01:31:12.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.349+0000 7f549e7fc640 1 -- 172.21.15.112:0/1825635278 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0009450 con 0x7f54a81567c0 2024-04-25T01:31:12.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.349+0000 7f54afe31640 1 -- 172.21.15.112:0/1825635278 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a8073030 con 0x7f54a81567c0 2024-04-25T01:31:12.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.349+0000 7f54afe31640 1 -- 172.21.15.112:0/1825635278 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a8150290 con 0x7f54a81567c0 2024-04-25T01:31:12.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.350+0000 7f54afe31640 1 --2- 172.21.15.112:0/1825635278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a809b3b0 0x7f54a81eeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.350+0000 7f54afe31640 1 -- 172.21.15.112:0/1825635278 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f54a809c960 con 0x7f54a809b3b0 2024-04-25T01:31:12.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.350+0000 7f54adbcd640 1 --2- 172.21.15.112:0/1825635278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a809b3b0 0x7f54a81eeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.350+0000 7f54adbcd640 1 -- 172.21.15.112:0/1825635278 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54a81ebd20 con 0x7f54a809b3b0 2024-04-25T01:31:12.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.350+0000 7f549e7fc640 1 -- 172.21.15.112:0/1825635278 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54a0039040 con 0x7f54a81567c0 2024-04-25T01:31:12.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.351+0000 7f54adbcd640 1 --2- 172.21.15.112:0/1825635278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a809b3b0 0x7f54a81eeed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f54a4001520 tx=0x7f54a40019e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:12.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.351+0000 7f549e7fc640 1 --2- 172.21.15.112:0/1825635278 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f548c06be30 0x7f548c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.351+0000 7f549e7fc640 1 -- 172.21.15.112:0/1825635278 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f54a00ac720 con 0x7f54a81567c0 2024-04-25T01:31:12.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.351+0000 7f54ad3cc640 1 --2- 172.21.15.112:0/1825635278 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f548c06be30 0x7f548c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.351+0000 7f549e7fc640 1 -- 172.21.15.112:0/1825635278 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4005ac0 con 0x7f54a809b3b0 2024-04-25T01:31:12.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.351+0000 7f549e7fc640 1 -- 172.21.15.112:0/1825635278 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f54a4018070 con 0x7f54a809b3b0 2024-04-25T01:31:12.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.351+0000 7f549e7fc640 1 -- 172.21.15.112:0/1825635278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a809b3b0 msgr2=0x7f54a81eeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.351+0000 7f549e7fc640 1 --2- 172.21.15.112:0/1825635278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a809b3b0 0x7f54a81eeed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f54a4001520 tx=0x7f54a40019e0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.351+0000 7f549e7fc640 1 -- 172.21.15.112:0/1825635278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a809b3b0 msgr2=0x7f54a81eeed0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:31:12.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.351+0000 7f549e7fc640 1 --2- 172.21.15.112:0/1825635278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a809b3b0 0x7f54a81eeed0 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f54a4001520 tx=0x7f54a40019e0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.352+0000 7f54ad3cc640 1 --2- 172.21.15.112:0/1825635278 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f548c06be30 0x7f548c06e2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f54a809b970 tx=0x7f5498009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:12.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.360+0000 7f54afe31640 1 --2- 172.21.15.112:0/1825635278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a81ef410 0x7f54a80ab570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.360+0000 7f54afe31640 1 -- 172.21.15.112:0/1825635278 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f54a8075980 con 0x7f54a81ef410 2024-04-25T01:31:12.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.360+0000 7f54adbcd640 1 --2- 172.21.15.112:0/1825635278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a81ef410 0x7f54a80ab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.360+0000 7f54adbcd640 1 -- 172.21.15.112:0/1825635278 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f54a80689b0 con 0x7f54a81ef410 2024-04-25T01:31:12.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.361+0000 7f54adbcd640 1 --2- 172.21.15.112:0/1825635278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a81ef410 0x7f54a80ab570 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f54a4001520 tx=0x7f54a4001150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:12.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.361+0000 7f549e7fc640 1 -- 172.21.15.112:0/1825635278 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4004cb0 con 0x7f54a81ef410 2024-04-25T01:31:12.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.361+0000 7f549e7fc640 1 -- 172.21.15.112:0/1825635278 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4898 (secure 0 0 0) 0x7f54a4004cb0 con 0x7f54a81ef410 2024-04-25T01:31:12.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.361+0000 7f549e7fc640 1 -- 172.21.15.112:0/1825635278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a81ef410 msgr2=0x7f54a80ab570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.361+0000 7f549e7fc640 1 --2- 172.21.15.112:0/1825635278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a81ef410 0x7f54a80ab570 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f54a4001520 tx=0x7f54a4001150 comp rx=0 tx=0).stop 2024-04-25T01:31:12.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.361+0000 7f549e7fc640 1 -- 172.21.15.112:0/1825635278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a81ef410 msgr2=0x7f54a80ab570 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:31:12.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.361+0000 7f549e7fc640 1 --2- 172.21.15.112:0/1825635278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a81ef410 0x7f54a80ab570 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.342 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:31:12.342 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:31:12.342 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:31:12.342 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:31:12.342 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 38, 2024-04-25T01:31:12.342 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 61100, 2024-04-25T01:31:12.342 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:31:12.343 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:31:12.343 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:31:12.343 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:31:12.343 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:12.343 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:31:12.343 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:31:12.343 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:31:12.343 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:31:12.343 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:12.343 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:12.343 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:12.343 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:12.343 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:12.343 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:12.343 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:12.344 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:12.344 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:12.344 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:12.344 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:12.344 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:12.344 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:31:12.344 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:31:12.344 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:12.344 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:12.344 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:12.344 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:12.344 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:12.344 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:12.344 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:12.344 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:12.345 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:12.345 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:12.345 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:12.345 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:12.345 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.345 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:31:12.345 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:31:12.345 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:31:12.345 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:31:12.345 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:31:12.345 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:31:12.345 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:31:12.345 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:31:12.345 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:31:12.345 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:31:12.346 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:31:12.346 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:31:12.346 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:31:12.346 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:31:12.346 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:31:12.346 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:31:12.346 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:31:12.346 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:12.346 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:12.346 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:12.346 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:12.346 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:12.346 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:12.346 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:12.346 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:12.347 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:12.347 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:12.347 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:12.347 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:12.347 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:31:12.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.347 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:31:12.347 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.347 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:31:12.347 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:31:12.347 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:12.347 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:12.347 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.347 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:12.347 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:31:12.348 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:12.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.348 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.348 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:12.348 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:31:12.348 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:12.348 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.348 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:12.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.348 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:31:12.348 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:31:12.348 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:12.348 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:12.348 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:12.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.349 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.349 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:31:12.349 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:31:12.349 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:12.349 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:12.349 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.349 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:12.349 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:31:12.349 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:12.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.349 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.349 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:12.349 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:31:12.349 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:12.349 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.350 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:12.350 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.350 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:31:12.350 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:31:12.350 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:12.350 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:12.350 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:12.350 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.350 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.350 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:31:12.350 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:31:12.351 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:12.351 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:12.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.351 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:12.351 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:31:12.351 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:12.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.351 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:12.351 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:31:12.351 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:12.351 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.352 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:12.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.352 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:31:12.352 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:31:12.352 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:12.352 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:12.352 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:12.352 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.352 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:12.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.352 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:31:12.353 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:31:12.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.353 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:12.353 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:12.353 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:31:12.353 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.353 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:12.353 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:31:12.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.353 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:12.353 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:12.353 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:31:12.353 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.353 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:12.354 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:31:12.354 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.354 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:12.354 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:12.354 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:31:12.354 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.354 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:12.354 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:31:12.354 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:12.354 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:12.354 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:12.354 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:31:12.354 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:12.354 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:12.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:12.355 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:31:12.355 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:31:12.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.362+0000 7f54afe31640 1 -- 172.21.15.112:0/1825635278 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f548c06be30 msgr2=0x7f548c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.362+0000 7f54afe31640 1 --2- 172.21.15.112:0/1825635278 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f548c06be30 0x7f548c06e2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f54a809b970 tx=0x7f5498009040 comp rx=0 tx=0).stop 2024-04-25T01:31:12.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.362+0000 7f54afe31640 1 -- 172.21.15.112:0/1825635278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54a81567c0 msgr2=0x7f54a81ee990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.362+0000 7f54afe31640 1 --2- 172.21.15.112:0/1825635278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54a81567c0 0x7f54a81ee990 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f54a0031000 tx=0x7f54a0004a40 comp rx=0 tx=0).stop 2024-04-25T01:31:12.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.362+0000 7f54ae3ce640 1 -- 172.21.15.112:0/1825635278 reap_dead start 2024-04-25T01:31:12.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.362+0000 7f54ae3ce640 1 -- 172.21.15.112:0/1825635278 reap_dead start 2024-04-25T01:31:12.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.362+0000 7f54afe31640 1 -- 172.21.15.112:0/1825635278 shutdown_connections 2024-04-25T01:31:12.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.362+0000 7f54afe31640 1 -- 172.21.15.112:0/1825635278 >> 172.21.15.112:0/1825635278 conn(0x7f54a806b160 msgr2=0x7f54a80a7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:12.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.362+0000 7f54afe31640 1 -- 172.21.15.112:0/1825635278 shutdown_connections 2024-04-25T01:31:12.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.362+0000 7f54afe31640 1 -- 172.21.15.112:0/1825635278 wait complete. 2024-04-25T01:31:12.357 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:31:12.357 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T01:31:12.357 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T01:31:12.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.488+0000 7fef01114640 1 Processor -- start 2024-04-25T01:31:12.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.488+0000 7fef01114640 1 -- start start 2024-04-25T01:31:12.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.489+0000 7fef01114640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feefc14f7e0 0x7feefc06bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.489+0000 7fef01114640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feefc1558a0 0x7feefc1574a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.489+0000 7fef01114640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feefc1579e0 con 0x7feefc0caa40 2024-04-25T01:31:12.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.489+0000 7fef01114640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7feefc159aa0 con 0x7feefc1558a0 2024-04-25T01:31:12.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.489+0000 7fef01114640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7feefc158e90 con 0x7feefc14f7e0 2024-04-25T01:31:12.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.489+0000 7feefb59e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feefc1558a0 0x7feefc1574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.489+0000 7feefa59c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7feefc0caa40 0x7feefc156d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:35934/0 (socket says 172.21.15.112:35934) 2024-04-25T01:31:12.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.489+0000 7feefb59e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feefc1558a0 0x7feefc1574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50294/0 (socket says 172.21.15.112:50294) 2024-04-25T01:31:12.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.489+0000 7feefa59c640 1 -- 172.21.15.112:0/3761134682 learned_addr learned my addr 172.21.15.112:0/3761134682 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:31:12.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.489+0000 7feefad9d640 1 --2- 172.21.15.112:0/3761134682 >> v2:172.21.15.187:3300/0 conn(0x7feefc14f7e0 0x7feefc06bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.490+0000 7feefb59e640 1 -- 172.21.15.112:0/3761134682 >> v2:172.21.15.187:3300/0 conn(0x7feefc14f7e0 msgr2=0x7feefc06bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:31:12.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.490+0000 7feefb59e640 1 --2- 172.21.15.112:0/3761134682 >> v2:172.21.15.187:3300/0 conn(0x7feefc14f7e0 0x7feefc06bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.490+0000 7feefb59e640 1 -- 172.21.15.112:0/3761134682 >> v1:172.21.15.112:6789/0 conn(0x7feefc0caa40 legacy=0x7feefc156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.490+0000 7feef9d9b640 1 -- 172.21.15.112:0/3761134682 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 751920824 0 0) 0x7feefc1579e0 con 0x7feefc0caa40 2024-04-25T01:31:12.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.490+0000 7feefb59e640 1 -- 172.21.15.112:0/3761134682 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc159170 con 0x7feefc1558a0 2024-04-25T01:31:12.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.490+0000 7feefb59e640 1 --2- 172.21.15.112:0/3761134682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc1558a0 0x7feefc1574a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7feefc0c8670 tx=0x7feef402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26e189f2a6fe34c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:12.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.491+0000 7feef9d9b640 1 -- 172.21.15.112:0/3761134682 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4007740 con 0x7feefc1558a0 2024-04-25T01:31:12.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.491+0000 7feef9d9b640 1 -- 172.21.15.112:0/3761134682 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef40078e0 con 0x7feefc1558a0 2024-04-25T01:31:12.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.491+0000 7feef9d9b640 1 -- 172.21.15.112:0/3761134682 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4007cb0 con 0x7feefc1558a0 2024-04-25T01:31:12.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.491+0000 7fef01114640 1 -- 172.21.15.112:0/3761134682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc1558a0 msgr2=0x7feefc1574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.491+0000 7fef01114640 1 --2- 172.21.15.112:0/3761134682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc1558a0 0x7feefc1574a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7feefc0c8670 tx=0x7feef402a680 comp rx=0 tx=0).stop 2024-04-25T01:31:12.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.491+0000 7fef01114640 1 -- 172.21.15.112:0/3761134682 shutdown_connections 2024-04-25T01:31:12.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.491+0000 7fef01114640 1 --2- 172.21.15.112:0/3761134682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc1558a0 0x7feefc1574a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.491+0000 7fef01114640 1 --2- 172.21.15.112:0/3761134682 >> v2:172.21.15.187:3300/0 conn(0x7feefc14f7e0 0x7feefc06bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.491+0000 7fef01114640 1 -- 172.21.15.112:0/3761134682 >> 172.21.15.112:0/3761134682 conn(0x7feefc0acc90 msgr2=0x7feefc0ad0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:12.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.491+0000 7fef01114640 1 -- 172.21.15.112:0/3761134682 shutdown_connections 2024-04-25T01:31:12.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.491+0000 7fef01114640 1 -- 172.21.15.112:0/3761134682 wait complete. 2024-04-25T01:31:12.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.493+0000 7fef01114640 1 Processor -- start 2024-04-25T01:31:12.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.493+0000 7fef01114640 1 -- start start 2024-04-25T01:31:12.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.494+0000 7fef01114640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feefc0caa40 0x7feefc0abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.494+0000 7fef01114640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc14f7e0 0x7feefc0c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.494+0000 7fef01114640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feefc1558a0 0x7feefc0c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.494+0000 7fef01114640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feefc153450 con 0x7feefc14f7e0 2024-04-25T01:31:12.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.494+0000 7fef01114640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feefc159730 con 0x7feefc1558a0 2024-04-25T01:31:12.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.494+0000 7fef01114640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feefc14fbc0 con 0x7feefc0caa40 2024-04-25T01:31:12.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.494+0000 7feefa59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc14f7e0 0x7feefc0c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.494+0000 7feefa59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc14f7e0 0x7feefc0c5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50300/0 (socket says 172.21.15.112:50300) 2024-04-25T01:31:12.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.494+0000 7feefb59e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feefc1558a0 0x7feefc0c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.494+0000 7feefa59c640 1 -- 172.21.15.112:0/357945296 learned_addr learned my addr 172.21.15.112:0/357945296 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:12.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.494+0000 7feefad9d640 1 --2- 172.21.15.112:0/357945296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feefc0caa40 0x7feefc0abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.494+0000 7feefb59e640 1 -- 172.21.15.112:0/357945296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feefc0caa40 msgr2=0x7feefc0abd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.494+0000 7feefb59e640 1 --2- 172.21.15.112:0/357945296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feefc0caa40 0x7feefc0abd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.494+0000 7feefb59e640 1 -- 172.21.15.112:0/357945296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc14f7e0 msgr2=0x7feefc0c5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.494+0000 7feefb59e640 1 --2- 172.21.15.112:0/357945296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc14f7e0 0x7feefc0c5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.494+0000 7feefb59e640 1 -- 172.21.15.112:0/357945296 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc0ae7a0 con 0x7feefc1558a0 2024-04-25T01:31:12.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.494+0000 7feefa59c640 1 --2- 172.21.15.112:0/357945296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc14f7e0 0x7feefc0c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:31:12.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.494+0000 7feefb59e640 1 --2- 172.21.15.112:0/357945296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feefc1558a0 0x7feefc0c64a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7feef4031000 tx=0x7feef4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:12.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.494+0000 7feee37fe640 1 -- 172.21.15.112:0/357945296 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef403f070 con 0x7feefc1558a0 2024-04-25T01:31:12.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.495+0000 7feee37fe640 1 -- 172.21.15.112:0/357945296 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef4031bc0 con 0x7feefc1558a0 2024-04-25T01:31:12.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.495+0000 7fef01114640 1 -- 172.21.15.112:0/357945296 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feefc0c5a80 con 0x7feefc1558a0 2024-04-25T01:31:12.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.495+0000 7feee37fe640 1 -- 172.21.15.112:0/357945296 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4009450 con 0x7feefc1558a0 2024-04-25T01:31:12.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.495+0000 7fef01114640 1 -- 172.21.15.112:0/357945296 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feefc0c77f0 con 0x7feefc1558a0 2024-04-25T01:31:12.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.496+0000 7fef01114640 1 -- 172.21.15.112:0/357945296 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feefc0c5c50 con 0x7feefc1558a0 2024-04-25T01:31:12.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.496+0000 7feee37fe640 1 -- 172.21.15.112:0/357945296 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feef403b410 con 0x7feefc1558a0 2024-04-25T01:31:12.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.496+0000 7feee37fe640 1 --2- 172.21.15.112:0/357945296 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feee806be60 0x7feee806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:12.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.497+0000 7feee37fe640 1 -- 172.21.15.112:0/357945296 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7feef40ac790 con 0x7feefc1558a0 2024-04-25T01:31:12.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.497+0000 7feefad9d640 1 --2- 172.21.15.112:0/357945296 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feee806be60 0x7feee806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:12.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.497+0000 7feefad9d640 1 --2- 172.21.15.112:0/357945296 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feee806be60 0x7feee806e320 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feefc1ef8c0 tx=0x7feeec009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:12.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.502+0000 7feee37fe640 1 -- 172.21.15.112:0/357945296 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feef407a750 con 0x7feefc1558a0 2024-04-25T01:31:12.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.812+0000 7fef01114640 1 -- 172.21.15.112:0/357945296 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7feefc0682a0 con 0x7feefc1558a0 2024-04-25T01:31:12.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.813+0000 7feee37fe640 1 -- 172.21.15.112:0/357945296 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7feef407ea50 con 0x7feefc1558a0 2024-04-25T01:31:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.816+0000 7fef01114640 1 -- 172.21.15.112:0/357945296 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feee806be60 msgr2=0x7feee806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.816+0000 7fef01114640 1 --2- 172.21.15.112:0/357945296 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feee806be60 0x7feee806e320 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feefc1ef8c0 tx=0x7feeec009190 comp rx=0 tx=0).stop 2024-04-25T01:31:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.816+0000 7fef01114640 1 -- 172.21.15.112:0/357945296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feefc1558a0 msgr2=0x7feefc0c64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.816+0000 7fef01114640 1 --2- 172.21.15.112:0/357945296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feefc1558a0 0x7feefc0c64a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7feef4031000 tx=0x7feef4004a40 comp rx=0 tx=0).stop 2024-04-25T01:31:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.816+0000 7fef01114640 1 -- 172.21.15.112:0/357945296 shutdown_connections 2024-04-25T01:31:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.816+0000 7fef01114640 1 --2- 172.21.15.112:0/357945296 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feee806be60 0x7feee806e320 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.816+0000 7fef01114640 1 --2- 172.21.15.112:0/357945296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feefc1558a0 0x7feefc0c64a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.816+0000 7fef01114640 1 --2- 172.21.15.112:0/357945296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc14f7e0 0x7feefc0c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.816+0000 7fef01114640 1 --2- 172.21.15.112:0/357945296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feefc0caa40 0x7feefc0abd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.816+0000 7fef01114640 1 -- 172.21.15.112:0/357945296 >> 172.21.15.112:0/357945296 conn(0x7feefc0acc90 msgr2=0x7feefc0ad070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.816+0000 7fef01114640 1 -- 172.21.15.112:0/357945296 shutdown_connections 2024-04-25T01:31:12.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:12.816+0000 7fef01114640 1 -- 172.21.15.112:0/357945296 wait complete. 2024-04-25T01:31:12.805 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T01:31:22.806 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:31:22.807 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:31:22.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.936+0000 7fa57aa79640 1 Processor -- start 2024-04-25T01:31:22.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.936+0000 7fa57aa79640 1 -- start start 2024-04-25T01:31:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.937+0000 7fa57aa79640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa57414fd30 0x7fa57406f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.937+0000 7fa57aa79640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa574153010 0x7fa57406f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.937+0000 7fa57aa79640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5740c2bd0 con 0x7fa574071e20 2024-04-25T01:31:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.937+0000 7fa57aa79640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa5740a97c0 con 0x7fa57414fd30 2024-04-25T01:31:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.937+0000 7fa57aa79640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa57406eef0 con 0x7fa574153010 2024-04-25T01:31:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.937+0000 7fa578815640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa57414fd30 0x7fa57406f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.937+0000 7fa579016640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa574071e20 0x7fa574072250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37520/0 (socket says 172.21.15.112:37520) 2024-04-25T01:31:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.937+0000 7fa579016640 1 -- 172.21.15.112:0/1857593011 learned_addr learned my addr 172.21.15.112:0/1857593011 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:31:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.937+0000 7fa573fff640 1 --2- 172.21.15.112:0/1857593011 >> v2:172.21.15.187:3300/0 conn(0x7fa574153010 0x7fa57406f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.937+0000 7fa5737fe640 1 -- 172.21.15.112:0/1857593011 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2051454025 0 0) 0x7fa5740c2bd0 con 0x7fa574071e20 2024-04-25T01:31:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.937+0000 7fa5737fe640 1 -- 172.21.15.112:0/1857593011 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa564003520 con 0x7fa574071e20 2024-04-25T01:31:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.937+0000 7fa578815640 1 -- 172.21.15.112:0/1857593011 >> v2:172.21.15.187:3300/0 conn(0x7fa574153010 msgr2=0x7fa57406f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:31:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.937+0000 7fa578815640 1 --2- 172.21.15.112:0/1857593011 >> v2:172.21.15.187:3300/0 conn(0x7fa574153010 0x7fa57406f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.937+0000 7fa578815640 1 -- 172.21.15.112:0/1857593011 >> v1:172.21.15.112:6789/0 conn(0x7fa574071e20 legacy=0x7fa574072250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.937+0000 7fa578815640 1 -- 172.21.15.112:0/1857593011 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5740c2e40 con 0x7fa57414fd30 2024-04-25T01:31:22.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.938+0000 7fa578815640 1 --2- 172.21.15.112:0/1857593011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa57414fd30 0x7fa57406f380 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa57406d5c0 tx=0x7fa56802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c01cf03857d765 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:22.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.938+0000 7fa5737fe640 1 -- 172.21.15.112:0/1857593011 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568037070 con 0x7fa57414fd30 2024-04-25T01:31:22.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.939+0000 7fa5737fe640 1 -- 172.21.15.112:0/1857593011 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa568032440 con 0x7fa57414fd30 2024-04-25T01:31:22.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.939+0000 7fa5737fe640 1 -- 172.21.15.112:0/1857593011 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56802ae60 con 0x7fa57414fd30 2024-04-25T01:31:22.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.939+0000 7fa57aa79640 1 -- 172.21.15.112:0/1857593011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa57414fd30 msgr2=0x7fa57406f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:22.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.939+0000 7fa57aa79640 1 --2- 172.21.15.112:0/1857593011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa57414fd30 0x7fa57406f380 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa57406d5c0 tx=0x7fa56802a110 comp rx=0 tx=0).stop 2024-04-25T01:31:22.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.939+0000 7fa57aa79640 1 -- 172.21.15.112:0/1857593011 shutdown_connections 2024-04-25T01:31:22.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.939+0000 7fa57aa79640 1 --2- 172.21.15.112:0/1857593011 >> v2:172.21.15.187:3300/0 conn(0x7fa574153010 0x7fa57406f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:22.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.939+0000 7fa57aa79640 1 --2- 172.21.15.112:0/1857593011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa57414fd30 0x7fa57406f380 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:22.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.939+0000 7fa57aa79640 1 -- 172.21.15.112:0/1857593011 >> 172.21.15.112:0/1857593011 conn(0x7fa5740c8130 msgr2=0x7fa5740c8510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:22.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.939+0000 7fa57aa79640 1 -- 172.21.15.112:0/1857593011 shutdown_connections 2024-04-25T01:31:22.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.939+0000 7fa57aa79640 1 -- 172.21.15.112:0/1857593011 wait complete. 2024-04-25T01:31:22.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.940+0000 7fa57aa79640 1 Processor -- start 2024-04-25T01:31:22.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.940+0000 7fa57aa79640 1 -- start start 2024-04-25T01:31:22.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.941+0000 7fa57aa79640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa574071e20 0x7fa57414f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:22.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.941+0000 7fa57aa79640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa57414fd30 0x7fa57414c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:22.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.941+0000 7fa57aa79640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa574149fd0 0x7fa5741409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:22.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.941+0000 7fa57aa79640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5741534a0 con 0x7fa57414fd30 2024-04-25T01:31:22.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.941+0000 7fa57aa79640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa57406fe00 con 0x7fa574149fd0 2024-04-25T01:31:22.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.941+0000 7fa57aa79640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa5740c4810 con 0x7fa574071e20 2024-04-25T01:31:22.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.941+0000 7fa573fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa57414fd30 0x7fa57414c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:22.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.941+0000 7fa573fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa57414fd30 0x7fa57414c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54456/0 (socket says 172.21.15.112:54456) 2024-04-25T01:31:22.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.941+0000 7fa578815640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa574071e20 0x7fa57414f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:22.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.941+0000 7fa573fff640 1 -- 172.21.15.112:0/632776786 learned_addr learned my addr 172.21.15.112:0/632776786 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:22.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.941+0000 7fa579016640 1 --2- 172.21.15.112:0/632776786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa574149fd0 0x7fa5741409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:22.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.942+0000 7fa573fff640 1 -- 172.21.15.112:0/632776786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa574071e20 msgr2=0x7fa57414f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:22.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.942+0000 7fa573fff640 1 --2- 172.21.15.112:0/632776786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa574071e20 0x7fa57414f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:22.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.942+0000 7fa573fff640 1 -- 172.21.15.112:0/632776786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa574149fd0 msgr2=0x7fa5741409f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:22.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.942+0000 7fa573fff640 1 --2- 172.21.15.112:0/632776786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa574149fd0 0x7fa5741409f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:22.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.942+0000 7fa573fff640 1 -- 172.21.15.112:0/632776786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5741e7130 con 0x7fa57414fd30 2024-04-25T01:31:22.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.942+0000 7fa578815640 1 --2- 172.21.15.112:0/632776786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa574071e20 0x7fa57414f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:31:22.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.942+0000 7fa573fff640 1 --2- 172.21.15.112:0/632776786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa57414fd30 0x7fa57414c390 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa56000e870 tx=0x7fa56000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:22.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.942+0000 7fa5717fa640 1 -- 172.21.15.112:0/632776786 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560017070 con 0x7fa57414fd30 2024-04-25T01:31:22.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.942+0000 7fa5717fa640 1 -- 172.21.15.112:0/632776786 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa56000c8e0 con 0x7fa57414fd30 2024-04-25T01:31:22.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.942+0000 7fa5717fa640 1 -- 172.21.15.112:0/632776786 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56000ccd0 con 0x7fa57414fd30 2024-04-25T01:31:22.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.942+0000 7fa57aa79640 1 -- 172.21.15.112:0/632776786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5741e7410 con 0x7fa57414fd30 2024-04-25T01:31:22.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.942+0000 7fa57aa79640 1 -- 172.21.15.112:0/632776786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5741e79d0 con 0x7fa57414fd30 2024-04-25T01:31:22.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.943+0000 7fa57aa79640 1 -- 172.21.15.112:0/632776786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5740c4030 con 0x7fa57414fd30 2024-04-25T01:31:22.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.944+0000 7fa5717fa640 1 -- 172.21.15.112:0/632776786 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa56001a070 con 0x7fa57414fd30 2024-04-25T01:31:22.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.944+0000 7fa5717fa640 1 --2- 172.21.15.112:0/632776786 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa55806be60 0x7fa55806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:22.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.944+0000 7fa5717fa640 1 -- 172.21.15.112:0/632776786 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fa56008b2f0 con 0x7fa57414fd30 2024-04-25T01:31:22.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.945+0000 7fa578815640 1 --2- 172.21.15.112:0/632776786 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa55806be60 0x7fa55806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:22.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.945+0000 7fa578815640 1 --2- 172.21.15.112:0/632776786 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa55806be60 0x7fa55806e320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa568034000 tx=0x7fa56802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:22.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:22.951+0000 7fa5717fa640 1 -- 172.21.15.112:0/632776786 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa5740c4030 con 0x7fa57414fd30 2024-04-25T01:31:23.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.251+0000 7fa57aa79640 1 -- 172.21.15.112:0/632776786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa5740689b0 con 0x7fa57414fd30 2024-04-25T01:31:23.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.252+0000 7fa5717fa640 1 -- 172.21.15.112:0/632776786 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7fa5600592b0 con 0x7fa57414fd30 2024-04-25T01:31:23.230 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:31:23.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.254+0000 7fa57aa79640 1 -- 172.21.15.112:0/632776786 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa55806be60 msgr2=0x7fa55806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.254+0000 7fa57aa79640 1 --2- 172.21.15.112:0/632776786 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa55806be60 0x7fa55806e320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa568034000 tx=0x7fa56802e000 comp rx=0 tx=0).stop 2024-04-25T01:31:23.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.254+0000 7fa57aa79640 1 -- 172.21.15.112:0/632776786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa57414fd30 msgr2=0x7fa57414c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.254+0000 7fa57aa79640 1 --2- 172.21.15.112:0/632776786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa57414fd30 0x7fa57414c390 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa56000e870 tx=0x7fa56000ed30 comp rx=0 tx=0).stop 2024-04-25T01:31:23.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.254+0000 7fa57aa79640 1 -- 172.21.15.112:0/632776786 shutdown_connections 2024-04-25T01:31:23.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.254+0000 7fa57aa79640 1 --2- 172.21.15.112:0/632776786 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa55806be60 0x7fa55806e320 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.254+0000 7fa57aa79640 1 --2- 172.21.15.112:0/632776786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa574149fd0 0x7fa5741409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.254+0000 7fa57aa79640 1 --2- 172.21.15.112:0/632776786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa57414fd30 0x7fa57414c390 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.254+0000 7fa57aa79640 1 --2- 172.21.15.112:0/632776786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa574071e20 0x7fa57414f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.254+0000 7fa57aa79640 1 -- 172.21.15.112:0/632776786 >> 172.21.15.112:0/632776786 conn(0x7fa5740c8130 msgr2=0x7fa5740aa880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:23.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.255+0000 7fa57aa79640 1 -- 172.21.15.112:0/632776786 shutdown_connections 2024-04-25T01:31:23.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.255+0000 7fa57aa79640 1 -- 172.21.15.112:0/632776786 wait complete. 2024-04-25T01:31:23.245 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:31:23.245 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T01:31:23.245 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:31:23.245 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:31:23.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.381+0000 7fcf47ba5640 1 Processor -- start 2024-04-25T01:31:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.381+0000 7fcf47ba5640 1 -- start start 2024-04-25T01:31:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.382+0000 7fcf47ba5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcf401596a0 0x7fcf40152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.382+0000 7fcf47ba5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf40159b90 con 0x7fcf400c72b0 2024-04-25T01:31:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.382+0000 7fcf47ba5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf4006c970 con 0x7fcf401592c0 2024-04-25T01:31:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.382+0000 7fcf47ba5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fcf400c6e20 con 0x7fcf401596a0 2024-04-25T01:31:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.382+0000 7fcf45140640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcf401596a0 0x7fcf40152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.382+0000 7fcf46142640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fcf400c72b0 0x7fcf40160380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37526/0 (socket says 172.21.15.112:37526) 2024-04-25T01:31:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.382+0000 7fcf46142640 1 -- 172.21.15.112:0/3058033216 learned_addr learned my addr 172.21.15.112:0/3058033216 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:31:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.382+0000 7fcf45140640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcf401596a0 0x7fcf40152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54464/0 (socket says 172.21.15.112:54464) 2024-04-25T01:31:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.382+0000 7fcf45140640 1 -- 172.21.15.112:0/3058033216 >> v1:172.21.15.187:6789/0 conn(0x7fcf401592c0 legacy=0x7fcf400c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.382+0000 7fcf45140640 1 -- 172.21.15.112:0/3058033216 >> v1:172.21.15.112:6789/0 conn(0x7fcf400c72b0 legacy=0x7fcf40160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.382+0000 7fcf45140640 1 -- 172.21.15.112:0/3058033216 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4006cc50 con 0x7fcf401596a0 2024-04-25T01:31:23.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.383+0000 7fcf45140640 1 --2- 172.21.15.112:0/3058033216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf401596a0 0x7fcf40152d00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcf300097a0 tx=0x7fcf3002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731aeb170c1677f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:23.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.383+0000 7fcf4493f640 1 -- 172.21.15.112:0/3058033216 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30037070 con 0x7fcf401596a0 2024-04-25T01:31:23.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.383+0000 7fcf4493f640 1 -- 172.21.15.112:0/3058033216 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf30032440 con 0x7fcf401596a0 2024-04-25T01:31:23.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.383+0000 7fcf4493f640 1 -- 172.21.15.112:0/3058033216 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3002ae60 con 0x7fcf401596a0 2024-04-25T01:31:23.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.384+0000 7fcf47ba5640 1 -- 172.21.15.112:0/3058033216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf401596a0 msgr2=0x7fcf40152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.384+0000 7fcf47ba5640 1 --2- 172.21.15.112:0/3058033216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf401596a0 0x7fcf40152d00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcf300097a0 tx=0x7fcf3002a110 comp rx=0 tx=0).stop 2024-04-25T01:31:23.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.384+0000 7fcf47ba5640 1 -- 172.21.15.112:0/3058033216 shutdown_connections 2024-04-25T01:31:23.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.384+0000 7fcf47ba5640 1 --2- 172.21.15.112:0/3058033216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf401596a0 0x7fcf40152d00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.384+0000 7fcf47ba5640 1 -- 172.21.15.112:0/3058033216 >> 172.21.15.112:0/3058033216 conn(0x7fcf400cb170 msgr2=0x7fcf400cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:23.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.384+0000 7fcf47ba5640 1 -- 172.21.15.112:0/3058033216 shutdown_connections 2024-04-25T01:31:23.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.384+0000 7fcf47ba5640 1 -- 172.21.15.112:0/3058033216 wait complete. 2024-04-25T01:31:23.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.385+0000 7fcf47ba5640 1 Processor -- start 2024-04-25T01:31:23.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.385+0000 7fcf47ba5640 1 -- start start 2024-04-25T01:31:23.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.386+0000 7fcf47ba5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf400c72b0 0x7fcf40147220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.386+0000 7fcf47ba5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf401592c0 0x7fcf40148530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.386+0000 7fcf47ba5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf401596a0 0x7fcf40158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.386+0000 7fcf45140640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf401592c0 0x7fcf40148530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.386+0000 7fcf47ba5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4006c510 con 0x7fcf401592c0 2024-04-25T01:31:23.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.386+0000 7fcf45140640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf401592c0 0x7fcf40148530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54472/0 (socket says 172.21.15.112:54472) 2024-04-25T01:31:23.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.386+0000 7fcf47ba5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4006c390 con 0x7fcf400c72b0 2024-04-25T01:31:23.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.386+0000 7fcf45140640 1 -- 172.21.15.112:0/2559752574 learned_addr learned my addr 172.21.15.112:0/2559752574 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:23.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.386+0000 7fcf47ba5640 1 -- 172.21.15.112:0/2559752574 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcf4006db20 con 0x7fcf401596a0 2024-04-25T01:31:23.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.386+0000 7fcf45941640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf400c72b0 0x7fcf40147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.386+0000 7fcf46142640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf401596a0 0x7fcf40158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.386+0000 7fcf45140640 1 -- 172.21.15.112:0/2559752574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf401596a0 msgr2=0x7fcf40158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.386+0000 7fcf45140640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf401596a0 0x7fcf40158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.386+0000 7fcf45140640 1 -- 172.21.15.112:0/2559752574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf400c72b0 msgr2=0x7fcf40147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.386+0000 7fcf45140640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf400c72b0 0x7fcf40147220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.386+0000 7fcf45140640 1 -- 172.21.15.112:0/2559752574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf40147ae0 con 0x7fcf401592c0 2024-04-25T01:31:23.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.386+0000 7fcf45140640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf401592c0 0x7fcf40148530 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcf3002a150 tx=0x7fcf300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:23.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.387+0000 7fcf367fc640 1 -- 172.21.15.112:0/2559752574 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30037040 con 0x7fcf401592c0 2024-04-25T01:31:23.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.387+0000 7fcf367fc640 1 -- 172.21.15.112:0/2559752574 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf30042070 con 0x7fcf401592c0 2024-04-25T01:31:23.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.387+0000 7fcf47ba5640 1 -- 172.21.15.112:0/2559752574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf40148e00 con 0x7fcf401592c0 2024-04-25T01:31:23.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.387+0000 7fcf367fc640 1 -- 172.21.15.112:0/2559752574 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30030c70 con 0x7fcf401592c0 2024-04-25T01:31:23.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.387+0000 7fcf47ba5640 1 -- 172.21.15.112:0/2559752574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf401e3010 con 0x7fcf401592c0 2024-04-25T01:31:23.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.388+0000 7fcf47ba5640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf400c1dd0 0x7fcf40158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.388+0000 7fcf47ba5640 1 -- 172.21.15.112:0/2559752574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcf400679a0 con 0x7fcf400c1dd0 2024-04-25T01:31:23.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.388+0000 7fcf45941640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf400c1dd0 0x7fcf40158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.388+0000 7fcf367fc640 1 -- 172.21.15.112:0/2559752574 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf30047460 con 0x7fcf401592c0 2024-04-25T01:31:23.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.388+0000 7fcf45941640 1 -- 172.21.15.112:0/2559752574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf400682a0 con 0x7fcf400c1dd0 2024-04-25T01:31:23.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.388+0000 7fcf45941640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf400c1dd0 0x7fcf40158760 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00f570 tx=0x7fcf3c00fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:23.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.388+0000 7fcf367fc640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcf2406be30 0x7fcf2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.389+0000 7fcf367fc640 1 -- 172.21.15.112:0/2559752574 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fcf300abd80 con 0x7fcf401592c0 2024-04-25T01:31:23.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.389+0000 7fcf46142640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcf2406be30 0x7fcf2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.389+0000 7fcf367fc640 1 -- 172.21.15.112:0/2559752574 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c019070 con 0x7fcf400c1dd0 2024-04-25T01:31:23.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.389+0000 7fcf367fc640 1 -- 172.21.15.112:0/2559752574 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fcf3c005320 con 0x7fcf400c1dd0 2024-04-25T01:31:23.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.389+0000 7fcf367fc640 1 -- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf400c1dd0 msgr2=0x7fcf40158760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.389+0000 7fcf367fc640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf400c1dd0 0x7fcf40158760 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00f570 tx=0x7fcf3c00fa30 comp rx=0 tx=0).stop 2024-04-25T01:31:23.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.389+0000 7fcf367fc640 1 -- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf400c1dd0 msgr2=0x7fcf40158760 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:31:23.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.389+0000 7fcf367fc640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf400c1dd0 0x7fcf40158760 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.389+0000 7fcf46142640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcf2406be30 0x7fcf2406e2f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcf38005c00 tx=0x7fcf38009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:23.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.397+0000 7fcf47ba5640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf4014b3a0 0x7fcf401eec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.397+0000 7fcf47ba5640 1 -- 172.21.15.112:0/2559752574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcf400c79d0 con 0x7fcf4014b3a0 2024-04-25T01:31:23.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.397+0000 7fcf45941640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf4014b3a0 0x7fcf401eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.397+0000 7fcf45941640 1 -- 172.21.15.112:0/2559752574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fcf400c5ca0 con 0x7fcf4014b3a0 2024-04-25T01:31:23.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.397+0000 7fcf45941640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf4014b3a0 0x7fcf401eec70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00f570 tx=0x7fcf3c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:23.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.397+0000 7fcf367fc640 1 -- 172.21.15.112:0/2559752574 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c019070 con 0x7fcf4014b3a0 2024-04-25T01:31:23.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.398+0000 7fcf367fc640 1 -- 172.21.15.112:0/2559752574 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4561 (secure 0 0 0) 0x7fcf3c006d20 con 0x7fcf4014b3a0 2024-04-25T01:31:23.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.398+0000 7fcf367fc640 1 -- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf4014b3a0 msgr2=0x7fcf401eec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.398+0000 7fcf367fc640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf4014b3a0 0x7fcf401eec70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00f570 tx=0x7fcf3c000ed0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.398+0000 7fcf367fc640 1 -- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf4014b3a0 msgr2=0x7fcf401eec70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:31:23.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.398+0000 7fcf367fc640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf4014b3a0 0x7fcf401eec70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.376 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:31:23.376 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:31:23.376 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:31:23.376 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:31:23.376 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 38, 2024-04-25T01:31:23.376 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 18766, 2024-04-25T01:31:23.376 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:31:23.377 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:31:23.377 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:31:23.377 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:31:23.377 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:23.377 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 12, 2024-04-25T01:31:23.377 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:31:23.377 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:31:23.377 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:31:23.377 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:23.377 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:23.377 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:23.377 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:23.377 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:23.377 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:23.378 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:23.378 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:23.378 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:23.378 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:23.378 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:23.378 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:23.378 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:31:23.378 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:31:23.378 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:23.378 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:23.378 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:23.378 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:23.378 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:23.378 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:23.379 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:23.379 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:23.379 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:23.379 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:23.379 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:23.379 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:23.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.379 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:31:23.379 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:31:23.379 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:31:23.379 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:31:23.379 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:31:23.379 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:31:23.379 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:31:23.379 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:31:23.379 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:31:23.380 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:31:23.380 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:31:23.380 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:31:23.380 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:31:23.380 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:31:23.380 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:31:23.380 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:31:23.380 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:31:23.380 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:23.380 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:23.380 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:23.380 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:23.380 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:23.380 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:23.380 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:23.381 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:23.381 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:23.381 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:23.381 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:23.381 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:23.381 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:31:23.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.381 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:31:23.381 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.381 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:31:23.381 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:31:23.381 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:23.381 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:23.381 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.381 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:23.381 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:31:23.382 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:23.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.382 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.382 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:23.382 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:31:23.382 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:23.382 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.382 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:23.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.382 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:31:23.382 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:31:23.382 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:23.382 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:23.383 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:23.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.383 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.383 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:31:23.383 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:31:23.383 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:23.383 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:23.383 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.383 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:23.383 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:31:23.383 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:23.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.383 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.383 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:23.383 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:31:23.383 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:23.384 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.384 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:23.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.384 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:31:23.384 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:31:23.384 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:23.384 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:23.384 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:23.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.384 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.384 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:31:23.384 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:31:23.384 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:23.384 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:23.384 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.384 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:23.385 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:31:23.385 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:23.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.385 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.385 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:23.385 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:31:23.385 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:23.385 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.385 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:23.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.385 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:31:23.385 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:31:23.385 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:23.385 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:23.385 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:23.385 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.386 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:23.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.386 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:31:23.386 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:31:23.386 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.386 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:23.386 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:23.386 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:31:23.386 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.386 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:23.386 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:31:23.386 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.386 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:23.386 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:23.386 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:31:23.387 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.387 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:23.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.387 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:31:23.387 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:31:23.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.398+0000 7fcf47ba5640 1 -- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcf2406be30 msgr2=0x7fcf2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.398+0000 7fcf47ba5640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcf2406be30 0x7fcf2406e2f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcf38005c00 tx=0x7fcf38009040 comp rx=0 tx=0).stop 2024-04-25T01:31:23.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.398+0000 7fcf47ba5640 1 -- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf401592c0 msgr2=0x7fcf40148530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.398+0000 7fcf47ba5640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf401592c0 0x7fcf40148530 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcf3002a150 tx=0x7fcf300046c0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.398+0000 7fcf46142640 1 -- 172.21.15.112:0/2559752574 reap_dead start 2024-04-25T01:31:23.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.398+0000 7fcf47ba5640 1 -- 172.21.15.112:0/2559752574 shutdown_connections 2024-04-25T01:31:23.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.398+0000 7fcf47ba5640 1 --2- 172.21.15.112:0/2559752574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf401592c0 0x7fcf40148530 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.398+0000 7fcf47ba5640 1 -- 172.21.15.112:0/2559752574 >> 172.21.15.112:0/2559752574 conn(0x7fcf400cb170 msgr2=0x7fcf400ac470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:23.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.398+0000 7fcf47ba5640 1 -- 172.21.15.112:0/2559752574 shutdown_connections 2024-04-25T01:31:23.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.398+0000 7fcf47ba5640 1 -- 172.21.15.112:0/2559752574 wait complete. 2024-04-25T01:31:23.389 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:31:23.389 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:31:23.389 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:31:23.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.519+0000 7f869f1e9640 1 Processor -- start 2024-04-25T01:31:23.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.519+0000 7f869f1e9640 1 -- start start 2024-04-25T01:31:23.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.520+0000 7f869f1e9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8698073070 0x7f86981539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.520+0000 7f869f1e9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86980c2ba0 con 0x7f86980aac00 2024-04-25T01:31:23.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.520+0000 7f869f1e9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86980c2e80 con 0x7f8698150e40 2024-04-25T01:31:23.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.520+0000 7f869f1e9640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f86980c37b0 con 0x7f8698073070 2024-04-25T01:31:23.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.520+0000 7f869d786640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f86980aac00 0x7f8698153f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37528/0 (socket says 172.21.15.112:37528) 2024-04-25T01:31:23.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.520+0000 7f869d786640 1 -- 172.21.15.112:0/2434639261 learned_addr learned my addr 172.21.15.112:0/2434639261 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:31:23.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.520+0000 7f8697fff640 1 --2- 172.21.15.112:0/2434639261 >> v2:172.21.15.187:3300/0 conn(0x7f8698073070 0x7f86981539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.520+0000 7f86977fe640 1 -- 172.21.15.112:0/2434639261 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 861763245 0 0) 0x7f86980c2ba0 con 0x7f86980aac00 2024-04-25T01:31:23.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.520+0000 7f86977fe640 1 -- 172.21.15.112:0/2434639261 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8688003520 con 0x7f86980aac00 2024-04-25T01:31:23.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.521+0000 7f8697fff640 1 -- 172.21.15.112:0/2434639261 >> v1:172.21.15.187:6789/0 conn(0x7f8698150e40 legacy=0x7f869806b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.521+0000 7f8697fff640 1 -- 172.21.15.112:0/2434639261 >> v1:172.21.15.112:6789/0 conn(0x7f86980aac00 legacy=0x7f8698153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.521+0000 7f8697fff640 1 -- 172.21.15.112:0/2434639261 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86980c3160 con 0x7f8698073070 2024-04-25T01:31:23.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.521+0000 7f8697fff640 1 --2- 172.21.15.112:0/2434639261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8698073070 0x7f86981539c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f86840097a0 tx=0x7f868402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e97f78f239c08d99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:23.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.521+0000 7f86977fe640 1 -- 172.21.15.112:0/2434639261 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684037070 con 0x7f8698073070 2024-04-25T01:31:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.521+0000 7f86977fe640 1 -- 172.21.15.112:0/2434639261 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8684032440 con 0x7f8698073070 2024-04-25T01:31:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.522+0000 7f86977fe640 1 -- 172.21.15.112:0/2434639261 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684032810 con 0x7f8698073070 2024-04-25T01:31:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.522+0000 7f869f1e9640 1 -- 172.21.15.112:0/2434639261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8698073070 msgr2=0x7f86981539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.522+0000 7f869f1e9640 1 --2- 172.21.15.112:0/2434639261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8698073070 0x7f86981539c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f86840097a0 tx=0x7f868402a1a0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.522+0000 7f869f1e9640 1 -- 172.21.15.112:0/2434639261 shutdown_connections 2024-04-25T01:31:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.522+0000 7f869f1e9640 1 --2- 172.21.15.112:0/2434639261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8698073070 0x7f86981539c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.522+0000 7f869f1e9640 1 -- 172.21.15.112:0/2434639261 >> 172.21.15.112:0/2434639261 conn(0x7f86980c4c00 msgr2=0x7f86980c4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.522+0000 7f869f1e9640 1 -- 172.21.15.112:0/2434639261 shutdown_connections 2024-04-25T01:31:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.522+0000 7f869f1e9640 1 -- 172.21.15.112:0/2434639261 wait complete. 2024-04-25T01:31:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.523+0000 7f869f1e9640 1 Processor -- start 2024-04-25T01:31:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.523+0000 7f869f1e9640 1 -- start start 2024-04-25T01:31:23.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.524+0000 7f869f1e9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8698073070 0x7f86980adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.524+0000 7f869f1e9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86980aac00 0x7f869813faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.524+0000 7f869f1e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8698150e40 0x7f86980c17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.524+0000 7f869f1e9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f869806b870 con 0x7f8698150e40 2024-04-25T01:31:23.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.524+0000 7f869f1e9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8698154610 con 0x7f8698073070 2024-04-25T01:31:23.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.524+0000 7f869f1e9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f869806b9f0 con 0x7f86980aac00 2024-04-25T01:31:23.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.524+0000 7f869d786640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8698150e40 0x7f86980c17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.524+0000 7f869cf85640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8698073070 0x7f86980adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.524+0000 7f869d786640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8698150e40 0x7f86980c17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54504/0 (socket says 172.21.15.112:54504) 2024-04-25T01:31:23.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.524+0000 7f869d786640 1 -- 172.21.15.112:0/1557724406 learned_addr learned my addr 172.21.15.112:0/1557724406 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:23.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.524+0000 7f8697fff640 1 --2- 172.21.15.112:0/1557724406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86980aac00 0x7f869813faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.525+0000 7f869d786640 1 -- 172.21.15.112:0/1557724406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86980aac00 msgr2=0x7f869813faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.525+0000 7f869d786640 1 --2- 172.21.15.112:0/1557724406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86980aac00 0x7f869813faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.525+0000 7f869d786640 1 -- 172.21.15.112:0/1557724406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8698073070 msgr2=0x7f86980adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.525+0000 7f869d786640 1 --2- 172.21.15.112:0/1557724406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8698073070 0x7f86980adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.525+0000 7f869d786640 1 -- 172.21.15.112:0/1557724406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86981ebd30 con 0x7f8698150e40 2024-04-25T01:31:23.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.525+0000 7f869d786640 1 --2- 172.21.15.112:0/1557724406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8698150e40 0x7f86980c17a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f869000cb70 tx=0x7f869000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:23.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.525+0000 7f86957fa640 1 -- 172.21.15.112:0/1557724406 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8690017070 con 0x7f8698150e40 2024-04-25T01:31:23.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.525+0000 7f86957fa640 1 -- 172.21.15.112:0/1557724406 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f869000ad40 con 0x7f8698150e40 2024-04-25T01:31:23.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.525+0000 7f869f1e9640 1 -- 172.21.15.112:0/1557724406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86980c5e20 con 0x7f8698150e40 2024-04-25T01:31:23.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.525+0000 7f86957fa640 1 -- 172.21.15.112:0/1557724406 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8690012620 con 0x7f8698150e40 2024-04-25T01:31:23.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.525+0000 7f869f1e9640 1 -- 172.21.15.112:0/1557724406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f869806fca0 con 0x7f8698150e40 2024-04-25T01:31:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.526+0000 7f869f1e9640 1 --2- 172.21.15.112:0/1557724406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86980c5ff0 0x7f86980c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.526+0000 7f869f1e9640 1 -- 172.21.15.112:0/1557724406 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f86980ab190 con 0x7f86980c5ff0 2024-04-25T01:31:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.526+0000 7f869cf85640 1 --2- 172.21.15.112:0/1557724406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86980c5ff0 0x7f86980c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.527+0000 7f869cf85640 1 -- 172.21.15.112:0/1557724406 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8698075980 con 0x7f86980c5ff0 2024-04-25T01:31:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.527+0000 7f869cf85640 1 --2- 172.21.15.112:0/1557724406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86980c5ff0 0x7f86980c7710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f86981404b0 tx=0x7f868c00f900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.527+0000 7f86957fa640 1 -- 172.21.15.112:0/1557724406 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f869001a070 con 0x7f8698150e40 2024-04-25T01:31:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.527+0000 7f86957fa640 1 --2- 172.21.15.112:0/1557724406 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f867c06be30 0x7f867c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.527+0000 7f86957fa640 1 -- 172.21.15.112:0/1557724406 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f869008af30 con 0x7f8698150e40 2024-04-25T01:31:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.527+0000 7f8697fff640 1 --2- 172.21.15.112:0/1557724406 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f867c06be30 0x7f867c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.528+0000 7f86957fa640 1 -- 172.21.15.112:0/1557724406 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c019070 con 0x7f86980c5ff0 2024-04-25T01:31:23.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.528+0000 7f86957fa640 1 -- 172.21.15.112:0/1557724406 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f868c005320 con 0x7f86980c5ff0 2024-04-25T01:31:23.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.528+0000 7f86957fa640 1 -- 172.21.15.112:0/1557724406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86980c5ff0 msgr2=0x7f86980c7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.528+0000 7f86957fa640 1 --2- 172.21.15.112:0/1557724406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86980c5ff0 0x7f86980c7710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f86981404b0 tx=0x7f868c00f900 comp rx=0 tx=0).stop 2024-04-25T01:31:23.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.528+0000 7f86957fa640 1 -- 172.21.15.112:0/1557724406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86980c5ff0 msgr2=0x7f86980c7710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:31:23.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.528+0000 7f86957fa640 1 --2- 172.21.15.112:0/1557724406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86980c5ff0 0x7f86980c7710 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.528+0000 7f8697fff640 1 --2- 172.21.15.112:0/1557724406 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f867c06be30 0x7f867c06e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8684034000 tx=0x7f868402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:23.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.536+0000 7f869f1e9640 1 --2- 172.21.15.112:0/1557724406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86981ee780 0x7f86981eeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.536+0000 7f869f1e9640 1 -- 172.21.15.112:0/1557724406 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f86980682a0 con 0x7f86981ee780 2024-04-25T01:31:23.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.536+0000 7f869cf85640 1 --2- 172.21.15.112:0/1557724406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86981ee780 0x7f86981eeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.536+0000 7f869cf85640 1 -- 172.21.15.112:0/1557724406 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f869814aec0 con 0x7f86981ee780 2024-04-25T01:31:23.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.536+0000 7f869cf85640 1 --2- 172.21.15.112:0/1557724406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86981ee780 0x7f86981eeb90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f86981404b0 tx=0x7f868c000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:23.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.537+0000 7f86957fa640 1 -- 172.21.15.112:0/1557724406 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c019030 con 0x7f86981ee780 2024-04-25T01:31:23.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.537+0000 7f86957fa640 1 -- 172.21.15.112:0/1557724406 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5070 (secure 0 0 0) 0x7f868c006d30 con 0x7f86981ee780 2024-04-25T01:31:23.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.537+0000 7f86957fa640 1 -- 172.21.15.112:0/1557724406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86981ee780 msgr2=0x7f86981eeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.537+0000 7f86957fa640 1 --2- 172.21.15.112:0/1557724406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86981ee780 0x7f86981eeb90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f86981404b0 tx=0x7f868c000ed0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.537+0000 7f86957fa640 1 -- 172.21.15.112:0/1557724406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86981ee780 msgr2=0x7f86981eeb90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:31:23.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.537+0000 7f86957fa640 1 --2- 172.21.15.112:0/1557724406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86981ee780 0x7f86981eeb90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.515 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:31:23.515 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:31:23.515 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:31:23.515 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:31:23.515 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 38, 2024-04-25T01:31:23.515 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 119227, 2024-04-25T01:31:23.516 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:31:23.516 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:31:23.516 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:31:23.516 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:31:23.516 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:23.516 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 12, 2024-04-25T01:31:23.516 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:31:23.516 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:31:23.516 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:31:23.516 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:23.516 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:23.516 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:23.516 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:23.516 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:23.516 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:23.517 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:23.517 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:23.517 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:23.517 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:23.517 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:23.517 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:23.517 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:31:23.517 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:31:23.517 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:23.517 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:23.517 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:23.517 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:23.517 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:23.517 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:23.517 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:23.518 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:23.518 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:23.518 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:23.518 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:23.518 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:23.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.518 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:31:23.518 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:31:23.518 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:31:23.519 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:31:23.519 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:31:23.519 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:31:23.519 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:31:23.519 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:31:23.519 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:31:23.519 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:31:23.519 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:31:23.520 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:31:23.520 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:31:23.520 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:31:23.520 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:31:23.520 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:31:23.520 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:31:23.520 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:23.520 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:23.520 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:23.520 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:23.520 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:23.520 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:23.520 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:23.520 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:23.520 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:23.521 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:23.521 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:23.521 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:23.521 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:31:23.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.521 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:31:23.521 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.521 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:31:23.521 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:31:23.521 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:23.521 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:23.521 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.521 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:23.521 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:31:23.521 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:23.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.522 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.522 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:23.522 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:31:23.522 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:23.522 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.522 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:23.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.522 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:31:23.522 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:31:23.522 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:23.522 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:23.522 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:23.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.523 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.523 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:31:23.523 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:31:23.523 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:23.523 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:23.523 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.523 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:23.523 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:31:23.523 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:23.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.523 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.523 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:23.523 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:31:23.523 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:23.523 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.523 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:23.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.524 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:31:23.524 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:31:23.524 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:23.524 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:23.524 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:23.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.524 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.524 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:31:23.524 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:31:23.524 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:23.524 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:23.524 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.524 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:23.524 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:31:23.525 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:23.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.525 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.525 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:23.525 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:31:23.525 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:23.525 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.525 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:23.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.525 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:31:23.525 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:31:23.525 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:23.525 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:23.525 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:23.525 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.526 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:23.526 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.526 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:31:23.526 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:31:23.526 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.526 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:23.526 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:23.526 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:31:23.526 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.526 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:23.526 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:31:23.526 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.526 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:23.526 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:23.527 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:31:23.527 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.527 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:23.527 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:31:23.527 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.527 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:23.527 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:23.527 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:31:23.527 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.527 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:23.527 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:31:23.527 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.527 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:23.527 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:23.527 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:31:23.527 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.528 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:23.528 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:31:23.528 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.528 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:23.528 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:23.528 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:31:23.528 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.528 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:23.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.528 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:31:23.528 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:31:23.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.537+0000 7f869f1e9640 1 -- 172.21.15.112:0/1557724406 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f867c06be30 msgr2=0x7f867c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.537+0000 7f869f1e9640 1 --2- 172.21.15.112:0/1557724406 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f867c06be30 0x7f867c06e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8684034000 tx=0x7f868402e000 comp rx=0 tx=0).stop 2024-04-25T01:31:23.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.537+0000 7f869f1e9640 1 -- 172.21.15.112:0/1557724406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8698150e40 msgr2=0x7f86980c17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.537+0000 7f869f1e9640 1 --2- 172.21.15.112:0/1557724406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8698150e40 0x7f86980c17a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f869000cb70 tx=0x7f869000a430 comp rx=0 tx=0).stop 2024-04-25T01:31:23.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.537+0000 7f869d786640 1 -- 172.21.15.112:0/1557724406 reap_dead start 2024-04-25T01:31:23.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.537+0000 7f869d786640 1 -- 172.21.15.112:0/1557724406 reap_dead start 2024-04-25T01:31:23.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.537+0000 7f869f1e9640 1 -- 172.21.15.112:0/1557724406 shutdown_connections 2024-04-25T01:31:23.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.537+0000 7f869f1e9640 1 -- 172.21.15.112:0/1557724406 >> 172.21.15.112:0/1557724406 conn(0x7f86980c4c00 msgr2=0x7f8698151ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:23.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.537+0000 7f869f1e9640 1 -- 172.21.15.112:0/1557724406 shutdown_connections 2024-04-25T01:31:23.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.537+0000 7f869f1e9640 1 -- 172.21.15.112:0/1557724406 wait complete. 2024-04-25T01:31:23.531 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:31:23.531 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:31:23.531 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:31:23.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.663+0000 7f08de530640 1 Processor -- start 2024-04-25T01:31:23.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.664+0000 7f08de530640 1 -- start start 2024-04-25T01:31:23.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.664+0000 7f08de530640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f08d80c42e0 0x7f08d806dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.664+0000 7f08de530640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f08d8156290 0x7f08d8158e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.664+0000 7f08de530640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d806e130 con 0x7f08d81593d0 2024-04-25T01:31:23.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.664+0000 7f08de530640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f08d81598c0 con 0x7f08d8156290 2024-04-25T01:31:23.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.664+0000 7f08de530640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f08d8159ba0 con 0x7f08d80c42e0 2024-04-25T01:31:23.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.664+0000 7f08d77fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f08d8156290 0x7f08d8158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.664+0000 7f08d77fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f08d8156290 0x7f08d8158e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54518/0 (socket says 172.21.15.112:54518) 2024-04-25T01:31:23.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.664+0000 7f08d77fe640 1 -- 172.21.15.112:0/1961261590 learned_addr learned my addr 172.21.15.112:0/1961261590 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:23.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.665+0000 7f08d7fff640 1 --2- 172.21.15.112:0/1961261590 >> v2:172.21.15.187:3300/0 conn(0x7f08d80c42e0 0x7f08d806dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.665+0000 7f08d77fe640 1 -- 172.21.15.112:0/1961261590 >> v2:172.21.15.187:3300/0 conn(0x7f08d80c42e0 msgr2=0x7f08d806dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:31:23.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.665+0000 7f08d77fe640 1 --2- 172.21.15.112:0/1961261590 >> v2:172.21.15.187:3300/0 conn(0x7f08d80c42e0 0x7f08d806dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.665+0000 7f08d77fe640 1 -- 172.21.15.112:0/1961261590 >> v1:172.21.15.187:6789/0 conn(0x7f08d81593d0 legacy=0x7f08d8152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.665+0000 7f08d77fe640 1 -- 172.21.15.112:0/1961261590 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d8155a90 con 0x7f08d8156290 2024-04-25T01:31:23.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.665+0000 7f08d77fe640 1 --2- 172.21.15.112:0/1961261590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d8156290 0x7f08d8158e90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f08d8072dc0 tx=0x7f08cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c28ddc9ed4fcf1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:23.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.666+0000 7f08d6ffd640 1 -- 172.21.15.112:0/1961261590 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc037070 con 0x7f08d8156290 2024-04-25T01:31:23.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.666+0000 7f08d6ffd640 1 -- 172.21.15.112:0/1961261590 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08cc032440 con 0x7f08d8156290 2024-04-25T01:31:23.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.666+0000 7f08d6ffd640 1 -- 172.21.15.112:0/1961261590 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc032810 con 0x7f08d8156290 2024-04-25T01:31:23.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.666+0000 7f08de530640 1 -- 172.21.15.112:0/1961261590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d8156290 msgr2=0x7f08d8158e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.666+0000 7f08de530640 1 --2- 172.21.15.112:0/1961261590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d8156290 0x7f08d8158e90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f08d8072dc0 tx=0x7f08cc02a270 comp rx=0 tx=0).stop 2024-04-25T01:31:23.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.666+0000 7f08de530640 1 -- 172.21.15.112:0/1961261590 shutdown_connections 2024-04-25T01:31:23.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.666+0000 7f08de530640 1 --2- 172.21.15.112:0/1961261590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d8156290 0x7f08d8158e90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.666+0000 7f08de530640 1 --2- 172.21.15.112:0/1961261590 >> v2:172.21.15.187:3300/0 conn(0x7f08d80c42e0 0x7f08d806dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.666+0000 7f08de530640 1 -- 172.21.15.112:0/1961261590 >> 172.21.15.112:0/1961261590 conn(0x7f08d80c4a90 msgr2=0x7f08d80c4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:23.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.666+0000 7f08de530640 1 -- 172.21.15.112:0/1961261590 shutdown_connections 2024-04-25T01:31:23.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.666+0000 7f08de530640 1 -- 172.21.15.112:0/1961261590 wait complete. 2024-04-25T01:31:23.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.667+0000 7f08de530640 1 Processor -- start 2024-04-25T01:31:23.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.668+0000 7f08de530640 1 -- start start 2024-04-25T01:31:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.668+0000 7f08de530640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d80c42e0 0x7f08d81ee5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.668+0000 7f08de530640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08d8156290 0x7f08d8149a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.668+0000 7f08de530640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d81593d0 0x7f08d8149f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.668+0000 7f08de530640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08d8156880 con 0x7f08d81593d0 2024-04-25T01:31:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.668+0000 7f08de530640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f08d8156700 con 0x7f08d8156290 2024-04-25T01:31:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.668+0000 7f08de530640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f08d81533b0 con 0x7f08d80c42e0 2024-04-25T01:31:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.669+0000 7f08dcacd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d81593d0 0x7f08d8149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.669+0000 7f08d7fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d80c42e0 0x7f08d81ee5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.669+0000 7f08d77fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08d8156290 0x7f08d8149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.669+0000 7f08d7fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d80c42e0 0x7f08d81ee5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:38716/0 (socket says 172.21.15.112:38716) 2024-04-25T01:31:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.669+0000 7f08dcacd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d81593d0 0x7f08d8149f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54528/0 (socket says 172.21.15.112:54528) 2024-04-25T01:31:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.669+0000 7f08d7fff640 1 -- 172.21.15.112:0/3625289914 learned_addr learned my addr 172.21.15.112:0/3625289914 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.669+0000 7f08d7fff640 1 -- 172.21.15.112:0/3625289914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08d8156290 msgr2=0x7f08d8149a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.669+0000 7f08d7fff640 1 --2- 172.21.15.112:0/3625289914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08d8156290 0x7f08d8149a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.669+0000 7f08d7fff640 1 -- 172.21.15.112:0/3625289914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d81593d0 msgr2=0x7f08d8149f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.669+0000 7f08d7fff640 1 --2- 172.21.15.112:0/3625289914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d81593d0 0x7f08d8149f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.669+0000 7f08d7fff640 1 -- 172.21.15.112:0/3625289914 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d814ffa0 con 0x7f08d80c42e0 2024-04-25T01:31:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.669+0000 7f08dcacd640 1 --2- 172.21.15.112:0/3625289914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d81593d0 0x7f08d8149f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:31:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.669+0000 7f08d7fff640 1 --2- 172.21.15.112:0/3625289914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d80c42e0 0x7f08d81ee5d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f08c800e9b0 tx=0x7f08c800ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.669+0000 7f08d4ff9640 1 -- 172.21.15.112:0/3625289914 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c8017070 con 0x7f08d80c42e0 2024-04-25T01:31:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.669+0000 7f08de530640 1 -- 172.21.15.112:0/3625289914 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d81eef10 con 0x7f08d80c42e0 2024-04-25T01:31:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.669+0000 7f08d4ff9640 1 -- 172.21.15.112:0/3625289914 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c800c9a0 con 0x7f08d80c42e0 2024-04-25T01:31:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.670+0000 7f08d4ff9640 1 -- 172.21.15.112:0/3625289914 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c800cd90 con 0x7f08d80c42e0 2024-04-25T01:31:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.670+0000 7f08de530640 1 -- 172.21.15.112:0/3625289914 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d80c3cd0 con 0x7f08d80c42e0 2024-04-25T01:31:23.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.671+0000 7f08de530640 1 --2- 172.21.15.112:0/3625289914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d81eeb10 0x7f08d814f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.671+0000 7f08de530640 1 -- 172.21.15.112:0/3625289914 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f08d814b710 con 0x7f08d81eeb10 2024-04-25T01:31:23.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.671+0000 7f08d77fe640 1 --2- 172.21.15.112:0/3625289914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d81eeb10 0x7f08d814f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.671+0000 7f08d4ff9640 1 -- 172.21.15.112:0/3625289914 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f08c801a070 con 0x7f08d80c42e0 2024-04-25T01:31:23.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.671+0000 7f08d77fe640 1 -- 172.21.15.112:0/3625289914 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f08d814fd50 con 0x7f08d81eeb10 2024-04-25T01:31:23.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.671+0000 7f08d77fe640 1 --2- 172.21.15.112:0/3625289914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d81eeb10 0x7f08d814f300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f08d8070640 tx=0x7f08cc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:23.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.671+0000 7f08d4ff9640 1 --2- 172.21.15.112:0/3625289914 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f08c406be60 0x7f08c406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.672+0000 7f08d4ff9640 1 -- 172.21.15.112:0/3625289914 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f08c808b460 con 0x7f08d80c42e0 2024-04-25T01:31:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.672+0000 7f08dcacd640 1 --2- 172.21.15.112:0/3625289914 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f08c406be60 0x7f08c406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.672+0000 7f08d4ff9640 1 -- 172.21.15.112:0/3625289914 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc043070 con 0x7f08d81eeb10 2024-04-25T01:31:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.672+0000 7f08d4ff9640 1 -- 172.21.15.112:0/3625289914 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f08cc009490 con 0x7f08d81eeb10 2024-04-25T01:31:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.672+0000 7f08d4ff9640 1 -- 172.21.15.112:0/3625289914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d81eeb10 msgr2=0x7f08d814f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.672+0000 7f08d4ff9640 1 --2- 172.21.15.112:0/3625289914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d81eeb10 0x7f08d814f300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f08d8070640 tx=0x7f08cc0046c0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.672+0000 7f08d4ff9640 1 -- 172.21.15.112:0/3625289914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d81eeb10 msgr2=0x7f08d814f300 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:31:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.672+0000 7f08d4ff9640 1 --2- 172.21.15.112:0/3625289914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d81eeb10 0x7f08d814f300 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f08d8070640 tx=0x7f08cc0046c0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.672+0000 7f08dcacd640 1 --2- 172.21.15.112:0/3625289914 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f08c406be60 0x7f08c406e320 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f08d00012e0 tx=0x7f08d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:23.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.681+0000 7f08de530640 1 --2- 172.21.15.112:0/3625289914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d814cb20 0x7f08d8147d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.681+0000 7f08de530640 1 -- 172.21.15.112:0/3625289914 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f08d8075980 con 0x7f08d814cb20 2024-04-25T01:31:23.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.681+0000 7f08d77fe640 1 --2- 172.21.15.112:0/3625289914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d814cb20 0x7f08d8147d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.682+0000 7f08d77fe640 1 -- 172.21.15.112:0/3625289914 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f08d80682a0 con 0x7f08d814cb20 2024-04-25T01:31:23.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.682+0000 7f08d77fe640 1 --2- 172.21.15.112:0/3625289914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d814cb20 0x7f08d8147d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f08cc02a2b0 tx=0x7f08cc008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.682+0000 7f08d4ff9640 1 -- 172.21.15.112:0/3625289914 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc037040 con 0x7f08d814cb20 2024-04-25T01:31:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.682+0000 7f08d4ff9640 1 -- 172.21.15.112:0/3625289914 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4899 (secure 0 0 0) 0x7f08cc03cda0 con 0x7f08d814cb20 2024-04-25T01:31:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.682+0000 7f08d4ff9640 1 -- 172.21.15.112:0/3625289914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d814cb20 msgr2=0x7f08d8147d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.682+0000 7f08d4ff9640 1 --2- 172.21.15.112:0/3625289914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d814cb20 0x7f08d8147d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f08cc02a2b0 tx=0x7f08cc008030 comp rx=0 tx=0).stop 2024-04-25T01:31:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.682+0000 7f08d4ff9640 1 -- 172.21.15.112:0/3625289914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d814cb20 msgr2=0x7f08d8147d80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:31:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.682+0000 7f08d4ff9640 1 --2- 172.21.15.112:0/3625289914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d814cb20 0x7f08d8147d80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.662 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:31:23.662 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:31:23.662 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:31:23.662 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:31:23.662 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 38, 2024-04-25T01:31:23.662 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 72421, 2024-04-25T01:31:23.662 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:31:23.663 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:31:23.663 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:31:23.663 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:31:23.663 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:23.663 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 12, 2024-04-25T01:31:23.663 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:31:23.663 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:31:23.663 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:31:23.663 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:23.664 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:23.664 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:23.664 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:23.664 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:23.664 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:23.664 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:23.664 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:23.664 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:23.664 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:23.665 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:23.665 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:23.665 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:31:23.665 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:31:23.665 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:23.665 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:23.665 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:23.665 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:23.666 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:23.666 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:23.666 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:23.666 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:23.666 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:23.666 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:23.666 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:23.666 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:23.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.666 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:31:23.666 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:31:23.667 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:31:23.667 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:31:23.667 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:31:23.667 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:31:23.667 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:31:23.667 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:31:23.668 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:31:23.668 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:31:23.668 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:31:23.668 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:31:23.668 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:31:23.668 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:31:23.668 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:31:23.669 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:31:23.669 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:31:23.669 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:23.669 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:23.669 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:23.669 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:23.669 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:23.669 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:23.669 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:23.670 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:23.670 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:23.670 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:23.670 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:23.670 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:23.670 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:31:23.670 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.670 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:31:23.670 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.670 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:31:23.671 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:31:23.671 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:23.671 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:23.671 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.671 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:23.671 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:31:23.671 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:23.671 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.671 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.671 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:23.671 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:31:23.671 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:23.672 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.672 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:23.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.672 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:31:23.672 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:31:23.672 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:23.672 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:23.673 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:23.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.673 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.673 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:31:23.673 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:31:23.673 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:23.673 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:23.673 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.673 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:23.673 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:31:23.673 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:23.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.674 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.674 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:23.674 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:31:23.674 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:23.674 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.674 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:23.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.674 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:31:23.674 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:31:23.674 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:23.674 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:23.674 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:23.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.675 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.675 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:31:23.675 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:31:23.675 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:23.675 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:23.675 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.675 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:23.675 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:31:23.675 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:23.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.676 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.676 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:23.676 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:31:23.676 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:23.676 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.676 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:23.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.676 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:31:23.676 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:31:23.676 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:23.676 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:23.677 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:23.677 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.677 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:23.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.677 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:31:23.677 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:31:23.677 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.677 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:23.677 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:23.677 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:31:23.677 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.678 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:23.678 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:31:23.678 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.678 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:23.678 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:23.678 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:31:23.678 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.678 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:23.678 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:31:23.678 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.678 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:23.678 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:23.679 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:31:23.679 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.679 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:23.679 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:31:23.679 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:23.679 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:23.679 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:23.679 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:31:23.679 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:23.679 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:23.679 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:23.679 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:31:23.680 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:31:23.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.683+0000 7f08de530640 1 -- 172.21.15.112:0/3625289914 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f08c406be60 msgr2=0x7f08c406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.683+0000 7f08de530640 1 --2- 172.21.15.112:0/3625289914 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f08c406be60 0x7f08c406e320 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f08d00012e0 tx=0x7f08d0009040 comp rx=0 tx=0).stop 2024-04-25T01:31:23.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.683+0000 7f08de530640 1 -- 172.21.15.112:0/3625289914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d80c42e0 msgr2=0x7f08d81ee5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.683+0000 7f08de530640 1 --2- 172.21.15.112:0/3625289914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d80c42e0 0x7f08d81ee5d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f08c800e9b0 tx=0x7f08c800ee70 comp rx=0 tx=0).stop 2024-04-25T01:31:23.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.683+0000 7f08dcacd640 1 -- 172.21.15.112:0/3625289914 reap_dead start 2024-04-25T01:31:23.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.683+0000 7f08dcacd640 1 -- 172.21.15.112:0/3625289914 reap_dead start 2024-04-25T01:31:23.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.683+0000 7f08de530640 1 -- 172.21.15.112:0/3625289914 shutdown_connections 2024-04-25T01:31:23.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.683+0000 7f08de530640 1 -- 172.21.15.112:0/3625289914 >> 172.21.15.112:0/3625289914 conn(0x7f08d80c4a90 msgr2=0x7f08d8072900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:23.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.683+0000 7f08de530640 1 -- 172.21.15.112:0/3625289914 shutdown_connections 2024-04-25T01:31:23.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.683+0000 7f08de530640 1 -- 172.21.15.112:0/3625289914 wait complete. 2024-04-25T01:31:23.682 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:31:23.682 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-04-25T01:31:23.683 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-25T01:31:23.683 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-04-25T01:31:23.683 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-04-25T01:31:23.683 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T01:31:23.683 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:31:23.695 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T01:31:23.696 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T01:31:23.696 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:31:23.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.870+0000 7fe9b6c5d640 1 Processor -- start 2024-04-25T01:31:23.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.870+0000 7fe9b6c5d640 1 -- start start 2024-04-25T01:31:23.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.871+0000 7fe9b6c5d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe9b00c9570 0x7fe9b00c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.871+0000 7fe9b6c5d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9b01532d0 con 0x7fe9b0152ef0 2024-04-25T01:31:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.871+0000 7fe9b6c5d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9b0155a70 con 0x7fe9b00c9060 2024-04-25T01:31:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.871+0000 7fe9b6c5d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fe9b00c9ba0 con 0x7fe9b00c9570 2024-04-25T01:31:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.871+0000 7fe9b49f9640 1 -- >> v1:172.21.15.112:6789/0 conn(0x7fe9b0152ef0 legacy=0x7fe9b00ca680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:31:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.871+0000 7fe9affff640 1 -- >> v2:172.21.15.112:3300/0 conn(0x7fe9b00c9570 msgr2=0x7fe9b00c5ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:31:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.871+0000 7fe9affff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe9b00c9570 0x7fe9b00c5ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:31:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.871+0000 7fe9b51fa640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fe9b00c9060 0x7fe9b0160200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:51668/0 (socket says 172.21.15.112:51668) 2024-04-25T01:31:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.871+0000 7fe9b51fa640 1 -- 172.21.15.112:0/587601794 learned_addr learned my addr 172.21.15.112:0/587601794 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:31:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.871+0000 7fe9af7fe640 1 -- 172.21.15.112:0/587601794 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3193352014 0 0) 0x7fe9b0155a70 con 0x7fe9b00c9060 2024-04-25T01:31:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.871+0000 7fe9af7fe640 1 -- 172.21.15.112:0/587601794 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9a0003520 con 0x7fe9b00c9060 2024-04-25T01:31:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9af7fe640 1 -- 172.21.15.112:0/587601794 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3417083109 0 0) 0x7fe9a0003520 con 0x7fe9b00c9060 2024-04-25T01:31:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9af7fe640 1 -- 172.21.15.112:0/587601794 >> v2:172.21.15.112:3300/0 conn(0x7fe9b00c9570 msgr2=0x7fe9b00c5ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:31:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9af7fe640 1 --2- 172.21.15.112:0/587601794 >> v2:172.21.15.112:3300/0 conn(0x7fe9b00c9570 0x7fe9b00c5ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9af7fe640 1 -- 172.21.15.112:0/587601794 >> v1:172.21.15.112:6789/0 conn(0x7fe9b0152ef0 legacy=0x7fe9b00ca680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:31:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9af7fe640 1 -- 172.21.15.112:0/587601794 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b00c9e80 con 0x7fe9b00c9060 2024-04-25T01:31:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9af7fe640 1 -- 172.21.15.112:0/587601794 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fe9a80019f0 con 0x7fe9b00c9060 2024-04-25T01:31:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9af7fe640 1 -- 172.21.15.112:0/587601794 >> v1:172.21.15.187:6789/0 conn(0x7fe9b00c9060 legacy=0x7fe9b0160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9af7fe640 1 --2- 172.21.15.112:0/587601794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9a00039b0 0x7fe9a0003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9af7fe640 1 --2- 172.21.15.112:0/587601794 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9a0004800 0x7fe9a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9af7fe640 1 --2- 172.21.15.112:0/587601794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9a0005230 0x7fe9a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9af7fe640 1 -- 172.21.15.112:0/587601794 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b00c9ba0 con 0x7fe9a0005230 2024-04-25T01:31:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9af7fe640 1 -- 172.21.15.112:0/587601794 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a000c190 con 0x7fe9a00039b0 2024-04-25T01:31:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9af7fe640 1 -- 172.21.15.112:0/587601794 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a000c390 con 0x7fe9a0004800 2024-04-25T01:31:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9b51fa640 1 -- 172.21.15.112:0/587601794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9a0005230 msgr2=0x7fe9a000b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:31:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9b51fa640 1 --2- 172.21.15.112:0/587601794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9a0005230 0x7fe9a000b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:31:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9b49f9640 1 --2- 172.21.15.112:0/587601794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9a00039b0 0x7fe9a0003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9affff640 1 --2- 172.21.15.112:0/587601794 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9a0004800 0x7fe9a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9b49f9640 1 -- 172.21.15.112:0/587601794 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9a0004800 msgr2=0x7fe9a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:31:23.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9b49f9640 1 --2- 172.21.15.112:0/587601794 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9a0004800 0x7fe9a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9b49f9640 1 -- 172.21.15.112:0/587601794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9a0005230 msgr2=0x7fe9a000b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:31:23.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9b49f9640 1 --2- 172.21.15.112:0/587601794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9a0005230 0x7fe9a000b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9b51fa640 1 -- 172.21.15.112:0/587601794 reap_dead start 2024-04-25T01:31:23.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.872+0000 7fe9b49f9640 1 -- 172.21.15.112:0/587601794 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a0003520 con 0x7fe9a00039b0 2024-04-25T01:31:23.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.873+0000 7fe9b49f9640 1 --2- 172.21.15.112:0/587601794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9a00039b0 0x7fe9a0003db0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009900 tx=0x7fe9a0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1638d64082114897 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:23.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.873+0000 7fe9af7fe640 1 -- 172.21.15.112:0/587601794 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4009500 con 0x7fe9a00039b0 2024-04-25T01:31:23.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.873+0000 7fe9af7fe640 1 -- 172.21.15.112:0/587601794 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9a402e070 con 0x7fe9a00039b0 2024-04-25T01:31:23.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.873+0000 7fe9af7fe640 1 -- 172.21.15.112:0/587601794 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4003df0 con 0x7fe9a00039b0 2024-04-25T01:31:23.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.874+0000 7fe9b6c5d640 1 -- 172.21.15.112:0/587601794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9a00039b0 msgr2=0x7fe9a0003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.874+0000 7fe9b6c5d640 1 --2- 172.21.15.112:0/587601794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9a00039b0 0x7fe9a0003db0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009900 tx=0x7fe9a0005830 comp rx=0 tx=0).stop 2024-04-25T01:31:23.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.874+0000 7fe9b6c5d640 1 -- 172.21.15.112:0/587601794 shutdown_connections 2024-04-25T01:31:23.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.874+0000 7fe9b6c5d640 1 --2- 172.21.15.112:0/587601794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9a00039b0 0x7fe9a0003db0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.874+0000 7fe9b6c5d640 1 -- 172.21.15.112:0/587601794 >> 172.21.15.112:0/587601794 conn(0x7fe9b00cb160 msgr2=0x7fe9b00cb570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:23.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.874+0000 7fe9b6c5d640 1 -- 172.21.15.112:0/587601794 shutdown_connections 2024-04-25T01:31:23.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.874+0000 7fe9b6c5d640 1 -- 172.21.15.112:0/587601794 wait complete. 2024-04-25T01:31:23.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.875+0000 7fe9b6c5d640 1 Processor -- start 2024-04-25T01:31:23.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.875+0000 7fe9b6c5d640 1 -- start start 2024-04-25T01:31:23.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.876+0000 7fe9b6c5d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9a00039b0 0x7fe9b01eedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.876+0000 7fe9b6c5d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9b01ef330 0x7fe9b00c3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.876+0000 7fe9b6c5d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9b00c3b90 0x7fe9b01eb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.876+0000 7fe9b6c5d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b006c0a0 con 0x7fe9a00039b0 2024-04-25T01:31:23.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.876+0000 7fe9b6c5d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b006bf20 con 0x7fe9b01ef330 2024-04-25T01:31:23.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.876+0000 7fe9b6c5d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b006c220 con 0x7fe9b00c3b90 2024-04-25T01:31:23.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.876+0000 7fe9b49f9640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9a00039b0 msgr2=0x7fe9b01eedf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:31:23.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.876+0000 7fe9b49f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9a00039b0 0x7fe9b01eedf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:31:23.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.876+0000 7fe9affff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9b01ef330 0x7fe9b00c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.876+0000 7fe9affff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9b01ef330 0x7fe9b00c3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:36458/0 (socket says 172.21.15.112:36458) 2024-04-25T01:31:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.876+0000 7fe9b51fa640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9b00c3b90 0x7fe9b01eb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.876+0000 7fe9affff640 1 -- 172.21.15.112:0/790666454 learned_addr learned my addr 172.21.15.112:0/790666454 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.876+0000 7fe9affff640 1 -- 172.21.15.112:0/790666454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9b00c3b90 msgr2=0x7fe9b01eb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.876+0000 7fe9affff640 1 --2- 172.21.15.112:0/790666454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9b00c3b90 0x7fe9b01eb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.876+0000 7fe9affff640 1 -- 172.21.15.112:0/790666454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9a00039b0 msgr2=0x7fe9b01eedf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:31:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.876+0000 7fe9affff640 1 --2- 172.21.15.112:0/790666454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9a00039b0 0x7fe9b01eedf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.876+0000 7fe9affff640 1 -- 172.21.15.112:0/790666454 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b00bf8d0 con 0x7fe9b01ef330 2024-04-25T01:31:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.876+0000 7fe9b51fa640 1 --2- 172.21.15.112:0/790666454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9b00c3b90 0x7fe9b01eb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:31:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.877+0000 7fe9affff640 1 --2- 172.21.15.112:0/790666454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9b01ef330 0x7fe9b00c3650 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009550 tx=0x7fe99c009a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.877+0000 7fe9ad7fa640 1 -- 172.21.15.112:0/790666454 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c016070 con 0x7fe9b01ef330 2024-04-25T01:31:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.877+0000 7fe9ad7fa640 1 -- 172.21.15.112:0/790666454 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe99c012600 con 0x7fe9b01ef330 2024-04-25T01:31:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.877+0000 7fe9b6c5d640 1 -- 172.21.15.112:0/790666454 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b00c0f90 con 0x7fe9b01ef330 2024-04-25T01:31:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.877+0000 7fe9ad7fa640 1 -- 172.21.15.112:0/790666454 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c0129f0 con 0x7fe9b01ef330 2024-04-25T01:31:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.877+0000 7fe9b6c5d640 1 -- 172.21.15.112:0/790666454 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b00bfe30 con 0x7fe9b01ef330 2024-04-25T01:31:23.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.878+0000 7fe9b6c5d640 1 -- 172.21.15.112:0/790666454 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9b00cbd80 con 0x7fe9b01ef330 2024-04-25T01:31:23.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.878+0000 7fe9ad7fa640 1 -- 172.21.15.112:0/790666454 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe99c019070 con 0x7fe9b01ef330 2024-04-25T01:31:23.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.879+0000 7fe9ad7fa640 1 --2- 172.21.15.112:0/790666454 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe99406bde0 0x7fe99406e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:23.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.879+0000 7fe9ad7fa640 1 -- 172.21.15.112:0/790666454 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fe99c08a9a0 con 0x7fe9b01ef330 2024-04-25T01:31:23.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.879+0000 7fe9b49f9640 1 --2- 172.21.15.112:0/790666454 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe99406bde0 0x7fe99406e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:23.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.879+0000 7fe9b49f9640 1 --2- 172.21.15.112:0/790666454 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe99406bde0 0x7fe99406e2a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4032000 tx=0x7fe9a4037040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:23.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:23.883+0000 7fe9ad7fa640 1 -- 172.21.15.112:0/790666454 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe9b00bf8d0 con 0x7fe9b01ef330 2024-04-25T01:31:24.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:24.172+0000 7fe9b6c5d640 1 -- 172.21.15.112:0/790666454 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe9b00c63d0 con 0x7fe9b01ef330 2024-04-25T01:31:24.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:24.172+0000 7fe9ad7fa640 1 -- 172.21.15.112:0/790666454 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7fe99c058940 con 0x7fe9b01ef330 2024-04-25T01:31:24.150 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:31:24.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:24.174+0000 7fe9b6c5d640 1 -- 172.21.15.112:0/790666454 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe99406bde0 msgr2=0x7fe99406e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:24.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:24.174+0000 7fe9b6c5d640 1 --2- 172.21.15.112:0/790666454 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe99406bde0 0x7fe99406e2a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4032000 tx=0x7fe9a4037040 comp rx=0 tx=0).stop 2024-04-25T01:31:24.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:24.174+0000 7fe9b6c5d640 1 -- 172.21.15.112:0/790666454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9b01ef330 msgr2=0x7fe9b00c3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:24.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:24.174+0000 7fe9b6c5d640 1 --2- 172.21.15.112:0/790666454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9b01ef330 0x7fe9b00c3650 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009550 tx=0x7fe99c009a10 comp rx=0 tx=0).stop 2024-04-25T01:31:24.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:24.174+0000 7fe9b6c5d640 1 -- 172.21.15.112:0/790666454 shutdown_connections 2024-04-25T01:31:24.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:24.174+0000 7fe9b6c5d640 1 --2- 172.21.15.112:0/790666454 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe99406bde0 0x7fe99406e2a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:24.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:24.174+0000 7fe9b6c5d640 1 --2- 172.21.15.112:0/790666454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9b00c3b90 0x7fe9b01eb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:24.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:24.174+0000 7fe9b6c5d640 1 --2- 172.21.15.112:0/790666454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9b01ef330 0x7fe9b00c3650 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:24.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:24.174+0000 7fe9b6c5d640 1 --2- 172.21.15.112:0/790666454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9a00039b0 0x7fe9b01eedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:24.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:24.174+0000 7fe9b6c5d640 1 -- 172.21.15.112:0/790666454 >> 172.21.15.112:0/790666454 conn(0x7fe9b00cb160 msgr2=0x7fe9b0151ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:24.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:24.175+0000 7fe9b6c5d640 1 -- 172.21.15.112:0/790666454 shutdown_connections 2024-04-25T01:31:24.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:24.175+0000 7fe9b6c5d640 1 -- 172.21.15.112:0/790666454 wait complete. 2024-04-25T01:31:25.730 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:31:25.732 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:31:27.165 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:31:27.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.303+0000 7f3a4a71b640 1 Processor -- start 2024-04-25T01:31:27.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.303+0000 7f3a4a71b640 1 -- start start 2024-04-25T01:31:27.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.304+0000 7f3a4a71b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a440c9b80 0x7f3a4406bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:27.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.304+0000 7f3a437fe640 1 -- >> v1:172.21.15.112:6789/0 conn(0x7f3a440c5d70 legacy=0x7f3a44156cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:31:27.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.304+0000 7f3a4a71b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a440c9fd0 con 0x7f3a440c5d70 2024-04-25T01:31:27.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.304+0000 7f3a4a71b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a440c90e0 con 0x7f3a44152fd0 2024-04-25T01:31:27.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.304+0000 7f3a4a71b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3a44157400 con 0x7f3a440c9b80 2024-04-25T01:31:27.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.304+0000 7f3a43fff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f3a44152fd0 0x7f3a44156280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:51672/0 (socket says 172.21.15.112:51672) 2024-04-25T01:31:27.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.304+0000 7f3a48cb8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a440c9b80 0x7f3a4406bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:27.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.304+0000 7f3a43fff640 1 -- 172.21.15.112:0/807178246 learned_addr learned my addr 172.21.15.112:0/807178246 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:31:27.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.305+0000 7f3a48cb8640 1 -- 172.21.15.112:0/807178246 >> v1:172.21.15.187:6789/0 conn(0x7f3a44152fd0 legacy=0x7f3a44156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:27.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.305+0000 7f3a48cb8640 1 -- 172.21.15.112:0/807178246 >> v1:172.21.15.112:6789/0 conn(0x7f3a440c5d70 legacy=0x7f3a44156cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:31:27.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.305+0000 7f3a42ffd640 1 -- 172.21.15.112:0/807178246 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 145097783 0 0) 0x7f3a440c90e0 con 0x7f3a44152fd0 2024-04-25T01:31:27.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.305+0000 7f3a48cb8640 1 -- 172.21.15.112:0/807178246 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a440cac00 con 0x7f3a440c9b80 2024-04-25T01:31:27.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.305+0000 7f3a48cb8640 1 --2- 172.21.15.112:0/807178246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a440c9b80 0x7f3a4406bda0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009dc0 tx=0x7f3a3c02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27b0c512edf25c81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:27.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.306+0000 7f3a42ffd640 1 -- 172.21.15.112:0/807178246 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c007740 con 0x7f3a440c9b80 2024-04-25T01:31:27.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.306+0000 7f3a42ffd640 1 -- 172.21.15.112:0/807178246 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a3c0078e0 con 0x7f3a440c9b80 2024-04-25T01:31:27.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.306+0000 7f3a42ffd640 1 -- 172.21.15.112:0/807178246 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c007cb0 con 0x7f3a440c9b80 2024-04-25T01:31:27.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.306+0000 7f3a4a71b640 1 -- 172.21.15.112:0/807178246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a440c9b80 msgr2=0x7f3a4406bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:27.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.306+0000 7f3a4a71b640 1 --2- 172.21.15.112:0/807178246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a440c9b80 0x7f3a4406bda0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009dc0 tx=0x7f3a3c02a5f0 comp rx=0 tx=0).stop 2024-04-25T01:31:27.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.306+0000 7f3a4a71b640 1 -- 172.21.15.112:0/807178246 shutdown_connections 2024-04-25T01:31:27.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.306+0000 7f3a4a71b640 1 --2- 172.21.15.112:0/807178246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a440c9b80 0x7f3a4406bda0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:27.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.306+0000 7f3a4a71b640 1 -- 172.21.15.112:0/807178246 >> 172.21.15.112:0/807178246 conn(0x7f3a440acc70 msgr2=0x7f3a440ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:27.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.306+0000 7f3a4a71b640 1 -- 172.21.15.112:0/807178246 shutdown_connections 2024-04-25T01:31:27.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.306+0000 7f3a4a71b640 1 -- 172.21.15.112:0/807178246 wait complete. 2024-04-25T01:31:27.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.308+0000 7f3a4a71b640 1 Processor -- start 2024-04-25T01:31:27.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.308+0000 7f3a4a71b640 1 -- start start 2024-04-25T01:31:27.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.308+0000 7f3a4a71b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a440c5d70 0x7f3a44154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:27.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.308+0000 7f3a4a71b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a440c9b80 0x7f3a4414d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:27.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.308+0000 7f3a4a71b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a44152fd0 0x7f3a441ee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:27.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.309+0000 7f3a4a71b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3a440c9660 con 0x7f3a440c9b80 2024-04-25T01:31:27.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.309+0000 7f3a4a71b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3a440ca9b0 con 0x7f3a440c5d70 2024-04-25T01:31:27.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.309+0000 7f3a4a71b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3a440caee0 con 0x7f3a44152fd0 2024-04-25T01:31:27.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.309+0000 7f3a437fe640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a440c9b80 msgr2=0x7f3a4414d270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:31:27.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.309+0000 7f3a437fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a440c9b80 0x7f3a4414d270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:31:27.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.309+0000 7f3a43fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a440c5d70 0x7f3a44154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:27.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.309+0000 7f3a43fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a440c5d70 0x7f3a44154f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:36472/0 (socket says 172.21.15.112:36472) 2024-04-25T01:31:27.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.309+0000 7f3a43fff640 1 -- 172.21.15.112:0/2274565890 learned_addr learned my addr 172.21.15.112:0/2274565890 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:27.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.309+0000 7f3a48cb8640 1 --2- 172.21.15.112:0/2274565890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a44152fd0 0x7f3a441ee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:27.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.309+0000 7f3a43fff640 1 -- 172.21.15.112:0/2274565890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a44152fd0 msgr2=0x7f3a441ee7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:27.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.309+0000 7f3a43fff640 1 --2- 172.21.15.112:0/2274565890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a44152fd0 0x7f3a441ee7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:27.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.309+0000 7f3a43fff640 1 -- 172.21.15.112:0/2274565890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a440c9b80 msgr2=0x7f3a4414d270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:31:27.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.309+0000 7f3a43fff640 1 --2- 172.21.15.112:0/2274565890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a440c9b80 0x7f3a4414d270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:27.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.309+0000 7f3a43fff640 1 -- 172.21.15.112:0/2274565890 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a441ef240 con 0x7f3a440c5d70 2024-04-25T01:31:27.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.309+0000 7f3a43fff640 1 --2- 172.21.15.112:0/2274565890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a440c5d70 0x7f3a44154f40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400c460 tx=0x7f3a3400c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:27.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.309+0000 7f3a40ff9640 1 -- 172.21.15.112:0/2274565890 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34018070 con 0x7f3a440c5d70 2024-04-25T01:31:27.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.309+0000 7f3a40ff9640 1 -- 172.21.15.112:0/2274565890 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a34007500 con 0x7f3a440c5d70 2024-04-25T01:31:27.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.309+0000 7f3a4a71b640 1 -- 172.21.15.112:0/2274565890 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a44154740 con 0x7f3a440c5d70 2024-04-25T01:31:27.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.309+0000 7f3a40ff9640 1 -- 172.21.15.112:0/2274565890 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34004a20 con 0x7f3a440c5d70 2024-04-25T01:31:27.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.310+0000 7f3a4a71b640 1 -- 172.21.15.112:0/2274565890 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4414f5a0 con 0x7f3a440c5d70 2024-04-25T01:31:27.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.311+0000 7f3a4a71b640 1 -- 172.21.15.112:0/2274565890 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a440682a0 con 0x7f3a440c5d70 2024-04-25T01:31:27.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.311+0000 7f3a40ff9640 1 -- 172.21.15.112:0/2274565890 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a34004bc0 con 0x7f3a440c5d70 2024-04-25T01:31:27.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.311+0000 7f3a40ff9640 1 --2- 172.21.15.112:0/2274565890 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a3006bde0 0x7f3a3006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:27.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.312+0000 7f3a40ff9640 1 -- 172.21.15.112:0/2274565890 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f3a3408a430 con 0x7f3a440c5d70 2024-04-25T01:31:27.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.312+0000 7f3a437fe640 1 --2- 172.21.15.112:0/2274565890 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a3006bde0 0x7f3a3006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:27.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.312+0000 7f3a437fe640 1 --2- 172.21.15.112:0/2274565890 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a3006bde0 0x7f3a3006e2a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3a440bf470 tx=0x7f3a38006d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:27.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.317+0000 7f3a40ff9640 1 -- 172.21.15.112:0/2274565890 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3a34058400 con 0x7f3a440c5d70 2024-04-25T01:31:27.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.607+0000 7f3a4a71b640 1 -- 172.21.15.112:0/2274565890 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3a440c5ac0 con 0x7f3a440c5d70 2024-04-25T01:31:27.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.608+0000 7f3a40ff9640 1 -- 172.21.15.112:0/2274565890 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f3a3405c700 con 0x7f3a440c5d70 2024-04-25T01:31:27.585 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:31:27.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.610+0000 7f3a4a71b640 1 -- 172.21.15.112:0/2274565890 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a3006bde0 msgr2=0x7f3a3006e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:27.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.610+0000 7f3a4a71b640 1 --2- 172.21.15.112:0/2274565890 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a3006bde0 0x7f3a3006e2a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3a440bf470 tx=0x7f3a38006d30 comp rx=0 tx=0).stop 2024-04-25T01:31:27.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.610+0000 7f3a4a71b640 1 -- 172.21.15.112:0/2274565890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a440c5d70 msgr2=0x7f3a44154f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:27.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.610+0000 7f3a4a71b640 1 --2- 172.21.15.112:0/2274565890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a440c5d70 0x7f3a44154f40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400c460 tx=0x7f3a3400c920 comp rx=0 tx=0).stop 2024-04-25T01:31:27.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.610+0000 7f3a4a71b640 1 -- 172.21.15.112:0/2274565890 shutdown_connections 2024-04-25T01:31:27.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.610+0000 7f3a4a71b640 1 --2- 172.21.15.112:0/2274565890 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a3006bde0 0x7f3a3006e2a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:27.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.610+0000 7f3a4a71b640 1 --2- 172.21.15.112:0/2274565890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a44152fd0 0x7f3a441ee7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:27.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.610+0000 7f3a4a71b640 1 --2- 172.21.15.112:0/2274565890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a440c9b80 0x7f3a4414d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:27.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.610+0000 7f3a4a71b640 1 --2- 172.21.15.112:0/2274565890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a440c5d70 0x7f3a44154f40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:27.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.610+0000 7f3a4a71b640 1 -- 172.21.15.112:0/2274565890 >> 172.21.15.112:0/2274565890 conn(0x7f3a440acc70 msgr2=0x7f3a4406c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:27.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.611+0000 7f3a4a71b640 1 -- 172.21.15.112:0/2274565890 shutdown_connections 2024-04-25T01:31:27.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:27.611+0000 7f3a4a71b640 1 -- 172.21.15.112:0/2274565890 wait complete. 2024-04-25T01:31:30.600 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:31:30.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.736+0000 7f75e7d9e640 1 Processor -- start 2024-04-25T01:31:30.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.736+0000 7f75e7d9e640 1 -- start start 2024-04-25T01:31:30.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.737+0000 7f75e7d9e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f75e00c4780 0x7f75e01565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:30.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.737+0000 7f75e7d9e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75e006ee60 0x7f75e006f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:30.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.737+0000 7f75e7d9e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75e0153500 con 0x7f75e014c7b0 2024-04-25T01:31:30.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.737+0000 7f75e7d9e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f75e0152fd0 con 0x7f75e00c4780 2024-04-25T01:31:30.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.737+0000 7f75e7d9e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f75e00c2c60 con 0x7f75e006ee60 2024-04-25T01:31:30.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.737+0000 7f75e5b3a640 1 -- >> v1:172.21.15.112:6789/0 conn(0x7f75e014c7b0 legacy=0x7f75e0155eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:31:30.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.737+0000 7f75e633b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75e006ee60 0x7f75e006f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:30.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.737+0000 7f75e5339640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f75e00c4780 0x7f75e01565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:30.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.737+0000 7f75e5339640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f75e00c4780 0x7f75e01565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:35708/0 (socket says 172.21.15.112:35708) 2024-04-25T01:31:30.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.737+0000 7f75e633b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75e006ee60 0x7f75e006f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:36770/0 (socket says 172.21.15.112:36770) 2024-04-25T01:31:30.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.737+0000 7f75e5339640 1 -- 172.21.15.112:0/515684419 learned_addr learned my addr 172.21.15.112:0/515684419 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:30.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.737+0000 7f75e633b640 1 -- 172.21.15.112:0/515684419 >> v2:172.21.15.187:3300/0 conn(0x7f75e00c4780 msgr2=0x7f75e01565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:31:30.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.737+0000 7f75e633b640 1 --2- 172.21.15.112:0/515684419 >> v2:172.21.15.187:3300/0 conn(0x7f75e00c4780 0x7f75e01565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:30.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.737+0000 7f75e633b640 1 -- 172.21.15.112:0/515684419 >> v1:172.21.15.112:6789/0 conn(0x7f75e014c7b0 legacy=0x7f75e0155eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:31:30.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.737+0000 7f75e633b640 1 -- 172.21.15.112:0/515684419 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e00c3270 con 0x7f75e006ee60 2024-04-25T01:31:30.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.737+0000 7f75e5339640 1 --2- 172.21.15.112:0/515684419 >> v2:172.21.15.187:3300/0 conn(0x7f75e00c4780 0x7f75e01565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:31:30.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.738+0000 7f75e633b640 1 --2- 172.21.15.112:0/515684419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75e006ee60 0x7f75e006f2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f75d800bfa0 tx=0x7f75d802c6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e54b7a87e7aadbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:30.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.738+0000 7f75e4b38640 1 -- 172.21.15.112:0/515684419 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8038070 con 0x7f75e006ee60 2024-04-25T01:31:30.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.738+0000 7f75e4b38640 1 -- 172.21.15.112:0/515684419 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75d8009540 con 0x7f75e006ee60 2024-04-25T01:31:30.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.738+0000 7f75e4b38640 1 -- 172.21.15.112:0/515684419 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8009910 con 0x7f75e006ee60 2024-04-25T01:31:30.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.739+0000 7f75e7d9e640 1 -- 172.21.15.112:0/515684419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75e006ee60 msgr2=0x7f75e006f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:30.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.739+0000 7f75e7d9e640 1 --2- 172.21.15.112:0/515684419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75e006ee60 0x7f75e006f2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f75d800bfa0 tx=0x7f75d802c6f0 comp rx=0 tx=0).stop 2024-04-25T01:31:30.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.739+0000 7f75e7d9e640 1 -- 172.21.15.112:0/515684419 shutdown_connections 2024-04-25T01:31:30.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.739+0000 7f75e7d9e640 1 --2- 172.21.15.112:0/515684419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75e006ee60 0x7f75e006f2f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:30.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.739+0000 7f75e7d9e640 1 --2- 172.21.15.112:0/515684419 >> v2:172.21.15.187:3300/0 conn(0x7f75e00c4780 0x7f75e01565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:30.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.739+0000 7f75e7d9e640 1 -- 172.21.15.112:0/515684419 >> 172.21.15.112:0/515684419 conn(0x7f75e00c8130 msgr2=0x7f75e00c8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:30.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.739+0000 7f75e7d9e640 1 -- 172.21.15.112:0/515684419 shutdown_connections 2024-04-25T01:31:30.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.739+0000 7f75e7d9e640 1 -- 172.21.15.112:0/515684419 wait complete. 2024-04-25T01:31:30.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.740+0000 7f75e7d9e640 1 Processor -- start 2024-04-25T01:31:30.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.740+0000 7f75e7d9e640 1 -- start start 2024-04-25T01:31:30.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.741+0000 7f75e7d9e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75e006ee60 0x7f75e01e6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:30.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.741+0000 7f75e7d9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e00c4780 0x7f75e0155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:30.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.741+0000 7f75e7d9e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75e014c7b0 0x7f75e01404a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:30.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.741+0000 7f75e7d9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75e006fdf0 con 0x7f75e00c4780 2024-04-25T01:31:30.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.741+0000 7f75e7d9e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75e00c2f00 con 0x7f75e014c7b0 2024-04-25T01:31:30.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.741+0000 7f75e7d9e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f75e01e7d40 con 0x7f75e006ee60 2024-04-25T01:31:30.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.741+0000 7f75e5339640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e00c4780 msgr2=0x7f75e0155360 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:31:30.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.741+0000 7f75e5339640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e00c4780 0x7f75e0155360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:31:30.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.741+0000 7f75e5b3a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75e006ee60 0x7f75e01e6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:30.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.741+0000 7f75e633b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75e014c7b0 0x7f75e01404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:30.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.741+0000 7f75e5b3a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75e006ee60 0x7f75e01e6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:36778/0 (socket says 172.21.15.112:36778) 2024-04-25T01:31:30.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.741+0000 7f75e5b3a640 1 -- 172.21.15.112:0/1747725690 learned_addr learned my addr 172.21.15.112:0/1747725690 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:30.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.742+0000 7f75e5b3a640 1 -- 172.21.15.112:0/1747725690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75e014c7b0 msgr2=0x7f75e01404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:30.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.742+0000 7f75e5b3a640 1 --2- 172.21.15.112:0/1747725690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75e014c7b0 0x7f75e01404a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:30.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.742+0000 7f75e5b3a640 1 -- 172.21.15.112:0/1747725690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e00c4780 msgr2=0x7f75e0155360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:31:30.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.742+0000 7f75e5b3a640 1 --2- 172.21.15.112:0/1747725690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e00c4780 0x7f75e0155360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:30.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.742+0000 7f75e5b3a640 1 -- 172.21.15.112:0/1747725690 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e01539e0 con 0x7f75e006ee60 2024-04-25T01:31:30.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.742+0000 7f75e633b640 1 --2- 172.21.15.112:0/1747725690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75e014c7b0 0x7f75e01404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:31:30.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.742+0000 7f75e5b3a640 1 --2- 172.21.15.112:0/1747725690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75e006ee60 0x7f75e01e6d60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00b580 tx=0x7f75dc00ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:30.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.742+0000 7f75d67fc640 1 -- 172.21.15.112:0/1747725690 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc003dc0 con 0x7f75e006ee60 2024-04-25T01:31:30.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.742+0000 7f75d67fc640 1 -- 172.21.15.112:0/1747725690 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75dc015070 con 0x7f75e006ee60 2024-04-25T01:31:30.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.742+0000 7f75e7d9e640 1 -- 172.21.15.112:0/1747725690 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e0153cc0 con 0x7f75e006ee60 2024-04-25T01:31:30.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.742+0000 7f75d67fc640 1 -- 172.21.15.112:0/1747725690 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc005330 con 0x7f75e006ee60 2024-04-25T01:31:30.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.742+0000 7f75e7d9e640 1 -- 172.21.15.112:0/1747725690 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e01558a0 con 0x7f75e006ee60 2024-04-25T01:31:30.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.743+0000 7f75e7d9e640 1 -- 172.21.15.112:0/1747725690 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75e00689b0 con 0x7f75e006ee60 2024-04-25T01:31:30.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.744+0000 7f75d67fc640 1 -- 172.21.15.112:0/1747725690 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f75dc0054d0 con 0x7f75e006ee60 2024-04-25T01:31:30.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.744+0000 7f75d67fc640 1 --2- 172.21.15.112:0/1747725690 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f75c406bde0 0x7f75c406e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:30.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.744+0000 7f75d67fc640 1 -- 172.21.15.112:0/1747725690 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f75dc08aeb0 con 0x7f75e006ee60 2024-04-25T01:31:30.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.744+0000 7f75e5339640 1 --2- 172.21.15.112:0/1747725690 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f75c406bde0 0x7f75c406e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:30.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.745+0000 7f75e5339640 1 --2- 172.21.15.112:0/1747725690 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f75c406bde0 0x7f75c406e2a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f75d0006c20 tx=0x7f75d0009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:30.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:30.749+0000 7f75d67fc640 1 -- 172.21.15.112:0/1747725690 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f75dc059550 con 0x7f75e006ee60 2024-04-25T01:31:31.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:31.029+0000 7f75e7d9e640 1 -- 172.21.15.112:0/1747725690 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f75e006ff70 con 0x7f75e006ee60 2024-04-25T01:31:31.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:31.029+0000 7f75d67fc640 1 -- 172.21.15.112:0/1747725690 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f75dc058e50 con 0x7f75e006ee60 2024-04-25T01:31:31.007 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:31:31.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:31.032+0000 7f75e7d9e640 1 -- 172.21.15.112:0/1747725690 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f75c406bde0 msgr2=0x7f75c406e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:31.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:31.032+0000 7f75e7d9e640 1 --2- 172.21.15.112:0/1747725690 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f75c406bde0 0x7f75c406e2a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f75d0006c20 tx=0x7f75d0009240 comp rx=0 tx=0).stop 2024-04-25T01:31:31.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:31.032+0000 7f75e7d9e640 1 -- 172.21.15.112:0/1747725690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75e006ee60 msgr2=0x7f75e01e6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:31.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:31.032+0000 7f75e7d9e640 1 --2- 172.21.15.112:0/1747725690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75e006ee60 0x7f75e01e6d60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00b580 tx=0x7f75dc00ba40 comp rx=0 tx=0).stop 2024-04-25T01:31:31.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:31.032+0000 7f75e7d9e640 1 -- 172.21.15.112:0/1747725690 shutdown_connections 2024-04-25T01:31:31.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:31.032+0000 7f75e7d9e640 1 --2- 172.21.15.112:0/1747725690 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f75c406bde0 0x7f75c406e2a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:31.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:31.032+0000 7f75e7d9e640 1 --2- 172.21.15.112:0/1747725690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75e014c7b0 0x7f75e01404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:31.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:31.032+0000 7f75e7d9e640 1 --2- 172.21.15.112:0/1747725690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e00c4780 0x7f75e0155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:31.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:31.032+0000 7f75e7d9e640 1 --2- 172.21.15.112:0/1747725690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75e006ee60 0x7f75e01e6d60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:31.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:31.032+0000 7f75e7d9e640 1 -- 172.21.15.112:0/1747725690 >> 172.21.15.112:0/1747725690 conn(0x7f75e00c8130 msgr2=0x7f75e0156380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:31.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:31.032+0000 7f75e7d9e640 1 -- 172.21.15.112:0/1747725690 shutdown_connections 2024-04-25T01:31:31.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:31.032+0000 7f75e7d9e640 1 -- 172.21.15.112:0/1747725690 wait complete. 2024-04-25T01:31:34.022 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:31:34.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.146+0000 7feb5703f640 1 Processor -- start 2024-04-25T01:31:34.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.146+0000 7feb5703f640 1 -- start start 2024-04-25T01:31:34.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.147+0000 7feb5703f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 0x7feb500c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:34.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.147+0000 7feb5703f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb500c90e0 con 0x7feb50156cf0 2024-04-25T01:31:34.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.147+0000 7feb5703f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb500cac00 con 0x7feb50152fd0 2024-04-25T01:31:34.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.147+0000 7feb5703f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7feb500ca9b0 con 0x7feb500c5d70 2024-04-25T01:31:34.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.147+0000 7feb555dc640 1 -- >> v1:172.21.15.112:6789/0 conn(0x7feb50156cf0 legacy=0x7feb5006bda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:31:34.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.147+0000 7feb4ffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 0x7feb500c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:34.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.147+0000 7feb4ffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 0x7feb500c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:35716/0 (socket says 172.21.15.112:35716) 2024-04-25T01:31:34.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.147+0000 7feb54ddb640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7feb50152fd0 0x7feb50156280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:40096/0 (socket says 172.21.15.112:40096) 2024-04-25T01:31:34.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.147+0000 7feb4ffff640 1 -- 172.21.15.112:0/706518719 learned_addr learned my addr 172.21.15.112:0/706518719 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:34.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.147+0000 7feb4ffff640 1 -- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 msgr2=0x7feb500c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:31:34.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.147+0000 7feb4ffff640 1 -- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 msgr2=0x7feb500c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:31:34.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.147+0000 7feb4ffff640 1 --2- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 0x7feb500c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:31:34.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.147+0000 7feb4ffff640 1 --2- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 0x7feb500c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:31:34.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.348+0000 7feb555dc640 1 -- 172.21.15.112:0/706518719 >> v1:172.21.15.112:6789/0 conn(0x7feb50156cf0 legacy=0x7feb5006bda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:31:34.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.348+0000 7feb4ffff640 1 --2- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 0x7feb500c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:34.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.348+0000 7feb4ffff640 1 -- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 msgr2=0x7feb500c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:31:34.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.348+0000 7feb4ffff640 1 -- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 msgr2=0x7feb500c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:31:34.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.348+0000 7feb4ffff640 1 --2- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 0x7feb500c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:31:34.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.348+0000 7feb4ffff640 1 --2- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 0x7feb500c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:31:34.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.749+0000 7feb555dc640 1 -- 172.21.15.112:0/706518719 >> v1:172.21.15.112:6789/0 conn(0x7feb50156cf0 legacy=0x7feb5006bda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:31:34.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.749+0000 7feb4ffff640 1 --2- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 0x7feb500c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:34.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.749+0000 7feb4ffff640 1 -- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 msgr2=0x7feb500c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:31:34.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.749+0000 7feb4ffff640 1 -- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 msgr2=0x7feb500c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:31:34.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.749+0000 7feb4ffff640 1 --2- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 0x7feb500c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:31:34.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:34.749+0000 7feb4ffff640 1 --2- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 0x7feb500c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:31:35.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:35.550+0000 7feb555dc640 1 -- 172.21.15.112:0/706518719 >> v1:172.21.15.112:6789/0 conn(0x7feb50156cf0 legacy=0x7feb5006bda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:31:35.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:35.550+0000 7feb4ffff640 1 --2- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 0x7feb500c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:35.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:35.550+0000 7feb4ffff640 1 -- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 msgr2=0x7feb500c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:31:35.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:35.550+0000 7feb4ffff640 1 -- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 msgr2=0x7feb500c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:31:35.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:35.550+0000 7feb4ffff640 1 --2- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 0x7feb500c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:31:35.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:35.550+0000 7feb4ffff640 1 --2- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 0x7feb500c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:31:37.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.147+0000 7feb4e7fc640 1 -- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 msgr2=0x7feb500c9b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:31:37.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.147+0000 7feb4e7fc640 1 --2- 172.21.15.112:0/706518719 >> v2:172.21.15.187:3300/0 conn(0x7feb500c5d70 0x7feb500c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:37.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.147+0000 7feb4e7fc640 1 -- 172.21.15.112:0/706518719 >> v1:172.21.15.187:6789/0 conn(0x7feb50152fd0 legacy=0x7feb50156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:37.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.147+0000 7feb4e7fc640 1 -- 172.21.15.112:0/706518719 >> v1:172.21.15.112:6789/0 conn(0x7feb50156cf0 legacy=0x7feb5006bda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:31:37.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.147+0000 7feb4e7fc640 1 --2- 172.21.15.112:0/706518719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb34003690 0x7feb34005ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:37.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.147+0000 7feb4e7fc640 1 --2- 172.21.15.112:0/706518719 >> v2:172.21.15.112:3300/0 conn(0x7feb34006170 0x7feb34008630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:37.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.147+0000 7feb4e7fc640 1 -- 172.21.15.112:0/706518719 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7feb500ca9b0 con 0x7feb34006170 2024-04-25T01:31:37.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.147+0000 7feb4e7fc640 1 -- 172.21.15.112:0/706518719 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feb34008cf0 con 0x7feb34003690 2024-04-25T01:31:37.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.147+0000 7feb4ffff640 1 -- 172.21.15.112:0/706518719 >> v2:172.21.15.112:3300/0 conn(0x7feb34006170 msgr2=0x7feb34008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:31:37.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.147+0000 7feb4ffff640 1 --2- 172.21.15.112:0/706518719 >> v2:172.21.15.112:3300/0 conn(0x7feb34006170 0x7feb34008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:31:37.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.147+0000 7feb54ddb640 1 --2- 172.21.15.112:0/706518719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb34003690 0x7feb34005ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:37.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.147+0000 7feb54ddb640 1 -- 172.21.15.112:0/706518719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb34003690 msgr2=0x7feb34005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:31:37.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.147+0000 7feb54ddb640 1 -- 172.21.15.112:0/706518719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb34003690 msgr2=0x7feb34005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:31:37.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.147+0000 7feb54ddb640 1 --2- 172.21.15.112:0/706518719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb34003690 0x7feb34005ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:31:37.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.147+0000 7feb54ddb640 1 --2- 172.21.15.112:0/706518719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb34003690 0x7feb34005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:31:37.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.348+0000 7feb4ffff640 1 -- 172.21.15.112:0/706518719 >> v2:172.21.15.112:3300/0 conn(0x7feb34006170 msgr2=0x7feb34008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:31:37.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.348+0000 7feb4ffff640 1 --2- 172.21.15.112:0/706518719 >> v2:172.21.15.112:3300/0 conn(0x7feb34006170 0x7feb34008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:31:37.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.348+0000 7feb54ddb640 1 --2- 172.21.15.112:0/706518719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb34003690 0x7feb34005ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:37.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.348+0000 7feb54ddb640 1 -- 172.21.15.112:0/706518719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb34003690 msgr2=0x7feb34005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:31:37.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.348+0000 7feb54ddb640 1 -- 172.21.15.112:0/706518719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb34003690 msgr2=0x7feb34005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:31:37.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.348+0000 7feb54ddb640 1 --2- 172.21.15.112:0/706518719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb34003690 0x7feb34005ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:31:37.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.348+0000 7feb54ddb640 1 --2- 172.21.15.112:0/706518719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb34003690 0x7feb34005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:31:37.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.749+0000 7feb4ffff640 1 -- 172.21.15.112:0/706518719 >> v2:172.21.15.112:3300/0 conn(0x7feb34006170 msgr2=0x7feb34008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:31:37.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.749+0000 7feb4ffff640 1 --2- 172.21.15.112:0/706518719 >> v2:172.21.15.112:3300/0 conn(0x7feb34006170 0x7feb34008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:31:37.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.749+0000 7feb54ddb640 1 --2- 172.21.15.112:0/706518719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb34003690 0x7feb34005ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:37.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.749+0000 7feb54ddb640 1 -- 172.21.15.112:0/706518719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb34003690 msgr2=0x7feb34005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:31:37.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.749+0000 7feb54ddb640 1 -- 172.21.15.112:0/706518719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb34003690 msgr2=0x7feb34005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:31:37.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.749+0000 7feb54ddb640 1 --2- 172.21.15.112:0/706518719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb34003690 0x7feb34005ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:31:37.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:37.749+0000 7feb54ddb640 1 --2- 172.21.15.112:0/706518719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb34003690 0x7feb34005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:31:38.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.550+0000 7feb4ffff640 1 -- 172.21.15.112:0/706518719 >> v2:172.21.15.112:3300/0 conn(0x7feb34006170 msgr2=0x7feb34008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:31:38.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.550+0000 7feb4ffff640 1 --2- 172.21.15.112:0/706518719 >> v2:172.21.15.112:3300/0 conn(0x7feb34006170 0x7feb34008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:31:38.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.550+0000 7feb54ddb640 1 --2- 172.21.15.112:0/706518719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb34003690 0x7feb34005ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:38.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.550+0000 7feb54ddb640 1 -- 172.21.15.112:0/706518719 >> v2:172.21.15.112:3300/0 conn(0x7feb34006170 msgr2=0x7feb34008630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:31:38.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.550+0000 7feb54ddb640 1 --2- 172.21.15.112:0/706518719 >> v2:172.21.15.112:3300/0 conn(0x7feb34006170 0x7feb34008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:38.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.550+0000 7feb54ddb640 1 -- 172.21.15.112:0/706518719 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb500c9380 con 0x7feb34003690 2024-04-25T01:31:38.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.551+0000 7feb54ddb640 1 --2- 172.21.15.112:0/706518719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb34003690 0x7feb34005ae0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feb44007fd0 tx=0x7feb4402a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c830ed7ae7adf464 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.551+0000 7feb4f7fe640 1 -- 172.21.15.112:0/706518719 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44038070 con 0x7feb34003690 2024-04-25T01:31:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.551+0000 7feb4f7fe640 1 -- 172.21.15.112:0/706518719 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb44033410 con 0x7feb34003690 2024-04-25T01:31:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.551+0000 7feb4f7fe640 1 -- 172.21.15.112:0/706518719 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44033800 con 0x7feb34003690 2024-04-25T01:31:38.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.552+0000 7feb5703f640 1 -- 172.21.15.112:0/706518719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb34003690 msgr2=0x7feb34005ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:38.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.552+0000 7feb5703f640 1 --2- 172.21.15.112:0/706518719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb34003690 0x7feb34005ae0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feb44007fd0 tx=0x7feb4402a730 comp rx=0 tx=0).stop 2024-04-25T01:31:38.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.552+0000 7feb555dc640 1 -- 172.21.15.112:0/706518719 reap_dead start 2024-04-25T01:31:38.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.552+0000 7feb5703f640 1 -- 172.21.15.112:0/706518719 shutdown_connections 2024-04-25T01:31:38.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.552+0000 7feb5703f640 1 -- 172.21.15.112:0/706518719 >> 172.21.15.112:0/706518719 conn(0x7feb500acc70 msgr2=0x7feb500ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:38.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.552+0000 7feb5703f640 1 -- 172.21.15.112:0/706518719 shutdown_connections 2024-04-25T01:31:38.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.552+0000 7feb5703f640 1 -- 172.21.15.112:0/706518719 wait complete. 2024-04-25T01:31:38.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.553+0000 7feb5703f640 1 Processor -- start 2024-04-25T01:31:38.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.553+0000 7feb5703f640 1 -- start start 2024-04-25T01:31:38.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.554+0000 7feb5703f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb34003690 0x7feb500b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:38.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.554+0000 7feb5703f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb50151970 0x7feb501eee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:38.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.554+0000 7feb5703f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb5014d270 0x7feb500c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:38.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.554+0000 7feb5703f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb5006c4b0 con 0x7feb34003690 2024-04-25T01:31:38.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.554+0000 7feb5703f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feb500caee0 con 0x7feb50151970 2024-04-25T01:31:38.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.554+0000 7feb5703f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feb5006c630 con 0x7feb5014d270 2024-04-25T01:31:38.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.554+0000 7feb54ddb640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb34003690 msgr2=0x7feb500b02a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:31:38.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.554+0000 7feb54ddb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb34003690 0x7feb500b02a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:31:38.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.554+0000 7feb555dc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb5014d270 0x7feb500c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:38.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.554+0000 7feb555dc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb5014d270 0x7feb500c6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:42846/0 (socket says 172.21.15.112:42846) 2024-04-25T01:31:38.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.554+0000 7feb555dc640 1 -- 172.21.15.112:0/2946871584 learned_addr learned my addr 172.21.15.112:0/2946871584 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:38.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.554+0000 7feb4ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb50151970 0x7feb501eee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:38.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.555+0000 7feb555dc640 1 -- 172.21.15.112:0/2946871584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb50151970 msgr2=0x7feb501eee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:38.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.555+0000 7feb555dc640 1 --2- 172.21.15.112:0/2946871584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb50151970 0x7feb501eee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:38.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.555+0000 7feb555dc640 1 -- 172.21.15.112:0/2946871584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb34003690 msgr2=0x7feb500b02a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:31:38.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.555+0000 7feb555dc640 1 --2- 172.21.15.112:0/2946871584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb34003690 0x7feb500b02a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:38.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.555+0000 7feb555dc640 1 -- 172.21.15.112:0/2946871584 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb501ef3c0 con 0x7feb5014d270 2024-04-25T01:31:38.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.555+0000 7feb4ffff640 1 --2- 172.21.15.112:0/2946871584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb50151970 0x7feb501eee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:31:38.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.555+0000 7feb555dc640 1 --2- 172.21.15.112:0/2946871584 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb5014d270 0x7feb500c6a70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb48003d00 tx=0x7feb4800bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:38.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.555+0000 7feb4d7fa640 1 -- 172.21.15.112:0/2946871584 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4800cdf0 con 0x7feb5014d270 2024-04-25T01:31:38.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.555+0000 7feb4d7fa640 1 -- 172.21.15.112:0/2946871584 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb48008030 con 0x7feb5014d270 2024-04-25T01:31:38.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.555+0000 7feb5703f640 1 -- 172.21.15.112:0/2946871584 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb50151ec0 con 0x7feb5014d270 2024-04-25T01:31:38.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.555+0000 7feb4d7fa640 1 -- 172.21.15.112:0/2946871584 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb480155d0 con 0x7feb5014d270 2024-04-25T01:31:38.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.555+0000 7feb5703f640 1 -- 172.21.15.112:0/2946871584 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feb501ee680 con 0x7feb5014d270 2024-04-25T01:31:38.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.556+0000 7feb5703f640 1 -- 172.21.15.112:0/2946871584 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb501523e0 con 0x7feb5014d270 2024-04-25T01:31:38.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.557+0000 7feb4d7fa640 1 -- 172.21.15.112:0/2946871584 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb4800c040 con 0x7feb5014d270 2024-04-25T01:31:38.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.557+0000 7feb4d7fa640 1 --2- 172.21.15.112:0/2946871584 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feb3406ccc0 0x7feb3406f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:38.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.557+0000 7feb54ddb640 1 --2- 172.21.15.112:0/2946871584 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feb3406ccc0 0x7feb3406f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:38.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.557+0000 7feb4d7fa640 1 -- 172.21.15.112:0/2946871584 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7feb4808ae50 con 0x7feb5014d270 2024-04-25T01:31:38.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.557+0000 7feb54ddb640 1 --2- 172.21.15.112:0/2946871584 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feb3406ccc0 0x7feb3406f180 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7feb44035000 tx=0x7feb44010040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:38.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.562+0000 7feb4d7fa640 1 -- 172.21.15.112:0/2946871584 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feb501523e0 con 0x7feb5014d270 2024-04-25T01:31:38.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.851+0000 7feb5703f640 1 -- 172.21.15.112:0/2946871584 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7feb500682a0 con 0x7feb5014d270 2024-04-25T01:31:38.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.852+0000 7feb4d7fa640 1 -- 172.21.15.112:0/2946871584 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7feb48058e20 con 0x7feb5014d270 2024-04-25T01:31:38.830 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":42,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:31:38.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.854+0000 7feb5703f640 1 -- 172.21.15.112:0/2946871584 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feb3406ccc0 msgr2=0x7feb3406f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:38.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.854+0000 7feb5703f640 1 --2- 172.21.15.112:0/2946871584 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feb3406ccc0 0x7feb3406f180 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7feb44035000 tx=0x7feb44010040 comp rx=0 tx=0).stop 2024-04-25T01:31:38.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.854+0000 7feb5703f640 1 -- 172.21.15.112:0/2946871584 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb5014d270 msgr2=0x7feb500c6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:38.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.854+0000 7feb5703f640 1 --2- 172.21.15.112:0/2946871584 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb5014d270 0x7feb500c6a70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb48003d00 tx=0x7feb4800bf00 comp rx=0 tx=0).stop 2024-04-25T01:31:38.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.854+0000 7feb5703f640 1 -- 172.21.15.112:0/2946871584 shutdown_connections 2024-04-25T01:31:38.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.854+0000 7feb5703f640 1 --2- 172.21.15.112:0/2946871584 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feb3406ccc0 0x7feb3406f180 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:38.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.854+0000 7feb5703f640 1 --2- 172.21.15.112:0/2946871584 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb5014d270 0x7feb500c6a70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:38.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.854+0000 7feb5703f640 1 --2- 172.21.15.112:0/2946871584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb50151970 0x7feb501eee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:38.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.854+0000 7feb5703f640 1 --2- 172.21.15.112:0/2946871584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb34003690 0x7feb500b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:38.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.854+0000 7feb5703f640 1 -- 172.21.15.112:0/2946871584 >> 172.21.15.112:0/2946871584 conn(0x7feb500acc70 msgr2=0x7feb5006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:38.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.854+0000 7feb5703f640 1 -- 172.21.15.112:0/2946871584 shutdown_connections 2024-04-25T01:31:38.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.854+0000 7feb5703f640 1 -- 172.21.15.112:0/2946871584 wait complete. 2024-04-25T01:31:38.843 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T01:31:38.843 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-04-25T01:31:38.844 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:31:38.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.971+0000 7fe434f63640 1 Processor -- start 2024-04-25T01:31:38.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.971+0000 7fe434f63640 1 -- start start 2024-04-25T01:31:38.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.972+0000 7fe434f63640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe4301596a0 0x7fe430152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:38.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.972+0000 7fe434f63640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe4300c72b0 0x7fe430160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:38.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.972+0000 7fe434f63640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe430159b90 con 0x7fe4301592c0 2024-04-25T01:31:38.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.972+0000 7fe434f63640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fe4300c6e20 con 0x7fe4301596a0 2024-04-25T01:31:38.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.972+0000 7fe434f63640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe43006c970 con 0x7fe4300c72b0 2024-04-25T01:31:38.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.972+0000 7fe42dd9b640 1 -- >> v2:172.21.15.112:3300/0 conn(0x7fe4301596a0 msgr2=0x7fe430152d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:31:38.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.972+0000 7fe42e59c640 1 -- >> v1:172.21.15.112:6789/0 conn(0x7fe4301592c0 legacy=0x7fe4300c9070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:31:38.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.973+0000 7fe42dd9b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe4301596a0 0x7fe430152d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:31:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.973+0000 7fe42ed9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe4300c72b0 0x7fe430160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.973+0000 7fe42ed9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe4300c72b0 0x7fe430160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:56480/0 (socket says 172.21.15.112:56480) 2024-04-25T01:31:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.973+0000 7fe42ed9d640 1 -- 172.21.15.112:0/1670627705 learned_addr learned my addr 172.21.15.112:0/1670627705 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.973+0000 7fe42ed9d640 1 -- 172.21.15.112:0/1670627705 >> v2:172.21.15.112:3300/0 conn(0x7fe4301596a0 msgr2=0x7fe430152d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:31:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.973+0000 7fe42ed9d640 1 --2- 172.21.15.112:0/1670627705 >> v2:172.21.15.112:3300/0 conn(0x7fe4301596a0 0x7fe430152d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.973+0000 7fe42ed9d640 1 -- 172.21.15.112:0/1670627705 >> v1:172.21.15.112:6789/0 conn(0x7fe4301592c0 legacy=0x7fe4300c9070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:31:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.973+0000 7fe42ed9d640 1 -- 172.21.15.112:0/1670627705 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43006cc50 con 0x7fe4300c72b0 2024-04-25T01:31:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.974+0000 7fe42ed9d640 1 --2- 172.21.15.112:0/1670627705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4300c72b0 0x7fe430160380 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe4300c21d0 tx=0x7fe42802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94a02de5edcc5a7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.974+0000 7fe42d59a640 1 -- 172.21.15.112:0/1670627705 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe428036070 con 0x7fe4300c72b0 2024-04-25T01:31:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.974+0000 7fe42d59a640 1 -- 172.21.15.112:0/1670627705 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe428031600 con 0x7fe4300c72b0 2024-04-25T01:31:38.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.974+0000 7fe42d59a640 1 -- 172.21.15.112:0/1670627705 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4280319d0 con 0x7fe4300c72b0 2024-04-25T01:31:38.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.974+0000 7fe434f63640 1 -- 172.21.15.112:0/1670627705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4300c72b0 msgr2=0x7fe430160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:38.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.974+0000 7fe434f63640 1 --2- 172.21.15.112:0/1670627705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4300c72b0 0x7fe430160380 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe4300c21d0 tx=0x7fe42802a5f0 comp rx=0 tx=0).stop 2024-04-25T01:31:38.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.975+0000 7fe434f63640 1 -- 172.21.15.112:0/1670627705 shutdown_connections 2024-04-25T01:31:38.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.975+0000 7fe434f63640 1 --2- 172.21.15.112:0/1670627705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4300c72b0 0x7fe430160380 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:38.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.975+0000 7fe434f63640 1 --2- 172.21.15.112:0/1670627705 >> v2:172.21.15.112:3300/0 conn(0x7fe4301596a0 0x7fe430152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:38.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.975+0000 7fe434f63640 1 -- 172.21.15.112:0/1670627705 >> 172.21.15.112:0/1670627705 conn(0x7fe4300cb170 msgr2=0x7fe4300cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:38.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.975+0000 7fe434f63640 1 -- 172.21.15.112:0/1670627705 shutdown_connections 2024-04-25T01:31:38.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.975+0000 7fe434f63640 1 -- 172.21.15.112:0/1670627705 wait complete. 2024-04-25T01:31:38.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.976+0000 7fe434f63640 1 Processor -- start 2024-04-25T01:31:38.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.976+0000 7fe434f63640 1 -- start start 2024-04-25T01:31:38.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.977+0000 7fe434f63640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4300c72b0 0x7fe4300c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:38.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.977+0000 7fe434f63640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4301592c0 0x7fe4301ec1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:38.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.977+0000 7fe434f63640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4301596a0 0x7fe430155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:38.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.977+0000 7fe434f63640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe43006c510 con 0x7fe4300c72b0 2024-04-25T01:31:38.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.977+0000 7fe434f63640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe43006c390 con 0x7fe4301596a0 2024-04-25T01:31:38.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.977+0000 7fe434f63640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe43006db20 con 0x7fe4301592c0 2024-04-25T01:31:38.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.977+0000 7fe42e59c640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4300c72b0 msgr2=0x7fe4300c50c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:31:38.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.977+0000 7fe42e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4300c72b0 0x7fe4300c50c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:31:38.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.977+0000 7fe42ed9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4301596a0 0x7fe430155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:38.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.977+0000 7fe42ed9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4301596a0 0x7fe430155620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:56490/0 (socket says 172.21.15.112:56490) 2024-04-25T01:31:38.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.977+0000 7fe42ed9d640 1 -- 172.21.15.112:0/3802854230 learned_addr learned my addr 172.21.15.112:0/3802854230 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:38.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.977+0000 7fe42dd9b640 1 --2- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4301592c0 0x7fe4301ec1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:38.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.978+0000 7fe42ed9d640 1 -- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4301592c0 msgr2=0x7fe4301ec1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:38.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.978+0000 7fe42ed9d640 1 --2- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4301592c0 0x7fe4301ec1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:38.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.978+0000 7fe42ed9d640 1 -- 172.21.15.112:0/3802854230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4300c72b0 msgr2=0x7fe4300c50c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:31:38.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.978+0000 7fe42ed9d640 1 --2- 172.21.15.112:0/3802854230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4300c72b0 0x7fe4300c50c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:38.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.978+0000 7fe42ed9d640 1 -- 172.21.15.112:0/3802854230 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4301eb720 con 0x7fe4301596a0 2024-04-25T01:31:38.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.978+0000 7fe42ed9d640 1 --2- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4301596a0 0x7fe430155620 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe428034040 tx=0x7fe428004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:38.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.978+0000 7fe41effd640 1 -- 172.21.15.112:0/3802854230 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe428041020 con 0x7fe4301596a0 2024-04-25T01:31:38.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.978+0000 7fe41effd640 1 -- 172.21.15.112:0/3802854230 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe428007780 con 0x7fe4301596a0 2024-04-25T01:31:38.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.978+0000 7fe434f63640 1 -- 172.21.15.112:0/3802854230 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4301ef740 con 0x7fe4301596a0 2024-04-25T01:31:38.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.978+0000 7fe41effd640 1 -- 172.21.15.112:0/3802854230 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe428036040 con 0x7fe4301596a0 2024-04-25T01:31:38.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.978+0000 7fe434f63640 1 -- 172.21.15.112:0/3802854230 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4301eb9e0 con 0x7fe4301596a0 2024-04-25T01:31:38.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.979+0000 7fe434f63640 1 --2- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4301ebbb0 0x7fe4300c0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:38.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.979+0000 7fe434f63640 1 -- 172.21.15.112:0/3802854230 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe4300679a0 con 0x7fe4301ebbb0 2024-04-25T01:31:38.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.979+0000 7fe42e59c640 1 --2- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4301ebbb0 0x7fe4300c0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:38.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.979+0000 7fe42e59c640 1 -- 172.21.15.112:0/3802854230 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe4300682a0 con 0x7fe4301ebbb0 2024-04-25T01:31:38.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.980+0000 7fe42e59c640 1 --2- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4301ebbb0 0x7fe4300c0e70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe42000b450 tx=0x7fe42000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:38.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.980+0000 7fe41effd640 1 -- 172.21.15.112:0/3802854230 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420003d40 con 0x7fe4301ebbb0 2024-04-25T01:31:38.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.980+0000 7fe41effd640 1 -- 172.21.15.112:0/3802854230 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe428044020 con 0x7fe4301596a0 2024-04-25T01:31:38.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.980+0000 7fe41effd640 1 --2- 172.21.15.112:0/3802854230 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe41006bd30 0x7fe41006e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:38.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.980+0000 7fe41effd640 1 -- 172.21.15.112:0/3802854230 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fe4280abf70 con 0x7fe4301596a0 2024-04-25T01:31:38.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.980+0000 7fe42dd9b640 1 --2- 172.21.15.112:0/3802854230 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe41006bd30 0x7fe41006e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:38.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.980+0000 7fe41effd640 1 -- 172.21.15.112:0/3802854230 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe420015070 con 0x7fe4301ebbb0 2024-04-25T01:31:38.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.980+0000 7fe41effd640 1 -- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4301ebbb0 msgr2=0x7fe4300c0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:38.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.980+0000 7fe41effd640 1 --2- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4301ebbb0 0x7fe4300c0e70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe42000b450 tx=0x7fe42000b910 comp rx=0 tx=0).stop 2024-04-25T01:31:38.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.980+0000 7fe41effd640 1 -- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4301ebbb0 msgr2=0x7fe4300c0e70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:31:38.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.980+0000 7fe41effd640 1 --2- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4301ebbb0 0x7fe4300c0e70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:38.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.981+0000 7fe42dd9b640 1 --2- 172.21.15.112:0/3802854230 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe41006bd30 0x7fe41006e1f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe430154b80 tx=0x7fe424006f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:38.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.987+0000 7fe434f63640 1 --2- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4300c13b0 0x7fe430155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:38.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.987+0000 7fe434f63640 1 -- 172.21.15.112:0/3802854230 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe4300c8190 con 0x7fe4300c13b0 2024-04-25T01:31:38.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.987+0000 7fe42e59c640 1 --2- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4300c13b0 0x7fe430155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:38.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.987+0000 7fe42e59c640 1 -- 172.21.15.112:0/3802854230 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe43006bef0 con 0x7fe4300c13b0 2024-04-25T01:31:38.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.988+0000 7fe42e59c640 1 --2- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4300c13b0 0x7fe430155fb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe42000b450 tx=0x7fe420005990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:38.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.988+0000 7fe41effd640 1 -- 172.21.15.112:0/3802854230 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420023410 con 0x7fe4300c13b0 2024-04-25T01:31:38.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.988+0000 7fe41effd640 1 -- 172.21.15.112:0/3802854230 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5060 (secure 0 0 0) 0x7fe420010400 con 0x7fe4300c13b0 2024-04-25T01:31:38.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.988+0000 7fe41effd640 1 -- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4300c13b0 msgr2=0x7fe430155fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:38.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.988+0000 7fe41effd640 1 --2- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4300c13b0 0x7fe430155fb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe42000b450 tx=0x7fe420005990 comp rx=0 tx=0).stop 2024-04-25T01:31:38.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.988+0000 7fe41effd640 1 -- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4300c13b0 msgr2=0x7fe430155fb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:31:38.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.988+0000 7fe41effd640 1 --2- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4300c13b0 0x7fe430155fb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:38.967 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:31:38.967 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:31:38.967 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:31:38.967 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:31:38.967 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 42, 2024-04-25T01:31:38.967 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 134677, 2024-04-25T01:31:38.967 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:31:38.967 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:31:38.967 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:31:38.967 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:38.967 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:31:38.967 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:31:38.967 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:31:38.967 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:31:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:31:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:31:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:31:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:31:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:31:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:31:38.970 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:31:38.970 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:31:38.970 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:31:38.970 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:31:38.970 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:31:38.970 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:31:38.970 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:31:38.970 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:31:38.970 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:31:38.970 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:31:38.970 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:31:38.970 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:31:38.970 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:31:38.970 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:38.970 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:31:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:31:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:31:38.972 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:31:38.972 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:38.972 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:38.972 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:38.972 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:38.972 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:31:38.972 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:38.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:38.972 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:38.972 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:38.972 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:31:38.972 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:38.972 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:38.972 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:38.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:38.973 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:31:38.973 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:31:38.973 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:38.973 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:38.973 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:38.973 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:38.973 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:38.973 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:31:38.973 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:31:38.973 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:38.973 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:38.973 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:38.973 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:38.973 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:31:38.973 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:38.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:38.974 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:38.974 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:38.974 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:31:38.974 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:38.974 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:38.974 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:38.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:38.974 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:31:38.974 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:31:38.974 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:38.974 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:38.974 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:38.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:38.975 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:38.975 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:31:38.975 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:31:38.975 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:38.975 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:38.975 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:38.975 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:38.975 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:31:38.975 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:38.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:38.975 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:38.975 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:38.975 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:31:38.975 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:31:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:31:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:31:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:31:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:38.977 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:38.977 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:31:38.977 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:38.977 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:38.977 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:31:38.977 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:38.977 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:38.977 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:38.977 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:31:38.977 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:38.977 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:38.978 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:31:38.978 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:38.978 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:38.978 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:38.978 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:31:38.978 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:38.978 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:38.978 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:31:38.978 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:38.978 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:38.978 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:38.978 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:31:38.978 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:38.978 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:38.979 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:31:38.979 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:38.979 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:38.979 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:38.979 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:31:38.979 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:38.979 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:38.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:38.979 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:31:38.979 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:31:38.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.988+0000 7fe434f63640 1 -- 172.21.15.112:0/3802854230 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe41006bd30 msgr2=0x7fe41006e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:38.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.988+0000 7fe434f63640 1 --2- 172.21.15.112:0/3802854230 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe41006bd30 0x7fe41006e1f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe430154b80 tx=0x7fe424006f90 comp rx=0 tx=0).stop 2024-04-25T01:31:38.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.988+0000 7fe434f63640 1 -- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4301596a0 msgr2=0x7fe430155620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:38.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.988+0000 7fe42ed9d640 1 -- 172.21.15.112:0/3802854230 reap_dead start 2024-04-25T01:31:38.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.988+0000 7fe434f63640 1 --2- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4301596a0 0x7fe430155620 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe428034040 tx=0x7fe428004a40 comp rx=0 tx=0).stop 2024-04-25T01:31:38.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.988+0000 7fe434f63640 1 -- 172.21.15.112:0/3802854230 shutdown_connections 2024-04-25T01:31:38.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.988+0000 7fe434f63640 1 --2- 172.21.15.112:0/3802854230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4301596a0 0x7fe430155620 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:38.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.988+0000 7fe434f63640 1 -- 172.21.15.112:0/3802854230 >> 172.21.15.112:0/3802854230 conn(0x7fe4300cb170 msgr2=0x7fe4300ac290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:38.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.988+0000 7fe434f63640 1 -- 172.21.15.112:0/3802854230 shutdown_connections 2024-04-25T01:31:38.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:38.988+0000 7fe434f63640 1 -- 172.21.15.112:0/3802854230 wait complete. 2024-04-25T01:31:38.981 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:31:38.981 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-04-25T01:31:38.982 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:31:39.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.111+0000 7f528519e640 1 Processor -- start 2024-04-25T01:31:39.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.111+0000 7f528519e640 1 -- start start 2024-04-25T01:31:39.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.111+0000 7f528519e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5280150d80 0x7f5280154470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:39.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.112+0000 7f528519e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5280071000 0x7f52800c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:39.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.112+0000 7f528519e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5280071850 con 0x7f52800c73c0 2024-04-25T01:31:39.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.112+0000 7f528519e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5280071570 con 0x7f5280071000 2024-04-25T01:31:39.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.112+0000 7f528519e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5280153990 con 0x7f5280150d80 2024-04-25T01:31:39.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.112+0000 7f527e59c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5280071000 0x7f52800c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:39.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.112+0000 7f527e59c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5280071000 0x7f52800c6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:56518/0 (socket says 172.21.15.112:56518) 2024-04-25T01:31:39.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.112+0000 7f527e59c640 1 -- 172.21.15.112:0/2215817840 learned_addr learned my addr 172.21.15.112:0/2215817840 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:39.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.112+0000 7f527ed9d640 1 --2- 172.21.15.112:0/2215817840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5280150d80 0x7f5280154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:39.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.112+0000 7f527e59c640 1 -- 172.21.15.112:0/2215817840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5280150d80 msgr2=0x7f5280154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:31:39.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.112+0000 7f527e59c640 1 --2- 172.21.15.112:0/2215817840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5280150d80 0x7f5280154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:39.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.112+0000 7f527e59c640 1 -- 172.21.15.112:0/2215817840 >> v1:172.21.15.187:6789/0 conn(0x7f52800c73c0 legacy=0x7f52800c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:39.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.112+0000 7f527e59c640 1 -- 172.21.15.112:0/2215817840 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5280153c70 con 0x7f5280071000 2024-04-25T01:31:39.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.113+0000 7f527e59c640 1 --2- 172.21.15.112:0/2215817840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5280071000 0x7f52800c6e80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5274009870 tx=0x7f527402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=363b68cf93cf3568 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.113+0000 7f527dd9b640 1 -- 172.21.15.112:0/2215817840 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274037070 con 0x7f5280071000 2024-04-25T01:31:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.113+0000 7f527dd9b640 1 -- 172.21.15.112:0/2215817840 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5274032440 con 0x7f5280071000 2024-04-25T01:31:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.113+0000 7f527dd9b640 1 -- 172.21.15.112:0/2215817840 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274032810 con 0x7f5280071000 2024-04-25T01:31:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.114+0000 7f528519e640 1 -- 172.21.15.112:0/2215817840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5280071000 msgr2=0x7f52800c6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.114+0000 7f528519e640 1 --2- 172.21.15.112:0/2215817840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5280071000 0x7f52800c6e80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5274009870 tx=0x7f527402a1c0 comp rx=0 tx=0).stop 2024-04-25T01:31:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.114+0000 7f528519e640 1 -- 172.21.15.112:0/2215817840 shutdown_connections 2024-04-25T01:31:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.114+0000 7f528519e640 1 --2- 172.21.15.112:0/2215817840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5280071000 0x7f52800c6e80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.114+0000 7f528519e640 1 --2- 172.21.15.112:0/2215817840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5280150d80 0x7f5280154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.114+0000 7f528519e640 1 -- 172.21.15.112:0/2215817840 >> 172.21.15.112:0/2215817840 conn(0x7f5280074a90 msgr2=0x7f52800c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.114+0000 7f528519e640 1 -- 172.21.15.112:0/2215817840 shutdown_connections 2024-04-25T01:31:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.114+0000 7f528519e640 1 -- 172.21.15.112:0/2215817840 wait complete. 2024-04-25T01:31:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.115+0000 7f528519e640 1 Processor -- start 2024-04-25T01:31:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.115+0000 7f528519e640 1 -- start start 2024-04-25T01:31:39.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.116+0000 7f528519e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5280071000 0x7f528014c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:39.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.116+0000 7f528519e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52800c73c0 0x7f5280157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:39.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.116+0000 7f528519e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5280150d80 0x7f5280157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:39.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.116+0000 7f528519e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52800c8d30 con 0x7f5280071000 2024-04-25T01:31:39.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.116+0000 7f528519e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f52800c8bb0 con 0x7f52800c73c0 2024-04-25T01:31:39.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.116+0000 7f528519e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5280151160 con 0x7f5280150d80 2024-04-25T01:31:39.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.116+0000 7f527ed9d640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5280071000 msgr2=0x7f528014c590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:31:39.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.116+0000 7f527ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5280071000 0x7f528014c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:31:39.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.116+0000 7f527e59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52800c73c0 0x7f5280157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:39.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.116+0000 7f527f59e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5280150d80 0x7f5280157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:39.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.116+0000 7f527e59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52800c73c0 0x7f5280157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:56528/0 (socket says 172.21.15.112:56528) 2024-04-25T01:31:39.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.116+0000 7f527e59c640 1 -- 172.21.15.112:0/2649761676 learned_addr learned my addr 172.21.15.112:0/2649761676 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:39.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.117+0000 7f527e59c640 1 -- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5280150d80 msgr2=0x7f5280157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:39.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.117+0000 7f527e59c640 1 --2- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5280150d80 0x7f5280157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:39.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.117+0000 7f527e59c640 1 -- 172.21.15.112:0/2649761676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5280071000 msgr2=0x7f528014c590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:31:39.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.117+0000 7f527e59c640 1 --2- 172.21.15.112:0/2649761676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5280071000 0x7f528014c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:39.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.117+0000 7f527e59c640 1 -- 172.21.15.112:0/2649761676 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528014d0e0 con 0x7f52800c73c0 2024-04-25T01:31:39.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.117+0000 7f527f59e640 1 --2- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5280150d80 0x7f5280157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:31:39.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.117+0000 7f527e59c640 1 --2- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52800c73c0 0x7f5280157350 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f527402a200 tx=0x7f52740046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:39.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.117+0000 7f52677fe640 1 -- 172.21.15.112:0/2649761676 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274037040 con 0x7f52800c73c0 2024-04-25T01:31:39.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.117+0000 7f52677fe640 1 -- 172.21.15.112:0/2649761676 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5274042070 con 0x7f52800c73c0 2024-04-25T01:31:39.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.117+0000 7f528519e640 1 -- 172.21.15.112:0/2649761676 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5280156330 con 0x7f52800c73c0 2024-04-25T01:31:39.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.117+0000 7f52677fe640 1 -- 172.21.15.112:0/2649761676 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274030880 con 0x7f52800c73c0 2024-04-25T01:31:39.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.117+0000 7f528519e640 1 -- 172.21.15.112:0/2649761676 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52801568f0 con 0x7f52800c73c0 2024-04-25T01:31:39.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.118+0000 7f528519e640 1 --2- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52800689b0 0x7f52801e9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:39.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.118+0000 7f528519e640 1 -- 172.21.15.112:0/2649761676 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5280156f40 con 0x7f52800689b0 2024-04-25T01:31:39.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.118+0000 7f527ed9d640 1 --2- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52800689b0 0x7f52801e9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:39.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.119+0000 7f52677fe640 1 -- 172.21.15.112:0/2649761676 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f527402ad10 con 0x7f52800c73c0 2024-04-25T01:31:39.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.119+0000 7f527ed9d640 1 -- 172.21.15.112:0/2649761676 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5280075980 con 0x7f52800689b0 2024-04-25T01:31:39.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.119+0000 7f527ed9d640 1 --2- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52800689b0 0x7f52801e9940 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5270009fd0 tx=0x7f527000ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:39.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.119+0000 7f52677fe640 1 --2- 172.21.15.112:0/2649761676 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f526c06bde0 0x7f526c06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:39.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.119+0000 7f52677fe640 1 -- 172.21.15.112:0/2649761676 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f52740ab630 con 0x7f52800c73c0 2024-04-25T01:31:39.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.119+0000 7f527f59e640 1 --2- 172.21.15.112:0/2649761676 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f526c06bde0 0x7f526c06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:39.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.119+0000 7f52677fe640 1 -- 172.21.15.112:0/2649761676 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270017070 con 0x7f52800689b0 2024-04-25T01:31:39.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.119+0000 7f52677fe640 1 -- 172.21.15.112:0/2649761676 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5270004020 con 0x7f52800689b0 2024-04-25T01:31:39.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.119+0000 7f52677fe640 1 -- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52800689b0 msgr2=0x7f52801e9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:39.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.119+0000 7f52677fe640 1 --2- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52800689b0 0x7f52801e9940 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5270009fd0 tx=0x7f527000ec70 comp rx=0 tx=0).stop 2024-04-25T01:31:39.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.119+0000 7f52677fe640 1 -- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52800689b0 msgr2=0x7f52801e9940 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:31:39.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.119+0000 7f52677fe640 1 --2- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52800689b0 0x7f52801e9940 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:39.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.120+0000 7f527f59e640 1 --2- 172.21.15.112:0/2649761676 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f526c06bde0 0x7f526c06e2a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f52780012e0 tx=0x7f5278009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:39.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.126+0000 7f528519e640 1 --2- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f528006e480 0x7f5280145fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:39.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.126+0000 7f528519e640 1 -- 172.21.15.112:0/2649761676 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5280070df0 con 0x7f528006e480 2024-04-25T01:31:39.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.126+0000 7f527ed9d640 1 --2- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f528006e480 0x7f5280145fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:39.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.127+0000 7f527ed9d640 1 -- 172.21.15.112:0/2649761676 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f52800c8280 con 0x7f528006e480 2024-04-25T01:31:39.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.127+0000 7f527ed9d640 1 --2- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f528006e480 0x7f5280145fd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5270009fd0 tx=0x7f5270004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:39.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.127+0000 7f52677fe640 1 -- 172.21.15.112:0/2649761676 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270017040 con 0x7f528006e480 2024-04-25T01:31:39.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.127+0000 7f52677fe640 1 -- 172.21.15.112:0/2649761676 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4887 (secure 0 0 0) 0x7f5270003bc0 con 0x7f528006e480 2024-04-25T01:31:39.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.127+0000 7f52677fe640 1 -- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f528006e480 msgr2=0x7f5280145fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:39.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.127+0000 7f52677fe640 1 --2- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f528006e480 0x7f5280145fd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5270009fd0 tx=0x7f5270004730 comp rx=0 tx=0).stop 2024-04-25T01:31:39.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.127+0000 7f52677fe640 1 -- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f528006e480 msgr2=0x7f5280145fd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:31:39.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.127+0000 7f52677fe640 1 --2- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f528006e480 0x7f5280145fd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:39.105 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:31:39.106 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:31:39.106 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:31:39.106 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:31:39.106 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 42, 2024-04-25T01:31:39.106 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 87866, 2024-04-25T01:31:39.106 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:31:39.106 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:31:39.106 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:31:39.106 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:39.106 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:31:39.106 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:31:39.106 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:31:39.106 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:31:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:31:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:31:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:39.108 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:39.108 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:39.108 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:39.108 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:39.108 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:39.108 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:39.108 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:39.108 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:39.108 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:39.108 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:39.108 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:39.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:39.108 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:31:39.108 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:31:39.108 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:31:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:31:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:31:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:31:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:31:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:31:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:31:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:31:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:31:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:31:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:31:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:31:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:31:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:31:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:31:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:31:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:31:39.110 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:31:39.110 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:31:39.110 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:31:39.110 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:31:39.110 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:31:39.110 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:31:39.110 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:31:39.110 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:31:39.110 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:31:39.110 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:39.110 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:31:39.110 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:39.110 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:31:39.110 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:39.111 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:31:39.111 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:31:39.111 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:39.111 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:39.111 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:39.111 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:39.111 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:31:39.111 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:39.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:39.111 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:39.111 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:39.111 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:31:39.111 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:39.111 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:39.111 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:31:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:31:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:31:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:31:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:31:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:39.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:39.113 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:39.113 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:39.113 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:31:39.113 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:39.113 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:39.113 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:39.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:39.113 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:31:39.113 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:31:39.113 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:39.113 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:39.113 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:39.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:39.114 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:39.114 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:31:39.114 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:31:39.114 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:31:39.114 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:31:39.114 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:39.114 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:31:39.114 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:31:39.114 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:39.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:39.114 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:39.114 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:31:39.114 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:31:39.114 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:31:39.115 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:39.115 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:39.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:39.115 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:31:39.115 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:31:39.115 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:31:39.115 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:31:39.115 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:31:39.115 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:39.115 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:39.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:39.115 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:31:39.115 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:31:39.115 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:31:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:31:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:31:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:31:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:39.117 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:31:39.117 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:39.117 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:31:39.117 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:31:39.117 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:31:39.117 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:31:39.117 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:31:39.117 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:31:39.117 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:31:39.117 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:31:39.117 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:31:39.117 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:31:39.117 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:31:39.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.128+0000 7f528519e640 1 -- 172.21.15.112:0/2649761676 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f526c06bde0 msgr2=0x7f526c06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:39.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.128+0000 7f528519e640 1 --2- 172.21.15.112:0/2649761676 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f526c06bde0 0x7f526c06e2a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f52780012e0 tx=0x7f5278009040 comp rx=0 tx=0).stop 2024-04-25T01:31:39.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.128+0000 7f528519e640 1 -- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52800c73c0 msgr2=0x7f5280157350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:39.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.128+0000 7f528519e640 1 --2- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52800c73c0 0x7f5280157350 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f527402a200 tx=0x7f52740046c0 comp rx=0 tx=0).stop 2024-04-25T01:31:39.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.128+0000 7f527f59e640 1 -- 172.21.15.112:0/2649761676 reap_dead start 2024-04-25T01:31:39.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.128+0000 7f528519e640 1 -- 172.21.15.112:0/2649761676 shutdown_connections 2024-04-25T01:31:39.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.128+0000 7f528519e640 1 --2- 172.21.15.112:0/2649761676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52800c73c0 0x7f5280157350 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:39.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.128+0000 7f528519e640 1 -- 172.21.15.112:0/2649761676 >> 172.21.15.112:0/2649761676 conn(0x7f5280074a90 msgr2=0x7f52800c77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:39.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.128+0000 7f528519e640 1 -- 172.21.15.112:0/2649761676 shutdown_connections 2024-04-25T01:31:39.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:39.128+0000 7f528519e640 1 -- 172.21.15.112:0/2649761676 wait complete. 2024-04-25T01:31:39.120 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:31:39.120 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T01:31:55.788 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:31:55.794 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:31:59.121 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-04-25T01:31:59.121 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-04-25T01:31:59.121 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T01:31:59.121 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T01:31:59.124 INFO:tasks.ceph.mon.a:Started 2024-04-25T01:31:59.124 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:31:59.124 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:31:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.262+0000 7f89c5abc640 1 Processor -- start 2024-04-25T01:31:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.262+0000 7f89c5abc640 1 -- start start 2024-04-25T01:31:59.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.263+0000 7f89c5abc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f89c00c9760 0x7f89c00caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:59.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.263+0000 7f89c5abc640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89c0153330 con 0x7f89c0152f50 2024-04-25T01:31:59.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.263+0000 7f89c5abc640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89c0159dc0 con 0x7f89c00c5cf0 2024-04-25T01:31:59.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.263+0000 7f89c5abc640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f89c00c6180 con 0x7f89c00c9760 2024-04-25T01:31:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.263+0000 7f89beffd640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f89c00c5cf0 0x7f89c00c9050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:36848/0 (socket says 172.21.15.112:36848) 2024-04-25T01:31:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.263+0000 7f89beffd640 1 -- 172.21.15.112:0/3333174763 learned_addr learned my addr 172.21.15.112:0/3333174763 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:31:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.263+0000 7f89bffff640 1 --2- 172.21.15.112:0/3333174763 >> v2:172.21.15.112:3300/0 conn(0x7f89c00c9760 0x7f89c00caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.263+0000 7f89bffff640 1 -- 172.21.15.112:0/3333174763 >> v2:172.21.15.112:3300/0 conn(0x7f89c00c9760 msgr2=0x7f89c00caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:31:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.263+0000 7f89bffff640 1 -- 172.21.15.112:0/3333174763 >> v2:172.21.15.112:3300/0 conn(0x7f89c00c9760 msgr2=0x7f89c00caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:31:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.263+0000 7f89bffff640 1 --2- 172.21.15.112:0/3333174763 >> v2:172.21.15.112:3300/0 conn(0x7f89c00c9760 0x7f89c00caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:31:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.263+0000 7f89bffff640 1 --2- 172.21.15.112:0/3333174763 >> v2:172.21.15.112:3300/0 conn(0x7f89c00c9760 0x7f89c00caa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:31:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.263+0000 7f89be7fc640 1 -- 172.21.15.112:0/3333174763 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 171154370 0 0) 0x7f89c0159dc0 con 0x7f89c00c5cf0 2024-04-25T01:31:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.263+0000 7f89be7fc640 1 -- 172.21.15.112:0/3333174763 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89ac003520 con 0x7f89c00c5cf0 2024-04-25T01:31:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.263+0000 7f89be7fc640 1 -- 172.21.15.112:0/3333174763 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2425457854 0 0) 0x7f89ac003520 con 0x7f89c00c5cf0 2024-04-25T01:31:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.263+0000 7f89be7fc640 1 -- 172.21.15.112:0/3333174763 >> v2:172.21.15.112:3300/0 conn(0x7f89c00c9760 msgr2=0x7f89c00caa70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:31:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.263+0000 7f89be7fc640 1 --2- 172.21.15.112:0/3333174763 >> v2:172.21.15.112:3300/0 conn(0x7f89c00c9760 0x7f89c00caa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.263+0000 7f89be7fc640 1 -- 172.21.15.112:0/3333174763 >> v1:172.21.15.112:6789/0 conn(0x7f89c0152f50 legacy=0x7f89c00c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.263+0000 7f89be7fc640 1 -- 172.21.15.112:0/3333174763 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c0159000 con 0x7f89c00c5cf0 2024-04-25T01:31:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.263+0000 7f89be7fc640 1 -- 172.21.15.112:0/3333174763 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f89b0001670 con 0x7f89c00c5cf0 2024-04-25T01:31:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.263+0000 7f89be7fc640 1 -- 172.21.15.112:0/3333174763 >> v1:172.21.15.187:6789/0 conn(0x7f89c00c5cf0 legacy=0x7f89c00c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.264+0000 7f89be7fc640 1 --2- 172.21.15.112:0/3333174763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89ac0039b0 0x7f89ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.264+0000 7f89be7fc640 1 --2- 172.21.15.112:0/3333174763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89ac004800 0x7f89ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.264+0000 7f89be7fc640 1 --2- 172.21.15.112:0/3333174763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89ac0052e0 0x7f89ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.264+0000 7f89be7fc640 1 -- 172.21.15.112:0/3333174763 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89ac00c180 con 0x7f89ac0039b0 2024-04-25T01:31:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.264+0000 7f89be7fc640 1 -- 172.21.15.112:0/3333174763 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f89ac00c330 con 0x7f89ac0052e0 2024-04-25T01:31:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.264+0000 7f89be7fc640 1 -- 172.21.15.112:0/3333174763 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f89ac00c530 con 0x7f89ac004800 2024-04-25T01:31:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.264+0000 7f89bf7fe640 1 --2- 172.21.15.112:0/3333174763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89ac0039b0 0x7f89ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.264+0000 7f89bffff640 1 --2- 172.21.15.112:0/3333174763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89ac0052e0 0x7f89ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.264+0000 7f89beffd640 1 --2- 172.21.15.112:0/3333174763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89ac004800 0x7f89ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.264+0000 7f89bffff640 1 -- 172.21.15.112:0/3333174763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89ac004800 msgr2=0x7f89ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:31:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.264+0000 7f89bffff640 1 --2- 172.21.15.112:0/3333174763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89ac004800 0x7f89ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.264+0000 7f89bffff640 1 -- 172.21.15.112:0/3333174763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89ac0039b0 msgr2=0x7f89ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:31:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.264+0000 7f89bffff640 1 --2- 172.21.15.112:0/3333174763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89ac0039b0 0x7f89ac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.264+0000 7f89bffff640 1 -- 172.21.15.112:0/3333174763 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ac003520 con 0x7f89ac0052e0 2024-04-25T01:31:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.265+0000 7f89bffff640 1 -- 172.21.15.112:0/3333174763 reap_dead start 2024-04-25T01:31:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.265+0000 7f89bffff640 1 --2- 172.21.15.112:0/3333174763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89ac0052e0 0x7f89ac00b9c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009610 tx=0x7f89ac005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70229e84a1215b11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.265+0000 7f89be7fc640 1 -- 172.21.15.112:0/3333174763 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8033070 con 0x7f89ac0052e0 2024-04-25T01:31:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.265+0000 7f89be7fc640 1 -- 172.21.15.112:0/3333174763 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b802f670 con 0x7f89ac0052e0 2024-04-25T01:31:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.265+0000 7f89be7fc640 1 -- 172.21.15.112:0/3333174763 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8009bd0 con 0x7f89ac0052e0 2024-04-25T01:31:59.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.266+0000 7f89c5abc640 1 -- 172.21.15.112:0/3333174763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89ac0052e0 msgr2=0x7f89ac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:59.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.266+0000 7f89c5abc640 1 --2- 172.21.15.112:0/3333174763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89ac0052e0 0x7f89ac00b9c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009610 tx=0x7f89ac005830 comp rx=0 tx=0).stop 2024-04-25T01:31:59.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.266+0000 7f89c5abc640 1 -- 172.21.15.112:0/3333174763 shutdown_connections 2024-04-25T01:31:59.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.266+0000 7f89c5abc640 1 --2- 172.21.15.112:0/3333174763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89ac0052e0 0x7f89ac00b9c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:59.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.266+0000 7f89c5abc640 1 -- 172.21.15.112:0/3333174763 >> 172.21.15.112:0/3333174763 conn(0x7f89c00ace40 msgr2=0x7f89c00ad250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:59.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.266+0000 7f89c5abc640 1 -- 172.21.15.112:0/3333174763 shutdown_connections 2024-04-25T01:31:59.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.266+0000 7f89c5abc640 1 -- 172.21.15.112:0/3333174763 wait complete. 2024-04-25T01:31:59.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.267+0000 7f89c5abc640 1 Processor -- start 2024-04-25T01:31:59.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.267+0000 7f89c5abc640 1 -- start start 2024-04-25T01:31:59.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89c5abc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89ac0052e0 0x7f89c00c0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:59.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89c5abc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89c014cb50 0x7f89c00abd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:59.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89c5abc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c00ac280 0x7f89c01eb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:59.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89c5abc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01598a0 con 0x7f89c00ac280 2024-04-25T01:31:59.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89c5abc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01595c0 con 0x7f89ac0052e0 2024-04-25T01:31:59.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89c5abc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f89c00caa70 con 0x7f89c014cb50 2024-04-25T01:31:59.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89bffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c00ac280 0x7f89c01eb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:59.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89bf7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89ac0052e0 0x7f89c00c0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:59.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89beffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89c014cb50 0x7f89c00abd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:59.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89bffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c00ac280 0x7f89c01eb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54654/0 (socket says 172.21.15.112:54654) 2024-04-25T01:31:59.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89bf7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89ac0052e0 0x7f89c00c0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43814/0 (socket says 172.21.15.112:43814) 2024-04-25T01:31:59.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89beffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89c014cb50 0x7f89c00abd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:33384/0 (socket says 172.21.15.112:33384) 2024-04-25T01:31:59.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89bffff640 1 -- 172.21.15.112:0/371332778 learned_addr learned my addr 172.21.15.112:0/371332778 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:31:59.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89bffff640 1 -- 172.21.15.112:0/371332778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c00ac280 msgr2=0x7f89c01eb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-25T01:31:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89bffff640 1 -- 172.21.15.112:0/371332778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c00ac280 msgr2=0x7f89c01eb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:31:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89bffff640 1 --2- 172.21.15.112:0/371332778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c00ac280 0x7f89c01eb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:31:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89bffff640 1 --2- 172.21.15.112:0/371332778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c00ac280 0x7f89c01eb800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:31:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89beffd640 1 -- 172.21.15.112:0/371332778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89ac0052e0 msgr2=0x7f89c00c0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89beffd640 1 --2- 172.21.15.112:0/371332778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89ac0052e0 0x7f89c00c0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89beffd640 1 -- 172.21.15.112:0/371332778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c00ac280 msgr2=0x7f89c01eb800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:31:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89beffd640 1 --2- 172.21.15.112:0/371332778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c00ac280 0x7f89c01eb800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89beffd640 1 -- 172.21.15.112:0/371332778 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c014c7c0 con 0x7f89c014cb50 2024-04-25T01:31:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89bf7fe640 1 --2- 172.21.15.112:0/371332778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89ac0052e0 0x7f89c00c0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:31:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.268+0000 7f89beffd640 1 --2- 172.21.15.112:0/371332778 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89c014cb50 0x7f89c00abd40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89b0007290 tx=0x7f89b0007690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.269+0000 7f89b7fff640 1 -- 172.21.15.112:0/371332778 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b000f8e0 con 0x7f89c014cb50 2024-04-25T01:31:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.269+0000 7f89b7fff640 1 -- 172.21.15.112:0/371332778 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b0018070 con 0x7f89c014cb50 2024-04-25T01:31:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.269+0000 7f89b7fff640 1 -- 172.21.15.112:0/371332778 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b000fce0 con 0x7f89c014cb50 2024-04-25T01:31:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.269+0000 7f89c5abc640 1 -- 172.21.15.112:0/371332778 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c01ef5f0 con 0x7f89c014cb50 2024-04-25T01:31:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.269+0000 7f89c5abc640 1 -- 172.21.15.112:0/371332778 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c00c1cc0 con 0x7f89c014cb50 2024-04-25T01:31:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.270+0000 7f89c5abc640 1 -- 172.21.15.112:0/371332778 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89c01ef3e0 con 0x7f89c014cb50 2024-04-25T01:31:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.270+0000 7f89b7fff640 1 -- 172.21.15.112:0/371332778 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f89b001d460 con 0x7f89c014cb50 2024-04-25T01:31:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.271+0000 7f89b7fff640 1 --2- 172.21.15.112:0/371332778 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f89a006be60 0x7f89a006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.271+0000 7f89b7fff640 1 -- 172.21.15.112:0/371332778 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f89b008b050 con 0x7f89c014cb50 2024-04-25T01:31:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.271+0000 7f89bf7fe640 1 --2- 172.21.15.112:0/371332778 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f89a006be60 0x7f89a006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.271+0000 7f89bf7fe640 1 --2- 172.21.15.112:0/371332778 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f89a006be60 0x7f89a006e320 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f89c00c7310 tx=0x7f89a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:59.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.276+0000 7f89b7fff640 1 -- 172.21.15.112:0/371332778 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f89b00590a0 con 0x7f89c014cb50 2024-04-25T01:31:59.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.581+0000 7f89c5abc640 1 -- 172.21.15.112:0/371332778 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f89c00c5ad0 con 0x7f89c014cb50 2024-04-25T01:31:59.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.581+0000 7f89b7fff640 1 -- 172.21.15.112:0/371332778 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1563 (secure 0 0 0) 0x7f89b005d3a0 con 0x7f89c014cb50 2024-04-25T01:31:59.559 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":42,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:31:59.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.584+0000 7f89c5abc640 1 -- 172.21.15.112:0/371332778 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f89a006be60 msgr2=0x7f89a006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:59.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.584+0000 7f89c5abc640 1 --2- 172.21.15.112:0/371332778 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f89a006be60 0x7f89a006e320 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f89c00c7310 tx=0x7f89a8009040 comp rx=0 tx=0).stop 2024-04-25T01:31:59.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.584+0000 7f89c5abc640 1 -- 172.21.15.112:0/371332778 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89c014cb50 msgr2=0x7f89c00abd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:59.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.584+0000 7f89c5abc640 1 --2- 172.21.15.112:0/371332778 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89c014cb50 0x7f89c00abd40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89b0007290 tx=0x7f89b0007690 comp rx=0 tx=0).stop 2024-04-25T01:31:59.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.584+0000 7f89c5abc640 1 -- 172.21.15.112:0/371332778 shutdown_connections 2024-04-25T01:31:59.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.584+0000 7f89c5abc640 1 --2- 172.21.15.112:0/371332778 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f89a006be60 0x7f89a006e320 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:59.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.584+0000 7f89c5abc640 1 --2- 172.21.15.112:0/371332778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c00ac280 0x7f89c01eb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:59.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.584+0000 7f89c5abc640 1 --2- 172.21.15.112:0/371332778 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89c014cb50 0x7f89c00abd40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:59.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.584+0000 7f89c5abc640 1 --2- 172.21.15.112:0/371332778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89ac0052e0 0x7f89c00c0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:59.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.584+0000 7f89c5abc640 1 -- 172.21.15.112:0/371332778 >> 172.21.15.112:0/371332778 conn(0x7f89c00ace40 msgr2=0x7f89c006c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:59.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.584+0000 7f89c5abc640 1 -- 172.21.15.112:0/371332778 shutdown_connections 2024-04-25T01:31:59.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:31:59.584+0000 7f89c5abc640 1 -- 172.21.15.112:0/371332778 wait complete. 2024-04-25T01:32:02.573 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:32:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.704+0000 7fe65384d640 1 Processor -- start 2024-04-25T01:32:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.704+0000 7fe65384d640 1 -- start start 2024-04-25T01:32:02.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.705+0000 7fe65384d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe64c0ab440 0x7fe64c151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:02.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.705+0000 7fe65384d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe64c0721f0 0x7fe64c0c2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:02.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.705+0000 7fe65384d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe64c0ab820 con 0x7fe64c0c2750 2024-04-25T01:32:02.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.705+0000 7fe65384d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fe64c1552a0 con 0x7fe64c0ab440 2024-04-25T01:32:02.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.705+0000 7fe65384d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe64c152290 con 0x7fe64c0721f0 2024-04-25T01:32:02.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.705+0000 7fe6515e9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe64c0ab440 0x7fe64c151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:02.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.705+0000 7fe651dea640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe64c0c2750 0x7fe64c155780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37998/0 (socket says 172.21.15.112:37998) 2024-04-25T01:32:02.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.705+0000 7fe6515e9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe64c0ab440 0x7fe64c151cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54670/0 (socket says 172.21.15.112:54670) 2024-04-25T01:32:02.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.705+0000 7fe651dea640 1 -- 172.21.15.112:0/204529870 learned_addr learned my addr 172.21.15.112:0/204529870 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:32:02.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.705+0000 7fe650de8640 1 --2- 172.21.15.112:0/204529870 >> v2:172.21.15.187:3300/0 conn(0x7fe64c0721f0 0x7fe64c0c2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:02.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.705+0000 7fe6515e9640 1 -- 172.21.15.112:0/204529870 >> v2:172.21.15.112:3300/0 conn(0x7fe64c0ab440 msgr2=0x7fe64c151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:32:02.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.705+0000 7fe6515e9640 1 -- 172.21.15.112:0/204529870 >> v2:172.21.15.112:3300/0 conn(0x7fe64c0ab440 msgr2=0x7fe64c151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:32:02.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.705+0000 7fe6515e9640 1 --2- 172.21.15.112:0/204529870 >> v2:172.21.15.112:3300/0 conn(0x7fe64c0ab440 0x7fe64c151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:02.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.705+0000 7fe6515e9640 1 --2- 172.21.15.112:0/204529870 >> v2:172.21.15.112:3300/0 conn(0x7fe64c0ab440 0x7fe64c151cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:32:02.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.705+0000 7fe650de8640 1 -- 172.21.15.112:0/204529870 >> v2:172.21.15.112:3300/0 conn(0x7fe64c0ab440 msgr2=0x7fe64c151cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:32:02.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.705+0000 7fe650de8640 1 --2- 172.21.15.112:0/204529870 >> v2:172.21.15.112:3300/0 conn(0x7fe64c0ab440 0x7fe64c151cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:02.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.705+0000 7fe650de8640 1 -- 172.21.15.112:0/204529870 >> v1:172.21.15.112:6789/0 conn(0x7fe64c0c2750 legacy=0x7fe64c155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:02.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.705+0000 7fe650de8640 1 -- 172.21.15.112:0/204529870 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64c152570 con 0x7fe64c0721f0 2024-04-25T01:32:02.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.706+0000 7fe650de8640 1 --2- 172.21.15.112:0/204529870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe64c0721f0 0x7fe64c0c2210 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe64c0c7f50 tx=0x7fe63c02c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56ce0f3009575383 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:02.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.706+0000 7fe643fff640 1 -- 172.21.15.112:0/204529870 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c038070 con 0x7fe64c0721f0 2024-04-25T01:32:02.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.706+0000 7fe643fff640 1 -- 172.21.15.112:0/204529870 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe63c031710 con 0x7fe64c0721f0 2024-04-25T01:32:02.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.706+0000 7fe643fff640 1 -- 172.21.15.112:0/204529870 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c031ae0 con 0x7fe64c0721f0 2024-04-25T01:32:02.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.707+0000 7fe65384d640 1 -- 172.21.15.112:0/204529870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe64c0721f0 msgr2=0x7fe64c0c2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:02.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.707+0000 7fe65384d640 1 --2- 172.21.15.112:0/204529870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe64c0721f0 0x7fe64c0c2210 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe64c0c7f50 tx=0x7fe63c02c530 comp rx=0 tx=0).stop 2024-04-25T01:32:02.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.707+0000 7fe65384d640 1 -- 172.21.15.112:0/204529870 shutdown_connections 2024-04-25T01:32:02.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.707+0000 7fe65384d640 1 --2- 172.21.15.112:0/204529870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe64c0721f0 0x7fe64c0c2210 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:02.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.707+0000 7fe65384d640 1 --2- 172.21.15.112:0/204529870 >> v2:172.21.15.112:3300/0 conn(0x7fe64c0ab440 0x7fe64c151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:02.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.707+0000 7fe65384d640 1 -- 172.21.15.112:0/204529870 >> 172.21.15.112:0/204529870 conn(0x7fe64c0c6f80 msgr2=0x7fe64c0c7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:02.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.707+0000 7fe65384d640 1 -- 172.21.15.112:0/204529870 shutdown_connections 2024-04-25T01:32:02.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.707+0000 7fe65384d640 1 -- 172.21.15.112:0/204529870 wait complete. 2024-04-25T01:32:02.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.708+0000 7fe65384d640 1 Processor -- start 2024-04-25T01:32:02.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.708+0000 7fe65384d640 1 -- start start 2024-04-25T01:32:02.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.709+0000 7fe65384d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64c0721f0 0x7fe64c1ec650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:02.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.709+0000 7fe65384d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe64c0ab440 0x7fe64c1ebbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:02.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.709+0000 7fe65384d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe64c0c2750 0x7fe64c0c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:02.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.709+0000 7fe65384d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c06c8a0 con 0x7fe64c0ab440 2024-04-25T01:32:02.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.709+0000 7fe65384d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c152b30 con 0x7fe64c0c2750 2024-04-25T01:32:02.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.709+0000 7fe65384d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe64c072680 con 0x7fe64c0721f0 2024-04-25T01:32:02.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.709+0000 7fe6515e9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64c0721f0 0x7fe64c1ec650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:02.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.709+0000 7fe650de8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe64c0ab440 0x7fe64c1ebbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:02.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.709+0000 7fe650de8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe64c0ab440 0x7fe64c1ebbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54678/0 (socket says 172.21.15.112:54678) 2024-04-25T01:32:02.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.709+0000 7fe650de8640 1 -- 172.21.15.112:0/3884041190 learned_addr learned my addr 172.21.15.112:0/3884041190 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:02.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.709+0000 7fe6515e9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64c0721f0 0x7fe64c1ec650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:33436/0 (socket says 172.21.15.112:33436) 2024-04-25T01:32:02.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.709+0000 7fe651dea640 1 --2- 172.21.15.112:0/3884041190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe64c0c2750 0x7fe64c0c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:02.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.709+0000 7fe650de8640 1 -- 172.21.15.112:0/3884041190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe64c0ab440 msgr2=0x7fe64c1ebbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T01:32:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.709+0000 7fe650de8640 1 -- 172.21.15.112:0/3884041190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe64c0ab440 msgr2=0x7fe64c1ebbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:32:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.709+0000 7fe650de8640 1 --2- 172.21.15.112:0/3884041190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe64c0ab440 0x7fe64c1ebbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.709+0000 7fe650de8640 1 --2- 172.21.15.112:0/3884041190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe64c0ab440 0x7fe64c1ebbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:32:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.710+0000 7fe651dea640 1 -- 172.21.15.112:0/3884041190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64c0721f0 msgr2=0x7fe64c1ec650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.710+0000 7fe651dea640 1 --2- 172.21.15.112:0/3884041190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64c0721f0 0x7fe64c1ec650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.710+0000 7fe651dea640 1 -- 172.21.15.112:0/3884041190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe64c0ab440 msgr2=0x7fe64c1ebbe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:32:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.710+0000 7fe651dea640 1 --2- 172.21.15.112:0/3884041190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe64c0ab440 0x7fe64c1ebbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.710+0000 7fe651dea640 1 -- 172.21.15.112:0/3884041190 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64c1ec120 con 0x7fe64c0c2750 2024-04-25T01:32:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.710+0000 7fe6515e9640 1 --2- 172.21.15.112:0/3884041190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64c0721f0 0x7fe64c1ec650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:32:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.710+0000 7fe651dea640 1 --2- 172.21.15.112:0/3884041190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe64c0c2750 0x7fe64c0c40c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe64400cb00 tx=0x7fe64400a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.710+0000 7fe641ffb640 1 -- 172.21.15.112:0/3884041190 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644017070 con 0x7fe64c0c2750 2024-04-25T01:32:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.710+0000 7fe641ffb640 1 -- 172.21.15.112:0/3884041190 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe64400aca0 con 0x7fe64c0c2750 2024-04-25T01:32:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.710+0000 7fe641ffb640 1 -- 172.21.15.112:0/3884041190 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644012600 con 0x7fe64c0c2750 2024-04-25T01:32:02.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.710+0000 7fe65384d640 1 -- 172.21.15.112:0/3884041190 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe64c0c1510 con 0x7fe64c0c2750 2024-04-25T01:32:02.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.710+0000 7fe65384d640 1 -- 172.21.15.112:0/3884041190 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe64c1ee7f0 con 0x7fe64c0c2750 2024-04-25T01:32:02.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.711+0000 7fe65384d640 1 -- 172.21.15.112:0/3884041190 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe64c0682a0 con 0x7fe64c0c2750 2024-04-25T01:32:02.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.712+0000 7fe641ffb640 1 -- 172.21.15.112:0/3884041190 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe64401a070 con 0x7fe64c0c2750 2024-04-25T01:32:02.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.712+0000 7fe641ffb640 1 --2- 172.21.15.112:0/3884041190 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe63006be30 0x7fe63006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:02.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.712+0000 7fe641ffb640 1 -- 172.21.15.112:0/3884041190 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fe64408ad20 con 0x7fe64c0c2750 2024-04-25T01:32:02.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.712+0000 7fe6515e9640 1 --2- 172.21.15.112:0/3884041190 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe63006be30 0x7fe63006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:02.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.713+0000 7fe6515e9640 1 --2- 172.21.15.112:0/3884041190 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe63006be30 0x7fe63006e2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe648006fd0 tx=0x7fe648008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:02.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:02.718+0000 7fe641ffb640 1 -- 172.21.15.112:0/3884041190 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe64c1ec120 con 0x7fe64c0c2750 2024-04-25T01:32:03.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:03.029+0000 7fe65384d640 1 -- 172.21.15.112:0/3884041190 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe64c071fd0 con 0x7fe64c0c2750 2024-04-25T01:32:03.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:03.029+0000 7fe641ffb640 1 -- 172.21.15.112:0/3884041190 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1563 (secure 0 0 0) 0x7fe644058d70 con 0x7fe64c0c2750 2024-04-25T01:32:03.007 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":42,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:32:03.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:03.031+0000 7fe65384d640 1 -- 172.21.15.112:0/3884041190 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe63006be30 msgr2=0x7fe63006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:03.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:03.031+0000 7fe65384d640 1 --2- 172.21.15.112:0/3884041190 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe63006be30 0x7fe63006e2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe648006fd0 tx=0x7fe648008040 comp rx=0 tx=0).stop 2024-04-25T01:32:03.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:03.031+0000 7fe65384d640 1 -- 172.21.15.112:0/3884041190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe64c0c2750 msgr2=0x7fe64c0c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:03.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:03.031+0000 7fe65384d640 1 --2- 172.21.15.112:0/3884041190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe64c0c2750 0x7fe64c0c40c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe64400cb00 tx=0x7fe64400a060 comp rx=0 tx=0).stop 2024-04-25T01:32:03.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:03.031+0000 7fe65384d640 1 -- 172.21.15.112:0/3884041190 shutdown_connections 2024-04-25T01:32:03.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:03.031+0000 7fe65384d640 1 --2- 172.21.15.112:0/3884041190 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe63006be30 0x7fe63006e2f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:03.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:03.031+0000 7fe65384d640 1 --2- 172.21.15.112:0/3884041190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe64c0c2750 0x7fe64c0c40c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:03.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:03.031+0000 7fe65384d640 1 --2- 172.21.15.112:0/3884041190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe64c0ab440 0x7fe64c1ebbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:03.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:03.031+0000 7fe65384d640 1 --2- 172.21.15.112:0/3884041190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64c0721f0 0x7fe64c1ec650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:03.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:03.031+0000 7fe65384d640 1 -- 172.21.15.112:0/3884041190 >> 172.21.15.112:0/3884041190 conn(0x7fe64c0c6f80 msgr2=0x7fe64c071740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:03.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:03.031+0000 7fe65384d640 1 -- 172.21.15.112:0/3884041190 shutdown_connections 2024-04-25T01:32:03.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:03.031+0000 7fe65384d640 1 -- 172.21.15.112:0/3884041190 wait complete. 2024-04-25T01:32:06.020 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:32:06.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.150+0000 7f1e7e714640 1 Processor -- start 2024-04-25T01:32:06.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.150+0000 7f1e7e714640 1 -- start start 2024-04-25T01:32:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.151+0000 7f1e7e714640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1e78152fd0 0x7f1e780c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.151+0000 7f1e7e714640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e780c90e0 con 0x7f1e780c5d70 2024-04-25T01:32:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.151+0000 7f1e7e714640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e780cac00 con 0x7f1e78156cf0 2024-04-25T01:32:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.151+0000 7f1e7e714640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f1e780ca9b0 con 0x7f1e78152fd0 2024-04-25T01:32:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.151+0000 7f1e77fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1e78152fd0 0x7f1e780c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.151+0000 7f1e777fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1e780c5d70 0x7f1e78156280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38010/0 (socket says 172.21.15.112:38010) 2024-04-25T01:32:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.151+0000 7f1e777fe640 1 -- 172.21.15.112:0/3814527075 learned_addr learned my addr 172.21.15.112:0/3814527075 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:32:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.151+0000 7f1e77fff640 1 -- 172.21.15.112:0/3814527075 >> v1:172.21.15.187:6789/0 conn(0x7f1e78156cf0 legacy=0x7f1e7806bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.151+0000 7f1e77fff640 1 -- 172.21.15.112:0/3814527075 >> v1:172.21.15.112:6789/0 conn(0x7f1e780c5d70 legacy=0x7f1e78156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.151+0000 7f1e77fff640 1 -- 172.21.15.112:0/3814527075 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e780c9380 con 0x7f1e78152fd0 2024-04-25T01:32:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.152+0000 7f1e77fff640 1 --2- 172.21.15.112:0/3814527075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e78152fd0 0x7f1e780c9b80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1e680097a0 tx=0x7f1e6802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e3d81a49726258 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.152+0000 7f1e76ffd640 1 -- 172.21.15.112:0/3814527075 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68037070 con 0x7f1e78152fd0 2024-04-25T01:32:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.152+0000 7f1e76ffd640 1 -- 172.21.15.112:0/3814527075 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e68032440 con 0x7f1e78152fd0 2024-04-25T01:32:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.152+0000 7f1e76ffd640 1 -- 172.21.15.112:0/3814527075 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68032810 con 0x7f1e78152fd0 2024-04-25T01:32:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.153+0000 7f1e7e714640 1 -- 172.21.15.112:0/3814527075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e78152fd0 msgr2=0x7f1e780c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.153+0000 7f1e7e714640 1 --2- 172.21.15.112:0/3814527075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e78152fd0 0x7f1e780c9b80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1e680097a0 tx=0x7f1e6802a1a0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.153+0000 7f1e7e714640 1 -- 172.21.15.112:0/3814527075 shutdown_connections 2024-04-25T01:32:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.153+0000 7f1e7e714640 1 --2- 172.21.15.112:0/3814527075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e78152fd0 0x7f1e780c9b80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.153+0000 7f1e7e714640 1 -- 172.21.15.112:0/3814527075 >> 172.21.15.112:0/3814527075 conn(0x7f1e780acc70 msgr2=0x7f1e780ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.153+0000 7f1e7e714640 1 -- 172.21.15.112:0/3814527075 shutdown_connections 2024-04-25T01:32:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.153+0000 7f1e7e714640 1 -- 172.21.15.112:0/3814527075 wait complete. 2024-04-25T01:32:06.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.154+0000 7f1e7e714640 1 Processor -- start 2024-04-25T01:32:06.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.154+0000 7f1e7e714640 1 -- start start 2024-04-25T01:32:06.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.155+0000 7f1e7e714640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e780c5d70 0x7f1e781ebe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.155+0000 7f1e7e714640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e78152fd0 0x7f1e7814feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.155+0000 7f1e7e714640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e78156cf0 0x7f1e781e2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.155+0000 7f1e7e714640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7806c4b0 con 0x7f1e78152fd0 2024-04-25T01:32:06.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.155+0000 7f1e7e714640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1e780caee0 con 0x7f1e780c5d70 2024-04-25T01:32:06.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.155+0000 7f1e7e714640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1e7806c630 con 0x7f1e78156cf0 2024-04-25T01:32:06.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.155+0000 7f1e777fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e78152fd0 0x7f1e7814feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.155+0000 7f1e777fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e78152fd0 0x7f1e7814feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54694/0 (socket says 172.21.15.112:54694) 2024-04-25T01:32:06.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.155+0000 7f1e777fe640 1 -- 172.21.15.112:0/2561889673 learned_addr learned my addr 172.21.15.112:0/2561889673 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.155+0000 7f1e77fff640 1 --2- 172.21.15.112:0/2561889673 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e780c5d70 0x7f1e781ebe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.155+0000 7f1e7ccb1640 1 --2- 172.21.15.112:0/2561889673 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e78156cf0 0x7f1e781e2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.155+0000 7f1e777fe640 1 -- 172.21.15.112:0/2561889673 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e78156cf0 msgr2=0x7f1e781e2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.155+0000 7f1e777fe640 1 --2- 172.21.15.112:0/2561889673 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e78156cf0 0x7f1e781e2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.155+0000 7f1e777fe640 1 -- 172.21.15.112:0/2561889673 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e780c5d70 msgr2=0x7f1e781ebe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.155+0000 7f1e777fe640 1 --2- 172.21.15.112:0/2561889673 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e780c5d70 0x7f1e781ebe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.155+0000 7f1e777fe640 1 -- 172.21.15.112:0/2561889673 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e781e06b0 con 0x7f1e78152fd0 2024-04-25T01:32:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.156+0000 7f1e777fe640 1 --2- 172.21.15.112:0/2561889673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e78152fd0 0x7f1e7814feb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00c500 tx=0x7f1e6c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.156+0000 7f1e74ff9640 1 -- 172.21.15.112:0/2561889673 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c017070 con 0x7f1e78152fd0 2024-04-25T01:32:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.156+0000 7f1e74ff9640 1 -- 172.21.15.112:0/2561889673 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e6c005ce0 con 0x7f1e78152fd0 2024-04-25T01:32:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.156+0000 7f1e7e714640 1 -- 172.21.15.112:0/2561889673 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e781e09c0 con 0x7f1e78152fd0 2024-04-25T01:32:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.156+0000 7f1e74ff9640 1 -- 172.21.15.112:0/2561889673 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c007310 con 0x7f1e78152fd0 2024-04-25T01:32:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.156+0000 7f1e7e714640 1 -- 172.21.15.112:0/2561889673 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e781e0f80 con 0x7f1e78152fd0 2024-04-25T01:32:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.157+0000 7f1e7e714640 1 -- 172.21.15.112:0/2561889673 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e780682a0 con 0x7f1e78152fd0 2024-04-25T01:32:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.157+0000 7f1e74ff9640 1 -- 172.21.15.112:0/2561889673 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e6c01a070 con 0x7f1e78152fd0 2024-04-25T01:32:06.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.158+0000 7f1e74ff9640 1 --2- 172.21.15.112:0/2561889673 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e6406be30 0x7f1e6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.158+0000 7f1e74ff9640 1 -- 172.21.15.112:0/2561889673 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f1e6c08a460 con 0x7f1e78152fd0 2024-04-25T01:32:06.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.158+0000 7f1e77fff640 1 --2- 172.21.15.112:0/2561889673 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e6406be30 0x7f1e6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.158+0000 7f1e77fff640 1 --2- 172.21.15.112:0/2561889673 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e6406be30 0x7f1e6406e2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1e68034000 tx=0x7f1e6802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.163+0000 7f1e74ff9640 1 -- 172.21.15.112:0/2561889673 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1e6c0584b0 con 0x7f1e78152fd0 2024-04-25T01:32:06.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.453+0000 7f1e7e714640 1 -- 172.21.15.112:0/2561889673 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1e780c5ac0 con 0x7f1e78152fd0 2024-04-25T01:32:06.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.453+0000 7f1e74ff9640 1 -- 172.21.15.112:0/2561889673 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1568 (secure 0 0 0) 0x7f1e6c05c7b0 con 0x7f1e78152fd0 2024-04-25T01:32:06.431 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:32:06.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.455+0000 7f1e7e714640 1 -- 172.21.15.112:0/2561889673 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e6406be30 msgr2=0x7f1e6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.455+0000 7f1e7e714640 1 --2- 172.21.15.112:0/2561889673 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e6406be30 0x7f1e6406e2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1e68034000 tx=0x7f1e6802e000 comp rx=0 tx=0).stop 2024-04-25T01:32:06.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.455+0000 7f1e7e714640 1 -- 172.21.15.112:0/2561889673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e78152fd0 msgr2=0x7f1e7814feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.455+0000 7f1e7e714640 1 --2- 172.21.15.112:0/2561889673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e78152fd0 0x7f1e7814feb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00c500 tx=0x7f1e6c00c9c0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.455+0000 7f1e7e714640 1 -- 172.21.15.112:0/2561889673 shutdown_connections 2024-04-25T01:32:06.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.455+0000 7f1e7e714640 1 --2- 172.21.15.112:0/2561889673 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e6406be30 0x7f1e6406e2f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.455+0000 7f1e7e714640 1 --2- 172.21.15.112:0/2561889673 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e78156cf0 0x7f1e781e2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.455+0000 7f1e7e714640 1 --2- 172.21.15.112:0/2561889673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e78152fd0 0x7f1e7814feb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.455+0000 7f1e7e714640 1 --2- 172.21.15.112:0/2561889673 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e780c5d70 0x7f1e781ebe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.455+0000 7f1e7e714640 1 -- 172.21.15.112:0/2561889673 >> 172.21.15.112:0/2561889673 conn(0x7f1e780acc70 msgr2=0x7f1e78158350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:06.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.455+0000 7f1e7e714640 1 -- 172.21.15.112:0/2561889673 shutdown_connections 2024-04-25T01:32:06.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.455+0000 7f1e7e714640 1 -- 172.21.15.112:0/2561889673 wait complete. 2024-04-25T01:32:06.446 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:32:06.446 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:32:06.446 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:32:06.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.578+0000 7feb1fde7640 1 Processor -- start 2024-04-25T01:32:06.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.578+0000 7feb1fde7640 1 -- start start 2024-04-25T01:32:06.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.579+0000 7feb1fde7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb180caa40 0x7feb18156d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.579+0000 7feb1fde7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb181579e0 con 0x7feb1814f7e0 2024-04-25T01:32:06.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.579+0000 7feb1fde7640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb18158e90 con 0x7feb181558a0 2024-04-25T01:32:06.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.579+0000 7feb1fde7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feb18159aa0 con 0x7feb180caa40 2024-04-25T01:32:06.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.579+0000 7feb1db83640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7feb1814f7e0 0x7feb1806bd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38018/0 (socket says 172.21.15.112:38018) 2024-04-25T01:32:06.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.579+0000 7feb1db83640 1 -- 172.21.15.112:0/207355631 learned_addr learned my addr 172.21.15.112:0/207355631 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:32:06.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.579+0000 7feb1d382640 1 --2- 172.21.15.112:0/207355631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb180caa40 0x7feb18156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.580+0000 7feb1cb81640 1 -- 172.21.15.112:0/207355631 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1016428573 0 0) 0x7feb181579e0 con 0x7feb1814f7e0 2024-04-25T01:32:06.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.580+0000 7feb1cb81640 1 -- 172.21.15.112:0/207355631 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb04003520 con 0x7feb1814f7e0 2024-04-25T01:32:06.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.580+0000 7feb1cb81640 1 -- 172.21.15.112:0/207355631 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4097255561 0 0) 0x7feb18158e90 con 0x7feb181558a0 2024-04-25T01:32:06.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.580+0000 7feb1cb81640 1 -- 172.21.15.112:0/207355631 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb181579e0 con 0x7feb181558a0 2024-04-25T01:32:06.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.580+0000 7feb1d382640 1 -- 172.21.15.112:0/207355631 >> v1:172.21.15.187:6789/0 conn(0x7feb181558a0 legacy=0x7feb181572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.580+0000 7feb1d382640 1 -- 172.21.15.112:0/207355631 >> v1:172.21.15.112:6789/0 conn(0x7feb1814f7e0 legacy=0x7feb1806bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.580+0000 7feb1d382640 1 -- 172.21.15.112:0/207355631 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb18159170 con 0x7feb180caa40 2024-04-25T01:32:06.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.581+0000 7feb1d382640 1 --2- 172.21.15.112:0/207355631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb180caa40 0x7feb18156d90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feb080097d0 tx=0x7feb0802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4d9a178c48cb0bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.581+0000 7feb1cb81640 1 -- 172.21.15.112:0/207355631 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08037070 con 0x7feb180caa40 2024-04-25T01:32:06.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.581+0000 7feb1cb81640 1 -- 172.21.15.112:0/207355631 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb08032440 con 0x7feb180caa40 2024-04-25T01:32:06.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.581+0000 7feb1cb81640 1 -- 172.21.15.112:0/207355631 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08032810 con 0x7feb180caa40 2024-04-25T01:32:06.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.581+0000 7feb1fde7640 1 -- 172.21.15.112:0/207355631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb180caa40 msgr2=0x7feb18156d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.581+0000 7feb1fde7640 1 --2- 172.21.15.112:0/207355631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb180caa40 0x7feb18156d90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feb080097d0 tx=0x7feb0802a210 comp rx=0 tx=0).stop 2024-04-25T01:32:06.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.581+0000 7feb1fde7640 1 -- 172.21.15.112:0/207355631 shutdown_connections 2024-04-25T01:32:06.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.581+0000 7feb1fde7640 1 --2- 172.21.15.112:0/207355631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb180caa40 0x7feb18156d90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.581+0000 7feb1fde7640 1 -- 172.21.15.112:0/207355631 >> 172.21.15.112:0/207355631 conn(0x7feb180acc90 msgr2=0x7feb180ad0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:06.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.582+0000 7feb1fde7640 1 -- 172.21.15.112:0/207355631 shutdown_connections 2024-04-25T01:32:06.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.582+0000 7feb1fde7640 1 -- 172.21.15.112:0/207355631 wait complete. 2024-04-25T01:32:06.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.583+0000 7feb1fde7640 1 Processor -- start 2024-04-25T01:32:06.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.583+0000 7feb1fde7640 1 -- start start 2024-04-25T01:32:06.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.583+0000 7feb1fde7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb180caa40 0x7feb1814e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.583+0000 7feb1fde7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb1814f7e0 0x7feb180c3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.583+0000 7feb1fde7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb181558a0 0x7feb1814d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.583+0000 7feb1fde7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb18153450 con 0x7feb181558a0 2024-04-25T01:32:06.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.583+0000 7feb1fde7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feb18159730 con 0x7feb1814f7e0 2024-04-25T01:32:06.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.583+0000 7feb1fde7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feb1806c460 con 0x7feb180caa40 2024-04-25T01:32:06.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.583+0000 7feb1e384640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb181558a0 0x7feb1814d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.583+0000 7feb1e384640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb181558a0 0x7feb1814d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54710/0 (socket says 172.21.15.112:54710) 2024-04-25T01:32:06.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.584+0000 7feb1e384640 1 -- 172.21.15.112:0/1797610221 learned_addr learned my addr 172.21.15.112:0/1797610221 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:06.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.584+0000 7feb1e384640 1 -- 172.21.15.112:0/1797610221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb180caa40 msgr2=0x7feb1814e4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:32:06.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.584+0000 7feb1e384640 1 --2- 172.21.15.112:0/1797610221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb180caa40 0x7feb1814e4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.584+0000 7feb1e384640 1 -- 172.21.15.112:0/1797610221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb1814f7e0 msgr2=0x7feb180c3ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:32:06.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.584+0000 7feb1e384640 1 --2- 172.21.15.112:0/1797610221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb1814f7e0 0x7feb180c3ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.584+0000 7feb1e384640 1 -- 172.21.15.112:0/1797610221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1814e200 con 0x7feb181558a0 2024-04-25T01:32:06.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.584+0000 7feb1e384640 1 --2- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb181558a0 0x7feb1814d750 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feb1000c6e0 tx=0x7feb1000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.584+0000 7feb0e7fc640 1 -- 172.21.15.112:0/1797610221 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10003b70 con 0x7feb181558a0 2024-04-25T01:32:06.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.584+0000 7feb0e7fc640 1 -- 172.21.15.112:0/1797610221 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb10003d10 con 0x7feb181558a0 2024-04-25T01:32:06.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.584+0000 7feb1fde7640 1 -- 172.21.15.112:0/1797610221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb180bef20 con 0x7feb181558a0 2024-04-25T01:32:06.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.584+0000 7feb0e7fc640 1 -- 172.21.15.112:0/1797610221 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb100052d0 con 0x7feb181558a0 2024-04-25T01:32:06.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.584+0000 7feb1fde7640 1 -- 172.21.15.112:0/1797610221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb180bf7e0 con 0x7feb181558a0 2024-04-25T01:32:06.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.586+0000 7feb1fde7640 1 --2- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb180b0d10 0x7feb1814dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.586+0000 7feb1fde7640 1 -- 172.21.15.112:0/1797610221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb180bf640 con 0x7feb180b0d10 2024-04-25T01:32:06.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.586+0000 7feb1db83640 1 --2- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb180b0d10 0x7feb1814dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.586+0000 7feb0e7fc640 1 -- 172.21.15.112:0/1797610221 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb10005470 con 0x7feb181558a0 2024-04-25T01:32:06.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.586+0000 7feb1db83640 1 -- 172.21.15.112:0/1797610221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feb18075980 con 0x7feb180b0d10 2024-04-25T01:32:06.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.586+0000 7feb1db83640 1 --2- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb180b0d10 0x7feb1814dc90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7feb1400c310 tx=0x7feb1400c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.587+0000 7feb0e7fc640 1 --2- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feafc06be60 0x7feafc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.587+0000 7feb1d382640 1 --2- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feafc06be60 0x7feafc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.587+0000 7feb0e7fc640 1 -- 172.21.15.112:0/1797610221 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7feb1008b050 con 0x7feb181558a0 2024-04-25T01:32:06.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.587+0000 7feb0e7fc640 1 -- 172.21.15.112:0/1797610221 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14017070 con 0x7feb180b0d10 2024-04-25T01:32:06.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.587+0000 7feb0e7fc640 1 -- 172.21.15.112:0/1797610221 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7feb14005ce0 con 0x7feb180b0d10 2024-04-25T01:32:06.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.587+0000 7feb0e7fc640 1 -- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb180b0d10 msgr2=0x7feb1814dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.587+0000 7feb0e7fc640 1 --2- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb180b0d10 0x7feb1814dc90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7feb1400c310 tx=0x7feb1400c7d0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.587+0000 7feb0e7fc640 1 -- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb180b0d10 msgr2=0x7feb1814dc90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:32:06.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.587+0000 7feb0e7fc640 1 --2- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb180b0d10 0x7feb1814dc90 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7feb1400c310 tx=0x7feb1400c7d0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.587+0000 7feb1d382640 1 --2- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feafc06be60 0x7feafc06e320 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feb08034000 tx=0x7feb0802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.597+0000 7feb1fde7640 1 --2- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb181eaff0 0x7feb181eb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.597+0000 7feb1fde7640 1 -- 172.21.15.112:0/1797610221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb180682a0 con 0x7feb181eaff0 2024-04-25T01:32:06.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.597+0000 7feb1db83640 1 --2- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb181eaff0 0x7feb181eb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.597+0000 7feb1db83640 1 -- 172.21.15.112:0/1797610221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7feb1806c0e0 con 0x7feb181eaff0 2024-04-25T01:32:06.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.598+0000 7feb1db83640 1 --2- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb181eaff0 0x7feb181eb400 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7feb181ec120 tx=0x7feb14000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.598+0000 7feb0e7fc640 1 -- 172.21.15.112:0/1797610221 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14017040 con 0x7feb181eaff0 2024-04-25T01:32:06.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.598+0000 7feb0e7fc640 1 -- 172.21.15.112:0/1797610221 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4559 (secure 0 0 0) 0x7feb14007730 con 0x7feb181eaff0 2024-04-25T01:32:06.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.598+0000 7feb0e7fc640 1 -- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb181eaff0 msgr2=0x7feb181eb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.598+0000 7feb0e7fc640 1 --2- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb181eaff0 0x7feb181eb400 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7feb181ec120 tx=0x7feb14000ed0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.598+0000 7feb0e7fc640 1 -- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb181eaff0 msgr2=0x7feb181eb400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:32:06.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.598+0000 7feb0e7fc640 1 --2- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb181eaff0 0x7feb181eb400 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7feb181ec120 tx=0x7feb14000ed0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.576 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:32:06.576 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:32:06.577 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:32:06.577 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:32:06.577 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 46, 2024-04-25T01:32:06.577 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 7363, 2024-04-25T01:32:06.577 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:32:06.577 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:32:06.577 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:32:06.577 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:32:06.577 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:06.577 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:32:06.577 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:32:06.577 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:32:06.577 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:32:06.577 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:06.578 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:06.578 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:06.578 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:06.578 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:06.578 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:06.578 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:06.578 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:06.578 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:06.578 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:06.578 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:06.578 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:06.578 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:32:06.578 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:32:06.578 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:06.579 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:06.579 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:06.579 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:06.579 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:06.579 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:06.579 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:06.579 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:06.579 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:06.579 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:06.579 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:06.579 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:06.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.579 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:32:06.579 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:32:06.579 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:32:06.580 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:32:06.580 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:32:06.580 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:32:06.580 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:32:06.580 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:32:06.580 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:32:06.580 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:32:06.580 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:32:06.580 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:32:06.580 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:32:06.580 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:32:06.580 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:32:06.580 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:32:06.580 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:32:06.580 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:06.581 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:06.581 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:06.581 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:06.581 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:06.581 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:06.581 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:06.581 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:06.581 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:06.581 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:06.581 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:06.581 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:06.581 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:32:06.581 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.581 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:32:06.581 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.582 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:32:06.582 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:32:06.582 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:06.582 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:06.582 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.582 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:06.582 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:32:06.582 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:06.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.582 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.582 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:06.582 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:32:06.582 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:06.582 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.583 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:06.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.583 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:32:06.583 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:32:06.583 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:06.583 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:06.583 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:06.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.583 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.584 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:32:06.584 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:32:06.584 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:06.584 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:06.584 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.584 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:06.584 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:32:06.584 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:06.584 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.584 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.584 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:06.584 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:32:06.584 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:06.584 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.584 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:06.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.585 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:32:06.585 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:32:06.585 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:06.585 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:06.585 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:06.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.585 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.585 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:32:06.585 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:32:06.585 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:06.585 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:06.585 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.585 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:06.585 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:32:06.586 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:06.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.586 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.586 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:06.586 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:32:06.586 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:06.586 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.586 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:06.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.586 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:32:06.586 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:32:06.586 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:06.586 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:06.586 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:06.586 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.587 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:06.587 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.587 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:32:06.587 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:32:06.587 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.587 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:06.587 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:06.587 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:32:06.587 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.587 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:06.587 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:32:06.587 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.587 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:06.587 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:06.587 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:32:06.588 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.588 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:06.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.588 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:32:06.588 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:32:06.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.599+0000 7feb1fde7640 1 -- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feafc06be60 msgr2=0x7feafc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.599+0000 7feb1fde7640 1 --2- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feafc06be60 0x7feafc06e320 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feb08034000 tx=0x7feb0802e000 comp rx=0 tx=0).stop 2024-04-25T01:32:06.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.599+0000 7feb1fde7640 1 -- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb181558a0 msgr2=0x7feb1814d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.599+0000 7feb1fde7640 1 --2- 172.21.15.112:0/1797610221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb181558a0 0x7feb1814d750 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feb1000c6e0 tx=0x7feb1000cba0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.599+0000 7feb1e384640 1 -- 172.21.15.112:0/1797610221 reap_dead start 2024-04-25T01:32:06.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.599+0000 7feb1e384640 1 -- 172.21.15.112:0/1797610221 reap_dead start 2024-04-25T01:32:06.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.599+0000 7feb1fde7640 1 -- 172.21.15.112:0/1797610221 shutdown_connections 2024-04-25T01:32:06.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.599+0000 7feb1fde7640 1 -- 172.21.15.112:0/1797610221 >> 172.21.15.112:0/1797610221 conn(0x7feb180acc90 msgr2=0x7feb18154cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:06.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.599+0000 7feb1fde7640 1 -- 172.21.15.112:0/1797610221 shutdown_connections 2024-04-25T01:32:06.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.599+0000 7feb1fde7640 1 -- 172.21.15.112:0/1797610221 wait complete. 2024-04-25T01:32:06.590 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:32:06.590 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:32:06.590 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:32:06.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.717+0000 7ff975763640 1 Processor -- start 2024-04-25T01:32:06.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.717+0000 7ff975763640 1 -- start start 2024-04-25T01:32:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.718+0000 7ff975763640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff970153030 0x7ff9700a9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.718+0000 7ff975763640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9700a9bd0 0x7ff97015cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.718+0000 7ff975763640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff97006f220 con 0x7ff970072090 2024-04-25T01:32:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.718+0000 7ff975763640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7ff970153470 con 0x7ff970153030 2024-04-25T01:32:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.718+0000 7ff975763640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff9700c4990 con 0x7ff9700a9bd0 2024-04-25T01:32:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.718+0000 7ff96e7fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff970153030 0x7ff9700a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.718+0000 7ff96e7fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff970153030 0x7ff9700a9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54748/0 (socket says 172.21.15.112:54748) 2024-04-25T01:32:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.718+0000 7ff96effd640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7ff970072090 0x7ff97014f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:36876/0 (socket says 172.21.15.112:36876) 2024-04-25T01:32:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.718+0000 7ff96e7fc640 1 -- 172.21.15.112:0/981471729 learned_addr learned my addr 172.21.15.112:0/981471729 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.718+0000 7ff96f7fe640 1 --2- 172.21.15.112:0/981471729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9700a9bd0 0x7ff97015cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.718+0000 7ff96e7fc640 1 -- 172.21.15.112:0/981471729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9700a9bd0 msgr2=0x7ff97015cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:32:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.718+0000 7ff96e7fc640 1 --2- 172.21.15.112:0/981471729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9700a9bd0 0x7ff97015cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.718+0000 7ff96e7fc640 1 -- 172.21.15.112:0/981471729 >> v1:172.21.15.187:6789/0 conn(0x7ff970072090 legacy=0x7ff97014f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.718+0000 7ff96dffb640 1 -- 172.21.15.112:0/981471729 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2619041082 0 0) 0x7ff97006f220 con 0x7ff970072090 2024-04-25T01:32:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.718+0000 7ff96e7fc640 1 -- 172.21.15.112:0/981471729 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97006f4c0 con 0x7ff970153030 2024-04-25T01:32:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.719+0000 7ff96e7fc640 1 --2- 172.21.15.112:0/981471729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff970153030 0x7ff9700a9690 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff96400bb10 tx=0x7ff96402c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650f7baba2c431fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.720+0000 7ff96dffb640 1 -- 172.21.15.112:0/981471729 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964036020 con 0x7ff970153030 2024-04-25T01:32:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.720+0000 7ff96dffb640 1 -- 172.21.15.112:0/981471729 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff964032820 con 0x7ff970153030 2024-04-25T01:32:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.720+0000 7ff96dffb640 1 -- 172.21.15.112:0/981471729 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964032bf0 con 0x7ff970153030 2024-04-25T01:32:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.720+0000 7ff975763640 1 -- 172.21.15.112:0/981471729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff970153030 msgr2=0x7ff9700a9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.720+0000 7ff975763640 1 --2- 172.21.15.112:0/981471729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff970153030 0x7ff9700a9690 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff96400bb10 tx=0x7ff96402c920 comp rx=0 tx=0).stop 2024-04-25T01:32:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.721+0000 7ff975763640 1 -- 172.21.15.112:0/981471729 shutdown_connections 2024-04-25T01:32:06.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.721+0000 7ff975763640 1 --2- 172.21.15.112:0/981471729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9700a9bd0 0x7ff97015cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.721+0000 7ff975763640 1 --2- 172.21.15.112:0/981471729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff970153030 0x7ff9700a9690 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.721+0000 7ff975763640 1 -- 172.21.15.112:0/981471729 >> 172.21.15.112:0/981471729 conn(0x7ff97006c8a0 msgr2=0x7ff9700c8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:06.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.721+0000 7ff975763640 1 -- 172.21.15.112:0/981471729 shutdown_connections 2024-04-25T01:32:06.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.721+0000 7ff975763640 1 -- 172.21.15.112:0/981471729 wait complete. 2024-04-25T01:32:06.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.722+0000 7ff975763640 1 Processor -- start 2024-04-25T01:32:06.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.722+0000 7ff975763640 1 -- start start 2024-04-25T01:32:06.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.723+0000 7ff975763640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff970072090 0x7ff970156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.723+0000 7ff975763640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9700a9bd0 0x7ff970151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.723+0000 7ff975763640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff970153030 0x7ff9701ee1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.723+0000 7ff975763640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff97006fa80 con 0x7ff970153030 2024-04-25T01:32:06.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.723+0000 7ff975763640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff97006fdf0 con 0x7ff9700a9bd0 2024-04-25T01:32:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.723+0000 7ff975763640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff97014fea0 con 0x7ff970072090 2024-04-25T01:32:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.723+0000 7ff96f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff970153030 0x7ff9701ee1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.723+0000 7ff96f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff970153030 0x7ff9701ee1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54760/0 (socket says 172.21.15.112:54760) 2024-04-25T01:32:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.723+0000 7ff96f7fe640 1 -- 172.21.15.112:0/1084802852 learned_addr learned my addr 172.21.15.112:0/1084802852 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.723+0000 7ff96e7fc640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9700a9bd0 0x7ff970151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.723+0000 7ff96effd640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff970072090 0x7ff970156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.723+0000 7ff96f7fe640 1 -- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff970072090 msgr2=0x7ff970156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.723+0000 7ff96f7fe640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff970072090 0x7ff970156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.723+0000 7ff96f7fe640 1 -- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9700a9bd0 msgr2=0x7ff970151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.723+0000 7ff96f7fe640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9700a9bd0 0x7ff970151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.723+0000 7ff96f7fe640 1 -- 172.21.15.112:0/1084802852 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97014d890 con 0x7ff970153030 2024-04-25T01:32:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.723+0000 7ff96e7fc640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9700a9bd0 0x7ff970151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:32:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.724+0000 7ff96f7fe640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff970153030 0x7ff9701ee1d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff968009dd0 tx=0x7ff96800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.724+0000 7ff9577fe640 1 -- 172.21.15.112:0/1084802852 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968018020 con 0x7ff970153030 2024-04-25T01:32:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.724+0000 7ff9577fe640 1 -- 172.21.15.112:0/1084802852 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff968014480 con 0x7ff970153030 2024-04-25T01:32:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.724+0000 7ff9577fe640 1 -- 172.21.15.112:0/1084802852 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968014870 con 0x7ff970153030 2024-04-25T01:32:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.724+0000 7ff975763640 1 -- 172.21.15.112:0/1084802852 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9701eacf0 con 0x7ff970153030 2024-04-25T01:32:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.724+0000 7ff975763640 1 -- 172.21.15.112:0/1084802852 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9701eb2b0 con 0x7ff970153030 2024-04-25T01:32:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.725+0000 7ff975763640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff97014d400 0x7ff9701ee710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.725+0000 7ff975763640 1 -- 172.21.15.112:0/1084802852 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff9701eec50 con 0x7ff97014d400 2024-04-25T01:32:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.725+0000 7ff96effd640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff97014d400 0x7ff9701ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.725+0000 7ff9577fe640 1 -- 172.21.15.112:0/1084802852 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff968005d10 con 0x7ff970153030 2024-04-25T01:32:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.726+0000 7ff96effd640 1 -- 172.21.15.112:0/1084802852 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff970075980 con 0x7ff97014d400 2024-04-25T01:32:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.726+0000 7ff96effd640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff97014d400 0x7ff9701ee710 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff9600011e0 tx=0x7ff960001990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.726+0000 7ff9577fe640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff95c06be30 0x7ff95c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.726+0000 7ff96e7fc640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff95c06be30 0x7ff95c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.726+0000 7ff9577fe640 1 -- 172.21.15.112:0/1084802852 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7ff96808bba0 con 0x7ff970153030 2024-04-25T01:32:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.726+0000 7ff9577fe640 1 -- 172.21.15.112:0/1084802852 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96000f600 con 0x7ff97014d400 2024-04-25T01:32:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.726+0000 7ff9577fe640 1 -- 172.21.15.112:0/1084802852 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff960018070 con 0x7ff97014d400 2024-04-25T01:32:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.726+0000 7ff9577fe640 1 -- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff97014d400 msgr2=0x7ff9701ee710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.726+0000 7ff9577fe640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff97014d400 0x7ff9701ee710 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff9600011e0 tx=0x7ff960001990 comp rx=0 tx=0).stop 2024-04-25T01:32:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.726+0000 7ff9577fe640 1 -- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff97014d400 msgr2=0x7ff9701ee710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:32:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.726+0000 7ff9577fe640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff97014d400 0x7ff9701ee710 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff9600011e0 tx=0x7ff960001990 comp rx=0 tx=0).stop 2024-04-25T01:32:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.726+0000 7ff96e7fc640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff95c06be30 0x7ff95c06e2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff964037000 tx=0x7ff964032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.735+0000 7ff975763640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9701eb4f0 0x7ff97014ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.735+0000 7ff975763640 1 -- 172.21.15.112:0/1084802852 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff9700689b0 con 0x7ff9701eb4f0 2024-04-25T01:32:06.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.735+0000 7ff96effd640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9701eb4f0 0x7ff97014ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.735+0000 7ff96effd640 1 -- 172.21.15.112:0/1084802852 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff970072ff0 con 0x7ff9701eb4f0 2024-04-25T01:32:06.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.735+0000 7ff96effd640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9701eb4f0 0x7ff97014ba70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff9600011e0 tx=0x7ff960001080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.735+0000 7ff9577fe640 1 -- 172.21.15.112:0/1084802852 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960026410 con 0x7ff9701eb4f0 2024-04-25T01:32:06.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.736+0000 7ff9577fe640 1 -- 172.21.15.112:0/1084802852 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5069 (secure 0 0 0) 0x7ff9600133f0 con 0x7ff9701eb4f0 2024-04-25T01:32:06.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.736+0000 7ff9577fe640 1 -- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9701eb4f0 msgr2=0x7ff97014ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.736+0000 7ff9577fe640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9701eb4f0 0x7ff97014ba70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff9600011e0 tx=0x7ff960001080 comp rx=0 tx=0).stop 2024-04-25T01:32:06.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.736+0000 7ff9577fe640 1 -- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9701eb4f0 msgr2=0x7ff97014ba70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:32:06.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.736+0000 7ff9577fe640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9701eb4f0 0x7ff97014ba70 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff9600011e0 tx=0x7ff960001080 comp rx=0 tx=0).stop 2024-04-25T01:32:06.714 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:32:06.714 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:32:06.714 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:32:06.714 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:32:06.714 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 46, 2024-04-25T01:32:06.714 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 162424, 2024-04-25T01:32:06.714 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:32:06.714 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:32:06.714 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:32:06.714 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:32:06.714 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:06.715 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:32:06.715 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:32:06.715 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:32:06.715 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:32:06.715 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:06.715 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:06.715 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:06.715 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:06.715 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:06.715 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:06.715 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:06.715 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:06.715 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:06.715 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:06.715 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:06.716 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:06.716 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:32:06.716 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:32:06.716 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:06.716 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:06.716 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:06.716 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:06.716 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:06.716 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:06.716 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:06.716 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:06.716 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:06.716 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:06.716 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:06.716 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:32:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:32:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:32:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:32:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:32:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:32:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:32:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:32:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:32:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:32:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:32:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:32:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:32:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:32:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:32:06.718 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:32:06.718 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:32:06.718 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:06.718 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:06.718 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:06.718 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:06.718 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:06.718 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:06.718 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:06.718 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:06.718 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:06.718 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:06.718 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:06.718 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:06.719 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:32:06.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.719 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:32:06.719 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.719 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:32:06.719 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:32:06.719 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:06.719 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:06.719 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.719 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:06.719 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:32:06.719 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:06.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.719 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.719 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:32:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:32:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:32:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:32:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:32:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:06.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.721 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:06.721 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:32:06.721 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:06.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.721 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:06.721 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:32:06.721 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:06.721 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.721 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:06.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.721 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:32:06.721 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:32:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:32:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:32:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:32:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:32:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:32:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:32:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:32:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:32:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:32:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:32:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:32:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:32:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:06.725 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:32:06.725 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.725 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:06.725 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:32:06.725 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.725 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:06.725 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:06.725 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:32:06.725 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.725 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:06.725 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:32:06.725 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.725 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:06.725 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:06.725 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:32:06.726 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.726 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:06.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.726 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:32:06.726 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:32:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.736+0000 7ff975763640 1 -- 172.21.15.112:0/1084802852 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff95c06be30 msgr2=0x7ff95c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.736+0000 7ff975763640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff95c06be30 0x7ff95c06e2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff964037000 tx=0x7ff964032000 comp rx=0 tx=0).stop 2024-04-25T01:32:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.736+0000 7ff975763640 1 -- 172.21.15.112:0/1084802852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff970153030 msgr2=0x7ff9701ee1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.736+0000 7ff975763640 1 --2- 172.21.15.112:0/1084802852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff970153030 0x7ff9701ee1d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff968009dd0 tx=0x7ff96800c490 comp rx=0 tx=0).stop 2024-04-25T01:32:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.736+0000 7ff96f7fe640 1 -- 172.21.15.112:0/1084802852 reap_dead start 2024-04-25T01:32:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.736+0000 7ff96f7fe640 1 -- 172.21.15.112:0/1084802852 reap_dead start 2024-04-25T01:32:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.736+0000 7ff975763640 1 -- 172.21.15.112:0/1084802852 shutdown_connections 2024-04-25T01:32:06.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.736+0000 7ff975763640 1 -- 172.21.15.112:0/1084802852 >> 172.21.15.112:0/1084802852 conn(0x7ff97006c8a0 msgr2=0x7ff97014f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:06.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.736+0000 7ff975763640 1 -- 172.21.15.112:0/1084802852 shutdown_connections 2024-04-25T01:32:06.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.736+0000 7ff975763640 1 -- 172.21.15.112:0/1084802852 wait complete. 2024-04-25T01:32:06.728 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:32:06.728 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:32:06.728 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:32:06.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.861+0000 7f4adbc38640 1 Processor -- start 2024-04-25T01:32:06.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.861+0000 7f4adbc38640 1 -- start start 2024-04-25T01:32:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.862+0000 7f4adbc38640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4ad41596a0 0x7f4ad4152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.862+0000 7f4adbc38640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4ad40c72b0 0x7f4ad4160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.862+0000 7f4adbc38640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ad4159b90 con 0x7f4ad41592c0 2024-04-25T01:32:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.862+0000 7f4adbc38640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f4ad40c6e20 con 0x7f4ad41596a0 2024-04-25T01:32:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.862+0000 7f4adbc38640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4ad406c970 con 0x7f4ad40c72b0 2024-04-25T01:32:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.862+0000 7f4ad91d3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4ad41596a0 0x7f4ad4152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.862+0000 7f4ad99d4640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4ad41592c0 0x7f4ad40c9070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38032/0 (socket says 172.21.15.112:38032) 2024-04-25T01:32:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.862+0000 7f4ad99d4640 1 -- 172.21.15.112:0/281843346 learned_addr learned my addr 172.21.15.112:0/281843346 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:32:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.862+0000 7f4ada1d5640 1 --2- 172.21.15.112:0/281843346 >> v2:172.21.15.187:3300/0 conn(0x7f4ad40c72b0 0x7f4ad4160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.863+0000 7f4ad91d3640 1 -- 172.21.15.112:0/281843346 >> v2:172.21.15.187:3300/0 conn(0x7f4ad40c72b0 msgr2=0x7f4ad4160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:32:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.863+0000 7f4ad91d3640 1 --2- 172.21.15.112:0/281843346 >> v2:172.21.15.187:3300/0 conn(0x7f4ad40c72b0 0x7f4ad4160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.863+0000 7f4ad91d3640 1 -- 172.21.15.112:0/281843346 >> v1:172.21.15.112:6789/0 conn(0x7f4ad41592c0 legacy=0x7f4ad40c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.863+0000 7f4ad91d3640 1 -- 172.21.15.112:0/281843346 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad406cc50 con 0x7f4ad41596a0 2024-04-25T01:32:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.863+0000 7f4ad89d2640 1 -- 172.21.15.112:0/281843346 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1659632802 0 0) 0x7f4ad4159b90 con 0x7f4ad41592c0 2024-04-25T01:32:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.863+0000 7f4ad91d3640 1 --2- 172.21.15.112:0/281843346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ad41596a0 0x7f4ad4152d00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4ad40c21d0 tx=0x7f4ac402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f0d7d10411abd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.863+0000 7f4ad89d2640 1 -- 172.21.15.112:0/281843346 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4037070 con 0x7f4ad41596a0 2024-04-25T01:32:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.864+0000 7f4ad89d2640 1 -- 172.21.15.112:0/281843346 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac402add0 con 0x7f4ad41596a0 2024-04-25T01:32:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.864+0000 7f4ad89d2640 1 -- 172.21.15.112:0/281843346 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4032660 con 0x7f4ad41596a0 2024-04-25T01:32:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.864+0000 7f4adbc38640 1 -- 172.21.15.112:0/281843346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ad41596a0 msgr2=0x7f4ad4152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.864+0000 7f4adbc38640 1 --2- 172.21.15.112:0/281843346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ad41596a0 0x7f4ad4152d00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4ad40c21d0 tx=0x7f4ac402a080 comp rx=0 tx=0).stop 2024-04-25T01:32:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.864+0000 7f4adbc38640 1 -- 172.21.15.112:0/281843346 shutdown_connections 2024-04-25T01:32:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.864+0000 7f4adbc38640 1 --2- 172.21.15.112:0/281843346 >> v2:172.21.15.187:3300/0 conn(0x7f4ad40c72b0 0x7f4ad4160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.864+0000 7f4adbc38640 1 --2- 172.21.15.112:0/281843346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ad41596a0 0x7f4ad4152d00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.864+0000 7f4adbc38640 1 -- 172.21.15.112:0/281843346 >> 172.21.15.112:0/281843346 conn(0x7f4ad40cb170 msgr2=0x7f4ad40cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.864+0000 7f4adbc38640 1 -- 172.21.15.112:0/281843346 shutdown_connections 2024-04-25T01:32:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.864+0000 7f4adbc38640 1 -- 172.21.15.112:0/281843346 wait complete. 2024-04-25T01:32:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.865+0000 7f4adbc38640 1 Processor -- start 2024-04-25T01:32:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.865+0000 7f4adbc38640 1 -- start start 2024-04-25T01:32:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.866+0000 7f4adbc38640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ad40c72b0 0x7f4ad41eeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.866+0000 7f4adbc38640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ad41592c0 0x7f4ad41546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.866+0000 7f4adbc38640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ad41596a0 0x7f4ad4154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.866+0000 7f4adbc38640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad406c510 con 0x7f4ad40c72b0 2024-04-25T01:32:06.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.866+0000 7f4adbc38640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad406c390 con 0x7f4ad41592c0 2024-04-25T01:32:06.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.866+0000 7f4adbc38640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad406db20 con 0x7f4ad41596a0 2024-04-25T01:32:06.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.866+0000 7f4ad99d4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ad40c72b0 0x7f4ad41eeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.866+0000 7f4ad99d4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ad40c72b0 0x7f4ad41eeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54780/0 (socket says 172.21.15.112:54780) 2024-04-25T01:32:06.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.866+0000 7f4ad99d4640 1 -- 172.21.15.112:0/993014335 learned_addr learned my addr 172.21.15.112:0/993014335 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:06.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.866+0000 7f4ad91d3640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ad41592c0 0x7f4ad41546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.866+0000 7f4ada1d5640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ad41596a0 0x7f4ad4154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.867+0000 7f4ad99d4640 1 -- 172.21.15.112:0/993014335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ad41596a0 msgr2=0x7f4ad4154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.867+0000 7f4ad99d4640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ad41596a0 0x7f4ad4154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.867+0000 7f4ad99d4640 1 -- 172.21.15.112:0/993014335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ad41592c0 msgr2=0x7f4ad41546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.867+0000 7f4ad99d4640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ad41592c0 0x7f4ad41546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.867+0000 7f4ad99d4640 1 -- 172.21.15.112:0/993014335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad41ef500 con 0x7f4ad40c72b0 2024-04-25T01:32:06.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.867+0000 7f4ad99d4640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ad40c72b0 0x7f4ad41eeb10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000c500 tx=0x7f4ad000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.867+0000 7f4aca7fc640 1 -- 172.21.15.112:0/993014335 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0018070 con 0x7f4ad40c72b0 2024-04-25T01:32:06.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.867+0000 7f4aca7fc640 1 -- 172.21.15.112:0/993014335 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad0007580 con 0x7f4ad40c72b0 2024-04-25T01:32:06.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.867+0000 7f4aca7fc640 1 -- 172.21.15.112:0/993014335 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0004a20 con 0x7f4ad40c72b0 2024-04-25T01:32:06.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.867+0000 7f4adbc38640 1 -- 172.21.15.112:0/993014335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad4153bf0 con 0x7f4ad40c72b0 2024-04-25T01:32:06.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.867+0000 7f4adbc38640 1 -- 172.21.15.112:0/993014335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad4159e20 con 0x7f4ad40c72b0 2024-04-25T01:32:06.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.868+0000 7f4adbc38640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ad41ef050 0x7f4ad4149230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.868+0000 7f4adbc38640 1 -- 172.21.15.112:0/993014335 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad4075980 con 0x7f4ad41ef050 2024-04-25T01:32:06.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.868+0000 7f4ad91d3640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ad41ef050 0x7f4ad4149230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.869+0000 7f4ad91d3640 1 -- 172.21.15.112:0/993014335 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4ad40679a0 con 0x7f4ad41ef050 2024-04-25T01:32:06.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.869+0000 7f4ad91d3640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ad41ef050 0x7f4ad4149230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4ad408cb00 tx=0x7f4ac40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.869+0000 7f4aca7fc640 1 -- 172.21.15.112:0/993014335 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ad001b070 con 0x7f4ad40c72b0 2024-04-25T01:32:06.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.869+0000 7f4aca7fc640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f4ab806be30 0x7f4ab806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.869+0000 7f4ada1d5640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f4ab806be30 0x7f4ab806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.870+0000 7f4aca7fc640 1 -- 172.21.15.112:0/993014335 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f4ad008a430 con 0x7f4ad40c72b0 2024-04-25T01:32:06.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.870+0000 7f4ada1d5640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f4ab806be30 0x7f4ab806e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4ad41557c0 tx=0x7f4acc00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.870+0000 7f4aca7fc640 1 -- 172.21.15.112:0/993014335 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4043070 con 0x7f4ad41ef050 2024-04-25T01:32:06.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.870+0000 7f4aca7fc640 1 -- 172.21.15.112:0/993014335 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4ac4032c90 con 0x7f4ad41ef050 2024-04-25T01:32:06.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.870+0000 7f4aca7fc640 1 -- 172.21.15.112:0/993014335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ad41ef050 msgr2=0x7f4ad4149230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.870+0000 7f4aca7fc640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ad41ef050 0x7f4ad4149230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4ad408cb00 tx=0x7f4ac40046c0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.870+0000 7f4aca7fc640 1 -- 172.21.15.112:0/993014335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ad41ef050 msgr2=0x7f4ad4149230 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:32:06.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.870+0000 7f4aca7fc640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ad41ef050 0x7f4ad4149230 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.880+0000 7f4adbc38640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ad4155150 0x7f4ad4149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.880+0000 7f4adbc38640 1 -- 172.21.15.112:0/993014335 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad40c79d0 con 0x7f4ad4155150 2024-04-25T01:32:06.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.880+0000 7f4ad91d3640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ad4155150 0x7f4ad4149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.880+0000 7f4ad91d3640 1 -- 172.21.15.112:0/993014335 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4ad40c5ca0 con 0x7f4ad4155150 2024-04-25T01:32:06.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.881+0000 7f4ad91d3640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ad4155150 0x7f4ad4149770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4ac402a0c0 tx=0x7f4ac4008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.881+0000 7f4aca7fc640 1 -- 172.21.15.112:0/993014335 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4037040 con 0x7f4ad4155150 2024-04-25T01:32:06.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.881+0000 7f4aca7fc640 1 -- 172.21.15.112:0/993014335 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4899 (secure 0 0 0) 0x7f4ac4049e50 con 0x7f4ad4155150 2024-04-25T01:32:06.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.881+0000 7f4aca7fc640 1 -- 172.21.15.112:0/993014335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ad4155150 msgr2=0x7f4ad4149770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.881+0000 7f4aca7fc640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ad4155150 0x7f4ad4149770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4ac402a0c0 tx=0x7f4ac4008030 comp rx=0 tx=0).stop 2024-04-25T01:32:06.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.881+0000 7f4aca7fc640 1 -- 172.21.15.112:0/993014335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ad4155150 msgr2=0x7f4ad4149770 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:32:06.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.881+0000 7f4aca7fc640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ad4155150 0x7f4ad4149770 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.859 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:32:06.859 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:32:06.859 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:32:06.860 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:32:06.860 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 46, 2024-04-25T01:32:06.860 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 115620, 2024-04-25T01:32:06.860 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:32:06.860 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:32:06.860 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:32:06.860 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:32:06.860 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:06.860 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:32:06.860 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:32:06.860 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:32:06.860 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:32:06.860 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:06.860 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:06.861 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:06.861 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:06.861 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:06.861 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:06.861 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:06.861 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:06.861 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:06.861 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:06.861 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:06.861 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:06.861 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:32:06.861 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:32:06.861 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:06.861 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:06.862 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:06.862 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:06.862 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:06.862 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:06.862 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:06.862 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:06.862 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:06.862 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:06.862 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:06.862 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:06.862 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.862 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:32:06.862 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:32:06.862 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:32:06.862 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:32:06.863 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:32:06.863 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:32:06.863 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:32:06.863 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:32:06.863 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:32:06.863 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:32:06.863 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:32:06.863 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:32:06.863 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:32:06.863 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:32:06.863 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:32:06.863 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:32:06.863 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:32:06.863 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:06.864 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:06.864 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:06.864 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:06.864 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:06.864 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:06.864 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:06.864 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:06.864 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:06.864 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:06.864 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:06.864 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:06.864 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:32:06.864 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.864 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:32:06.864 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.865 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:32:06.865 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:32:06.865 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:06.865 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:06.865 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.865 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:06.865 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:32:06.865 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:06.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.865 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.865 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:06.865 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:32:06.865 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:06.865 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.865 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:06.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.866 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:32:06.866 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:32:06.866 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:06.866 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:06.866 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:06.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.866 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.866 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:32:06.866 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:32:06.866 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:06.866 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:06.866 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.867 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:06.867 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:32:06.867 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:06.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.867 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.867 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:06.867 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:32:06.867 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:06.867 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.867 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:06.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.867 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:32:06.867 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:32:06.867 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:06.867 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:06.868 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:06.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.868 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.868 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:32:06.868 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:32:06.868 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:06.868 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:06.868 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.868 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:06.868 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:32:06.868 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:06.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.868 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.868 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:06.869 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:32:06.869 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:06.869 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.869 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:06.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.869 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:32:06.869 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:32:06.869 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:06.869 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:06.869 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:06.869 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.869 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:06.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.869 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:32:06.869 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:32:06.870 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.870 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:06.870 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:06.870 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:32:06.870 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.870 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:06.870 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:32:06.870 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.870 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:06.870 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:06.870 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:32:06.870 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.870 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:06.871 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:32:06.871 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.871 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:06.871 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:06.871 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:32:06.871 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.871 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:06.871 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:32:06.871 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:06.871 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:06.871 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:06.871 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:32:06.871 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:06.871 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:06.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:06.872 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:32:06.872 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:32:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.882+0000 7f4adbc38640 1 -- 172.21.15.112:0/993014335 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f4ab806be30 msgr2=0x7f4ab806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.882+0000 7f4adbc38640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f4ab806be30 0x7f4ab806e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4ad41557c0 tx=0x7f4acc00b040 comp rx=0 tx=0).stop 2024-04-25T01:32:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.882+0000 7f4adbc38640 1 -- 172.21.15.112:0/993014335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ad40c72b0 msgr2=0x7f4ad41eeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.882+0000 7f4adbc38640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ad40c72b0 0x7f4ad41eeb10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000c500 tx=0x7f4ad000c9c0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.882+0000 7f4ada1d5640 1 -- 172.21.15.112:0/993014335 reap_dead start 2024-04-25T01:32:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.882+0000 7f4adbc38640 1 -- 172.21.15.112:0/993014335 shutdown_connections 2024-04-25T01:32:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.882+0000 7f4adbc38640 1 --2- 172.21.15.112:0/993014335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ad40c72b0 0x7f4ad41eeb10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.882+0000 7f4adbc38640 1 -- 172.21.15.112:0/993014335 >> 172.21.15.112:0/993014335 conn(0x7f4ad40cb170 msgr2=0x7f4ad40ab1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:06.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.882+0000 7f4adbc38640 1 -- 172.21.15.112:0/993014335 shutdown_connections 2024-04-25T01:32:06.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:06.882+0000 7f4adbc38640 1 -- 172.21.15.112:0/993014335 wait complete. 2024-04-25T01:32:06.874 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:32:06.874 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T01:32:06.874 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T01:32:06.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.001+0000 7fca9d5b7640 1 Processor -- start 2024-04-25T01:32:06.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.001+0000 7fca9d5b7640 1 -- start start 2024-04-25T01:32:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.002+0000 7fca9d5b7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca9806b3d0 0x7fca98158e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.002+0000 7fca9d5b7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fca981560d0 0x7fca98152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.002+0000 7fca9d5b7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca980c2c30 con 0x7fca980a9520 2024-04-25T01:32:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.002+0000 7fca9d5b7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fca980c2f10 con 0x7fca981560d0 2024-04-25T01:32:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.002+0000 7fca9d5b7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fca980c31f0 con 0x7fca9806b3d0 2024-04-25T01:32:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.002+0000 7fca977fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fca981560d0 0x7fca98152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.002+0000 7fca967fc640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fca980a9520 0x7fca980c3510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38048/0 (socket says 172.21.15.112:38048) 2024-04-25T01:32:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.002+0000 7fca967fc640 1 -- 172.21.15.112:0/2971594264 learned_addr learned my addr 172.21.15.112:0/2971594264 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:32:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.002+0000 7fca977fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fca981560d0 0x7fca98152ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54794/0 (socket says 172.21.15.112:54794) 2024-04-25T01:32:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.002+0000 7fca96ffd640 1 --2- 172.21.15.112:0/2971594264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca9806b3d0 0x7fca98158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.002+0000 7fca977fe640 1 -- 172.21.15.112:0/2971594264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca9806b3d0 msgr2=0x7fca98158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:32:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.002+0000 7fca977fe640 1 --2- 172.21.15.112:0/2971594264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca9806b3d0 0x7fca98158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.002+0000 7fca977fe640 1 -- 172.21.15.112:0/2971594264 >> v1:172.21.15.112:6789/0 conn(0x7fca980a9520 legacy=0x7fca980c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.002+0000 7fca977fe640 1 -- 172.21.15.112:0/2971594264 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca98155bb0 con 0x7fca981560d0 2024-04-25T01:32:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.002+0000 7fca95ffb640 1 -- 172.21.15.112:0/2971594264 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3396829301 0 0) 0x7fca980c2c30 con 0x7fca980a9520 2024-04-25T01:32:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.003+0000 7fca977fe640 1 --2- 172.21.15.112:0/2971594264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca981560d0 0x7fca98152ba0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fca90009d90 tx=0x7fca9002a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b15f82b325abbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.003+0000 7fca95ffb640 1 -- 172.21.15.112:0/2971594264 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90007740 con 0x7fca981560d0 2024-04-25T01:32:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.003+0000 7fca95ffb640 1 -- 172.21.15.112:0/2971594264 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca900078e0 con 0x7fca981560d0 2024-04-25T01:32:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.003+0000 7fca95ffb640 1 -- 172.21.15.112:0/2971594264 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90007cb0 con 0x7fca981560d0 2024-04-25T01:32:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.004+0000 7fca9d5b7640 1 -- 172.21.15.112:0/2971594264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca981560d0 msgr2=0x7fca98152ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.004+0000 7fca9d5b7640 1 --2- 172.21.15.112:0/2971594264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca981560d0 0x7fca98152ba0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fca90009d90 tx=0x7fca9002a530 comp rx=0 tx=0).stop 2024-04-25T01:32:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.004+0000 7fca9d5b7640 1 -- 172.21.15.112:0/2971594264 shutdown_connections 2024-04-25T01:32:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.004+0000 7fca9d5b7640 1 --2- 172.21.15.112:0/2971594264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca981560d0 0x7fca98152ba0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.004+0000 7fca9d5b7640 1 --2- 172.21.15.112:0/2971594264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca9806b3d0 0x7fca98158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.004+0000 7fca9d5b7640 1 -- 172.21.15.112:0/2971594264 >> 172.21.15.112:0/2971594264 conn(0x7fca980c4c00 msgr2=0x7fca980c5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.004+0000 7fca9d5b7640 1 -- 172.21.15.112:0/2971594264 shutdown_connections 2024-04-25T01:32:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.004+0000 7fca9d5b7640 1 -- 172.21.15.112:0/2971594264 wait complete. 2024-04-25T01:32:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.006+0000 7fca9d5b7640 1 Processor -- start 2024-04-25T01:32:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.006+0000 7fca9d5b7640 1 -- start start 2024-04-25T01:32:06.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca9d5b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca9806b3d0 0x7fca9814e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca9d5b7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca980a9520 0x7fca9814ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca9d5b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca981560d0 0x7fca980ab470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca9d5b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fca9806b930 con 0x7fca9806b3d0 2024-04-25T01:32:06.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca9d5b7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fca9806b7b0 con 0x7fca981560d0 2024-04-25T01:32:06.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca9d5b7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fca98156530 con 0x7fca980a9520 2024-04-25T01:32:06.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca96ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca9806b3d0 0x7fca9814e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca96ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca9806b3d0 0x7fca9814e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54796/0 (socket says 172.21.15.112:54796) 2024-04-25T01:32:06.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca96ffd640 1 -- 172.21.15.112:0/3804099754 learned_addr learned my addr 172.21.15.112:0/3804099754 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:06.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca967fc640 1 --2- 172.21.15.112:0/3804099754 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca980a9520 0x7fca9814ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca977fe640 1 --2- 172.21.15.112:0/3804099754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca981560d0 0x7fca980ab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca96ffd640 1 -- 172.21.15.112:0/3804099754 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca980a9520 msgr2=0x7fca9814ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca96ffd640 1 --2- 172.21.15.112:0/3804099754 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca980a9520 0x7fca9814ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca96ffd640 1 -- 172.21.15.112:0/3804099754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca981560d0 msgr2=0x7fca980ab470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:06.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca96ffd640 1 --2- 172.21.15.112:0/3804099754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca981560d0 0x7fca980ab470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:06.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca96ffd640 1 -- 172.21.15.112:0/3804099754 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca981eb9a0 con 0x7fca9806b3d0 2024-04-25T01:32:06.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca96ffd640 1 --2- 172.21.15.112:0/3804099754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca9806b3d0 0x7fca9814e780 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fca8800e9b0 tx=0x7fca8800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca7f7fe640 1 -- 172.21.15.112:0/3804099754 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88017070 con 0x7fca9806b3d0 2024-04-25T01:32:06.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca7f7fe640 1 -- 172.21.15.112:0/3804099754 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca8800c9a0 con 0x7fca9806b3d0 2024-04-25T01:32:06.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca7f7fe640 1 -- 172.21.15.112:0/3804099754 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8800cd90 con 0x7fca9806b3d0 2024-04-25T01:32:06.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.007+0000 7fca9d5b7640 1 -- 172.21.15.112:0/3804099754 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca981e2000 con 0x7fca9806b3d0 2024-04-25T01:32:06.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.008+0000 7fca9d5b7640 1 -- 172.21.15.112:0/3804099754 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca981ebc80 con 0x7fca9806b3d0 2024-04-25T01:32:06.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.009+0000 7fca9d5b7640 1 -- 172.21.15.112:0/3804099754 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca980689b0 con 0x7fca9806b3d0 2024-04-25T01:32:06.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.009+0000 7fca7f7fe640 1 -- 172.21.15.112:0/3804099754 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fca8801a070 con 0x7fca9806b3d0 2024-04-25T01:32:06.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.009+0000 7fca7f7fe640 1 --2- 172.21.15.112:0/3804099754 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fca8406be60 0x7fca8406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:06.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.009+0000 7fca967fc640 1 --2- 172.21.15.112:0/3804099754 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fca8406be60 0x7fca8406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:06.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.009+0000 7fca7f7fe640 1 -- 172.21.15.112:0/3804099754 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fca8808b440 con 0x7fca9806b3d0 2024-04-25T01:32:06.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.010+0000 7fca967fc640 1 --2- 172.21.15.112:0/3804099754 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fca8406be60 0x7fca8406e320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fca981ef420 tx=0x7fca8c007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:06.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.014+0000 7fca7f7fe640 1 -- 172.21.15.112:0/3804099754 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fca88059490 con 0x7fca9806b3d0 2024-04-25T01:32:07.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.304+0000 7fca9d5b7640 1 -- 172.21.15.112:0/3804099754 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fca98072fb0 con 0x7fca9806b3d0 2024-04-25T01:32:07.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.305+0000 7fca7f7fe640 1 -- 172.21.15.112:0/3804099754 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fca8805d790 con 0x7fca9806b3d0 2024-04-25T01:32:07.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.309+0000 7fca9d5b7640 1 -- 172.21.15.112:0/3804099754 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fca8406be60 msgr2=0x7fca8406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:07.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.309+0000 7fca9d5b7640 1 --2- 172.21.15.112:0/3804099754 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fca8406be60 0x7fca8406e320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fca981ef420 tx=0x7fca8c007d50 comp rx=0 tx=0).stop 2024-04-25T01:32:07.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.309+0000 7fca9d5b7640 1 -- 172.21.15.112:0/3804099754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca9806b3d0 msgr2=0x7fca9814e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:07.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.309+0000 7fca9d5b7640 1 --2- 172.21.15.112:0/3804099754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca9806b3d0 0x7fca9814e780 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fca8800e9b0 tx=0x7fca8800ee70 comp rx=0 tx=0).stop 2024-04-25T01:32:07.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.309+0000 7fca9d5b7640 1 -- 172.21.15.112:0/3804099754 shutdown_connections 2024-04-25T01:32:07.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.309+0000 7fca9d5b7640 1 --2- 172.21.15.112:0/3804099754 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fca8406be60 0x7fca8406e320 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:07.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.309+0000 7fca9d5b7640 1 --2- 172.21.15.112:0/3804099754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca981560d0 0x7fca980ab470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:07.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.309+0000 7fca9d5b7640 1 --2- 172.21.15.112:0/3804099754 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca980a9520 0x7fca9814ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:07.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.309+0000 7fca9d5b7640 1 --2- 172.21.15.112:0/3804099754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca9806b3d0 0x7fca9814e780 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:07.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.309+0000 7fca9d5b7640 1 -- 172.21.15.112:0/3804099754 >> 172.21.15.112:0/3804099754 conn(0x7fca980c4c00 msgr2=0x7fca98071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:07.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.309+0000 7fca9d5b7640 1 -- 172.21.15.112:0/3804099754 shutdown_connections 2024-04-25T01:32:07.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:07.309+0000 7fca9d5b7640 1 -- 172.21.15.112:0/3804099754 wait complete. 2024-04-25T01:32:07.300 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T01:32:17.300 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:32:17.301 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:32:17.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.433+0000 7fd295cba640 1 Processor -- start 2024-04-25T01:32:17.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.433+0000 7fd295cba640 1 -- start start 2024-04-25T01:32:17.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.434+0000 7fd295cba640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd2900ab440 0x7fd290151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:17.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.434+0000 7fd295cba640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2900ab820 con 0x7fd290155780 2024-04-25T01:32:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.434+0000 7fd295cba640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2901552a0 con 0x7fd2900721f0 2024-04-25T01:32:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.434+0000 7fd295cba640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fd290152290 con 0x7fd2900ab440 2024-04-25T01:32:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.434+0000 7fd28f7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd2900ab440 0x7fd290151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.434+0000 7fd28f7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd2900ab440 0x7fd290151cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55642/0 (socket says 172.21.15.112:55642) 2024-04-25T01:32:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.434+0000 7fd28ffff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd290155780 0x7fd290155bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:40390/0 (socket says 172.21.15.112:40390) 2024-04-25T01:32:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.434+0000 7fd28f7fe640 1 -- 172.21.15.112:0/2734053970 learned_addr learned my addr 172.21.15.112:0/2734053970 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.435+0000 7fd28f7fe640 1 -- 172.21.15.112:0/2734053970 >> v1:172.21.15.187:6789/0 conn(0x7fd2900721f0 legacy=0x7fd2900c2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.435+0000 7fd28f7fe640 1 -- 172.21.15.112:0/2734053970 >> v1:172.21.15.112:6789/0 conn(0x7fd290155780 legacy=0x7fd290155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.435+0000 7fd28e7fc640 1 -- 172.21.15.112:0/2734053970 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3226650020 0 0) 0x7fd2901552a0 con 0x7fd2900721f0 2024-04-25T01:32:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.435+0000 7fd28f7fe640 1 -- 172.21.15.112:0/2734053970 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd290152570 con 0x7fd2900ab440 2024-04-25T01:32:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.435+0000 7fd28f7fe640 1 --2- 172.21.15.112:0/2734053970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2900ab440 0x7fd290151cb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd280009870 tx=0x7fd28002a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938e2ed7e9147556 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.435+0000 7fd28e7fc640 1 -- 172.21.15.112:0/2734053970 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd280037070 con 0x7fd2900ab440 2024-04-25T01:32:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.436+0000 7fd28e7fc640 1 -- 172.21.15.112:0/2734053970 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd28002ad60 con 0x7fd2900ab440 2024-04-25T01:32:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.436+0000 7fd28e7fc640 1 -- 172.21.15.112:0/2734053970 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd280032660 con 0x7fd2900ab440 2024-04-25T01:32:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.436+0000 7fd295cba640 1 -- 172.21.15.112:0/2734053970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2900ab440 msgr2=0x7fd290151cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.436+0000 7fd295cba640 1 --2- 172.21.15.112:0/2734053970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2900ab440 0x7fd290151cb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd280009870 tx=0x7fd28002a0d0 comp rx=0 tx=0).stop 2024-04-25T01:32:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.436+0000 7fd295cba640 1 -- 172.21.15.112:0/2734053970 shutdown_connections 2024-04-25T01:32:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.436+0000 7fd295cba640 1 --2- 172.21.15.112:0/2734053970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2900ab440 0x7fd290151cb0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.436+0000 7fd295cba640 1 -- 172.21.15.112:0/2734053970 >> 172.21.15.112:0/2734053970 conn(0x7fd2900c6f80 msgr2=0x7fd2900c7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.436+0000 7fd295cba640 1 -- 172.21.15.112:0/2734053970 shutdown_connections 2024-04-25T01:32:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.436+0000 7fd295cba640 1 -- 172.21.15.112:0/2734053970 wait complete. 2024-04-25T01:32:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.438+0000 7fd295cba640 1 Processor -- start 2024-04-25T01:32:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.438+0000 7fd295cba640 1 -- start start 2024-04-25T01:32:17.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.439+0000 7fd295cba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2900721f0 0x7fd29014cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:17.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.439+0000 7fd295cba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2900ab440 0x7fd29014a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:17.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.439+0000 7fd295cba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd290155780 0x7fd29014a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:17.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.439+0000 7fd295cba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd29006c8a0 con 0x7fd290155780 2024-04-25T01:32:17.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.439+0000 7fd295cba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd290152b30 con 0x7fd2900721f0 2024-04-25T01:32:17.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.439+0000 7fd295cba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd2900c2920 con 0x7fd2900ab440 2024-04-25T01:32:17.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.439+0000 7fd28ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd290155780 0x7fd29014a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:17.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.439+0000 7fd28ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd290155780 0x7fd29014a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55644/0 (socket says 172.21.15.112:55644) 2024-04-25T01:32:17.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.439+0000 7fd28ffff640 1 -- 172.21.15.112:0/1216291533 learned_addr learned my addr 172.21.15.112:0/1216291533 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:17.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.439+0000 7fd28f7fe640 1 --2- 172.21.15.112:0/1216291533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2900721f0 0x7fd29014cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:17.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.439+0000 7fd28effd640 1 --2- 172.21.15.112:0/1216291533 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2900ab440 0x7fd29014a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:17.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.439+0000 7fd28ffff640 1 -- 172.21.15.112:0/1216291533 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2900ab440 msgr2=0x7fd29014a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:17.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.439+0000 7fd28ffff640 1 --2- 172.21.15.112:0/1216291533 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2900ab440 0x7fd29014a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:17.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.439+0000 7fd28ffff640 1 -- 172.21.15.112:0/1216291533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2900721f0 msgr2=0x7fd29014cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:17.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.439+0000 7fd28ffff640 1 --2- 172.21.15.112:0/1216291533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2900721f0 0x7fd29014cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:17.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.439+0000 7fd28ffff640 1 -- 172.21.15.112:0/1216291533 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd290149a00 con 0x7fd290155780 2024-04-25T01:32:17.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.439+0000 7fd28ffff640 1 --2- 172.21.15.112:0/1216291533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd290155780 0x7fd29014a9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd28800caf0 tx=0x7fd2880020b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:17.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.439+0000 7fd277fff640 1 -- 172.21.15.112:0/1216291533 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288018070 con 0x7fd290155780 2024-04-25T01:32:17.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.439+0000 7fd277fff640 1 -- 172.21.15.112:0/1216291533 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd28800a6e0 con 0x7fd290155780 2024-04-25T01:32:17.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.439+0000 7fd277fff640 1 -- 172.21.15.112:0/1216291533 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28800aad0 con 0x7fd290155780 2024-04-25T01:32:17.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.440+0000 7fd295cba640 1 -- 172.21.15.112:0/1216291533 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd290148570 con 0x7fd290155780 2024-04-25T01:32:17.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.440+0000 7fd295cba640 1 -- 172.21.15.112:0/1216291533 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd290149f50 con 0x7fd290155780 2024-04-25T01:32:17.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.441+0000 7fd295cba640 1 -- 172.21.15.112:0/1216291533 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2900682a0 con 0x7fd290155780 2024-04-25T01:32:17.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.441+0000 7fd277fff640 1 -- 172.21.15.112:0/1216291533 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd28801b070 con 0x7fd290155780 2024-04-25T01:32:17.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.442+0000 7fd277fff640 1 --2- 172.21.15.112:0/1216291533 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd27c06be30 0x7fd27c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:17.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.442+0000 7fd277fff640 1 -- 172.21.15.112:0/1216291533 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fd28808a890 con 0x7fd290155780 2024-04-25T01:32:17.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.442+0000 7fd28f7fe640 1 --2- 172.21.15.112:0/1216291533 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd27c06be30 0x7fd27c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:17.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.442+0000 7fd28f7fe640 1 --2- 172.21.15.112:0/1216291533 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd27c06be30 0x7fd27c06e2f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd280034000 tx=0x7fd28002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:17.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.447+0000 7fd277fff640 1 -- 172.21.15.112:0/1216291533 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd28800ac70 con 0x7fd290155780 2024-04-25T01:32:17.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.758+0000 7fd295cba640 1 -- 172.21.15.112:0/1216291533 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd290071fd0 con 0x7fd290155780 2024-04-25T01:32:17.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.759+0000 7fd277fff640 1 -- 172.21.15.112:0/1216291533 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7fd2880588e0 con 0x7fd290155780 2024-04-25T01:32:17.737 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:32:17.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.761+0000 7fd295cba640 1 -- 172.21.15.112:0/1216291533 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd27c06be30 msgr2=0x7fd27c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:17.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.761+0000 7fd295cba640 1 --2- 172.21.15.112:0/1216291533 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd27c06be30 0x7fd27c06e2f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd280034000 tx=0x7fd28002e000 comp rx=0 tx=0).stop 2024-04-25T01:32:17.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.761+0000 7fd295cba640 1 -- 172.21.15.112:0/1216291533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd290155780 msgr2=0x7fd29014a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:17.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.761+0000 7fd295cba640 1 --2- 172.21.15.112:0/1216291533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd290155780 0x7fd29014a9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd28800caf0 tx=0x7fd2880020b0 comp rx=0 tx=0).stop 2024-04-25T01:32:17.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.761+0000 7fd295cba640 1 -- 172.21.15.112:0/1216291533 shutdown_connections 2024-04-25T01:32:17.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.761+0000 7fd295cba640 1 --2- 172.21.15.112:0/1216291533 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd27c06be30 0x7fd27c06e2f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:17.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.761+0000 7fd295cba640 1 --2- 172.21.15.112:0/1216291533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd290155780 0x7fd29014a9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:17.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.761+0000 7fd295cba640 1 --2- 172.21.15.112:0/1216291533 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2900ab440 0x7fd29014a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:17.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.761+0000 7fd295cba640 1 --2- 172.21.15.112:0/1216291533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2900721f0 0x7fd29014cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:17.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.761+0000 7fd295cba640 1 -- 172.21.15.112:0/1216291533 >> 172.21.15.112:0/1216291533 conn(0x7fd2900c6f80 msgr2=0x7fd290155bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:17.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.761+0000 7fd295cba640 1 -- 172.21.15.112:0/1216291533 shutdown_connections 2024-04-25T01:32:17.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.761+0000 7fd295cba640 1 -- 172.21.15.112:0/1216291533 wait complete. 2024-04-25T01:32:17.749 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:32:17.749 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T01:32:17.750 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:32:17.750 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:32:17.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.887+0000 7f87b742a640 1 Processor -- start 2024-04-25T01:32:17.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.887+0000 7f87b742a640 1 -- start start 2024-04-25T01:32:17.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.888+0000 7f87b742a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87b00c42e0 0x7f87b006dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:17.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.888+0000 7f87b742a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f87b0156290 0x7f87b0158e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:17.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.888+0000 7f87b742a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87b006e130 con 0x7f87b01593d0 2024-04-25T01:32:17.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.888+0000 7f87b742a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f87b01598c0 con 0x7f87b0156290 2024-04-25T01:32:17.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.888+0000 7f87b742a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f87b0159ba0 con 0x7f87b00c42e0 2024-04-25T01:32:17.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.888+0000 7f87b49c5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f87b0156290 0x7f87b0158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:17.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.888+0000 7f87b51c6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87b00c42e0 0x7f87b006dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:17.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.888+0000 7f87b49c5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f87b0156290 0x7f87b0158e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:35650/0 (socket says 172.21.15.112:35650) 2024-04-25T01:32:17.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.889+0000 7f87b49c5640 1 -- 172.21.15.112:0/698361397 learned_addr learned my addr 172.21.15.112:0/698361397 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:17.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.889+0000 7f87b49c5640 1 -- 172.21.15.112:0/698361397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87b00c42e0 msgr2=0x7f87b006dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:32:17.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.889+0000 7f87b49c5640 1 --2- 172.21.15.112:0/698361397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87b00c42e0 0x7f87b006dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:17.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.889+0000 7f87b49c5640 1 -- 172.21.15.112:0/698361397 >> v1:172.21.15.187:6789/0 conn(0x7f87b01593d0 legacy=0x7f87b0152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:17.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.889+0000 7f87b49c5640 1 -- 172.21.15.112:0/698361397 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b0155a90 con 0x7f87b0156290 2024-04-25T01:32:17.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.889+0000 7f87b51c6640 1 --2- 172.21.15.112:0/698361397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87b00c42e0 0x7f87b006dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:32:17.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.890+0000 7f87b49c5640 1 --2- 172.21.15.112:0/698361397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87b0156290 0x7f87b0158e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009870 tx=0x7f87a002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d5ea73dccf267f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:17.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.890+0000 7f87a7fff640 1 -- 172.21.15.112:0/698361397 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a0037070 con 0x7f87b0156290 2024-04-25T01:32:17.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.890+0000 7f87a7fff640 1 -- 172.21.15.112:0/698361397 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87a0032440 con 0x7f87b0156290 2024-04-25T01:32:17.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.890+0000 7f87a7fff640 1 -- 172.21.15.112:0/698361397 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a0032810 con 0x7f87b0156290 2024-04-25T01:32:17.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.890+0000 7f87b742a640 1 -- 172.21.15.112:0/698361397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87b0156290 msgr2=0x7f87b0158e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:17.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.890+0000 7f87b742a640 1 --2- 172.21.15.112:0/698361397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87b0156290 0x7f87b0158e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009870 tx=0x7f87a002a250 comp rx=0 tx=0).stop 2024-04-25T01:32:17.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.890+0000 7f87b742a640 1 -- 172.21.15.112:0/698361397 shutdown_connections 2024-04-25T01:32:17.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.890+0000 7f87b742a640 1 --2- 172.21.15.112:0/698361397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87b0156290 0x7f87b0158e90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:17.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.890+0000 7f87b742a640 1 --2- 172.21.15.112:0/698361397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87b00c42e0 0x7f87b006dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:17.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.890+0000 7f87b742a640 1 -- 172.21.15.112:0/698361397 >> 172.21.15.112:0/698361397 conn(0x7f87b00c4a90 msgr2=0x7f87b00c4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:17.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.890+0000 7f87b742a640 1 -- 172.21.15.112:0/698361397 shutdown_connections 2024-04-25T01:32:17.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.890+0000 7f87b742a640 1 -- 172.21.15.112:0/698361397 wait complete. 2024-04-25T01:32:17.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.892+0000 7f87b742a640 1 Processor -- start 2024-04-25T01:32:17.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.892+0000 7f87b742a640 1 -- start start 2024-04-25T01:32:17.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87b742a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b00c42e0 0x7f87b014c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:17.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87b742a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87b0156290 0x7f87b0157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:17.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87b742a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87b01593d0 0x7f87b0149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:17.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87b742a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f87b0156850 con 0x7f87b00c42e0 2024-04-25T01:32:17.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87b742a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f87b01566d0 con 0x7f87b01593d0 2024-04-25T01:32:17.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87b742a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f87b01533b0 con 0x7f87b0156290 2024-04-25T01:32:17.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87b51c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b00c42e0 0x7f87b014c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:17.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87b49c5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87b0156290 0x7f87b0157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:17.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87b51c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b00c42e0 0x7f87b014c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38374/0 (socket says 172.21.15.112:38374) 2024-04-25T01:32:17.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87b51c6640 1 -- 172.21.15.112:0/1311536779 learned_addr learned my addr 172.21.15.112:0/1311536779 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:17.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87b59c7640 1 --2- 172.21.15.112:0/1311536779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87b01593d0 0x7f87b0149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:17.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87b51c6640 1 -- 172.21.15.112:0/1311536779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87b0156290 msgr2=0x7f87b0157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:17.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87b51c6640 1 --2- 172.21.15.112:0/1311536779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87b0156290 0x7f87b0157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:17.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87b51c6640 1 -- 172.21.15.112:0/1311536779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87b01593d0 msgr2=0x7f87b0149c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:17.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87b51c6640 1 --2- 172.21.15.112:0/1311536779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87b01593d0 0x7f87b0149c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:17.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87b51c6640 1 -- 172.21.15.112:0/1311536779 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b01ef650 con 0x7f87b00c42e0 2024-04-25T01:32:17.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87b49c5640 1 --2- 172.21.15.112:0/1311536779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87b0156290 0x7f87b0157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:32:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87b51c6640 1 --2- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b00c42e0 0x7f87b014c240 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00ec10 tx=0x7f87ac00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87a5ffb640 1 -- 172.21.15.112:0/1311536779 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac017070 con 0x7f87b00c42e0 2024-04-25T01:32:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87a5ffb640 1 -- 172.21.15.112:0/1311536779 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87ac00cc30 con 0x7f87b00c42e0 2024-04-25T01:32:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87a5ffb640 1 -- 172.21.15.112:0/1311536779 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac013470 con 0x7f87b00c42e0 2024-04-25T01:32:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.893+0000 7f87b742a640 1 -- 172.21.15.112:0/1311536779 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b01eb620 con 0x7f87b00c42e0 2024-04-25T01:32:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.894+0000 7f87b742a640 1 -- 172.21.15.112:0/1311536779 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b01ebbe0 con 0x7f87b00c42e0 2024-04-25T01:32:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.895+0000 7f87b742a640 1 --2- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b01ec100 0x7f87b014f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.895+0000 7f87b742a640 1 -- 172.21.15.112:0/1311536779 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f87b01eb480 con 0x7f87b01ec100 2024-04-25T01:32:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.895+0000 7f87b49c5640 1 --2- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b01ec100 0x7f87b014f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.895+0000 7f87b49c5640 1 -- 172.21.15.112:0/1311536779 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f87b01ef400 con 0x7f87b01ec100 2024-04-25T01:32:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.895+0000 7f87b49c5640 1 --2- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b01ec100 0x7f87b014f2f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f87b00c2840 tx=0x7f87a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.895+0000 7f87a5ffb640 1 -- 172.21.15.112:0/1311536779 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f87ac01a070 con 0x7f87b00c42e0 2024-04-25T01:32:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.896+0000 7f87a5ffb640 1 --2- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f879406be30 0x7f879406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.896+0000 7f87a5ffb640 1 -- 172.21.15.112:0/1311536779 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f87ac08b4e0 con 0x7f87b00c42e0 2024-04-25T01:32:17.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.896+0000 7f87b59c7640 1 --2- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f879406be30 0x7f879406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:17.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.896+0000 7f87a5ffb640 1 -- 172.21.15.112:0/1311536779 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a0037040 con 0x7f87b01ec100 2024-04-25T01:32:17.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.896+0000 7f87a5ffb640 1 -- 172.21.15.112:0/1311536779 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f87a0032440 con 0x7f87b01ec100 2024-04-25T01:32:17.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.896+0000 7f87a5ffb640 1 -- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b01ec100 msgr2=0x7f87b014f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:17.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.896+0000 7f87a5ffb640 1 --2- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b01ec100 0x7f87b014f2f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f87b00c2840 tx=0x7f87a00046c0 comp rx=0 tx=0).stop 2024-04-25T01:32:17.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.896+0000 7f87a5ffb640 1 -- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b01ec100 msgr2=0x7f87b014f2f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:32:17.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.896+0000 7f87a5ffb640 1 --2- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b01ec100 0x7f87b014f2f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:17.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.896+0000 7f87b59c7640 1 --2- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f879406be30 0x7f879406e2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f87b00c7dd0 tx=0x7f87a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:17.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.906+0000 7f87b742a640 1 --2- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b014a150 0x7f87b014f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:17.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.906+0000 7f87b742a640 1 -- 172.21.15.112:0/1311536779 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f87b01ef930 con 0x7f87b014a150 2024-04-25T01:32:17.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.906+0000 7f87b49c5640 1 --2- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b014a150 0x7f87b014f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:17.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.906+0000 7f87b49c5640 1 -- 172.21.15.112:0/1311536779 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f87b0075980 con 0x7f87b014a150 2024-04-25T01:32:17.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.906+0000 7f87b49c5640 1 --2- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b014a150 0x7f87b014f830 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009870 tx=0x7f87a0008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:17.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.906+0000 7f87a5ffb640 1 -- 172.21.15.112:0/1311536779 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a0037040 con 0x7f87b014a150 2024-04-25T01:32:17.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.907+0000 7f87a5ffb640 1 -- 172.21.15.112:0/1311536779 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4561 (secure 0 0 0) 0x7f87a0003ca0 con 0x7f87b014a150 2024-04-25T01:32:17.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.907+0000 7f87a5ffb640 1 -- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b014a150 msgr2=0x7f87b014f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:17.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.907+0000 7f87a5ffb640 1 --2- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b014a150 0x7f87b014f830 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009870 tx=0x7f87a0008030 comp rx=0 tx=0).stop 2024-04-25T01:32:17.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.907+0000 7f87a5ffb640 1 -- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b014a150 msgr2=0x7f87b014f830 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:32:17.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.907+0000 7f87a5ffb640 1 --2- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b014a150 0x7f87b014f830 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:17.885 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:32:17.885 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:32:17.885 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:32:17.885 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:32:17.885 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 46, 2024-04-25T01:32:17.885 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 18671, 2024-04-25T01:32:17.885 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:32:17.885 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:32:17.885 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:32:17.885 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:32:17.885 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:17.886 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 12, 2024-04-25T01:32:17.886 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:32:17.886 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:32:17.886 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:32:17.886 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:17.886 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:17.886 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:17.886 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:17.886 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:17.886 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:17.886 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:17.886 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:17.886 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:17.886 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:17.887 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:17.887 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:17.887 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:32:17.887 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:32:17.887 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:17.887 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:17.887 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:17.887 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:17.887 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:17.887 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:17.887 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:17.887 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:17.887 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:17.887 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:17.887 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:17.888 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:17.888 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:17.888 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:32:17.888 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:32:17.888 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:32:17.888 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:32:17.888 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:32:17.888 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:32:17.888 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:32:17.888 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:32:17.888 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:32:17.888 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:32:17.888 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:32:17.888 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:32:17.888 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:32:17.889 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:32:17.889 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:32:17.889 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:32:17.889 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:32:17.889 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:17.889 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:17.889 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:17.889 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:17.889 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:17.889 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:17.889 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:17.889 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:17.889 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:17.890 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:17.890 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:17.890 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:17.890 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:32:17.890 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:17.890 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:32:17.890 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:17.890 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:32:17.890 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:32:17.890 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:17.890 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:17.890 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:17.890 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:17.891 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:32:17.891 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:17.891 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:17.891 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:17.891 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:17.891 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:32:17.891 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:17.891 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:17.891 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:17.891 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:17.891 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:32:17.891 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:32:17.891 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:17.891 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:17.892 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:17.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:17.892 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:17.892 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:32:17.892 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:32:17.892 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:17.892 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:17.892 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:17.892 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:17.892 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:32:17.892 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:17.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:17.892 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:17.892 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:17.892 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:32:17.893 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:17.893 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:17.893 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:17.893 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:17.893 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:32:17.893 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:32:17.893 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:17.893 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:17.893 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:17.893 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:17.893 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:17.893 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:32:17.893 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:32:17.893 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:17.894 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:17.894 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:17.894 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:17.894 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:32:17.894 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:17.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:17.894 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:17.894 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:17.894 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:32:17.894 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:17.894 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:17.894 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:17.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:17.894 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:32:17.895 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:32:17.895 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:17.895 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:17.895 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:17.895 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:17.895 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:17.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:17.895 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:32:17.895 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:32:17.895 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:17.895 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:17.895 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:17.895 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:32:17.895 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:17.895 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:17.896 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:32:17.896 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:17.896 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:17.896 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:17.896 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:32:17.896 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:17.896 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:17.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:17.896 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:32:17.896 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:32:17.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.907+0000 7f87b742a640 1 -- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f879406be30 msgr2=0x7f879406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:17.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.907+0000 7f87b742a640 1 --2- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f879406be30 0x7f879406e2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f87b00c7dd0 tx=0x7f87a8009040 comp rx=0 tx=0).stop 2024-04-25T01:32:17.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.907+0000 7f87b742a640 1 -- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b00c42e0 msgr2=0x7f87b014c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:17.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.907+0000 7f87b742a640 1 --2- 172.21.15.112:0/1311536779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b00c42e0 0x7f87b014c240 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00ec10 tx=0x7f87ac00c520 comp rx=0 tx=0).stop 2024-04-25T01:32:17.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.907+0000 7f87b59c7640 1 -- 172.21.15.112:0/1311536779 reap_dead start 2024-04-25T01:32:17.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.907+0000 7f87b59c7640 1 -- 172.21.15.112:0/1311536779 reap_dead start 2024-04-25T01:32:17.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.907+0000 7f87b742a640 1 -- 172.21.15.112:0/1311536779 shutdown_connections 2024-04-25T01:32:17.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.907+0000 7f87b742a640 1 -- 172.21.15.112:0/1311536779 >> 172.21.15.112:0/1311536779 conn(0x7f87b00c4a90 msgr2=0x7f87b0150730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:17.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.907+0000 7f87b742a640 1 -- 172.21.15.112:0/1311536779 shutdown_connections 2024-04-25T01:32:17.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:17.907+0000 7f87b742a640 1 -- 172.21.15.112:0/1311536779 wait complete. 2024-04-25T01:32:17.898 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:32:17.899 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:32:17.899 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:32:18.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.031+0000 7f28b26bd640 1 Processor -- start 2024-04-25T01:32:18.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.032+0000 7f28b26bd640 1 -- start start 2024-04-25T01:32:18.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.032+0000 7f28b26bd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f28ac155b90 0x7f28ac155f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.032+0000 7f28b26bd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f28ac1564b0 0x7f28ac153c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.032+0000 7f28b26bd640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28ac06e010 con 0x7f28ac154180 2024-04-25T01:32:18.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.032+0000 7f28b26bd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f28ac1568f0 con 0x7f28ac155b90 2024-04-25T01:32:18.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.032+0000 7f28b26bd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f28ac0c36d0 con 0x7f28ac1564b0 2024-04-25T01:32:18.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.032+0000 7f28abfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f28ac155b90 0x7f28ac155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.032+0000 7f28abfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f28ac155b90 0x7f28ac155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38406/0 (socket says 172.21.15.112:38406) 2024-04-25T01:32:18.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.032+0000 7f28abfff640 1 -- 172.21.15.112:0/3120708904 learned_addr learned my addr 172.21.15.112:0/3120708904 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:18.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.033+0000 7f28ab7fe640 1 --2- 172.21.15.112:0/3120708904 >> v2:172.21.15.187:3300/0 conn(0x7f28ac1564b0 0x7f28ac153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.033+0000 7f28abfff640 1 -- 172.21.15.112:0/3120708904 >> v2:172.21.15.187:3300/0 conn(0x7f28ac1564b0 msgr2=0x7f28ac153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:32:18.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.033+0000 7f28abfff640 1 --2- 172.21.15.112:0/3120708904 >> v2:172.21.15.187:3300/0 conn(0x7f28ac1564b0 0x7f28ac153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.033+0000 7f28abfff640 1 -- 172.21.15.112:0/3120708904 >> v1:172.21.15.187:6789/0 conn(0x7f28ac154180 legacy=0x7f28ac1525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.033+0000 7f28abfff640 1 -- 172.21.15.112:0/3120708904 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ac0c39b0 con 0x7f28ac155b90 2024-04-25T01:32:18.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.034+0000 7f28abfff640 1 --2- 172.21.15.112:0/3120708904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ac155b90 0x7f28ac155f70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f28ac0c8f30 tx=0x7f289c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f584190f9105faf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:18.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.034+0000 7f28aaffd640 1 -- 172.21.15.112:0/3120708904 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c037070 con 0x7f28ac155b90 2024-04-25T01:32:18.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.034+0000 7f28aaffd640 1 -- 172.21.15.112:0/3120708904 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f289c032440 con 0x7f28ac155b90 2024-04-25T01:32:18.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.034+0000 7f28aaffd640 1 -- 172.21.15.112:0/3120708904 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c032810 con 0x7f28ac155b90 2024-04-25T01:32:18.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.034+0000 7f28b26bd640 1 -- 172.21.15.112:0/3120708904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ac155b90 msgr2=0x7f28ac155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.034+0000 7f28b26bd640 1 --2- 172.21.15.112:0/3120708904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ac155b90 0x7f28ac155f70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f28ac0c8f30 tx=0x7f289c02a1e0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.034+0000 7f28b26bd640 1 -- 172.21.15.112:0/3120708904 shutdown_connections 2024-04-25T01:32:18.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.034+0000 7f28b26bd640 1 --2- 172.21.15.112:0/3120708904 >> v2:172.21.15.187:3300/0 conn(0x7f28ac1564b0 0x7f28ac153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.034+0000 7f28b26bd640 1 --2- 172.21.15.112:0/3120708904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ac155b90 0x7f28ac155f70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.034+0000 7f28b26bd640 1 -- 172.21.15.112:0/3120708904 >> 172.21.15.112:0/3120708904 conn(0x7f28ac0c4a90 msgr2=0x7f28ac0c4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:18.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.034+0000 7f28b26bd640 1 -- 172.21.15.112:0/3120708904 shutdown_connections 2024-04-25T01:32:18.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.034+0000 7f28b26bd640 1 -- 172.21.15.112:0/3120708904 wait complete. 2024-04-25T01:32:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.036+0000 7f28b26bd640 1 Processor -- start 2024-04-25T01:32:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.036+0000 7f28b26bd640 1 -- start start 2024-04-25T01:32:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.037+0000 7f28b26bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ac154180 0x7f28ac147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.037+0000 7f28b26bd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28ac155b90 0x7f28ac152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.037+0000 7f28b26bd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28ac1564b0 0x7f28ac14e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.037+0000 7f28b26bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac0c33a0 con 0x7f28ac154180 2024-04-25T01:32:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.037+0000 7f28b26bd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac0c3f70 con 0x7f28ac1564b0 2024-04-25T01:32:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.037+0000 7f28b26bd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f28ac14fe50 con 0x7f28ac155b90 2024-04-25T01:32:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.037+0000 7f28abfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ac154180 0x7f28ac147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.037+0000 7f28abfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ac154180 0x7f28ac147430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38412/0 (socket says 172.21.15.112:38412) 2024-04-25T01:32:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.037+0000 7f28abfff640 1 -- 172.21.15.112:0/2346978624 learned_addr learned my addr 172.21.15.112:0/2346978624 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.037+0000 7f28ab7fe640 1 --2- 172.21.15.112:0/2346978624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28ac155b90 0x7f28ac152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.037+0000 7f28b0c5a640 1 --2- 172.21.15.112:0/2346978624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28ac1564b0 0x7f28ac14e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.037+0000 7f28abfff640 1 -- 172.21.15.112:0/2346978624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28ac155b90 msgr2=0x7f28ac152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.037+0000 7f28abfff640 1 --2- 172.21.15.112:0/2346978624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28ac155b90 0x7f28ac152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.037+0000 7f28abfff640 1 -- 172.21.15.112:0/2346978624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28ac1564b0 msgr2=0x7f28ac14e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.037+0000 7f28abfff640 1 --2- 172.21.15.112:0/2346978624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28ac1564b0 0x7f28ac14e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.037+0000 7f28abfff640 1 -- 172.21.15.112:0/2346978624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ac0ac4c0 con 0x7f28ac154180 2024-04-25T01:32:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.037+0000 7f28abfff640 1 --2- 172.21.15.112:0/2346978624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ac154180 0x7f28ac147430 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f289c02a220 tx=0x7f289c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:18.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.038+0000 7f28a8ff9640 1 -- 172.21.15.112:0/2346978624 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c037040 con 0x7f28ac154180 2024-04-25T01:32:18.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.038+0000 7f28a8ff9640 1 -- 172.21.15.112:0/2346978624 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f289c042070 con 0x7f28ac154180 2024-04-25T01:32:18.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.038+0000 7f28b26bd640 1 -- 172.21.15.112:0/2346978624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ac147970 con 0x7f28ac154180 2024-04-25T01:32:18.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.038+0000 7f28a8ff9640 1 -- 172.21.15.112:0/2346978624 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c030880 con 0x7f28ac154180 2024-04-25T01:32:18.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.038+0000 7f28b26bd640 1 -- 172.21.15.112:0/2346978624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ac0c6f50 con 0x7f28ac154180 2024-04-25T01:32:18.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.039+0000 7f28b26bd640 1 --2- 172.21.15.112:0/2346978624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28ac14ef30 0x7f28ac0c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.039+0000 7f28b26bd640 1 -- 172.21.15.112:0/2346978624 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac075980 con 0x7f28ac14ef30 2024-04-25T01:32:18.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.039+0000 7f28ab7fe640 1 --2- 172.21.15.112:0/2346978624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28ac14ef30 0x7f28ac0c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.039+0000 7f28ab7fe640 1 -- 172.21.15.112:0/2346978624 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f28ac0682a0 con 0x7f28ac14ef30 2024-04-25T01:32:18.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.039+0000 7f28a8ff9640 1 -- 172.21.15.112:0/2346978624 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f289c030a20 con 0x7f28ac154180 2024-04-25T01:32:18.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.039+0000 7f28ab7fe640 1 --2- 172.21.15.112:0/2346978624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28ac14ef30 0x7f28ac0c7150 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f28ac068e90 tx=0x7f28a0011d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:18.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.040+0000 7f28a8ff9640 1 --2- 172.21.15.112:0/2346978624 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f289806be30 0x7f289806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.040+0000 7f28a8ff9640 1 -- 172.21.15.112:0/2346978624 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f289c0ab860 con 0x7f28ac154180 2024-04-25T01:32:18.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.040+0000 7f28b0c5a640 1 --2- 172.21.15.112:0/2346978624 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f289806be30 0x7f289806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.040+0000 7f28a8ff9640 1 -- 172.21.15.112:0/2346978624 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a000fb60 con 0x7f28ac14ef30 2024-04-25T01:32:18.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.040+0000 7f28a8ff9640 1 -- 172.21.15.112:0/2346978624 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f28a000fd00 con 0x7f28ac14ef30 2024-04-25T01:32:18.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.040+0000 7f28a8ff9640 1 -- 172.21.15.112:0/2346978624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28ac14ef30 msgr2=0x7f28ac0c7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.040+0000 7f28a8ff9640 1 --2- 172.21.15.112:0/2346978624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28ac14ef30 0x7f28ac0c7150 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f28ac068e90 tx=0x7f28a0011d40 comp rx=0 tx=0).stop 2024-04-25T01:32:18.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.040+0000 7f28a8ff9640 1 -- 172.21.15.112:0/2346978624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28ac14ef30 msgr2=0x7f28ac0c7150 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:32:18.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.040+0000 7f28a8ff9640 1 --2- 172.21.15.112:0/2346978624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28ac14ef30 0x7f28ac0c7150 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f28ac068e90 tx=0x7f28a0011d40 comp rx=0 tx=0).stop 2024-04-25T01:32:18.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.040+0000 7f28b0c5a640 1 --2- 172.21.15.112:0/2346978624 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f289806be30 0x7f289806e2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f28ac1e9250 tx=0x7f28a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:18.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.047+0000 7f28b26bd640 1 --2- 172.21.15.112:0/2346978624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28ac1ec880 0x7f28ac1ecc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.047+0000 7f28b26bd640 1 -- 172.21.15.112:0/2346978624 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac0c7950 con 0x7f28ac1ec880 2024-04-25T01:32:18.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.047+0000 7f28ab7fe640 1 --2- 172.21.15.112:0/2346978624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28ac1ec880 0x7f28ac1ecc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.047+0000 7f28ab7fe640 1 -- 172.21.15.112:0/2346978624 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f28ac152b10 con 0x7f28ac1ec880 2024-04-25T01:32:18.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.047+0000 7f28ab7fe640 1 --2- 172.21.15.112:0/2346978624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28ac1ec880 0x7f28ac1ecc90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f28ac068e90 tx=0x7f28a00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:18.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.048+0000 7f28a8ff9640 1 -- 172.21.15.112:0/2346978624 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a000fb60 con 0x7f28ac1ec880 2024-04-25T01:32:18.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.048+0000 7f28a8ff9640 1 -- 172.21.15.112:0/2346978624 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5070 (secure 0 0 0) 0x7f28a0003da0 con 0x7f28ac1ec880 2024-04-25T01:32:18.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.048+0000 7f28a8ff9640 1 -- 172.21.15.112:0/2346978624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28ac1ec880 msgr2=0x7f28ac1ecc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.048+0000 7f28a8ff9640 1 --2- 172.21.15.112:0/2346978624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28ac1ec880 0x7f28ac1ecc90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f28ac068e90 tx=0x7f28a00046c0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.048+0000 7f28a8ff9640 1 -- 172.21.15.112:0/2346978624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28ac1ec880 msgr2=0x7f28ac1ecc90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:32:18.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.048+0000 7f28a8ff9640 1 --2- 172.21.15.112:0/2346978624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28ac1ec880 0x7f28ac1ecc90 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f28ac068e90 tx=0x7f28a00046c0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.031 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:32:18.032 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:32:18.032 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:32:18.032 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:32:18.032 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 46, 2024-04-25T01:32:18.032 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 173737, 2024-04-25T01:32:18.032 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:32:18.033 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:32:18.033 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:32:18.033 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:32:18.033 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:18.033 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 12, 2024-04-25T01:32:18.033 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:32:18.033 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:32:18.034 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:32:18.034 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:18.034 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:18.034 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:18.034 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:18.034 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:18.034 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:18.035 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:18.035 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:18.035 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:18.035 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:18.035 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:18.035 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:18.035 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:32:18.036 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:32:18.036 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:18.036 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:18.036 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:18.036 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:18.036 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:18.036 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:18.037 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:18.037 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:18.037 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:18.037 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:18.037 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:18.037 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:18.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.037 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:32:18.038 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:32:18.038 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:32:18.038 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:32:18.038 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:32:18.038 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:32:18.038 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:32:18.038 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:32:18.039 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:32:18.039 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:32:18.039 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:32:18.039 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:32:18.039 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:32:18.039 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:32:18.040 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:32:18.040 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:32:18.040 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:32:18.040 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:18.040 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:18.040 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:18.040 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:18.040 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:18.041 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:18.041 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:18.041 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:18.041 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:18.041 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:18.041 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:18.041 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:18.041 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:32:18.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.042 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:32:18.042 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.042 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:32:18.042 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:32:18.042 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:18.043 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:18.043 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.043 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:18.043 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:32:18.043 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:18.043 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.043 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.044 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:18.044 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:32:18.044 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:18.044 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:18.044 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:18.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.044 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:32:18.045 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:32:18.045 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:18.045 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:18.045 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:18.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.045 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:32:18.045 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:32:18.045 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:18.045 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:18.046 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.046 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:18.046 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:32:18.046 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:18.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.046 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.046 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:18.046 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:32:18.046 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:18.046 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:18.046 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:18.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.046 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:32:18.047 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:32:18.047 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:18.047 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:18.047 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:18.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.047 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.047 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:32:18.047 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:32:18.047 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:18.047 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:18.047 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.047 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:18.047 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:32:18.047 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:32:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:32:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:32:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:18.049 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.049 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:32:18.049 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:32:18.049 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.049 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:18.049 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:18.049 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:32:18.049 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:18.049 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:18.049 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:32:18.049 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.049 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:18.049 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:18.049 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:32:18.049 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:18.050 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:18.050 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:32:18.050 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.050 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:18.050 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:18.050 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:32:18.050 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:18.050 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:18.050 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:32:18.050 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.050 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:18.050 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:18.050 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:32:18.050 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:18.051 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:18.051 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:32:18.051 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.051 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:18.051 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:18.051 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:32:18.051 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:18.051 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:18.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.051 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:32:18.051 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:32:18.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.048+0000 7f28b26bd640 1 -- 172.21.15.112:0/2346978624 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f289806be30 msgr2=0x7f289806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.048+0000 7f28b26bd640 1 --2- 172.21.15.112:0/2346978624 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f289806be30 0x7f289806e2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f28ac1e9250 tx=0x7f28a4009040 comp rx=0 tx=0).stop 2024-04-25T01:32:18.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.048+0000 7f28b26bd640 1 -- 172.21.15.112:0/2346978624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ac154180 msgr2=0x7f28ac147430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.048+0000 7f28b26bd640 1 --2- 172.21.15.112:0/2346978624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ac154180 0x7f28ac147430 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f289c02a220 tx=0x7f289c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.048+0000 7f28b0c5a640 1 -- 172.21.15.112:0/2346978624 reap_dead start 2024-04-25T01:32:18.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.048+0000 7f28b0c5a640 1 -- 172.21.15.112:0/2346978624 reap_dead start 2024-04-25T01:32:18.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.048+0000 7f28b26bd640 1 -- 172.21.15.112:0/2346978624 shutdown_connections 2024-04-25T01:32:18.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.048+0000 7f28b26bd640 1 -- 172.21.15.112:0/2346978624 >> 172.21.15.112:0/2346978624 conn(0x7f28ac0c4a90 msgr2=0x7f28ac14dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:18.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.048+0000 7f28b26bd640 1 -- 172.21.15.112:0/2346978624 shutdown_connections 2024-04-25T01:32:18.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.049+0000 7f28b26bd640 1 -- 172.21.15.112:0/2346978624 wait complete. 2024-04-25T01:32:18.053 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:32:18.054 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:32:18.054 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:32:18.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.186+0000 7fa82dabd640 1 Processor -- start 2024-04-25T01:32:18.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.187+0000 7fa82dabd640 1 -- start start 2024-04-25T01:32:18.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.187+0000 7fa82dabd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa828152e50 0x7fa8280c9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.187+0000 7fa82dabd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa82806bda0 0x7fa82806c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.187+0000 7fa82dabd640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8280c9840 con 0x7fa82806d6f0 2024-04-25T01:32:18.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.187+0000 7fa82dabd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa828156690 con 0x7fa828152e50 2024-04-25T01:32:18.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.187+0000 7fa82dabd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa828155a80 con 0x7fa82806bda0 2024-04-25T01:32:18.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.187+0000 7fa826ffd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa82806d6f0 0x7fa8280c7bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50042/0 (socket says 172.21.15.112:50042) 2024-04-25T01:32:18.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.187+0000 7fa826ffd640 1 -- 172.21.15.112:0/2050458479 learned_addr learned my addr 172.21.15.112:0/2050458479 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:32:18.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.188+0000 7fa827fff640 1 --2- 172.21.15.112:0/2050458479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa82806bda0 0x7fa82806c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.188+0000 7fa8277fe640 1 --2- 172.21.15.112:0/2050458479 >> v2:172.21.15.187:3300/0 conn(0x7fa828152e50 0x7fa8280c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.188+0000 7fa8267fc640 1 -- 172.21.15.112:0/2050458479 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3659938990 0 0) 0x7fa8280c9840 con 0x7fa82806d6f0 2024-04-25T01:32:18.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.188+0000 7fa8267fc640 1 -- 172.21.15.112:0/2050458479 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa810003520 con 0x7fa82806d6f0 2024-04-25T01:32:18.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.188+0000 7fa8267fc640 1 -- 172.21.15.112:0/2050458479 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1621356621 0 0) 0x7fa810003520 con 0x7fa82806d6f0 2024-04-25T01:32:18.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.188+0000 7fa8267fc640 1 -- 172.21.15.112:0/2050458479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa82806bda0 msgr2=0x7fa82806c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:32:18.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.188+0000 7fa8267fc640 1 --2- 172.21.15.112:0/2050458479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa82806bda0 0x7fa82806c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.188+0000 7fa8267fc640 1 -- 172.21.15.112:0/2050458479 >> v2:172.21.15.187:3300/0 conn(0x7fa828152e50 msgr2=0x7fa8280c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:32:18.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.188+0000 7fa8267fc640 1 --2- 172.21.15.112:0/2050458479 >> v2:172.21.15.187:3300/0 conn(0x7fa828152e50 0x7fa8280c9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.188+0000 7fa8267fc640 1 -- 172.21.15.112:0/2050458479 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa828155d60 con 0x7fa82806d6f0 2024-04-25T01:32:18.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.188+0000 7fa8267fc640 1 -- 172.21.15.112:0/2050458479 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fa81c001670 con 0x7fa82806d6f0 2024-04-25T01:32:18.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.188+0000 7fa8267fc640 1 -- 172.21.15.112:0/2050458479 >> v1:172.21.15.112:6789/0 conn(0x7fa82806d6f0 legacy=0x7fa8280c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.188+0000 7fa8267fc640 1 --2- 172.21.15.112:0/2050458479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa82806bda0 0x7fa8100039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.188+0000 7fa8267fc640 1 --2- 172.21.15.112:0/2050458479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa810003f30 0x7fa810004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.188+0000 7fa8267fc640 1 --2- 172.21.15.112:0/2050458479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa810004d40 0x7fa8100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.188+0000 7fa8267fc640 1 -- 172.21.15.112:0/2050458479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa828156690 con 0x7fa810003f30 2024-04-25T01:32:18.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.188+0000 7fa8267fc640 1 -- 172.21.15.112:0/2050458479 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa828155a80 con 0x7fa810004d40 2024-04-25T01:32:18.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.188+0000 7fa8267fc640 1 -- 172.21.15.112:0/2050458479 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa81000baa0 con 0x7fa82806bda0 2024-04-25T01:32:18.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.188+0000 7fa826ffd640 1 --2- 172.21.15.112:0/2050458479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa810003f30 0x7fa810004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.189+0000 7fa827fff640 1 --2- 172.21.15.112:0/2050458479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa810004d40 0x7fa8100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.189+0000 7fa826ffd640 1 -- 172.21.15.112:0/2050458479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa82806bda0 msgr2=0x7fa8100039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:32:18.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.189+0000 7fa826ffd640 1 --2- 172.21.15.112:0/2050458479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa82806bda0 0x7fa8100039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.189+0000 7fa826ffd640 1 -- 172.21.15.112:0/2050458479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa810004d40 msgr2=0x7fa8100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:32:18.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.189+0000 7fa826ffd640 1 --2- 172.21.15.112:0/2050458479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa810004d40 0x7fa8100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.189+0000 7fa826ffd640 1 -- 172.21.15.112:0/2050458479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa810003520 con 0x7fa810003f30 2024-04-25T01:32:18.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.189+0000 7fa827fff640 1 --2- 172.21.15.112:0/2050458479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa810004d40 0x7fa8100051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:32:18.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.189+0000 7fa826ffd640 1 --2- 172.21.15.112:0/2050458479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa810003f30 0x7fa810004800 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00c240 tx=0x7fa810005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a695bc8bc6c69ae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:18.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.190+0000 7fa8267fc640 1 -- 172.21.15.112:0/2050458479 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c02e070 con 0x7fa810003f30 2024-04-25T01:32:18.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.190+0000 7fa8267fc640 1 -- 172.21.15.112:0/2050458479 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81c005af0 con 0x7fa810003f30 2024-04-25T01:32:18.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.190+0000 7fa8267fc640 1 -- 172.21.15.112:0/2050458479 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c007d30 con 0x7fa810003f30 2024-04-25T01:32:18.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.190+0000 7fa82dabd640 1 -- 172.21.15.112:0/2050458479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa810003f30 msgr2=0x7fa810004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.190+0000 7fa82dabd640 1 --2- 172.21.15.112:0/2050458479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa810003f30 0x7fa810004800 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00c240 tx=0x7fa810005830 comp rx=0 tx=0).stop 2024-04-25T01:32:18.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.190+0000 7fa827fff640 1 -- 172.21.15.112:0/2050458479 reap_dead start 2024-04-25T01:32:18.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.190+0000 7fa82dabd640 1 -- 172.21.15.112:0/2050458479 shutdown_connections 2024-04-25T01:32:18.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.190+0000 7fa82dabd640 1 -- 172.21.15.112:0/2050458479 >> 172.21.15.112:0/2050458479 conn(0x7fa8280acd30 msgr2=0x7fa8280ad110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:18.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.191+0000 7fa82dabd640 1 -- 172.21.15.112:0/2050458479 shutdown_connections 2024-04-25T01:32:18.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.191+0000 7fa82dabd640 1 -- 172.21.15.112:0/2050458479 wait complete. 2024-04-25T01:32:18.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.192+0000 7fa82dabd640 1 Processor -- start 2024-04-25T01:32:18.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.192+0000 7fa82dabd640 1 -- start start 2024-04-25T01:32:18.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.193+0000 7fa82dabd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa810003f30 0x7fa8280c3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.193+0000 7fa82dabd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8280c22b0 0x7fa82814a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.193+0000 7fa82dabd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa82814a8b0 0x7fa8280c4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.193+0000 7fa82dabd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8280c8300 con 0x7fa810003f30 2024-04-25T01:32:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.193+0000 7fa82dabd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa828156320 con 0x7fa82814a8b0 2024-04-25T01:32:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.193+0000 7fa82dabd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa8280c8480 con 0x7fa8280c22b0 2024-04-25T01:32:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.193+0000 7fa8277fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa810003f30 0x7fa8280c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.193+0000 7fa827fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa82814a8b0 0x7fa8280c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.193+0000 7fa8277fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa810003f30 0x7fa8280c3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38430/0 (socket says 172.21.15.112:38430) 2024-04-25T01:32:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.193+0000 7fa8277fe640 1 -- 172.21.15.112:0/3059624723 learned_addr learned my addr 172.21.15.112:0/3059624723 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.193+0000 7fa826ffd640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8280c22b0 0x7fa82814a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.194+0000 7fa8277fe640 1 -- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8280c22b0 msgr2=0x7fa82814a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.194+0000 7fa8277fe640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8280c22b0 0x7fa82814a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.194+0000 7fa8277fe640 1 -- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa82814a8b0 msgr2=0x7fa8280c4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.194+0000 7fa8277fe640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa82814a8b0 0x7fa8280c4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.194+0000 7fa8277fe640 1 -- 172.21.15.112:0/3059624723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8281eb7e0 con 0x7fa810003f30 2024-04-25T01:32:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.194+0000 7fa826ffd640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8280c22b0 0x7fa82814a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:32:18.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.194+0000 7fa827fff640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa82814a8b0 0x7fa8280c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:32:18.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.194+0000 7fa8277fe640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa810003f30 0x7fa8280c3070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa81800e890 tx=0x7fa81800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:18.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.194+0000 7fa80ffff640 1 -- 172.21.15.112:0/3059624723 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818017070 con 0x7fa810003f30 2024-04-25T01:32:18.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.194+0000 7fa80ffff640 1 -- 172.21.15.112:0/3059624723 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81800c940 con 0x7fa810003f30 2024-04-25T01:32:18.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.194+0000 7fa82dabd640 1 -- 172.21.15.112:0/3059624723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8281ebac0 con 0x7fa810003f30 2024-04-25T01:32:18.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.194+0000 7fa80ffff640 1 -- 172.21.15.112:0/3059624723 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81800cd30 con 0x7fa810003f30 2024-04-25T01:32:18.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.194+0000 7fa82dabd640 1 -- 172.21.15.112:0/3059624723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8281ec080 con 0x7fa810003f30 2024-04-25T01:32:18.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.196+0000 7fa82dabd640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8280c4780 0x7fa8281eeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.196+0000 7fa82dabd640 1 -- 172.21.15.112:0/3059624723 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa8281ec5a0 con 0x7fa8280c4780 2024-04-25T01:32:18.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.196+0000 7fa826ffd640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8280c4780 0x7fa8281eeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.196+0000 7fa826ffd640 1 -- 172.21.15.112:0/3059624723 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa828075980 con 0x7fa8280c4780 2024-04-25T01:32:18.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.196+0000 7fa80ffff640 1 -- 172.21.15.112:0/3059624723 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa81801a070 con 0x7fa810003f30 2024-04-25T01:32:18.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.196+0000 7fa826ffd640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8280c4780 0x7fa8281eeb40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00c210 tx=0x7fa828155970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:18.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.197+0000 7fa80ffff640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa81406be30 0x7fa81406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.197+0000 7fa80ffff640 1 -- 172.21.15.112:0/3059624723 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fa81808b040 con 0x7fa810003f30 2024-04-25T01:32:18.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.197+0000 7fa827fff640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa81406be30 0x7fa81406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.197+0000 7fa80ffff640 1 -- 172.21.15.112:0/3059624723 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c007480 con 0x7fa8280c4780 2024-04-25T01:32:18.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.197+0000 7fa80ffff640 1 -- 172.21.15.112:0/3059624723 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa81c043070 con 0x7fa8280c4780 2024-04-25T01:32:18.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.197+0000 7fa80ffff640 1 -- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8280c4780 msgr2=0x7fa8281eeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.197+0000 7fa80ffff640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8280c4780 0x7fa8281eeb40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00c210 tx=0x7fa828155970 comp rx=0 tx=0).stop 2024-04-25T01:32:18.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.197+0000 7fa80ffff640 1 -- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8280c4780 msgr2=0x7fa8281eeb40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:32:18.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.197+0000 7fa80ffff640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8280c4780 0x7fa8281eeb40 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00c210 tx=0x7fa828155970 comp rx=0 tx=0).stop 2024-04-25T01:32:18.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.197+0000 7fa827fff640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa81406be30 0x7fa81406e2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa82808d230 tx=0x7fa820004ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:18.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.205+0000 7fa82dabd640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8280c4b60 0x7fa8281ef080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.205+0000 7fa82dabd640 1 -- 172.21.15.112:0/3059624723 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa828151c60 con 0x7fa8280c4b60 2024-04-25T01:32:18.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.205+0000 7fa826ffd640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8280c4b60 0x7fa8281ef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.206+0000 7fa826ffd640 1 -- 172.21.15.112:0/3059624723 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa8280682a0 con 0x7fa8280c4b60 2024-04-25T01:32:18.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.206+0000 7fa826ffd640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8280c4b60 0x7fa8281ef080 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa82814c980 tx=0x7fa81c02e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:18.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.206+0000 7fa80ffff640 1 -- 172.21.15.112:0/3059624723 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c049670 con 0x7fa8280c4b60 2024-04-25T01:32:18.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.206+0000 7fa80ffff640 1 -- 172.21.15.112:0/3059624723 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4900 (secure 0 0 0) 0x7fa81c049670 con 0x7fa8280c4b60 2024-04-25T01:32:18.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.206+0000 7fa80ffff640 1 -- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8280c4b60 msgr2=0x7fa8281ef080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.206+0000 7fa80ffff640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8280c4b60 0x7fa8281ef080 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa82814c980 tx=0x7fa81c02e000 comp rx=0 tx=0).stop 2024-04-25T01:32:18.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.206+0000 7fa80ffff640 1 -- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8280c4b60 msgr2=0x7fa8281ef080 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:32:18.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.206+0000 7fa80ffff640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8280c4b60 0x7fa8281ef080 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa82814c980 tx=0x7fa81c02e000 comp rx=0 tx=0).stop 2024-04-25T01:32:18.184 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:32:18.184 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:32:18.184 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:32:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:32:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 46, 2024-04-25T01:32:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 126945, 2024-04-25T01:32:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:32:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:32:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:32:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:32:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 12, 2024-04-25T01:32:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:32:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:32:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:32:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:18.186 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:18.186 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:18.186 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:18.186 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:18.186 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:18.186 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:18.186 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:18.186 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:18.186 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:18.186 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:32:18.186 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:32:18.186 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:18.186 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:18.186 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:32:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:32:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:32:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:32:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:32:18.188 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:32:18.188 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:32:18.188 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:32:18.188 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:32:18.188 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:32:18.188 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:32:18.188 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:32:18.188 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:32:18.188 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:32:18.188 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:32:18.188 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:32:18.188 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:32:18.188 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:18.188 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:18.188 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:18.189 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:18.189 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:18.189 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:18.189 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:18.189 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:18.189 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:18.189 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:18.189 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:18.189 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:18.189 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:32:18.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.189 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:32:18.189 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.189 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:32:18.190 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:32:18.190 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:18.190 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:18.190 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.190 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:18.190 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:32:18.190 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:18.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.190 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.190 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:18.190 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:32:18.190 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:18.190 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:18.190 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:18.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.191 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:32:18.191 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:32:18.191 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:18.191 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:18.191 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:18.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.191 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.191 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:32:18.191 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:32:18.191 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:18.191 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:18.191 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.191 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:18.191 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:32:18.192 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:18.192 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.192 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.192 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:18.192 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:32:18.192 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:18.192 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:18.192 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:18.192 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.192 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:32:18.192 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:32:18.192 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:18.192 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:18.192 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:18.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.193 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.193 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:32:18.193 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:32:18.193 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:18.193 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:18.193 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.193 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:18.193 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:32:18.193 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:18.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.193 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.193 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:18.193 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:32:18.193 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:18.194 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:18.194 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:18.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.194 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:32:18.194 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:32:18.194 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:18.194 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:18.194 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:18.194 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:18.194 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:18.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.194 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:32:18.194 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:32:18.194 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.195 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:18.195 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:18.195 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:32:18.195 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:18.195 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:18.195 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:32:18.195 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.195 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:18.195 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:18.195 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:32:18.195 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:18.195 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:18.195 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:32:18.195 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.195 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:18.196 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:18.196 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:32:18.196 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:18.196 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:18.196 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:32:18.196 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:18.196 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:18.196 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:18.196 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:32:18.196 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:18.196 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:18.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:18.196 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:32:18.197 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:32:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.207+0000 7fa82dabd640 1 -- 172.21.15.112:0/3059624723 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa81406be30 msgr2=0x7fa81406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.207+0000 7fa82dabd640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa81406be30 0x7fa81406e2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa82808d230 tx=0x7fa820004ab0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.207+0000 7fa82dabd640 1 -- 172.21.15.112:0/3059624723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa810003f30 msgr2=0x7fa8280c3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.207+0000 7fa82dabd640 1 --2- 172.21.15.112:0/3059624723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa810003f30 0x7fa8280c3070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa81800e890 tx=0x7fa81800ed50 comp rx=0 tx=0).stop 2024-04-25T01:32:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.207+0000 7fa827fff640 1 -- 172.21.15.112:0/3059624723 reap_dead start 2024-04-25T01:32:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.207+0000 7fa827fff640 1 -- 172.21.15.112:0/3059624723 reap_dead start 2024-04-25T01:32:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.207+0000 7fa82dabd640 1 -- 172.21.15.112:0/3059624723 shutdown_connections 2024-04-25T01:32:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.207+0000 7fa82dabd640 1 -- 172.21.15.112:0/3059624723 >> 172.21.15.112:0/3059624723 conn(0x7fa8280acd30 msgr2=0x7fa82806c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.207+0000 7fa82dabd640 1 -- 172.21.15.112:0/3059624723 shutdown_connections 2024-04-25T01:32:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.207+0000 7fa82dabd640 1 -- 172.21.15.112:0/3059624723 wait complete. 2024-04-25T01:32:18.199 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:32:18.199 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-04-25T01:32:18.199 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-25T01:32:18.199 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-04-25T01:32:18.199 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-25T01:32:18.200 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T01:32:18.200 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:32:18.212 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T01:32:18.212 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T01:32:18.212 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:32:18.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.341+0000 7f1eb4e9a640 1 Processor -- start 2024-04-25T01:32:18.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.341+0000 7f1eb4e9a640 1 -- start start 2024-04-25T01:32:18.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.342+0000 7f1eb4e9a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1eb014fbe0 0x7f1eb006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.342+0000 7f1eb4e9a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1eb00c7250 0x7f1eb01525b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.342+0000 7f1eb4e9a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eb00c6a70 con 0x7f1eb00746a0 2024-04-25T01:32:18.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.342+0000 7f1eb4e9a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f1eb0155ce0 con 0x7f1eb014fbe0 2024-04-25T01:32:18.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.342+0000 7f1eb4e9a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1eb0155fc0 con 0x7f1eb00c7250 2024-04-25T01:32:18.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.342+0000 7f1eae59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1eb014fbe0 0x7f1eb006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.342+0000 7f1eaed9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1eb00746a0 0x7f1eb0152af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50054/0 (socket says 172.21.15.112:50054) 2024-04-25T01:32:18.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.342+0000 7f1eaed9d640 1 -- 172.21.15.112:0/2747757833 learned_addr learned my addr 172.21.15.112:0/2747757833 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:32:18.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.342+0000 7f1eadd9b640 1 --2- 172.21.15.112:0/2747757833 >> v2:172.21.15.187:3300/0 conn(0x7f1eb00c7250 0x7f1eb01525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.342+0000 7f1eae59c640 1 -- 172.21.15.112:0/2747757833 >> v2:172.21.15.187:3300/0 conn(0x7f1eb00c7250 msgr2=0x7f1eb01525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:32:18.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.342+0000 7f1eae59c640 1 --2- 172.21.15.112:0/2747757833 >> v2:172.21.15.187:3300/0 conn(0x7f1eb00c7250 0x7f1eb01525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.342+0000 7f1eae59c640 1 -- 172.21.15.112:0/2747757833 >> v1:172.21.15.112:6789/0 conn(0x7f1eb00746a0 legacy=0x7f1eb0152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.342+0000 7f1eae59c640 1 -- 172.21.15.112:0/2747757833 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb01562a0 con 0x7f1eb014fbe0 2024-04-25T01:32:18.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.343+0000 7f1eae59c640 1 --2- 172.21.15.112:0/2747757833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb014fbe0 0x7f1eb006c170 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0070f10 tx=0x7f1ea002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9c77f06c9d17c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:18.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.343+0000 7f1ead59a640 1 -- 172.21.15.112:0/2747757833 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0037070 con 0x7f1eb014fbe0 2024-04-25T01:32:18.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.343+0000 7f1ead59a640 1 -- 172.21.15.112:0/2747757833 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea002add0 con 0x7f1eb014fbe0 2024-04-25T01:32:18.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.343+0000 7f1ead59a640 1 -- 172.21.15.112:0/2747757833 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0032660 con 0x7f1eb014fbe0 2024-04-25T01:32:18.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.344+0000 7f1eb4e9a640 1 -- 172.21.15.112:0/2747757833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb014fbe0 msgr2=0x7f1eb006c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.344+0000 7f1eb4e9a640 1 --2- 172.21.15.112:0/2747757833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb014fbe0 0x7f1eb006c170 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0070f10 tx=0x7f1ea002a080 comp rx=0 tx=0).stop 2024-04-25T01:32:18.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.344+0000 7f1eb4e9a640 1 -- 172.21.15.112:0/2747757833 shutdown_connections 2024-04-25T01:32:18.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.344+0000 7f1eb4e9a640 1 --2- 172.21.15.112:0/2747757833 >> v2:172.21.15.187:3300/0 conn(0x7f1eb00c7250 0x7f1eb01525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.344+0000 7f1eb4e9a640 1 --2- 172.21.15.112:0/2747757833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb014fbe0 0x7f1eb006c170 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.344+0000 7f1eb4e9a640 1 -- 172.21.15.112:0/2747757833 >> 172.21.15.112:0/2747757833 conn(0x7f1eb00c42d0 msgr2=0x7f1eb00c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:18.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.344+0000 7f1eb4e9a640 1 -- 172.21.15.112:0/2747757833 shutdown_connections 2024-04-25T01:32:18.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.344+0000 7f1eb4e9a640 1 -- 172.21.15.112:0/2747757833 wait complete. 2024-04-25T01:32:18.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.345+0000 7f1eb4e9a640 1 Processor -- start 2024-04-25T01:32:18.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.345+0000 7f1eb4e9a640 1 -- start start 2024-04-25T01:32:18.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.346+0000 7f1eb4e9a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1eb00746a0 0x7f1eb01516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.346+0000 7f1eb4e9a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb00c7250 0x7f1eb0153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.346+0000 7f1eb4e9a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb014fbe0 0x7f1eb014e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.346+0000 7f1eb4e9a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb014ffc0 con 0x7f1eb00c7250 2024-04-25T01:32:18.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.346+0000 7f1eb4e9a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb0156860 con 0x7f1eb00746a0 2024-04-25T01:32:18.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.346+0000 7f1eb4e9a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb0153200 con 0x7f1eb014fbe0 2024-04-25T01:32:18.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.346+0000 7f1eaed9d640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb014fbe0 msgr2=0x7f1eb014e960 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:32:18.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.346+0000 7f1eaed9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb014fbe0 0x7f1eb014e960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:32:18.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.346+0000 7f1eae59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1eb00746a0 0x7f1eb01516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.346+0000 7f1eadd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb00c7250 0x7f1eb0153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.346+0000 7f1eae59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1eb00746a0 0x7f1eb01516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:35824/0 (socket says 172.21.15.112:35824) 2024-04-25T01:32:18.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.346+0000 7f1eae59c640 1 -- 172.21.15.112:0/2495028284 learned_addr learned my addr 172.21.15.112:0/2495028284 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:18.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.346+0000 7f1eadd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb00c7250 0x7f1eb0153960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38500/0 (socket says 172.21.15.112:38500) 2024-04-25T01:32:18.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.347+0000 7f1eadd9b640 1 -- 172.21.15.112:0/2495028284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb014fbe0 msgr2=0x7f1eb014e960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:32:18.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.347+0000 7f1eadd9b640 1 --2- 172.21.15.112:0/2495028284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb014fbe0 0x7f1eb014e960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.347+0000 7f1eadd9b640 1 -- 172.21.15.112:0/2495028284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1eb00746a0 msgr2=0x7f1eb01516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.347+0000 7f1eadd9b640 1 --2- 172.21.15.112:0/2495028284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1eb00746a0 0x7f1eb01516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.347+0000 7f1eadd9b640 1 -- 172.21.15.112:0/2495028284 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb0155440 con 0x7f1eb00c7250 2024-04-25T01:32:18.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.347+0000 7f1eadd9b640 1 --2- 172.21.15.112:0/2495028284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb00c7250 0x7f1eb0153960 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400e850 tx=0x7f1ea400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:18.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.347+0000 7f1e9effd640 1 -- 172.21.15.112:0/2495028284 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4017070 con 0x7f1eb00c7250 2024-04-25T01:32:18.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.347+0000 7f1e9effd640 1 -- 172.21.15.112:0/2495028284 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea400c940 con 0x7f1eb00c7250 2024-04-25T01:32:18.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.347+0000 7f1eb4e9a640 1 -- 172.21.15.112:0/2495028284 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb0155160 con 0x7f1eb00c7250 2024-04-25T01:32:18.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.347+0000 7f1e9effd640 1 -- 172.21.15.112:0/2495028284 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea400cd10 con 0x7f1eb00c7250 2024-04-25T01:32:18.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.347+0000 7f1eb4e9a640 1 -- 172.21.15.112:0/2495028284 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb01eb990 con 0x7f1eb00c7250 2024-04-25T01:32:18.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.348+0000 7f1eb4e9a640 1 -- 172.21.15.112:0/2495028284 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb00689b0 con 0x7f1eb00c7250 2024-04-25T01:32:18.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.348+0000 7f1e9effd640 1 -- 172.21.15.112:0/2495028284 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ea401a070 con 0x7f1eb00c7250 2024-04-25T01:32:18.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.349+0000 7f1e9effd640 1 --2- 172.21.15.112:0/2495028284 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e9006bde0 0x7f1e9006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:18.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.349+0000 7f1e9effd640 1 -- 172.21.15.112:0/2495028284 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f1ea408afa0 con 0x7f1eb00c7250 2024-04-25T01:32:18.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.349+0000 7f1eae59c640 1 --2- 172.21.15.112:0/2495028284 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e9006bde0 0x7f1e9006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:18.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.350+0000 7f1eae59c640 1 --2- 172.21.15.112:0/2495028284 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e9006bde0 0x7f1e9006e2a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1ea002a0c0 tx=0x7f1ea002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:18.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.354+0000 7f1e9effd640 1 -- 172.21.15.112:0/2495028284 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1ea4058ff0 con 0x7f1eb00c7250 2024-04-25T01:32:18.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.673+0000 7f1eb4e9a640 1 -- 172.21.15.112:0/2495028284 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1eb00c67e0 con 0x7f1eb00c7250 2024-04-25T01:32:18.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.674+0000 7f1e9effd640 1 -- 172.21.15.112:0/2495028284 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f1ea405d2f0 con 0x7f1eb00c7250 2024-04-25T01:32:18.651 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:32:18.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.677+0000 7f1eb4e9a640 1 -- 172.21.15.112:0/2495028284 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e9006bde0 msgr2=0x7f1e9006e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.677+0000 7f1eb4e9a640 1 --2- 172.21.15.112:0/2495028284 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e9006bde0 0x7f1e9006e2a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1ea002a0c0 tx=0x7f1ea002e000 comp rx=0 tx=0).stop 2024-04-25T01:32:18.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.677+0000 7f1eb4e9a640 1 -- 172.21.15.112:0/2495028284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb00c7250 msgr2=0x7f1eb0153960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:18.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.677+0000 7f1eb4e9a640 1 --2- 172.21.15.112:0/2495028284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb00c7250 0x7f1eb0153960 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400e850 tx=0x7f1ea400ed10 comp rx=0 tx=0).stop 2024-04-25T01:32:18.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.677+0000 7f1eb4e9a640 1 -- 172.21.15.112:0/2495028284 shutdown_connections 2024-04-25T01:32:18.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.677+0000 7f1eb4e9a640 1 --2- 172.21.15.112:0/2495028284 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e9006bde0 0x7f1e9006e2a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.677+0000 7f1eb4e9a640 1 --2- 172.21.15.112:0/2495028284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb014fbe0 0x7f1eb014e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.677+0000 7f1eb4e9a640 1 --2- 172.21.15.112:0/2495028284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb00c7250 0x7f1eb0153960 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.677+0000 7f1eb4e9a640 1 --2- 172.21.15.112:0/2495028284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1eb00746a0 0x7f1eb01516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:18.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.677+0000 7f1eb4e9a640 1 -- 172.21.15.112:0/2495028284 >> 172.21.15.112:0/2495028284 conn(0x7f1eb00c42d0 msgr2=0x7f1eb00a7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:18.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.677+0000 7f1eb4e9a640 1 -- 172.21.15.112:0/2495028284 shutdown_connections 2024-04-25T01:32:18.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:18.677+0000 7f1eb4e9a640 1 -- 172.21.15.112:0/2495028284 wait complete. 2024-04-25T01:32:21.666 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:32:21.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.795+0000 7f1ccee09640 1 Processor -- start 2024-04-25T01:32:21.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.795+0000 7f1ccee09640 1 -- start start 2024-04-25T01:32:21.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.796+0000 7f1ccee09640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1cc8153610 0x7f1cc806df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:21.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.796+0000 7f1ccee09640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cc806d1a0 0x7f1cc806e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:21.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.796+0000 7f1ccee09640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cc806cd90 con 0x7f1cc8150ac0 2024-04-25T01:32:21.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.796+0000 7f1ccee09640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f1cc806e9d0 con 0x7f1cc8153610 2024-04-25T01:32:21.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.796+0000 7f1ccee09640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc80ac4b0 con 0x7f1cc806d1a0 2024-04-25T01:32:21.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.796+0000 7f1ccd3a6640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cc806d1a0 msgr2=0x7f1cc806e490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:32:21.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.796+0000 7f1ccd3a6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cc806d1a0 0x7f1cc806e490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:32:21.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.796+0000 7f1cc7fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1cc8153610 0x7f1cc806df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:21.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.796+0000 7f1cc7fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1cc8153610 0x7f1cc806df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38506/0 (socket says 172.21.15.112:38506) 2024-04-25T01:32:21.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.796+0000 7f1cc7fff640 1 -- 172.21.15.112:0/1597743513 learned_addr learned my addr 172.21.15.112:0/1597743513 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:21.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.796+0000 7f1cc7fff640 1 -- 172.21.15.112:0/1597743513 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cc806d1a0 msgr2=0x7f1cc806e490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:32:21.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.796+0000 7f1cc7fff640 1 --2- 172.21.15.112:0/1597743513 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cc806d1a0 0x7f1cc806e490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:21.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.796+0000 7f1cc7fff640 1 -- 172.21.15.112:0/1597743513 >> v1:172.21.15.187:6789/0 conn(0x7f1cc8150ac0 legacy=0x7f1cc80a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:21.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.797+0000 7f1cc7fff640 1 -- 172.21.15.112:0/1597743513 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc80ab410 con 0x7f1cc8153610 2024-04-25T01:32:21.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.797+0000 7f1cc7fff640 1 --2- 172.21.15.112:0/1597743513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cc8153610 0x7f1cc806df50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009870 tx=0x7f1cb402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9686822a57b208e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:21.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.797+0000 7f1cc77fe640 1 -- 172.21.15.112:0/1597743513 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4037070 con 0x7f1cc8153610 2024-04-25T01:32:21.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.798+0000 7f1cc77fe640 1 -- 172.21.15.112:0/1597743513 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb4032440 con 0x7f1cc8153610 2024-04-25T01:32:21.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.798+0000 7f1cc77fe640 1 -- 172.21.15.112:0/1597743513 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4032810 con 0x7f1cc8153610 2024-04-25T01:32:21.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.798+0000 7f1ccee09640 1 -- 172.21.15.112:0/1597743513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cc8153610 msgr2=0x7f1cc806df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:21.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.798+0000 7f1ccee09640 1 --2- 172.21.15.112:0/1597743513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cc8153610 0x7f1cc806df50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009870 tx=0x7f1cb402a1e0 comp rx=0 tx=0).stop 2024-04-25T01:32:21.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.798+0000 7f1ccee09640 1 -- 172.21.15.112:0/1597743513 shutdown_connections 2024-04-25T01:32:21.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.798+0000 7f1ccee09640 1 --2- 172.21.15.112:0/1597743513 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cc806d1a0 0x7f1cc806e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:21.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.798+0000 7f1ccee09640 1 --2- 172.21.15.112:0/1597743513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cc8153610 0x7f1cc806df50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:21.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.798+0000 7f1ccee09640 1 -- 172.21.15.112:0/1597743513 >> 172.21.15.112:0/1597743513 conn(0x7f1cc80c6f80 msgr2=0x7f1cc80c7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:21.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.798+0000 7f1ccee09640 1 -- 172.21.15.112:0/1597743513 shutdown_connections 2024-04-25T01:32:21.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.798+0000 7f1ccee09640 1 -- 172.21.15.112:0/1597743513 wait complete. 2024-04-25T01:32:21.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.799+0000 7f1ccee09640 1 Processor -- start 2024-04-25T01:32:21.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.799+0000 7f1ccee09640 1 -- start start 2024-04-25T01:32:21.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.800+0000 7f1ccee09640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cc806d1a0 0x7f1cc8149e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:21.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.800+0000 7f1ccee09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cc8150ac0 0x7f1cc80c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.800+0000 7f1ccee09640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cc8153610 0x7f1cc81eb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.800+0000 7f1ccee09640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc80abcb0 con 0x7f1cc8150ac0 2024-04-25T01:32:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.800+0000 7f1ccee09640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc80ab9d0 con 0x7f1cc806d1a0 2024-04-25T01:32:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.800+0000 7f1ccee09640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc80abf90 con 0x7f1cc8153610 2024-04-25T01:32:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.800+0000 7f1cc7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cc8150ac0 0x7f1cc80c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.800+0000 7f1ccd3a6640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cc8153610 msgr2=0x7f1cc81eb2e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:32:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.800+0000 7f1ccd3a6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cc8153610 0x7f1cc81eb2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:32:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.800+0000 7f1cc7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cc8150ac0 0x7f1cc80c5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38508/0 (socket says 172.21.15.112:38508) 2024-04-25T01:32:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.800+0000 7f1cc7fff640 1 -- 172.21.15.112:0/268309532 learned_addr learned my addr 172.21.15.112:0/268309532 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.800+0000 7f1cccba5640 1 --2- 172.21.15.112:0/268309532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cc806d1a0 0x7f1cc8149e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.801+0000 7f1cc7fff640 1 -- 172.21.15.112:0/268309532 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cc8153610 msgr2=0x7f1cc81eb2e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:32:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.801+0000 7f1cc7fff640 1 --2- 172.21.15.112:0/268309532 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cc8153610 0x7f1cc81eb2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.801+0000 7f1cc7fff640 1 -- 172.21.15.112:0/268309532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cc806d1a0 msgr2=0x7f1cc8149e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:21.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.801+0000 7f1cc7fff640 1 --2- 172.21.15.112:0/268309532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cc806d1a0 0x7f1cc8149e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:21.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.801+0000 7f1cc7fff640 1 -- 172.21.15.112:0/268309532 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc81defe0 con 0x7f1cc8150ac0 2024-04-25T01:32:21.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.801+0000 7f1cc7fff640 1 --2- 172.21.15.112:0/268309532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cc8150ac0 0x7f1cc80c5e30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4000c00 tx=0x7f1cb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:21.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.801+0000 7f1cc57fa640 1 -- 172.21.15.112:0/268309532 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4041070 con 0x7f1cc8150ac0 2024-04-25T01:32:21.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.801+0000 7f1cc57fa640 1 -- 172.21.15.112:0/268309532 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb403c870 con 0x7f1cc8150ac0 2024-04-25T01:32:21.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.801+0000 7f1ccee09640 1 -- 172.21.15.112:0/268309532 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc8072960 con 0x7f1cc8150ac0 2024-04-25T01:32:21.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.801+0000 7f1cc57fa640 1 -- 172.21.15.112:0/268309532 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4037040 con 0x7f1cc8150ac0 2024-04-25T01:32:21.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.801+0000 7f1ccee09640 1 -- 172.21.15.112:0/268309532 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc80c6370 con 0x7f1cc8150ac0 2024-04-25T01:32:21.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.802+0000 7f1ccee09640 1 -- 172.21.15.112:0/268309532 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc80682a0 con 0x7f1cc8150ac0 2024-04-25T01:32:21.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.803+0000 7f1cc57fa640 1 -- 172.21.15.112:0/268309532 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1cb4044070 con 0x7f1cc8150ac0 2024-04-25T01:32:21.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.803+0000 7f1cc57fa640 1 --2- 172.21.15.112:0/268309532 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1cac06bde0 0x7f1cac06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:21.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.803+0000 7f1cc57fa640 1 -- 172.21.15.112:0/268309532 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f1cb40abd80 con 0x7f1cc8150ac0 2024-04-25T01:32:21.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.803+0000 7f1cccba5640 1 --2- 172.21.15.112:0/268309532 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1cac06bde0 0x7f1cac06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:21.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.804+0000 7f1cccba5640 1 --2- 172.21.15.112:0/268309532 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1cac06bde0 0x7f1cac06e2a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0016e0 tx=0x7f1cbc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:21.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:21.809+0000 7f1cc57fa640 1 -- 172.21.15.112:0/268309532 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1cb4079e70 con 0x7f1cc8150ac0 2024-04-25T01:32:22.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:22.099+0000 7f1ccee09640 1 -- 172.21.15.112:0/268309532 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1cc806cf60 con 0x7f1cc8150ac0 2024-04-25T01:32:22.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:22.099+0000 7f1cc57fa640 1 -- 172.21.15.112:0/268309532 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f1cb407e170 con 0x7f1cc8150ac0 2024-04-25T01:32:22.078 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:32:22.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:22.101+0000 7f1ccee09640 1 -- 172.21.15.112:0/268309532 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1cac06bde0 msgr2=0x7f1cac06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:22.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:22.101+0000 7f1ccee09640 1 --2- 172.21.15.112:0/268309532 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1cac06bde0 0x7f1cac06e2a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0016e0 tx=0x7f1cbc009040 comp rx=0 tx=0).stop 2024-04-25T01:32:22.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:22.101+0000 7f1ccee09640 1 -- 172.21.15.112:0/268309532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cc8150ac0 msgr2=0x7f1cc80c5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:22.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:22.101+0000 7f1ccee09640 1 --2- 172.21.15.112:0/268309532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cc8150ac0 0x7f1cc80c5e30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4000c00 tx=0x7f1cb40046c0 comp rx=0 tx=0).stop 2024-04-25T01:32:22.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:22.102+0000 7f1ccee09640 1 -- 172.21.15.112:0/268309532 shutdown_connections 2024-04-25T01:32:22.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:22.102+0000 7f1ccee09640 1 --2- 172.21.15.112:0/268309532 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1cac06bde0 0x7f1cac06e2a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:22.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:22.102+0000 7f1ccee09640 1 --2- 172.21.15.112:0/268309532 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cc8153610 0x7f1cc81eb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:22.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:22.102+0000 7f1ccee09640 1 --2- 172.21.15.112:0/268309532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cc8150ac0 0x7f1cc80c5e30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:22.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:22.102+0000 7f1ccee09640 1 --2- 172.21.15.112:0/268309532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cc806d1a0 0x7f1cc8149e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:22.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:22.102+0000 7f1ccee09640 1 -- 172.21.15.112:0/268309532 >> 172.21.15.112:0/268309532 conn(0x7f1cc80c6f80 msgr2=0x7f1cc806d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:22.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:22.102+0000 7f1ccee09640 1 -- 172.21.15.112:0/268309532 shutdown_connections 2024-04-25T01:32:22.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:22.102+0000 7f1ccee09640 1 -- 172.21.15.112:0/268309532 wait complete. 2024-04-25T01:32:25.093 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:32:25.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.223+0000 7f1b44061640 1 Processor -- start 2024-04-25T01:32:25.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.223+0000 7f1b44061640 1 -- start start 2024-04-25T01:32:25.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.224+0000 7f1b44061640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1b3c0c5db0 0x7f1b3c0c6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:25.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.224+0000 7f1b44061640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1b3c0c33e0 0x7f1b3c1602d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:25.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.224+0000 7f1b44061640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b3c070670 con 0x7f1b3c0c7900 2024-04-25T01:32:25.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.224+0000 7f1b44061640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f1b3c070420 con 0x7f1b3c0c5db0 2024-04-25T01:32:25.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.224+0000 7f1b44061640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1b3c0c7100 con 0x7f1b3c0c33e0 2024-04-25T01:32:25.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.224+0000 7f1b41dfd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1b3c0c5db0 0x7f1b3c0c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:25.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.224+0000 7f1b41dfd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1b3c0c5db0 0x7f1b3c0c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38522/0 (socket says 172.21.15.112:38522) 2024-04-25T01:32:25.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.224+0000 7f1b41dfd640 1 -- 172.21.15.112:0/2442765715 learned_addr learned my addr 172.21.15.112:0/2442765715 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:25.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.224+0000 7f1b425fe640 1 --2- 172.21.15.112:0/2442765715 >> v2:172.21.15.187:3300/0 conn(0x7f1b3c0c33e0 0x7f1b3c1602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:25.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.224+0000 7f1b41dfd640 1 -- 172.21.15.112:0/2442765715 >> v2:172.21.15.187:3300/0 conn(0x7f1b3c0c33e0 msgr2=0x7f1b3c1602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:32:25.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.224+0000 7f1b41dfd640 1 --2- 172.21.15.112:0/2442765715 >> v2:172.21.15.187:3300/0 conn(0x7f1b3c0c33e0 0x7f1b3c1602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:25.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.224+0000 7f1b41dfd640 1 -- 172.21.15.112:0/2442765715 >> v1:172.21.15.187:6789/0 conn(0x7f1b3c0c7900 legacy=0x7f1b3c155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:25.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.224+0000 7f1b41dfd640 1 -- 172.21.15.112:0/2442765715 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c0c73e0 con 0x7f1b3c0c5db0 2024-04-25T01:32:25.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.225+0000 7f1b41dfd640 1 --2- 172.21.15.112:0/2442765715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c0c5db0 0x7f1b3c0c6190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c06fd80 tx=0x7f1b3802a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5eb829299ff155 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:25.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.225+0000 7f1b40dfb640 1 -- 172.21.15.112:0/2442765715 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38037070 con 0x7f1b3c0c5db0 2024-04-25T01:32:25.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.225+0000 7f1b40dfb640 1 -- 172.21.15.112:0/2442765715 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b38032440 con 0x7f1b3c0c5db0 2024-04-25T01:32:25.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.225+0000 7f1b40dfb640 1 -- 172.21.15.112:0/2442765715 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38032810 con 0x7f1b3c0c5db0 2024-04-25T01:32:25.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.226+0000 7f1b44061640 1 -- 172.21.15.112:0/2442765715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c0c5db0 msgr2=0x7f1b3c0c6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:25.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.226+0000 7f1b44061640 1 --2- 172.21.15.112:0/2442765715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c0c5db0 0x7f1b3c0c6190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c06fd80 tx=0x7f1b3802a150 comp rx=0 tx=0).stop 2024-04-25T01:32:25.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.226+0000 7f1b44061640 1 -- 172.21.15.112:0/2442765715 shutdown_connections 2024-04-25T01:32:25.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.226+0000 7f1b44061640 1 --2- 172.21.15.112:0/2442765715 >> v2:172.21.15.187:3300/0 conn(0x7f1b3c0c33e0 0x7f1b3c1602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:25.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.226+0000 7f1b44061640 1 --2- 172.21.15.112:0/2442765715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c0c5db0 0x7f1b3c0c6190 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:25.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.226+0000 7f1b44061640 1 -- 172.21.15.112:0/2442765715 >> 172.21.15.112:0/2442765715 conn(0x7f1b3c0c91d0 msgr2=0x7f1b3c0c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:25.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.226+0000 7f1b44061640 1 -- 172.21.15.112:0/2442765715 shutdown_connections 2024-04-25T01:32:25.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.226+0000 7f1b44061640 1 -- 172.21.15.112:0/2442765715 wait complete. 2024-04-25T01:32:25.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.227+0000 7f1b44061640 1 Processor -- start 2024-04-25T01:32:25.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.227+0000 7f1b44061640 1 -- start start 2024-04-25T01:32:25.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.228+0000 7f1b44061640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0c33e0 0x7f1b3c06d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:25.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.228+0000 7f1b44061640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b3c0c5db0 0x7f1b3c154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:25.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.228+0000 7f1b44061640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c0c7900 0x7f1b3c155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:25.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.228+0000 7f1b44061640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c153320 con 0x7f1b3c0c7900 2024-04-25T01:32:25.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.228+0000 7f1b44061640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c06ebd0 con 0x7f1b3c0c5db0 2024-04-25T01:32:25.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.228+0000 7f1b44061640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3c1534a0 con 0x7f1b3c0c33e0 2024-04-25T01:32:25.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.228+0000 7f1b41dfd640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0c33e0 msgr2=0x7f1b3c06d2c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:32:25.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.228+0000 7f1b41dfd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0c33e0 0x7f1b3c06d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:32:25.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.228+0000 7f1b425fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c0c7900 0x7f1b3c155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:25.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.228+0000 7f1b425fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c0c7900 0x7f1b3c155120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38538/0 (socket says 172.21.15.112:38538) 2024-04-25T01:32:25.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.228+0000 7f1b425fe640 1 -- 172.21.15.112:0/3379070870 learned_addr learned my addr 172.21.15.112:0/3379070870 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:25.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.228+0000 7f1b415fc640 1 --2- 172.21.15.112:0/3379070870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b3c0c5db0 0x7f1b3c154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:25.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.228+0000 7f1b425fe640 1 -- 172.21.15.112:0/3379070870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0c33e0 msgr2=0x7f1b3c06d2c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:32:25.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.228+0000 7f1b425fe640 1 --2- 172.21.15.112:0/3379070870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0c33e0 0x7f1b3c06d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:25.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.228+0000 7f1b425fe640 1 -- 172.21.15.112:0/3379070870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b3c0c5db0 msgr2=0x7f1b3c154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:25.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.228+0000 7f1b425fe640 1 --2- 172.21.15.112:0/3379070870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b3c0c5db0 0x7f1b3c154be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:25.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.228+0000 7f1b425fe640 1 -- 172.21.15.112:0/3379070870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c14f5f0 con 0x7f1b3c0c7900 2024-04-25T01:32:25.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.228+0000 7f1b425fe640 1 --2- 172.21.15.112:0/3379070870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c0c7900 0x7f1b3c155120 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400efc0 tx=0x7f1b3400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:25.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.229+0000 7f1b327fc640 1 -- 172.21.15.112:0/3379070870 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34018020 con 0x7f1b3c0c7900 2024-04-25T01:32:25.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.229+0000 7f1b327fc640 1 -- 172.21.15.112:0/3379070870 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b34014480 con 0x7f1b3c0c7900 2024-04-25T01:32:25.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.229+0000 7f1b327fc640 1 -- 172.21.15.112:0/3379070870 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34014870 con 0x7f1b3c0c7900 2024-04-25T01:32:25.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.229+0000 7f1b44061640 1 -- 172.21.15.112:0/3379070870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3c145340 con 0x7f1b3c0c7900 2024-04-25T01:32:25.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.229+0000 7f1b44061640 1 -- 172.21.15.112:0/3379070870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3c14f310 con 0x7f1b3c0c7900 2024-04-25T01:32:25.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.230+0000 7f1b44061640 1 -- 172.21.15.112:0/3379070870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b3c0689b0 con 0x7f1b3c0c7900 2024-04-25T01:32:25.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.230+0000 7f1b327fc640 1 -- 172.21.15.112:0/3379070870 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b34005d10 con 0x7f1b3c0c7900 2024-04-25T01:32:25.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.230+0000 7f1b327fc640 1 --2- 172.21.15.112:0/3379070870 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1b2006be10 0x7f1b2006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:25.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.230+0000 7f1b327fc640 1 -- 172.21.15.112:0/3379070870 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f1b3408bba0 con 0x7f1b3c0c7900 2024-04-25T01:32:25.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.230+0000 7f1b41dfd640 1 --2- 172.21.15.112:0/3379070870 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1b2006be10 0x7f1b2006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:25.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.231+0000 7f1b41dfd640 1 --2- 172.21.15.112:0/3379070870 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1b2006be10 0x7f1b2006e2d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1b38034000 tx=0x7f1b3802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:25.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.236+0000 7f1b327fc640 1 -- 172.21.15.112:0/3379070870 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1b34022070 con 0x7f1b3c0c7900 2024-04-25T01:32:25.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.536+0000 7f1b44061640 1 -- 172.21.15.112:0/3379070870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1b3c0c31c0 con 0x7f1b3c0c7900 2024-04-25T01:32:25.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.536+0000 7f1b327fc640 1 -- 172.21.15.112:0/3379070870 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f1b3c14f5f0 con 0x7f1b3c0c7900 2024-04-25T01:32:25.514 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:32:25.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.538+0000 7f1b44061640 1 -- 172.21.15.112:0/3379070870 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1b2006be10 msgr2=0x7f1b2006e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:25.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.538+0000 7f1b44061640 1 --2- 172.21.15.112:0/3379070870 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1b2006be10 0x7f1b2006e2d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1b38034000 tx=0x7f1b3802e000 comp rx=0 tx=0).stop 2024-04-25T01:32:25.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.538+0000 7f1b44061640 1 -- 172.21.15.112:0/3379070870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c0c7900 msgr2=0x7f1b3c155120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:25.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.538+0000 7f1b44061640 1 --2- 172.21.15.112:0/3379070870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c0c7900 0x7f1b3c155120 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400efc0 tx=0x7f1b3400c490 comp rx=0 tx=0).stop 2024-04-25T01:32:25.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.538+0000 7f1b44061640 1 -- 172.21.15.112:0/3379070870 shutdown_connections 2024-04-25T01:32:25.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.538+0000 7f1b44061640 1 --2- 172.21.15.112:0/3379070870 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1b2006be10 0x7f1b2006e2d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:25.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.538+0000 7f1b44061640 1 --2- 172.21.15.112:0/3379070870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c0c7900 0x7f1b3c155120 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:25.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.538+0000 7f1b44061640 1 --2- 172.21.15.112:0/3379070870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b3c0c5db0 0x7f1b3c154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:25.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.538+0000 7f1b44061640 1 --2- 172.21.15.112:0/3379070870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0c33e0 0x7f1b3c06d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:25.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.538+0000 7f1b44061640 1 -- 172.21.15.112:0/3379070870 >> 172.21.15.112:0/3379070870 conn(0x7f1b3c0c91d0 msgr2=0x7f1b3c0aad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:25.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.538+0000 7f1b44061640 1 -- 172.21.15.112:0/3379070870 shutdown_connections 2024-04-25T01:32:25.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:25.538+0000 7f1b44061640 1 -- 172.21.15.112:0/3379070870 wait complete. 2024-04-25T01:32:25.911 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:32:25.913 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:32:28.527 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:32:28.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.662+0000 7f3a013b6640 1 Processor -- start 2024-04-25T01:32:28.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.662+0000 7f3a013b6640 1 -- start start 2024-04-25T01:32:28.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.663+0000 7f3a013b6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39fc14c7b0 0x7f39fc155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:28.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.663+0000 7f3a013b6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 0x7f39fc1563f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:28.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.663+0000 7f3a013b6640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39fc153500 con 0x7f39fc06ee60 2024-04-25T01:32:28.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.663+0000 7f3a013b6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f39fc152fd0 con 0x7f39fc0c4780 2024-04-25T01:32:28.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.663+0000 7f3a013b6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f39fc0c2c60 con 0x7f39fc14c7b0 2024-04-25T01:32:28.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.663+0000 7f39faffd640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39fc14c7b0 msgr2=0x7f39fc155eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:32:28.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.663+0000 7f39faffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39fc14c7b0 0x7f39fc155eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:32:28.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.663+0000 7f39f3fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 0x7f39fc1563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:28.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.663+0000 7f39f3fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 0x7f39fc1563f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:37350/0 (socket says 172.21.15.112:37350) 2024-04-25T01:32:28.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.663+0000 7f39f3fff640 1 -- 172.21.15.112:0/3007872621 learned_addr learned my addr 172.21.15.112:0/3007872621 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:28.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.663+0000 7f39fb7fe640 1 --1- 172.21.15.112:0/3007872621 >> v1:172.21.15.187:6789/0 conn(0x7f39fc06ee60 0x7f39fc06f310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:42814/0 (socket says 172.21.15.112:42814) 2024-04-25T01:32:28.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.663+0000 7f39f3fff640 1 -- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 msgr2=0x7f39fc1563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:32:28.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.663+0000 7f39f3fff640 1 -- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 msgr2=0x7f39fc1563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:32:28.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.663+0000 7f39f3fff640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 0x7f39fc1563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:28.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.663+0000 7f39f3fff640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 0x7f39fc1563f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:32:28.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.864+0000 7f39faffd640 1 -- 172.21.15.112:0/3007872621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39fc14c7b0 msgr2=0x7f39fc155eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:32:28.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.864+0000 7f39faffd640 1 --2- 172.21.15.112:0/3007872621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39fc14c7b0 0x7f39fc155eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:32:28.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.864+0000 7f39f3fff640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 0x7f39fc1563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:28.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.864+0000 7f39f3fff640 1 -- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 msgr2=0x7f39fc1563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:32:28.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.864+0000 7f39f3fff640 1 -- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 msgr2=0x7f39fc1563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:32:28.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.864+0000 7f39f3fff640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 0x7f39fc1563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:28.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:28.864+0000 7f39f3fff640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 0x7f39fc1563f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:32:29.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:29.265+0000 7f39faffd640 1 -- 172.21.15.112:0/3007872621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39fc14c7b0 msgr2=0x7f39fc155eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:32:29.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:29.265+0000 7f39faffd640 1 --2- 172.21.15.112:0/3007872621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39fc14c7b0 0x7f39fc155eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:32:29.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:29.265+0000 7f39f3fff640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 0x7f39fc1563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:29.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:29.265+0000 7f39f3fff640 1 -- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 msgr2=0x7f39fc1563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:32:29.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:29.265+0000 7f39f3fff640 1 -- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 msgr2=0x7f39fc1563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:32:29.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:29.265+0000 7f39f3fff640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 0x7f39fc1563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:29.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:29.265+0000 7f39f3fff640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 0x7f39fc1563f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:32:30.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:30.066+0000 7f39faffd640 1 -- 172.21.15.112:0/3007872621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39fc14c7b0 msgr2=0x7f39fc155eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:32:30.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:30.066+0000 7f39faffd640 1 --2- 172.21.15.112:0/3007872621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39fc14c7b0 0x7f39fc155eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:32:30.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:30.066+0000 7f39f3fff640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 0x7f39fc1563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:30.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:30.066+0000 7f39f3fff640 1 -- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 msgr2=0x7f39fc1563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:32:30.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:30.066+0000 7f39f3fff640 1 -- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 msgr2=0x7f39fc1563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:32:30.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:30.066+0000 7f39f3fff640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 0x7f39fc1563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:30.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:30.066+0000 7f39f3fff640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 0x7f39fc1563f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:32:31.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:31.663+0000 7f39f97fa640 1 -- 172.21.15.112:0/3007872621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39fc14c7b0 msgr2=0x7f39fc155eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:32:31.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:31.663+0000 7f39f97fa640 1 --2- 172.21.15.112:0/3007872621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39fc14c7b0 0x7f39fc155eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:31.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:31.663+0000 7f39f97fa640 1 -- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 msgr2=0x7f39fc1563f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:32:31.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:31.663+0000 7f39f97fa640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.187:3300/0 conn(0x7f39fc0c4780 0x7f39fc1563f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:31.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:31.663+0000 7f39f97fa640 1 -- 172.21.15.112:0/3007872621 >> v1:172.21.15.187:6789/0 conn(0x7f39fc06ee60 legacy=0x7f39fc06f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:31.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:31.663+0000 7f39f97fa640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.112:3300/0 conn(0x7f39dc0036c0 0x7f39dc005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:31.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:31.663+0000 7f39f97fa640 1 -- 172.21.15.112:0/3007872621 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39fc0c3270 con 0x7f39dc006230 2024-04-25T01:32:31.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:31.663+0000 7f39f97fa640 1 -- 172.21.15.112:0/3007872621 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f39fc152fd0 con 0x7f39dc0036c0 2024-04-25T01:32:31.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:31.663+0000 7f39faffd640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.112:3300/0 conn(0x7f39dc0036c0 0x7f39dc005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:31.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:31.663+0000 7f39faffd640 1 -- 172.21.15.112:0/3007872621 >> v2:172.21.15.112:3300/0 conn(0x7f39dc0036c0 msgr2=0x7f39dc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:32:31.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:31.663+0000 7f39faffd640 1 -- 172.21.15.112:0/3007872621 >> v2:172.21.15.112:3300/0 conn(0x7f39dc0036c0 msgr2=0x7f39dc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:32:31.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:31.663+0000 7f39faffd640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.112:3300/0 conn(0x7f39dc0036c0 0x7f39dc005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:31.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:31.663+0000 7f39faffd640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.112:3300/0 conn(0x7f39dc0036c0 0x7f39dc005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:32:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:31.864+0000 7f39faffd640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.112:3300/0 conn(0x7f39dc0036c0 0x7f39dc005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:31.864+0000 7f39faffd640 1 -- 172.21.15.112:0/3007872621 >> v2:172.21.15.112:3300/0 conn(0x7f39dc0036c0 msgr2=0x7f39dc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:32:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:31.864+0000 7f39faffd640 1 -- 172.21.15.112:0/3007872621 >> v2:172.21.15.112:3300/0 conn(0x7f39dc0036c0 msgr2=0x7f39dc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:32:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:31.864+0000 7f39faffd640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.112:3300/0 conn(0x7f39dc0036c0 0x7f39dc005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:31.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:31.864+0000 7f39faffd640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.112:3300/0 conn(0x7f39dc0036c0 0x7f39dc005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:32:32.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.265+0000 7f39faffd640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.112:3300/0 conn(0x7f39dc0036c0 0x7f39dc005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:32.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.265+0000 7f39faffd640 1 -- 172.21.15.112:0/3007872621 >> v2:172.21.15.112:3300/0 conn(0x7f39dc0036c0 msgr2=0x7f39dc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:32:32.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.265+0000 7f39faffd640 1 -- 172.21.15.112:0/3007872621 >> v2:172.21.15.112:3300/0 conn(0x7f39dc0036c0 msgr2=0x7f39dc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:32:32.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.265+0000 7f39faffd640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.112:3300/0 conn(0x7f39dc0036c0 0x7f39dc005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:32.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.265+0000 7f39faffd640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.112:3300/0 conn(0x7f39dc0036c0 0x7f39dc005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:32:32.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.574+0000 7f39fa7fc640 1 -- 172.21.15.112:0/3007872621 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 619323540 0 0) 0x7f39fc0c3270 con 0x7f39dc006230 2024-04-25T01:32:32.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.574+0000 7f39fa7fc640 1 -- 172.21.15.112:0/3007872621 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39e8003520 con 0x7f39dc006230 2024-04-25T01:32:32.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.576+0000 7f39fa7fc640 1 -- 172.21.15.112:0/3007872621 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2124261331 0 0) 0x7f39e8003520 con 0x7f39dc006230 2024-04-25T01:32:32.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.576+0000 7f39fa7fc640 1 -- 172.21.15.112:0/3007872621 >> v2:172.21.15.112:3300/0 conn(0x7f39dc0036c0 msgr2=0x7f39dc005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:32:32.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.576+0000 7f39fa7fc640 1 --2- 172.21.15.112:0/3007872621 >> v2:172.21.15.112:3300/0 conn(0x7f39dc0036c0 0x7f39dc005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:32.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.576+0000 7f39fa7fc640 1 -- 172.21.15.112:0/3007872621 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39dc00afc0 con 0x7f39dc006230 2024-04-25T01:32:32.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.576+0000 7f39fa7fc640 1 -- 172.21.15.112:0/3007872621 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f39e400f040 con 0x7f39dc006230 2024-04-25T01:32:32.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.576+0000 7f39fa7fc640 1 -- 172.21.15.112:0/3007872621 >> v1:172.21.15.112:6789/0 conn(0x7f39dc006230 legacy=0x7f39dc0086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:32.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.576+0000 7f39fb7fe640 1 -- 172.21.15.112:0/3007872621 reap_dead start 2024-04-25T01:32:32.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.576+0000 7f39fa7fc640 1 --2- 172.21.15.112:0/3007872621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39e80039b0 0x7f39e8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:32.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.576+0000 7f39fa7fc640 1 --2- 172.21.15.112:0/3007872621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39e8004800 0x7f39e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:32.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.576+0000 7f39fa7fc640 1 --2- 172.21.15.112:0/3007872621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39e8005210 0x7f39e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:32.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.576+0000 7f39fa7fc640 1 -- 172.21.15.112:0/3007872621 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc152fd0 con 0x7f39e80039b0 2024-04-25T01:32:32.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.576+0000 7f39fa7fc640 1 -- 172.21.15.112:0/3007872621 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f39e800c140 con 0x7f39e8004800 2024-04-25T01:32:32.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.576+0000 7f39fa7fc640 1 -- 172.21.15.112:0/3007872621 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f39e800c340 con 0x7f39e8005210 2024-04-25T01:32:32.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.576+0000 7f39fb7fe640 1 -- 172.21.15.112:0/3007872621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39e8005210 msgr2=0x7f39e800b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:32:32.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.576+0000 7f39fb7fe640 1 --2- 172.21.15.112:0/3007872621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39e8005210 0x7f39e800b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:32:32.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.576+0000 7f39f3fff640 1 --2- 172.21.15.112:0/3007872621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39e8004800 0x7f39e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:32.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.577+0000 7f39f3fff640 1 -- 172.21.15.112:0/3007872621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39e8005210 msgr2=0x7f39e800b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:32:32.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.577+0000 7f39f3fff640 1 --2- 172.21.15.112:0/3007872621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39e8005210 0x7f39e800b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:32.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.577+0000 7f39f3fff640 1 -- 172.21.15.112:0/3007872621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39e80039b0 msgr2=0x7f39e8003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:32:32.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.577+0000 7f39f3fff640 1 --2- 172.21.15.112:0/3007872621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39e80039b0 0x7f39e8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:32.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.577+0000 7f39f3fff640 1 -- 172.21.15.112:0/3007872621 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e8003520 con 0x7f39e8004800 2024-04-25T01:32:32.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.577+0000 7f39f3fff640 1 --2- 172.21.15.112:0/3007872621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39e8004800 0x7f39e8004c80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0c4c00 tx=0x7f39e8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eed0c3cb5c8d826d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:32.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.578+0000 7f39fa7fc640 1 -- 172.21.15.112:0/3007872621 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e4031070 con 0x7f39e8004800 2024-04-25T01:32:32.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.578+0000 7f39fa7fc640 1 -- 172.21.15.112:0/3007872621 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39e402be50 con 0x7f39e8004800 2024-04-25T01:32:32.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.578+0000 7f39fa7fc640 1 -- 172.21.15.112:0/3007872621 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e400cb60 con 0x7f39e8004800 2024-04-25T01:32:32.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.578+0000 7f3a013b6640 1 -- 172.21.15.112:0/3007872621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39e8004800 msgr2=0x7f39e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:32.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.578+0000 7f3a013b6640 1 --2- 172.21.15.112:0/3007872621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39e8004800 0x7f39e8004c80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0c4c00 tx=0x7f39e8005830 comp rx=0 tx=0).stop 2024-04-25T01:32:32.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.578+0000 7f3a013b6640 1 -- 172.21.15.112:0/3007872621 shutdown_connections 2024-04-25T01:32:32.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.578+0000 7f3a013b6640 1 --2- 172.21.15.112:0/3007872621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39e8005210 0x7f39e800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:32.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.578+0000 7f3a013b6640 1 --2- 172.21.15.112:0/3007872621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39e8004800 0x7f39e8004c80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:32.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.578+0000 7f3a013b6640 1 --2- 172.21.15.112:0/3007872621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39e80039b0 0x7f39e8003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:32.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.578+0000 7f3a013b6640 1 -- 172.21.15.112:0/3007872621 >> 172.21.15.112:0/3007872621 conn(0x7f39fc0c8130 msgr2=0x7f39fc0c8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:32.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.578+0000 7f3a013b6640 1 -- 172.21.15.112:0/3007872621 shutdown_connections 2024-04-25T01:32:32.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.578+0000 7f3a013b6640 1 -- 172.21.15.112:0/3007872621 wait complete. 2024-04-25T01:32:32.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.580+0000 7f3a013b6640 1 Processor -- start 2024-04-25T01:32:32.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.580+0000 7f3a013b6640 1 -- start start 2024-04-25T01:32:32.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.581+0000 7f3a013b6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39e80039b0 0x7f39fc1e8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:32.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.581+0000 7f3a013b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39e8004800 0x7f39fc14f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:32.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.581+0000 7f3a013b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39e8005210 0x7f39fc1ea8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:32.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.581+0000 7f39faffd640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39e80039b0 msgr2=0x7f39fc1e8530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:32:32.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.581+0000 7f39faffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39e80039b0 0x7f39fc1e8530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:32:32.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.581+0000 7f3a013b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc06fdf0 con 0x7f39e8005210 2024-04-25T01:32:32.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.581+0000 7f3a013b6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc0c2f00 con 0x7f39e8004800 2024-04-25T01:32:32.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.581+0000 7f3a013b6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f39fc14eec0 con 0x7f39e80039b0 2024-04-25T01:32:32.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.581+0000 7f39f3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39e8004800 0x7f39fc14f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:32.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.581+0000 7f39fb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39e8005210 0x7f39fc1ea8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:32.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.581+0000 7f39f3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39e8004800 0x7f39fc14f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:37396/0 (socket says 172.21.15.112:37396) 2024-04-25T01:32:32.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.581+0000 7f39f3fff640 1 -- 172.21.15.112:0/813235176 learned_addr learned my addr 172.21.15.112:0/813235176 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:32.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.581+0000 7f39fb7fe640 1 -- 172.21.15.112:0/813235176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39e80039b0 msgr2=0x7f39fc1e8530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:32:32.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.581+0000 7f39fb7fe640 1 --2- 172.21.15.112:0/813235176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39e80039b0 0x7f39fc1e8530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:32.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.581+0000 7f39fb7fe640 1 -- 172.21.15.112:0/813235176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39e8004800 msgr2=0x7f39fc14f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:32.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.581+0000 7f39fb7fe640 1 --2- 172.21.15.112:0/813235176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39e8004800 0x7f39fc14f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:32.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.581+0000 7f39fb7fe640 1 -- 172.21.15.112:0/813235176 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39fc14e8b0 con 0x7f39e8005210 2024-04-25T01:32:32.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.581+0000 7f39f3fff640 1 --2- 172.21.15.112:0/813235176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39e8004800 0x7f39fc14f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:32:32.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.582+0000 7f39fb7fe640 1 --2- 172.21.15.112:0/813235176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39e8005210 0x7f39fc1ea8a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f39fc073870 tx=0x7f39f400ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:32.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.584+0000 7f39f37fe640 1 -- 172.21.15.112:0/813235176 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4019070 con 0x7f39e8005210 2024-04-25T01:32:32.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.584+0000 7f39f37fe640 1 -- 172.21.15.112:0/813235176 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39f400ca20 con 0x7f39e8005210 2024-04-25T01:32:32.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.584+0000 7f3a013b6640 1 -- 172.21.15.112:0/813235176 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39fc1eb1c0 con 0x7f39e8005210 2024-04-25T01:32:32.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.584+0000 7f39f37fe640 1 -- 172.21.15.112:0/813235176 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f400cdf0 con 0x7f39e8005210 2024-04-25T01:32:32.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.584+0000 7f3a013b6640 1 -- 172.21.15.112:0/813235176 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39fc14ce00 con 0x7f39e8005210 2024-04-25T01:32:32.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.585+0000 7f3a013b6640 1 -- 172.21.15.112:0/813235176 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39fc0689b0 con 0x7f39e8005210 2024-04-25T01:32:32.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.585+0000 7f39f37fe640 1 -- 172.21.15.112:0/813235176 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f39f401c070 con 0x7f39e8005210 2024-04-25T01:32:32.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.586+0000 7f39f37fe640 1 --2- 172.21.15.112:0/813235176 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f39dc06c720 0x7f39dc06ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:32.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.586+0000 7f39f37fe640 1 -- 172.21.15.112:0/813235176 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f39f408c890 con 0x7f39e8005210 2024-04-25T01:32:32.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.586+0000 7f39faffd640 1 --2- 172.21.15.112:0/813235176 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f39dc06c720 0x7f39dc06ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:32.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.586+0000 7f39faffd640 1 --2- 172.21.15.112:0/813235176 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f39dc06c720 0x7f39dc06ebe0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f39fc1ea590 tx=0x7f39ec005040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:32.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.591+0000 7f39f37fe640 1 -- 172.21.15.112:0/813235176 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f39f405a8e0 con 0x7f39e8005210 2024-04-25T01:32:32.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.897+0000 7f3a013b6640 1 -- 172.21.15.112:0/813235176 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f39fc06ff70 con 0x7f39e8005210 2024-04-25T01:32:32.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.898+0000 7f39f37fe640 1 -- 172.21.15.112:0/813235176 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7f39f405ebe0 con 0x7f39e8005210 2024-04-25T01:32:32.875 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":50,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:32:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.900+0000 7f3a013b6640 1 -- 172.21.15.112:0/813235176 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f39dc06c720 msgr2=0x7f39dc06ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.900+0000 7f3a013b6640 1 --2- 172.21.15.112:0/813235176 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f39dc06c720 0x7f39dc06ebe0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f39fc1ea590 tx=0x7f39ec005040 comp rx=0 tx=0).stop 2024-04-25T01:32:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.900+0000 7f3a013b6640 1 -- 172.21.15.112:0/813235176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39e8005210 msgr2=0x7f39fc1ea8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.900+0000 7f3a013b6640 1 --2- 172.21.15.112:0/813235176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39e8005210 0x7f39fc1ea8a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f39fc073870 tx=0x7f39f400ef00 comp rx=0 tx=0).stop 2024-04-25T01:32:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.900+0000 7f3a013b6640 1 -- 172.21.15.112:0/813235176 shutdown_connections 2024-04-25T01:32:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.900+0000 7f3a013b6640 1 --2- 172.21.15.112:0/813235176 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f39dc06c720 0x7f39dc06ebe0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.900+0000 7f3a013b6640 1 --2- 172.21.15.112:0/813235176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39e8005210 0x7f39fc1ea8a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.900+0000 7f3a013b6640 1 --2- 172.21.15.112:0/813235176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39e8004800 0x7f39fc14f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.900+0000 7f3a013b6640 1 --2- 172.21.15.112:0/813235176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39e80039b0 0x7f39fc1e8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.900+0000 7f3a013b6640 1 -- 172.21.15.112:0/813235176 >> 172.21.15.112:0/813235176 conn(0x7f39fc0c8130 msgr2=0x7f39fc14b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.900+0000 7f3a013b6640 1 -- 172.21.15.112:0/813235176 shutdown_connections 2024-04-25T01:32:32.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:32.900+0000 7f3a013b6640 1 -- 172.21.15.112:0/813235176 wait complete. 2024-04-25T01:32:32.890 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T01:32:32.891 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-04-25T01:32:32.891 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:32:32.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.019+0000 7f7d90f14640 1 Processor -- start 2024-04-25T01:32:32.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.019+0000 7f7d90f14640 1 -- start start 2024-04-25T01:32:32.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.020+0000 7f7d90f14640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7d8c1592c0 0x7f7d8c0c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.020+0000 7f7d90f14640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7d8c0c72b0 0x7f7d8c160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.020+0000 7f7d90f14640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d8c159b90 con 0x7f7d8c1596a0 2024-04-25T01:32:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.020+0000 7f7d90f14640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f7d8c0c6e20 con 0x7f7d8c1592c0 2024-04-25T01:32:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.020+0000 7f7d90f14640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7d8c06c970 con 0x7f7d8c0c72b0 2024-04-25T01:32:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.020+0000 7f7d8a59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7d8c1592c0 0x7f7d8c0c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.020+0000 7f7d83fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7d8c1596a0 0x7f7d8c152d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:36546/0 (socket says 172.21.15.112:36546) 2024-04-25T01:32:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.020+0000 7f7d83fff640 1 -- 172.21.15.112:0/3394269224 learned_addr learned my addr 172.21.15.112:0/3394269224 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:32:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.020+0000 7f7d8ad9d640 1 --2- 172.21.15.112:0/3394269224 >> v2:172.21.15.187:3300/0 conn(0x7f7d8c0c72b0 0x7f7d8c160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.020+0000 7f7d8a59c640 1 -- 172.21.15.112:0/3394269224 >> v2:172.21.15.187:3300/0 conn(0x7f7d8c0c72b0 msgr2=0x7f7d8c160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:32:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.020+0000 7f7d8a59c640 1 --2- 172.21.15.112:0/3394269224 >> v2:172.21.15.187:3300/0 conn(0x7f7d8c0c72b0 0x7f7d8c160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.020+0000 7f7d8a59c640 1 -- 172.21.15.112:0/3394269224 >> v1:172.21.15.112:6789/0 conn(0x7f7d8c1596a0 legacy=0x7f7d8c152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.020+0000 7f7d8a59c640 1 -- 172.21.15.112:0/3394269224 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8c06cc50 con 0x7f7d8c1592c0 2024-04-25T01:32:32.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.020+0000 7f7d89d9b640 1 -- 172.21.15.112:0/3394269224 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2222384304 0 0) 0x7f7d8c159b90 con 0x7f7d8c1596a0 2024-04-25T01:32:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.021+0000 7f7d8a59c640 1 --2- 172.21.15.112:0/3394269224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c1592c0 0x7f7d8c0c9070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0c21d0 tx=0x7f7d7c02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6381e6f44098d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.021+0000 7f7d89d9b640 1 -- 172.21.15.112:0/3394269224 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c037070 con 0x7f7d8c1592c0 2024-04-25T01:32:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.021+0000 7f7d89d9b640 1 -- 172.21.15.112:0/3394269224 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d7c02add0 con 0x7f7d8c1592c0 2024-04-25T01:32:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.021+0000 7f7d89d9b640 1 -- 172.21.15.112:0/3394269224 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c032660 con 0x7f7d8c1592c0 2024-04-25T01:32:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.022+0000 7f7d90f14640 1 -- 172.21.15.112:0/3394269224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c1592c0 msgr2=0x7f7d8c0c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.022+0000 7f7d90f14640 1 --2- 172.21.15.112:0/3394269224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c1592c0 0x7f7d8c0c9070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0c21d0 tx=0x7f7d7c02a080 comp rx=0 tx=0).stop 2024-04-25T01:32:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.022+0000 7f7d90f14640 1 -- 172.21.15.112:0/3394269224 shutdown_connections 2024-04-25T01:32:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.022+0000 7f7d90f14640 1 --2- 172.21.15.112:0/3394269224 >> v2:172.21.15.187:3300/0 conn(0x7f7d8c0c72b0 0x7f7d8c160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.022+0000 7f7d90f14640 1 --2- 172.21.15.112:0/3394269224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c1592c0 0x7f7d8c0c9070 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.022+0000 7f7d90f14640 1 -- 172.21.15.112:0/3394269224 >> 172.21.15.112:0/3394269224 conn(0x7f7d8c0cb170 msgr2=0x7f7d8c0cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.023+0000 7f7d90f14640 1 -- 172.21.15.112:0/3394269224 shutdown_connections 2024-04-25T01:32:33.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.023+0000 7f7d90f14640 1 -- 172.21.15.112:0/3394269224 wait complete. 2024-04-25T01:32:33.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.024+0000 7f7d90f14640 1 Processor -- start 2024-04-25T01:32:33.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.024+0000 7f7d90f14640 1 -- start start 2024-04-25T01:32:33.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.025+0000 7f7d90f14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c0c72b0 0x7f7d8c1eed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:33.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.025+0000 7f7d90f14640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d8c1592c0 0x7f7d8c1ef2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:33.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.025+0000 7f7d90f14640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d8c1596a0 0x7f7d8c1eca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:33.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.025+0000 7f7d90f14640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c153590 con 0x7f7d8c0c72b0 2024-04-25T01:32:33.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.025+0000 7f7d90f14640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c153410 con 0x7f7d8c1592c0 2024-04-25T01:32:33.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.025+0000 7f7d90f14640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8c06db20 con 0x7f7d8c1596a0 2024-04-25T01:32:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.025+0000 7f7d8ad9d640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d8c1596a0 msgr2=0x7f7d8c1eca20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:32:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.025+0000 7f7d8ad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d8c1596a0 0x7f7d8c1eca20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:32:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.026+0000 7f7d83fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d8c1592c0 0x7f7d8c1ef2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.026+0000 7f7d8a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c0c72b0 0x7f7d8c1eed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.026+0000 7f7d83fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d8c1592c0 0x7f7d8c1ef2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:37416/0 (socket says 172.21.15.112:37416) 2024-04-25T01:32:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.026+0000 7f7d83fff640 1 -- 172.21.15.112:0/4091655743 learned_addr learned my addr 172.21.15.112:0/4091655743 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.026+0000 7f7d8a59c640 1 -- 172.21.15.112:0/4091655743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d8c1596a0 msgr2=0x7f7d8c1eca20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:32:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.026+0000 7f7d8a59c640 1 --2- 172.21.15.112:0/4091655743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d8c1596a0 0x7f7d8c1eca20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.026+0000 7f7d8a59c640 1 -- 172.21.15.112:0/4091655743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d8c1592c0 msgr2=0x7f7d8c1ef2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.026+0000 7f7d8a59c640 1 --2- 172.21.15.112:0/4091655743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d8c1592c0 0x7f7d8c1ef2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.026+0000 7f7d8a59c640 1 -- 172.21.15.112:0/4091655743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8c1ef810 con 0x7f7d8c0c72b0 2024-04-25T01:32:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.026+0000 7f7d8a59c640 1 --2- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c0c72b0 0x7f7d8c1eed90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c031000 tx=0x7f7d7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.026+0000 7f7d82ffd640 1 -- 172.21.15.112:0/4091655743 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c041030 con 0x7f7d8c0c72b0 2024-04-25T01:32:33.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.026+0000 7f7d82ffd640 1 -- 172.21.15.112:0/4091655743 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d7c02abd0 con 0x7f7d8c0c72b0 2024-04-25T01:32:33.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.026+0000 7f7d82ffd640 1 -- 172.21.15.112:0/4091655743 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c037040 con 0x7f7d8c0c72b0 2024-04-25T01:32:33.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.026+0000 7f7d90f14640 1 -- 172.21.15.112:0/4091655743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8c155740 con 0x7f7d8c0c72b0 2024-04-25T01:32:33.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.027+0000 7f7d90f14640 1 -- 172.21.15.112:0/4091655743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d8c154c60 con 0x7f7d8c0c72b0 2024-04-25T01:32:33.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.028+0000 7f7d90f14640 1 --2- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c14a090 0x7f7d8c1ecf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:33.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.028+0000 7f7d90f14640 1 -- 172.21.15.112:0/4091655743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c0c2e70 con 0x7f7d8c14a090 2024-04-25T01:32:33.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.028+0000 7f7d83fff640 1 --2- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c14a090 0x7f7d8c1ecf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:33.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.028+0000 7f7d83fff640 1 -- 172.21.15.112:0/4091655743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7d8c075980 con 0x7f7d8c14a090 2024-04-25T01:32:33.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.028+0000 7f7d83fff640 1 --2- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c14a090 0x7f7d8c1ecf60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c158ea0 tx=0x7f7d74001950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:33.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.028+0000 7f7d82ffd640 1 -- 172.21.15.112:0/4091655743 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d7c03d640 con 0x7f7d8c0c72b0 2024-04-25T01:32:33.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.029+0000 7f7d82ffd640 1 --2- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7d6c06be10 0x7f7d6c06e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:33.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.029+0000 7f7d82ffd640 1 -- 172.21.15.112:0/4091655743 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f7d7c0ab060 con 0x7f7d8c0c72b0 2024-04-25T01:32:33.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.029+0000 7f7d8ad9d640 1 --2- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7d6c06be10 0x7f7d6c06e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:33.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.029+0000 7f7d82ffd640 1 -- 172.21.15.112:0/4091655743 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7400f5e0 con 0x7f7d8c14a090 2024-04-25T01:32:33.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.029+0000 7f7d82ffd640 1 -- 172.21.15.112:0/4091655743 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7d74018070 con 0x7f7d8c14a090 2024-04-25T01:32:33.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.029+0000 7f7d82ffd640 1 -- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c14a090 msgr2=0x7f7d8c1ecf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:33.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.029+0000 7f7d82ffd640 1 --2- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c14a090 0x7f7d8c1ecf60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c158ea0 tx=0x7f7d74001950 comp rx=0 tx=0).stop 2024-04-25T01:32:33.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.029+0000 7f7d82ffd640 1 -- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c14a090 msgr2=0x7f7d8c1ecf60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:32:33.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.029+0000 7f7d82ffd640 1 --2- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c14a090 0x7f7d8c1ecf60 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c158ea0 tx=0x7f7d74001950 comp rx=0 tx=0).stop 2024-04-25T01:32:33.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.029+0000 7f7d8ad9d640 1 --2- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7d6c06be10 0x7f7d6c06e2d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009b10 tx=0x7f7d8400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:33.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.036+0000 7f7d90f14640 1 --2- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c149010 0x7f7d8c1493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:33.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.036+0000 7f7d90f14640 1 -- 172.21.15.112:0/4091655743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c0cc050 con 0x7f7d8c149010 2024-04-25T01:32:33.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.036+0000 7f7d83fff640 1 --2- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c149010 0x7f7d8c1493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:33.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.037+0000 7f7d83fff640 1 -- 172.21.15.112:0/4091655743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7d8c1ed4a0 con 0x7f7d8c149010 2024-04-25T01:32:33.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.037+0000 7f7d83fff640 1 --2- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c149010 0x7f7d8c1493f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c153920 tx=0x7f7d74001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:33.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.037+0000 7f7d82ffd640 1 -- 172.21.15.112:0/4091655743 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7400fc10 con 0x7f7d8c149010 2024-04-25T01:32:33.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.037+0000 7f7d82ffd640 1 -- 172.21.15.112:0/4091655743 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4719 (secure 0 0 0) 0x7f7d7400fc10 con 0x7f7d8c149010 2024-04-25T01:32:33.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.037+0000 7f7d82ffd640 1 -- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c149010 msgr2=0x7f7d8c1493f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:33.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.037+0000 7f7d82ffd640 1 --2- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c149010 0x7f7d8c1493f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c153920 tx=0x7f7d74001080 comp rx=0 tx=0).stop 2024-04-25T01:32:33.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.037+0000 7f7d82ffd640 1 -- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c149010 msgr2=0x7f7d8c1493f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:32:33.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.037+0000 7f7d82ffd640 1 --2- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c149010 0x7f7d8c1493f0 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c153920 tx=0x7f7d74001080 comp rx=0 tx=0).stop 2024-04-25T01:32:33.015 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:32:33.015 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:32:33.015 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:32:33.016 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:32:33.016 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 50, 2024-04-25T01:32:33.016 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 33803, 2024-04-25T01:32:33.016 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:32:33.016 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:32:33.016 INFO:teuthology.orchestra.run.smithi112.stdout: 1 2024-04-25T01:32:33.016 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:33.016 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:32:33.016 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:32:33.016 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:32:33.016 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:32:33.016 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:33.016 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:33.016 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:33.016 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:33.017 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:33.017 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:33.017 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:33.017 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:33.017 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:33.017 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:33.017 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:33.017 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:33.017 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:32:33.017 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:32:33.017 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:33.017 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:33.017 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:33.017 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:33.017 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:33.018 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:33.018 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:33.018 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:33.018 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:33.018 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:33.018 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:33.018 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:33.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.018 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:32:33.018 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:32:33.018 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:32:33.018 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:32:33.018 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:32:33.018 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:32:33.018 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:32:33.019 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:32:33.019 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:32:33.019 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:32:33.019 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:32:33.019 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:32:33.019 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:32:33.019 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:32:33.019 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:32:33.019 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:32:33.019 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:32:33.019 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:33.019 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:33.019 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:33.019 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:33.020 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:33.020 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:33.020 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:33.020 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:33.020 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:33.020 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:33.020 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:33.020 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:33.020 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:32:33.020 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.020 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:32:33.020 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.020 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:32:33.020 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:32:33.020 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:33.021 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:33.021 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.021 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:33.021 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:32:33.021 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:33.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.021 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.021 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:33.021 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:32:33.021 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:33.021 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:33.021 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:33.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.021 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:32:33.022 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:32:33.022 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:33.022 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:33.022 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:33.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.022 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.022 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:32:33.022 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:32:33.022 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:33.022 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:33.022 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.022 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:33.022 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:32:33.022 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:33.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.023 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.023 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:33.023 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:32:33.023 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:33.023 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:33.023 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:33.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.023 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:32:33.023 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:32:33.023 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:33.023 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:33.023 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:33.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.023 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.023 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:32:33.024 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:32:33.024 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:33.024 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:33.024 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.024 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:33.024 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:32:33.024 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:33.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.024 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.024 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:33.024 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:32:33.024 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:33.024 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:33.024 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:33.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.025 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:32:33.025 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:32:33.025 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:33.025 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:33.025 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:33.025 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:33.025 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:33.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.025 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:32:33.025 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:32:33.025 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.025 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:33.025 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:33.025 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:32:33.025 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:33.026 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:33.026 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:32:33.026 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.026 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:33.026 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:33.026 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:32:33.026 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:33.026 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:33.026 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:32:33.026 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.026 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:33.026 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:33.026 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:32:33.027 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:33.027 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:33.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.027 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:32:33.027 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:32:33.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.038+0000 7f7d90f14640 1 -- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7d6c06be10 msgr2=0x7f7d6c06e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:33.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.038+0000 7f7d90f14640 1 --2- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7d6c06be10 0x7f7d6c06e2d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009b10 tx=0x7f7d8400b040 comp rx=0 tx=0).stop 2024-04-25T01:32:33.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.038+0000 7f7d90f14640 1 -- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c0c72b0 msgr2=0x7f7d8c1eed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:33.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.038+0000 7f7d90f14640 1 --2- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c0c72b0 0x7f7d8c1eed90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c031000 tx=0x7f7d7c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:32:33.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.038+0000 7f7d8ad9d640 1 -- 172.21.15.112:0/4091655743 reap_dead start 2024-04-25T01:32:33.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.038+0000 7f7d90f14640 1 -- 172.21.15.112:0/4091655743 shutdown_connections 2024-04-25T01:32:33.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.038+0000 7f7d90f14640 1 --2- 172.21.15.112:0/4091655743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c0c72b0 0x7f7d8c1eed90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:33.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.038+0000 7f7d90f14640 1 -- 172.21.15.112:0/4091655743 >> 172.21.15.112:0/4091655743 conn(0x7f7d8c0cb170 msgr2=0x7f7d8c0ab3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:33.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.038+0000 7f7d90f14640 1 -- 172.21.15.112:0/4091655743 shutdown_connections 2024-04-25T01:32:33.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.038+0000 7f7d90f14640 1 -- 172.21.15.112:0/4091655743 wait complete. 2024-04-25T01:32:33.030 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:32:33.030 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-04-25T01:32:33.030 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:32:33.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.167+0000 7fa5d3f12640 1 Processor -- start 2024-04-25T01:32:33.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.168+0000 7fa5d3f12640 1 -- start start 2024-04-25T01:32:33.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.169+0000 7fa5d3f12640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5cc072660 0x7fa5cc158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:33.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.169+0000 7fa5d3f12640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa5cc071e00 0x7fa5cc0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:33.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.169+0000 7fa5d3f12640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5cc155a10 con 0x7fa5cc152f80 2024-04-25T01:32:33.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.169+0000 7fa5d3f12640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa5cc072d10 con 0x7fa5cc071e00 2024-04-25T01:32:33.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.169+0000 7fa5d3f12640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa5cc072270 con 0x7fa5cc072660 2024-04-25T01:32:33.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.169+0000 7fa5d14ad640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5cc072660 msgr2=0x7fa5cc158ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:32:33.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.169+0000 7fa5d14ad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5cc072660 0x7fa5cc158ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:32:33.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.169+0000 7fa5d24af640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa5cc071e00 0x7fa5cc0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:33.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.169+0000 7fa5d24af640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa5cc071e00 0x7fa5cc0c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:37424/0 (socket says 172.21.15.112:37424) 2024-04-25T01:32:33.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.169+0000 7fa5d24af640 1 -- 172.21.15.112:0/2597516288 learned_addr learned my addr 172.21.15.112:0/2597516288 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:33.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.169+0000 7fa5d24af640 1 -- 172.21.15.112:0/2597516288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5cc072660 msgr2=0x7fa5cc158ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:32:33.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.169+0000 7fa5d24af640 1 --2- 172.21.15.112:0/2597516288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5cc072660 0x7fa5cc158ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:33.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.169+0000 7fa5d24af640 1 -- 172.21.15.112:0/2597516288 >> v1:172.21.15.187:6789/0 conn(0x7fa5cc152f80 legacy=0x7fa5cc1561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:33.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.169+0000 7fa5d24af640 1 -- 172.21.15.112:0/2597516288 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5cc0aa060 con 0x7fa5cc071e00 2024-04-25T01:32:33.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.170+0000 7fa5d24af640 1 --2- 172.21.15.112:0/2597516288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc071e00 0x7fa5cc0c50a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009e50 tx=0x7fa5c402a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae75a8312316714e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:33.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.170+0000 7fa5d0cac640 1 -- 172.21.15.112:0/2597516288 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4036070 con 0x7fa5cc071e00 2024-04-25T01:32:33.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.170+0000 7fa5d0cac640 1 -- 172.21.15.112:0/2597516288 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5c4031600 con 0x7fa5cc071e00 2024-04-25T01:32:33.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.170+0000 7fa5d0cac640 1 -- 172.21.15.112:0/2597516288 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c40319d0 con 0x7fa5cc071e00 2024-04-25T01:32:33.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.171+0000 7fa5d3f12640 1 -- 172.21.15.112:0/2597516288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc071e00 msgr2=0x7fa5cc0c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:33.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.171+0000 7fa5d3f12640 1 --2- 172.21.15.112:0/2597516288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc071e00 0x7fa5cc0c50a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009e50 tx=0x7fa5c402a610 comp rx=0 tx=0).stop 2024-04-25T01:32:33.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.171+0000 7fa5d3f12640 1 -- 172.21.15.112:0/2597516288 shutdown_connections 2024-04-25T01:32:33.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.171+0000 7fa5d3f12640 1 --2- 172.21.15.112:0/2597516288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc071e00 0x7fa5cc0c50a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:33.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.171+0000 7fa5d3f12640 1 --2- 172.21.15.112:0/2597516288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5cc072660 0x7fa5cc158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:33.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.171+0000 7fa5d3f12640 1 -- 172.21.15.112:0/2597516288 >> 172.21.15.112:0/2597516288 conn(0x7fa5cc06c6c0 msgr2=0x7fa5cc06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:33.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.171+0000 7fa5d3f12640 1 -- 172.21.15.112:0/2597516288 shutdown_connections 2024-04-25T01:32:33.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.171+0000 7fa5d3f12640 1 -- 172.21.15.112:0/2597516288 wait complete. 2024-04-25T01:32:33.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.172+0000 7fa5d3f12640 1 Processor -- start 2024-04-25T01:32:33.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.172+0000 7fa5d3f12640 1 -- start start 2024-04-25T01:32:33.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.172+0000 7fa5d3f12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5cc071e00 0x7fa5cc14fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:33.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.172+0000 7fa5d3f12640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5cc072660 0x7fa5cc0c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:33.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.172+0000 7fa5d3f12640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc152f80 0x7fa5cc151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:33.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.172+0000 7fa5d3f12640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc1568e0 con 0x7fa5cc071e00 2024-04-25T01:32:33.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.172+0000 7fa5d3f12640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc153360 con 0x7fa5cc152f80 2024-04-25T01:32:33.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.172+0000 7fa5d3f12640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa5cc155cb0 con 0x7fa5cc072660 2024-04-25T01:32:33.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.173+0000 7fa5d14ad640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5cc072660 msgr2=0x7fa5cc0c8060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:32:33.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.173+0000 7fa5d14ad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5cc072660 0x7fa5cc0c8060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:32:33.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.173+0000 7fa5d1cae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5cc071e00 0x7fa5cc14fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:33.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.173+0000 7fa5d24af640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc152f80 0x7fa5cc151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:33.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.173+0000 7fa5d1cae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5cc071e00 0x7fa5cc14fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55250/0 (socket says 172.21.15.112:55250) 2024-04-25T01:32:33.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.173+0000 7fa5d1cae640 1 -- 172.21.15.112:0/2983774298 learned_addr learned my addr 172.21.15.112:0/2983774298 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:33.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.173+0000 7fa5d24af640 1 -- 172.21.15.112:0/2983774298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5cc072660 msgr2=0x7fa5cc0c8060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:32:33.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.173+0000 7fa5d24af640 1 --2- 172.21.15.112:0/2983774298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5cc072660 0x7fa5cc0c8060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:33.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.173+0000 7fa5d24af640 1 -- 172.21.15.112:0/2983774298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5cc071e00 msgr2=0x7fa5cc14fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:33.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.173+0000 7fa5d24af640 1 --2- 172.21.15.112:0/2983774298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5cc071e00 0x7fa5cc14fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:33.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.173+0000 7fa5d24af640 1 -- 172.21.15.112:0/2983774298 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5cc1ef4a0 con 0x7fa5cc152f80 2024-04-25T01:32:33.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.173+0000 7fa5d1cae640 1 --2- 172.21.15.112:0/2983774298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5cc071e00 0x7fa5cc14fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:32:33.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.173+0000 7fa5d24af640 1 --2- 172.21.15.112:0/2983774298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc152f80 0x7fa5cc151d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4030000 tx=0x7fa5c4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:33.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.173+0000 7fa5c27fc640 1 -- 172.21.15.112:0/2983774298 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4041020 con 0x7fa5cc152f80 2024-04-25T01:32:33.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.173+0000 7fa5c27fc640 1 -- 172.21.15.112:0/2983774298 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5c40077a0 con 0x7fa5cc152f80 2024-04-25T01:32:33.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.173+0000 7fa5c27fc640 1 -- 172.21.15.112:0/2983774298 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4036040 con 0x7fa5cc152f80 2024-04-25T01:32:33.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.173+0000 7fa5d3f12640 1 -- 172.21.15.112:0/2983774298 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5cc1ef740 con 0x7fa5cc152f80 2024-04-25T01:32:33.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.174+0000 7fa5d3f12640 1 -- 172.21.15.112:0/2983774298 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5cc0c85a0 con 0x7fa5cc152f80 2024-04-25T01:32:33.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.174+0000 7fa5d3f12640 1 --2- 172.21.15.112:0/2983774298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc0c8cd0 0x7fa5cc1eb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:33.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.174+0000 7fa5d3f12640 1 -- 172.21.15.112:0/2983774298 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc1500d0 con 0x7fa5cc0c8cd0 2024-04-25T01:32:33.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.175+0000 7fa5c27fc640 1 -- 172.21.15.112:0/2983774298 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa5c4044020 con 0x7fa5cc152f80 2024-04-25T01:32:33.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.175+0000 7fa5d1cae640 1 --2- 172.21.15.112:0/2983774298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc0c8cd0 0x7fa5cc1eb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:33.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.175+0000 7fa5c27fc640 1 --2- 172.21.15.112:0/2983774298 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa5b006bde0 0x7fa5b006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:33.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.175+0000 7fa5c27fc640 1 -- 172.21.15.112:0/2983774298 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fa5c40abd40 con 0x7fa5cc152f80 2024-04-25T01:32:33.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.176+0000 7fa5d14ad640 1 --2- 172.21.15.112:0/2983774298 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa5b006bde0 0x7fa5b006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:33.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.176+0000 7fa5d1cae640 1 -- 172.21.15.112:0/2983774298 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa5cc075980 con 0x7fa5cc0c8cd0 2024-04-25T01:32:33.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.176+0000 7fa5d14ad640 1 --2- 172.21.15.112:0/2983774298 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa5b006bde0 0x7fa5b006e2a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc006fd0 tx=0x7fa5bc006d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:33.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.176+0000 7fa5d1cae640 1 --2- 172.21.15.112:0/2983774298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc0c8cd0 0x7fa5cc1eb480 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800fae0 tx=0x7fa5c8001db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:33.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.176+0000 7fa5c27fc640 1 -- 172.21.15.112:0/2983774298 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c80052e0 con 0x7fa5cc0c8cd0 2024-04-25T01:32:33.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.177+0000 7fa5c27fc640 1 -- 172.21.15.112:0/2983774298 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa5c80052e0 con 0x7fa5cc0c8cd0 2024-04-25T01:32:33.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.177+0000 7fa5c27fc640 1 -- 172.21.15.112:0/2983774298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc0c8cd0 msgr2=0x7fa5cc1eb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:33.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.177+0000 7fa5c27fc640 1 --2- 172.21.15.112:0/2983774298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc0c8cd0 0x7fa5cc1eb480 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800fae0 tx=0x7fa5c8001db0 comp rx=0 tx=0).stop 2024-04-25T01:32:33.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.177+0000 7fa5c27fc640 1 -- 172.21.15.112:0/2983774298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc0c8cd0 msgr2=0x7fa5cc1eb480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:32:33.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.177+0000 7fa5c27fc640 1 --2- 172.21.15.112:0/2983774298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc0c8cd0 0x7fa5cc1eb480 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:33.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.184+0000 7fa5d3f12640 1 --2- 172.21.15.112:0/2983774298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc0ad1e0 0x7fa5cc1eb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:33.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.184+0000 7fa5d3f12640 1 -- 172.21.15.112:0/2983774298 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc0acf00 con 0x7fa5cc0ad1e0 2024-04-25T01:32:33.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.184+0000 7fa5d1cae640 1 --2- 172.21.15.112:0/2983774298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc0ad1e0 0x7fa5cc1eb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:33.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.184+0000 7fa5d1cae640 1 -- 172.21.15.112:0/2983774298 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa5cc0689b0 con 0x7fa5cc0ad1e0 2024-04-25T01:32:33.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.184+0000 7fa5d1cae640 1 --2- 172.21.15.112:0/2983774298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc0ad1e0 0x7fa5cc1eb9c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800fae0 tx=0x7fa5c8000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:33.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.185+0000 7fa5c27fc640 1 -- 172.21.15.112:0/2983774298 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8005490 con 0x7fa5cc0ad1e0 2024-04-25T01:32:33.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.185+0000 7fa5c27fc640 1 -- 172.21.15.112:0/2983774298 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5058 (secure 0 0 0) 0x7fa5c8005490 con 0x7fa5cc0ad1e0 2024-04-25T01:32:33.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.185+0000 7fa5c27fc640 1 -- 172.21.15.112:0/2983774298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc0ad1e0 msgr2=0x7fa5cc1eb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:33.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.185+0000 7fa5c27fc640 1 --2- 172.21.15.112:0/2983774298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc0ad1e0 0x7fa5cc1eb9c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800fae0 tx=0x7fa5c8000ed0 comp rx=0 tx=0).stop 2024-04-25T01:32:33.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.185+0000 7fa5c27fc640 1 -- 172.21.15.112:0/2983774298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc0ad1e0 msgr2=0x7fa5cc1eb9c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:32:33.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.185+0000 7fa5c27fc640 1 --2- 172.21.15.112:0/2983774298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc0ad1e0 0x7fa5cc1eb9c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:33.163 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:32:33.163 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:32:33.163 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:32:33.163 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:32:33.163 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 50, 2024-04-25T01:32:33.164 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 188875, 2024-04-25T01:32:33.164 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:32:33.164 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:32:33.164 INFO:teuthology.orchestra.run.smithi112.stdout: 1 2024-04-25T01:32:33.164 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:33.164 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:32:33.164 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:32:33.164 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:32:33.164 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:32:33.164 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:33.164 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:33.164 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:33.164 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:33.164 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:33.164 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:33.165 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:33.165 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:33.165 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:33.165 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:33.165 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:33.165 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:33.165 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:32:33.165 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:32:33.165 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:33.165 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:33.165 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:33.165 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:33.165 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:33.165 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:33.165 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:33.166 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:33.166 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:33.166 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:33.166 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:33.166 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:33.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.166 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:32:33.166 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:32:33.166 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:32:33.166 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:32:33.166 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:32:33.166 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:32:33.166 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:32:33.166 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:32:33.167 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:32:33.167 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:32:33.167 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:32:33.167 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:32:33.167 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:32:33.167 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:32:33.167 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:32:33.167 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:32:33.167 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:32:33.167 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:32:33.168 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:32:33.168 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:32:33.168 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:32:33.168 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:32:33.168 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:32:33.168 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:32:33.168 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:32:33.168 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:32:33.169 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:32:33.169 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:32:33.169 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:33.169 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:32:33.169 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.169 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:32:33.169 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.169 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:32:33.169 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:32:33.169 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:33.169 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:33.169 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.169 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:33.169 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:32:33.169 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:33.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.170 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.170 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:33.170 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:32:33.170 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:33.170 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:33.170 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:33.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.170 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:32:33.170 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:32:33.170 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:33.170 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:33.170 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:33.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.171 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.171 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:32:33.171 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:32:33.171 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:33.171 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:33.171 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.171 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:33.171 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:32:33.171 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:33.171 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.171 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.171 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:33.171 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:32:33.171 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:33.172 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:33.172 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:33.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.172 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:32:33.172 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:32:33.172 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:33.172 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:33.172 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:33.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.172 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.172 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:32:33.172 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:32:33.173 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:32:33.173 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:32:33.173 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.173 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:32:33.173 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:32:33.173 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:33.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.173 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.174 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:32:33.174 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:32:33.174 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:32:33.174 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:33.174 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:33.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.174 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:32:33.174 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:32:33.174 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:32:33.174 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:32:33.174 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:32:33.174 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:33.174 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:33.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.175 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:32:33.175 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:32:33.175 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.175 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:33.175 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:33.175 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:32:33.175 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:33.175 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:33.175 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:32:33.175 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.175 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:33.175 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:33.175 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:32:33.175 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:33.175 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:33.176 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:32:33.176 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.176 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:33.176 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:33.176 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:32:33.176 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:33.176 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:33.176 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:32:33.176 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.176 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:33.176 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:33.176 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 6 2024-04-25T01:32:33.176 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:33.176 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:32:33.176 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:32:33.177 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:32:33.177 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:32:33.177 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:32:33.177 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:32:33.177 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:32:33.177 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:32:33.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:32:33.177 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:32:33.177 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:32:33.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.185+0000 7fa5d3f12640 1 -- 172.21.15.112:0/2983774298 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa5b006bde0 msgr2=0x7fa5b006e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:33.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.185+0000 7fa5d3f12640 1 --2- 172.21.15.112:0/2983774298 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa5b006bde0 0x7fa5b006e2a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc006fd0 tx=0x7fa5bc006d80 comp rx=0 tx=0).stop 2024-04-25T01:32:33.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.185+0000 7fa5d3f12640 1 -- 172.21.15.112:0/2983774298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc152f80 msgr2=0x7fa5cc151d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:33.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.185+0000 7fa5d3f12640 1 --2- 172.21.15.112:0/2983774298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5cc152f80 0x7fa5cc151d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4030000 tx=0x7fa5c4004a40 comp rx=0 tx=0).stop 2024-04-25T01:32:33.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.185+0000 7fa5d24af640 1 -- 172.21.15.112:0/2983774298 reap_dead start 2024-04-25T01:32:33.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.185+0000 7fa5d24af640 1 -- 172.21.15.112:0/2983774298 reap_dead start 2024-04-25T01:32:33.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.185+0000 7fa5d3f12640 1 -- 172.21.15.112:0/2983774298 shutdown_connections 2024-04-25T01:32:33.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.185+0000 7fa5d3f12640 1 -- 172.21.15.112:0/2983774298 >> 172.21.15.112:0/2983774298 conn(0x7fa5cc06c6c0 msgr2=0x7fa5cc1e2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:33.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.185+0000 7fa5d3f12640 1 -- 172.21.15.112:0/2983774298 shutdown_connections 2024-04-25T01:32:33.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:33.185+0000 7fa5d3f12640 1 -- 172.21.15.112:0/2983774298 wait complete. 2024-04-25T01:32:33.179 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:32:33.180 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T01:32:53.181 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-25T01:32:53.181 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-04-25T01:32:53.181 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T01:32:53.181 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T01:32:53.184 INFO:tasks.ceph.mon.c:Started 2024-04-25T01:32:53.184 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:32:53.184 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:32:53.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.315+0000 7fd6658c4640 1 Processor -- start 2024-04-25T01:32:53.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.315+0000 7fd6658c4640 1 -- start start 2024-04-25T01:32:53.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd6658c4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd660152fd0 0x7fd6600c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:53.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd6658c4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6600c90e0 con 0x7fd6600c5d70 2024-04-25T01:32:53.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd6658c4640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6600cac00 con 0x7fd660156cf0 2024-04-25T01:32:53.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd6658c4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd6600ca9b0 con 0x7fd660152fd0 2024-04-25T01:32:53.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd657fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd6600c5d70 0x7fd660156280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:36760/0 (socket says 172.21.15.112:36760) 2024-04-25T01:32:53.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd657fff640 1 -- 172.21.15.112:0/679560402 learned_addr learned my addr 172.21.15.112:0/679560402 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:32:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd65effd640 1 --2- 172.21.15.112:0/679560402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd660152fd0 0x7fd6600c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd65effd640 1 -- 172.21.15.112:0/679560402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd660152fd0 msgr2=0x7fd6600c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:32:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd65effd640 1 -- 172.21.15.112:0/679560402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd660152fd0 msgr2=0x7fd6600c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:32:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd65effd640 1 --2- 172.21.15.112:0/679560402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd660152fd0 0x7fd6600c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd65effd640 1 --2- 172.21.15.112:0/679560402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd660152fd0 0x7fd6600c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:32:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd65e7fc640 1 -- 172.21.15.112:0/679560402 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 726361141 0 0) 0x7fd6600c90e0 con 0x7fd6600c5d70 2024-04-25T01:32:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd65e7fc640 1 -- 172.21.15.112:0/679560402 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd64c003520 con 0x7fd6600c5d70 2024-04-25T01:32:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd65e7fc640 1 -- 172.21.15.112:0/679560402 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3183220062 0 0) 0x7fd6600cac00 con 0x7fd660156cf0 2024-04-25T01:32:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd65e7fc640 1 -- 172.21.15.112:0/679560402 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6600c90e0 con 0x7fd660156cf0 2024-04-25T01:32:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd65e7fc640 1 -- 172.21.15.112:0/679560402 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3344749491 0 0) 0x7fd64c003520 con 0x7fd6600c5d70 2024-04-25T01:32:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd65e7fc640 1 -- 172.21.15.112:0/679560402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd660152fd0 msgr2=0x7fd6600c9b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:32:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd65e7fc640 1 --2- 172.21.15.112:0/679560402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd660152fd0 0x7fd6600c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:53.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd65e7fc640 1 -- 172.21.15.112:0/679560402 >> v1:172.21.15.187:6789/0 conn(0x7fd660156cf0 legacy=0x7fd66006bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:53.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd65e7fc640 1 -- 172.21.15.112:0/679560402 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6600c9380 con 0x7fd6600c5d70 2024-04-25T01:32:53.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd65e7fc640 1 -- 172.21.15.112:0/679560402 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fd648001670 con 0x7fd6600c5d70 2024-04-25T01:32:53.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.316+0000 7fd65e7fc640 1 -- 172.21.15.112:0/679560402 >> v1:172.21.15.112:6789/0 conn(0x7fd6600c5d70 legacy=0x7fd660156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:53.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.317+0000 7fd65e7fc640 1 --2- 172.21.15.112:0/679560402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd64c003b70 0x7fd64c003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:53.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.317+0000 7fd65e7fc640 1 --2- 172.21.15.112:0/679560402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd64c004800 0x7fd64c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:53.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.317+0000 7fd65e7fc640 1 --2- 172.21.15.112:0/679560402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd660152fd0 0x7fd64c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:53.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.317+0000 7fd65e7fc640 1 -- 172.21.15.112:0/679560402 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6600ca9b0 con 0x7fd64c003b70 2024-04-25T01:32:53.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.317+0000 7fd65e7fc640 1 -- 172.21.15.112:0/679560402 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd64c00bb80 con 0x7fd64c004800 2024-04-25T01:32:53.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.317+0000 7fd65e7fc640 1 -- 172.21.15.112:0/679560402 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd64c00bd80 con 0x7fd660152fd0 2024-04-25T01:32:53.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.317+0000 7fd65effd640 1 --2- 172.21.15.112:0/679560402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd64c003b70 0x7fd64c003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:53.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.317+0000 7fd657fff640 1 --2- 172.21.15.112:0/679560402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd64c004800 0x7fd64c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:53.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.317+0000 7fd65f7fe640 1 --2- 172.21.15.112:0/679560402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd660152fd0 0x7fd64c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:53.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.317+0000 7fd65effd640 1 -- 172.21.15.112:0/679560402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd660152fd0 msgr2=0x7fd64c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:32:53.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.317+0000 7fd65effd640 1 --2- 172.21.15.112:0/679560402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd660152fd0 0x7fd64c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:53.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.317+0000 7fd65effd640 1 -- 172.21.15.112:0/679560402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd64c004800 msgr2=0x7fd64c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:32:53.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.317+0000 7fd65effd640 1 --2- 172.21.15.112:0/679560402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd64c004800 0x7fd64c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:53.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.317+0000 7fd65effd640 1 -- 172.21.15.112:0/679560402 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64c003520 con 0x7fd64c003b70 2024-04-25T01:32:53.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.318+0000 7fd65effd640 1 --2- 172.21.15.112:0/679560402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd64c003b70 0x7fd64c003f50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd65000b9e0 tx=0x7fd64c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c26aad46cafd38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:53.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.318+0000 7fd65e7fc640 1 -- 172.21.15.112:0/679560402 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650030070 con 0x7fd64c003b70 2024-04-25T01:32:53.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.318+0000 7fd65e7fc640 1 -- 172.21.15.112:0/679560402 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd65002ada0 con 0x7fd64c003b70 2024-04-25T01:32:53.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.318+0000 7fd65e7fc640 1 -- 172.21.15.112:0/679560402 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65000b5e0 con 0x7fd64c003b70 2024-04-25T01:32:53.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.319+0000 7fd6658c4640 1 -- 172.21.15.112:0/679560402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd64c003b70 msgr2=0x7fd64c003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:53.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.319+0000 7fd6658c4640 1 --2- 172.21.15.112:0/679560402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd64c003b70 0x7fd64c003f50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd65000b9e0 tx=0x7fd64c005830 comp rx=0 tx=0).stop 2024-04-25T01:32:53.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.319+0000 7fd65f7fe640 1 -- 172.21.15.112:0/679560402 reap_dead start 2024-04-25T01:32:53.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.319+0000 7fd6658c4640 1 -- 172.21.15.112:0/679560402 shutdown_connections 2024-04-25T01:32:53.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.319+0000 7fd6658c4640 1 -- 172.21.15.112:0/679560402 >> 172.21.15.112:0/679560402 conn(0x7fd6600acc70 msgr2=0x7fd6600ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:53.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.319+0000 7fd6658c4640 1 -- 172.21.15.112:0/679560402 shutdown_connections 2024-04-25T01:32:53.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.319+0000 7fd6658c4640 1 -- 172.21.15.112:0/679560402 wait complete. 2024-04-25T01:32:53.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.320+0000 7fd6658c4640 1 Processor -- start 2024-04-25T01:32:53.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.320+0000 7fd6658c4640 1 -- start start 2024-04-25T01:32:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.320+0000 7fd6658c4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd64c003b70 0x7fd660158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.320+0000 7fd6658c4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6600c1d00 0x7fd6600c2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd6658c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6600c2850 0x7fd66014bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd6658c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd66006c4b0 con 0x7fd6600c2850 2024-04-25T01:32:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd6658c4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd6600caee0 con 0x7fd64c003b70 2024-04-25T01:32:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd6658c4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd66006c630 con 0x7fd6600c1d00 2024-04-25T01:32:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd65f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6600c2850 0x7fd66014bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd65f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6600c2850 0x7fd66014bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50320/0 (socket says 172.21.15.112:50320) 2024-04-25T01:32:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd65f7fe640 1 -- 172.21.15.112:0/1222995873 learned_addr learned my addr 172.21.15.112:0/1222995873 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:32:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd65effd640 1 --2- 172.21.15.112:0/1222995873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd64c003b70 0x7fd660158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd657fff640 1 --2- 172.21.15.112:0/1222995873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6600c1d00 0x7fd6600c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd657fff640 1 -- 172.21.15.112:0/1222995873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6600c1d00 msgr2=0x7fd6600c2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T01:32:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd657fff640 1 -- 172.21.15.112:0/1222995873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6600c1d00 msgr2=0x7fd6600c2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:32:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd657fff640 1 --2- 172.21.15.112:0/1222995873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6600c1d00 0x7fd6600c2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:53.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd657fff640 1 --2- 172.21.15.112:0/1222995873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6600c1d00 0x7fd6600c2310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:32:53.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd65f7fe640 1 -- 172.21.15.112:0/1222995873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6600c1d00 msgr2=0x7fd6600c2310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:32:53.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd65f7fe640 1 --2- 172.21.15.112:0/1222995873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6600c1d00 0x7fd6600c2310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:53.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd65f7fe640 1 -- 172.21.15.112:0/1222995873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd64c003b70 msgr2=0x7fd660158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:53.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd65f7fe640 1 --2- 172.21.15.112:0/1222995873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd64c003b70 0x7fd660158f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:53.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd65f7fe640 1 -- 172.21.15.112:0/1222995873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6601ee840 con 0x7fd6600c2850 2024-04-25T01:32:53.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd65effd640 1 --2- 172.21.15.112:0/1222995873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd64c003b70 0x7fd660158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:32:53.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd65f7fe640 1 --2- 172.21.15.112:0/1222995873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6600c2850 0x7fd66014bb60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd6580016c0 tx=0x7fd65800ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:53.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd65cff9640 1 -- 172.21.15.112:0/1222995873 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd658019070 con 0x7fd6600c2850 2024-04-25T01:32:53.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd65cff9640 1 -- 172.21.15.112:0/1222995873 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd658006d60 con 0x7fd6600c2850 2024-04-25T01:32:53.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd65cff9640 1 -- 172.21.15.112:0/1222995873 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd658003b90 con 0x7fd6600c2850 2024-04-25T01:32:53.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd6658c4640 1 -- 172.21.15.112:0/1222995873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6601eeb20 con 0x7fd6600c2850 2024-04-25T01:32:53.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.321+0000 7fd6658c4640 1 -- 172.21.15.112:0/1222995873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6601ef0e0 con 0x7fd6600c2850 2024-04-25T01:32:53.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.322+0000 7fd6658c4640 1 -- 172.21.15.112:0/1222995873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd66006bda0 con 0x7fd6600c2850 2024-04-25T01:32:53.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.323+0000 7fd65cff9640 1 -- 172.21.15.112:0/1222995873 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd65801c070 con 0x7fd6600c2850 2024-04-25T01:32:53.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.324+0000 7fd65cff9640 1 --2- 172.21.15.112:0/1222995873 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd64006be30 0x7fd64006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:53.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.324+0000 7fd65cff9640 1 -- 172.21.15.112:0/1222995873 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fd65808a6f0 con 0x7fd6600c2850 2024-04-25T01:32:53.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.324+0000 7fd65effd640 1 --2- 172.21.15.112:0/1222995873 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd64006be30 0x7fd64006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:53.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.324+0000 7fd65effd640 1 --2- 172.21.15.112:0/1222995873 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd64006be30 0x7fd64006e2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd650035000 tx=0x7fd650030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:53.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.329+0000 7fd65cff9640 1 -- 172.21.15.112:0/1222995873 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd66006bda0 con 0x7fd6600c2850 2024-04-25T01:32:53.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.613+0000 7fd6658c4640 1 -- 172.21.15.112:0/1222995873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd6600682a0 con 0x7fd6600c2850 2024-04-25T01:32:53.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.613+0000 7fd65cff9640 1 -- 172.21.15.112:0/1222995873 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1563 (secure 0 0 0) 0x7fd658058760 con 0x7fd6600c2850 2024-04-25T01:32:53.590 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":50,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:32:53.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.616+0000 7fd6658c4640 1 -- 172.21.15.112:0/1222995873 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd64006be30 msgr2=0x7fd64006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:53.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.616+0000 7fd6658c4640 1 --2- 172.21.15.112:0/1222995873 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd64006be30 0x7fd64006e2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd650035000 tx=0x7fd650030000 comp rx=0 tx=0).stop 2024-04-25T01:32:53.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.616+0000 7fd6658c4640 1 -- 172.21.15.112:0/1222995873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6600c2850 msgr2=0x7fd66014bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:53.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.616+0000 7fd6658c4640 1 --2- 172.21.15.112:0/1222995873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6600c2850 0x7fd66014bb60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd6580016c0 tx=0x7fd65800ff70 comp rx=0 tx=0).stop 2024-04-25T01:32:53.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.616+0000 7fd6658c4640 1 -- 172.21.15.112:0/1222995873 shutdown_connections 2024-04-25T01:32:53.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.616+0000 7fd6658c4640 1 --2- 172.21.15.112:0/1222995873 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd64006be30 0x7fd64006e2f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:53.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.616+0000 7fd6658c4640 1 --2- 172.21.15.112:0/1222995873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6600c2850 0x7fd66014bb60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:53.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.616+0000 7fd6658c4640 1 --2- 172.21.15.112:0/1222995873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6600c1d00 0x7fd6600c2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:53.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.616+0000 7fd6658c4640 1 --2- 172.21.15.112:0/1222995873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd64c003b70 0x7fd660158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:53.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.616+0000 7fd6658c4640 1 -- 172.21.15.112:0/1222995873 >> 172.21.15.112:0/1222995873 conn(0x7fd6600acc70 msgr2=0x7fd6600aabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:53.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.616+0000 7fd6658c4640 1 -- 172.21.15.112:0/1222995873 shutdown_connections 2024-04-25T01:32:53.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:53.616+0000 7fd6658c4640 1 -- 172.21.15.112:0/1222995873 wait complete. 2024-04-25T01:32:56.314 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:32:56.317 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:32:56.607 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:32:56.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.735+0000 7f025ea88640 1 Processor -- start 2024-04-25T01:32:56.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.735+0000 7f025ea88640 1 -- start start 2024-04-25T01:32:56.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.736+0000 7f025ea88640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 0x7f025814c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:56.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.736+0000 7f025ea88640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 0x7f0258153a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:56.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.736+0000 7f025ea88640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f025806dfb0 con 0x7f0258155d80 2024-04-25T01:32:56.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.737+0000 7f025ea88640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f0258156870 con 0x7f025806dab0 2024-04-25T01:32:56.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.737+0000 7f025ea88640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f025806ebc0 con 0x7f025814c5c0 2024-04-25T01:32:56.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.737+0000 7f025c824640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0258155d80 0x7f0258156160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:36762/0 (socket says 172.21.15.112:36762) 2024-04-25T01:32:56.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.737+0000 7f025c824640 1 -- 172.21.15.112:0/86036335 learned_addr learned my addr 172.21.15.112:0/86036335 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:32:56.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.737+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 0x7f025814c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:56.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.737+0000 7f025d025640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 0x7f0258153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:56.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.737+0000 7f0257fff640 1 -- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 msgr2=0x7f025814c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:32:56.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.737+0000 7f0257fff640 1 -- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 msgr2=0x7f025814c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:32:56.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.737+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 0x7f025814c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:56.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.737+0000 7f025d025640 1 -- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 msgr2=0x7f0258153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:32:56.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.737+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 0x7f025814c080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:32:56.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.737+0000 7f025d025640 1 -- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 msgr2=0x7f0258153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:32:56.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.737+0000 7f025d025640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 0x7f0258153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:56.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.737+0000 7f025d025640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 0x7f0258153a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:32:56.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.938+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 0x7f025814c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:56.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.938+0000 7f025d025640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 0x7f0258153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:56.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.938+0000 7f0257fff640 1 -- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 msgr2=0x7f025814c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:32:56.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.938+0000 7f0257fff640 1 -- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 msgr2=0x7f025814c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:32:56.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.938+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 0x7f025814c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:56.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.938+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 0x7f025814c080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:32:56.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.938+0000 7f025d025640 1 -- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 msgr2=0x7f0258153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:32:56.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.938+0000 7f025d025640 1 -- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 msgr2=0x7f0258153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:32:56.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.938+0000 7f025d025640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 0x7f0258153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:56.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:56.938+0000 7f025d025640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 0x7f0258153a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:32:57.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:57.339+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 0x7f025814c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:57.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:57.339+0000 7f025d025640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 0x7f0258153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:57.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:57.339+0000 7f0257fff640 1 -- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 msgr2=0x7f025814c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:32:57.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:57.339+0000 7f0257fff640 1 -- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 msgr2=0x7f025814c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:32:57.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:57.339+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 0x7f025814c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:57.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:57.339+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 0x7f025814c080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:32:57.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:57.339+0000 7f025d025640 1 -- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 msgr2=0x7f0258153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:32:57.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:57.339+0000 7f025d025640 1 -- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 msgr2=0x7f0258153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:32:57.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:57.339+0000 7f025d025640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 0x7f0258153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:57.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:57.339+0000 7f025d025640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 0x7f0258153a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:32:58.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:58.139+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 0x7f025814c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:58.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:58.139+0000 7f025d025640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 0x7f0258153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:58.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:58.139+0000 7f0257fff640 1 -- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 msgr2=0x7f025814c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:32:58.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:58.139+0000 7f0257fff640 1 -- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 msgr2=0x7f025814c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:32:58.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:58.139+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 0x7f025814c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:58.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:58.139+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 0x7f025814c080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:32:58.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:58.139+0000 7f025d025640 1 -- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 msgr2=0x7f0258153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:32:58.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:58.139+0000 7f025d025640 1 -- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 msgr2=0x7f0258153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:32:58.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:58.139+0000 7f025d025640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 0x7f0258153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:58.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:58.139+0000 7f025d025640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 0x7f0258153a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:32:59.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:59.736+0000 7f02567fc640 1 -- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 msgr2=0x7f0258153a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:32:59.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:59.736+0000 7f02567fc640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025814c5c0 0x7f0258153a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:59.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:59.736+0000 7f02567fc640 1 -- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 msgr2=0x7f025814c080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:32:59.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:59.736+0000 7f02567fc640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.112:3300/0 conn(0x7f025806dab0 0x7f025814c080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:59.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:59.736+0000 7f02567fc640 1 -- 172.21.15.112:0/86036335 >> v1:172.21.15.112:6789/0 conn(0x7f0258155d80 legacy=0x7f0258156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:59.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:59.736+0000 7f02567fc640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 0x7f023c009810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:59.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:59.736+0000 7f02567fc640 1 -- 172.21.15.112:0/86036335 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f025806e290 con 0x7f023c0036a0 2024-04-25T01:32:59.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:59.736+0000 7f02567fc640 1 -- 172.21.15.112:0/86036335 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f025806ebc0 con 0x7f023c007350 2024-04-25T01:32:59.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:59.736+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 0x7f023c009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:59.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:59.736+0000 7f0257fff640 1 -- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 msgr2=0x7f023c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:32:59.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:59.736+0000 7f0257fff640 1 -- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 msgr2=0x7f023c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:32:59.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:59.736+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 0x7f023c009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:59.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:59.736+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 0x7f023c009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:32:59.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:59.937+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 0x7f023c009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:59.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:59.937+0000 7f0257fff640 1 -- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 msgr2=0x7f023c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:32:59.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:59.937+0000 7f0257fff640 1 -- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 msgr2=0x7f023c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:32:59.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:59.937+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 0x7f023c009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:32:59.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:32:59.937+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 0x7f023c009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:33:00.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:00.338+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 0x7f023c009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:00.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:00.338+0000 7f0257fff640 1 -- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 msgr2=0x7f023c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:33:00.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:00.338+0000 7f0257fff640 1 -- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 msgr2=0x7f023c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:00.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:00.338+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 0x7f023c009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:00.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:00.338+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 0x7f023c009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:33:01.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.139+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 0x7f023c009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:01.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.139+0000 7f0257fff640 1 -- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 msgr2=0x7f023c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:33:01.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.139+0000 7f0257fff640 1 -- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 msgr2=0x7f023c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:01.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.139+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 0x7f023c009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:01.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.139+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 0x7f023c009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:33:01.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.327+0000 7f02577fe640 1 -- 172.21.15.112:0/86036335 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4041490186 0 0) 0x7f025806e290 con 0x7f023c0036a0 2024-04-25T01:33:01.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.327+0000 7f02577fe640 1 -- 172.21.15.112:0/86036335 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0248003520 con 0x7f023c0036a0 2024-04-25T01:33:01.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.328+0000 7f02577fe640 1 -- 172.21.15.112:0/86036335 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1610118291 0 0) 0x7f0248003520 con 0x7f023c0036a0 2024-04-25T01:33:01.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.328+0000 7f02577fe640 1 -- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 msgr2=0x7f023c009810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:33:01.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.328+0000 7f02577fe640 1 --2- 172.21.15.112:0/86036335 >> v2:172.21.15.187:3300/0 conn(0x7f023c007350 0x7f023c009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:01.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.328+0000 7f02577fe640 1 -- 172.21.15.112:0/86036335 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023c00af80 con 0x7f023c0036a0 2024-04-25T01:33:01.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.328+0000 7f02577fe640 1 -- 172.21.15.112:0/86036335 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f024c0016f0 con 0x7f023c0036a0 2024-04-25T01:33:01.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.328+0000 7f02577fe640 1 -- 172.21.15.112:0/86036335 >> v1:172.21.15.187:6789/0 conn(0x7f023c0036a0 legacy=0x7f023c005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:01.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.328+0000 7f025d025640 1 -- 172.21.15.112:0/86036335 reap_dead start 2024-04-25T01:33:01.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.328+0000 7f02577fe640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0248003a00 0x7f0248003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:01.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.328+0000 7f02577fe640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0248004800 0x7f0248004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:01.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.328+0000 7f02577fe640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02480053c0 0x7f024800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:01.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.328+0000 7f02577fe640 1 -- 172.21.15.112:0/86036335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f025806ebc0 con 0x7f0248003a00 2024-04-25T01:33:01.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.328+0000 7f02577fe640 1 -- 172.21.15.112:0/86036335 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f024800c280 con 0x7f02480053c0 2024-04-25T01:33:01.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.328+0000 7f02577fe640 1 -- 172.21.15.112:0/86036335 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f024800c480 con 0x7f0248004800 2024-04-25T01:33:01.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.328+0000 7f025d025640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02480053c0 0x7f024800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:01.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.328+0000 7f0257fff640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0248004800 0x7f0248004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:01.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.328+0000 7f025d025640 1 -- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0248004800 msgr2=0x7f0248004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:33:01.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.328+0000 7f025d025640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0248004800 0x7f0248004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:01.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.328+0000 7f025d025640 1 -- 172.21.15.112:0/86036335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0248003a00 msgr2=0x7f0248003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:33:01.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.328+0000 7f025d025640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0248003a00 0x7f0248003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:01.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.328+0000 7f025d025640 1 -- 172.21.15.112:0/86036335 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0248003520 con 0x7f02480053c0 2024-04-25T01:33:01.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.329+0000 7f025d025640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02480053c0 0x7f024800b9c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0258150360 tx=0x7f0248005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e4aab537742a7f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:01.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.330+0000 7f02577fe640 1 -- 172.21.15.112:0/86036335 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0250031070 con 0x7f02480053c0 2024-04-25T01:33:01.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.330+0000 7f02577fe640 1 -- 172.21.15.112:0/86036335 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f025002b780 con 0x7f02480053c0 2024-04-25T01:33:01.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.330+0000 7f02577fe640 1 -- 172.21.15.112:0/86036335 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0250003db0 con 0x7f02480053c0 2024-04-25T01:33:01.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.330+0000 7f025ea88640 1 -- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02480053c0 msgr2=0x7f024800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:01.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.330+0000 7f025ea88640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02480053c0 0x7f024800b9c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0258150360 tx=0x7f0248005830 comp rx=0 tx=0).stop 2024-04-25T01:33:01.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.330+0000 7f025ea88640 1 -- 172.21.15.112:0/86036335 shutdown_connections 2024-04-25T01:33:01.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.330+0000 7f025ea88640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02480053c0 0x7f024800b9c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:01.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.330+0000 7f025ea88640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0248004800 0x7f0248004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:01.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.330+0000 7f025ea88640 1 --2- 172.21.15.112:0/86036335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0248003a00 0x7f0248003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:01.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.330+0000 7f025ea88640 1 -- 172.21.15.112:0/86036335 >> 172.21.15.112:0/86036335 conn(0x7f02580c6f80 msgr2=0x7f02580c7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:01.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.331+0000 7f025ea88640 1 -- 172.21.15.112:0/86036335 shutdown_connections 2024-04-25T01:33:01.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.331+0000 7f025ea88640 1 -- 172.21.15.112:0/86036335 wait complete. 2024-04-25T01:33:01.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.332+0000 7f025ea88640 1 Processor -- start 2024-04-25T01:33:01.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.332+0000 7f025ea88640 1 -- start start 2024-04-25T01:33:01.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.333+0000 7f025ea88640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0248003a00 0x7f02580c0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:01.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.333+0000 7f025ea88640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0248004800 0x7f0258143940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:01.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.333+0000 7f025ea88640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02480053c0 0x7f0258143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:01.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.333+0000 7f025ea88640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f025806c910 con 0x7f0248003a00 2024-04-25T01:33:01.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.333+0000 7f025ea88640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f025806e850 con 0x7f02480053c0 2024-04-25T01:33:01.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.333+0000 7f025ea88640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f025806ca90 con 0x7f0248004800 2024-04-25T01:33:01.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.333+0000 7f025c824640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0248003a00 0x7f02580c0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:01.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.333+0000 7f025c824640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0248003a00 0x7f02580c0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44708/0 (socket says 172.21.15.112:44708) 2024-04-25T01:33:01.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.333+0000 7f025c824640 1 -- 172.21.15.112:0/154836991 learned_addr learned my addr 172.21.15.112:0/154836991 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:01.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.333+0000 7f0257fff640 1 --2- 172.21.15.112:0/154836991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0248004800 0x7f0258143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:01.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.333+0000 7f025d025640 1 --2- 172.21.15.112:0/154836991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02480053c0 0x7f0258143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:01.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.333+0000 7f0257fff640 1 -- 172.21.15.112:0/154836991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0248004800 msgr2=0x7f0258143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T01:33:01.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.333+0000 7f0257fff640 1 -- 172.21.15.112:0/154836991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0248004800 msgr2=0x7f0258143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:33:01.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.333+0000 7f0257fff640 1 --2- 172.21.15.112:0/154836991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0248004800 0x7f0258143940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:01.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.333+0000 7f0257fff640 1 --2- 172.21.15.112:0/154836991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0248004800 0x7f0258143940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:01.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.333+0000 7f025c824640 1 -- 172.21.15.112:0/154836991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0248004800 msgr2=0x7f0258143940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:33:01.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.333+0000 7f025c824640 1 --2- 172.21.15.112:0/154836991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0248004800 0x7f0258143940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:01.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.333+0000 7f025c824640 1 -- 172.21.15.112:0/154836991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02480053c0 msgr2=0x7f0258143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:01.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.333+0000 7f025c824640 1 --2- 172.21.15.112:0/154836991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02480053c0 0x7f0258143e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:01.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.334+0000 7f025c824640 1 -- 172.21.15.112:0/154836991 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02581554b0 con 0x7f0248003a00 2024-04-25T01:33:01.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.334+0000 7f025c824640 1 --2- 172.21.15.112:0/154836991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0248003a00 0x7f02580c0620 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f024c00c6c0 tx=0x7f024c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:01.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.334+0000 7f02557fa640 1 -- 172.21.15.112:0/154836991 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c017070 con 0x7f0248003a00 2024-04-25T01:33:01.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.334+0000 7f025ea88640 1 -- 172.21.15.112:0/154836991 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02581eb5a0 con 0x7f0248003a00 2024-04-25T01:33:01.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.334+0000 7f025ea88640 1 -- 172.21.15.112:0/154836991 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02580c0b60 con 0x7f0248003a00 2024-04-25T01:33:01.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.335+0000 7f02557fa640 1 -- 172.21.15.112:0/154836991 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f024c005ad0 con 0x7f0248003a00 2024-04-25T01:33:01.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.335+0000 7f02557fa640 1 -- 172.21.15.112:0/154836991 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c021060 con 0x7f0248003a00 2024-04-25T01:33:01.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.335+0000 7f025ea88640 1 -- 172.21.15.112:0/154836991 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0238005290 con 0x7f0248003a00 2024-04-25T01:33:01.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.336+0000 7f02557fa640 1 -- 172.21.15.112:0/154836991 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f024c01a070 con 0x7f0248003a00 2024-04-25T01:33:01.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.336+0000 7f02557fa640 1 --2- 172.21.15.112:0/154836991 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f023c06c5a0 0x7f023c06ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:01.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.336+0000 7f02557fa640 1 -- 172.21.15.112:0/154836991 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f024c08a6b0 con 0x7f0248003a00 2024-04-25T01:33:01.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.336+0000 7f0257fff640 1 --2- 172.21.15.112:0/154836991 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f023c06c5a0 0x7f023c06ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:01.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.337+0000 7f0257fff640 1 --2- 172.21.15.112:0/154836991 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f023c06c5a0 0x7f023c06ea60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0258155ae0 tx=0x7f0244008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:01.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.342+0000 7f02557fa640 1 -- 172.21.15.112:0/154836991 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f024c0585f0 con 0x7f0248003a00 2024-04-25T01:33:01.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.636+0000 7f025ea88640 1 -- 172.21.15.112:0/154836991 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0238005850 con 0x7f0248003a00 2024-04-25T01:33:01.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.636+0000 7f02557fa640 1 -- 172.21.15.112:0/154836991 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7f024c05c8f0 con 0x7f0248003a00 2024-04-25T01:33:01.615 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":54,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:33:01.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.639+0000 7f025ea88640 1 -- 172.21.15.112:0/154836991 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f023c06c5a0 msgr2=0x7f023c06ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:01.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.639+0000 7f025ea88640 1 --2- 172.21.15.112:0/154836991 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f023c06c5a0 0x7f023c06ea60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0258155ae0 tx=0x7f0244008040 comp rx=0 tx=0).stop 2024-04-25T01:33:01.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.639+0000 7f025ea88640 1 -- 172.21.15.112:0/154836991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0248003a00 msgr2=0x7f02580c0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:01.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.639+0000 7f025ea88640 1 --2- 172.21.15.112:0/154836991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0248003a00 0x7f02580c0620 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f024c00c6c0 tx=0x7f024c00cb80 comp rx=0 tx=0).stop 2024-04-25T01:33:01.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.639+0000 7f025ea88640 1 -- 172.21.15.112:0/154836991 shutdown_connections 2024-04-25T01:33:01.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.639+0000 7f025ea88640 1 --2- 172.21.15.112:0/154836991 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f023c06c5a0 0x7f023c06ea60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:01.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.639+0000 7f025ea88640 1 --2- 172.21.15.112:0/154836991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02480053c0 0x7f0258143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:01.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.639+0000 7f025ea88640 1 --2- 172.21.15.112:0/154836991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0248004800 0x7f0258143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:01.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.639+0000 7f025ea88640 1 --2- 172.21.15.112:0/154836991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0248003a00 0x7f02580c0620 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:01.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.639+0000 7f025ea88640 1 -- 172.21.15.112:0/154836991 >> 172.21.15.112:0/154836991 conn(0x7f02580c6f80 msgr2=0x7f025814a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:01.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.639+0000 7f025ea88640 1 -- 172.21.15.112:0/154836991 shutdown_connections 2024-04-25T01:33:01.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:01.639+0000 7f025ea88640 1 -- 172.21.15.112:0/154836991 wait complete. 2024-04-25T01:33:04.630 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:33:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.758+0000 7fa2c10b3640 1 Processor -- start 2024-04-25T01:33:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.758+0000 7fa2c10b3640 1 -- start start 2024-04-25T01:33:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.759+0000 7fa2c10b3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa2bc0c6d00 0x7fa2bc0c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.759+0000 7fa2c10b3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2bc0c77f0 0x7fa2bc0c9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.759+0000 7fa2c10b3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2bc06c970 con 0x7fa2bc1565e0 2024-04-25T01:33:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.759+0000 7fa2c10b3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa2bc0c6b60 con 0x7fa2bc0c6d00 2024-04-25T01:33:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.759+0000 7fa2c10b3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa2bc0c63f0 con 0x7fa2bc0c77f0 2024-04-25T01:33:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.759+0000 7fa2b3fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2bc0c77f0 0x7fa2bc0c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.759+0000 7fa2bad9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa2bc0c6d00 0x7fa2bc0c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.759+0000 7fa2b3fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2bc0c77f0 0x7fa2bc0c9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:58504/0 (socket says 172.21.15.112:58504) 2024-04-25T01:33:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.759+0000 7fa2b3fff640 1 -- 172.21.15.112:0/4270181235 learned_addr learned my addr 172.21.15.112:0/4270181235 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.759+0000 7fa2bad9d640 1 --2- 172.21.15.112:0/4270181235 >> v2:172.21.15.187:3300/0 conn(0x7fa2bc0c6d00 0x7fa2bc0c70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41496/0 (socket says 172.21.15.112:41496) 2024-04-25T01:33:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.759+0000 7fa2bb59e640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fa2bc1565e0 0x7fa2bc0c9590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:40984/0 (socket says 172.21.15.112:40984) 2024-04-25T01:33:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.759+0000 7fa2b3fff640 1 -- 172.21.15.112:0/4270181235 >> v2:172.21.15.187:3300/0 conn(0x7fa2bc0c6d00 msgr2=0x7fa2bc0c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:33:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.759+0000 7fa2b3fff640 1 --2- 172.21.15.112:0/4270181235 >> v2:172.21.15.187:3300/0 conn(0x7fa2bc0c6d00 0x7fa2bc0c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.759+0000 7fa2b3fff640 1 -- 172.21.15.112:0/4270181235 >> v1:172.21.15.187:6789/0 conn(0x7fa2bc1565e0 legacy=0x7fa2bc0c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.759+0000 7fa2b3fff640 1 -- 172.21.15.112:0/4270181235 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2bc06cc50 con 0x7fa2bc0c77f0 2024-04-25T01:33:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.759+0000 7fa2bad9d640 1 --2- 172.21.15.112:0/4270181235 >> v2:172.21.15.187:3300/0 conn(0x7fa2bc0c6d00 0x7fa2bc0c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:33:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.760+0000 7fa2b3fff640 1 --2- 172.21.15.112:0/4270181235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2bc0c77f0 0x7fa2bc0c9050 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4009870 tx=0x7fa2a402a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2989a6e4f2321b1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.760+0000 7fa2ba59c640 1 -- 172.21.15.112:0/4270181235 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4037070 con 0x7fa2bc0c77f0 2024-04-25T01:33:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.760+0000 7fa2ba59c640 1 -- 172.21.15.112:0/4270181235 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a4032440 con 0x7fa2bc0c77f0 2024-04-25T01:33:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.760+0000 7fa2ba59c640 1 -- 172.21.15.112:0/4270181235 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4032810 con 0x7fa2bc0c77f0 2024-04-25T01:33:04.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.761+0000 7fa2c10b3640 1 -- 172.21.15.112:0/4270181235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2bc0c77f0 msgr2=0x7fa2bc0c9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:04.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.761+0000 7fa2c10b3640 1 --2- 172.21.15.112:0/4270181235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2bc0c77f0 0x7fa2bc0c9050 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4009870 tx=0x7fa2a402a130 comp rx=0 tx=0).stop 2024-04-25T01:33:04.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.761+0000 7fa2c10b3640 1 -- 172.21.15.112:0/4270181235 shutdown_connections 2024-04-25T01:33:04.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.761+0000 7fa2c10b3640 1 --2- 172.21.15.112:0/4270181235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2bc0c77f0 0x7fa2bc0c9050 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:04.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.761+0000 7fa2c10b3640 1 --2- 172.21.15.112:0/4270181235 >> v2:172.21.15.187:3300/0 conn(0x7fa2bc0c6d00 0x7fa2bc0c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:04.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.761+0000 7fa2c10b3640 1 -- 172.21.15.112:0/4270181235 >> 172.21.15.112:0/4270181235 conn(0x7fa2bc0cb330 msgr2=0x7fa2bc0cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:04.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.761+0000 7fa2c10b3640 1 -- 172.21.15.112:0/4270181235 shutdown_connections 2024-04-25T01:33:04.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.761+0000 7fa2c10b3640 1 -- 172.21.15.112:0/4270181235 wait complete. 2024-04-25T01:33:04.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.762+0000 7fa2c10b3640 1 Processor -- start 2024-04-25T01:33:04.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.762+0000 7fa2c10b3640 1 -- start start 2024-04-25T01:33:04.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.763+0000 7fa2c10b3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2bc0c6d00 0x7fa2bc0ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:04.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.763+0000 7fa2c10b3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2bc0c77f0 0x7fa2bc0c4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:04.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.763+0000 7fa2c10b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2bc1565e0 0x7fa2bc0c3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:04.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.763+0000 7fa2c10b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc153430 con 0x7fa2bc1565e0 2024-04-25T01:33:04.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.763+0000 7fa2c10b3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc0ca940 con 0x7fa2bc0c6d00 2024-04-25T01:33:04.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.763+0000 7fa2c10b3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa2bc1535b0 con 0x7fa2bc0c77f0 2024-04-25T01:33:04.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.763+0000 7fa2bb59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2bc1565e0 0x7fa2bc0c3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:04.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.763+0000 7fa2bb59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2bc1565e0 0x7fa2bc0c3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44722/0 (socket says 172.21.15.112:44722) 2024-04-25T01:33:04.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.763+0000 7fa2bb59e640 1 -- 172.21.15.112:0/3418152633 learned_addr learned my addr 172.21.15.112:0/3418152633 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:04.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.763+0000 7fa2b3fff640 1 --2- 172.21.15.112:0/3418152633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2bc0c77f0 0x7fa2bc0c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:04.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.763+0000 7fa2bad9d640 1 --2- 172.21.15.112:0/3418152633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2bc0c6d00 0x7fa2bc0ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:04.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.763+0000 7fa2bb59e640 1 -- 172.21.15.112:0/3418152633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2bc0c77f0 msgr2=0x7fa2bc0c4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:04.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.763+0000 7fa2bb59e640 1 --2- 172.21.15.112:0/3418152633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2bc0c77f0 0x7fa2bc0c4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:04.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.763+0000 7fa2bb59e640 1 -- 172.21.15.112:0/3418152633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2bc0c6d00 msgr2=0x7fa2bc0ab6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:04.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.763+0000 7fa2bb59e640 1 --2- 172.21.15.112:0/3418152633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2bc0c6d00 0x7fa2bc0ab6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:04.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.763+0000 7fa2bb59e640 1 -- 172.21.15.112:0/3418152633 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2bc14c300 con 0x7fa2bc1565e0 2024-04-25T01:33:04.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.763+0000 7fa2bad9d640 1 --2- 172.21.15.112:0/3418152633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2bc0c6d00 0x7fa2bc0ab6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:33:04.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.764+0000 7fa2bb59e640 1 --2- 172.21.15.112:0/3418152633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2bc1565e0 0x7fa2bc0c3270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400c7b0 tx=0x7fa2b400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:04.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.764+0000 7fa2b37fe640 1 -- 172.21.15.112:0/3418152633 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4017070 con 0x7fa2bc1565e0 2024-04-25T01:33:04.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.764+0000 7fa2b37fe640 1 -- 172.21.15.112:0/3418152633 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b400a640 con 0x7fa2bc1565e0 2024-04-25T01:33:04.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.764+0000 7fa2c10b3640 1 -- 172.21.15.112:0/3418152633 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2bc14c020 con 0x7fa2bc1565e0 2024-04-25T01:33:04.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.764+0000 7fa2b37fe640 1 -- 172.21.15.112:0/3418152633 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b400a9f0 con 0x7fa2bc1565e0 2024-04-25T01:33:04.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.764+0000 7fa2c10b3640 1 -- 172.21.15.112:0/3418152633 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2bc0c54f0 con 0x7fa2bc1565e0 2024-04-25T01:33:04.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.765+0000 7fa2c10b3640 1 -- 172.21.15.112:0/3418152633 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2bc0682a0 con 0x7fa2bc1565e0 2024-04-25T01:33:04.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.765+0000 7fa2b37fe640 1 -- 172.21.15.112:0/3418152633 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa2b401a070 con 0x7fa2bc1565e0 2024-04-25T01:33:04.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.766+0000 7fa2b37fe640 1 --2- 172.21.15.112:0/3418152633 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa29c06be30 0x7fa29c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:04.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.766+0000 7fa2b37fe640 1 -- 172.21.15.112:0/3418152633 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fa2b408a720 con 0x7fa2bc1565e0 2024-04-25T01:33:04.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.766+0000 7fa2bad9d640 1 --2- 172.21.15.112:0/3418152633 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa29c06be30 0x7fa29c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:04.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.766+0000 7fa2bad9d640 1 --2- 172.21.15.112:0/3418152633 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa29c06be30 0x7fa29c06e2f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac009560 tx=0x7fa2ac009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:04.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:04.772+0000 7fa2b37fe640 1 -- 172.21.15.112:0/3418152633 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa2b40586e0 con 0x7fa2bc1565e0 2024-04-25T01:33:05.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.080+0000 7fa2c10b3640 1 -- 172.21.15.112:0/3418152633 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa2bc0c5c80 con 0x7fa2bc1565e0 2024-04-25T01:33:05.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.080+0000 7fa2b37fe640 1 -- 172.21.15.112:0/3418152633 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1568 (secure 0 0 0) 0x7fa2b405c9e0 con 0x7fa2bc1565e0 2024-04-25T01:33:05.058 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":56,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:33:05.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.082+0000 7fa2c10b3640 1 -- 172.21.15.112:0/3418152633 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa29c06be30 msgr2=0x7fa29c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.082+0000 7fa2c10b3640 1 --2- 172.21.15.112:0/3418152633 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa29c06be30 0x7fa29c06e2f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac009560 tx=0x7fa2ac009190 comp rx=0 tx=0).stop 2024-04-25T01:33:05.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.082+0000 7fa2c10b3640 1 -- 172.21.15.112:0/3418152633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2bc1565e0 msgr2=0x7fa2bc0c3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.082+0000 7fa2c10b3640 1 --2- 172.21.15.112:0/3418152633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2bc1565e0 0x7fa2bc0c3270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400c7b0 tx=0x7fa2b400cc70 comp rx=0 tx=0).stop 2024-04-25T01:33:05.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.083+0000 7fa2c10b3640 1 -- 172.21.15.112:0/3418152633 shutdown_connections 2024-04-25T01:33:05.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.083+0000 7fa2c10b3640 1 --2- 172.21.15.112:0/3418152633 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa29c06be30 0x7fa29c06e2f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.083+0000 7fa2c10b3640 1 --2- 172.21.15.112:0/3418152633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2bc1565e0 0x7fa2bc0c3270 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.083+0000 7fa2c10b3640 1 --2- 172.21.15.112:0/3418152633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2bc0c77f0 0x7fa2bc0c4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.083+0000 7fa2c10b3640 1 --2- 172.21.15.112:0/3418152633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2bc0c6d00 0x7fa2bc0ab6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.083+0000 7fa2c10b3640 1 -- 172.21.15.112:0/3418152633 >> 172.21.15.112:0/3418152633 conn(0x7fa2bc0cb330 msgr2=0x7fa2bc0cb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:05.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.083+0000 7fa2c10b3640 1 -- 172.21.15.112:0/3418152633 shutdown_connections 2024-04-25T01:33:05.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.083+0000 7fa2c10b3640 1 -- 172.21.15.112:0/3418152633 wait complete. 2024-04-25T01:33:05.073 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:33:05.073 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:33:05.073 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:33:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.212+0000 7f60b3046640 1 Processor -- start 2024-04-25T01:33:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.212+0000 7f60b3046640 1 -- start start 2024-04-25T01:33:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.213+0000 7f60b3046640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60ac1524d0 0x7f60ac06f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.213+0000 7f60b3046640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f60ac155fa0 0x7f60ac1563d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.213+0000 7f60b3046640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60ac155ac0 con 0x7f60ac0ad430 2024-04-25T01:33:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.213+0000 7f60b3046640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f60ac156910 con 0x7f60ac155fa0 2024-04-25T01:33:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.213+0000 7f60b3046640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f60ac152a20 con 0x7f60ac1524d0 2024-04-25T01:33:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.213+0000 7f60abfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60ac1524d0 0x7f60ac06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.213+0000 7f60b15e3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f60ac155fa0 0x7f60ac1563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.213+0000 7f60abfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60ac1524d0 0x7f60ac06f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:58520/0 (socket says 172.21.15.112:58520) 2024-04-25T01:33:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.213+0000 7f60abfff640 1 -- 172.21.15.112:0/2230266029 learned_addr learned my addr 172.21.15.112:0/2230266029 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.214+0000 7f60abfff640 1 -- 172.21.15.112:0/2230266029 >> v2:172.21.15.187:3300/0 conn(0x7f60ac155fa0 msgr2=0x7f60ac1563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:33:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.214+0000 7f60abfff640 1 --2- 172.21.15.112:0/2230266029 >> v2:172.21.15.187:3300/0 conn(0x7f60ac155fa0 0x7f60ac1563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.214+0000 7f60abfff640 1 -- 172.21.15.112:0/2230266029 >> v1:172.21.15.187:6789/0 conn(0x7f60ac0ad430 legacy=0x7f60ac0c50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.214+0000 7f60abfff640 1 -- 172.21.15.112:0/2230266029 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac152d00 con 0x7f60ac1524d0 2024-04-25T01:33:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.214+0000 7f60abfff640 1 --2- 172.21.15.112:0/2230266029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60ac1524d0 0x7f60ac06f130 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f60ac155c90 tx=0x7f609802a450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8380ea37fb20ae9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:05.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.215+0000 7f60ab7fe640 1 -- 172.21.15.112:0/2230266029 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6098036070 con 0x7f60ac1524d0 2024-04-25T01:33:05.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.215+0000 7f60ab7fe640 1 -- 172.21.15.112:0/2230266029 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f609802f710 con 0x7f60ac1524d0 2024-04-25T01:33:05.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.215+0000 7f60ab7fe640 1 -- 172.21.15.112:0/2230266029 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609802fae0 con 0x7f60ac1524d0 2024-04-25T01:33:05.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.215+0000 7f60b3046640 1 -- 172.21.15.112:0/2230266029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60ac1524d0 msgr2=0x7f60ac06f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.215+0000 7f60b3046640 1 --2- 172.21.15.112:0/2230266029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60ac1524d0 0x7f60ac06f130 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f60ac155c90 tx=0x7f609802a450 comp rx=0 tx=0).stop 2024-04-25T01:33:05.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.215+0000 7f60b3046640 1 -- 172.21.15.112:0/2230266029 shutdown_connections 2024-04-25T01:33:05.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.215+0000 7f60b3046640 1 --2- 172.21.15.112:0/2230266029 >> v2:172.21.15.187:3300/0 conn(0x7f60ac155fa0 0x7f60ac1563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.215+0000 7f60b3046640 1 --2- 172.21.15.112:0/2230266029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60ac1524d0 0x7f60ac06f130 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.215+0000 7f60b3046640 1 -- 172.21.15.112:0/2230266029 >> 172.21.15.112:0/2230266029 conn(0x7f60ac06c6c0 msgr2=0x7f60ac06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:05.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.215+0000 7f60b3046640 1 -- 172.21.15.112:0/2230266029 shutdown_connections 2024-04-25T01:33:05.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.215+0000 7f60b3046640 1 -- 172.21.15.112:0/2230266029 wait complete. 2024-04-25T01:33:05.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.216+0000 7f60b3046640 1 Processor -- start 2024-04-25T01:33:05.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.217+0000 7f60b3046640 1 -- start start 2024-04-25T01:33:05.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.217+0000 7f60b3046640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac0ad430 0x7f60ac0c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.217+0000 7f60b3046640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60ac1524d0 0x7f60ac154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.217+0000 7f60b3046640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60ac155fa0 0x7f60ac13a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.217+0000 7f60b3046640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac0c57e0 con 0x7f60ac0ad430 2024-04-25T01:33:05.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.217+0000 7f60b3046640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac1532c0 con 0x7f60ac1524d0 2024-04-25T01:33:05.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.217+0000 7f60b3046640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f60ac0c5960 con 0x7f60ac155fa0 2024-04-25T01:33:05.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.217+0000 7f60b0de2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac0ad430 0x7f60ac0c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.217+0000 7f60b15e3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60ac155fa0 0x7f60ac13a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.217+0000 7f60abfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60ac1524d0 0x7f60ac154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.218+0000 7f60b0de2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac0ad430 0x7f60ac0c68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44734/0 (socket says 172.21.15.112:44734) 2024-04-25T01:33:05.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.218+0000 7f60b0de2640 1 -- 172.21.15.112:0/4009629218 learned_addr learned my addr 172.21.15.112:0/4009629218 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:05.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.218+0000 7f60b0de2640 1 -- 172.21.15.112:0/4009629218 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60ac155fa0 msgr2=0x7f60ac13a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.218+0000 7f60b0de2640 1 --2- 172.21.15.112:0/4009629218 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60ac155fa0 0x7f60ac13a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.218+0000 7f60b0de2640 1 -- 172.21.15.112:0/4009629218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60ac1524d0 msgr2=0x7f60ac154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.218+0000 7f60b0de2640 1 --2- 172.21.15.112:0/4009629218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60ac1524d0 0x7f60ac154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.218+0000 7f60b0de2640 1 -- 172.21.15.112:0/4009629218 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac1e75b0 con 0x7f60ac0ad430 2024-04-25T01:33:05.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.218+0000 7f60abfff640 1 --2- 172.21.15.112:0/4009629218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60ac1524d0 0x7f60ac154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:33:05.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.218+0000 7f60b15e3640 1 --2- 172.21.15.112:0/4009629218 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60ac155fa0 0x7f60ac13a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:33:05.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.218+0000 7f60b0de2640 1 --2- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac0ad430 0x7f60ac0c68f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f60a000c400 tx=0x7f60a000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:05.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.218+0000 7f60a97fa640 1 -- 172.21.15.112:0/4009629218 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a00052d0 con 0x7f60ac0ad430 2024-04-25T01:33:05.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.218+0000 7f60b3046640 1 -- 172.21.15.112:0/4009629218 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ac1e7890 con 0x7f60ac0ad430 2024-04-25T01:33:05.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.218+0000 7f60a97fa640 1 -- 172.21.15.112:0/4009629218 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a0005490 con 0x7f60ac0ad430 2024-04-25T01:33:05.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.218+0000 7f60a97fa640 1 -- 172.21.15.112:0/4009629218 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a0007260 con 0x7f60ac0ad430 2024-04-25T01:33:05.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.218+0000 7f60b3046640 1 -- 172.21.15.112:0/4009629218 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ac1e7e50 con 0x7f60ac0ad430 2024-04-25T01:33:05.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.219+0000 7f60b3046640 1 --2- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac149110 0x7f60ac13ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.219+0000 7f60b3046640 1 -- 172.21.15.112:0/4009629218 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac13b1c0 con 0x7f60ac149110 2024-04-25T01:33:05.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.219+0000 7f60abfff640 1 --2- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac149110 0x7f60ac13ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.219+0000 7f60abfff640 1 -- 172.21.15.112:0/4009629218 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f60ac075980 con 0x7f60ac149110 2024-04-25T01:33:05.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.220+0000 7f60abfff640 1 --2- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac149110 0x7f60ac13ac80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0c5bf0 tx=0x7f60980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:05.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.220+0000 7f60a97fa640 1 -- 172.21.15.112:0/4009629218 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60a0007400 con 0x7f60ac0ad430 2024-04-25T01:33:05.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.220+0000 7f60a97fa640 1 --2- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f609006be30 0x7f609006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.220+0000 7f60b15e3640 1 --2- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f609006be30 0x7f609006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.220+0000 7f60a97fa640 1 -- 172.21.15.112:0/4009629218 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f60a008a6d0 con 0x7f60ac0ad430 2024-04-25T01:33:05.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.220+0000 7f60a97fa640 1 -- 172.21.15.112:0/4009629218 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6098036040 con 0x7f60ac149110 2024-04-25T01:33:05.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.220+0000 7f60a97fa640 1 -- 172.21.15.112:0/4009629218 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f609803b8e0 con 0x7f60ac149110 2024-04-25T01:33:05.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.220+0000 7f60a97fa640 1 -- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac149110 msgr2=0x7f60ac13ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.220+0000 7f60a97fa640 1 --2- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac149110 0x7f60ac13ac80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0c5bf0 tx=0x7f60980046c0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.220+0000 7f60a97fa640 1 -- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac149110 msgr2=0x7f60ac13ac80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:33:05.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.220+0000 7f60a97fa640 1 --2- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac149110 0x7f60ac13ac80 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0c5bf0 tx=0x7f60980046c0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.221+0000 7f60b15e3640 1 --2- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f609006be30 0x7f609006e2f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009fd0 tx=0x7f60a4011040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:05.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.227+0000 7f60b3046640 1 --2- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac1ea910 0x7f60ac1eacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.227+0000 7f60b3046640 1 -- 172.21.15.112:0/4009629218 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac1e8370 con 0x7f60ac1ea910 2024-04-25T01:33:05.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.227+0000 7f60abfff640 1 --2- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac1ea910 0x7f60ac1eacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.227+0000 7f60abfff640 1 -- 172.21.15.112:0/4009629218 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f60ac0736e0 con 0x7f60ac1ea910 2024-04-25T01:33:05.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.227+0000 7f60abfff640 1 --2- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac1ea910 0x7f60ac1eacf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f609802a490 tx=0x7f609802f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:05.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.227+0000 7f60a97fa640 1 -- 172.21.15.112:0/4009629218 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6098051070 con 0x7f60ac1ea910 2024-04-25T01:33:05.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.228+0000 7f60a97fa640 1 -- 172.21.15.112:0/4009629218 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4730 (secure 0 0 0) 0x7f6098003a20 con 0x7f60ac1ea910 2024-04-25T01:33:05.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.228+0000 7f60a97fa640 1 -- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac1ea910 msgr2=0x7f60ac1eacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.228+0000 7f60a97fa640 1 --2- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac1ea910 0x7f60ac1eacf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f609802a490 tx=0x7f609802f000 comp rx=0 tx=0).stop 2024-04-25T01:33:05.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.228+0000 7f60a97fa640 1 -- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac1ea910 msgr2=0x7f60ac1eacf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:33:05.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.228+0000 7f60a97fa640 1 --2- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac1ea910 0x7f60ac1eacf0 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f609802a490 tx=0x7f609802f000 comp rx=0 tx=0).stop 2024-04-25T01:33:05.206 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:33:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:33:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:33:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:33:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 56, 2024-04-25T01:33:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 65993, 2024-04-25T01:33:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:33:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:33:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:33:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:33:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 2, 2024-04-25T01:33:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:33:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:33:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:33:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:33:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:33:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:33:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:33:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:33:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:33:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:33:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:33:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:33:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:33:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:33:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:33:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:33:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:33:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:33:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:33:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:33:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:33:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:33:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:33:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:33:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:33:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:33:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:33:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:33:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:33:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:33:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:33:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:33:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:33:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:33:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.215 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:05.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.215 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:33:05.215 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:33:05.215 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:05.215 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:05.215 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:05.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.215 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.216 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:33:05.216 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:33:05.216 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:05.216 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:05.216 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.216 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:05.216 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:33:05.216 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:05.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.216 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.216 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:05.216 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:33:05.216 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:05.216 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:33:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:33:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:33:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:33:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:33:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:33:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:33:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:33:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:05.219 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:33:05.219 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.219 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:05.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.219 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:33:05.219 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:33:05.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.228+0000 7f60b3046640 1 -- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f609006be30 msgr2=0x7f609006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.228+0000 7f60b3046640 1 --2- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f609006be30 0x7f609006e2f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009fd0 tx=0x7f60a4011040 comp rx=0 tx=0).stop 2024-04-25T01:33:05.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.228+0000 7f60b3046640 1 -- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac0ad430 msgr2=0x7f60ac0c68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.228+0000 7f60b3046640 1 --2- 172.21.15.112:0/4009629218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac0ad430 0x7f60ac0c68f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f60a000c400 tx=0x7f60a000c8c0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.228+0000 7f60b15e3640 1 -- 172.21.15.112:0/4009629218 reap_dead start 2024-04-25T01:33:05.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.228+0000 7f60b15e3640 1 -- 172.21.15.112:0/4009629218 reap_dead start 2024-04-25T01:33:05.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.228+0000 7f60b3046640 1 -- 172.21.15.112:0/4009629218 shutdown_connections 2024-04-25T01:33:05.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.228+0000 7f60b3046640 1 -- 172.21.15.112:0/4009629218 >> 172.21.15.112:0/4009629218 conn(0x7f60ac06c6c0 msgr2=0x7f60ac0c50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:05.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.228+0000 7f60b3046640 1 -- 172.21.15.112:0/4009629218 shutdown_connections 2024-04-25T01:33:05.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.228+0000 7f60b3046640 1 -- 172.21.15.112:0/4009629218 wait complete. 2024-04-25T01:33:05.224 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:33:05.224 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:33:05.224 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:33:05.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.352+0000 7efc66753640 1 Processor -- start 2024-04-25T01:33:05.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.352+0000 7efc66753640 1 -- start start 2024-04-25T01:33:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.353+0000 7efc66753640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efc601592c0 0x7efc600c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.353+0000 7efc66753640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc600c72b0 0x7efc60160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.353+0000 7efc66753640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc60159b90 con 0x7efc601596a0 2024-04-25T01:33:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.353+0000 7efc66753640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7efc600c6e20 con 0x7efc601592c0 2024-04-25T01:33:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.353+0000 7efc66753640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efc6006c970 con 0x7efc600c72b0 2024-04-25T01:33:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.353+0000 7efc64cf0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc600c72b0 0x7efc60160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.353+0000 7efc5ffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efc601592c0 0x7efc600c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.353+0000 7efc5ffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efc601592c0 0x7efc600c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41534/0 (socket says 172.21.15.112:41534) 2024-04-25T01:33:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.353+0000 7efc64cf0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc600c72b0 0x7efc60160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:58538/0 (socket says 172.21.15.112:58538) 2024-04-25T01:33:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.353+0000 7efc5ffff640 1 -- 172.21.15.112:0/1364830966 learned_addr learned my addr 172.21.15.112:0/1364830966 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.353+0000 7efc5f7fe640 1 --1- 172.21.15.112:0/1364830966 >> v1:172.21.15.187:6789/0 conn(0x7efc601596a0 0x7efc60152d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:40994/0 (socket says 172.21.15.112:40994) 2024-04-25T01:33:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.353+0000 7efc64cf0640 1 -- 172.21.15.112:0/1364830966 >> v2:172.21.15.187:3300/0 conn(0x7efc601592c0 msgr2=0x7efc600c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:33:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.353+0000 7efc64cf0640 1 --2- 172.21.15.112:0/1364830966 >> v2:172.21.15.187:3300/0 conn(0x7efc601592c0 0x7efc600c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.353+0000 7efc64cf0640 1 -- 172.21.15.112:0/1364830966 >> v1:172.21.15.187:6789/0 conn(0x7efc601596a0 legacy=0x7efc60152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.353+0000 7efc64cf0640 1 -- 172.21.15.112:0/1364830966 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6006cc50 con 0x7efc600c72b0 2024-04-25T01:33:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.353+0000 7efc5ffff640 1 --2- 172.21.15.112:0/1364830966 >> v2:172.21.15.187:3300/0 conn(0x7efc601592c0 0x7efc600c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:33:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.354+0000 7efc64cf0640 1 --2- 172.21.15.112:0/1364830966 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc600c72b0 0x7efc60160380 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efc58009d90 tx=0x7efc5802a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f7962b5369bd817 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.354+0000 7efc5effd640 1 -- 172.21.15.112:0/1364830966 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58007740 con 0x7efc600c72b0 2024-04-25T01:33:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.354+0000 7efc5effd640 1 -- 172.21.15.112:0/1364830966 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc580078e0 con 0x7efc600c72b0 2024-04-25T01:33:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.354+0000 7efc5effd640 1 -- 172.21.15.112:0/1364830966 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58007cb0 con 0x7efc600c72b0 2024-04-25T01:33:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.355+0000 7efc66753640 1 -- 172.21.15.112:0/1364830966 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc600c72b0 msgr2=0x7efc60160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.355+0000 7efc66753640 1 --2- 172.21.15.112:0/1364830966 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc600c72b0 0x7efc60160380 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efc58009d90 tx=0x7efc5802a530 comp rx=0 tx=0).stop 2024-04-25T01:33:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.355+0000 7efc66753640 1 -- 172.21.15.112:0/1364830966 shutdown_connections 2024-04-25T01:33:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.355+0000 7efc66753640 1 --2- 172.21.15.112:0/1364830966 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc600c72b0 0x7efc60160380 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.355+0000 7efc66753640 1 --2- 172.21.15.112:0/1364830966 >> v2:172.21.15.187:3300/0 conn(0x7efc601592c0 0x7efc600c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.355+0000 7efc66753640 1 -- 172.21.15.112:0/1364830966 >> 172.21.15.112:0/1364830966 conn(0x7efc600cb170 msgr2=0x7efc600cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.355+0000 7efc66753640 1 -- 172.21.15.112:0/1364830966 shutdown_connections 2024-04-25T01:33:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.355+0000 7efc66753640 1 -- 172.21.15.112:0/1364830966 wait complete. 2024-04-25T01:33:05.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.356+0000 7efc66753640 1 Processor -- start 2024-04-25T01:33:05.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.356+0000 7efc66753640 1 -- start start 2024-04-25T01:33:05.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.357+0000 7efc66753640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc600c72b0 0x7efc60158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.357+0000 7efc66753640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc601592c0 0x7efc601eca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.357+0000 7efc66753640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc601596a0 0x7efc600b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.357+0000 7efc66753640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efc60153590 con 0x7efc600c72b0 2024-04-25T01:33:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.357+0000 7efc66753640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efc60153410 con 0x7efc601592c0 2024-04-25T01:33:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.357+0000 7efc66753640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efc6006db20 con 0x7efc601596a0 2024-04-25T01:33:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.357+0000 7efc5ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc600c72b0 0x7efc60158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.357+0000 7efc64cf0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc601596a0 0x7efc600b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.357+0000 7efc5ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc600c72b0 0x7efc60158200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44764/0 (socket says 172.21.15.112:44764) 2024-04-25T01:33:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.357+0000 7efc5ffff640 1 -- 172.21.15.112:0/2111662018 learned_addr learned my addr 172.21.15.112:0/2111662018 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.357+0000 7efc5f7fe640 1 --2- 172.21.15.112:0/2111662018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc601592c0 0x7efc601eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.357+0000 7efc64cf0640 1 -- 172.21.15.112:0/2111662018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc601592c0 msgr2=0x7efc601eca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.357+0000 7efc64cf0640 1 --2- 172.21.15.112:0/2111662018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc601592c0 0x7efc601eca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.357+0000 7efc64cf0640 1 -- 172.21.15.112:0/2111662018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc600c72b0 msgr2=0x7efc60158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.357+0000 7efc64cf0640 1 --2- 172.21.15.112:0/2111662018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc600c72b0 0x7efc60158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.357+0000 7efc64cf0640 1 -- 172.21.15.112:0/2111662018 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc600ae6b0 con 0x7efc601596a0 2024-04-25T01:33:05.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.357+0000 7efc64cf0640 1 --2- 172.21.15.112:0/2111662018 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc601596a0 0x7efc600b04a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efc58031000 tx=0x7efc58004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:05.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.358+0000 7efc5cff9640 1 -- 172.21.15.112:0/2111662018 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5803f070 con 0x7efc601596a0 2024-04-25T01:33:05.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.358+0000 7efc5cff9640 1 -- 172.21.15.112:0/2111662018 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc58031c00 con 0x7efc601596a0 2024-04-25T01:33:05.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.358+0000 7efc66753640 1 -- 172.21.15.112:0/2111662018 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc600ae990 con 0x7efc601596a0 2024-04-25T01:33:05.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.358+0000 7efc5cff9640 1 -- 172.21.15.112:0/2111662018 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58009450 con 0x7efc601596a0 2024-04-25T01:33:05.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.358+0000 7efc66753640 1 -- 172.21.15.112:0/2111662018 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efc60157ce0 con 0x7efc601596a0 2024-04-25T01:33:05.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.359+0000 7efc66753640 1 --2- 172.21.15.112:0/2111662018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc60156050 0x7efc6014c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.359+0000 7efc66753640 1 -- 172.21.15.112:0/2111662018 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efc60157f20 con 0x7efc60156050 2024-04-25T01:33:05.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.359+0000 7efc5ffff640 1 --2- 172.21.15.112:0/2111662018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc60156050 0x7efc6014c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.359+0000 7efc5ffff640 1 -- 172.21.15.112:0/2111662018 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efc60075980 con 0x7efc60156050 2024-04-25T01:33:05.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.359+0000 7efc5cff9640 1 -- 172.21.15.112:0/2111662018 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc58044460 con 0x7efc601596a0 2024-04-25T01:33:05.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.359+0000 7efc5ffff640 1 --2- 172.21.15.112:0/2111662018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc60156050 0x7efc6014c290 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efc500119b0 tx=0x7efc50011e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:05.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.360+0000 7efc5cff9640 1 --2- 172.21.15.112:0/2111662018 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efc4c06be60 0x7efc4c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.360+0000 7efc5cff9640 1 -- 172.21.15.112:0/2111662018 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7efc580ac710 con 0x7efc601596a0 2024-04-25T01:33:05.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.360+0000 7efc5f7fe640 1 --2- 172.21.15.112:0/2111662018 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efc4c06be60 0x7efc4c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.360+0000 7efc5cff9640 1 -- 172.21.15.112:0/2111662018 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5000fb60 con 0x7efc60156050 2024-04-25T01:33:05.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.360+0000 7efc5cff9640 1 -- 172.21.15.112:0/2111662018 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7efc5000fb60 con 0x7efc60156050 2024-04-25T01:33:05.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.360+0000 7efc5cff9640 1 -- 172.21.15.112:0/2111662018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc60156050 msgr2=0x7efc6014c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.360+0000 7efc5cff9640 1 --2- 172.21.15.112:0/2111662018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc60156050 0x7efc6014c290 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efc500119b0 tx=0x7efc50011e70 comp rx=0 tx=0).stop 2024-04-25T01:33:05.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.360+0000 7efc5cff9640 1 -- 172.21.15.112:0/2111662018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc60156050 msgr2=0x7efc6014c290 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:33:05.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.360+0000 7efc5cff9640 1 --2- 172.21.15.112:0/2111662018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc60156050 0x7efc6014c290 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efc500119b0 tx=0x7efc50011e70 comp rx=0 tx=0).stop 2024-04-25T01:33:05.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.360+0000 7efc5f7fe640 1 --2- 172.21.15.112:0/2111662018 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efc4c06be60 0x7efc4c06e320 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efc6014da70 tx=0x7efc54007e20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:05.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.367+0000 7efc66753640 1 --2- 172.21.15.112:0/2111662018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc600b09e0 0x7efc6014c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.367+0000 7efc66753640 1 -- 172.21.15.112:0/2111662018 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efc600682a0 con 0x7efc600b09e0 2024-04-25T01:33:05.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.367+0000 7efc5ffff640 1 --2- 172.21.15.112:0/2111662018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc600b09e0 0x7efc6014c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.367+0000 7efc5ffff640 1 -- 172.21.15.112:0/2111662018 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7efc601577f0 con 0x7efc600b09e0 2024-04-25T01:33:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.367+0000 7efc5ffff640 1 --2- 172.21.15.112:0/2111662018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc600b09e0 0x7efc6014c7d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efc500119b0 tx=0x7efc500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.368+0000 7efc5cff9640 1 -- 172.21.15.112:0/2111662018 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50014660 con 0x7efc600b09e0 2024-04-25T01:33:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.368+0000 7efc5cff9640 1 -- 172.21.15.112:0/2111662018 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5069 (secure 0 0 0) 0x7efc5001c3a0 con 0x7efc600b09e0 2024-04-25T01:33:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.368+0000 7efc5cff9640 1 -- 172.21.15.112:0/2111662018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc600b09e0 msgr2=0x7efc6014c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.368+0000 7efc5cff9640 1 --2- 172.21.15.112:0/2111662018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc600b09e0 0x7efc6014c7d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efc500119b0 tx=0x7efc500046c0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.368+0000 7efc5cff9640 1 -- 172.21.15.112:0/2111662018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc600b09e0 msgr2=0x7efc6014c7d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:33:05.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.368+0000 7efc5cff9640 1 --2- 172.21.15.112:0/2111662018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc600b09e0 0x7efc6014c7d0 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efc500119b0 tx=0x7efc500046c0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.347 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:33:05.347 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:33:05.347 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:33:05.347 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:33:05.347 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 56, 2024-04-25T01:33:05.347 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 221057, 2024-04-25T01:33:05.347 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:33:05.347 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:33:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:33:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:33:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 3, 2024-04-25T01:33:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:33:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:33:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:33:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:33:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:33:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:33:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:33:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:33:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:33:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:33:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:33:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:33:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:33:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:33:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:33:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:33:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:33:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:33:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:33:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:33:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:33:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:33:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:33:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:33:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:33:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:33:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:33:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:33:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:33:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:33:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:33:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:33:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:33:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:33:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:33:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:33:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:33:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:33:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:33:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:33:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:05.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.357 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:33:05.357 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:33:05.357 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:05.357 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:05.357 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:05.357 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.357 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:05.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.357 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:33:05.357 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:33:05.357 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.357 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:05.357 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:33:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:33:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:33:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:33:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:33:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:33:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:33:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:33:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:33:05.360 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.360 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:05.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.360 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:33:05.360 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:33:05.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.368+0000 7efc66753640 1 -- 172.21.15.112:0/2111662018 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efc4c06be60 msgr2=0x7efc4c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.368+0000 7efc66753640 1 --2- 172.21.15.112:0/2111662018 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efc4c06be60 0x7efc4c06e320 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efc6014da70 tx=0x7efc54007e20 comp rx=0 tx=0).stop 2024-04-25T01:33:05.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.368+0000 7efc66753640 1 -- 172.21.15.112:0/2111662018 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc601596a0 msgr2=0x7efc600b04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.368+0000 7efc66753640 1 --2- 172.21.15.112:0/2111662018 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc601596a0 0x7efc600b04a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efc58031000 tx=0x7efc58004a40 comp rx=0 tx=0).stop 2024-04-25T01:33:05.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.368+0000 7efc64cf0640 1 -- 172.21.15.112:0/2111662018 reap_dead start 2024-04-25T01:33:05.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.368+0000 7efc64cf0640 1 -- 172.21.15.112:0/2111662018 reap_dead start 2024-04-25T01:33:05.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.368+0000 7efc66753640 1 -- 172.21.15.112:0/2111662018 shutdown_connections 2024-04-25T01:33:05.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.368+0000 7efc66753640 1 -- 172.21.15.112:0/2111662018 >> 172.21.15.112:0/2111662018 conn(0x7efc600cb170 msgr2=0x7efc600cb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:05.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.368+0000 7efc66753640 1 -- 172.21.15.112:0/2111662018 shutdown_connections 2024-04-25T01:33:05.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.368+0000 7efc66753640 1 -- 172.21.15.112:0/2111662018 wait complete. 2024-04-25T01:33:05.362 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:33:05.362 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:33:05.362 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:33:05.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.501+0000 7f93f4258640 1 Processor -- start 2024-04-25T01:33:05.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.502+0000 7f93f4258640 1 -- start start 2024-04-25T01:33:05.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.502+0000 7f93f4258640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f93ec06c840 0x7f93ec0c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.502+0000 7f93f4258640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f93ec0c86d0 0x7f93ec158e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.502+0000 7f93f4258640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93ec155ab0 con 0x7f93ec152fa0 2024-04-25T01:33:05.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.502+0000 7f93f4258640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f93ec156870 con 0x7f93ec0c86d0 2024-04-25T01:33:05.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.502+0000 7f93f4258640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f93ec1563e0 con 0x7f93ec06c840 2024-04-25T01:33:05.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.502+0000 7f93f27f5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f93ec0c86d0 0x7f93ec158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.502+0000 7f93f27f5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f93ec0c86d0 0x7f93ec158e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44774/0 (socket says 172.21.15.112:44774) 2024-04-25T01:33:05.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.503+0000 7f93f27f5640 1 -- 172.21.15.112:0/3928812423 learned_addr learned my addr 172.21.15.112:0/3928812423 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:05.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.503+0000 7f93f17f3640 1 --2- 172.21.15.112:0/3928812423 >> v2:172.21.15.187:3300/0 conn(0x7f93ec06c840 0x7f93ec0c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.503+0000 7f93f27f5640 1 -- 172.21.15.112:0/3928812423 >> v2:172.21.15.187:3300/0 conn(0x7f93ec06c840 msgr2=0x7f93ec0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:33:05.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.503+0000 7f93f27f5640 1 --2- 172.21.15.112:0/3928812423 >> v2:172.21.15.187:3300/0 conn(0x7f93ec06c840 0x7f93ec0c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.503+0000 7f93f27f5640 1 -- 172.21.15.112:0/3928812423 >> v1:172.21.15.187:6789/0 conn(0x7f93ec152fa0 legacy=0x7f93ec0c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.503+0000 7f93f27f5640 1 -- 172.21.15.112:0/3928812423 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ec155d90 con 0x7f93ec0c86d0 2024-04-25T01:33:05.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.503+0000 7f93f17f3640 1 --2- 172.21.15.112:0/3928812423 >> v2:172.21.15.187:3300/0 conn(0x7f93ec06c840 0x7f93ec0c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:33:05.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.504+0000 7f93f27f5640 1 --2- 172.21.15.112:0/3928812423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93ec0c86d0 0x7f93ec158e30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0cc220 tx=0x7f93e402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92b85504bcf85e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:05.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.504+0000 7f93f0ff2640 1 -- 172.21.15.112:0/3928812423 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4036070 con 0x7f93ec0c86d0 2024-04-25T01:33:05.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.504+0000 7f93f0ff2640 1 -- 172.21.15.112:0/3928812423 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93e4031600 con 0x7f93ec0c86d0 2024-04-25T01:33:05.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.504+0000 7f93f0ff2640 1 -- 172.21.15.112:0/3928812423 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e40319d0 con 0x7f93ec0c86d0 2024-04-25T01:33:05.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.505+0000 7f93f4258640 1 -- 172.21.15.112:0/3928812423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93ec0c86d0 msgr2=0x7f93ec158e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.505+0000 7f93f4258640 1 --2- 172.21.15.112:0/3928812423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93ec0c86d0 0x7f93ec158e30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0cc220 tx=0x7f93e402a6a0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.505+0000 7f93f4258640 1 -- 172.21.15.112:0/3928812423 shutdown_connections 2024-04-25T01:33:05.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.505+0000 7f93f4258640 1 --2- 172.21.15.112:0/3928812423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93ec0c86d0 0x7f93ec158e30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.505+0000 7f93f4258640 1 --2- 172.21.15.112:0/3928812423 >> v2:172.21.15.187:3300/0 conn(0x7f93ec06c840 0x7f93ec0c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.505+0000 7f93f4258640 1 -- 172.21.15.112:0/3928812423 >> 172.21.15.112:0/3928812423 conn(0x7f93ec0ad150 msgr2=0x7f93ec0ad560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:05.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.505+0000 7f93f4258640 1 -- 172.21.15.112:0/3928812423 shutdown_connections 2024-04-25T01:33:05.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.505+0000 7f93f4258640 1 -- 172.21.15.112:0/3928812423 wait complete. 2024-04-25T01:33:05.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.506+0000 7f93f4258640 1 Processor -- start 2024-04-25T01:33:05.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.506+0000 7f93f4258640 1 -- start start 2024-04-25T01:33:05.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.507+0000 7f93f4258640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93ec06c840 0x7f93ec1549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.507+0000 7f93f4258640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93ec0c86d0 0x7f93ec1edc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.507+0000 7f93f4258640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93ec152fa0 0x7f93ec1ee140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.507+0000 7f93f4258640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec153380 con 0x7f93ec0c86d0 2024-04-25T01:33:05.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.507+0000 7f93f4258640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec06cdb0 con 0x7f93ec06c840 2024-04-25T01:33:05.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.507+0000 7f93f4258640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f93ec153500 con 0x7f93ec152fa0 2024-04-25T01:33:05.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.507+0000 7f93f17f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93ec0c86d0 0x7f93ec1edc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.507+0000 7f93f17f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93ec0c86d0 0x7f93ec1edc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44780/0 (socket says 172.21.15.112:44780) 2024-04-25T01:33:05.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.507+0000 7f93f17f3640 1 -- 172.21.15.112:0/111414016 learned_addr learned my addr 172.21.15.112:0/111414016 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:05.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.507+0000 7f93f1ff4640 1 --2- 172.21.15.112:0/111414016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93ec06c840 0x7f93ec1549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.507+0000 7f93f27f5640 1 --2- 172.21.15.112:0/111414016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93ec152fa0 0x7f93ec1ee140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.507+0000 7f93f17f3640 1 -- 172.21.15.112:0/111414016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93ec152fa0 msgr2=0x7f93ec1ee140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.507+0000 7f93f17f3640 1 --2- 172.21.15.112:0/111414016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93ec152fa0 0x7f93ec1ee140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.507+0000 7f93f17f3640 1 -- 172.21.15.112:0/111414016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93ec06c840 msgr2=0x7f93ec1549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.507+0000 7f93f17f3640 1 --2- 172.21.15.112:0/111414016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93ec06c840 0x7f93ec1549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.507+0000 7f93f17f3640 1 -- 172.21.15.112:0/111414016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ec152270 con 0x7f93ec0c86d0 2024-04-25T01:33:05.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.508+0000 7f93f17f3640 1 --2- 172.21.15.112:0/111414016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93ec0c86d0 0x7f93ec1edc00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00e990 tx=0x7f93dc00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:05.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.508+0000 7f93e27fc640 1 -- 172.21.15.112:0/111414016 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc017070 con 0x7f93ec0c86d0 2024-04-25T01:33:05.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.508+0000 7f93e27fc640 1 -- 172.21.15.112:0/111414016 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93dc00c9e0 con 0x7f93ec0c86d0 2024-04-25T01:33:05.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.508+0000 7f93e27fc640 1 -- 172.21.15.112:0/111414016 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc00cdb0 con 0x7f93ec0c86d0 2024-04-25T01:33:05.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.508+0000 7f93f4258640 1 -- 172.21.15.112:0/111414016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ec1ef240 con 0x7f93ec0c86d0 2024-04-25T01:33:05.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.508+0000 7f93f4258640 1 -- 172.21.15.112:0/111414016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ec152530 con 0x7f93ec0c86d0 2024-04-25T01:33:05.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.509+0000 7f93f4258640 1 --2- 172.21.15.112:0/111414016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93ec1555d0 0x7f93ec1eb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.509+0000 7f93f4258640 1 -- 172.21.15.112:0/111414016 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f93ec1520f0 con 0x7f93ec1555d0 2024-04-25T01:33:05.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.509+0000 7f93f1ff4640 1 --2- 172.21.15.112:0/111414016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93ec1555d0 0x7f93ec1eb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.509+0000 7f93f1ff4640 1 -- 172.21.15.112:0/111414016 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93ec075980 con 0x7f93ec1555d0 2024-04-25T01:33:05.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.509+0000 7f93f1ff4640 1 --2- 172.21.15.112:0/111414016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93ec1555d0 0x7f93ec1eb710 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f93e800f540 tx=0x7f93e800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:05.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.509+0000 7f93e27fc640 1 -- 172.21.15.112:0/111414016 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e8019070 con 0x7f93ec1555d0 2024-04-25T01:33:05.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.510+0000 7f93e27fc640 1 -- 172.21.15.112:0/111414016 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f93dc01a070 con 0x7f93ec0c86d0 2024-04-25T01:33:05.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.510+0000 7f93e27fc640 1 --2- 172.21.15.112:0/111414016 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f93d006bd30 0x7f93d006e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.510+0000 7f93e27fc640 1 -- 172.21.15.112:0/111414016 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f93dc08af60 con 0x7f93ec0c86d0 2024-04-25T01:33:05.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.510+0000 7f93f27f5640 1 --2- 172.21.15.112:0/111414016 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f93d006bd30 0x7f93d006e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.510+0000 7f93e27fc640 1 -- 172.21.15.112:0/111414016 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f93e8005320 con 0x7f93ec1555d0 2024-04-25T01:33:05.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.510+0000 7f93e27fc640 1 -- 172.21.15.112:0/111414016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93ec1555d0 msgr2=0x7f93ec1eb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.510+0000 7f93e27fc640 1 --2- 172.21.15.112:0/111414016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93ec1555d0 0x7f93ec1eb710 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f93e800f540 tx=0x7f93e800fa00 comp rx=0 tx=0).stop 2024-04-25T01:33:05.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.510+0000 7f93e27fc640 1 -- 172.21.15.112:0/111414016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93ec1555d0 msgr2=0x7f93ec1eb710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:33:05.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.510+0000 7f93e27fc640 1 --2- 172.21.15.112:0/111414016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93ec1555d0 0x7f93ec1eb710 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.511+0000 7f93f27f5640 1 --2- 172.21.15.112:0/111414016 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f93d006bd30 0x7f93d006e1f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f93e4033000 tx=0x7f93e4009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:05.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.518+0000 7f93f4258640 1 --2- 172.21.15.112:0/111414016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93ec0be510 0x7f93ec0c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.518+0000 7f93f4258640 1 -- 172.21.15.112:0/111414016 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f93ec1ec730 con 0x7f93ec0be510 2024-04-25T01:33:05.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.518+0000 7f93f1ff4640 1 --2- 172.21.15.112:0/111414016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93ec0be510 0x7f93ec0c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.518+0000 7f93f1ff4640 1 -- 172.21.15.112:0/111414016 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f93ec0ab9f0 con 0x7f93ec0be510 2024-04-25T01:33:05.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.518+0000 7f93f1ff4640 1 --2- 172.21.15.112:0/111414016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93ec0be510 0x7f93ec0c8f80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f93e800f540 tx=0x7f93e8000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:05.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.518+0000 7f93e27fc640 1 -- 172.21.15.112:0/111414016 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e801c6c0 con 0x7f93ec0be510 2024-04-25T01:33:05.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.519+0000 7f93e27fc640 1 -- 172.21.15.112:0/111414016 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4558 (secure 0 0 0) 0x7f93e801c6c0 con 0x7f93ec0be510 2024-04-25T01:33:05.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.519+0000 7f93e27fc640 1 -- 172.21.15.112:0/111414016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93ec0be510 msgr2=0x7f93ec0c8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.519+0000 7f93e27fc640 1 --2- 172.21.15.112:0/111414016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93ec0be510 0x7f93ec0c8f80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f93e800f540 tx=0x7f93e8000ed0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.519+0000 7f93e27fc640 1 -- 172.21.15.112:0/111414016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93ec0be510 msgr2=0x7f93ec0c8f80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:33:05.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.519+0000 7f93e27fc640 1 --2- 172.21.15.112:0/111414016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93ec0be510 0x7f93ec0c8f80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.496 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:33:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:33:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:33:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:33:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 56, 2024-04-25T01:33:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 12219, 2024-04-25T01:33:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:33:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:33:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:33:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:33:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 3, 2024-04-25T01:33:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:33:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:33:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:33:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:05.499 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:05.499 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:33:05.499 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:33:05.499 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:05.499 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:05.499 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:05.499 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:05.499 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:05.499 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:05.499 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:05.499 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:05.499 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:05.499 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:05.499 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:05.499 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:05.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.500 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:33:05.500 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:33:05.500 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:33:05.500 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:33:05.500 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:33:05.500 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:33:05.500 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:33:05.500 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:33:05.500 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:33:05.500 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:33:05.500 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:33:05.500 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:33:05.500 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:33:05.501 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:33:05.501 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:33:05.501 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:33:05.501 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:33:05.501 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:05.501 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:05.501 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:05.501 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:05.501 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:05.501 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:05.501 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:05.501 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:05.501 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:05.501 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:05.501 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:33:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:33:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:33:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:33:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:33:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:33:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:33:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:33:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:33:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:33:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:33:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:33:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:33:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:33:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:33:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:33:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:33:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:33:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:33:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:33:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:33:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:33:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:33:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:33:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:05.508 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:33:05.508 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:05.508 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:05.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:05.508 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:33:05.508 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:33:05.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.519+0000 7f93f4258640 1 -- 172.21.15.112:0/111414016 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f93d006bd30 msgr2=0x7f93d006e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.519+0000 7f93f4258640 1 --2- 172.21.15.112:0/111414016 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f93d006bd30 0x7f93d006e1f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f93e4033000 tx=0x7f93e4009000 comp rx=0 tx=0).stop 2024-04-25T01:33:05.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.519+0000 7f93f4258640 1 -- 172.21.15.112:0/111414016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93ec0c86d0 msgr2=0x7f93ec1edc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.519+0000 7f93f4258640 1 --2- 172.21.15.112:0/111414016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93ec0c86d0 0x7f93ec1edc00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00e990 tx=0x7f93dc00ee50 comp rx=0 tx=0).stop 2024-04-25T01:33:05.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.519+0000 7f93f27f5640 1 -- 172.21.15.112:0/111414016 reap_dead start 2024-04-25T01:33:05.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.519+0000 7f93f27f5640 1 -- 172.21.15.112:0/111414016 reap_dead start 2024-04-25T01:33:05.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.519+0000 7f93f4258640 1 -- 172.21.15.112:0/111414016 shutdown_connections 2024-04-25T01:33:05.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.519+0000 7f93f4258640 1 -- 172.21.15.112:0/111414016 >> 172.21.15.112:0/111414016 conn(0x7f93ec0ad150 msgr2=0x7f93ec0c5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:05.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.519+0000 7f93f4258640 1 -- 172.21.15.112:0/111414016 shutdown_connections 2024-04-25T01:33:05.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.519+0000 7f93f4258640 1 -- 172.21.15.112:0/111414016 wait complete. 2024-04-25T01:33:05.510 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:33:05.510 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T01:33:05.510 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T01:33:05.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.649+0000 7fabfc44c640 1 Processor -- start 2024-04-25T01:33:05.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.649+0000 7fabfc44c640 1 -- start start 2024-04-25T01:33:05.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.650+0000 7fabfc44c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fabf41592c0 0x7fabf40c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.650+0000 7fabfc44c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fabf40c72b0 0x7fabf4160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.650+0000 7fabfc44c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf4159b90 con 0x7fabf41596a0 2024-04-25T01:33:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.650+0000 7fabfc44c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fabf40c6e20 con 0x7fabf40c72b0 2024-04-25T01:33:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.650+0000 7fabfc44c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fabf406c970 con 0x7fabf41592c0 2024-04-25T01:33:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.650+0000 7fabfa9e9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fabf40c72b0 0x7fabf4160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.650+0000 7fabfa1e8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fabf41592c0 0x7fabf40c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.650+0000 7fabfa1e8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fabf41592c0 0x7fabf40c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:58580/0 (socket says 172.21.15.112:58580) 2024-04-25T01:33:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.650+0000 7fabfa9e9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fabf40c72b0 0x7fabf4160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41610/0 (socket says 172.21.15.112:41610) 2024-04-25T01:33:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.650+0000 7fabfa1e8640 1 -- 172.21.15.112:0/2267136776 learned_addr learned my addr 172.21.15.112:0/2267136776 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.650+0000 7fabfa1e8640 1 -- 172.21.15.112:0/2267136776 >> v2:172.21.15.187:3300/0 conn(0x7fabf40c72b0 msgr2=0x7fabf4160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:33:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.650+0000 7fabfa1e8640 1 --2- 172.21.15.112:0/2267136776 >> v2:172.21.15.187:3300/0 conn(0x7fabf40c72b0 0x7fabf4160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.650+0000 7fabfa1e8640 1 -- 172.21.15.112:0/2267136776 >> v1:172.21.15.187:6789/0 conn(0x7fabf41596a0 legacy=0x7fabf4152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.650+0000 7fabfa1e8640 1 -- 172.21.15.112:0/2267136776 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf406cc50 con 0x7fabf41592c0 2024-04-25T01:33:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.650+0000 7fabfa9e9640 1 --2- 172.21.15.112:0/2267136776 >> v2:172.21.15.187:3300/0 conn(0x7fabf40c72b0 0x7fabf4160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:33:05.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.651+0000 7fabfa1e8640 1 --2- 172.21.15.112:0/2267136776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fabf41592c0 0x7fabf40c9070 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009870 tx=0x7fabf002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe03c8405c94bad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:05.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.651+0000 7fabf91e6640 1 -- 172.21.15.112:0/2267136776 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0037070 con 0x7fabf41592c0 2024-04-25T01:33:05.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.652+0000 7fabf91e6640 1 -- 172.21.15.112:0/2267136776 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabf0032440 con 0x7fabf41592c0 2024-04-25T01:33:05.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.652+0000 7fabf91e6640 1 -- 172.21.15.112:0/2267136776 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0032810 con 0x7fabf41592c0 2024-04-25T01:33:05.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.652+0000 7fabfc44c640 1 -- 172.21.15.112:0/2267136776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fabf41592c0 msgr2=0x7fabf40c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.652+0000 7fabfc44c640 1 --2- 172.21.15.112:0/2267136776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fabf41592c0 0x7fabf40c9070 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009870 tx=0x7fabf002a130 comp rx=0 tx=0).stop 2024-04-25T01:33:05.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.652+0000 7fabfc44c640 1 -- 172.21.15.112:0/2267136776 shutdown_connections 2024-04-25T01:33:05.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.652+0000 7fabfc44c640 1 --2- 172.21.15.112:0/2267136776 >> v2:172.21.15.187:3300/0 conn(0x7fabf40c72b0 0x7fabf4160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.652+0000 7fabfc44c640 1 --2- 172.21.15.112:0/2267136776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fabf41592c0 0x7fabf40c9070 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.652+0000 7fabfc44c640 1 -- 172.21.15.112:0/2267136776 >> 172.21.15.112:0/2267136776 conn(0x7fabf40cb170 msgr2=0x7fabf40cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:05.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.652+0000 7fabfc44c640 1 -- 172.21.15.112:0/2267136776 shutdown_connections 2024-04-25T01:33:05.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.652+0000 7fabfc44c640 1 -- 172.21.15.112:0/2267136776 wait complete. 2024-04-25T01:33:05.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.653+0000 7fabfc44c640 1 Processor -- start 2024-04-25T01:33:05.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.653+0000 7fabfc44c640 1 -- start start 2024-04-25T01:33:05.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.654+0000 7fabfc44c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabf40c72b0 0x7fabf40bf400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.654+0000 7fabfc44c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fabf41592c0 0x7fabf40abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.654+0000 7fabfc44c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabf41596a0 0x7fabf41ed6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.654+0000 7fabfc44c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fabf4153590 con 0x7fabf41596a0 2024-04-25T01:33:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.654+0000 7fabfc44c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fabf4153410 con 0x7fabf40c72b0 2024-04-25T01:33:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.654+0000 7fabfc44c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fabf406db20 con 0x7fabf41592c0 2024-04-25T01:33:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.654+0000 7fabfa1e8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabf40c72b0 0x7fabf40bf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.654+0000 7fabf99e7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fabf41592c0 0x7fabf40abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.654+0000 7fabfa9e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabf41596a0 0x7fabf41ed6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.654+0000 7fabfa1e8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabf40c72b0 0x7fabf40bf400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41614/0 (socket says 172.21.15.112:41614) 2024-04-25T01:33:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.654+0000 7fabfa1e8640 1 -- 172.21.15.112:0/1670807425 learned_addr learned my addr 172.21.15.112:0/1670807425 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.654+0000 7fabf99e7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fabf41592c0 0x7fabf40abda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:58596/0 (socket says 172.21.15.112:58596) 2024-04-25T01:33:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.655+0000 7fabfa9e9640 1 -- 172.21.15.112:0/1670807425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fabf41592c0 msgr2=0x7fabf40abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.655+0000 7fabfa9e9640 1 --2- 172.21.15.112:0/1670807425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fabf41592c0 0x7fabf40abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.655+0000 7fabfa9e9640 1 -- 172.21.15.112:0/1670807425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabf40c72b0 msgr2=0x7fabf40bf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.655+0000 7fabfa9e9640 1 --2- 172.21.15.112:0/1670807425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabf40c72b0 0x7fabf40bf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.655+0000 7fabfa9e9640 1 -- 172.21.15.112:0/1670807425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf41ec940 con 0x7fabf41596a0 2024-04-25T01:33:05.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.655+0000 7fabfa1e8640 1 --2- 172.21.15.112:0/1670807425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabf40c72b0 0x7fabf40bf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:33:05.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.655+0000 7fabfa9e9640 1 --2- 172.21.15.112:0/1670807425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabf41596a0 0x7fabf41ed6c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fabec009db0 tx=0x7fabec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:05.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.655+0000 7fabeaffd640 1 -- 172.21.15.112:0/1670807425 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec018030 con 0x7fabf41596a0 2024-04-25T01:33:05.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.655+0000 7fabeaffd640 1 -- 172.21.15.112:0/1670807425 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabec00cd30 con 0x7fabf41596a0 2024-04-25T01:33:05.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.655+0000 7fabfc44c640 1 -- 172.21.15.112:0/1670807425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf41ecbe0 con 0x7fabf41596a0 2024-04-25T01:33:05.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.655+0000 7fabeaffd640 1 -- 172.21.15.112:0/1670807425 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec003bb0 con 0x7fabf41596a0 2024-04-25T01:33:05.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.655+0000 7fabfc44c640 1 -- 172.21.15.112:0/1670807425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf40ac2e0 con 0x7fabf41596a0 2024-04-25T01:33:05.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.656+0000 7fabfc44c640 1 -- 172.21.15.112:0/1670807425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf40679a0 con 0x7fabf41596a0 2024-04-25T01:33:05.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.657+0000 7fabeaffd640 1 -- 172.21.15.112:0/1670807425 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fabec005d10 con 0x7fabf41596a0 2024-04-25T01:33:05.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.657+0000 7fabeaffd640 1 --2- 172.21.15.112:0/1670807425 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fabd806be30 0x7fabd806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:05.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.657+0000 7fabeaffd640 1 -- 172.21.15.112:0/1670807425 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fabec08b9a0 con 0x7fabf41596a0 2024-04-25T01:33:05.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.657+0000 7fabfa1e8640 1 --2- 172.21.15.112:0/1670807425 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fabd806be30 0x7fabd806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:05.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.658+0000 7fabfa1e8640 1 --2- 172.21.15.112:0/1670807425 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fabd806be30 0x7fabd806e2f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fabf00090b0 tx=0x7fabf002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:05.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.661+0000 7fabeaffd640 1 -- 172.21.15.112:0/1670807425 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fabec022070 con 0x7fabf41596a0 2024-04-25T01:33:05.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.972+0000 7fabfc44c640 1 -- 172.21.15.112:0/1670807425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fabf40c79d0 con 0x7fabf41596a0 2024-04-25T01:33:05.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.973+0000 7fabeaffd640 1 -- 172.21.15.112:0/1670807425 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fabec010040 con 0x7fabf41596a0 2024-04-25T01:33:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.975+0000 7fabfc44c640 1 -- 172.21.15.112:0/1670807425 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fabd806be30 msgr2=0x7fabd806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.975+0000 7fabfc44c640 1 --2- 172.21.15.112:0/1670807425 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fabd806be30 0x7fabd806e2f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fabf00090b0 tx=0x7fabf002e000 comp rx=0 tx=0).stop 2024-04-25T01:33:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.975+0000 7fabfc44c640 1 -- 172.21.15.112:0/1670807425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabf41596a0 msgr2=0x7fabf41ed6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.975+0000 7fabfc44c640 1 --2- 172.21.15.112:0/1670807425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabf41596a0 0x7fabf41ed6c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fabec009db0 tx=0x7fabec00c490 comp rx=0 tx=0).stop 2024-04-25T01:33:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.976+0000 7fabfc44c640 1 -- 172.21.15.112:0/1670807425 shutdown_connections 2024-04-25T01:33:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.976+0000 7fabfc44c640 1 --2- 172.21.15.112:0/1670807425 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fabd806be30 0x7fabd806e2f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.976+0000 7fabfc44c640 1 --2- 172.21.15.112:0/1670807425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabf41596a0 0x7fabf41ed6c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.976+0000 7fabfc44c640 1 --2- 172.21.15.112:0/1670807425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fabf41592c0 0x7fabf40abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.976+0000 7fabfc44c640 1 --2- 172.21.15.112:0/1670807425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabf40c72b0 0x7fabf40bf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:05.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.976+0000 7fabfc44c640 1 -- 172.21.15.112:0/1670807425 >> 172.21.15.112:0/1670807425 conn(0x7fabf40cb170 msgr2=0x7fabf4152150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:05.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.976+0000 7fabfc44c640 1 -- 172.21.15.112:0/1670807425 shutdown_connections 2024-04-25T01:33:05.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:05.976+0000 7fabfc44c640 1 -- 172.21.15.112:0/1670807425 wait complete. 2024-04-25T01:33:05.965 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T01:33:15.966 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:33:15.967 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:33:16.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.100+0000 7f00791e9640 1 Processor -- start 2024-04-25T01:33:16.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.100+0000 7f00791e9640 1 -- start start 2024-04-25T01:33:16.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.101+0000 7f00791e9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f00740c3980 0x7f0074153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.101+0000 7f00791e9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00740aeac0 con 0x7f00740c5f40 2024-04-25T01:33:16.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.101+0000 7f00791e9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00741578c0 con 0x7f007406f010 2024-04-25T01:33:16.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.101+0000 7f00791e9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f0074071090 con 0x7f00740c3980 2024-04-25T01:33:16.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.101+0000 7f006bfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f00740c3980 0x7f0074153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.101+0000 7f0072d9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f00740c5f40 0x7f00741535e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38846/0 (socket says 172.21.15.112:38846) 2024-04-25T01:33:16.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.101+0000 7f0072d9d640 1 -- 172.21.15.112:0/1650666206 learned_addr learned my addr 172.21.15.112:0/1650666206 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:33:16.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.101+0000 7f006bfff640 1 -- 172.21.15.112:0/1650666206 >> v1:172.21.15.187:6789/0 conn(0x7f007406f010 legacy=0x7f007406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.101+0000 7f007259c640 1 -- 172.21.15.112:0/1650666206 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 448890726 0 0) 0x7f00741578c0 con 0x7f007406f010 2024-04-25T01:33:16.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.101+0000 7f006bfff640 1 -- 172.21.15.112:0/1650666206 >> v1:172.21.15.112:6789/0 conn(0x7f00740c5f40 legacy=0x7f00741535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.101+0000 7f006bfff640 1 -- 172.21.15.112:0/1650666206 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00740712c0 con 0x7f00740c3980 2024-04-25T01:33:16.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.102+0000 7f006bfff640 1 --2- 172.21.15.112:0/1650666206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00740c3980 0x7f0074153cf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f005c0097a0 tx=0x7f005c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a66696c6994796 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:16.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.102+0000 7f007259c640 1 -- 172.21.15.112:0/1650666206 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c037070 con 0x7f00740c3980 2024-04-25T01:33:16.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.102+0000 7f007259c640 1 -- 172.21.15.112:0/1650666206 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f005c02adc0 con 0x7f00740c3980 2024-04-25T01:33:16.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.102+0000 7f007259c640 1 -- 172.21.15.112:0/1650666206 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c032660 con 0x7f00740c3980 2024-04-25T01:33:16.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.103+0000 7f00791e9640 1 -- 172.21.15.112:0/1650666206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00740c3980 msgr2=0x7f0074153cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.103+0000 7f00791e9640 1 --2- 172.21.15.112:0/1650666206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00740c3980 0x7f0074153cf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f005c0097a0 tx=0x7f005c02a130 comp rx=0 tx=0).stop 2024-04-25T01:33:16.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.103+0000 7f00791e9640 1 -- 172.21.15.112:0/1650666206 shutdown_connections 2024-04-25T01:33:16.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.103+0000 7f00791e9640 1 --2- 172.21.15.112:0/1650666206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00740c3980 0x7f0074153cf0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.103+0000 7f00791e9640 1 -- 172.21.15.112:0/1650666206 >> 172.21.15.112:0/1650666206 conn(0x7f00740c8f80 msgr2=0x7f00740c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:16.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.103+0000 7f00791e9640 1 -- 172.21.15.112:0/1650666206 shutdown_connections 2024-04-25T01:33:16.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.103+0000 7f00791e9640 1 -- 172.21.15.112:0/1650666206 wait complete. 2024-04-25T01:33:16.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.104+0000 7f00791e9640 1 Processor -- start 2024-04-25T01:33:16.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.104+0000 7f00791e9640 1 -- start start 2024-04-25T01:33:16.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.105+0000 7f00791e9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f007406f010 0x7f00740ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.105+0000 7f00791e9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00740c3980 0x7f00741ec3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.105+0000 7f00791e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00740c5f40 0x7f00741ec900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.105+0000 7f00791e9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f00740717e0 con 0x7f00740c5f40 2024-04-25T01:33:16.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.105+0000 7f00791e9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0074157590 con 0x7f00740c3980 2024-04-25T01:33:16.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.105+0000 7f00791e9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f00740aef30 con 0x7f007406f010 2024-04-25T01:33:16.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.105+0000 7f007359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00740c5f40 0x7f00741ec900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.105+0000 7f006bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00740c3980 0x7f00741ec3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.105+0000 7f007359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00740c5f40 0x7f00741ec900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56336/0 (socket says 172.21.15.112:56336) 2024-04-25T01:33:16.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.105+0000 7f006bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00740c3980 0x7f00741ec3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:36420/0 (socket says 172.21.15.112:36420) 2024-04-25T01:33:16.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.105+0000 7f007359e640 1 -- 172.21.15.112:0/3576586280 learned_addr learned my addr 172.21.15.112:0/3576586280 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:16.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.105+0000 7f0072d9d640 1 --2- 172.21.15.112:0/3576586280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f007406f010 0x7f00740ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.106+0000 7f007359e640 1 -- 172.21.15.112:0/3576586280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f007406f010 msgr2=0x7f00740ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.106+0000 7f007359e640 1 --2- 172.21.15.112:0/3576586280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f007406f010 0x7f00740ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.106+0000 7f007359e640 1 -- 172.21.15.112:0/3576586280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00740c3980 msgr2=0x7f00741ec3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.106+0000 7f007359e640 1 --2- 172.21.15.112:0/3576586280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00740c3980 0x7f00741ec3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.106+0000 7f007359e640 1 -- 172.21.15.112:0/3576586280 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0074151df0 con 0x7f00740c5f40 2024-04-25T01:33:16.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.106+0000 7f007359e640 1 --2- 172.21.15.112:0/3576586280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00740c5f40 0x7f00741ec900 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f006c00c790 tx=0x7f006c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:16.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.106+0000 7f006b7fe640 1 -- 172.21.15.112:0/3576586280 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c003da0 con 0x7f00740c5f40 2024-04-25T01:33:16.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.106+0000 7f006b7fe640 1 -- 172.21.15.112:0/3576586280 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f006c005070 con 0x7f00740c5f40 2024-04-25T01:33:16.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.106+0000 7f00791e9640 1 -- 172.21.15.112:0/3576586280 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00741520d0 con 0x7f00740c5f40 2024-04-25T01:33:16.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.106+0000 7f006b7fe640 1 -- 172.21.15.112:0/3576586280 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c005460 con 0x7f00740c5f40 2024-04-25T01:33:16.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.106+0000 7f00791e9640 1 -- 172.21.15.112:0/3576586280 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0074153710 con 0x7f00740c5f40 2024-04-25T01:33:16.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.107+0000 7f00791e9640 1 -- 172.21.15.112:0/3576586280 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00740682a0 con 0x7f00740c5f40 2024-04-25T01:33:16.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.107+0000 7f006b7fe640 1 -- 172.21.15.112:0/3576586280 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f006c005600 con 0x7f00740c5f40 2024-04-25T01:33:16.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.108+0000 7f006b7fe640 1 --2- 172.21.15.112:0/3576586280 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f005406be30 0x7f005406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.108+0000 7f006b7fe640 1 -- 172.21.15.112:0/3576586280 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f006c08b090 con 0x7f00740c5f40 2024-04-25T01:33:16.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.108+0000 7f0072d9d640 1 --2- 172.21.15.112:0/3576586280 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f005406be30 0x7f005406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.108+0000 7f0072d9d640 1 --2- 172.21.15.112:0/3576586280 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f005406be30 0x7f005406e2f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0064000f60 tx=0x7f0064007e20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:16.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.112+0000 7f006b7fe640 1 -- 172.21.15.112:0/3576586280 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f006c059050 con 0x7f00740c5f40 2024-04-25T01:33:16.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.461+0000 7f00791e9640 1 -- 172.21.15.112:0/3576586280 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f00740c8a50 con 0x7f00740c5f40 2024-04-25T01:33:16.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.461+0000 7f006b7fe640 1 -- 172.21.15.112:0/3576586280 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f006c05d350 con 0x7f00740c5f40 2024-04-25T01:33:16.438 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":56,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:33:16.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.463+0000 7f00791e9640 1 -- 172.21.15.112:0/3576586280 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f005406be30 msgr2=0x7f005406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.463+0000 7f00791e9640 1 --2- 172.21.15.112:0/3576586280 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f005406be30 0x7f005406e2f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0064000f60 tx=0x7f0064007e20 comp rx=0 tx=0).stop 2024-04-25T01:33:16.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.463+0000 7f00791e9640 1 -- 172.21.15.112:0/3576586280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00740c5f40 msgr2=0x7f00741ec900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.463+0000 7f00791e9640 1 --2- 172.21.15.112:0/3576586280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00740c5f40 0x7f00741ec900 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f006c00c790 tx=0x7f006c00cc50 comp rx=0 tx=0).stop 2024-04-25T01:33:16.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.463+0000 7f00791e9640 1 -- 172.21.15.112:0/3576586280 shutdown_connections 2024-04-25T01:33:16.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.463+0000 7f00791e9640 1 --2- 172.21.15.112:0/3576586280 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f005406be30 0x7f005406e2f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.463+0000 7f00791e9640 1 --2- 172.21.15.112:0/3576586280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00740c5f40 0x7f00741ec900 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.463+0000 7f00791e9640 1 --2- 172.21.15.112:0/3576586280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00740c3980 0x7f00741ec3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.463+0000 7f00791e9640 1 --2- 172.21.15.112:0/3576586280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f007406f010 0x7f00740ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.463+0000 7f00791e9640 1 -- 172.21.15.112:0/3576586280 >> 172.21.15.112:0/3576586280 conn(0x7f00740c8f80 msgr2=0x7f00740aa230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:16.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.463+0000 7f00791e9640 1 -- 172.21.15.112:0/3576586280 shutdown_connections 2024-04-25T01:33:16.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.463+0000 7f00791e9640 1 -- 172.21.15.112:0/3576586280 wait complete. 2024-04-25T01:33:16.452 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:33:16.452 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T01:33:16.452 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:33:16.453 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:33:16.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.583+0000 7fa493f7e640 1 Processor -- start 2024-04-25T01:33:16.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.583+0000 7fa493f7e640 1 -- start start 2024-04-25T01:33:16.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.584+0000 7fa493f7e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa48c06ed60 0x7fa48c157810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.584+0000 7fa493f7e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa48c1541c0 0x7fa48c0ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.584+0000 7fa493f7e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa48c0a9630 con 0x7fa48c072ec0 2024-04-25T01:33:16.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.584+0000 7fa493f7e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa48c0723f0 con 0x7fa48c06ed60 2024-04-25T01:33:16.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.584+0000 7fa493f7e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa48c0a9910 con 0x7fa48c1541c0 2024-04-25T01:33:16.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.584+0000 7fa491519640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa48c06ed60 0x7fa48c157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.584+0000 7fa491519640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa48c06ed60 0x7fa48c157810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56340/0 (socket says 172.21.15.112:56340) 2024-04-25T01:33:16.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.584+0000 7fa491519640 1 -- 172.21.15.112:0/368326200 learned_addr learned my addr 172.21.15.112:0/368326200 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:16.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.584+0000 7fa49251b640 1 --2- 172.21.15.112:0/368326200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa48c1541c0 0x7fa48c0ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.584+0000 7fa491519640 1 -- 172.21.15.112:0/368326200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa48c1541c0 msgr2=0x7fa48c0ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:33:16.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.584+0000 7fa491519640 1 --2- 172.21.15.112:0/368326200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa48c1541c0 0x7fa48c0ad4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.584+0000 7fa491519640 1 -- 172.21.15.112:0/368326200 >> v1:172.21.15.187:6789/0 conn(0x7fa48c072ec0 legacy=0x7fa48c071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.584+0000 7fa491519640 1 -- 172.21.15.112:0/368326200 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c156d30 con 0x7fa48c06ed60 2024-04-25T01:33:16.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.585+0000 7fa491519640 1 --2- 172.21.15.112:0/368326200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c06ed60 0x7fa48c157810 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009870 tx=0x7fa47c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f3f5c7a74cd57b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:16.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.585+0000 7fa490d18640 1 -- 172.21.15.112:0/368326200 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c037070 con 0x7fa48c06ed60 2024-04-25T01:33:16.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.585+0000 7fa490d18640 1 -- 172.21.15.112:0/368326200 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47c032440 con 0x7fa48c06ed60 2024-04-25T01:33:16.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.585+0000 7fa490d18640 1 -- 172.21.15.112:0/368326200 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c032810 con 0x7fa48c06ed60 2024-04-25T01:33:16.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.586+0000 7fa493f7e640 1 -- 172.21.15.112:0/368326200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c06ed60 msgr2=0x7fa48c157810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.586+0000 7fa493f7e640 1 --2- 172.21.15.112:0/368326200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c06ed60 0x7fa48c157810 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009870 tx=0x7fa47c02a1c0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.586+0000 7fa493f7e640 1 -- 172.21.15.112:0/368326200 shutdown_connections 2024-04-25T01:33:16.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.586+0000 7fa493f7e640 1 --2- 172.21.15.112:0/368326200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa48c1541c0 0x7fa48c0ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.586+0000 7fa493f7e640 1 --2- 172.21.15.112:0/368326200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c06ed60 0x7fa48c157810 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.586+0000 7fa493f7e640 1 -- 172.21.15.112:0/368326200 >> 172.21.15.112:0/368326200 conn(0x7fa48c0c8090 msgr2=0x7fa48c0c8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:16.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.586+0000 7fa493f7e640 1 -- 172.21.15.112:0/368326200 shutdown_connections 2024-04-25T01:33:16.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.586+0000 7fa493f7e640 1 -- 172.21.15.112:0/368326200 wait complete. 2024-04-25T01:33:16.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.587+0000 7fa493f7e640 1 Processor -- start 2024-04-25T01:33:16.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.587+0000 7fa493f7e640 1 -- start start 2024-04-25T01:33:16.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.588+0000 7fa493f7e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa48c06ed60 0x7fa48c0c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.588+0000 7fa493f7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c072ec0 0x7fa48c1517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.588+0000 7fa493f7e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa48c1541c0 0x7fa48c149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.588+0000 7fa493f7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c0732a0 con 0x7fa48c072ec0 2024-04-25T01:33:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.588+0000 7fa493f7e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1572f0 con 0x7fa48c06ed60 2024-04-25T01:33:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.588+0000 7fa493f7e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa48c1545f0 con 0x7fa48c1541c0 2024-04-25T01:33:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.588+0000 7fa491519640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c072ec0 0x7fa48c1517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.588+0000 7fa491519640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c072ec0 0x7fa48c1517d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56346/0 (socket says 172.21.15.112:56346) 2024-04-25T01:33:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.588+0000 7fa491519640 1 -- 172.21.15.112:0/3200266719 learned_addr learned my addr 172.21.15.112:0/3200266719 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.588+0000 7fa491d1a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa48c06ed60 0x7fa48c0c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.588+0000 7fa49251b640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa48c1541c0 0x7fa48c149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.589+0000 7fa491519640 1 -- 172.21.15.112:0/3200266719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa48c1541c0 msgr2=0x7fa48c149850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.589+0000 7fa491519640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa48c1541c0 0x7fa48c149850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.589+0000 7fa491519640 1 -- 172.21.15.112:0/3200266719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa48c06ed60 msgr2=0x7fa48c0c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.589+0000 7fa491519640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa48c06ed60 0x7fa48c0c6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.589+0000 7fa491519640 1 -- 172.21.15.112:0/3200266719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c0c7200 con 0x7fa48c072ec0 2024-04-25T01:33:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.589+0000 7fa491d1a640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa48c06ed60 0x7fa48c0c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:33:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.589+0000 7fa49251b640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa48c1541c0 0x7fa48c149850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:33:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.589+0000 7fa491519640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c072ec0 0x7fa48c1517d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa47c000c00 tx=0x7fa47c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.589+0000 7fa4827fc640 1 -- 172.21.15.112:0/3200266719 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c037040 con 0x7fa48c072ec0 2024-04-25T01:33:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.589+0000 7fa4827fc640 1 -- 172.21.15.112:0/3200266719 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47c042070 con 0x7fa48c072ec0 2024-04-25T01:33:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.589+0000 7fa493f7e640 1 -- 172.21.15.112:0/3200266719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48c0c74e0 con 0x7fa48c072ec0 2024-04-25T01:33:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.589+0000 7fa4827fc640 1 -- 172.21.15.112:0/3200266719 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c030880 con 0x7fa48c072ec0 2024-04-25T01:33:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.589+0000 7fa493f7e640 1 -- 172.21.15.112:0/3200266719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48c14e250 con 0x7fa48c072ec0 2024-04-25T01:33:16.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.590+0000 7fa493f7e640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c14e490 0x7fa48c149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.590+0000 7fa493f7e640 1 -- 172.21.15.112:0/3200266719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c075980 con 0x7fa48c14e490 2024-04-25T01:33:16.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.591+0000 7fa491d1a640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c14e490 0x7fa48c149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.591+0000 7fa491d1a640 1 -- 172.21.15.112:0/3200266719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa48c0689b0 con 0x7fa48c14e490 2024-04-25T01:33:16.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.591+0000 7fa4827fc640 1 -- 172.21.15.112:0/3200266719 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa47c02ad10 con 0x7fa48c072ec0 2024-04-25T01:33:16.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.591+0000 7fa491d1a640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c14e490 0x7fa48c149d90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa48800fb10 tx=0x7fa488001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:16.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.591+0000 7fa4827fc640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa47006be60 0x7fa47006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.591+0000 7fa4827fc640 1 -- 172.21.15.112:0/3200266719 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fa47c0ab5a0 con 0x7fa48c072ec0 2024-04-25T01:33:16.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.591+0000 7fa49251b640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa47006be60 0x7fa47006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.592+0000 7fa4827fc640 1 -- 172.21.15.112:0/3200266719 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4880052e0 con 0x7fa48c14e490 2024-04-25T01:33:16.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.592+0000 7fa4827fc640 1 -- 172.21.15.112:0/3200266719 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa488005480 con 0x7fa48c14e490 2024-04-25T01:33:16.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.592+0000 7fa4827fc640 1 -- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c14e490 msgr2=0x7fa48c149d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.592+0000 7fa4827fc640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c14e490 0x7fa48c149d90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa48800fb10 tx=0x7fa488001cf0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.592+0000 7fa4827fc640 1 -- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c14e490 msgr2=0x7fa48c149d90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:33:16.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.592+0000 7fa4827fc640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c14e490 0x7fa48c149d90 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa48800fb10 tx=0x7fa488001cf0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.592+0000 7fa49251b640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa47006be60 0x7fa47006e320 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa484009fd0 tx=0x7fa48400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:16.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.600+0000 7fa493f7e640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c14ea50 0x7fa48c14ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.600+0000 7fa493f7e640 1 -- 172.21.15.112:0/3200266719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c0c5a60 con 0x7fa48c14ea50 2024-04-25T01:33:16.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.600+0000 7fa491d1a640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c14ea50 0x7fa48c14ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.601+0000 7fa491d1a640 1 -- 172.21.15.112:0/3200266719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa48c0c8f40 con 0x7fa48c14ea50 2024-04-25T01:33:16.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.601+0000 7fa491d1a640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c14ea50 0x7fa48c14ee60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa488016000 tx=0x7fa488000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:16.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.601+0000 7fa4827fc640 1 -- 172.21.15.112:0/3200266719 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa488006c10 con 0x7fa48c14ea50 2024-04-25T01:33:16.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.601+0000 7fa4827fc640 1 -- 172.21.15.112:0/3200266719 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4731 (secure 0 0 0) 0x7fa488006c10 con 0x7fa48c14ea50 2024-04-25T01:33:16.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.601+0000 7fa4827fc640 1 -- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c14ea50 msgr2=0x7fa48c14ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.601+0000 7fa4827fc640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c14ea50 0x7fa48c14ee60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa488016000 tx=0x7fa488000ed0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.601+0000 7fa4827fc640 1 -- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c14ea50 msgr2=0x7fa48c14ee60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:33:16.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.601+0000 7fa4827fc640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c14ea50 0x7fa48c14ee60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.579 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:33:16.579 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:33:16.579 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:33:16.579 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:33:16.579 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 56, 2024-04-25T01:33:16.580 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 77366, 2024-04-25T01:33:16.580 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:33:16.580 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:33:16.580 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:33:16.580 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:33:16.580 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:16.580 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 14, 2024-04-25T01:33:16.580 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:33:16.580 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:33:16.580 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:33:16.580 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:16.580 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:16.580 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:16.580 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:16.580 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:16.581 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:16.581 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:16.581 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:16.581 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:16.581 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:16.581 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:16.581 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:16.581 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:33:16.581 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:33:16.581 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:16.581 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:16.581 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:16.581 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:16.581 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:16.581 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:16.582 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:16.582 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:16.582 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:16.582 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:16.582 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:16.582 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:16.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.582 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:33:16.582 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:33:16.582 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:33:16.582 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:33:16.582 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:33:16.582 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:33:16.582 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:33:16.582 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:33:16.583 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:33:16.583 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:33:16.583 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:33:16.583 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:33:16.583 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:33:16.583 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:33:16.583 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:33:16.583 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:33:16.583 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:33:16.583 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:16.583 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:16.583 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:16.583 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:16.583 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:16.583 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:16.584 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:16.584 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:16.584 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:16.584 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:16.584 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:16.584 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:16.584 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:33:16.584 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.584 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:33:16.584 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.585 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:33:16.585 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:33:16.585 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:16.585 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:16.585 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.585 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:16.585 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:33:16.585 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:16.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.585 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.585 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:16.585 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:33:16.585 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:16.585 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.585 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:16.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.586 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:33:16.586 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:33:16.586 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:16.586 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:16.586 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:16.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.586 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.586 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:33:16.586 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:33:16.586 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:16.586 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:16.586 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.586 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:16.587 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:33:16.587 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:16.587 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.587 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.587 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:16.587 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:33:16.587 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:16.587 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.587 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:16.587 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.587 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:33:16.587 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:33:16.587 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:16.587 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:16.587 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:16.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.588 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.588 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:33:16.588 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:33:16.588 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:16.588 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:16.588 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.588 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:16.588 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:33:16.588 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:16.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.588 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.588 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:16.588 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:33:16.588 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:16.589 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.589 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:16.589 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.589 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:33:16.589 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:33:16.589 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:16.589 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:16.589 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:16.589 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.589 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:16.589 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.589 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:33:16.589 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:33:16.589 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.590 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:16.590 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:16.590 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:33:16.590 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.590 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:16.590 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:33:16.590 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.590 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:16.590 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:16.590 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:33:16.590 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.590 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:16.590 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:33:16.590 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.591 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:16.591 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:16.591 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:33:16.591 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.591 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:16.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.591 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:33:16.591 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:33:16.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.602+0000 7fa493f7e640 1 -- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa47006be60 msgr2=0x7fa47006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.602+0000 7fa493f7e640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa47006be60 0x7fa47006e320 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa484009fd0 tx=0x7fa48400b040 comp rx=0 tx=0).stop 2024-04-25T01:33:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.602+0000 7fa493f7e640 1 -- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c072ec0 msgr2=0x7fa48c1517d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.602+0000 7fa493f7e640 1 --2- 172.21.15.112:0/3200266719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c072ec0 0x7fa48c1517d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa47c000c00 tx=0x7fa47c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.602+0000 7fa49251b640 1 -- 172.21.15.112:0/3200266719 reap_dead start 2024-04-25T01:33:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.602+0000 7fa49251b640 1 -- 172.21.15.112:0/3200266719 reap_dead start 2024-04-25T01:33:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.602+0000 7fa493f7e640 1 -- 172.21.15.112:0/3200266719 shutdown_connections 2024-04-25T01:33:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.602+0000 7fa493f7e640 1 -- 172.21.15.112:0/3200266719 >> 172.21.15.112:0/3200266719 conn(0x7fa48c0c8090 msgr2=0x7fa48c0c4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.602+0000 7fa493f7e640 1 -- 172.21.15.112:0/3200266719 shutdown_connections 2024-04-25T01:33:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.602+0000 7fa493f7e640 1 -- 172.21.15.112:0/3200266719 wait complete. 2024-04-25T01:33:16.593 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:33:16.593 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:33:16.594 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:33:16.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.715+0000 7fc9eb963640 1 Processor -- start 2024-04-25T01:33:16.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.716+0000 7fc9eb963640 1 -- start start 2024-04-25T01:33:16.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.716+0000 7fc9eb963640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9e41596a0 0x7fc9e4152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.716+0000 7fc9eb963640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e4159b90 con 0x7fc9e40c72b0 2024-04-25T01:33:16.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.716+0000 7fc9eb963640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e406c970 con 0x7fc9e41592c0 2024-04-25T01:33:16.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.716+0000 7fc9eb963640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e40c6e20 con 0x7fc9e41596a0 2024-04-25T01:33:16.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.717+0000 7fc9e9f00640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc9e40c72b0 0x7fc9e4160380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38856/0 (socket says 172.21.15.112:38856) 2024-04-25T01:33:16.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.717+0000 7fc9e9f00640 1 -- 172.21.15.112:0/578603381 learned_addr learned my addr 172.21.15.112:0/578603381 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:33:16.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.717+0000 7fc9e8efe640 1 --2- 172.21.15.112:0/578603381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9e41596a0 0x7fc9e4152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.717+0000 7fc9d7fff640 1 -- 172.21.15.112:0/578603381 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3138689001 0 0) 0x7fc9e4159b90 con 0x7fc9e40c72b0 2024-04-25T01:33:16.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.717+0000 7fc9d7fff640 1 -- 172.21.15.112:0/578603381 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9d0003520 con 0x7fc9e40c72b0 2024-04-25T01:33:16.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.717+0000 7fc9d7fff640 1 -- 172.21.15.112:0/578603381 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 363507534 0 0) 0x7fc9e406c970 con 0x7fc9e41592c0 2024-04-25T01:33:16.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.717+0000 7fc9e8efe640 1 -- 172.21.15.112:0/578603381 >> v1:172.21.15.187:6789/0 conn(0x7fc9e41592c0 legacy=0x7fc9e40c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.717+0000 7fc9e8efe640 1 -- 172.21.15.112:0/578603381 >> v1:172.21.15.112:6789/0 conn(0x7fc9e40c72b0 legacy=0x7fc9e4160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.717+0000 7fc9e8efe640 1 -- 172.21.15.112:0/578603381 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e406cc50 con 0x7fc9e41596a0 2024-04-25T01:33:16.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.718+0000 7fc9e8efe640 1 --2- 172.21.15.112:0/578603381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9e41596a0 0x7fc9e4152d00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9d80097d0 tx=0x7fc9d802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d51f5b72a6fb656 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:16.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.718+0000 7fc9d7fff640 1 -- 172.21.15.112:0/578603381 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8037070 con 0x7fc9e41596a0 2024-04-25T01:33:16.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.718+0000 7fc9d7fff640 1 -- 172.21.15.112:0/578603381 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d802ae50 con 0x7fc9e41596a0 2024-04-25T01:33:16.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.718+0000 7fc9d7fff640 1 -- 172.21.15.112:0/578603381 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8032640 con 0x7fc9e41596a0 2024-04-25T01:33:16.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.719+0000 7fc9eb963640 1 -- 172.21.15.112:0/578603381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9e41596a0 msgr2=0x7fc9e4152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.719+0000 7fc9eb963640 1 --2- 172.21.15.112:0/578603381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9e41596a0 0x7fc9e4152d00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9d80097d0 tx=0x7fc9d802a100 comp rx=0 tx=0).stop 2024-04-25T01:33:16.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.719+0000 7fc9eb963640 1 -- 172.21.15.112:0/578603381 shutdown_connections 2024-04-25T01:33:16.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.719+0000 7fc9eb963640 1 --2- 172.21.15.112:0/578603381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9e41596a0 0x7fc9e4152d00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.719+0000 7fc9eb963640 1 -- 172.21.15.112:0/578603381 >> 172.21.15.112:0/578603381 conn(0x7fc9e40cb170 msgr2=0x7fc9e40cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:16.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.719+0000 7fc9eb963640 1 -- 172.21.15.112:0/578603381 shutdown_connections 2024-04-25T01:33:16.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.719+0000 7fc9eb963640 1 -- 172.21.15.112:0/578603381 wait complete. 2024-04-25T01:33:16.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.721+0000 7fc9eb963640 1 Processor -- start 2024-04-25T01:33:16.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.721+0000 7fc9eb963640 1 -- start start 2024-04-25T01:33:16.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.721+0000 7fc9eb963640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9e40c72b0 0x7fc9e40c11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.721+0000 7fc9eb963640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e41592c0 0x7fc9e414d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.721+0000 7fc9eb963640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9e41596a0 0x7fc9e414d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.721+0000 7fc9eb963640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e406c510 con 0x7fc9e41596a0 2024-04-25T01:33:16.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.721+0000 7fc9eb963640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e406c390 con 0x7fc9e41592c0 2024-04-25T01:33:16.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.721+0000 7fc9eb963640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e406db20 con 0x7fc9e40c72b0 2024-04-25T01:33:16.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.721+0000 7fc9e9f00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9e41596a0 0x7fc9e414d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.721+0000 7fc9e9f00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9e41596a0 0x7fc9e414d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56390/0 (socket says 172.21.15.112:56390) 2024-04-25T01:33:16.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.721+0000 7fc9e9f00640 1 -- 172.21.15.112:0/2841534617 learned_addr learned my addr 172.21.15.112:0/2841534617 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:16.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.722+0000 7fc9e8efe640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e41592c0 0x7fc9e414d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.722+0000 7fc9e96ff640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9e40c72b0 0x7fc9e40c11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.722+0000 7fc9e9f00640 1 -- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9e40c72b0 msgr2=0x7fc9e40c11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.722+0000 7fc9e9f00640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9e40c72b0 0x7fc9e40c11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.722+0000 7fc9e9f00640 1 -- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e41592c0 msgr2=0x7fc9e414d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.722+0000 7fc9e9f00640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e41592c0 0x7fc9e414d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.722+0000 7fc9e9f00640 1 -- 172.21.15.112:0/2841534617 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e40c1710 con 0x7fc9e41596a0 2024-04-25T01:33:16.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.722+0000 7fc9e8efe640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e41592c0 0x7fc9e414d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:33:16.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.722+0000 7fc9e9f00640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9e41596a0 0x7fc9e414d750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00cae0 tx=0x7fc9dc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:16.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.722+0000 7fc9d5ffb640 1 -- 172.21.15.112:0/2841534617 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc017070 con 0x7fc9e41596a0 2024-04-25T01:33:16.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.722+0000 7fc9d5ffb640 1 -- 172.21.15.112:0/2841534617 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9dc00ac40 con 0x7fc9e41596a0 2024-04-25T01:33:16.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.722+0000 7fc9eb963640 1 -- 172.21.15.112:0/2841534617 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e4153ad0 con 0x7fc9e41596a0 2024-04-25T01:33:16.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.722+0000 7fc9d5ffb640 1 -- 172.21.15.112:0/2841534617 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc012600 con 0x7fc9e41596a0 2024-04-25T01:33:16.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.723+0000 7fc9eb963640 1 -- 172.21.15.112:0/2841534617 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e4154d60 con 0x7fc9e41596a0 2024-04-25T01:33:16.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.724+0000 7fc9eb963640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e414dc90 0x7fc9e414e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.724+0000 7fc9eb963640 1 -- 172.21.15.112:0/2841534617 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e414a090 con 0x7fc9e414dc90 2024-04-25T01:33:16.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.724+0000 7fc9d5ffb640 1 -- 172.21.15.112:0/2841534617 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc9dc01a070 con 0x7fc9e41596a0 2024-04-25T01:33:16.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.724+0000 7fc9e96ff640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e414dc90 0x7fc9e414e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.724+0000 7fc9d5ffb640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc9c806be30 0x7fc9c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.725+0000 7fc9d5ffb640 1 -- 172.21.15.112:0/2841534617 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fc9dc08ade0 con 0x7fc9e41596a0 2024-04-25T01:33:16.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.725+0000 7fc9e96ff640 1 -- 172.21.15.112:0/2841534617 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc9e4075980 con 0x7fc9e414dc90 2024-04-25T01:33:16.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.725+0000 7fc9e8efe640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc9c806be30 0x7fc9c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.725+0000 7fc9e96ff640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e414dc90 0x7fc9e414e3f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0000f60 tx=0x7fc9e000e960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:16.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.725+0000 7fc9e8efe640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc9c806be30 0x7fc9c806e2f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8034000 tx=0x7fc9d802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:16.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.725+0000 7fc9d5ffb640 1 -- 172.21.15.112:0/2841534617 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e00052e0 con 0x7fc9e414dc90 2024-04-25T01:33:16.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.726+0000 7fc9d5ffb640 1 -- 172.21.15.112:0/2841534617 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc9e0018070 con 0x7fc9e414dc90 2024-04-25T01:33:16.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.726+0000 7fc9d5ffb640 1 -- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e414dc90 msgr2=0x7fc9e414e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.726+0000 7fc9d5ffb640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e414dc90 0x7fc9e414e3f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0000f60 tx=0x7fc9e000e960 comp rx=0 tx=0).stop 2024-04-25T01:33:16.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.726+0000 7fc9d5ffb640 1 -- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e414dc90 msgr2=0x7fc9e414e3f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:33:16.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.726+0000 7fc9d5ffb640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e414dc90 0x7fc9e414e3f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.735+0000 7fc9eb963640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e414e930 0x7fc9e414ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.735+0000 7fc9eb963640 1 -- 172.21.15.112:0/2841534617 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e406d890 con 0x7fc9e414e930 2024-04-25T01:33:16.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.736+0000 7fc9e96ff640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e414e930 0x7fc9e414ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.736+0000 7fc9e96ff640 1 -- 172.21.15.112:0/2841534617 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc9e40682a0 con 0x7fc9e414e930 2024-04-25T01:33:16.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.736+0000 7fc9e96ff640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e414e930 0x7fc9e414ed40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0000f60 tx=0x7fc9e0001080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:16.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.736+0000 7fc9d5ffb640 1 -- 172.21.15.112:0/2841534617 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0003b40 con 0x7fc9e414e930 2024-04-25T01:33:16.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.737+0000 7fc9d5ffb640 1 -- 172.21.15.112:0/2841534617 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5070 (secure 0 0 0) 0x7fc9e0003b40 con 0x7fc9e414e930 2024-04-25T01:33:16.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.737+0000 7fc9d5ffb640 1 -- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e414e930 msgr2=0x7fc9e414ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.737+0000 7fc9d5ffb640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e414e930 0x7fc9e414ed40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0000f60 tx=0x7fc9e0001080 comp rx=0 tx=0).stop 2024-04-25T01:33:16.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.737+0000 7fc9d5ffb640 1 -- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e414e930 msgr2=0x7fc9e414ed40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:33:16.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.737+0000 7fc9d5ffb640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e414e930 0x7fc9e414ed40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.715 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:33:16.715 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:33:16.715 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:33:16.715 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:33:16.715 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 56, 2024-04-25T01:33:16.715 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 232425, 2024-04-25T01:33:16.715 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:33:16.715 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:33:16.715 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:33:16.715 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:33:16.716 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:16.716 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 14, 2024-04-25T01:33:16.716 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:33:16.716 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:33:16.716 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:33:16.716 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:16.716 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:16.716 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:16.716 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:16.716 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:16.716 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:16.716 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:16.716 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:16.716 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:16.717 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:16.717 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:16.717 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:16.717 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:33:16.717 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:33:16.717 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:16.717 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:16.717 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:16.717 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:16.717 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:16.717 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:16.717 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:16.717 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:16.717 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:16.717 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:16.717 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:16.718 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:16.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.718 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:33:16.718 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:33:16.718 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:33:16.718 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:33:16.718 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:33:16.718 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:33:16.718 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:33:16.718 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:33:16.718 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:33:16.718 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:33:16.718 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:33:16.718 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:33:16.719 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:33:16.719 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:33:16.719 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:33:16.719 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:33:16.719 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:33:16.719 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:16.719 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:16.719 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:16.719 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:16.719 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:16.719 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:16.719 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:16.719 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:16.719 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:16.719 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:16.720 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:16.720 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:16.720 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:33:16.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.720 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:33:16.720 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.720 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:33:16.720 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:33:16.720 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:16.720 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:16.720 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.720 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:16.720 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:33:16.720 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:16.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.721 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:16.721 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:33:16.721 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:16.721 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.721 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:16.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.721 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:33:16.721 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:33:16.721 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:16.721 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:16.721 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:16.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.721 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:33:16.722 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:33:16.722 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:16.722 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:16.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.722 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:16.722 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:33:16.722 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:16.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.723 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.723 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:16.723 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:33:16.723 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:16.723 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.723 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:16.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.723 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:33:16.723 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:33:16.723 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:16.723 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:16.723 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:16.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.723 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.724 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:33:16.724 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:33:16.724 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:16.724 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:16.724 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.724 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:16.724 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:33:16.724 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:16.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.724 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.724 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:16.724 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:33:16.724 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:16.724 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.725 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:16.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.725 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:33:16.725 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:33:16.725 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:16.725 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:16.725 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:16.725 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.725 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:16.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.725 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:33:16.725 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:33:16.725 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.725 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:16.725 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:16.726 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:33:16.726 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.726 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:16.726 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:33:16.726 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.726 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:16.726 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:16.726 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:33:16.726 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.726 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:16.726 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:33:16.726 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.726 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:16.726 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:16.726 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:33:16.727 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.727 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:16.727 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:33:16.727 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.727 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:16.727 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:16.727 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:33:16.727 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.727 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:16.727 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:33:16.728 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.728 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:16.728 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:16.728 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:33:16.728 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.728 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:16.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.728 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:33:16.728 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:33:16.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.737+0000 7fc9eb963640 1 -- 172.21.15.112:0/2841534617 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc9c806be30 msgr2=0x7fc9c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.737+0000 7fc9eb963640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc9c806be30 0x7fc9c806e2f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8034000 tx=0x7fc9d802e000 comp rx=0 tx=0).stop 2024-04-25T01:33:16.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.737+0000 7fc9eb963640 1 -- 172.21.15.112:0/2841534617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9e41596a0 msgr2=0x7fc9e414d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.737+0000 7fc9eb963640 1 --2- 172.21.15.112:0/2841534617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9e41596a0 0x7fc9e414d750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00cae0 tx=0x7fc9dc00a060 comp rx=0 tx=0).stop 2024-04-25T01:33:16.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.737+0000 7fc9e9f00640 1 -- 172.21.15.112:0/2841534617 reap_dead start 2024-04-25T01:33:16.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.737+0000 7fc9e9f00640 1 -- 172.21.15.112:0/2841534617 reap_dead start 2024-04-25T01:33:16.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.737+0000 7fc9eb963640 1 -- 172.21.15.112:0/2841534617 shutdown_connections 2024-04-25T01:33:16.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.737+0000 7fc9eb963640 1 -- 172.21.15.112:0/2841534617 >> 172.21.15.112:0/2841534617 conn(0x7fc9e40cb170 msgr2=0x7fc9e40cb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:16.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.737+0000 7fc9eb963640 1 -- 172.21.15.112:0/2841534617 shutdown_connections 2024-04-25T01:33:16.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.737+0000 7fc9eb963640 1 -- 172.21.15.112:0/2841534617 wait complete. 2024-04-25T01:33:16.731 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:33:16.731 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:33:16.731 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:33:16.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.872+0000 7ff61c9a9640 1 Processor -- start 2024-04-25T01:33:16.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.873+0000 7ff61c9a9640 1 -- start start 2024-04-25T01:33:16.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.874+0000 7ff61c9a9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff618074930 0x7ff61806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.874+0000 7ff61c9a9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6180c23b0 con 0x7ff618152f60 2024-04-25T01:33:16.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.874+0000 7ff61c9a9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6180c2690 con 0x7ff61806c6b0 2024-04-25T01:33:16.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.874+0000 7ff61c9a9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7ff618155a20 con 0x7ff618074930 2024-04-25T01:33:16.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.874+0000 7ff615d9b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff618074930 0x7ff61806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.874+0000 7ff61659c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff618152f60 0x7ff6180c6ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38864/0 (socket says 172.21.15.112:38864) 2024-04-25T01:33:16.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.874+0000 7ff61659c640 1 -- 172.21.15.112:0/631806624 learned_addr learned my addr 172.21.15.112:0/631806624 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:33:16.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.874+0000 7ff61559a640 1 -- 172.21.15.112:0/631806624 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2320158238 0 0) 0x7ff6180c2690 con 0x7ff61806c6b0 2024-04-25T01:33:16.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.874+0000 7ff615d9b640 1 -- 172.21.15.112:0/631806624 >> v1:172.21.15.187:6789/0 conn(0x7ff61806c6b0 legacy=0x7ff6181592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.874+0000 7ff615d9b640 1 -- 172.21.15.112:0/631806624 >> v1:172.21.15.112:6789/0 conn(0x7ff618152f60 legacy=0x7ff6180c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.874+0000 7ff615d9b640 1 -- 172.21.15.112:0/631806624 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6180c2970 con 0x7ff618074930 2024-04-25T01:33:16.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.875+0000 7ff615d9b640 1 --2- 172.21.15.112:0/631806624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618074930 0x7ff61806c170 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0097a0 tx=0x7ff60c02a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b30b612c940b809 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:16.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.875+0000 7ff61559a640 1 -- 172.21.15.112:0/631806624 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c037070 con 0x7ff618074930 2024-04-25T01:33:16.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.875+0000 7ff61559a640 1 -- 172.21.15.112:0/631806624 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff60c02ac90 con 0x7ff618074930 2024-04-25T01:33:16.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.875+0000 7ff61559a640 1 -- 172.21.15.112:0/631806624 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c032660 con 0x7ff618074930 2024-04-25T01:33:16.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.876+0000 7ff61c9a9640 1 -- 172.21.15.112:0/631806624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618074930 msgr2=0x7ff61806c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.876+0000 7ff61c9a9640 1 --2- 172.21.15.112:0/631806624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618074930 0x7ff61806c170 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0097a0 tx=0x7ff60c02a000 comp rx=0 tx=0).stop 2024-04-25T01:33:16.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.876+0000 7ff61c9a9640 1 -- 172.21.15.112:0/631806624 shutdown_connections 2024-04-25T01:33:16.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.876+0000 7ff61c9a9640 1 --2- 172.21.15.112:0/631806624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618074930 0x7ff61806c170 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.876+0000 7ff61c9a9640 1 -- 172.21.15.112:0/631806624 >> 172.21.15.112:0/631806624 conn(0x7ff6180c4260 msgr2=0x7ff6180c4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:16.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.876+0000 7ff61c9a9640 1 -- 172.21.15.112:0/631806624 shutdown_connections 2024-04-25T01:33:16.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.876+0000 7ff61c9a9640 1 -- 172.21.15.112:0/631806624 wait complete. 2024-04-25T01:33:16.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.878+0000 7ff61c9a9640 1 Processor -- start 2024-04-25T01:33:16.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.878+0000 7ff61c9a9640 1 -- start start 2024-04-25T01:33:16.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.878+0000 7ff61c9a9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff61806c6b0 0x7ff61809aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.878+0000 7ff61c9a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618074930 0x7ff618155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.878+0000 7ff61c9a9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff618152f60 0x7ff6181eafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.878+0000 7ff61c9a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff618153340 con 0x7ff618074930 2024-04-25T01:33:16.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.878+0000 7ff61c9a9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff6180c2160 con 0x7ff61806c6b0 2024-04-25T01:33:16.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.878+0000 7ff61c9a9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff6181534c0 con 0x7ff618152f60 2024-04-25T01:33:16.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.879+0000 7ff615d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618074930 0x7ff618155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.879+0000 7ff615d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618074930 0x7ff618155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56396/0 (socket says 172.21.15.112:56396) 2024-04-25T01:33:16.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.879+0000 7ff615d9b640 1 -- 172.21.15.112:0/2392300440 learned_addr learned my addr 172.21.15.112:0/2392300440 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:16.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.879+0000 7ff61659c640 1 --2- 172.21.15.112:0/2392300440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff61806c6b0 0x7ff61809aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.879+0000 7ff616d9d640 1 --2- 172.21.15.112:0/2392300440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff618152f60 0x7ff6181eafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.879+0000 7ff615d9b640 1 -- 172.21.15.112:0/2392300440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff618152f60 msgr2=0x7ff6181eafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.879+0000 7ff615d9b640 1 --2- 172.21.15.112:0/2392300440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff618152f60 0x7ff6181eafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.879+0000 7ff615d9b640 1 -- 172.21.15.112:0/2392300440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff61806c6b0 msgr2=0x7ff61809aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.879+0000 7ff615d9b640 1 --2- 172.21.15.112:0/2392300440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff61806c6b0 0x7ff61809aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.879+0000 7ff615d9b640 1 -- 172.21.15.112:0/2392300440 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff618158b40 con 0x7ff618074930 2024-04-25T01:33:16.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.879+0000 7ff615d9b640 1 --2- 172.21.15.112:0/2392300440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618074930 0x7ff618155e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff60c035040 tx=0x7ff60c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:16.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.879+0000 7ff606ffd640 1 -- 172.21.15.112:0/2392300440 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c037040 con 0x7ff618074930 2024-04-25T01:33:16.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.879+0000 7ff606ffd640 1 -- 172.21.15.112:0/2392300440 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff60c004030 con 0x7ff618074930 2024-04-25T01:33:16.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.879+0000 7ff61c9a9640 1 -- 172.21.15.112:0/2392300440 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff618158e20 con 0x7ff618074930 2024-04-25T01:33:16.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.879+0000 7ff606ffd640 1 -- 172.21.15.112:0/2392300440 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c004460 con 0x7ff618074930 2024-04-25T01:33:16.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.879+0000 7ff61c9a9640 1 -- 172.21.15.112:0/2392300440 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff61806da20 con 0x7ff618074930 2024-04-25T01:33:16.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.880+0000 7ff61c9a9640 1 --2- 172.21.15.112:0/2392300440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff61806df20 0x7ff6181eb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.880+0000 7ff61c9a9640 1 -- 172.21.15.112:0/2392300440 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff6181eba60 con 0x7ff61806df20 2024-04-25T01:33:16.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.881+0000 7ff61659c640 1 --2- 172.21.15.112:0/2392300440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff61806df20 0x7ff6181eb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.881+0000 7ff61659c640 1 -- 172.21.15.112:0/2392300440 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff618075980 con 0x7ff61806df20 2024-04-25T01:33:16.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.881+0000 7ff61659c640 1 --2- 172.21.15.112:0/2392300440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff61806df20 0x7ff6181eb520 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff608007fd0 tx=0x7ff60800fbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:16.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.881+0000 7ff606ffd640 1 -- 172.21.15.112:0/2392300440 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff60c03a070 con 0x7ff618074930 2024-04-25T01:33:16.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.882+0000 7ff606ffd640 1 --2- 172.21.15.112:0/2392300440 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff5f806be60 0x7ff5f806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.882+0000 7ff606ffd640 1 -- 172.21.15.112:0/2392300440 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7ff60c0ab6a0 con 0x7ff618074930 2024-04-25T01:33:16.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.882+0000 7ff616d9d640 1 --2- 172.21.15.112:0/2392300440 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff5f806be60 0x7ff5f806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.882+0000 7ff606ffd640 1 -- 172.21.15.112:0/2392300440 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608005850 con 0x7ff61806df20 2024-04-25T01:33:16.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.882+0000 7ff606ffd640 1 -- 172.21.15.112:0/2392300440 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff608018070 con 0x7ff61806df20 2024-04-25T01:33:16.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.882+0000 7ff606ffd640 1 -- 172.21.15.112:0/2392300440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff61806df20 msgr2=0x7ff6181eb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.882+0000 7ff606ffd640 1 --2- 172.21.15.112:0/2392300440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff61806df20 0x7ff6181eb520 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff608007fd0 tx=0x7ff60800fbe0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.882+0000 7ff606ffd640 1 -- 172.21.15.112:0/2392300440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff61806df20 msgr2=0x7ff6181eb520 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:33:16.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.882+0000 7ff606ffd640 1 --2- 172.21.15.112:0/2392300440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff61806df20 0x7ff6181eb520 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.882+0000 7ff616d9d640 1 --2- 172.21.15.112:0/2392300440 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff5f806be60 0x7ff5f806e320 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff6181569c0 tx=0x7ff610009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:16.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.888+0000 7ff61c9a9640 1 --2- 172.21.15.112:0/2392300440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff618156de0 0x7ff6181571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.889+0000 7ff61c9a9640 1 -- 172.21.15.112:0/2392300440 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff6180689b0 con 0x7ff618156de0 2024-04-25T01:33:16.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.889+0000 7ff61659c640 1 --2- 172.21.15.112:0/2392300440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff618156de0 0x7ff6181571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.889+0000 7ff61659c640 1 -- 172.21.15.112:0/2392300440 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff6180c5660 con 0x7ff618156de0 2024-04-25T01:33:16.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.889+0000 7ff61659c640 1 --2- 172.21.15.112:0/2392300440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff618156de0 0x7ff6181571f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff608007fd0 tx=0x7ff608000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:16.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.889+0000 7ff606ffd640 1 -- 172.21.15.112:0/2392300440 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608007720 con 0x7ff618156de0 2024-04-25T01:33:16.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.890+0000 7ff606ffd640 1 -- 172.21.15.112:0/2392300440 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4559 (secure 0 0 0) 0x7ff608007720 con 0x7ff618156de0 2024-04-25T01:33:16.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.890+0000 7ff606ffd640 1 -- 172.21.15.112:0/2392300440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff618156de0 msgr2=0x7ff6181571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.890+0000 7ff606ffd640 1 --2- 172.21.15.112:0/2392300440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff618156de0 0x7ff6181571f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff608007fd0 tx=0x7ff608000ed0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.890+0000 7ff606ffd640 1 -- 172.21.15.112:0/2392300440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff618156de0 msgr2=0x7ff6181571f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:33:16.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.890+0000 7ff606ffd640 1 --2- 172.21.15.112:0/2392300440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff618156de0 0x7ff6181571f0 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff608007fd0 tx=0x7ff608000ed0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.867 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:33:16.867 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:33:16.868 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:33:16.868 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:33:16.868 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 56, 2024-04-25T01:33:16.868 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 23589, 2024-04-25T01:33:16.868 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:33:16.868 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:33:16.868 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:33:16.868 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:33:16.868 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:16.868 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 14, 2024-04-25T01:33:16.868 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:33:16.868 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:33:16.868 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:33:16.868 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:16.868 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:16.869 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:16.869 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:16.869 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:16.869 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:16.869 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:16.869 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:16.869 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:16.869 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:16.869 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:16.869 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:16.869 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:33:16.869 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:33:16.869 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:16.869 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:16.869 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:16.870 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:16.870 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:16.870 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:16.870 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:16.870 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:16.870 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:16.870 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:16.870 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:16.870 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:16.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.870 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:33:16.871 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:33:16.871 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:33:16.871 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:33:16.871 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:33:16.871 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:33:16.871 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:33:16.871 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:33:16.871 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:33:16.871 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:33:16.871 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:33:16.871 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:33:16.871 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:33:16.872 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:33:16.872 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:33:16.872 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:33:16.872 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:33:16.872 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:16.872 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:16.872 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:16.872 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:16.872 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:16.872 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:16.872 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:16.872 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:16.872 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:16.872 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:16.872 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:16.873 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:16.873 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:33:16.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.873 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:33:16.873 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.873 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:33:16.873 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:33:16.874 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:16.874 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:16.874 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.874 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:16.874 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:33:16.874 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:16.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.874 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.874 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:16.874 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:33:16.874 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:16.874 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.874 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:16.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.875 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:33:16.875 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:33:16.875 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:16.875 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:16.875 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:16.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.875 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:33:16.875 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:33:16.875 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:16.875 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:16.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.875 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:16.875 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:33:16.876 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:16.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.876 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.876 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:16.876 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:33:16.876 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:16.876 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.876 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:16.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.876 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:33:16.876 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:33:16.876 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:16.876 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:16.876 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:16.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.877 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.877 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:33:16.877 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:33:16.877 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:16.877 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:16.877 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.877 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:16.877 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:33:16.877 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:16.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.877 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.877 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:16.877 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:33:16.877 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:16.877 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.878 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:16.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.878 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:33:16.878 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:33:16.878 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:16.878 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:16.878 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:16.878 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.878 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:16.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.878 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:33:16.878 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:33:16.878 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.878 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:16.878 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:33:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:33:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:33:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:33:16.879 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:33:16.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.890+0000 7ff61c9a9640 1 -- 172.21.15.112:0/2392300440 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff5f806be60 msgr2=0x7ff5f806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.890+0000 7ff61c9a9640 1 --2- 172.21.15.112:0/2392300440 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff5f806be60 0x7ff5f806e320 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff6181569c0 tx=0x7ff610009040 comp rx=0 tx=0).stop 2024-04-25T01:33:16.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.890+0000 7ff61c9a9640 1 -- 172.21.15.112:0/2392300440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618074930 msgr2=0x7ff618155e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.890+0000 7ff61c9a9640 1 --2- 172.21.15.112:0/2392300440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618074930 0x7ff618155e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff60c035040 tx=0x7ff60c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.890+0000 7ff616d9d640 1 -- 172.21.15.112:0/2392300440 reap_dead start 2024-04-25T01:33:16.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.890+0000 7ff616d9d640 1 -- 172.21.15.112:0/2392300440 reap_dead start 2024-04-25T01:33:16.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.890+0000 7ff61c9a9640 1 -- 172.21.15.112:0/2392300440 shutdown_connections 2024-04-25T01:33:16.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.890+0000 7ff61c9a9640 1 -- 172.21.15.112:0/2392300440 >> 172.21.15.112:0/2392300440 conn(0x7ff6180c4260 msgr2=0x7ff6181592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:16.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.890+0000 7ff61c9a9640 1 -- 172.21.15.112:0/2392300440 shutdown_connections 2024-04-25T01:33:16.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:16.890+0000 7ff61c9a9640 1 -- 172.21.15.112:0/2392300440 wait complete. 2024-04-25T01:33:16.882 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:33:16.882 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-04-25T01:33:16.882 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-25T01:33:16.882 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-04-25T01:33:16.882 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-25T01:33:16.882 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T01:33:16.883 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:33:16.893 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T01:33:16.893 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T01:33:16.894 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:33:17.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.023+0000 7fc489b4f640 1 Processor -- start 2024-04-25T01:33:17.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.023+0000 7fc489b4f640 1 -- start start 2024-04-25T01:33:17.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.024+0000 7fc489b4f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc4840c5f40 0x7fc4841535e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:17.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.024+0000 7fc489b4f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc4840c3980 0x7fc484153b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:17.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.024+0000 7fc489b4f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc48406f010 0x7fc48406f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:17.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.024+0000 7fc489b4f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fc484071090 con 0x7fc4840c5f40 2024-04-25T01:33:17.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.024+0000 7fc489b4f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc484154060 con 0x7fc4840c3980 2024-04-25T01:33:17.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.024+0000 7fc489b4f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc4840aeac0 con 0x7fc48406f010 2024-04-25T01:33:17.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.024+0000 7fc483fff640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc48406f010 msgr2=0x7fc48406f460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:33:17.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.024+0000 7fc483fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc48406f010 0x7fc48406f460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:17.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.024+0000 7fc4837fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc4840c5f40 0x7fc4841535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:17.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.024+0000 7fc4837fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc4840c5f40 0x7fc4841535e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56400/0 (socket says 172.21.15.112:56400) 2024-04-25T01:33:17.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.024+0000 7fc4837fe640 1 -- 172.21.15.112:0/2825525846 learned_addr learned my addr 172.21.15.112:0/2825525846 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:17.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.024+0000 7fc482ffd640 1 --2- 172.21.15.112:0/2825525846 >> v2:172.21.15.187:3300/0 conn(0x7fc4840c3980 0x7fc484153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:17.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.024+0000 7fc4837fe640 1 -- 172.21.15.112:0/2825525846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc48406f010 msgr2=0x7fc48406f460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:33:17.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.024+0000 7fc4837fe640 1 --2- 172.21.15.112:0/2825525846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc48406f010 0x7fc48406f460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:17.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.024+0000 7fc4837fe640 1 -- 172.21.15.112:0/2825525846 >> v2:172.21.15.187:3300/0 conn(0x7fc4840c3980 msgr2=0x7fc484153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:33:17.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.024+0000 7fc4837fe640 1 --2- 172.21.15.112:0/2825525846 >> v2:172.21.15.187:3300/0 conn(0x7fc4840c3980 0x7fc484153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:17.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.024+0000 7fc4837fe640 1 -- 172.21.15.112:0/2825525846 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4841578c0 con 0x7fc4840c5f40 2024-04-25T01:33:17.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.025+0000 7fc4837fe640 1 --2- 172.21.15.112:0/2825525846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4840c5f40 0x7fc4841535e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc4840ae6a0 tx=0x7fc47402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd58e10656dac281 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:17.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.025+0000 7fc4827fc640 1 -- 172.21.15.112:0/2825525846 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474037070 con 0x7fc4840c5f40 2024-04-25T01:33:17.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.025+0000 7fc4827fc640 1 -- 172.21.15.112:0/2825525846 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc474032440 con 0x7fc4840c5f40 2024-04-25T01:33:17.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.025+0000 7fc4827fc640 1 -- 172.21.15.112:0/2825525846 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474032810 con 0x7fc4840c5f40 2024-04-25T01:33:17.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.026+0000 7fc489b4f640 1 -- 172.21.15.112:0/2825525846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4840c5f40 msgr2=0x7fc4841535e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:17.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.026+0000 7fc489b4f640 1 --2- 172.21.15.112:0/2825525846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4840c5f40 0x7fc4841535e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc4840ae6a0 tx=0x7fc47402a200 comp rx=0 tx=0).stop 2024-04-25T01:33:17.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.026+0000 7fc489b4f640 1 -- 172.21.15.112:0/2825525846 shutdown_connections 2024-04-25T01:33:17.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.026+0000 7fc489b4f640 1 --2- 172.21.15.112:0/2825525846 >> v2:172.21.15.187:3300/0 conn(0x7fc4840c3980 0x7fc484153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:17.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.026+0000 7fc489b4f640 1 --2- 172.21.15.112:0/2825525846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc48406f010 0x7fc48406f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:17.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.026+0000 7fc489b4f640 1 --2- 172.21.15.112:0/2825525846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4840c5f40 0x7fc4841535e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:17.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.026+0000 7fc489b4f640 1 -- 172.21.15.112:0/2825525846 >> 172.21.15.112:0/2825525846 conn(0x7fc4840c8f80 msgr2=0x7fc4840c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:17.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.026+0000 7fc489b4f640 1 -- 172.21.15.112:0/2825525846 shutdown_connections 2024-04-25T01:33:17.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.026+0000 7fc489b4f640 1 -- 172.21.15.112:0/2825525846 wait complete. 2024-04-25T01:33:17.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.027+0000 7fc489b4f640 1 Processor -- start 2024-04-25T01:33:17.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.027+0000 7fc489b4f640 1 -- start start 2024-04-25T01:33:17.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.028+0000 7fc489b4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc48406f010 0x7fc484156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:17.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.028+0000 7fc489b4f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4840c3980 0x7fc4841ec200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:17.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.028+0000 7fc489b4f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4840c5f40 0x7fc4841ec740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:17.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.028+0000 7fc489b4f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc4840717e0 con 0x7fc48406f010 2024-04-25T01:33:17.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.028+0000 7fc489b4f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc484157590 con 0x7fc4840c5f40 2024-04-25T01:33:17.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.028+0000 7fc489b4f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc4840aef30 con 0x7fc4840c3980 2024-04-25T01:33:17.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.028+0000 7fc482ffd640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4840c3980 msgr2=0x7fc4841ec200 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:33:17.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.028+0000 7fc482ffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4840c3980 0x7fc4841ec200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:17.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.028+0000 7fc4837fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc48406f010 0x7fc484156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:17.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.028+0000 7fc4837fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc48406f010 0x7fc484156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56416/0 (socket says 172.21.15.112:56416) 2024-04-25T01:33:17.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.028+0000 7fc4837fe640 1 -- 172.21.15.112:0/2035785695 learned_addr learned my addr 172.21.15.112:0/2035785695 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:17.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.028+0000 7fc483fff640 1 --2- 172.21.15.112:0/2035785695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4840c5f40 0x7fc4841ec740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:17.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.028+0000 7fc4837fe640 1 -- 172.21.15.112:0/2035785695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4840c3980 msgr2=0x7fc4841ec200 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:33:17.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.029+0000 7fc4837fe640 1 --2- 172.21.15.112:0/2035785695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4840c3980 0x7fc4841ec200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:17.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.029+0000 7fc4837fe640 1 -- 172.21.15.112:0/2035785695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4840c5f40 msgr2=0x7fc4841ec740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:17.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.029+0000 7fc4837fe640 1 --2- 172.21.15.112:0/2035785695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4840c5f40 0x7fc4841ec740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:17.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.029+0000 7fc4837fe640 1 -- 172.21.15.112:0/2035785695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4841df6f0 con 0x7fc48406f010 2024-04-25T01:33:17.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.029+0000 7fc4837fe640 1 --2- 172.21.15.112:0/2035785695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc48406f010 0x7fc484156160 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc4740090b0 tx=0x7fc4740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:17.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.029+0000 7fc46bfff640 1 -- 172.21.15.112:0/2035785695 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474041020 con 0x7fc48406f010 2024-04-25T01:33:17.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.029+0000 7fc46bfff640 1 -- 172.21.15.112:0/2035785695 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc47403d620 con 0x7fc48406f010 2024-04-25T01:33:17.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.029+0000 7fc489b4f640 1 -- 172.21.15.112:0/2035785695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4841df990 con 0x7fc48406f010 2024-04-25T01:33:17.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.029+0000 7fc46bfff640 1 -- 172.21.15.112:0/2035785695 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474037040 con 0x7fc48406f010 2024-04-25T01:33:17.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.029+0000 7fc489b4f640 1 -- 172.21.15.112:0/2035785695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4841dd630 con 0x7fc48406f010 2024-04-25T01:33:17.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.030+0000 7fc489b4f640 1 -- 172.21.15.112:0/2035785695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4840682a0 con 0x7fc48406f010 2024-04-25T01:33:17.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.030+0000 7fc46bfff640 1 -- 172.21.15.112:0/2035785695 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc474044020 con 0x7fc48406f010 2024-04-25T01:33:17.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.031+0000 7fc46bfff640 1 --2- 172.21.15.112:0/2035785695 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc47006bde0 0x7fc47006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:17.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.031+0000 7fc46bfff640 1 -- 172.21.15.112:0/2035785695 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fc4740ab8b0 con 0x7fc48406f010 2024-04-25T01:33:17.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.031+0000 7fc482ffd640 1 --2- 172.21.15.112:0/2035785695 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc47006bde0 0x7fc47006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:17.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.031+0000 7fc482ffd640 1 --2- 172.21.15.112:0/2035785695 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc47006bde0 0x7fc47006e2a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc4841569e0 tx=0x7fc478009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:17.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.037+0000 7fc46bfff640 1 -- 172.21.15.112:0/2035785695 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc47404b030 con 0x7fc48406f010 2024-04-25T01:33:17.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.336+0000 7fc489b4f640 1 -- 172.21.15.112:0/2035785695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc4841dd100 con 0x7fc48406f010 2024-04-25T01:33:17.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.337+0000 7fc46bfff640 1 -- 172.21.15.112:0/2035785695 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7fc474079ed0 con 0x7fc48406f010 2024-04-25T01:33:17.314 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":56,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:33:17.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.339+0000 7fc489b4f640 1 -- 172.21.15.112:0/2035785695 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc47006bde0 msgr2=0x7fc47006e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:17.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.339+0000 7fc489b4f640 1 --2- 172.21.15.112:0/2035785695 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc47006bde0 0x7fc47006e2a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc4841569e0 tx=0x7fc478009190 comp rx=0 tx=0).stop 2024-04-25T01:33:17.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.339+0000 7fc489b4f640 1 -- 172.21.15.112:0/2035785695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc48406f010 msgr2=0x7fc484156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:17.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.339+0000 7fc489b4f640 1 --2- 172.21.15.112:0/2035785695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc48406f010 0x7fc484156160 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc4740090b0 tx=0x7fc4740046c0 comp rx=0 tx=0).stop 2024-04-25T01:33:17.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.339+0000 7fc489b4f640 1 -- 172.21.15.112:0/2035785695 shutdown_connections 2024-04-25T01:33:17.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.339+0000 7fc489b4f640 1 --2- 172.21.15.112:0/2035785695 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc47006bde0 0x7fc47006e2a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:17.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.339+0000 7fc489b4f640 1 --2- 172.21.15.112:0/2035785695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4840c5f40 0x7fc4841ec740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:17.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.339+0000 7fc489b4f640 1 --2- 172.21.15.112:0/2035785695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4840c3980 0x7fc4841ec200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:17.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.339+0000 7fc489b4f640 1 --2- 172.21.15.112:0/2035785695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc48406f010 0x7fc484156160 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:17.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.339+0000 7fc489b4f640 1 -- 172.21.15.112:0/2035785695 >> 172.21.15.112:0/2035785695 conn(0x7fc4840c8f80 msgr2=0x7fc4840c6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:17.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.339+0000 7fc489b4f640 1 -- 172.21.15.112:0/2035785695 shutdown_connections 2024-04-25T01:33:17.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:17.339+0000 7fc489b4f640 1 -- 172.21.15.112:0/2035785695 wait complete. 2024-04-25T01:33:20.328 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:33:20.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.465+0000 7f424fdd5640 1 Processor -- start 2024-04-25T01:33:20.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.465+0000 7f424fdd5640 1 -- start start 2024-04-25T01:33:20.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424fdd5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42480c77f0 0x7f42480c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:20.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424fdd5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f424806c970 con 0x7f42480c6d00 2024-04-25T01:33:20.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424fdd5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f424806cc50 con 0x7f42481565e0 2024-04-25T01:33:20.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424fdd5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f42480c6b60 con 0x7f42480c77f0 2024-04-25T01:33:20.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424d370640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42480c77f0 msgr2=0x7f42480c9760 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:33:20.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424d370640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42480c77f0 0x7f42480c9760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:20.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424db71640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f42480c6d00 0x7f42480c9050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58268/0 (socket says 172.21.15.112:58268) 2024-04-25T01:33:20.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424db71640 1 -- 172.21.15.112:0/240564368 learned_addr learned my addr 172.21.15.112:0/240564368 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:33:20.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424cb6f640 1 -- 172.21.15.112:0/240564368 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1872694407 0 0) 0x7f424806c970 con 0x7f42480c6d00 2024-04-25T01:33:20.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424cb6f640 1 -- 172.21.15.112:0/240564368 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4234003520 con 0x7f42480c6d00 2024-04-25T01:33:20.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424cb6f640 1 -- 172.21.15.112:0/240564368 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2595405067 0 0) 0x7f4234003520 con 0x7f42480c6d00 2024-04-25T01:33:20.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424cb6f640 1 -- 172.21.15.112:0/240564368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42480c77f0 msgr2=0x7f42480c9760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:33:20.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424cb6f640 1 --2- 172.21.15.112:0/240564368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42480c77f0 0x7f42480c9760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:20.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424cb6f640 1 -- 172.21.15.112:0/240564368 >> v1:172.21.15.187:6789/0 conn(0x7f42481565e0 legacy=0x7f4248160400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:20.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424cb6f640 1 -- 172.21.15.112:0/240564368 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42480c6640 con 0x7f42480c6d00 2024-04-25T01:33:20.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424cb6f640 1 -- 172.21.15.112:0/240564368 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f4244001670 con 0x7f42480c6d00 2024-04-25T01:33:20.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424cb6f640 1 -- 172.21.15.112:0/240564368 >> v1:172.21.15.112:6789/0 conn(0x7f42480c6d00 legacy=0x7f42480c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:20.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424cb6f640 1 --2- 172.21.15.112:0/240564368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42480c77f0 0x7f4234003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:20.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424cb6f640 1 --2- 172.21.15.112:0/240564368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4234004110 0x7f4234004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:20.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424cb6f640 1 --2- 172.21.15.112:0/240564368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4234004e60 0x7f423400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:20.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424cb6f640 1 -- 172.21.15.112:0/240564368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f42480c6b60 con 0x7f4234004e60 2024-04-25T01:33:20.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424cb6f640 1 -- 172.21.15.112:0/240564368 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4234005650 con 0x7f4234004110 2024-04-25T01:33:20.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424cb6f640 1 -- 172.21.15.112:0/240564368 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f423400bf50 con 0x7f42480c77f0 2024-04-25T01:33:20.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.466+0000 7f424db71640 1 -- 172.21.15.112:0/240564368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42480c77f0 msgr2=0x7f4234003a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:33:20.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.467+0000 7f424db71640 1 --2- 172.21.15.112:0/240564368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42480c77f0 0x7f4234003a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:20.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.467+0000 7f424e372640 1 --2- 172.21.15.112:0/240564368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4234004e60 0x7f423400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:20.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.467+0000 7f424d370640 1 --2- 172.21.15.112:0/240564368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4234004110 0x7f4234004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:20.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.467+0000 7f424e372640 1 -- 172.21.15.112:0/240564368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42480c77f0 msgr2=0x7f4234003a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:33:20.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.467+0000 7f424e372640 1 --2- 172.21.15.112:0/240564368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42480c77f0 0x7f4234003a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:20.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.467+0000 7f424e372640 1 -- 172.21.15.112:0/240564368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4234004110 msgr2=0x7f4234004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:33:20.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.467+0000 7f424e372640 1 --2- 172.21.15.112:0/240564368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4234004110 0x7f4234004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:20.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.467+0000 7f424e372640 1 -- 172.21.15.112:0/240564368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4234003520 con 0x7f4234004e60 2024-04-25T01:33:20.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.468+0000 7f424e372640 1 --2- 172.21.15.112:0/240564368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4234004e60 0x7f423400b9c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4240001e80 tx=0x7f4234005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ae3d5899c6d7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:20.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.468+0000 7f424cb6f640 1 -- 172.21.15.112:0/240564368 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240001a70 con 0x7f4234004e60 2024-04-25T01:33:20.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.468+0000 7f424cb6f640 1 -- 172.21.15.112:0/240564368 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f424002ae50 con 0x7f4234004e60 2024-04-25T01:33:20.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.468+0000 7f424cb6f640 1 -- 172.21.15.112:0/240564368 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240003bb0 con 0x7f4234004e60 2024-04-25T01:33:20.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.468+0000 7f424fdd5640 1 -- 172.21.15.112:0/240564368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4234004e60 msgr2=0x7f423400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:20.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.468+0000 7f424fdd5640 1 --2- 172.21.15.112:0/240564368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4234004e60 0x7f423400b9c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4240001e80 tx=0x7f4234005830 comp rx=0 tx=0).stop 2024-04-25T01:33:20.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.468+0000 7f424e372640 1 -- 172.21.15.112:0/240564368 reap_dead start 2024-04-25T01:33:20.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.468+0000 7f424fdd5640 1 -- 172.21.15.112:0/240564368 shutdown_connections 2024-04-25T01:33:20.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.468+0000 7f424fdd5640 1 -- 172.21.15.112:0/240564368 >> 172.21.15.112:0/240564368 conn(0x7f42480cb330 msgr2=0x7f42480cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:20.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.469+0000 7f424fdd5640 1 -- 172.21.15.112:0/240564368 shutdown_connections 2024-04-25T01:33:20.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.469+0000 7f424fdd5640 1 -- 172.21.15.112:0/240564368 wait complete. 2024-04-25T01:33:20.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.470+0000 7f424fdd5640 1 Processor -- start 2024-04-25T01:33:20.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.470+0000 7f424fdd5640 1 -- start start 2024-04-25T01:33:20.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.471+0000 7f424fdd5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42480c2490 0x7f42480add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:20.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.471+0000 7f424fdd5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4248158fa0 0x7f42480ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:20.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.471+0000 7f424fdd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4248151200 0x7f4248151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:20.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.471+0000 7f424fdd5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f42480c63f0 con 0x7f4248151200 2024-04-25T01:33:20.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.471+0000 7f424fdd5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4248153640 con 0x7f4248158fa0 2024-04-25T01:33:20.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.471+0000 7f424fdd5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f42480ca940 con 0x7f42480c2490 2024-04-25T01:33:20.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.471+0000 7f424db71640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42480c2490 msgr2=0x7f42480add50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:33:20.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.471+0000 7f424db71640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42480c2490 0x7f42480add50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:20.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.471+0000 7f424e372640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4248151200 0x7f4248151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:20.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.471+0000 7f424e372640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4248151200 0x7f4248151690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51320/0 (socket says 172.21.15.112:51320) 2024-04-25T01:33:20.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.471+0000 7f424e372640 1 -- 172.21.15.112:0/182907003 learned_addr learned my addr 172.21.15.112:0/182907003 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:20.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.471+0000 7f424d370640 1 --2- 172.21.15.112:0/182907003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4248158fa0 0x7f42480ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:20.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.471+0000 7f424e372640 1 -- 172.21.15.112:0/182907003 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42480c2490 msgr2=0x7f42480add50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:33:20.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.471+0000 7f424e372640 1 --2- 172.21.15.112:0/182907003 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42480c2490 0x7f42480add50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:20.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.471+0000 7f424e372640 1 -- 172.21.15.112:0/182907003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4248158fa0 msgr2=0x7f42480ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:20.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.471+0000 7f424e372640 1 --2- 172.21.15.112:0/182907003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4248158fa0 0x7f42480ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:20.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.471+0000 7f424e372640 1 -- 172.21.15.112:0/182907003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42481522a0 con 0x7f4248151200 2024-04-25T01:33:20.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.471+0000 7f424d370640 1 --2- 172.21.15.112:0/182907003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4248158fa0 0x7f42480ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:33:20.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.471+0000 7f424e372640 1 --2- 172.21.15.112:0/182907003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4248151200 0x7f4248151690 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4240001fa0 tx=0x7f42400014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:20.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.472+0000 7f423e7fc640 1 -- 172.21.15.112:0/182907003 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240042070 con 0x7f4248151200 2024-04-25T01:33:20.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.472+0000 7f423e7fc640 1 -- 172.21.15.112:0/182907003 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4240001250 con 0x7f4248151200 2024-04-25T01:33:20.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.472+0000 7f423e7fc640 1 -- 172.21.15.112:0/182907003 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240037040 con 0x7f4248151200 2024-04-25T01:33:20.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.472+0000 7f424fdd5640 1 -- 172.21.15.112:0/182907003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4248152570 con 0x7f4248151200 2024-04-25T01:33:20.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.472+0000 7f424fdd5640 1 -- 172.21.15.112:0/182907003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42481eeca0 con 0x7f4248151200 2024-04-25T01:33:20.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.473+0000 7f424fdd5640 1 -- 172.21.15.112:0/182907003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42480682a0 con 0x7f4248151200 2024-04-25T01:33:20.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.473+0000 7f423e7fc640 1 -- 172.21.15.112:0/182907003 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4240045070 con 0x7f4248151200 2024-04-25T01:33:20.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.474+0000 7f423e7fc640 1 --2- 172.21.15.112:0/182907003 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f422c06bde0 0x7f422c06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:20.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.474+0000 7f423e7fc640 1 -- 172.21.15.112:0/182907003 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f42400aa7b0 con 0x7f4248151200 2024-04-25T01:33:20.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.474+0000 7f424db71640 1 --2- 172.21.15.112:0/182907003 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f422c06bde0 0x7f422c06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:20.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.474+0000 7f424db71640 1 --2- 172.21.15.112:0/182907003 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f422c06bde0 0x7f422c06e2a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4244001a00 tx=0x7f42440014d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:20.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.479+0000 7f423e7fc640 1 -- 172.21.15.112:0/182907003 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f42400786e0 con 0x7f4248151200 2024-04-25T01:33:20.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.812+0000 7f424fdd5640 1 -- 172.21.15.112:0/182907003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f42480c5c80 con 0x7f4248151200 2024-04-25T01:33:20.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.812+0000 7f423e7fc640 1 -- 172.21.15.112:0/182907003 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f424007c9e0 con 0x7f4248151200 2024-04-25T01:33:20.789 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":56,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:33:20.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.814+0000 7f424fdd5640 1 -- 172.21.15.112:0/182907003 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f422c06bde0 msgr2=0x7f422c06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:20.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.814+0000 7f424fdd5640 1 --2- 172.21.15.112:0/182907003 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f422c06bde0 0x7f422c06e2a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4244001a00 tx=0x7f42440014d0 comp rx=0 tx=0).stop 2024-04-25T01:33:20.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.814+0000 7f424fdd5640 1 -- 172.21.15.112:0/182907003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4248151200 msgr2=0x7f4248151690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:20.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.814+0000 7f424fdd5640 1 --2- 172.21.15.112:0/182907003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4248151200 0x7f4248151690 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4240001fa0 tx=0x7f42400014a0 comp rx=0 tx=0).stop 2024-04-25T01:33:20.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.814+0000 7f424fdd5640 1 -- 172.21.15.112:0/182907003 shutdown_connections 2024-04-25T01:33:20.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.814+0000 7f424fdd5640 1 --2- 172.21.15.112:0/182907003 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f422c06bde0 0x7f422c06e2a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:20.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.814+0000 7f424fdd5640 1 --2- 172.21.15.112:0/182907003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4248151200 0x7f4248151690 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:20.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.814+0000 7f424fdd5640 1 --2- 172.21.15.112:0/182907003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4248158fa0 0x7f42480ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:20.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.814+0000 7f424fdd5640 1 --2- 172.21.15.112:0/182907003 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42480c2490 0x7f42480add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:20.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.814+0000 7f424fdd5640 1 -- 172.21.15.112:0/182907003 >> 172.21.15.112:0/182907003 conn(0x7f42480cb330 msgr2=0x7f4248154950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:20.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.815+0000 7f424fdd5640 1 -- 172.21.15.112:0/182907003 shutdown_connections 2024-04-25T01:33:20.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:20.815+0000 7f424fdd5640 1 -- 172.21.15.112:0/182907003 wait complete. 2024-04-25T01:33:23.804 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:33:23.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.934+0000 7facbf5f4640 1 Processor -- start 2024-04-25T01:33:23.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.934+0000 7facbf5f4640 1 -- start start 2024-04-25T01:33:23.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.935+0000 7facbf5f4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7facb80c5d70 0x7facb80c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:23.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.935+0000 7facbf5f4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7facb8156cf0 0x7facb806bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:23.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.935+0000 7facbf5f4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facb80c90e0 con 0x7facb8152fd0 2024-04-25T01:33:23.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.935+0000 7facbf5f4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7facb80ca9b0 con 0x7facb8156cf0 2024-04-25T01:33:23.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.935+0000 7facbf5f4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7facb80cac00 con 0x7facb80c5d70 2024-04-25T01:33:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.935+0000 7facbdb91640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7facb8156cf0 0x7facb806bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.935+0000 7facbd390640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7facb8152fd0 0x7facb8156280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58284/0 (socket says 172.21.15.112:58284) 2024-04-25T01:33:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.935+0000 7facbd390640 1 -- 172.21.15.112:0/1517645256 learned_addr learned my addr 172.21.15.112:0/1517645256 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:33:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.935+0000 7facbcb8f640 1 --2- 172.21.15.112:0/1517645256 >> v2:172.21.15.187:3300/0 conn(0x7facb80c5d70 0x7facb80c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.935+0000 7facbdb91640 1 -- 172.21.15.112:0/1517645256 >> v2:172.21.15.187:3300/0 conn(0x7facb80c5d70 msgr2=0x7facb80c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:33:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.935+0000 7facbdb91640 1 --2- 172.21.15.112:0/1517645256 >> v2:172.21.15.187:3300/0 conn(0x7facb80c5d70 0x7facb80c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.935+0000 7facbdb91640 1 -- 172.21.15.112:0/1517645256 >> v1:172.21.15.112:6789/0 conn(0x7facb8152fd0 legacy=0x7facb8156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.935+0000 7facbdb91640 1 -- 172.21.15.112:0/1517645256 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb80c9380 con 0x7facb8156cf0 2024-04-25T01:33:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.936+0000 7facbdb91640 1 --2- 172.21.15.112:0/1517645256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facb8156cf0 0x7facb806bda0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7facb80bec40 tx=0x7facb002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f9b8f36dd77de3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.936+0000 7facaffff640 1 -- 172.21.15.112:0/1517645256 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0007740 con 0x7facb8156cf0 2024-04-25T01:33:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.936+0000 7facaffff640 1 -- 172.21.15.112:0/1517645256 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facb00078e0 con 0x7facb8156cf0 2024-04-25T01:33:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.936+0000 7facaffff640 1 -- 172.21.15.112:0/1517645256 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0007cb0 con 0x7facb8156cf0 2024-04-25T01:33:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.937+0000 7facbf5f4640 1 -- 172.21.15.112:0/1517645256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facb8156cf0 msgr2=0x7facb806bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.937+0000 7facbf5f4640 1 --2- 172.21.15.112:0/1517645256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facb8156cf0 0x7facb806bda0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7facb80bec40 tx=0x7facb002a5f0 comp rx=0 tx=0).stop 2024-04-25T01:33:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.937+0000 7facbf5f4640 1 -- 172.21.15.112:0/1517645256 shutdown_connections 2024-04-25T01:33:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.937+0000 7facbf5f4640 1 --2- 172.21.15.112:0/1517645256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facb8156cf0 0x7facb806bda0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.937+0000 7facbf5f4640 1 --2- 172.21.15.112:0/1517645256 >> v2:172.21.15.187:3300/0 conn(0x7facb80c5d70 0x7facb80c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.937+0000 7facbf5f4640 1 -- 172.21.15.112:0/1517645256 >> 172.21.15.112:0/1517645256 conn(0x7facb80acc70 msgr2=0x7facb80ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.937+0000 7facbf5f4640 1 -- 172.21.15.112:0/1517645256 shutdown_connections 2024-04-25T01:33:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.937+0000 7facbf5f4640 1 -- 172.21.15.112:0/1517645256 wait complete. 2024-04-25T01:33:23.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.938+0000 7facbf5f4640 1 Processor -- start 2024-04-25T01:33:23.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.938+0000 7facbf5f4640 1 -- start start 2024-04-25T01:33:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.939+0000 7facbf5f4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7facb80c5d70 0x7facb814d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.939+0000 7facbf5f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facb8152fd0 0x7facb81ec5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.939+0000 7facbf5f4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facb8156cf0 0x7facb81ecaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.939+0000 7facbf5f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7facb8157100 con 0x7facb8152fd0 2024-04-25T01:33:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.939+0000 7facbf5f4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7facb80caee0 con 0x7facb8156cf0 2024-04-25T01:33:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.939+0000 7facbf5f4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7facb8157280 con 0x7facb80c5d70 2024-04-25T01:33:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.939+0000 7facbd390640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7facb80c5d70 msgr2=0x7facb814d710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:33:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.939+0000 7facbd390640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7facb80c5d70 0x7facb814d710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.939+0000 7facbcb8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facb8152fd0 0x7facb81ec5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.939+0000 7facbcb8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facb8152fd0 0x7facb81ec5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51342/0 (socket says 172.21.15.112:51342) 2024-04-25T01:33:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.939+0000 7facbcb8f640 1 -- 172.21.15.112:0/2895680770 learned_addr learned my addr 172.21.15.112:0/2895680770 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.939+0000 7facbdb91640 1 --2- 172.21.15.112:0/2895680770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facb8156cf0 0x7facb81ecaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.939+0000 7facbcb8f640 1 -- 172.21.15.112:0/2895680770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7facb80c5d70 msgr2=0x7facb814d710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:33:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.939+0000 7facbcb8f640 1 --2- 172.21.15.112:0/2895680770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7facb80c5d70 0x7facb814d710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.939+0000 7facbcb8f640 1 -- 172.21.15.112:0/2895680770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facb8156cf0 msgr2=0x7facb81ecaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.939+0000 7facbcb8f640 1 --2- 172.21.15.112:0/2895680770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facb8156cf0 0x7facb81ecaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.939+0000 7facbcb8f640 1 -- 172.21.15.112:0/2895680770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb814dd20 con 0x7facb8152fd0 2024-04-25T01:33:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.939+0000 7facbdb91640 1 --2- 172.21.15.112:0/2895680770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facb8156cf0 0x7facb81ecaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:33:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.940+0000 7facbcb8f640 1 --2- 172.21.15.112:0/2895680770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facb8152fd0 0x7facb81ec5b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faca800e9b0 tx=0x7faca800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.940+0000 7facadffb640 1 -- 172.21.15.112:0/2895680770 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca8017070 con 0x7facb8152fd0 2024-04-25T01:33:23.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.940+0000 7facadffb640 1 -- 172.21.15.112:0/2895680770 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faca800c9a0 con 0x7facb8152fd0 2024-04-25T01:33:23.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.940+0000 7facadffb640 1 -- 172.21.15.112:0/2895680770 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca800cd70 con 0x7facb8152fd0 2024-04-25T01:33:23.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.940+0000 7facbf5f4640 1 -- 172.21.15.112:0/2895680770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb814df90 con 0x7facb8152fd0 2024-04-25T01:33:23.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.940+0000 7facbf5f4640 1 -- 172.21.15.112:0/2895680770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb814e550 con 0x7facb8152fd0 2024-04-25T01:33:23.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.941+0000 7facbf5f4640 1 -- 172.21.15.112:0/2895680770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb80682a0 con 0x7facb8152fd0 2024-04-25T01:33:23.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.942+0000 7facadffb640 1 -- 172.21.15.112:0/2895680770 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faca801a070 con 0x7facb8152fd0 2024-04-25T01:33:23.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.942+0000 7facadffb640 1 --2- 172.21.15.112:0/2895680770 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fac9c06bde0 0x7fac9c06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:23.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.942+0000 7facadffb640 1 -- 172.21.15.112:0/2895680770 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7faca808b500 con 0x7facb8152fd0 2024-04-25T01:33:23.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.942+0000 7facbd390640 1 --2- 172.21.15.112:0/2895680770 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fac9c06bde0 0x7fac9c06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:23.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.943+0000 7facbd390640 1 --2- 172.21.15.112:0/2895680770 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fac9c06bde0 0x7fac9c06e2a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7facb4000f60 tx=0x7facb4007ae0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:23.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:23.948+0000 7facadffb640 1 -- 172.21.15.112:0/2895680770 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faca8059b20 con 0x7facb8152fd0 2024-04-25T01:33:24.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:24.233+0000 7facbf5f4640 1 -- 172.21.15.112:0/2895680770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7facb80c1f80 con 0x7facb8152fd0 2024-04-25T01:33:24.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:24.233+0000 7facadffb640 1 -- 172.21.15.112:0/2895680770 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7faca8059420 con 0x7facb8152fd0 2024-04-25T01:33:24.211 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":56,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:33:24.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:24.236+0000 7facbf5f4640 1 -- 172.21.15.112:0/2895680770 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fac9c06bde0 msgr2=0x7fac9c06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:24.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:24.236+0000 7facbf5f4640 1 --2- 172.21.15.112:0/2895680770 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fac9c06bde0 0x7fac9c06e2a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7facb4000f60 tx=0x7facb4007ae0 comp rx=0 tx=0).stop 2024-04-25T01:33:24.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:24.236+0000 7facbf5f4640 1 -- 172.21.15.112:0/2895680770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facb8152fd0 msgr2=0x7facb81ec5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:24.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:24.236+0000 7facbf5f4640 1 --2- 172.21.15.112:0/2895680770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facb8152fd0 0x7facb81ec5b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faca800e9b0 tx=0x7faca800ee70 comp rx=0 tx=0).stop 2024-04-25T01:33:24.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:24.236+0000 7facbf5f4640 1 -- 172.21.15.112:0/2895680770 shutdown_connections 2024-04-25T01:33:24.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:24.236+0000 7facbf5f4640 1 --2- 172.21.15.112:0/2895680770 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fac9c06bde0 0x7fac9c06e2a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:24.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:24.236+0000 7facbf5f4640 1 --2- 172.21.15.112:0/2895680770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facb8156cf0 0x7facb81ecaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:24.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:24.236+0000 7facbf5f4640 1 --2- 172.21.15.112:0/2895680770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facb8152fd0 0x7facb81ec5b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:24.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:24.236+0000 7facbf5f4640 1 --2- 172.21.15.112:0/2895680770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7facb80c5d70 0x7facb814d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:24.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:24.236+0000 7facbf5f4640 1 -- 172.21.15.112:0/2895680770 >> 172.21.15.112:0/2895680770 conn(0x7facb80acc70 msgr2=0x7facb8157ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:24.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:24.236+0000 7facbf5f4640 1 -- 172.21.15.112:0/2895680770 shutdown_connections 2024-04-25T01:33:24.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:24.236+0000 7facbf5f4640 1 -- 172.21.15.112:0/2895680770 wait complete. 2024-04-25T01:33:26.377 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:33:26.379 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:33:27.227 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:33:27.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.357+0000 7fa67d996640 1 Processor -- start 2024-04-25T01:33:27.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.357+0000 7fa67d996640 1 -- start start 2024-04-25T01:33:27.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.358+0000 7fa67d996640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa678153160 0x7fa6780c3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:27.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.358+0000 7fa67d996640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 0x7fa67806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:27.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.358+0000 7fa67d996640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 0x7fa678158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:27.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.358+0000 7fa67d996640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa678074740 con 0x7fa67806c6b0 2024-04-25T01:33:27.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.358+0000 7fa67d996640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa6780c21d0 con 0x7fa6780a6d00 2024-04-25T01:33:27.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.358+0000 7fa67d996640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa6780c24b0 con 0x7fa678153160 2024-04-25T01:33:27.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.358+0000 7fa676ffd640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa678153160 msgr2=0x7fa6780c3c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:33:27.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.358+0000 7fa676ffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa678153160 0x7fa6780c3c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:27.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.358+0000 7fa6777fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 0x7fa678158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:27.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.358+0000 7fa6777fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 0x7fa678158ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51348/0 (socket says 172.21.15.112:51348) 2024-04-25T01:33:27.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.358+0000 7fa6777fe640 1 -- 172.21.15.112:0/2294242419 learned_addr learned my addr 172.21.15.112:0/2294242419 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:27.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.358+0000 7fa6767fc640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 0x7fa67806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:27.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.358+0000 7fa6777fe640 1 -- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 msgr2=0x7fa678158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:33:27.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.358+0000 7fa6777fe640 1 -- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 msgr2=0x7fa678158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:27.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.358+0000 7fa6777fe640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 0x7fa678158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:27.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.358+0000 7fa6777fe640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 0x7fa678158ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:27.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.358+0000 7fa6767fc640 1 -- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 msgr2=0x7fa67806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:33:27.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.358+0000 7fa6767fc640 1 -- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 msgr2=0x7fa67806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:27.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.358+0000 7fa6767fc640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 0x7fa67806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:27.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.358+0000 7fa6767fc640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 0x7fa67806c170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:27.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.559+0000 7fa676ffd640 1 -- 172.21.15.112:0/2294242419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa678153160 msgr2=0x7fa6780c3c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:33:27.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.559+0000 7fa676ffd640 1 --2- 172.21.15.112:0/2294242419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa678153160 0x7fa6780c3c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:33:27.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.559+0000 7fa6777fe640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 0x7fa678158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:27.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.559+0000 7fa6767fc640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 0x7fa67806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:27.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.559+0000 7fa6777fe640 1 -- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 msgr2=0x7fa678158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:33:27.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.559+0000 7fa6777fe640 1 -- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 msgr2=0x7fa678158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:27.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.559+0000 7fa6777fe640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 0x7fa678158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:27.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.559+0000 7fa6777fe640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 0x7fa678158ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:33:27.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.559+0000 7fa6767fc640 1 -- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 msgr2=0x7fa67806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:33:27.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.559+0000 7fa6767fc640 1 -- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 msgr2=0x7fa67806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:27.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.559+0000 7fa6767fc640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 0x7fa67806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:27.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.559+0000 7fa6767fc640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 0x7fa67806c170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:33:27.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.960+0000 7fa676ffd640 1 -- 172.21.15.112:0/2294242419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa678153160 msgr2=0x7fa6780c3c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:33:27.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.960+0000 7fa676ffd640 1 --2- 172.21.15.112:0/2294242419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa678153160 0x7fa6780c3c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:33:27.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.960+0000 7fa6777fe640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 0x7fa678158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:27.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.960+0000 7fa6767fc640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 0x7fa67806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:27.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.960+0000 7fa6777fe640 1 -- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 msgr2=0x7fa678158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:33:27.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.960+0000 7fa6777fe640 1 -- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 msgr2=0x7fa678158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:27.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.960+0000 7fa6777fe640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 0x7fa678158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:27.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.960+0000 7fa6777fe640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 0x7fa678158ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:33:27.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.960+0000 7fa6767fc640 1 -- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 msgr2=0x7fa67806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:33:27.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.960+0000 7fa6767fc640 1 -- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 msgr2=0x7fa67806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:27.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.960+0000 7fa6767fc640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 0x7fa67806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:27.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:27.960+0000 7fa6767fc640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 0x7fa67806c170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:33:28.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:28.761+0000 7fa676ffd640 1 -- 172.21.15.112:0/2294242419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa678153160 msgr2=0x7fa6780c3c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:33:28.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:28.761+0000 7fa676ffd640 1 --2- 172.21.15.112:0/2294242419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa678153160 0x7fa6780c3c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:33:28.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:28.761+0000 7fa6777fe640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 0x7fa678158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:28.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:28.761+0000 7fa6767fc640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 0x7fa67806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:28.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:28.761+0000 7fa6777fe640 1 -- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 msgr2=0x7fa678158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:33:28.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:28.761+0000 7fa6777fe640 1 -- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 msgr2=0x7fa678158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:28.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:28.761+0000 7fa6777fe640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 0x7fa678158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:28.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:28.761+0000 7fa6777fe640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 0x7fa678158ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:33:28.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:28.761+0000 7fa6767fc640 1 -- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 msgr2=0x7fa67806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:33:28.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:28.761+0000 7fa6767fc640 1 -- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 msgr2=0x7fa67806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:28.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:28.761+0000 7fa6767fc640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 0x7fa67806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:28.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:28.761+0000 7fa6767fc640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 0x7fa67806c170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:33:30.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:30.358+0000 7fa674ff9640 1 -- 172.21.15.112:0/2294242419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa678153160 msgr2=0x7fa6780c3c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:33:30.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:30.358+0000 7fa674ff9640 1 --2- 172.21.15.112:0/2294242419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa678153160 0x7fa6780c3c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:30.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:30.358+0000 7fa674ff9640 1 -- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 msgr2=0x7fa67806c170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:33:30.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:30.358+0000 7fa674ff9640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.187:3300/0 conn(0x7fa6780a6d00 0x7fa67806c170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:30.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:30.358+0000 7fa674ff9640 1 -- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 msgr2=0x7fa678158ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:33:30.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:30.358+0000 7fa674ff9640 1 --2- 172.21.15.112:0/2294242419 >> v2:172.21.15.112:3300/0 conn(0x7fa67806c6b0 0x7fa678158ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:30.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:30.358+0000 7fa674ff9640 1 -- 172.21.15.112:0/2294242419 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa678153540 con 0x7fa664003690 2024-04-25T01:33:30.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:30.358+0000 7fa674ff9640 1 -- 172.21.15.112:0/2294242419 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa66400c0a0 con 0x7fa664007320 2024-04-25T01:33:31.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.428+0000 7fa675ffb640 1 -- 172.21.15.112:0/2294242419 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 64870593 0 0) 0x7fa66400c0a0 con 0x7fa664007320 2024-04-25T01:33:31.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.429+0000 7fa675ffb640 1 -- 172.21.15.112:0/2294242419 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa660003520 con 0x7fa664007320 2024-04-25T01:33:31.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.432+0000 7fa675ffb640 1 -- 172.21.15.112:0/2294242419 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2833768223 0 0) 0x7fa678153540 con 0x7fa664003690 2024-04-25T01:33:31.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.432+0000 7fa675ffb640 1 -- 172.21.15.112:0/2294242419 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa66400c0a0 con 0x7fa664003690 2024-04-25T01:33:31.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.469+0000 7fa675ffb640 1 -- 172.21.15.112:0/2294242419 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1321331804 0 0) 0x7fa66400c0a0 con 0x7fa664003690 2024-04-25T01:33:31.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.469+0000 7fa675ffb640 1 -- 172.21.15.112:0/2294242419 >> v1:172.21.15.187:6789/0 conn(0x7fa664007320 legacy=0x7fa6640097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:31.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.469+0000 7fa675ffb640 1 -- 172.21.15.112:0/2294242419 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66400d2e0 con 0x7fa664003690 2024-04-25T01:33:31.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.469+0000 7fa675ffb640 1 -- 172.21.15.112:0/2294242419 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fa668004300 con 0x7fa664003690 2024-04-25T01:33:31.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.469+0000 7fa675ffb640 1 -- 172.21.15.112:0/2294242419 >> v1:172.21.15.112:6789/0 conn(0x7fa664003690 legacy=0x7fa664005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:31.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.469+0000 7fa6777fe640 1 -- 172.21.15.112:0/2294242419 reap_dead start 2024-04-25T01:33:31.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.469+0000 7fa675ffb640 1 --2- 172.21.15.112:0/2294242419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa660003b70 0x7fa660003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.469+0000 7fa675ffb640 1 --2- 172.21.15.112:0/2294242419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa660004800 0x7fa660004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.469+0000 7fa675ffb640 1 --2- 172.21.15.112:0/2294242419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6600053c0 0x7fa66000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.469+0000 7fa675ffb640 1 -- 172.21.15.112:0/2294242419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa66000c220 con 0x7fa6600053c0 2024-04-25T01:33:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.469+0000 7fa675ffb640 1 -- 172.21.15.112:0/2294242419 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa66000c3d0 con 0x7fa660004800 2024-04-25T01:33:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.469+0000 7fa675ffb640 1 -- 172.21.15.112:0/2294242419 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa66000c5d0 con 0x7fa660003b70 2024-04-25T01:33:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.469+0000 7fa676ffd640 1 -- 172.21.15.112:0/2294242419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa660003b70 msgr2=0x7fa660003f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:33:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.469+0000 7fa676ffd640 1 --2- 172.21.15.112:0/2294242419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa660003b70 0x7fa660003f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.469+0000 7fa6777fe640 1 --2- 172.21.15.112:0/2294242419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6600053c0 0x7fa66000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.470+0000 7fa6767fc640 1 --2- 172.21.15.112:0/2294242419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa660004800 0x7fa660004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.470+0000 7fa6777fe640 1 -- 172.21.15.112:0/2294242419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa660003b70 msgr2=0x7fa660003f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:33:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.470+0000 7fa6777fe640 1 --2- 172.21.15.112:0/2294242419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa660003b70 0x7fa660003f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.470+0000 7fa6777fe640 1 -- 172.21.15.112:0/2294242419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa660004800 msgr2=0x7fa660004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:33:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.470+0000 7fa6777fe640 1 --2- 172.21.15.112:0/2294242419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa660004800 0x7fa660004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.470+0000 7fa6777fe640 1 -- 172.21.15.112:0/2294242419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66400c0a0 con 0x7fa6600053c0 2024-04-25T01:33:31.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.470+0000 7fa6777fe640 1 --2- 172.21.15.112:0/2294242419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6600053c0 0x7fa66000b9c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa6780aa3f0 tx=0x7fa660005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd6672696c3891 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:31.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.472+0000 7fa675ffb640 1 -- 172.21.15.112:0/2294242419 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670031070 con 0x7fa6600053c0 2024-04-25T01:33:31.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.473+0000 7fa675ffb640 1 -- 172.21.15.112:0/2294242419 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa67002b510 con 0x7fa6600053c0 2024-04-25T01:33:31.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.473+0000 7fa675ffb640 1 -- 172.21.15.112:0/2294242419 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670009700 con 0x7fa6600053c0 2024-04-25T01:33:31.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.473+0000 7fa67d996640 1 -- 172.21.15.112:0/2294242419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6600053c0 msgr2=0x7fa66000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:31.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.473+0000 7fa67d996640 1 --2- 172.21.15.112:0/2294242419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6600053c0 0x7fa66000b9c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa6780aa3f0 tx=0x7fa660005830 comp rx=0 tx=0).stop 2024-04-25T01:33:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.473+0000 7fa67d996640 1 -- 172.21.15.112:0/2294242419 shutdown_connections 2024-04-25T01:33:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.473+0000 7fa67d996640 1 --2- 172.21.15.112:0/2294242419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6600053c0 0x7fa66000b9c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.473+0000 7fa67d996640 1 --2- 172.21.15.112:0/2294242419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa660004800 0x7fa660004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.473+0000 7fa67d996640 1 --2- 172.21.15.112:0/2294242419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa660003b70 0x7fa660003f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.473+0000 7fa67d996640 1 -- 172.21.15.112:0/2294242419 >> 172.21.15.112:0/2294242419 conn(0x7fa6780c42d0 msgr2=0x7fa6780c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.474+0000 7fa67d996640 1 -- 172.21.15.112:0/2294242419 shutdown_connections 2024-04-25T01:33:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.474+0000 7fa67d996640 1 -- 172.21.15.112:0/2294242419 wait complete. 2024-04-25T01:33:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.475+0000 7fa67d996640 1 Processor -- start 2024-04-25T01:33:31.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.475+0000 7fa67d996640 1 -- start start 2024-04-25T01:33:31.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.476+0000 7fa67d996640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa660003b70 0x7fa678072870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:31.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.476+0000 7fa67d996640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa660004800 0x7fa6781eaa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:31.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.476+0000 7fa67d996640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6600053c0 0x7fa6781eafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:31.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.476+0000 7fa67d996640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa67806c2f0 con 0x7fa660004800 2024-04-25T01:33:31.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.476+0000 7fa67d996640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa67806c170 con 0x7fa6600053c0 2024-04-25T01:33:31.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.476+0000 7fa67d996640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa67806c470 con 0x7fa660003b70 2024-04-25T01:33:31.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.476+0000 7fa676ffd640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa660003b70 msgr2=0x7fa678072870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:33:31.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.476+0000 7fa676ffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa660003b70 0x7fa678072870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:31.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.476+0000 7fa6767fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa660004800 0x7fa6781eaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:31.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.476+0000 7fa6767fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa660004800 0x7fa6781eaa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54022/0 (socket says 172.21.15.112:54022) 2024-04-25T01:33:31.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.476+0000 7fa6767fc640 1 -- 172.21.15.112:0/3582809093 learned_addr learned my addr 172.21.15.112:0/3582809093 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:31.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.476+0000 7fa6777fe640 1 --2- 172.21.15.112:0/3582809093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6600053c0 0x7fa6781eafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:31.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.477+0000 7fa6767fc640 1 -- 172.21.15.112:0/3582809093 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa660003b70 msgr2=0x7fa678072870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:33:31.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.477+0000 7fa6767fc640 1 --2- 172.21.15.112:0/3582809093 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa660003b70 0x7fa678072870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.477+0000 7fa6767fc640 1 -- 172.21.15.112:0/3582809093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6600053c0 msgr2=0x7fa6781eafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:31.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.477+0000 7fa6767fc640 1 --2- 172.21.15.112:0/3582809093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6600053c0 0x7fa6781eafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.477+0000 7fa6767fc640 1 -- 172.21.15.112:0/3582809093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6780aa960 con 0x7fa660004800 2024-04-25T01:33:31.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.477+0000 7fa6777fe640 1 --2- 172.21.15.112:0/3582809093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6600053c0 0x7fa6781eafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:33:31.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.477+0000 7fa6767fc640 1 --2- 172.21.15.112:0/3582809093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa660004800 0x7fa6781eaa80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa66c004630 tx=0x7fa66c003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:31.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.477+0000 7fa65ffff640 1 -- 172.21.15.112:0/3582809093 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c0166d0 con 0x7fa660004800 2024-04-25T01:33:31.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.477+0000 7fa67d996640 1 -- 172.21.15.112:0/3582809093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa678157dd0 con 0x7fa660004800 2024-04-25T01:33:31.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.477+0000 7fa65ffff640 1 -- 172.21.15.112:0/3582809093 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa66c016870 con 0x7fa660004800 2024-04-25T01:33:31.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.477+0000 7fa65ffff640 1 -- 172.21.15.112:0/3582809093 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c016c60 con 0x7fa660004800 2024-04-25T01:33:31.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.477+0000 7fa67d996640 1 -- 172.21.15.112:0/3582809093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa678158390 con 0x7fa660004800 2024-04-25T01:33:31.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.478+0000 7fa67d996640 1 -- 172.21.15.112:0/3582809093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6780689b0 con 0x7fa660004800 2024-04-25T01:33:31.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.478+0000 7fa65ffff640 1 -- 172.21.15.112:0/3582809093 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa66c029440 con 0x7fa660004800 2024-04-25T01:33:31.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.479+0000 7fa65ffff640 1 --2- 172.21.15.112:0/3582809093 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa66406c790 0x7fa66406ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:31.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.479+0000 7fa65ffff640 1 -- 172.21.15.112:0/3582809093 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fa66c08c050 con 0x7fa660004800 2024-04-25T01:33:31.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.479+0000 7fa676ffd640 1 --2- 172.21.15.112:0/3582809093 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa66406c790 0x7fa66406ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:31.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.479+0000 7fa676ffd640 1 --2- 172.21.15.112:0/3582809093 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa66406c790 0x7fa66406ec50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa668003930 tx=0x7fa668004100 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:31.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.483+0000 7fa65ffff640 1 -- 172.21.15.112:0/3582809093 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa6780aa960 con 0x7fa660004800 2024-04-25T01:33:31.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.793+0000 7fa67d996640 1 -- 172.21.15.112:0/3582809093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa6780c6ac0 con 0x7fa660004800 2024-04-25T01:33:31.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.793+0000 7fa65ffff640 1 -- 172.21.15.112:0/3582809093 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7fa66c05a5e0 con 0x7fa660004800 2024-04-25T01:33:31.770 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":60,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:33:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.795+0000 7fa67d996640 1 -- 172.21.15.112:0/3582809093 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa66406c790 msgr2=0x7fa66406ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.795+0000 7fa67d996640 1 --2- 172.21.15.112:0/3582809093 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa66406c790 0x7fa66406ec50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa668003930 tx=0x7fa668004100 comp rx=0 tx=0).stop 2024-04-25T01:33:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.795+0000 7fa67d996640 1 -- 172.21.15.112:0/3582809093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa660004800 msgr2=0x7fa6781eaa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.795+0000 7fa67d996640 1 --2- 172.21.15.112:0/3582809093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa660004800 0x7fa6781eaa80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa66c004630 tx=0x7fa66c003aa0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.795+0000 7fa67d996640 1 -- 172.21.15.112:0/3582809093 shutdown_connections 2024-04-25T01:33:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.795+0000 7fa67d996640 1 --2- 172.21.15.112:0/3582809093 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa66406c790 0x7fa66406ec50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.795+0000 7fa67d996640 1 --2- 172.21.15.112:0/3582809093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6600053c0 0x7fa6781eafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.795+0000 7fa67d996640 1 --2- 172.21.15.112:0/3582809093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa660004800 0x7fa6781eaa80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.795+0000 7fa67d996640 1 --2- 172.21.15.112:0/3582809093 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa660003b70 0x7fa678072870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.795+0000 7fa67d996640 1 -- 172.21.15.112:0/3582809093 >> 172.21.15.112:0/3582809093 conn(0x7fa6780c42d0 msgr2=0x7fa678159020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.795+0000 7fa67d996640 1 -- 172.21.15.112:0/3582809093 shutdown_connections 2024-04-25T01:33:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.795+0000 7fa67d996640 1 -- 172.21.15.112:0/3582809093 wait complete. 2024-04-25T01:33:31.786 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T01:33:31.787 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-04-25T01:33:31.788 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:33:31.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.924+0000 7f9bd857e640 1 Processor -- start 2024-04-25T01:33:31.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.924+0000 7f9bd857e640 1 -- start start 2024-04-25T01:33:31.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.925+0000 7f9bd857e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9bd01592c0 0x7f9bd00c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:31.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.925+0000 7f9bd857e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bd01596a0 0x7f9bd0152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:31.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.925+0000 7f9bd857e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9bd00c72b0 0x7f9bd0155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:31.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.925+0000 7f9bd857e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f9bd00c6e20 con 0x7f9bd00c72b0 2024-04-25T01:33:31.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.925+0000 7f9bd857e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9bd0159b90 con 0x7f9bd01592c0 2024-04-25T01:33:31.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.925+0000 7f9bd857e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9bd006c970 con 0x7f9bd01596a0 2024-04-25T01:33:31.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.925+0000 7f9bd6b1b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9bd00c72b0 0x7f9bd0155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:31.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.925+0000 7f9bd6b1b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9bd00c72b0 0x7f9bd0155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54026/0 (socket says 172.21.15.112:54026) 2024-04-25T01:33:31.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.925+0000 7f9bd6b1b640 1 -- 172.21.15.112:0/3457331368 learned_addr learned my addr 172.21.15.112:0/3457331368 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:31.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.925+0000 7f9bd5b19640 1 -- 172.21.15.112:0/3457331368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bd01596a0 msgr2=0x7f9bd0152d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:33:31.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.925+0000 7f9bd5b19640 1 --2- 172.21.15.112:0/3457331368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bd01596a0 0x7f9bd0152d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:31.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.926+0000 7f9bd6b1b640 1 -- 172.21.15.112:0/3457331368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bd01596a0 msgr2=0x7f9bd0152d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:33:31.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.926+0000 7f9bd6b1b640 1 --2- 172.21.15.112:0/3457331368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bd01596a0 0x7f9bd0152d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.926+0000 7f9bd6b1b640 1 -- 172.21.15.112:0/3457331368 >> v2:172.21.15.187:3300/0 conn(0x7f9bd01592c0 msgr2=0x7f9bd00c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:33:31.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.926+0000 7f9bd6b1b640 1 --2- 172.21.15.112:0/3457331368 >> v2:172.21.15.187:3300/0 conn(0x7f9bd01592c0 0x7f9bd00c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.926+0000 7f9bd6b1b640 1 -- 172.21.15.112:0/3457331368 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd006cc50 con 0x7f9bd00c72b0 2024-04-25T01:33:31.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.926+0000 7f9bd6b1b640 1 --2- 172.21.15.112:0/3457331368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd00c72b0 0x7f9bd0155ae0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9bd008cb00 tx=0x7f9bc802a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6eeeba723e4574 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:31.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.927+0000 7f9bd5318640 1 -- 172.21.15.112:0/3457331368 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8036070 con 0x7f9bd00c72b0 2024-04-25T01:33:31.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.927+0000 7f9bd5318640 1 -- 172.21.15.112:0/3457331368 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bc8031600 con 0x7f9bd00c72b0 2024-04-25T01:33:31.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.927+0000 7f9bd5318640 1 -- 172.21.15.112:0/3457331368 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80319d0 con 0x7f9bd00c72b0 2024-04-25T01:33:31.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.927+0000 7f9bd857e640 1 -- 172.21.15.112:0/3457331368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd00c72b0 msgr2=0x7f9bd0155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:31.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.927+0000 7f9bd857e640 1 --2- 172.21.15.112:0/3457331368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd00c72b0 0x7f9bd0155ae0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9bd008cb00 tx=0x7f9bc802a590 comp rx=0 tx=0).stop 2024-04-25T01:33:31.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.927+0000 7f9bd857e640 1 -- 172.21.15.112:0/3457331368 shutdown_connections 2024-04-25T01:33:31.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.927+0000 7f9bd857e640 1 --2- 172.21.15.112:0/3457331368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd00c72b0 0x7f9bd0155ae0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.927+0000 7f9bd857e640 1 --2- 172.21.15.112:0/3457331368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bd01596a0 0x7f9bd0152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.927+0000 7f9bd857e640 1 --2- 172.21.15.112:0/3457331368 >> v2:172.21.15.187:3300/0 conn(0x7f9bd01592c0 0x7f9bd00c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.927+0000 7f9bd857e640 1 -- 172.21.15.112:0/3457331368 >> 172.21.15.112:0/3457331368 conn(0x7f9bd00cb170 msgr2=0x7f9bd00cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:31.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.927+0000 7f9bd857e640 1 -- 172.21.15.112:0/3457331368 shutdown_connections 2024-04-25T01:33:31.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.927+0000 7f9bd857e640 1 -- 172.21.15.112:0/3457331368 wait complete. 2024-04-25T01:33:31.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.929+0000 7f9bd857e640 1 Processor -- start 2024-04-25T01:33:31.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.929+0000 7f9bd857e640 1 -- start start 2024-04-25T01:33:31.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.929+0000 7f9bd857e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd00c72b0 0x7f9bd01511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:31.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.929+0000 7f9bd857e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bd01592c0 0x7f9bd01eb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:31.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.929+0000 7f9bd857e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bd01596a0 0x7f9bd01ebac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:31.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.929+0000 7f9bd857e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd006dca0 con 0x7f9bd00c72b0 2024-04-25T01:33:31.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.929+0000 7f9bd857e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd006db20 con 0x7f9bd01592c0 2024-04-25T01:33:31.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.929+0000 7f9bd857e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9bd00c95b0 con 0x7f9bd01596a0 2024-04-25T01:33:31.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.929+0000 7f9bd631a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd00c72b0 0x7f9bd01511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:31.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.929+0000 7f9bd631a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd00c72b0 0x7f9bd01511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54038/0 (socket says 172.21.15.112:54038) 2024-04-25T01:33:31.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.929+0000 7f9bd631a640 1 -- 172.21.15.112:0/1544047322 learned_addr learned my addr 172.21.15.112:0/1544047322 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:31.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.929+0000 7f9bd6b1b640 1 -- 172.21.15.112:0/1544047322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bd01596a0 msgr2=0x7f9bd01ebac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:33:31.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.929+0000 7f9bd6b1b640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bd01596a0 0x7f9bd01ebac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:31.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.930+0000 7f9bd5b19640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bd01592c0 0x7f9bd01eb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:31.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.930+0000 7f9bd631a640 1 -- 172.21.15.112:0/1544047322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bd01596a0 msgr2=0x7f9bd01ebac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:33:31.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.930+0000 7f9bd631a640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bd01596a0 0x7f9bd01ebac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.930+0000 7f9bd631a640 1 -- 172.21.15.112:0/1544047322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bd01592c0 msgr2=0x7f9bd01eb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:31.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.930+0000 7f9bd631a640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bd01592c0 0x7f9bd01eb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.930+0000 7f9bd631a640 1 -- 172.21.15.112:0/1544047322 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd01ef9e0 con 0x7f9bd00c72b0 2024-04-25T01:33:31.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.930+0000 7f9bd5b19640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bd01592c0 0x7f9bd01eb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:33:31.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.930+0000 7f9bd631a640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd00c72b0 0x7f9bd01511c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc00b4a0 tx=0x7f9bcc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:31.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.930+0000 7f9bc6ffd640 1 -- 172.21.15.112:0/1544047322 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc014070 con 0x7f9bd00c72b0 2024-04-25T01:33:31.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.930+0000 7f9bc6ffd640 1 -- 172.21.15.112:0/1544047322 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bcc010610 con 0x7f9bd00c72b0 2024-04-25T01:33:31.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.930+0000 7f9bc6ffd640 1 -- 172.21.15.112:0/1544047322 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc010a00 con 0x7f9bd00c72b0 2024-04-25T01:33:31.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.930+0000 7f9bd857e640 1 -- 172.21.15.112:0/1544047322 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd0146eb0 con 0x7f9bd00c72b0 2024-04-25T01:33:31.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.930+0000 7f9bd857e640 1 -- 172.21.15.112:0/1544047322 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd01eef00 con 0x7f9bd00c72b0 2024-04-25T01:33:31.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.931+0000 7f9bd857e640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd01468f0 0x7f9bd014f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:31.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.931+0000 7f9bd857e640 1 -- 172.21.15.112:0/1544047322 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd0075980 con 0x7f9bd01468f0 2024-04-25T01:33:31.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.931+0000 7f9bd5b19640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd01468f0 0x7f9bd014f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:31.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.932+0000 7f9bd5b19640 1 -- 172.21.15.112:0/1544047322 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9bd00679a0 con 0x7f9bd01468f0 2024-04-25T01:33:31.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.932+0000 7f9bd5b19640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd01468f0 0x7f9bd014f850 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9bd00cbc60 tx=0x7f9bc000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:31.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.932+0000 7f9bc6ffd640 1 -- 172.21.15.112:0/1544047322 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9bcc017070 con 0x7f9bd00c72b0 2024-04-25T01:33:31.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.932+0000 7f9bc6ffd640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9bb406bde0 0x7f9bb406e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:31.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.932+0000 7f9bc6ffd640 1 -- 172.21.15.112:0/1544047322 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f9bcc08abb0 con 0x7f9bd00c72b0 2024-04-25T01:33:31.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.932+0000 7f9bd6b1b640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9bb406bde0 0x7f9bb406e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:31.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.932+0000 7f9bc6ffd640 1 -- 172.21.15.112:0/1544047322 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc00039b0 con 0x7f9bd01468f0 2024-04-25T01:33:31.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.932+0000 7f9bc6ffd640 1 -- 172.21.15.112:0/1544047322 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9bc0017070 con 0x7f9bd01468f0 2024-04-25T01:33:31.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.932+0000 7f9bc6ffd640 1 -- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd01468f0 msgr2=0x7f9bd014f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:31.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.932+0000 7f9bc6ffd640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd01468f0 0x7f9bd014f850 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9bd00cbc60 tx=0x7f9bc000db90 comp rx=0 tx=0).stop 2024-04-25T01:33:31.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.932+0000 7f9bc6ffd640 1 -- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd01468f0 msgr2=0x7f9bd014f850 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:33:31.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.932+0000 7f9bc6ffd640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd01468f0 0x7f9bd014f850 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.933+0000 7f9bd6b1b640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9bb406bde0 0x7f9bb406e2a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8033000 tx=0x7f9bc8009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:31.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.940+0000 7f9bd857e640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd014fd90 0x7f9bd01e00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:31.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.940+0000 7f9bd857e640 1 -- 172.21.15.112:0/1544047322 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd00abd20 con 0x7f9bd014fd90 2024-04-25T01:33:31.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.940+0000 7f9bd5b19640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd014fd90 0x7f9bd01e00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:31.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.941+0000 7f9bd5b19640 1 -- 172.21.15.112:0/1544047322 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9bd00c79d0 con 0x7f9bd014fd90 2024-04-25T01:33:31.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.941+0000 7f9bd5b19640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd014fd90 0x7f9bd01e00b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0013000 tx=0x7f9bc00059d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:31.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.941+0000 7f9bc6ffd640 1 -- 172.21.15.112:0/1544047322 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc001ccd0 con 0x7f9bd014fd90 2024-04-25T01:33:31.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.941+0000 7f9bc6ffd640 1 -- 172.21.15.112:0/1544047322 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4719 (secure 0 0 0) 0x7f9bc001ccd0 con 0x7f9bd014fd90 2024-04-25T01:33:31.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.941+0000 7f9bc6ffd640 1 -- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd014fd90 msgr2=0x7f9bd01e00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:31.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.941+0000 7f9bc6ffd640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd014fd90 0x7f9bd01e00b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0013000 tx=0x7f9bc00059d0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.941+0000 7f9bc6ffd640 1 -- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd014fd90 msgr2=0x7f9bd01e00b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:33:31.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.941+0000 7f9bc6ffd640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd014fd90 0x7f9bd01e00b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.919 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:33:31.919 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:33:31.919 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:33:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:33:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 60, 2024-04-25T01:33:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 92706, 2024-04-25T01:33:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:33:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:33:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: 1 2024-04-25T01:33:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:33:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:33:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:33:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:33:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:33:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:33:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:33:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:33:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:33:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:33:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:33:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:33:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:33:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:33:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:33:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:33:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:33:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:33:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:33:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:33:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:33:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:33:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:33:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:33:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:33:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:33:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:33:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:33:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:33:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:33:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:33:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:33:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:33:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:33:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:33:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:33:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:33:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:33:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:33:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:33:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:33:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:33:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:33:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:33:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:33:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:33:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:33:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:33:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:33:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:33:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:33:31.931 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:33:31.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.941+0000 7f9bd857e640 1 -- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9bb406bde0 msgr2=0x7f9bb406e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:31.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.941+0000 7f9bd857e640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9bb406bde0 0x7f9bb406e2a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8033000 tx=0x7f9bc8009000 comp rx=0 tx=0).stop 2024-04-25T01:33:31.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.941+0000 7f9bd857e640 1 -- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd00c72b0 msgr2=0x7f9bd01511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:31.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.941+0000 7f9bd857e640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd00c72b0 0x7f9bd01511c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc00b4a0 tx=0x7f9bcc00b960 comp rx=0 tx=0).stop 2024-04-25T01:33:31.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.941+0000 7f9bd6b1b640 1 -- 172.21.15.112:0/1544047322 reap_dead start 2024-04-25T01:33:31.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.941+0000 7f9bd857e640 1 -- 172.21.15.112:0/1544047322 shutdown_connections 2024-04-25T01:33:31.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.941+0000 7f9bd857e640 1 --2- 172.21.15.112:0/1544047322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd00c72b0 0x7f9bd01511c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:31.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.941+0000 7f9bd857e640 1 -- 172.21.15.112:0/1544047322 >> 172.21.15.112:0/1544047322 conn(0x7f9bd00cb170 msgr2=0x7f9bd0152ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:31.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.942+0000 7f9bd857e640 1 -- 172.21.15.112:0/1544047322 shutdown_connections 2024-04-25T01:33:31.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:31.942+0000 7f9bd857e640 1 -- 172.21.15.112:0/1544047322 wait complete. 2024-04-25T01:33:31.933 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:33:31.933 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-04-25T01:33:31.934 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:33:32.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.060+0000 7f01e3631640 1 Processor -- start 2024-04-25T01:33:32.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.061+0000 7f01e3631640 1 -- start start 2024-04-25T01:33:32.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.062+0000 7f01e3631640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01dc1596a0 0x7f01dc152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:32.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.062+0000 7f01e3631640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f01dc0c72b0 0x7f01dc160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:32.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.062+0000 7f01e3631640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01dc159b90 con 0x7f01dc1592c0 2024-04-25T01:33:32.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.062+0000 7f01e3631640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f01dc0c6e20 con 0x7f01dc0c72b0 2024-04-25T01:33:32.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.062+0000 7f01e3631640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f01dc06c970 con 0x7f01dc1596a0 2024-04-25T01:33:32.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.062+0000 7f01e0bcc640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01dc1596a0 msgr2=0x7f01dc152d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:33:32.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.062+0000 7f01e0bcc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01dc1596a0 0x7f01dc152d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:32.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.062+0000 7f01e13cd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f01dc1592c0 0x7f01dc0c9070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:35886/0 (socket says 172.21.15.112:35886) 2024-04-25T01:33:32.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.062+0000 7f01e13cd640 1 -- 172.21.15.112:0/2759079066 learned_addr learned my addr 172.21.15.112:0/2759079066 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:33:32.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.062+0000 7f01e1bce640 1 --2- 172.21.15.112:0/2759079066 >> v2:172.21.15.112:3300/0 conn(0x7f01dc0c72b0 0x7f01dc160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:32.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.062+0000 7f01d3fff640 1 -- 172.21.15.112:0/2759079066 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3364257261 0 0) 0x7f01dc159b90 con 0x7f01dc1592c0 2024-04-25T01:33:32.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.062+0000 7f01d3fff640 1 -- 172.21.15.112:0/2759079066 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01c8003520 con 0x7f01dc1592c0 2024-04-25T01:33:32.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.062+0000 7f01e1bce640 1 -- 172.21.15.112:0/2759079066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01dc1596a0 msgr2=0x7f01dc152d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:33:32.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.062+0000 7f01e1bce640 1 --2- 172.21.15.112:0/2759079066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01dc1596a0 0x7f01dc152d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:32.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.062+0000 7f01e1bce640 1 -- 172.21.15.112:0/2759079066 >> v1:172.21.15.112:6789/0 conn(0x7f01dc1592c0 legacy=0x7f01dc0c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:32.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.062+0000 7f01e1bce640 1 -- 172.21.15.112:0/2759079066 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01dc06cc50 con 0x7f01dc0c72b0 2024-04-25T01:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.063+0000 7f01e1bce640 1 --2- 172.21.15.112:0/2759079066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01dc0c72b0 0x7f01dc160380 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009d90 tx=0x7f01d402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2be5fbfb61c393 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.063+0000 7f01d3fff640 1 -- 172.21.15.112:0/2759079066 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4007740 con 0x7f01dc0c72b0 2024-04-25T01:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.063+0000 7f01d3fff640 1 -- 172.21.15.112:0/2759079066 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d40078e0 con 0x7f01dc0c72b0 2024-04-25T01:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.063+0000 7f01d3fff640 1 -- 172.21.15.112:0/2759079066 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4007c90 con 0x7f01dc0c72b0 2024-04-25T01:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.064+0000 7f01e3631640 1 -- 172.21.15.112:0/2759079066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01dc0c72b0 msgr2=0x7f01dc160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.064+0000 7f01e3631640 1 --2- 172.21.15.112:0/2759079066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01dc0c72b0 0x7f01dc160380 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009d90 tx=0x7f01d402a560 comp rx=0 tx=0).stop 2024-04-25T01:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.064+0000 7f01e3631640 1 -- 172.21.15.112:0/2759079066 shutdown_connections 2024-04-25T01:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.064+0000 7f01e3631640 1 --2- 172.21.15.112:0/2759079066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01dc0c72b0 0x7f01dc160380 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.064+0000 7f01e3631640 1 --2- 172.21.15.112:0/2759079066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01dc1596a0 0x7f01dc152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.064+0000 7f01e3631640 1 -- 172.21.15.112:0/2759079066 >> 172.21.15.112:0/2759079066 conn(0x7f01dc0cb170 msgr2=0x7f01dc0cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.064+0000 7f01e3631640 1 -- 172.21.15.112:0/2759079066 shutdown_connections 2024-04-25T01:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.064+0000 7f01e3631640 1 -- 172.21.15.112:0/2759079066 wait complete. 2024-04-25T01:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.065+0000 7f01e3631640 1 Processor -- start 2024-04-25T01:33:32.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.065+0000 7f01e3631640 1 -- start start 2024-04-25T01:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.066+0000 7f01e3631640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01dc0c72b0 0x7f01dc0c0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.066+0000 7f01e3631640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01dc1592c0 0x7f01dc1e9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.066+0000 7f01e3631640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01dc1596a0 0x7f01dc1eee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.066+0000 7f01e3631640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc06c510 con 0x7f01dc1596a0 2024-04-25T01:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.066+0000 7f01e3631640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc06c390 con 0x7f01dc0c72b0 2024-04-25T01:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.066+0000 7f01e3631640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f01dc06db20 con 0x7f01dc1592c0 2024-04-25T01:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.066+0000 7f01e1bce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01dc1596a0 0x7f01dc1eee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.066+0000 7f01e1bce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01dc1596a0 0x7f01dc1eee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54072/0 (socket says 172.21.15.112:54072) 2024-04-25T01:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.066+0000 7f01e0bcc640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01dc1592c0 msgr2=0x7f01dc1e9620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.066+0000 7f01e1bce640 1 -- 172.21.15.112:0/2250620497 learned_addr learned my addr 172.21.15.112:0/2250620497 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.066+0000 7f01e0bcc640 1 --2- 172.21.15.112:0/2250620497 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01dc1592c0 0x7f01dc1e9620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.066+0000 7f01e13cd640 1 --2- 172.21.15.112:0/2250620497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01dc0c72b0 0x7f01dc0c0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.066+0000 7f01e1bce640 1 -- 172.21.15.112:0/2250620497 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01dc1592c0 msgr2=0x7f01dc1e9620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:33:32.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.066+0000 7f01e1bce640 1 --2- 172.21.15.112:0/2250620497 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01dc1592c0 0x7f01dc1e9620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:32.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.066+0000 7f01e1bce640 1 -- 172.21.15.112:0/2250620497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01dc0c72b0 msgr2=0x7f01dc0c0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:32.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.066+0000 7f01e1bce640 1 --2- 172.21.15.112:0/2250620497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01dc0c72b0 0x7f01dc0c0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:32.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.066+0000 7f01e1bce640 1 -- 172.21.15.112:0/2250620497 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01dc0c07d0 con 0x7f01dc1596a0 2024-04-25T01:33:32.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.066+0000 7f01e1bce640 1 --2- 172.21.15.112:0/2250620497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01dc1596a0 0x7f01dc1eee20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009eb0 tx=0x7f01d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:32.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.067+0000 7f01d1ffb640 1 -- 172.21.15.112:0/2250620497 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4009450 con 0x7f01dc1596a0 2024-04-25T01:33:32.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.067+0000 7f01d1ffb640 1 -- 172.21.15.112:0/2250620497 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d4004240 con 0x7f01dc1596a0 2024-04-25T01:33:32.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.067+0000 7f01e3631640 1 -- 172.21.15.112:0/2250620497 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01dc1ef8a0 con 0x7f01dc1596a0 2024-04-25T01:33:32.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.067+0000 7f01d1ffb640 1 -- 172.21.15.112:0/2250620497 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d40045f0 con 0x7f01dc1596a0 2024-04-25T01:33:32.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.067+0000 7f01e3631640 1 -- 172.21.15.112:0/2250620497 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01dc0bfd90 con 0x7f01dc1596a0 2024-04-25T01:33:32.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.068+0000 7f01e3631640 1 --2- 172.21.15.112:0/2250620497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01dc0c1a60 0x7f01dc1eb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:32.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.068+0000 7f01e3631640 1 -- 172.21.15.112:0/2250620497 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc0bf910 con 0x7f01dc0c1a60 2024-04-25T01:33:32.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.068+0000 7f01e13cd640 1 --2- 172.21.15.112:0/2250620497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01dc0c1a60 0x7f01dc1eb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:32.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.068+0000 7f01e13cd640 1 -- 172.21.15.112:0/2250620497 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01dc1542a0 con 0x7f01dc0c1a60 2024-04-25T01:33:32.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.069+0000 7f01e13cd640 1 --2- 172.21.15.112:0/2250620497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01dc0c1a60 0x7f01dc1eb640 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0c54e0 tx=0x7f01d800e960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:32.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.069+0000 7f01d1ffb640 1 -- 172.21.15.112:0/2250620497 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f01d4039040 con 0x7f01dc1596a0 2024-04-25T01:33:32.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.069+0000 7f01d1ffb640 1 --2- 172.21.15.112:0/2250620497 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f01c006bde0 0x7f01c006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:32.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.069+0000 7f01d1ffb640 1 -- 172.21.15.112:0/2250620497 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8005a60 con 0x7f01dc0c1a60 2024-04-25T01:33:32.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.069+0000 7f01e0bcc640 1 --2- 172.21.15.112:0/2250620497 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f01c006bde0 0x7f01c006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:32.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.069+0000 7f01d1ffb640 1 -- 172.21.15.112:0/2250620497 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f01d40ac4e0 con 0x7f01dc1596a0 2024-04-25T01:33:32.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.069+0000 7f01d1ffb640 1 -- 172.21.15.112:0/2250620497 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f01d8018070 con 0x7f01dc0c1a60 2024-04-25T01:33:32.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.069+0000 7f01d1ffb640 1 -- 172.21.15.112:0/2250620497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01dc0c1a60 msgr2=0x7f01dc1eb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:32.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.069+0000 7f01d1ffb640 1 --2- 172.21.15.112:0/2250620497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01dc0c1a60 0x7f01dc1eb640 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0c54e0 tx=0x7f01d800e960 comp rx=0 tx=0).stop 2024-04-25T01:33:32.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.069+0000 7f01d1ffb640 1 -- 172.21.15.112:0/2250620497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01dc0c1a60 msgr2=0x7f01dc1eb640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:33:32.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.069+0000 7f01d1ffb640 1 --2- 172.21.15.112:0/2250620497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01dc0c1a60 0x7f01dc1eb640 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0c54e0 tx=0x7f01d800e960 comp rx=0 tx=0).stop 2024-04-25T01:33:32.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.070+0000 7f01e0bcc640 1 --2- 172.21.15.112:0/2250620497 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f01c006bde0 0x7f01c006e2a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f01cc006fd0 tx=0x7f01cc006d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:32.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.076+0000 7f01e3631640 1 --2- 172.21.15.112:0/2250620497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01dc1ef360 0x7f01dc1ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:32.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.076+0000 7f01e3631640 1 -- 172.21.15.112:0/2250620497 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc0679a0 con 0x7f01dc1ef360 2024-04-25T01:33:32.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.076+0000 7f01e13cd640 1 --2- 172.21.15.112:0/2250620497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01dc1ef360 0x7f01dc1ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:32.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.076+0000 7f01e13cd640 1 -- 172.21.15.112:0/2250620497 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f01dc06d320 con 0x7f01dc1ef360 2024-04-25T01:33:32.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.077+0000 7f01e13cd640 1 --2- 172.21.15.112:0/2250620497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01dc1ef360 0x7f01dc1ebb80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f01d8007fd0 tx=0x7f01d80010a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:32.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.077+0000 7f01d1ffb640 1 -- 172.21.15.112:0/2250620497 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d801d7c0 con 0x7f01dc1ef360 2024-04-25T01:33:32.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.077+0000 7f01d1ffb640 1 -- 172.21.15.112:0/2250620497 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5058 (secure 0 0 0) 0x7f01d801d7c0 con 0x7f01dc1ef360 2024-04-25T01:33:32.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.077+0000 7f01d1ffb640 1 -- 172.21.15.112:0/2250620497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01dc1ef360 msgr2=0x7f01dc1ebb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:32.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.077+0000 7f01d1ffb640 1 --2- 172.21.15.112:0/2250620497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01dc1ef360 0x7f01dc1ebb80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f01d8007fd0 tx=0x7f01d80010a0 comp rx=0 tx=0).stop 2024-04-25T01:33:32.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.077+0000 7f01d1ffb640 1 -- 172.21.15.112:0/2250620497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01dc1ef360 msgr2=0x7f01dc1ebb80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:33:32.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.077+0000 7f01d1ffb640 1 --2- 172.21.15.112:0/2250620497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01dc1ef360 0x7f01dc1ebb80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:32.055 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:33:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:33:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:33:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:33:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 60, 2024-04-25T01:33:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 247766, 2024-04-25T01:33:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:33:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:33:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: 1 2024-04-25T01:33:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:33:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:33:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:33:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:33:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:33:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:33:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:33:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:33:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:33:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:33:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:33:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:33:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:33:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:33:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:33:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:33:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:33:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:33:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:33:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:33:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:33:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:33:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:33:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:33:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:33:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:33:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:33:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:33:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:33:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:33:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:33:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:33:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:33:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:33:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:33:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:33:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:33:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:33:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:33:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:33:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:33:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:33:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:33:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:33:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:33:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:33:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:33:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:33:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:33:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:33:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:33:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:33:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:33:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:33:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:33:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:33:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:33:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:33:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:33:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:33:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:33:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:33:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:33:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:33:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:33:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:33:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:33:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:33:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:33:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:33:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:33:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:33:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:33:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:33:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:33:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:33:32.069 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:33:32.069 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:33:32.069 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:33:32.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:33:32.069 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:33:32.069 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:33:32.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.078+0000 7f01e3631640 1 -- 172.21.15.112:0/2250620497 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f01c006bde0 msgr2=0x7f01c006e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:32.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.078+0000 7f01e3631640 1 --2- 172.21.15.112:0/2250620497 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f01c006bde0 0x7f01c006e2a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f01cc006fd0 tx=0x7f01cc006d40 comp rx=0 tx=0).stop 2024-04-25T01:33:32.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.078+0000 7f01e3631640 1 -- 172.21.15.112:0/2250620497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01dc1596a0 msgr2=0x7f01dc1eee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:32.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.078+0000 7f01e3631640 1 --2- 172.21.15.112:0/2250620497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01dc1596a0 0x7f01dc1eee20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009eb0 tx=0x7f01d4004a40 comp rx=0 tx=0).stop 2024-04-25T01:33:32.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.078+0000 7f01e1bce640 1 -- 172.21.15.112:0/2250620497 reap_dead start 2024-04-25T01:33:32.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.078+0000 7f01e1bce640 1 -- 172.21.15.112:0/2250620497 reap_dead start 2024-04-25T01:33:32.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.078+0000 7f01e3631640 1 -- 172.21.15.112:0/2250620497 shutdown_connections 2024-04-25T01:33:32.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.078+0000 7f01e3631640 1 -- 172.21.15.112:0/2250620497 >> 172.21.15.112:0/2250620497 conn(0x7f01dc0cb170 msgr2=0x7f01dc0cb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:32.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.078+0000 7f01e3631640 1 -- 172.21.15.112:0/2250620497 shutdown_connections 2024-04-25T01:33:32.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:32.078+0000 7f01e3631640 1 -- 172.21.15.112:0/2250620497 wait complete. 2024-04-25T01:33:32.071 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:33:32.071 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T01:33:52.072 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-25T01:33:52.073 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-04-25T01:33:52.073 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T01:33:52.073 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T01:33:52.075 INFO:tasks.ceph.mon.c:Started 2024-04-25T01:33:52.076 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:33:52.076 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:33:52.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.208+0000 7f22ff241640 1 Processor -- start 2024-04-25T01:33:52.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.208+0000 7f22ff241640 1 -- start start 2024-04-25T01:33:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.209+0000 7f22ff241640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f81531d0 0x7f22f80c1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.209+0000 7f22ff241640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f22f80c1560 0x7f22f8155990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.209+0000 7f22ff241640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22f80aaa50 con 0x7f22f806bd90 2024-04-25T01:33:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.209+0000 7f22ff241640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f22f80aafd0 con 0x7f22f80c1560 2024-04-25T01:33:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.209+0000 7f22ff241640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f22f8159ab0 con 0x7f22f81531d0 2024-04-25T01:33:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.209+0000 7f22f7fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f22f80c1560 0x7f22f8155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.209+0000 7f22fcfdd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f81531d0 0x7f22f80c1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.209+0000 7f22f7fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f22f80c1560 0x7f22f8155990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33626/0 (socket says 172.21.15.112:33626) 2024-04-25T01:33:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.209+0000 7f22fd7de640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f22f806bd90 0x7f22f806b160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:53518/0 (socket says 172.21.15.112:53518) 2024-04-25T01:33:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.209+0000 7f22f7fff640 1 -- 172.21.15.112:0/3462998968 learned_addr learned my addr 172.21.15.112:0/3462998968 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.209+0000 7f22fcfdd640 1 -- 172.21.15.112:0/3462998968 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f81531d0 msgr2=0x7f22f80c1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:33:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.209+0000 7f22fcfdd640 1 -- 172.21.15.112:0/3462998968 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f81531d0 msgr2=0x7f22f80c1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.209+0000 7f22fcfdd640 1 --2- 172.21.15.112:0/3462998968 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f81531d0 0x7f22f80c1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.209+0000 7f22fcfdd640 1 --2- 172.21.15.112:0/3462998968 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f81531d0 0x7f22f80c1de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.209+0000 7f22f7fff640 1 -- 172.21.15.112:0/3462998968 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f81531d0 msgr2=0x7f22f80c1de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:33:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.209+0000 7f22f7fff640 1 --2- 172.21.15.112:0/3462998968 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f81531d0 0x7f22f80c1de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.209+0000 7f22f7fff640 1 -- 172.21.15.112:0/3462998968 >> v1:172.21.15.187:6789/0 conn(0x7f22f806bd90 legacy=0x7f22f806b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.209+0000 7f22f7fff640 1 -- 172.21.15.112:0/3462998968 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f80aacf0 con 0x7f22f80c1560 2024-04-25T01:33:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.210+0000 7f22f7fff640 1 --2- 172.21.15.112:0/3462998968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f80c1560 0x7f22f8155990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009870 tx=0x7f22e402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3032100b8caf3a8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.210+0000 7f22f77fe640 1 -- 172.21.15.112:0/3462998968 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4037070 con 0x7f22f80c1560 2024-04-25T01:33:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.210+0000 7f22f77fe640 1 -- 172.21.15.112:0/3462998968 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e4032440 con 0x7f22f80c1560 2024-04-25T01:33:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.210+0000 7f22f77fe640 1 -- 172.21.15.112:0/3462998968 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4032810 con 0x7f22f80c1560 2024-04-25T01:33:52.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.211+0000 7f22ff241640 1 -- 172.21.15.112:0/3462998968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f80c1560 msgr2=0x7f22f8155990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:52.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.211+0000 7f22ff241640 1 --2- 172.21.15.112:0/3462998968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f80c1560 0x7f22f8155990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009870 tx=0x7f22e402a130 comp rx=0 tx=0).stop 2024-04-25T01:33:52.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.211+0000 7f22ff241640 1 -- 172.21.15.112:0/3462998968 shutdown_connections 2024-04-25T01:33:52.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.211+0000 7f22ff241640 1 --2- 172.21.15.112:0/3462998968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f80c1560 0x7f22f8155990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:52.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.211+0000 7f22ff241640 1 --2- 172.21.15.112:0/3462998968 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f81531d0 0x7f22f80c1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:52.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.211+0000 7f22ff241640 1 -- 172.21.15.112:0/3462998968 >> 172.21.15.112:0/3462998968 conn(0x7f22f80c8130 msgr2=0x7f22f80c8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:52.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.211+0000 7f22ff241640 1 -- 172.21.15.112:0/3462998968 shutdown_connections 2024-04-25T01:33:52.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.211+0000 7f22ff241640 1 -- 172.21.15.112:0/3462998968 wait complete. 2024-04-25T01:33:52.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.212+0000 7f22ff241640 1 Processor -- start 2024-04-25T01:33:52.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.212+0000 7f22ff241640 1 -- start start 2024-04-25T01:33:52.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.213+0000 7f22ff241640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f806bd90 0x7f22f814edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:52.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.213+0000 7f22ff241640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f80c1560 0x7f22f81ec340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:52.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.213+0000 7f22ff241640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22f81531d0 0x7f22f806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:52.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.213+0000 7f22ff241640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f22f806b870 con 0x7f22f806bd90 2024-04-25T01:33:52.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.213+0000 7f22ff241640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f22f80c4900 con 0x7f22f81531d0 2024-04-25T01:33:52.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.213+0000 7f22ff241640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f22f806b9f0 con 0x7f22f80c1560 2024-04-25T01:33:52.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.214+0000 7f22fcfdd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f806bd90 0x7f22f814edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:52.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.214+0000 7f22f7fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f80c1560 0x7f22f81ec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:52.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.214+0000 7f22fcfdd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f806bd90 0x7f22f814edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33642/0 (socket says 172.21.15.112:33642) 2024-04-25T01:33:52.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.214+0000 7f22fcfdd640 1 -- 172.21.15.112:0/2601642196 learned_addr learned my addr 172.21.15.112:0/2601642196 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:33:52.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.214+0000 7f22fd7de640 1 --2- 172.21.15.112:0/2601642196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22f81531d0 0x7f22f806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:52.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.214+0000 7f22f7fff640 1 -- 172.21.15.112:0/2601642196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f80c1560 msgr2=0x7f22f81ec340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T01:33:52.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.214+0000 7f22f7fff640 1 -- 172.21.15.112:0/2601642196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f80c1560 msgr2=0x7f22f81ec340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:33:52.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.214+0000 7f22f7fff640 1 --2- 172.21.15.112:0/2601642196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f80c1560 0x7f22f81ec340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:52.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.214+0000 7f22f7fff640 1 --2- 172.21.15.112:0/2601642196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f80c1560 0x7f22f81ec340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:52.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.214+0000 7f22fcfdd640 1 -- 172.21.15.112:0/2601642196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f80c1560 msgr2=0x7f22f81ec340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:33:52.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.214+0000 7f22fcfdd640 1 --2- 172.21.15.112:0/2601642196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f80c1560 0x7f22f81ec340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:52.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.214+0000 7f22fcfdd640 1 -- 172.21.15.112:0/2601642196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22f81531d0 msgr2=0x7f22f806f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:52.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.214+0000 7f22fcfdd640 1 --2- 172.21.15.112:0/2601642196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22f81531d0 0x7f22f806f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:52.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.214+0000 7f22fcfdd640 1 -- 172.21.15.112:0/2601642196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f81553d0 con 0x7f22f806bd90 2024-04-25T01:33:52.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.214+0000 7f22fcfdd640 1 --2- 172.21.15.112:0/2601642196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f806bd90 0x7f22f814edd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00d800 tx=0x7f22ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:52.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.214+0000 7f22f57fa640 1 -- 172.21.15.112:0/2601642196 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec00b6c0 con 0x7f22f806bd90 2024-04-25T01:33:52.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.214+0000 7f22f57fa640 1 -- 172.21.15.112:0/2601642196 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22ec019070 con 0x7f22f806bd90 2024-04-25T01:33:52.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.214+0000 7f22ff241640 1 -- 172.21.15.112:0/2601642196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f8153650 con 0x7f22f806bd90 2024-04-25T01:33:52.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.214+0000 7f22f57fa640 1 -- 172.21.15.112:0/2601642196 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec00bac0 con 0x7f22f806bd90 2024-04-25T01:33:52.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.214+0000 7f22ff241640 1 -- 172.21.15.112:0/2601642196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f806f610 con 0x7f22f806bd90 2024-04-25T01:33:52.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.215+0000 7f22ff241640 1 -- 172.21.15.112:0/2601642196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f80689b0 con 0x7f22f806bd90 2024-04-25T01:33:52.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.216+0000 7f22f57fa640 1 -- 172.21.15.112:0/2601642196 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f22ec00bc60 con 0x7f22f806bd90 2024-04-25T01:33:52.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.216+0000 7f22f57fa640 1 --2- 172.21.15.112:0/2601642196 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f22dc06be60 0x7f22dc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:52.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.217+0000 7f22f57fa640 1 -- 172.21.15.112:0/2601642196 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f22ec08b5f0 con 0x7f22f806bd90 2024-04-25T01:33:52.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.217+0000 7f22f7fff640 1 --2- 172.21.15.112:0/2601642196 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f22dc06be60 0x7f22dc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:52.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.217+0000 7f22f7fff640 1 --2- 172.21.15.112:0/2601642196 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f22dc06be60 0x7f22dc06e320 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f22e4034000 tx=0x7f22e402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:52.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.223+0000 7f22f57fa640 1 -- 172.21.15.112:0/2601642196 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f22ec059c00 con 0x7f22f806bd90 2024-04-25T01:33:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.548+0000 7f22ff241640 1 -- 172.21.15.112:0/2601642196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f22f8150250 con 0x7f22f806bd90 2024-04-25T01:33:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.549+0000 7f22f57fa640 1 -- 172.21.15.112:0/2601642196 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1563 (secure 0 0 0) 0x7f22ec059500 con 0x7f22f806bd90 2024-04-25T01:33:52.526 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":60,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:33:52.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.551+0000 7f22ff241640 1 -- 172.21.15.112:0/2601642196 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f22dc06be60 msgr2=0x7f22dc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:52.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.551+0000 7f22ff241640 1 --2- 172.21.15.112:0/2601642196 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f22dc06be60 0x7f22dc06e320 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f22e4034000 tx=0x7f22e402e000 comp rx=0 tx=0).stop 2024-04-25T01:33:52.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.551+0000 7f22ff241640 1 -- 172.21.15.112:0/2601642196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f806bd90 msgr2=0x7f22f814edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:52.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.551+0000 7f22ff241640 1 --2- 172.21.15.112:0/2601642196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f806bd90 0x7f22f814edd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00d800 tx=0x7f22ec00dcc0 comp rx=0 tx=0).stop 2024-04-25T01:33:52.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.551+0000 7f22ff241640 1 -- 172.21.15.112:0/2601642196 shutdown_connections 2024-04-25T01:33:52.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.551+0000 7f22ff241640 1 --2- 172.21.15.112:0/2601642196 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f22dc06be60 0x7f22dc06e320 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:52.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.551+0000 7f22ff241640 1 --2- 172.21.15.112:0/2601642196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22f81531d0 0x7f22f806f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:52.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.551+0000 7f22ff241640 1 --2- 172.21.15.112:0/2601642196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22f80c1560 0x7f22f81ec340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:52.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.551+0000 7f22ff241640 1 --2- 172.21.15.112:0/2601642196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f806bd90 0x7f22f814edd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:52.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.551+0000 7f22ff241640 1 -- 172.21.15.112:0/2601642196 >> 172.21.15.112:0/2601642196 conn(0x7f22f80c8130 msgr2=0x7f22f8072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:52.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.552+0000 7f22ff241640 1 -- 172.21.15.112:0/2601642196 shutdown_connections 2024-04-25T01:33:52.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:52.552+0000 7f22ff241640 1 -- 172.21.15.112:0/2601642196 wait complete. 2024-04-25T01:33:55.544 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:33:55.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:55.674+0000 7f9bf0b70640 1 Processor -- start 2024-04-25T01:33:55.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:55.674+0000 7f9bf0b70640 1 -- start start 2024-04-25T01:33:55.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:55.675+0000 7f9bf0b70640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 0x7f9bec0c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:55.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:55.675+0000 7f9bf0b70640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bec0c90e0 con 0x7f9bec0c5d70 2024-04-25T01:33:55.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:55.675+0000 7f9bf0b70640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bec0cac00 con 0x7f9bec156cf0 2024-04-25T01:33:55.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:55.675+0000 7f9bf0b70640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f9bec0ca9b0 con 0x7f9bec152fd0 2024-04-25T01:33:55.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:55.675+0000 7f9bea59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 0x7f9bec0c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:55.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:55.675+0000 7f9be9d9b640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9bec0c5d70 0x7f9bec156280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:46236/0 (socket says 172.21.15.112:46236) 2024-04-25T01:33:55.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:55.675+0000 7f9be9d9b640 1 -- 172.21.15.112:0/3655332938 learned_addr learned my addr 172.21.15.112:0/3655332938 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:33:55.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:55.675+0000 7f9bea59c640 1 -- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 msgr2=0x7f9bec0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:33:55.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:55.675+0000 7f9bea59c640 1 -- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 msgr2=0x7f9bec0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:55.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:55.675+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 0x7f9bec0c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:55.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:55.675+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 0x7f9bec0c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:55.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:55.876+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 0x7f9bec0c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:55.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:55.876+0000 7f9bea59c640 1 -- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 msgr2=0x7f9bec0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:33:55.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:55.876+0000 7f9bea59c640 1 -- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 msgr2=0x7f9bec0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:55.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:55.876+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 0x7f9bec0c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:55.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:55.876+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 0x7f9bec0c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:33:56.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:56.277+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 0x7f9bec0c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:56.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:56.277+0000 7f9bea59c640 1 -- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 msgr2=0x7f9bec0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:33:56.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:56.277+0000 7f9bea59c640 1 -- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 msgr2=0x7f9bec0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:56.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:56.277+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 0x7f9bec0c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:56.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:56.277+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 0x7f9bec0c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:33:56.500 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:33:56.503 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:33:57.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:57.078+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 0x7f9bec0c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:57.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:57.078+0000 7f9bea59c640 1 -- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 msgr2=0x7f9bec0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:33:57.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:57.078+0000 7f9bea59c640 1 -- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 msgr2=0x7f9bec0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:57.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:57.078+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 0x7f9bec0c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:57.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:57.078+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 0x7f9bec0c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:33:58.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.675+0000 7f9bdbfff640 1 -- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 msgr2=0x7f9bec0c9b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:33:58.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.675+0000 7f9bdbfff640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bec152fd0 0x7f9bec0c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:58.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.675+0000 7f9bdbfff640 1 -- 172.21.15.112:0/3655332938 >> v1:172.21.15.187:6789/0 conn(0x7f9bec156cf0 legacy=0x7f9bec06bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:58.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.675+0000 7f9bdbfff640 1 -- 172.21.15.112:0/3655332938 >> v1:172.21.15.112:6789/0 conn(0x7f9bec0c5d70 legacy=0x7f9bec156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:58.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.675+0000 7f9bdbfff640 1 --2- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 0x7f9bcc005b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:58.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.675+0000 7f9bdbfff640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 0x7f9bcc008670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:58.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.675+0000 7f9bdbfff640 1 -- 172.21.15.112:0/3655332938 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9bec0ca9b0 con 0x7f9bcc0061d0 2024-04-25T01:33:58.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.675+0000 7f9bdbfff640 1 -- 172.21.15.112:0/3655332938 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9bcc008d30 con 0x7f9bcc003710 2024-04-25T01:33:58.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.675+0000 7f9be9d9b640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 0x7f9bcc008670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:58.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.675+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 0x7f9bcc005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:58.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.676+0000 7f9be9d9b640 1 -- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 msgr2=0x7f9bcc008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:33:58.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.676+0000 7f9bea59c640 1 -- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 msgr2=0x7f9bcc005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:33:58.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.676+0000 7f9be9d9b640 1 -- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 msgr2=0x7f9bcc008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:58.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.676+0000 7f9bea59c640 1 -- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 msgr2=0x7f9bcc005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:58.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.676+0000 7f9be9d9b640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 0x7f9bcc008670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:58.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.676+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 0x7f9bcc005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:58.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.676+0000 7f9be9d9b640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 0x7f9bcc008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:58.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.676+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 0x7f9bcc005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:33:58.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.877+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 0x7f9bcc005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:58.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.877+0000 7f9be9d9b640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 0x7f9bcc008670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:58.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.877+0000 7f9be9d9b640 1 -- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 msgr2=0x7f9bcc008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:33:58.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.877+0000 7f9be9d9b640 1 -- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 msgr2=0x7f9bcc008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:58.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.877+0000 7f9be9d9b640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 0x7f9bcc008670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:58.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.877+0000 7f9be9d9b640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 0x7f9bcc008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:33:58.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.877+0000 7f9bea59c640 1 -- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 msgr2=0x7f9bcc005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:33:58.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.877+0000 7f9bea59c640 1 -- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 msgr2=0x7f9bcc005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:58.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.877+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 0x7f9bcc005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:58.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:58.877+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 0x7f9bcc005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:33:59.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:59.278+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 0x7f9bcc005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:59.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:59.278+0000 7f9be9d9b640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 0x7f9bcc008670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:59.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:59.278+0000 7f9bea59c640 1 -- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 msgr2=0x7f9bcc005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:33:59.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:59.278+0000 7f9bea59c640 1 -- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 msgr2=0x7f9bcc005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:59.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:59.278+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 0x7f9bcc005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:59.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:59.278+0000 7f9be9d9b640 1 -- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 msgr2=0x7f9bcc008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:33:59.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:59.278+0000 7f9be9d9b640 1 -- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 msgr2=0x7f9bcc008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:33:59.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:59.278+0000 7f9be9d9b640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 0x7f9bcc008670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:33:59.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:59.278+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 0x7f9bcc005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:33:59.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:33:59.278+0000 7f9be9d9b640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 0x7f9bcc008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:34:00.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:00.079+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 0x7f9bcc005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:00.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:00.079+0000 7f9be9d9b640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 0x7f9bcc008670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:00.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:00.079+0000 7f9be9d9b640 1 -- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 msgr2=0x7f9bcc008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:34:00.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:00.079+0000 7f9be9d9b640 1 -- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 msgr2=0x7f9bcc008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:34:00.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:00.079+0000 7f9be9d9b640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 0x7f9bcc008670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:34:00.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:00.079+0000 7f9bea59c640 1 -- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 msgr2=0x7f9bcc005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:34:00.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:00.079+0000 7f9bea59c640 1 -- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 msgr2=0x7f9bcc005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:34:00.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:00.079+0000 7f9be9d9b640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 0x7f9bcc008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:34:00.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:00.079+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 0x7f9bcc005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:34:00.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:00.079+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 0x7f9bcc005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:34:01.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.675+0000 7f9bdbfff640 1 -- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 msgr2=0x7f9bcc005b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:34:01.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.675+0000 7f9bdbfff640 1 --2- 172.21.15.112:0/3655332938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc003710 0x7f9bcc005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:01.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.675+0000 7f9bdbfff640 1 -- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 msgr2=0x7f9bcc008670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:34:01.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.675+0000 7f9bdbfff640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 0x7f9bcc008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:01.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.675+0000 7f9bead9d640 1 -- 172.21.15.112:0/3655332938 reap_dead start 2024-04-25T01:34:01.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.675+0000 7f9bdbfff640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 0x7f9bcc0085f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:01.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.675+0000 7f9bdbfff640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bcc00a8f0 0x7f9bcc00cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:01.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.675+0000 7f9bdbfff640 1 -- 172.21.15.112:0/3655332938 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bec0c9380 con 0x7f9bcc003b30 2024-04-25T01:34:01.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.675+0000 7f9bdbfff640 1 -- 172.21.15.112:0/3655332938 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f9bec0ca9b0 con 0x7f9bcc00a8f0 2024-04-25T01:34:01.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.675+0000 7f9bdbfff640 1 -- 172.21.15.112:0/3655332938 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9bcc008d30 con 0x7f9bcc0061d0 2024-04-25T01:34:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.676+0000 7f9bead9d640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.112:3300/0 conn(0x7f9bcc00a8f0 0x7f9bcc00cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.676+0000 7f9bea59c640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 0x7f9bcc0085f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.676+0000 7f9bead9d640 1 -- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 msgr2=0x7f9bcc0085f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:34:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.676+0000 7f9bead9d640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 0x7f9bcc0085f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.676+0000 7f9bead9d640 1 -- 172.21.15.112:0/3655332938 >> v1:172.21.15.112:6789/0 conn(0x7f9bcc003b30 legacy=0x7f9bcc0091d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.676+0000 7f9bead9d640 1 -- 172.21.15.112:0/3655332938 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bcc00e510 con 0x7f9bcc00a8f0 2024-04-25T01:34:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.676+0000 7f9be959a640 1 -- 172.21.15.112:0/3655332938 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2925669111 0 0) 0x7f9bec0c9380 con 0x7f9bcc003b30 2024-04-25T01:34:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.677+0000 7f9bead9d640 1 --2- 172.21.15.112:0/3655332938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bcc00a8f0 0x7f9bcc00cd60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0adcd0 tx=0x7f9be402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6221e2a634c822e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.677+0000 7f9be959a640 1 -- 172.21.15.112:0/3655332938 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be402aae0 con 0x7f9bcc00a8f0 2024-04-25T01:34:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.677+0000 7f9be959a640 1 -- 172.21.15.112:0/3655332938 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be402e070 con 0x7f9bcc00a8f0 2024-04-25T01:34:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.677+0000 7f9be959a640 1 -- 172.21.15.112:0/3655332938 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4007070 con 0x7f9bcc00a8f0 2024-04-25T01:34:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.677+0000 7f9bf0b70640 1 -- 172.21.15.112:0/3655332938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bcc00a8f0 msgr2=0x7f9bcc00cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.677+0000 7f9bf0b70640 1 --2- 172.21.15.112:0/3655332938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bcc00a8f0 0x7f9bcc00cd60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0adcd0 tx=0x7f9be402a1e0 comp rx=0 tx=0).stop 2024-04-25T01:34:01.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.678+0000 7f9bf0b70640 1 -- 172.21.15.112:0/3655332938 shutdown_connections 2024-04-25T01:34:01.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.678+0000 7f9bf0b70640 1 --2- 172.21.15.112:0/3655332938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bcc00a8f0 0x7f9bcc00cd60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:01.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.678+0000 7f9bf0b70640 1 --2- 172.21.15.112:0/3655332938 >> v2:172.21.15.187:3300/0 conn(0x7f9bcc0061d0 0x7f9bcc0085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:01.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.678+0000 7f9bf0b70640 1 -- 172.21.15.112:0/3655332938 >> 172.21.15.112:0/3655332938 conn(0x7f9bec0acc70 msgr2=0x7f9bec0ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:01.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.678+0000 7f9bf0b70640 1 -- 172.21.15.112:0/3655332938 shutdown_connections 2024-04-25T01:34:01.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.678+0000 7f9bf0b70640 1 -- 172.21.15.112:0/3655332938 wait complete. 2024-04-25T01:34:01.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.679+0000 7f9bf0b70640 1 Processor -- start 2024-04-25T01:34:01.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.679+0000 7f9bf0b70640 1 -- start start 2024-04-25T01:34:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.680+0000 7f9bf0b70640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bcc003b30 0x7f9bec155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.680+0000 7f9bf0b70640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bcc0061d0 0x7f9bec1ec610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.680+0000 7f9bf0b70640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc00a8f0 0x7f9bec1ecb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.680+0000 7f9bf0b70640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec157100 con 0x7f9bcc0061d0 2024-04-25T01:34:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.680+0000 7f9bf0b70640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec0caee0 con 0x7f9bcc003b30 2024-04-25T01:34:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.680+0000 7f9bf0b70640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9bec157280 con 0x7f9bcc00a8f0 2024-04-25T01:34:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.680+0000 7f9be9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bcc0061d0 0x7f9bec1ec610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.680+0000 7f9be9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bcc0061d0 0x7f9bec1ec610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50482/0 (socket says 172.21.15.112:50482) 2024-04-25T01:34:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.680+0000 7f9be9d9b640 1 -- 172.21.15.112:0/29050267 learned_addr learned my addr 172.21.15.112:0/29050267 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.680+0000 7f9bea59c640 1 --2- 172.21.15.112:0/29050267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bcc003b30 0x7f9bec155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.680+0000 7f9bead9d640 1 --2- 172.21.15.112:0/29050267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc00a8f0 0x7f9bec1ecb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.680+0000 7f9bead9d640 1 -- 172.21.15.112:0/29050267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc00a8f0 msgr2=0x7f9bec1ecb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-25T01:34:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.680+0000 7f9bead9d640 1 -- 172.21.15.112:0/29050267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc00a8f0 msgr2=0x7f9bec1ecb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:34:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.680+0000 7f9bead9d640 1 --2- 172.21.15.112:0/29050267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc00a8f0 0x7f9bec1ecb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:34:01.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.680+0000 7f9bead9d640 1 --2- 172.21.15.112:0/29050267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc00a8f0 0x7f9bec1ecb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:34:01.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.680+0000 7f9be9d9b640 1 -- 172.21.15.112:0/29050267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc00a8f0 msgr2=0x7f9bec1ecb50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:34:01.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.680+0000 7f9be9d9b640 1 --2- 172.21.15.112:0/29050267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc00a8f0 0x7f9bec1ecb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:01.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.680+0000 7f9be9d9b640 1 -- 172.21.15.112:0/29050267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bcc003b30 msgr2=0x7f9bec155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:01.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.680+0000 7f9be9d9b640 1 --2- 172.21.15.112:0/29050267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bcc003b30 0x7f9bec155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:01.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.680+0000 7f9be9d9b640 1 -- 172.21.15.112:0/29050267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bec1ee890 con 0x7f9bcc0061d0 2024-04-25T01:34:01.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.681+0000 7f9be9d9b640 1 --2- 172.21.15.112:0/29050267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bcc0061d0 0x7f9bec1ec610 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9be0000f60 tx=0x7f9be0001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:01.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.681+0000 7f9bdaffd640 1 -- 172.21.15.112:0/29050267 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be001a070 con 0x7f9bcc0061d0 2024-04-25T01:34:01.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.681+0000 7f9bdaffd640 1 -- 172.21.15.112:0/29050267 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be0016450 con 0x7f9bcc0061d0 2024-04-25T01:34:01.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.681+0000 7f9bf0b70640 1 -- 172.21.15.112:0/29050267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bec1eeb70 con 0x7f9bcc0061d0 2024-04-25T01:34:01.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.681+0000 7f9bdaffd640 1 -- 172.21.15.112:0/29050267 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0016820 con 0x7f9bcc0061d0 2024-04-25T01:34:01.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.681+0000 7f9bf0b70640 1 -- 172.21.15.112:0/29050267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bec1ef130 con 0x7f9bcc0061d0 2024-04-25T01:34:01.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.682+0000 7f9bf0b70640 1 -- 172.21.15.112:0/29050267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bec0682a0 con 0x7f9bcc0061d0 2024-04-25T01:34:01.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.682+0000 7f9bdaffd640 1 -- 172.21.15.112:0/29050267 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9be001d070 con 0x7f9bcc0061d0 2024-04-25T01:34:01.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.683+0000 7f9bdaffd640 1 --2- 172.21.15.112:0/29050267 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9bcc06d5c0 0x7f9bcc06fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:01.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.683+0000 7f9bdaffd640 1 -- 172.21.15.112:0/29050267 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f9be008b6d0 con 0x7f9bcc0061d0 2024-04-25T01:34:01.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.683+0000 7f9bea59c640 1 --2- 172.21.15.112:0/29050267 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9bcc06d5c0 0x7f9bcc06fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:01.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.683+0000 7f9bea59c640 1 --2- 172.21.15.112:0/29050267 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9bcc06d5c0 0x7f9bcc06fa80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc006fd0 tx=0x7f9bdc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:01.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.688+0000 7f9bdaffd640 1 -- 172.21.15.112:0/29050267 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9be0059690 con 0x7f9bcc0061d0 2024-04-25T01:34:01.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.984+0000 7f9bf0b70640 1 -- 172.21.15.112:0/29050267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9bec0c5ac0 con 0x7f9bcc0061d0 2024-04-25T01:34:02.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.984+0000 7f9bdaffd640 1 -- 172.21.15.112:0/29050267 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1568 (secure 0 0 0) 0x7f9be005d990 con 0x7f9bcc0061d0 2024-04-25T01:34:02.079 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":66,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:34:02.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.986+0000 7f9bf0b70640 1 -- 172.21.15.112:0/29050267 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9bcc06d5c0 msgr2=0x7f9bcc06fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.986+0000 7f9bf0b70640 1 --2- 172.21.15.112:0/29050267 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9bcc06d5c0 0x7f9bcc06fa80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc006fd0 tx=0x7f9bdc008040 comp rx=0 tx=0).stop 2024-04-25T01:34:02.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.986+0000 7f9bf0b70640 1 -- 172.21.15.112:0/29050267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bcc0061d0 msgr2=0x7f9bec1ec610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.986+0000 7f9bf0b70640 1 --2- 172.21.15.112:0/29050267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bcc0061d0 0x7f9bec1ec610 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9be0000f60 tx=0x7f9be0001630 comp rx=0 tx=0).stop 2024-04-25T01:34:02.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.986+0000 7f9bf0b70640 1 -- 172.21.15.112:0/29050267 shutdown_connections 2024-04-25T01:34:02.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.986+0000 7f9bf0b70640 1 --2- 172.21.15.112:0/29050267 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9bcc06d5c0 0x7f9bcc06fa80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.986+0000 7f9bf0b70640 1 --2- 172.21.15.112:0/29050267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bcc00a8f0 0x7f9bec1ecb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.986+0000 7f9bf0b70640 1 --2- 172.21.15.112:0/29050267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bcc0061d0 0x7f9bec1ec610 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.986+0000 7f9bf0b70640 1 --2- 172.21.15.112:0/29050267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bcc003b30 0x7f9bec155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.986+0000 7f9bf0b70640 1 -- 172.21.15.112:0/29050267 >> 172.21.15.112:0/29050267 conn(0x7f9bec0acc70 msgr2=0x7f9bec0c6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:02.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.986+0000 7f9bf0b70640 1 -- 172.21.15.112:0/29050267 shutdown_connections 2024-04-25T01:34:02.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:01.986+0000 7f9bf0b70640 1 -- 172.21.15.112:0/29050267 wait complete. 2024-04-25T01:34:02.082 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:34:02.082 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:34:02.082 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:34:02.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.218+0000 7f5c683c8640 1 Processor -- start 2024-04-25T01:34:02.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.218+0000 7f5c683c8640 1 -- start start 2024-04-25T01:34:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.219+0000 7f5c683c8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c601596a0 0x7f5c60152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.219+0000 7f5c683c8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5c600c72b0 0x7f5c60160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.219+0000 7f5c683c8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c60159b90 con 0x7f5c601592c0 2024-04-25T01:34:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.219+0000 7f5c683c8640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f5c600c6e20 con 0x7f5c601596a0 2024-04-25T01:34:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.219+0000 7f5c683c8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5c6006c970 con 0x7f5c600c72b0 2024-04-25T01:34:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.219+0000 7f5c65963640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c601596a0 0x7f5c60152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.219+0000 7f5c65963640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c601596a0 0x7f5c60152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50492/0 (socket says 172.21.15.112:50492) 2024-04-25T01:34:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.219+0000 7f5c65963640 1 -- 172.21.15.112:0/4185134609 learned_addr learned my addr 172.21.15.112:0/4185134609 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.219+0000 7f5c66965640 1 --2- 172.21.15.112:0/4185134609 >> v2:172.21.15.187:3300/0 conn(0x7f5c600c72b0 0x7f5c60160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.219+0000 7f5c65963640 1 -- 172.21.15.112:0/4185134609 >> v2:172.21.15.187:3300/0 conn(0x7f5c600c72b0 msgr2=0x7f5c60160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:34:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.219+0000 7f5c65963640 1 --2- 172.21.15.112:0/4185134609 >> v2:172.21.15.187:3300/0 conn(0x7f5c600c72b0 0x7f5c60160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.219+0000 7f5c65963640 1 -- 172.21.15.112:0/4185134609 >> v1:172.21.15.187:6789/0 conn(0x7f5c601592c0 legacy=0x7f5c600c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.219+0000 7f5c65963640 1 -- 172.21.15.112:0/4185134609 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6006cc50 con 0x7f5c601596a0 2024-04-25T01:34:02.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.220+0000 7f5c65963640 1 --2- 172.21.15.112:0/4185134609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c601596a0 0x7f5c60152d00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5c600c21d0 tx=0x7f5c5002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ef014aeec268c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:02.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.220+0000 7f5c65162640 1 -- 172.21.15.112:0/4185134609 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50037070 con 0x7f5c601596a0 2024-04-25T01:34:02.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.220+0000 7f5c65162640 1 -- 172.21.15.112:0/4185134609 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c50032440 con 0x7f5c601596a0 2024-04-25T01:34:02.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.220+0000 7f5c65162640 1 -- 172.21.15.112:0/4185134609 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50032810 con 0x7f5c601596a0 2024-04-25T01:34:02.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.221+0000 7f5c683c8640 1 -- 172.21.15.112:0/4185134609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c601596a0 msgr2=0x7f5c60152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.221+0000 7f5c683c8640 1 --2- 172.21.15.112:0/4185134609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c601596a0 0x7f5c60152d00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5c600c21d0 tx=0x7f5c5002a150 comp rx=0 tx=0).stop 2024-04-25T01:34:02.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.221+0000 7f5c683c8640 1 -- 172.21.15.112:0/4185134609 shutdown_connections 2024-04-25T01:34:02.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.221+0000 7f5c683c8640 1 --2- 172.21.15.112:0/4185134609 >> v2:172.21.15.187:3300/0 conn(0x7f5c600c72b0 0x7f5c60160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.221+0000 7f5c683c8640 1 --2- 172.21.15.112:0/4185134609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c601596a0 0x7f5c60152d00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.221+0000 7f5c683c8640 1 -- 172.21.15.112:0/4185134609 >> 172.21.15.112:0/4185134609 conn(0x7f5c600cb170 msgr2=0x7f5c600cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:02.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.221+0000 7f5c683c8640 1 -- 172.21.15.112:0/4185134609 shutdown_connections 2024-04-25T01:34:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.221+0000 7f5c683c8640 1 -- 172.21.15.112:0/4185134609 wait complete. 2024-04-25T01:34:02.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.222+0000 7f5c683c8640 1 Processor -- start 2024-04-25T01:34:02.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.222+0000 7f5c683c8640 1 -- start start 2024-04-25T01:34:02.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c683c8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c600c72b0 0x7f5c601ef050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c683c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c601592c0 0x7f5c60155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c683c8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c601596a0 0x7f5c60156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c683c8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6006c510 con 0x7f5c601592c0 2024-04-25T01:34:02.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c683c8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6006c390 con 0x7f5c600c72b0 2024-04-25T01:34:02.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c683c8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5c6006db20 con 0x7f5c601596a0 2024-04-25T01:34:02.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c65963640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c601592c0 0x7f5c60155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c65963640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c601592c0 0x7f5c60155e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50508/0 (socket says 172.21.15.112:50508) 2024-04-25T01:34:02.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c65963640 1 -- 172.21.15.112:0/1811360461 learned_addr learned my addr 172.21.15.112:0/1811360461 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:02.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c66164640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c600c72b0 0x7f5c601ef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c66965640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c601596a0 0x7f5c60156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c65963640 1 -- 172.21.15.112:0/1811360461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c601596a0 msgr2=0x7f5c60156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c65963640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c601596a0 0x7f5c60156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c65963640 1 -- 172.21.15.112:0/1811360461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c600c72b0 msgr2=0x7f5c601ef050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c65963640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c600c72b0 0x7f5c601ef050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c65963640 1 -- 172.21.15.112:0/1811360461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c601568c0 con 0x7f5c601592c0 2024-04-25T01:34:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c65963640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c601592c0 0x7f5c60155e40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5c50000c00 tx=0x7f5c500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c56ffd640 1 -- 172.21.15.112:0/1811360461 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50041030 con 0x7f5c601592c0 2024-04-25T01:34:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c56ffd640 1 -- 172.21.15.112:0/1811360461 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c50030440 con 0x7f5c601592c0 2024-04-25T01:34:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c683c8640 1 -- 172.21.15.112:0/1811360461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c601efb40 con 0x7f5c601592c0 2024-04-25T01:34:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.223+0000 7f5c56ffd640 1 -- 172.21.15.112:0/1811360461 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50037040 con 0x7f5c601592c0 2024-04-25T01:34:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.224+0000 7f5c683c8640 1 -- 172.21.15.112:0/1811360461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c600bff90 con 0x7f5c601592c0 2024-04-25T01:34:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.224+0000 7f5c683c8640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c601ef590 0x7f5c601e0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.224+0000 7f5c683c8640 1 -- 172.21.15.112:0/1811360461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c600c5610 con 0x7f5c601ef590 2024-04-25T01:34:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.225+0000 7f5c66164640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c601ef590 0x7f5c601e0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.225+0000 7f5c66164640 1 -- 172.21.15.112:0/1811360461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5c60075980 con 0x7f5c601ef590 2024-04-25T01:34:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.225+0000 7f5c66164640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c601ef590 0x7f5c601e0350 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00f570 tx=0x7f5c5c00fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.225+0000 7f5c56ffd640 1 -- 172.21.15.112:0/1811360461 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c019070 con 0x7f5c601ef590 2024-04-25T01:34:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.225+0000 7f5c56ffd640 1 -- 172.21.15.112:0/1811360461 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c50044070 con 0x7f5c601592c0 2024-04-25T01:34:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.226+0000 7f5c56ffd640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c4406bd30 0x7f5c4406e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.226+0000 7f5c56ffd640 1 -- 172.21.15.112:0/1811360461 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5c5c005320 con 0x7f5c601ef590 2024-04-25T01:34:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.226+0000 7f5c56ffd640 1 -- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c601ef590 msgr2=0x7f5c601e0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.226+0000 7f5c66965640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c4406bd30 0x7f5c4406e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.226+0000 7f5c56ffd640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c601ef590 0x7f5c601e0350 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00f570 tx=0x7f5c5c00fa30 comp rx=0 tx=0).stop 2024-04-25T01:34:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.226+0000 7f5c56ffd640 1 -- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c601ef590 msgr2=0x7f5c601e0350 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:34:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.226+0000 7f5c56ffd640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c601ef590 0x7f5c601e0350 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00f570 tx=0x7f5c5c00fa30 comp rx=0 tx=0).stop 2024-04-25T01:34:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.226+0000 7f5c56ffd640 1 -- 172.21.15.112:0/1811360461 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f5c500ab730 con 0x7f5c601592c0 2024-04-25T01:34:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.226+0000 7f5c66965640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c4406bd30 0x7f5c4406e1f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5c580113e0 tx=0x7f5c5800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:02.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.234+0000 7f5c683c8640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c60151460 0x7f5c601e0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.234+0000 7f5c683c8640 1 -- 172.21.15.112:0/1811360461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c600c0300 con 0x7f5c60151460 2024-04-25T01:34:02.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.234+0000 7f5c66164640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c60151460 0x7f5c601e0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.234+0000 7f5c66164640 1 -- 172.21.15.112:0/1811360461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5c600679a0 con 0x7f5c60151460 2024-04-25T01:34:02.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.235+0000 7f5c66164640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c60151460 0x7f5c601e0890 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00f570 tx=0x7f5c5c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:02.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.235+0000 7f5c56ffd640 1 -- 172.21.15.112:0/1811360461 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c006ce0 con 0x7f5c60151460 2024-04-25T01:34:02.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.235+0000 7f5c56ffd640 1 -- 172.21.15.112:0/1811360461 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4731 (secure 0 0 0) 0x7f5c5c006ce0 con 0x7f5c60151460 2024-04-25T01:34:02.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.235+0000 7f5c56ffd640 1 -- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c60151460 msgr2=0x7f5c601e0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.235+0000 7f5c56ffd640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c60151460 0x7f5c601e0890 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00f570 tx=0x7f5c5c000ed0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.235+0000 7f5c56ffd640 1 -- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c60151460 msgr2=0x7f5c601e0890 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:34:02.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.235+0000 7f5c56ffd640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c60151460 0x7f5c601e0890 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.213 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:34:02.213 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:34:02.213 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:34:02.213 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:34:02.213 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 66, 2024-04-25T01:34:02.213 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 123000, 2024-04-25T01:34:02.213 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:34:02.213 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:34:02.213 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:34:02.214 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:34:02.214 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:02.214 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:34:02.214 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:34:02.214 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:34:02.214 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:34:02.214 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:02.214 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:02.214 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:02.214 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:02.214 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:02.215 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:02.215 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:02.215 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:02.215 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:02.215 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:02.215 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:02.215 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:02.215 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:34:02.215 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:34:02.215 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:02.215 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:02.215 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:02.215 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:02.215 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:02.216 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:02.216 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:02.216 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:02.216 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:02.216 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:02.217 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:02.217 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:02.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.217 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:34:02.217 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:34:02.217 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:34:02.217 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:34:02.217 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:34:02.217 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:34:02.217 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:34:02.218 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:34:02.218 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:34:02.218 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:34:02.218 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:34:02.218 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:34:02.218 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:34:02.218 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:34:02.218 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:34:02.218 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:34:02.218 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:34:02.218 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:02.218 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:02.219 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:02.219 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:02.219 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:02.219 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:02.219 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:02.219 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:02.219 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:02.219 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:02.219 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:02.219 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:02.219 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:34:02.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.219 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:34:02.219 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.219 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:34:02.220 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:34:02.220 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:02.220 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:02.220 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.220 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:02.220 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:34:02.220 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:02.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.220 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.220 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:02.220 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:34:02.220 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:02.220 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.220 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:02.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.221 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:34:02.221 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:34:02.222 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:02.222 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:02.222 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:02.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.222 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.222 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:34:02.222 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:34:02.222 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:02.222 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:02.222 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.222 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:02.222 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:34:02.222 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:02.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.223 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.223 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:02.223 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:34:02.223 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:02.223 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.223 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:02.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.223 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:34:02.223 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:34:02.223 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:02.223 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:02.223 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:02.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.223 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.224 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:34:02.224 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:34:02.224 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:02.224 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:02.224 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.224 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:02.224 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:34:02.224 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:02.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.224 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.224 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:02.224 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:34:02.224 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:02.225 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.225 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:02.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.225 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:34:02.225 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:34:02.225 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:02.225 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:02.225 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:02.225 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.226 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:02.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.226 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:34:02.226 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:34:02.226 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.226 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:02.226 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:02.226 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:34:02.226 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.226 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:02.226 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:34:02.226 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.226 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:02.226 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:02.226 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:34:02.227 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.227 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:02.227 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:34:02.227 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.227 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:02.227 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:02.227 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:34:02.227 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.227 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:02.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.227 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:34:02.227 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:34:02.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.235+0000 7f5c683c8640 1 -- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c4406bd30 msgr2=0x7f5c4406e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.235+0000 7f5c683c8640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c4406bd30 0x7f5c4406e1f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5c580113e0 tx=0x7f5c5800b040 comp rx=0 tx=0).stop 2024-04-25T01:34:02.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.235+0000 7f5c683c8640 1 -- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c601592c0 msgr2=0x7f5c60155e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.235+0000 7f5c683c8640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c601592c0 0x7f5c60155e40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5c50000c00 tx=0x7f5c500046c0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.235+0000 7f5c66965640 1 -- 172.21.15.112:0/1811360461 reap_dead start 2024-04-25T01:34:02.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.235+0000 7f5c683c8640 1 -- 172.21.15.112:0/1811360461 shutdown_connections 2024-04-25T01:34:02.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.235+0000 7f5c683c8640 1 --2- 172.21.15.112:0/1811360461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c601592c0 0x7f5c60155e40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.235+0000 7f5c683c8640 1 -- 172.21.15.112:0/1811360461 >> 172.21.15.112:0/1811360461 conn(0x7f5c600cb170 msgr2=0x7f5c60067110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:02.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.235+0000 7f5c683c8640 1 -- 172.21.15.112:0/1811360461 shutdown_connections 2024-04-25T01:34:02.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.235+0000 7f5c683c8640 1 -- 172.21.15.112:0/1811360461 wait complete. 2024-04-25T01:34:02.229 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:34:02.230 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:34:02.230 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:34:02.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.366+0000 7f2f727ce640 1 Processor -- start 2024-04-25T01:34:02.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.367+0000 7f2f727ce640 1 -- start start 2024-04-25T01:34:02.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.367+0000 7f2f727ce640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2f6c1592c0 0x7f2f6c0c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.367+0000 7f2f727ce640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2f6c1596a0 0x7f2f6c152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.367+0000 7f2f727ce640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f6c0c72b0 0x7f2f6c155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.367+0000 7f2f727ce640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f2f6c0c6e20 con 0x7f2f6c1592c0 2024-04-25T01:34:02.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.367+0000 7f2f727ce640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2f6c159b90 con 0x7f2f6c1596a0 2024-04-25T01:34:02.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.367+0000 7f2f727ce640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6c06c970 con 0x7f2f6c0c72b0 2024-04-25T01:34:02.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.367+0000 7f2f6bfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2f6c1592c0 0x7f2f6c0c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.367+0000 7f2f6bfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2f6c1592c0 0x7f2f6c0c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50514/0 (socket says 172.21.15.112:50514) 2024-04-25T01:34:02.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.367+0000 7f2f6bfff640 1 -- 172.21.15.112:0/1729077081 learned_addr learned my addr 172.21.15.112:0/1729077081 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:02.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.368+0000 7f2f6b7fe640 1 --2- 172.21.15.112:0/1729077081 >> v2:172.21.15.187:3300/0 conn(0x7f2f6c1596a0 0x7f2f6c152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.368+0000 7f2f70d6b640 1 --2- 172.21.15.112:0/1729077081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f6c0c72b0 0x7f2f6c155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.368+0000 7f2f6bfff640 1 -- 172.21.15.112:0/1729077081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f6c0c72b0 msgr2=0x7f2f6c155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:34:02.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.368+0000 7f2f6bfff640 1 --2- 172.21.15.112:0/1729077081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f6c0c72b0 0x7f2f6c155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.368+0000 7f2f6bfff640 1 -- 172.21.15.112:0/1729077081 >> v2:172.21.15.187:3300/0 conn(0x7f2f6c1596a0 msgr2=0x7f2f6c152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:34:02.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.368+0000 7f2f6bfff640 1 --2- 172.21.15.112:0/1729077081 >> v2:172.21.15.187:3300/0 conn(0x7f2f6c1596a0 0x7f2f6c152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.368+0000 7f2f6bfff640 1 -- 172.21.15.112:0/1729077081 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c06cc50 con 0x7f2f6c1592c0 2024-04-25T01:34:02.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.369+0000 7f2f6bfff640 1 --2- 172.21.15.112:0/1729077081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c1592c0 0x7f2f6c0c9070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0bea40 tx=0x7f2f5c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b04844b9f1f3e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:02.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.369+0000 7f2f6affd640 1 -- 172.21.15.112:0/1729077081 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c037070 con 0x7f2f6c1592c0 2024-04-25T01:34:02.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.369+0000 7f2f6affd640 1 -- 172.21.15.112:0/1729077081 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f5c02ae00 con 0x7f2f6c1592c0 2024-04-25T01:34:02.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.369+0000 7f2f6affd640 1 -- 172.21.15.112:0/1729077081 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c032660 con 0x7f2f6c1592c0 2024-04-25T01:34:02.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.369+0000 7f2f727ce640 1 -- 172.21.15.112:0/1729077081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c1592c0 msgr2=0x7f2f6c0c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.369+0000 7f2f727ce640 1 --2- 172.21.15.112:0/1729077081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c1592c0 0x7f2f6c0c9070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0bea40 tx=0x7f2f5c02a0b0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.369+0000 7f2f727ce640 1 -- 172.21.15.112:0/1729077081 shutdown_connections 2024-04-25T01:34:02.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.369+0000 7f2f727ce640 1 --2- 172.21.15.112:0/1729077081 >> v2:172.21.15.187:3300/0 conn(0x7f2f6c1596a0 0x7f2f6c152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.369+0000 7f2f727ce640 1 --2- 172.21.15.112:0/1729077081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f6c0c72b0 0x7f2f6c155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.369+0000 7f2f727ce640 1 --2- 172.21.15.112:0/1729077081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c1592c0 0x7f2f6c0c9070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.369+0000 7f2f727ce640 1 -- 172.21.15.112:0/1729077081 >> 172.21.15.112:0/1729077081 conn(0x7f2f6c0cb170 msgr2=0x7f2f6c0cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:02.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.369+0000 7f2f727ce640 1 -- 172.21.15.112:0/1729077081 shutdown_connections 2024-04-25T01:34:02.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.369+0000 7f2f727ce640 1 -- 172.21.15.112:0/1729077081 wait complete. 2024-04-25T01:34:02.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.371+0000 7f2f727ce640 1 Processor -- start 2024-04-25T01:34:02.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.371+0000 7f2f727ce640 1 -- start start 2024-04-25T01:34:02.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.371+0000 7f2f727ce640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c0c72b0 0x7f2f6c14b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.371+0000 7f2f727ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c1592c0 0x7f2f6c1ee230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.371+0000 7f2f727ce640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f6c1596a0 0x7f2f6c1ee770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.371+0000 7f2f727ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c06dca0 con 0x7f2f6c1592c0 2024-04-25T01:34:02.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.371+0000 7f2f727ce640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c06db20 con 0x7f2f6c0c72b0 2024-04-25T01:34:02.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.371+0000 7f2f727ce640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6c0c95b0 con 0x7f2f6c1596a0 2024-04-25T01:34:02.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.372+0000 7f2f6b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c1592c0 0x7f2f6c1ee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.372+0000 7f2f6bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c0c72b0 0x7f2f6c14b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.372+0000 7f2f6b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c1592c0 0x7f2f6c1ee230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50530/0 (socket says 172.21.15.112:50530) 2024-04-25T01:34:02.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.372+0000 7f2f6b7fe640 1 -- 172.21.15.112:0/1336242935 learned_addr learned my addr 172.21.15.112:0/1336242935 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:02.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.372+0000 7f2f70d6b640 1 --2- 172.21.15.112:0/1336242935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f6c1596a0 0x7f2f6c1ee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.372+0000 7f2f6bfff640 1 -- 172.21.15.112:0/1336242935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f6c1596a0 msgr2=0x7f2f6c1ee770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.372+0000 7f2f6bfff640 1 --2- 172.21.15.112:0/1336242935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f6c1596a0 0x7f2f6c1ee770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.372+0000 7f2f6bfff640 1 -- 172.21.15.112:0/1336242935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c1592c0 msgr2=0x7f2f6c1ee230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.372+0000 7f2f6bfff640 1 --2- 172.21.15.112:0/1336242935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c1592c0 0x7f2f6c1ee230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.372+0000 7f2f6bfff640 1 -- 172.21.15.112:0/1336242935 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c1efa70 con 0x7f2f6c0c72b0 2024-04-25T01:34:02.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.372+0000 7f2f6b7fe640 1 --2- 172.21.15.112:0/1336242935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c1592c0 0x7f2f6c1ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:34:02.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.372+0000 7f2f6bfff640 1 --2- 172.21.15.112:0/1336242935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c0c72b0 0x7f2f6c14b530 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c031000 tx=0x7f2f5c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:02.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.372+0000 7f2f68ff9640 1 -- 172.21.15.112:0/1336242935 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c041070 con 0x7f2f6c0c72b0 2024-04-25T01:34:02.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.372+0000 7f2f68ff9640 1 -- 172.21.15.112:0/1336242935 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f5c03c460 con 0x7f2f6c0c72b0 2024-04-25T01:34:02.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.372+0000 7f2f727ce640 1 -- 172.21.15.112:0/1336242935 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6c158230 con 0x7f2f6c0c72b0 2024-04-25T01:34:02.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.372+0000 7f2f68ff9640 1 -- 172.21.15.112:0/1336242935 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c037040 con 0x7f2f6c0c72b0 2024-04-25T01:34:02.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.372+0000 7f2f727ce640 1 -- 172.21.15.112:0/1336242935 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f6c1587f0 con 0x7f2f6c0c72b0 2024-04-25T01:34:02.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.373+0000 7f2f727ce640 1 --2- 172.21.15.112:0/1336242935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c14fe90 0x7f2f6c1ebbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.373+0000 7f2f727ce640 1 -- 172.21.15.112:0/1336242935 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c1ef820 con 0x7f2f6c14fe90 2024-04-25T01:34:02.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.374+0000 7f2f6b7fe640 1 --2- 172.21.15.112:0/1336242935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c14fe90 0x7f2f6c1ebbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.374+0000 7f2f6b7fe640 1 -- 172.21.15.112:0/1336242935 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2f6c1efd50 con 0x7f2f6c14fe90 2024-04-25T01:34:02.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.374+0000 7f2f6b7fe640 1 --2- 172.21.15.112:0/1336242935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c14fe90 0x7f2f6c1ebbf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2f60011c40 tx=0x7f2f6000f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:02.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.374+0000 7f2f68ff9640 1 -- 172.21.15.112:0/1336242935 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f5c03a070 con 0x7f2f6c0c72b0 2024-04-25T01:34:02.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.374+0000 7f2f68ff9640 1 --2- 172.21.15.112:0/1336242935 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2f5806be30 0x7f2f5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.375+0000 7f2f68ff9640 1 -- 172.21.15.112:0/1336242935 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f2f5c0ab520 con 0x7f2f6c0c72b0 2024-04-25T01:34:02.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.375+0000 7f2f70d6b640 1 --2- 172.21.15.112:0/1336242935 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2f5806be30 0x7f2f5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.375+0000 7f2f68ff9640 1 -- 172.21.15.112:0/1336242935 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f60019070 con 0x7f2f6c14fe90 2024-04-25T01:34:02.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.375+0000 7f2f68ff9640 1 -- 172.21.15.112:0/1336242935 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2f60004030 con 0x7f2f6c14fe90 2024-04-25T01:34:02.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.375+0000 7f2f68ff9640 1 -- 172.21.15.112:0/1336242935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c14fe90 msgr2=0x7f2f6c1ebbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.375+0000 7f2f68ff9640 1 --2- 172.21.15.112:0/1336242935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c14fe90 0x7f2f6c1ebbf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2f60011c40 tx=0x7f2f6000f610 comp rx=0 tx=0).stop 2024-04-25T01:34:02.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.375+0000 7f2f68ff9640 1 -- 172.21.15.112:0/1336242935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c14fe90 msgr2=0x7f2f6c1ebbf0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:34:02.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.375+0000 7f2f68ff9640 1 --2- 172.21.15.112:0/1336242935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c14fe90 0x7f2f6c1ebbf0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.375+0000 7f2f70d6b640 1 --2- 172.21.15.112:0/1336242935 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2f5806be30 0x7f2f5806e2f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2f640113e0 tx=0x7f2f6400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:02.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.385+0000 7f2f727ce640 1 --2- 172.21.15.112:0/1336242935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c1ec130 0x7f2f6c156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.385+0000 7f2f727ce640 1 -- 172.21.15.112:0/1336242935 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c0679a0 con 0x7f2f6c1ec130 2024-04-25T01:34:02.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.385+0000 7f2f6b7fe640 1 --2- 172.21.15.112:0/1336242935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c1ec130 0x7f2f6c156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.385+0000 7f2f6b7fe640 1 -- 172.21.15.112:0/1336242935 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2f6c0682a0 con 0x7f2f6c1ec130 2024-04-25T01:34:02.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.385+0000 7f2f6b7fe640 1 --2- 172.21.15.112:0/1336242935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c1ec130 0x7f2f6c156e50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2f60011c40 tx=0x7f2f600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:02.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.386+0000 7f2f68ff9640 1 -- 172.21.15.112:0/1336242935 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f60019040 con 0x7f2f6c1ec130 2024-04-25T01:34:02.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.386+0000 7f2f68ff9640 1 -- 172.21.15.112:0/1336242935 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5069 (secure 0 0 0) 0x7f2f6001e660 con 0x7f2f6c1ec130 2024-04-25T01:34:02.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.386+0000 7f2f68ff9640 1 -- 172.21.15.112:0/1336242935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c1ec130 msgr2=0x7f2f6c156e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.386+0000 7f2f68ff9640 1 --2- 172.21.15.112:0/1336242935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c1ec130 0x7f2f6c156e50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2f60011c40 tx=0x7f2f600046c0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.386+0000 7f2f68ff9640 1 -- 172.21.15.112:0/1336242935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c1ec130 msgr2=0x7f2f6c156e50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:34:02.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.386+0000 7f2f68ff9640 1 --2- 172.21.15.112:0/1336242935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c1ec130 0x7f2f6c156e50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.363 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:34:02.363 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:34:02.364 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:34:02.364 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:34:02.364 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 66, 2024-04-25T01:34:02.364 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 278075, 2024-04-25T01:34:02.364 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:34:02.364 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:34:02.364 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:34:02.364 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:34:02.364 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:02.364 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:34:02.364 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:34:02.364 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:34:02.364 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:34:02.364 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:02.365 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:02.365 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:02.365 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:02.365 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:02.365 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:02.365 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:02.365 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:02.365 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:02.365 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:02.365 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:02.365 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:02.365 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:34:02.365 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:34:02.365 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:02.365 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:02.365 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:02.366 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:02.366 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:02.366 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:02.366 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:02.366 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:02.366 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:02.366 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:02.366 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:02.366 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:02.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.366 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:34:02.366 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:34:02.366 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:34:02.366 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:34:02.367 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:34:02.367 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:34:02.367 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:34:02.367 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:34:02.367 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:34:02.367 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:34:02.367 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:34:02.367 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:34:02.367 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:34:02.367 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:34:02.367 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:34:02.367 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:34:02.367 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:34:02.367 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:02.367 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:02.368 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:02.368 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:02.368 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:02.368 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:02.368 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:02.368 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:02.368 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:02.368 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:02.368 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:02.368 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:02.368 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:34:02.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.368 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:34:02.368 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.368 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:34:02.369 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:34:02.369 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:02.369 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:02.369 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.369 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:02.369 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:34:02.369 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:02.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.369 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.369 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:02.369 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:34:02.369 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:02.369 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.369 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:02.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.369 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:34:02.370 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:34:02.370 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:02.370 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:02.370 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:02.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.370 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.370 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:34:02.370 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:34:02.370 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:02.370 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:02.370 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.370 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:02.370 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:34:02.370 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:02.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.371 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.371 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:02.371 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:34:02.371 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:02.371 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.371 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:02.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.371 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:34:02.371 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:34:02.371 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:02.371 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:02.371 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:02.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.371 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.372 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:34:02.372 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:34:02.372 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:02.372 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:02.372 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.372 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:02.372 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:34:02.372 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:02.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.372 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.372 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:02.372 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:34:02.372 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:02.372 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.372 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:02.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.373 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:34:02.373 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:34:02.373 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:02.373 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:02.373 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:02.373 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.373 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:02.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.373 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:34:02.373 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:34:02.373 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.373 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:02.373 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:02.373 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:34:02.373 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.374 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:02.374 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:34:02.374 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.374 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:02.374 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:02.374 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:34:02.374 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.374 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:02.374 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:34:02.374 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.374 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:02.374 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:02.374 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:34:02.374 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.375 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:02.375 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:34:02.375 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.375 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:02.375 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:02.375 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 6 2024-04-25T01:34:02.375 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.375 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:02.375 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:34:02.375 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.375 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:02.375 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:02.375 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:34:02.375 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.375 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:02.376 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.376 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:34:02.376 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:34:02.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.386+0000 7f2f727ce640 1 -- 172.21.15.112:0/1336242935 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2f5806be30 msgr2=0x7f2f5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.386+0000 7f2f727ce640 1 --2- 172.21.15.112:0/1336242935 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2f5806be30 0x7f2f5806e2f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2f640113e0 tx=0x7f2f6400b040 comp rx=0 tx=0).stop 2024-04-25T01:34:02.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.386+0000 7f2f727ce640 1 -- 172.21.15.112:0/1336242935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c0c72b0 msgr2=0x7f2f6c14b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.386+0000 7f2f727ce640 1 --2- 172.21.15.112:0/1336242935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c0c72b0 0x7f2f6c14b530 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c031000 tx=0x7f2f5c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.386+0000 7f2f70d6b640 1 -- 172.21.15.112:0/1336242935 reap_dead start 2024-04-25T01:34:02.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.386+0000 7f2f70d6b640 1 -- 172.21.15.112:0/1336242935 reap_dead start 2024-04-25T01:34:02.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.386+0000 7f2f727ce640 1 -- 172.21.15.112:0/1336242935 shutdown_connections 2024-04-25T01:34:02.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.386+0000 7f2f727ce640 1 -- 172.21.15.112:0/1336242935 >> 172.21.15.112:0/1336242935 conn(0x7f2f6c0cb170 msgr2=0x7f2f6c0ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:02.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.387+0000 7f2f727ce640 1 -- 172.21.15.112:0/1336242935 shutdown_connections 2024-04-25T01:34:02.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.387+0000 7f2f727ce640 1 -- 172.21.15.112:0/1336242935 wait complete. 2024-04-25T01:34:02.378 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:34:02.378 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:34:02.378 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:34:02.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.512+0000 7f0ce41e7640 1 Processor -- start 2024-04-25T01:34:02.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.512+0000 7f0ce41e7640 1 -- start start 2024-04-25T01:34:02.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.513+0000 7f0ce41e7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0cdc06e210 0x7f0cdc0c7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.513+0000 7f0ce41e7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0cdc0c10c0 0x7f0cdc152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.513+0000 7f0ce41e7640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cdc155ac0 con 0x7f0cdc1531f0 2024-04-25T01:34:02.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.513+0000 7f0ce41e7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f0cdc0c2ad0 con 0x7f0cdc06e210 2024-04-25T01:34:02.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.513+0000 7f0ce41e7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0cdc156880 con 0x7f0cdc0c10c0 2024-04-25T01:34:02.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.513+0000 7f0ce1f83640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0cdc06e210 0x7f0cdc0c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.513+0000 7f0ce1f83640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0cdc06e210 0x7f0cdc0c7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50536/0 (socket says 172.21.15.112:50536) 2024-04-25T01:34:02.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.513+0000 7f0ce1f83640 1 -- 172.21.15.112:0/3012429398 learned_addr learned my addr 172.21.15.112:0/3012429398 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:02.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.513+0000 7f0ce1782640 1 --2- 172.21.15.112:0/3012429398 >> v2:172.21.15.187:3300/0 conn(0x7f0cdc0c10c0 0x7f0cdc152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.513+0000 7f0ce1f83640 1 -- 172.21.15.112:0/3012429398 >> v2:172.21.15.187:3300/0 conn(0x7f0cdc0c10c0 msgr2=0x7f0cdc152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:34:02.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.513+0000 7f0ce1f83640 1 --2- 172.21.15.112:0/3012429398 >> v2:172.21.15.187:3300/0 conn(0x7f0cdc0c10c0 0x7f0cdc152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.513+0000 7f0ce1f83640 1 -- 172.21.15.112:0/3012429398 >> v1:172.21.15.187:6789/0 conn(0x7f0cdc1531f0 legacy=0x7f0cdc158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.513+0000 7f0ce1f83640 1 -- 172.21.15.112:0/3012429398 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cdc155da0 con 0x7f0cdc06e210 2024-04-25T01:34:02.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.513+0000 7f0ce1782640 1 --2- 172.21.15.112:0/3012429398 >> v2:172.21.15.187:3300/0 conn(0x7f0cdc0c10c0 0x7f0cdc152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:34:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.514+0000 7f0ce1f83640 1 --2- 172.21.15.112:0/3012429398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cdc06e210 0x7f0cdc0c7810 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc06fd10 tx=0x7f0cd802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cb7f5a9cfec499 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.515+0000 7f0ce0f81640 1 -- 172.21.15.112:0/3012429398 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8036070 con 0x7f0cdc06e210 2024-04-25T01:34:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.515+0000 7f0ce0f81640 1 -- 172.21.15.112:0/3012429398 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cd8031600 con 0x7f0cdc06e210 2024-04-25T01:34:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.515+0000 7f0ce0f81640 1 -- 172.21.15.112:0/3012429398 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd80319d0 con 0x7f0cdc06e210 2024-04-25T01:34:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.515+0000 7f0ce41e7640 1 -- 172.21.15.112:0/3012429398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cdc06e210 msgr2=0x7f0cdc0c7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.515+0000 7f0ce41e7640 1 --2- 172.21.15.112:0/3012429398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cdc06e210 0x7f0cdc0c7810 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc06fd10 tx=0x7f0cd802a730 comp rx=0 tx=0).stop 2024-04-25T01:34:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.515+0000 7f0ce41e7640 1 -- 172.21.15.112:0/3012429398 shutdown_connections 2024-04-25T01:34:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.516+0000 7f0ce41e7640 1 --2- 172.21.15.112:0/3012429398 >> v2:172.21.15.187:3300/0 conn(0x7f0cdc0c10c0 0x7f0cdc152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.516+0000 7f0ce41e7640 1 --2- 172.21.15.112:0/3012429398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cdc06e210 0x7f0cdc0c7810 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.516+0000 7f0ce41e7640 1 -- 172.21.15.112:0/3012429398 >> 172.21.15.112:0/3012429398 conn(0x7f0cdc0a9670 msgr2=0x7f0cdc0a9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.516+0000 7f0ce41e7640 1 -- 172.21.15.112:0/3012429398 shutdown_connections 2024-04-25T01:34:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.516+0000 7f0ce41e7640 1 -- 172.21.15.112:0/3012429398 wait complete. 2024-04-25T01:34:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.517+0000 7f0ce41e7640 1 Processor -- start 2024-04-25T01:34:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.517+0000 7f0ce41e7640 1 -- start start 2024-04-25T01:34:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.518+0000 7f0ce41e7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cdc06e210 0x7f0cdc0c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.518+0000 7f0ce41e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cdc0c10c0 0x7f0cdc14b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.518+0000 7f0ce41e7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cdc1531f0 0x7f0cdc14b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.518+0000 7f0ce41e7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc0c7d50 con 0x7f0cdc0c10c0 2024-04-25T01:34:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.518+0000 7f0ce41e7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc1563f0 con 0x7f0cdc06e210 2024-04-25T01:34:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.518+0000 7f0ce41e7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0cdc06e5f0 con 0x7f0cdc1531f0 2024-04-25T01:34:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.518+0000 7f0ce1782640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cdc0c10c0 0x7f0cdc14b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.518+0000 7f0ce1782640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cdc0c10c0 0x7f0cdc14b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50538/0 (socket says 172.21.15.112:50538) 2024-04-25T01:34:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.518+0000 7f0ce1782640 1 -- 172.21.15.112:0/1810669511 learned_addr learned my addr 172.21.15.112:0/1810669511 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.518+0000 7f0ce1f83640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cdc06e210 0x7f0cdc0c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.518+0000 7f0ce2784640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cdc1531f0 0x7f0cdc14b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.518+0000 7f0ce1782640 1 -- 172.21.15.112:0/1810669511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cdc1531f0 msgr2=0x7f0cdc14b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.518+0000 7f0ce1782640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cdc1531f0 0x7f0cdc14b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.518+0000 7f0ce1782640 1 -- 172.21.15.112:0/1810669511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cdc06e210 msgr2=0x7f0cdc0c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.518+0000 7f0ce1782640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cdc06e210 0x7f0cdc0c6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.518+0000 7f0ce1782640 1 -- 172.21.15.112:0/1810669511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cdc14bc20 con 0x7f0cdc0c10c0 2024-04-25T01:34:02.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.519+0000 7f0ce1782640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cdc0c10c0 0x7f0cdc14b1a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00ebe0 tx=0x7f0ccc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:02.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.519+0000 7f0cd27fc640 1 -- 172.21.15.112:0/1810669511 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc017070 con 0x7f0cdc0c10c0 2024-04-25T01:34:02.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.519+0000 7f0cd27fc640 1 -- 172.21.15.112:0/1810669511 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ccc00cc30 con 0x7f0cdc0c10c0 2024-04-25T01:34:02.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.519+0000 7f0cd27fc640 1 -- 172.21.15.112:0/1810669511 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc013470 con 0x7f0cdc0c10c0 2024-04-25T01:34:02.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.519+0000 7f0ce41e7640 1 -- 172.21.15.112:0/1810669511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cdc1504a0 con 0x7f0cdc0c10c0 2024-04-25T01:34:02.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.519+0000 7f0ce41e7640 1 -- 172.21.15.112:0/1810669511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cdc072340 con 0x7f0cdc0c10c0 2024-04-25T01:34:02.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.520+0000 7f0ce41e7640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cdc14be60 0x7f0cdc072680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.520+0000 7f0ce41e7640 1 -- 172.21.15.112:0/1810669511 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0cdc075980 con 0x7f0cdc14be60 2024-04-25T01:34:02.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.520+0000 7f0ce1f83640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cdc14be60 0x7f0cdc072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.520+0000 7f0ce1f83640 1 -- 172.21.15.112:0/1810669511 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0cdc0682a0 con 0x7f0cdc14be60 2024-04-25T01:34:02.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.520+0000 7f0cd27fc640 1 -- 172.21.15.112:0/1810669511 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ccc01a070 con 0x7f0cdc0c10c0 2024-04-25T01:34:02.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.521+0000 7f0ce1f83640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cdc14be60 0x7f0cdc072680 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc06cdd0 tx=0x7f0cd8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:02.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.521+0000 7f0cd27fc640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0cc006be30 0x7f0cc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.521+0000 7f0ce2784640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0cc006be30 0x7f0cc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.521+0000 7f0cd27fc640 1 -- 172.21.15.112:0/1810669511 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f0ccc08b070 con 0x7f0cdc0c10c0 2024-04-25T01:34:02.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.521+0000 7f0cd27fc640 1 -- 172.21.15.112:0/1810669511 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8036040 con 0x7f0cdc14be60 2024-04-25T01:34:02.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.521+0000 7f0cd27fc640 1 -- 172.21.15.112:0/1810669511 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0cd8037040 con 0x7f0cdc14be60 2024-04-25T01:34:02.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.521+0000 7f0cd27fc640 1 -- 172.21.15.112:0/1810669511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cdc14be60 msgr2=0x7f0cdc072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.521+0000 7f0cd27fc640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cdc14be60 0x7f0cdc072680 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc06cdd0 tx=0x7f0cd8004a40 comp rx=0 tx=0).stop 2024-04-25T01:34:02.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.521+0000 7f0cd27fc640 1 -- 172.21.15.112:0/1810669511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cdc14be60 msgr2=0x7f0cdc072680 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:34:02.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.521+0000 7f0cd27fc640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cdc14be60 0x7f0cdc072680 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc06cdd0 tx=0x7f0cd8004a40 comp rx=0 tx=0).stop 2024-04-25T01:34:02.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.522+0000 7f0ce2784640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0cc006be30 0x7f0cc006e2f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4000f60 tx=0x7f0cd4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:02.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.528+0000 7f0ce41e7640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cdc1eb070 0x7f0cdc1eb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.529+0000 7f0ce41e7640 1 -- 172.21.15.112:0/1810669511 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0cdc067760 con 0x7f0cdc1eb070 2024-04-25T01:34:02.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.529+0000 7f0ce1f83640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cdc1eb070 0x7f0cdc1eb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.529+0000 7f0ce1f83640 1 -- 172.21.15.112:0/1810669511 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0cdc067480 con 0x7f0cdc1eb070 2024-04-25T01:34:02.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.529+0000 7f0ce1f83640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cdc1eb070 0x7f0cdc1eb450 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0cd802a770 tx=0x7f0cd802f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:02.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.529+0000 7f0cd27fc640 1 -- 172.21.15.112:0/1810669511 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8036040 con 0x7f0cdc1eb070 2024-04-25T01:34:02.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.530+0000 7f0cd27fc640 1 -- 172.21.15.112:0/1810669511 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4558 (secure 0 0 0) 0x7f0cd8048e50 con 0x7f0cdc1eb070 2024-04-25T01:34:02.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.530+0000 7f0cd27fc640 1 -- 172.21.15.112:0/1810669511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cdc1eb070 msgr2=0x7f0cdc1eb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.530+0000 7f0cd27fc640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cdc1eb070 0x7f0cdc1eb450 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0cd802a770 tx=0x7f0cd802f000 comp rx=0 tx=0).stop 2024-04-25T01:34:02.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.530+0000 7f0cd27fc640 1 -- 172.21.15.112:0/1810669511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cdc1eb070 msgr2=0x7f0cdc1eb450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:34:02.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.530+0000 7f0cd27fc640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0cdc1eb070 0x7f0cdc1eb450 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0cd802a770 tx=0x7f0cd802f000 comp rx=0 tx=0).stop 2024-04-25T01:34:02.507 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:34:02.507 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:34:02.507 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:34:02.507 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:34:02.507 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 66, 2024-04-25T01:34:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 10348, 2024-04-25T01:34:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:34:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:34:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:34:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:34:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:34:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:34:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:34:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:34:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:02.509 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:02.509 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:02.509 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:02.509 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:02.509 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:02.509 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:02.509 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:02.509 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:34:02.509 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:34:02.509 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:02.509 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:02.509 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:02.509 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:02.509 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:02.509 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:02.510 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:02.510 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:02.510 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:02.510 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:02.510 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:02.510 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:02.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.510 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:34:02.510 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:34:02.510 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:34:02.510 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:34:02.510 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:34:02.510 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:34:02.510 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:34:02.511 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:34:02.511 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:34:02.511 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:34:02.511 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:34:02.511 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:34:02.511 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:34:02.511 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:34:02.511 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:34:02.511 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:34:02.511 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:34:02.511 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:02.511 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:02.511 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:02.511 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:02.511 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:02.512 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:02.512 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:02.512 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:02.512 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:02.512 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:02.512 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:02.512 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:02.512 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:34:02.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.512 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:34:02.512 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.512 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:34:02.512 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:34:02.512 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:02.513 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:02.513 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.513 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:02.513 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:34:02.513 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:02.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.513 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.513 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:02.513 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:34:02.513 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:02.513 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.514 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:02.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.514 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:34:02.514 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:34:02.514 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:02.514 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:02.514 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:02.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.514 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.514 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:34:02.514 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:34:02.514 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:02.514 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:02.514 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.515 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:02.515 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:34:02.515 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:02.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.515 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.515 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:02.515 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:34:02.515 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:02.515 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.515 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:02.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.515 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:34:02.515 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:34:02.515 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:02.515 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:34:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:34:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:34:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:34:02.517 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:02.517 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.517 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:02.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.517 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:34:02.517 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:34:02.517 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:02.517 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:02.517 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:02.517 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.517 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:02.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.517 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:34:02.517 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:34:02.517 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.518 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:02.518 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:02.518 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:34:02.518 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.518 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:02.518 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:34:02.518 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:02.518 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:02.518 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:02.518 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:34:02.518 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:02.518 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:02.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:02.518 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:34:02.519 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:34:02.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.530+0000 7f0ce41e7640 1 -- 172.21.15.112:0/1810669511 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0cc006be30 msgr2=0x7f0cc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.530+0000 7f0ce41e7640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0cc006be30 0x7f0cc006e2f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4000f60 tx=0x7f0cd4009040 comp rx=0 tx=0).stop 2024-04-25T01:34:02.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.530+0000 7f0ce41e7640 1 -- 172.21.15.112:0/1810669511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cdc0c10c0 msgr2=0x7f0cdc14b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.530+0000 7f0ce41e7640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cdc0c10c0 0x7f0cdc14b1a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00ebe0 tx=0x7f0ccc00c520 comp rx=0 tx=0).stop 2024-04-25T01:34:02.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.530+0000 7f0ce2784640 1 -- 172.21.15.112:0/1810669511 reap_dead start 2024-04-25T01:34:02.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.530+0000 7f0ce41e7640 1 -- 172.21.15.112:0/1810669511 shutdown_connections 2024-04-25T01:34:02.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.530+0000 7f0ce41e7640 1 --2- 172.21.15.112:0/1810669511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cdc0c10c0 0x7f0cdc14b1a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.530+0000 7f0ce41e7640 1 -- 172.21.15.112:0/1810669511 >> 172.21.15.112:0/1810669511 conn(0x7f0cdc0a9670 msgr2=0x7f0cdc14c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:02.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.530+0000 7f0ce41e7640 1 -- 172.21.15.112:0/1810669511 shutdown_connections 2024-04-25T01:34:02.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.530+0000 7f0ce41e7640 1 -- 172.21.15.112:0/1810669511 wait complete. 2024-04-25T01:34:02.521 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:34:02.521 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T01:34:02.521 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T01:34:02.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.656+0000 7f0fb207d640 1 Processor -- start 2024-04-25T01:34:02.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.656+0000 7f0fb207d640 1 -- start start 2024-04-25T01:34:02.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.657+0000 7f0fb207d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0fac156290 0x7f0fac158e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.657+0000 7f0fb207d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fac1593d0 0x7f0fac152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.657+0000 7f0fb207d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fac1598c0 con 0x7f0fac0c42e0 2024-04-25T01:34:02.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.657+0000 7f0fb207d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f0fac159ba0 con 0x7f0fac156290 2024-04-25T01:34:02.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.657+0000 7f0fb207d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0fac155a90 con 0x7f0fac1593d0 2024-04-25T01:34:02.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.657+0000 7f0faaffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0fac156290 0x7f0fac158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.657+0000 7f0faaffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0fac156290 0x7f0fac158e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50548/0 (socket says 172.21.15.112:50548) 2024-04-25T01:34:02.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.657+0000 7f0faaffd640 1 -- 172.21.15.112:0/2388297610 learned_addr learned my addr 172.21.15.112:0/2388297610 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:02.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.657+0000 7f0fabfff640 1 --2- 172.21.15.112:0/2388297610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fac1593d0 0x7f0fac152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.657+0000 7f0faaffd640 1 -- 172.21.15.112:0/2388297610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fac1593d0 msgr2=0x7f0fac152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:34:02.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.657+0000 7f0faaffd640 1 --2- 172.21.15.112:0/2388297610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fac1593d0 0x7f0fac152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.657+0000 7f0faaffd640 1 -- 172.21.15.112:0/2388297610 >> v1:172.21.15.187:6789/0 conn(0x7f0fac0c42e0 legacy=0x7f0fac06dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.657+0000 7f0faaffd640 1 -- 172.21.15.112:0/2388297610 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fac155d70 con 0x7f0fac156290 2024-04-25T01:34:02.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.658+0000 7f0faaffd640 1 --2- 172.21.15.112:0/2388297610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fac156290 0x7f0fac158e90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009870 tx=0x7f0fa002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4017b72a2996b87b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:02.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.658+0000 7f0faa7fc640 1 -- 172.21.15.112:0/2388297610 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0037070 con 0x7f0fac156290 2024-04-25T01:34:02.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.658+0000 7f0faa7fc640 1 -- 172.21.15.112:0/2388297610 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa0032440 con 0x7f0fac156290 2024-04-25T01:34:02.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.658+0000 7f0faa7fc640 1 -- 172.21.15.112:0/2388297610 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0032810 con 0x7f0fac156290 2024-04-25T01:34:02.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.659+0000 7f0fb207d640 1 -- 172.21.15.112:0/2388297610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fac156290 msgr2=0x7f0fac158e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.659+0000 7f0fb207d640 1 --2- 172.21.15.112:0/2388297610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fac156290 0x7f0fac158e90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009870 tx=0x7f0fa002a270 comp rx=0 tx=0).stop 2024-04-25T01:34:02.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.659+0000 7f0fb207d640 1 -- 172.21.15.112:0/2388297610 shutdown_connections 2024-04-25T01:34:02.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.659+0000 7f0fb207d640 1 --2- 172.21.15.112:0/2388297610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fac1593d0 0x7f0fac152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.659+0000 7f0fb207d640 1 --2- 172.21.15.112:0/2388297610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fac156290 0x7f0fac158e90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.659+0000 7f0fb207d640 1 -- 172.21.15.112:0/2388297610 >> 172.21.15.112:0/2388297610 conn(0x7f0fac0c4a90 msgr2=0x7f0fac0c4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:02.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.659+0000 7f0fb207d640 1 -- 172.21.15.112:0/2388297610 shutdown_connections 2024-04-25T01:34:02.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.659+0000 7f0fb207d640 1 -- 172.21.15.112:0/2388297610 wait complete. 2024-04-25T01:34:02.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.660+0000 7f0fb207d640 1 Processor -- start 2024-04-25T01:34:02.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.660+0000 7f0fb207d640 1 -- start start 2024-04-25T01:34:02.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.661+0000 7f0fb207d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fac0c42e0 0x7f0fac145980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.661+0000 7f0fb207d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fac156290 0x7f0fac1489c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.661+0000 7f0fb207d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fac1593d0 0x7f0fac148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.661+0000 7f0fb207d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac06b420 con 0x7f0fac1593d0 2024-04-25T01:34:02.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.661+0000 7f0fb207d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac06b2a0 con 0x7f0fac0c42e0 2024-04-25T01:34:02.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.661+0000 7f0fb207d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0fac0c46c0 con 0x7f0fac156290 2024-04-25T01:34:02.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.661+0000 7f0fabfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fac1593d0 0x7f0fac148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.661+0000 7f0faaffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fac156290 0x7f0fac1489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.661+0000 7f0fab7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fac0c42e0 0x7f0fac145980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.661+0000 7f0fabfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fac1593d0 0x7f0fac148f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50556/0 (socket says 172.21.15.112:50556) 2024-04-25T01:34:02.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.661+0000 7f0fabfff640 1 -- 172.21.15.112:0/2012936770 learned_addr learned my addr 172.21.15.112:0/2012936770 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:02.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.662+0000 7f0fabfff640 1 -- 172.21.15.112:0/2012936770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fac156290 msgr2=0x7f0fac1489c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.662+0000 7f0fabfff640 1 --2- 172.21.15.112:0/2012936770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fac156290 0x7f0fac1489c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.662+0000 7f0fabfff640 1 -- 172.21.15.112:0/2012936770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fac0c42e0 msgr2=0x7f0fac145980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.662+0000 7f0fabfff640 1 --2- 172.21.15.112:0/2012936770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fac0c42e0 0x7f0fac145980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.662+0000 7f0fabfff640 1 -- 172.21.15.112:0/2012936770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fac1ec210 con 0x7f0fac1593d0 2024-04-25T01:34:02.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.662+0000 7f0fabfff640 1 --2- 172.21.15.112:0/2012936770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fac1593d0 0x7f0fac148f00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009d30 tx=0x7f0fa400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:02.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.662+0000 7f0f93fff640 1 -- 172.21.15.112:0/2012936770 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4018020 con 0x7f0fac1593d0 2024-04-25T01:34:02.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.662+0000 7f0f93fff640 1 -- 172.21.15.112:0/2012936770 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa4014480 con 0x7f0fac1593d0 2024-04-25T01:34:02.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.662+0000 7f0fb207d640 1 -- 172.21.15.112:0/2012936770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fac146f30 con 0x7f0fac1593d0 2024-04-25T01:34:02.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.662+0000 7f0f93fff640 1 -- 172.21.15.112:0/2012936770 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4014870 con 0x7f0fac1593d0 2024-04-25T01:34:02.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.662+0000 7f0fb207d640 1 -- 172.21.15.112:0/2012936770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fac14f840 con 0x7f0fac1593d0 2024-04-25T01:34:02.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.663+0000 7f0fb207d640 1 -- 172.21.15.112:0/2012936770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fac147450 con 0x7f0fac1593d0 2024-04-25T01:34:02.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.664+0000 7f0f93fff640 1 -- 172.21.15.112:0/2012936770 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0fa4005d10 con 0x7f0fac1593d0 2024-04-25T01:34:02.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.664+0000 7f0f93fff640 1 --2- 172.21.15.112:0/2012936770 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0f9806be60 0x7f0f9806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:02.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.664+0000 7f0f93fff640 1 -- 172.21.15.112:0/2012936770 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f0fa408ba00 con 0x7f0fac1593d0 2024-04-25T01:34:02.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.664+0000 7f0fab7fe640 1 --2- 172.21.15.112:0/2012936770 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0f9806be60 0x7f0f9806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:02.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.665+0000 7f0fab7fe640 1 --2- 172.21.15.112:0/2012936770 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0f9806be60 0x7f0f9806e320 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0fac153dd0 tx=0x7f0f9c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:02.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.669+0000 7f0f93fff640 1 -- 172.21.15.112:0/2012936770 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0fac147450 con 0x7f0fac1593d0 2024-04-25T01:34:02.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.964+0000 7f0fb207d640 1 -- 172.21.15.112:0/2012936770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f0fac0682a0 con 0x7f0fac1593d0 2024-04-25T01:34:02.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.965+0000 7f0f93fff640 1 -- 172.21.15.112:0/2012936770 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f0fa4022070 con 0x7f0fac1593d0 2024-04-25T01:34:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.968+0000 7f0fb207d640 1 -- 172.21.15.112:0/2012936770 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0f9806be60 msgr2=0x7f0f9806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.968+0000 7f0fb207d640 1 --2- 172.21.15.112:0/2012936770 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0f9806be60 0x7f0f9806e320 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0fac153dd0 tx=0x7f0f9c009040 comp rx=0 tx=0).stop 2024-04-25T01:34:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.968+0000 7f0fb207d640 1 -- 172.21.15.112:0/2012936770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fac1593d0 msgr2=0x7f0fac148f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.968+0000 7f0fb207d640 1 --2- 172.21.15.112:0/2012936770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fac1593d0 0x7f0fac148f00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009d30 tx=0x7f0fa400c490 comp rx=0 tx=0).stop 2024-04-25T01:34:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.968+0000 7f0fb207d640 1 -- 172.21.15.112:0/2012936770 shutdown_connections 2024-04-25T01:34:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.968+0000 7f0fb207d640 1 --2- 172.21.15.112:0/2012936770 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0f9806be60 0x7f0f9806e320 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.968+0000 7f0fb207d640 1 --2- 172.21.15.112:0/2012936770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fac1593d0 0x7f0fac148f00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.968+0000 7f0fb207d640 1 --2- 172.21.15.112:0/2012936770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fac156290 0x7f0fac1489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.968+0000 7f0fb207d640 1 --2- 172.21.15.112:0/2012936770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fac0c42e0 0x7f0fac145980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.968+0000 7f0fb207d640 1 -- 172.21.15.112:0/2012936770 >> 172.21.15.112:0/2012936770 conn(0x7f0fac0c4a90 msgr2=0x7f0fac06b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.968+0000 7f0fb207d640 1 -- 172.21.15.112:0/2012936770 shutdown_connections 2024-04-25T01:34:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:02.968+0000 7f0fb207d640 1 -- 172.21.15.112:0/2012936770 wait complete. 2024-04-25T01:34:02.956 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T01:34:12.957 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:34:12.958 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:34:13.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.093+0000 7f8caf841640 1 Processor -- start 2024-04-25T01:34:13.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.093+0000 7f8caf841640 1 -- start start 2024-04-25T01:34:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.094+0000 7f8caf841640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8ca80c5d70 0x7f8ca80c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.094+0000 7f8caf841640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8ca8156cf0 0x7f8ca806bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.094+0000 7f8caf841640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ca80c90e0 con 0x7f8ca8152fd0 2024-04-25T01:34:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.094+0000 7f8caf841640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f8ca80ca9b0 con 0x7f8ca8156cf0 2024-04-25T01:34:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.094+0000 7f8caf841640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8ca80cac00 con 0x7f8ca80c5d70 2024-04-25T01:34:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.094+0000 7f8caddde640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8ca8156cf0 0x7f8ca806bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.094+0000 7f8cad5dd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8ca8152fd0 0x7f8ca8156280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:43976/0 (socket says 172.21.15.112:43976) 2024-04-25T01:34:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.094+0000 7f8cad5dd640 1 -- 172.21.15.112:0/2538386140 learned_addr learned my addr 172.21.15.112:0/2538386140 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:34:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.094+0000 7f8cacddc640 1 --2- 172.21.15.112:0/2538386140 >> v2:172.21.15.187:3300/0 conn(0x7f8ca80c5d70 0x7f8ca80c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.094+0000 7f8caddde640 1 -- 172.21.15.112:0/2538386140 >> v2:172.21.15.187:3300/0 conn(0x7f8ca80c5d70 msgr2=0x7f8ca80c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:34:13.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.094+0000 7f8caddde640 1 --2- 172.21.15.112:0/2538386140 >> v2:172.21.15.187:3300/0 conn(0x7f8ca80c5d70 0x7f8ca80c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.094+0000 7f8caddde640 1 -- 172.21.15.112:0/2538386140 >> v1:172.21.15.112:6789/0 conn(0x7f8ca8152fd0 legacy=0x7f8ca8156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.094+0000 7f8c9ffff640 1 -- 172.21.15.112:0/2538386140 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3283012728 0 0) 0x7f8ca80c90e0 con 0x7f8ca8152fd0 2024-04-25T01:34:13.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.094+0000 7f8caddde640 1 -- 172.21.15.112:0/2538386140 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca80c9380 con 0x7f8ca8156cf0 2024-04-25T01:34:13.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.095+0000 7f8caddde640 1 --2- 172.21.15.112:0/2538386140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ca8156cf0 0x7f8ca806bda0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80bec40 tx=0x7f8ca002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf97742c623d2c14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:13.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.095+0000 7f8c9ffff640 1 -- 172.21.15.112:0/2538386140 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0007740 con 0x7f8ca8156cf0 2024-04-25T01:34:13.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.095+0000 7f8c9ffff640 1 -- 172.21.15.112:0/2538386140 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ca00078e0 con 0x7f8ca8156cf0 2024-04-25T01:34:13.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.095+0000 7f8c9ffff640 1 -- 172.21.15.112:0/2538386140 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0007cb0 con 0x7f8ca8156cf0 2024-04-25T01:34:13.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.096+0000 7f8caf841640 1 -- 172.21.15.112:0/2538386140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ca8156cf0 msgr2=0x7f8ca806bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.096+0000 7f8caf841640 1 --2- 172.21.15.112:0/2538386140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ca8156cf0 0x7f8ca806bda0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80bec40 tx=0x7f8ca002a5f0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.096+0000 7f8caf841640 1 -- 172.21.15.112:0/2538386140 shutdown_connections 2024-04-25T01:34:13.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.096+0000 7f8caf841640 1 --2- 172.21.15.112:0/2538386140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ca8156cf0 0x7f8ca806bda0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.096+0000 7f8caf841640 1 --2- 172.21.15.112:0/2538386140 >> v2:172.21.15.187:3300/0 conn(0x7f8ca80c5d70 0x7f8ca80c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.096+0000 7f8caf841640 1 -- 172.21.15.112:0/2538386140 >> 172.21.15.112:0/2538386140 conn(0x7f8ca80acc70 msgr2=0x7f8ca80ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:13.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.096+0000 7f8caf841640 1 -- 172.21.15.112:0/2538386140 shutdown_connections 2024-04-25T01:34:13.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.096+0000 7f8caf841640 1 -- 172.21.15.112:0/2538386140 wait complete. 2024-04-25T01:34:13.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.097+0000 7f8caf841640 1 Processor -- start 2024-04-25T01:34:13.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.097+0000 7f8caf841640 1 -- start start 2024-04-25T01:34:13.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.098+0000 7f8caf841640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ca80c5d70 0x7f8ca81ef020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.098+0000 7f8caf841640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ca8152fd0 0x7f8ca8151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.098+0000 7f8caf841640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ca8156cf0 0x7f8ca814e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.098+0000 7f8caf841640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca8157100 con 0x7f8ca8156cf0 2024-04-25T01:34:13.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.098+0000 7f8caf841640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca80caee0 con 0x7f8ca80c5d70 2024-04-25T01:34:13.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.098+0000 7f8caf841640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca8157280 con 0x7f8ca8152fd0 2024-04-25T01:34:13.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.098+0000 7f8caddde640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ca8156cf0 0x7f8ca814e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.098+0000 7f8caddde640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ca8156cf0 0x7f8ca814e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59526/0 (socket says 172.21.15.112:59526) 2024-04-25T01:34:13.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.098+0000 7f8caddde640 1 -- 172.21.15.112:0/1318669861 learned_addr learned my addr 172.21.15.112:0/1318669861 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:13.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.098+0000 7f8cad5dd640 1 --2- 172.21.15.112:0/1318669861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ca80c5d70 0x7f8ca81ef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.098+0000 7f8cacddc640 1 --2- 172.21.15.112:0/1318669861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ca8152fd0 0x7f8ca8151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.098+0000 7f8caddde640 1 -- 172.21.15.112:0/1318669861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ca8152fd0 msgr2=0x7f8ca8151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.098+0000 7f8caddde640 1 --2- 172.21.15.112:0/1318669861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ca8152fd0 0x7f8ca8151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.098+0000 7f8caddde640 1 -- 172.21.15.112:0/1318669861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ca80c5d70 msgr2=0x7f8ca81ef020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.098+0000 7f8caddde640 1 --2- 172.21.15.112:0/1318669861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ca80c5d70 0x7f8ca81ef020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.098+0000 7f8caddde640 1 -- 172.21.15.112:0/1318669861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca81548d0 con 0x7f8ca8156cf0 2024-04-25T01:34:13.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.098+0000 7f8caddde640 1 --2- 172.21.15.112:0/1318669861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ca8156cf0 0x7f8ca814e490 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009e80 tx=0x7f8ca0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:13.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.099+0000 7f8c9dffb640 1 -- 172.21.15.112:0/1318669861 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca003f070 con 0x7f8ca8156cf0 2024-04-25T01:34:13.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.099+0000 7f8c9dffb640 1 -- 172.21.15.112:0/1318669861 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ca0031c40 con 0x7f8ca8156cf0 2024-04-25T01:34:13.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.099+0000 7f8caf841640 1 -- 172.21.15.112:0/1318669861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca81ef560 con 0x7f8ca8156cf0 2024-04-25T01:34:13.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.099+0000 7f8c9dffb640 1 -- 172.21.15.112:0/1318669861 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0009450 con 0x7f8ca8156cf0 2024-04-25T01:34:13.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.099+0000 7f8caf841640 1 -- 172.21.15.112:0/1318669861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca81ebe90 con 0x7f8ca8156cf0 2024-04-25T01:34:13.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.100+0000 7f8caf841640 1 -- 172.21.15.112:0/1318669861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca80682a0 con 0x7f8ca8156cf0 2024-04-25T01:34:13.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.100+0000 7f8c9dffb640 1 -- 172.21.15.112:0/1318669861 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ca0003990 con 0x7f8ca8156cf0 2024-04-25T01:34:13.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.101+0000 7f8c9dffb640 1 --2- 172.21.15.112:0/1318669861 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8c8c06be30 0x7f8c8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.101+0000 7f8c9dffb640 1 -- 172.21.15.112:0/1318669861 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f8ca00abe70 con 0x7f8ca8156cf0 2024-04-25T01:34:13.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.101+0000 7f8cad5dd640 1 --2- 172.21.15.112:0/1318669861 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8c8c06be30 0x7f8c8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.101+0000 7f8cad5dd640 1 --2- 172.21.15.112:0/1318669861 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8c8c06be30 0x7f8c8c06e2f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4000f60 tx=0x7f8ca4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:13.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.106+0000 7f8c9dffb640 1 -- 172.21.15.112:0/1318669861 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8ca81548d0 con 0x7f8ca8156cf0 2024-04-25T01:34:13.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.448+0000 7f8caf841640 1 -- 172.21.15.112:0/1318669861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8ca80c5ac0 con 0x7f8ca8156cf0 2024-04-25T01:34:13.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.449+0000 7f8c9dffb640 1 -- 172.21.15.112:0/1318669861 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f8ca0079ec0 con 0x7f8ca8156cf0 2024-04-25T01:34:13.425 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":66,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:34:13.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.451+0000 7f8caf841640 1 -- 172.21.15.112:0/1318669861 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8c8c06be30 msgr2=0x7f8c8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.451+0000 7f8caf841640 1 --2- 172.21.15.112:0/1318669861 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8c8c06be30 0x7f8c8c06e2f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4000f60 tx=0x7f8ca4009040 comp rx=0 tx=0).stop 2024-04-25T01:34:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.451+0000 7f8caf841640 1 -- 172.21.15.112:0/1318669861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ca8156cf0 msgr2=0x7f8ca814e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.451+0000 7f8caf841640 1 --2- 172.21.15.112:0/1318669861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ca8156cf0 0x7f8ca814e490 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009e80 tx=0x7f8ca0004a40 comp rx=0 tx=0).stop 2024-04-25T01:34:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.451+0000 7f8caf841640 1 -- 172.21.15.112:0/1318669861 shutdown_connections 2024-04-25T01:34:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.451+0000 7f8caf841640 1 --2- 172.21.15.112:0/1318669861 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8c8c06be30 0x7f8c8c06e2f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.451+0000 7f8caf841640 1 --2- 172.21.15.112:0/1318669861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ca8156cf0 0x7f8ca814e490 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.451+0000 7f8caf841640 1 --2- 172.21.15.112:0/1318669861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ca8152fd0 0x7f8ca8151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.451+0000 7f8caf841640 1 --2- 172.21.15.112:0/1318669861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ca80c5d70 0x7f8ca81ef020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.451+0000 7f8caf841640 1 -- 172.21.15.112:0/1318669861 >> 172.21.15.112:0/1318669861 conn(0x7f8ca80acc70 msgr2=0x7f8ca80cb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.452+0000 7f8caf841640 1 -- 172.21.15.112:0/1318669861 shutdown_connections 2024-04-25T01:34:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.452+0000 7f8caf841640 1 -- 172.21.15.112:0/1318669861 wait complete. 2024-04-25T01:34:13.439 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:34:13.440 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T01:34:13.440 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:34:13.440 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:34:13.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.576+0000 7f9052361640 1 Processor -- start 2024-04-25T01:34:13.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.576+0000 7f9052361640 1 -- start start 2024-04-25T01:34:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.577+0000 7f9052361640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f904c152f80 0x7f904c158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.577+0000 7f9052361640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f904c072660 0x7f904c071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.577+0000 7f9052361640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f904c1561d0 0x7f904c0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.577+0000 7f9052361640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f904c072d10 con 0x7f904c1561d0 2024-04-25T01:34:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.577+0000 7f9052361640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f904c153360 con 0x7f904c072660 2024-04-25T01:34:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.577+0000 7f9052361640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f904c155a10 con 0x7f904c152f80 2024-04-25T01:34:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.577+0000 7f90508fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f904c1561d0 0x7f904c0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.577+0000 7f90508fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f904c1561d0 0x7f904c0c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59540/0 (socket says 172.21.15.112:59540) 2024-04-25T01:34:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.577+0000 7f90508fe640 1 -- 172.21.15.112:0/3299929509 learned_addr learned my addr 172.21.15.112:0/3299929509 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.577+0000 7f904bfff640 1 --2- 172.21.15.112:0/3299929509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f904c152f80 0x7f904c158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.577+0000 7f904b7fe640 1 --2- 172.21.15.112:0/3299929509 >> v2:172.21.15.187:3300/0 conn(0x7f904c072660 0x7f904c071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.577+0000 7f90508fe640 1 -- 172.21.15.112:0/3299929509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f904c152f80 msgr2=0x7f904c158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:34:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.577+0000 7f90508fe640 1 --2- 172.21.15.112:0/3299929509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f904c152f80 0x7f904c158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.577+0000 7f90508fe640 1 -- 172.21.15.112:0/3299929509 >> v2:172.21.15.187:3300/0 conn(0x7f904c072660 msgr2=0x7f904c071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:34:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.577+0000 7f90508fe640 1 --2- 172.21.15.112:0/3299929509 >> v2:172.21.15.187:3300/0 conn(0x7f904c072660 0x7f904c071e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.577+0000 7f90508fe640 1 -- 172.21.15.112:0/3299929509 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c0aa060 con 0x7f904c1561d0 2024-04-25T01:34:13.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.578+0000 7f90508fe640 1 --2- 172.21.15.112:0/3299929509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c1561d0 0x7f904c0c50a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f904c0c29d0 tx=0x7f904402a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb1c0a256f62ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:13.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.578+0000 7f904affd640 1 -- 172.21.15.112:0/3299929509 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044036070 con 0x7f904c1561d0 2024-04-25T01:34:13.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.578+0000 7f904affd640 1 -- 172.21.15.112:0/3299929509 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9044031600 con 0x7f904c1561d0 2024-04-25T01:34:13.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.578+0000 7f904affd640 1 -- 172.21.15.112:0/3299929509 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90440319d0 con 0x7f904c1561d0 2024-04-25T01:34:13.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.578+0000 7f9052361640 1 -- 172.21.15.112:0/3299929509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c1561d0 msgr2=0x7f904c0c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.578+0000 7f9052361640 1 --2- 172.21.15.112:0/3299929509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c1561d0 0x7f904c0c50a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f904c0c29d0 tx=0x7f904402a570 comp rx=0 tx=0).stop 2024-04-25T01:34:13.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.579+0000 7f9052361640 1 -- 172.21.15.112:0/3299929509 shutdown_connections 2024-04-25T01:34:13.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.579+0000 7f9052361640 1 --2- 172.21.15.112:0/3299929509 >> v2:172.21.15.187:3300/0 conn(0x7f904c072660 0x7f904c071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.579+0000 7f9052361640 1 --2- 172.21.15.112:0/3299929509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c1561d0 0x7f904c0c50a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.579+0000 7f9052361640 1 --2- 172.21.15.112:0/3299929509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f904c152f80 0x7f904c158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.579+0000 7f9052361640 1 -- 172.21.15.112:0/3299929509 >> 172.21.15.112:0/3299929509 conn(0x7f904c06c6c0 msgr2=0x7f904c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:13.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.579+0000 7f9052361640 1 -- 172.21.15.112:0/3299929509 shutdown_connections 2024-04-25T01:34:13.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.579+0000 7f9052361640 1 -- 172.21.15.112:0/3299929509 wait complete. 2024-04-25T01:34:13.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.580+0000 7f9052361640 1 Processor -- start 2024-04-25T01:34:13.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.580+0000 7f9052361640 1 -- start start 2024-04-25T01:34:13.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.580+0000 7f9052361640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f904c072660 0x7f904c14e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.581+0000 7f9052361640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f904c152f80 0x7f904c14ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.581+0000 7f9052361640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c1561d0 0x7f904c0c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.581+0000 7f9052361640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f904c0aa620 con 0x7f904c1561d0 2024-04-25T01:34:13.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.581+0000 7f9052361640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f904c155cb0 con 0x7f904c152f80 2024-04-25T01:34:13.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.581+0000 7f9052361640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f904c0aa7a0 con 0x7f904c072660 2024-04-25T01:34:13.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.581+0000 7f90508fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c1561d0 0x7f904c0c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.581+0000 7f90508fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c1561d0 0x7f904c0c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59550/0 (socket says 172.21.15.112:59550) 2024-04-25T01:34:13.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.581+0000 7f904b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f904c152f80 0x7f904c14ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.581+0000 7f90508fe640 1 -- 172.21.15.112:0/1244693438 learned_addr learned my addr 172.21.15.112:0/1244693438 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:13.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.581+0000 7f904bfff640 1 --2- 172.21.15.112:0/1244693438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f904c072660 0x7f904c14e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.581+0000 7f90508fe640 1 -- 172.21.15.112:0/1244693438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f904c072660 msgr2=0x7f904c14e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.581+0000 7f90508fe640 1 --2- 172.21.15.112:0/1244693438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f904c072660 0x7f904c14e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.581+0000 7f90508fe640 1 -- 172.21.15.112:0/1244693438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f904c152f80 msgr2=0x7f904c14ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.581+0000 7f90508fe640 1 --2- 172.21.15.112:0/1244693438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f904c152f80 0x7f904c14ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.581+0000 7f90508fe640 1 -- 172.21.15.112:0/1244693438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c152650 con 0x7f904c1561d0 2024-04-25T01:34:13.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.581+0000 7f90508fe640 1 --2- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c1561d0 0x7f904c0c5fc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f904402ab60 tx=0x7f9044004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:13.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.581+0000 7f9048ff9640 1 -- 172.21.15.112:0/1244693438 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904403f020 con 0x7f904c1561d0 2024-04-25T01:34:13.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.581+0000 7f9048ff9640 1 -- 172.21.15.112:0/1244693438 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9044037040 con 0x7f904c1561d0 2024-04-25T01:34:13.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.581+0000 7f9052361640 1 -- 172.21.15.112:0/1244693438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f904c1e2b60 con 0x7f904c1561d0 2024-04-25T01:34:13.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.581+0000 7f9052361640 1 -- 172.21.15.112:0/1244693438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f904c14fff0 con 0x7f904c1561d0 2024-04-25T01:34:13.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.582+0000 7f9048ff9640 1 -- 172.21.15.112:0/1244693438 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044036070 con 0x7f904c1561d0 2024-04-25T01:34:13.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.582+0000 7f9052361640 1 --2- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c1e25c0 0x7f904c157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.582+0000 7f9052361640 1 -- 172.21.15.112:0/1244693438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f904c075980 con 0x7f904c1e25c0 2024-04-25T01:34:13.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.583+0000 7f904bfff640 1 --2- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c1e25c0 0x7f904c157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.583+0000 7f904bfff640 1 -- 172.21.15.112:0/1244693438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f904c0689b0 con 0x7f904c1e25c0 2024-04-25T01:34:13.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.583+0000 7f904bfff640 1 --2- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c1e25c0 0x7f904c157930 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f903c011890 tx=0x7f903c011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:13.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.583+0000 7f9048ff9640 1 -- 172.21.15.112:0/1244693438 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c019070 con 0x7f904c1e25c0 2024-04-25T01:34:13.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.583+0000 7f9048ff9640 1 -- 172.21.15.112:0/1244693438 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9044042020 con 0x7f904c1561d0 2024-04-25T01:34:13.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.584+0000 7f9048ff9640 1 --2- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f903806bd30 0x7f903806e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.584+0000 7f9048ff9640 1 -- 172.21.15.112:0/1244693438 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f90440ab4a0 con 0x7f904c1561d0 2024-04-25T01:34:13.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.584+0000 7f904b7fe640 1 --2- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f903806bd30 0x7f903806e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.584+0000 7f9048ff9640 1 -- 172.21.15.112:0/1244693438 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f903c004240 con 0x7f904c1e25c0 2024-04-25T01:34:13.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.584+0000 7f9048ff9640 1 -- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c1e25c0 msgr2=0x7f904c157930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.584+0000 7f9048ff9640 1 --2- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c1e25c0 0x7f904c157930 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f903c011890 tx=0x7f903c011d50 comp rx=0 tx=0).stop 2024-04-25T01:34:13.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.584+0000 7f9048ff9640 1 -- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c1e25c0 msgr2=0x7f904c157930 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:34:13.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.584+0000 7f9048ff9640 1 --2- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c1e25c0 0x7f904c157930 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.584+0000 7f904b7fe640 1 --2- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f903806bd30 0x7f903806e1f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f904c1591c0 tx=0x7f9040009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:13.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.589+0000 7f9052361640 1 --2- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c157e70 0x7f904c158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.589+0000 7f9052361640 1 -- 172.21.15.112:0/1244693438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f904c157690 con 0x7f904c157e70 2024-04-25T01:34:13.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.589+0000 7f904bfff640 1 --2- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c157e70 0x7f904c158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.590+0000 7f904bfff640 1 -- 172.21.15.112:0/1244693438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f904c0c2590 con 0x7f904c157e70 2024-04-25T01:34:13.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.590+0000 7f904bfff640 1 --2- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c157e70 0x7f904c158250 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f903c00f6b0 tx=0x7f903c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:13.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.590+0000 7f9048ff9640 1 -- 172.21.15.112:0/1244693438 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c003df0 con 0x7f904c157e70 2024-04-25T01:34:13.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.590+0000 7f9048ff9640 1 -- 172.21.15.112:0/1244693438 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4732 (secure 0 0 0) 0x7f903c003df0 con 0x7f904c157e70 2024-04-25T01:34:13.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.590+0000 7f9048ff9640 1 -- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c157e70 msgr2=0x7f904c158250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.590+0000 7f9048ff9640 1 --2- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c157e70 0x7f904c158250 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f903c00f6b0 tx=0x7f903c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.590+0000 7f9048ff9640 1 -- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c157e70 msgr2=0x7f904c158250 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:34:13.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.590+0000 7f9048ff9640 1 --2- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c157e70 0x7f904c158250 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f903c00f6b0 tx=0x7f903c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.569 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:34:13.569 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:34:13.569 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:34:13.569 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:34:13.569 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 66, 2024-04-25T01:34:13.569 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 134356, 2024-04-25T01:34:13.569 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:34:13.569 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:34:13.569 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:34:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:34:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 12, 2024-04-25T01:34:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:34:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:34:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:34:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:34:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:34:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:34:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:34:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:34:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:34:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:34:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:34:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:34:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:34:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:34:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:34:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:34:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:34:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:34:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:34:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:34:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:34:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:34:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:34:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:34:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:34:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:34:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:34:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:34:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:34:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:34:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:34:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:34:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:34:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:34:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:34:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:34:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:34:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:34:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:34:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:34:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:34:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:34:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:34:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:34:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:34:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:34:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:34:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:34:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:34:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.581 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:34:13.581 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:34:13.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.591+0000 7f9052361640 1 -- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f903806bd30 msgr2=0x7f903806e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.591+0000 7f9052361640 1 --2- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f903806bd30 0x7f903806e1f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f904c1591c0 tx=0x7f9040009190 comp rx=0 tx=0).stop 2024-04-25T01:34:13.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.591+0000 7f9052361640 1 -- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c1561d0 msgr2=0x7f904c0c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.591+0000 7f9052361640 1 --2- 172.21.15.112:0/1244693438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f904c1561d0 0x7f904c0c5fc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f904402ab60 tx=0x7f9044004a40 comp rx=0 tx=0).stop 2024-04-25T01:34:13.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.591+0000 7f90508fe640 1 -- 172.21.15.112:0/1244693438 reap_dead start 2024-04-25T01:34:13.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.591+0000 7f90508fe640 1 -- 172.21.15.112:0/1244693438 reap_dead start 2024-04-25T01:34:13.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.591+0000 7f9052361640 1 -- 172.21.15.112:0/1244693438 shutdown_connections 2024-04-25T01:34:13.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.591+0000 7f9052361640 1 -- 172.21.15.112:0/1244693438 >> 172.21.15.112:0/1244693438 conn(0x7f904c06c6c0 msgr2=0x7f904c151e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:13.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.591+0000 7f9052361640 1 -- 172.21.15.112:0/1244693438 shutdown_connections 2024-04-25T01:34:13.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.591+0000 7f9052361640 1 -- 172.21.15.112:0/1244693438 wait complete. 2024-04-25T01:34:13.585 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:34:13.585 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:34:13.585 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:34:13.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.720+0000 7f9252eb2640 1 Processor -- start 2024-04-25T01:34:13.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.720+0000 7f9252eb2640 1 -- start start 2024-04-25T01:34:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.721+0000 7f9252eb2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f924c153280 0x7f924c0c1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.721+0000 7f9252eb2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f924c06f680 0x7f924c155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.721+0000 7f9252eb2640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f924c0a6f50 con 0x7f924c06fbb0 2024-04-25T01:34:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.721+0000 7f9252eb2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f924c0a6d00 con 0x7f924c153280 2024-04-25T01:34:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.721+0000 7f9252eb2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f924c159b90 con 0x7f924c06f680 2024-04-25T01:34:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.721+0000 7f9250c4e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f924c153280 0x7f924c0c1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.721+0000 7f925144f640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f924c06fbb0 0x7f924c0743d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:43980/0 (socket says 172.21.15.112:43980) 2024-04-25T01:34:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.722+0000 7f925144f640 1 -- 172.21.15.112:0/3438091157 learned_addr learned my addr 172.21.15.112:0/3438091157 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:34:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.722+0000 7f924bfff640 1 --2- 172.21.15.112:0/3438091157 >> v2:172.21.15.187:3300/0 conn(0x7f924c06f680 0x7f924c155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.722+0000 7f9250c4e640 1 -- 172.21.15.112:0/3438091157 >> v2:172.21.15.187:3300/0 conn(0x7f924c06f680 msgr2=0x7f924c155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:34:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.722+0000 7f9250c4e640 1 --2- 172.21.15.112:0/3438091157 >> v2:172.21.15.187:3300/0 conn(0x7f924c06f680 0x7f924c155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.722+0000 7f9250c4e640 1 -- 172.21.15.112:0/3438091157 >> v1:172.21.15.112:6789/0 conn(0x7f924c06fbb0 legacy=0x7f924c0743d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.722+0000 7f9250c4e640 1 -- 172.21.15.112:0/3438091157 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c074ae0 con 0x7f924c153280 2024-04-25T01:34:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.722+0000 7f924b7fe640 1 -- 172.21.15.112:0/3438091157 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3673263332 0 0) 0x7f924c0a6f50 con 0x7f924c06fbb0 2024-04-25T01:34:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.723+0000 7f9250c4e640 1 --2- 172.21.15.112:0/3438091157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c153280 0x7f924c0c1a00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f924c066f10 tx=0x7f924002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aeba32cdc9f9c76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.723+0000 7f924b7fe640 1 -- 172.21.15.112:0/3438091157 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240037070 con 0x7f924c153280 2024-04-25T01:34:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.723+0000 7f924b7fe640 1 -- 172.21.15.112:0/3438091157 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f924002add0 con 0x7f924c153280 2024-04-25T01:34:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.723+0000 7f924b7fe640 1 -- 172.21.15.112:0/3438091157 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240032660 con 0x7f924c153280 2024-04-25T01:34:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.723+0000 7f9252eb2640 1 -- 172.21.15.112:0/3438091157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c153280 msgr2=0x7f924c0c1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.723+0000 7f9252eb2640 1 --2- 172.21.15.112:0/3438091157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c153280 0x7f924c0c1a00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f924c066f10 tx=0x7f924002a080 comp rx=0 tx=0).stop 2024-04-25T01:34:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.723+0000 7f9252eb2640 1 -- 172.21.15.112:0/3438091157 shutdown_connections 2024-04-25T01:34:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.723+0000 7f9252eb2640 1 --2- 172.21.15.112:0/3438091157 >> v2:172.21.15.187:3300/0 conn(0x7f924c06f680 0x7f924c155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.723+0000 7f9252eb2640 1 --2- 172.21.15.112:0/3438091157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c153280 0x7f924c0c1a00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.723+0000 7f9252eb2640 1 -- 172.21.15.112:0/3438091157 >> 172.21.15.112:0/3438091157 conn(0x7f924c06b160 msgr2=0x7f924c06b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:13.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.723+0000 7f9252eb2640 1 -- 172.21.15.112:0/3438091157 shutdown_connections 2024-04-25T01:34:13.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.723+0000 7f9252eb2640 1 -- 172.21.15.112:0/3438091157 wait complete. 2024-04-25T01:34:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.725+0000 7f9252eb2640 1 Processor -- start 2024-04-25T01:34:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.725+0000 7f9252eb2640 1 -- start start 2024-04-25T01:34:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.726+0000 7f9252eb2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c06fbb0 0x7f924c14ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.726+0000 7f9252eb2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f924c153280 0x7f924c14f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.726+0000 7f9252eb2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f924c14fbf0 0x7f924c1ebc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.726+0000 7f9252eb2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f924c0c0f10 con 0x7f924c06fbb0 2024-04-25T01:34:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.726+0000 7f9252eb2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f924c0c0d90 con 0x7f924c153280 2024-04-25T01:34:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.726+0000 7f9252eb2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f924c1602a0 con 0x7f924c14fbf0 2024-04-25T01:34:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.726+0000 7f9250c4e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c06fbb0 0x7f924c14ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.726+0000 7f9250c4e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c06fbb0 0x7f924c14ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59574/0 (socket says 172.21.15.112:59574) 2024-04-25T01:34:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.726+0000 7f9250c4e640 1 -- 172.21.15.112:0/2382034013 learned_addr learned my addr 172.21.15.112:0/2382034013 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.726+0000 7f924bfff640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f924c153280 0x7f924c14f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.726+0000 7f925144f640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f924c14fbf0 0x7f924c1ebc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.726+0000 7f9250c4e640 1 -- 172.21.15.112:0/2382034013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f924c14fbf0 msgr2=0x7f924c1ebc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.726+0000 7f9250c4e640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f924c14fbf0 0x7f924c1ebc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.726+0000 7f9250c4e640 1 -- 172.21.15.112:0/2382034013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f924c153280 msgr2=0x7f924c14f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.726+0000 7f9250c4e640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f924c153280 0x7f924c14f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.726+0000 7f9250c4e640 1 -- 172.21.15.112:0/2382034013 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c1587c0 con 0x7f924c06fbb0 2024-04-25T01:34:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.726+0000 7f9250c4e640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c06fbb0 0x7f924c14ccc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9240009890 tx=0x7f92400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.726+0000 7f92497fa640 1 -- 172.21.15.112:0/2382034013 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240037040 con 0x7f924c06fbb0 2024-04-25T01:34:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.727+0000 7f92497fa640 1 -- 172.21.15.112:0/2382034013 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9240042070 con 0x7f924c06fbb0 2024-04-25T01:34:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.727+0000 7f92497fa640 1 -- 172.21.15.112:0/2382034013 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924003d660 con 0x7f924c06fbb0 2024-04-25T01:34:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.727+0000 7f9252eb2640 1 -- 172.21.15.112:0/2382034013 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924c158aa0 con 0x7f924c06fbb0 2024-04-25T01:34:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.727+0000 7f9252eb2640 1 -- 172.21.15.112:0/2382034013 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f924c0c5820 con 0x7f924c06fbb0 2024-04-25T01:34:13.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.728+0000 7f9252eb2640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f924c06f680 0x7f924c1ec1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.728+0000 7f9252eb2640 1 -- 172.21.15.112:0/2382034013 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f924c0c53c0 con 0x7f924c06f680 2024-04-25T01:34:13.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.728+0000 7f924bfff640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f924c06f680 0x7f924c1ec1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.728+0000 7f924bfff640 1 -- 172.21.15.112:0/2382034013 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f924c1efba0 con 0x7f924c06f680 2024-04-25T01:34:13.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.728+0000 7f92497fa640 1 -- 172.21.15.112:0/2382034013 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f924003d800 con 0x7f924c06fbb0 2024-04-25T01:34:13.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.728+0000 7f924bfff640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f924c06f680 0x7f924c1ec1a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f924c0736a0 tx=0x7f9238011cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:13.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.729+0000 7f92497fa640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f923006be30 0x7f923006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.729+0000 7f92497fa640 1 -- 172.21.15.112:0/2382034013 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f92400ab590 con 0x7f924c06fbb0 2024-04-25T01:34:13.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.729+0000 7f925144f640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f923006be30 0x7f923006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.729+0000 7f92497fa640 1 -- 172.21.15.112:0/2382034013 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923800fa70 con 0x7f924c06f680 2024-04-25T01:34:13.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.729+0000 7f92497fa640 1 -- 172.21.15.112:0/2382034013 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f923800fc10 con 0x7f924c06f680 2024-04-25T01:34:13.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.729+0000 7f92497fa640 1 -- 172.21.15.112:0/2382034013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f924c06f680 msgr2=0x7f924c1ec1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.729+0000 7f92497fa640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f924c06f680 0x7f924c1ec1a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f924c0736a0 tx=0x7f9238011cf0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.729+0000 7f92497fa640 1 -- 172.21.15.112:0/2382034013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f924c06f680 msgr2=0x7f924c1ec1a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:34:13.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.729+0000 7f92497fa640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f924c06f680 0x7f924c1ec1a0 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f924c0736a0 tx=0x7f9238011cf0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.729+0000 7f925144f640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f923006be30 0x7f923006e2f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f92440012e0 tx=0x7f9244004d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:13.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.739+0000 7f9252eb2640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f924c153870 0x7f924c072530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.739+0000 7f9252eb2640 1 -- 172.21.15.112:0/2382034013 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f924c075980 con 0x7f924c153870 2024-04-25T01:34:13.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.739+0000 7f924bfff640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f924c153870 0x7f924c072530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.740+0000 7f924bfff640 1 -- 172.21.15.112:0/2382034013 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f924c0682a0 con 0x7f924c153870 2024-04-25T01:34:13.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.740+0000 7f924bfff640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f924c153870 0x7f924c072530 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f924c0736a0 tx=0x7f92380046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:13.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.740+0000 7f92497fa640 1 -- 172.21.15.112:0/2382034013 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238003be0 con 0x7f924c153870 2024-04-25T01:34:13.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.740+0000 7f92497fa640 1 -- 172.21.15.112:0/2382034013 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5070 (secure 0 0 0) 0x7f9238003be0 con 0x7f924c153870 2024-04-25T01:34:13.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.740+0000 7f92497fa640 1 -- 172.21.15.112:0/2382034013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f924c153870 msgr2=0x7f924c072530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.740+0000 7f92497fa640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f924c153870 0x7f924c072530 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f924c0736a0 tx=0x7f92380046c0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.740+0000 7f92497fa640 1 -- 172.21.15.112:0/2382034013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f924c153870 msgr2=0x7f924c072530 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:34:13.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.740+0000 7f92497fa640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f924c153870 0x7f924c072530 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.719 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:34:13.719 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:34:13.719 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:34:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:34:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 66, 2024-04-25T01:34:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 289430, 2024-04-25T01:34:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:34:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:34:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:34:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:34:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 12, 2024-04-25T01:34:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:34:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:34:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:34:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:34:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:34:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:13.722 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:13.722 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:13.722 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:13.722 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:13.722 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:13.722 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:13.722 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:13.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:34:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:34:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:34:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:34:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:34:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:34:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:34:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:34:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:34:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:34:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:34:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:34:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:34:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:34:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:34:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:34:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:34:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:34:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:34:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:34:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:34:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:34:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:34:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:34:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:34:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:34:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:34:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:34:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:34:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:34:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:34:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:34:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:34:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:34:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:34:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:34:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:34:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:34:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:34:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:34:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:34:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:34:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:34:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:34:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:34:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:34:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:34:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:34:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:34:13.733 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:34:13.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.740+0000 7f9252eb2640 1 -- 172.21.15.112:0/2382034013 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f923006be30 msgr2=0x7f923006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.740+0000 7f9252eb2640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f923006be30 0x7f923006e2f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f92440012e0 tx=0x7f9244004d40 comp rx=0 tx=0).stop 2024-04-25T01:34:13.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.741+0000 7f9252eb2640 1 -- 172.21.15.112:0/2382034013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c06fbb0 msgr2=0x7f924c14ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.741+0000 7f925144f640 1 -- 172.21.15.112:0/2382034013 reap_dead start 2024-04-25T01:34:13.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.741+0000 7f9252eb2640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c06fbb0 0x7f924c14ccc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9240009890 tx=0x7f92400046c0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.741+0000 7f9252eb2640 1 -- 172.21.15.112:0/2382034013 shutdown_connections 2024-04-25T01:34:13.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.741+0000 7f9252eb2640 1 --2- 172.21.15.112:0/2382034013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c06fbb0 0x7f924c14ccc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.741+0000 7f9252eb2640 1 -- 172.21.15.112:0/2382034013 >> 172.21.15.112:0/2382034013 conn(0x7f924c06b160 msgr2=0x7f924c15f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:13.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.741+0000 7f9252eb2640 1 -- 172.21.15.112:0/2382034013 shutdown_connections 2024-04-25T01:34:13.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.741+0000 7f9252eb2640 1 -- 172.21.15.112:0/2382034013 wait complete. 2024-04-25T01:34:13.735 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:34:13.735 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:34:13.735 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:34:13.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.867+0000 7f425f453640 1 Processor -- start 2024-04-25T01:34:13.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.867+0000 7f425f453640 1 -- start start 2024-04-25T01:34:13.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.868+0000 7f425f453640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f42580c9af0 0x7f42580c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.868+0000 7f425f453640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4258156370 0x7f42580c5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.868+0000 7f425f453640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42580c8910 con 0x7f4258153030 2024-04-25T01:34:13.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.868+0000 7f425f453640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f4258159160 con 0x7f42580c9af0 2024-04-25T01:34:13.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.868+0000 7f425f453640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f42580c8bf0 con 0x7f4258156370 2024-04-25T01:34:13.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.868+0000 7f425c9ee640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f42580c9af0 0x7f42580c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.868+0000 7f425c9ee640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f42580c9af0 0x7f42580c70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59586/0 (socket says 172.21.15.112:59586) 2024-04-25T01:34:13.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.868+0000 7f425c9ee640 1 -- 172.21.15.112:0/4229546702 learned_addr learned my addr 172.21.15.112:0/4229546702 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:13.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.868+0000 7f425d9f0640 1 --2- 172.21.15.112:0/4229546702 >> v2:172.21.15.187:3300/0 conn(0x7f4258156370 0x7f42580c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.868+0000 7f425c9ee640 1 -- 172.21.15.112:0/4229546702 >> v2:172.21.15.187:3300/0 conn(0x7f4258156370 msgr2=0x7f42580c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:34:13.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.868+0000 7f425c9ee640 1 --2- 172.21.15.112:0/4229546702 >> v2:172.21.15.187:3300/0 conn(0x7f4258156370 0x7f42580c5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.868+0000 7f425c9ee640 1 -- 172.21.15.112:0/4229546702 >> v1:172.21.15.187:6789/0 conn(0x7f4258153030 legacy=0x7f42581595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.868+0000 7f425c9ee640 1 -- 172.21.15.112:0/4229546702 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4258159d00 con 0x7f42580c9af0 2024-04-25T01:34:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.869+0000 7f425c9ee640 1 --2- 172.21.15.112:0/4229546702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42580c9af0 0x7f42580c70a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f42580a8c50 tx=0x7f424802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9faaa82dfb66969e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.869+0000 7f424ffff640 1 -- 172.21.15.112:0/4229546702 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248037070 con 0x7f42580c9af0 2024-04-25T01:34:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.869+0000 7f424ffff640 1 -- 172.21.15.112:0/4229546702 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f424802ae50 con 0x7f42580c9af0 2024-04-25T01:34:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.869+0000 7f424ffff640 1 -- 172.21.15.112:0/4229546702 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248032860 con 0x7f42580c9af0 2024-04-25T01:34:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.870+0000 7f425f453640 1 -- 172.21.15.112:0/4229546702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42580c9af0 msgr2=0x7f42580c70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.870+0000 7f425f453640 1 --2- 172.21.15.112:0/4229546702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42580c9af0 0x7f42580c70a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f42580a8c50 tx=0x7f424802a300 comp rx=0 tx=0).stop 2024-04-25T01:34:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.870+0000 7f425f453640 1 -- 172.21.15.112:0/4229546702 shutdown_connections 2024-04-25T01:34:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.870+0000 7f425f453640 1 --2- 172.21.15.112:0/4229546702 >> v2:172.21.15.187:3300/0 conn(0x7f4258156370 0x7f42580c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.870+0000 7f425f453640 1 --2- 172.21.15.112:0/4229546702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42580c9af0 0x7f42580c70a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.870+0000 7f425f453640 1 -- 172.21.15.112:0/4229546702 >> 172.21.15.112:0/4229546702 conn(0x7f4258074a00 msgr2=0x7f42580c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.870+0000 7f425f453640 1 -- 172.21.15.112:0/4229546702 shutdown_connections 2024-04-25T01:34:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.870+0000 7f425f453640 1 -- 172.21.15.112:0/4229546702 wait complete. 2024-04-25T01:34:13.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.872+0000 7f425f453640 1 Processor -- start 2024-04-25T01:34:13.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.872+0000 7f425f453640 1 -- start start 2024-04-25T01:34:13.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.873+0000 7f425f453640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42580c9af0 0x7f4258154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.873+0000 7f425f453640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4258153030 0x7f425814f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.873+0000 7f425f453640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4258156370 0x7f42580ad3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.873+0000 7f425f453640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f42580c6020 con 0x7f42580c9af0 2024-04-25T01:34:13.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.873+0000 7f425f453640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f42580c5ea0 con 0x7f4258153030 2024-04-25T01:34:13.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.873+0000 7f425f453640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f42580c61a0 con 0x7f4258156370 2024-04-25T01:34:13.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.873+0000 7f425d1ef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42580c9af0 0x7f4258154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.873+0000 7f425d1ef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42580c9af0 0x7f4258154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59602/0 (socket says 172.21.15.112:59602) 2024-04-25T01:34:13.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.873+0000 7f425d1ef640 1 -- 172.21.15.112:0/2607931926 learned_addr learned my addr 172.21.15.112:0/2607931926 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:13.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.873+0000 7f425c9ee640 1 --2- 172.21.15.112:0/2607931926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4258153030 0x7f425814f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.873+0000 7f425d9f0640 1 --2- 172.21.15.112:0/2607931926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4258156370 0x7f42580ad3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.873+0000 7f425d1ef640 1 -- 172.21.15.112:0/2607931926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4258156370 msgr2=0x7f42580ad3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.873+0000 7f425d1ef640 1 --2- 172.21.15.112:0/2607931926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4258156370 0x7f42580ad3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.873+0000 7f425d1ef640 1 -- 172.21.15.112:0/2607931926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4258153030 msgr2=0x7f425814f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.873+0000 7f425d1ef640 1 --2- 172.21.15.112:0/2607931926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4258153030 0x7f425814f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.873+0000 7f425d1ef640 1 -- 172.21.15.112:0/2607931926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425806e1f0 con 0x7f42580c9af0 2024-04-25T01:34:13.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.873+0000 7f425d1ef640 1 --2- 172.21.15.112:0/2607931926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42580c9af0 0x7f4258154a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f425000c460 tx=0x7f425000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:13.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.873+0000 7f424dffb640 1 -- 172.21.15.112:0/2607931926 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42500052d0 con 0x7f42580c9af0 2024-04-25T01:34:13.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.874+0000 7f424dffb640 1 -- 172.21.15.112:0/2607931926 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4250012630 con 0x7f42580c9af0 2024-04-25T01:34:13.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.874+0000 7f424dffb640 1 -- 172.21.15.112:0/2607931926 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42500054b0 con 0x7f42580c9af0 2024-04-25T01:34:13.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.874+0000 7f425f453640 1 -- 172.21.15.112:0/2607931926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42581ec130 con 0x7f42580c9af0 2024-04-25T01:34:13.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.874+0000 7f425f453640 1 -- 172.21.15.112:0/2607931926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42581df2b0 con 0x7f42580c9af0 2024-04-25T01:34:13.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.875+0000 7f425f453640 1 --2- 172.21.15.112:0/2607931926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f425814fc50 0x7f42580ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.875+0000 7f425f453640 1 -- 172.21.15.112:0/2607931926 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f42581df480 con 0x7f425814fc50 2024-04-25T01:34:13.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.875+0000 7f425c9ee640 1 --2- 172.21.15.112:0/2607931926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f425814fc50 0x7f42580ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.875+0000 7f424dffb640 1 -- 172.21.15.112:0/2607931926 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4250012a20 con 0x7f42580c9af0 2024-04-25T01:34:13.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.875+0000 7f425c9ee640 1 -- 172.21.15.112:0/2607931926 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4258075980 con 0x7f425814fc50 2024-04-25T01:34:13.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.875+0000 7f425c9ee640 1 --2- 172.21.15.112:0/2607931926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f425814fc50 0x7f42580ad920 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f42580ae6a0 tx=0x7f42480046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:13.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.876+0000 7f424dffb640 1 --2- 172.21.15.112:0/2607931926 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f423c06be30 0x7f423c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.876+0000 7f425d9f0640 1 --2- 172.21.15.112:0/2607931926 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f423c06be30 0x7f423c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.876+0000 7f424dffb640 1 -- 172.21.15.112:0/2607931926 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f425008ab80 con 0x7f42580c9af0 2024-04-25T01:34:13.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.876+0000 7f424dffb640 1 -- 172.21.15.112:0/2607931926 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248037040 con 0x7f425814fc50 2024-04-25T01:34:13.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.876+0000 7f424dffb640 1 -- 172.21.15.112:0/2607931926 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f424802ae50 con 0x7f425814fc50 2024-04-25T01:34:13.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.876+0000 7f424dffb640 1 -- 172.21.15.112:0/2607931926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f425814fc50 msgr2=0x7f42580ad920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.876+0000 7f424dffb640 1 --2- 172.21.15.112:0/2607931926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f425814fc50 0x7f42580ad920 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f42580ae6a0 tx=0x7f42480046c0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.876+0000 7f424dffb640 1 -- 172.21.15.112:0/2607931926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f425814fc50 msgr2=0x7f42580ad920 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:34:13.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.876+0000 7f424dffb640 1 --2- 172.21.15.112:0/2607931926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f425814fc50 0x7f42580ad920 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f42580ae6a0 tx=0x7f42480046c0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.876+0000 7f425d9f0640 1 --2- 172.21.15.112:0/2607931926 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f423c06be30 0x7f423c06e2f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f42540113e0 tx=0x7f425400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:13.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.883+0000 7f425f453640 1 --2- 172.21.15.112:0/2607931926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42581dfa90 0x7f42581dfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.883+0000 7f425f453640 1 -- 172.21.15.112:0/2607931926 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f42580682a0 con 0x7f42581dfa90 2024-04-25T01:34:13.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.883+0000 7f425c9ee640 1 --2- 172.21.15.112:0/2607931926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42581dfa90 0x7f42581dfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.883+0000 7f425c9ee640 1 -- 172.21.15.112:0/2607931926 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f42580ade60 con 0x7f42581dfa90 2024-04-25T01:34:13.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.884+0000 7f425c9ee640 1 --2- 172.21.15.112:0/2607931926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42581dfa90 0x7f42581dfeb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f424802a340 tx=0x7f4248008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:13.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.884+0000 7f424dffb640 1 -- 172.21.15.112:0/2607931926 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248037030 con 0x7f42581dfa90 2024-04-25T01:34:13.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.884+0000 7f424dffb640 1 -- 172.21.15.112:0/2607931926 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4559 (secure 0 0 0) 0x7f4248048230 con 0x7f42581dfa90 2024-04-25T01:34:13.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.884+0000 7f424dffb640 1 -- 172.21.15.112:0/2607931926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42581dfa90 msgr2=0x7f42581dfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.884+0000 7f424dffb640 1 --2- 172.21.15.112:0/2607931926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42581dfa90 0x7f42581dfeb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f424802a340 tx=0x7f4248008030 comp rx=0 tx=0).stop 2024-04-25T01:34:13.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.884+0000 7f424dffb640 1 -- 172.21.15.112:0/2607931926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42581dfa90 msgr2=0x7f42581dfeb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:34:13.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.884+0000 7f424dffb640 1 --2- 172.21.15.112:0/2607931926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42581dfa90 0x7f42581dfeb0 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f424802a340 tx=0x7f4248008030 comp rx=0 tx=0).stop 2024-04-25T01:34:13.863 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:34:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:34:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:34:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:34:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 66, 2024-04-25T01:34:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 21704, 2024-04-25T01:34:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:34:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:34:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:34:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:34:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 12, 2024-04-25T01:34:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:34:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:34:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:34:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:34:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:34:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:34:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:34:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:34:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:34:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:34:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:34:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:34:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:34:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:34:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:34:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:34:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:34:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:34:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:34:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:34:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:34:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:34:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:34:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:34:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:34:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:34:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:34:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:34:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:34:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:34:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:34:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:34:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:34:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:34:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:34:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:34:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:34:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:34:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:34:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:34:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:34:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:34:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:34:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:34:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:34:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:34:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:34:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:34:13.874 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:34:13.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.884+0000 7f425f453640 1 -- 172.21.15.112:0/2607931926 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f423c06be30 msgr2=0x7f423c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.884+0000 7f425f453640 1 --2- 172.21.15.112:0/2607931926 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f423c06be30 0x7f423c06e2f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f42540113e0 tx=0x7f425400b040 comp rx=0 tx=0).stop 2024-04-25T01:34:13.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.884+0000 7f425f453640 1 -- 172.21.15.112:0/2607931926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42580c9af0 msgr2=0x7f4258154a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.884+0000 7f425f453640 1 --2- 172.21.15.112:0/2607931926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42580c9af0 0x7f4258154a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f425000c460 tx=0x7f425000c920 comp rx=0 tx=0).stop 2024-04-25T01:34:13.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.884+0000 7f425d9f0640 1 -- 172.21.15.112:0/2607931926 reap_dead start 2024-04-25T01:34:13.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.884+0000 7f425d9f0640 1 -- 172.21.15.112:0/2607931926 reap_dead start 2024-04-25T01:34:13.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.884+0000 7f425f453640 1 -- 172.21.15.112:0/2607931926 shutdown_connections 2024-04-25T01:34:13.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.884+0000 7f425f453640 1 -- 172.21.15.112:0/2607931926 >> 172.21.15.112:0/2607931926 conn(0x7f4258074a00 msgr2=0x7f4258157250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:13.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.884+0000 7f425f453640 1 -- 172.21.15.112:0/2607931926 shutdown_connections 2024-04-25T01:34:13.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:13.884+0000 7f425f453640 1 -- 172.21.15.112:0/2607931926 wait complete. 2024-04-25T01:34:13.877 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:34:13.877 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-04-25T01:34:13.877 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-25T01:34:13.877 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-04-25T01:34:13.877 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-25T01:34:13.877 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T01:34:13.878 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:34:13.888 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T01:34:13.888 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T01:34:13.888 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:34:13.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.018+0000 7f1b4344d640 1 Processor -- start 2024-04-25T01:34:13.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.019+0000 7f1b4344d640 1 -- start start 2024-04-25T01:34:13.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.019+0000 7f1b4344d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0c5d70 0x7f1b3c0c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.019+0000 7f1b4344d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1b3c156cf0 0x7f1b3c06bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:13.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.019+0000 7f1b4344d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b3c0c90e0 con 0x7f1b3c152fd0 2024-04-25T01:34:13.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.019+0000 7f1b4344d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f1b3c0ca9b0 con 0x7f1b3c156cf0 2024-04-25T01:34:13.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.019+0000 7f1b4344d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3c0cac00 con 0x7f1b3c0c5d70 2024-04-25T01:34:13.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.019+0000 7f1b409e8640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0c5d70 msgr2=0x7f1b3c0c9b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:34:13.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.019+0000 7f1b409e8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0c5d70 0x7f1b3c0c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:34:13.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.019+0000 7f1b419ea640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1b3c156cf0 0x7f1b3c06bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:13.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.020+0000 7f1b419ea640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1b3c156cf0 0x7f1b3c06bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59606/0 (socket says 172.21.15.112:59606) 2024-04-25T01:34:13.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.020+0000 7f1b419ea640 1 -- 172.21.15.112:0/1497395126 learned_addr learned my addr 172.21.15.112:0/1497395126 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:13.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.020+0000 7f1b419ea640 1 -- 172.21.15.112:0/1497395126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0c5d70 msgr2=0x7f1b3c0c9b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:34:13.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.020+0000 7f1b419ea640 1 --2- 172.21.15.112:0/1497395126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0c5d70 0x7f1b3c0c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:13.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.020+0000 7f1b33fff640 1 -- 172.21.15.112:0/1497395126 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2273062813 0 0) 0x7f1b3c0c90e0 con 0x7f1b3c152fd0 2024-04-25T01:34:13.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.020+0000 7f1b419ea640 1 -- 172.21.15.112:0/1497395126 >> v1:172.21.15.187:6789/0 conn(0x7f1b3c152fd0 legacy=0x7f1b3c156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:13.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.020+0000 7f1b419ea640 1 -- 172.21.15.112:0/1497395126 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c0c9380 con 0x7f1b3c156cf0 2024-04-25T01:34:13.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.021+0000 7f1b419ea640 1 --2- 172.21.15.112:0/1497395126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c156cf0 0x7f1b3c06bda0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009e50 tx=0x7f1b3402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bea4850b4e56b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:14.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.021+0000 7f1b33fff640 1 -- 172.21.15.112:0/1497395126 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34036070 con 0x7f1b3c156cf0 2024-04-25T01:34:14.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.021+0000 7f1b33fff640 1 -- 172.21.15.112:0/1497395126 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b34031600 con 0x7f1b3c156cf0 2024-04-25T01:34:14.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.021+0000 7f1b33fff640 1 -- 172.21.15.112:0/1497395126 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b340319d0 con 0x7f1b3c156cf0 2024-04-25T01:34:14.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.021+0000 7f1b4344d640 1 -- 172.21.15.112:0/1497395126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c156cf0 msgr2=0x7f1b3c06bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.021+0000 7f1b4344d640 1 --2- 172.21.15.112:0/1497395126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c156cf0 0x7f1b3c06bda0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009e50 tx=0x7f1b3402a6a0 comp rx=0 tx=0).stop 2024-04-25T01:34:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.021+0000 7f1b4344d640 1 -- 172.21.15.112:0/1497395126 shutdown_connections 2024-04-25T01:34:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.021+0000 7f1b4344d640 1 --2- 172.21.15.112:0/1497395126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c156cf0 0x7f1b3c06bda0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.021+0000 7f1b4344d640 1 --2- 172.21.15.112:0/1497395126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0c5d70 0x7f1b3c0c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.021+0000 7f1b4344d640 1 -- 172.21.15.112:0/1497395126 >> 172.21.15.112:0/1497395126 conn(0x7f1b3c0acc70 msgr2=0x7f1b3c0ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.022+0000 7f1b4344d640 1 -- 172.21.15.112:0/1497395126 shutdown_connections 2024-04-25T01:34:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.022+0000 7f1b4344d640 1 -- 172.21.15.112:0/1497395126 wait complete. 2024-04-25T01:34:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.023+0000 7f1b4344d640 1 Processor -- start 2024-04-25T01:34:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.023+0000 7f1b4344d640 1 -- start start 2024-04-25T01:34:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b4344d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c0c5d70 0x7f1b3c154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b4344d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b3c152fd0 0x7f1b3c0af970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b4344d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c156cf0 0x7f1b3c0afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:14.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b4344d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c157160 con 0x7f1b3c0c5d70 2024-04-25T01:34:14.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b4344d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c0caee0 con 0x7f1b3c152fd0 2024-04-25T01:34:14.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b4344d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3c1572e0 con 0x7f1b3c156cf0 2024-04-25T01:34:14.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b419ea640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c156cf0 msgr2=0x7f1b3c0afeb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:34:14.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b419ea640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c156cf0 0x7f1b3c0afeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:34:14.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b411e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c0c5d70 0x7f1b3c154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:14.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b411e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c0c5d70 0x7f1b3c154680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59622/0 (socket says 172.21.15.112:59622) 2024-04-25T01:34:14.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b411e9640 1 -- 172.21.15.112:0/852258143 learned_addr learned my addr 172.21.15.112:0/852258143 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:14.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b409e8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b3c152fd0 0x7f1b3c0af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:14.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b411e9640 1 -- 172.21.15.112:0/852258143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c156cf0 msgr2=0x7f1b3c0afeb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:34:14.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b411e9640 1 --2- 172.21.15.112:0/852258143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c156cf0 0x7f1b3c0afeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:14.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b411e9640 1 -- 172.21.15.112:0/852258143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b3c152fd0 msgr2=0x7f1b3c0af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b411e9640 1 --2- 172.21.15.112:0/852258143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b3c152fd0 0x7f1b3c0af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b411e9640 1 -- 172.21.15.112:0/852258143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c153c30 con 0x7f1b3c0c5d70 2024-04-25T01:34:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b411e9640 1 --2- 172.21.15.112:0/852258143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c0c5d70 0x7f1b3c154680 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c4e0 tx=0x7f1b3800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b31ffb640 1 -- 172.21.15.112:0/852258143 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38017070 con 0x7f1b3c0c5d70 2024-04-25T01:34:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b31ffb640 1 -- 172.21.15.112:0/852258143 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b380035f0 con 0x7f1b3c0c5d70 2024-04-25T01:34:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b4344d640 1 -- 172.21.15.112:0/852258143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3c0b0910 con 0x7f1b3c0c5d70 2024-04-25T01:34:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.024+0000 7f1b31ffb640 1 -- 172.21.15.112:0/852258143 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b380039e0 con 0x7f1b3c0c5d70 2024-04-25T01:34:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.025+0000 7f1b4344d640 1 -- 172.21.15.112:0/852258143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3c154bc0 con 0x7f1b3c0c5d70 2024-04-25T01:34:14.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.026+0000 7f1b4344d640 1 -- 172.21.15.112:0/852258143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b3c0682a0 con 0x7f1b3c0c5d70 2024-04-25T01:34:14.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.026+0000 7f1b31ffb640 1 -- 172.21.15.112:0/852258143 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b3801a070 con 0x7f1b3c0c5d70 2024-04-25T01:34:14.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.027+0000 7f1b31ffb640 1 --2- 172.21.15.112:0/852258143 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1b2006be10 0x7f1b2006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:14.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.027+0000 7f1b409e8640 1 --2- 172.21.15.112:0/852258143 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1b2006be10 0x7f1b2006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:14.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.027+0000 7f1b31ffb640 1 -- 172.21.15.112:0/852258143 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f1b3808a3a0 con 0x7f1b3c0c5d70 2024-04-25T01:34:14.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.027+0000 7f1b409e8640 1 --2- 172.21.15.112:0/852258143 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1b2006be10 0x7f1b2006e2d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c006b20 tx=0x7f1b2c006ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:14.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.032+0000 7f1b31ffb640 1 -- 172.21.15.112:0/852258143 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1b380583f0 con 0x7f1b3c0c5d70 2024-04-25T01:34:14.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.330+0000 7f1b4344d640 1 -- 172.21.15.112:0/852258143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1b3c0c5ac0 con 0x7f1b3c0c5d70 2024-04-25T01:34:14.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.331+0000 7f1b31ffb640 1 -- 172.21.15.112:0/852258143 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f1b3805c6f0 con 0x7f1b3c0c5d70 2024-04-25T01:34:14.309 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":66,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:34:14.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.334+0000 7f1b4344d640 1 -- 172.21.15.112:0/852258143 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1b2006be10 msgr2=0x7f1b2006e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:14.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.334+0000 7f1b4344d640 1 --2- 172.21.15.112:0/852258143 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1b2006be10 0x7f1b2006e2d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c006b20 tx=0x7f1b2c006ab0 comp rx=0 tx=0).stop 2024-04-25T01:34:14.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.334+0000 7f1b4344d640 1 -- 172.21.15.112:0/852258143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c0c5d70 msgr2=0x7f1b3c154680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:14.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.334+0000 7f1b4344d640 1 --2- 172.21.15.112:0/852258143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c0c5d70 0x7f1b3c154680 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c4e0 tx=0x7f1b3800c9a0 comp rx=0 tx=0).stop 2024-04-25T01:34:14.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.334+0000 7f1b4344d640 1 -- 172.21.15.112:0/852258143 shutdown_connections 2024-04-25T01:34:14.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.334+0000 7f1b4344d640 1 --2- 172.21.15.112:0/852258143 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1b2006be10 0x7f1b2006e2d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:14.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.334+0000 7f1b4344d640 1 --2- 172.21.15.112:0/852258143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c156cf0 0x7f1b3c0afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:14.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.334+0000 7f1b4344d640 1 --2- 172.21.15.112:0/852258143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b3c152fd0 0x7f1b3c0af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:14.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.334+0000 7f1b4344d640 1 --2- 172.21.15.112:0/852258143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b3c0c5d70 0x7f1b3c154680 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:14.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.334+0000 7f1b4344d640 1 -- 172.21.15.112:0/852258143 >> 172.21.15.112:0/852258143 conn(0x7f1b3c0acc70 msgr2=0x7f1b3c0ad050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:14.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.334+0000 7f1b4344d640 1 -- 172.21.15.112:0/852258143 shutdown_connections 2024-04-25T01:34:14.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:14.334+0000 7f1b4344d640 1 -- 172.21.15.112:0/852258143 wait complete. 2024-04-25T01:34:17.325 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:34:17.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.465+0000 7f7a57838640 1 Processor -- start 2024-04-25T01:34:17.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.466+0000 7f7a57838640 1 -- start start 2024-04-25T01:34:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.466+0000 7f7a57838640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7a50152fd0 0x7f7a500c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.466+0000 7f7a57838640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7a500c5d70 0x7f7a50156280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.467+0000 7f7a57838640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a501567c0 con 0x7f7a50156cf0 2024-04-25T01:34:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.467+0000 7f7a57838640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f7a500ca9b0 con 0x7f7a500c5d70 2024-04-25T01:34:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.467+0000 7f7a57838640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7a500c90e0 con 0x7f7a50152fd0 2024-04-25T01:34:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.467+0000 7f7a54dd3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7a500c5d70 0x7f7a50156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.467+0000 7f7a54dd3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7a500c5d70 0x7f7a50156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48920/0 (socket says 172.21.15.112:48920) 2024-04-25T01:34:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.467+0000 7f7a54dd3640 1 -- 172.21.15.112:0/1155324976 learned_addr learned my addr 172.21.15.112:0/1155324976 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.467+0000 7f7a555d4640 1 --2- 172.21.15.112:0/1155324976 >> v2:172.21.15.187:3300/0 conn(0x7f7a50152fd0 0x7f7a500c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.467+0000 7f7a54dd3640 1 -- 172.21.15.112:0/1155324976 >> v2:172.21.15.187:3300/0 conn(0x7f7a50152fd0 msgr2=0x7f7a500c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:34:17.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.467+0000 7f7a54dd3640 1 --2- 172.21.15.112:0/1155324976 >> v2:172.21.15.187:3300/0 conn(0x7f7a50152fd0 0x7f7a500c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:17.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.467+0000 7f7a54dd3640 1 -- 172.21.15.112:0/1155324976 >> v1:172.21.15.187:6789/0 conn(0x7f7a50156cf0 legacy=0x7f7a5006bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:17.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.467+0000 7f7a54dd3640 1 -- 172.21.15.112:0/1155324976 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a500cac00 con 0x7f7a500c5d70 2024-04-25T01:34:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.468+0000 7f7a54dd3640 1 --2- 172.21.15.112:0/1155324976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a500c5d70 0x7f7a50156280 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7a500adcd0 tx=0x7f7a4002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b98999650e186a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.468+0000 7f7a47fff640 1 -- 172.21.15.112:0/1155324976 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40037070 con 0x7f7a500c5d70 2024-04-25T01:34:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.468+0000 7f7a47fff640 1 -- 172.21.15.112:0/1155324976 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a40032440 con 0x7f7a500c5d70 2024-04-25T01:34:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.468+0000 7f7a47fff640 1 -- 172.21.15.112:0/1155324976 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40032810 con 0x7f7a500c5d70 2024-04-25T01:34:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.469+0000 7f7a57838640 1 -- 172.21.15.112:0/1155324976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a500c5d70 msgr2=0x7f7a50156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.469+0000 7f7a57838640 1 --2- 172.21.15.112:0/1155324976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a500c5d70 0x7f7a50156280 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7a500adcd0 tx=0x7f7a4002a1e0 comp rx=0 tx=0).stop 2024-04-25T01:34:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.469+0000 7f7a57838640 1 -- 172.21.15.112:0/1155324976 shutdown_connections 2024-04-25T01:34:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.469+0000 7f7a57838640 1 --2- 172.21.15.112:0/1155324976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a500c5d70 0x7f7a50156280 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.469+0000 7f7a57838640 1 --2- 172.21.15.112:0/1155324976 >> v2:172.21.15.187:3300/0 conn(0x7f7a50152fd0 0x7f7a500c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.469+0000 7f7a57838640 1 -- 172.21.15.112:0/1155324976 >> 172.21.15.112:0/1155324976 conn(0x7f7a500acc70 msgr2=0x7f7a500ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.469+0000 7f7a57838640 1 -- 172.21.15.112:0/1155324976 shutdown_connections 2024-04-25T01:34:17.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.469+0000 7f7a57838640 1 -- 172.21.15.112:0/1155324976 wait complete. 2024-04-25T01:34:17.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.470+0000 7f7a57838640 1 Processor -- start 2024-04-25T01:34:17.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.470+0000 7f7a57838640 1 -- start start 2024-04-25T01:34:17.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.471+0000 7f7a57838640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a500c5d70 0x7f7a501ebc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:17.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.471+0000 7f7a57838640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a50152fd0 0x7f7a50154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:17.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.471+0000 7f7a57838640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a50156cf0 0x7f7a500c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:17.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.471+0000 7f7a57838640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7a500caee0 con 0x7f7a500c5d70 2024-04-25T01:34:17.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.471+0000 7f7a57838640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7a500c9660 con 0x7f7a50152fd0 2024-04-25T01:34:17.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.471+0000 7f7a57838640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7a5006c4b0 con 0x7f7a50156cf0 2024-04-25T01:34:17.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.471+0000 7f7a555d4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a500c5d70 0x7f7a501ebc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:17.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.471+0000 7f7a55dd5640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a50156cf0 msgr2=0x7f7a500c68c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:34:17.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.471+0000 7f7a555d4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a500c5d70 0x7f7a501ebc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48936/0 (socket says 172.21.15.112:48936) 2024-04-25T01:34:17.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.471+0000 7f7a55dd5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a50156cf0 0x7f7a500c68c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:34:17.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.471+0000 7f7a555d4640 1 -- 172.21.15.112:0/326362805 learned_addr learned my addr 172.21.15.112:0/326362805 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:17.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.471+0000 7f7a54dd3640 1 --2- 172.21.15.112:0/326362805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a50152fd0 0x7f7a50154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:17.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.471+0000 7f7a555d4640 1 -- 172.21.15.112:0/326362805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a50156cf0 msgr2=0x7f7a500c68c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:34:17.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.471+0000 7f7a555d4640 1 --2- 172.21.15.112:0/326362805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a50156cf0 0x7f7a500c68c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:17.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.471+0000 7f7a555d4640 1 -- 172.21.15.112:0/326362805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a50152fd0 msgr2=0x7f7a50154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:17.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.471+0000 7f7a555d4640 1 --2- 172.21.15.112:0/326362805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a50152fd0 0x7f7a50154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:17.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.471+0000 7f7a555d4640 1 -- 172.21.15.112:0/326362805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a500c7e10 con 0x7f7a500c5d70 2024-04-25T01:34:17.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.471+0000 7f7a555d4640 1 --2- 172.21.15.112:0/326362805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a500c5d70 0x7f7a501ebc20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00e9b0 tx=0x7f7a4c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:17.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.471+0000 7f7a45ffb640 1 -- 172.21.15.112:0/326362805 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c017070 con 0x7f7a500c5d70 2024-04-25T01:34:17.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.471+0000 7f7a45ffb640 1 -- 172.21.15.112:0/326362805 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a4c00c9a0 con 0x7f7a500c5d70 2024-04-25T01:34:17.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.472+0000 7f7a57838640 1 -- 172.21.15.112:0/326362805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a500c80f0 con 0x7f7a500c5d70 2024-04-25T01:34:17.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.472+0000 7f7a45ffb640 1 -- 172.21.15.112:0/326362805 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c00cd90 con 0x7f7a500c5d70 2024-04-25T01:34:17.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.472+0000 7f7a57838640 1 -- 172.21.15.112:0/326362805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a50154e70 con 0x7f7a500c5d70 2024-04-25T01:34:17.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.473+0000 7f7a57838640 1 -- 172.21.15.112:0/326362805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a500682a0 con 0x7f7a500c5d70 2024-04-25T01:34:17.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.473+0000 7f7a45ffb640 1 -- 172.21.15.112:0/326362805 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a4c01a070 con 0x7f7a500c5d70 2024-04-25T01:34:17.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.474+0000 7f7a45ffb640 1 --2- 172.21.15.112:0/326362805 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7a3406be10 0x7f7a3406e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:17.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.474+0000 7f7a45ffb640 1 -- 172.21.15.112:0/326362805 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f7a4c08b3f0 con 0x7f7a500c5d70 2024-04-25T01:34:17.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.474+0000 7f7a54dd3640 1 --2- 172.21.15.112:0/326362805 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7a3406be10 0x7f7a3406e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:17.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.474+0000 7f7a54dd3640 1 --2- 172.21.15.112:0/326362805 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7a3406be10 0x7f7a3406e2d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7a40034000 tx=0x7f7a4002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:17.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.479+0000 7f7a45ffb640 1 -- 172.21.15.112:0/326362805 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7a4c059440 con 0x7f7a500c5d70 2024-04-25T01:34:17.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.778+0000 7f7a57838640 1 -- 172.21.15.112:0/326362805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f7a500c5ac0 con 0x7f7a500c5d70 2024-04-25T01:34:17.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.778+0000 7f7a45ffb640 1 -- 172.21.15.112:0/326362805 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f7a4c05d740 con 0x7f7a500c5d70 2024-04-25T01:34:17.756 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":66,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:34:17.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.781+0000 7f7a57838640 1 -- 172.21.15.112:0/326362805 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7a3406be10 msgr2=0x7f7a3406e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:17.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.781+0000 7f7a57838640 1 --2- 172.21.15.112:0/326362805 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7a3406be10 0x7f7a3406e2d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7a40034000 tx=0x7f7a4002e000 comp rx=0 tx=0).stop 2024-04-25T01:34:17.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.781+0000 7f7a57838640 1 -- 172.21.15.112:0/326362805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a500c5d70 msgr2=0x7f7a501ebc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:17.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.781+0000 7f7a57838640 1 --2- 172.21.15.112:0/326362805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a500c5d70 0x7f7a501ebc20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00e9b0 tx=0x7f7a4c00ee70 comp rx=0 tx=0).stop 2024-04-25T01:34:17.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.781+0000 7f7a57838640 1 -- 172.21.15.112:0/326362805 shutdown_connections 2024-04-25T01:34:17.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.781+0000 7f7a57838640 1 --2- 172.21.15.112:0/326362805 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7a3406be10 0x7f7a3406e2d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:17.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.781+0000 7f7a57838640 1 --2- 172.21.15.112:0/326362805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a50156cf0 0x7f7a500c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:17.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.781+0000 7f7a57838640 1 --2- 172.21.15.112:0/326362805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a50152fd0 0x7f7a50154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:17.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.781+0000 7f7a57838640 1 --2- 172.21.15.112:0/326362805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a500c5d70 0x7f7a501ebc20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:17.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.781+0000 7f7a57838640 1 -- 172.21.15.112:0/326362805 >> 172.21.15.112:0/326362805 conn(0x7f7a500acc70 msgr2=0x7f7a500ad050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:17.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.781+0000 7f7a57838640 1 -- 172.21.15.112:0/326362805 shutdown_connections 2024-04-25T01:34:17.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:17.781+0000 7f7a57838640 1 -- 172.21.15.112:0/326362805 wait complete. 2024-04-25T01:34:20.773 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:34:20.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.910+0000 7ff36f45d640 1 Processor -- start 2024-04-25T01:34:20.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.911+0000 7ff36f45d640 1 -- start start 2024-04-25T01:34:20.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.912+0000 7ff36f45d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff368152fd0 0x7ff3680c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:20.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.912+0000 7ff36f45d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff3680c5d70 0x7ff368156280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:20.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.912+0000 7ff36f45d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3681567c0 con 0x7ff368156cf0 2024-04-25T01:34:20.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.912+0000 7ff36f45d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7ff3680ca9b0 con 0x7ff3680c5d70 2024-04-25T01:34:20.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.912+0000 7ff36f45d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff3680c90e0 con 0x7ff368152fd0 2024-04-25T01:34:20.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.912+0000 7ff36d1f9640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff368152fd0 msgr2=0x7ff3680c9b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:34:20.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.912+0000 7ff36d1f9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff368152fd0 0x7ff3680c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:34:20.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.912+0000 7ff36c9f8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff3680c5d70 0x7ff368156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:20.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.912+0000 7ff36c9f8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff3680c5d70 0x7ff368156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48942/0 (socket says 172.21.15.112:48942) 2024-04-25T01:34:20.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.912+0000 7ff36c9f8640 1 -- 172.21.15.112:0/968086489 learned_addr learned my addr 172.21.15.112:0/968086489 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:20.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.912+0000 7ff36c9f8640 1 -- 172.21.15.112:0/968086489 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff368152fd0 msgr2=0x7ff3680c9b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:34:20.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.912+0000 7ff36c9f8640 1 --2- 172.21.15.112:0/968086489 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff368152fd0 0x7ff3680c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:20.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.912+0000 7ff36c9f8640 1 -- 172.21.15.112:0/968086489 >> v1:172.21.15.187:6789/0 conn(0x7ff368156cf0 legacy=0x7ff36806bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:20.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.912+0000 7ff36c9f8640 1 -- 172.21.15.112:0/968086489 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3680cac00 con 0x7ff3680c5d70 2024-04-25T01:34:20.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.913+0000 7ff36c9f8640 1 --2- 172.21.15.112:0/968086489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3680c5d70 0x7ff368156280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff358009870 tx=0x7ff35802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f9aa659260ff42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:20.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.913+0000 7ff35ffff640 1 -- 172.21.15.112:0/968086489 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff358037070 con 0x7ff3680c5d70 2024-04-25T01:34:20.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.913+0000 7ff35ffff640 1 -- 172.21.15.112:0/968086489 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff358032440 con 0x7ff3680c5d70 2024-04-25T01:34:20.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.913+0000 7ff35ffff640 1 -- 172.21.15.112:0/968086489 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff358032810 con 0x7ff3680c5d70 2024-04-25T01:34:20.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.914+0000 7ff36f45d640 1 -- 172.21.15.112:0/968086489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3680c5d70 msgr2=0x7ff368156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:20.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.914+0000 7ff36f45d640 1 --2- 172.21.15.112:0/968086489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3680c5d70 0x7ff368156280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff358009870 tx=0x7ff35802a1e0 comp rx=0 tx=0).stop 2024-04-25T01:34:20.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.914+0000 7ff36f45d640 1 -- 172.21.15.112:0/968086489 shutdown_connections 2024-04-25T01:34:20.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.914+0000 7ff36f45d640 1 --2- 172.21.15.112:0/968086489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3680c5d70 0x7ff368156280 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:20.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.914+0000 7ff36f45d640 1 --2- 172.21.15.112:0/968086489 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff368152fd0 0x7ff3680c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:20.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.914+0000 7ff36f45d640 1 -- 172.21.15.112:0/968086489 >> 172.21.15.112:0/968086489 conn(0x7ff3680acc70 msgr2=0x7ff3680ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:20.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.914+0000 7ff36f45d640 1 -- 172.21.15.112:0/968086489 shutdown_connections 2024-04-25T01:34:20.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.914+0000 7ff36f45d640 1 -- 172.21.15.112:0/968086489 wait complete. 2024-04-25T01:34:20.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.915+0000 7ff36f45d640 1 Processor -- start 2024-04-25T01:34:20.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.916+0000 7ff36f45d640 1 -- start start 2024-04-25T01:34:20.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.916+0000 7ff36f45d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3680c5d70 0x7ff36814c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:20.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.916+0000 7ff36f45d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff368152fd0 0x7ff368157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.916+0000 7ff36f45d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff368156cf0 0x7ff3681eec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.916+0000 7ff36f45d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff3680caee0 con 0x7ff368156cf0 2024-04-25T01:34:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.916+0000 7ff36f45d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff3680c9660 con 0x7ff368152fd0 2024-04-25T01:34:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.916+0000 7ff36f45d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff36806c4b0 con 0x7ff3680c5d70 2024-04-25T01:34:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.916+0000 7ff36d1f9640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3680c5d70 msgr2=0x7ff36814c460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:34:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.916+0000 7ff36d1f9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3680c5d70 0x7ff36814c460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:34:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.916+0000 7ff36d9fa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff368156cf0 0x7ff3681eec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.916+0000 7ff36d9fa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff368156cf0 0x7ff3681eec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48952/0 (socket says 172.21.15.112:48952) 2024-04-25T01:34:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.916+0000 7ff36d9fa640 1 -- 172.21.15.112:0/3506960186 learned_addr learned my addr 172.21.15.112:0/3506960186 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.917+0000 7ff36c9f8640 1 --2- 172.21.15.112:0/3506960186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff368152fd0 0x7ff368157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.917+0000 7ff36d9fa640 1 -- 172.21.15.112:0/3506960186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3680c5d70 msgr2=0x7ff36814c460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:34:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.917+0000 7ff36d9fa640 1 --2- 172.21.15.112:0/3506960186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3680c5d70 0x7ff36814c460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.917+0000 7ff36d9fa640 1 -- 172.21.15.112:0/3506960186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff368152fd0 msgr2=0x7ff368157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:20.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.917+0000 7ff36d9fa640 1 --2- 172.21.15.112:0/3506960186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff368152fd0 0x7ff368157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:20.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.917+0000 7ff36d9fa640 1 -- 172.21.15.112:0/3506960186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3680ab3b0 con 0x7ff368156cf0 2024-04-25T01:34:20.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.917+0000 7ff36d9fa640 1 --2- 172.21.15.112:0/3506960186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff368156cf0 0x7ff3681eec00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff36000c6b0 tx=0x7ff36000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:20.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.917+0000 7ff35dffb640 1 -- 172.21.15.112:0/3506960186 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360003b70 con 0x7ff368156cf0 2024-04-25T01:34:20.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.917+0000 7ff35dffb640 1 -- 172.21.15.112:0/3506960186 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff360003d30 con 0x7ff368156cf0 2024-04-25T01:34:20.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.917+0000 7ff35dffb640 1 -- 172.21.15.112:0/3506960186 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360005310 con 0x7ff368156cf0 2024-04-25T01:34:20.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.917+0000 7ff36f45d640 1 -- 172.21.15.112:0/3506960186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff368153450 con 0x7ff368156cf0 2024-04-25T01:34:20.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.917+0000 7ff36f45d640 1 -- 172.21.15.112:0/3506960186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3681ef5f0 con 0x7ff368156cf0 2024-04-25T01:34:20.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.919+0000 7ff36f45d640 1 -- 172.21.15.112:0/3506960186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3680682a0 con 0x7ff368156cf0 2024-04-25T01:34:20.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.919+0000 7ff35dffb640 1 -- 172.21.15.112:0/3506960186 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff3600054b0 con 0x7ff368156cf0 2024-04-25T01:34:20.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.919+0000 7ff35dffb640 1 --2- 172.21.15.112:0/3506960186 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff34c06bde0 0x7ff34c06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:20.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.919+0000 7ff35dffb640 1 -- 172.21.15.112:0/3506960186 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7ff36008b0a0 con 0x7ff368156cf0 2024-04-25T01:34:20.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.920+0000 7ff36d1f9640 1 --2- 172.21.15.112:0/3506960186 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff34c06bde0 0x7ff34c06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:20.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.920+0000 7ff36d1f9640 1 --2- 172.21.15.112:0/3506960186 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff34c06bde0 0x7ff34c06e2a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff364006fd0 tx=0x7ff364006d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:20.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:20.924+0000 7ff35dffb640 1 -- 172.21.15.112:0/3506960186 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff3681567c0 con 0x7ff368156cf0 2024-04-25T01:34:21.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:21.261+0000 7ff36f45d640 1 -- 172.21.15.112:0/3506960186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff3680c5ac0 con 0x7ff368156cf0 2024-04-25T01:34:21.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:21.262+0000 7ff35dffb640 1 -- 172.21.15.112:0/3506960186 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7ff360059610 con 0x7ff368156cf0 2024-04-25T01:34:21.239 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":66,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:34:21.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:21.264+0000 7ff36f45d640 1 -- 172.21.15.112:0/3506960186 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff34c06bde0 msgr2=0x7ff34c06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:21.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:21.264+0000 7ff36f45d640 1 --2- 172.21.15.112:0/3506960186 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff34c06bde0 0x7ff34c06e2a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff364006fd0 tx=0x7ff364006d80 comp rx=0 tx=0).stop 2024-04-25T01:34:21.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:21.264+0000 7ff36f45d640 1 -- 172.21.15.112:0/3506960186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff368156cf0 msgr2=0x7ff3681eec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:21.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:21.264+0000 7ff36f45d640 1 --2- 172.21.15.112:0/3506960186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff368156cf0 0x7ff3681eec00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff36000c6b0 tx=0x7ff36000cb70 comp rx=0 tx=0).stop 2024-04-25T01:34:21.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:21.264+0000 7ff36f45d640 1 -- 172.21.15.112:0/3506960186 shutdown_connections 2024-04-25T01:34:21.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:21.264+0000 7ff36f45d640 1 --2- 172.21.15.112:0/3506960186 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff34c06bde0 0x7ff34c06e2a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:21.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:21.264+0000 7ff36f45d640 1 --2- 172.21.15.112:0/3506960186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff368156cf0 0x7ff3681eec00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:21.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:21.264+0000 7ff36f45d640 1 --2- 172.21.15.112:0/3506960186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff368152fd0 0x7ff368157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:21.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:21.264+0000 7ff36f45d640 1 --2- 172.21.15.112:0/3506960186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3680c5d70 0x7ff36814c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:21.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:21.264+0000 7ff36f45d640 1 -- 172.21.15.112:0/3506960186 >> 172.21.15.112:0/3506960186 conn(0x7ff3680acc70 msgr2=0x7ff3680ab5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:21.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:21.264+0000 7ff36f45d640 1 -- 172.21.15.112:0/3506960186 shutdown_connections 2024-04-25T01:34:21.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:21.264+0000 7ff36f45d640 1 -- 172.21.15.112:0/3506960186 wait complete. 2024-04-25T01:34:24.252 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:34:24.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.388+0000 7fa63d3b7640 1 Processor -- start 2024-04-25T01:34:24.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.388+0000 7fa63d3b7640 1 -- start start 2024-04-25T01:34:24.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.389+0000 7fa63d3b7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6380c5cf0 0x7fa6380caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:24.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.389+0000 7fa63d3b7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 0x7fa6380c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:24.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.389+0000 7fa63d3b7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa638153330 con 0x7fa638152f50 2024-04-25T01:34:24.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.389+0000 7fa63d3b7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa6380cafb0 con 0x7fa6380c9050 2024-04-25T01:34:24.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.389+0000 7fa63d3b7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa6380c6150 con 0x7fa6380c5cf0 2024-04-25T01:34:24.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.389+0000 7fa62ffff640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6380c5cf0 msgr2=0x7fa6380caa70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:34:24.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.389+0000 7fa62ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6380c5cf0 0x7fa6380caa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:34:24.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.389+0000 7fa6377fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 0x7fa6380c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:24.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.389+0000 7fa636ffd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa638152f50 0x7fa6380c7b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:60306/0 (socket says 172.21.15.112:60306) 2024-04-25T01:34:24.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.389+0000 7fa636ffd640 1 -- 172.21.15.112:0/3010337021 learned_addr learned my addr 172.21.15.112:0/3010337021 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:34:24.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.389+0000 7fa6377fe640 1 -- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 msgr2=0x7fa6380c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:34:24.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.389+0000 7fa6377fe640 1 -- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 msgr2=0x7fa6380c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:34:24.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.389+0000 7fa6377fe640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 0x7fa6380c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:34:24.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.389+0000 7fa6377fe640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 0x7fa6380c94a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:34:24.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.590+0000 7fa62ffff640 1 -- 172.21.15.112:0/3010337021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6380c5cf0 msgr2=0x7fa6380caa70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:34:24.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.590+0000 7fa62ffff640 1 --2- 172.21.15.112:0/3010337021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6380c5cf0 0x7fa6380caa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:34:24.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.590+0000 7fa6377fe640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 0x7fa6380c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:24.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.590+0000 7fa6377fe640 1 -- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 msgr2=0x7fa6380c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:34:24.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.590+0000 7fa6377fe640 1 -- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 msgr2=0x7fa6380c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:34:24.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.590+0000 7fa6377fe640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 0x7fa6380c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:34:24.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.590+0000 7fa6377fe640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 0x7fa6380c94a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:34:24.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.991+0000 7fa62ffff640 1 -- 172.21.15.112:0/3010337021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6380c5cf0 msgr2=0x7fa6380caa70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:34:24.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.991+0000 7fa62ffff640 1 --2- 172.21.15.112:0/3010337021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6380c5cf0 0x7fa6380caa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:34:24.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.991+0000 7fa6377fe640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 0x7fa6380c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:24.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.991+0000 7fa6377fe640 1 -- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 msgr2=0x7fa6380c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:34:24.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.991+0000 7fa6377fe640 1 -- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 msgr2=0x7fa6380c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:34:24.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.991+0000 7fa6377fe640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 0x7fa6380c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:34:24.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:24.991+0000 7fa6377fe640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 0x7fa6380c94a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:34:25.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:25.792+0000 7fa62ffff640 1 -- 172.21.15.112:0/3010337021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6380c5cf0 msgr2=0x7fa6380caa70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:34:25.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:25.792+0000 7fa62ffff640 1 --2- 172.21.15.112:0/3010337021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6380c5cf0 0x7fa6380caa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:34:25.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:25.792+0000 7fa6377fe640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 0x7fa6380c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:25.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:25.792+0000 7fa6377fe640 1 -- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 msgr2=0x7fa6380c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:34:25.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:25.792+0000 7fa6377fe640 1 -- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 msgr2=0x7fa6380c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:34:25.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:25.792+0000 7fa6377fe640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 0x7fa6380c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:34:25.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:25.792+0000 7fa6377fe640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 0x7fa6380c94a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:34:26.574 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:34:26.577 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:34:27.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.389+0000 7fa6357fa640 1 -- 172.21.15.112:0/3010337021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6380c5cf0 msgr2=0x7fa6380caa70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:34:27.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.389+0000 7fa6357fa640 1 --2- 172.21.15.112:0/3010337021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6380c5cf0 0x7fa6380caa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:27.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.389+0000 7fa6357fa640 1 -- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 msgr2=0x7fa6380c94a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:34:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.389+0000 7fa6357fa640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.112:3300/0 conn(0x7fa6380c9050 0x7fa6380c94a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.389+0000 7fa6357fa640 1 -- 172.21.15.112:0/3010337021 >> v1:172.21.15.112:6789/0 conn(0x7fa638152f50 legacy=0x7fa6380c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.389+0000 7fa6357fa640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.187:3300/0 conn(0x7fa6180036f0 0x7fa618005b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.389+0000 7fa6357fa640 1 -- 172.21.15.112:0/3010337021 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa638159dc0 con 0x7fa6180061c0 2024-04-25T01:34:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.389+0000 7fa6357fa640 1 -- 172.21.15.112:0/3010337021 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa6380cafb0 con 0x7fa6180036f0 2024-04-25T01:34:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.389+0000 7fa636ffd640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.187:3300/0 conn(0x7fa6180036f0 0x7fa618005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.390+0000 7fa636ffd640 1 -- 172.21.15.112:0/3010337021 >> v2:172.21.15.187:3300/0 conn(0x7fa6180036f0 msgr2=0x7fa618005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:34:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.390+0000 7fa636ffd640 1 -- 172.21.15.112:0/3010337021 >> v2:172.21.15.187:3300/0 conn(0x7fa6180036f0 msgr2=0x7fa618005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:34:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.390+0000 7fa636ffd640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.187:3300/0 conn(0x7fa6180036f0 0x7fa618005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:34:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.390+0000 7fa636ffd640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.187:3300/0 conn(0x7fa6180036f0 0x7fa618005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:34:27.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.591+0000 7fa636ffd640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.187:3300/0 conn(0x7fa6180036f0 0x7fa618005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:27.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.591+0000 7fa636ffd640 1 -- 172.21.15.112:0/3010337021 >> v2:172.21.15.187:3300/0 conn(0x7fa6180036f0 msgr2=0x7fa618005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:34:27.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.591+0000 7fa636ffd640 1 -- 172.21.15.112:0/3010337021 >> v2:172.21.15.187:3300/0 conn(0x7fa6180036f0 msgr2=0x7fa618005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:34:27.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.591+0000 7fa636ffd640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.187:3300/0 conn(0x7fa6180036f0 0x7fa618005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:34:27.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.591+0000 7fa636ffd640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.187:3300/0 conn(0x7fa6180036f0 0x7fa618005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:34:27.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.992+0000 7fa636ffd640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.187:3300/0 conn(0x7fa6180036f0 0x7fa618005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:27.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.993+0000 7fa636ffd640 1 -- 172.21.15.112:0/3010337021 >> v2:172.21.15.187:3300/0 conn(0x7fa6180036f0 msgr2=0x7fa618005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:34:27.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.993+0000 7fa636ffd640 1 -- 172.21.15.112:0/3010337021 >> v2:172.21.15.187:3300/0 conn(0x7fa6180036f0 msgr2=0x7fa618005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:34:27.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.993+0000 7fa636ffd640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.187:3300/0 conn(0x7fa6180036f0 0x7fa618005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:34:27.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:27.993+0000 7fa636ffd640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.187:3300/0 conn(0x7fa6180036f0 0x7fa618005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:34:28.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.314+0000 7fa6367fc640 1 -- 172.21.15.112:0/3010337021 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3276124173 0 0) 0x7fa638159dc0 con 0x7fa6180061c0 2024-04-25T01:34:28.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.314+0000 7fa6367fc640 1 -- 172.21.15.112:0/3010337021 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa624003520 con 0x7fa6180061c0 2024-04-25T01:34:28.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.321+0000 7fa6367fc640 1 -- 172.21.15.112:0/3010337021 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2320958232 0 0) 0x7fa624003520 con 0x7fa6180061c0 2024-04-25T01:34:28.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.321+0000 7fa6367fc640 1 -- 172.21.15.112:0/3010337021 >> v2:172.21.15.187:3300/0 conn(0x7fa6180036f0 msgr2=0x7fa618005b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:34:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.321+0000 7fa6367fc640 1 --2- 172.21.15.112:0/3010337021 >> v2:172.21.15.187:3300/0 conn(0x7fa6180036f0 0x7fa618005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.321+0000 7fa6367fc640 1 -- 172.21.15.112:0/3010337021 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61800af30 con 0x7fa6180061c0 2024-04-25T01:34:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.321+0000 7fa6367fc640 1 -- 172.21.15.112:0/3010337021 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fa620004340 con 0x7fa6180061c0 2024-04-25T01:34:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.321+0000 7fa6367fc640 1 -- 172.21.15.112:0/3010337021 >> v1:172.21.15.187:6789/0 conn(0x7fa6180061c0 legacy=0x7fa618008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.321+0000 7fa6377fe640 1 -- 172.21.15.112:0/3010337021 reap_dead start 2024-04-25T01:34:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.321+0000 7fa6367fc640 1 --2- 172.21.15.112:0/3010337021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6240039b0 0x7fa624003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.321+0000 7fa6367fc640 1 --2- 172.21.15.112:0/3010337021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa624004800 0x7fa624004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.321+0000 7fa6367fc640 1 --2- 172.21.15.112:0/3010337021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa624005210 0x7fa62400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.321+0000 7fa6367fc640 1 -- 172.21.15.112:0/3010337021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa6380cafb0 con 0x7fa624004800 2024-04-25T01:34:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.321+0000 7fa6367fc640 1 -- 172.21.15.112:0/3010337021 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa62400c140 con 0x7fa6240039b0 2024-04-25T01:34:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.321+0000 7fa6367fc640 1 -- 172.21.15.112:0/3010337021 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa62400c340 con 0x7fa624005210 2024-04-25T01:34:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.322+0000 7fa6377fe640 1 -- 172.21.15.112:0/3010337021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa624005210 msgr2=0x7fa62400b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:34:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.322+0000 7fa6377fe640 1 --2- 172.21.15.112:0/3010337021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa624005210 0x7fa62400b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:34:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.322+0000 7fa62ffff640 1 --2- 172.21.15.112:0/3010337021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa624004800 0x7fa624004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.322+0000 7fa636ffd640 1 --2- 172.21.15.112:0/3010337021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6240039b0 0x7fa624003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.322+0000 7fa62ffff640 1 -- 172.21.15.112:0/3010337021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa624005210 msgr2=0x7fa62400b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:34:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.322+0000 7fa62ffff640 1 --2- 172.21.15.112:0/3010337021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa624005210 0x7fa62400b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.322+0000 7fa62ffff640 1 -- 172.21.15.112:0/3010337021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6240039b0 msgr2=0x7fa624003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:34:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.322+0000 7fa62ffff640 1 --2- 172.21.15.112:0/3010337021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6240039b0 0x7fa624003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.322+0000 7fa62ffff640 1 -- 172.21.15.112:0/3010337021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa624003520 con 0x7fa624004800 2024-04-25T01:34:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.323+0000 7fa62ffff640 1 --2- 172.21.15.112:0/3010337021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa624004800 0x7fa624004c80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa620003930 tx=0x7fa624005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46071791df511e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.323+0000 7fa6367fc640 1 -- 172.21.15.112:0/3010337021 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62000c0e0 con 0x7fa624004800 2024-04-25T01:34:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.323+0000 7fa6367fc640 1 -- 172.21.15.112:0/3010337021 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa620034070 con 0x7fa624004800 2024-04-25T01:34:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.323+0000 7fa6367fc640 1 -- 172.21.15.112:0/3010337021 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62002f870 con 0x7fa624004800 2024-04-25T01:34:28.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.323+0000 7fa63d3b7640 1 -- 172.21.15.112:0/3010337021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa624004800 msgr2=0x7fa624004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.323+0000 7fa63d3b7640 1 --2- 172.21.15.112:0/3010337021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa624004800 0x7fa624004c80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa620003930 tx=0x7fa624005830 comp rx=0 tx=0).stop 2024-04-25T01:34:28.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.324+0000 7fa63d3b7640 1 -- 172.21.15.112:0/3010337021 shutdown_connections 2024-04-25T01:34:28.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.324+0000 7fa63d3b7640 1 --2- 172.21.15.112:0/3010337021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa624005210 0x7fa62400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.324+0000 7fa63d3b7640 1 --2- 172.21.15.112:0/3010337021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa624004800 0x7fa624004c80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.324+0000 7fa63d3b7640 1 --2- 172.21.15.112:0/3010337021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6240039b0 0x7fa624003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.324+0000 7fa63d3b7640 1 -- 172.21.15.112:0/3010337021 >> 172.21.15.112:0/3010337021 conn(0x7fa6380ace40 msgr2=0x7fa6380ad250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:28.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.324+0000 7fa63d3b7640 1 -- 172.21.15.112:0/3010337021 shutdown_connections 2024-04-25T01:34:28.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.324+0000 7fa63d3b7640 1 -- 172.21.15.112:0/3010337021 wait complete. 2024-04-25T01:34:28.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.325+0000 7fa63d3b7640 1 Processor -- start 2024-04-25T01:34:28.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.325+0000 7fa63d3b7640 1 -- start start 2024-04-25T01:34:28.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.325+0000 7fa63d3b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6240039b0 0x7fa6381ec180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.325+0000 7fa63d3b7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa624004800 0x7fa63814c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa63d3b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa624005210 0x7fa6380c3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa63d3b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa6381595c0 con 0x7fa6240039b0 2024-04-25T01:34:28.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa63d3b7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa6381592e0 con 0x7fa624005210 2024-04-25T01:34:28.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa63d3b7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa6381598a0 con 0x7fa624004800 2024-04-25T01:34:28.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa62ffff640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa624004800 msgr2=0x7fa63814c780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:34:28.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa62ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa624004800 0x7fa63814c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:34:28.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa636ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6240039b0 0x7fa6381ec180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa636ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6240039b0 0x7fa6381ec180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58676/0 (socket says 172.21.15.112:58676) 2024-04-25T01:34:28.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa636ffd640 1 -- 172.21.15.112:0/1387103107 learned_addr learned my addr 172.21.15.112:0/1387103107 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:28.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa6377fe640 1 --2- 172.21.15.112:0/1387103107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa624005210 0x7fa6380c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa636ffd640 1 -- 172.21.15.112:0/1387103107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa624004800 msgr2=0x7fa63814c780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:34:28.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa636ffd640 1 --2- 172.21.15.112:0/1387103107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa624004800 0x7fa63814c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa636ffd640 1 -- 172.21.15.112:0/1387103107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa624005210 msgr2=0x7fa6380c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa636ffd640 1 --2- 172.21.15.112:0/1387103107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa624005210 0x7fa6380c3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa636ffd640 1 -- 172.21.15.112:0/1387103107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6380c2470 con 0x7fa6240039b0 2024-04-25T01:34:28.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa636ffd640 1 --2- 172.21.15.112:0/1387103107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6240039b0 0x7fa6381ec180 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa628007870 tx=0x7fa6280016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:28.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa62f7fe640 1 -- 172.21.15.112:0/1387103107 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62801b020 con 0x7fa6240039b0 2024-04-25T01:34:28.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa62f7fe640 1 -- 172.21.15.112:0/1387103107 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa628017670 con 0x7fa6240039b0 2024-04-25T01:34:28.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa63d3b7640 1 -- 172.21.15.112:0/1387103107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6380c2750 con 0x7fa6240039b0 2024-04-25T01:34:28.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa62f7fe640 1 -- 172.21.15.112:0/1387103107 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628017a40 con 0x7fa6240039b0 2024-04-25T01:34:28.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.326+0000 7fa63d3b7640 1 -- 172.21.15.112:0/1387103107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6380c2d10 con 0x7fa6240039b0 2024-04-25T01:34:28.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.328+0000 7fa63d3b7640 1 -- 172.21.15.112:0/1387103107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6380c2200 con 0x7fa6240039b0 2024-04-25T01:34:28.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.328+0000 7fa62f7fe640 1 -- 172.21.15.112:0/1387103107 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa62801e020 con 0x7fa6240039b0 2024-04-25T01:34:28.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.328+0000 7fa62f7fe640 1 --2- 172.21.15.112:0/1387103107 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa61806c680 0x7fa61806eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.328+0000 7fa62f7fe640 1 -- 172.21.15.112:0/1387103107 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fa62808b950 con 0x7fa6240039b0 2024-04-25T01:34:28.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.328+0000 7fa62ffff640 1 --2- 172.21.15.112:0/1387103107 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa61806c680 0x7fa61806eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.329+0000 7fa62ffff640 1 --2- 172.21.15.112:0/1387103107 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa61806c680 0x7fa61806eb40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa620033000 tx=0x7fa62002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:28.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.333+0000 7fa62f7fe640 1 -- 172.21.15.112:0/1387103107 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa628025030 con 0x7fa6240039b0 2024-04-25T01:34:28.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.638+0000 7fa63d3b7640 1 -- 172.21.15.112:0/1387103107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa6380682a0 con 0x7fa6240039b0 2024-04-25T01:34:28.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.638+0000 7fa62f7fe640 1 -- 172.21.15.112:0/1387103107 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7fa638153330 con 0x7fa6240039b0 2024-04-25T01:34:28.615 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":70,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:34:28.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.640+0000 7fa63d3b7640 1 -- 172.21.15.112:0/1387103107 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa61806c680 msgr2=0x7fa61806eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.640+0000 7fa63d3b7640 1 --2- 172.21.15.112:0/1387103107 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa61806c680 0x7fa61806eb40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa620033000 tx=0x7fa62002e000 comp rx=0 tx=0).stop 2024-04-25T01:34:28.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.640+0000 7fa63d3b7640 1 -- 172.21.15.112:0/1387103107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6240039b0 msgr2=0x7fa6381ec180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.640+0000 7fa63d3b7640 1 --2- 172.21.15.112:0/1387103107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6240039b0 0x7fa6381ec180 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa628007870 tx=0x7fa6280016f0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.640+0000 7fa63d3b7640 1 -- 172.21.15.112:0/1387103107 shutdown_connections 2024-04-25T01:34:28.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.640+0000 7fa63d3b7640 1 --2- 172.21.15.112:0/1387103107 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa61806c680 0x7fa61806eb40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.640+0000 7fa63d3b7640 1 --2- 172.21.15.112:0/1387103107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa624005210 0x7fa6380c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.640+0000 7fa63d3b7640 1 --2- 172.21.15.112:0/1387103107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa624004800 0x7fa63814c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.640+0000 7fa63d3b7640 1 --2- 172.21.15.112:0/1387103107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6240039b0 0x7fa6381ec180 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.640+0000 7fa63d3b7640 1 -- 172.21.15.112:0/1387103107 >> 172.21.15.112:0/1387103107 conn(0x7fa6380ace40 msgr2=0x7fa63814f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:28.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.640+0000 7fa63d3b7640 1 -- 172.21.15.112:0/1387103107 shutdown_connections 2024-04-25T01:34:28.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.640+0000 7fa63d3b7640 1 -- 172.21.15.112:0/1387103107 wait complete. 2024-04-25T01:34:28.629 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T01:34:28.629 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-04-25T01:34:28.629 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:34:28.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.778+0000 7f0722a4d640 1 Processor -- start 2024-04-25T01:34:28.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.778+0000 7f0722a4d640 1 -- start start 2024-04-25T01:34:28.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.779+0000 7f0722a4d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f071c1592c0 0x7f071c0c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.779+0000 7f0722a4d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c1596a0 0x7f071c152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.779+0000 7f0722a4d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f071c0c72b0 0x7f071c155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.779+0000 7f0722a4d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f071c0c6e20 con 0x7f071c1592c0 2024-04-25T01:34:28.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.779+0000 7f0722a4d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f071c159b90 con 0x7f071c0c72b0 2024-04-25T01:34:28.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.779+0000 7f0722a4d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f071c06c970 con 0x7f071c1596a0 2024-04-25T01:34:28.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.779+0000 7f071b7fe640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c1596a0 msgr2=0x7f071c152d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:34:28.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.779+0000 7f071b7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c1596a0 0x7f071c152d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:34:28.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.779+0000 7f071bfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f071c1592c0 0x7f071c0c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.779+0000 7f071bfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f071c1592c0 0x7f071c0c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58682/0 (socket says 172.21.15.112:58682) 2024-04-25T01:34:28.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.779+0000 7f071bfff640 1 -- 172.21.15.112:0/2171351441 learned_addr learned my addr 172.21.15.112:0/2171351441 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:28.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.779+0000 7f0720fea640 1 --2- 172.21.15.112:0/2171351441 >> v2:172.21.15.187:3300/0 conn(0x7f071c0c72b0 0x7f071c155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.779+0000 7f071bfff640 1 -- 172.21.15.112:0/2171351441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c1596a0 msgr2=0x7f071c152d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:34:28.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.779+0000 7f071bfff640 1 --2- 172.21.15.112:0/2171351441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c1596a0 0x7f071c152d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.779+0000 7f071bfff640 1 -- 172.21.15.112:0/2171351441 >> v2:172.21.15.187:3300/0 conn(0x7f071c0c72b0 msgr2=0x7f071c155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:34:28.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.779+0000 7f071bfff640 1 --2- 172.21.15.112:0/2171351441 >> v2:172.21.15.187:3300/0 conn(0x7f071c0c72b0 0x7f071c155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.779+0000 7f071bfff640 1 -- 172.21.15.112:0/2171351441 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071c06cc50 con 0x7f071c1592c0 2024-04-25T01:34:28.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.779+0000 7f0720fea640 1 --2- 172.21.15.112:0/2171351441 >> v2:172.21.15.187:3300/0 conn(0x7f071c0c72b0 0x7f071c155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:34:28.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.780+0000 7f071bfff640 1 --2- 172.21.15.112:0/2171351441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c1592c0 0x7f071c0c9070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f071c0aac40 tx=0x7f070c02a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c176f48a4b88a4d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:28.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.780+0000 7f071affd640 1 -- 172.21.15.112:0/2171351441 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c037070 con 0x7f071c1592c0 2024-04-25T01:34:28.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.780+0000 7f071affd640 1 -- 172.21.15.112:0/2171351441 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f070c02ae20 con 0x7f071c1592c0 2024-04-25T01:34:28.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.780+0000 7f071affd640 1 -- 172.21.15.112:0/2171351441 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c032660 con 0x7f071c1592c0 2024-04-25T01:34:28.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.781+0000 7f0722a4d640 1 -- 172.21.15.112:0/2171351441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c1592c0 msgr2=0x7f071c0c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.781+0000 7f0722a4d640 1 --2- 172.21.15.112:0/2171351441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c1592c0 0x7f071c0c9070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f071c0aac40 tx=0x7f070c02a0d0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.781+0000 7f0722a4d640 1 -- 172.21.15.112:0/2171351441 shutdown_connections 2024-04-25T01:34:28.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.781+0000 7f0722a4d640 1 --2- 172.21.15.112:0/2171351441 >> v2:172.21.15.187:3300/0 conn(0x7f071c0c72b0 0x7f071c155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.781+0000 7f0722a4d640 1 --2- 172.21.15.112:0/2171351441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c1596a0 0x7f071c152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.781+0000 7f0722a4d640 1 --2- 172.21.15.112:0/2171351441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c1592c0 0x7f071c0c9070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.781+0000 7f0722a4d640 1 -- 172.21.15.112:0/2171351441 >> 172.21.15.112:0/2171351441 conn(0x7f071c0cb170 msgr2=0x7f071c0cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:28.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.781+0000 7f0722a4d640 1 -- 172.21.15.112:0/2171351441 shutdown_connections 2024-04-25T01:34:28.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.781+0000 7f0722a4d640 1 -- 172.21.15.112:0/2171351441 wait complete. 2024-04-25T01:34:28.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.782+0000 7f0722a4d640 1 Processor -- start 2024-04-25T01:34:28.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.782+0000 7f0722a4d640 1 -- start start 2024-04-25T01:34:28.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.783+0000 7f0722a4d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f071c0c72b0 0x7f071c0ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.783+0000 7f0722a4d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c1592c0 0x7f071c1542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.783+0000 7f0722a4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c1596a0 0x7f071c0c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.783+0000 7f0722a4d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f071c06dca0 con 0x7f071c1596a0 2024-04-25T01:34:28.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.783+0000 7f0722a4d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f071c06db20 con 0x7f071c0c72b0 2024-04-25T01:34:28.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.783+0000 7f0722a4d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f071c0c95b0 con 0x7f071c1592c0 2024-04-25T01:34:28.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.783+0000 7f071b7fe640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c1592c0 msgr2=0x7f071c1542c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:34:28.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.783+0000 7f071b7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c1592c0 0x7f071c1542c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:34:28.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.783+0000 7f0720fea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c1596a0 0x7f071c0c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.783+0000 7f071bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f071c0c72b0 0x7f071c0ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.783+0000 7f0720fea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c1596a0 0x7f071c0c4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58694/0 (socket says 172.21.15.112:58694) 2024-04-25T01:34:28.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.783+0000 7f0720fea640 1 -- 172.21.15.112:0/3450938103 learned_addr learned my addr 172.21.15.112:0/3450938103 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:28.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.783+0000 7f0720fea640 1 -- 172.21.15.112:0/3450938103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c1592c0 msgr2=0x7f071c1542c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:34:28.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.783+0000 7f0720fea640 1 --2- 172.21.15.112:0/3450938103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f071c1592c0 0x7f071c1542c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.783+0000 7f0720fea640 1 -- 172.21.15.112:0/3450938103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f071c0c72b0 msgr2=0x7f071c0ab0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.783+0000 7f0720fea640 1 --2- 172.21.15.112:0/3450938103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f071c0c72b0 0x7f071c0ab0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.783+0000 7f0720fea640 1 -- 172.21.15.112:0/3450938103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071c1ec210 con 0x7f071c1596a0 2024-04-25T01:34:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.783+0000 7f0720fea640 1 --2- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c1596a0 0x7f071c0c4520 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0714009f00 tx=0x7f071400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.784+0000 7f0718ff9640 1 -- 172.21.15.112:0/3450938103 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714018030 con 0x7f071c1596a0 2024-04-25T01:34:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.784+0000 7f0718ff9640 1 -- 172.21.15.112:0/3450938103 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f071400ce50 con 0x7f071c1596a0 2024-04-25T01:34:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.784+0000 7f0722a4d640 1 -- 172.21.15.112:0/3450938103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f071c154cb0 con 0x7f071c1596a0 2024-04-25T01:34:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.784+0000 7f0718ff9640 1 -- 172.21.15.112:0/3450938103 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714003bb0 con 0x7f071c1596a0 2024-04-25T01:34:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.784+0000 7f0722a4d640 1 -- 172.21.15.112:0/3450938103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f071c0ab610 con 0x7f071c1596a0 2024-04-25T01:34:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.785+0000 7f0722a4d640 1 --2- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c154800 0x7f071c0c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.785+0000 7f0722a4d640 1 -- 172.21.15.112:0/3450938103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f071c0c53e0 con 0x7f071c154800 2024-04-25T01:34:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.785+0000 7f071bfff640 1 --2- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c154800 0x7f071c0c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.785+0000 7f071bfff640 1 -- 172.21.15.112:0/3450938103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f071c075980 con 0x7f071c154800 2024-04-25T01:34:28.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.785+0000 7f071bfff640 1 --2- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c154800 0x7f071c0c4a60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f071c08cb00 tx=0x7f070c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:28.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.785+0000 7f0718ff9640 1 -- 172.21.15.112:0/3450938103 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0714005d10 con 0x7f071c1596a0 2024-04-25T01:34:28.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.786+0000 7f0718ff9640 1 --2- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f070806be10 0x7f070806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.786+0000 7f0718ff9640 1 -- 172.21.15.112:0/3450938103 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f071408bb20 con 0x7f071c1596a0 2024-04-25T01:34:28.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.786+0000 7f071b7fe640 1 --2- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f070806be10 0x7f070806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.786+0000 7f0718ff9640 1 -- 172.21.15.112:0/3450938103 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c043070 con 0x7f071c154800 2024-04-25T01:34:28.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.786+0000 7f0718ff9640 1 -- 172.21.15.112:0/3450938103 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f070c03c640 con 0x7f071c154800 2024-04-25T01:34:28.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.786+0000 7f0718ff9640 1 -- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c154800 msgr2=0x7f071c0c4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.786+0000 7f0718ff9640 1 --2- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c154800 0x7f071c0c4a60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f071c08cb00 tx=0x7f070c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.786+0000 7f0718ff9640 1 -- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c154800 msgr2=0x7f071c0c4a60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:34:28.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.786+0000 7f0718ff9640 1 --2- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c154800 0x7f071c0c4a60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.786+0000 7f071b7fe640 1 --2- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f070806be10 0x7f070806e2d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0710006fd0 tx=0x7f0710006d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:28.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.793+0000 7f0722a4d640 1 --2- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c158900 0x7f071c1eed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.793+0000 7f0722a4d640 1 -- 172.21.15.112:0/3450938103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f071c0c8f50 con 0x7f071c158900 2024-04-25T01:34:28.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.793+0000 7f071bfff640 1 --2- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c158900 0x7f071c1eed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.793+0000 7f071bfff640 1 -- 172.21.15.112:0/3450938103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f071c0679a0 con 0x7f071c158900 2024-04-25T01:34:28.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.794+0000 7f071bfff640 1 --2- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c158900 0x7f071c1eed00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f071c155bc0 tx=0x7f070c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:28.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.794+0000 7f0718ff9640 1 -- 172.21.15.112:0/3450938103 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c037040 con 0x7f071c158900 2024-04-25T01:34:28.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.794+0000 7f0718ff9640 1 -- 172.21.15.112:0/3450938103 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4720 (secure 0 0 0) 0x7f070c003820 con 0x7f071c158900 2024-04-25T01:34:28.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.794+0000 7f0718ff9640 1 -- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c158900 msgr2=0x7f071c1eed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.794+0000 7f0718ff9640 1 --2- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c158900 0x7f071c1eed00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f071c155bc0 tx=0x7f070c008030 comp rx=0 tx=0).stop 2024-04-25T01:34:28.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.794+0000 7f0718ff9640 1 -- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c158900 msgr2=0x7f071c1eed00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:34:28.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.794+0000 7f0718ff9640 1 --2- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c158900 0x7f071c1eed00 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f071c155bc0 tx=0x7f070c008030 comp rx=0 tx=0).stop 2024-04-25T01:34:28.772 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:34:28.772 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:34:28.772 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:34:28.772 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:34:28.772 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 70, 2024-04-25T01:34:28.772 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 149560, 2024-04-25T01:34:28.772 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:34:28.772 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:34:28.772 INFO:teuthology.orchestra.run.smithi112.stdout: 1 2024-04-25T01:34:28.772 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:28.773 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:34:28.773 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:34:28.773 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:34:28.773 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:34:28.773 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:28.773 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:28.773 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:28.773 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:28.773 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:28.773 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:28.773 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:28.773 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:28.773 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:28.774 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:28.774 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:28.774 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:28.774 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:34:28.774 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:34:28.774 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:28.774 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:28.774 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:28.774 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:28.774 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:28.774 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:28.774 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:28.774 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:28.774 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:28.774 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:28.775 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:28.775 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:28.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.775 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:34:28.775 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:34:28.775 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:34:28.775 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:34:28.775 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:34:28.775 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:34:28.775 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:34:28.775 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:34:28.775 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:34:28.775 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:34:28.775 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:34:28.776 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:34:28.776 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:34:28.776 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:34:28.776 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:34:28.776 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:34:28.776 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:34:28.776 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:28.776 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:28.776 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:28.776 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:28.776 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:28.776 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:28.776 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:28.776 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:28.776 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:28.777 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:28.777 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:28.777 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:28.777 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:34:28.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.777 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:34:28.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.777 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:34:28.777 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:34:28.777 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:28.777 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:28.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.777 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:28.777 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:34:28.777 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:28.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.778 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.778 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:28.778 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:34:28.778 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:28.778 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:28.778 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:28.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.778 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:34:28.778 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:34:28.778 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:28.778 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:28.778 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:28.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.779 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:34:28.779 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:34:28.779 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:28.779 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:28.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.779 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:28.779 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:34:28.779 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:28.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.780 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:28.780 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:34:28.780 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:28.780 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:28.780 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:28.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.780 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:34:28.780 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:34:28.780 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:28.780 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:28.780 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:28.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.781 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:34:28.781 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:34:28.781 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:28.781 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:28.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.781 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:28.781 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:34:28.781 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:28.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.781 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:28.781 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:34:28.781 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:28.782 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:28.782 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:28.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.782 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:34:28.782 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:34:28.782 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:28.782 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:28.782 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:28.782 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:28.782 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:28.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.782 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:34:28.782 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:34:28.782 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.783 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:28.783 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:28.783 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:34:28.783 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:28.783 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:28.783 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:34:28.783 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.783 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:28.783 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:28.783 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:34:28.783 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:28.783 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:28.784 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:34:28.784 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.784 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:28.784 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:28.784 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:34:28.784 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:28.784 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:28.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.784 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:34:28.784 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:34:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.794+0000 7f0722a4d640 1 -- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f070806be10 msgr2=0x7f070806e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.794+0000 7f0722a4d640 1 --2- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f070806be10 0x7f070806e2d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0710006fd0 tx=0x7f0710006d80 comp rx=0 tx=0).stop 2024-04-25T01:34:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.794+0000 7f0722a4d640 1 -- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c1596a0 msgr2=0x7f071c0c4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.794+0000 7f0722a4d640 1 --2- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c1596a0 0x7f071c0c4520 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0714009f00 tx=0x7f071400c490 comp rx=0 tx=0).stop 2024-04-25T01:34:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.794+0000 7f0720fea640 1 -- 172.21.15.112:0/3450938103 reap_dead start 2024-04-25T01:34:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.794+0000 7f0722a4d640 1 -- 172.21.15.112:0/3450938103 shutdown_connections 2024-04-25T01:34:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.794+0000 7f0722a4d640 1 --2- 172.21.15.112:0/3450938103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071c1596a0 0x7f071c0c4520 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.794+0000 7f0722a4d640 1 -- 172.21.15.112:0/3450938103 >> 172.21.15.112:0/3450938103 conn(0x7f071c0cb170 msgr2=0x7f071c152170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:28.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.794+0000 7f0722a4d640 1 -- 172.21.15.112:0/3450938103 shutdown_connections 2024-04-25T01:34:28.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.794+0000 7f0722a4d640 1 -- 172.21.15.112:0/3450938103 wait complete. 2024-04-25T01:34:28.787 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:34:28.787 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-04-25T01:34:28.787 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:34:28.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.922+0000 7fab6067b640 1 Processor -- start 2024-04-25T01:34:28.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.922+0000 7fab6067b640 1 -- start start 2024-04-25T01:34:28.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.923+0000 7fab6067b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fab5806bdc0 0x7fab5806c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.923+0000 7fab6067b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab580c9290 con 0x7fab580c5b70 2024-04-25T01:34:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.923+0000 7fab6067b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab581565b0 con 0x7fab580cadf0 2024-04-25T01:34:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.923+0000 7fab6067b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fab58156890 con 0x7fab5806bdc0 2024-04-25T01:34:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.923+0000 7fab5ec18640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fab580c5b70 0x7fab581601d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57112/0 (socket says 172.21.15.112:57112) 2024-04-25T01:34:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.923+0000 7fab5e417640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fab5806bdc0 0x7fab5806c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.923+0000 7fab5ec18640 1 -- 172.21.15.112:0/3846137553 learned_addr learned my addr 172.21.15.112:0/3846137553 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:34:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.924+0000 7fab5d415640 1 -- 172.21.15.112:0/3846137553 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1181970118 0 0) 0x7fab580c9290 con 0x7fab580c5b70 2024-04-25T01:34:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.924+0000 7fab5d415640 1 -- 172.21.15.112:0/3846137553 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab44003520 con 0x7fab580c5b70 2024-04-25T01:34:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.924+0000 7fab5e417640 1 -- 172.21.15.112:0/3846137553 >> v1:172.21.15.187:6789/0 conn(0x7fab580cadf0 legacy=0x7fab580c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.924+0000 7fab5e417640 1 -- 172.21.15.112:0/3846137553 >> v1:172.21.15.112:6789/0 conn(0x7fab580c5b70 legacy=0x7fab581601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.924+0000 7fab5e417640 1 -- 172.21.15.112:0/3846137553 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab580c9530 con 0x7fab5806bdc0 2024-04-25T01:34:28.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.925+0000 7fab5e417640 1 --2- 172.21.15.112:0/3846137553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab5806bdc0 0x7fab5806c1a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fab540097a0 tx=0x7fab5402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ff7e5a918308f21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:28.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.925+0000 7fab5d415640 1 -- 172.21.15.112:0/3846137553 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54037070 con 0x7fab5806bdc0 2024-04-25T01:34:28.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.925+0000 7fab5d415640 1 -- 172.21.15.112:0/3846137553 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab54032440 con 0x7fab5806bdc0 2024-04-25T01:34:28.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.925+0000 7fab5d415640 1 -- 172.21.15.112:0/3846137553 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5402ae60 con 0x7fab5806bdc0 2024-04-25T01:34:28.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.925+0000 7fab6067b640 1 -- 172.21.15.112:0/3846137553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab5806bdc0 msgr2=0x7fab5806c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.925+0000 7fab6067b640 1 --2- 172.21.15.112:0/3846137553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab5806bdc0 0x7fab5806c1a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fab540097a0 tx=0x7fab5402a110 comp rx=0 tx=0).stop 2024-04-25T01:34:28.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.925+0000 7fab6067b640 1 -- 172.21.15.112:0/3846137553 shutdown_connections 2024-04-25T01:34:28.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.925+0000 7fab6067b640 1 --2- 172.21.15.112:0/3846137553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab5806bdc0 0x7fab5806c1a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.925+0000 7fab6067b640 1 -- 172.21.15.112:0/3846137553 >> 172.21.15.112:0/3846137553 conn(0x7fab580acd20 msgr2=0x7fab580ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:28.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.925+0000 7fab6067b640 1 -- 172.21.15.112:0/3846137553 shutdown_connections 2024-04-25T01:34:28.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.926+0000 7fab6067b640 1 -- 172.21.15.112:0/3846137553 wait complete. 2024-04-25T01:34:28.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.927+0000 7fab6067b640 1 Processor -- start 2024-04-25T01:34:28.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.927+0000 7fab6067b640 1 -- start start 2024-04-25T01:34:28.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.927+0000 7fab6067b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab5806bdc0 0x7fab580c3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.927+0000 7fab6067b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab580c5b70 0x7fab581eef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.927+0000 7fab6067b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab580cadf0 0x7fab580c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.927+0000 7fab6067b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fab580b0b50 con 0x7fab5806bdc0 2024-04-25T01:34:28.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.927+0000 7fab6067b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fab58156280 con 0x7fab580c5b70 2024-04-25T01:34:28.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.927+0000 7fab6067b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fab580b0cd0 con 0x7fab580cadf0 2024-04-25T01:34:28.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.927+0000 7fab5ec18640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab580cadf0 msgr2=0x7fab580c73b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:34:28.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.927+0000 7fab5e417640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab5806bdc0 0x7fab580c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.928+0000 7fab5ec18640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab580cadf0 0x7fab580c73b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:34:28.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.928+0000 7fab5e417640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab5806bdc0 0x7fab580c3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58726/0 (socket says 172.21.15.112:58726) 2024-04-25T01:34:28.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.928+0000 7fab5e417640 1 -- 172.21.15.112:0/3450795454 learned_addr learned my addr 172.21.15.112:0/3450795454 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:28.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.928+0000 7fab5dc16640 1 --2- 172.21.15.112:0/3450795454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab580c5b70 0x7fab581eef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.928+0000 7fab5e417640 1 -- 172.21.15.112:0/3450795454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab580cadf0 msgr2=0x7fab580c73b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:34:28.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.928+0000 7fab5e417640 1 --2- 172.21.15.112:0/3450795454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab580cadf0 0x7fab580c73b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.928+0000 7fab5e417640 1 -- 172.21.15.112:0/3450795454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab580c5b70 msgr2=0x7fab581eef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.928+0000 7fab5e417640 1 --2- 172.21.15.112:0/3450795454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab580c5b70 0x7fab581eef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.928+0000 7fab5e417640 1 -- 172.21.15.112:0/3450795454 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab580c38c0 con 0x7fab5806bdc0 2024-04-25T01:34:28.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.928+0000 7fab5e417640 1 --2- 172.21.15.112:0/3450795454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab5806bdc0 0x7fab580c3380 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fab540097a0 tx=0x7fab540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:28.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.928+0000 7fab4effd640 1 -- 172.21.15.112:0/3450795454 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54037050 con 0x7fab5806bdc0 2024-04-25T01:34:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.928+0000 7fab4effd640 1 -- 172.21.15.112:0/3450795454 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab54038040 con 0x7fab5806bdc0 2024-04-25T01:34:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.928+0000 7fab4effd640 1 -- 172.21.15.112:0/3450795454 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54004470 con 0x7fab5806bdc0 2024-04-25T01:34:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.928+0000 7fab6067b640 1 -- 172.21.15.112:0/3450795454 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab580bf6b0 con 0x7fab5806bdc0 2024-04-25T01:34:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.928+0000 7fab6067b640 1 -- 172.21.15.112:0/3450795454 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab580bf950 con 0x7fab5806bdc0 2024-04-25T01:34:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.929+0000 7fab6067b640 1 --2- 172.21.15.112:0/3450795454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab580c78f0 0x7fab580c2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.929+0000 7fab6067b640 1 -- 172.21.15.112:0/3450795454 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fab58075980 con 0x7fab580c78f0 2024-04-25T01:34:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.929+0000 7fab5dc16640 1 --2- 172.21.15.112:0/3450795454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab580c78f0 0x7fab580c2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.930+0000 7fab5dc16640 1 -- 172.21.15.112:0/3450795454 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fab580682a0 con 0x7fab580c78f0 2024-04-25T01:34:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.930+0000 7fab5dc16640 1 --2- 172.21.15.112:0/3450795454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab580c78f0 0x7fab580c2280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fab4800f570 tx=0x7fab4800fa30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.930+0000 7fab4effd640 1 -- 172.21.15.112:0/3450795454 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fab5403c6d0 con 0x7fab5806bdc0 2024-04-25T01:34:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.930+0000 7fab4effd640 1 --2- 172.21.15.112:0/3450795454 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fab3c06bde0 0x7fab3c06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.930+0000 7fab4effd640 1 -- 172.21.15.112:0/3450795454 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fab540aba90 con 0x7fab5806bdc0 2024-04-25T01:34:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.930+0000 7fab5ec18640 1 --2- 172.21.15.112:0/3450795454 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fab3c06bde0 0x7fab3c06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.931+0000 7fab4effd640 1 -- 172.21.15.112:0/3450795454 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48019070 con 0x7fab580c78f0 2024-04-25T01:34:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.931+0000 7fab4effd640 1 -- 172.21.15.112:0/3450795454 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fab48005320 con 0x7fab580c78f0 2024-04-25T01:34:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.931+0000 7fab4effd640 1 -- 172.21.15.112:0/3450795454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab580c78f0 msgr2=0x7fab580c2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.931+0000 7fab4effd640 1 --2- 172.21.15.112:0/3450795454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab580c78f0 0x7fab580c2280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fab4800f570 tx=0x7fab4800fa30 comp rx=0 tx=0).stop 2024-04-25T01:34:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.931+0000 7fab4effd640 1 -- 172.21.15.112:0/3450795454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab580c78f0 msgr2=0x7fab580c2280 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:34:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.931+0000 7fab4effd640 1 --2- 172.21.15.112:0/3450795454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab580c78f0 0x7fab580c2280 secure :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fab4800f570 tx=0x7fab4800fa30 comp rx=0 tx=0).stop 2024-04-25T01:34:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.931+0000 7fab5ec18640 1 --2- 172.21.15.112:0/3450795454 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fab3c06bde0 0x7fab3c06e2a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fab5814ef10 tx=0x7fab50001670 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:28.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.939+0000 7fab6067b640 1 --2- 172.21.15.112:0/3450795454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab581eeab0 0x7fab580c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.939+0000 7fab6067b640 1 -- 172.21.15.112:0/3450795454 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fab580cabd0 con 0x7fab581eeab0 2024-04-25T01:34:28.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.939+0000 7fab5dc16640 1 --2- 172.21.15.112:0/3450795454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab581eeab0 0x7fab580c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.939+0000 7fab5dc16640 1 -- 172.21.15.112:0/3450795454 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fab58152bb0 con 0x7fab581eeab0 2024-04-25T01:34:28.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.939+0000 7fab5dc16640 1 --2- 172.21.15.112:0/3450795454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab581eeab0 0x7fab580c27c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fab580c0690 tx=0x7fab48000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:28.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.939+0000 7fab4effd640 1 -- 172.21.15.112:0/3450795454 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48019030 con 0x7fab581eeab0 2024-04-25T01:34:28.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.940+0000 7fab4effd640 1 -- 172.21.15.112:0/3450795454 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5058 (secure 0 0 0) 0x7fab48006af0 con 0x7fab581eeab0 2024-04-25T01:34:28.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.940+0000 7fab4effd640 1 -- 172.21.15.112:0/3450795454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab581eeab0 msgr2=0x7fab580c27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.940+0000 7fab4effd640 1 --2- 172.21.15.112:0/3450795454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab581eeab0 0x7fab580c27c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fab580c0690 tx=0x7fab48000ed0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.940+0000 7fab4effd640 1 -- 172.21.15.112:0/3450795454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab581eeab0 msgr2=0x7fab580c27c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:34:28.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.940+0000 7fab4effd640 1 --2- 172.21.15.112:0/3450795454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab581eeab0 0x7fab580c27c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.917 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:34:28.917 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:34:28.917 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:34:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:34:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 70, 2024-04-25T01:34:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 304628, 2024-04-25T01:34:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:34:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:34:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: 1 2024-04-25T01:34:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:34:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:34:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:34:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:34:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:34:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:34:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:34:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:34:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:34:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:34:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:34:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:34:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:34:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:34:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:34:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:34:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:34:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:34:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:34:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:34:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:34:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:34:28.922 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:34:28.922 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:34:28.922 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:34:28.922 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:34:28.922 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:34:28.922 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:34:28.922 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:34:28.922 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:34:28.922 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:34:28.922 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:34:28.922 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:34:28.922 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:34:28.922 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:28.923 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:34:28.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.923 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:34:28.923 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.923 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:34:28.923 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:34:28.923 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:28.923 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:28.923 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.923 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:28.923 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:34:28.923 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:28.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.923 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.923 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:28.923 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:34:28.924 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:28.924 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:28.924 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:28.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.924 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:34:28.924 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:34:28.924 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:28.924 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:28.924 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:28.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.924 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.924 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:34:28.925 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:34:28.925 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:28.925 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:28.925 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.925 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:28.925 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:34:28.925 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:28.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.925 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.925 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:28.925 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:34:28.925 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:28.925 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:28.926 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:28.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.926 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:34:28.926 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:34:28.926 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:28.926 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:28.926 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:28.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.926 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.926 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:34:28.926 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:34:28.926 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:34:28.926 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:34:28.927 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.927 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:34:28.927 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:34:28.927 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:28.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.927 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.927 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:34:28.927 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:34:28.927 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:34:28.927 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:28.927 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:28.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.927 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:34:28.927 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:34:28.928 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:34:28.928 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:34:28.928 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:34:28.928 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:28.928 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:28.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.928 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:34:28.928 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:34:28.928 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.928 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:28.928 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:28.928 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:34:28.928 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:28.928 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:28.929 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:34:28.929 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.929 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:28.929 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:28.929 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:34:28.929 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:28.929 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:28.929 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:34:28.929 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.929 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:28.929 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:28.929 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:34:28.929 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:28.930 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:28.930 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:34:28.930 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.930 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:28.930 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:28.930 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:34:28.930 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:28.930 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:34:28.930 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:34:28.930 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:34:28.930 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:34:28.930 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:34:28.930 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:34:28.931 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:34:28.931 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:34:28.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:34:28.931 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:34:28.931 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:34:28.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.940+0000 7fab6067b640 1 -- 172.21.15.112:0/3450795454 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fab3c06bde0 msgr2=0x7fab3c06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.940+0000 7fab6067b640 1 --2- 172.21.15.112:0/3450795454 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fab3c06bde0 0x7fab3c06e2a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fab5814ef10 tx=0x7fab50001670 comp rx=0 tx=0).stop 2024-04-25T01:34:28.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.940+0000 7fab6067b640 1 -- 172.21.15.112:0/3450795454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab5806bdc0 msgr2=0x7fab580c3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.940+0000 7fab6067b640 1 --2- 172.21.15.112:0/3450795454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab5806bdc0 0x7fab580c3380 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fab540097a0 tx=0x7fab540046c0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.940+0000 7fab5ec18640 1 -- 172.21.15.112:0/3450795454 reap_dead start 2024-04-25T01:34:28.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.940+0000 7fab6067b640 1 -- 172.21.15.112:0/3450795454 shutdown_connections 2024-04-25T01:34:28.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.940+0000 7fab6067b640 1 --2- 172.21.15.112:0/3450795454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab5806bdc0 0x7fab580c3380 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.940+0000 7fab6067b640 1 -- 172.21.15.112:0/3450795454 >> 172.21.15.112:0/3450795454 conn(0x7fab580acd20 msgr2=0x7fab580af300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:28.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.940+0000 7fab6067b640 1 -- 172.21.15.112:0/3450795454 shutdown_connections 2024-04-25T01:34:28.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:28.940+0000 7fab6067b640 1 -- 172.21.15.112:0/3450795454 wait complete. 2024-04-25T01:34:28.933 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:34:28.933 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T01:34:48.934 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-25T01:34:48.938 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-04-25T01:34:48.938 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T01:34:48.938 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T01:34:48.941 INFO:tasks.ceph.mon.c:Started 2024-04-25T01:34:48.941 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:34:48.942 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:34:54.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:54.963+0000 7f94c961c640 1 Processor -- start 2024-04-25T01:34:54.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:54.963+0000 7f94c961c640 1 -- start start 2024-04-25T01:34:54.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:54.964+0000 7f94c961c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 0x7f94c406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:54.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:54.964+0000 7f94c961c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 0x7f94c41525b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:54.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:54.964+0000 7f94c961c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c40c6a70 con 0x7f94c40746a0 2024-04-25T01:34:54.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:54.964+0000 7f94c961c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f94c4155ce0 con 0x7f94c414fbe0 2024-04-25T01:34:54.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:54.964+0000 7f94c961c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f94c4155fc0 con 0x7f94c40c7250 2024-04-25T01:34:54.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:54.964+0000 7f94c2ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 0x7f94c406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:54.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:54.964+0000 7f94c2ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 0x7f94c406c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47910/0 (socket says 172.21.15.112:47910) 2024-04-25T01:34:54.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:54.965+0000 7f94c2ffd640 1 -- 172.21.15.112:0/3267295341 learned_addr learned my addr 172.21.15.112:0/3267295341 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:54.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:54.965+0000 7f94c27fc640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 0x7f94c41525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:54.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:54.965+0000 7f94c2ffd640 1 -- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 msgr2=0x7f94c406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:34:54.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:54.965+0000 7f94c2ffd640 1 -- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 msgr2=0x7f94c406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:34:54.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:54.965+0000 7f94c2ffd640 1 --2- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 0x7f94c406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:34:54.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:54.965+0000 7f94c2ffd640 1 --2- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 0x7f94c406c170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:34:54.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:54.965+0000 7f94c27fc640 1 -- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 msgr2=0x7f94c41525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:34:54.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:54.965+0000 7f94c27fc640 1 -- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 msgr2=0x7f94c41525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:34:54.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:54.965+0000 7f94c27fc640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 0x7f94c41525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:34:54.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:54.965+0000 7f94c27fc640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 0x7f94c41525b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:34:55.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:55.166+0000 7f94c2ffd640 1 --2- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 0x7f94c406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:55.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:55.166+0000 7f94c27fc640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 0x7f94c41525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:55.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:55.166+0000 7f94c2ffd640 1 -- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 msgr2=0x7f94c406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:34:55.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:55.166+0000 7f94c2ffd640 1 -- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 msgr2=0x7f94c406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:34:55.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:55.166+0000 7f94c2ffd640 1 --2- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 0x7f94c406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:34:55.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:55.166+0000 7f94c2ffd640 1 --2- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 0x7f94c406c170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:34:55.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:55.166+0000 7f94c27fc640 1 -- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 msgr2=0x7f94c41525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:34:55.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:55.166+0000 7f94c27fc640 1 -- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 msgr2=0x7f94c41525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:34:55.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:55.166+0000 7f94c27fc640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 0x7f94c41525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:34:55.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:55.166+0000 7f94c27fc640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 0x7f94c41525b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:34:55.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:55.567+0000 7f94c2ffd640 1 --2- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 0x7f94c406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:55.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:55.567+0000 7f94c27fc640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 0x7f94c41525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:55.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:55.567+0000 7f94c2ffd640 1 -- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 msgr2=0x7f94c406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:34:55.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:55.567+0000 7f94c2ffd640 1 -- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 msgr2=0x7f94c406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:34:55.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:55.567+0000 7f94c2ffd640 1 --2- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 0x7f94c406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:34:55.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:55.567+0000 7f94c2ffd640 1 --2- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 0x7f94c406c170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:34:55.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:55.567+0000 7f94c27fc640 1 -- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 msgr2=0x7f94c41525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:34:55.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:55.567+0000 7f94c27fc640 1 -- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 msgr2=0x7f94c41525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:34:55.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:55.567+0000 7f94c27fc640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 0x7f94c41525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:34:55.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:55.567+0000 7f94c27fc640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 0x7f94c41525b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:34:56.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:56.369+0000 7f94c2ffd640 1 --2- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 0x7f94c406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:56.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:56.369+0000 7f94c27fc640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 0x7f94c41525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:56.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:56.369+0000 7f94c2ffd640 1 -- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 msgr2=0x7f94c406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:34:56.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:56.369+0000 7f94c2ffd640 1 -- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 msgr2=0x7f94c406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:34:56.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:56.369+0000 7f94c2ffd640 1 --2- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 0x7f94c406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:34:56.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:56.369+0000 7f94c2ffd640 1 --2- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 0x7f94c406c170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:34:56.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:56.369+0000 7f94c27fc640 1 -- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 msgr2=0x7f94c41525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:34:56.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:56.369+0000 7f94c27fc640 1 -- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 msgr2=0x7f94c41525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:34:56.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:56.369+0000 7f94c27fc640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 0x7f94c41525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:34:56.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:56.369+0000 7f94c27fc640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 0x7f94c41525b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:34:56.631 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:34:56.634 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:34:57.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.364+0000 7f94c1ffb640 1 -- 172.21.15.112:0/3267295341 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 364301 0 0) 0x7f94c40c6a70 con 0x7f94c40746a0 2024-04-25T01:34:57.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.364+0000 7f94c1ffb640 1 -- 172.21.15.112:0/3267295341 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94ac003520 con 0x7f94c40746a0 2024-04-25T01:34:57.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.364+0000 7f94c1ffb640 1 -- 172.21.15.112:0/3267295341 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 324348780 0 0) 0x7f94ac003520 con 0x7f94c40746a0 2024-04-25T01:34:57.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.364+0000 7f94c1ffb640 1 -- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 msgr2=0x7f94c41525b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:34:57.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.364+0000 7f94c1ffb640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 0x7f94c41525b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:57.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.364+0000 7f94c1ffb640 1 -- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 msgr2=0x7f94c406c170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:34:57.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.364+0000 7f94c1ffb640 1 --2- 172.21.15.112:0/3267295341 >> v2:172.21.15.112:3300/0 conn(0x7f94c414fbe0 0x7f94c406c170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:57.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.364+0000 7f94c1ffb640 1 -- 172.21.15.112:0/3267295341 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c41562a0 con 0x7f94c40746a0 2024-04-25T01:34:57.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.364+0000 7f94c1ffb640 1 -- 172.21.15.112:0/3267295341 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f94bc0019f0 con 0x7f94c40746a0 2024-04-25T01:34:57.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.364+0000 7f94c1ffb640 1 -- 172.21.15.112:0/3267295341 >> v1:172.21.15.187:6789/0 conn(0x7f94c40746a0 legacy=0x7f94c4152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:57.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.364+0000 7f94c1ffb640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94ac0039b0 0x7f94ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:57.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.364+0000 7f94c1ffb640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 0x7f94ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:57.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.364+0000 7f94c1ffb640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94ac004d40 0x7f94ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:57.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.364+0000 7f94c1ffb640 1 -- 172.21.15.112:0/3267295341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f94c4155ce0 con 0x7f94ac0039b0 2024-04-25T01:34:57.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.364+0000 7f94c1ffb640 1 -- 172.21.15.112:0/3267295341 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f94c4155fc0 con 0x7f94ac004d40 2024-04-25T01:34:57.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.364+0000 7f94c1ffb640 1 -- 172.21.15.112:0/3267295341 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f94ac00bbd0 con 0x7f94c40c7250 2024-04-25T01:34:57.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.365+0000 7f94c37fe640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94ac004d40 0x7f94ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:57.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.365+0000 7f94c27fc640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 0x7f94ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:57.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.365+0000 7f94c37fe640 1 -- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 msgr2=0x7f94ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:34:57.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.365+0000 7f94c37fe640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40c7250 0x7f94ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:57.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.365+0000 7f94c37fe640 1 -- 172.21.15.112:0/3267295341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94ac0039b0 msgr2=0x7f94ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:34:57.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.365+0000 7f94c37fe640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94ac0039b0 0x7f94ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:57.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.365+0000 7f94c37fe640 1 -- 172.21.15.112:0/3267295341 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ac003520 con 0x7f94ac004d40 2024-04-25T01:34:57.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.366+0000 7f94c37fe640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94ac004d40 0x7f94ac0051c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00c490 tx=0x7f94ac005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8702cde8762fd4bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:57.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.366+0000 7f94c1ffb640 1 -- 172.21.15.112:0/3267295341 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc00c0e0 con 0x7f94ac004d40 2024-04-25T01:34:57.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.366+0000 7f94c1ffb640 1 -- 172.21.15.112:0/3267295341 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94bc02c660 con 0x7f94ac004d40 2024-04-25T01:34:57.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.366+0000 7f94c1ffb640 1 -- 172.21.15.112:0/3267295341 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc02ca50 con 0x7f94ac004d40 2024-04-25T01:34:57.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.367+0000 7f94c961c640 1 -- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94ac004d40 msgr2=0x7f94ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:57.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.367+0000 7f94c961c640 1 --2- 172.21.15.112:0/3267295341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94ac004d40 0x7f94ac0051c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00c490 tx=0x7f94ac005830 comp rx=0 tx=0).stop 2024-04-25T01:34:57.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.367+0000 7f94c37fe640 1 -- 172.21.15.112:0/3267295341 reap_dead start 2024-04-25T01:34:57.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.367+0000 7f94c961c640 1 -- 172.21.15.112:0/3267295341 shutdown_connections 2024-04-25T01:34:57.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.367+0000 7f94c961c640 1 -- 172.21.15.112:0/3267295341 >> 172.21.15.112:0/3267295341 conn(0x7f94c40c42d0 msgr2=0x7f94c40c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:57.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.367+0000 7f94c961c640 1 -- 172.21.15.112:0/3267295341 shutdown_connections 2024-04-25T01:34:57.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.367+0000 7f94c961c640 1 -- 172.21.15.112:0/3267295341 wait complete. 2024-04-25T01:34:57.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.369+0000 7f94c961c640 1 Processor -- start 2024-04-25T01:34:57.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.369+0000 7f94c961c640 1 -- start start 2024-04-25T01:34:57.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.370+0000 7f94c961c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94ac004d40 0x7f94c414f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:57.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.370+0000 7f94c961c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40bc7f0 0x7f94c40c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:57.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.370+0000 7f94c961c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94c40c5c10 0x7f94c414e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:57.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.370+0000 7f94c961c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f94c414ffc0 con 0x7f94ac004d40 2024-04-25T01:34:57.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.370+0000 7f94c961c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f94c4156860 con 0x7f94c40c5c10 2024-04-25T01:34:57.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.370+0000 7f94c961c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f94c4153200 con 0x7f94c40bc7f0 2024-04-25T01:34:57.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.370+0000 7f94c2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94ac004d40 0x7f94c414f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:57.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.370+0000 7f94c2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94ac004d40 0x7f94c414f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47938/0 (socket says 172.21.15.112:47938) 2024-04-25T01:34:57.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.370+0000 7f94c2ffd640 1 -- 172.21.15.112:0/2762915398 learned_addr learned my addr 172.21.15.112:0/2762915398 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:34:57.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.370+0000 7f94c27fc640 1 --2- 172.21.15.112:0/2762915398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40bc7f0 0x7f94c40c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:57.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.370+0000 7f94c37fe640 1 --2- 172.21.15.112:0/2762915398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94c40c5c10 0x7f94c414e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:57.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.370+0000 7f94c27fc640 1 -- 172.21.15.112:0/2762915398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40bc7f0 msgr2=0x7f94c40c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T01:34:57.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.370+0000 7f94c27fc640 1 -- 172.21.15.112:0/2762915398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40bc7f0 msgr2=0x7f94c40c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:34:57.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.370+0000 7f94c27fc640 1 --2- 172.21.15.112:0/2762915398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40bc7f0 0x7f94c40c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:34:57.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.370+0000 7f94c27fc640 1 --2- 172.21.15.112:0/2762915398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40bc7f0 0x7f94c40c56d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:34:57.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.370+0000 7f94c2ffd640 1 -- 172.21.15.112:0/2762915398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40bc7f0 msgr2=0x7f94c40c56d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:34:57.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.370+0000 7f94c2ffd640 1 --2- 172.21.15.112:0/2762915398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40bc7f0 0x7f94c40c56d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:57.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.370+0000 7f94c2ffd640 1 -- 172.21.15.112:0/2762915398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94c40c5c10 msgr2=0x7f94c414e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:57.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.370+0000 7f94c2ffd640 1 --2- 172.21.15.112:0/2762915398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94c40c5c10 0x7f94c414e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:57.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.370+0000 7f94c2ffd640 1 -- 172.21.15.112:0/2762915398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c41e9e20 con 0x7f94ac004d40 2024-04-25T01:34:57.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.371+0000 7f94c2ffd640 1 --2- 172.21.15.112:0/2762915398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94ac004d40 0x7f94c414f040 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f94b40146a0 tx=0x7f94b4014b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:57.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.371+0000 7f94ab7fe640 1 -- 172.21.15.112:0/2762915398 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b401d070 con 0x7f94ac004d40 2024-04-25T01:34:57.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.372+0000 7f94c961c640 1 -- 172.21.15.112:0/2762915398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c41dd0c0 con 0x7f94ac004d40 2024-04-25T01:34:57.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.372+0000 7f94c961c640 1 -- 172.21.15.112:0/2762915398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c41dd680 con 0x7f94ac004d40 2024-04-25T01:34:57.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.372+0000 7f94ab7fe640 1 -- 172.21.15.112:0/2762915398 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b4010a30 con 0x7f94ac004d40 2024-04-25T01:34:57.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.372+0000 7f94ab7fe640 1 -- 172.21.15.112:0/2762915398 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b4027060 con 0x7f94ac004d40 2024-04-25T01:34:57.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.373+0000 7f94c961c640 1 -- 172.21.15.112:0/2762915398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a4005290 con 0x7f94ac004d40 2024-04-25T01:34:57.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.374+0000 7f94ab7fe640 1 -- 172.21.15.112:0/2762915398 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f94b4020070 con 0x7f94ac004d40 2024-04-25T01:34:57.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.374+0000 7f94ab7fe640 1 --2- 172.21.15.112:0/2762915398 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f94b006be30 0x7f94b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:57.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.374+0000 7f94ab7fe640 1 -- 172.21.15.112:0/2762915398 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f94b408b5d0 con 0x7f94ac004d40 2024-04-25T01:34:57.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.374+0000 7f94c27fc640 1 --2- 172.21.15.112:0/2762915398 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f94b006be30 0x7f94b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:57.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.375+0000 7f94c27fc640 1 --2- 172.21.15.112:0/2762915398 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f94b006be30 0x7f94b006e2f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f94c41dfbe0 tx=0x7f94b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:57.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.379+0000 7f94ab7fe640 1 -- 172.21.15.112:0/2762915398 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f94b40595a0 con 0x7f94ac004d40 2024-04-25T01:34:57.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.698+0000 7f94c961c640 1 -- 172.21.15.112:0/2762915398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f94a4005850 con 0x7f94ac004d40 2024-04-25T01:34:57.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.698+0000 7f94ab7fe640 1 -- 172.21.15.112:0/2762915398 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7f94b405d8a0 con 0x7f94ac004d40 2024-04-25T01:34:57.675 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":74,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:34:57.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.701+0000 7f94c961c640 1 -- 172.21.15.112:0/2762915398 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f94b006be30 msgr2=0x7f94b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:57.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.701+0000 7f94c961c640 1 --2- 172.21.15.112:0/2762915398 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f94b006be30 0x7f94b006e2f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f94c41dfbe0 tx=0x7f94b8008040 comp rx=0 tx=0).stop 2024-04-25T01:34:57.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.701+0000 7f94c961c640 1 -- 172.21.15.112:0/2762915398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94ac004d40 msgr2=0x7f94c414f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:57.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.701+0000 7f94c961c640 1 --2- 172.21.15.112:0/2762915398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94ac004d40 0x7f94c414f040 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f94b40146a0 tx=0x7f94b4014b60 comp rx=0 tx=0).stop 2024-04-25T01:34:57.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.701+0000 7f94c961c640 1 -- 172.21.15.112:0/2762915398 shutdown_connections 2024-04-25T01:34:57.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.701+0000 7f94c961c640 1 --2- 172.21.15.112:0/2762915398 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f94b006be30 0x7f94b006e2f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:57.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.701+0000 7f94c961c640 1 --2- 172.21.15.112:0/2762915398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94c40c5c10 0x7f94c414e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:57.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.701+0000 7f94c961c640 1 --2- 172.21.15.112:0/2762915398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94c40bc7f0 0x7f94c40c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:57.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.701+0000 7f94c961c640 1 --2- 172.21.15.112:0/2762915398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94ac004d40 0x7f94c414f040 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:57.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.701+0000 7f94c961c640 1 -- 172.21.15.112:0/2762915398 >> 172.21.15.112:0/2762915398 conn(0x7f94c40c42d0 msgr2=0x7f94c40c3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:57.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.701+0000 7f94c961c640 1 -- 172.21.15.112:0/2762915398 shutdown_connections 2024-04-25T01:34:57.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:34:57.701+0000 7f94c961c640 1 -- 172.21.15.112:0/2762915398 wait complete. 2024-04-25T01:35:00.691 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:35:00.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.861+0000 7f285533b640 1 Processor -- start 2024-04-25T01:35:00.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.861+0000 7f285533b640 1 -- start start 2024-04-25T01:35:00.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.862+0000 7f285533b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f28500c2bc0 0x7f2850157290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:00.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.862+0000 7f285533b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2850154620 0x7f28501577d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.862+0000 7f285533b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28500c28e0 con 0x7f285006c3e0 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.862+0000 7f285533b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f28500c3cd0 con 0x7f2850154620 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.862+0000 7f285533b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f28500c2690 con 0x7f28500c2bc0 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.862+0000 7f284f7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2850154620 0x7f28501577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.862+0000 7f284effd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f285006c3e0 0x7f2850153f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:44018/0 (socket says 172.21.15.112:44018) 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.862+0000 7f284effd640 1 -- 172.21.15.112:0/2505871227 learned_addr learned my addr 172.21.15.112:0/2505871227 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.862+0000 7f284e7fc640 1 --2- 172.21.15.112:0/2505871227 >> v2:172.21.15.187:3300/0 conn(0x7f28500c2bc0 0x7f2850157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.863+0000 7f284f7fe640 1 -- 172.21.15.112:0/2505871227 >> v2:172.21.15.187:3300/0 conn(0x7f28500c2bc0 msgr2=0x7f2850157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.863+0000 7f284f7fe640 1 --2- 172.21.15.112:0/2505871227 >> v2:172.21.15.187:3300/0 conn(0x7f28500c2bc0 0x7f2850157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.863+0000 7f284f7fe640 1 -- 172.21.15.112:0/2505871227 >> v1:172.21.15.112:6789/0 conn(0x7f285006c3e0 legacy=0x7f2850153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.863+0000 7f284dffb640 1 -- 172.21.15.112:0/2505871227 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2143534743 0 0) 0x7f28500c28e0 con 0x7f285006c3e0 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.863+0000 7f284f7fe640 1 -- 172.21.15.112:0/2505871227 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2850074560 con 0x7f2850154620 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.863+0000 7f284f7fe640 1 --2- 172.21.15.112:0/2505871227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2850154620 0x7f28501577d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2850073940 tx=0x7f284802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadb647ca7da8487 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.864+0000 7f284dffb640 1 -- 172.21.15.112:0/2505871227 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848007740 con 0x7f2850154620 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.864+0000 7f284dffb640 1 -- 172.21.15.112:0/2505871227 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28480078e0 con 0x7f2850154620 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.864+0000 7f284dffb640 1 -- 172.21.15.112:0/2505871227 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848007cb0 con 0x7f2850154620 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.864+0000 7f285533b640 1 -- 172.21.15.112:0/2505871227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2850154620 msgr2=0x7f28501577d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.864+0000 7f285533b640 1 --2- 172.21.15.112:0/2505871227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2850154620 0x7f28501577d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2850073940 tx=0x7f284802a5f0 comp rx=0 tx=0).stop 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.864+0000 7f285533b640 1 -- 172.21.15.112:0/2505871227 shutdown_connections 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.864+0000 7f285533b640 1 --2- 172.21.15.112:0/2505871227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2850154620 0x7f28501577d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.864+0000 7f285533b640 1 --2- 172.21.15.112:0/2505871227 >> v2:172.21.15.187:3300/0 conn(0x7f28500c2bc0 0x7f2850157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.864+0000 7f285533b640 1 -- 172.21.15.112:0/2505871227 >> 172.21.15.112:0/2505871227 conn(0x7f28500c42d0 msgr2=0x7f28500c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:00.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.864+0000 7f285533b640 1 -- 172.21.15.112:0/2505871227 shutdown_connections 2024-04-25T01:35:00.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.864+0000 7f285533b640 1 -- 172.21.15.112:0/2505871227 wait complete. 2024-04-25T01:35:00.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.865+0000 7f285533b640 1 Processor -- start 2024-04-25T01:35:00.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.866+0000 7f285533b640 1 -- start start 2024-04-25T01:35:00.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.866+0000 7f285533b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f285006c3e0 0x7f28501ef020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:00.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.866+0000 7f285533b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28500c2bc0 0x7f28500c0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:00.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.866+0000 7f285533b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2850154620 0x7f285014fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:00.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.866+0000 7f285533b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2850157d10 con 0x7f2850154620 2024-04-25T01:35:00.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.866+0000 7f285533b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2850153900 con 0x7f28500c2bc0 2024-04-25T01:35:00.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.866+0000 7f285533b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f28500c7350 con 0x7f285006c3e0 2024-04-25T01:35:00.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.866+0000 7f284f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2850154620 0x7f285014fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:00.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.866+0000 7f284f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2850154620 0x7f285014fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59660/0 (socket says 172.21.15.112:59660) 2024-04-25T01:35:00.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.866+0000 7f284f7fe640 1 -- 172.21.15.112:0/3269583406 learned_addr learned my addr 172.21.15.112:0/3269583406 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:00.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.866+0000 7f284e7fc640 1 --2- 172.21.15.112:0/3269583406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28500c2bc0 0x7f28500c0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:00.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.866+0000 7f284effd640 1 --2- 172.21.15.112:0/3269583406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f285006c3e0 0x7f28501ef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:00.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.867+0000 7f284f7fe640 1 -- 172.21.15.112:0/3269583406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f285006c3e0 msgr2=0x7f28501ef020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:00.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.867+0000 7f284f7fe640 1 --2- 172.21.15.112:0/3269583406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f285006c3e0 0x7f28501ef020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:00.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.867+0000 7f284f7fe640 1 -- 172.21.15.112:0/3269583406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28500c2bc0 msgr2=0x7f28500c0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:00.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.867+0000 7f284f7fe640 1 --2- 172.21.15.112:0/3269583406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28500c2bc0 0x7f28500c0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:00.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.867+0000 7f284f7fe640 1 -- 172.21.15.112:0/3269583406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285009b330 con 0x7f2850154620 2024-04-25T01:35:00.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.867+0000 7f284f7fe640 1 --2- 172.21.15.112:0/3269583406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2850154620 0x7f285014fba0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f284802abe0 tx=0x7f2848004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:00.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.867+0000 7f28377fe640 1 -- 172.21.15.112:0/3269583406 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284803f070 con 0x7f2850154620 2024-04-25T01:35:00.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.867+0000 7f28377fe640 1 -- 172.21.15.112:0/3269583406 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2848031c40 con 0x7f2850154620 2024-04-25T01:35:00.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.867+0000 7f28377fe640 1 -- 172.21.15.112:0/3269583406 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848009450 con 0x7f2850154620 2024-04-25T01:35:00.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.867+0000 7f285533b640 1 -- 172.21.15.112:0/3269583406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285009b610 con 0x7f2850154620 2024-04-25T01:35:00.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.867+0000 7f285533b640 1 -- 172.21.15.112:0/3269583406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28500c13b0 con 0x7f2850154620 2024-04-25T01:35:00.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.868+0000 7f285533b640 1 -- 172.21.15.112:0/3269583406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28500689b0 con 0x7f2850154620 2024-04-25T01:35:00.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.869+0000 7f28377fe640 1 -- 172.21.15.112:0/3269583406 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2848003990 con 0x7f2850154620 2024-04-25T01:35:00.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.869+0000 7f28377fe640 1 --2- 172.21.15.112:0/3269583406 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f283c06be30 0x7f283c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:00.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.869+0000 7f28377fe640 1 -- 172.21.15.112:0/3269583406 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f28480abe30 con 0x7f2850154620 2024-04-25T01:35:00.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.869+0000 7f284effd640 1 --2- 172.21.15.112:0/3269583406 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f283c06be30 0x7f283c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:00.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.869+0000 7f284effd640 1 --2- 172.21.15.112:0/3269583406 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f283c06be30 0x7f283c06e2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2840000f60 tx=0x7f2840007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:00.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:00.874+0000 7f28377fe640 1 -- 172.21.15.112:0/3269583406 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2848079e80 con 0x7f2850154620 2024-04-25T01:35:01.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.167+0000 7f285533b640 1 -- 172.21.15.112:0/3269583406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f28500c66e0 con 0x7f2850154620 2024-04-25T01:35:01.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.167+0000 7f28377fe640 1 -- 172.21.15.112:0/3269583406 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1568 (secure 0 0 0) 0x7f284807e180 con 0x7f2850154620 2024-04-25T01:35:01.144 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":76,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:35:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.169+0000 7f285533b640 1 -- 172.21.15.112:0/3269583406 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f283c06be30 msgr2=0x7f283c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.169+0000 7f285533b640 1 --2- 172.21.15.112:0/3269583406 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f283c06be30 0x7f283c06e2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2840000f60 tx=0x7f2840007d50 comp rx=0 tx=0).stop 2024-04-25T01:35:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.169+0000 7f285533b640 1 -- 172.21.15.112:0/3269583406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2850154620 msgr2=0x7f285014fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.169+0000 7f285533b640 1 --2- 172.21.15.112:0/3269583406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2850154620 0x7f285014fba0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f284802abe0 tx=0x7f2848004a40 comp rx=0 tx=0).stop 2024-04-25T01:35:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.169+0000 7f285533b640 1 -- 172.21.15.112:0/3269583406 shutdown_connections 2024-04-25T01:35:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.169+0000 7f285533b640 1 --2- 172.21.15.112:0/3269583406 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f283c06be30 0x7f283c06e2f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.169+0000 7f285533b640 1 --2- 172.21.15.112:0/3269583406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2850154620 0x7f285014fba0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.169+0000 7f285533b640 1 --2- 172.21.15.112:0/3269583406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28500c2bc0 0x7f28500c0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.169+0000 7f285533b640 1 --2- 172.21.15.112:0/3269583406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f285006c3e0 0x7f28501ef020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.169+0000 7f285533b640 1 -- 172.21.15.112:0/3269583406 >> 172.21.15.112:0/3269583406 conn(0x7f28500c42d0 msgr2=0x7f2850151080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:01.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.169+0000 7f285533b640 1 -- 172.21.15.112:0/3269583406 shutdown_connections 2024-04-25T01:35:01.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.169+0000 7f285533b640 1 -- 172.21.15.112:0/3269583406 wait complete. 2024-04-25T01:35:01.157 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:35:01.158 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:35:01.158 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:35:01.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.289+0000 7f1e0d514640 1 Processor -- start 2024-04-25T01:35:01.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.289+0000 7f1e0d514640 1 -- start start 2024-04-25T01:35:01.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.290+0000 7f1e0d514640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1e081592c0 0x7f1e080c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.290+0000 7f1e0d514640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e081596a0 0x7f1e08152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.290+0000 7f1e0d514640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e08159b90 con 0x7f1e080c72b0 2024-04-25T01:35:01.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.290+0000 7f1e0d514640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f1e080c6e20 con 0x7f1e081592c0 2024-04-25T01:35:01.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.290+0000 7f1e0d514640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1e0806c970 con 0x7f1e081596a0 2024-04-25T01:35:01.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.290+0000 7f1e06ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1e081592c0 0x7f1e080c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.290+0000 7f1e06ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1e081592c0 0x7f1e080c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59672/0 (socket says 172.21.15.112:59672) 2024-04-25T01:35:01.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.290+0000 7f1e06ffd640 1 -- 172.21.15.112:0/3498853769 learned_addr learned my addr 172.21.15.112:0/3498853769 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:01.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.290+0000 7f1dfffff640 1 --2- 172.21.15.112:0/3498853769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e081596a0 0x7f1e08152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.290+0000 7f1e06ffd640 1 -- 172.21.15.112:0/3498853769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e081596a0 msgr2=0x7f1e08152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:35:01.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.290+0000 7f1e06ffd640 1 --2- 172.21.15.112:0/3498853769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e081596a0 0x7f1e08152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.290+0000 7f1e06ffd640 1 -- 172.21.15.112:0/3498853769 >> v1:172.21.15.187:6789/0 conn(0x7f1e080c72b0 legacy=0x7f1e08155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.290+0000 7f1e06ffd640 1 -- 172.21.15.112:0/3498853769 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0806cc50 con 0x7f1e081592c0 2024-04-25T01:35:01.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.291+0000 7f1e06ffd640 1 --2- 172.21.15.112:0/3498853769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e081592c0 0x7f1e080c9070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009870 tx=0x7f1df802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac413e912e066a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.291+0000 7f1e067fc640 1 -- 172.21.15.112:0/3498853769 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8037070 con 0x7f1e081592c0 2024-04-25T01:35:01.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.291+0000 7f1e067fc640 1 -- 172.21.15.112:0/3498853769 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df8032440 con 0x7f1e081592c0 2024-04-25T01:35:01.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.291+0000 7f1e067fc640 1 -- 172.21.15.112:0/3498853769 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8032810 con 0x7f1e081592c0 2024-04-25T01:35:01.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.292+0000 7f1e0d514640 1 -- 172.21.15.112:0/3498853769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e081592c0 msgr2=0x7f1e080c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.292+0000 7f1e0d514640 1 --2- 172.21.15.112:0/3498853769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e081592c0 0x7f1e080c9070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009870 tx=0x7f1df802a130 comp rx=0 tx=0).stop 2024-04-25T01:35:01.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.292+0000 7f1e0d514640 1 -- 172.21.15.112:0/3498853769 shutdown_connections 2024-04-25T01:35:01.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.292+0000 7f1e0d514640 1 --2- 172.21.15.112:0/3498853769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e081596a0 0x7f1e08152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.292+0000 7f1e0d514640 1 --2- 172.21.15.112:0/3498853769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e081592c0 0x7f1e080c9070 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.292+0000 7f1e0d514640 1 -- 172.21.15.112:0/3498853769 >> 172.21.15.112:0/3498853769 conn(0x7f1e080cb170 msgr2=0x7f1e080cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:01.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.292+0000 7f1e0d514640 1 -- 172.21.15.112:0/3498853769 shutdown_connections 2024-04-25T01:35:01.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.292+0000 7f1e0d514640 1 -- 172.21.15.112:0/3498853769 wait complete. 2024-04-25T01:35:01.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.293+0000 7f1e0d514640 1 Processor -- start 2024-04-25T01:35:01.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.293+0000 7f1e0d514640 1 -- start start 2024-04-25T01:35:01.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.294+0000 7f1e0d514640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e080c72b0 0x7f1e0814d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.294+0000 7f1e0d514640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e081592c0 0x7f1e0814d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.294+0000 7f1e0d514640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e081596a0 0x7f1e08155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.294+0000 7f1e0d514640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0806dca0 con 0x7f1e081592c0 2024-04-25T01:35:01.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.294+0000 7f1e0d514640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0806db20 con 0x7f1e080c72b0 2024-04-25T01:35:01.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.294+0000 7f1e0d514640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1e080c95b0 con 0x7f1e081596a0 2024-04-25T01:35:01.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.294+0000 7f1dfffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e081592c0 0x7f1e0814d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.294+0000 7f1dfffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e081592c0 0x7f1e0814d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59676/0 (socket says 172.21.15.112:59676) 2024-04-25T01:35:01.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.294+0000 7f1dfffff640 1 -- 172.21.15.112:0/797333835 learned_addr learned my addr 172.21.15.112:0/797333835 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:01.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.294+0000 7f1e06ffd640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e080c72b0 0x7f1e0814d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.294+0000 7f1e077fe640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e081596a0 0x7f1e08155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.294+0000 7f1dfffff640 1 -- 172.21.15.112:0/797333835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e081596a0 msgr2=0x7f1e08155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.294+0000 7f1dfffff640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e081596a0 0x7f1e08155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.294+0000 7f1dfffff640 1 -- 172.21.15.112:0/797333835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e080c72b0 msgr2=0x7f1e0814d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.294+0000 7f1dfffff640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e080c72b0 0x7f1e0814d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.294+0000 7f1dfffff640 1 -- 172.21.15.112:0/797333835 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e080ac7e0 con 0x7f1e081592c0 2024-04-25T01:35:01.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.294+0000 7f1e06ffd640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e080c72b0 0x7f1e0814d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:35:01.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.294+0000 7f1dfffff640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e081592c0 0x7f1e0814d750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1df000e870 tx=0x7f1df000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.295+0000 7f1dff7fe640 1 -- 172.21.15.112:0/797333835 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0017070 con 0x7f1e081592c0 2024-04-25T01:35:01.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.295+0000 7f1dff7fe640 1 -- 172.21.15.112:0/797333835 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df000c940 con 0x7f1e081592c0 2024-04-25T01:35:01.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.295+0000 7f1e0d514640 1 -- 172.21.15.112:0/797333835 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e080be1d0 con 0x7f1e081592c0 2024-04-25T01:35:01.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.295+0000 7f1dff7fe640 1 -- 172.21.15.112:0/797333835 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df000cd10 con 0x7f1e081592c0 2024-04-25T01:35:01.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.295+0000 7f1e0d514640 1 -- 172.21.15.112:0/797333835 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e081547c0 con 0x7f1e081592c0 2024-04-25T01:35:01.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.296+0000 7f1e0d514640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e080682a0 0x7f1e08156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.296+0000 7f1e0d514640 1 -- 172.21.15.112:0/797333835 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e08075980 con 0x7f1e080682a0 2024-04-25T01:35:01.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.296+0000 7f1e06ffd640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e080682a0 0x7f1e08156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.296+0000 7f1e06ffd640 1 -- 172.21.15.112:0/797333835 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1e08154310 con 0x7f1e080682a0 2024-04-25T01:35:01.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.296+0000 7f1e06ffd640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e080682a0 0x7f1e08156420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1e080aac40 tx=0x7f1df80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.296+0000 7f1dff7fe640 1 -- 172.21.15.112:0/797333835 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8037040 con 0x7f1e080682a0 2024-04-25T01:35:01.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.296+0000 7f1dff7fe640 1 -- 172.21.15.112:0/797333835 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1df001a070 con 0x7f1e081592c0 2024-04-25T01:35:01.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.297+0000 7f1dff7fe640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1de806bd30 0x7f1de806e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.297+0000 7f1e077fe640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1de806bd30 0x7f1de806e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.297+0000 7f1dff7fe640 1 -- 172.21.15.112:0/797333835 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f1df008afd0 con 0x7f1e081592c0 2024-04-25T01:35:01.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.297+0000 7f1dff7fe640 1 -- 172.21.15.112:0/797333835 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1df8032440 con 0x7f1e080682a0 2024-04-25T01:35:01.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.297+0000 7f1dff7fe640 1 -- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e080682a0 msgr2=0x7f1e08156420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.297+0000 7f1dff7fe640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e080682a0 0x7f1e08156420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1e080aac40 tx=0x7f1df80046c0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.297+0000 7f1dff7fe640 1 -- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e080682a0 msgr2=0x7f1e08156420 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:35:01.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.297+0000 7f1dff7fe640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e080682a0 0x7f1e08156420 secure :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1e080aac40 tx=0x7f1df80046c0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.297+0000 7f1e077fe640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1de806bd30 0x7f1de806e1f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1e000012e0 tx=0x7f1e00009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.305+0000 7f1e0d514640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e08154b30 0x7f1e0814c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.305+0000 7f1e0d514640 1 -- 172.21.15.112:0/797333835 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e080679a0 con 0x7f1e08154b30 2024-04-25T01:35:01.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.305+0000 7f1e06ffd640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e08154b30 0x7f1e0814c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.305+0000 7f1e06ffd640 1 -- 172.21.15.112:0/797333835 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1e08156960 con 0x7f1e08154b30 2024-04-25T01:35:01.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.305+0000 7f1e06ffd640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e08154b30 0x7f1e0814c170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009870 tx=0x7f1df8008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.305+0000 7f1dff7fe640 1 -- 172.21.15.112:0/797333835 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8003cf0 con 0x7f1e08154b30 2024-04-25T01:35:01.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.306+0000 7f1dff7fe640 1 -- 172.21.15.112:0/797333835 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4731 (secure 0 0 0) 0x7f1df8003cf0 con 0x7f1e08154b30 2024-04-25T01:35:01.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.306+0000 7f1dff7fe640 1 -- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e08154b30 msgr2=0x7f1e0814c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.306+0000 7f1dff7fe640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e08154b30 0x7f1e0814c170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009870 tx=0x7f1df8008030 comp rx=0 tx=0).stop 2024-04-25T01:35:01.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.306+0000 7f1dff7fe640 1 -- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e08154b30 msgr2=0x7f1e0814c170 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:35:01.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.306+0000 7f1dff7fe640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e08154b30 0x7f1e0814c170 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.283 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:35:01.283 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:35:01.283 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:35:01.283 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:35:01.283 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 76, 2024-04-25T01:35:01.283 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 182072, 2024-04-25T01:35:01.283 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:35:01.284 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:35:01.284 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:35:01.284 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:35:01.284 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:01.284 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 2, 2024-04-25T01:35:01.284 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:35:01.284 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:35:01.284 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:35:01.284 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:01.284 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:01.284 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:01.284 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:01.284 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:01.284 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:01.285 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:01.285 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:01.285 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:01.285 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:01.285 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:01.285 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:01.285 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:35:01.285 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:35:01.285 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:01.285 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:01.285 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:01.285 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:01.285 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:01.285 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:01.285 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:01.286 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:01.286 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:01.286 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:01.286 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:01.286 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:01.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.286 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:35:01.286 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:35:01.286 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:35:01.286 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:35:01.286 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:35:01.286 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:35:01.286 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:35:01.286 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:35:01.287 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:35:01.287 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:35:01.287 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:35:01.287 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:35:01.287 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:35:01.287 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:35:01.287 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:35:01.287 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:35:01.287 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:35:01.287 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:01.287 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:01.287 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:01.287 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:01.287 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:35:01.293 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:35:01.293 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:01.293 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:01.293 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:01.293 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.293 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:01.293 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.293 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:35:01.293 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:35:01.293 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.293 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:01.293 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:01.293 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:35:01.293 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.293 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:01.294 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:35:01.294 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.294 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:01.294 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:01.294 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:35:01.294 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.294 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:01.294 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:35:01.294 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.294 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:01.294 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:01.294 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:35:01.294 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.294 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:01.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.294 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:35:01.295 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:35:01.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.306+0000 7f1e0d514640 1 -- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1de806bd30 msgr2=0x7f1de806e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.306+0000 7f1e0d514640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1de806bd30 0x7f1de806e1f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1e000012e0 tx=0x7f1e00009040 comp rx=0 tx=0).stop 2024-04-25T01:35:01.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.306+0000 7f1e0d514640 1 -- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e081592c0 msgr2=0x7f1e0814d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.306+0000 7f1e0d514640 1 --2- 172.21.15.112:0/797333835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e081592c0 0x7f1e0814d750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1df000e870 tx=0x7f1df000ed30 comp rx=0 tx=0).stop 2024-04-25T01:35:01.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.306+0000 7f1e077fe640 1 -- 172.21.15.112:0/797333835 reap_dead start 2024-04-25T01:35:01.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.306+0000 7f1e077fe640 1 -- 172.21.15.112:0/797333835 reap_dead start 2024-04-25T01:35:01.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.306+0000 7f1e0d514640 1 -- 172.21.15.112:0/797333835 shutdown_connections 2024-04-25T01:35:01.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.306+0000 7f1e0d514640 1 -- 172.21.15.112:0/797333835 >> 172.21.15.112:0/797333835 conn(0x7f1e080cb170 msgr2=0x7f1e08152ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:01.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.306+0000 7f1e0d514640 1 -- 172.21.15.112:0/797333835 shutdown_connections 2024-04-25T01:35:01.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.306+0000 7f1e0d514640 1 -- 172.21.15.112:0/797333835 wait complete. 2024-04-25T01:35:01.297 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:35:01.297 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:35:01.297 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:35:01.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.426+0000 7fd2883ee640 1 Processor -- start 2024-04-25T01:35:01.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.426+0000 7fd2883ee640 1 -- start start 2024-04-25T01:35:01.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.427+0000 7fd2883ee640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd2801592c0 0x7fd2800c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.427+0000 7fd2883ee640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2801596a0 0x7fd280152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.427+0000 7fd2883ee640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd280159b90 con 0x7fd2800c72b0 2024-04-25T01:35:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.427+0000 7fd2883ee640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd2800c6e20 con 0x7fd2801592c0 2024-04-25T01:35:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.427+0000 7fd2883ee640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd28006c970 con 0x7fd2801596a0 2024-04-25T01:35:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.428+0000 7fd28698b640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd2800c72b0 0x7fd280155ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:44034/0 (socket says 172.21.15.112:44034) 2024-04-25T01:35:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.428+0000 7fd28698b640 1 -- 172.21.15.112:0/1697240508 learned_addr learned my addr 172.21.15.112:0/1697240508 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:35:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.428+0000 7fd28618a640 1 --2- 172.21.15.112:0/1697240508 >> v2:172.21.15.187:3300/0 conn(0x7fd2801592c0 0x7fd2800c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.428+0000 7fd285989640 1 --2- 172.21.15.112:0/1697240508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2801596a0 0x7fd280152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.428+0000 7fd285188640 1 -- 172.21.15.112:0/1697240508 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3675788283 0 0) 0x7fd280159b90 con 0x7fd2800c72b0 2024-04-25T01:35:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.428+0000 7fd285188640 1 -- 172.21.15.112:0/1697240508 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd26c003520 con 0x7fd2800c72b0 2024-04-25T01:35:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.428+0000 7fd28618a640 1 -- 172.21.15.112:0/1697240508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2801596a0 msgr2=0x7fd280152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:35:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.428+0000 7fd28618a640 1 --2- 172.21.15.112:0/1697240508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2801596a0 0x7fd280152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.428+0000 7fd28618a640 1 -- 172.21.15.112:0/1697240508 >> v1:172.21.15.112:6789/0 conn(0x7fd2800c72b0 legacy=0x7fd280155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.428+0000 7fd28618a640 1 -- 172.21.15.112:0/1697240508 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28006cc50 con 0x7fd2801592c0 2024-04-25T01:35:01.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.429+0000 7fd28618a640 1 --2- 172.21.15.112:0/1697240508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2801592c0 0x7fd2800c9070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0097a0 tx=0x7fd27c02a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30a214bcff776ad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.430+0000 7fd285188640 1 -- 172.21.15.112:0/1697240508 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c037070 con 0x7fd2801592c0 2024-04-25T01:35:01.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.430+0000 7fd285188640 1 -- 172.21.15.112:0/1697240508 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd27c02adb0 con 0x7fd2801592c0 2024-04-25T01:35:01.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.430+0000 7fd285188640 1 -- 172.21.15.112:0/1697240508 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c032660 con 0x7fd2801592c0 2024-04-25T01:35:01.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.431+0000 7fd2883ee640 1 -- 172.21.15.112:0/1697240508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2801592c0 msgr2=0x7fd2800c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.431+0000 7fd2883ee640 1 --2- 172.21.15.112:0/1697240508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2801592c0 0x7fd2800c9070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0097a0 tx=0x7fd27c02a060 comp rx=0 tx=0).stop 2024-04-25T01:35:01.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.431+0000 7fd2883ee640 1 -- 172.21.15.112:0/1697240508 shutdown_connections 2024-04-25T01:35:01.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.431+0000 7fd2883ee640 1 --2- 172.21.15.112:0/1697240508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2801596a0 0x7fd280152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.431+0000 7fd2883ee640 1 --2- 172.21.15.112:0/1697240508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2801592c0 0x7fd2800c9070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.431+0000 7fd2883ee640 1 -- 172.21.15.112:0/1697240508 >> 172.21.15.112:0/1697240508 conn(0x7fd2800cb170 msgr2=0x7fd2800cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:01.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.431+0000 7fd2883ee640 1 -- 172.21.15.112:0/1697240508 shutdown_connections 2024-04-25T01:35:01.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.431+0000 7fd2883ee640 1 -- 172.21.15.112:0/1697240508 wait complete. 2024-04-25T01:35:01.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.433+0000 7fd2883ee640 1 Processor -- start 2024-04-25T01:35:01.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.433+0000 7fd2883ee640 1 -- start start 2024-04-25T01:35:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.434+0000 7fd2883ee640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2800c72b0 0x7fd280146620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.434+0000 7fd2883ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2801592c0 0x7fd2801511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.434+0000 7fd2883ee640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2801596a0 0x7fd2801ebab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.434+0000 7fd285989640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2801592c0 0x7fd2801511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.434+0000 7fd285989640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2801592c0 0x7fd2801511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59692/0 (socket says 172.21.15.112:59692) 2024-04-25T01:35:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.434+0000 7fd285989640 1 -- 172.21.15.112:0/2816054074 learned_addr learned my addr 172.21.15.112:0/2816054074 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.434+0000 7fd2883ee640 1 -- 172.21.15.112:0/2816054074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd28006dca0 con 0x7fd2801592c0 2024-04-25T01:35:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.434+0000 7fd2883ee640 1 -- 172.21.15.112:0/2816054074 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd28006db20 con 0x7fd2800c72b0 2024-04-25T01:35:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.434+0000 7fd2883ee640 1 -- 172.21.15.112:0/2816054074 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd2800c95b0 con 0x7fd2801596a0 2024-04-25T01:35:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.434+0000 7fd28618a640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2800c72b0 0x7fd280146620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.434+0000 7fd28698b640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2801596a0 0x7fd2801ebab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.434+0000 7fd285989640 1 -- 172.21.15.112:0/2816054074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2801596a0 msgr2=0x7fd2801ebab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.434+0000 7fd285989640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2801596a0 0x7fd2801ebab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.434+0000 7fd285989640 1 -- 172.21.15.112:0/2816054074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2800c72b0 msgr2=0x7fd280146620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.434+0000 7fd285989640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2800c72b0 0x7fd280146620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.434+0000 7fd285989640 1 -- 172.21.15.112:0/2816054074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd280146340 con 0x7fd2801592c0 2024-04-25T01:35:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.434+0000 7fd285989640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2801592c0 0x7fd2801511c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd27000e840 tx=0x7fd27000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.434+0000 7fd276ffd640 1 -- 172.21.15.112:0/2816054074 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd270017070 con 0x7fd2801592c0 2024-04-25T01:35:01.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.435+0000 7fd276ffd640 1 -- 172.21.15.112:0/2816054074 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd27000c940 con 0x7fd2801592c0 2024-04-25T01:35:01.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.435+0000 7fd2883ee640 1 -- 172.21.15.112:0/2816054074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2801ef530 con 0x7fd2801592c0 2024-04-25T01:35:01.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.435+0000 7fd276ffd640 1 -- 172.21.15.112:0/2816054074 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27000cd10 con 0x7fd2801592c0 2024-04-25T01:35:01.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.435+0000 7fd2883ee640 1 -- 172.21.15.112:0/2816054074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2801ef810 con 0x7fd2801592c0 2024-04-25T01:35:01.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.436+0000 7fd2883ee640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2800c1dd0 0x7fd2801ebff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.436+0000 7fd2883ee640 1 -- 172.21.15.112:0/2816054074 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd280075980 con 0x7fd2800c1dd0 2024-04-25T01:35:01.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.436+0000 7fd28618a640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2800c1dd0 0x7fd2801ebff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.436+0000 7fd28618a640 1 -- 172.21.15.112:0/2816054074 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd2800679a0 con 0x7fd2800c1dd0 2024-04-25T01:35:01.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.436+0000 7fd276ffd640 1 -- 172.21.15.112:0/2816054074 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd27001a070 con 0x7fd2801592c0 2024-04-25T01:35:01.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.436+0000 7fd28618a640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2800c1dd0 0x7fd2801ebff0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd2800aac40 tx=0x7fd27c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.437+0000 7fd276ffd640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd26406be30 0x7fd26406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.437+0000 7fd276ffd640 1 -- 172.21.15.112:0/2816054074 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fd27008aff0 con 0x7fd2801592c0 2024-04-25T01:35:01.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.437+0000 7fd28698b640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd26406be30 0x7fd26406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.437+0000 7fd28698b640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd26406be30 0x7fd26406e2f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd28014f5d0 tx=0x7fd278009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.439+0000 7fd276ffd640 1 -- 172.21.15.112:0/2816054074 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c037040 con 0x7fd2800c1dd0 2024-04-25T01:35:01.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.439+0000 7fd276ffd640 1 -- 172.21.15.112:0/2816054074 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd27c032bd0 con 0x7fd2800c1dd0 2024-04-25T01:35:01.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.439+0000 7fd276ffd640 1 -- 172.21.15.112:0/2816054074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2800c1dd0 msgr2=0x7fd2801ebff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.439+0000 7fd276ffd640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2800c1dd0 0x7fd2801ebff0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd2800aac40 tx=0x7fd27c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.439+0000 7fd276ffd640 1 -- 172.21.15.112:0/2816054074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2800c1dd0 msgr2=0x7fd2801ebff0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:35:01.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.439+0000 7fd276ffd640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2800c1dd0 0x7fd2801ebff0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.447+0000 7fd2883ee640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd280145160 0x7fd280145540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.447+0000 7fd2883ee640 1 -- 172.21.15.112:0/2816054074 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd2800c79d0 con 0x7fd280145160 2024-04-25T01:35:01.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.447+0000 7fd28618a640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd280145160 0x7fd280145540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.448+0000 7fd28618a640 1 -- 172.21.15.112:0/2816054074 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd2800c5ca0 con 0x7fd280145160 2024-04-25T01:35:01.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.448+0000 7fd28618a640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd280145160 0x7fd280145540 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd27c02a0a0 tx=0x7fd27c008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.448+0000 7fd276ffd640 1 -- 172.21.15.112:0/2816054074 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c037030 con 0x7fd280145160 2024-04-25T01:35:01.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.448+0000 7fd276ffd640 1 -- 172.21.15.112:0/2816054074 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5069 (secure 0 0 0) 0x7fd27c050450 con 0x7fd280145160 2024-04-25T01:35:01.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.448+0000 7fd276ffd640 1 -- 172.21.15.112:0/2816054074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd280145160 msgr2=0x7fd280145540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.448+0000 7fd276ffd640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd280145160 0x7fd280145540 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd27c02a0a0 tx=0x7fd27c008030 comp rx=0 tx=0).stop 2024-04-25T01:35:01.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.448+0000 7fd276ffd640 1 -- 172.21.15.112:0/2816054074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd280145160 msgr2=0x7fd280145540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:35:01.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.448+0000 7fd276ffd640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd280145160 0x7fd280145540 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.426 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:35:01.426 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:35:01.426 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:35:01.426 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:35:01.426 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 76, 2024-04-25T01:35:01.426 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 337137, 2024-04-25T01:35:01.426 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:35:01.426 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:35:01.427 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:35:01.427 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:35:01.427 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:01.427 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 3, 2024-04-25T01:35:01.427 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:35:01.427 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:35:01.427 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:35:01.427 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:01.427 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:01.427 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:01.428 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:01.428 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:01.428 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:01.428 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:01.428 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:01.428 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:01.428 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:01.428 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:01.428 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:01.428 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:35:01.428 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:35:01.429 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:01.429 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:01.429 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:01.429 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:01.429 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:01.429 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:01.429 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:01.429 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:01.429 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:01.429 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:01.430 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:01.430 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:01.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.430 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:35:01.430 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:35:01.430 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:35:01.430 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:35:01.430 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:35:01.430 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:35:01.431 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:35:01.431 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:35:01.431 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:35:01.431 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:35:01.431 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:35:01.431 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:35:01.431 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:35:01.431 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:35:01.431 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:35:01.431 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:35:01.431 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:35:01.431 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:01.431 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:01.432 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:01.432 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:01.432 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:01.432 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:01.432 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:01.432 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:01.432 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:01.432 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:01.432 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:01.432 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:01.432 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:35:01.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.432 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:35:01.432 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.433 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:35:01.433 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:35:01.433 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:01.433 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:01.433 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.433 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:01.433 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:35:01.433 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:01.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.433 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.433 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:01.433 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:35:01.433 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:01.433 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.434 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:01.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.434 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:35:01.434 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:35:01.434 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:01.434 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:01.434 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:01.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.434 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.434 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:35:01.434 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:35:01.434 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:01.434 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:01.434 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.434 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:01.435 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:35:01.435 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:01.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.435 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.435 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:01.435 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:35:01.435 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:01.435 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.435 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:01.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.435 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:35:01.435 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:35:01.435 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:01.436 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:01.436 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:01.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.436 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.436 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:35:01.436 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:35:01.436 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:01.436 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:01.436 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.436 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:01.436 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:35:01.436 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:01.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.436 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.437 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:01.437 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:35:01.437 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:01.437 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.437 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:01.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.437 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:35:01.437 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:35:01.437 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:01.437 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:01.437 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:01.437 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.437 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:01.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.437 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:35:01.438 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:35:01.438 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.438 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:01.438 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:01.438 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:35:01.438 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.438 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:01.438 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:35:01.438 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.438 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:01.438 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:01.438 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:35:01.438 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.439 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:01.439 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:35:01.439 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.439 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:01.439 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:01.439 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:35:01.439 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.439 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:01.439 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:35:01.439 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.439 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:01.439 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:01.439 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:35:01.439 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.439 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:01.440 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:35:01.440 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.440 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:01.440 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:01.440 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:35:01.440 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.440 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:01.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.440 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:35:01.440 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:35:01.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.448+0000 7fd2883ee640 1 -- 172.21.15.112:0/2816054074 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd26406be30 msgr2=0x7fd26406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.448+0000 7fd2883ee640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd26406be30 0x7fd26406e2f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd28014f5d0 tx=0x7fd278009040 comp rx=0 tx=0).stop 2024-04-25T01:35:01.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.448+0000 7fd2883ee640 1 -- 172.21.15.112:0/2816054074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2801592c0 msgr2=0x7fd2801511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.449+0000 7fd2883ee640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2801592c0 0x7fd2801511c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd27000e840 tx=0x7fd27000ed00 comp rx=0 tx=0).stop 2024-04-25T01:35:01.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.449+0000 7fd28698b640 1 -- 172.21.15.112:0/2816054074 reap_dead start 2024-04-25T01:35:01.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.449+0000 7fd2883ee640 1 -- 172.21.15.112:0/2816054074 shutdown_connections 2024-04-25T01:35:01.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.449+0000 7fd2883ee640 1 --2- 172.21.15.112:0/2816054074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2801592c0 0x7fd2801511c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.449+0000 7fd2883ee640 1 -- 172.21.15.112:0/2816054074 >> 172.21.15.112:0/2816054074 conn(0x7fd2800cb170 msgr2=0x7fd2800ca3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:01.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.449+0000 7fd2883ee640 1 -- 172.21.15.112:0/2816054074 shutdown_connections 2024-04-25T01:35:01.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.449+0000 7fd2883ee640 1 -- 172.21.15.112:0/2816054074 wait complete. 2024-04-25T01:35:01.443 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:35:01.443 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:35:01.443 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:35:01.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.578+0000 7f3f1b2fc640 1 Processor -- start 2024-04-25T01:35:01.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.578+0000 7f3f1b2fc640 1 -- start start 2024-04-25T01:35:01.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.579+0000 7f3f1b2fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3f141592c0 0x7f3f140c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.579+0000 7f3f1b2fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f141596a0 0x7f3f14152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.579+0000 7f3f1b2fc640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f14159b90 con 0x7f3f140c72b0 2024-04-25T01:35:01.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.579+0000 7f3f1b2fc640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f3f140c6e20 con 0x7f3f141592c0 2024-04-25T01:35:01.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.579+0000 7f3f1b2fc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1406c970 con 0x7f3f141596a0 2024-04-25T01:35:01.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.579+0000 7f3f19098640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3f141592c0 0x7f3f140c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.579+0000 7f3f19098640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3f141592c0 0x7f3f140c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59700/0 (socket says 172.21.15.112:59700) 2024-04-25T01:35:01.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.579+0000 7f3f19098640 1 -- 172.21.15.112:0/2042970972 learned_addr learned my addr 172.21.15.112:0/2042970972 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:01.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.579+0000 7f3f18897640 1 --2- 172.21.15.112:0/2042970972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f141596a0 0x7f3f14152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.579+0000 7f3f19098640 1 -- 172.21.15.112:0/2042970972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f141596a0 msgr2=0x7f3f14152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:35:01.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.579+0000 7f3f19098640 1 --2- 172.21.15.112:0/2042970972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f141596a0 0x7f3f14152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.579+0000 7f3f19098640 1 -- 172.21.15.112:0/2042970972 >> v1:172.21.15.187:6789/0 conn(0x7f3f140c72b0 legacy=0x7f3f14155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:35:01.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.579+0000 7f3f19098640 1 -- 172.21.15.112:0/2042970972 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1406cc50 con 0x7f3f141592c0 2024-04-25T01:35:01.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.580+0000 7f3f19098640 1 --2- 172.21.15.112:0/2042970972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f141592c0 0x7f3f140c9070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3f1408cb00 tx=0x7f3f1002a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ced91be4e41f213 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.580+0000 7f3f0bfff640 1 -- 172.21.15.112:0/2042970972 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10037070 con 0x7f3f141592c0 2024-04-25T01:35:01.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.580+0000 7f3f0bfff640 1 -- 172.21.15.112:0/2042970972 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f1002ade0 con 0x7f3f141592c0 2024-04-25T01:35:01.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.580+0000 7f3f0bfff640 1 -- 172.21.15.112:0/2042970972 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10032660 con 0x7f3f141592c0 2024-04-25T01:35:01.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.581+0000 7f3f1b2fc640 1 -- 172.21.15.112:0/2042970972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f141592c0 msgr2=0x7f3f140c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.581+0000 7f3f1b2fc640 1 --2- 172.21.15.112:0/2042970972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f141592c0 0x7f3f140c9070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3f1408cb00 tx=0x7f3f1002a0d0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.581+0000 7f3f1b2fc640 1 -- 172.21.15.112:0/2042970972 shutdown_connections 2024-04-25T01:35:01.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.581+0000 7f3f1b2fc640 1 --2- 172.21.15.112:0/2042970972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f141596a0 0x7f3f14152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.581+0000 7f3f1b2fc640 1 --2- 172.21.15.112:0/2042970972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f141592c0 0x7f3f140c9070 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.581+0000 7f3f1b2fc640 1 -- 172.21.15.112:0/2042970972 >> 172.21.15.112:0/2042970972 conn(0x7f3f140cb170 msgr2=0x7f3f140cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:01.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.582+0000 7f3f1b2fc640 1 -- 172.21.15.112:0/2042970972 shutdown_connections 2024-04-25T01:35:01.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.582+0000 7f3f1b2fc640 1 -- 172.21.15.112:0/2042970972 wait complete. 2024-04-25T01:35:01.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.583+0000 7f3f1b2fc640 1 Processor -- start 2024-04-25T01:35:01.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.583+0000 7f3f1b2fc640 1 -- start start 2024-04-25T01:35:01.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.584+0000 7f3f1b2fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f140c72b0 0x7f3f14158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.584+0000 7f3f1b2fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f141592c0 0x7f3f14154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.584+0000 7f3f1b2fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f141596a0 0x7f3f14156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.584+0000 7f3f1b2fc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1406dca0 con 0x7f3f140c72b0 2024-04-25T01:35:01.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.584+0000 7f3f1b2fc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1406db20 con 0x7f3f141596a0 2024-04-25T01:35:01.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.584+0000 7f3f1b2fc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f140c95b0 con 0x7f3f141592c0 2024-04-25T01:35:01.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.584+0000 7f3f19098640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f140c72b0 0x7f3f14158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.584+0000 7f3f19098640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f140c72b0 0x7f3f14158af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59706/0 (socket says 172.21.15.112:59706) 2024-04-25T01:35:01.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.584+0000 7f3f19098640 1 -- 172.21.15.112:0/2098321272 learned_addr learned my addr 172.21.15.112:0/2098321272 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:01.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.584+0000 7f3f18897640 1 --2- 172.21.15.112:0/2098321272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f141592c0 0x7f3f14154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.584+0000 7f3f19899640 1 --2- 172.21.15.112:0/2098321272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f141596a0 0x7f3f14156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.584+0000 7f3f19098640 1 -- 172.21.15.112:0/2098321272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f141592c0 msgr2=0x7f3f14154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.584+0000 7f3f19098640 1 --2- 172.21.15.112:0/2098321272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f141592c0 0x7f3f14154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.584+0000 7f3f19098640 1 -- 172.21.15.112:0/2098321272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f141596a0 msgr2=0x7f3f14156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.584+0000 7f3f19098640 1 --2- 172.21.15.112:0/2098321272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f141596a0 0x7f3f14156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.584+0000 7f3f19098640 1 -- 172.21.15.112:0/2098321272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f14155c00 con 0x7f3f140c72b0 2024-04-25T01:35:01.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.584+0000 7f3f19098640 1 --2- 172.21.15.112:0/2098321272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f140c72b0 0x7f3f14158af0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3f100097c0 tx=0x7f3f10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.585+0000 7f3f09ffb640 1 -- 172.21.15.112:0/2098321272 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10041070 con 0x7f3f140c72b0 2024-04-25T01:35:01.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.585+0000 7f3f09ffb640 1 -- 172.21.15.112:0/2098321272 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f1003c460 con 0x7f3f140c72b0 2024-04-25T01:35:01.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.585+0000 7f3f09ffb640 1 -- 172.21.15.112:0/2098321272 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10037040 con 0x7f3f140c72b0 2024-04-25T01:35:01.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.585+0000 7f3f1b2fc640 1 -- 172.21.15.112:0/2098321272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f14157390 con 0x7f3f140c72b0 2024-04-25T01:35:01.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.585+0000 7f3f1b2fc640 1 -- 172.21.15.112:0/2098321272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f14153b60 con 0x7f3f140c72b0 2024-04-25T01:35:01.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.586+0000 7f3f1b2fc640 1 --2- 172.21.15.112:0/2098321272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f14155dd0 0x7f3f14148860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.586+0000 7f3f1b2fc640 1 -- 172.21.15.112:0/2098321272 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f140679a0 con 0x7f3f14155dd0 2024-04-25T01:35:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.586+0000 7f3f18897640 1 --2- 172.21.15.112:0/2098321272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f14155dd0 0x7f3f14148860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.586+0000 7f3f18897640 1 -- 172.21.15.112:0/2098321272 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f140cc050 con 0x7f3f14155dd0 2024-04-25T01:35:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.586+0000 7f3f18897640 1 --2- 172.21.15.112:0/2098321272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f14155dd0 0x7f3f14148860 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3f140ab920 tx=0x7f3f04010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.587+0000 7f3f09ffb640 1 -- 172.21.15.112:0/2098321272 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f10044070 con 0x7f3f140c72b0 2024-04-25T01:35:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.587+0000 7f3f09ffb640 1 --2- 172.21.15.112:0/2098321272 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3ef806be60 0x7f3ef806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.587+0000 7f3f09ffb640 1 -- 172.21.15.112:0/2098321272 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f3f100ab800 con 0x7f3f140c72b0 2024-04-25T01:35:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.587+0000 7f3f19899640 1 --2- 172.21.15.112:0/2098321272 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3ef806be60 0x7f3ef806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.587+0000 7f3f09ffb640 1 -- 172.21.15.112:0/2098321272 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04010e70 con 0x7f3f14155dd0 2024-04-25T01:35:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.587+0000 7f3f09ffb640 1 -- 172.21.15.112:0/2098321272 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3f0400ea00 con 0x7f3f14155dd0 2024-04-25T01:35:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.587+0000 7f3f09ffb640 1 -- 172.21.15.112:0/2098321272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f14155dd0 msgr2=0x7f3f14148860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.587+0000 7f3f09ffb640 1 --2- 172.21.15.112:0/2098321272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f14155dd0 0x7f3f14148860 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3f140ab920 tx=0x7f3f04010c80 comp rx=0 tx=0).stop 2024-04-25T01:35:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.587+0000 7f3f09ffb640 1 -- 172.21.15.112:0/2098321272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f14155dd0 msgr2=0x7f3f14148860 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:35:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.587+0000 7f3f09ffb640 1 --2- 172.21.15.112:0/2098321272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f14155dd0 0x7f3f14148860 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.588+0000 7f3f19899640 1 --2- 172.21.15.112:0/2098321272 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3ef806be60 0x7f3ef806e320 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c001e70 tx=0x7f3f0c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.596+0000 7f3f1b2fc640 1 --2- 172.21.15.112:0/2098321272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f14156390 0x7f3f1414bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.596+0000 7f3f1b2fc640 1 -- 172.21.15.112:0/2098321272 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f140682a0 con 0x7f3f14156390 2024-04-25T01:35:01.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.596+0000 7f3f18897640 1 --2- 172.21.15.112:0/2098321272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f14156390 0x7f3f1414bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.596+0000 7f3f18897640 1 -- 172.21.15.112:0/2098321272 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3f141588d0 con 0x7f3f14156390 2024-04-25T01:35:01.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.596+0000 7f3f18897640 1 --2- 172.21.15.112:0/2098321272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f14156390 0x7f3f1414bd50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3f04007fd0 tx=0x7f3f040046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.596+0000 7f3f09ffb640 1 -- 172.21.15.112:0/2098321272 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04003cc0 con 0x7f3f14156390 2024-04-25T01:35:01.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.597+0000 7f3f09ffb640 1 -- 172.21.15.112:0/2098321272 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4558 (secure 0 0 0) 0x7f3f04003ca0 con 0x7f3f14156390 2024-04-25T01:35:01.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.597+0000 7f3f09ffb640 1 -- 172.21.15.112:0/2098321272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f14156390 msgr2=0x7f3f1414bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.597+0000 7f3f09ffb640 1 --2- 172.21.15.112:0/2098321272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f14156390 0x7f3f1414bd50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3f04007fd0 tx=0x7f3f040046c0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.597+0000 7f3f09ffb640 1 -- 172.21.15.112:0/2098321272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f14156390 msgr2=0x7f3f1414bd50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:35:01.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.597+0000 7f3f09ffb640 1 --2- 172.21.15.112:0/2098321272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f14156390 0x7f3f1414bd50 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3f04007fd0 tx=0x7f3f040046c0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.574 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:35:01.574 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:35:01.574 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:35:01.574 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:35:01.574 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 76, 2024-04-25T01:35:01.574 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 12256, 2024-04-25T01:35:01.574 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:35:01.574 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:35:01.574 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:35:01.574 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:35:01.575 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:01.575 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 3, 2024-04-25T01:35:01.575 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:35:01.575 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:35:01.575 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:35:01.575 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:01.575 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:01.575 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:01.575 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:01.575 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:01.575 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:01.575 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:01.575 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:01.575 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:01.575 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:01.575 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:01.576 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:01.576 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:35:01.576 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:35:01.576 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:01.576 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:01.576 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:01.576 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:01.576 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:01.576 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:01.576 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:01.576 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:01.576 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:01.576 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:01.576 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:01.577 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:01.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.577 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:35:01.577 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:35:01.577 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:35:01.577 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:35:01.577 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:35:01.577 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:35:01.577 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:35:01.577 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:35:01.577 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:35:01.577 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:35:01.577 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:35:01.577 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:35:01.577 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:35:01.578 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:35:01.578 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:35:01.578 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:35:01.578 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:35:01.578 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:01.578 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:01.578 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:01.578 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:01.578 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:01.578 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:01.578 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:01.578 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:01.578 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:01.578 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:01.579 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:01.579 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:01.579 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:35:01.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.579 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:35:01.579 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.579 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:35:01.579 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:35:01.579 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:01.579 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:01.579 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.579 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:01.579 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:35:01.579 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:01.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.580 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.580 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:01.580 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:35:01.580 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:01.580 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.580 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:01.580 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.580 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:35:01.580 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:35:01.580 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:01.580 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:01.580 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:01.580 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.580 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.580 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:35:01.581 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:35:01.581 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:01.581 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:01.581 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.581 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:01.581 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:35:01.581 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:01.581 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.581 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.581 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:01.581 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:35:01.581 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:01.581 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.581 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:01.581 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.582 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:35:01.582 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:35:01.582 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:01.582 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:01.582 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:01.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.582 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.582 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:35:01.582 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:35:01.582 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:01.582 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:01.582 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.582 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:01.582 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:35:01.583 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:01.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.583 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.583 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:01.583 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:35:01.583 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:01.583 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.583 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:01.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.583 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:35:01.583 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:35:01.583 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:01.583 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:01.583 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:01.583 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.584 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:01.584 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.584 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:35:01.584 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:35:01.584 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.584 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:01.584 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:01.584 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:35:01.584 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.584 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:01.584 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:35:01.584 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:01.584 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:01.584 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:01.585 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:35:01.585 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:01.585 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:01.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:01.585 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:35:01.585 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:35:01.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.597+0000 7f3f1b2fc640 1 -- 172.21.15.112:0/2098321272 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3ef806be60 msgr2=0x7f3ef806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.597+0000 7f3f1b2fc640 1 --2- 172.21.15.112:0/2098321272 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3ef806be60 0x7f3ef806e320 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c001e70 tx=0x7f3f0c009040 comp rx=0 tx=0).stop 2024-04-25T01:35:01.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.597+0000 7f3f1b2fc640 1 -- 172.21.15.112:0/2098321272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f140c72b0 msgr2=0x7f3f14158af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.597+0000 7f3f1b2fc640 1 --2- 172.21.15.112:0/2098321272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f140c72b0 0x7f3f14158af0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3f100097c0 tx=0x7f3f10004730 comp rx=0 tx=0).stop 2024-04-25T01:35:01.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.597+0000 7f3f19899640 1 -- 172.21.15.112:0/2098321272 reap_dead start 2024-04-25T01:35:01.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.597+0000 7f3f19899640 1 -- 172.21.15.112:0/2098321272 reap_dead start 2024-04-25T01:35:01.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.597+0000 7f3f1b2fc640 1 -- 172.21.15.112:0/2098321272 shutdown_connections 2024-04-25T01:35:01.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.597+0000 7f3f1b2fc640 1 -- 172.21.15.112:0/2098321272 >> 172.21.15.112:0/2098321272 conn(0x7f3f140cb170 msgr2=0x7f3f140ca660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:01.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.597+0000 7f3f1b2fc640 1 -- 172.21.15.112:0/2098321272 shutdown_connections 2024-04-25T01:35:01.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.597+0000 7f3f1b2fc640 1 -- 172.21.15.112:0/2098321272 wait complete. 2024-04-25T01:35:01.588 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:35:01.588 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T01:35:01.588 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T01:35:01.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.719+0000 7f4a10291640 1 Processor -- start 2024-04-25T01:35:01.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.719+0000 7f4a10291640 1 -- start start 2024-04-25T01:35:01.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.720+0000 7f4a10291640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4a080c72b0 0x7f4a08161390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.720+0000 7f4a10291640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a08159b90 con 0x7f4a081596a0 2024-04-25T01:35:01.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.720+0000 7f4a10291640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a0806c970 con 0x7f4a081592c0 2024-04-25T01:35:01.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.720+0000 7f4a10291640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f4a080c6e20 con 0x7f4a080c72b0 2024-04-25T01:35:01.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.720+0000 7f4a0e82e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4a080c72b0 0x7f4a08161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.720+0000 7f4a0d82c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4a081596a0 0x7f4a08152d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:44046/0 (socket says 172.21.15.112:44046) 2024-04-25T01:35:01.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.720+0000 7f4a0d82c640 1 -- 172.21.15.112:0/2750501360 learned_addr learned my addr 172.21.15.112:0/2750501360 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:35:01.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.721+0000 7f4a0e82e640 1 -- 172.21.15.112:0/2750501360 >> v1:172.21.15.187:6789/0 conn(0x7f4a081592c0 legacy=0x7f4a080c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:35:01.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.721+0000 7f4a0e82e640 1 -- 172.21.15.112:0/2750501360 >> v1:172.21.15.112:6789/0 conn(0x7f4a081596a0 legacy=0x7f4a08152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.721+0000 7f4a0e82e640 1 -- 172.21.15.112:0/2750501360 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0806cc50 con 0x7f4a080c72b0 2024-04-25T01:35:01.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.721+0000 7f4a0d02b640 1 -- 172.21.15.112:0/2750501360 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 430216800 0 0) 0x7f4a08159b90 con 0x7f4a081596a0 2024-04-25T01:35:01.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.721+0000 7f4a0e82e640 1 --2- 172.21.15.112:0/2750501360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a080c72b0 0x7f4a08161390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009d90 tx=0x7f4a0002a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151797482b3bea19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.721+0000 7f4a0d02b640 1 -- 172.21.15.112:0/2750501360 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00007740 con 0x7f4a080c72b0 2024-04-25T01:35:01.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.722+0000 7f4a0d02b640 1 -- 172.21.15.112:0/2750501360 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a000078e0 con 0x7f4a080c72b0 2024-04-25T01:35:01.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.722+0000 7f4a0d02b640 1 -- 172.21.15.112:0/2750501360 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00007cb0 con 0x7f4a080c72b0 2024-04-25T01:35:01.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.722+0000 7f4a10291640 1 -- 172.21.15.112:0/2750501360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a080c72b0 msgr2=0x7f4a08161390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.722+0000 7f4a10291640 1 --2- 172.21.15.112:0/2750501360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a080c72b0 0x7f4a08161390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009d90 tx=0x7f4a0002a580 comp rx=0 tx=0).stop 2024-04-25T01:35:01.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.722+0000 7f4a10291640 1 -- 172.21.15.112:0/2750501360 shutdown_connections 2024-04-25T01:35:01.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.722+0000 7f4a10291640 1 --2- 172.21.15.112:0/2750501360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a080c72b0 0x7f4a08161390 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.722+0000 7f4a10291640 1 -- 172.21.15.112:0/2750501360 >> 172.21.15.112:0/2750501360 conn(0x7f4a080cb170 msgr2=0x7f4a080cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:01.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.722+0000 7f4a10291640 1 -- 172.21.15.112:0/2750501360 shutdown_connections 2024-04-25T01:35:01.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.722+0000 7f4a10291640 1 -- 172.21.15.112:0/2750501360 wait complete. 2024-04-25T01:35:01.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.723+0000 7f4a10291640 1 Processor -- start 2024-04-25T01:35:01.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.724+0000 7f4a10291640 1 -- start start 2024-04-25T01:35:01.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.724+0000 7f4a10291640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a080c72b0 0x7f4a081ec030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.724+0000 7f4a10291640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a081592c0 0x7f4a08155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.724+0000 7f4a10291640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a081596a0 0x7f4a08154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.724+0000 7f4a10291640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0806c510 con 0x7f4a080c72b0 2024-04-25T01:35:01.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.724+0000 7f4a10291640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0806c390 con 0x7f4a081596a0 2024-04-25T01:35:01.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.724+0000 7f4a10291640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4a08153410 con 0x7f4a081592c0 2024-04-25T01:35:01.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.724+0000 7f4a0e02d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a080c72b0 0x7f4a081ec030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.724+0000 7f4a0e02d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a080c72b0 0x7f4a081ec030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59726/0 (socket says 172.21.15.112:59726) 2024-04-25T01:35:01.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.724+0000 7f4a0e02d640 1 -- 172.21.15.112:0/3789500873 learned_addr learned my addr 172.21.15.112:0/3789500873 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:01.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.725+0000 7f4a0d82c640 1 --2- 172.21.15.112:0/3789500873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a081592c0 0x7f4a08155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.725+0000 7f4a0e82e640 1 --2- 172.21.15.112:0/3789500873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a081596a0 0x7f4a08154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.725+0000 7f4a0e02d640 1 -- 172.21.15.112:0/3789500873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a081592c0 msgr2=0x7f4a08155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.725+0000 7f4a0e02d640 1 --2- 172.21.15.112:0/3789500873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a081592c0 0x7f4a08155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.725+0000 7f4a0e02d640 1 -- 172.21.15.112:0/3789500873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a081596a0 msgr2=0x7f4a08154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.725+0000 7f4a0e02d640 1 --2- 172.21.15.112:0/3789500873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a081596a0 0x7f4a08154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.725+0000 7f4a0e02d640 1 -- 172.21.15.112:0/3789500873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a080c2010 con 0x7f4a080c72b0 2024-04-25T01:35:01.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.725+0000 7f4a0e02d640 1 --2- 172.21.15.112:0/3789500873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a080c72b0 0x7f4a081ec030 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400c440 tx=0x7f4a0400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.725+0000 7f49feffd640 1 -- 172.21.15.112:0/3789500873 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04017070 con 0x7f4a080c72b0 2024-04-25T01:35:01.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.725+0000 7f49feffd640 1 -- 172.21.15.112:0/3789500873 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a040035f0 con 0x7f4a080c72b0 2024-04-25T01:35:01.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.725+0000 7f49feffd640 1 -- 172.21.15.112:0/3789500873 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a040039e0 con 0x7f4a080c72b0 2024-04-25T01:35:01.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.725+0000 7f4a10291640 1 -- 172.21.15.112:0/3789500873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a081539f0 con 0x7f4a080c72b0 2024-04-25T01:35:01.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.726+0000 7f4a10291640 1 -- 172.21.15.112:0/3789500873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a08153fb0 con 0x7f4a080c72b0 2024-04-25T01:35:01.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.727+0000 7f4a10291640 1 -- 172.21.15.112:0/3789500873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a080679a0 con 0x7f4a080c72b0 2024-04-25T01:35:01.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.727+0000 7f49feffd640 1 -- 172.21.15.112:0/3789500873 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a0401a070 con 0x7f4a080c72b0 2024-04-25T01:35:01.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.728+0000 7f49feffd640 1 --2- 172.21.15.112:0/3789500873 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f49ec06be80 0x7f49ec06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.728+0000 7f49feffd640 1 -- 172.21.15.112:0/3789500873 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f4a0408a350 con 0x7f4a080c72b0 2024-04-25T01:35:01.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.728+0000 7f4a0d82c640 1 --2- 172.21.15.112:0/3789500873 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f49ec06be80 0x7f49ec06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.728+0000 7f4a0d82c640 1 --2- 172.21.15.112:0/3789500873 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f49ec06be80 0x7f49ec06e340 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4a080c24b0 tx=0x7f49f8007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:01.734+0000 7f49feffd640 1 -- 172.21.15.112:0/3789500873 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4a040010c0 con 0x7f4a080c72b0 2024-04-25T01:35:02.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:02.046+0000 7f4a10291640 1 -- 172.21.15.112:0/3789500873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f4a080682a0 con 0x7f4a080c72b0 2024-04-25T01:35:02.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:02.046+0000 7f49feffd640 1 -- 172.21.15.112:0/3789500873 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f4a04058480 con 0x7f4a080c72b0 2024-04-25T01:35:02.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:02.049+0000 7f4a10291640 1 -- 172.21.15.112:0/3789500873 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f49ec06be80 msgr2=0x7f49ec06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:02.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:02.049+0000 7f4a10291640 1 --2- 172.21.15.112:0/3789500873 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f49ec06be80 0x7f49ec06e340 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4a080c24b0 tx=0x7f49f8007de0 comp rx=0 tx=0).stop 2024-04-25T01:35:02.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:02.049+0000 7f4a10291640 1 -- 172.21.15.112:0/3789500873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a080c72b0 msgr2=0x7f4a081ec030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:02.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:02.049+0000 7f4a10291640 1 --2- 172.21.15.112:0/3789500873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a080c72b0 0x7f4a081ec030 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400c440 tx=0x7f4a0400c900 comp rx=0 tx=0).stop 2024-04-25T01:35:02.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:02.049+0000 7f4a10291640 1 -- 172.21.15.112:0/3789500873 shutdown_connections 2024-04-25T01:35:02.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:02.049+0000 7f4a10291640 1 --2- 172.21.15.112:0/3789500873 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f49ec06be80 0x7f49ec06e340 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:02.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:02.049+0000 7f4a10291640 1 --2- 172.21.15.112:0/3789500873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a081596a0 0x7f4a08154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:02.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:02.049+0000 7f4a10291640 1 --2- 172.21.15.112:0/3789500873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a081592c0 0x7f4a08155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:02.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:02.049+0000 7f4a10291640 1 --2- 172.21.15.112:0/3789500873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a080c72b0 0x7f4a081ec030 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:02.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:02.049+0000 7f4a10291640 1 -- 172.21.15.112:0/3789500873 >> 172.21.15.112:0/3789500873 conn(0x7f4a080cb170 msgr2=0x7f4a08152cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:02.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:02.049+0000 7f4a10291640 1 -- 172.21.15.112:0/3789500873 shutdown_connections 2024-04-25T01:35:02.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:02.049+0000 7f4a10291640 1 -- 172.21.15.112:0/3789500873 wait complete. 2024-04-25T01:35:02.040 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T01:35:11.363 INFO:tasks.workunit.client.1.smithi187.stdout:fs setup took 424 secs 2024-04-25T01:35:12.041 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:35:12.041 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:35:12.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.203+0000 7f5c8c091640 1 Processor -- start 2024-04-25T01:35:12.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.203+0000 7f5c8c091640 1 -- start start 2024-04-25T01:35:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.204+0000 7f5c8c091640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5c840c5d70 0x7f5c840c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.204+0000 7f5c8c091640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c84156cf0 0x7f5c8406bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.204+0000 7f5c8c091640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c840c90e0 con 0x7f5c84152fd0 2024-04-25T01:35:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.204+0000 7f5c8c091640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f5c840ca9b0 con 0x7f5c84156cf0 2024-04-25T01:35:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.204+0000 7f5c8c091640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5c840cac00 con 0x7f5c840c5d70 2024-04-25T01:35:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.204+0000 7f5c8a62e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c84156cf0 0x7f5c8406bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.204+0000 7f5c89e2d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5c84152fd0 0x7f5c84156280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54972/0 (socket says 172.21.15.112:54972) 2024-04-25T01:35:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.204+0000 7f5c89e2d640 1 -- 172.21.15.112:0/804206446 learned_addr learned my addr 172.21.15.112:0/804206446 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:35:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.204+0000 7f5c8962c640 1 --2- 172.21.15.112:0/804206446 >> v2:172.21.15.187:3300/0 conn(0x7f5c840c5d70 0x7f5c840c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.205+0000 7f5c8a62e640 1 -- 172.21.15.112:0/804206446 >> v2:172.21.15.187:3300/0 conn(0x7f5c840c5d70 msgr2=0x7f5c840c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:35:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.205+0000 7f5c8a62e640 1 --2- 172.21.15.112:0/804206446 >> v2:172.21.15.187:3300/0 conn(0x7f5c840c5d70 0x7f5c840c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.205+0000 7f5c8a62e640 1 -- 172.21.15.112:0/804206446 >> v1:172.21.15.112:6789/0 conn(0x7f5c84152fd0 legacy=0x7f5c84156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.205+0000 7f5c8a62e640 1 -- 172.21.15.112:0/804206446 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c840c9380 con 0x7f5c84156cf0 2024-04-25T01:35:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.205+0000 7f5c88e2b640 1 -- 172.21.15.112:0/804206446 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1354210956 0 0) 0x7f5c840c90e0 con 0x7f5c84152fd0 2024-04-25T01:35:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.205+0000 7f5c8a62e640 1 --2- 172.21.15.112:0/804206446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c84156cf0 0x7f5c8406bda0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5c840bec40 tx=0x7f5c7c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4e1d9831bb605c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.205+0000 7f5c88e2b640 1 -- 172.21.15.112:0/804206446 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c007740 con 0x7f5c84156cf0 2024-04-25T01:35:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.205+0000 7f5c88e2b640 1 -- 172.21.15.112:0/804206446 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c7c0078e0 con 0x7f5c84156cf0 2024-04-25T01:35:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.205+0000 7f5c88e2b640 1 -- 172.21.15.112:0/804206446 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c007cb0 con 0x7f5c84156cf0 2024-04-25T01:35:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.206+0000 7f5c8c091640 1 -- 172.21.15.112:0/804206446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c84156cf0 msgr2=0x7f5c8406bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.206+0000 7f5c8c091640 1 --2- 172.21.15.112:0/804206446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c84156cf0 0x7f5c8406bda0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5c840bec40 tx=0x7f5c7c02a5f0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.206+0000 7f5c8c091640 1 -- 172.21.15.112:0/804206446 shutdown_connections 2024-04-25T01:35:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.206+0000 7f5c8c091640 1 --2- 172.21.15.112:0/804206446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c84156cf0 0x7f5c8406bda0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.206+0000 7f5c8c091640 1 --2- 172.21.15.112:0/804206446 >> v2:172.21.15.187:3300/0 conn(0x7f5c840c5d70 0x7f5c840c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.206+0000 7f5c8c091640 1 -- 172.21.15.112:0/804206446 >> 172.21.15.112:0/804206446 conn(0x7f5c840acc70 msgr2=0x7f5c840ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.206+0000 7f5c8c091640 1 -- 172.21.15.112:0/804206446 shutdown_connections 2024-04-25T01:35:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.206+0000 7f5c8c091640 1 -- 172.21.15.112:0/804206446 wait complete. 2024-04-25T01:35:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.207+0000 7f5c8c091640 1 Processor -- start 2024-04-25T01:35:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.207+0000 7f5c8c091640 1 -- start start 2024-04-25T01:35:12.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.208+0000 7f5c8c091640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c840c5d70 0x7f5c841ef020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.208+0000 7f5c8c091640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c84152fd0 0x7f5c84151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.208+0000 7f5c8c091640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c84156cf0 0x7f5c8414e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.208+0000 7f5c8c091640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c84157100 con 0x7f5c84156cf0 2024-04-25T01:35:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.208+0000 7f5c8c091640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5c840caee0 con 0x7f5c840c5d70 2024-04-25T01:35:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.208+0000 7f5c8c091640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5c84157280 con 0x7f5c84152fd0 2024-04-25T01:35:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.208+0000 7f5c8a62e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c84156cf0 0x7f5c8414e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.208+0000 7f5c8a62e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c84156cf0 0x7f5c8414e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48232/0 (socket says 172.21.15.112:48232) 2024-04-25T01:35:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.208+0000 7f5c8a62e640 1 -- 172.21.15.112:0/1891165711 learned_addr learned my addr 172.21.15.112:0/1891165711 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.208+0000 7f5c8962c640 1 --2- 172.21.15.112:0/1891165711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c84152fd0 0x7f5c84151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.208+0000 7f5c89e2d640 1 --2- 172.21.15.112:0/1891165711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c840c5d70 0x7f5c841ef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.208+0000 7f5c8a62e640 1 -- 172.21.15.112:0/1891165711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c84152fd0 msgr2=0x7f5c84151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.208+0000 7f5c8a62e640 1 --2- 172.21.15.112:0/1891165711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c84152fd0 0x7f5c84151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.208+0000 7f5c8a62e640 1 -- 172.21.15.112:0/1891165711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c840c5d70 msgr2=0x7f5c841ef020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.208+0000 7f5c8a62e640 1 --2- 172.21.15.112:0/1891165711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c840c5d70 0x7f5c841ef020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.208+0000 7f5c8a62e640 1 -- 172.21.15.112:0/1891165711 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c841548d0 con 0x7f5c84156cf0 2024-04-25T01:35:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.209+0000 7f5c8a62e640 1 --2- 172.21.15.112:0/1891165711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c84156cf0 0x7f5c8414e490 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c02abe0 tx=0x7f5c7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.209+0000 7f5c7a7fc640 1 -- 172.21.15.112:0/1891165711 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c03f070 con 0x7f5c84156cf0 2024-04-25T01:35:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.209+0000 7f5c7a7fc640 1 -- 172.21.15.112:0/1891165711 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c7c031bc0 con 0x7f5c84156cf0 2024-04-25T01:35:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.209+0000 7f5c7a7fc640 1 -- 172.21.15.112:0/1891165711 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c044460 con 0x7f5c84156cf0 2024-04-25T01:35:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.209+0000 7f5c8c091640 1 -- 172.21.15.112:0/1891165711 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c841ef560 con 0x7f5c84156cf0 2024-04-25T01:35:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.209+0000 7f5c8c091640 1 -- 172.21.15.112:0/1891165711 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c841ebe90 con 0x7f5c84156cf0 2024-04-25T01:35:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.210+0000 7f5c8c091640 1 -- 172.21.15.112:0/1891165711 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c840682a0 con 0x7f5c84156cf0 2024-04-25T01:35:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.210+0000 7f5c7a7fc640 1 -- 172.21.15.112:0/1891165711 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c7c03b410 con 0x7f5c84156cf0 2024-04-25T01:35:12.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.211+0000 7f5c7a7fc640 1 --2- 172.21.15.112:0/1891165711 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c6806be30 0x7f5c6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.211+0000 7f5c7a7fc640 1 -- 172.21.15.112:0/1891165711 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f5c7c0ac9b0 con 0x7f5c84156cf0 2024-04-25T01:35:12.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.211+0000 7f5c89e2d640 1 --2- 172.21.15.112:0/1891165711 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c6806be30 0x7f5c6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.211+0000 7f5c89e2d640 1 --2- 172.21.15.112:0/1891165711 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c6806be30 0x7f5c6806e2f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5c80000f60 tx=0x7f5c80009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.215+0000 7f5c7a7fc640 1 -- 172.21.15.112:0/1891165711 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5c841548d0 con 0x7f5c84156cf0 2024-04-25T01:35:12.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.518+0000 7f5c8c091640 1 -- 172.21.15.112:0/1891165711 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5c840c5ac0 con 0x7f5c84156cf0 2024-04-25T01:35:12.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.519+0000 7f5c7a7fc640 1 -- 172.21.15.112:0/1891165711 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f5c7c07a9e0 con 0x7f5c84156cf0 2024-04-25T01:35:12.495 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":76,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:35:12.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.521+0000 7f5c8c091640 1 -- 172.21.15.112:0/1891165711 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c6806be30 msgr2=0x7f5c6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.521+0000 7f5c8c091640 1 --2- 172.21.15.112:0/1891165711 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c6806be30 0x7f5c6806e2f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5c80000f60 tx=0x7f5c80009040 comp rx=0 tx=0).stop 2024-04-25T01:35:12.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.521+0000 7f5c8c091640 1 -- 172.21.15.112:0/1891165711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c84156cf0 msgr2=0x7f5c8414e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.521+0000 7f5c8c091640 1 --2- 172.21.15.112:0/1891165711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c84156cf0 0x7f5c8414e490 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c02abe0 tx=0x7f5c7c004a40 comp rx=0 tx=0).stop 2024-04-25T01:35:12.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.521+0000 7f5c8c091640 1 -- 172.21.15.112:0/1891165711 shutdown_connections 2024-04-25T01:35:12.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.521+0000 7f5c8c091640 1 --2- 172.21.15.112:0/1891165711 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c6806be30 0x7f5c6806e2f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.521+0000 7f5c8c091640 1 --2- 172.21.15.112:0/1891165711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c84156cf0 0x7f5c8414e490 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.521+0000 7f5c8c091640 1 --2- 172.21.15.112:0/1891165711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c84152fd0 0x7f5c84151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.521+0000 7f5c8c091640 1 --2- 172.21.15.112:0/1891165711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c840c5d70 0x7f5c841ef020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.521+0000 7f5c8c091640 1 -- 172.21.15.112:0/1891165711 >> 172.21.15.112:0/1891165711 conn(0x7f5c840acc70 msgr2=0x7f5c840cb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:12.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.521+0000 7f5c8c091640 1 -- 172.21.15.112:0/1891165711 shutdown_connections 2024-04-25T01:35:12.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.521+0000 7f5c8c091640 1 -- 172.21.15.112:0/1891165711 wait complete. 2024-04-25T01:35:12.508 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:35:12.509 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T01:35:12.509 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:35:12.509 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:35:12.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.655+0000 7fb217db7640 1 Processor -- start 2024-04-25T01:35:12.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.655+0000 7fb217db7640 1 -- start start 2024-04-25T01:35:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.656+0000 7fb217db7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb210072d80 0x7fb2101536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.656+0000 7fb217db7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb21006e0f0 con 0x7fb210071db0 2024-04-25T01:35:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.656+0000 7fb217db7640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2100c2180 con 0x7fb2100c1a80 2024-04-25T01:35:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.656+0000 7fb217db7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb2100c1f30 con 0x7fb210072d80 2024-04-25T01:35:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.656+0000 7fb216354640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb210071db0 0x7fb2100721c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54974/0 (socket says 172.21.15.112:54974) 2024-04-25T01:35:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.656+0000 7fb216354640 1 -- 172.21.15.112:0/3477101935 learned_addr learned my addr 172.21.15.112:0/3477101935 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:35:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb215b53640 1 --2- 172.21.15.112:0/3477101935 >> v2:172.21.15.187:3300/0 conn(0x7fb210072d80 0x7fb2101536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb214b51640 1 -- 172.21.15.112:0/3477101935 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1357529957 0 0) 0x7fb208001490 con 0x7fb210071db0 2024-04-25T01:35:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb214b51640 1 -- 172.21.15.112:0/3477101935 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1fc003520 con 0x7fb210071db0 2024-04-25T01:35:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb214b51640 1 -- 172.21.15.112:0/3477101935 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 338023012 0 0) 0x7fb1fc003520 con 0x7fb210071db0 2024-04-25T01:35:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb214b51640 1 -- 172.21.15.112:0/3477101935 >> v2:172.21.15.187:3300/0 conn(0x7fb210072d80 msgr2=0x7fb2101536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:35:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb214b51640 1 --2- 172.21.15.112:0/3477101935 >> v2:172.21.15.187:3300/0 conn(0x7fb210072d80 0x7fb2101536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb214b51640 1 -- 172.21.15.112:0/3477101935 >> v1:172.21.15.187:6789/0 conn(0x7fb2100c1a80 legacy=0x7fb21014d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb214b51640 1 -- 172.21.15.112:0/3477101935 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2100c2460 con 0x7fb210071db0 2024-04-25T01:35:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb214b51640 1 -- 172.21.15.112:0/3477101935 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fb208001bc0 con 0x7fb210071db0 2024-04-25T01:35:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb214b51640 1 -- 172.21.15.112:0/3477101935 >> v1:172.21.15.112:6789/0 conn(0x7fb210071db0 legacy=0x7fb2100721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb214b51640 1 --2- 172.21.15.112:0/3477101935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1fc0039b0 0x7fb1fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb214b51640 1 --2- 172.21.15.112:0/3477101935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1fc004800 0x7fb1fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb214b51640 1 --2- 172.21.15.112:0/3477101935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1fc0051c0 0x7fb1fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb214b51640 1 -- 172.21.15.112:0/3477101935 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2100c1f30 con 0x7fb1fc0039b0 2024-04-25T01:35:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb214b51640 1 -- 172.21.15.112:0/3477101935 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc00c060 con 0x7fb1fc0051c0 2024-04-25T01:35:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb214b51640 1 -- 172.21.15.112:0/3477101935 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb1fc00c260 con 0x7fb1fc004800 2024-04-25T01:35:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb215b53640 1 --2- 172.21.15.112:0/3477101935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1fc0039b0 0x7fb1fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb216354640 1 --2- 172.21.15.112:0/3477101935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1fc0051c0 0x7fb1fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb215352640 1 --2- 172.21.15.112:0/3477101935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1fc004800 0x7fb1fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb215b53640 1 -- 172.21.15.112:0/3477101935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1fc004800 msgr2=0x7fb1fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:35:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb215b53640 1 --2- 172.21.15.112:0/3477101935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1fc004800 0x7fb1fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb215b53640 1 -- 172.21.15.112:0/3477101935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1fc0051c0 msgr2=0x7fb1fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:35:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb215b53640 1 --2- 172.21.15.112:0/3477101935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1fc0051c0 0x7fb1fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb215b53640 1 -- 172.21.15.112:0/3477101935 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1fc003520 con 0x7fb1fc0039b0 2024-04-25T01:35:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb216354640 1 -- 172.21.15.112:0/3477101935 reap_dead start 2024-04-25T01:35:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.657+0000 7fb215352640 1 --2- 172.21.15.112:0/3477101935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1fc004800 0x7fb1fc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:35:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.658+0000 7fb215b53640 1 --2- 172.21.15.112:0/3477101935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1fc0039b0 0x7fb1fc003dd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009ac0 tx=0x7fb1fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977ce551337564fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.658+0000 7fb214b51640 1 -- 172.21.15.112:0/3477101935 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c036070 con 0x7fb1fc0039b0 2024-04-25T01:35:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.658+0000 7fb214b51640 1 -- 172.21.15.112:0/3477101935 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb20c031400 con 0x7fb1fc0039b0 2024-04-25T01:35:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.658+0000 7fb214b51640 1 -- 172.21.15.112:0/3477101935 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c009690 con 0x7fb1fc0039b0 2024-04-25T01:35:12.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.659+0000 7fb217db7640 1 -- 172.21.15.112:0/3477101935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1fc0039b0 msgr2=0x7fb1fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.659+0000 7fb217db7640 1 --2- 172.21.15.112:0/3477101935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1fc0039b0 0x7fb1fc003dd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009ac0 tx=0x7fb1fc005830 comp rx=0 tx=0).stop 2024-04-25T01:35:12.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.659+0000 7fb217db7640 1 -- 172.21.15.112:0/3477101935 shutdown_connections 2024-04-25T01:35:12.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.659+0000 7fb217db7640 1 --2- 172.21.15.112:0/3477101935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1fc0039b0 0x7fb1fc003dd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.659+0000 7fb217db7640 1 -- 172.21.15.112:0/3477101935 >> 172.21.15.112:0/3477101935 conn(0x7fb21006b160 msgr2=0x7fb21006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:12.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.659+0000 7fb217db7640 1 -- 172.21.15.112:0/3477101935 shutdown_connections 2024-04-25T01:35:12.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.659+0000 7fb217db7640 1 -- 172.21.15.112:0/3477101935 wait complete. 2024-04-25T01:35:12.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.660+0000 7fb217db7640 1 Processor -- start 2024-04-25T01:35:12.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.660+0000 7fb217db7640 1 -- start start 2024-04-25T01:35:12.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.661+0000 7fb217db7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1fc0039b0 0x7fb210143e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.661+0000 7fb217db7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb210148bd0 0x7fb210149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.661+0000 7fb217db7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb210149590 0x7fb2101ec230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.661+0000 7fb217db7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb210071890 con 0x7fb1fc0039b0 2024-04-25T01:35:12.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.661+0000 7fb217db7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2100c2a20 con 0x7fb210148bd0 2024-04-25T01:35:12.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.661+0000 7fb217db7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb210073160 con 0x7fb210149590 2024-04-25T01:35:12.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.661+0000 7fb215b53640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1fc0039b0 0x7fb210143e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.661+0000 7fb215b53640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1fc0039b0 0x7fb210143e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48258/0 (socket says 172.21.15.112:48258) 2024-04-25T01:35:12.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.661+0000 7fb215b53640 1 -- 172.21.15.112:0/3364180567 learned_addr learned my addr 172.21.15.112:0/3364180567 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:12.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.661+0000 7fb216354640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb210149590 0x7fb2101ec230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.661+0000 7fb215352640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb210148bd0 0x7fb210149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.661+0000 7fb215b53640 1 -- 172.21.15.112:0/3364180567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb210149590 msgr2=0x7fb2101ec230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.661+0000 7fb215b53640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb210149590 0x7fb2101ec230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.661+0000 7fb215b53640 1 -- 172.21.15.112:0/3364180567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb210148bd0 msgr2=0x7fb210149050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.661+0000 7fb215b53640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb210148bd0 0x7fb210149050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.661+0000 7fb215b53640 1 -- 172.21.15.112:0/3364180567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2101463d0 con 0x7fb1fc0039b0 2024-04-25T01:35:12.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.662+0000 7fb215b53640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1fc0039b0 0x7fb210143e20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009be0 tx=0x7fb20c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:12.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.662+0000 7fb2067fc640 1 -- 172.21.15.112:0/3364180567 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c03f020 con 0x7fb1fc0039b0 2024-04-25T01:35:12.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.662+0000 7fb2067fc640 1 -- 172.21.15.112:0/3364180567 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb20c037040 con 0x7fb1fc0039b0 2024-04-25T01:35:12.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.662+0000 7fb217db7640 1 -- 172.21.15.112:0/3364180567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb210146930 con 0x7fb1fc0039b0 2024-04-25T01:35:12.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.662+0000 7fb2067fc640 1 -- 172.21.15.112:0/3364180567 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c036070 con 0x7fb1fc0039b0 2024-04-25T01:35:12.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.662+0000 7fb217db7640 1 -- 172.21.15.112:0/3364180567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb210146650 con 0x7fb1fc0039b0 2024-04-25T01:35:12.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.663+0000 7fb217db7640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb210148100 0x7fb2101ec770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.663+0000 7fb217db7640 1 -- 172.21.15.112:0/3364180567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2100c6db0 con 0x7fb210148100 2024-04-25T01:35:12.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.663+0000 7fb215352640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb210148100 0x7fb2101ec770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.663+0000 7fb215352640 1 -- 172.21.15.112:0/3364180567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb210075980 con 0x7fb210148100 2024-04-25T01:35:12.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.663+0000 7fb215352640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb210148100 0x7fb2101ec770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb21006b7e0 tx=0x7fb20000f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:12.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.664+0000 7fb2067fc640 1 -- 172.21.15.112:0/3364180567 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2000052e0 con 0x7fb210148100 2024-04-25T01:35:12.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.664+0000 7fb2067fc640 1 -- 172.21.15.112:0/3364180567 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb20c042020 con 0x7fb1fc0039b0 2024-04-25T01:35:12.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.664+0000 7fb2067fc640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb1f406bd60 0x7fb1f406e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.664+0000 7fb2067fc640 1 -- 172.21.15.112:0/3364180567 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fb20c0ab180 con 0x7fb1fc0039b0 2024-04-25T01:35:12.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.664+0000 7fb216354640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb1f406bd60 0x7fb1f406e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.664+0000 7fb2067fc640 1 -- 172.21.15.112:0/3364180567 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb200018070 con 0x7fb210148100 2024-04-25T01:35:12.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.664+0000 7fb2067fc640 1 -- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb210148100 msgr2=0x7fb2101ec770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.664+0000 7fb2067fc640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb210148100 0x7fb2101ec770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb21006b7e0 tx=0x7fb20000f890 comp rx=0 tx=0).stop 2024-04-25T01:35:12.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.664+0000 7fb2067fc640 1 -- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb210148100 msgr2=0x7fb2101ec770 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:35:12.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.664+0000 7fb2067fc640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb210148100 0x7fb2101ec770 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb21006b7e0 tx=0x7fb20000f890 comp rx=0 tx=0).stop 2024-04-25T01:35:12.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.665+0000 7fb216354640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb1f406bd60 0x7fb1f406e220 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb2100c4bd0 tx=0x7fb208006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:12.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.671+0000 7fb217db7640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2101eccb0 0x7fb210141f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.671+0000 7fb217db7640 1 -- 172.21.15.112:0/3364180567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2100c6780 con 0x7fb2101eccb0 2024-04-25T01:35:12.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.671+0000 7fb215352640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2101eccb0 0x7fb210141f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.671+0000 7fb215352640 1 -- 172.21.15.112:0/3364180567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb2101435c0 con 0x7fb2101eccb0 2024-04-25T01:35:12.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.671+0000 7fb215352640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2101eccb0 0x7fb210141f70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb200017000 tx=0x7fb200001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:12.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.671+0000 7fb2067fc640 1 -- 172.21.15.112:0/3364180567 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200003ad0 con 0x7fb2101eccb0 2024-04-25T01:35:12.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.672+0000 7fb2067fc640 1 -- 172.21.15.112:0/3364180567 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4732 (secure 0 0 0) 0x7fb200003ad0 con 0x7fb2101eccb0 2024-04-25T01:35:12.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.672+0000 7fb2067fc640 1 -- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2101eccb0 msgr2=0x7fb210141f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.672+0000 7fb2067fc640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2101eccb0 0x7fb210141f70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb200017000 tx=0x7fb200001120 comp rx=0 tx=0).stop 2024-04-25T01:35:12.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.672+0000 7fb2067fc640 1 -- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2101eccb0 msgr2=0x7fb210141f70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:35:12.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.672+0000 7fb2067fc640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2101eccb0 0x7fb210141f70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.649 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:35:12.649 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:35:12.649 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:35:12.649 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:35:12.649 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 76, 2024-04-25T01:35:12.649 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 193436, 2024-04-25T01:35:12.649 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:35:12.649 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:35:12.649 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:35:12.649 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:35:12.650 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:12.650 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 14, 2024-04-25T01:35:12.650 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:35:12.650 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:35:12.650 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:35:12.650 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:12.650 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:12.650 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:12.650 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:12.650 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:12.650 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:12.650 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:12.650 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:12.650 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:12.650 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:12.651 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:12.651 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:12.651 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:35:12.651 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:35:12.651 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:12.651 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:12.651 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:12.651 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:12.651 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:12.651 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:12.651 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:12.651 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:12.651 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:12.651 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:12.651 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:12.652 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:12.652 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.652 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:35:12.652 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:35:12.652 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:35:12.652 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:35:12.652 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:35:12.652 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:35:12.652 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:35:12.652 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:35:12.652 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:35:12.652 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:35:12.652 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:35:12.652 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:35:12.652 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:35:12.653 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:35:12.653 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:35:12.653 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:35:12.653 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:35:12.653 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:12.653 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:12.653 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:12.653 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:12.653 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:12.653 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:12.653 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:12.653 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:12.653 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:12.653 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:12.653 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:12.654 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:12.654 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:35:12.654 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.654 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:35:12.654 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.654 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:35:12.654 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:35:12.654 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:12.654 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:12.654 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.654 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:12.654 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:35:12.654 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:12.654 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.654 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.655 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:12.655 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:35:12.655 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:12.655 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.655 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:12.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.655 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:35:12.655 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:35:12.655 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:12.655 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:12.655 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:12.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.655 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.655 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:35:12.655 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:35:12.656 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:12.656 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:12.656 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.656 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:12.656 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:35:12.656 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:12.656 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.656 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.656 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:12.656 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:35:12.656 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:12.656 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.656 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:12.656 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.656 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:35:12.657 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:35:12.657 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:12.657 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:12.657 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:12.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.657 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.657 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:35:12.657 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:35:12.657 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:12.657 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:12.657 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.657 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:12.657 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:35:12.657 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:12.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.658 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.658 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:12.658 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:35:12.658 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:12.658 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.658 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:12.658 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.658 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:35:12.658 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:35:12.658 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:12.658 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:12.658 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:12.658 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.658 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:12.658 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.659 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:35:12.659 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:35:12.659 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.659 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:12.659 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:12.659 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:35:12.659 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.659 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:12.659 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:35:12.659 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.659 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:12.659 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:12.659 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:35:12.659 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.660 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:12.660 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:35:12.660 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.660 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:12.660 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:12.660 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:35:12.660 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.660 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:12.660 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.660 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:35:12.660 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:35:12.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.672+0000 7fb217db7640 1 -- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb1f406bd60 msgr2=0x7fb1f406e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.672+0000 7fb217db7640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb1f406bd60 0x7fb1f406e220 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb2100c4bd0 tx=0x7fb208006040 comp rx=0 tx=0).stop 2024-04-25T01:35:12.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.672+0000 7fb217db7640 1 -- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1fc0039b0 msgr2=0x7fb210143e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.672+0000 7fb217db7640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1fc0039b0 0x7fb210143e20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009be0 tx=0x7fb20c004730 comp rx=0 tx=0).stop 2024-04-25T01:35:12.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.672+0000 7fb216354640 1 -- 172.21.15.112:0/3364180567 reap_dead start 2024-04-25T01:35:12.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.672+0000 7fb217db7640 1 -- 172.21.15.112:0/3364180567 shutdown_connections 2024-04-25T01:35:12.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.672+0000 7fb217db7640 1 --2- 172.21.15.112:0/3364180567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1fc0039b0 0x7fb210143e20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.672+0000 7fb217db7640 1 -- 172.21.15.112:0/3364180567 >> 172.21.15.112:0/3364180567 conn(0x7fb21006b160 msgr2=0x7fb210151860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:12.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.672+0000 7fb217db7640 1 -- 172.21.15.112:0/3364180567 shutdown_connections 2024-04-25T01:35:12.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.672+0000 7fb217db7640 1 -- 172.21.15.112:0/3364180567 wait complete. 2024-04-25T01:35:12.662 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:35:12.663 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:35:12.663 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:35:12.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.793+0000 7ff6b0ec3640 1 Processor -- start 2024-04-25T01:35:12.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.793+0000 7ff6b0ec3640 1 -- start start 2024-04-25T01:35:12.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.794+0000 7ff6b0ec3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff6ac1592c0 0x7ff6ac0c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.794+0000 7ff6b0ec3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff6ac0c72b0 0x7ff6ac160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.794+0000 7ff6b0ec3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6ac159b90 con 0x7ff6ac1596a0 2024-04-25T01:35:12.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.794+0000 7ff6b0ec3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7ff6ac0c6e20 con 0x7ff6ac0c72b0 2024-04-25T01:35:12.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.794+0000 7ff6b0ec3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff6ac06c970 con 0x7ff6ac1592c0 2024-04-25T01:35:12.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.794+0000 7ff6aad9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff6ac0c72b0 0x7ff6ac160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.794+0000 7ff6aad9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff6ac0c72b0 0x7ff6ac160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48290/0 (socket says 172.21.15.112:48290) 2024-04-25T01:35:12.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.794+0000 7ff6aad9d640 1 -- 172.21.15.112:0/1356135509 learned_addr learned my addr 172.21.15.112:0/1356135509 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:12.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.794+0000 7ff6aa59c640 1 --2- 172.21.15.112:0/1356135509 >> v2:172.21.15.187:3300/0 conn(0x7ff6ac1592c0 0x7ff6ac0c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.794+0000 7ff6aad9d640 1 -- 172.21.15.112:0/1356135509 >> v2:172.21.15.187:3300/0 conn(0x7ff6ac1592c0 msgr2=0x7ff6ac0c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:35:12.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.794+0000 7ff6aad9d640 1 --2- 172.21.15.112:0/1356135509 >> v2:172.21.15.187:3300/0 conn(0x7ff6ac1592c0 0x7ff6ac0c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.794+0000 7ff6aad9d640 1 -- 172.21.15.112:0/1356135509 >> v1:172.21.15.187:6789/0 conn(0x7ff6ac1596a0 legacy=0x7ff6ac152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.794+0000 7ff6aad9d640 1 -- 172.21.15.112:0/1356135509 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ac06cc50 con 0x7ff6ac0c72b0 2024-04-25T01:35:12.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.795+0000 7ff6aad9d640 1 --2- 172.21.15.112:0/1356135509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6ac0c72b0 0x7ff6ac160380 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac08cb00 tx=0x7ff6a402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1c8199d49215bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:12.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.795+0000 7ff6a9d9b640 1 -- 172.21.15.112:0/1356135509 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4036070 con 0x7ff6ac0c72b0 2024-04-25T01:35:12.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.796+0000 7ff6a9d9b640 1 -- 172.21.15.112:0/1356135509 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6a4031600 con 0x7ff6ac0c72b0 2024-04-25T01:35:12.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.796+0000 7ff6a9d9b640 1 -- 172.21.15.112:0/1356135509 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a40319d0 con 0x7ff6ac0c72b0 2024-04-25T01:35:12.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.796+0000 7ff6b0ec3640 1 -- 172.21.15.112:0/1356135509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6ac0c72b0 msgr2=0x7ff6ac160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.796+0000 7ff6b0ec3640 1 --2- 172.21.15.112:0/1356135509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6ac0c72b0 0x7ff6ac160380 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac08cb00 tx=0x7ff6a402a610 comp rx=0 tx=0).stop 2024-04-25T01:35:12.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.796+0000 7ff6b0ec3640 1 -- 172.21.15.112:0/1356135509 shutdown_connections 2024-04-25T01:35:12.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.796+0000 7ff6b0ec3640 1 --2- 172.21.15.112:0/1356135509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6ac0c72b0 0x7ff6ac160380 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.796+0000 7ff6b0ec3640 1 --2- 172.21.15.112:0/1356135509 >> v2:172.21.15.187:3300/0 conn(0x7ff6ac1592c0 0x7ff6ac0c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.796+0000 7ff6b0ec3640 1 -- 172.21.15.112:0/1356135509 >> 172.21.15.112:0/1356135509 conn(0x7ff6ac0cb170 msgr2=0x7ff6ac0cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:12.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.796+0000 7ff6b0ec3640 1 -- 172.21.15.112:0/1356135509 shutdown_connections 2024-04-25T01:35:12.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.796+0000 7ff6b0ec3640 1 -- 172.21.15.112:0/1356135509 wait complete. 2024-04-25T01:35:12.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.797+0000 7ff6b0ec3640 1 Processor -- start 2024-04-25T01:35:12.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.798+0000 7ff6b0ec3640 1 -- start start 2024-04-25T01:35:12.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.798+0000 7ff6b0ec3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6ac0c72b0 0x7ff6ac155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.798+0000 7ff6b0ec3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ac1592c0 0x7ff6ac1ef210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.798+0000 7ff6b0ec3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6ac1596a0 0x7ff6ac0c02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.798+0000 7ff6b0ec3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac153590 con 0x7ff6ac1596a0 2024-04-25T01:35:12.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.798+0000 7ff6b0ec3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac153410 con 0x7ff6ac1592c0 2024-04-25T01:35:12.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.798+0000 7ff6b0ec3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff6ac06db20 con 0x7ff6ac0c72b0 2024-04-25T01:35:12.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.798+0000 7ff6aad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6ac1596a0 0x7ff6ac0c02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.798+0000 7ff6aad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6ac1596a0 0x7ff6ac0c02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48294/0 (socket says 172.21.15.112:48294) 2024-04-25T01:35:12.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.798+0000 7ff6aad9d640 1 -- 172.21.15.112:0/1914633745 learned_addr learned my addr 172.21.15.112:0/1914633745 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:12.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.798+0000 7ff6aa59c640 1 --2- 172.21.15.112:0/1914633745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6ac0c72b0 0x7ff6ac155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.799+0000 7ff6a3fff640 1 --2- 172.21.15.112:0/1914633745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ac1592c0 0x7ff6ac1ef210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.799+0000 7ff6aad9d640 1 -- 172.21.15.112:0/1914633745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6ac0c72b0 msgr2=0x7ff6ac155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.799+0000 7ff6aad9d640 1 --2- 172.21.15.112:0/1914633745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6ac0c72b0 0x7ff6ac155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.799+0000 7ff6aad9d640 1 -- 172.21.15.112:0/1914633745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ac1592c0 msgr2=0x7ff6ac1ef210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.799+0000 7ff6aad9d640 1 --2- 172.21.15.112:0/1914633745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ac1592c0 0x7ff6ac1ef210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.799+0000 7ff6aad9d640 1 -- 172.21.15.112:0/1914633745 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ac154b80 con 0x7ff6ac1596a0 2024-04-25T01:35:12.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.799+0000 7ff6aad9d640 1 --2- 172.21.15.112:0/1914633745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6ac1596a0 0x7ff6ac0c02f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4009f40 tx=0x7ff6a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:12.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.799+0000 7ff6a2ffd640 1 -- 172.21.15.112:0/1914633745 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4036050 con 0x7ff6ac1596a0 2024-04-25T01:35:12.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.799+0000 7ff6a2ffd640 1 -- 172.21.15.112:0/1914633745 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6a4042070 con 0x7ff6ac1596a0 2024-04-25T01:35:12.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.799+0000 7ff6b0ec3640 1 -- 172.21.15.112:0/1914633745 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6ac1efa50 con 0x7ff6ac1596a0 2024-04-25T01:35:12.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.799+0000 7ff6a2ffd640 1 -- 172.21.15.112:0/1914633745 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4007bd0 con 0x7ff6ac1596a0 2024-04-25T01:35:12.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.799+0000 7ff6b0ec3640 1 -- 172.21.15.112:0/1914633745 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6ac1efcf0 con 0x7ff6ac1596a0 2024-04-25T01:35:12.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.800+0000 7ff6b0ec3640 1 --2- 172.21.15.112:0/1914633745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ac0c0830 0x7ff6ac0c4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.800+0000 7ff6b0ec3640 1 -- 172.21.15.112:0/1914633745 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac075980 con 0x7ff6ac0c0830 2024-04-25T01:35:12.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.800+0000 7ff6aa59c640 1 --2- 172.21.15.112:0/1914633745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ac0c0830 0x7ff6ac0c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.800+0000 7ff6aa59c640 1 -- 172.21.15.112:0/1914633745 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff6ac0679a0 con 0x7ff6ac0c0830 2024-04-25T01:35:12.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.801+0000 7ff6a2ffd640 1 -- 172.21.15.112:0/1914633745 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff6a4047460 con 0x7ff6ac1596a0 2024-04-25T01:35:12.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.801+0000 7ff6aa59c640 1 --2- 172.21.15.112:0/1914633745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ac0c0830 0x7ff6ac0c4930 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00f4a0 tx=0x7ff69c00f960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:12.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.801+0000 7ff6a2ffd640 1 --2- 172.21.15.112:0/1914633745 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff68c06be60 0x7ff68c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.801+0000 7ff6a2ffd640 1 -- 172.21.15.112:0/1914633745 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7ff6a40ac050 con 0x7ff6ac1596a0 2024-04-25T01:35:12.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.801+0000 7ff6a3fff640 1 --2- 172.21.15.112:0/1914633745 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff68c06be60 0x7ff68c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.801+0000 7ff6a2ffd640 1 -- 172.21.15.112:0/1914633745 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c019030 con 0x7ff6ac0c0830 2024-04-25T01:35:12.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.801+0000 7ff6a2ffd640 1 -- 172.21.15.112:0/1914633745 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff69c004020 con 0x7ff6ac0c0830 2024-04-25T01:35:12.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.801+0000 7ff6a2ffd640 1 -- 172.21.15.112:0/1914633745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ac0c0830 msgr2=0x7ff6ac0c4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.801+0000 7ff6a2ffd640 1 --2- 172.21.15.112:0/1914633745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ac0c0830 0x7ff6ac0c4930 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00f4a0 tx=0x7ff69c00f960 comp rx=0 tx=0).stop 2024-04-25T01:35:12.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.801+0000 7ff6a2ffd640 1 -- 172.21.15.112:0/1914633745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ac0c0830 msgr2=0x7ff6ac0c4930 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:35:12.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.801+0000 7ff6a2ffd640 1 --2- 172.21.15.112:0/1914633745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ac0c0830 0x7ff6ac0c4930 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.802+0000 7ff6a3fff640 1 --2- 172.21.15.112:0/1914633745 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff68c06be60 0x7ff68c06e320 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff694007fd0 tx=0x7ff694009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:12.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.808+0000 7ff6b0ec3640 1 --2- 172.21.15.112:0/1914633745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ac0bec20 0x7ff6ac1eba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.808+0000 7ff6b0ec3640 1 -- 172.21.15.112:0/1914633745 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac0682a0 con 0x7ff6ac0bec20 2024-04-25T01:35:12.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.808+0000 7ff6aa59c640 1 --2- 172.21.15.112:0/1914633745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ac0bec20 0x7ff6ac1eba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.809+0000 7ff6aa59c640 1 -- 172.21.15.112:0/1914633745 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff6ac0c79d0 con 0x7ff6ac0bec20 2024-04-25T01:35:12.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.809+0000 7ff6aa59c640 1 --2- 172.21.15.112:0/1914633745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ac0bec20 0x7ff6ac1eba30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00f4a0 tx=0x7ff69c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:12.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.809+0000 7ff6a2ffd640 1 -- 172.21.15.112:0/1914633745 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c014d00 con 0x7ff6ac0bec20 2024-04-25T01:35:12.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.809+0000 7ff6a2ffd640 1 -- 172.21.15.112:0/1914633745 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5070 (secure 0 0 0) 0x7ff69c014d00 con 0x7ff6ac0bec20 2024-04-25T01:35:12.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.809+0000 7ff6a2ffd640 1 -- 172.21.15.112:0/1914633745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ac0bec20 msgr2=0x7ff6ac1eba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.809+0000 7ff6a2ffd640 1 --2- 172.21.15.112:0/1914633745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ac0bec20 0x7ff6ac1eba30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00f4a0 tx=0x7ff69c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.809+0000 7ff6a2ffd640 1 -- 172.21.15.112:0/1914633745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ac0bec20 msgr2=0x7ff6ac1eba30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:35:12.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.809+0000 7ff6a2ffd640 1 --2- 172.21.15.112:0/1914633745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ac0bec20 0x7ff6ac1eba30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.786 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:35:12.786 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:35:12.786 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:35:12.786 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:35:12.786 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 76, 2024-04-25T01:35:12.786 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 348498, 2024-04-25T01:35:12.786 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:35:12.787 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:35:12.787 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:35:12.787 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:35:12.787 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:12.787 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 14, 2024-04-25T01:35:12.787 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:35:12.787 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:35:12.787 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:35:12.787 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:12.787 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:12.787 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:12.787 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:12.787 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:12.787 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:12.787 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:12.788 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:12.788 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:12.788 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:12.788 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:12.788 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:12.788 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:35:12.788 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:35:12.788 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:12.788 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:12.788 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:12.788 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:12.788 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:12.788 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:12.788 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:12.788 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:12.788 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:12.789 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:12.789 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:12.789 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:12.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.789 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:35:12.789 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:35:12.789 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:35:12.789 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:35:12.789 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:35:12.789 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:35:12.789 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:35:12.789 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:35:12.789 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:35:12.789 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:35:12.789 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:35:12.790 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:35:12.790 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:35:12.790 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:35:12.790 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:35:12.790 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:35:12.790 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:35:12.790 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:12.790 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:12.790 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:12.790 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:12.790 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:12.790 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:12.790 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:12.790 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:12.791 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:12.791 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:12.791 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:12.791 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:12.791 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:35:12.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.791 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:35:12.791 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.791 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:35:12.791 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:35:12.791 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:12.791 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:12.791 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.791 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:12.791 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:35:12.791 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:12.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.792 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.792 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:12.792 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:35:12.792 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:12.792 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.792 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:12.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.792 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:35:12.792 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:35:12.792 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:12.792 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:12.792 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:12.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.792 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.793 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:35:12.793 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:35:12.793 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:12.793 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:12.793 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.793 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:12.793 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:35:12.793 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:12.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.793 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.793 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:12.793 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:35:12.793 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:12.793 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.793 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:12.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.793 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:35:12.794 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:35:12.794 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:12.794 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:12.794 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:12.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.794 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.794 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:35:12.794 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:35:12.794 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:12.794 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:12.794 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.794 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:12.794 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:35:12.794 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:12.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.795 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:12.795 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:35:12.795 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:12.795 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.795 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:12.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.795 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:35:12.795 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:35:12.795 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:12.795 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:12.795 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:12.795 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.795 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:12.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.795 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:35:12.795 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:35:12.796 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.796 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:12.796 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:12.796 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:35:12.796 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.796 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:12.796 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:35:12.796 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.796 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:12.796 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:12.796 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:35:12.796 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.796 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:12.796 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:35:12.796 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.797 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:12.797 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:12.797 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:35:12.797 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.797 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:12.797 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:35:12.797 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.797 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:12.797 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:12.797 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:35:12.797 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.797 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:12.797 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:35:12.797 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.797 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:12.797 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:12.797 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:35:12.798 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.798 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:12.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.798 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:35:12.798 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:35:12.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.809+0000 7ff6b0ec3640 1 -- 172.21.15.112:0/1914633745 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff68c06be60 msgr2=0x7ff68c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.809+0000 7ff6b0ec3640 1 --2- 172.21.15.112:0/1914633745 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff68c06be60 0x7ff68c06e320 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff694007fd0 tx=0x7ff694009040 comp rx=0 tx=0).stop 2024-04-25T01:35:12.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.810+0000 7ff6b0ec3640 1 -- 172.21.15.112:0/1914633745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6ac1596a0 msgr2=0x7ff6ac0c02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.810+0000 7ff6b0ec3640 1 --2- 172.21.15.112:0/1914633745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6ac1596a0 0x7ff6ac0c02f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4009f40 tx=0x7ff6a4004a40 comp rx=0 tx=0).stop 2024-04-25T01:35:12.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.810+0000 7ff6aad9d640 1 -- 172.21.15.112:0/1914633745 reap_dead start 2024-04-25T01:35:12.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.810+0000 7ff6aad9d640 1 -- 172.21.15.112:0/1914633745 reap_dead start 2024-04-25T01:35:12.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.810+0000 7ff6b0ec3640 1 -- 172.21.15.112:0/1914633745 shutdown_connections 2024-04-25T01:35:12.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.810+0000 7ff6b0ec3640 1 -- 172.21.15.112:0/1914633745 >> 172.21.15.112:0/1914633745 conn(0x7ff6ac0cb170 msgr2=0x7ff6ac1e1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:12.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.810+0000 7ff6b0ec3640 1 -- 172.21.15.112:0/1914633745 shutdown_connections 2024-04-25T01:35:12.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.810+0000 7ff6b0ec3640 1 -- 172.21.15.112:0/1914633745 wait complete. 2024-04-25T01:35:12.800 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:35:12.800 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:35:12.800 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:35:12.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.933+0000 7fe7795ce640 1 Processor -- start 2024-04-25T01:35:12.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.933+0000 7fe7795ce640 1 -- start start 2024-04-25T01:35:12.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.934+0000 7fe7795ce640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe7740c5660 0x7fe77406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.934+0000 7fe7795ce640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7741578a0 con 0x7fe77406fbe0 2024-04-25T01:35:12.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.934+0000 7fe7795ce640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7740c3a70 con 0x7fe774150f40 2024-04-25T01:35:12.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.934+0000 7fe7795ce640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fe77406f260 con 0x7fe7740c5660 2024-04-25T01:35:12.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.934+0000 7fe7737fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe7740c5660 0x7fe77406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.934+0000 7fe7727fc640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe77406fbe0 0x7fe77406e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54986/0 (socket says 172.21.15.112:54986) 2024-04-25T01:35:12.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.934+0000 7fe7737fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe7740c5660 0x7fe77406ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48296/0 (socket says 172.21.15.112:48296) 2024-04-25T01:35:12.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.934+0000 7fe7727fc640 1 -- 172.21.15.112:0/3662787968 learned_addr learned my addr 172.21.15.112:0/3662787968 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:35:12.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.934+0000 7fe7737fe640 1 -- 172.21.15.112:0/3662787968 >> v1:172.21.15.187:6789/0 conn(0x7fe774150f40 legacy=0x7fe77406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.934+0000 7fe7737fe640 1 -- 172.21.15.112:0/3662787968 >> v1:172.21.15.112:6789/0 conn(0x7fe77406fbe0 legacy=0x7fe77406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.934+0000 7fe7737fe640 1 -- 172.21.15.112:0/3662787968 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe774151320 con 0x7fe7740c5660 2024-04-25T01:35:12.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.934+0000 7fe771ffb640 1 -- 172.21.15.112:0/3662787968 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2790816120 0 0) 0x7fe7741578a0 con 0x7fe77406fbe0 2024-04-25T01:35:12.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.935+0000 7fe7737fe640 1 --2- 172.21.15.112:0/3662787968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7740c5660 0x7fe77406ed20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009d90 tx=0x7fe76c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1864cf2ea360f10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:12.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.935+0000 7fe771ffb640 1 -- 172.21.15.112:0/3662787968 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c031600 con 0x7fe7740c5660 2024-04-25T01:35:12.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.935+0000 7fe771ffb640 1 -- 172.21.15.112:0/3662787968 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe76c0317a0 con 0x7fe7740c5660 2024-04-25T01:35:12.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.935+0000 7fe771ffb640 1 -- 172.21.15.112:0/3662787968 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c031b70 con 0x7fe7740c5660 2024-04-25T01:35:12.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.936+0000 7fe7795ce640 1 -- 172.21.15.112:0/3662787968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7740c5660 msgr2=0x7fe77406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.936+0000 7fe7795ce640 1 --2- 172.21.15.112:0/3662787968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7740c5660 0x7fe77406ed20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009d90 tx=0x7fe76c02a5d0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.936+0000 7fe7795ce640 1 -- 172.21.15.112:0/3662787968 shutdown_connections 2024-04-25T01:35:12.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.936+0000 7fe7795ce640 1 --2- 172.21.15.112:0/3662787968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7740c5660 0x7fe77406ed20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.936+0000 7fe7795ce640 1 -- 172.21.15.112:0/3662787968 >> 172.21.15.112:0/3662787968 conn(0x7fe7740c4260 msgr2=0x7fe7740c4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:12.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.936+0000 7fe7795ce640 1 -- 172.21.15.112:0/3662787968 shutdown_connections 2024-04-25T01:35:12.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.936+0000 7fe7795ce640 1 -- 172.21.15.112:0/3662787968 wait complete. 2024-04-25T01:35:12.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.937+0000 7fe7795ce640 1 Processor -- start 2024-04-25T01:35:12.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.938+0000 7fe7795ce640 1 -- start start 2024-04-25T01:35:12.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.938+0000 7fe7795ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe77406fbe0 0x7fe7740c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.938+0000 7fe7795ce640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7740c5660 0x7fe7741466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.938+0000 7fe7795ce640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe774150f40 0x7fe774146c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.938+0000 7fe7795ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe77406ca00 con 0x7fe77406fbe0 2024-04-25T01:35:12.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.938+0000 7fe7795ce640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe77406c880 con 0x7fe774150f40 2024-04-25T01:35:12.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.938+0000 7fe7795ce640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe7740c19e0 con 0x7fe7740c5660 2024-04-25T01:35:12.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.938+0000 7fe772ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe77406fbe0 0x7fe7740c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.938+0000 7fe772ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe77406fbe0 0x7fe7740c6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48302/0 (socket says 172.21.15.112:48302) 2024-04-25T01:35:12.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.938+0000 7fe772ffd640 1 -- 172.21.15.112:0/3062129340 learned_addr learned my addr 172.21.15.112:0/3062129340 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:12.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.939+0000 7fe7727fc640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7740c5660 0x7fe7741466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.939+0000 7fe7737fe640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe774150f40 0x7fe774146c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.939+0000 7fe772ffd640 1 -- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7740c5660 msgr2=0x7fe7741466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.939+0000 7fe772ffd640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7740c5660 0x7fe7741466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.939+0000 7fe772ffd640 1 -- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe774150f40 msgr2=0x7fe774146c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.939+0000 7fe772ffd640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe774150f40 0x7fe774146c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.939+0000 7fe772ffd640 1 -- 172.21.15.112:0/3062129340 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7741eceb0 con 0x7fe77406fbe0 2024-04-25T01:35:12.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.939+0000 7fe7727fc640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7740c5660 0x7fe7741466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:35:12.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.939+0000 7fe772ffd640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe77406fbe0 0x7fe7740c6180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe76400c450 tx=0x7fe76400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:12.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.939+0000 7fe75b7fe640 1 -- 172.21.15.112:0/3062129340 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7640052d0 con 0x7fe77406fbe0 2024-04-25T01:35:12.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.939+0000 7fe75b7fe640 1 -- 172.21.15.112:0/3062129340 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe764005470 con 0x7fe77406fbe0 2024-04-25T01:35:12.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.939+0000 7fe75b7fe640 1 -- 172.21.15.112:0/3062129340 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764012870 con 0x7fe77406fbe0 2024-04-25T01:35:12.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.939+0000 7fe7795ce640 1 -- 172.21.15.112:0/3062129340 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7741dfa40 con 0x7fe77406fbe0 2024-04-25T01:35:12.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.939+0000 7fe7795ce640 1 -- 172.21.15.112:0/3062129340 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7741ed150 con 0x7fe77406fbe0 2024-04-25T01:35:12.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.940+0000 7fe7795ce640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe774147410 0x7fe7741ddeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.940+0000 7fe7795ce640 1 -- 172.21.15.112:0/3062129340 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe774075980 con 0x7fe774147410 2024-04-25T01:35:12.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.940+0000 7fe7727fc640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe774147410 0x7fe7741ddeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.941+0000 7fe7727fc640 1 -- 172.21.15.112:0/3062129340 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7740689b0 con 0x7fe774147410 2024-04-25T01:35:12.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.941+0000 7fe75b7fe640 1 -- 172.21.15.112:0/3062129340 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe764012a10 con 0x7fe77406fbe0 2024-04-25T01:35:12.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.941+0000 7fe75b7fe640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe76006be30 0x7fe76006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.941+0000 7fe7737fe640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe76006be30 0x7fe76006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.941+0000 7fe75b7fe640 1 -- 172.21.15.112:0/3062129340 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fe76408ad00 con 0x7fe77406fbe0 2024-04-25T01:35:12.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.941+0000 7fe7727fc640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe774147410 0x7fe7741ddeb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe774147b60 tx=0x7fe7680017a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:12.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.942+0000 7fe75b7fe640 1 -- 172.21.15.112:0/3062129340 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768012590 con 0x7fe774147410 2024-04-25T01:35:12.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.942+0000 7fe7737fe640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe76006be30 0x7fe76006e2f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe76c034000 tx=0x7fe76c02f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:12.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.942+0000 7fe75b7fe640 1 -- 172.21.15.112:0/3062129340 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe768018070 con 0x7fe774147410 2024-04-25T01:35:12.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.942+0000 7fe75b7fe640 1 -- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe774147410 msgr2=0x7fe7741ddeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.942+0000 7fe75b7fe640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe774147410 0x7fe7741ddeb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe774147b60 tx=0x7fe7680017a0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.942+0000 7fe75b7fe640 1 -- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe774147410 msgr2=0x7fe7741ddeb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:35:12.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.942+0000 7fe75b7fe640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe774147410 0x7fe7741ddeb0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.948+0000 7fe7795ce640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7741de3f0 0x7fe7741de7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:12.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.948+0000 7fe7795ce640 1 -- 172.21.15.112:0/3062129340 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe774157890 con 0x7fe7741de3f0 2024-04-25T01:35:12.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.949+0000 7fe7727fc640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7741de3f0 0x7fe7741de7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:12.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.949+0000 7fe7727fc640 1 -- 172.21.15.112:0/3062129340 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe77406c2a0 con 0x7fe7741de3f0 2024-04-25T01:35:12.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.949+0000 7fe7727fc640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7741de3f0 0x7fe7741de7d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe768014000 tx=0x7fe768000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:12.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.949+0000 7fe75b7fe640 1 -- 172.21.15.112:0/3062129340 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768028440 con 0x7fe7741de3f0 2024-04-25T01:35:12.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.950+0000 7fe75b7fe640 1 -- 172.21.15.112:0/3062129340 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4559 (secure 0 0 0) 0x7fe768013420 con 0x7fe7741de3f0 2024-04-25T01:35:12.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.950+0000 7fe75b7fe640 1 -- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7741de3f0 msgr2=0x7fe7741de7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.950+0000 7fe75b7fe640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7741de3f0 0x7fe7741de7d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe768014000 tx=0x7fe768000ed0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.950+0000 7fe75b7fe640 1 -- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7741de3f0 msgr2=0x7fe7741de7d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:35:12.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.950+0000 7fe75b7fe640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7741de3f0 0x7fe7741de7d0 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe768014000 tx=0x7fe768000ed0 comp rx=0 tx=0).stop 2024-04-25T01:35:12.927 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:35:12.927 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:35:12.927 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:35:12.927 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:35:12.927 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 76, 2024-04-25T01:35:12.927 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 23609, 2024-04-25T01:35:12.927 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:35:12.927 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:35:12.927 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:35:12.927 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:35:12.927 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:12.927 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 14, 2024-04-25T01:35:12.927 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:35:12.927 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:35:12.928 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:35:12.928 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:12.928 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:12.928 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:12.928 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:12.928 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:12.928 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:12.928 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:12.928 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:12.928 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:12.928 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:12.928 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:12.928 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:12.928 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:35:12.928 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:35:12.928 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:12.929 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:12.929 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:12.929 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:12.929 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:12.929 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:12.929 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:12.929 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:12.929 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:12.929 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:12.929 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:12.929 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:12.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.929 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:35:12.929 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:35:12.930 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:35:12.930 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:35:12.930 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:35:12.930 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:35:12.930 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:35:12.930 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:35:12.930 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:35:12.930 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:35:12.930 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:35:12.930 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:35:12.930 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:35:12.930 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:35:12.930 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:35:12.930 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:35:12.930 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:35:12.931 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:12.931 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:12.931 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:12.931 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:12.931 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:12.931 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:12.931 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:12.931 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:12.931 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:12.931 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:12.931 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:12.931 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:12.931 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:35:12.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.932 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:35:12.932 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.932 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:35:12.932 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:35:12.932 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:12.932 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:12.932 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.932 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:12.932 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:35:12.932 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:12.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.932 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.932 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:12.932 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:35:12.932 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:12.932 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.933 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:12.933 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.933 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:35:12.933 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:35:12.933 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:12.933 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:12.933 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:12.933 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.933 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.933 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:35:12.933 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:35:12.933 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:12.933 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:12.933 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.933 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:12.933 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:35:12.934 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:12.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.934 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.934 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:12.934 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:35:12.934 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:12.934 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.934 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:12.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.934 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:35:12.934 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:35:12.934 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:12.934 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:12.934 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:12.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.935 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.935 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:35:12.935 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:35:12.935 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:12.935 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:12.935 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.935 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:12.935 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:35:12.935 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:12.935 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.935 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.935 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:12.935 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:35:12.935 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:12.935 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.936 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:12.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.936 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:35:12.936 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:35:12.936 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:12.936 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:12.936 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:12.936 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.936 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:12.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.936 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:35:12.936 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:35:12.936 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.936 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:12.937 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:12.937 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:35:12.937 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.937 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:12.937 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:35:12.937 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:12.937 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:12.937 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:12.937 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:35:12.937 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:12.937 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:12.937 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:12.937 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:35:12.937 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:35:12.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.950+0000 7fe7795ce640 1 -- 172.21.15.112:0/3062129340 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe76006be30 msgr2=0x7fe76006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.950+0000 7fe7795ce640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe76006be30 0x7fe76006e2f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe76c034000 tx=0x7fe76c02f000 comp rx=0 tx=0).stop 2024-04-25T01:35:12.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.950+0000 7fe7795ce640 1 -- 172.21.15.112:0/3062129340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe77406fbe0 msgr2=0x7fe7740c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:12.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.950+0000 7fe7795ce640 1 --2- 172.21.15.112:0/3062129340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe77406fbe0 0x7fe7740c6180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe76400c450 tx=0x7fe76400c910 comp rx=0 tx=0).stop 2024-04-25T01:35:12.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.950+0000 7fe7737fe640 1 -- 172.21.15.112:0/3062129340 reap_dead start 2024-04-25T01:35:12.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.950+0000 7fe7737fe640 1 -- 172.21.15.112:0/3062129340 reap_dead start 2024-04-25T01:35:12.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.950+0000 7fe7795ce640 1 -- 172.21.15.112:0/3062129340 shutdown_connections 2024-04-25T01:35:12.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.950+0000 7fe7795ce640 1 -- 172.21.15.112:0/3062129340 >> 172.21.15.112:0/3062129340 conn(0x7fe7740c4260 msgr2=0x7fe77414eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:12.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.950+0000 7fe7795ce640 1 -- 172.21.15.112:0/3062129340 shutdown_connections 2024-04-25T01:35:12.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:12.950+0000 7fe7795ce640 1 -- 172.21.15.112:0/3062129340 wait complete. 2024-04-25T01:35:12.940 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:35:12.940 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-04-25T01:35:12.940 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-25T01:35:12.940 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-04-25T01:35:12.940 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-04-25T01:35:12.940 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T01:35:12.940 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:35:12.952 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T01:35:12.952 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T01:35:12.953 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:35:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.124+0000 7f8a24716640 1 Processor -- start 2024-04-25T01:35:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.124+0000 7f8a24716640 1 -- start start 2024-04-25T01:35:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.124+0000 7f8a24716640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a1c152fd0 0x7f8a1c0c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.125+0000 7f8a24716640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8a1c0c5d70 0x7f8a1c156280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.125+0000 7f8a24716640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8a1c156cf0 0x7f8a1c06bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.125+0000 7f8a24716640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f8a1c0ca9b0 con 0x7f8a1c156cf0 2024-04-25T01:35:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.125+0000 7f8a24716640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8a1c1567c0 con 0x7f8a1c0c5d70 2024-04-25T01:35:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.125+0000 7f8a24716640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8a1c0c90e0 con 0x7f8a1c152fd0 2024-04-25T01:35:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.125+0000 7f8a22cb3640 1 -- >> v2:172.21.15.112:3300/0 conn(0x7f8a1c156cf0 msgr2=0x7f8a1c06bda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:35:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.125+0000 7f8a22cb3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8a1c156cf0 0x7f8a1c06bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:35:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.125+0000 7f8a21cb1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8a1c0c5d70 0x7f8a1c156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.125+0000 7f8a224b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a1c152fd0 0x7f8a1c0c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.125+0000 7f8a21cb1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8a1c0c5d70 0x7f8a1c156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:51878/0 (socket says 172.21.15.112:51878) 2024-04-25T01:35:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.125+0000 7f8a21cb1640 1 -- 172.21.15.112:0/684468549 learned_addr learned my addr 172.21.15.112:0/684468549 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.125+0000 7f8a224b2640 1 -- 172.21.15.112:0/684468549 >> v2:172.21.15.187:3300/0 conn(0x7f8a1c0c5d70 msgr2=0x7f8a1c156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:35:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.125+0000 7f8a224b2640 1 --2- 172.21.15.112:0/684468549 >> v2:172.21.15.187:3300/0 conn(0x7f8a1c0c5d70 0x7f8a1c156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.125+0000 7f8a224b2640 1 -- 172.21.15.112:0/684468549 >> v2:172.21.15.112:3300/0 conn(0x7f8a1c156cf0 msgr2=0x7f8a1c06bda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:35:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.125+0000 7f8a224b2640 1 --2- 172.21.15.112:0/684468549 >> v2:172.21.15.112:3300/0 conn(0x7f8a1c156cf0 0x7f8a1c06bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.125+0000 7f8a224b2640 1 -- 172.21.15.112:0/684468549 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1c0cac00 con 0x7f8a1c152fd0 2024-04-25T01:35:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.125+0000 7f8a21cb1640 1 --2- 172.21.15.112:0/684468549 >> v2:172.21.15.187:3300/0 conn(0x7f8a1c0c5d70 0x7f8a1c156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:35:13.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.126+0000 7f8a224b2640 1 --2- 172.21.15.112:0/684468549 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a1c152fd0 0x7f8a1c0c9b80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0c86e0 tx=0x7f8a1802a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b4f39110f1be83b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:13.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.126+0000 7f8a214b0640 1 -- 172.21.15.112:0/684468549 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18037070 con 0x7f8a1c152fd0 2024-04-25T01:35:13.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.126+0000 7f8a214b0640 1 -- 172.21.15.112:0/684468549 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a1802adf0 con 0x7f8a1c152fd0 2024-04-25T01:35:13.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.126+0000 7f8a214b0640 1 -- 172.21.15.112:0/684468549 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18032660 con 0x7f8a1c152fd0 2024-04-25T01:35:13.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.127+0000 7f8a24716640 1 -- 172.21.15.112:0/684468549 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a1c152fd0 msgr2=0x7f8a1c0c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:13.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.127+0000 7f8a24716640 1 --2- 172.21.15.112:0/684468549 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a1c152fd0 0x7f8a1c0c9b80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0c86e0 tx=0x7f8a1802a0a0 comp rx=0 tx=0).stop 2024-04-25T01:35:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.127+0000 7f8a24716640 1 -- 172.21.15.112:0/684468549 shutdown_connections 2024-04-25T01:35:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.127+0000 7f8a24716640 1 --2- 172.21.15.112:0/684468549 >> v2:172.21.15.187:3300/0 conn(0x7f8a1c0c5d70 0x7f8a1c156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.127+0000 7f8a24716640 1 --2- 172.21.15.112:0/684468549 >> v2:172.21.15.112:3300/0 conn(0x7f8a1c156cf0 0x7f8a1c06bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.127+0000 7f8a24716640 1 --2- 172.21.15.112:0/684468549 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a1c152fd0 0x7f8a1c0c9b80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.127+0000 7f8a24716640 1 -- 172.21.15.112:0/684468549 >> 172.21.15.112:0/684468549 conn(0x7f8a1c0acc70 msgr2=0x7f8a1c0ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.127+0000 7f8a24716640 1 -- 172.21.15.112:0/684468549 shutdown_connections 2024-04-25T01:35:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.127+0000 7f8a24716640 1 -- 172.21.15.112:0/684468549 wait complete. 2024-04-25T01:35:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.128+0000 7f8a24716640 1 Processor -- start 2024-04-25T01:35:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.128+0000 7f8a24716640 1 -- start start 2024-04-25T01:35:13.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.129+0000 7f8a24716640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a1c0c5d70 0x7f8a1c1ef5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:13.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.129+0000 7f8a24716640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a1c152fd0 0x7f8a1c0c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:13.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.129+0000 7f8a24716640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a1c156cf0 0x7f8a1c14d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:13.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.129+0000 7f8a24716640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1c0caee0 con 0x7f8a1c156cf0 2024-04-25T01:35:13.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.129+0000 7f8a24716640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1c0c9660 con 0x7f8a1c152fd0 2024-04-25T01:35:13.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.129+0000 7f8a24716640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8a1c157160 con 0x7f8a1c0c5d70 2024-04-25T01:35:13.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.129+0000 7f8a22cb3640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a1c156cf0 msgr2=0x7f8a1c14d700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:35:13.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.129+0000 7f8a22cb3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a1c156cf0 0x7f8a1c14d700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:35:13.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.129+0000 7f8a224b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a1c0c5d70 0x7f8a1c1ef5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:13.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.129+0000 7f8a224b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a1c0c5d70 0x7f8a1c1ef5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:46928/0 (socket says 172.21.15.112:46928) 2024-04-25T01:35:13.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.129+0000 7f8a224b2640 1 -- 172.21.15.112:0/355262634 learned_addr learned my addr 172.21.15.112:0/355262634 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:13.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.129+0000 7f8a21cb1640 1 --2- 172.21.15.112:0/355262634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a1c152fd0 0x7f8a1c0c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:13.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.129+0000 7f8a224b2640 1 -- 172.21.15.112:0/355262634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a1c152fd0 msgr2=0x7f8a1c0c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:13.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.129+0000 7f8a224b2640 1 --2- 172.21.15.112:0/355262634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a1c152fd0 0x7f8a1c0c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:13.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.129+0000 7f8a224b2640 1 -- 172.21.15.112:0/355262634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a1c156cf0 msgr2=0x7f8a1c14d700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:35:13.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.129+0000 7f8a224b2640 1 --2- 172.21.15.112:0/355262634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a1c156cf0 0x7f8a1c14d700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:13.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.129+0000 7f8a224b2640 1 -- 172.21.15.112:0/355262634 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1c0c3720 con 0x7f8a1c0c5d70 2024-04-25T01:35:13.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.130+0000 7f8a224b2640 1 --2- 172.21.15.112:0/355262634 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a1c0c5d70 0x7f8a1c1ef5f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8a18000c00 tx=0x7f8a180046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:13.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.130+0000 7f8a12ffd640 1 -- 172.21.15.112:0/355262634 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18037040 con 0x7f8a1c0c5d70 2024-04-25T01:35:13.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.130+0000 7f8a12ffd640 1 -- 172.21.15.112:0/355262634 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a18042070 con 0x7f8a1c0c5d70 2024-04-25T01:35:13.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.130+0000 7f8a24716640 1 -- 172.21.15.112:0/355262634 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a1c0c4790 con 0x7f8a1c0c5d70 2024-04-25T01:35:13.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.130+0000 7f8a12ffd640 1 -- 172.21.15.112:0/355262634 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1803b850 con 0x7f8a1c0c5d70 2024-04-25T01:35:13.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.130+0000 7f8a24716640 1 -- 172.21.15.112:0/355262634 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a1c1ebb60 con 0x7f8a1c0c5d70 2024-04-25T01:35:13.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.131+0000 7f8a24716640 1 -- 172.21.15.112:0/355262634 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a1c1eb670 con 0x7f8a1c0c5d70 2024-04-25T01:35:13.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.131+0000 7f8a12ffd640 1 -- 172.21.15.112:0/355262634 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a1803a070 con 0x7f8a1c0c5d70 2024-04-25T01:35:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.132+0000 7f8a12ffd640 1 --2- 172.21.15.112:0/355262634 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8a0006bde0 0x7f8a0006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.132+0000 7f8a21cb1640 1 --2- 172.21.15.112:0/355262634 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8a0006bde0 0x7f8a0006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.132+0000 7f8a12ffd640 1 -- 172.21.15.112:0/355262634 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f8a180ab1f0 con 0x7f8a1c0c5d70 2024-04-25T01:35:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.132+0000 7f8a21cb1640 1 --2- 172.21.15.112:0/355262634 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8a0006bde0 0x7f8a0006e2a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0c3290 tx=0x7f8a0c009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.137+0000 7f8a12ffd640 1 -- 172.21.15.112:0/355262634 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8a1c0c3720 con 0x7f8a1c0c5d70 2024-04-25T01:35:13.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.428+0000 7f8a24716640 1 -- 172.21.15.112:0/355262634 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8a1c0682a0 con 0x7f8a1c0c5d70 2024-04-25T01:35:13.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.428+0000 7f8a12ffd640 1 -- 172.21.15.112:0/355262634 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f8a180799b0 con 0x7f8a1c0c5d70 2024-04-25T01:35:13.405 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":76,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:35:13.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.430+0000 7f8a24716640 1 -- 172.21.15.112:0/355262634 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8a0006bde0 msgr2=0x7f8a0006e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:13.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.430+0000 7f8a24716640 1 --2- 172.21.15.112:0/355262634 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8a0006bde0 0x7f8a0006e2a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0c3290 tx=0x7f8a0c009240 comp rx=0 tx=0).stop 2024-04-25T01:35:13.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.430+0000 7f8a24716640 1 -- 172.21.15.112:0/355262634 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a1c0c5d70 msgr2=0x7f8a1c1ef5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:13.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.430+0000 7f8a24716640 1 --2- 172.21.15.112:0/355262634 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a1c0c5d70 0x7f8a1c1ef5f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8a18000c00 tx=0x7f8a180046c0 comp rx=0 tx=0).stop 2024-04-25T01:35:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.430+0000 7f8a24716640 1 -- 172.21.15.112:0/355262634 shutdown_connections 2024-04-25T01:35:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.430+0000 7f8a24716640 1 --2- 172.21.15.112:0/355262634 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8a0006bde0 0x7f8a0006e2a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.430+0000 7f8a24716640 1 --2- 172.21.15.112:0/355262634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a1c156cf0 0x7f8a1c14d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.430+0000 7f8a24716640 1 --2- 172.21.15.112:0/355262634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a1c152fd0 0x7f8a1c0c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.430+0000 7f8a24716640 1 --2- 172.21.15.112:0/355262634 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a1c0c5d70 0x7f8a1c1ef5f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.430+0000 7f8a24716640 1 -- 172.21.15.112:0/355262634 >> 172.21.15.112:0/355262634 conn(0x7f8a1c0acc70 msgr2=0x7f8a1c0aab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.430+0000 7f8a24716640 1 -- 172.21.15.112:0/355262634 shutdown_connections 2024-04-25T01:35:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:13.431+0000 7f8a24716640 1 -- 172.21.15.112:0/355262634 wait complete. 2024-04-25T01:35:13.656 INFO:tasks.workunit.client.1.smithi187.stdout:Syncing()...2 sec 2024-04-25T01:35:13.658 INFO:tasks.workunit.client.1.smithi187.stdout:Starting Actual Benchmark At: Thu Apr 25 01:35:13 2024 2024-04-25T01:35:13.658 INFO:tasks.workunit.client.1.smithi187.stdout: 2024-04-25T01:35:16.420 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:35:16.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.580+0000 7f25f3da5640 1 Processor -- start 2024-04-25T01:35:16.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.580+0000 7f25f3da5640 1 -- start start 2024-04-25T01:35:16.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.581+0000 7f25f3da5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f25ec0c4be0 0x7f25ec161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:16.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.581+0000 7f25f3da5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25ec0c4ff0 con 0x7f25ec0c8790 2024-04-25T01:35:16.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.581+0000 7f25f3da5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25ec0c3b20 con 0x7f25ec152ff0 2024-04-25T01:35:16.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.581+0000 7f25f3da5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f25ec0c40e0 con 0x7f25ec0c4be0 2024-04-25T01:35:16.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.581+0000 7f25f1340640 1 -- >> v1:172.21.15.112:6789/0 conn(0x7f25ec0c8790 legacy=0x7f25ec0c5a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:35:16.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.581+0000 7f25f1b41640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f25ec152ff0 0x7f25ec155860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:53286/0 (socket says 172.21.15.112:53286) 2024-04-25T01:35:16.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.581+0000 7f25f1b41640 1 -- 172.21.15.112:0/133122690 learned_addr learned my addr 172.21.15.112:0/133122690 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:35:16.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.581+0000 7f25f2342640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f25ec0c4be0 0x7f25ec161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:16.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.582+0000 7f25f0b3f640 1 -- 172.21.15.112:0/133122690 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1400028214 0 0) 0x7f25ec0c3b20 con 0x7f25ec152ff0 2024-04-25T01:35:16.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.582+0000 7f25f0b3f640 1 -- 172.21.15.112:0/133122690 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25d8003520 con 0x7f25ec152ff0 2024-04-25T01:35:16.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.582+0000 7f25f2342640 1 -- 172.21.15.112:0/133122690 >> v1:172.21.15.187:6789/0 conn(0x7f25ec152ff0 legacy=0x7f25ec155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:16.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.582+0000 7f25f2342640 1 -- 172.21.15.112:0/133122690 >> v1:172.21.15.112:6789/0 conn(0x7f25ec0c8790 legacy=0x7f25ec0c5a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:35:16.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.582+0000 7f25f2342640 1 -- 172.21.15.112:0/133122690 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ec0c3e00 con 0x7f25ec0c4be0 2024-04-25T01:35:16.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.584+0000 7f25f2342640 1 --2- 172.21.15.112:0/133122690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25ec0c4be0 0x7f25ec161150 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009d90 tx=0x7f25e402a580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d32c5c66be73f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:16.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.584+0000 7f25f0b3f640 1 -- 172.21.15.112:0/133122690 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e4007740 con 0x7f25ec0c4be0 2024-04-25T01:35:16.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.585+0000 7f25f0b3f640 1 -- 172.21.15.112:0/133122690 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25e40078e0 con 0x7f25ec0c4be0 2024-04-25T01:35:16.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.585+0000 7f25f0b3f640 1 -- 172.21.15.112:0/133122690 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e4007c90 con 0x7f25ec0c4be0 2024-04-25T01:35:16.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.585+0000 7f25f3da5640 1 -- 172.21.15.112:0/133122690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25ec0c4be0 msgr2=0x7f25ec161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:16.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.586+0000 7f25f3da5640 1 --2- 172.21.15.112:0/133122690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25ec0c4be0 0x7f25ec161150 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009d90 tx=0x7f25e402a580 comp rx=0 tx=0).stop 2024-04-25T01:35:16.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.586+0000 7f25f3da5640 1 -- 172.21.15.112:0/133122690 shutdown_connections 2024-04-25T01:35:16.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.586+0000 7f25f3da5640 1 --2- 172.21.15.112:0/133122690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25ec0c4be0 0x7f25ec161150 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:16.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.586+0000 7f25f3da5640 1 -- 172.21.15.112:0/133122690 >> 172.21.15.112:0/133122690 conn(0x7f25ec0c8f80 msgr2=0x7f25ec0c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:16.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.586+0000 7f25f3da5640 1 -- 172.21.15.112:0/133122690 shutdown_connections 2024-04-25T01:35:16.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.586+0000 7f25f3da5640 1 -- 172.21.15.112:0/133122690 wait complete. 2024-04-25T01:35:16.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.587+0000 7f25f3da5640 1 Processor -- start 2024-04-25T01:35:16.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.588+0000 7f25f3da5640 1 -- start start 2024-04-25T01:35:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.589+0000 7f25f3da5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25ec0c4be0 0x7f25ec156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.589+0000 7f25f3da5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25ec0c8790 0x7f25ec153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.589+0000 7f25f3da5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25ec152ff0 0x7f25ec155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.589+0000 7f25f3da5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec155f70 con 0x7f25ec0c4be0 2024-04-25T01:35:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.589+0000 7f25f3da5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec0c4750 con 0x7f25ec152ff0 2024-04-25T01:35:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.589+0000 7f25f1b41640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25ec0c4be0 msgr2=0x7f25ec156e40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:35:16.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.589+0000 7f25f3da5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f25ec071790 con 0x7f25ec0c8790 2024-04-25T01:35:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.589+0000 7f25f1b41640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25ec0c4be0 0x7f25ec156e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:35:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.589+0000 7f25f2342640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25ec152ff0 0x7f25ec155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.589+0000 7f25f1340640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25ec0c8790 0x7f25ec153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.589+0000 7f25f2342640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25ec152ff0 0x7f25ec155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:51916/0 (socket says 172.21.15.112:51916) 2024-04-25T01:35:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.589+0000 7f25f2342640 1 -- 172.21.15.112:0/3999049473 learned_addr learned my addr 172.21.15.112:0/3999049473 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.590+0000 7f25f2342640 1 -- 172.21.15.112:0/3999049473 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25ec0c8790 msgr2=0x7f25ec153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.590+0000 7f25f2342640 1 --2- 172.21.15.112:0/3999049473 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25ec0c8790 0x7f25ec153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.590+0000 7f25f2342640 1 -- 172.21.15.112:0/3999049473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25ec0c4be0 msgr2=0x7f25ec156e40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:35:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.590+0000 7f25f2342640 1 --2- 172.21.15.112:0/3999049473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25ec0c4be0 0x7f25ec156e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.590+0000 7f25f2342640 1 -- 172.21.15.112:0/3999049473 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ec1596d0 con 0x7f25ec152ff0 2024-04-25T01:35:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.590+0000 7f25f2342640 1 --2- 172.21.15.112:0/3999049473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25ec152ff0 0x7f25ec155450 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009d90 tx=0x7f25e4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.590+0000 7f25e27fc640 1 -- 172.21.15.112:0/3999049473 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e4040070 con 0x7f25ec152ff0 2024-04-25T01:35:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.590+0000 7f25e27fc640 1 -- 172.21.15.112:0/3999049473 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25e4007e30 con 0x7f25ec152ff0 2024-04-25T01:35:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.591+0000 7f25e27fc640 1 -- 172.21.15.112:0/3999049473 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e4007c90 con 0x7f25ec152ff0 2024-04-25T01:35:16.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.591+0000 7f25f3da5640 1 -- 172.21.15.112:0/3999049473 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ec1599b0 con 0x7f25ec152ff0 2024-04-25T01:35:16.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.591+0000 7f25f3da5640 1 -- 172.21.15.112:0/3999049473 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ec157380 con 0x7f25ec152ff0 2024-04-25T01:35:16.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.592+0000 7f25f3da5640 1 -- 172.21.15.112:0/3999049473 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25ec0682a0 con 0x7f25ec152ff0 2024-04-25T01:35:16.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.593+0000 7f25e27fc640 1 -- 172.21.15.112:0/3999049473 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f25e4039040 con 0x7f25ec152ff0 2024-04-25T01:35:16.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.593+0000 7f25e27fc640 1 --2- 172.21.15.112:0/3999049473 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f25d006bde0 0x7f25d006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:16.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.593+0000 7f25e27fc640 1 -- 172.21.15.112:0/3999049473 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f25e40ac260 con 0x7f25ec152ff0 2024-04-25T01:35:16.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.593+0000 7f25f1b41640 1 --2- 172.21.15.112:0/3999049473 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f25d006bde0 0x7f25d006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:16.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.594+0000 7f25f1b41640 1 --2- 172.21.15.112:0/3999049473 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f25d006bde0 0x7f25d006e2a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f25e8000c00 tx=0x7f25e8007a10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:16.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:16.601+0000 7f25e27fc640 1 -- 172.21.15.112:0/3999049473 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f25e407a860 con 0x7f25ec152ff0 2024-04-25T01:35:16.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:17.007+0000 7f25f3da5640 1 -- 172.21.15.112:0/3999049473 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f25ec0c75b0 con 0x7f25ec152ff0 2024-04-25T01:35:16.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:17.008+0000 7f25e27fc640 1 -- 172.21.15.112:0/3999049473 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f25e407a160 con 0x7f25ec152ff0 2024-04-25T01:35:16.984 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":76,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:35:16.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:17.011+0000 7f25f3da5640 1 -- 172.21.15.112:0/3999049473 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f25d006bde0 msgr2=0x7f25d006e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:16.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:17.011+0000 7f25f3da5640 1 --2- 172.21.15.112:0/3999049473 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f25d006bde0 0x7f25d006e2a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f25e8000c00 tx=0x7f25e8007a10 comp rx=0 tx=0).stop 2024-04-25T01:35:16.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:17.011+0000 7f25f3da5640 1 -- 172.21.15.112:0/3999049473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25ec152ff0 msgr2=0x7f25ec155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:16.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:17.011+0000 7f25f3da5640 1 --2- 172.21.15.112:0/3999049473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25ec152ff0 0x7f25ec155450 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009d90 tx=0x7f25e4004a40 comp rx=0 tx=0).stop 2024-04-25T01:35:16.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:17.011+0000 7f25f3da5640 1 -- 172.21.15.112:0/3999049473 shutdown_connections 2024-04-25T01:35:16.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:17.011+0000 7f25f3da5640 1 --2- 172.21.15.112:0/3999049473 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f25d006bde0 0x7f25d006e2a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:16.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:17.011+0000 7f25f3da5640 1 --2- 172.21.15.112:0/3999049473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25ec152ff0 0x7f25ec155450 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:16.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:17.011+0000 7f25f3da5640 1 --2- 172.21.15.112:0/3999049473 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25ec0c8790 0x7f25ec153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:16.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:17.011+0000 7f25f3da5640 1 --2- 172.21.15.112:0/3999049473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25ec0c4be0 0x7f25ec156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:16.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:17.011+0000 7f25f3da5640 1 -- 172.21.15.112:0/3999049473 >> 172.21.15.112:0/3999049473 conn(0x7f25ec0c8f80 msgr2=0x7f25ec0c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:16.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:17.011+0000 7f25f3da5640 1 -- 172.21.15.112:0/3999049473 shutdown_connections 2024-04-25T01:35:16.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:17.011+0000 7f25f3da5640 1 -- 172.21.15.112:0/3999049473 wait complete. 2024-04-25T01:35:20.001 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:35:20.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.163+0000 7fb42c3e7640 1 Processor -- start 2024-04-25T01:35:20.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.163+0000 7fb42c3e7640 1 -- start start 2024-04-25T01:35:20.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.164+0000 7fb42c3e7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb424155d80 0x7fb424156160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:20.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.164+0000 7fb42c3e7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb42406dab0 0x7fb42414c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:20.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.164+0000 7fb42c3e7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb42406dfb0 con 0x7fb42414c5c0 2024-04-25T01:35:20.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.164+0000 7fb42c3e7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fb42406ebc0 con 0x7fb424155d80 2024-04-25T01:35:20.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.164+0000 7fb42c3e7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb42406e290 con 0x7fb42406dab0 2024-04-25T01:35:20.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.164+0000 7fb42a183640 1 -- >> v2:172.21.15.112:3300/0 conn(0x7fb424155d80 msgr2=0x7fb424156160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:35:20.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.164+0000 7fb42a183640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb424155d80 0x7fb424156160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:35:20.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.164+0000 7fb42a984640 1 -- >> v1:172.21.15.112:6789/0 conn(0x7fb42414c5c0 legacy=0x7fb424153a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:35:20.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.165+0000 7fb429982640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb42406dab0 0x7fb42414c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:20.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.165+0000 7fb429982640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb42406dab0 0x7fb42414c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:52272/0 (socket says 172.21.15.112:52272) 2024-04-25T01:35:20.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.165+0000 7fb429982640 1 -- 172.21.15.112:0/1547931168 learned_addr learned my addr 172.21.15.112:0/1547931168 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:20.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.167+0000 7fb429982640 1 -- 172.21.15.112:0/1547931168 >> v2:172.21.15.112:3300/0 conn(0x7fb424155d80 msgr2=0x7fb424156160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:35:20.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.167+0000 7fb429982640 1 --2- 172.21.15.112:0/1547931168 >> v2:172.21.15.112:3300/0 conn(0x7fb424155d80 0x7fb424156160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:20.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.167+0000 7fb429982640 1 -- 172.21.15.112:0/1547931168 >> v1:172.21.15.112:6789/0 conn(0x7fb42414c5c0 legacy=0x7fb424153a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:35:20.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.167+0000 7fb429982640 1 -- 172.21.15.112:0/1547931168 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42406e570 con 0x7fb42406dab0 2024-04-25T01:35:20.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.169+0000 7fb429982640 1 --2- 172.21.15.112:0/1547931168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb42406dab0 0x7fb42414c080 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb414009870 tx=0x7fb41402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a10867bdd99a66a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:20.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.170+0000 7fb429181640 1 -- 172.21.15.112:0/1547931168 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414037070 con 0x7fb42406dab0 2024-04-25T01:35:20.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.170+0000 7fb429181640 1 -- 172.21.15.112:0/1547931168 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb414032440 con 0x7fb42406dab0 2024-04-25T01:35:20.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.170+0000 7fb429181640 1 -- 172.21.15.112:0/1547931168 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414032810 con 0x7fb42406dab0 2024-04-25T01:35:20.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.171+0000 7fb42c3e7640 1 -- 172.21.15.112:0/1547931168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb42406dab0 msgr2=0x7fb42414c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:20.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.171+0000 7fb42c3e7640 1 --2- 172.21.15.112:0/1547931168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb42406dab0 0x7fb42414c080 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb414009870 tx=0x7fb41402a1a0 comp rx=0 tx=0).stop 2024-04-25T01:35:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.171+0000 7fb42c3e7640 1 -- 172.21.15.112:0/1547931168 shutdown_connections 2024-04-25T01:35:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.171+0000 7fb42c3e7640 1 --2- 172.21.15.112:0/1547931168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb42406dab0 0x7fb42414c080 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.171+0000 7fb42c3e7640 1 --2- 172.21.15.112:0/1547931168 >> v2:172.21.15.112:3300/0 conn(0x7fb424155d80 0x7fb424156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.171+0000 7fb42c3e7640 1 -- 172.21.15.112:0/1547931168 >> 172.21.15.112:0/1547931168 conn(0x7fb4240c6f80 msgr2=0x7fb4240c7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.171+0000 7fb42c3e7640 1 -- 172.21.15.112:0/1547931168 shutdown_connections 2024-04-25T01:35:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.171+0000 7fb42c3e7640 1 -- 172.21.15.112:0/1547931168 wait complete. 2024-04-25T01:35:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.172+0000 7fb42c3e7640 1 Processor -- start 2024-04-25T01:35:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.172+0000 7fb42c3e7640 1 -- start start 2024-04-25T01:35:20.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.173+0000 7fb42c3e7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb42406dab0 0x7fb4241eeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.173+0000 7fb42c3e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb42414c5c0 0x7fb424145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.173+0000 7fb42c3e7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb424155d80 0x7fb424144690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.173+0000 7fb42c3e7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb424156820 con 0x7fb42414c5c0 2024-04-25T01:35:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.173+0000 7fb42c3e7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241566a0 con 0x7fb42406dab0 2024-04-25T01:35:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.173+0000 7fb42c3e7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb42406c910 con 0x7fb424155d80 2024-04-25T01:35:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.173+0000 7fb429982640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb42414c5c0 msgr2=0x7fb424145860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:35:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.173+0000 7fb429982640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb42414c5c0 0x7fb424145860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:35:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.173+0000 7fb42a183640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb42406dab0 0x7fb4241eeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.173+0000 7fb42a183640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb42406dab0 0x7fb4241eeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43260/0 (socket says 172.21.15.112:43260) 2024-04-25T01:35:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.173+0000 7fb42a183640 1 -- 172.21.15.112:0/470541142 learned_addr learned my addr 172.21.15.112:0/470541142 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.173+0000 7fb42a984640 1 --2- 172.21.15.112:0/470541142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb424155d80 0x7fb424144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.174+0000 7fb42a183640 1 -- 172.21.15.112:0/470541142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb424155d80 msgr2=0x7fb424144690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.174+0000 7fb42a183640 1 --2- 172.21.15.112:0/470541142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb424155d80 0x7fb424144690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.174+0000 7fb42a183640 1 -- 172.21.15.112:0/470541142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb42414c5c0 msgr2=0x7fb424145860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:35:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.174+0000 7fb42a183640 1 --2- 172.21.15.112:0/470541142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb42414c5c0 0x7fb424145860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.174+0000 7fb42a183640 1 -- 172.21.15.112:0/470541142 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb424151790 con 0x7fb42406dab0 2024-04-25T01:35:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.174+0000 7fb42a183640 1 --2- 172.21.15.112:0/470541142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb42406dab0 0x7fb4241eeeb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb42000b580 tx=0x7fb42000ba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.174+0000 7fb41affd640 1 -- 172.21.15.112:0/470541142 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb420003bc0 con 0x7fb42406dab0 2024-04-25T01:35:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.174+0000 7fb41affd640 1 -- 172.21.15.112:0/470541142 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb420015070 con 0x7fb42406dab0 2024-04-25T01:35:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.174+0000 7fb42c3e7640 1 -- 172.21.15.112:0/470541142 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4240abd60 con 0x7fb42406dab0 2024-04-25T01:35:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.174+0000 7fb41affd640 1 -- 172.21.15.112:0/470541142 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4200050f0 con 0x7fb42406dab0 2024-04-25T01:35:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.174+0000 7fb42c3e7640 1 -- 172.21.15.112:0/470541142 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4240ac320 con 0x7fb42406dab0 2024-04-25T01:35:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.176+0000 7fb42c3e7640 1 -- 172.21.15.112:0/470541142 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4240682a0 con 0x7fb42406dab0 2024-04-25T01:35:20.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.177+0000 7fb41affd640 1 -- 172.21.15.112:0/470541142 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb420005290 con 0x7fb42406dab0 2024-04-25T01:35:20.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.177+0000 7fb41affd640 1 --2- 172.21.15.112:0/470541142 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb40806bde0 0x7fb40806e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:20.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.177+0000 7fb41affd640 1 -- 172.21.15.112:0/470541142 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fb42008ab50 con 0x7fb42406dab0 2024-04-25T01:35:20.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.177+0000 7fb429982640 1 --2- 172.21.15.112:0/470541142 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb40806bde0 0x7fb40806e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:20.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.178+0000 7fb429982640 1 --2- 172.21.15.112:0/470541142 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb40806bde0 0x7fb40806e2a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb414034000 tx=0x7fb41402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:20.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.184+0000 7fb41affd640 1 -- 172.21.15.112:0/470541142 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb420059160 con 0x7fb42406dab0 2024-04-25T01:35:20.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.595+0000 7fb42c3e7640 1 -- 172.21.15.112:0/470541142 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb4240c7770 con 0x7fb42406dab0 2024-04-25T01:35:20.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.595+0000 7fb41affd640 1 -- 172.21.15.112:0/470541142 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7fb420058a60 con 0x7fb42406dab0 2024-04-25T01:35:20.572 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":76,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:35:20.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.598+0000 7fb42c3e7640 1 -- 172.21.15.112:0/470541142 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb40806bde0 msgr2=0x7fb40806e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:20.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.598+0000 7fb42c3e7640 1 --2- 172.21.15.112:0/470541142 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb40806bde0 0x7fb40806e2a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb414034000 tx=0x7fb41402e000 comp rx=0 tx=0).stop 2024-04-25T01:35:20.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.598+0000 7fb42c3e7640 1 -- 172.21.15.112:0/470541142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb42406dab0 msgr2=0x7fb4241eeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:20.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.598+0000 7fb42c3e7640 1 --2- 172.21.15.112:0/470541142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb42406dab0 0x7fb4241eeeb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb42000b580 tx=0x7fb42000ba40 comp rx=0 tx=0).stop 2024-04-25T01:35:20.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.598+0000 7fb42c3e7640 1 -- 172.21.15.112:0/470541142 shutdown_connections 2024-04-25T01:35:20.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.598+0000 7fb42c3e7640 1 --2- 172.21.15.112:0/470541142 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb40806bde0 0x7fb40806e2a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:20.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.598+0000 7fb42c3e7640 1 --2- 172.21.15.112:0/470541142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb424155d80 0x7fb424144690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:20.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.598+0000 7fb42c3e7640 1 --2- 172.21.15.112:0/470541142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb42414c5c0 0x7fb424145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:20.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.598+0000 7fb42c3e7640 1 --2- 172.21.15.112:0/470541142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb42406dab0 0x7fb4241eeeb0 secure :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb42000b580 tx=0x7fb42000ba40 comp rx=0 tx=0).stop 2024-04-25T01:35:20.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.598+0000 7fb42c3e7640 1 -- 172.21.15.112:0/470541142 >> 172.21.15.112:0/470541142 conn(0x7fb4240c6f80 msgr2=0x7fb424148350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:20.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.598+0000 7fb42c3e7640 1 -- 172.21.15.112:0/470541142 shutdown_connections 2024-04-25T01:35:20.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:20.598+0000 7fb42c3e7640 1 -- 172.21.15.112:0/470541142 wait complete. 2024-04-25T01:35:23.591 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:35:23.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:23.752+0000 7f79ab8a5640 1 Processor -- start 2024-04-25T01:35:23.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:23.752+0000 7f79ab8a5640 1 -- start start 2024-04-25T01:35:23.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:23.753+0000 7f79ab8a5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f79a4152f80 0x7f79a40c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:23.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:23.753+0000 7f79ab8a5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79a40c4290 con 0x7f79a40c6180 2024-04-25T01:35:23.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:23.753+0000 7f79ab8a5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79a40c8760 con 0x7f79a4159530 2024-04-25T01:35:23.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:23.753+0000 7f79ab8a5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f79a40c83c0 con 0x7f79a4152f80 2024-04-25T01:35:23.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:23.753+0000 7f79a8e40640 1 -- >> v1:172.21.15.112:6789/0 conn(0x7f79a40c6180 legacy=0x7f79a4158e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:35:23.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:23.753+0000 7f79a9641640 1 -- >> v2:172.21.15.112:3300/0 conn(0x7f79a4152f80 msgr2=0x7f79a40c7780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:35:23.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:23.753+0000 7f79a9641640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f79a4152f80 0x7f79a40c7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:35:23.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:23.754+0000 7f79a9e42640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f79a4159530 0x7f79a41601b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:59926/0 (socket says 172.21.15.112:59926) 2024-04-25T01:35:23.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:23.754+0000 7f79a9e42640 1 -- 172.21.15.112:0/193206877 learned_addr learned my addr 172.21.15.112:0/193206877 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:35:23.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:23.954+0000 7f79a9641640 1 -- 172.21.15.112:0/193206877 >> v2:172.21.15.112:3300/0 conn(0x7f79a4152f80 msgr2=0x7f79a40c7780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:35:23.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:23.954+0000 7f79a8e40640 1 -- 172.21.15.112:0/193206877 >> v1:172.21.15.112:6789/0 conn(0x7f79a40c6180 legacy=0x7f79a4158e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:35:23.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:23.954+0000 7f79a9641640 1 --2- 172.21.15.112:0/193206877 >> v2:172.21.15.112:3300/0 conn(0x7f79a4152f80 0x7f79a40c7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:35:24.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:24.355+0000 7f79a8e40640 1 -- 172.21.15.112:0/193206877 >> v1:172.21.15.112:6789/0 conn(0x7f79a40c6180 legacy=0x7f79a4158e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:35:24.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:24.355+0000 7f79a9641640 1 -- 172.21.15.112:0/193206877 >> v2:172.21.15.112:3300/0 conn(0x7f79a4152f80 msgr2=0x7f79a40c7780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:35:24.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:24.355+0000 7f79a9641640 1 --2- 172.21.15.112:0/193206877 >> v2:172.21.15.112:3300/0 conn(0x7f79a4152f80 0x7f79a40c7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:35:25.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:25.156+0000 7f79a8e40640 1 -- 172.21.15.112:0/193206877 >> v1:172.21.15.112:6789/0 conn(0x7f79a40c6180 legacy=0x7f79a4158e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:35:25.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:25.156+0000 7f79a9641640 1 -- 172.21.15.112:0/193206877 >> v2:172.21.15.112:3300/0 conn(0x7f79a4152f80 msgr2=0x7f79a40c7780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:35:25.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:25.156+0000 7f79a9641640 1 --2- 172.21.15.112:0/193206877 >> v2:172.21.15.112:3300/0 conn(0x7f79a4152f80 0x7f79a40c7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:35:26.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.753+0000 7f799affd640 1 -- 172.21.15.112:0/193206877 >> v2:172.21.15.112:3300/0 conn(0x7f79a4152f80 msgr2=0x7f79a40c7780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:35:26.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.753+0000 7f799affd640 1 --2- 172.21.15.112:0/193206877 >> v2:172.21.15.112:3300/0 conn(0x7f79a4152f80 0x7f79a40c7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:26.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.753+0000 7f799affd640 1 -- 172.21.15.112:0/193206877 >> v1:172.21.15.187:6789/0 conn(0x7f79a4159530 legacy=0x7f79a41601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:26.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.753+0000 7f799affd640 1 -- 172.21.15.112:0/193206877 >> v1:172.21.15.112:6789/0 conn(0x7f79a40c6180 legacy=0x7f79a4158e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:35:26.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.753+0000 7f799affd640 1 --2- 172.21.15.112:0/193206877 >> v2:172.21.15.187:3300/0 conn(0x7f79880036b0 0x7f7988005b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:26.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.753+0000 7f799affd640 1 --2- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7988006120 0x7f79880085e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:26.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.753+0000 7f799affd640 1 -- 172.21.15.112:0/193206877 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f79a40c83c0 con 0x7f79880036b0 2024-04-25T01:35:26.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.753+0000 7f799affd640 1 -- 172.21.15.112:0/193206877 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7988008d10 con 0x7f7988006120 2024-04-25T01:35:26.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.753+0000 7f79a9641640 1 --2- 172.21.15.112:0/193206877 >> v2:172.21.15.187:3300/0 conn(0x7f79880036b0 0x7f7988005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:26.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.753+0000 7f79a8e40640 1 --2- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7988006120 0x7f79880085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:26.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.753+0000 7f79a9641640 1 -- 172.21.15.112:0/193206877 >> v2:172.21.15.187:3300/0 conn(0x7f79880036b0 msgr2=0x7f7988005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:35:26.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.753+0000 7f79a9641640 1 -- 172.21.15.112:0/193206877 >> v2:172.21.15.187:3300/0 conn(0x7f79880036b0 msgr2=0x7f7988005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:35:26.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.753+0000 7f79a9641640 1 --2- 172.21.15.112:0/193206877 >> v2:172.21.15.187:3300/0 conn(0x7f79880036b0 0x7f7988005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:35:26.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.753+0000 7f79a9641640 1 --2- 172.21.15.112:0/193206877 >> v2:172.21.15.187:3300/0 conn(0x7f79880036b0 0x7f7988005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:35:26.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.753+0000 7f79a8e40640 1 -- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7988006120 msgr2=0x7f79880085e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:35:26.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.753+0000 7f79a8e40640 1 -- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7988006120 msgr2=0x7f79880085e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:35:26.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.753+0000 7f79a8e40640 1 --2- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7988006120 0x7f79880085e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:35:26.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.753+0000 7f79a8e40640 1 --2- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7988006120 0x7f79880085e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:35:26.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.954+0000 7f79a9641640 1 --2- 172.21.15.112:0/193206877 >> v2:172.21.15.187:3300/0 conn(0x7f79880036b0 0x7f7988005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:26.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.954+0000 7f79a9641640 1 -- 172.21.15.112:0/193206877 >> v2:172.21.15.187:3300/0 conn(0x7f79880036b0 msgr2=0x7f7988005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:35:26.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.954+0000 7f79a9641640 1 -- 172.21.15.112:0/193206877 >> v2:172.21.15.187:3300/0 conn(0x7f79880036b0 msgr2=0x7f7988005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:35:26.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.954+0000 7f79a9641640 1 --2- 172.21.15.112:0/193206877 >> v2:172.21.15.187:3300/0 conn(0x7f79880036b0 0x7f7988005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:35:26.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.954+0000 7f79a9641640 1 --2- 172.21.15.112:0/193206877 >> v2:172.21.15.187:3300/0 conn(0x7f79880036b0 0x7f7988005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:35:26.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.955+0000 7f79a8e40640 1 --2- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7988006120 0x7f79880085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:26.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.955+0000 7f79a8e40640 1 -- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7988006120 msgr2=0x7f79880085e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:35:26.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.955+0000 7f79a8e40640 1 -- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7988006120 msgr2=0x7f79880085e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:35:26.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.955+0000 7f79a8e40640 1 --2- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7988006120 0x7f79880085e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:35:26.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:26.955+0000 7f79a8e40640 1 --2- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7988006120 0x7f79880085e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:35:27.188 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:35:27.191 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:35:27.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:27.355+0000 7f79a9641640 1 --2- 172.21.15.112:0/193206877 >> v2:172.21.15.187:3300/0 conn(0x7f79880036b0 0x7f7988005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:27.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:27.355+0000 7f79a9641640 1 -- 172.21.15.112:0/193206877 >> v2:172.21.15.187:3300/0 conn(0x7f79880036b0 msgr2=0x7f7988005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:35:27.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:27.355+0000 7f79a9641640 1 -- 172.21.15.112:0/193206877 >> v2:172.21.15.187:3300/0 conn(0x7f79880036b0 msgr2=0x7f7988005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:35:27.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:27.355+0000 7f79a9641640 1 --2- 172.21.15.112:0/193206877 >> v2:172.21.15.187:3300/0 conn(0x7f79880036b0 0x7f7988005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:35:27.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:27.355+0000 7f79a9641640 1 --2- 172.21.15.112:0/193206877 >> v2:172.21.15.187:3300/0 conn(0x7f79880036b0 0x7f7988005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:35:27.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:27.356+0000 7f79a8e40640 1 --2- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7988006120 0x7f79880085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:27.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:27.356+0000 7f79a8e40640 1 -- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7988006120 msgr2=0x7f79880085e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:35:27.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:27.356+0000 7f79a8e40640 1 -- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7988006120 msgr2=0x7f79880085e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:35:27.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:27.356+0000 7f79a8e40640 1 --2- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7988006120 0x7f79880085e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:35:27.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:27.356+0000 7f79a8e40640 1 --2- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7988006120 0x7f79880085e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:35:28.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.156+0000 7f79a9641640 1 --2- 172.21.15.112:0/193206877 >> v2:172.21.15.187:3300/0 conn(0x7f79880036b0 0x7f7988005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:28.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.156+0000 7f79a9641640 1 -- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7988006120 msgr2=0x7f79880085e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:35:28.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.156+0000 7f79a9641640 1 --2- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7988006120 0x7f79880085e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.156+0000 7f79a9641640 1 -- 172.21.15.112:0/193206877 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a40c8a40 con 0x7f79880036b0 2024-04-25T01:35:28.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.157+0000 7f79a9641640 1 --2- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79880036b0 0x7f7988005b30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f79a0003980 tx=0x7f79a002b640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d6d0917ed2c9ad5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:28.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.158+0000 7f799bfff640 1 -- 172.21.15.112:0/193206877 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0031480 con 0x7f79880036b0 2024-04-25T01:35:28.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.158+0000 7f799bfff640 1 -- 172.21.15.112:0/193206877 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a0031620 con 0x7f79880036b0 2024-04-25T01:35:28.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.158+0000 7f799bfff640 1 -- 172.21.15.112:0/193206877 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0031a10 con 0x7f79880036b0 2024-04-25T01:35:28.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.159+0000 7f79ab8a5640 1 -- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79880036b0 msgr2=0x7f7988005b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:28.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.159+0000 7f79ab8a5640 1 --2- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79880036b0 0x7f7988005b30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f79a0003980 tx=0x7f79a002b640 comp rx=0 tx=0).stop 2024-04-25T01:35:28.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.159+0000 7f79a9e42640 1 -- 172.21.15.112:0/193206877 reap_dead start 2024-04-25T01:35:28.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.159+0000 7f79ab8a5640 1 -- 172.21.15.112:0/193206877 shutdown_connections 2024-04-25T01:35:28.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.159+0000 7f79ab8a5640 1 --2- 172.21.15.112:0/193206877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79880036b0 0x7f7988005b30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.159+0000 7f79ab8a5640 1 -- 172.21.15.112:0/193206877 >> 172.21.15.112:0/193206877 conn(0x7f79a40c8f80 msgr2=0x7f79a40c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:28.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.159+0000 7f79ab8a5640 1 -- 172.21.15.112:0/193206877 shutdown_connections 2024-04-25T01:35:28.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.159+0000 7f79ab8a5640 1 -- 172.21.15.112:0/193206877 wait complete. 2024-04-25T01:35:28.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.161+0000 7f79ab8a5640 1 Processor -- start 2024-04-25T01:35:28.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.161+0000 7f79ab8a5640 1 -- start start 2024-04-25T01:35:28.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.162+0000 7f79ab8a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79880036b0 0x7f79a41524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.162+0000 7f79ab8a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79a40adca0 0x7f79a40ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.162+0000 7f79ab8a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79a4156c20 0x7f79a40ac760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.162+0000 7f79ab8a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f79a40c8d20 con 0x7f79a4156c20 2024-04-25T01:35:28.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.162+0000 7f79ab8a5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f79a40c48a0 con 0x7f79880036b0 2024-04-25T01:35:28.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.162+0000 7f79ab8a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f79a40c4d30 con 0x7f79a40adca0 2024-04-25T01:35:28.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.162+0000 7f79a9e42640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79a4156c20 msgr2=0x7f79a40ac760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:35:28.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.162+0000 7f79a9e42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79a4156c20 0x7f79a40ac760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:35:28.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.162+0000 7f79a9641640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79880036b0 0x7f79a41524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:28.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.162+0000 7f79a8e40640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79a40adca0 0x7f79a40ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:28.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.162+0000 7f79a9641640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79880036b0 0x7f79a41524c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:59446/0 (socket says 172.21.15.112:59446) 2024-04-25T01:35:28.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.162+0000 7f79a8e40640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79a40adca0 0x7f79a40ae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:43036/0 (socket says 172.21.15.112:43036) 2024-04-25T01:35:28.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.162+0000 7f79a9641640 1 -- 172.21.15.112:0/1552656380 learned_addr learned my addr 172.21.15.112:0/1552656380 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:28.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.162+0000 7f79a8e40640 1 -- 172.21.15.112:0/1552656380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79880036b0 msgr2=0x7f79a41524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:28.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.162+0000 7f79a8e40640 1 --2- 172.21.15.112:0/1552656380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79880036b0 0x7f79a41524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.162+0000 7f79a8e40640 1 -- 172.21.15.112:0/1552656380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79a4156c20 msgr2=0x7f79a40ac760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:35:28.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.162+0000 7f79a8e40640 1 --2- 172.21.15.112:0/1552656380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79a4156c20 0x7f79a40ac760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.162+0000 7f79a8e40640 1 -- 172.21.15.112:0/1552656380 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a40aec10 con 0x7f79a40adca0 2024-04-25T01:35:28.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.162+0000 7f79a9641640 1 --2- 172.21.15.112:0/1552656380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79880036b0 0x7f79a41524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:35:28.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.163+0000 7f79a8e40640 1 --2- 172.21.15.112:0/1552656380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79a40adca0 0x7f79a40ae5d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f79a40c65c0 tx=0x7f799400a7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:28.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.163+0000 7f7999ffb640 1 -- 172.21.15.112:0/1552656380 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799401b070 con 0x7f79a40adca0 2024-04-25T01:35:28.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.163+0000 7f79ab8a5640 1 -- 172.21.15.112:0/1552656380 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a40aeef0 con 0x7f79a40adca0 2024-04-25T01:35:28.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.163+0000 7f7999ffb640 1 -- 172.21.15.112:0/1552656380 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7994016630 con 0x7f79a40adca0 2024-04-25T01:35:28.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.163+0000 7f7999ffb640 1 -- 172.21.15.112:0/1552656380 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994016a20 con 0x7f79a40adca0 2024-04-25T01:35:28.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.163+0000 7f79ab8a5640 1 -- 172.21.15.112:0/1552656380 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a4158b80 con 0x7f79a40adca0 2024-04-25T01:35:28.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.164+0000 7f79ab8a5640 1 -- 172.21.15.112:0/1552656380 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a40682a0 con 0x7f79a40adca0 2024-04-25T01:35:28.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.165+0000 7f7999ffb640 1 -- 172.21.15.112:0/1552656380 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f799401e070 con 0x7f79a40adca0 2024-04-25T01:35:28.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.165+0000 7f7999ffb640 1 --2- 172.21.15.112:0/1552656380 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f798806cd10 0x7f798806f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.165+0000 7f7999ffb640 1 -- 172.21.15.112:0/1552656380 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f799408b390 con 0x7f79a40adca0 2024-04-25T01:35:28.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.165+0000 7f79a9641640 1 --2- 172.21.15.112:0/1552656380 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f798806cd10 0x7f798806f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:28.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.166+0000 7f79a9641640 1 --2- 172.21.15.112:0/1552656380 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f798806cd10 0x7f798806f1d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009bf0 tx=0x7f79a002f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:28.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.172+0000 7f7999ffb640 1 -- 172.21.15.112:0/1552656380 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f79940593e0 con 0x7f79a40adca0 2024-04-25T01:35:28.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.559+0000 7f79ab8a5640 1 -- 172.21.15.112:0/1552656380 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f79a40c8540 con 0x7f79a40adca0 2024-04-25T01:35:28.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.559+0000 7f7999ffb640 1 -- 172.21.15.112:0/1552656380 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7f799405d6e0 con 0x7f79a40adca0 2024-04-25T01:35:28.536 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":78,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:35:28.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.562+0000 7f79ab8a5640 1 -- 172.21.15.112:0/1552656380 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f798806cd10 msgr2=0x7f798806f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:28.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.562+0000 7f79ab8a5640 1 --2- 172.21.15.112:0/1552656380 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f798806cd10 0x7f798806f1d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009bf0 tx=0x7f79a002f000 comp rx=0 tx=0).stop 2024-04-25T01:35:28.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.562+0000 7f79ab8a5640 1 -- 172.21.15.112:0/1552656380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79a40adca0 msgr2=0x7f79a40ae5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:28.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.562+0000 7f79ab8a5640 1 --2- 172.21.15.112:0/1552656380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79a40adca0 0x7f79a40ae5d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f79a40c65c0 tx=0x7f799400a7d0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.562+0000 7f79ab8a5640 1 -- 172.21.15.112:0/1552656380 shutdown_connections 2024-04-25T01:35:28.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.562+0000 7f79ab8a5640 1 --2- 172.21.15.112:0/1552656380 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f798806cd10 0x7f798806f1d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.562+0000 7f79ab8a5640 1 --2- 172.21.15.112:0/1552656380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79a4156c20 0x7f79a40ac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.562+0000 7f79ab8a5640 1 --2- 172.21.15.112:0/1552656380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79a40adca0 0x7f79a40ae5d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.562+0000 7f79ab8a5640 1 --2- 172.21.15.112:0/1552656380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79880036b0 0x7f79a41524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.562+0000 7f79ab8a5640 1 -- 172.21.15.112:0/1552656380 >> 172.21.15.112:0/1552656380 conn(0x7f79a40c8f80 msgr2=0x7f79a406da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:28.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.562+0000 7f79ab8a5640 1 -- 172.21.15.112:0/1552656380 shutdown_connections 2024-04-25T01:35:28.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.562+0000 7f79ab8a5640 1 -- 172.21.15.112:0/1552656380 wait complete. 2024-04-25T01:35:28.554 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T01:35:28.554 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-04-25T01:35:28.554 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:35:28.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.718+0000 7fb340a15640 1 Processor -- start 2024-04-25T01:35:28.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.718+0000 7fb340a15640 1 -- start start 2024-04-25T01:35:28.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.719+0000 7fb340a15640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb33c14fa90 0x7fb33c1563c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.719+0000 7fb340a15640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb33c0727f0 0x7fb33c06b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.719+0000 7fb340a15640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb33c06b6a0 0x7fb33c0c3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.719+0000 7fb340a15640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fb33c14fe70 con 0x7fb33c14fa90 2024-04-25T01:35:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.719+0000 7fb340a15640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb33c0720e0 con 0x7fb33c0727f0 2024-04-25T01:35:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.719+0000 7fb340a15640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb33c158f30 con 0x7fb33c06b6a0 2024-04-25T01:35:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.720+0000 7fb33ad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb33c06b6a0 0x7fb33c0c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.720+0000 7fb33ad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb33c06b6a0 0x7fb33c0c3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:43046/0 (socket says 172.21.15.112:43046) 2024-04-25T01:35:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.720+0000 7fb33ad9d640 1 -- 172.21.15.112:0/3327940818 learned_addr learned my addr 172.21.15.112:0/3327940818 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.720+0000 7fb333fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb33c0727f0 0x7fb33c06b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.720+0000 7fb33ad9d640 1 -- 172.21.15.112:0/3327940818 >> v2:172.21.15.187:3300/0 conn(0x7fb33c0727f0 msgr2=0x7fb33c06b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:35:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.720+0000 7fb33ad9d640 1 --2- 172.21.15.112:0/3327940818 >> v2:172.21.15.187:3300/0 conn(0x7fb33c0727f0 0x7fb33c06b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.720+0000 7fb33ad9d640 1 -- 172.21.15.112:0/3327940818 >> v2:172.21.15.112:3300/0 conn(0x7fb33c14fa90 msgr2=0x7fb33c1563c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:35:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.720+0000 7fb33a59c640 1 -- 172.21.15.112:0/3327940818 >> v2:172.21.15.112:3300/0 conn(0x7fb33c14fa90 msgr2=0x7fb33c1563c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:35:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.720+0000 7fb33a59c640 1 --2- 172.21.15.112:0/3327940818 >> v2:172.21.15.112:3300/0 conn(0x7fb33c14fa90 0x7fb33c1563c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:35:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.720+0000 7fb33ad9d640 1 --2- 172.21.15.112:0/3327940818 >> v2:172.21.15.112:3300/0 conn(0x7fb33c14fa90 0x7fb33c1563c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.720+0000 7fb33ad9d640 1 -- 172.21.15.112:0/3327940818 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33c1591f0 con 0x7fb33c06b6a0 2024-04-25T01:35:28.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.721+0000 7fb33ad9d640 1 --2- 172.21.15.112:0/3327940818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb33c06b6a0 0x7fb33c0c3860 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0c8dc0 tx=0x7fb33402a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee1cfbdda9497d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:28.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.721+0000 7fb339d9b640 1 -- 172.21.15.112:0/3327940818 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334036070 con 0x7fb33c06b6a0 2024-04-25T01:35:28.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.722+0000 7fb339d9b640 1 -- 172.21.15.112:0/3327940818 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb334031600 con 0x7fb33c06b6a0 2024-04-25T01:35:28.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.722+0000 7fb339d9b640 1 -- 172.21.15.112:0/3327940818 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3340319d0 con 0x7fb33c06b6a0 2024-04-25T01:35:28.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.722+0000 7fb340a15640 1 -- 172.21.15.112:0/3327940818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb33c06b6a0 msgr2=0x7fb33c0c3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:28.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.722+0000 7fb340a15640 1 --2- 172.21.15.112:0/3327940818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb33c06b6a0 0x7fb33c0c3860 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0c8dc0 tx=0x7fb33402a6c0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.723+0000 7fb340a15640 1 -- 172.21.15.112:0/3327940818 shutdown_connections 2024-04-25T01:35:28.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.723+0000 7fb340a15640 1 --2- 172.21.15.112:0/3327940818 >> v2:172.21.15.187:3300/0 conn(0x7fb33c0727f0 0x7fb33c06b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.723+0000 7fb340a15640 1 --2- 172.21.15.112:0/3327940818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb33c06b6a0 0x7fb33c0c3860 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.723+0000 7fb340a15640 1 --2- 172.21.15.112:0/3327940818 >> v2:172.21.15.112:3300/0 conn(0x7fb33c14fa90 0x7fb33c1563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.723+0000 7fb340a15640 1 -- 172.21.15.112:0/3327940818 >> 172.21.15.112:0/3327940818 conn(0x7fb33c06c8a0 msgr2=0x7fb33c0c8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:28.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.723+0000 7fb340a15640 1 -- 172.21.15.112:0/3327940818 shutdown_connections 2024-04-25T01:35:28.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.723+0000 7fb340a15640 1 -- 172.21.15.112:0/3327940818 wait complete. 2024-04-25T01:35:28.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.724+0000 7fb340a15640 1 Processor -- start 2024-04-25T01:35:28.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.725+0000 7fb340a15640 1 -- start start 2024-04-25T01:35:28.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.726+0000 7fb340a15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb33c06b6a0 0x7fb33c1ec960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.726+0000 7fb340a15640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb33c0727f0 0x7fb33c1ef470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.726+0000 7fb340a15640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb33c14fa90 0x7fb33c06f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.726+0000 7fb340a15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c159a90 con 0x7fb33c06b6a0 2024-04-25T01:35:28.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.726+0000 7fb340a15640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c1597b0 con 0x7fb33c0727f0 2024-04-25T01:35:28.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.726+0000 7fb340a15640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb33c0c3da0 con 0x7fb33c14fa90 2024-04-25T01:35:28.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.726+0000 7fb33a59c640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb33c06b6a0 msgr2=0x7fb33c1ec960 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:35:28.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.726+0000 7fb33a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb33c06b6a0 0x7fb33c1ec960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:35:28.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.726+0000 7fb333fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb33c0727f0 0x7fb33c1ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:28.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.726+0000 7fb333fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb33c0727f0 0x7fb33c1ef470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:59472/0 (socket says 172.21.15.112:59472) 2024-04-25T01:35:28.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.726+0000 7fb333fff640 1 -- 172.21.15.112:0/2165675295 learned_addr learned my addr 172.21.15.112:0/2165675295 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:28.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.726+0000 7fb33ad9d640 1 --2- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb33c14fa90 0x7fb33c06f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:28.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.726+0000 7fb333fff640 1 -- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb33c14fa90 msgr2=0x7fb33c06f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:28.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.727+0000 7fb333fff640 1 --2- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb33c14fa90 0x7fb33c06f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.727+0000 7fb333fff640 1 -- 172.21.15.112:0/2165675295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb33c06b6a0 msgr2=0x7fb33c1ec960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:35:28.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.727+0000 7fb333fff640 1 --2- 172.21.15.112:0/2165675295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb33c06b6a0 0x7fb33c1ec960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.727+0000 7fb333fff640 1 -- 172.21.15.112:0/2165675295 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33c1eb600 con 0x7fb33c0727f0 2024-04-25T01:35:28.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.727+0000 7fb33ad9d640 1 --2- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb33c14fa90 0x7fb33c06f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:35:28.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.727+0000 7fb333fff640 1 --2- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb33c0727f0 0x7fb33c1ef470 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb32400e9e0 tx=0x7fb32400eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:28.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.727+0000 7fb332ffd640 1 -- 172.21.15.112:0/2165675295 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324017070 con 0x7fb33c0727f0 2024-04-25T01:35:28.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.727+0000 7fb340a15640 1 -- 172.21.15.112:0/2165675295 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33c155220 con 0x7fb33c0727f0 2024-04-25T01:35:28.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.728+0000 7fb340a15640 1 -- 172.21.15.112:0/2165675295 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb33c1eb8a0 con 0x7fb33c0727f0 2024-04-25T01:35:28.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.728+0000 7fb332ffd640 1 -- 172.21.15.112:0/2165675295 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb32400c9d0 con 0x7fb33c0727f0 2024-04-25T01:35:28.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.728+0000 7fb332ffd640 1 -- 172.21.15.112:0/2165675295 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324021060 con 0x7fb33c0727f0 2024-04-25T01:35:28.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.729+0000 7fb340a15640 1 --2- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb320002a70 0x7fb320004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.729+0000 7fb340a15640 1 -- 172.21.15.112:0/2165675295 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb320007ec0 con 0x7fb320002a70 2024-04-25T01:35:28.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.729+0000 7fb33a59c640 1 --2- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb320002a70 0x7fb320004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:28.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.729+0000 7fb33a59c640 1 -- 172.21.15.112:0/2165675295 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb3200080c0 con 0x7fb320002a70 2024-04-25T01:35:28.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.730+0000 7fb33a59c640 1 --2- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb320002a70 0x7fb320004f30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00b450 tx=0x7fb32c00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:28.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.730+0000 7fb332ffd640 1 -- 172.21.15.112:0/2165675295 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c003dc0 con 0x7fb320002a70 2024-04-25T01:35:28.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.730+0000 7fb332ffd640 1 -- 172.21.15.112:0/2165675295 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb32401a070 con 0x7fb33c0727f0 2024-04-25T01:35:28.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.731+0000 7fb332ffd640 1 --2- 172.21.15.112:0/2165675295 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb31c06bd60 0x7fb31c06e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.731+0000 7fb332ffd640 1 -- 172.21.15.112:0/2165675295 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb32c015070 con 0x7fb320002a70 2024-04-25T01:35:28.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.731+0000 7fb332ffd640 1 -- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb320002a70 msgr2=0x7fb320004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:28.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.731+0000 7fb332ffd640 1 --2- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb320002a70 0x7fb320004f30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00b450 tx=0x7fb32c00b910 comp rx=0 tx=0).stop 2024-04-25T01:35:28.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.731+0000 7fb332ffd640 1 -- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb320002a70 msgr2=0x7fb320004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:35:28.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.731+0000 7fb332ffd640 1 --2- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb320002a70 0x7fb320004f30 secure :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00b450 tx=0x7fb32c00b910 comp rx=0 tx=0).stop 2024-04-25T01:35:28.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.731+0000 7fb332ffd640 1 -- 172.21.15.112:0/2165675295 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fb32408b800 con 0x7fb33c0727f0 2024-04-25T01:35:28.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.731+0000 7fb33ad9d640 1 --2- 172.21.15.112:0/2165675295 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb31c06bd60 0x7fb31c06e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:28.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.733+0000 7fb33ad9d640 1 --2- 172.21.15.112:0/2165675295 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb31c06bd60 0x7fb31c06e220 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb334033000 tx=0x7fb334009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:28.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.744+0000 7fb340a15640 1 --2- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb320001640 0x7fb320001ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.744+0000 7fb340a15640 1 -- 172.21.15.112:0/2165675295 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb32000a3e0 con 0x7fb320001640 2024-04-25T01:35:28.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.744+0000 7fb33a59c640 1 --2- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb320001640 0x7fb320001ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:28.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.745+0000 7fb33a59c640 1 -- 172.21.15.112:0/2165675295 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb320002010 con 0x7fb320001640 2024-04-25T01:35:28.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.745+0000 7fb33a59c640 1 --2- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb320001640 0x7fb320001ad0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00b450 tx=0x7fb32c0059d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:28.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.746+0000 7fb332ffd640 1 -- 172.21.15.112:0/2165675295 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c0056a0 con 0x7fb320001640 2024-04-25T01:35:28.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.746+0000 7fb332ffd640 1 -- 172.21.15.112:0/2165675295 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5060 (secure 0 0 0) 0x7fb32c010400 con 0x7fb320001640 2024-04-25T01:35:28.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.746+0000 7fb332ffd640 1 -- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb320001640 msgr2=0x7fb320001ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:28.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.746+0000 7fb332ffd640 1 --2- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb320001640 0x7fb320001ad0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00b450 tx=0x7fb32c0059d0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.723 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:35:28.723 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:35:28.723 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:35:28.723 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:35:28.723 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 78, 2024-04-25T01:35:28.723 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 364435, 2024-04-25T01:35:28.723 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:35:28.723 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:35:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:35:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:35:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:35:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:35:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:35:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:35:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:35:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:35:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:35:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:35:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:35:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:35:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:35:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:35:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:35:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:35:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:35:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:35:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:35:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:35:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:35:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:35:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:35:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:35:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:35:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:35:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:35:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:35:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:35:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:35:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:35:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:35:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:35:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:35:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:35:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:35:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:28.731 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:28.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.731 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:35:28.731 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:35:28.731 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:28.731 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:28.731 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:28.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.731 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.731 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:35:28.731 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:35:28.731 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:28.731 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:28.731 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.731 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:28.731 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:35:28.731 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:28.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.732 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.732 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:28.732 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:35:28.732 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:28.732 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:28.732 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:28.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.732 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:35:28.732 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:35:28.732 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:28.732 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:28.732 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:28.732 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:28.733 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:28.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.733 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:35:28.733 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:35:28.733 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.733 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:28.733 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:28.733 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:35:28.733 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:28.733 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:28.733 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:35:28.733 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.733 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:28.733 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:28.733 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:35:28.733 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:28.734 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:28.734 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:35:28.734 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.734 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:28.734 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:28.734 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 7 2024-04-25T01:35:28.734 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:28.734 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:28.734 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:35:28.734 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.734 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:28.734 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:28.734 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 6 2024-04-25T01:35:28.734 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:28.734 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:28.735 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:35:28.735 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.735 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:28.735 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:28.735 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:35:28.735 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:28.735 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:28.735 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.735 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:35:28.735 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:35:28.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.746+0000 7fb332ffd640 1 -- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb320001640 msgr2=0x7fb320001ad0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:35:28.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.746+0000 7fb332ffd640 1 --2- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb320001640 0x7fb320001ad0 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00b450 tx=0x7fb32c0059d0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.747+0000 7fb340a15640 1 -- 172.21.15.112:0/2165675295 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb31c06bd60 msgr2=0x7fb31c06e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:28.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.747+0000 7fb340a15640 1 --2- 172.21.15.112:0/2165675295 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb31c06bd60 0x7fb31c06e220 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb334033000 tx=0x7fb334009000 comp rx=0 tx=0).stop 2024-04-25T01:35:28.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.747+0000 7fb340a15640 1 -- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb33c0727f0 msgr2=0x7fb33c1ef470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:28.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.747+0000 7fb340a15640 1 --2- 172.21.15.112:0/2165675295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb33c0727f0 0x7fb33c1ef470 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb32400e9e0 tx=0x7fb32400eea0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.747+0000 7fb33ad9d640 1 -- 172.21.15.112:0/2165675295 reap_dead start 2024-04-25T01:35:28.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.747+0000 7fb33ad9d640 1 -- 172.21.15.112:0/2165675295 reap_dead start 2024-04-25T01:35:28.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.747+0000 7fb340a15640 1 -- 172.21.15.112:0/2165675295 shutdown_connections 2024-04-25T01:35:28.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.747+0000 7fb340a15640 1 -- 172.21.15.112:0/2165675295 >> 172.21.15.112:0/2165675295 conn(0x7fb33c06c8a0 msgr2=0x7fb33c1562c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:28.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.747+0000 7fb340a15640 1 -- 172.21.15.112:0/2165675295 shutdown_connections 2024-04-25T01:35:28.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.747+0000 7fb340a15640 1 -- 172.21.15.112:0/2165675295 wait complete. 2024-04-25T01:35:28.739 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:35:28.739 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-04-25T01:35:28.739 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:35:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.905+0000 7fb3ec4df640 1 Processor -- start 2024-04-25T01:35:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.905+0000 7fb3ec4df640 1 -- start start 2024-04-25T01:35:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.907+0000 7fb3ec4df640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb3e40c9af0 0x7fb3e40c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.907+0000 7fb3ec4df640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e4156370 0x7fb3e40c5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.907+0000 7fb3ec4df640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3e40c8910 con 0x7fb3e4153030 2024-04-25T01:35:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.907+0000 7fb3ec4df640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fb3e4159160 con 0x7fb3e40c9af0 2024-04-25T01:35:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.907+0000 7fb3ec4df640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e40c8bf0 con 0x7fb3e4156370 2024-04-25T01:35:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.907+0000 7fb3e9a7a640 1 -- >> v2:172.21.15.112:3300/0 conn(0x7fb3e40c9af0 msgr2=0x7fb3e40c70a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:35:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.907+0000 7fb3e9a7a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb3e40c9af0 0x7fb3e40c70a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:35:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.907+0000 7fb3ea27b640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fb3e4153030 0x7fb3e41595f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:37436/0 (socket says 172.21.15.112:37436) 2024-04-25T01:35:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.907+0000 7fb3ea27b640 1 -- 172.21.15.112:0/4111539222 learned_addr learned my addr 172.21.15.112:0/4111539222 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:35:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.907+0000 7fb3eaa7c640 1 --2- 172.21.15.112:0/4111539222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e4156370 0x7fb3e40c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.908+0000 7fb3e9279640 1 -- 172.21.15.112:0/4111539222 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 645905473 0 0) 0x7fb3e40c8910 con 0x7fb3e4153030 2024-04-25T01:35:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.908+0000 7fb3e9279640 1 -- 172.21.15.112:0/4111539222 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3d0003520 con 0x7fb3e4153030 2024-04-25T01:35:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.908+0000 7fb3eaa7c640 1 -- 172.21.15.112:0/4111539222 >> v2:172.21.15.112:3300/0 conn(0x7fb3e40c9af0 msgr2=0x7fb3e40c70a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:35:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.908+0000 7fb3eaa7c640 1 --2- 172.21.15.112:0/4111539222 >> v2:172.21.15.112:3300/0 conn(0x7fb3e40c9af0 0x7fb3e40c70a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.908+0000 7fb3eaa7c640 1 -- 172.21.15.112:0/4111539222 >> v1:172.21.15.187:6789/0 conn(0x7fb3e4153030 legacy=0x7fb3e41595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.908+0000 7fb3eaa7c640 1 -- 172.21.15.112:0/4111539222 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e4159d00 con 0x7fb3e4156370 2024-04-25T01:35:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.909+0000 7fb3eaa7c640 1 --2- 172.21.15.112:0/4111539222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e4156370 0x7fb3e40c5960 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc009d90 tx=0x7fb3dc02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c6967438650b8fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.909+0000 7fb3e9279640 1 -- 172.21.15.112:0/4111539222 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc007740 con 0x7fb3e4156370 2024-04-25T01:35:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.909+0000 7fb3e9279640 1 -- 172.21.15.112:0/4111539222 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3dc0078e0 con 0x7fb3e4156370 2024-04-25T01:35:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.909+0000 7fb3e9279640 1 -- 172.21.15.112:0/4111539222 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc007cb0 con 0x7fb3e4156370 2024-04-25T01:35:28.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.910+0000 7fb3ec4df640 1 -- 172.21.15.112:0/4111539222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e4156370 msgr2=0x7fb3e40c5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:28.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.910+0000 7fb3ec4df640 1 --2- 172.21.15.112:0/4111539222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e4156370 0x7fb3e40c5960 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc009d90 tx=0x7fb3dc02a680 comp rx=0 tx=0).stop 2024-04-25T01:35:28.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.910+0000 7fb3ec4df640 1 -- 172.21.15.112:0/4111539222 shutdown_connections 2024-04-25T01:35:28.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.910+0000 7fb3ec4df640 1 --2- 172.21.15.112:0/4111539222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e4156370 0x7fb3e40c5960 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.910+0000 7fb3ec4df640 1 --2- 172.21.15.112:0/4111539222 >> v2:172.21.15.112:3300/0 conn(0x7fb3e40c9af0 0x7fb3e40c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.910+0000 7fb3ec4df640 1 -- 172.21.15.112:0/4111539222 >> 172.21.15.112:0/4111539222 conn(0x7fb3e4074a00 msgr2=0x7fb3e40c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:28.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.911+0000 7fb3ec4df640 1 -- 172.21.15.112:0/4111539222 shutdown_connections 2024-04-25T01:35:28.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.911+0000 7fb3ec4df640 1 -- 172.21.15.112:0/4111539222 wait complete. 2024-04-25T01:35:28.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.913+0000 7fb3ec4df640 1 Processor -- start 2024-04-25T01:35:28.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.913+0000 7fb3ec4df640 1 -- start start 2024-04-25T01:35:28.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.914+0000 7fb3ec4df640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e40c9af0 0x7fb3e40a8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.914+0000 7fb3ec4df640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e4153030 0x7fb3e414feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.914+0000 7fb3ec4df640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3e4156370 0x7fb3e41eb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.914+0000 7fb3ec4df640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e40c6020 con 0x7fb3e4153030 2024-04-25T01:35:28.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.914+0000 7fb3ec4df640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e40c5ea0 con 0x7fb3e4156370 2024-04-25T01:35:28.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.914+0000 7fb3ec4df640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e40c61a0 con 0x7fb3e40c9af0 2024-04-25T01:35:28.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.914+0000 7fb3e9a7a640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e4153030 msgr2=0x7fb3e414feb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:35:28.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.914+0000 7fb3e9a7a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e4153030 0x7fb3e414feb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:35:28.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.914+0000 7fb3ea27b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e40c9af0 0x7fb3e40a8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:28.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.914+0000 7fb3eaa7c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3e4156370 0x7fb3e41eb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:28.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.914+0000 7fb3ea27b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e40c9af0 0x7fb3e40a8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:43068/0 (socket says 172.21.15.112:43068) 2024-04-25T01:35:28.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.914+0000 7fb3ea27b640 1 -- 172.21.15.112:0/2722126641 learned_addr learned my addr 172.21.15.112:0/2722126641 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:28.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.914+0000 7fb3ea27b640 1 -- 172.21.15.112:0/2722126641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3e4156370 msgr2=0x7fb3e41eb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:28.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.914+0000 7fb3ea27b640 1 --2- 172.21.15.112:0/2722126641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3e4156370 0x7fb3e41eb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.914+0000 7fb3ea27b640 1 -- 172.21.15.112:0/2722126641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e4153030 msgr2=0x7fb3e414feb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:35:28.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.914+0000 7fb3ea27b640 1 --2- 172.21.15.112:0/2722126641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3e4153030 0x7fb3e414feb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.915+0000 7fb3ea27b640 1 -- 172.21.15.112:0/2722126641 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e41e0530 con 0x7fb3e40c9af0 2024-04-25T01:35:28.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.915+0000 7fb3ea27b640 1 --2- 172.21.15.112:0/2722126641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e40c9af0 0x7fb3e40a8ca0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000c530 tx=0x7fb3e000c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:28.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.915+0000 7fb3daffd640 1 -- 172.21.15.112:0/2722126641 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0018070 con 0x7fb3e40c9af0 2024-04-25T01:35:28.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.915+0000 7fb3daffd640 1 -- 172.21.15.112:0/2722126641 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3e00074e0 con 0x7fb3e40c9af0 2024-04-25T01:35:28.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.915+0000 7fb3daffd640 1 -- 172.21.15.112:0/2722126641 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0004a20 con 0x7fb3e40c9af0 2024-04-25T01:35:28.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.915+0000 7fb3ec4df640 1 -- 172.21.15.112:0/2722126641 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e41e0810 con 0x7fb3e40c9af0 2024-04-25T01:35:28.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.916+0000 7fb3ec4df640 1 -- 172.21.15.112:0/2722126641 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e41eeaf0 con 0x7fb3e40c9af0 2024-04-25T01:35:28.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.917+0000 7fb3ec4df640 1 --2- 172.21.15.112:0/2722126641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e41eed30 0x7fb3e4154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.917+0000 7fb3ec4df640 1 -- 172.21.15.112:0/2722126641 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e4075980 con 0x7fb3e41eed30 2024-04-25T01:35:28.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.917+0000 7fb3e9a7a640 1 --2- 172.21.15.112:0/2722126641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e41eed30 0x7fb3e4154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:28.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.918+0000 7fb3e9a7a640 1 -- 172.21.15.112:0/2722126641 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb3e4067da0 con 0x7fb3e41eed30 2024-04-25T01:35:28.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.918+0000 7fb3daffd640 1 -- 172.21.15.112:0/2722126641 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb3e0004bc0 con 0x7fb3e40c9af0 2024-04-25T01:35:28.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.918+0000 7fb3e9a7a640 1 --2- 172.21.15.112:0/2722126641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e41eed30 0x7fb3e4154940 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4155720 tx=0x7fb3d400b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:28.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.918+0000 7fb3daffd640 1 --2- 172.21.15.112:0/2722126641 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb3c806bde0 0x7fb3c806e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.918+0000 7fb3eaa7c640 1 --2- 172.21.15.112:0/2722126641 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb3c806bde0 0x7fb3c806e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:28.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.918+0000 7fb3daffd640 1 -- 172.21.15.112:0/2722126641 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fb3e008a410 con 0x7fb3e40c9af0 2024-04-25T01:35:28.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.919+0000 7fb3daffd640 1 -- 172.21.15.112:0/2722126641 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4003e10 con 0x7fb3e41eed30 2024-04-25T01:35:28.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.919+0000 7fb3daffd640 1 -- 172.21.15.112:0/2722126641 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb3d4015070 con 0x7fb3e41eed30 2024-04-25T01:35:28.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.919+0000 7fb3daffd640 1 -- 172.21.15.112:0/2722126641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e41eed30 msgr2=0x7fb3e4154940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:28.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.919+0000 7fb3daffd640 1 --2- 172.21.15.112:0/2722126641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e41eed30 0x7fb3e4154940 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4155720 tx=0x7fb3d400b8c0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.919+0000 7fb3daffd640 1 -- 172.21.15.112:0/2722126641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e41eed30 msgr2=0x7fb3e4154940 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:35:28.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.919+0000 7fb3daffd640 1 --2- 172.21.15.112:0/2722126641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e41eed30 0x7fb3e4154940 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.919+0000 7fb3eaa7c640 1 --2- 172.21.15.112:0/2722126641 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb3c806bde0 0x7fb3c806e2a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc034000 tx=0x7fb3dc02f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:28.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.929+0000 7fb3ec4df640 1 --2- 172.21.15.112:0/2722126641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e41ef2f0 0x7fb3e40ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.929+0000 7fb3ec4df640 1 -- 172.21.15.112:0/2722126641 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e406e240 con 0x7fb3e41ef2f0 2024-04-25T01:35:28.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.930+0000 7fb3e9a7a640 1 --2- 172.21.15.112:0/2722126641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e41ef2f0 0x7fb3e40ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:28.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.930+0000 7fb3e9a7a640 1 -- 172.21.15.112:0/2722126641 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb3e40682a0 con 0x7fb3e41ef2f0 2024-04-25T01:35:28.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.931+0000 7fb3e9a7a640 1 --2- 172.21.15.112:0/2722126641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e41ef2f0 0x7fb3e40ac7d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4014000 tx=0x7fb3d4005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:28.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.931+0000 7fb3daffd640 1 -- 172.21.15.112:0/2722126641 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4005820 con 0x7fb3e41ef2f0 2024-04-25T01:35:28.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.931+0000 7fb3daffd640 1 -- 172.21.15.112:0/2722126641 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4717 (secure 0 0 0) 0x7fb3d4005820 con 0x7fb3e41ef2f0 2024-04-25T01:35:28.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.931+0000 7fb3daffd640 1 -- 172.21.15.112:0/2722126641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e41ef2f0 msgr2=0x7fb3e40ac7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:28.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.931+0000 7fb3daffd640 1 --2- 172.21.15.112:0/2722126641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e41ef2f0 0x7fb3e40ac7d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4014000 tx=0x7fb3d4005a20 comp rx=0 tx=0).stop 2024-04-25T01:35:28.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.931+0000 7fb3daffd640 1 -- 172.21.15.112:0/2722126641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e41ef2f0 msgr2=0x7fb3e40ac7d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:35:28.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.931+0000 7fb3daffd640 1 --2- 172.21.15.112:0/2722126641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e41ef2f0 0x7fb3e40ac7d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.909 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:35:28.909 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:35:28.910 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:35:28.910 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:35:28.910 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 78, 2024-04-25T01:35:28.910 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 39591, 2024-04-25T01:35:28.910 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:35:28.910 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:35:28.910 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:35:28.910 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:28.910 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:35:28.910 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:35:28.910 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:35:28.910 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:35:28.910 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:28.910 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:28.911 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:28.911 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:28.911 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:28.911 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:28.911 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:28.911 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:28.911 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:28.911 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:28.911 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:28.911 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:28.911 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:35:28.911 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:35:28.911 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:28.911 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:28.912 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:28.912 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:28.912 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:28.912 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:28.912 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:28.912 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:28.912 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:28.912 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:28.912 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:28.912 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:28.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.912 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:35:28.912 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:35:28.913 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:35:28.913 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:35:28.913 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:35:28.913 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:35:28.913 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:35:28.913 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:35:28.913 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:35:28.913 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:35:28.913 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:35:28.913 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:35:28.913 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:35:28.914 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:35:28.914 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:35:28.914 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:35:28.914 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:35:28.914 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:35:28.914 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:35:28.914 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:35:28.914 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:35:28.914 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:35:28.914 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:35:28.914 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:35:28.914 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:35:28.914 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:35:28.915 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:35:28.915 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:35:28.915 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:28.915 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:35:28.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.915 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:35:28.915 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.915 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:35:28.915 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:35:28.915 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:28.915 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:28.915 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.916 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:28.916 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:35:28.916 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:28.916 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.916 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.916 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:28.916 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:35:28.916 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:28.916 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:28.916 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:28.916 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.917 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:35:28.917 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:35:28.917 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:28.917 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:28.917 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:28.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.917 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.917 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:35:28.917 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:35:28.917 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:28.917 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:28.917 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.917 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:28.917 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:35:28.917 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:35:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:35:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:35:28.918 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:35:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:35:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:35:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:35:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:35:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:35:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.919 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:35:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:35:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:35:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:35:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:35:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:35:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:35:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:35:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:35:28.920 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:35:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:35:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:35:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:35:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:35:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:35:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:35:28.921 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:35:28.922 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:35:28.922 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:35:28.922 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:35:28.922 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:35:28.922 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:35:28.922 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:35:28.922 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:35:28.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.932+0000 7fb3ec4df640 1 -- 172.21.15.112:0/2722126641 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb3c806bde0 msgr2=0x7fb3c806e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:28.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.932+0000 7fb3ec4df640 1 --2- 172.21.15.112:0/2722126641 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb3c806bde0 0x7fb3c806e2a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc034000 tx=0x7fb3dc02f000 comp rx=0 tx=0).stop 2024-04-25T01:35:28.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.932+0000 7fb3ec4df640 1 -- 172.21.15.112:0/2722126641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e40c9af0 msgr2=0x7fb3e40a8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:28.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.932+0000 7fb3ec4df640 1 --2- 172.21.15.112:0/2722126641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e40c9af0 0x7fb3e40a8ca0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000c530 tx=0x7fb3e000c9f0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.932+0000 7fb3eaa7c640 1 -- 172.21.15.112:0/2722126641 reap_dead start 2024-04-25T01:35:28.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.932+0000 7fb3eaa7c640 1 -- 172.21.15.112:0/2722126641 reap_dead start 2024-04-25T01:35:28.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.932+0000 7fb3ec4df640 1 -- 172.21.15.112:0/2722126641 shutdown_connections 2024-04-25T01:35:28.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.932+0000 7fb3ec4df640 1 -- 172.21.15.112:0/2722126641 >> 172.21.15.112:0/2722126641 conn(0x7fb3e4074a00 msgr2=0x7fb3e40c70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:28.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.932+0000 7fb3ec4df640 1 -- 172.21.15.112:0/2722126641 shutdown_connections 2024-04-25T01:35:28.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:28.932+0000 7fb3ec4df640 1 -- 172.21.15.112:0/2722126641 wait complete. 2024-04-25T01:35:28.926 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:35:28.926 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T01:35:34.765 INFO:tasks.workunit.client.0.smithi112.stdout:fs setup took 444 secs 2024-04-25T01:35:37.314 INFO:tasks.workunit.client.0.smithi112.stdout:Syncing()...2 sec 2024-04-25T01:35:37.314 INFO:tasks.workunit.client.0.smithi112.stdout:Starting Actual Benchmark At: Thu Apr 25 01:35:37 2024 2024-04-25T01:35:37.314 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T01:35:48.927 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-04-25T01:35:48.927 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-04-25T01:35:48.927 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T01:35:48.927 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T01:35:48.930 INFO:tasks.ceph.mon.a:Started 2024-04-25T01:35:48.931 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:35:48.931 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:35:49.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.112+0000 7fd49b121640 1 Processor -- start 2024-04-25T01:35:49.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.112+0000 7fd49b121640 1 -- start start 2024-04-25T01:35:49.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.114+0000 7fd49b121640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd494152fd0 0x7fd4940c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:49.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.114+0000 7fd49b121640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4940c90e0 con 0x7fd4940c5d70 2024-04-25T01:35:49.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.114+0000 7fd493fff640 1 -- >> v1:172.21.15.112:6789/0 conn(0x7fd4940c5d70 legacy=0x7fd494156280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:35:49.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.114+0000 7fd49b121640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4940cac00 con 0x7fd494156cf0 2024-04-25T01:35:49.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.114+0000 7fd49b121640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd4940ca9b0 con 0x7fd494152fd0 2024-04-25T01:35:49.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.114+0000 7fd498ebd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd494152fd0 0x7fd4940c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:49.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.114+0000 7fd498ebd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd494152fd0 0x7fd4940c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:47810/0 (socket says 172.21.15.112:47810) 2024-04-25T01:35:49.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.114+0000 7fd498ebd640 1 -- 172.21.15.112:0/4259023303 learned_addr learned my addr 172.21.15.112:0/4259023303 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:49.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.114+0000 7fd498ebd640 1 -- 172.21.15.112:0/4259023303 >> v1:172.21.15.187:6789/0 conn(0x7fd494156cf0 legacy=0x7fd49406bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:49.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.114+0000 7fd498ebd640 1 -- 172.21.15.112:0/4259023303 >> v1:172.21.15.112:6789/0 conn(0x7fd4940c5d70 legacy=0x7fd494156280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:35:49.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.114+0000 7fd498ebd640 1 -- 172.21.15.112:0/4259023303 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4940c9380 con 0x7fd494152fd0 2024-04-25T01:35:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.117+0000 7fd498ebd640 1 --2- 172.21.15.112:0/4259023303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd494152fd0 0x7fd4940c9b80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd4880098a0 tx=0x7fd48802a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61b0aa2624244d6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.117+0000 7fd4937fe640 1 -- 172.21.15.112:0/4259023303 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488037070 con 0x7fd494152fd0 2024-04-25T01:35:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.117+0000 7fd4937fe640 1 -- 172.21.15.112:0/4259023303 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd488032440 con 0x7fd494152fd0 2024-04-25T01:35:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.117+0000 7fd4937fe640 1 -- 172.21.15.112:0/4259023303 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488032810 con 0x7fd494152fd0 2024-04-25T01:35:49.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.118+0000 7fd49b121640 1 -- 172.21.15.112:0/4259023303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd494152fd0 msgr2=0x7fd4940c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:49.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.118+0000 7fd49b121640 1 --2- 172.21.15.112:0/4259023303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd494152fd0 0x7fd4940c9b80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd4880098a0 tx=0x7fd48802a200 comp rx=0 tx=0).stop 2024-04-25T01:35:49.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.119+0000 7fd49b121640 1 -- 172.21.15.112:0/4259023303 shutdown_connections 2024-04-25T01:35:49.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.119+0000 7fd49b121640 1 --2- 172.21.15.112:0/4259023303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd494152fd0 0x7fd4940c9b80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:49.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.119+0000 7fd49b121640 1 -- 172.21.15.112:0/4259023303 >> 172.21.15.112:0/4259023303 conn(0x7fd4940acc70 msgr2=0x7fd4940ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:49.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.119+0000 7fd49b121640 1 -- 172.21.15.112:0/4259023303 shutdown_connections 2024-04-25T01:35:49.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.119+0000 7fd49b121640 1 -- 172.21.15.112:0/4259023303 wait complete. 2024-04-25T01:35:49.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.121+0000 7fd49b121640 1 Processor -- start 2024-04-25T01:35:49.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.121+0000 7fd49b121640 1 -- start start 2024-04-25T01:35:49.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.123+0000 7fd49b121640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4940c5d70 0x7fd4940af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:49.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.123+0000 7fd49b121640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd494152fd0 0x7fd494151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:49.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.123+0000 7fd493fff640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd494152fd0 msgr2=0x7fd494151450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:35:49.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.123+0000 7fd493fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd494152fd0 0x7fd494151450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:35:49.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.123+0000 7fd49b121640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd494156cf0 0x7fd49414c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:49.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.123+0000 7fd49b121640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd49406c4b0 con 0x7fd494152fd0 2024-04-25T01:35:49.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.123+0000 7fd49b121640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd4940caee0 con 0x7fd494156cf0 2024-04-25T01:35:49.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.123+0000 7fd49b121640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd49406c630 con 0x7fd4940c5d70 2024-04-25T01:35:49.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.124+0000 7fd498ebd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4940c5d70 0x7fd4940af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:49.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.124+0000 7fd498ebd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4940c5d70 0x7fd4940af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:47812/0 (socket says 172.21.15.112:47812) 2024-04-25T01:35:49.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.124+0000 7fd498ebd640 1 -- 172.21.15.112:0/1181136610 learned_addr learned my addr 172.21.15.112:0/1181136610 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:49.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.124+0000 7fd4996be640 1 --2- 172.21.15.112:0/1181136610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd494156cf0 0x7fd49414c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:49.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.124+0000 7fd498ebd640 1 -- 172.21.15.112:0/1181136610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd494156cf0 msgr2=0x7fd49414c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:49.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.124+0000 7fd498ebd640 1 --2- 172.21.15.112:0/1181136610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd494156cf0 0x7fd49414c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:49.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.124+0000 7fd498ebd640 1 -- 172.21.15.112:0/1181136610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd494152fd0 msgr2=0x7fd494151450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:35:49.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.124+0000 7fd498ebd640 1 --2- 172.21.15.112:0/1181136610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd494152fd0 0x7fd494151450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:49.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.124+0000 7fd498ebd640 1 -- 172.21.15.112:0/1181136610 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4940b0780 con 0x7fd4940c5d70 2024-04-25T01:35:49.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.124+0000 7fd4996be640 1 --2- 172.21.15.112:0/1181136610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd494156cf0 0x7fd49414c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:35:49.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.125+0000 7fd498ebd640 1 --2- 172.21.15.112:0/1181136610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4940c5d70 0x7fd4940af730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd4880098a0 tx=0x7fd4880046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:49.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.125+0000 7fd4917fa640 1 -- 172.21.15.112:0/1181136610 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488041020 con 0x7fd4940c5d70 2024-04-25T01:35:49.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.125+0000 7fd4917fa640 1 -- 172.21.15.112:0/1181136610 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd488003610 con 0x7fd4940c5d70 2024-04-25T01:35:49.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.125+0000 7fd4917fa640 1 -- 172.21.15.112:0/1181136610 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488037040 con 0x7fd4940c5d70 2024-04-25T01:35:49.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.125+0000 7fd49b121640 1 -- 172.21.15.112:0/1181136610 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4940b0a60 con 0x7fd4940c5d70 2024-04-25T01:35:49.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.126+0000 7fd49b121640 1 -- 172.21.15.112:0/1181136610 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4940afc70 con 0x7fd4940c5d70 2024-04-25T01:35:49.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.127+0000 7fd49b121640 1 -- 172.21.15.112:0/1181136610 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4941522e0 con 0x7fd4940c5d70 2024-04-25T01:35:49.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.128+0000 7fd4917fa640 1 -- 172.21.15.112:0/1181136610 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd488044020 con 0x7fd4940c5d70 2024-04-25T01:35:49.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.129+0000 7fd4917fa640 1 --2- 172.21.15.112:0/1181136610 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd47806bde0 0x7fd47806e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:49.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.130+0000 7fd493fff640 1 --2- 172.21.15.112:0/1181136610 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd47806bde0 0x7fd47806e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:49.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.130+0000 7fd493fff640 1 --2- 172.21.15.112:0/1181136610 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd47806bde0 0x7fd47806e2a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd480006fd0 tx=0x7fd480006d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:49.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.131+0000 7fd4917fa640 1 -- 172.21.15.112:0/1181136610 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fd4880abb40 con 0x7fd4940c5d70 2024-04-25T01:35:49.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.137+0000 7fd4917fa640 1 -- 172.21.15.112:0/1181136610 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd4941522e0 con 0x7fd4940c5d70 2024-04-25T01:35:49.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.614+0000 7fd49b121640 1 -- 172.21.15.112:0/1181136610 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd4940682a0 con 0x7fd4940c5d70 2024-04-25T01:35:49.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.616+0000 7fd4917fa640 1 -- 172.21.15.112:0/1181136610 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1563 (secure 0 0 0) 0x7fd48804b030 con 0x7fd4940c5d70 2024-04-25T01:35:49.592 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":78,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:35:49.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.620+0000 7fd49b121640 1 -- 172.21.15.112:0/1181136610 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd47806bde0 msgr2=0x7fd47806e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:49.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.620+0000 7fd49b121640 1 --2- 172.21.15.112:0/1181136610 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd47806bde0 0x7fd47806e2a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd480006fd0 tx=0x7fd480006d80 comp rx=0 tx=0).stop 2024-04-25T01:35:49.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.620+0000 7fd49b121640 1 -- 172.21.15.112:0/1181136610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4940c5d70 msgr2=0x7fd4940af730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:49.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.620+0000 7fd49b121640 1 --2- 172.21.15.112:0/1181136610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4940c5d70 0x7fd4940af730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd4880098a0 tx=0x7fd4880046c0 comp rx=0 tx=0).stop 2024-04-25T01:35:49.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.621+0000 7fd49b121640 1 -- 172.21.15.112:0/1181136610 shutdown_connections 2024-04-25T01:35:49.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.621+0000 7fd49b121640 1 --2- 172.21.15.112:0/1181136610 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd47806bde0 0x7fd47806e2a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:49.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.621+0000 7fd49b121640 1 --2- 172.21.15.112:0/1181136610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd494156cf0 0x7fd49414c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:49.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.621+0000 7fd49b121640 1 --2- 172.21.15.112:0/1181136610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd494152fd0 0x7fd494151450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:49.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.621+0000 7fd49b121640 1 --2- 172.21.15.112:0/1181136610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4940c5d70 0x7fd4940af730 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:49.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.621+0000 7fd49b121640 1 -- 172.21.15.112:0/1181136610 >> 172.21.15.112:0/1181136610 conn(0x7fd4940acc70 msgr2=0x7fd4940c6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:49.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.623+0000 7fd49b121640 1 -- 172.21.15.112:0/1181136610 shutdown_connections 2024-04-25T01:35:49.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:49.623+0000 7fd49b121640 1 -- 172.21.15.112:0/1181136610 wait complete. 2024-04-25T01:35:52.620 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:35:52.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.802+0000 7efccd672640 1 Processor -- start 2024-04-25T01:35:52.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.803+0000 7efccd672640 1 -- start start 2024-04-25T01:35:52.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.804+0000 7efccd672640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efcc00d3b40 0x7efcc0046b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:52.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.804+0000 7efccd672640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcc004b110 0x7efcc0006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:52.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.804+0000 7efccd672640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcc0047050 con 0x7efcc000bd50 2024-04-25T01:35:52.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.804+0000 7efccd672640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7efcc00d9c60 con 0x7efcc00d3b40 2024-04-25T01:35:52.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.804+0000 7efccd672640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efcc00d9f40 con 0x7efcc004b110 2024-04-25T01:35:52.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.804+0000 7efcc7fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efcc00d3b40 0x7efcc0046b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:52.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.804+0000 7efcc7fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efcc00d3b40 0x7efcc0046b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:39124/0 (socket says 172.21.15.112:39124) 2024-04-25T01:35:52.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.804+0000 7efcc7fff640 1 -- 172.21.15.112:0/3185090576 learned_addr learned my addr 172.21.15.112:0/3185090576 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:52.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.804+0000 7efcc77fe640 1 --2- 172.21.15.112:0/3185090576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcc004b110 0x7efcc0006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:52.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.805+0000 7efcc77fe640 1 -- 172.21.15.112:0/3185090576 >> v2:172.21.15.187:3300/0 conn(0x7efcc00d3b40 msgr2=0x7efcc0046b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:35:52.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.805+0000 7efcc77fe640 1 --2- 172.21.15.112:0/3185090576 >> v2:172.21.15.187:3300/0 conn(0x7efcc00d3b40 0x7efcc0046b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:52.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.805+0000 7efcc77fe640 1 -- 172.21.15.112:0/3185090576 >> v1:172.21.15.112:6789/0 conn(0x7efcc000bd50 legacy=0x7efcc0009fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:52.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.805+0000 7efcc77fe640 1 -- 172.21.15.112:0/3185090576 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc00da220 con 0x7efcc004b110 2024-04-25T01:35:52.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.807+0000 7efcc77fe640 1 --2- 172.21.15.112:0/3185090576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcc004b110 0x7efcc0006ea0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efcb40097a0 tx=0x7efcb402a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f4be7b86384c696 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:52.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.808+0000 7efcc6ffd640 1 -- 172.21.15.112:0/3185090576 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4037070 con 0x7efcc004b110 2024-04-25T01:35:52.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.808+0000 7efcc6ffd640 1 -- 172.21.15.112:0/3185090576 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb402ae40 con 0x7efcc004b110 2024-04-25T01:35:52.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.808+0000 7efcc6ffd640 1 -- 172.21.15.112:0/3185090576 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4032660 con 0x7efcc004b110 2024-04-25T01:35:52.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.809+0000 7efccd672640 1 -- 172.21.15.112:0/3185090576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcc004b110 msgr2=0x7efcc0006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:52.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.809+0000 7efccd672640 1 --2- 172.21.15.112:0/3185090576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcc004b110 0x7efcc0006ea0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efcb40097a0 tx=0x7efcb402a0f0 comp rx=0 tx=0).stop 2024-04-25T01:35:52.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.809+0000 7efccd672640 1 -- 172.21.15.112:0/3185090576 shutdown_connections 2024-04-25T01:35:52.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.809+0000 7efccd672640 1 --2- 172.21.15.112:0/3185090576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcc004b110 0x7efcc0006ea0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:52.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.810+0000 7efccd672640 1 --2- 172.21.15.112:0/3185090576 >> v2:172.21.15.187:3300/0 conn(0x7efcc00d3b40 0x7efcc0046b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:52.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.810+0000 7efccd672640 1 -- 172.21.15.112:0/3185090576 >> 172.21.15.112:0/3185090576 conn(0x7efcc0048250 msgr2=0x7efcc0048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:52.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.810+0000 7efccd672640 1 -- 172.21.15.112:0/3185090576 shutdown_connections 2024-04-25T01:35:52.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.811+0000 7efccd672640 1 -- 172.21.15.112:0/3185090576 wait complete. 2024-04-25T01:35:52.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.813+0000 7efccd672640 1 Processor -- start 2024-04-25T01:35:52.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.813+0000 7efccd672640 1 -- start start 2024-04-25T01:35:52.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.815+0000 7efccd672640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcc000bd50 0x7efcc00c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:52.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.815+0000 7efccd672640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcc004b110 0x7efcc00c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:52.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.815+0000 7efccd672640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcc00d3b40 0x7efcc002be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:52.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.815+0000 7efccd672640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efcc00d9930 con 0x7efcc004b110 2024-04-25T01:35:52.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.815+0000 7efccd672640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efcc00da7e0 con 0x7efcc000bd50 2024-04-25T01:35:52.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.815+0000 7efccd672640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efcc00d3f20 con 0x7efcc00d3b40 2024-04-25T01:35:52.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.815+0000 7efcc7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcc000bd50 0x7efcc00c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:52.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.815+0000 7efcc7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcc000bd50 0x7efcc00c8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:39126/0 (socket says 172.21.15.112:39126) 2024-04-25T01:35:52.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.815+0000 7efcc77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcc004b110 0x7efcc00c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:52.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.815+0000 7efcc7fff640 1 -- 172.21.15.112:0/1212821745 learned_addr learned my addr 172.21.15.112:0/1212821745 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:52.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.815+0000 7efccce71640 1 --2- 172.21.15.112:0/1212821745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcc00d3b40 0x7efcc002be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:52.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.815+0000 7efcc77fe640 1 -- 172.21.15.112:0/1212821745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcc004b110 msgr2=0x7efcc00c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T01:35:52.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.815+0000 7efcc77fe640 1 -- 172.21.15.112:0/1212821745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcc004b110 msgr2=0x7efcc00c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:35:52.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.815+0000 7efcc77fe640 1 --2- 172.21.15.112:0/1212821745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcc004b110 0x7efcc00c87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:35:52.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.815+0000 7efcc77fe640 1 --2- 172.21.15.112:0/1212821745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcc004b110 0x7efcc00c87d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:35:52.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.816+0000 7efcc7fff640 1 -- 172.21.15.112:0/1212821745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcc00d3b40 msgr2=0x7efcc002be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:52.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.816+0000 7efcc7fff640 1 --2- 172.21.15.112:0/1212821745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcc00d3b40 0x7efcc002be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:52.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.816+0000 7efcc7fff640 1 -- 172.21.15.112:0/1212821745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcc004b110 msgr2=0x7efcc00c87d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:35:52.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.816+0000 7efcc7fff640 1 --2- 172.21.15.112:0/1212821745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcc004b110 0x7efcc00c87d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:52.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.816+0000 7efcc7fff640 1 -- 172.21.15.112:0/1212821745 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc00421f0 con 0x7efcc000bd50 2024-04-25T01:35:52.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.816+0000 7efccce71640 1 --2- 172.21.15.112:0/1212821745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcc00d3b40 0x7efcc002be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:35:52.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.816+0000 7efcc7fff640 1 --2- 172.21.15.112:0/1212821745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcc000bd50 0x7efcc00c8290 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efcbc009fd0 tx=0x7efcbc00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:52.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.817+0000 7efcc4ff9640 1 -- 172.21.15.112:0/1212821745 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc017070 con 0x7efcc000bd50 2024-04-25T01:35:52.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.817+0000 7efcc4ff9640 1 -- 172.21.15.112:0/1212821745 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcbc00c940 con 0x7efcc000bd50 2024-04-25T01:35:52.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.817+0000 7efcc4ff9640 1 -- 172.21.15.112:0/1212821745 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc00cd10 con 0x7efcc000bd50 2024-04-25T01:35:52.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.817+0000 7efccd672640 1 -- 172.21.15.112:0/1212821745 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc002b740 con 0x7efcc000bd50 2024-04-25T01:35:52.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.817+0000 7efccd672640 1 -- 172.21.15.112:0/1212821745 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc00ca7e0 con 0x7efcc000bd50 2024-04-25T01:35:52.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.819+0000 7efccd672640 1 -- 172.21.15.112:0/1212821745 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc0005390 con 0x7efcc000bd50 2024-04-25T01:35:52.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.820+0000 7efcc4ff9640 1 -- 172.21.15.112:0/1212821745 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efcbc01a070 con 0x7efcc000bd50 2024-04-25T01:35:52.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.820+0000 7efcc4ff9640 1 --2- 172.21.15.112:0/1212821745 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efcac06be60 0x7efcac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:52.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.821+0000 7efcc4ff9640 1 -- 172.21.15.112:0/1212821745 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7efcbc08b430 con 0x7efcc000bd50 2024-04-25T01:35:52.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.821+0000 7efcc77fe640 1 --2- 172.21.15.112:0/1212821745 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efcac06be60 0x7efcac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:52.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.821+0000 7efcc77fe640 1 --2- 172.21.15.112:0/1212821745 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efcac06be60 0x7efcac06e320 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efcb4034000 tx=0x7efcb402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:52.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:52.828+0000 7efcc4ff9640 1 -- 172.21.15.112:0/1212821745 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efcbc0593e0 con 0x7efcc000bd50 2024-04-25T01:35:53.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:53.301+0000 7efccd672640 1 -- 172.21.15.112:0/1212821745 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7efcc0048030 con 0x7efcc000bd50 2024-04-25T01:35:53.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:53.302+0000 7efcc4ff9640 1 -- 172.21.15.112:0/1212821745 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1563 (secure 0 0 0) 0x7efcbc05d6e0 con 0x7efcc000bd50 2024-04-25T01:35:53.279 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":78,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:35:53.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:53.305+0000 7efccd672640 1 -- 172.21.15.112:0/1212821745 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efcac06be60 msgr2=0x7efcac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:53.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:53.305+0000 7efccd672640 1 --2- 172.21.15.112:0/1212821745 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efcac06be60 0x7efcac06e320 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efcb4034000 tx=0x7efcb402e000 comp rx=0 tx=0).stop 2024-04-25T01:35:53.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:53.305+0000 7efccd672640 1 -- 172.21.15.112:0/1212821745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcc000bd50 msgr2=0x7efcc00c8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:53.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:53.305+0000 7efccd672640 1 --2- 172.21.15.112:0/1212821745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcc000bd50 0x7efcc00c8290 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efcbc009fd0 tx=0x7efcbc00ed40 comp rx=0 tx=0).stop 2024-04-25T01:35:53.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:53.306+0000 7efccd672640 1 -- 172.21.15.112:0/1212821745 shutdown_connections 2024-04-25T01:35:53.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:53.306+0000 7efccd672640 1 --2- 172.21.15.112:0/1212821745 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7efcac06be60 0x7efcac06e320 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:53.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:53.306+0000 7efccd672640 1 --2- 172.21.15.112:0/1212821745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efcc00d3b40 0x7efcc002be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:53.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:53.306+0000 7efccd672640 1 --2- 172.21.15.112:0/1212821745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcc004b110 0x7efcc00c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:53.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:53.306+0000 7efccd672640 1 --2- 172.21.15.112:0/1212821745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efcc000bd50 0x7efcc00c8290 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:53.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:53.306+0000 7efccd672640 1 -- 172.21.15.112:0/1212821745 >> 172.21.15.112:0/1212821745 conn(0x7efcc0048250 msgr2=0x7efcc0048630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:53.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:53.306+0000 7efccd672640 1 -- 172.21.15.112:0/1212821745 shutdown_connections 2024-04-25T01:35:53.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:53.306+0000 7efccd672640 1 -- 172.21.15.112:0/1212821745 wait complete. 2024-04-25T01:35:56.299 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:35:56.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.479+0000 7f2fa0b61640 1 Processor -- start 2024-04-25T01:35:56.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.479+0000 7f2fa0b61640 1 -- start start 2024-04-25T01:35:56.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.480+0000 7f2fa0b61640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f9c0c9b80 0x7f2f9c06bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:56.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.480+0000 7f2fa0b61640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f9c0c9fd0 con 0x7f2f9c152fd0 2024-04-25T01:35:56.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.480+0000 7f2fa0b61640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f9c0c90e0 con 0x7f2f9c0c5d70 2024-04-25T01:35:56.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.480+0000 7f2fa0b61640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2f9c157400 con 0x7f2f9c0c9b80 2024-04-25T01:35:56.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.480+0000 7f2f9a59c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2f9c152fd0 0x7f2f9c156280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54734/0 (socket says 172.21.15.112:54734) 2024-04-25T01:35:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.480+0000 7f2f9a59c640 1 -- 172.21.15.112:0/2730353945 learned_addr learned my addr 172.21.15.112:0/2730353945 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:35:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.480+0000 7f2f9ad9d640 1 --2- 172.21.15.112:0/2730353945 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f9c0c9b80 0x7f2f9c06bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.482+0000 7f2f9ad9d640 1 -- 172.21.15.112:0/2730353945 >> v1:172.21.15.187:6789/0 conn(0x7f2f9c0c5d70 legacy=0x7f2f9c156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.482+0000 7f2f9ad9d640 1 -- 172.21.15.112:0/2730353945 >> v1:172.21.15.112:6789/0 conn(0x7f2f9c152fd0 legacy=0x7f2f9c156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.482+0000 7f2f9ad9d640 1 -- 172.21.15.112:0/2730353945 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9c0cac00 con 0x7f2f9c0c9b80 2024-04-25T01:35:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.484+0000 7f2f9ad9d640 1 --2- 172.21.15.112:0/2730353945 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f9c0c9b80 0x7f2f9c06bda0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2f94009dc0 tx=0x7f2f9402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2d0cd6cc97d913e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.484+0000 7f2f9959a640 1 -- 172.21.15.112:0/2730353945 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f94007740 con 0x7f2f9c0c9b80 2024-04-25T01:35:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.484+0000 7f2f9959a640 1 -- 172.21.15.112:0/2730353945 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f940078e0 con 0x7f2f9c0c9b80 2024-04-25T01:35:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.484+0000 7f2f9959a640 1 -- 172.21.15.112:0/2730353945 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f94007cf0 con 0x7f2f9c0c9b80 2024-04-25T01:35:56.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.485+0000 7f2fa0b61640 1 -- 172.21.15.112:0/2730353945 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f9c0c9b80 msgr2=0x7f2f9c06bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:56.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.485+0000 7f2fa0b61640 1 --2- 172.21.15.112:0/2730353945 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f9c0c9b80 0x7f2f9c06bda0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2f94009dc0 tx=0x7f2f9402a680 comp rx=0 tx=0).stop 2024-04-25T01:35:56.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.485+0000 7f2fa0b61640 1 -- 172.21.15.112:0/2730353945 shutdown_connections 2024-04-25T01:35:56.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.485+0000 7f2fa0b61640 1 --2- 172.21.15.112:0/2730353945 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f9c0c9b80 0x7f2f9c06bda0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:56.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.486+0000 7f2fa0b61640 1 -- 172.21.15.112:0/2730353945 >> 172.21.15.112:0/2730353945 conn(0x7f2f9c0acc70 msgr2=0x7f2f9c0ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:56.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.486+0000 7f2fa0b61640 1 -- 172.21.15.112:0/2730353945 shutdown_connections 2024-04-25T01:35:56.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.486+0000 7f2fa0b61640 1 -- 172.21.15.112:0/2730353945 wait complete. 2024-04-25T01:35:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.488+0000 7f2fa0b61640 1 Processor -- start 2024-04-25T01:35:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.488+0000 7f2fa0b61640 1 -- start start 2024-04-25T01:35:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.489+0000 7f2fa0b61640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f9c0c5d70 0x7f2f9c1560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:56.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.490+0000 7f2fa0b61640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f9c0c9b80 0x7f2f9c14fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:56.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.490+0000 7f2fa0b61640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f9c152fd0 0x7f2f9c148320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.490+0000 7f2fa0b61640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c0c9660 con 0x7f2f9c0c9b80 2024-04-25T01:35:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.490+0000 7f2fa0b61640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c0ca9b0 con 0x7f2f9c152fd0 2024-04-25T01:35:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.490+0000 7f2fa0b61640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2f9c0caee0 con 0x7f2f9c0c5d70 2024-04-25T01:35:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.490+0000 7f2f99d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f9c0c9b80 0x7f2f9c14fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.490+0000 7f2f99d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f9c0c9b80 0x7f2f9c14fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59732/0 (socket says 172.21.15.112:59732) 2024-04-25T01:35:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.490+0000 7f2f99d9b640 1 -- 172.21.15.112:0/2800316265 learned_addr learned my addr 172.21.15.112:0/2800316265 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:35:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.490+0000 7f2f9a59c640 1 --2- 172.21.15.112:0/2800316265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f9c0c5d70 0x7f2f9c1560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.490+0000 7f2f9ad9d640 1 --2- 172.21.15.112:0/2800316265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f9c152fd0 0x7f2f9c148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.490+0000 7f2f99d9b640 1 -- 172.21.15.112:0/2800316265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f9c0c9b80 msgr2=0x7f2f9c14fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T01:35:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.490+0000 7f2f99d9b640 1 -- 172.21.15.112:0/2800316265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f9c0c9b80 msgr2=0x7f2f9c14fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:35:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.490+0000 7f2f99d9b640 1 --2- 172.21.15.112:0/2800316265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f9c0c9b80 0x7f2f9c14fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:35:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.490+0000 7f2f99d9b640 1 --2- 172.21.15.112:0/2800316265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f9c0c9b80 0x7f2f9c14fb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:35:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.490+0000 7f2f9a59c640 1 -- 172.21.15.112:0/2800316265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f9c152fd0 msgr2=0x7f2f9c148320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.490+0000 7f2f9a59c640 1 --2- 172.21.15.112:0/2800316265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f9c152fd0 0x7f2f9c148320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.490+0000 7f2f9a59c640 1 -- 172.21.15.112:0/2800316265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f9c0c9b80 msgr2=0x7f2f9c14fb00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:35:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.490+0000 7f2f9a59c640 1 --2- 172.21.15.112:0/2800316265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f9c0c9b80 0x7f2f9c14fb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.490+0000 7f2f9a59c640 1 -- 172.21.15.112:0/2800316265 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9c0c2210 con 0x7f2f9c0c5d70 2024-04-25T01:35:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.491+0000 7f2f9a59c640 1 --2- 172.21.15.112:0/2800316265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f9c0c5d70 0x7f2f9c1560c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000c470 tx=0x7f2f9000c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.492+0000 7f2f8affd640 1 -- 172.21.15.112:0/2800316265 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90017070 con 0x7f2f9c0c5d70 2024-04-25T01:35:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.492+0000 7f2f8affd640 1 -- 172.21.15.112:0/2800316265 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f90005ce0 con 0x7f2f9c0c5d70 2024-04-25T01:35:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.492+0000 7f2f8affd640 1 -- 172.21.15.112:0/2800316265 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f900072d0 con 0x7f2f9c0c5d70 2024-04-25T01:35:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.492+0000 7f2fa0b61640 1 -- 172.21.15.112:0/2800316265 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f9c0c24f0 con 0x7f2f9c0c5d70 2024-04-25T01:35:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.492+0000 7f2fa0b61640 1 -- 172.21.15.112:0/2800316265 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f9c0c2ab0 con 0x7f2f9c0c5d70 2024-04-25T01:35:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.494+0000 7f2fa0b61640 1 -- 172.21.15.112:0/2800316265 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f9c157e10 con 0x7f2f9c0c5d70 2024-04-25T01:35:56.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.495+0000 7f2f8affd640 1 -- 172.21.15.112:0/2800316265 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f9001a070 con 0x7f2f9c0c5d70 2024-04-25T01:35:56.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.495+0000 7f2f8affd640 1 --2- 172.21.15.112:0/2800316265 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2f7c06bf30 0x7f2f7c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:56.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.496+0000 7f2f8affd640 1 -- 172.21.15.112:0/2800316265 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f2f9008a370 con 0x7f2f9c0c5d70 2024-04-25T01:35:56.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.496+0000 7f2f99d9b640 1 --2- 172.21.15.112:0/2800316265 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2f7c06bf30 0x7f2f7c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:56.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.496+0000 7f2f99d9b640 1 --2- 172.21.15.112:0/2800316265 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2f7c06bf30 0x7f2f7c06e3f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0c4300 tx=0x7f2f8c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:56.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.504+0000 7f2f8affd640 1 -- 172.21.15.112:0/2800316265 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2f9c157e10 con 0x7f2f9c0c5d70 2024-04-25T01:35:56.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.989+0000 7f2fa0b61640 1 -- 172.21.15.112:0/2800316265 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f2f9c0c5ac0 con 0x7f2f9c0c5d70 2024-04-25T01:35:56.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.990+0000 7f2f8affd640 1 -- 172.21.15.112:0/2800316265 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1563 (secure 0 0 0) 0x7f2f900583a0 con 0x7f2f9c0c5d70 2024-04-25T01:35:56.966 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":78,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":29,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:35:56.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.993+0000 7f2fa0b61640 1 -- 172.21.15.112:0/2800316265 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2f7c06bf30 msgr2=0x7f2f7c06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:56.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.994+0000 7f2fa0b61640 1 --2- 172.21.15.112:0/2800316265 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2f7c06bf30 0x7f2f7c06e3f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0c4300 tx=0x7f2f8c009040 comp rx=0 tx=0).stop 2024-04-25T01:35:56.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.994+0000 7f2fa0b61640 1 -- 172.21.15.112:0/2800316265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f9c0c5d70 msgr2=0x7f2f9c1560c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:56.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.994+0000 7f2fa0b61640 1 --2- 172.21.15.112:0/2800316265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f9c0c5d70 0x7f2f9c1560c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000c470 tx=0x7f2f9000c930 comp rx=0 tx=0).stop 2024-04-25T01:35:56.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.994+0000 7f2fa0b61640 1 -- 172.21.15.112:0/2800316265 shutdown_connections 2024-04-25T01:35:56.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.994+0000 7f2fa0b61640 1 --2- 172.21.15.112:0/2800316265 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2f7c06bf30 0x7f2f7c06e3f0 secure :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0c4300 tx=0x7f2f8c009040 comp rx=0 tx=0).stop 2024-04-25T01:35:56.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.994+0000 7f2fa0b61640 1 --2- 172.21.15.112:0/2800316265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f9c152fd0 0x7f2f9c148320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:56.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.994+0000 7f2fa0b61640 1 --2- 172.21.15.112:0/2800316265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f9c0c9b80 0x7f2f9c14fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:56.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.994+0000 7f2fa0b61640 1 --2- 172.21.15.112:0/2800316265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f9c0c5d70 0x7f2f9c1560c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:56.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.994+0000 7f2fa0b61640 1 -- 172.21.15.112:0/2800316265 >> 172.21.15.112:0/2800316265 conn(0x7f2f9c0acc70 msgr2=0x7f2f9c0ad050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:56.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.995+0000 7f2fa0b61640 1 -- 172.21.15.112:0/2800316265 shutdown_connections 2024-04-25T01:35:56.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:35:56.996+0000 7f2fa0b61640 1 -- 172.21.15.112:0/2800316265 wait complete. 2024-04-25T01:35:57.462 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:35:57.465 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:35:59.993 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:36:00.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.182+0000 7f3775f96640 1 Processor -- start 2024-04-25T01:36:00.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.182+0000 7f3775f96640 1 -- start start 2024-04-25T01:36:00.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.183+0000 7f3775f96640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37700b06f0 0x7f37700c9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:00.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.183+0000 7f3775f96640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f37700c5b20 0x7f37700c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:00.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.183+0000 7f3775f96640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3770158f60 con 0x7f3770155970 2024-04-25T01:36:00.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.183+0000 7f3775f96640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3770159b50 con 0x7f37700c5b20 2024-04-25T01:36:00.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.183+0000 7f3775f96640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3770159220 con 0x7f37700b06f0 2024-04-25T01:36:00.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.183+0000 7f376ffff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3770155970 0x7f3770155e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:42172/0 (socket says 172.21.15.112:42172) 2024-04-25T01:36:00.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.183+0000 7f376f7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37700b06f0 0x7f37700c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:00.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.183+0000 7f376ffff640 1 -- 172.21.15.112:0/2160389601 learned_addr learned my addr 172.21.15.112:0/2160389601 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:36:00.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.183+0000 7f376effd640 1 --2- 172.21.15.112:0/2160389601 >> v2:172.21.15.187:3300/0 conn(0x7f37700c5b20 0x7f37700c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:00.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.184+0000 7f376f7fe640 1 -- 172.21.15.112:0/2160389601 >> v2:172.21.15.187:3300/0 conn(0x7f37700c5b20 msgr2=0x7f37700c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:00.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.184+0000 7f376f7fe640 1 --2- 172.21.15.112:0/2160389601 >> v2:172.21.15.187:3300/0 conn(0x7f37700c5b20 0x7f37700c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:00.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.184+0000 7f376f7fe640 1 -- 172.21.15.112:0/2160389601 >> v1:172.21.15.112:6789/0 conn(0x7f3770155970 legacy=0x7f3770155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:00.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.184+0000 7f376f7fe640 1 -- 172.21.15.112:0/2160389601 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3770159500 con 0x7f37700b06f0 2024-04-25T01:36:00.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.184+0000 7f376effd640 1 --2- 172.21.15.112:0/2160389601 >> v2:172.21.15.187:3300/0 conn(0x7f37700c5b20 0x7f37700c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:36:00.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.185+0000 7f376f7fe640 1 --2- 172.21.15.112:0/2160389601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37700b06f0 0x7f37700c9150 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f37600097a0 tx=0x7f376002a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=812d3f8c7e9b9429 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:00.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.186+0000 7f376e7fc640 1 -- 172.21.15.112:0/2160389601 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760037070 con 0x7f37700b06f0 2024-04-25T01:36:00.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.186+0000 7f376e7fc640 1 -- 172.21.15.112:0/2160389601 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f376002adb0 con 0x7f37700b06f0 2024-04-25T01:36:00.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.186+0000 7f376e7fc640 1 -- 172.21.15.112:0/2160389601 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760032660 con 0x7f37700b06f0 2024-04-25T01:36:00.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.187+0000 7f3775f96640 1 -- 172.21.15.112:0/2160389601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37700b06f0 msgr2=0x7f37700c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:00.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.187+0000 7f3775f96640 1 --2- 172.21.15.112:0/2160389601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37700b06f0 0x7f37700c9150 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f37600097a0 tx=0x7f376002a060 comp rx=0 tx=0).stop 2024-04-25T01:36:00.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.187+0000 7f3775f96640 1 -- 172.21.15.112:0/2160389601 shutdown_connections 2024-04-25T01:36:00.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.187+0000 7f3775f96640 1 --2- 172.21.15.112:0/2160389601 >> v2:172.21.15.187:3300/0 conn(0x7f37700c5b20 0x7f37700c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:00.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.187+0000 7f3775f96640 1 --2- 172.21.15.112:0/2160389601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37700b06f0 0x7f37700c9150 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:00.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.187+0000 7f3775f96640 1 -- 172.21.15.112:0/2160389601 >> 172.21.15.112:0/2160389601 conn(0x7f3770074d80 msgr2=0x7f37700cb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:00.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.187+0000 7f3775f96640 1 -- 172.21.15.112:0/2160389601 shutdown_connections 2024-04-25T01:36:00.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.187+0000 7f3775f96640 1 -- 172.21.15.112:0/2160389601 wait complete. 2024-04-25T01:36:00.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.188+0000 7f3775f96640 1 Processor -- start 2024-04-25T01:36:00.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.188+0000 7f3775f96640 1 -- start start 2024-04-25T01:36:00.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.189+0000 7f3775f96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37700b06f0 0x7f3770151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:00.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.190+0000 7f3775f96640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37700c5b20 0x7f37701ec670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:00.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.190+0000 7f3775f96640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3770155970 0x7f37701ecbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:00.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.190+0000 7f3775f96640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f37700b0ad0 con 0x7f37700b06f0 2024-04-25T01:36:00.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.190+0000 7f3775f96640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f377006db80 con 0x7f3770155970 2024-04-25T01:36:00.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.190+0000 7f3775f96640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3770156510 con 0x7f37700c5b20 2024-04-25T01:36:00.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.191+0000 7f376f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37700b06f0 0x7f3770151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:00.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.191+0000 7f376f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37700b06f0 0x7f3770151b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52700/0 (socket says 172.21.15.112:52700) 2024-04-25T01:36:00.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.191+0000 7f376f7fe640 1 -- 172.21.15.112:0/2713535245 learned_addr learned my addr 172.21.15.112:0/2713535245 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:36:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.191+0000 7f376effd640 1 --2- 172.21.15.112:0/2713535245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37700c5b20 0x7f37701ec670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.191+0000 7f376f7fe640 1 -- 172.21.15.112:0/2713535245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37700b06f0 msgr2=0x7f3770151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T01:36:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.191+0000 7f376f7fe640 1 -- 172.21.15.112:0/2713535245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37700b06f0 msgr2=0x7f3770151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:36:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.191+0000 7f376f7fe640 1 --2- 172.21.15.112:0/2713535245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37700b06f0 0x7f3770151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.191+0000 7f376f7fe640 1 --2- 172.21.15.112:0/2713535245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37700b06f0 0x7f3770151b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:36:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.191+0000 7f376ffff640 1 --2- 172.21.15.112:0/2713535245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3770155970 0x7f37701ecbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.192+0000 7f376ffff640 1 -- 172.21.15.112:0/2713535245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37700c5b20 msgr2=0x7f37701ec670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.192+0000 7f376ffff640 1 --2- 172.21.15.112:0/2713535245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37700c5b20 0x7f37701ec670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.192+0000 7f376ffff640 1 -- 172.21.15.112:0/2713535245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37700b06f0 msgr2=0x7f3770151b60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:36:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.192+0000 7f376ffff640 1 --2- 172.21.15.112:0/2713535245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37700b06f0 0x7f3770151b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.192+0000 7f376ffff640 1 -- 172.21.15.112:0/2713535245 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37701ed0f0 con 0x7f3770155970 2024-04-25T01:36:00.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.192+0000 7f376ffff640 1 --2- 172.21.15.112:0/2713535245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3770155970 0x7f37701ecbb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f376800ca70 tx=0x7f376800cf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:00.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.192+0000 7f3757fff640 1 -- 172.21.15.112:0/2713535245 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768017070 con 0x7f3770155970 2024-04-25T01:36:00.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.193+0000 7f3775f96640 1 -- 172.21.15.112:0/2713535245 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37701ec140 con 0x7f3770155970 2024-04-25T01:36:00.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.193+0000 7f3775f96640 1 -- 172.21.15.112:0/2713535245 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37701eb3a0 con 0x7f3770155970 2024-04-25T01:36:00.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.193+0000 7f3757fff640 1 -- 172.21.15.112:0/2713535245 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f376800aa90 con 0x7f3770155970 2024-04-25T01:36:00.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.193+0000 7f3757fff640 1 -- 172.21.15.112:0/2713535245 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768012400 con 0x7f3770155970 2024-04-25T01:36:00.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.194+0000 7f3775f96640 1 -- 172.21.15.112:0/2713535245 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3750005290 con 0x7f3770155970 2024-04-25T01:36:00.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.195+0000 7f3757fff640 1 -- 172.21.15.112:0/2713535245 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f376801a070 con 0x7f3770155970 2024-04-25T01:36:00.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.196+0000 7f3757fff640 1 --2- 172.21.15.112:0/2713535245 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f375c06be80 0x7f375c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:00.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.196+0000 7f376f7fe640 1 --2- 172.21.15.112:0/2713535245 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f375c06be80 0x7f375c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:00.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.196+0000 7f3757fff640 1 -- 172.21.15.112:0/2713535245 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f376808ad30 con 0x7f3770155970 2024-04-25T01:36:00.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.197+0000 7f376f7fe640 1 --2- 172.21.15.112:0/2713535245 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f375c06be80 0x7f375c06e340 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3760009560 tx=0x7f376002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:00.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.204+0000 7f3757fff640 1 -- 172.21.15.112:0/2713535245 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3768058d60 con 0x7f3770155970 2024-04-25T01:36:00.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.700+0000 7f3775f96640 1 -- 172.21.15.112:0/2713535245 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3750005580 con 0x7f3770155970 2024-04-25T01:36:00.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.700+0000 7f3757fff640 1 -- 172.21.15.112:0/2713535245 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1563 (secure 0 0 0) 0x7f376805d060 con 0x7f3770155970 2024-04-25T01:36:00.679 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":78,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":33,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:36:00.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.704+0000 7f3775f96640 1 -- 172.21.15.112:0/2713535245 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f375c06be80 msgr2=0x7f375c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:00.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.704+0000 7f3775f96640 1 --2- 172.21.15.112:0/2713535245 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f375c06be80 0x7f375c06e340 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3760009560 tx=0x7f376002e000 comp rx=0 tx=0).stop 2024-04-25T01:36:00.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.704+0000 7f3775f96640 1 -- 172.21.15.112:0/2713535245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3770155970 msgr2=0x7f37701ecbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:00.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.704+0000 7f3775f96640 1 --2- 172.21.15.112:0/2713535245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3770155970 0x7f37701ecbb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f376800ca70 tx=0x7f376800cf30 comp rx=0 tx=0).stop 2024-04-25T01:36:00.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.704+0000 7f3775f96640 1 -- 172.21.15.112:0/2713535245 shutdown_connections 2024-04-25T01:36:00.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.704+0000 7f3775f96640 1 --2- 172.21.15.112:0/2713535245 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f375c06be80 0x7f375c06e340 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:00.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.704+0000 7f3775f96640 1 --2- 172.21.15.112:0/2713535245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3770155970 0x7f37701ecbb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:00.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.704+0000 7f3775f96640 1 --2- 172.21.15.112:0/2713535245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37700c5b20 0x7f37701ec670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:00.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.704+0000 7f3775f96640 1 --2- 172.21.15.112:0/2713535245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37700b06f0 0x7f3770151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:00.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.704+0000 7f3775f96640 1 -- 172.21.15.112:0/2713535245 >> 172.21.15.112:0/2713535245 conn(0x7f3770074d80 msgr2=0x7f3770155e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:00.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.705+0000 7f3775f96640 1 -- 172.21.15.112:0/2713535245 shutdown_connections 2024-04-25T01:36:00.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:00.705+0000 7f3775f96640 1 -- 172.21.15.112:0/2713535245 wait complete. 2024-04-25T01:36:03.699 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:36:03.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:03.887+0000 7ff7bc514640 1 Processor -- start 2024-04-25T01:36:03.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:03.887+0000 7ff7bc514640 1 -- start start 2024-04-25T01:36:03.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:03.889+0000 7ff7bc514640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 0x7ff7b406be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:03.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:03.889+0000 7ff7bc514640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7b406c410 con 0x7ff7b40be1a0 2024-04-25T01:36:03.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:03.889+0000 7ff7bc514640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7b406c6f0 con 0x7ff7b40c7450 2024-04-25T01:36:03.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:03.889+0000 7ff7bc514640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7ff7b40c5cb0 con 0x7ff7b415d440 2024-04-25T01:36:03.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:03.889+0000 7ff7b9aaf640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 0x7ff7b406be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:03.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:03.889+0000 7ff7ba2b0640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7ff7b40c7450 0x7ff7b40c6140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:37108/0 (socket says 172.21.15.112:37108) 2024-04-25T01:36:03.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:03.889+0000 7ff7ba2b0640 1 -- 172.21.15.112:0/3520798308 learned_addr learned my addr 172.21.15.112:0/3520798308 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:36:03.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:03.889+0000 7ff7baab1640 1 --1- 172.21.15.112:0/3520798308 >> v1:172.21.15.112:6789/0 conn(0x7ff7b40be1a0 0x7ff7b415a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:42184/0 (socket says 172.21.15.112:42184) 2024-04-25T01:36:03.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:03.889+0000 7ff7b9aaf640 1 -- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 msgr2=0x7ff7b406be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:36:03.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:03.889+0000 7ff7b9aaf640 1 -- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 msgr2=0x7ff7b406be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:36:03.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:03.889+0000 7ff7b9aaf640 1 --2- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 0x7ff7b406be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:03.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:03.889+0000 7ff7b9aaf640 1 --2- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 0x7ff7b406be30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:36:04.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:04.090+0000 7ff7b9aaf640 1 --2- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 0x7ff7b406be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:04.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:04.090+0000 7ff7b9aaf640 1 -- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 msgr2=0x7ff7b406be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:36:04.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:04.090+0000 7ff7b9aaf640 1 -- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 msgr2=0x7ff7b406be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:36:04.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:04.090+0000 7ff7b9aaf640 1 --2- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 0x7ff7b406be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:04.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:04.090+0000 7ff7b9aaf640 1 --2- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 0x7ff7b406be30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:36:04.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:04.199+0000 7ff7baab1640 1 -- 172.21.15.112:0/3520798308 >> v1:172.21.15.112:6789/0 conn(0x7ff7b40be1a0 legacy=0x7ff7b415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-25T01:36:04.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:04.199+0000 7ff7baab1640 1 -- 172.21.15.112:0/3520798308 >> v1:172.21.15.112:6789/0 conn(0x7ff7b40be1a0 legacy=0x7ff7b415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:36:04.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:04.199+0000 7ff7baab1640 1 --1- 172.21.15.112:0/3520798308 >> v1:172.21.15.112:6789/0 conn(0x7ff7b40be1a0 0x7ff7b415a500 :-1 s=OPENED pgs=31 cs=1 l=1).handle_message read tag failed 2024-04-25T01:36:04.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:04.199+0000 7ff7baab1640 1 --1- 172.21.15.112:0/3520798308 >> v1:172.21.15.112:6789/0 conn(0x7ff7b40be1a0 0x7ff7b415a500 :-1 s=OPENED pgs=31 cs=1 l=1).fault on lossy channel, failing 2024-04-25T01:36:04.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:04.491+0000 7ff7b9aaf640 1 --2- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 0x7ff7b406be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:04.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:04.491+0000 7ff7b9aaf640 1 -- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 msgr2=0x7ff7b406be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:36:04.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:04.491+0000 7ff7b9aaf640 1 -- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 msgr2=0x7ff7b406be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:36:04.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:04.491+0000 7ff7b9aaf640 1 --2- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 0x7ff7b406be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:04.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:04.491+0000 7ff7b9aaf640 1 --2- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 0x7ff7b406be30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:36:05.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:05.292+0000 7ff7b9aaf640 1 --2- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 0x7ff7b406be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:05.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:05.292+0000 7ff7b9aaf640 1 -- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 msgr2=0x7ff7b406be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:36:05.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:05.292+0000 7ff7b9aaf640 1 -- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 msgr2=0x7ff7b406be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:36:05.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:05.292+0000 7ff7b9aaf640 1 --2- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 0x7ff7b406be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:05.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:05.292+0000 7ff7b9aaf640 1 --2- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 0x7ff7b406be30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:36:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.854+0000 7ff7b92ae640 1 -- 172.21.15.112:0/3520798308 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 508913553 0 0) 0x7ff7b406c6f0 con 0x7ff7b40c7450 2024-04-25T01:36:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.854+0000 7ff7b92ae640 1 -- 172.21.15.112:0/3520798308 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7a0003520 con 0x7ff7b40c7450 2024-04-25T01:36:06.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.856+0000 7ff7b92ae640 1 -- 172.21.15.112:0/3520798308 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1376590184 0 0) 0x7ff7a0003520 con 0x7ff7b40c7450 2024-04-25T01:36:06.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.856+0000 7ff7b92ae640 1 -- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 msgr2=0x7ff7b406be30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:36:06.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.856+0000 7ff7b92ae640 1 --2- 172.21.15.112:0/3520798308 >> v2:172.21.15.112:3300/0 conn(0x7ff7b415d440 0x7ff7b406be30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:06.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.856+0000 7ff7b92ae640 1 -- 172.21.15.112:0/3520798308 >> v1:172.21.15.112:6789/0 conn(0x7ff7b40be1a0 legacy=0x7ff7b415a500 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:36:06.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.856+0000 7ff7b92ae640 1 -- 172.21.15.112:0/3520798308 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b406c9d0 con 0x7ff7b40c7450 2024-04-25T01:36:06.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.856+0000 7ff7b92ae640 1 -- 172.21.15.112:0/3520798308 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7ff7b0001650 con 0x7ff7b40c7450 2024-04-25T01:36:06.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.856+0000 7ff7b92ae640 1 -- 172.21.15.112:0/3520798308 >> v1:172.21.15.187:6789/0 conn(0x7ff7b40c7450 legacy=0x7ff7b40c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:06.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.856+0000 7ff7b92ae640 1 --2- 172.21.15.112:0/3520798308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7a00039b0 0x7ff7a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:06.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.856+0000 7ff7b92ae640 1 --2- 172.21.15.112:0/3520798308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a0004800 0x7ff7a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:06.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.856+0000 7ff7b92ae640 1 --2- 172.21.15.112:0/3520798308 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7a00052e0 0x7ff7a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:06.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.856+0000 7ff7b92ae640 1 -- 172.21.15.112:0/3520798308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b40c5cb0 con 0x7ff7a0004800 2024-04-25T01:36:06.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.856+0000 7ff7b92ae640 1 -- 172.21.15.112:0/3520798308 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a000c1b0 con 0x7ff7a00039b0 2024-04-25T01:36:06.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.856+0000 7ff7b92ae640 1 -- 172.21.15.112:0/3520798308 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff7a000c3b0 con 0x7ff7a00052e0 2024-04-25T01:36:06.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.856+0000 7ff7b9aaf640 1 --2- 172.21.15.112:0/3520798308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a0004800 0x7ff7a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:06.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.856+0000 7ff7baab1640 1 --2- 172.21.15.112:0/3520798308 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7a00052e0 0x7ff7a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.857+0000 7ff7b9aaf640 1 -- 172.21.15.112:0/3520798308 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7a00052e0 msgr2=0x7ff7a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.857+0000 7ff7b9aaf640 1 --2- 172.21.15.112:0/3520798308 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7a00052e0 0x7ff7a000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.857+0000 7ff7b9aaf640 1 -- 172.21.15.112:0/3520798308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7a00039b0 msgr2=0x7ff7a0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.857+0000 7ff7b9aaf640 1 --2- 172.21.15.112:0/3520798308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7a00039b0 0x7ff7a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.857+0000 7ff7b9aaf640 1 -- 172.21.15.112:0/3520798308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b406c9d0 con 0x7ff7a0004800 2024-04-25T01:36:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.857+0000 7ff7baab1640 1 -- 172.21.15.112:0/3520798308 reap_dead start 2024-04-25T01:36:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.858+0000 7ff7b9aaf640 1 --2- 172.21.15.112:0/3520798308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a0004800 0x7ff7a0004c80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4006fd0 tx=0x7ff7a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0933dacc322d1db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:06.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.859+0000 7ff7b92ae640 1 -- 172.21.15.112:0/3520798308 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4031070 con 0x7ff7a0004800 2024-04-25T01:36:06.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.862+0000 7ff7b92ae640 1 -- 172.21.15.112:0/3520798308 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a402b730 con 0x7ff7a0004800 2024-04-25T01:36:06.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.862+0000 7ff7b92ae640 1 -- 172.21.15.112:0/3520798308 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4031070 con 0x7ff7a0004800 2024-04-25T01:36:06.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.863+0000 7ff7bc514640 1 -- 172.21.15.112:0/3520798308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a0004800 msgr2=0x7ff7a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:06.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.863+0000 7ff7bc514640 1 --2- 172.21.15.112:0/3520798308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a0004800 0x7ff7a0004c80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4006fd0 tx=0x7ff7a0005830 comp rx=0 tx=0).stop 2024-04-25T01:36:06.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.863+0000 7ff7bc514640 1 -- 172.21.15.112:0/3520798308 shutdown_connections 2024-04-25T01:36:06.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.863+0000 7ff7bc514640 1 --2- 172.21.15.112:0/3520798308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a0004800 0x7ff7a0004c80 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4006fd0 tx=0x7ff7a0005830 comp rx=0 tx=0).stop 2024-04-25T01:36:06.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.863+0000 7ff7bc514640 1 -- 172.21.15.112:0/3520798308 >> 172.21.15.112:0/3520798308 conn(0x7ff7b40ad2d0 msgr2=0x7ff7b40cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:06.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.863+0000 7ff7bc514640 1 -- 172.21.15.112:0/3520798308 shutdown_connections 2024-04-25T01:36:06.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.863+0000 7ff7bc514640 1 -- 172.21.15.112:0/3520798308 wait complete. 2024-04-25T01:36:06.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.865+0000 7ff7bc514640 1 Processor -- start 2024-04-25T01:36:06.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.865+0000 7ff7bc514640 1 -- start start 2024-04-25T01:36:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.866+0000 7ff7bc514640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7b4157150 0x7ff7b40c2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.866+0000 7ff7bc514640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7b40c2780 0x7ff7b41510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.866+0000 7ff7bc514640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7b41f3340 0x7ff7b41515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.866+0000 7ff7bc514640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b40c7830 con 0x7ff7b4157150 2024-04-25T01:36:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.866+0000 7ff7bc514640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b40caef0 con 0x7ff7b41f3340 2024-04-25T01:36:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.866+0000 7ff7bc514640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b40c79b0 con 0x7ff7b40c2780 2024-04-25T01:36:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.866+0000 7ff7b9aaf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7b40c2780 0x7ff7b41510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.866+0000 7ff7b9aaf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7b40c2780 0x7ff7b41510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:51196/0 (socket says 172.21.15.112:51196) 2024-04-25T01:36:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.866+0000 7ff7b9aaf640 1 -- 172.21.15.112:0/1215449974 learned_addr learned my addr 172.21.15.112:0/1215449974 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:36:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.866+0000 7ff7ba2b0640 1 --2- 172.21.15.112:0/1215449974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7b4157150 0x7ff7b40c2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.866+0000 7ff7b9aaf640 1 -- 172.21.15.112:0/1215449974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7b40c2780 msgr2=0x7ff7b41510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-25T01:36:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.866+0000 7ff7b9aaf640 1 -- 172.21.15.112:0/1215449974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7b40c2780 msgr2=0x7ff7b41510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:36:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.866+0000 7ff7b9aaf640 1 --2- 172.21.15.112:0/1215449974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7b40c2780 0x7ff7b41510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.866+0000 7ff7b9aaf640 1 --2- 172.21.15.112:0/1215449974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7b40c2780 0x7ff7b41510a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:36:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.867+0000 7ff7baab1640 1 --2- 172.21.15.112:0/1215449974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7b41f3340 0x7ff7b41515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.867+0000 7ff7ba2b0640 1 -- 172.21.15.112:0/1215449974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7b40c2780 msgr2=0x7ff7b41510a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:36:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.867+0000 7ff7ba2b0640 1 --2- 172.21.15.112:0/1215449974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7b40c2780 0x7ff7b41510a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.867+0000 7ff7ba2b0640 1 -- 172.21.15.112:0/1215449974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7b41f3340 msgr2=0x7ff7b41515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.867+0000 7ff7ba2b0640 1 --2- 172.21.15.112:0/1215449974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7b41f3340 0x7ff7b41515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.867+0000 7ff7ba2b0640 1 -- 172.21.15.112:0/1215449974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b415be50 con 0x7ff7b4157150 2024-04-25T01:36:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.867+0000 7ff7baab1640 1 --2- 172.21.15.112:0/1215449974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7b41f3340 0x7ff7b41515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:36:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.867+0000 7ff7ba2b0640 1 --2- 172.21.15.112:0/1215449974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7b4157150 0x7ff7b40c2240 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40699b0 tx=0x7ff7b000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.869+0000 7ff7aaffd640 1 -- 172.21.15.112:0/1215449974 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0018070 con 0x7ff7b4157150 2024-04-25T01:36:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.869+0000 7ff7bc514640 1 -- 172.21.15.112:0/1215449974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b415c130 con 0x7ff7b4157150 2024-04-25T01:36:06.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.869+0000 7ff7bc514640 1 -- 172.21.15.112:0/1215449974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b415c6f0 con 0x7ff7b4157150 2024-04-25T01:36:06.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.869+0000 7ff7aaffd640 1 -- 172.21.15.112:0/1215449974 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b00072f0 con 0x7ff7b4157150 2024-04-25T01:36:06.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.870+0000 7ff7aaffd640 1 -- 172.21.15.112:0/1215449974 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b00047e0 con 0x7ff7b4157150 2024-04-25T01:36:06.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.871+0000 7ff7bc514640 1 -- 172.21.15.112:0/1215449974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff79c005290 con 0x7ff7b4157150 2024-04-25T01:36:06.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.874+0000 7ff7aaffd640 1 -- 172.21.15.112:0/1215449974 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff7b001b070 con 0x7ff7b4157150 2024-04-25T01:36:06.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.875+0000 7ff7aaffd640 1 --2- 172.21.15.112:0/1215449974 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff79806be30 0x7ff79806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:06.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.875+0000 7ff7b9aaf640 1 --2- 172.21.15.112:0/1215449974 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff79806be30 0x7ff79806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:06.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.875+0000 7ff7aaffd640 1 -- 172.21.15.112:0/1215449974 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7ff7b008a590 con 0x7ff7b4157150 2024-04-25T01:36:06.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.876+0000 7ff7b9aaf640 1 --2- 172.21.15.112:0/1215449974 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff79806be30 0x7ff79806e2f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff7b406d370 tx=0x7ff7a4031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:06.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:06.883+0000 7ff7aaffd640 1 -- 172.21.15.112:0/1215449974 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff7b0058c30 con 0x7ff7b4157150 2024-04-25T01:36:07.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:07.376+0000 7ff7bc514640 1 -- 172.21.15.112:0/1215449974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff79c005580 con 0x7ff7b4157150 2024-04-25T01:36:07.354 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":82,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:36:07.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:07.378+0000 7ff7aaffd640 1 -- 172.21.15.112:0/1215449974 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7ff7b0058c30 con 0x7ff7b4157150 2024-04-25T01:36:07.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:07.381+0000 7ff7bc514640 1 -- 172.21.15.112:0/1215449974 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff79806be30 msgr2=0x7ff79806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:07.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:07.381+0000 7ff7bc514640 1 --2- 172.21.15.112:0/1215449974 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff79806be30 0x7ff79806e2f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff7b406d370 tx=0x7ff7a4031000 comp rx=0 tx=0).stop 2024-04-25T01:36:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:07.381+0000 7ff7bc514640 1 -- 172.21.15.112:0/1215449974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7b4157150 msgr2=0x7ff7b40c2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:07.381+0000 7ff7bc514640 1 --2- 172.21.15.112:0/1215449974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7b4157150 0x7ff7b40c2240 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40699b0 tx=0x7ff7b000cb90 comp rx=0 tx=0).stop 2024-04-25T01:36:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:07.381+0000 7ff7bc514640 1 -- 172.21.15.112:0/1215449974 shutdown_connections 2024-04-25T01:36:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:07.381+0000 7ff7bc514640 1 --2- 172.21.15.112:0/1215449974 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff79806be30 0x7ff79806e2f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:07.381+0000 7ff7bc514640 1 --2- 172.21.15.112:0/1215449974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7b41f3340 0x7ff7b41515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:07.381+0000 7ff7bc514640 1 --2- 172.21.15.112:0/1215449974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7b40c2780 0x7ff7b41510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:07.381+0000 7ff7bc514640 1 --2- 172.21.15.112:0/1215449974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7b4157150 0x7ff7b40c2240 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:07.381+0000 7ff7bc514640 1 -- 172.21.15.112:0/1215449974 >> 172.21.15.112:0/1215449974 conn(0x7ff7b40ad2d0 msgr2=0x7ff7b40cb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:07.381+0000 7ff7bc514640 1 -- 172.21.15.112:0/1215449974 shutdown_connections 2024-04-25T01:36:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:07.381+0000 7ff7bc514640 1 -- 172.21.15.112:0/1215449974 wait complete. 2024-04-25T01:36:10.377 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:36:10.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.556+0000 7fbcfaff1640 1 Processor -- start 2024-04-25T01:36:10.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.556+0000 7fbcfaff1640 1 -- start start 2024-04-25T01:36:10.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.557+0000 7fbcf9fef640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fbcec0d5650 0x7fbcec048360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:46960/0 (socket says 172.21.15.112:46960) 2024-04-25T01:36:10.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.557+0000 7fbcfaff1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 0x7fbcec0d8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:10.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.557+0000 7fbcf9fef640 1 -- 172.21.15.112:0/3339463983 learned_addr learned my addr 172.21.15.112:0/3339463983 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:36:10.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.557+0000 7fbcfaff1640 1 --2- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 0x7fbcec04aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:10.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.557+0000 7fbcfaff1640 1 -- 172.21.15.112:0/3339463983 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcec02d300 con 0x7fbcec0d5650 2024-04-25T01:36:10.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.557+0000 7fbcfaff1640 1 -- 172.21.15.112:0/3339463983 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fbcec0d86c0 con 0x7fbcec04aa40 2024-04-25T01:36:10.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.557+0000 7fbcfaff1640 1 -- 172.21.15.112:0/3339463983 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbcec04b3f0 con 0x7fbcec0db770 2024-04-25T01:36:10.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.558+0000 7fbcf97ee640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 0x7fbcec0d8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:10.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.558+0000 7fbcfa7f0640 1 --2- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 0x7fbcec04aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:10.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.558+0000 7fbcfa7f0640 1 -- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 msgr2=0x7fbcec04aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:36:10.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.558+0000 7fbcfa7f0640 1 -- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 msgr2=0x7fbcec04aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:36:10.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.558+0000 7fbcfa7f0640 1 --2- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 0x7fbcec04aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:10.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.558+0000 7fbcfa7f0640 1 --2- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 0x7fbcec04aeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:36:10.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.558+0000 7fbcf97ee640 1 -- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 msgr2=0x7fbcec0d8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:36:10.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.558+0000 7fbcf97ee640 1 -- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 msgr2=0x7fbcec0d8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:36:10.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.558+0000 7fbcf97ee640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 0x7fbcec0d8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:10.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.558+0000 7fbcf97ee640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 0x7fbcec0d8180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:36:10.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.759+0000 7fbcfa7f0640 1 --2- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 0x7fbcec04aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:10.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.759+0000 7fbcfa7f0640 1 -- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 msgr2=0x7fbcec04aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:36:10.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.759+0000 7fbcfa7f0640 1 -- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 msgr2=0x7fbcec04aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:36:10.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.759+0000 7fbcfa7f0640 1 --2- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 0x7fbcec04aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:10.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.759+0000 7fbcfa7f0640 1 --2- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 0x7fbcec04aeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:36:10.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.760+0000 7fbcf97ee640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 0x7fbcec0d8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:10.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.760+0000 7fbcf97ee640 1 -- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 msgr2=0x7fbcec0d8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:36:10.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.760+0000 7fbcf97ee640 1 -- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 msgr2=0x7fbcec0d8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:36:10.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.760+0000 7fbcf97ee640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 0x7fbcec0d8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:10.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:10.760+0000 7fbcf97ee640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 0x7fbcec0d8180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:36:11.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:11.160+0000 7fbcfa7f0640 1 --2- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 0x7fbcec04aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:11.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:11.160+0000 7fbcfa7f0640 1 -- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 msgr2=0x7fbcec04aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:36:11.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:11.160+0000 7fbcfa7f0640 1 -- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 msgr2=0x7fbcec04aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:36:11.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:11.160+0000 7fbcfa7f0640 1 --2- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 0x7fbcec04aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:11.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:11.160+0000 7fbcfa7f0640 1 --2- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 0x7fbcec04aeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:36:11.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:11.161+0000 7fbcf97ee640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 0x7fbcec0d8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:11.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:11.161+0000 7fbcf97ee640 1 -- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 msgr2=0x7fbcec0d8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:36:11.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:11.161+0000 7fbcf97ee640 1 -- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 msgr2=0x7fbcec0d8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:36:11.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:11.161+0000 7fbcf97ee640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 0x7fbcec0d8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:11.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:11.161+0000 7fbcf97ee640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 0x7fbcec0d8180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:36:11.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:11.961+0000 7fbcfa7f0640 1 --2- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 0x7fbcec04aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:11.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:11.961+0000 7fbcfa7f0640 1 -- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 msgr2=0x7fbcec04aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:36:11.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:11.961+0000 7fbcfa7f0640 1 -- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 msgr2=0x7fbcec04aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:36:11.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:11.961+0000 7fbcfa7f0640 1 --2- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 0x7fbcec04aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:11.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:11.961+0000 7fbcfa7f0640 1 --2- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 0x7fbcec04aeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:36:11.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:11.962+0000 7fbcf97ee640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 0x7fbcec0d8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:11.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:11.962+0000 7fbcf97ee640 1 -- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 msgr2=0x7fbcec0d8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:36:11.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:11.962+0000 7fbcf97ee640 1 -- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 msgr2=0x7fbcec0d8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:36:11.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:11.962+0000 7fbcf97ee640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 0x7fbcec0d8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:11.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:11.962+0000 7fbcf97ee640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 0x7fbcec0d8180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:36:12.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.854+0000 7fbcf8fed640 1 -- 172.21.15.112:0/3339463983 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3335730362 0 0) 0x7fbcec02d300 con 0x7fbcec0d5650 2024-04-25T01:36:12.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.854+0000 7fbcf8fed640 1 -- 172.21.15.112:0/3339463983 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbce0003520 con 0x7fbcec0d5650 2024-04-25T01:36:12.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.857+0000 7fbcf8fed640 1 -- 172.21.15.112:0/3339463983 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1011684072 0 0) 0x7fbce0003520 con 0x7fbcec0d5650 2024-04-25T01:36:12.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.857+0000 7fbcf8fed640 1 -- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 msgr2=0x7fbcec0d8180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:36:12.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.857+0000 7fbcf8fed640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec0db770 0x7fbcec0d8180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:12.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.857+0000 7fbcf8fed640 1 -- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 msgr2=0x7fbcec04aeb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:36:12.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.857+0000 7fbcf8fed640 1 --2- 172.21.15.112:0/3339463983 >> v2:172.21.15.112:3300/0 conn(0x7fbcec04aa40 0x7fbcec04aeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:12.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.857+0000 7fbcf8fed640 1 -- 172.21.15.112:0/3339463983 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcec0d88f0 con 0x7fbcec0d5650 2024-04-25T01:36:12.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.857+0000 7fbcf8fed640 1 -- 172.21.15.112:0/3339463983 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fbcf0001670 con 0x7fbcec0d5650 2024-04-25T01:36:12.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.857+0000 7fbcf8fed640 1 -- 172.21.15.112:0/3339463983 >> v1:172.21.15.112:6789/0 conn(0x7fbcec0d5650 legacy=0x7fbcec048360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:12.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.857+0000 7fbcf8fed640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbce00039b0 0x7fbce0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:12.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.857+0000 7fbcf8fed640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbce0004800 0x7fbce0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:12.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.857+0000 7fbcf8fed640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce00052e0 0x7fbce000fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:12.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.857+0000 7fbcf8fed640 1 -- 172.21.15.112:0/3339463983 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec04b3f0 con 0x7fbce00052e0 2024-04-25T01:36:12.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.857+0000 7fbcf8fed640 1 -- 172.21.15.112:0/3339463983 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbce0010610 con 0x7fbce0004800 2024-04-25T01:36:12.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.857+0000 7fbcf8fed640 1 -- 172.21.15.112:0/3339463983 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbce0010810 con 0x7fbce00039b0 2024-04-25T01:36:12.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.857+0000 7fbcf9fef640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbce00039b0 0x7fbce0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:12.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.857+0000 7fbcfa7f0640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce00052e0 0x7fbce000fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:12.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.857+0000 7fbcf97ee640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbce0004800 0x7fbce0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:12.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.858+0000 7fbcf9fef640 1 -- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbce0004800 msgr2=0x7fbce0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:12.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.858+0000 7fbcf9fef640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbce0004800 0x7fbce0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:12.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.858+0000 7fbcf9fef640 1 -- 172.21.15.112:0/3339463983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce00052e0 msgr2=0x7fbce000fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:12.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.858+0000 7fbcf9fef640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce00052e0 0x7fbce000fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:12.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.858+0000 7fbcf9fef640 1 -- 172.21.15.112:0/3339463983 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce0003520 con 0x7fbce00039b0 2024-04-25T01:36:12.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.859+0000 7fbcf9fef640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbce00039b0 0x7fbce0003dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000c160 tx=0x7fbce0005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a723f7e5b842d179 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:12.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.859+0000 7fbcf8fed640 1 -- 172.21.15.112:0/3339463983 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0007c80 con 0x7fbce00039b0 2024-04-25T01:36:12.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.859+0000 7fbcf8fed640 1 -- 172.21.15.112:0/3339463983 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcf0005ad0 con 0x7fbce00039b0 2024-04-25T01:36:12.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.859+0000 7fbcf8fed640 1 -- 172.21.15.112:0/3339463983 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0007070 con 0x7fbce00039b0 2024-04-25T01:36:12.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.860+0000 7fbcfaff1640 1 -- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbce00039b0 msgr2=0x7fbce0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:12.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.860+0000 7fbcfaff1640 1 --2- 172.21.15.112:0/3339463983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbce00039b0 0x7fbce0003dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000c160 tx=0x7fbce0005830 comp rx=0 tx=0).stop 2024-04-25T01:36:12.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.860+0000 7fbcfa7f0640 1 -- 172.21.15.112:0/3339463983 reap_dead start 2024-04-25T01:36:12.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.860+0000 7fbcfaff1640 1 -- 172.21.15.112:0/3339463983 shutdown_connections 2024-04-25T01:36:12.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.860+0000 7fbcfaff1640 1 -- 172.21.15.112:0/3339463983 >> 172.21.15.112:0/3339463983 conn(0x7fbcec02f5b0 msgr2=0x7fbcec02f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:12.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.861+0000 7fbcfaff1640 1 -- 172.21.15.112:0/3339463983 shutdown_connections 2024-04-25T01:36:12.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.861+0000 7fbcfaff1640 1 -- 172.21.15.112:0/3339463983 wait complete. 2024-04-25T01:36:12.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.873+0000 7fbcfaff1640 1 Processor -- start 2024-04-25T01:36:12.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.873+0000 7fbcfaff1640 1 -- start start 2024-04-25T01:36:12.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.874+0000 7fbcfaff1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce00039b0 0x7fbcec174b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:12.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.874+0000 7fbcfaff1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcec008c90 0x7fbcec0d9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:12.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.874+0000 7fbcfaff1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec007be0 0x7fbcec0cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:12.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.874+0000 7fbcfaff1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec0d8eb0 con 0x7fbce00039b0 2024-04-25T01:36:12.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.874+0000 7fbcfaff1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec0d9190 con 0x7fbcec008c90 2024-04-25T01:36:12.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.874+0000 7fbcfaff1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbcec04a520 con 0x7fbcec007be0 2024-04-25T01:36:12.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.874+0000 7fbcf9fef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce00039b0 0x7fbcec174b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:12.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.874+0000 7fbcf9fef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce00039b0 0x7fbcec174b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36586/0 (socket says 172.21.15.112:36586) 2024-04-25T01:36:12.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.874+0000 7fbcf9fef640 1 -- 172.21.15.112:0/2743132795 learned_addr learned my addr 172.21.15.112:0/2743132795 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:36:12.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.874+0000 7fbcf97ee640 1 --2- 172.21.15.112:0/2743132795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcec008c90 0x7fbcec0d9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:12.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.874+0000 7fbcfa7f0640 1 --2- 172.21.15.112:0/2743132795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec007be0 0x7fbcec0cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:12.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.874+0000 7fbcfa7f0640 1 -- 172.21.15.112:0/2743132795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec007be0 msgr2=0x7fbcec0cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-25T01:36:12.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.874+0000 7fbcfa7f0640 1 -- 172.21.15.112:0/2743132795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec007be0 msgr2=0x7fbcec0cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:36:12.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.874+0000 7fbcfa7f0640 1 --2- 172.21.15.112:0/2743132795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec007be0 0x7fbcec0cf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:12.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.874+0000 7fbcfa7f0640 1 --2- 172.21.15.112:0/2743132795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec007be0 0x7fbcec0cf9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:36:12.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.875+0000 7fbcf9fef640 1 -- 172.21.15.112:0/2743132795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec007be0 msgr2=0x7fbcec0cf9a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:36:12.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.875+0000 7fbcf9fef640 1 --2- 172.21.15.112:0/2743132795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec007be0 0x7fbcec0cf9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:12.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.875+0000 7fbcf9fef640 1 -- 172.21.15.112:0/2743132795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcec008c90 msgr2=0x7fbcec0d9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:12.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.875+0000 7fbcf9fef640 1 --2- 172.21.15.112:0/2743132795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcec008c90 0x7fbcec0d9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:12.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.875+0000 7fbcf9fef640 1 -- 172.21.15.112:0/2743132795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcec0d0740 con 0x7fbce00039b0 2024-04-25T01:36:12.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.875+0000 7fbcf9fef640 1 --2- 172.21.15.112:0/2743132795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce00039b0 0x7fbcec174b50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000c250 tx=0x7fbcec0d8ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:12.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.875+0000 7fbcea7fc640 1 -- 172.21.15.112:0/2743132795 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0041020 con 0x7fbce00039b0 2024-04-25T01:36:12.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.876+0000 7fbcfaff1640 1 -- 172.21.15.112:0/2743132795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcec0db220 con 0x7fbce00039b0 2024-04-25T01:36:12.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.876+0000 7fbcfaff1640 1 -- 172.21.15.112:0/2743132795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcec174070 con 0x7fbce00039b0 2024-04-25T01:36:12.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.876+0000 7fbcea7fc640 1 -- 172.21.15.112:0/2743132795 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcf00013b0 con 0x7fbce00039b0 2024-04-25T01:36:12.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.876+0000 7fbcea7fc640 1 -- 172.21.15.112:0/2743132795 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0007070 con 0x7fbce00039b0 2024-04-25T01:36:12.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.877+0000 7fbcfaff1640 1 -- 172.21.15.112:0/2743132795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcec0057e0 con 0x7fbce00039b0 2024-04-25T01:36:12.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.878+0000 7fbcea7fc640 1 -- 172.21.15.112:0/2743132795 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbcf0044020 con 0x7fbce00039b0 2024-04-25T01:36:12.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.878+0000 7fbcea7fc640 1 --2- 172.21.15.112:0/2743132795 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbcd806be60 0x7fbcd806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.878+0000 7fbcf97ee640 1 --2- 172.21.15.112:0/2743132795 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbcd806be60 0x7fbcd806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.879+0000 7fbcea7fc640 1 -- 172.21.15.112:0/2743132795 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fbcf00ab5e0 con 0x7fbce00039b0 2024-04-25T01:36:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.879+0000 7fbcf97ee640 1 --2- 172.21.15.112:0/2743132795 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbcd806be60 0x7fbcd806e320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0d03d0 tx=0x7fbce4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:12.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:12.885+0000 7fbcea7fc640 1 -- 172.21.15.112:0/2743132795 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbcf004b020 con 0x7fbce00039b0 2024-04-25T01:36:13.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:13.367+0000 7fbcfaff1640 1 -- 172.21.15.112:0/2743132795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fbcec04ca60 con 0x7fbce00039b0 2024-04-25T01:36:13.345 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":86,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:36:13.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:13.369+0000 7fbcea7fc640 1 -- 172.21.15.112:0/2743132795 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7fbcec0d0740 con 0x7fbce00039b0 2024-04-25T01:36:13.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:13.373+0000 7fbcfaff1640 1 -- 172.21.15.112:0/2743132795 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbcd806be60 msgr2=0x7fbcd806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:13.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:13.373+0000 7fbcfaff1640 1 --2- 172.21.15.112:0/2743132795 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbcd806be60 0x7fbcd806e320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0d03d0 tx=0x7fbce4008040 comp rx=0 tx=0).stop 2024-04-25T01:36:13.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:13.373+0000 7fbcfaff1640 1 -- 172.21.15.112:0/2743132795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce00039b0 msgr2=0x7fbcec174b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:13.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:13.373+0000 7fbcfaff1640 1 --2- 172.21.15.112:0/2743132795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce00039b0 0x7fbcec174b50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000c250 tx=0x7fbcec0d8ac0 comp rx=0 tx=0).stop 2024-04-25T01:36:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:13.373+0000 7fbcfaff1640 1 -- 172.21.15.112:0/2743132795 shutdown_connections 2024-04-25T01:36:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:13.373+0000 7fbcfaff1640 1 --2- 172.21.15.112:0/2743132795 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbcd806be60 0x7fbcd806e320 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:13.373+0000 7fbcfaff1640 1 --2- 172.21.15.112:0/2743132795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcec007be0 0x7fbcec0cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:13.373+0000 7fbcfaff1640 1 --2- 172.21.15.112:0/2743132795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcec008c90 0x7fbcec0d9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:13.373+0000 7fbcfaff1640 1 --2- 172.21.15.112:0/2743132795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce00039b0 0x7fbcec174b50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:13.373+0000 7fbcfaff1640 1 -- 172.21.15.112:0/2743132795 >> 172.21.15.112:0/2743132795 conn(0x7fbcec02f5b0 msgr2=0x7fbcec0c71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:13.374+0000 7fbcfaff1640 1 -- 172.21.15.112:0/2743132795 shutdown_connections 2024-04-25T01:36:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:13.374+0000 7fbcfaff1640 1 -- 172.21.15.112:0/2743132795 wait complete. 2024-04-25T01:36:16.366 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:36:16.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.549+0000 7f3e42909640 1 Processor -- start 2024-04-25T01:36:16.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.549+0000 7f3e42909640 1 -- start start 2024-04-25T01:36:16.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.550+0000 7f3e42909640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3e3c070370 0x7f3e3c158e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:16.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.550+0000 7f3e42909640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3e3c070a80 0x7f3e3c0c9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:16.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.550+0000 7f3e42909640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e3c0c7500 con 0x7f3e3c153040 2024-04-25T01:36:16.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.550+0000 7f3e42909640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f3e3c155da0 con 0x7f3e3c070a80 2024-04-25T01:36:16.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.550+0000 7f3e42909640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3e3c159340 con 0x7f3e3c070370 2024-04-25T01:36:16.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.550+0000 7f3e3b7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3e3c070370 0x7f3e3c158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:16.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.550+0000 7f3e3b7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3e3c070370 0x7f3e3c158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49932/0 (socket says 172.21.15.112:49932) 2024-04-25T01:36:16.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.550+0000 7f3e3b7fe640 1 -- 172.21.15.112:0/1019795052 learned_addr learned my addr 172.21.15.112:0/1019795052 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:36:16.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.550+0000 7f3e40ea6640 1 --2- 172.21.15.112:0/1019795052 >> v2:172.21.15.112:3300/0 conn(0x7f3e3c070a80 0x7f3e3c0c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:16.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.551+0000 7f3e40ea6640 1 -- 172.21.15.112:0/1019795052 >> v2:172.21.15.187:3300/0 conn(0x7f3e3c070370 msgr2=0x7f3e3c158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:16.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.551+0000 7f3e40ea6640 1 --2- 172.21.15.112:0/1019795052 >> v2:172.21.15.187:3300/0 conn(0x7f3e3c070370 0x7f3e3c158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:16.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.551+0000 7f3e40ea6640 1 -- 172.21.15.112:0/1019795052 >> v1:172.21.15.187:6789/0 conn(0x7f3e3c153040 legacy=0x7f3e3c156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:16.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.551+0000 7f3e40ea6640 1 -- 172.21.15.112:0/1019795052 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3c0c77e0 con 0x7f3e3c070a80 2024-04-25T01:36:16.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.551+0000 7f3e3b7fe640 1 --2- 172.21.15.112:0/1019795052 >> v2:172.21.15.187:3300/0 conn(0x7f3e3c070370 0x7f3e3c158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:36:16.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.552+0000 7f3e40ea6640 1 --2- 172.21.15.112:0/1019795052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3c070a80 0x7f3e3c0c9470 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0c2eb0 tx=0x7f3e3402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487cee8e623131a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:16.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.553+0000 7f3e3affd640 1 -- 172.21.15.112:0/1019795052 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34037070 con 0x7f3e3c070a80 2024-04-25T01:36:16.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.553+0000 7f3e3affd640 1 -- 172.21.15.112:0/1019795052 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e34032440 con 0x7f3e3c070a80 2024-04-25T01:36:16.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.553+0000 7f3e3affd640 1 -- 172.21.15.112:0/1019795052 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3402ae60 con 0x7f3e3c070a80 2024-04-25T01:36:16.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.554+0000 7f3e42909640 1 -- 172.21.15.112:0/1019795052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3c070a80 msgr2=0x7f3e3c0c9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:16.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.554+0000 7f3e42909640 1 --2- 172.21.15.112:0/1019795052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3c070a80 0x7f3e3c0c9470 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0c2eb0 tx=0x7f3e3402a110 comp rx=0 tx=0).stop 2024-04-25T01:36:16.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.555+0000 7f3e42909640 1 -- 172.21.15.112:0/1019795052 shutdown_connections 2024-04-25T01:36:16.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.555+0000 7f3e42909640 1 --2- 172.21.15.112:0/1019795052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3c070a80 0x7f3e3c0c9470 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:16.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.555+0000 7f3e42909640 1 --2- 172.21.15.112:0/1019795052 >> v2:172.21.15.187:3300/0 conn(0x7f3e3c070370 0x7f3e3c158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:16.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.555+0000 7f3e42909640 1 -- 172.21.15.112:0/1019795052 >> 172.21.15.112:0/1019795052 conn(0x7f3e3c0747f0 msgr2=0x7f3e3c074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:16.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.555+0000 7f3e42909640 1 -- 172.21.15.112:0/1019795052 shutdown_connections 2024-04-25T01:36:16.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.555+0000 7f3e42909640 1 -- 172.21.15.112:0/1019795052 wait complete. 2024-04-25T01:36:16.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.558+0000 7f3e42909640 1 Processor -- start 2024-04-25T01:36:16.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.558+0000 7f3e42909640 1 -- start start 2024-04-25T01:36:16.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.559+0000 7f3e42909640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3c070370 0x7f3e3c153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:16.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.559+0000 7f3e42909640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e3c070a80 0x7f3e3c1461e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:16.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.559+0000 7f3e42909640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3c153040 0x7f3e3c146720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:16.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.559+0000 7f3e42909640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c06f320 con 0x7f3e3c153040 2024-04-25T01:36:16.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.559+0000 7f3e42909640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c0c7ac0 con 0x7f3e3c070370 2024-04-25T01:36:16.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.559+0000 7f3e42909640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3e3c06f4a0 con 0x7f3e3c070a80 2024-04-25T01:36:16.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.560+0000 7f3e40ea6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3c153040 0x7f3e3c146720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:16.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.560+0000 7f3e3bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3c070370 0x7f3e3c153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:16.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.560+0000 7f3e3bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3c070370 0x7f3e3c153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49942/0 (socket says 172.21.15.112:49942) 2024-04-25T01:36:16.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.560+0000 7f3e3bfff640 1 -- 172.21.15.112:0/2707732397 learned_addr learned my addr 172.21.15.112:0/2707732397 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:36:16.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.560+0000 7f3e3bfff640 1 -- 172.21.15.112:0/2707732397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e3c070a80 msgr2=0x7f3e3c1461e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:16.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.560+0000 7f3e3bfff640 1 --2- 172.21.15.112:0/2707732397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e3c070a80 0x7f3e3c1461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:16.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.560+0000 7f3e3bfff640 1 -- 172.21.15.112:0/2707732397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3c153040 msgr2=0x7f3e3c146720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:16.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.560+0000 7f3e3bfff640 1 --2- 172.21.15.112:0/2707732397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3c153040 0x7f3e3c146720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:16.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.560+0000 7f3e3bfff640 1 -- 172.21.15.112:0/2707732397 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3c150f80 con 0x7f3e3c070370 2024-04-25T01:36:16.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.560+0000 7f3e40ea6640 1 --2- 172.21.15.112:0/2707732397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3c153040 0x7f3e3c146720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:36:16.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.561+0000 7f3e3bfff640 1 --2- 172.21.15.112:0/2707732397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3c070370 0x7f3e3c153b00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00c400 tx=0x7f3e2c00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:16.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.562+0000 7f3e38ff9640 1 -- 172.21.15.112:0/2707732397 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c017070 con 0x7f3e3c070370 2024-04-25T01:36:16.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.562+0000 7f3e38ff9640 1 -- 172.21.15.112:0/2707732397 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e2c0052d0 con 0x7f3e3c070370 2024-04-25T01:36:16.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.562+0000 7f3e38ff9640 1 -- 172.21.15.112:0/2707732397 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c003830 con 0x7f3e3c070370 2024-04-25T01:36:16.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.562+0000 7f3e42909640 1 -- 172.21.15.112:0/2707732397 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e3c1e7c30 con 0x7f3e3c070370 2024-04-25T01:36:16.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.562+0000 7f3e42909640 1 -- 172.21.15.112:0/2707732397 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e3c06ccb0 con 0x7f3e3c070370 2024-04-25T01:36:16.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.563+0000 7f3e42909640 1 -- 172.21.15.112:0/2707732397 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e3c1534b0 con 0x7f3e3c070370 2024-04-25T01:36:16.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.564+0000 7f3e38ff9640 1 -- 172.21.15.112:0/2707732397 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e2c01a070 con 0x7f3e3c070370 2024-04-25T01:36:16.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.565+0000 7f3e38ff9640 1 --2- 172.21.15.112:0/2707732397 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3e2006bf30 0x7f3e2006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:16.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.565+0000 7f3e38ff9640 1 -- 172.21.15.112:0/2707732397 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f3e2c08a4e0 con 0x7f3e3c070370 2024-04-25T01:36:16.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.565+0000 7f3e3b7fe640 1 --2- 172.21.15.112:0/2707732397 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3e2006bf30 0x7f3e2006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:16.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.566+0000 7f3e3b7fe640 1 --2- 172.21.15.112:0/2707732397 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3e2006bf30 0x7f3e2006e3f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3e240147c0 tx=0x7f3e24016040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:16.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:16.576+0000 7f3e38ff9640 1 -- 172.21.15.112:0/2707732397 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3e3c1534b0 con 0x7f3e3c070370 2024-04-25T01:36:17.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.060+0000 7f3e42909640 1 -- 172.21.15.112:0/2707732397 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3e3c0c6b40 con 0x7f3e3c070370 2024-04-25T01:36:17.038 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":90,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:36:17.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.062+0000 7f3e38ff9640 1 -- 172.21.15.112:0/2707732397 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1568 (secure 0 0 0) 0x7f3e3c0c6b40 con 0x7f3e3c070370 2024-04-25T01:36:17.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.066+0000 7f3e42909640 1 -- 172.21.15.112:0/2707732397 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3e2006bf30 msgr2=0x7f3e2006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.066+0000 7f3e42909640 1 --2- 172.21.15.112:0/2707732397 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3e2006bf30 0x7f3e2006e3f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3e240147c0 tx=0x7f3e24016040 comp rx=0 tx=0).stop 2024-04-25T01:36:17.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.066+0000 7f3e42909640 1 -- 172.21.15.112:0/2707732397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3c070370 msgr2=0x7f3e3c153b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.066+0000 7f3e42909640 1 --2- 172.21.15.112:0/2707732397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3c070370 0x7f3e3c153b00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00c400 tx=0x7f3e2c00c8c0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.066+0000 7f3e42909640 1 -- 172.21.15.112:0/2707732397 shutdown_connections 2024-04-25T01:36:17.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.067+0000 7f3e42909640 1 --2- 172.21.15.112:0/2707732397 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3e2006bf30 0x7f3e2006e3f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.067+0000 7f3e42909640 1 --2- 172.21.15.112:0/2707732397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3c153040 0x7f3e3c146720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.067+0000 7f3e42909640 1 --2- 172.21.15.112:0/2707732397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e3c070a80 0x7f3e3c1461e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.067+0000 7f3e42909640 1 --2- 172.21.15.112:0/2707732397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3c070370 0x7f3e3c153b00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.067+0000 7f3e42909640 1 -- 172.21.15.112:0/2707732397 >> 172.21.15.112:0/2707732397 conn(0x7f3e3c0747f0 msgr2=0x7f3e3c1594f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:17.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.067+0000 7f3e42909640 1 -- 172.21.15.112:0/2707732397 shutdown_connections 2024-04-25T01:36:17.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.067+0000 7f3e42909640 1 -- 172.21.15.112:0/2707732397 wait complete. 2024-04-25T01:36:17.061 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:36:17.061 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:36:17.061 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:36:17.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.245+0000 7fd3ce274640 1 Processor -- start 2024-04-25T01:36:17.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.245+0000 7fd3ce274640 1 -- start start 2024-04-25T01:36:17.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.246+0000 7fd3ce274640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3c8152e90 0x7fd3c806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.246+0000 7fd3ce274640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd3c806e860 0x7fd3c8156120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.246+0000 7fd3ce274640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3c8159ca0 con 0x7fd3c806f670 2024-04-25T01:36:17.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.246+0000 7fd3ce274640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd3c81599c0 con 0x7fd3c806e860 2024-04-25T01:36:17.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.246+0000 7fd3ce274640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c806ef80 con 0x7fd3c8152e90 2024-04-25T01:36:17.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.246+0000 7fd3cca71640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd3c806f670 0x7fd3c8155a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:46964/0 (socket says 172.21.15.112:46964) 2024-04-25T01:36:17.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.246+0000 7fd3cca71640 1 -- 172.21.15.112:0/3410884577 learned_addr learned my addr 172.21.15.112:0/3410884577 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:36:17.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.246+0000 7fd3cda73640 1 --2- 172.21.15.112:0/3410884577 >> v2:172.21.15.187:3300/0 conn(0x7fd3c806e860 0x7fd3c8156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.247+0000 7fd3cd272640 1 --2- 172.21.15.112:0/3410884577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3c8152e90 0x7fd3c806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.247+0000 7fd3cda73640 1 -- 172.21.15.112:0/3410884577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3c8152e90 msgr2=0x7fd3c806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:17.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.247+0000 7fd3cda73640 1 --2- 172.21.15.112:0/3410884577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3c8152e90 0x7fd3c806c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.247+0000 7fd3bffff640 1 -- 172.21.15.112:0/3410884577 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 783369960 0 0) 0x7fd3c8159ca0 con 0x7fd3c806f670 2024-04-25T01:36:17.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.247+0000 7fd3cda73640 1 -- 172.21.15.112:0/3410884577 >> v1:172.21.15.112:6789/0 conn(0x7fd3c806f670 legacy=0x7fd3c8155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.247+0000 7fd3cda73640 1 -- 172.21.15.112:0/3410884577 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c8153270 con 0x7fd3c806e860 2024-04-25T01:36:17.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.248+0000 7fd3cda73640 1 --2- 172.21.15.112:0/3410884577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c806e860 0x7fd3c8156120 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009d90 tx=0x7fd3c002a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4950d8319001249c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:17.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.248+0000 7fd3bffff640 1 -- 172.21.15.112:0/3410884577 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0031600 con 0x7fd3c806e860 2024-04-25T01:36:17.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.248+0000 7fd3bffff640 1 -- 172.21.15.112:0/3410884577 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c00317a0 con 0x7fd3c806e860 2024-04-25T01:36:17.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.248+0000 7fd3bffff640 1 -- 172.21.15.112:0/3410884577 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0031b70 con 0x7fd3c806e860 2024-04-25T01:36:17.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.249+0000 7fd3ce274640 1 -- 172.21.15.112:0/3410884577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c806e860 msgr2=0x7fd3c8156120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.249+0000 7fd3ce274640 1 --2- 172.21.15.112:0/3410884577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c806e860 0x7fd3c8156120 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009d90 tx=0x7fd3c002a5d0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.250+0000 7fd3ce274640 1 -- 172.21.15.112:0/3410884577 shutdown_connections 2024-04-25T01:36:17.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.250+0000 7fd3ce274640 1 --2- 172.21.15.112:0/3410884577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c806e860 0x7fd3c8156120 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.250+0000 7fd3ce274640 1 --2- 172.21.15.112:0/3410884577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3c8152e90 0x7fd3c806c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.250+0000 7fd3ce274640 1 -- 172.21.15.112:0/3410884577 >> 172.21.15.112:0/3410884577 conn(0x7fd3c806d090 msgr2=0x7fd3c806d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:17.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.251+0000 7fd3ce274640 1 -- 172.21.15.112:0/3410884577 shutdown_connections 2024-04-25T01:36:17.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.251+0000 7fd3ce274640 1 -- 172.21.15.112:0/3410884577 wait complete. 2024-04-25T01:36:17.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.253+0000 7fd3ce274640 1 Processor -- start 2024-04-25T01:36:17.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.253+0000 7fd3ce274640 1 -- start start 2024-04-25T01:36:17.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.254+0000 7fd3ce274640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c806e860 0x7fd3c80a7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.254+0000 7fd3ce274640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c806f670 0x7fd3c8072c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.254+0000 7fd3ce274640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3c8152e90 0x7fd3c8073170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.254+0000 7fd3ce274640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c80ac560 con 0x7fd3c806e860 2024-04-25T01:36:17.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.254+0000 7fd3ce274640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c80ac3e0 con 0x7fd3c806f670 2024-04-25T01:36:17.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.254+0000 7fd3ce274640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c806c6b0 con 0x7fd3c8152e90 2024-04-25T01:36:17.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.254+0000 7fd3cd272640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c806e860 0x7fd3c80a7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.254+0000 7fd3cda73640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3c8152e90 0x7fd3c8073170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.255+0000 7fd3cda73640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3c8152e90 0x7fd3c8073170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:33640/0 (socket says 172.21.15.112:33640) 2024-04-25T01:36:17.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.255+0000 7fd3cd272640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c806e860 0x7fd3c80a7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36628/0 (socket says 172.21.15.112:36628) 2024-04-25T01:36:17.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.255+0000 7fd3cda73640 1 -- 172.21.15.112:0/1469886990 learned_addr learned my addr 172.21.15.112:0/1469886990 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:36:17.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.255+0000 7fd3cca71640 1 --2- 172.21.15.112:0/1469886990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c806f670 0x7fd3c8072c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.255+0000 7fd3cd272640 1 -- 172.21.15.112:0/1469886990 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3c8152e90 msgr2=0x7fd3c8073170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.255+0000 7fd3cd272640 1 --2- 172.21.15.112:0/1469886990 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3c8152e90 0x7fd3c8073170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.255+0000 7fd3cd272640 1 -- 172.21.15.112:0/1469886990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c806f670 msgr2=0x7fd3c8072c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.255+0000 7fd3cd272640 1 --2- 172.21.15.112:0/1469886990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c806f670 0x7fd3c8072c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.255+0000 7fd3cd272640 1 -- 172.21.15.112:0/1469886990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c8071980 con 0x7fd3c806e860 2024-04-25T01:36:17.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.256+0000 7fd3cd272640 1 --2- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c806e860 0x7fd3c80a7a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400c5f0 tx=0x7fd3c400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:17.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.256+0000 7fd3bdffb640 1 -- 172.21.15.112:0/1469886990 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4015070 con 0x7fd3c806e860 2024-04-25T01:36:17.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.256+0000 7fd3bdffb640 1 -- 172.21.15.112:0/1469886990 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c4011800 con 0x7fd3c806e860 2024-04-25T01:36:17.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.257+0000 7fd3bdffb640 1 -- 172.21.15.112:0/1469886990 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4011bd0 con 0x7fd3c806e860 2024-04-25T01:36:17.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.257+0000 7fd3ce274640 1 -- 172.21.15.112:0/1469886990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c8071c60 con 0x7fd3c806e860 2024-04-25T01:36:17.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.257+0000 7fd3ce274640 1 -- 172.21.15.112:0/1469886990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c8072710 con 0x7fd3c806e860 2024-04-25T01:36:17.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.258+0000 7fd3ce274640 1 --2- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c81f24f0 0x7fd3c81f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.258+0000 7fd3ce274640 1 -- 172.21.15.112:0/1469886990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c80a9270 con 0x7fd3c81f24f0 2024-04-25T01:36:17.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.258+0000 7fd3cca71640 1 --2- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c81f24f0 0x7fd3c81f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.259+0000 7fd3cca71640 1 -- 172.21.15.112:0/1469886990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd3c8072910 con 0x7fd3c81f24f0 2024-04-25T01:36:17.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.259+0000 7fd3cca71640 1 --2- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c81f24f0 0x7fd3c81f28d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80c18f0 tx=0x7fd3b8001990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:17.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.259+0000 7fd3bdffb640 1 -- 172.21.15.112:0/1469886990 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd3c4018070 con 0x7fd3c806e860 2024-04-25T01:36:17.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.260+0000 7fd3bdffb640 1 --2- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd3ac06beb0 0x7fd3ac06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.260+0000 7fd3bdffb640 1 -- 172.21.15.112:0/1469886990 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fd3c408af60 con 0x7fd3c806e860 2024-04-25T01:36:17.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.260+0000 7fd3bdffb640 1 -- 172.21.15.112:0/1469886990 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b800f5e0 con 0x7fd3c81f24f0 2024-04-25T01:36:17.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.260+0000 7fd3bdffb640 1 -- 172.21.15.112:0/1469886990 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd3b8018070 con 0x7fd3c81f24f0 2024-04-25T01:36:17.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.260+0000 7fd3bdffb640 1 -- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c81f24f0 msgr2=0x7fd3c81f28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.260+0000 7fd3bdffb640 1 --2- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c81f24f0 0x7fd3c81f28d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80c18f0 tx=0x7fd3b8001990 comp rx=0 tx=0).stop 2024-04-25T01:36:17.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.260+0000 7fd3bdffb640 1 -- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c81f24f0 msgr2=0x7fd3c81f28d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:36:17.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.260+0000 7fd3bdffb640 1 --2- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c81f24f0 0x7fd3c81f28d0 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80c18f0 tx=0x7fd3b8001990 comp rx=0 tx=0).stop 2024-04-25T01:36:17.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.260+0000 7fd3cda73640 1 --2- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd3ac06beb0 0x7fd3ac06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.261+0000 7fd3cda73640 1 --2- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd3ac06beb0 0x7fd3ac06e370 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009bd0 tx=0x7fd3c002f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:17.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.281+0000 7fd3ce274640 1 --2- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c81f2e10 0x7fd3c814dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.281+0000 7fd3ce274640 1 -- 172.21.15.112:0/1469886990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c804f500 con 0x7fd3c81f2e10 2024-04-25T01:36:17.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.281+0000 7fd3cca71640 1 --2- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c81f2e10 0x7fd3c814dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.282+0000 7fd3cca71640 1 -- 172.21.15.112:0/1469886990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd3c806e490 con 0x7fd3c81f2e10 2024-04-25T01:36:17.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.282+0000 7fd3cca71640 1 --2- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c81f2e10 0x7fd3c814dde0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8158480 tx=0x7fd3b8001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:17.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.283+0000 7fd3bdffb640 1 -- 172.21.15.112:0/1469886990 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b800fe50 con 0x7fd3c81f2e10 2024-04-25T01:36:17.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.283+0000 7fd3bdffb640 1 -- 172.21.15.112:0/1469886990 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4560 (secure 0 0 0) 0x7fd3b800fe50 con 0x7fd3c81f2e10 2024-04-25T01:36:17.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.283+0000 7fd3bdffb640 1 -- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c81f2e10 msgr2=0x7fd3c814dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.283+0000 7fd3bdffb640 1 --2- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c81f2e10 0x7fd3c814dde0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8158480 tx=0x7fd3b8001080 comp rx=0 tx=0).stop 2024-04-25T01:36:17.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.283+0000 7fd3bdffb640 1 -- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c81f2e10 msgr2=0x7fd3c814dde0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:36:17.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.283+0000 7fd3bdffb640 1 --2- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c81f2e10 0x7fd3c814dde0 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8158480 tx=0x7fd3b8001080 comp rx=0 tx=0).stop 2024-04-25T01:36:17.260 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:36:17.260 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:36:17.260 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:36:17.260 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:36:17.260 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 90, 2024-04-25T01:36:17.260 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 28098, 2024-04-25T01:36:17.260 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:36:17.260 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:36:17.260 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:36:17.260 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:36:17.260 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:17.260 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 2, 2024-04-25T01:36:17.260 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:36:17.260 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:36:17.261 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:36:17.261 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:17.261 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:17.261 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:17.261 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:17.261 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:17.261 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:17.261 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:17.261 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:17.261 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:17.261 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:17.261 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:17.261 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:17.262 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:36:17.262 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:36:17.262 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:17.262 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:17.262 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:17.262 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:17.262 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:17.262 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:17.262 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:17.262 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:17.262 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:17.262 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:17.262 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:17.263 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:17.263 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.263 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:36:17.263 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:36:17.263 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:36:17.263 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:36:17.263 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:36:17.263 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:36:17.263 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:36:17.263 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:36:17.263 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:36:17.263 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:36:17.263 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:36:17.263 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:36:17.264 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:36:17.264 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:36:17.264 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:36:17.264 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:36:17.264 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:36:17.264 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:17.264 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:17.264 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:17.264 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:17.264 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:17.264 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:17.264 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:17.264 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:17.264 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:17.264 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:17.265 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:17.265 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:17.265 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:36:17.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.265 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:36:17.265 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.265 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:36:17.265 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:36:17.265 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:17.265 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:17.265 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.265 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:17.265 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:36:17.265 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:17.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.266 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.266 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:17.266 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:36:17.266 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:17.266 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.266 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:17.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.266 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:36:17.266 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:36:17.266 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:17.266 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:17.266 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:17.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.267 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.267 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:36:17.267 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:36:17.267 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:17.267 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:17.267 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.267 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:17.267 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:36:17.267 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:17.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.267 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.267 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:17.267 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:36:17.267 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:17.268 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.268 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:17.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.268 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:36:17.268 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:36:17.268 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:17.268 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:17.268 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:17.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.268 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.268 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:36:17.268 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:36:17.268 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:17.268 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:17.269 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.269 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:17.269 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:36:17.269 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:17.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.269 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.269 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:17.269 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:36:17.269 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:17.269 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.269 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:17.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.269 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:36:17.269 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:36:17.269 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:17.270 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:17.270 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:17.270 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.270 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:17.270 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.270 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:36:17.270 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:36:17.270 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.270 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:17.270 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:17.270 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:36:17.270 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.270 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:17.271 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:36:17.271 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.271 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:17.271 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:17.271 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:36:17.271 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.271 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:17.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.271 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:36:17.271 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:36:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.284+0000 7fd3ce274640 1 -- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd3ac06beb0 msgr2=0x7fd3ac06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.284+0000 7fd3ce274640 1 --2- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd3ac06beb0 0x7fd3ac06e370 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009bd0 tx=0x7fd3c002f000 comp rx=0 tx=0).stop 2024-04-25T01:36:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.284+0000 7fd3cda73640 1 -- 172.21.15.112:0/1469886990 reap_dead start 2024-04-25T01:36:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.284+0000 7fd3ce274640 1 -- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c806e860 msgr2=0x7fd3c80a7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.284+0000 7fd3ce274640 1 --2- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c806e860 0x7fd3c80a7a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400c5f0 tx=0x7fd3c400cab0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.284+0000 7fd3ce274640 1 -- 172.21.15.112:0/1469886990 shutdown_connections 2024-04-25T01:36:17.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.284+0000 7fd3ce274640 1 --2- 172.21.15.112:0/1469886990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c806e860 0x7fd3c80a7a60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.284+0000 7fd3ce274640 1 -- 172.21.15.112:0/1469886990 >> 172.21.15.112:0/1469886990 conn(0x7fd3c806d090 msgr2=0x7fd3c80ab970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:17.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.285+0000 7fd3ce274640 1 -- 172.21.15.112:0/1469886990 shutdown_connections 2024-04-25T01:36:17.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.285+0000 7fd3ce274640 1 -- 172.21.15.112:0/1469886990 wait complete. 2024-04-25T01:36:17.277 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:36:17.277 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:36:17.278 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:36:17.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.464+0000 7f5b92483640 1 Processor -- start 2024-04-25T01:36:17.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.464+0000 7f5b92483640 1 -- start start 2024-04-25T01:36:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.465+0000 7f5b92483640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5b8c0c56d0 0x7f5b8c06f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.465+0000 7f5b92483640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b8c0c5b80 con 0x7f5b8c0c6630 2024-04-25T01:36:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.465+0000 7f5b92483640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b8c0c1e50 con 0x7f5b8c06c170 2024-04-25T01:36:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.465+0000 7f5b92483640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5b8c0c6d70 con 0x7f5b8c0c56d0 2024-04-25T01:36:17.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.465+0000 7f5b90a20640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f5b8c06c170 0x7f5b8c1646d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:45668/0 (socket says 172.21.15.112:45668) 2024-04-25T01:36:17.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.465+0000 7f5b90a20640 1 -- 172.21.15.112:0/217242080 learned_addr learned my addr 172.21.15.112:0/217242080 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:36:17.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.465+0000 7f5b8b7fe640 1 --2- 172.21.15.112:0/217242080 >> v2:172.21.15.187:3300/0 conn(0x7f5b8c0c56d0 0x7f5b8c06f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.466+0000 7f5b8b7fe640 1 -- 172.21.15.112:0/217242080 >> v1:172.21.15.187:6789/0 conn(0x7f5b8c06c170 legacy=0x7f5b8c1646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.466+0000 7f5b8b7fe640 1 -- 172.21.15.112:0/217242080 >> v1:172.21.15.112:6789/0 conn(0x7f5b8c0c6630 legacy=0x7f5b8c15dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:17.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.466+0000 7f5b8b7fe640 1 -- 172.21.15.112:0/217242080 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c0c7050 con 0x7f5b8c0c56d0 2024-04-25T01:36:17.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.467+0000 7f5b8b7fe640 1 --2- 172.21.15.112:0/217242080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c0c56d0 0x7f5b8c06f7b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0097a0 tx=0x7f5b7c02a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=875d339bd25c573 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:17.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.468+0000 7f5b8affd640 1 -- 172.21.15.112:0/217242080 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c037070 con 0x7f5b8c0c56d0 2024-04-25T01:36:17.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.468+0000 7f5b8affd640 1 -- 172.21.15.112:0/217242080 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7c032440 con 0x7f5b8c0c56d0 2024-04-25T01:36:17.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.468+0000 7f5b8affd640 1 -- 172.21.15.112:0/217242080 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c032810 con 0x7f5b8c0c56d0 2024-04-25T01:36:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.469+0000 7f5b92483640 1 -- 172.21.15.112:0/217242080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c0c56d0 msgr2=0x7f5b8c06f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.469+0000 7f5b92483640 1 --2- 172.21.15.112:0/217242080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c0c56d0 0x7f5b8c06f7b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0097a0 tx=0x7f5b7c02a1e0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.471+0000 7f5b92483640 1 -- 172.21.15.112:0/217242080 shutdown_connections 2024-04-25T01:36:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.471+0000 7f5b92483640 1 --2- 172.21.15.112:0/217242080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c0c56d0 0x7f5b8c06f7b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.471+0000 7f5b92483640 1 -- 172.21.15.112:0/217242080 >> 172.21.15.112:0/217242080 conn(0x7f5b8c0c42d0 msgr2=0x7f5b8c0c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.471+0000 7f5b92483640 1 -- 172.21.15.112:0/217242080 shutdown_connections 2024-04-25T01:36:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.471+0000 7f5b92483640 1 -- 172.21.15.112:0/217242080 wait complete. 2024-04-25T01:36:17.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.473+0000 7f5b92483640 1 Processor -- start 2024-04-25T01:36:17.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.473+0000 7f5b92483640 1 -- start start 2024-04-25T01:36:17.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.474+0000 7f5b92483640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c06c170 0x7f5b8c14f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.475+0000 7f5b92483640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b8c0c56d0 0x7f5b8c0ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.475+0000 7f5b92483640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b8c0c6630 0x7f5b8c1efe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.475+0000 7f5b92483640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c06fcf0 con 0x7f5b8c0c6630 2024-04-25T01:36:17.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.475+0000 7f5b92483640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c06f290 con 0x7f5b8c06c170 2024-04-25T01:36:17.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.475+0000 7f5b92483640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8c0c74f0 con 0x7f5b8c0c56d0 2024-04-25T01:36:17.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.475+0000 7f5b90a20640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b8c0c6630 0x7f5b8c1efe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.475+0000 7f5b90a20640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b8c0c6630 0x7f5b8c1efe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57910/0 (socket says 172.21.15.112:57910) 2024-04-25T01:36:17.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.475+0000 7f5b90a20640 1 -- 172.21.15.112:0/3357551881 learned_addr learned my addr 172.21.15.112:0/3357551881 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:36:17.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.475+0000 7f5b8b7fe640 1 --2- 172.21.15.112:0/3357551881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b8c0c56d0 0x7f5b8c0ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.476+0000 7f5b8bfff640 1 --2- 172.21.15.112:0/3357551881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c06c170 0x7f5b8c14f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.476+0000 7f5b90a20640 1 -- 172.21.15.112:0/3357551881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b8c0c56d0 msgr2=0x7f5b8c0ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.476+0000 7f5b90a20640 1 --2- 172.21.15.112:0/3357551881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b8c0c56d0 0x7f5b8c0ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.476+0000 7f5b90a20640 1 -- 172.21.15.112:0/3357551881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c06c170 msgr2=0x7f5b8c14f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.476+0000 7f5b90a20640 1 --2- 172.21.15.112:0/3357551881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c06c170 0x7f5b8c14f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.476+0000 7f5b90a20640 1 -- 172.21.15.112:0/3357551881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c152950 con 0x7f5b8c0c6630 2024-04-25T01:36:17.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.476+0000 7f5b90a20640 1 --2- 172.21.15.112:0/3357551881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b8c0c6630 0x7f5b8c1efe80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400cae0 tx=0x7f5b8400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:17.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.477+0000 7f5b88ff9640 1 -- 172.21.15.112:0/3357551881 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84017070 con 0x7f5b8c0c6630 2024-04-25T01:36:17.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.477+0000 7f5b88ff9640 1 -- 172.21.15.112:0/3357551881 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b8400ac40 con 0x7f5b8c0c6630 2024-04-25T01:36:17.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.477+0000 7f5b88ff9640 1 -- 172.21.15.112:0/3357551881 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84012400 con 0x7f5b8c0c6630 2024-04-25T01:36:17.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.477+0000 7f5b92483640 1 -- 172.21.15.112:0/3357551881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8c0ac450 con 0x7f5b8c0c6630 2024-04-25T01:36:17.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.478+0000 7f5b92483640 1 -- 172.21.15.112:0/3357551881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b8c152bf0 con 0x7f5b8c0c6630 2024-04-25T01:36:17.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.480+0000 7f5b88ff9640 1 -- 172.21.15.112:0/3357551881 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b8401a070 con 0x7f5b8c0c6630 2024-04-25T01:36:17.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.481+0000 7f5b88ff9640 1 --2- 172.21.15.112:0/3357551881 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b7806bf00 0x7f5b7806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.481+0000 7f5b88ff9640 1 -- 172.21.15.112:0/3357551881 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f5b8408b040 con 0x7f5b8c0c6630 2024-04-25T01:36:17.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.482+0000 7f5b8bfff640 1 --2- 172.21.15.112:0/3357551881 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b7806bf00 0x7f5b7806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.483+0000 7f5b8bfff640 1 --2- 172.21.15.112:0/3357551881 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b7806bf00 0x7f5b7806e3c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5b80006fd0 tx=0x7f5b80008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:17.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.483+0000 7f5b92483640 1 --2- 172.21.15.112:0/3357551881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c15b160 0x7f5b8c1f03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.483+0000 7f5b92483640 1 -- 172.21.15.112:0/3357551881 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c1f0900 con 0x7f5b8c15b160 2024-04-25T01:36:17.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.484+0000 7f5b8b7fe640 1 --2- 172.21.15.112:0/3357551881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c15b160 0x7f5b8c1f03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.484+0000 7f5b8b7fe640 1 -- 172.21.15.112:0/3357551881 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b8c075980 con 0x7f5b8c15b160 2024-04-25T01:36:17.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.485+0000 7f5b8b7fe640 1 --2- 172.21.15.112:0/3357551881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c15b160 0x7f5b8c1f03c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0755b0 tx=0x7f5b7c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:17.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.485+0000 7f5b88ff9640 1 -- 172.21.15.112:0/3357551881 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c042070 con 0x7f5b8c15b160 2024-04-25T01:36:17.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.487+0000 7f5b88ff9640 1 -- 172.21.15.112:0/3357551881 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5b7c03e660 con 0x7f5b8c15b160 2024-04-25T01:36:17.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.487+0000 7f5b88ff9640 1 -- 172.21.15.112:0/3357551881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c15b160 msgr2=0x7f5b8c1f03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.487+0000 7f5b88ff9640 1 --2- 172.21.15.112:0/3357551881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c15b160 0x7f5b8c1f03c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0755b0 tx=0x7f5b7c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.487+0000 7f5b88ff9640 1 -- 172.21.15.112:0/3357551881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c15b160 msgr2=0x7f5b8c1f03c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:36:17.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.487+0000 7f5b88ff9640 1 --2- 172.21.15.112:0/3357551881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c15b160 0x7f5b8c1f03c0 secure :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0755b0 tx=0x7f5b7c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.500+0000 7f5b92483640 1 --2- 172.21.15.112:0/3357551881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c14b8c0 0x7f5b8c14bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.500+0000 7f5b92483640 1 -- 172.21.15.112:0/3357551881 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c152e30 con 0x7f5b8c14b8c0 2024-04-25T01:36:17.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.501+0000 7f5b8b7fe640 1 --2- 172.21.15.112:0/3357551881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c14b8c0 0x7f5b8c14bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.502+0000 7f5b8b7fe640 1 -- 172.21.15.112:0/3357551881 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5b8c0689b0 con 0x7f5b8c14b8c0 2024-04-25T01:36:17.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.502+0000 7f5b8b7fe640 1 --2- 172.21.15.112:0/3357551881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c14b8c0 0x7f5b8c14bcd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c02a220 tx=0x7f5b7c008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:17.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.503+0000 7f5b88ff9640 1 -- 172.21.15.112:0/3357551881 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c042070 con 0x7f5b8c14b8c0 2024-04-25T01:36:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.503+0000 7f5b88ff9640 1 -- 172.21.15.112:0/3357551881 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5069 (secure 0 0 0) 0x7f5b7c003820 con 0x7f5b8c14b8c0 2024-04-25T01:36:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.503+0000 7f5b88ff9640 1 -- 172.21.15.112:0/3357551881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c14b8c0 msgr2=0x7f5b8c14bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.503+0000 7f5b88ff9640 1 --2- 172.21.15.112:0/3357551881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c14b8c0 0x7f5b8c14bcd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c02a220 tx=0x7f5b7c008030 comp rx=0 tx=0).stop 2024-04-25T01:36:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.503+0000 7f5b88ff9640 1 -- 172.21.15.112:0/3357551881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c14b8c0 msgr2=0x7f5b8c14bcd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:36:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.503+0000 7f5b88ff9640 1 --2- 172.21.15.112:0/3357551881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c14b8c0 0x7f5b8c14bcd0 secure :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c02a220 tx=0x7f5b7c008030 comp rx=0 tx=0).stop 2024-04-25T01:36:17.482 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:36:17.482 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:36:17.482 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:36:17.482 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:36:17.482 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 90, 2024-04-25T01:36:17.483 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 413192, 2024-04-25T01:36:17.483 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:36:17.483 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:36:17.483 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:36:17.483 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:36:17.483 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:17.483 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 2, 2024-04-25T01:36:17.483 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:36:17.483 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:36:17.483 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:36:17.483 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:17.483 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:17.483 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:17.483 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:17.484 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:17.484 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:17.484 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:17.484 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:17.484 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:17.484 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:17.484 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:17.484 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:17.484 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:36:17.484 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:36:17.484 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:17.484 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:17.484 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:17.484 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:17.484 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:17.485 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:17.485 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:17.485 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:17.485 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:17.485 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:17.485 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:17.485 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:17.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.485 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:36:17.485 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:36:17.485 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:36:17.485 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:36:17.485 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:36:17.485 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:36:17.486 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:36:17.486 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:36:17.486 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:36:17.486 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:36:17.486 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:36:17.486 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:36:17.486 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:36:17.486 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:36:17.486 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:36:17.486 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:36:17.486 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:36:17.486 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:17.486 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:17.486 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:17.487 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:17.487 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:17.487 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:17.487 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:17.487 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:17.487 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:17.487 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:17.487 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:17.487 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:17.487 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:36:17.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.487 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:36:17.487 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.487 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:36:17.487 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:36:17.488 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:17.488 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:17.488 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.488 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:17.488 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:36:17.488 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:17.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.488 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.488 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:17.488 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:36:17.488 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:17.488 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.488 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:17.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.489 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:36:17.489 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:36:17.489 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:17.489 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:17.489 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:17.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.489 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.489 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:36:17.489 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:36:17.489 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:17.489 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:17.489 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.489 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:17.489 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:36:17.489 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:17.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.490 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.490 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:17.490 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:36:17.490 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:17.490 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.490 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:17.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.490 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:36:17.490 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:36:17.490 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:17.490 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:17.490 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:17.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.490 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.491 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:36:17.491 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:36:17.491 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:17.491 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:17.491 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.491 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:17.491 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:36:17.491 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:17.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.491 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.491 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:17.491 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:36:17.491 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:17.491 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.491 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:17.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.492 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:36:17.492 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:36:17.492 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:17.492 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:17.492 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:17.492 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.492 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:17.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.492 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:36:17.492 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:36:17.492 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.492 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:17.492 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:17.493 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:36:17.493 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.493 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:17.493 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:36:17.493 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.493 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:17.493 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:17.493 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:36:17.493 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.493 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:17.493 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:36:17.493 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.493 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:17.493 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:17.493 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 8 2024-04-25T01:36:17.494 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.494 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:17.494 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:36:17.494 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.494 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:17.494 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:17.494 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 6 2024-04-25T01:36:17.494 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.494 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:17.494 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:36:17.494 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.494 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:17.494 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:17.494 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:36:17.495 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.495 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:17.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.495 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:36:17.495 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:36:17.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.507+0000 7f5b92483640 1 -- 172.21.15.112:0/3357551881 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b7806bf00 msgr2=0x7f5b7806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.507+0000 7f5b92483640 1 --2- 172.21.15.112:0/3357551881 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5b7806bf00 0x7f5b7806e3c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5b80006fd0 tx=0x7f5b80008040 comp rx=0 tx=0).stop 2024-04-25T01:36:17.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.508+0000 7f5b92483640 1 -- 172.21.15.112:0/3357551881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b8c0c6630 msgr2=0x7f5b8c1efe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.508+0000 7f5b92483640 1 --2- 172.21.15.112:0/3357551881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b8c0c6630 0x7f5b8c1efe80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400cae0 tx=0x7f5b8400a060 comp rx=0 tx=0).stop 2024-04-25T01:36:17.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.508+0000 7f5b90a20640 1 -- 172.21.15.112:0/3357551881 reap_dead start 2024-04-25T01:36:17.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.508+0000 7f5b90a20640 1 -- 172.21.15.112:0/3357551881 reap_dead start 2024-04-25T01:36:17.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.508+0000 7f5b92483640 1 -- 172.21.15.112:0/3357551881 shutdown_connections 2024-04-25T01:36:17.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.508+0000 7f5b92483640 1 -- 172.21.15.112:0/3357551881 >> 172.21.15.112:0/3357551881 conn(0x7f5b8c0c42d0 msgr2=0x7f5b8c15cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:17.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.508+0000 7f5b92483640 1 -- 172.21.15.112:0/3357551881 shutdown_connections 2024-04-25T01:36:17.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.508+0000 7f5b92483640 1 -- 172.21.15.112:0/3357551881 wait complete. 2024-04-25T01:36:17.501 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:36:17.501 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:36:17.501 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:36:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.692+0000 7f087dd61640 1 Processor -- start 2024-04-25T01:36:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.692+0000 7f087dd61640 1 -- start start 2024-04-25T01:36:17.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.693+0000 7f087dd61640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f08780c1e50 0x7f08780c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.693+0000 7f087dd61640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f087806e400 0x7f08781601b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.693+0000 7f087dd61640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08780abfc0 con 0x7f08780abbe0 2024-04-25T01:36:17.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.693+0000 7f087dd61640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f08780c25c0 con 0x7f08780c1e50 2024-04-25T01:36:17.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.693+0000 7f087dd61640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f08780c22e0 con 0x7f087806e400 2024-04-25T01:36:17.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.693+0000 7f0876ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f08780c1e50 0x7f08780c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.693+0000 7f0876ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f08780c1e50 0x7f08780c3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57914/0 (socket says 172.21.15.112:57914) 2024-04-25T01:36:17.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.694+0000 7f0876ffd640 1 -- 172.21.15.112:0/4064714311 learned_addr learned my addr 172.21.15.112:0/4064714311 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:36:17.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.694+0000 7f0876ffd640 1 -- 172.21.15.112:0/4064714311 >> v2:172.21.15.187:3300/0 conn(0x7f087806e400 msgr2=0x7f08781601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:17.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.694+0000 7f0876ffd640 1 --2- 172.21.15.112:0/4064714311 >> v2:172.21.15.187:3300/0 conn(0x7f087806e400 0x7f08781601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.694+0000 7f0876ffd640 1 -- 172.21.15.112:0/4064714311 >> v1:172.21.15.187:6789/0 conn(0x7f08780abbe0 legacy=0x7f0878158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:17.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.694+0000 7f0876ffd640 1 -- 172.21.15.112:0/4064714311 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08781559d0 con 0x7f08780c1e50 2024-04-25T01:36:17.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.695+0000 7f0876ffd640 1 --2- 172.21.15.112:0/4064714311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08780c1e50 0x7f08780c3ee0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0878073900 tx=0x7f086c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bbdf5299987769 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:17.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.696+0000 7f08767fc640 1 -- 172.21.15.112:0/4064714311 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c037070 con 0x7f08780c1e50 2024-04-25T01:36:17.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.696+0000 7f08767fc640 1 -- 172.21.15.112:0/4064714311 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f086c032440 con 0x7f08780c1e50 2024-04-25T01:36:17.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.696+0000 7f08767fc640 1 -- 172.21.15.112:0/4064714311 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c032810 con 0x7f08780c1e50 2024-04-25T01:36:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.697+0000 7f087dd61640 1 -- 172.21.15.112:0/4064714311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08780c1e50 msgr2=0x7f08780c3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.697+0000 7f087dd61640 1 --2- 172.21.15.112:0/4064714311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08780c1e50 0x7f08780c3ee0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0878073900 tx=0x7f086c02a1a0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.697+0000 7f087dd61640 1 -- 172.21.15.112:0/4064714311 shutdown_connections 2024-04-25T01:36:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.697+0000 7f087dd61640 1 --2- 172.21.15.112:0/4064714311 >> v2:172.21.15.187:3300/0 conn(0x7f087806e400 0x7f08781601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.697+0000 7f087dd61640 1 --2- 172.21.15.112:0/4064714311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08780c1e50 0x7f08780c3ee0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.697+0000 7f087dd61640 1 -- 172.21.15.112:0/4064714311 >> 172.21.15.112:0/4064714311 conn(0x7f08780747f0 msgr2=0x7f08780c7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.698+0000 7f087dd61640 1 -- 172.21.15.112:0/4064714311 shutdown_connections 2024-04-25T01:36:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.698+0000 7f087dd61640 1 -- 172.21.15.112:0/4064714311 wait complete. 2024-04-25T01:36:17.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.699+0000 7f087dd61640 1 Processor -- start 2024-04-25T01:36:17.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.700+0000 7f087dd61640 1 -- start start 2024-04-25T01:36:17.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.700+0000 7f087dd61640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f087806e400 0x7f08780c0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.701+0000 7f087dd61640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08780abbe0 0x7f0878152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.701+0000 7f087dd61640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08780a8cc0 0x7f08781527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.701+0000 7f087dd61640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0878159530 con 0x7f08780a8cc0 2024-04-25T01:36:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.701+0000 7f087dd61640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0878155f90 con 0x7f087806e400 2024-04-25T01:36:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.701+0000 7f087dd61640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f08780ac880 con 0x7f08780abbe0 2024-04-25T01:36:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.701+0000 7f0877fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08780a8cc0 0x7f08781527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.701+0000 7f0877fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08780a8cc0 0x7f08781527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57926/0 (socket says 172.21.15.112:57926) 2024-04-25T01:36:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.701+0000 7f0877fff640 1 -- 172.21.15.112:0/1082830708 learned_addr learned my addr 172.21.15.112:0/1082830708 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:36:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.701+0000 7f08777fe640 1 --2- 172.21.15.112:0/1082830708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f087806e400 0x7f08780c0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.702+0000 7f0877fff640 1 -- 172.21.15.112:0/1082830708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08780abbe0 msgr2=0x7f0878152290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:36:17.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.702+0000 7f0877fff640 1 --2- 172.21.15.112:0/1082830708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08780abbe0 0x7f0878152290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.702+0000 7f0877fff640 1 -- 172.21.15.112:0/1082830708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f087806e400 msgr2=0x7f08780c0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.702+0000 7f0877fff640 1 --2- 172.21.15.112:0/1082830708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f087806e400 0x7f08780c0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.702+0000 7f0877fff640 1 -- 172.21.15.112:0/1082830708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0878152d10 con 0x7f08780a8cc0 2024-04-25T01:36:17.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.703+0000 7f0877fff640 1 --2- 172.21.15.112:0/1082830708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08780a8cc0 0x7f08781527d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f087000bd90 tx=0x7f087000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:17.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.703+0000 7f085ffff640 1 -- 172.21.15.112:0/1082830708 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870019070 con 0x7f08780a8cc0 2024-04-25T01:36:17.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.703+0000 7f087dd61640 1 -- 172.21.15.112:0/1082830708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08781eb880 con 0x7f08780a8cc0 2024-04-25T01:36:17.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.703+0000 7f087dd61640 1 -- 172.21.15.112:0/1082830708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08781ebe40 con 0x7f08780a8cc0 2024-04-25T01:36:17.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.704+0000 7f085ffff640 1 -- 172.21.15.112:0/1082830708 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f087000cc60 con 0x7f08780a8cc0 2024-04-25T01:36:17.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.704+0000 7f085ffff640 1 -- 172.21.15.112:0/1082830708 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870014660 con 0x7f08780a8cc0 2024-04-25T01:36:17.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.705+0000 7f087dd61640 1 --2- 172.21.15.112:0/1082830708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0858002a70 0x7f0858004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.705+0000 7f087dd61640 1 -- 172.21.15.112:0/1082830708 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0858007ec0 con 0x7f0858002a70 2024-04-25T01:36:17.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.705+0000 7f08777fe640 1 --2- 172.21.15.112:0/1082830708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0858002a70 0x7f0858004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.706+0000 7f085ffff640 1 -- 172.21.15.112:0/1082830708 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f087001c070 con 0x7f08780a8cc0 2024-04-25T01:36:17.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.706+0000 7f085ffff640 1 --2- 172.21.15.112:0/1082830708 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f086406be60 0x7f086406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.706+0000 7f085ffff640 1 -- 172.21.15.112:0/1082830708 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f087008ae20 con 0x7f08780a8cc0 2024-04-25T01:36:17.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.707+0000 7f0876ffd640 1 --2- 172.21.15.112:0/1082830708 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f086406be60 0x7f086406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.708+0000 7f08777fe640 1 -- 172.21.15.112:0/1082830708 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f08580080c0 con 0x7f0858002a70 2024-04-25T01:36:17.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.708+0000 7f0876ffd640 1 --2- 172.21.15.112:0/1082830708 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f086406be60 0x7f086406e320 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f086c034000 tx=0x7f086c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:17.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.708+0000 7f08777fe640 1 --2- 172.21.15.112:0/1082830708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0858002a70 0x7f0858004f30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f086800f520 tx=0x7f086800f9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.711+0000 7f085ffff640 1 -- 172.21.15.112:0/1082830708 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08680052e0 con 0x7f0858002a70 2024-04-25T01:36:17.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.712+0000 7f085ffff640 1 -- 172.21.15.112:0/1082830708 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0868018070 con 0x7f0858002a70 2024-04-25T01:36:17.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.712+0000 7f085ffff640 1 -- 172.21.15.112:0/1082830708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0858002a70 msgr2=0x7f0858004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.712+0000 7f085ffff640 1 --2- 172.21.15.112:0/1082830708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0858002a70 0x7f0858004f30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f086800f520 tx=0x7f086800f9e0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.712+0000 7f085ffff640 1 -- 172.21.15.112:0/1082830708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0858002a70 msgr2=0x7f0858004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:36:17.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.712+0000 7f085ffff640 1 --2- 172.21.15.112:0/1082830708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0858002a70 0x7f0858004f30 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f086800f520 tx=0x7f086800f9e0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.725+0000 7f087dd61640 1 --2- 172.21.15.112:0/1082830708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0858006eb0 0x7f0858007340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.725+0000 7f087dd61640 1 -- 172.21.15.112:0/1082830708 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f085800a4a0 con 0x7f0858006eb0 2024-04-25T01:36:17.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.726+0000 7f08777fe640 1 --2- 172.21.15.112:0/1082830708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0858006eb0 0x7f0858007340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.727+0000 7f08777fe640 1 -- 172.21.15.112:0/1082830708 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0858007880 con 0x7f0858006eb0 2024-04-25T01:36:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.727+0000 7f08777fe640 1 --2- 172.21.15.112:0/1082830708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0858006eb0 0x7f0858007340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f086800f520 tx=0x7f0868001020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:17.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.728+0000 7f085ffff640 1 -- 172.21.15.112:0/1082830708 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08680052e0 con 0x7f0858006eb0 2024-04-25T01:36:17.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.728+0000 7f085ffff640 1 -- 172.21.15.112:0/1082830708 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4558 (secure 0 0 0) 0x7f0868003ad0 con 0x7f0858006eb0 2024-04-25T01:36:17.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.728+0000 7f085ffff640 1 -- 172.21.15.112:0/1082830708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0858006eb0 msgr2=0x7f0858007340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.728+0000 7f085ffff640 1 --2- 172.21.15.112:0/1082830708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0858006eb0 0x7f0858007340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f086800f520 tx=0x7f0868001020 comp rx=0 tx=0).stop 2024-04-25T01:36:17.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.728+0000 7f085ffff640 1 -- 172.21.15.112:0/1082830708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0858006eb0 msgr2=0x7f0858007340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:36:17.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.728+0000 7f085ffff640 1 --2- 172.21.15.112:0/1082830708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0858006eb0 0x7f0858007340 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f086800f520 tx=0x7f0868001020 comp rx=0 tx=0).stop 2024-04-25T01:36:17.705 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:36:17.705 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:36:17.705 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:36:17.705 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:36:17.705 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 90, 2024-04-25T01:36:17.705 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 88388, 2024-04-25T01:36:17.705 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:36:17.705 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:36:17.705 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:36:17.705 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:36:17.706 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:17.706 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 3, 2024-04-25T01:36:17.706 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:36:17.706 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:36:17.706 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:36:17.706 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:17.706 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:17.706 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:17.706 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:17.706 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:17.706 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:17.706 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:17.706 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:17.706 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:17.706 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:17.707 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:17.707 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:17.707 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:36:17.707 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:36:17.707 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:17.707 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:17.707 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:17.707 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:17.707 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:17.707 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:17.707 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:17.707 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:17.707 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:17.707 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:17.707 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:17.707 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:17.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.708 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:36:17.708 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:36:17.708 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:36:17.708 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:36:17.708 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:36:17.708 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:36:17.708 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:36:17.708 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:36:17.708 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:36:17.708 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:36:17.708 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:36:17.708 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:36:17.708 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:36:17.709 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:36:17.709 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:36:17.709 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:36:17.709 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:36:17.709 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:17.709 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:17.709 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:17.709 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:17.709 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:17.709 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:17.709 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:17.709 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:17.709 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:17.709 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:17.710 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:17.710 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:17.710 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:36:17.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.710 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:36:17.710 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.710 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:36:17.710 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:36:17.710 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:17.710 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:17.710 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.710 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:17.710 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:36:17.710 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:17.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.711 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.711 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:17.711 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:36:17.711 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:17.711 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.711 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:17.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.711 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:36:17.711 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:36:17.711 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:17.711 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:17.711 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:17.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.711 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.712 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:36:17.712 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:36:17.712 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:17.712 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:17.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.712 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:17.712 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:36:17.712 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:17.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.712 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:17.712 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:36:17.712 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:17.712 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.712 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:36:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:36:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:36:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:36:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.714 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:17.714 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:36:17.714 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:17.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.714 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.714 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:17.714 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:36:17.714 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:17.714 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.714 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:17.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.714 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:36:17.714 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:36:17.714 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:17.715 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:17.715 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:17.715 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.715 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:17.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.715 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:36:17.715 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:36:17.715 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.715 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:17.715 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:17.715 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:36:17.715 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.715 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:17.715 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:36:17.715 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:17.716 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:17.716 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:17.716 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:36:17.716 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:17.716 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:17.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:17.716 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:36:17.716 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:36:17.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.729+0000 7f087dd61640 1 -- 172.21.15.112:0/1082830708 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f086406be60 msgr2=0x7f086406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.729+0000 7f087dd61640 1 --2- 172.21.15.112:0/1082830708 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f086406be60 0x7f086406e320 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f086c034000 tx=0x7f086c02e000 comp rx=0 tx=0).stop 2024-04-25T01:36:17.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.729+0000 7f087dd61640 1 -- 172.21.15.112:0/1082830708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08780a8cc0 msgr2=0x7f08781527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.729+0000 7f087dd61640 1 --2- 172.21.15.112:0/1082830708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08780a8cc0 0x7f08781527d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f087000bd90 tx=0x7f087000be80 comp rx=0 tx=0).stop 2024-04-25T01:36:17.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.729+0000 7f0877fff640 1 -- 172.21.15.112:0/1082830708 reap_dead start 2024-04-25T01:36:17.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.729+0000 7f0877fff640 1 -- 172.21.15.112:0/1082830708 reap_dead start 2024-04-25T01:36:17.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.729+0000 7f087dd61640 1 -- 172.21.15.112:0/1082830708 shutdown_connections 2024-04-25T01:36:17.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.729+0000 7f087dd61640 1 -- 172.21.15.112:0/1082830708 >> 172.21.15.112:0/1082830708 conn(0x7f08780747f0 msgr2=0x7f08780ab1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:17.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.730+0000 7f087dd61640 1 -- 172.21.15.112:0/1082830708 shutdown_connections 2024-04-25T01:36:17.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.730+0000 7f087dd61640 1 -- 172.21.15.112:0/1082830708 wait complete. 2024-04-25T01:36:17.722 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:36:17.723 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T01:36:17.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T01:36:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.901+0000 7f23d8790640 1 Processor -- start 2024-04-25T01:36:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.901+0000 7f23d8790640 1 -- start start 2024-04-25T01:36:17.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.903+0000 7f23d5d2b640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f23d0073490 0x7f23d006c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38152/0 (socket says 172.21.15.112:38152) 2024-04-25T01:36:17.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.903+0000 7f23d5d2b640 1 -- 172.21.15.112:0/302417667 learned_addr learned my addr 172.21.15.112:0/302417667 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:36:17.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.903+0000 7f23d8790640 1 --2- 172.21.15.112:0/302417667 >> v2:172.21.15.187:3300/0 conn(0x7f23d006deb0 0x7f23d0151e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.903+0000 7f23d8790640 1 -- 172.21.15.112:0/302417667 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23d00c6fa0 con 0x7f23d0073490 2024-04-25T01:36:17.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.903+0000 7f23d8790640 1 -- 172.21.15.112:0/302417667 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23d00c7280 con 0x7f23d00c3c90 2024-04-25T01:36:17.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.903+0000 7f23d8790640 1 -- 172.21.15.112:0/302417667 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f23d01523d0 con 0x7f23d006deb0 2024-04-25T01:36:17.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.903+0000 7f23d6d2d640 1 --2- 172.21.15.112:0/302417667 >> v2:172.21.15.187:3300/0 conn(0x7f23d006deb0 0x7f23d0151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.903+0000 7f23d552a640 1 -- 172.21.15.112:0/302417667 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3899550454 0 0) 0x7f23d00c6fa0 con 0x7f23d0073490 2024-04-25T01:36:17.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.903+0000 7f23d552a640 1 -- 172.21.15.112:0/302417667 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23bc003520 con 0x7f23d0073490 2024-04-25T01:36:17.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.903+0000 7f23d552a640 1 -- 172.21.15.112:0/302417667 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2023866754 0 0) 0x7f23d00c7280 con 0x7f23d00c3c90 2024-04-25T01:36:17.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.903+0000 7f23d552a640 1 -- 172.21.15.112:0/302417667 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23d00c6fa0 con 0x7f23d00c3c90 2024-04-25T01:36:17.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d552a640 1 -- 172.21.15.112:0/302417667 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 682578197 0 0) 0x7f23bc003520 con 0x7f23d0073490 2024-04-25T01:36:17.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d552a640 1 -- 172.21.15.112:0/302417667 >> v2:172.21.15.187:3300/0 conn(0x7f23d006deb0 msgr2=0x7f23d0151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:17.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d552a640 1 --2- 172.21.15.112:0/302417667 >> v2:172.21.15.187:3300/0 conn(0x7f23d006deb0 0x7f23d0151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d552a640 1 -- 172.21.15.112:0/302417667 >> v1:172.21.15.187:6789/0 conn(0x7f23d00c3c90 legacy=0x7f23d0155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d552a640 1 -- 172.21.15.112:0/302417667 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d00c7560 con 0x7f23d0073490 2024-04-25T01:36:17.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d552a640 1 -- 172.21.15.112:0/302417667 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f23c80019f0 con 0x7f23d0073490 2024-04-25T01:36:17.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d552a640 1 -- 172.21.15.112:0/302417667 >> v1:172.21.15.112:6789/0 conn(0x7f23d0073490 legacy=0x7f23d006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d552a640 1 --2- 172.21.15.112:0/302417667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23bc003b40 0x7f23bc003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d552a640 1 --2- 172.21.15.112:0/302417667 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23bc004800 0x7f23bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d552a640 1 --2- 172.21.15.112:0/302417667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23bc0051c0 0x7f23bc00fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d552a640 1 -- 172.21.15.112:0/302417667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc0104d0 con 0x7f23bc0051c0 2024-04-25T01:36:17.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d552a640 1 -- 172.21.15.112:0/302417667 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc010680 con 0x7f23bc003b40 2024-04-25T01:36:17.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d552a640 1 -- 172.21.15.112:0/302417667 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f23bc010880 con 0x7f23bc004800 2024-04-25T01:36:17.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d6d2d640 1 --2- 172.21.15.112:0/302417667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23bc0051c0 0x7f23bc00fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d5d2b640 1 --2- 172.21.15.112:0/302417667 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23bc004800 0x7f23bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d652c640 1 --2- 172.21.15.112:0/302417667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23bc003b40 0x7f23bc003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d6d2d640 1 -- 172.21.15.112:0/302417667 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23bc004800 msgr2=0x7f23bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:17.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d6d2d640 1 --2- 172.21.15.112:0/302417667 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23bc004800 0x7f23bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d6d2d640 1 -- 172.21.15.112:0/302417667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23bc003b40 msgr2=0x7f23bc003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:17.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d6d2d640 1 --2- 172.21.15.112:0/302417667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23bc003b40 0x7f23bc003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.904+0000 7f23d6d2d640 1 -- 172.21.15.112:0/302417667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23bc003520 con 0x7f23bc0051c0 2024-04-25T01:36:17.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.905+0000 7f23d6d2d640 1 -- 172.21.15.112:0/302417667 reap_dead start 2024-04-25T01:36:17.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.906+0000 7f23d6d2d640 1 --2- 172.21.15.112:0/302417667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23bc0051c0 0x7f23bc00fdd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009aa0 tx=0x7f23bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6bf2f7688b3fad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:17.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.906+0000 7f23d552a640 1 -- 172.21.15.112:0/302417667 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc036070 con 0x7f23bc0051c0 2024-04-25T01:36:17.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.906+0000 7f23d552a640 1 -- 172.21.15.112:0/302417667 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23cc031400 con 0x7f23bc0051c0 2024-04-25T01:36:17.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.906+0000 7f23d552a640 1 -- 172.21.15.112:0/302417667 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc009670 con 0x7f23bc0051c0 2024-04-25T01:36:17.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.908+0000 7f23d8790640 1 -- 172.21.15.112:0/302417667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23bc0051c0 msgr2=0x7f23bc00fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.908+0000 7f23d8790640 1 --2- 172.21.15.112:0/302417667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23bc0051c0 0x7f23bc00fdd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009aa0 tx=0x7f23bc005830 comp rx=0 tx=0).stop 2024-04-25T01:36:17.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.909+0000 7f23d8790640 1 -- 172.21.15.112:0/302417667 shutdown_connections 2024-04-25T01:36:17.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.909+0000 7f23d8790640 1 --2- 172.21.15.112:0/302417667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23bc0051c0 0x7f23bc00fdd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.909+0000 7f23d8790640 1 -- 172.21.15.112:0/302417667 >> 172.21.15.112:0/302417667 conn(0x7f23d00c42d0 msgr2=0x7f23d00c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:17.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.909+0000 7f23d8790640 1 -- 172.21.15.112:0/302417667 shutdown_connections 2024-04-25T01:36:17.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.909+0000 7f23d8790640 1 -- 172.21.15.112:0/302417667 wait complete. 2024-04-25T01:36:17.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.911+0000 7f23d8790640 1 Processor -- start 2024-04-25T01:36:17.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.911+0000 7f23d8790640 1 -- start start 2024-04-25T01:36:17.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.912+0000 7f23d8790640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23bc0051c0 0x7f23d00bca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.913+0000 7f23d8790640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d01e7690 0x7f23d014e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.913+0000 7f23d8790640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d014f090 0x7f23d014ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.913+0000 7f23d8790640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23d00c4070 con 0x7f23bc0051c0 2024-04-25T01:36:17.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.913+0000 7f23d8790640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f23d0155780 con 0x7f23d014f090 2024-04-25T01:36:17.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.913+0000 7f23d8790640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f23d00c1e20 con 0x7f23d01e7690 2024-04-25T01:36:17.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.913+0000 7f23d5d2b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d01e7690 0x7f23d014e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.913+0000 7f23d5d2b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d01e7690 0x7f23d014e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:41304/0 (socket says 172.21.15.112:41304) 2024-04-25T01:36:17.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.913+0000 7f23d5d2b640 1 -- 172.21.15.112:0/4199189410 learned_addr learned my addr 172.21.15.112:0/4199189410 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:36:17.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.913+0000 7f23d6d2d640 1 --2- 172.21.15.112:0/4199189410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d014f090 0x7f23d014ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.913+0000 7f23d652c640 1 --2- 172.21.15.112:0/4199189410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23bc0051c0 0x7f23d00bca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.914+0000 7f23d5d2b640 1 -- 172.21.15.112:0/4199189410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d014f090 msgr2=0x7f23d014ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.914+0000 7f23d5d2b640 1 --2- 172.21.15.112:0/4199189410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d014f090 0x7f23d014ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.914+0000 7f23d5d2b640 1 -- 172.21.15.112:0/4199189410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23bc0051c0 msgr2=0x7f23d00bca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:17.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.914+0000 7f23d5d2b640 1 --2- 172.21.15.112:0/4199189410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23bc0051c0 0x7f23d00bca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:17.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.914+0000 7f23d5d2b640 1 -- 172.21.15.112:0/4199189410 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d00aa5b0 con 0x7f23d01e7690 2024-04-25T01:36:17.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.914+0000 7f23d5d2b640 1 --2- 172.21.15.112:0/4199189410 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d01e7690 0x7f23d014e4f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23c8001550 tx=0x7f23c800a310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:17.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.915+0000 7f23c6ffd640 1 -- 172.21.15.112:0/4199189410 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8018d80 con 0x7f23d01e7690 2024-04-25T01:36:17.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.915+0000 7f23d8790640 1 -- 172.21.15.112:0/4199189410 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d014fdc0 con 0x7f23d01e7690 2024-04-25T01:36:17.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.915+0000 7f23c6ffd640 1 -- 172.21.15.112:0/4199189410 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c801c070 con 0x7f23d01e7690 2024-04-25T01:36:17.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.915+0000 7f23c6ffd640 1 -- 172.21.15.112:0/4199189410 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8026070 con 0x7f23d01e7690 2024-04-25T01:36:17.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.916+0000 7f23d8790640 1 -- 172.21.15.112:0/4199189410 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d0150380 con 0x7f23d01e7690 2024-04-25T01:36:17.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.918+0000 7f23c4ff9640 1 -- 172.21.15.112:0/4199189410 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23ac005290 con 0x7f23d01e7690 2024-04-25T01:36:17.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.919+0000 7f23c6ffd640 1 -- 172.21.15.112:0/4199189410 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f23c800abc0 con 0x7f23d01e7690 2024-04-25T01:36:17.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.919+0000 7f23c6ffd640 1 --2- 172.21.15.112:0/4199189410 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f23b406be30 0x7f23b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:17.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.919+0000 7f23c6ffd640 1 -- 172.21.15.112:0/4199189410 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f23c8093fb0 con 0x7f23d01e7690 2024-04-25T01:36:17.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.920+0000 7f23d652c640 1 --2- 172.21.15.112:0/4199189410 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f23b406be30 0x7f23b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:17.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.921+0000 7f23d652c640 1 --2- 172.21.15.112:0/4199189410 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f23b406be30 0x7f23b406e2f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f23c0007850 tx=0x7f23c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:17.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:17.927+0000 7f23c6ffd640 1 -- 172.21.15.112:0/4199189410 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f23c80625b0 con 0x7f23d01e7690 2024-04-25T01:36:18.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:18.407+0000 7f23c4ff9640 1 -- 172.21.15.112:0/4199189410 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f23ac005d90 con 0x7f23d01e7690 2024-04-25T01:36:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:18.411+0000 7f23c6ffd640 1 -- 172.21.15.112:0/4199189410 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f23c8061eb0 con 0x7f23d01e7690 2024-04-25T01:36:18.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:18.415+0000 7f23c4ff9640 1 -- 172.21.15.112:0/4199189410 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f23b406be30 msgr2=0x7f23b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:18.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:18.415+0000 7f23c4ff9640 1 --2- 172.21.15.112:0/4199189410 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f23b406be30 0x7f23b406e2f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f23c0007850 tx=0x7f23c0009040 comp rx=0 tx=0).stop 2024-04-25T01:36:18.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:18.415+0000 7f23c4ff9640 1 -- 172.21.15.112:0/4199189410 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d01e7690 msgr2=0x7f23d014e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:18.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:18.415+0000 7f23c4ff9640 1 --2- 172.21.15.112:0/4199189410 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d01e7690 0x7f23d014e4f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23c8001550 tx=0x7f23c800a310 comp rx=0 tx=0).stop 2024-04-25T01:36:18.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:18.415+0000 7f23c4ff9640 1 -- 172.21.15.112:0/4199189410 shutdown_connections 2024-04-25T01:36:18.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:18.415+0000 7f23c4ff9640 1 --2- 172.21.15.112:0/4199189410 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f23b406be30 0x7f23b406e2f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:18.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:18.415+0000 7f23c4ff9640 1 --2- 172.21.15.112:0/4199189410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d014f090 0x7f23d014ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:18.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:18.415+0000 7f23c4ff9640 1 --2- 172.21.15.112:0/4199189410 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d01e7690 0x7f23d014e4f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:18.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:18.415+0000 7f23c4ff9640 1 --2- 172.21.15.112:0/4199189410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23bc0051c0 0x7f23d00bca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:18.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:18.415+0000 7f23c4ff9640 1 -- 172.21.15.112:0/4199189410 >> 172.21.15.112:0/4199189410 conn(0x7f23d00c42d0 msgr2=0x7f23d006ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:18.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:18.415+0000 7f23c4ff9640 1 -- 172.21.15.112:0/4199189410 shutdown_connections 2024-04-25T01:36:18.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:18.415+0000 7f23c4ff9640 1 -- 172.21.15.112:0/4199189410 wait complete. 2024-04-25T01:36:18.409 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T01:36:27.595 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:36:27.598 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:36:28.409 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:36:28.410 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:36:28.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.591+0000 7f07dcbce640 1 Processor -- start 2024-04-25T01:36:28.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.591+0000 7f07dcbce640 1 -- start start 2024-04-25T01:36:28.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.593+0000 7f07dcbce640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f07d813ab90 0x7f07d81519a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:28.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.593+0000 7f07dcbce640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07d806c9a0 0x7f07d806cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:28.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.593+0000 7f07d5d9b640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f07d8072140 0x7f07d80c2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59850/0 (socket says 172.21.15.112:59850) 2024-04-25T01:36:28.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.593+0000 7f07d5d9b640 1 -- 172.21.15.112:0/2431706157 learned_addr learned my addr 172.21.15.112:0/2431706157 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:36:28.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.593+0000 7f07dcbce640 1 -- 172.21.15.112:0/2431706157 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07d8070cb0 con 0x7f07d8072140 2024-04-25T01:36:28.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.593+0000 7f07dcbce640 1 -- 172.21.15.112:0/2431706157 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f07d8070f90 con 0x7f07d813ab90 2024-04-25T01:36:28.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.593+0000 7f07dcbce640 1 -- 172.21.15.112:0/2431706157 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f07d8071270 con 0x7f07d806c9a0 2024-04-25T01:36:28.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.593+0000 7f07d6d9d640 1 --2- 172.21.15.112:0/2431706157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07d806c9a0 0x7f07d806cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:28.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.594+0000 7f07d659c640 1 --2- 172.21.15.112:0/2431706157 >> v2:172.21.15.112:3300/0 conn(0x7f07d813ab90 0x7f07d81519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:28.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.594+0000 7f07d559a640 1 -- 172.21.15.112:0/2431706157 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3661937249 0 0) 0x7f07d8070cb0 con 0x7f07d8072140 2024-04-25T01:36:28.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.594+0000 7f07d559a640 1 -- 172.21.15.112:0/2431706157 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07c0007930 con 0x7f07d8072140 2024-04-25T01:36:28.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.594+0000 7f07d659c640 1 -- 172.21.15.112:0/2431706157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07d806c9a0 msgr2=0x7f07d806cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:28.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.594+0000 7f07d659c640 1 --2- 172.21.15.112:0/2431706157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07d806c9a0 0x7f07d806cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:28.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.594+0000 7f07d659c640 1 -- 172.21.15.112:0/2431706157 >> v1:172.21.15.112:6789/0 conn(0x7f07d8072140 legacy=0x7f07d80c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:28.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.594+0000 7f07d559a640 1 -- 172.21.15.112:0/2431706157 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1893847831 0 0) 0x7f07c0007930 con 0x7f07d8072140 2024-04-25T01:36:28.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.595+0000 7f07d659c640 1 -- 172.21.15.112:0/2431706157 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d8071550 con 0x7f07d813ab90 2024-04-25T01:36:28.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.596+0000 7f07d6d9d640 1 --2- 172.21.15.112:0/2431706157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07d806c9a0 0x7f07d806cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:36:28.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.596+0000 7f07d659c640 1 --2- 172.21.15.112:0/2431706157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07d813ab90 0x7f07d81519a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f07d000be30 tx=0x7f07d000a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d165c36b1a6da535 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:28.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.596+0000 7f07d559a640 1 -- 172.21.15.112:0/2431706157 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0038070 con 0x7f07d813ab90 2024-04-25T01:36:28.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.597+0000 7f07d559a640 1 -- 172.21.15.112:0/2431706157 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07d0030480 con 0x7f07d813ab90 2024-04-25T01:36:28.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.597+0000 7f07d559a640 1 -- 172.21.15.112:0/2431706157 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c8001670 con 0x7f07d813ab90 2024-04-25T01:36:28.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.598+0000 7f07dcbce640 1 -- 172.21.15.112:0/2431706157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07d813ab90 msgr2=0x7f07d81519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:28.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.598+0000 7f07dcbce640 1 --2- 172.21.15.112:0/2431706157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07d813ab90 0x7f07d81519a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f07d000be30 tx=0x7f07d000a400 comp rx=0 tx=0).stop 2024-04-25T01:36:28.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.598+0000 7f07dcbce640 1 -- 172.21.15.112:0/2431706157 shutdown_connections 2024-04-25T01:36:28.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.598+0000 7f07dcbce640 1 --2- 172.21.15.112:0/2431706157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07d806c9a0 0x7f07d806cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:28.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.598+0000 7f07dcbce640 1 --2- 172.21.15.112:0/2431706157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07d813ab90 0x7f07d81519a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:28.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.598+0000 7f07dcbce640 1 -- 172.21.15.112:0/2431706157 >> 172.21.15.112:0/2431706157 conn(0x7f07d80c3a80 msgr2=0x7f07d80c3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:28.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.598+0000 7f07dcbce640 1 -- 172.21.15.112:0/2431706157 shutdown_connections 2024-04-25T01:36:28.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.598+0000 7f07dcbce640 1 -- 172.21.15.112:0/2431706157 wait complete. 2024-04-25T01:36:28.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.600+0000 7f07dcbce640 1 Processor -- start 2024-04-25T01:36:28.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.600+0000 7f07dcbce640 1 -- start start 2024-04-25T01:36:28.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.602+0000 7f07dcbce640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07d806c9a0 0x7f07d8134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:28.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.602+0000 7f07dcbce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07d8072140 0x7f07d80bbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:28.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.602+0000 7f07dcbce640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07d813ab90 0x7f07d8152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:28.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.602+0000 7f07d659c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07d806c9a0 0x7f07d8134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:28.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.602+0000 7f07d659c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07d806c9a0 0x7f07d8134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:36654/0 (socket says 172.21.15.112:36654) 2024-04-25T01:36:28.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.603+0000 7f07d6d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07d813ab90 0x7f07d8152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:28.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.603+0000 7f07d6d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07d813ab90 0x7f07d8152050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:42912/0 (socket says 172.21.15.112:42912) 2024-04-25T01:36:28.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.603+0000 7f07d6d9d640 1 -- 172.21.15.112:0/3624045803 learned_addr learned my addr 172.21.15.112:0/3624045803 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:36:28.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.603+0000 7f07d5d9b640 1 --2- 172.21.15.112:0/3624045803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07d8072140 0x7f07d80bbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:28.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.603+0000 7f07dcbce640 1 -- 172.21.15.112:0/3624045803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f07d80c30a0 con 0x7f07d8072140 2024-04-25T01:36:28.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.603+0000 7f07dcbce640 1 -- 172.21.15.112:0/3624045803 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f07d8071b10 con 0x7f07d813ab90 2024-04-25T01:36:28.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.603+0000 7f07dcbce640 1 -- 172.21.15.112:0/3624045803 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f07d80c3220 con 0x7f07d806c9a0 2024-04-25T01:36:28.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.603+0000 7f07d5d9b640 1 -- 172.21.15.112:0/3624045803 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07d806c9a0 msgr2=0x7f07d8134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.603+0000 7f07d5d9b640 1 --2- 172.21.15.112:0/3624045803 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07d806c9a0 0x7f07d8134ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.603+0000 7f07d5d9b640 1 -- 172.21.15.112:0/3624045803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07d813ab90 msgr2=0x7f07d8152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.603+0000 7f07d5d9b640 1 --2- 172.21.15.112:0/3624045803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07d813ab90 0x7f07d8152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.603+0000 7f07d5d9b640 1 -- 172.21.15.112:0/3624045803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d8155ad0 con 0x7f07d8072140 2024-04-25T01:36:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.604+0000 7f07d5d9b640 1 --2- 172.21.15.112:0/3624045803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07d8072140 0x7f07d80bbfa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f07c800c530 tx=0x7f07c800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.604+0000 7f07c6ffd640 1 -- 172.21.15.112:0/3624045803 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c8017070 con 0x7f07d8072140 2024-04-25T01:36:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.604+0000 7f07c6ffd640 1 -- 172.21.15.112:0/3624045803 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c8005ce0 con 0x7f07d8072140 2024-04-25T01:36:28.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.604+0000 7f07c6ffd640 1 -- 172.21.15.112:0/3624045803 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c80072b0 con 0x7f07d8072140 2024-04-25T01:36:28.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.604+0000 7f07dcbce640 1 -- 172.21.15.112:0/3624045803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d8155db0 con 0x7f07d8072140 2024-04-25T01:36:28.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.604+0000 7f07dcbce640 1 -- 172.21.15.112:0/3624045803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d8154d40 con 0x7f07d8072140 2024-04-25T01:36:28.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.606+0000 7f07dcbce640 1 -- 172.21.15.112:0/3624045803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07d8154f80 con 0x7f07d8072140 2024-04-25T01:36:28.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.607+0000 7f07c6ffd640 1 -- 172.21.15.112:0/3624045803 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f07c801a070 con 0x7f07d8072140 2024-04-25T01:36:28.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.607+0000 7f07c6ffd640 1 --2- 172.21.15.112:0/3624045803 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f07b806bf30 0x7f07b806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:28.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.608+0000 7f07c6ffd640 1 -- 172.21.15.112:0/3624045803 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f07c808a570 con 0x7f07d8072140 2024-04-25T01:36:28.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.608+0000 7f07d659c640 1 --2- 172.21.15.112:0/3624045803 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f07b806bf30 0x7f07b806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:28.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.609+0000 7f07d659c640 1 --2- 172.21.15.112:0/3624045803 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f07b806bf30 0x7f07b806e3f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f07d0035000 tx=0x7f07d002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:28.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:28.616+0000 7f07c6ffd640 1 -- 172.21.15.112:0/3624045803 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f07c80585a0 con 0x7f07d8072140 2024-04-25T01:36:29.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.112+0000 7f07dcbce640 1 -- 172.21.15.112:0/3624045803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f07d80682a0 con 0x7f07d8072140 2024-04-25T01:36:29.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.114+0000 7f07c6ffd640 1 -- 172.21.15.112:0/3624045803 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f07c805c8a0 con 0x7f07d8072140 2024-04-25T01:36:29.093 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":90,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:36:29.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.121+0000 7f07dcbce640 1 -- 172.21.15.112:0/3624045803 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f07b806bf30 msgr2=0x7f07b806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.121+0000 7f07dcbce640 1 --2- 172.21.15.112:0/3624045803 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f07b806bf30 0x7f07b806e3f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f07d0035000 tx=0x7f07d002e000 comp rx=0 tx=0).stop 2024-04-25T01:36:29.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.121+0000 7f07dcbce640 1 -- 172.21.15.112:0/3624045803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07d8072140 msgr2=0x7f07d80bbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.121+0000 7f07dcbce640 1 --2- 172.21.15.112:0/3624045803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07d8072140 0x7f07d80bbfa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f07c800c530 tx=0x7f07c800c9f0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.122+0000 7f07dcbce640 1 -- 172.21.15.112:0/3624045803 shutdown_connections 2024-04-25T01:36:29.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.122+0000 7f07dcbce640 1 --2- 172.21.15.112:0/3624045803 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f07b806bf30 0x7f07b806e3f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.122+0000 7f07dcbce640 1 --2- 172.21.15.112:0/3624045803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07d813ab90 0x7f07d8152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.122+0000 7f07dcbce640 1 --2- 172.21.15.112:0/3624045803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07d8072140 0x7f07d80bbfa0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.122+0000 7f07dcbce640 1 --2- 172.21.15.112:0/3624045803 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07d806c9a0 0x7f07d8134ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.122+0000 7f07dcbce640 1 -- 172.21.15.112:0/3624045803 >> 172.21.15.112:0/3624045803 conn(0x7f07d80c3a80 msgr2=0x7f07d80c2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:29.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.122+0000 7f07dcbce640 1 -- 172.21.15.112:0/3624045803 shutdown_connections 2024-04-25T01:36:29.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.122+0000 7f07dcbce640 1 -- 172.21.15.112:0/3624045803 wait complete. 2024-04-25T01:36:29.116 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:36:29.116 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T01:36:29.116 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:36:29.117 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:36:29.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.304+0000 7fa54afeb640 1 Processor -- start 2024-04-25T01:36:29.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.306+0000 7fa54afeb640 1 -- start start 2024-04-25T01:36:29.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.307+0000 7fa54afeb640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa53c00ba30 0x7fa53c0d8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.307+0000 7fa54afeb640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa53c0d3b50 0x7fa53c0e0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.307+0000 7fa54afeb640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa53c0d6ef0 con 0x7fa53c00b2b0 2024-04-25T01:36:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.307+0000 7fa54afeb640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa53c007420 con 0x7fa53c0d3b50 2024-04-25T01:36:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.307+0000 7fa54afeb640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa53c0d7470 con 0x7fa53c00ba30 2024-04-25T01:36:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.308+0000 7fa5497e8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa53c00ba30 0x7fa53c0d8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.308+0000 7fa5497e8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa53c00ba30 0x7fa53c0d8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:42928/0 (socket says 172.21.15.112:42928) 2024-04-25T01:36:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.308+0000 7fa5497e8640 1 -- 172.21.15.112:0/3695403645 learned_addr learned my addr 172.21.15.112:0/3695403645 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:36:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.308+0000 7fa54c24d640 1 --2- 172.21.15.112:0/3695403645 >> v2:172.21.15.112:3300/0 conn(0x7fa53c0d3b50 0x7fa53c0e0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.308+0000 7fa54c24d640 1 -- 172.21.15.112:0/3695403645 >> v2:172.21.15.187:3300/0 conn(0x7fa53c00ba30 msgr2=0x7fa53c0d8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:29.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.308+0000 7fa54c24d640 1 --2- 172.21.15.112:0/3695403645 >> v2:172.21.15.187:3300/0 conn(0x7fa53c00ba30 0x7fa53c0d8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.308+0000 7fa54c24d640 1 -- 172.21.15.112:0/3695403645 >> v1:172.21.15.112:6789/0 conn(0x7fa53c00b2b0 legacy=0x7fa53c0d7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:29.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.308+0000 7fa54c24d640 1 -- 172.21.15.112:0/3695403645 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53c0d7190 con 0x7fa53c0d3b50 2024-04-25T01:36:29.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.310+0000 7fa54c24d640 1 --2- 172.21.15.112:0/3695403645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c0d3b50 0x7fa53c0e0d80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa53c009950 tx=0x7fa53402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbcb3859e6a3856 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:29.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.310+0000 7fa548fe7640 1 -- 172.21.15.112:0/3695403645 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa534037070 con 0x7fa53c0d3b50 2024-04-25T01:36:29.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.310+0000 7fa548fe7640 1 -- 172.21.15.112:0/3695403645 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa534032440 con 0x7fa53c0d3b50 2024-04-25T01:36:29.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.311+0000 7fa548fe7640 1 -- 172.21.15.112:0/3695403645 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa534032810 con 0x7fa53c0d3b50 2024-04-25T01:36:29.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.311+0000 7fa54afeb640 1 -- 172.21.15.112:0/3695403645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c0d3b50 msgr2=0x7fa53c0e0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.311+0000 7fa54afeb640 1 --2- 172.21.15.112:0/3695403645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c0d3b50 0x7fa53c0e0d80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa53c009950 tx=0x7fa53402a160 comp rx=0 tx=0).stop 2024-04-25T01:36:29.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.312+0000 7fa54afeb640 1 -- 172.21.15.112:0/3695403645 shutdown_connections 2024-04-25T01:36:29.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.312+0000 7fa54afeb640 1 --2- 172.21.15.112:0/3695403645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c0d3b50 0x7fa53c0e0d80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.312+0000 7fa54afeb640 1 --2- 172.21.15.112:0/3695403645 >> v2:172.21.15.187:3300/0 conn(0x7fa53c00ba30 0x7fa53c0d8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.312+0000 7fa54afeb640 1 -- 172.21.15.112:0/3695403645 >> 172.21.15.112:0/3695403645 conn(0x7fa53c006070 msgr2=0x7fa53c006480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:29.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.312+0000 7fa54afeb640 1 -- 172.21.15.112:0/3695403645 shutdown_connections 2024-04-25T01:36:29.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.312+0000 7fa54afeb640 1 -- 172.21.15.112:0/3695403645 wait complete. 2024-04-25T01:36:29.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.314+0000 7fa54afeb640 1 Processor -- start 2024-04-25T01:36:29.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.314+0000 7fa54afeb640 1 -- start start 2024-04-25T01:36:29.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.315+0000 7fa54afeb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c00b2b0 0x7fa53c0d6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.315+0000 7fa54afeb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa53c00ba30 0x7fa53c04a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.315+0000 7fa54afeb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa53c0d3b50 0x7fa53c02cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.315+0000 7fa54afeb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c046a20 con 0x7fa53c00b2b0 2024-04-25T01:36:29.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.315+0000 7fa54afeb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c046740 con 0x7fa53c0d3b50 2024-04-25T01:36:29.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.315+0000 7fa54afeb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa53c02e2f0 con 0x7fa53c00ba30 2024-04-25T01:36:29.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.315+0000 7fa549fe9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c00b2b0 0x7fa53c0d6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.315+0000 7fa549fe9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c00b2b0 0x7fa53c0d6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51894/0 (socket says 172.21.15.112:51894) 2024-04-25T01:36:29.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.315+0000 7fa549fe9640 1 -- 172.21.15.112:0/1045492443 learned_addr learned my addr 172.21.15.112:0/1045492443 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:36:29.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.315+0000 7fa5497e8640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa53c00ba30 0x7fa53c04a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.316+0000 7fa54c24d640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa53c0d3b50 0x7fa53c02cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.316+0000 7fa549fe9640 1 -- 172.21.15.112:0/1045492443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa53c00ba30 msgr2=0x7fa53c04a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.316+0000 7fa549fe9640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa53c00ba30 0x7fa53c04a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.316+0000 7fa549fe9640 1 -- 172.21.15.112:0/1045492443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa53c0d3b50 msgr2=0x7fa53c02cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.316+0000 7fa549fe9640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa53c0d3b50 0x7fa53c02cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.316+0000 7fa549fe9640 1 -- 172.21.15.112:0/1045492443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53c04ad20 con 0x7fa53c00b2b0 2024-04-25T01:36:29.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.316+0000 7fa549fe9640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c00b2b0 0x7fa53c0d6140 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa5440699b0 tx=0x7fa544068670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:29.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.317+0000 7fa53a7fc640 1 -- 172.21.15.112:0/1045492443 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5440733d0 con 0x7fa53c00b2b0 2024-04-25T01:36:29.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.317+0000 7fa53a7fc640 1 -- 172.21.15.112:0/1045492443 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa54408f020 con 0x7fa53c00b2b0 2024-04-25T01:36:29.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.317+0000 7fa53a7fc640 1 -- 172.21.15.112:0/1045492443 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5440737d0 con 0x7fa53c00b2b0 2024-04-25T01:36:29.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.317+0000 7fa54c24d640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa53c0d3b50 0x7fa53c02cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:36:29.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.318+0000 7fa54afeb640 1 -- 172.21.15.112:0/1045492443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53c04b000 con 0x7fa53c00b2b0 2024-04-25T01:36:29.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.318+0000 7fa54afeb640 1 -- 172.21.15.112:0/1045492443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa53c049c30 con 0x7fa53c00b2b0 2024-04-25T01:36:29.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.320+0000 7fa54afeb640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c0d87c0 0x7fa53c16fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.320+0000 7fa54afeb640 1 -- 172.21.15.112:0/1045492443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c04aad0 con 0x7fa53c0d87c0 2024-04-25T01:36:29.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.321+0000 7fa53a7fc640 1 -- 172.21.15.112:0/1045492443 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa54406f400 con 0x7fa53c00b2b0 2024-04-25T01:36:29.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.321+0000 7fa5497e8640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c0d87c0 0x7fa53c16fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.321+0000 7fa5497e8640 1 -- 172.21.15.112:0/1045492443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa53c049790 con 0x7fa53c0d87c0 2024-04-25T01:36:29.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.322+0000 7fa5497e8640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c0d87c0 0x7fa53c16fac0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa53c0ce5d0 tx=0x7fa540012e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:29.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.322+0000 7fa53a7fc640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa52806bf00 0x7fa52806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.322+0000 7fa53a7fc640 1 -- 172.21.15.112:0/1045492443 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fa544105640 con 0x7fa53c00b2b0 2024-04-25T01:36:29.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.322+0000 7fa54c24d640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa52806bf00 0x7fa52806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.323+0000 7fa53a7fc640 1 -- 172.21.15.112:0/1045492443 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540010ea0 con 0x7fa53c0d87c0 2024-04-25T01:36:29.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.323+0000 7fa53a7fc640 1 -- 172.21.15.112:0/1045492443 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa540013440 con 0x7fa53c0d87c0 2024-04-25T01:36:29.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.323+0000 7fa53a7fc640 1 -- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c0d87c0 msgr2=0x7fa53c16fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.323+0000 7fa53a7fc640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c0d87c0 0x7fa53c16fac0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa53c0ce5d0 tx=0x7fa540012e70 comp rx=0 tx=0).stop 2024-04-25T01:36:29.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.323+0000 7fa53a7fc640 1 -- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c0d87c0 msgr2=0x7fa53c16fac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:36:29.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.323+0000 7fa53a7fc640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c0d87c0 0x7fa53c16fac0 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa53c0ce5d0 tx=0x7fa540012e70 comp rx=0 tx=0).stop 2024-04-25T01:36:29.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.323+0000 7fa54c24d640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa52806bf00 0x7fa52806e3c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa534034000 tx=0x7fa53402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:29.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.335+0000 7fa54afeb640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c0d9c40 0x7fa53c0da020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.335+0000 7fa54afeb640 1 -- 172.21.15.112:0/1045492443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c005570 con 0x7fa53c0d9c40 2024-04-25T01:36:29.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.336+0000 7fa5497e8640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c0d9c40 0x7fa53c0da020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.336+0000 7fa5497e8640 1 -- 172.21.15.112:0/1045492443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa53c0075a0 con 0x7fa53c0d9c40 2024-04-25T01:36:29.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.336+0000 7fa5497e8640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c0d9c40 0x7fa53c0da020 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa540019000 tx=0x7fa5400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:29.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.337+0000 7fa53a7fc640 1 -- 172.21.15.112:0/1045492443 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540003610 con 0x7fa53c0d9c40 2024-04-25T01:36:29.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.337+0000 7fa53a7fc640 1 -- 172.21.15.112:0/1045492443 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4561 (secure 0 0 0) 0x7fa5400135f0 con 0x7fa53c0d9c40 2024-04-25T01:36:29.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.337+0000 7fa53a7fc640 1 -- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c0d9c40 msgr2=0x7fa53c0da020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.337+0000 7fa53a7fc640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c0d9c40 0x7fa53c0da020 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa540019000 tx=0x7fa5400046c0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.337+0000 7fa53a7fc640 1 -- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c0d9c40 msgr2=0x7fa53c0da020 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:36:29.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.337+0000 7fa53a7fc640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c0d9c40 0x7fa53c0da020 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa540019000 tx=0x7fa5400046c0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.314 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:36:29.314 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:36:29.314 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:36:29.314 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:36:29.314 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 90, 2024-04-25T01:36:29.314 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 40151, 2024-04-25T01:36:29.314 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:36:29.314 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:36:29.314 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:36:29.314 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:36:29.314 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:29.315 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 14, 2024-04-25T01:36:29.315 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:36:29.315 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:36:29.315 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:36:29.315 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:29.315 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:29.315 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:29.315 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:29.315 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:29.315 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:29.315 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:29.316 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:29.316 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:29.316 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:29.316 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:29.316 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:29.316 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:36:29.316 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:36:29.316 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:29.316 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:29.316 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:29.317 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:29.317 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:29.317 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:29.317 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:29.317 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:29.317 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:29.317 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:29.317 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:29.317 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:29.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.318 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:36:29.318 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:36:29.318 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:36:29.318 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:36:29.318 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:36:29.318 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:36:29.318 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:36:29.318 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:36:29.318 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:36:29.318 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:36:29.318 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:36:29.318 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:36:29.318 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:36:29.318 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:36:29.319 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:36:29.319 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:36:29.319 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:36:29.319 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:29.319 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:29.319 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:29.319 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:29.319 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:29.319 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:29.319 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:29.319 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:29.319 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:29.319 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:29.319 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:29.319 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:29.320 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:36:29.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.320 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:36:29.320 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.320 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:36:29.320 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:36:29.320 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:29.320 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:29.320 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.320 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:29.320 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:36:29.320 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:29.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.320 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.321 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:29.321 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:36:29.321 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:29.321 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.321 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:29.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.321 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:36:29.321 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:36:29.321 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:29.321 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:29.321 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:29.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.321 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.321 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:36:29.322 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:36:29.322 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:29.322 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:29.322 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.322 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:29.322 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:36:29.322 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:29.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.322 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.322 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:29.322 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:36:29.322 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:29.323 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.323 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:29.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.323 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:36:29.323 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:36:29.323 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:29.323 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:29.323 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:29.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.323 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.323 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:36:29.323 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:36:29.323 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:29.323 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:29.324 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.324 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:29.324 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:36:29.324 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:29.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.324 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.324 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:29.324 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:36:29.324 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:29.324 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.324 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:29.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.324 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:36:29.324 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:36:29.324 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:29.324 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:29.325 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:29.325 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.325 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:29.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.325 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:36:29.325 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:36:29.325 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.325 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:29.325 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:29.325 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:36:29.325 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.325 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:29.325 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:36:29.325 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.325 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:29.326 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:29.326 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:36:29.326 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.326 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:29.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.326 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:36:29.326 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:36:29.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.338+0000 7fa54afeb640 1 -- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa52806bf00 msgr2=0x7fa52806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.338+0000 7fa54afeb640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa52806bf00 0x7fa52806e3c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa534034000 tx=0x7fa53402e000 comp rx=0 tx=0).stop 2024-04-25T01:36:29.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.338+0000 7fa54afeb640 1 -- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c00b2b0 msgr2=0x7fa53c0d6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.338+0000 7fa54afeb640 1 --2- 172.21.15.112:0/1045492443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa53c00b2b0 0x7fa53c0d6140 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa5440699b0 tx=0x7fa544068670 comp rx=0 tx=0).stop 2024-04-25T01:36:29.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.338+0000 7fa54c24d640 1 -- 172.21.15.112:0/1045492443 reap_dead start 2024-04-25T01:36:29.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.338+0000 7fa54c24d640 1 -- 172.21.15.112:0/1045492443 reap_dead start 2024-04-25T01:36:29.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.338+0000 7fa54afeb640 1 -- 172.21.15.112:0/1045492443 shutdown_connections 2024-04-25T01:36:29.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.338+0000 7fa54afeb640 1 -- 172.21.15.112:0/1045492443 >> 172.21.15.112:0/1045492443 conn(0x7fa53c006070 msgr2=0x7fa53c0cd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:29.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.338+0000 7fa54afeb640 1 -- 172.21.15.112:0/1045492443 shutdown_connections 2024-04-25T01:36:29.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.338+0000 7fa54afeb640 1 -- 172.21.15.112:0/1045492443 wait complete. 2024-04-25T01:36:29.334 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:36:29.334 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:36:29.334 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:36:29.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.529+0000 7ff6ac3ab640 1 Processor -- start 2024-04-25T01:36:29.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.529+0000 7ff6ac3ab640 1 -- start start 2024-04-25T01:36:29.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.531+0000 7ff6ac3ab640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6a40c6e20 0x7ff6a406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.531+0000 7ff6ac3ab640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff6a40c3d30 0x7ff6a40c0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.531+0000 7ff6ac3ab640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a40c2870 con 0x7ff6a406c6b0 2024-04-25T01:36:29.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.531+0000 7ff6ac3ab640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a40c2bd0 con 0x7ff6a40c3d30 2024-04-25T01:36:29.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.531+0000 7ff6ac3ab640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff6a4156830 con 0x7ff6a40c6e20 2024-04-25T01:36:29.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.531+0000 7ff6a9946640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff6a40c3d30 0x7ff6a40c0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.531+0000 7ff6a9946640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff6a40c3d30 0x7ff6a40c0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:42954/0 (socket says 172.21.15.112:42954) 2024-04-25T01:36:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.531+0000 7ff6aa948640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7ff6a406c6b0 0x7ff6a40c1140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:39890/0 (socket says 172.21.15.112:39890) 2024-04-25T01:36:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.531+0000 7ff6aa948640 1 -- 172.21.15.112:0/2494415166 learned_addr learned my addr 172.21.15.112:0/2494415166 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:36:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.532+0000 7ff6aa147640 1 --2- 172.21.15.112:0/2494415166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6a40c6e20 0x7ff6a406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.532+0000 7ff6a9145640 1 -- 172.21.15.112:0/2494415166 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1746008757 0 0) 0x7ff6a40c2870 con 0x7ff6a406c6b0 2024-04-25T01:36:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.532+0000 7ff6a9145640 1 -- 172.21.15.112:0/2494415166 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff690003520 con 0x7ff6a406c6b0 2024-04-25T01:36:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.532+0000 7ff6a9946640 1 -- 172.21.15.112:0/2494415166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6a40c6e20 msgr2=0x7ff6a406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.532+0000 7ff6a9946640 1 --2- 172.21.15.112:0/2494415166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6a40c6e20 0x7ff6a406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.532+0000 7ff6a9946640 1 -- 172.21.15.112:0/2494415166 >> v1:172.21.15.187:6789/0 conn(0x7ff6a406c6b0 legacy=0x7ff6a40c1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.532+0000 7ff6a9946640 1 -- 172.21.15.112:0/2494415166 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a40a6e30 con 0x7ff6a40c3d30 2024-04-25T01:36:29.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.533+0000 7ff6a9946640 1 --2- 172.21.15.112:0/2494415166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6a40c3d30 0x7ff6a40c0c00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff6980097a0 tx=0x7ff69802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b63efe8eea7b8a77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:29.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.533+0000 7ff6a9145640 1 -- 172.21.15.112:0/2494415166 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff698037070 con 0x7ff6a40c3d30 2024-04-25T01:36:29.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.534+0000 7ff6a9145640 1 -- 172.21.15.112:0/2494415166 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff698032440 con 0x7ff6a40c3d30 2024-04-25T01:36:29.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.534+0000 7ff6a9145640 1 -- 172.21.15.112:0/2494415166 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff698037070 con 0x7ff6a40c3d30 2024-04-25T01:36:29.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.535+0000 7ff6ac3ab640 1 -- 172.21.15.112:0/2494415166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6a40c3d30 msgr2=0x7ff6a40c0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.535+0000 7ff6ac3ab640 1 --2- 172.21.15.112:0/2494415166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6a40c3d30 0x7ff6a40c0c00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff6980097a0 tx=0x7ff69802a180 comp rx=0 tx=0).stop 2024-04-25T01:36:29.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.535+0000 7ff6ac3ab640 1 -- 172.21.15.112:0/2494415166 shutdown_connections 2024-04-25T01:36:29.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.535+0000 7ff6ac3ab640 1 --2- 172.21.15.112:0/2494415166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6a40c3d30 0x7ff6a40c0c00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.535+0000 7ff6ac3ab640 1 --2- 172.21.15.112:0/2494415166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6a40c6e20 0x7ff6a406c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.535+0000 7ff6ac3ab640 1 -- 172.21.15.112:0/2494415166 >> 172.21.15.112:0/2494415166 conn(0x7ff6a40c42d0 msgr2=0x7ff6a40c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:29.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.536+0000 7ff6ac3ab640 1 -- 172.21.15.112:0/2494415166 shutdown_connections 2024-04-25T01:36:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.537+0000 7ff6ac3ab640 1 -- 172.21.15.112:0/2494415166 wait complete. 2024-04-25T01:36:29.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.538+0000 7ff6ac3ab640 1 Processor -- start 2024-04-25T01:36:29.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.538+0000 7ff6ac3ab640 1 -- start start 2024-04-25T01:36:29.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.539+0000 7ff6ac3ab640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6a406c6b0 0x7ff6a41f8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.539+0000 7ff6ac3ab640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6a40c3d30 0x7ff6a41f83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.539+0000 7ff6ac3ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a40c6e20 0x7ff6a40c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.539+0000 7ff6ac3ab640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a40c7200 con 0x7ff6a40c6e20 2024-04-25T01:36:29.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.539+0000 7ff6ac3ab640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a40c2350 con 0x7ff6a40c3d30 2024-04-25T01:36:29.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.539+0000 7ff6ac3ab640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff6a40c7380 con 0x7ff6a406c6b0 2024-04-25T01:36:29.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.540+0000 7ff6aa948640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a40c6e20 0x7ff6a40c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.540+0000 7ff6aa948640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a40c6e20 0x7ff6a40c5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51914/0 (socket says 172.21.15.112:51914) 2024-04-25T01:36:29.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.540+0000 7ff6aa948640 1 -- 172.21.15.112:0/1052473519 learned_addr learned my addr 172.21.15.112:0/1052473519 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:36:29.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.541+0000 7ff6aa948640 1 -- 172.21.15.112:0/1052473519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6a406c6b0 msgr2=0x7ff6a41f8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.541+0000 7ff6aa147640 1 --2- 172.21.15.112:0/1052473519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6a406c6b0 0x7ff6a41f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.541+0000 7ff6aa948640 1 --2- 172.21.15.112:0/1052473519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6a406c6b0 0x7ff6a41f8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.541+0000 7ff6aa948640 1 -- 172.21.15.112:0/1052473519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6a40c3d30 msgr2=0x7ff6a41f83a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:36:29.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.541+0000 7ff6aa948640 1 --2- 172.21.15.112:0/1052473519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6a40c3d30 0x7ff6a41f83a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.541+0000 7ff6aa948640 1 -- 172.21.15.112:0/1052473519 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a41f88e0 con 0x7ff6a40c6e20 2024-04-25T01:36:29.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.541+0000 7ff6aa147640 1 --2- 172.21.15.112:0/1052473519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6a406c6b0 0x7ff6a41f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:36:29.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.542+0000 7ff6aa948640 1 --2- 172.21.15.112:0/1052473519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a40c6e20 0x7ff6a40c5bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000c780 tx=0x7ff6a000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:29.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.542+0000 7ff696ffd640 1 -- 172.21.15.112:0/1052473519 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0017070 con 0x7ff6a40c6e20 2024-04-25T01:36:29.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.542+0000 7ff6ac3ab640 1 -- 172.21.15.112:0/1052473519 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a40bc7f0 con 0x7ff6a40c6e20 2024-04-25T01:36:29.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.542+0000 7ff6ac3ab640 1 -- 172.21.15.112:0/1052473519 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a40bcd90 con 0x7ff6a40c6e20 2024-04-25T01:36:29.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.543+0000 7ff696ffd640 1 -- 172.21.15.112:0/1052473519 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6a00058b0 con 0x7ff6a40c6e20 2024-04-25T01:36:29.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.543+0000 7ff696ffd640 1 -- 172.21.15.112:0/1052473519 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0017070 con 0x7ff6a40c6e20 2024-04-25T01:36:29.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.545+0000 7ff6ac3ab640 1 --2- 172.21.15.112:0/1052473519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff68c002a70 0x7ff68c004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.545+0000 7ff6ac3ab640 1 -- 172.21.15.112:0/1052473519 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c007ec0 con 0x7ff68c002a70 2024-04-25T01:36:29.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.545+0000 7ff696ffd640 1 -- 172.21.15.112:0/1052473519 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff6a001a070 con 0x7ff6a40c6e20 2024-04-25T01:36:29.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.546+0000 7ff696ffd640 1 --2- 172.21.15.112:0/1052473519 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff68806c030 0x7ff68806e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.546+0000 7ff6aa147640 1 --2- 172.21.15.112:0/1052473519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff68c002a70 0x7ff68c004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.546+0000 7ff696ffd640 1 -- 172.21.15.112:0/1052473519 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7ff6a008a060 con 0x7ff6a40c6e20 2024-04-25T01:36:29.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.547+0000 7ff6a9946640 1 --2- 172.21.15.112:0/1052473519 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff68806c030 0x7ff68806e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.547+0000 7ff6aa147640 1 -- 172.21.15.112:0/1052473519 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff68c0080c0 con 0x7ff68c002a70 2024-04-25T01:36:29.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.550+0000 7ff6aa147640 1 --2- 172.21.15.112:0/1052473519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff68c002a70 0x7ff68c004f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff69c007d90 tx=0x7ff69c00f520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:29.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.550+0000 7ff696ffd640 1 -- 172.21.15.112:0/1052473519 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c003b90 con 0x7ff68c002a70 2024-04-25T01:36:29.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.551+0000 7ff696ffd640 1 -- 172.21.15.112:0/1052473519 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff69c019070 con 0x7ff68c002a70 2024-04-25T01:36:29.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.551+0000 7ff696ffd640 1 -- 172.21.15.112:0/1052473519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff68c002a70 msgr2=0x7ff68c004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.551+0000 7ff696ffd640 1 --2- 172.21.15.112:0/1052473519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff68c002a70 0x7ff68c004f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff69c007d90 tx=0x7ff69c00f520 comp rx=0 tx=0).stop 2024-04-25T01:36:29.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.551+0000 7ff696ffd640 1 -- 172.21.15.112:0/1052473519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff68c002a70 msgr2=0x7ff68c004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:36:29.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.551+0000 7ff696ffd640 1 --2- 172.21.15.112:0/1052473519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff68c002a70 0x7ff68c004f30 secure :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff69c007d90 tx=0x7ff69c00f520 comp rx=0 tx=0).stop 2024-04-25T01:36:29.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.552+0000 7ff6a9946640 1 --2- 172.21.15.112:0/1052473519 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff68806c030 0x7ff68806e4f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff698034000 tx=0x7ff69802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:29.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.564+0000 7ff6ac3ab640 1 --2- 172.21.15.112:0/1052473519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff68c006eb0 0x7ff68c007340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.564+0000 7ff6ac3ab640 1 -- 172.21.15.112:0/1052473519 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c00a4a0 con 0x7ff68c006eb0 2024-04-25T01:36:29.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.565+0000 7ff6aa147640 1 --2- 172.21.15.112:0/1052473519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff68c006eb0 0x7ff68c007340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.566+0000 7ff6aa147640 1 -- 172.21.15.112:0/1052473519 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff68c007880 con 0x7ff68c006eb0 2024-04-25T01:36:29.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.566+0000 7ff6aa147640 1 --2- 172.21.15.112:0/1052473519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff68c006eb0 0x7ff68c007340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff69c007d90 tx=0x7ff69c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:29.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.566+0000 7ff696ffd640 1 -- 172.21.15.112:0/1052473519 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c004240 con 0x7ff68c006eb0 2024-04-25T01:36:29.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.567+0000 7ff696ffd640 1 -- 172.21.15.112:0/1052473519 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5070 (secure 0 0 0) 0x7ff69c004240 con 0x7ff68c006eb0 2024-04-25T01:36:29.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.567+0000 7ff696ffd640 1 -- 172.21.15.112:0/1052473519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff68c006eb0 msgr2=0x7ff68c007340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.567+0000 7ff696ffd640 1 --2- 172.21.15.112:0/1052473519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff68c006eb0 0x7ff68c007340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff69c007d90 tx=0x7ff69c004a40 comp rx=0 tx=0).stop 2024-04-25T01:36:29.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.567+0000 7ff696ffd640 1 -- 172.21.15.112:0/1052473519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff68c006eb0 msgr2=0x7ff68c007340 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:36:29.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.567+0000 7ff696ffd640 1 --2- 172.21.15.112:0/1052473519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff68c006eb0 0x7ff68c007340 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.543 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:36:29.543 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:36:29.543 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:36:29.543 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:36:29.543 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 90, 2024-04-25T01:36:29.544 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 425255, 2024-04-25T01:36:29.544 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:36:29.544 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:36:29.544 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:36:29.544 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:36:29.544 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:29.544 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 14, 2024-04-25T01:36:29.544 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:36:29.544 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:36:29.544 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:36:29.544 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:29.544 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:29.544 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:29.544 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:29.544 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:29.545 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:29.545 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:29.545 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:29.545 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:29.545 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:29.545 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:29.545 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:29.545 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:36:29.545 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:36:29.545 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:29.545 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:29.545 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:29.545 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:29.545 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:29.546 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:29.546 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:29.546 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:29.546 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:29.546 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:29.546 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:29.546 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:29.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.546 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:36:29.546 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:36:29.546 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:36:29.546 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:36:29.546 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:36:29.546 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:36:29.547 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:36:29.547 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:36:29.547 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:36:29.547 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:36:29.547 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:36:29.547 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:36:29.547 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:36:29.547 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:36:29.547 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:36:29.547 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:36:29.547 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:36:29.547 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:29.547 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:29.547 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:29.547 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:29.548 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:29.548 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:29.548 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:29.548 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:29.548 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:29.548 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:29.548 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:29.548 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:29.548 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:36:29.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.548 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:36:29.548 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.548 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:36:29.548 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:36:29.549 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:29.549 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:29.549 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.549 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:29.549 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:36:29.549 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:29.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.549 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.549 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:29.549 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:36:29.549 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:29.549 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.549 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:29.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.550 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:36:29.550 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:36:29.550 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:29.550 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:29.550 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:29.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.550 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.550 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:36:29.550 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:36:29.550 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:29.550 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:29.550 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.550 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:29.550 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:36:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:36:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:36:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:36:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.552 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.552 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:36:29.552 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:36:29.552 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:29.552 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:29.552 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.552 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:29.552 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:36:29.552 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:29.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.552 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.552 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:29.552 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:36:29.552 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:29.552 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:36:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:36:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:36:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:36:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:36:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:36:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:36:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:36:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 8 2024-04-25T01:36:29.555 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.555 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:29.555 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:36:29.555 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.555 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:29.555 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:29.555 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 6 2024-04-25T01:36:29.555 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.555 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:29.555 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:36:29.555 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.555 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:29.555 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:29.555 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:36:29.556 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.556 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:29.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.556 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:36:29.556 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:36:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.567+0000 7ff6ac3ab640 1 -- 172.21.15.112:0/1052473519 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff68806c030 msgr2=0x7ff68806e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.567+0000 7ff6ac3ab640 1 --2- 172.21.15.112:0/1052473519 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff68806c030 0x7ff68806e4f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff698034000 tx=0x7ff69802e000 comp rx=0 tx=0).stop 2024-04-25T01:36:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.567+0000 7ff6ac3ab640 1 -- 172.21.15.112:0/1052473519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a40c6e20 msgr2=0x7ff6a40c5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.567+0000 7ff6ac3ab640 1 --2- 172.21.15.112:0/1052473519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a40c6e20 0x7ff6a40c5bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000c780 tx=0x7ff6a000cc40 comp rx=0 tx=0).stop 2024-04-25T01:36:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.567+0000 7ff6aa948640 1 -- 172.21.15.112:0/1052473519 reap_dead start 2024-04-25T01:36:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.567+0000 7ff6aa948640 1 -- 172.21.15.112:0/1052473519 reap_dead start 2024-04-25T01:36:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.567+0000 7ff6ac3ab640 1 -- 172.21.15.112:0/1052473519 shutdown_connections 2024-04-25T01:36:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.567+0000 7ff6ac3ab640 1 -- 172.21.15.112:0/1052473519 >> 172.21.15.112:0/1052473519 conn(0x7ff6a40c42d0 msgr2=0x7ff6a40a7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.568+0000 7ff6ac3ab640 1 -- 172.21.15.112:0/1052473519 shutdown_connections 2024-04-25T01:36:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.568+0000 7ff6ac3ab640 1 -- 172.21.15.112:0/1052473519 wait complete. 2024-04-25T01:36:29.565 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:36:29.565 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:36:29.565 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:36:29.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.753+0000 7f5e91efc640 1 Processor -- start 2024-04-25T01:36:29.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.753+0000 7f5e91efc640 1 -- start start 2024-04-25T01:36:29.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.754+0000 7f5e91efc640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5e8c153030 0x7f5e8c0a9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.754+0000 7f5e91efc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c0a9bd0 0x7f5e8c15cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.755+0000 7f5e8b7fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5e8c072090 0x7f5e8c14f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59870/0 (socket says 172.21.15.112:59870) 2024-04-25T01:36:29.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.755+0000 7f5e8b7fe640 1 -- 172.21.15.112:0/1789535620 learned_addr learned my addr 172.21.15.112:0/1789535620 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:36:29.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.755+0000 7f5e91efc640 1 -- 172.21.15.112:0/1789535620 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e8c06f220 con 0x7f5e8c072090 2024-04-25T01:36:29.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.755+0000 7f5e91efc640 1 -- 172.21.15.112:0/1789535620 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5e8c153470 con 0x7f5e8c153030 2024-04-25T01:36:29.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.755+0000 7f5e91efc640 1 -- 172.21.15.112:0/1789535620 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5e8c0c4990 con 0x7f5e8c0a9bd0 2024-04-25T01:36:29.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.755+0000 7f5e8affd640 1 --2- 172.21.15.112:0/1789535620 >> v2:172.21.15.187:3300/0 conn(0x7f5e8c153030 0x7f5e8c0a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.755+0000 7f5e8bfff640 1 --2- 172.21.15.112:0/1789535620 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c0a9bd0 0x7f5e8c15cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.755+0000 7f5e8affd640 1 -- 172.21.15.112:0/1789535620 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c0a9bd0 msgr2=0x7f5e8c15cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:29.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.755+0000 7f5e8affd640 1 --2- 172.21.15.112:0/1789535620 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c0a9bd0 0x7f5e8c15cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.755+0000 7f5e8affd640 1 -- 172.21.15.112:0/1789535620 >> v1:172.21.15.112:6789/0 conn(0x7f5e8c072090 legacy=0x7f5e8c14f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.755+0000 7f5e8affd640 1 -- 172.21.15.112:0/1789535620 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8c06f4c0 con 0x7f5e8c153030 2024-04-25T01:36:29.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.756+0000 7f5e8affd640 1 --2- 172.21.15.112:0/1789535620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e8c153030 0x7f5e8c0a9690 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5e800097a0 tx=0x7f5e8002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b87683d7f7c5aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:29.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.756+0000 7f5e8a7fc640 1 -- 172.21.15.112:0/1789535620 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80037070 con 0x7f5e8c153030 2024-04-25T01:36:29.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.756+0000 7f5e8a7fc640 1 -- 172.21.15.112:0/1789535620 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e80032440 con 0x7f5e8c153030 2024-04-25T01:36:29.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.756+0000 7f5e8a7fc640 1 -- 172.21.15.112:0/1789535620 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80037070 con 0x7f5e8c153030 2024-04-25T01:36:29.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.757+0000 7f5e91efc640 1 -- 172.21.15.112:0/1789535620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e8c153030 msgr2=0x7f5e8c0a9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.757+0000 7f5e91efc640 1 --2- 172.21.15.112:0/1789535620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e8c153030 0x7f5e8c0a9690 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5e800097a0 tx=0x7f5e8002a180 comp rx=0 tx=0).stop 2024-04-25T01:36:29.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.757+0000 7f5e91efc640 1 -- 172.21.15.112:0/1789535620 shutdown_connections 2024-04-25T01:36:29.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.757+0000 7f5e91efc640 1 --2- 172.21.15.112:0/1789535620 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c0a9bd0 0x7f5e8c15cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.757+0000 7f5e91efc640 1 --2- 172.21.15.112:0/1789535620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e8c153030 0x7f5e8c0a9690 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.757+0000 7f5e91efc640 1 -- 172.21.15.112:0/1789535620 >> 172.21.15.112:0/1789535620 conn(0x7f5e8c06c8a0 msgr2=0x7f5e8c0c8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:29.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.758+0000 7f5e91efc640 1 -- 172.21.15.112:0/1789535620 shutdown_connections 2024-04-25T01:36:29.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.759+0000 7f5e91efc640 1 -- 172.21.15.112:0/1789535620 wait complete. 2024-04-25T01:36:29.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.761+0000 7f5e91efc640 1 Processor -- start 2024-04-25T01:36:29.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.762+0000 7f5e91efc640 1 -- start start 2024-04-25T01:36:29.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.763+0000 7f5e91efc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c072090 0x7f5e8c154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.763+0000 7f5e91efc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e8c0a9bd0 0x7f5e8c1efde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.763+0000 7f5e91efc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e8c153030 0x7f5e8c0c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.763+0000 7f5e91efc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c06fa80 con 0x7f5e8c153030 2024-04-25T01:36:29.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.763+0000 7f5e91efc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c06fdf0 con 0x7f5e8c0a9bd0 2024-04-25T01:36:29.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.763+0000 7f5e91efc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5e8c14fea0 con 0x7f5e8c072090 2024-04-25T01:36:29.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.763+0000 7f5e8b7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c072090 0x7f5e8c154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.763+0000 7f5e8b7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c072090 0x7f5e8c154710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:36698/0 (socket says 172.21.15.112:36698) 2024-04-25T01:36:29.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.763+0000 7f5e8b7fe640 1 -- 172.21.15.112:0/2616446629 learned_addr learned my addr 172.21.15.112:0/2616446629 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:36:29.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.764+0000 7f5e8bfff640 1 --2- 172.21.15.112:0/2616446629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e8c153030 0x7f5e8c0c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.764+0000 7f5e8bfff640 1 -- 172.21.15.112:0/2616446629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c072090 msgr2=0x7f5e8c154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.764+0000 7f5e8bfff640 1 --2- 172.21.15.112:0/2616446629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c072090 0x7f5e8c154710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.764+0000 7f5e8bfff640 1 -- 172.21.15.112:0/2616446629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e8c0a9bd0 msgr2=0x7f5e8c1efde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.764+0000 7f5e8bfff640 1 --2- 172.21.15.112:0/2616446629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e8c0a9bd0 0x7f5e8c1efde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.764+0000 7f5e8bfff640 1 -- 172.21.15.112:0/2616446629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8c154c50 con 0x7f5e8c153030 2024-04-25T01:36:29.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.765+0000 7f5e8bfff640 1 --2- 172.21.15.112:0/2616446629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e8c153030 0x7f5e8c0c37d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400efc0 tx=0x7f5e8400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:29.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.765+0000 7f5e73fff640 1 -- 172.21.15.112:0/2616446629 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e84018020 con 0x7f5e8c153030 2024-04-25T01:36:29.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.766+0000 7f5e73fff640 1 -- 172.21.15.112:0/2616446629 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e84014480 con 0x7f5e8c153030 2024-04-25T01:36:29.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.766+0000 7f5e73fff640 1 -- 172.21.15.112:0/2616446629 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e84014870 con 0x7f5e8c153030 2024-04-25T01:36:29.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.766+0000 7f5e91efc640 1 -- 172.21.15.112:0/2616446629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8c1ef620 con 0x7f5e8c153030 2024-04-25T01:36:29.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.766+0000 7f5e91efc640 1 -- 172.21.15.112:0/2616446629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e8c1f36f0 con 0x7f5e8c153030 2024-04-25T01:36:29.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.767+0000 7f5e91efc640 1 --2- 172.21.15.112:0/2616446629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c154250 0x7f5e8c147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.767+0000 7f5e91efc640 1 -- 172.21.15.112:0/2616446629 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5e8c075980 con 0x7f5e8c154250 2024-04-25T01:36:29.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.767+0000 7f5e8b7fe640 1 --2- 172.21.15.112:0/2616446629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c154250 0x7f5e8c147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.770+0000 7f5e73fff640 1 -- 172.21.15.112:0/2616446629 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e84005d10 con 0x7f5e8c153030 2024-04-25T01:36:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.771+0000 7f5e8b7fe640 1 -- 172.21.15.112:0/2616446629 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e8c1ef410 con 0x7f5e8c154250 2024-04-25T01:36:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.771+0000 7f5e73fff640 1 --2- 172.21.15.112:0/2616446629 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5e7806bf00 0x7f5e7806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.771+0000 7f5e73fff640 1 -- 172.21.15.112:0/2616446629 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f5e8408c020 con 0x7f5e8c153030 2024-04-25T01:36:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.771+0000 7f5e8affd640 1 --2- 172.21.15.112:0/2616446629 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5e7806bf00 0x7f5e7806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.772+0000 7f5e8affd640 1 --2- 172.21.15.112:0/2616446629 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5e7806bf00 0x7f5e7806e3c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5e80034000 tx=0x7f5e8002c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.772+0000 7f5e8b7fe640 1 --2- 172.21.15.112:0/2616446629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c154250 0x7f5e8c147830 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00e500 tx=0x7f5e7c00e9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:29.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.772+0000 7f5e73fff640 1 -- 172.21.15.112:0/2616446629 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c005ac0 con 0x7f5e8c154250 2024-04-25T01:36:29.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.773+0000 7f5e73fff640 1 -- 172.21.15.112:0/2616446629 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5e7c018070 con 0x7f5e8c154250 2024-04-25T01:36:29.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.773+0000 7f5e73fff640 1 -- 172.21.15.112:0/2616446629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c154250 msgr2=0x7f5e8c147830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.773+0000 7f5e73fff640 1 --2- 172.21.15.112:0/2616446629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c154250 0x7f5e8c147830 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00e500 tx=0x7f5e7c00e9c0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.773+0000 7f5e73fff640 1 -- 172.21.15.112:0/2616446629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c154250 msgr2=0x7f5e8c147830 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:36:29.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.773+0000 7f5e73fff640 1 --2- 172.21.15.112:0/2616446629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c154250 0x7f5e8c147830 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00e500 tx=0x7f5e7c00e9c0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.789+0000 7f5e91efc640 1 --2- 172.21.15.112:0/2616446629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c153980 0x7f5e8c147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:29.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.789+0000 7f5e91efc640 1 -- 172.21.15.112:0/2616446629 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5e8c1f3930 con 0x7f5e8c153980 2024-04-25T01:36:29.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.792+0000 7f5e8b7fe640 1 --2- 172.21.15.112:0/2616446629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c153980 0x7f5e8c147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:29.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.792+0000 7f5e8b7fe640 1 -- 172.21.15.112:0/2616446629 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5e8c072ff0 con 0x7f5e8c153980 2024-04-25T01:36:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.794+0000 7f5e8b7fe640 1 --2- 172.21.15.112:0/2616446629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c153980 0x7f5e8c147d70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00e500 tx=0x7f5e7c000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.794+0000 7f5e73fff640 1 -- 172.21.15.112:0/2616446629 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c0049e0 con 0x7f5e8c153980 2024-04-25T01:36:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.794+0000 7f5e73fff640 1 -- 172.21.15.112:0/2616446629 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4560 (secure 0 0 0) 0x7f5e7c005690 con 0x7f5e8c153980 2024-04-25T01:36:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.794+0000 7f5e73fff640 1 -- 172.21.15.112:0/2616446629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c153980 msgr2=0x7f5e8c147d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.794+0000 7f5e73fff640 1 --2- 172.21.15.112:0/2616446629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c153980 0x7f5e8c147d70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00e500 tx=0x7f5e7c000ed0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.794+0000 7f5e73fff640 1 -- 172.21.15.112:0/2616446629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c153980 msgr2=0x7f5e8c147d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:36:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.794+0000 7f5e73fff640 1 --2- 172.21.15.112:0/2616446629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e8c153980 0x7f5e8c147d70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:29.771 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:36:29.771 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:36:29.771 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:36:29.771 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:36:29.771 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 90, 2024-04-25T01:36:29.771 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 100454, 2024-04-25T01:36:29.771 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:36:29.771 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:36:29.771 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:36:29.771 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:36:29.771 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:29.772 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 15, 2024-04-25T01:36:29.772 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:36:29.772 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:36:29.772 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:36:29.772 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:29.772 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:29.772 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:29.772 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:29.772 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:29.772 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:29.772 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:29.772 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:29.772 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:29.772 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:29.773 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:29.773 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:29.773 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:36:29.773 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:36:29.773 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:29.773 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:29.773 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:29.773 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:29.773 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:29.773 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:29.773 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:29.773 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:29.773 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:29.773 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:29.773 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:29.774 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:29.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.774 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:36:29.774 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:36:29.774 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:36:29.774 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:36:29.774 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:36:29.774 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:36:29.774 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:36:29.774 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:36:29.774 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:36:29.774 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:36:29.774 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:36:29.774 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:36:29.775 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:36:29.775 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:36:29.775 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:36:29.775 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:36:29.775 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:36:29.775 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:29.775 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:29.775 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:29.775 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:29.776 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:29.776 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:29.776 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:29.776 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:29.776 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:29.776 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:29.776 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:29.776 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:29.776 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:36:29.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.776 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:36:29.776 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.776 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:36:29.776 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:36:29.776 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:29.777 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:29.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.777 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:29.777 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:36:29.777 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:29.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.777 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:29.777 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:36:29.777 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:29.777 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.777 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:29.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.777 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:36:29.777 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:36:29.778 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:29.778 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:29.778 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:29.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.778 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.778 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:36:29.778 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:36:29.778 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:29.778 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:29.778 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.778 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:29.778 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:36:29.778 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:29.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.779 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:29.779 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:36:29.779 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:29.779 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.779 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:29.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.779 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:36:29.779 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:36:29.779 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:29.779 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:29.779 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:29.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.779 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:36:29.780 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:36:29.780 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:29.780 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:29.780 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.780 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:29.780 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:36:29.780 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:29.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.780 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.780 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:29.780 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:36:29.780 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:29.780 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.780 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:29.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.781 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:36:29.781 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:36:29.781 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:29.781 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:29.781 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:29.781 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.781 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:29.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.781 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:36:29.781 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:36:29.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.781 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:29.781 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:29.781 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:36:29.781 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.782 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:29.782 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:36:29.782 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:29.782 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:29.782 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:29.782 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:36:29.782 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:29.782 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:29.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:29.782 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:36:29.782 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:36:29.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.796+0000 7f5e91efc640 1 -- 172.21.15.112:0/2616446629 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5e7806bf00 msgr2=0x7f5e7806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.796+0000 7f5e91efc640 1 --2- 172.21.15.112:0/2616446629 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5e7806bf00 0x7f5e7806e3c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5e80034000 tx=0x7f5e8002c000 comp rx=0 tx=0).stop 2024-04-25T01:36:29.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.796+0000 7f5e91efc640 1 -- 172.21.15.112:0/2616446629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e8c153030 msgr2=0x7f5e8c0c37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:29.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.796+0000 7f5e91efc640 1 --2- 172.21.15.112:0/2616446629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e8c153030 0x7f5e8c0c37d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400efc0 tx=0x7f5e8400c490 comp rx=0 tx=0).stop 2024-04-25T01:36:29.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.796+0000 7f5e8bfff640 1 -- 172.21.15.112:0/2616446629 reap_dead start 2024-04-25T01:36:29.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.796+0000 7f5e8bfff640 1 -- 172.21.15.112:0/2616446629 reap_dead start 2024-04-25T01:36:29.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.796+0000 7f5e91efc640 1 -- 172.21.15.112:0/2616446629 shutdown_connections 2024-04-25T01:36:29.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.796+0000 7f5e91efc640 1 -- 172.21.15.112:0/2616446629 >> 172.21.15.112:0/2616446629 conn(0x7f5e8c06c8a0 msgr2=0x7f5e8c0c68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:29.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.797+0000 7f5e91efc640 1 -- 172.21.15.112:0/2616446629 shutdown_connections 2024-04-25T01:36:29.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:29.797+0000 7f5e91efc640 1 -- 172.21.15.112:0/2616446629 wait complete. 2024-04-25T01:36:29.793 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:36:29.793 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-04-25T01:36:29.793 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['b'] 2024-04-25T01:36:29.794 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-04-25T01:36:29.794 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-25T01:36:29.794 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T01:36:29.794 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:36:29.810 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T01:36:29.811 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-04-25T01:36:29.811 INFO:tasks.ceph.mon.b:Sent signal 19 2024-04-25T01:36:29.811 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-04-25T01:36:44.812 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-04-25T01:36:44.813 INFO:tasks.ceph.mon.b:Sent signal 18 2024-04-25T01:36:44.813 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T01:36:44.813 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:36:44.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.000+0000 7f50f17c1640 1 Processor -- start 2024-04-25T01:36:44.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.000+0000 7f50f17c1640 1 -- start start 2024-04-25T01:36:44.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.002+0000 7f50f17c1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50ec154300 0x7f50ec0c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:44.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.002+0000 7f50f17c1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 0x7f50ec0c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:44.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.002+0000 7f50e3fff640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50ec154300 msgr2=0x7f50ec0c4f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:36:44.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.002+0000 7f50e3fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50ec154300 0x7f50ec0c4f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:36:44.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.002+0000 7f50f17c1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50ec14daa0 con 0x7f50ec14d6c0 2024-04-25T01:36:44.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.002+0000 7f50f17c1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f50ec153e70 con 0x7f50ec0713e0 2024-04-25T01:36:44.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.002+0000 7f50f17c1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f50ec0c5580 con 0x7f50ec154300 2024-04-25T01:36:44.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.002+0000 7f50eaffd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f50ec14d6c0 0x7f50ec06eb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53536/0 (socket says 172.21.15.112:53536) 2024-04-25T01:36:44.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.002+0000 7f50eaffd640 1 -- 172.21.15.112:0/3111668605 learned_addr learned my addr 172.21.15.112:0/3111668605 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:36:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.003+0000 7f50eb7fe640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 0x7f50ec0c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.004+0000 7f50eb7fe640 1 -- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 msgr2=0x7f50ec0c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:36:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.004+0000 7f50eb7fe640 1 -- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 msgr2=0x7f50ec0c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:36:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.004+0000 7f50eb7fe640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 0x7f50ec0c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.004+0000 7f50eb7fe640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 0x7f50ec0c5db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:36:45.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.203+0000 7f50e3fff640 1 -- 172.21.15.112:0/3111668605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50ec154300 msgr2=0x7f50ec0c4f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:36:45.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.203+0000 7f50e3fff640 1 --2- 172.21.15.112:0/3111668605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50ec154300 0x7f50ec0c4f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:36:45.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.205+0000 7f50eb7fe640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 0x7f50ec0c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:45.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.205+0000 7f50eb7fe640 1 -- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 msgr2=0x7f50ec0c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:36:45.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.205+0000 7f50eb7fe640 1 -- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 msgr2=0x7f50ec0c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:36:45.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.205+0000 7f50eb7fe640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 0x7f50ec0c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:45.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.205+0000 7f50eb7fe640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 0x7f50ec0c5db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:36:45.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.604+0000 7f50e3fff640 1 -- 172.21.15.112:0/3111668605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50ec154300 msgr2=0x7f50ec0c4f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:36:45.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.604+0000 7f50e3fff640 1 --2- 172.21.15.112:0/3111668605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50ec154300 0x7f50ec0c4f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:36:45.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.606+0000 7f50eb7fe640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 0x7f50ec0c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:45.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.606+0000 7f50eb7fe640 1 -- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 msgr2=0x7f50ec0c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:36:45.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.606+0000 7f50eb7fe640 1 -- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 msgr2=0x7f50ec0c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:36:45.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.606+0000 7f50eb7fe640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 0x7f50ec0c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:45.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:45.606+0000 7f50eb7fe640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 0x7f50ec0c5db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:36:46.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:46.404+0000 7f50e3fff640 1 -- 172.21.15.112:0/3111668605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50ec154300 msgr2=0x7f50ec0c4f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:36:46.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:46.404+0000 7f50e3fff640 1 --2- 172.21.15.112:0/3111668605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50ec154300 0x7f50ec0c4f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:36:46.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:46.406+0000 7f50eb7fe640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 0x7f50ec0c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:46.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:46.406+0000 7f50eb7fe640 1 -- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 msgr2=0x7f50ec0c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:36:46.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:46.406+0000 7f50eb7fe640 1 -- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 msgr2=0x7f50ec0c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:36:46.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:46.406+0000 7f50eb7fe640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 0x7f50ec0c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:46.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:46.406+0000 7f50eb7fe640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 0x7f50ec0c5db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:36:47.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.001+0000 7f50e97fa640 1 -- 172.21.15.112:0/3111668605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50ec154300 msgr2=0x7f50ec0c4f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:36:47.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.001+0000 7f50e97fa640 1 --2- 172.21.15.112:0/3111668605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50ec154300 0x7f50ec0c4f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:47.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.001+0000 7f50e97fa640 1 -- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 msgr2=0x7f50ec0c5db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:36:47.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.001+0000 7f50e97fa640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.112:3300/0 conn(0x7f50ec0713e0 0x7f50ec0c5db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:47.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.001+0000 7f50e97fa640 1 -- 172.21.15.112:0/3111668605 >> v1:172.21.15.112:6789/0 conn(0x7f50ec14d6c0 legacy=0x7f50ec06eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:47.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.001+0000 7f50e97fa640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 0x7f50cc005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:47.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.001+0000 7f50e97fa640 1 -- 172.21.15.112:0/3111668605 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50ec153820 con 0x7f50cc006190 2024-04-25T01:36:47.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.001+0000 7f50e97fa640 1 -- 172.21.15.112:0/3111668605 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f50ec153e70 con 0x7f50cc0036c0 2024-04-25T01:36:47.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.001+0000 7f50eaffd640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 0x7f50cc005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:47.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.001+0000 7f50eaffd640 1 -- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 msgr2=0x7f50cc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:36:47.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.001+0000 7f50eaffd640 1 -- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 msgr2=0x7f50cc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:36:47.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.001+0000 7f50eaffd640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 0x7f50cc005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:47.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.001+0000 7f50eaffd640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 0x7f50cc005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:36:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.202+0000 7f50eaffd640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 0x7f50cc005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.202+0000 7f50eaffd640 1 -- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 msgr2=0x7f50cc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:36:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.202+0000 7f50eaffd640 1 -- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 msgr2=0x7f50cc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:36:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.202+0000 7f50eaffd640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 0x7f50cc005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.202+0000 7f50eaffd640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 0x7f50cc005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:36:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.603+0000 7f50eaffd640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 0x7f50cc005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.603+0000 7f50eaffd640 1 -- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 msgr2=0x7f50cc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:36:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.603+0000 7f50eaffd640 1 -- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 msgr2=0x7f50cc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:36:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.603+0000 7f50eaffd640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 0x7f50cc005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:48.603+0000 7f50eaffd640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 0x7f50cc005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:36:49.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.405+0000 7f50eaffd640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 0x7f50cc005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:49.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.405+0000 7f50eaffd640 1 -- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 msgr2=0x7f50cc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:36:49.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.405+0000 7f50eaffd640 1 -- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 msgr2=0x7f50cc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:36:49.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.405+0000 7f50eaffd640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 0x7f50cc005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:36:49.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.405+0000 7f50eaffd640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 0x7f50cc005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:36:49.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.855+0000 7f50ea7fc640 1 -- 172.21.15.112:0/3111668605 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 651535364 0 0) 0x7f50ec153820 con 0x7f50cc006190 2024-04-25T01:36:49.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.856+0000 7f50ea7fc640 1 -- 172.21.15.112:0/3111668605 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50d8003520 con 0x7f50cc006190 2024-04-25T01:36:49.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.863+0000 7f50ea7fc640 1 -- 172.21.15.112:0/3111668605 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2872863994 0 0) 0x7f50d8003520 con 0x7f50cc006190 2024-04-25T01:36:49.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.863+0000 7f50ea7fc640 1 -- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 msgr2=0x7f50cc005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:36:49.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.863+0000 7f50ea7fc640 1 --2- 172.21.15.112:0/3111668605 >> v2:172.21.15.187:3300/0 conn(0x7f50cc0036c0 0x7f50cc005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:49.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.863+0000 7f50ea7fc640 1 -- 172.21.15.112:0/3111668605 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50cc00afa0 con 0x7f50cc006190 2024-04-25T01:36:49.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.863+0000 7f50ea7fc640 1 -- 172.21.15.112:0/3111668605 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f50d4004330 con 0x7f50cc006190 2024-04-25T01:36:49.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.863+0000 7f50ea7fc640 1 -- 172.21.15.112:0/3111668605 >> v1:172.21.15.187:6789/0 conn(0x7f50cc006190 legacy=0x7f50cc008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:49.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.863+0000 7f50eb7fe640 1 -- 172.21.15.112:0/3111668605 reap_dead start 2024-04-25T01:36:49.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.863+0000 7f50ea7fc640 1 --2- 172.21.15.112:0/3111668605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d80039b0 0x7f50d8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:49.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.863+0000 7f50ea7fc640 1 --2- 172.21.15.112:0/3111668605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d8004800 0x7f50d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:49.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.863+0000 7f50ea7fc640 1 --2- 172.21.15.112:0/3111668605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50d8005210 0x7f50d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:49.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.863+0000 7f50ea7fc640 1 -- 172.21.15.112:0/3111668605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec153e70 con 0x7f50d8005210 2024-04-25T01:36:49.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.863+0000 7f50ea7fc640 1 -- 172.21.15.112:0/3111668605 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f50d800c140 con 0x7f50d8004800 2024-04-25T01:36:49.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.863+0000 7f50ea7fc640 1 -- 172.21.15.112:0/3111668605 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f50d800c340 con 0x7f50d80039b0 2024-04-25T01:36:49.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.863+0000 7f50eaffd640 1 -- 172.21.15.112:0/3111668605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d80039b0 msgr2=0x7f50d8003db0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:36:49.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.863+0000 7f50eaffd640 1 --2- 172.21.15.112:0/3111668605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d80039b0 0x7f50d8003db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:36:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.864+0000 7f50eb7fe640 1 --2- 172.21.15.112:0/3111668605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50d8005210 0x7f50d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.864+0000 7f50e3fff640 1 --2- 172.21.15.112:0/3111668605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d8004800 0x7f50d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.864+0000 7f50eb7fe640 1 -- 172.21.15.112:0/3111668605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d80039b0 msgr2=0x7f50d8003db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:36:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.864+0000 7f50eb7fe640 1 --2- 172.21.15.112:0/3111668605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d80039b0 0x7f50d8003db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.864+0000 7f50eb7fe640 1 -- 172.21.15.112:0/3111668605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d8004800 msgr2=0x7f50d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.864+0000 7f50eb7fe640 1 --2- 172.21.15.112:0/3111668605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d8004800 0x7f50d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.864+0000 7f50eb7fe640 1 -- 172.21.15.112:0/3111668605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d8003520 con 0x7f50d8005210 2024-04-25T01:36:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.865+0000 7f50eb7fe640 1 --2- 172.21.15.112:0/3111668605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50d8005210 0x7f50d800b9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0c4930 tx=0x7f50d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c9a50327be0811 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.865+0000 7f50ea7fc640 1 -- 172.21.15.112:0/3111668605 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4031070 con 0x7f50d8005210 2024-04-25T01:36:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.865+0000 7f50ea7fc640 1 -- 172.21.15.112:0/3111668605 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e402b7a0 con 0x7f50d8005210 2024-04-25T01:36:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.865+0000 7f50ea7fc640 1 -- 172.21.15.112:0/3111668605 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4003e30 con 0x7f50d8005210 2024-04-25T01:36:49.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.865+0000 7f50f17c1640 1 -- 172.21.15.112:0/3111668605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50d8005210 msgr2=0x7f50d800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:49.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.865+0000 7f50f17c1640 1 --2- 172.21.15.112:0/3111668605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50d8005210 0x7f50d800b9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0c4930 tx=0x7f50d8005830 comp rx=0 tx=0).stop 2024-04-25T01:36:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.865+0000 7f50f17c1640 1 -- 172.21.15.112:0/3111668605 shutdown_connections 2024-04-25T01:36:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.865+0000 7f50f17c1640 1 --2- 172.21.15.112:0/3111668605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50d8005210 0x7f50d800b9c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.865+0000 7f50f17c1640 1 --2- 172.21.15.112:0/3111668605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d8004800 0x7f50d8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.865+0000 7f50f17c1640 1 --2- 172.21.15.112:0/3111668605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d80039b0 0x7f50d8003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.865+0000 7f50f17c1640 1 -- 172.21.15.112:0/3111668605 >> 172.21.15.112:0/3111668605 conn(0x7f50ec0c9140 msgr2=0x7f50ec0c9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.866+0000 7f50f17c1640 1 -- 172.21.15.112:0/3111668605 shutdown_connections 2024-04-25T01:36:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.866+0000 7f50f17c1640 1 -- 172.21.15.112:0/3111668605 wait complete. 2024-04-25T01:36:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.867+0000 7f50f17c1640 1 Processor -- start 2024-04-25T01:36:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.867+0000 7f50f17c1640 1 -- start start 2024-04-25T01:36:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50f17c1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d80039b0 0x7f50ec141d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50f17c1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d8004800 0x7f50ec0ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50f17c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50d8005210 0x7f50ec0c2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50f17c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec0c4f10 con 0x7f50d8005210 2024-04-25T01:36:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50f17c1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec071060 con 0x7f50d80039b0 2024-04-25T01:36:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50f17c1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f50ec0c5090 con 0x7f50d8004800 2024-04-25T01:36:49.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50d8005210 0x7f50ec0c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:49.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50e3fff640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d8004800 msgr2=0x7f50ec0ace00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:36:49.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50e3fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d8004800 0x7f50ec0ace00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:36:49.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50d8005210 0x7f50ec0c2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36224/0 (socket says 172.21.15.112:36224) 2024-04-25T01:36:49.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50eb7fe640 1 -- 172.21.15.112:0/2768409988 learned_addr learned my addr 172.21.15.112:0/2768409988 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:36:49.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50eaffd640 1 --2- 172.21.15.112:0/2768409988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d80039b0 0x7f50ec141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:49.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50eb7fe640 1 -- 172.21.15.112:0/2768409988 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d8004800 msgr2=0x7f50ec0ace00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:36:49.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50eb7fe640 1 --2- 172.21.15.112:0/2768409988 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d8004800 0x7f50ec0ace00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:49.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50eb7fe640 1 -- 172.21.15.112:0/2768409988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d80039b0 msgr2=0x7f50ec141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:49.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50eb7fe640 1 --2- 172.21.15.112:0/2768409988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d80039b0 0x7f50ec141d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:49.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50eb7fe640 1 -- 172.21.15.112:0/2768409988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ec1423c0 con 0x7f50d8005210 2024-04-25T01:36:49.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50eaffd640 1 --2- 172.21.15.112:0/2768409988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d80039b0 0x7f50ec141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:36:49.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50eb7fe640 1 --2- 172.21.15.112:0/2768409988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50d8005210 0x7f50ec0c2600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f50e402b490 tx=0x7f50e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:49.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50e37fe640 1 -- 172.21.15.112:0/2768409988 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4003e30 con 0x7f50d8005210 2024-04-25T01:36:49.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50e37fe640 1 -- 172.21.15.112:0/2768409988 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e4044070 con 0x7f50d8005210 2024-04-25T01:36:49.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50e37fe640 1 -- 172.21.15.112:0/2768409988 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4035de0 con 0x7f50d8005210 2024-04-25T01:36:49.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50f17c1640 1 -- 172.21.15.112:0/2768409988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50ec1426a0 con 0x7f50d8005210 2024-04-25T01:36:49.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.868+0000 7f50f17c1640 1 -- 172.21.15.112:0/2768409988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50ec1eb880 con 0x7f50d8005210 2024-04-25T01:36:49.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.869+0000 7f50f17c1640 1 -- 172.21.15.112:0/2768409988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50ec0c54e0 con 0x7f50d8005210 2024-04-25T01:36:49.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.870+0000 7f50e37fe640 1 -- 172.21.15.112:0/2768409988 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f50e4049460 con 0x7f50d8005210 2024-04-25T01:36:49.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.870+0000 7f50e37fe640 1 --2- 172.21.15.112:0/2768409988 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f50cc06c590 0x7f50cc06ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:49.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.870+0000 7f50eaffd640 1 --2- 172.21.15.112:0/2768409988 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f50cc06c590 0x7f50cc06ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:49.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.871+0000 7f50e37fe640 1 -- 172.21.15.112:0/2768409988 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f50e40ac2b0 con 0x7f50d8005210 2024-04-25T01:36:49.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.871+0000 7f50eaffd640 1 --2- 172.21.15.112:0/2768409988 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f50cc06c590 0x7f50cc06ea50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f50dc007fd0 tx=0x7f50dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:49.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:49.876+0000 7f50e37fe640 1 -- 172.21.15.112:0/2768409988 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f50ec0c54e0 con 0x7f50d8005210 2024-04-25T01:36:50.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.114+0000 7f50f17c1640 1 -- 172.21.15.112:0/2768409988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f50ec0689b0 con 0x7f50d8005210 2024-04-25T01:36:50.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.114+0000 7f50e37fe640 1 -- 172.21.15.112:0/2768409988 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7f50e407a830 con 0x7f50d8005210 2024-04-25T01:36:50.091 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":92,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:36:50.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.116+0000 7f50f17c1640 1 -- 172.21.15.112:0/2768409988 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f50cc06c590 msgr2=0x7f50cc06ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.116+0000 7f50f17c1640 1 --2- 172.21.15.112:0/2768409988 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f50cc06c590 0x7f50cc06ea50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f50dc007fd0 tx=0x7f50dc009040 comp rx=0 tx=0).stop 2024-04-25T01:36:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.116+0000 7f50f17c1640 1 -- 172.21.15.112:0/2768409988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50d8005210 msgr2=0x7f50ec0c2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.116+0000 7f50f17c1640 1 --2- 172.21.15.112:0/2768409988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50d8005210 0x7f50ec0c2600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f50e402b490 tx=0x7f50e4004a40 comp rx=0 tx=0).stop 2024-04-25T01:36:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.116+0000 7f50f17c1640 1 -- 172.21.15.112:0/2768409988 shutdown_connections 2024-04-25T01:36:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.117+0000 7f50f17c1640 1 --2- 172.21.15.112:0/2768409988 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f50cc06c590 0x7f50cc06ea50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.117+0000 7f50f17c1640 1 --2- 172.21.15.112:0/2768409988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50d8005210 0x7f50ec0c2600 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.117+0000 7f50f17c1640 1 --2- 172.21.15.112:0/2768409988 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d8004800 0x7f50ec0ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.117+0000 7f50f17c1640 1 --2- 172.21.15.112:0/2768409988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d80039b0 0x7f50ec141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.117+0000 7f50f17c1640 1 -- 172.21.15.112:0/2768409988 >> 172.21.15.112:0/2768409988 conn(0x7f50ec0c9140 msgr2=0x7f50ec0c7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.117+0000 7f50f17c1640 1 -- 172.21.15.112:0/2768409988 shutdown_connections 2024-04-25T01:36:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.117+0000 7f50f17c1640 1 -- 172.21.15.112:0/2768409988 wait complete. 2024-04-25T01:36:50.104 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T01:36:50.104 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-04-25T01:36:50.105 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:36:50.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.226+0000 7fbfaf9e7640 1 Processor -- start 2024-04-25T01:36:50.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.226+0000 7fbfaf9e7640 1 -- start start 2024-04-25T01:36:50.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.227+0000 7fbfaf9e7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbfa814c3f0 0x7fbfa80c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:50.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.227+0000 7fbfaf9e7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbfa80ac3b0 0x7fbfa8155f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:50.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.227+0000 7fbfaf9e7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfa81524a0 0x7fbfa8156460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:50.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.227+0000 7fbfaf9e7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fbfa8155a90 con 0x7fbfa814c3f0 2024-04-25T01:36:50.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.227+0000 7fbfaf9e7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbfa81569a0 con 0x7fbfa80ac3b0 2024-04-25T01:36:50.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.227+0000 7fbfaf9e7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbfa81529b0 con 0x7fbfa81524a0 2024-04-25T01:36:50.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.227+0000 7fbfad783640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbfa814c3f0 0x7fbfa80c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:50.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.227+0000 7fbfadf84640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfa81524a0 msgr2=0x7fbfa8156460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:36:50.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.227+0000 7fbfad783640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbfa814c3f0 0x7fbfa80c2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36228/0 (socket says 172.21.15.112:36228) 2024-04-25T01:36:50.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.227+0000 7fbfadf84640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfa81524a0 0x7fbfa8156460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:36:50.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.227+0000 7fbfad783640 1 -- 172.21.15.112:0/3691307881 learned_addr learned my addr 172.21.15.112:0/3691307881 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:36:50.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.227+0000 7fbfacf82640 1 --2- 172.21.15.112:0/3691307881 >> v2:172.21.15.187:3300/0 conn(0x7fbfa80ac3b0 0x7fbfa8155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:50.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.227+0000 7fbfad783640 1 -- 172.21.15.112:0/3691307881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfa81524a0 msgr2=0x7fbfa8156460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:36:50.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.227+0000 7fbfad783640 1 --2- 172.21.15.112:0/3691307881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfa81524a0 0x7fbfa8156460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.227+0000 7fbfad783640 1 -- 172.21.15.112:0/3691307881 >> v2:172.21.15.187:3300/0 conn(0x7fbfa80ac3b0 msgr2=0x7fbfa8155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.227+0000 7fbfad783640 1 --2- 172.21.15.112:0/3691307881 >> v2:172.21.15.187:3300/0 conn(0x7fbfa80ac3b0 0x7fbfa8155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.227+0000 7fbfad783640 1 -- 172.21.15.112:0/3691307881 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa8152c70 con 0x7fbfa814c3f0 2024-04-25T01:36:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.227+0000 7fbfacf82640 1 --2- 172.21.15.112:0/3691307881 >> v2:172.21.15.187:3300/0 conn(0x7fbfa80ac3b0 0x7fbfa8155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:36:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.228+0000 7fbfad783640 1 --2- 172.21.15.112:0/3691307881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa814c3f0 0x7fbfa80c2b90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80ac800 tx=0x7fbfa402c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917129c4f31842ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.228+0000 7fbf9ffff640 1 -- 172.21.15.112:0/3691307881 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4039070 con 0x7fbfa814c3f0 2024-04-25T01:36:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.228+0000 7fbf9ffff640 1 -- 172.21.15.112:0/3691307881 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfa4032520 con 0x7fbfa814c3f0 2024-04-25T01:36:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.228+0000 7fbf9ffff640 1 -- 172.21.15.112:0/3691307881 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa40328f0 con 0x7fbfa814c3f0 2024-04-25T01:36:50.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.228+0000 7fbfaf9e7640 1 -- 172.21.15.112:0/3691307881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa814c3f0 msgr2=0x7fbfa80c2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:50.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.228+0000 7fbfaf9e7640 1 --2- 172.21.15.112:0/3691307881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa814c3f0 0x7fbfa80c2b90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80ac800 tx=0x7fbfa402c820 comp rx=0 tx=0).stop 2024-04-25T01:36:50.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.228+0000 7fbfaf9e7640 1 -- 172.21.15.112:0/3691307881 shutdown_connections 2024-04-25T01:36:50.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.228+0000 7fbfaf9e7640 1 --2- 172.21.15.112:0/3691307881 >> v2:172.21.15.187:3300/0 conn(0x7fbfa80ac3b0 0x7fbfa8155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.228+0000 7fbfaf9e7640 1 --2- 172.21.15.112:0/3691307881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfa81524a0 0x7fbfa8156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.228+0000 7fbfaf9e7640 1 --2- 172.21.15.112:0/3691307881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa814c3f0 0x7fbfa80c2b90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.228+0000 7fbfaf9e7640 1 -- 172.21.15.112:0/3691307881 >> 172.21.15.112:0/3691307881 conn(0x7fbfa806b160 msgr2=0x7fbfa806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:50.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.229+0000 7fbfaf9e7640 1 -- 172.21.15.112:0/3691307881 shutdown_connections 2024-04-25T01:36:50.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.229+0000 7fbfaf9e7640 1 -- 172.21.15.112:0/3691307881 wait complete. 2024-04-25T01:36:50.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.230+0000 7fbfaf9e7640 1 Processor -- start 2024-04-25T01:36:50.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.230+0000 7fbfaf9e7640 1 -- start start 2024-04-25T01:36:50.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.230+0000 7fbfaf9e7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfa80ac3b0 0x7fbfa8144040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:50.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.230+0000 7fbfaf9e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa814c3f0 0x7fbfa81dce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:50.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.230+0000 7fbfaf9e7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfa81524a0 0x7fbfa81dd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:50.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.230+0000 7fbfaf9e7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa8070d60 con 0x7fbfa814c3f0 2024-04-25T01:36:50.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.230+0000 7fbfaf9e7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa8153230 con 0x7fbfa81524a0 2024-04-25T01:36:50.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.230+0000 7fbfaf9e7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbfa814c7d0 con 0x7fbfa80ac3b0 2024-04-25T01:36:50.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.230+0000 7fbfacf82640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa814c3f0 0x7fbfa81dce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:50.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.230+0000 7fbfad783640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfa80ac3b0 msgr2=0x7fbfa8144040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:36:50.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.230+0000 7fbfacf82640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa814c3f0 0x7fbfa81dce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36232/0 (socket says 172.21.15.112:36232) 2024-04-25T01:36:50.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.230+0000 7fbfacf82640 1 -- 172.21.15.112:0/928597331 learned_addr learned my addr 172.21.15.112:0/928597331 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:36:50.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.230+0000 7fbfad783640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfa80ac3b0 0x7fbfa8144040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:36:50.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.231+0000 7fbfadf84640 1 --2- 172.21.15.112:0/928597331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfa81524a0 0x7fbfa81dd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:50.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.231+0000 7fbfacf82640 1 -- 172.21.15.112:0/928597331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfa80ac3b0 msgr2=0x7fbfa8144040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:36:50.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.231+0000 7fbfacf82640 1 --2- 172.21.15.112:0/928597331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfa80ac3b0 0x7fbfa8144040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.231+0000 7fbfacf82640 1 -- 172.21.15.112:0/928597331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfa81524a0 msgr2=0x7fbfa81dd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:50.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.231+0000 7fbfacf82640 1 --2- 172.21.15.112:0/928597331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfa81524a0 0x7fbfa81dd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.231+0000 7fbfacf82640 1 -- 172.21.15.112:0/928597331 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa81de600 con 0x7fbfa814c3f0 2024-04-25T01:36:50.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.231+0000 7fbfacf82640 1 --2- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa814c3f0 0x7fbfa81dce20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800c7f0 tx=0x7fbf9800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:50.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.231+0000 7fbf9dffb640 1 -- 172.21.15.112:0/928597331 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98015070 con 0x7fbfa814c3f0 2024-04-25T01:36:50.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.231+0000 7fbf9dffb640 1 -- 172.21.15.112:0/928597331 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf98007a30 con 0x7fbfa814c3f0 2024-04-25T01:36:50.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.231+0000 7fbf9dffb640 1 -- 172.21.15.112:0/928597331 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98007e00 con 0x7fbfa814c3f0 2024-04-25T01:36:50.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.231+0000 7fbfaf9e7640 1 -- 172.21.15.112:0/928597331 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa814aa70 con 0x7fbfa814c3f0 2024-04-25T01:36:50.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.231+0000 7fbfaf9e7640 1 -- 172.21.15.112:0/928597331 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa814ad10 con 0x7fbfa814c3f0 2024-04-25T01:36:50.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.232+0000 7fbfaf9e7640 1 --2- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa8144580 0x7fbfa81dbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:50.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.232+0000 7fbfaf9e7640 1 -- 172.21.15.112:0/928597331 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa8075980 con 0x7fbfa8144580 2024-04-25T01:36:50.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.232+0000 7fbfad783640 1 --2- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa8144580 0x7fbfa81dbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:50.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.232+0000 7fbf9dffb640 1 -- 172.21.15.112:0/928597331 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf98018070 con 0x7fbfa814c3f0 2024-04-25T01:36:50.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.232+0000 7fbfad783640 1 -- 172.21.15.112:0/928597331 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbfa814b0f0 con 0x7fbfa8144580 2024-04-25T01:36:50.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.232+0000 7fbfad783640 1 --2- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa8144580 0x7fbfa81dbbb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80c4c10 tx=0x7fbfa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:50.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.233+0000 7fbf9dffb640 1 --2- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbf8c06be10 0x7fbf8c06e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:50.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.233+0000 7fbf9dffb640 1 -- 172.21.15.112:0/928597331 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fbf9808b130 con 0x7fbfa814c3f0 2024-04-25T01:36:50.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.233+0000 7fbfadf84640 1 --2- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbf8c06be10 0x7fbf8c06e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:50.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.233+0000 7fbf9dffb640 1 -- 172.21.15.112:0/928597331 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4039040 con 0x7fbfa8144580 2024-04-25T01:36:50.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.233+0000 7fbf9dffb640 1 -- 172.21.15.112:0/928597331 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbfa400b730 con 0x7fbfa8144580 2024-04-25T01:36:50.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.233+0000 7fbf9dffb640 1 -- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa8144580 msgr2=0x7fbfa81dbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:50.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.233+0000 7fbf9dffb640 1 --2- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa8144580 0x7fbfa81dbbb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80c4c10 tx=0x7fbfa40046c0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.233+0000 7fbf9dffb640 1 -- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa8144580 msgr2=0x7fbfa81dbbb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:36:50.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.233+0000 7fbf9dffb640 1 --2- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa8144580 0x7fbfa81dbbb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.233+0000 7fbfadf84640 1 --2- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbf8c06be10 0x7fbf8c06e2d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0003cb0 tx=0x7fbfa0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:50.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.239+0000 7fbfaf9e7640 1 --2- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa814b880 0x7fbfa81dc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:50.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.239+0000 7fbfaf9e7640 1 -- 172.21.15.112:0/928597331 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa80682a0 con 0x7fbfa814b880 2024-04-25T01:36:50.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.239+0000 7fbfad783640 1 --2- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa814b880 0x7fbfa81dc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.240+0000 7fbfad783640 1 -- 172.21.15.112:0/928597331 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbfa80aaf90 con 0x7fbfa814b880 2024-04-25T01:36:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.240+0000 7fbfad783640 1 --2- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa814b880 0x7fbfa81dc0f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4000c00 tx=0x7fbfa400b450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.240+0000 7fbf9dffb640 1 -- 172.21.15.112:0/928597331 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4039070 con 0x7fbfa814b880 2024-04-25T01:36:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.240+0000 7fbf9dffb640 1 -- 172.21.15.112:0/928597331 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4549 (secure 0 0 0) 0x7fbfa4003e80 con 0x7fbfa814b880 2024-04-25T01:36:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.240+0000 7fbf9dffb640 1 -- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa814b880 msgr2=0x7fbfa81dc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.240+0000 7fbf9dffb640 1 --2- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa814b880 0x7fbfa81dc0f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4000c00 tx=0x7fbfa400b450 comp rx=0 tx=0).stop 2024-04-25T01:36:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.240+0000 7fbf9dffb640 1 -- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa814b880 msgr2=0x7fbfa81dc0f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:36:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.240+0000 7fbf9dffb640 1 --2- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa814b880 0x7fbfa81dc0f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.217 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:36:50.217 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:36:50.217 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:36:50.217 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:36:50.217 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 92, 2024-04-25T01:36:50.217 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 61055, 2024-04-25T01:36:50.217 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:36:50.218 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:36:50.218 INFO:teuthology.orchestra.run.smithi112.stdout: 1 2024-04-25T01:36:50.218 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:50.218 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:36:50.218 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:36:50.218 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:36:50.218 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:36:50.218 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:50.218 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:50.218 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:50.218 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:50.218 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:50.218 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:50.218 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:50.219 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:50.219 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:50.219 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:50.219 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:50.219 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:50.219 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:36:50.219 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:36:50.219 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:50.219 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:50.219 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:50.219 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:50.219 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:50.219 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:50.219 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:50.219 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:50.220 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:50.220 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:50.220 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:50.220 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:50.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.220 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:36:50.220 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:36:50.220 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:36:50.220 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:36:50.220 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:36:50.220 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:36:50.220 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:36:50.220 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:36:50.220 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:36:50.220 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:36:50.221 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:36:50.221 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:36:50.221 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:36:50.221 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:36:50.221 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:36:50.221 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:36:50.221 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:36:50.221 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:50.221 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:50.221 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:50.221 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:50.221 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:50.221 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:50.221 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:50.222 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:50.222 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:50.222 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:50.222 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:50.222 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:50.222 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:36:50.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.222 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:36:50.222 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.222 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:36:50.222 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:36:50.222 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:50.222 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:50.222 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.222 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:50.223 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:36:50.223 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:50.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.223 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.223 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:50.223 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:36:50.223 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:50.223 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:50.223 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:50.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.223 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:36:50.223 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:36:50.223 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:50.223 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:50.223 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:50.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.224 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.224 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:36:50.224 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:36:50.224 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:50.224 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:50.224 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.224 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:50.224 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:36:50.224 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:50.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.224 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.224 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:50.224 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:36:50.224 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:50.224 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:50.224 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:50.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.225 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:36:50.225 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:36:50.225 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:50.225 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:50.225 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:50.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.225 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.225 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:36:50.225 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:36:50.225 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:50.225 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:50.225 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.225 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:50.225 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:36:50.225 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:50.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.226 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.226 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:50.226 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:36:50.226 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:50.226 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:50.226 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:50.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.226 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:36:50.226 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:36:50.226 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:50.226 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:50.226 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:50.226 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:50.226 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:50.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.227 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:36:50.227 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:36:50.227 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.227 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:50.227 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:50.227 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:36:50.227 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:50.227 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:50.227 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:36:50.227 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.227 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:50.227 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:50.227 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:36:50.227 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:50.228 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:50.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.228 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:36:50.228 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:36:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.240+0000 7fbfaf9e7640 1 -- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbf8c06be10 msgr2=0x7fbf8c06e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.240+0000 7fbfaf9e7640 1 --2- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbf8c06be10 0x7fbf8c06e2d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0003cb0 tx=0x7fbfa0008040 comp rx=0 tx=0).stop 2024-04-25T01:36:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.240+0000 7fbfaf9e7640 1 -- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa814c3f0 msgr2=0x7fbfa81dce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.240+0000 7fbfaf9e7640 1 --2- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa814c3f0 0x7fbfa81dce20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800c7f0 tx=0x7fbf9800cbf0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.240+0000 7fbfadf84640 1 -- 172.21.15.112:0/928597331 reap_dead start 2024-04-25T01:36:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.241+0000 7fbfaf9e7640 1 -- 172.21.15.112:0/928597331 shutdown_connections 2024-04-25T01:36:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.241+0000 7fbfaf9e7640 1 --2- 172.21.15.112:0/928597331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa814c3f0 0x7fbfa81dce20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.241+0000 7fbfaf9e7640 1 -- 172.21.15.112:0/928597331 >> 172.21.15.112:0/928597331 conn(0x7fbfa806b160 msgr2=0x7fbfa8071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.241+0000 7fbfaf9e7640 1 -- 172.21.15.112:0/928597331 shutdown_connections 2024-04-25T01:36:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.241+0000 7fbfaf9e7640 1 -- 172.21.15.112:0/928597331 wait complete. 2024-04-25T01:36:50.230 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:36:50.230 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-04-25T01:36:50.230 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:36:50.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.351+0000 7fefb51ce640 1 Processor -- start 2024-04-25T01:36:50.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.351+0000 7fefb51ce640 1 -- start start 2024-04-25T01:36:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.352+0000 7fefb51ce640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fefb014c3f0 0x7fefb00c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.352+0000 7fefb51ce640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefb01529b0 con 0x7fefb00ac3b0 2024-04-25T01:36:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.352+0000 7fefb51ce640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefb0152c70 con 0x7fefb01524a0 2024-04-25T01:36:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.352+0000 7fefb51ce640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fefb0155a90 con 0x7fefb014c3f0 2024-04-25T01:36:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.352+0000 7fefae59c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fefb00ac3b0 0x7fefb0155f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57720/0 (socket says 172.21.15.112:57720) 2024-04-25T01:36:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.352+0000 7fefae59c640 1 -- 172.21.15.112:0/3756019734 learned_addr learned my addr 172.21.15.112:0/3756019734 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:36:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.352+0000 7fefadd9b640 1 -- 172.21.15.112:0/3756019734 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 59276211 0 0) 0x7fefb01529b0 con 0x7fefb00ac3b0 2024-04-25T01:36:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.352+0000 7fefaed9d640 1 --2- 172.21.15.112:0/3756019734 >> v2:172.21.15.187:3300/0 conn(0x7fefb014c3f0 0x7fefb00c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.352+0000 7fefadd9b640 1 -- 172.21.15.112:0/3756019734 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef98003520 con 0x7fefb00ac3b0 2024-04-25T01:36:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefadd9b640 1 -- 172.21.15.112:0/3756019734 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1366716113 0 0) 0x7fef98003520 con 0x7fefb00ac3b0 2024-04-25T01:36:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefadd9b640 1 -- 172.21.15.112:0/3756019734 >> v2:172.21.15.187:3300/0 conn(0x7fefb014c3f0 msgr2=0x7fefb00c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefadd9b640 1 --2- 172.21.15.112:0/3756019734 >> v2:172.21.15.187:3300/0 conn(0x7fefb014c3f0 0x7fefb00c2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefadd9b640 1 -- 172.21.15.112:0/3756019734 >> v1:172.21.15.187:6789/0 conn(0x7fefb01524a0 legacy=0x7fefb015ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:50.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefadd9b640 1 -- 172.21.15.112:0/3756019734 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb0152f50 con 0x7fefb00ac3b0 2024-04-25T01:36:50.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefadd9b640 1 -- 172.21.15.112:0/3756019734 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fefa4001670 con 0x7fefb00ac3b0 2024-04-25T01:36:50.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefadd9b640 1 -- 172.21.15.112:0/3756019734 >> v1:172.21.15.112:6789/0 conn(0x7fefb00ac3b0 legacy=0x7fefb0155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:50.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefadd9b640 1 --2- 172.21.15.112:0/3756019734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef980039b0 0x7fef98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:50.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefadd9b640 1 --2- 172.21.15.112:0/3756019734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef98004800 0x7fef98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:50.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefadd9b640 1 --2- 172.21.15.112:0/3756019734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef980051c0 0x7fef9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:50.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefadd9b640 1 -- 172.21.15.112:0/3756019734 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fefb0155a90 con 0x7fef980051c0 2024-04-25T01:36:50.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefadd9b640 1 -- 172.21.15.112:0/3756019734 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fef9800c160 con 0x7fef980039b0 2024-04-25T01:36:50.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefadd9b640 1 -- 172.21.15.112:0/3756019734 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fef9800c360 con 0x7fef98004800 2024-04-25T01:36:50.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefae59c640 1 -- 172.21.15.112:0/3756019734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef98004800 msgr2=0x7fef98004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:36:50.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefae59c640 1 --2- 172.21.15.112:0/3756019734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef98004800 0x7fef98004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:36:50.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefaf59e640 1 --2- 172.21.15.112:0/3756019734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef980051c0 0x7fef9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:50.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefaed9d640 1 --2- 172.21.15.112:0/3756019734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef980039b0 0x7fef98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:50.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefaf59e640 1 -- 172.21.15.112:0/3756019734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef98004800 msgr2=0x7fef98004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:36:50.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefaf59e640 1 --2- 172.21.15.112:0/3756019734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef98004800 0x7fef98004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefaf59e640 1 -- 172.21.15.112:0/3756019734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef980039b0 msgr2=0x7fef98003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:36:50.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefaf59e640 1 --2- 172.21.15.112:0/3756019734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef980039b0 0x7fef98003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.353+0000 7fefaf59e640 1 -- 172.21.15.112:0/3756019734 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef98003520 con 0x7fef980051c0 2024-04-25T01:36:50.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.354+0000 7fefaf59e640 1 -- 172.21.15.112:0/3756019734 reap_dead start 2024-04-25T01:36:50.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.354+0000 7fefaf59e640 1 --2- 172.21.15.112:0/3756019734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef980051c0 0x7fef9800b9c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fefa8001750 tx=0x7fef98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49eac254ff4efdfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:50.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.354+0000 7fefadd9b640 1 -- 172.21.15.112:0/3756019734 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa8001370 con 0x7fef980051c0 2024-04-25T01:36:50.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.354+0000 7fefadd9b640 1 -- 172.21.15.112:0/3756019734 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefa80052f0 con 0x7fef980051c0 2024-04-25T01:36:50.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.354+0000 7fefadd9b640 1 -- 172.21.15.112:0/3756019734 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa802abe0 con 0x7fef980051c0 2024-04-25T01:36:50.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.354+0000 7fefb51ce640 1 -- 172.21.15.112:0/3756019734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef980051c0 msgr2=0x7fef9800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:50.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.354+0000 7fefb51ce640 1 --2- 172.21.15.112:0/3756019734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef980051c0 0x7fef9800b9c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fefa8001750 tx=0x7fef98005830 comp rx=0 tx=0).stop 2024-04-25T01:36:50.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.354+0000 7fefb51ce640 1 -- 172.21.15.112:0/3756019734 shutdown_connections 2024-04-25T01:36:50.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.354+0000 7fefb51ce640 1 --2- 172.21.15.112:0/3756019734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef980051c0 0x7fef9800b9c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.354+0000 7fefb51ce640 1 -- 172.21.15.112:0/3756019734 >> 172.21.15.112:0/3756019734 conn(0x7fefb006b160 msgr2=0x7fefb006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:50.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.354+0000 7fefb51ce640 1 -- 172.21.15.112:0/3756019734 shutdown_connections 2024-04-25T01:36:50.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.354+0000 7fefb51ce640 1 -- 172.21.15.112:0/3756019734 wait complete. 2024-04-25T01:36:50.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.356+0000 7fefb51ce640 1 Processor -- start 2024-04-25T01:36:50.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.356+0000 7fefb51ce640 1 -- start start 2024-04-25T01:36:50.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.356+0000 7fefb51ce640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef980051c0 0x7fefb0141e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:50.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.356+0000 7fefb51ce640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefb014f940 0x7fefb006d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:50.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fefb51ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefb01e8650 0x7fefb01e8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:50.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fefb51ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fefb014c7d0 con 0x7fefb01e8650 2024-04-25T01:36:50.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fefb51ce640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fefb0070d60 con 0x7fef980051c0 2024-04-25T01:36:50.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fefb51ce640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fefb014c950 con 0x7fefb014f940 2024-04-25T01:36:50.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fefaf59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefb01e8650 0x7fefb01e8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:50.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fefaf59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefb01e8650 0x7fefb01e8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36264/0 (socket says 172.21.15.112:36264) 2024-04-25T01:36:50.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fefae59c640 1 -- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefb014f940 msgr2=0x7fefb006d580 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3301/0 2024-04-25T01:36:50.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fefaf59e640 1 -- 172.21.15.112:0/4178773758 learned_addr learned my addr 172.21.15.112:0/4178773758 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:36:50.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fefae59c640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefb014f940 0x7fefb006d580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:36:50.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fefaed9d640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef980051c0 0x7fefb0141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:50.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fefaf59e640 1 -- 172.21.15.112:0/4178773758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefb014f940 msgr2=0x7fefb006d580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:36:50.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fefaf59e640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefb014f940 0x7fefb006d580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fefaf59e640 1 -- 172.21.15.112:0/4178773758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef980051c0 msgr2=0x7fefb0141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:50.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fefaf59e640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef980051c0 0x7fefb0141e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fefaf59e640 1 -- 172.21.15.112:0/4178773758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb01418d0 con 0x7fefb01e8650 2024-04-25T01:36:50.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fefaf59e640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefb01e8650 0x7fefb01e8ae0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fefa802a590 tx=0x7fefa8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:50.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fef977fe640 1 -- 172.21.15.112:0/4178773758 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa8007a30 con 0x7fefb01e8650 2024-04-25T01:36:50.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fef977fe640 1 -- 172.21.15.112:0/4178773758 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefa80054a0 con 0x7fefb01e8650 2024-04-25T01:36:50.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fefb51ce640 1 -- 172.21.15.112:0/4178773758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb01e8150 con 0x7fefb01e8650 2024-04-25T01:36:50.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fef977fe640 1 -- 172.21.15.112:0/4178773758 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa8035de0 con 0x7fefb01e8650 2024-04-25T01:36:50.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.357+0000 7fefb51ce640 1 -- 172.21.15.112:0/4178773758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb014d4e0 con 0x7fefb01e8650 2024-04-25T01:36:50.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.358+0000 7fefb51ce640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefb014d6b0 0x7fefb0140c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:50.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.358+0000 7fefb51ce640 1 -- 172.21.15.112:0/4178773758 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fefb01e8370 con 0x7fefb014d6b0 2024-04-25T01:36:50.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.358+0000 7fefaed9d640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefb014d6b0 0x7fefb0140c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:50.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.359+0000 7fef977fe640 1 -- 172.21.15.112:0/4178773758 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fefa8007030 con 0x7fefb01e8650 2024-04-25T01:36:50.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.359+0000 7fefaed9d640 1 -- 172.21.15.112:0/4178773758 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fefb0075980 con 0x7fefb014d6b0 2024-04-25T01:36:50.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.359+0000 7fef977fe640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fef9c06be10 0x7fef9c06e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:50.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.359+0000 7fefae59c640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fef9c06be10 0x7fef9c06e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:50.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.359+0000 7fef977fe640 1 -- 172.21.15.112:0/4178773758 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fefa80ac5a0 con 0x7fefb01e8650 2024-04-25T01:36:50.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.359+0000 7fefaed9d640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefb014d6b0 0x7fefb0140c10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fefa0014ef0 tx=0x7fefa0012670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:50.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.359+0000 7fef977fe640 1 -- 172.21.15.112:0/4178773758 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0016810 con 0x7fefb014d6b0 2024-04-25T01:36:50.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.359+0000 7fefae59c640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fef9c06be10 0x7fef9c06e2d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fefa40018e0 tx=0x7fefa40015f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:50.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.360+0000 7fef977fe640 1 -- 172.21.15.112:0/4178773758 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fefa0012e90 con 0x7fefb014d6b0 2024-04-25T01:36:50.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.360+0000 7fef977fe640 1 -- 172.21.15.112:0/4178773758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefb014d6b0 msgr2=0x7fefb0140c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:50.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.360+0000 7fef977fe640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefb014d6b0 0x7fefb0140c10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fefa0014ef0 tx=0x7fefa0012670 comp rx=0 tx=0).stop 2024-04-25T01:36:50.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.360+0000 7fef977fe640 1 -- 172.21.15.112:0/4178773758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefb014d6b0 msgr2=0x7fefb0140c10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:36:50.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.360+0000 7fef977fe640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefb014d6b0 0x7fefb0140c10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.366+0000 7fefb51ce640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefb0141150 0x7fefb01eb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:50.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.366+0000 7fefb51ce640 1 -- 172.21.15.112:0/4178773758 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fefb00682a0 con 0x7fefb0141150 2024-04-25T01:36:50.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.366+0000 7fefaed9d640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefb0141150 0x7fefb01eb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:50.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.367+0000 7fefaed9d640 1 -- 172.21.15.112:0/4178773758 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fefb00aaf90 con 0x7fefb0141150 2024-04-25T01:36:50.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.367+0000 7fefaed9d640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefb0141150 0x7fefb01eb6c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fefa0014ef0 tx=0x7fefa0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:50.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.367+0000 7fef977fe640 1 -- 172.21.15.112:0/4178773758 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa00252b0 con 0x7fefb0141150 2024-04-25T01:36:50.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.367+0000 7fef977fe640 1 -- 172.21.15.112:0/4178773758 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5058 (secure 0 0 0) 0x7fefa00252b0 con 0x7fefb0141150 2024-04-25T01:36:50.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.368+0000 7fef977fe640 1 -- 172.21.15.112:0/4178773758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefb0141150 msgr2=0x7fefb01eb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:50.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.368+0000 7fef977fe640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefb0141150 0x7fefb01eb6c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fefa0014ef0 tx=0x7fefa0004ab0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.368+0000 7fef977fe640 1 -- 172.21.15.112:0/4178773758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefb0141150 msgr2=0x7fefb01eb6c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:36:50.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.368+0000 7fef977fe640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefb0141150 0x7fefb01eb6c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.344 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:36:50.344 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:36:50.344 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:36:50.344 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:36:50.344 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 92, 2024-04-25T01:36:50.344 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 446056, 2024-04-25T01:36:50.345 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:36:50.345 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:36:50.345 INFO:teuthology.orchestra.run.smithi112.stdout: 1 2024-04-25T01:36:50.345 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:50.345 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:36:50.345 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:36:50.345 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:36:50.345 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:36:50.345 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:50.345 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:50.345 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:50.345 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:50.345 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:50.345 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:50.346 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:50.346 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:50.346 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:50.346 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:50.346 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:50.346 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:50.346 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:36:50.346 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:36:50.346 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:50.346 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:50.346 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:50.346 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:50.346 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:50.346 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:50.347 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:50.347 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:50.347 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:50.347 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:50.347 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:50.347 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:50.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.347 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:36:50.347 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:36:50.347 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:36:50.347 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:36:50.347 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:36:50.347 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:36:50.347 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:36:50.347 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:36:50.348 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:36:50.348 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:36:50.348 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:36:50.348 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:36:50.348 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:36:50.348 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:36:50.348 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:36:50.348 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:36:50.348 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:36:50.348 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:36:50.348 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:36:50.348 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:36:50.348 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:36:50.348 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:36:50.348 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:36:50.349 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:36:50.349 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:36:50.349 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:36:50.349 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:36:50.349 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:36:50.349 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:50.349 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:36:50.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.349 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:36:50.349 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.349 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:36:50.349 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:36:50.349 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:50.349 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:50.349 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.349 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:50.350 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:36:50.350 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:50.350 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.350 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.350 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:50.350 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:36:50.350 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:50.350 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:50.350 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:50.350 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.350 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:36:50.350 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:36:50.350 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:50.350 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:50.351 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:50.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.351 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:36:50.351 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:36:50.351 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:50.351 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:50.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.351 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:50.351 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:36:50.351 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:50.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.351 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:50.351 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:36:50.352 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:50.352 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:50.352 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:50.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.352 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:36:50.352 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:36:50.352 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:50.352 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:50.352 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:50.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.352 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.352 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:36:50.352 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:36:50.352 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:36:50.352 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:36:50.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.353 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:36:50.353 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:36:50.353 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:50.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.353 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:36:50.353 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:36:50.353 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:36:50.353 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:50.353 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:50.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.353 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:36:50.353 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:36:50.353 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:36:50.353 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:36:50.354 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:36:50.354 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:50.354 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:50.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.354 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:36:50.354 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:36:50.354 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.354 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:50.354 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:50.354 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:36:50.354 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:50.354 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:50.354 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:36:50.354 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.355 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:50.355 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:50.355 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:36:50.355 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:50.355 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:50.355 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:36:50.355 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.355 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:50.355 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:50.355 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 8 2024-04-25T01:36:50.355 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:50.355 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:50.355 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:36:50.355 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.355 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:50.356 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:50.356 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 6 2024-04-25T01:36:50.356 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:50.356 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:36:50.356 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:36:50.356 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:36:50.356 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:36:50.356 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:36:50.356 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:36:50.356 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:36:50.356 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:36:50.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:36:50.356 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:36:50.356 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:36:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.368+0000 7fefb51ce640 1 -- 172.21.15.112:0/4178773758 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fef9c06be10 msgr2=0x7fef9c06e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.368+0000 7fefb51ce640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fef9c06be10 0x7fef9c06e2d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fefa40018e0 tx=0x7fefa40015f0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.368+0000 7fefb51ce640 1 -- 172.21.15.112:0/4178773758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefb01e8650 msgr2=0x7fefb01e8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.368+0000 7fefb51ce640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefb01e8650 0x7fefb01e8ae0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fefa802a590 tx=0x7fefa8001120 comp rx=0 tx=0).stop 2024-04-25T01:36:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.368+0000 7fefaf59e640 1 -- 172.21.15.112:0/4178773758 reap_dead start 2024-04-25T01:36:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.368+0000 7fefb51ce640 1 -- 172.21.15.112:0/4178773758 shutdown_connections 2024-04-25T01:36:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.368+0000 7fefb51ce640 1 --2- 172.21.15.112:0/4178773758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefb01e8650 0x7fefb01e8ae0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.368+0000 7fefb51ce640 1 -- 172.21.15.112:0/4178773758 >> 172.21.15.112:0/4178773758 conn(0x7fefb006b160 msgr2=0x7fefb00c3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.368+0000 7fefb51ce640 1 -- 172.21.15.112:0/4178773758 shutdown_connections 2024-04-25T01:36:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:36:50.368+0000 7fefb51ce640 1 -- 172.21.15.112:0/4178773758 wait complete. 2024-04-25T01:36:50.359 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:36:50.359 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T01:36:57.695 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:36:57.698 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:37:10.360 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-25T01:37:10.360 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-04-25T01:37:10.360 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T01:37:10.360 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T01:37:10.363 INFO:tasks.ceph.mon.c:Started 2024-04-25T01:37:10.363 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-04-25T01:37:10.364 INFO:tasks.ceph.mon.b:Sent signal 19 2024-04-25T01:37:10.364 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-04-25T01:37:25.365 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-04-25T01:37:25.366 INFO:tasks.ceph.mon.b:Sent signal 18 2024-04-25T01:37:25.366 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:37:25.366 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:37:25.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.556+0000 7f69f0b8c640 1 Processor -- start 2024-04-25T01:37:25.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.556+0000 7f69f0b8c640 1 -- start start 2024-04-25T01:37:25.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.557+0000 7f69f0b8c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69e40dc160 0x7f69e40409a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:25.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.557+0000 7f69eb7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69e40dc160 0x7f69e40409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:25.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.557+0000 7f69eb7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69e40dc160 0x7f69e40409a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:36240/0 (socket says 172.21.15.112:36240) 2024-04-25T01:37:25.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.557+0000 7f69f0b8c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f69e400da90 0x7f69e400c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:25.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.557+0000 7f69eb7fe640 1 -- 172.21.15.112:0/3263458250 learned_addr learned my addr 172.21.15.112:0/3263458250 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:25.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.558+0000 7f69f0b8c640 1 -- 172.21.15.112:0/3263458250 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69e404e5a0 con 0x7f69e4009640 2024-04-25T01:37:25.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.558+0000 7f69f0b8c640 1 -- 172.21.15.112:0/3263458250 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f69e40deb20 con 0x7f69e400da90 2024-04-25T01:37:25.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.558+0000 7f69f0b8c640 1 -- 172.21.15.112:0/3263458250 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f69e40dee00 con 0x7f69e40dc160 2024-04-25T01:37:25.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.558+0000 7f69eaffd640 1 --2- 172.21.15.112:0/3263458250 >> v2:172.21.15.112:3300/0 conn(0x7f69e400da90 0x7f69e400c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:25.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.559+0000 7f69eb7fe640 1 -- 172.21.15.112:0/3263458250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69e40dc160 msgr2=0x7f69e40409a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:37:25.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.559+0000 7f69eb7fe640 1 -- 172.21.15.112:0/3263458250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69e40dc160 msgr2=0x7f69e40409a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:37:25.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.559+0000 7f69eb7fe640 1 --2- 172.21.15.112:0/3263458250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69e40dc160 0x7f69e40409a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:37:25.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.559+0000 7f69eb7fe640 1 --2- 172.21.15.112:0/3263458250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69e40dc160 0x7f69e40409a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:37:25.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.559+0000 7f69eaffd640 1 -- 172.21.15.112:0/3263458250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69e40dc160 msgr2=0x7f69e40409a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:37:25.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.559+0000 7f69eaffd640 1 --2- 172.21.15.112:0/3263458250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69e40dc160 0x7f69e40409a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:25.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.559+0000 7f69eaffd640 1 -- 172.21.15.112:0/3263458250 >> v1:172.21.15.187:6789/0 conn(0x7f69e4009640 legacy=0x7f69e400caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:25.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.559+0000 7f69eaffd640 1 -- 172.21.15.112:0/3263458250 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e40df0e0 con 0x7f69e400da90 2024-04-25T01:37:25.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.560+0000 7f69eaffd640 1 --2- 172.21.15.112:0/3263458250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69e400da90 0x7f69e400c560 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f69d80097a0 tx=0x7f69d802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71b7afffde65d9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:25.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.561+0000 7f69ea7fc640 1 -- 172.21.15.112:0/3263458250 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d8037070 con 0x7f69e400da90 2024-04-25T01:37:25.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.561+0000 7f69ea7fc640 1 -- 172.21.15.112:0/3263458250 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d8032440 con 0x7f69e400da90 2024-04-25T01:37:25.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.561+0000 7f69ea7fc640 1 -- 172.21.15.112:0/3263458250 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d8032810 con 0x7f69e400da90 2024-04-25T01:37:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.562+0000 7f69f0b8c640 1 -- 172.21.15.112:0/3263458250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69e400da90 msgr2=0x7f69e400c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.562+0000 7f69f0b8c640 1 --2- 172.21.15.112:0/3263458250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69e400da90 0x7f69e400c560 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f69d80097a0 tx=0x7f69d802a180 comp rx=0 tx=0).stop 2024-04-25T01:37:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.562+0000 7f69f0b8c640 1 -- 172.21.15.112:0/3263458250 shutdown_connections 2024-04-25T01:37:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.562+0000 7f69f0b8c640 1 --2- 172.21.15.112:0/3263458250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69e400da90 0x7f69e400c560 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.562+0000 7f69f0b8c640 1 --2- 172.21.15.112:0/3263458250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69e40dc160 0x7f69e40409a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.562+0000 7f69f0b8c640 1 -- 172.21.15.112:0/3263458250 >> 172.21.15.112:0/3263458250 conn(0x7f69e404d9d0 msgr2=0x7f69e404dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.563+0000 7f69f0b8c640 1 -- 172.21.15.112:0/3263458250 shutdown_connections 2024-04-25T01:37:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.563+0000 7f69f0b8c640 1 -- 172.21.15.112:0/3263458250 wait complete. 2024-04-25T01:37:25.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.564+0000 7f69f0b8c640 1 Processor -- start 2024-04-25T01:37:25.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.565+0000 7f69f0b8c640 1 -- start start 2024-04-25T01:37:25.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.566+0000 7f69f0b8c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69e4009640 0x7f69e4174710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:25.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.566+0000 7f69f0b8c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69e400da90 0x7f69e4042d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:25.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.566+0000 7f69f0b8c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69e40dc160 0x7f69e40e0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:25.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.566+0000 7f69f0b8c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f69e400d1b0 con 0x7f69e400da90 2024-04-25T01:37:25.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.566+0000 7f69f0b8c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f69e40df6a0 con 0x7f69e4009640 2024-04-25T01:37:25.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.566+0000 7f69f0b8c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f69e4040ee0 con 0x7f69e40dc160 2024-04-25T01:37:25.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.566+0000 7f69eaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69e400da90 0x7f69e4042d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:25.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.566+0000 7f69eaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69e400da90 0x7f69e4042d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40698/0 (socket says 172.21.15.112:40698) 2024-04-25T01:37:25.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.566+0000 7f69ebfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69e40dc160 0x7f69e40e0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:25.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.567+0000 7f69eaffd640 1 -- 172.21.15.112:0/3010799319 learned_addr learned my addr 172.21.15.112:0/3010799319 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:25.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.567+0000 7f69eb7fe640 1 --2- 172.21.15.112:0/3010799319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69e4009640 0x7f69e4174710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:25.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.567+0000 7f69eb7fe640 1 -- 172.21.15.112:0/3010799319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69e40dc160 msgr2=0x7f69e40e0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:25.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.567+0000 7f69eb7fe640 1 --2- 172.21.15.112:0/3010799319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69e40dc160 0x7f69e40e0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:25.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.567+0000 7f69eb7fe640 1 -- 172.21.15.112:0/3010799319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69e400da90 msgr2=0x7f69e4042d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:25.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.567+0000 7f69eb7fe640 1 --2- 172.21.15.112:0/3010799319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69e400da90 0x7f69e4042d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:25.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.567+0000 7f69eb7fe640 1 -- 172.21.15.112:0/3010799319 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e40439e0 con 0x7f69e4009640 2024-04-25T01:37:25.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.568+0000 7f69eb7fe640 1 --2- 172.21.15.112:0/3010799319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69e4009640 0x7f69e4174710 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f69e000e7e0 tx=0x7f69e000eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:25.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.568+0000 7f69d7fff640 1 -- 172.21.15.112:0/3010799319 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e0017070 con 0x7f69e4009640 2024-04-25T01:37:25.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.568+0000 7f69f0b8c640 1 -- 172.21.15.112:0/3010799319 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e4041cf0 con 0x7f69e4009640 2024-04-25T01:37:25.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.568+0000 7f69f0b8c640 1 -- 172.21.15.112:0/3010799319 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e40422b0 con 0x7f69e4009640 2024-04-25T01:37:25.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.569+0000 7f69d7fff640 1 -- 172.21.15.112:0/3010799319 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e0004020 con 0x7f69e4009640 2024-04-25T01:37:25.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.570+0000 7f69d7fff640 1 -- 172.21.15.112:0/3010799319 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e000f040 con 0x7f69e4009640 2024-04-25T01:37:25.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.571+0000 7f69f0b8c640 1 -- 172.21.15.112:0/3010799319 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e4005380 con 0x7f69e4009640 2024-04-25T01:37:25.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.572+0000 7f69d7fff640 1 -- 172.21.15.112:0/3010799319 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f69e000c6d0 con 0x7f69e4009640 2024-04-25T01:37:25.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.573+0000 7f69d7fff640 1 --2- 172.21.15.112:0/3010799319 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f69d006be60 0x7f69d006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:25.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.573+0000 7f69d7fff640 1 -- 172.21.15.112:0/3010799319 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f69e008b0c0 con 0x7f69e4009640 2024-04-25T01:37:25.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.573+0000 7f69eaffd640 1 --2- 172.21.15.112:0/3010799319 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f69d006be60 0x7f69d006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:25.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.574+0000 7f69eaffd640 1 --2- 172.21.15.112:0/3010799319 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f69d006be60 0x7f69d006e320 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f69e40434e0 tx=0x7f69d802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:25.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:25.582+0000 7f69d7fff640 1 -- 172.21.15.112:0/3010799319 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f69e00590f0 con 0x7f69e4009640 2024-04-25T01:37:26.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.075+0000 7f69f0b8c640 1 -- 172.21.15.112:0/3010799319 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f69e404d1f0 con 0x7f69e4009640 2024-04-25T01:37:26.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.076+0000 7f69d7fff640 1 -- 172.21.15.112:0/3010799319 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1568 (secure 0 0 0) 0x7f69e005d3f0 con 0x7f69e4009640 2024-04-25T01:37:26.055 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":94,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:37:26.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.080+0000 7f69f0b8c640 1 -- 172.21.15.112:0/3010799319 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f69d006be60 msgr2=0x7f69d006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.080+0000 7f69f0b8c640 1 --2- 172.21.15.112:0/3010799319 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f69d006be60 0x7f69d006e320 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f69e40434e0 tx=0x7f69d802e000 comp rx=0 tx=0).stop 2024-04-25T01:37:26.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.080+0000 7f69f0b8c640 1 -- 172.21.15.112:0/3010799319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69e4009640 msgr2=0x7f69e4174710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.080+0000 7f69f0b8c640 1 --2- 172.21.15.112:0/3010799319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69e4009640 0x7f69e4174710 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f69e000e7e0 tx=0x7f69e000eca0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.081+0000 7f69f0b8c640 1 -- 172.21.15.112:0/3010799319 shutdown_connections 2024-04-25T01:37:26.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.081+0000 7f69f0b8c640 1 --2- 172.21.15.112:0/3010799319 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f69d006be60 0x7f69d006e320 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.081+0000 7f69f0b8c640 1 --2- 172.21.15.112:0/3010799319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69e40dc160 0x7f69e40e0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.081+0000 7f69f0b8c640 1 --2- 172.21.15.112:0/3010799319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69e400da90 0x7f69e4042d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.081+0000 7f69f0b8c640 1 --2- 172.21.15.112:0/3010799319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69e4009640 0x7f69e4174710 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.081+0000 7f69f0b8c640 1 -- 172.21.15.112:0/3010799319 >> 172.21.15.112:0/3010799319 conn(0x7f69e404d9d0 msgr2=0x7f69e400c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:26.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.084+0000 7f69f0b8c640 1 -- 172.21.15.112:0/3010799319 shutdown_connections 2024-04-25T01:37:26.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.084+0000 7f69f0b8c640 1 -- 172.21.15.112:0/3010799319 wait complete. 2024-04-25T01:37:26.080 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:37:26.081 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:37:26.081 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:37:26.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.275+0000 7f7ea4814640 1 Processor -- start 2024-04-25T01:37:26.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.275+0000 7f7ea4814640 1 -- start start 2024-04-25T01:37:26.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.276+0000 7f7ea4814640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ea006ee40 0x7f7ea00abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.276+0000 7f7ea4814640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7ea006db50 0x7f7ea006dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.276+0000 7f7ea4814640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ea01576d0 con 0x7f7ea014da70 2024-04-25T01:37:26.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.276+0000 7f7ea4814640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7ea0154550 con 0x7f7ea006db50 2024-04-25T01:37:26.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.277+0000 7f7e9e59c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ea006ee40 0x7f7ea00abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.277+0000 7f7e9e59c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ea006ee40 0x7f7ea00abca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:36266/0 (socket says 172.21.15.112:36266) 2024-04-25T01:37:26.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.277+0000 7f7e9e59c640 1 -- 172.21.15.112:0/105851688 learned_addr learned my addr 172.21.15.112:0/105851688 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:26.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.277+0000 7f7ea4814640 1 -- 172.21.15.112:0/105851688 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea00730b0 con 0x7f7ea006ee40 2024-04-25T01:37:26.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.278+0000 7f7e9f59e640 1 --2- 172.21.15.112:0/105851688 >> v2:172.21.15.187:3300/0 conn(0x7f7ea006db50 0x7f7ea006dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.278+0000 7f7e9f59e640 1 -- 172.21.15.112:0/105851688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ea006ee40 msgr2=0x7f7ea00abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:37:26.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.278+0000 7f7e9f59e640 1 --2- 172.21.15.112:0/105851688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ea006ee40 0x7f7ea00abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.278+0000 7f7e9f59e640 1 -- 172.21.15.112:0/105851688 >> v1:172.21.15.187:6789/0 conn(0x7f7ea014da70 legacy=0x7f7ea0153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.278+0000 7f7e9f59e640 1 -- 172.21.15.112:0/105851688 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea01579b0 con 0x7f7ea006db50 2024-04-25T01:37:26.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.279+0000 7f7e9f59e640 1 --2- 172.21.15.112:0/105851688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ea006db50 0x7f7ea006dfb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009d90 tx=0x7f7e9802ea70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bf66fafb064bdce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:26.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.280+0000 7f7e9dd9b640 1 -- 172.21.15.112:0/105851688 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98032830 con 0x7f7ea006db50 2024-04-25T01:37:26.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.280+0000 7f7e9dd9b640 1 -- 172.21.15.112:0/105851688 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e980329d0 con 0x7f7ea006db50 2024-04-25T01:37:26.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.280+0000 7f7e9dd9b640 1 -- 172.21.15.112:0/105851688 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98032da0 con 0x7f7ea006db50 2024-04-25T01:37:26.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.281+0000 7f7ea4814640 1 -- 172.21.15.112:0/105851688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ea006db50 msgr2=0x7f7ea006dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.281+0000 7f7ea4814640 1 --2- 172.21.15.112:0/105851688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ea006db50 0x7f7ea006dfb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009d90 tx=0x7f7e9802ea70 comp rx=0 tx=0).stop 2024-04-25T01:37:26.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.281+0000 7f7ea4814640 1 -- 172.21.15.112:0/105851688 shutdown_connections 2024-04-25T01:37:26.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.281+0000 7f7ea4814640 1 --2- 172.21.15.112:0/105851688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ea006db50 0x7f7ea006dfb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.281+0000 7f7ea4814640 1 --2- 172.21.15.112:0/105851688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ea006ee40 0x7f7ea00abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.281+0000 7f7ea4814640 1 -- 172.21.15.112:0/105851688 >> 172.21.15.112:0/105851688 conn(0x7f7ea00c6da0 msgr2=0x7f7ea00c7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:26.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.281+0000 7f7ea4814640 1 -- 172.21.15.112:0/105851688 shutdown_connections 2024-04-25T01:37:26.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.281+0000 7f7ea4814640 1 -- 172.21.15.112:0/105851688 wait complete. 2024-04-25T01:37:26.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.283+0000 7f7ea4814640 1 Processor -- start 2024-04-25T01:37:26.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.283+0000 7f7ea4814640 1 -- start start 2024-04-25T01:37:26.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.284+0000 7f7ea4814640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ea006db50 0x7f7ea00aa210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.284+0000 7f7ea4814640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea006ee40 0x7f7ea014b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.284+0000 7f7ea4814640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ea014da70 0x7f7ea014b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.284+0000 7f7ea4814640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea00c3140 con 0x7f7ea006ee40 2024-04-25T01:37:26.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.284+0000 7f7e9e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea006ee40 0x7f7ea014b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.285+0000 7f7e9e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea006ee40 0x7f7ea014b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40714/0 (socket says 172.21.15.112:40714) 2024-04-25T01:37:26.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.285+0000 7f7e9e59c640 1 -- 172.21.15.112:0/81822395 learned_addr learned my addr 172.21.15.112:0/81822395 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:26.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.285+0000 7f7ea4814640 1 -- 172.21.15.112:0/81822395 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea0153920 con 0x7f7ea014da70 2024-04-25T01:37:26.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.285+0000 7f7ea4814640 1 -- 172.21.15.112:0/81822395 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea006ea50 con 0x7f7ea006db50 2024-04-25T01:37:26.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.285+0000 7f7e9f59e640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ea014da70 0x7f7ea014b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.285+0000 7f7e9ed9d640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ea006db50 0x7f7ea00aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.285+0000 7f7e9e59c640 1 -- 172.21.15.112:0/81822395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ea006db50 msgr2=0x7f7ea00aa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.285+0000 7f7e9e59c640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ea006db50 0x7f7ea00aa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.285+0000 7f7e9e59c640 1 -- 172.21.15.112:0/81822395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ea014da70 msgr2=0x7f7ea014b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.285+0000 7f7e9e59c640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ea014da70 0x7f7ea014b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.285+0000 7f7e9e59c640 1 -- 172.21.15.112:0/81822395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea00c2100 con 0x7f7ea006ee40 2024-04-25T01:37:26.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.285+0000 7f7e9f59e640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ea014da70 0x7f7ea014b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:37:26.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.286+0000 7f7e9e59c640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea006ee40 0x7f7ea014b490 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400e8b0 tx=0x7f7e9400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:26.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.286+0000 7f7e877fe640 1 -- 172.21.15.112:0/81822395 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94017070 con 0x7f7ea006ee40 2024-04-25T01:37:26.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.286+0000 7f7e877fe640 1 -- 172.21.15.112:0/81822395 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9400c940 con 0x7f7ea006ee40 2024-04-25T01:37:26.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.286+0000 7f7e877fe640 1 -- 172.21.15.112:0/81822395 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400cd10 con 0x7f7ea006ee40 2024-04-25T01:37:26.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.286+0000 7f7ea4814640 1 -- 172.21.15.112:0/81822395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea014bf10 con 0x7f7ea006ee40 2024-04-25T01:37:26.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.286+0000 7f7ea4814640 1 -- 172.21.15.112:0/81822395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea014f230 con 0x7f7ea006ee40 2024-04-25T01:37:26.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.287+0000 7f7ea4814640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea00c1b40 0x7f7ea01ea2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.287+0000 7f7ea4814640 1 -- 172.21.15.112:0/81822395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea004f500 con 0x7f7ea00c1b40 2024-04-25T01:37:26.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.288+0000 7f7e9ed9d640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea00c1b40 0x7f7ea01ea2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.288+0000 7f7e9ed9d640 1 -- 172.21.15.112:0/81822395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ea006e6d0 con 0x7f7ea00c1b40 2024-04-25T01:37:26.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.288+0000 7f7e877fe640 1 -- 172.21.15.112:0/81822395 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e9401a070 con 0x7f7ea006ee40 2024-04-25T01:37:26.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.288+0000 7f7e9ed9d640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea00c1b40 0x7f7ea01ea2b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7e90000f60 tx=0x7f7e9000f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:26.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.288+0000 7f7e877fe640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7e8c06bf00 0x7f7e8c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.289+0000 7f7e877fe640 1 -- 172.21.15.112:0/81822395 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f7e9408b0e0 con 0x7f7ea006ee40 2024-04-25T01:37:26.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.289+0000 7f7e877fe640 1 -- 172.21.15.112:0/81822395 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e900052e0 con 0x7f7ea00c1b40 2024-04-25T01:37:26.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.289+0000 7f7e877fe640 1 -- 172.21.15.112:0/81822395 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7e90005480 con 0x7f7ea00c1b40 2024-04-25T01:37:26.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.289+0000 7f7e877fe640 1 -- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea00c1b40 msgr2=0x7f7ea01ea2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.289+0000 7f7e877fe640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea00c1b40 0x7f7ea01ea2b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7e90000f60 tx=0x7f7e9000f9d0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.289+0000 7f7e877fe640 1 -- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea00c1b40 msgr2=0x7f7ea01ea2b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:37:26.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.289+0000 7f7e877fe640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea00c1b40 0x7f7ea01ea2b0 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7e90000f60 tx=0x7f7e9000f9d0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.289+0000 7f7e9f59e640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7e8c06bf00 0x7f7e8c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.289+0000 7f7e9f59e640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7e8c06bf00 0x7f7e8c06e3c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7e98039000 tx=0x7f7e98034000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:26.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.298+0000 7f7ea4814640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea00bbe50 0x7f7ea01ea7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.298+0000 7f7ea4814640 1 -- 172.21.15.112:0/81822395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea014c130 con 0x7f7ea00bbe50 2024-04-25T01:37:26.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.298+0000 7f7e9ed9d640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea00bbe50 0x7f7ea01ea7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.299+0000 7f7e9ed9d640 1 -- 172.21.15.112:0/81822395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7ea00abd90 con 0x7f7ea00bbe50 2024-04-25T01:37:26.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.299+0000 7f7e9ed9d640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea00bbe50 0x7f7ea01ea7f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e90000f60 tx=0x7f7e90001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:26.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.299+0000 7f7e877fe640 1 -- 172.21.15.112:0/81822395 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e90006c60 con 0x7f7ea00bbe50 2024-04-25T01:37:26.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.300+0000 7f7e877fe640 1 -- 172.21.15.112:0/81822395 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4560 (secure 0 0 0) 0x7f7e90006c60 con 0x7f7ea00bbe50 2024-04-25T01:37:26.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.300+0000 7f7e877fe640 1 -- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea00bbe50 msgr2=0x7f7ea01ea7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.279 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:37:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:37:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:37:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:37:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 94, 2024-04-25T01:37:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 97114, 2024-04-25T01:37:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:37:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:37:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:37:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:37:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:37:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:37:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:37:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:37:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:37:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:37:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:37:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:37:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:37:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:37:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:37:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:37:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:37:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:37:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:37:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:37:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:37:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:37:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:37:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:37:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:37:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:37:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:37:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:37:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:37:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:37:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:37:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:37:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:37:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:37:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:37:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:37:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:37:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:37:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:37:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:37:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:37:26.287 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:26.287 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:26.287 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:26.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.287 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.287 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:37:26.287 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:37:26.287 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:26.287 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:26.287 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:37:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:37:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:37:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:37:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:37:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:37:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:37:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:37:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:26.290 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:37:26.290 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.290 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:26.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.290 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:37:26.290 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:37:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.300+0000 7f7e877fe640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea00bbe50 0x7f7ea01ea7f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e90000f60 tx=0x7f7e90001080 comp rx=0 tx=0).stop 2024-04-25T01:37:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.300+0000 7f7e877fe640 1 -- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea00bbe50 msgr2=0x7f7ea01ea7f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:37:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.300+0000 7f7e877fe640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea00bbe50 0x7f7ea01ea7f0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e90000f60 tx=0x7f7e90001080 comp rx=0 tx=0).stop 2024-04-25T01:37:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.300+0000 7f7ea4814640 1 -- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7e8c06bf00 msgr2=0x7f7e8c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.300+0000 7f7ea4814640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7e8c06bf00 0x7f7e8c06e3c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7e98039000 tx=0x7f7e98034000 comp rx=0 tx=0).stop 2024-04-25T01:37:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.300+0000 7f7ea4814640 1 -- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea006ee40 msgr2=0x7f7ea014b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.300+0000 7f7ea4814640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea006ee40 0x7f7ea014b490 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400e8b0 tx=0x7f7e9400ed70 comp rx=0 tx=0).stop 2024-04-25T01:37:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.300+0000 7f7e9f59e640 1 -- 172.21.15.112:0/81822395 reap_dead start 2024-04-25T01:37:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.300+0000 7f7ea4814640 1 -- 172.21.15.112:0/81822395 shutdown_connections 2024-04-25T01:37:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.300+0000 7f7ea4814640 1 --2- 172.21.15.112:0/81822395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea006ee40 0x7f7ea014b490 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.300+0000 7f7ea4814640 1 -- 172.21.15.112:0/81822395 >> 172.21.15.112:0/81822395 conn(0x7f7ea00c6da0 msgr2=0x7f7ea01569a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.300+0000 7f7ea4814640 1 -- 172.21.15.112:0/81822395 shutdown_connections 2024-04-25T01:37:26.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.300+0000 7f7ea4814640 1 -- 172.21.15.112:0/81822395 wait complete. 2024-04-25T01:37:26.293 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:37:26.293 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:37:26.293 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:37:26.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.489+0000 7f3f53841640 1 Processor -- start 2024-04-25T01:37:26.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.489+0000 7f3f53841640 1 -- start start 2024-04-25T01:37:26.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.490+0000 7f3f53841640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f4c070fd0 0x7f3f4c0ac000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.490+0000 7f3f53841640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f4c15ac20 con 0x7f3f4c072ce0 2024-04-25T01:37:26.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.490+0000 7f3f53841640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f4c0ab520 con 0x7f3f4c1571b0 2024-04-25T01:37:26.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.490+0000 7f3f53841640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f4c0c2190 con 0x7f3f4c070fd0 2024-04-25T01:37:26.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.491+0000 7f3f50ddc640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3f4c072ce0 0x7f3f4c0c1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:35498/0 (socket says 172.21.15.112:35498) 2024-04-25T01:37:26.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.491+0000 7f3f50ddc640 1 -- 172.21.15.112:0/1436761688 learned_addr learned my addr 172.21.15.112:0/1436761688 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:37:26.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.491+0000 7f3f515dd640 1 --2- 172.21.15.112:0/1436761688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f4c070fd0 0x7f3f4c0ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.491+0000 7f3f43fff640 1 -- 172.21.15.112:0/1436761688 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1869285905 0 0) 0x7f3f4c15ac20 con 0x7f3f4c072ce0 2024-04-25T01:37:26.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.491+0000 7f3f43fff640 1 -- 172.21.15.112:0/1436761688 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f38003520 con 0x7f3f4c072ce0 2024-04-25T01:37:26.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.492+0000 7f3f43fff640 1 -- 172.21.15.112:0/1436761688 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2194459405 0 0) 0x7f3f4c0ab520 con 0x7f3f4c1571b0 2024-04-25T01:37:26.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.492+0000 7f3f43fff640 1 -- 172.21.15.112:0/1436761688 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f4c15ac20 con 0x7f3f4c1571b0 2024-04-25T01:37:26.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.492+0000 7f3f43fff640 1 -- 172.21.15.112:0/1436761688 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2012899685 0 0) 0x7f3f38003520 con 0x7f3f4c072ce0 2024-04-25T01:37:26.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.492+0000 7f3f43fff640 1 -- 172.21.15.112:0/1436761688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f4c070fd0 msgr2=0x7f3f4c0ac000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:37:26.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.492+0000 7f3f43fff640 1 --2- 172.21.15.112:0/1436761688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f4c070fd0 0x7f3f4c0ac000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.492+0000 7f3f43fff640 1 -- 172.21.15.112:0/1436761688 >> v1:172.21.15.187:6789/0 conn(0x7f3f4c1571b0 legacy=0x7f3f4c164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.492+0000 7f3f43fff640 1 -- 172.21.15.112:0/1436761688 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4c0ab800 con 0x7f3f4c072ce0 2024-04-25T01:37:26.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.492+0000 7f3f43fff640 1 -- 172.21.15.112:0/1436761688 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f3f3c0015f0 con 0x7f3f4c072ce0 2024-04-25T01:37:26.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.492+0000 7f3f43fff640 1 -- 172.21.15.112:0/1436761688 >> v1:172.21.15.112:6789/0 conn(0x7f3f4c072ce0 legacy=0x7f3f4c0c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.492+0000 7f3f43fff640 1 --2- 172.21.15.112:0/1436761688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f38003bc0 0x7f3f38003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.492+0000 7f3f43fff640 1 --2- 172.21.15.112:0/1436761688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f38004800 0x7f3f38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.492+0000 7f3f43fff640 1 --2- 172.21.15.112:0/1436761688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f380051c0 0x7f3f3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.492+0000 7f3f43fff640 1 -- 172.21.15.112:0/1436761688 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c0c2190 con 0x7f3f38004800 2024-04-25T01:37:26.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.492+0000 7f3f43fff640 1 -- 172.21.15.112:0/1436761688 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3800c030 con 0x7f3f38003bc0 2024-04-25T01:37:26.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.492+0000 7f3f43fff640 1 -- 172.21.15.112:0/1436761688 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f3800c230 con 0x7f3f380051c0 2024-04-25T01:37:26.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.492+0000 7f3f51dde640 1 --2- 172.21.15.112:0/1436761688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f380051c0 0x7f3f3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.493+0000 7f3f50ddc640 1 --2- 172.21.15.112:0/1436761688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f38004800 0x7f3f38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.493+0000 7f3f50ddc640 1 -- 172.21.15.112:0/1436761688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f380051c0 msgr2=0x7f3f3800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:37:26.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.493+0000 7f3f50ddc640 1 --2- 172.21.15.112:0/1436761688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f380051c0 0x7f3f3800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.493+0000 7f3f50ddc640 1 -- 172.21.15.112:0/1436761688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f38003bc0 msgr2=0x7f3f38003fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:37:26.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.493+0000 7f3f50ddc640 1 --2- 172.21.15.112:0/1436761688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f38003bc0 0x7f3f38003fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.493+0000 7f3f50ddc640 1 -- 172.21.15.112:0/1436761688 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f38003520 con 0x7f3f38004800 2024-04-25T01:37:26.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.494+0000 7f3f50ddc640 1 --2- 172.21.15.112:0/1436761688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f38004800 0x7f3f38004c80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c007f80 tx=0x7f3f38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9037521a7f03aaa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:26.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.495+0000 7f3f43fff640 1 -- 172.21.15.112:0/1436761688 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c007ba0 con 0x7f3f38004800 2024-04-25T01:37:26.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.495+0000 7f3f43fff640 1 -- 172.21.15.112:0/1436761688 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f3c005bb0 con 0x7f3f38004800 2024-04-25T01:37:26.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.495+0000 7f3f43fff640 1 -- 172.21.15.112:0/1436761688 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c007070 con 0x7f3f38004800 2024-04-25T01:37:26.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.496+0000 7f3f53841640 1 -- 172.21.15.112:0/1436761688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f38004800 msgr2=0x7f3f38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.496+0000 7f3f53841640 1 --2- 172.21.15.112:0/1436761688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f38004800 0x7f3f38004c80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c007f80 tx=0x7f3f38005830 comp rx=0 tx=0).stop 2024-04-25T01:37:26.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.496+0000 7f3f51dde640 1 -- 172.21.15.112:0/1436761688 reap_dead start 2024-04-25T01:37:26.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.496+0000 7f3f53841640 1 -- 172.21.15.112:0/1436761688 shutdown_connections 2024-04-25T01:37:26.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.496+0000 7f3f53841640 1 -- 172.21.15.112:0/1436761688 >> 172.21.15.112:0/1436761688 conn(0x7f3f4c0c7080 msgr2=0x7f3f4c0c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:26.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.497+0000 7f3f53841640 1 -- 172.21.15.112:0/1436761688 shutdown_connections 2024-04-25T01:37:26.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.497+0000 7f3f53841640 1 -- 172.21.15.112:0/1436761688 wait complete. 2024-04-25T01:37:26.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.499+0000 7f3f53841640 1 Processor -- start 2024-04-25T01:37:26.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.499+0000 7f3f53841640 1 -- start start 2024-04-25T01:37:26.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.500+0000 7f3f53841640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f38004800 0x7f3f4c159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.500+0000 7f3f53841640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f4c1efb20 0x7f3f4c1effa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.500+0000 7f3f53841640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f4c1e3260 0x7f3f4c1e36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.500+0000 7f3f53841640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c0abb70 con 0x7f3f4c1efb20 2024-04-25T01:37:26.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.500+0000 7f3f53841640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c0ab2d0 con 0x7f3f38004800 2024-04-25T01:37:26.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.500+0000 7f3f53841640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f4c15a840 con 0x7f3f4c1e3260 2024-04-25T01:37:26.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.501+0000 7f3f51dde640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f4c1e3260 0x7f3f4c1e36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.501+0000 7f3f51dde640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f4c1e3260 0x7f3f4c1e36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:36298/0 (socket says 172.21.15.112:36298) 2024-04-25T01:37:26.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.501+0000 7f3f51dde640 1 -- 172.21.15.112:0/860482082 learned_addr learned my addr 172.21.15.112:0/860482082 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:26.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.501+0000 7f3f515dd640 1 --2- 172.21.15.112:0/860482082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f38004800 0x7f3f4c159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.501+0000 7f3f50ddc640 1 --2- 172.21.15.112:0/860482082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f4c1efb20 0x7f3f4c1effa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.501+0000 7f3f51dde640 1 -- 172.21.15.112:0/860482082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f38004800 msgr2=0x7f3f4c159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.501+0000 7f3f51dde640 1 --2- 172.21.15.112:0/860482082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f38004800 0x7f3f4c159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.501+0000 7f3f51dde640 1 -- 172.21.15.112:0/860482082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f4c1efb20 msgr2=0x7f3f4c1effa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.501+0000 7f3f51dde640 1 --2- 172.21.15.112:0/860482082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f4c1efb20 0x7f3f4c1effa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.501+0000 7f3f51dde640 1 -- 172.21.15.112:0/860482082 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4c1ef640 con 0x7f3f4c1e3260 2024-04-25T01:37:26.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.502+0000 7f3f51dde640 1 --2- 172.21.15.112:0/860482082 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f4c1e3260 0x7f3f4c1e36e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0c17d0 tx=0x7f3f4400dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:26.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.502+0000 7f3f41ffb640 1 -- 172.21.15.112:0/860482082 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f44018070 con 0x7f3f4c1e3260 2024-04-25T01:37:26.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.502+0000 7f3f53841640 1 -- 172.21.15.112:0/860482082 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f4c15b250 con 0x7f3f4c1e3260 2024-04-25T01:37:26.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.502+0000 7f3f41ffb640 1 -- 172.21.15.112:0/860482082 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f44003b70 con 0x7f3f4c1e3260 2024-04-25T01:37:26.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.502+0000 7f3f41ffb640 1 -- 172.21.15.112:0/860482082 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f44005070 con 0x7f3f4c1e3260 2024-04-25T01:37:26.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.503+0000 7f3f53841640 1 -- 172.21.15.112:0/860482082 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f4c15e0c0 con 0x7f3f4c1e3260 2024-04-25T01:37:26.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.505+0000 7f3f41ffb640 1 -- 172.21.15.112:0/860482082 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f4401b070 con 0x7f3f4c1e3260 2024-04-25T01:37:26.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.505+0000 7f3f41ffb640 1 --2- 172.21.15.112:0/860482082 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3f3006be30 0x7f3f3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.506+0000 7f3f41ffb640 1 -- 172.21.15.112:0/860482082 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f3f4408a990 con 0x7f3f4c1e3260 2024-04-25T01:37:26.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.506+0000 7f3f377fe640 1 --2- 172.21.15.112:0/860482082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4c151430 0x7f3f4c06c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.506+0000 7f3f377fe640 1 -- 172.21.15.112:0/860482082 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c1f34b0 con 0x7f3f4c151430 2024-04-25T01:37:26.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.508+0000 7f3f515dd640 1 --2- 172.21.15.112:0/860482082 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3f3006be30 0x7f3f3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.508+0000 7f3f515dd640 1 --2- 172.21.15.112:0/860482082 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3f3006be30 0x7f3f3006e2f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c06f3c0 tx=0x7f3f48009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:26.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.510+0000 7f3f50ddc640 1 --2- 172.21.15.112:0/860482082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4c151430 0x7f3f4c06c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.511+0000 7f3f50ddc640 1 -- 172.21.15.112:0/860482082 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f4c1f04e0 con 0x7f3f4c151430 2024-04-25T01:37:26.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.513+0000 7f3f50ddc640 1 --2- 172.21.15.112:0/860482082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4c151430 0x7f3f4c06c950 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c02f000 tx=0x7f3f3c0018f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:26.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.514+0000 7f3f41ffb640 1 -- 172.21.15.112:0/860482082 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c005990 con 0x7f3f4c151430 2024-04-25T01:37:26.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.514+0000 7f3f41ffb640 1 -- 172.21.15.112:0/860482082 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3f3c044070 con 0x7f3f4c151430 2024-04-25T01:37:26.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.514+0000 7f3f41ffb640 1 -- 172.21.15.112:0/860482082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4c151430 msgr2=0x7f3f4c06c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.514+0000 7f3f41ffb640 1 --2- 172.21.15.112:0/860482082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4c151430 0x7f3f4c06c950 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c02f000 tx=0x7f3f3c0018f0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.515+0000 7f3f41ffb640 1 -- 172.21.15.112:0/860482082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4c151430 msgr2=0x7f3f4c06c950 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:37:26.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.515+0000 7f3f41ffb640 1 --2- 172.21.15.112:0/860482082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4c151430 0x7f3f4c06c950 secure :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c02f000 tx=0x7f3f3c0018f0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.527+0000 7f3f377fe640 1 --2- 172.21.15.112:0/860482082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4c06ce90 0x7f3f4c06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.527+0000 7f3f377fe640 1 -- 172.21.15.112:0/860482082 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c06d7e0 con 0x7f3f4c06ce90 2024-04-25T01:37:26.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.528+0000 7f3f50ddc640 1 --2- 172.21.15.112:0/860482082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4c06ce90 0x7f3f4c06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.528+0000 7f3f50ddc640 1 -- 172.21.15.112:0/860482082 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3f4c075980 con 0x7f3f4c06ce90 2024-04-25T01:37:26.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.529+0000 7f3f50ddc640 1 --2- 172.21.15.112:0/860482082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4c06ce90 0x7f3f4c06d2a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c1520a0 tx=0x7f3f3c02e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:26.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.529+0000 7f3f41ffb640 1 -- 172.21.15.112:0/860482082 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0499d0 con 0x7f3f4c06ce90 2024-04-25T01:37:26.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.529+0000 7f3f41ffb640 1 -- 172.21.15.112:0/860482082 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5069 (secure 0 0 0) 0x7f3f3c0499d0 con 0x7f3f4c06ce90 2024-04-25T01:37:26.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.529+0000 7f3f41ffb640 1 -- 172.21.15.112:0/860482082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4c06ce90 msgr2=0x7f3f4c06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.529+0000 7f3f41ffb640 1 --2- 172.21.15.112:0/860482082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4c06ce90 0x7f3f4c06d2a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c1520a0 tx=0x7f3f3c02e000 comp rx=0 tx=0).stop 2024-04-25T01:37:26.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.530+0000 7f3f41ffb640 1 -- 172.21.15.112:0/860482082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4c06ce90 msgr2=0x7f3f4c06d2a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:37:26.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.530+0000 7f3f41ffb640 1 --2- 172.21.15.112:0/860482082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4c06ce90 0x7f3f4c06d2a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.508 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:37:26.508 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:37:26.508 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:37:26.508 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:37:26.508 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 94, 2024-04-25T01:37:26.508 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 482219, 2024-04-25T01:37:26.509 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:37:26.509 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:37:26.509 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:37:26.509 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:37:26.509 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:26.509 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:37:26.509 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:37:26.509 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:37:26.509 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:37:26.509 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:26.509 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:26.509 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:26.509 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:26.509 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:26.509 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:26.510 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:26.510 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:26.510 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:26.510 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:26.510 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:26.510 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:26.510 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:37:26.510 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:37:26.510 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:26.510 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:26.510 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:26.510 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:26.510 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:26.510 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:26.511 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:26.511 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:26.511 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:26.511 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:26.511 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:26.511 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:26.511 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.511 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:37:26.511 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:37:26.511 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:37:26.511 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:37:26.511 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:37:26.511 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:37:26.511 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:37:26.511 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:37:26.512 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:37:26.512 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:37:26.512 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:37:26.512 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:37:26.512 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:37:26.512 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:37:26.512 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:37:26.512 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:37:26.512 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:37:26.512 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:26.512 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:26.512 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:26.512 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:26.512 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:26.512 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:26.513 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:26.513 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:26.513 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:26.513 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:26.513 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:26.513 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:26.513 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:37:26.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.513 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:37:26.513 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.513 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:37:26.513 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:37:26.513 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:26.513 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:26.513 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.514 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:26.514 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:37:26.514 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:26.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.514 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.514 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:26.514 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:37:26.514 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:26.514 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.514 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:26.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.514 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:37:26.514 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:37:26.515 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:26.515 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:26.515 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:26.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.515 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.515 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:37:26.515 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:37:26.515 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:26.515 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:26.515 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.515 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:26.515 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:37:26.515 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:26.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.516 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.516 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:26.516 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:37:26.516 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:26.516 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.516 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:26.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.516 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:37:26.516 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:37:26.516 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:26.516 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:26.516 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:26.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.517 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.517 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:37:26.517 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:37:26.517 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:26.517 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:26.517 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.517 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:26.517 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:37:26.517 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:26.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.517 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.517 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:26.517 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:37:26.517 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:26.517 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.518 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:26.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.518 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:37:26.518 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:37:26.518 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:26.518 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:26.518 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:26.518 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.518 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:26.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.518 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:37:26.518 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:37:26.518 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.519 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:26.519 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:26.519 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:37:26.519 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.519 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:26.519 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:37:26.519 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.519 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:26.519 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:26.519 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:37:26.519 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.519 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:26.520 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:37:26.520 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.520 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:26.520 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:26.520 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 8 2024-04-25T01:37:26.520 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.520 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:26.520 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:37:26.520 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.520 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:26.520 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:26.520 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 6 2024-04-25T01:37:26.520 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.520 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:26.521 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:37:26.521 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.521 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:26.521 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:26.521 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:37:26.521 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.521 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:26.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.521 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:37:26.521 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:37:26.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.531+0000 7f3f53841640 1 -- 172.21.15.112:0/860482082 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3f3006be30 msgr2=0x7f3f3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.531+0000 7f3f53841640 1 --2- 172.21.15.112:0/860482082 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3f3006be30 0x7f3f3006e2f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c06f3c0 tx=0x7f3f48009190 comp rx=0 tx=0).stop 2024-04-25T01:37:26.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.531+0000 7f3f53841640 1 -- 172.21.15.112:0/860482082 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f4c1e3260 msgr2=0x7f3f4c1e36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.531+0000 7f3f53841640 1 --2- 172.21.15.112:0/860482082 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f4c1e3260 0x7f3f4c1e36e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0c17d0 tx=0x7f3f4400dc00 comp rx=0 tx=0).stop 2024-04-25T01:37:26.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.531+0000 7f3f51dde640 1 -- 172.21.15.112:0/860482082 reap_dead start 2024-04-25T01:37:26.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.531+0000 7f3f51dde640 1 -- 172.21.15.112:0/860482082 reap_dead start 2024-04-25T01:37:26.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.531+0000 7f3f53841640 1 -- 172.21.15.112:0/860482082 shutdown_connections 2024-04-25T01:37:26.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.531+0000 7f3f53841640 1 -- 172.21.15.112:0/860482082 >> 172.21.15.112:0/860482082 conn(0x7f3f4c0c7080 msgr2=0x7f3f4c0c1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:26.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.531+0000 7f3f53841640 1 -- 172.21.15.112:0/860482082 shutdown_connections 2024-04-25T01:37:26.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.531+0000 7f3f53841640 1 -- 172.21.15.112:0/860482082 wait complete. 2024-04-25T01:37:26.525 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:37:26.525 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:37:26.526 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:37:26.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.737+0000 7f0bea815640 1 Processor -- start 2024-04-25T01:37:26.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.737+0000 7f0bea815640 1 -- start start 2024-04-25T01:37:26.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.738+0000 7f0bea815640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0be406f0a0 0x7f0be4070d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.738+0000 7f0bea815640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be40aa1a0 0x7f0be414ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.738+0000 7f0bea815640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0be4151910 con 0x7f0be40c3b10 2024-04-25T01:37:26.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.738+0000 7f0bea815640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f0be4152520 con 0x7f0be406f0a0 2024-04-25T01:37:26.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.738+0000 7f0bea815640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0be4151bf0 con 0x7f0be40aa1a0 2024-04-25T01:37:26.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.739+0000 7f0be3fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0be406f0a0 0x7f0be4070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.739+0000 7f0be8db2640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f0be40c3b10 0x7f0be40c1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:59196/0 (socket says 172.21.15.112:59196) 2024-04-25T01:37:26.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.739+0000 7f0be8db2640 1 -- 172.21.15.112:0/3211810344 learned_addr learned my addr 172.21.15.112:0/3211810344 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:37:26.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.739+0000 7f0be3fff640 1 -- 172.21.15.112:0/3211810344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be40aa1a0 msgr2=0x7f0be414ec60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:37:26.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.739+0000 7f0be3fff640 1 --2- 172.21.15.112:0/3211810344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be40aa1a0 0x7f0be414ec60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.739+0000 7f0be3fff640 1 -- 172.21.15.112:0/3211810344 >> v1:172.21.15.187:6789/0 conn(0x7f0be40c3b10 legacy=0x7f0be40c1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.739+0000 7f0be3fff640 1 -- 172.21.15.112:0/3211810344 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be4151ed0 con 0x7f0be406f0a0 2024-04-25T01:37:26.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.740+0000 7f0be3fff640 1 --2- 172.21.15.112:0/3211810344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be406f0a0 0x7f0be4070d60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80097a0 tx=0x7f0bd802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4482490e7b0f83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:26.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.741+0000 7f0be2ffd640 1 -- 172.21.15.112:0/3211810344 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8037070 con 0x7f0be406f0a0 2024-04-25T01:37:26.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.741+0000 7f0be2ffd640 1 -- 172.21.15.112:0/3211810344 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd8032440 con 0x7f0be406f0a0 2024-04-25T01:37:26.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.741+0000 7f0be2ffd640 1 -- 172.21.15.112:0/3211810344 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8032810 con 0x7f0be406f0a0 2024-04-25T01:37:26.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.742+0000 7f0bea815640 1 -- 172.21.15.112:0/3211810344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be406f0a0 msgr2=0x7f0be4070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.742+0000 7f0bea815640 1 --2- 172.21.15.112:0/3211810344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be406f0a0 0x7f0be4070d60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80097a0 tx=0x7f0bd802a250 comp rx=0 tx=0).stop 2024-04-25T01:37:26.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.742+0000 7f0bea815640 1 -- 172.21.15.112:0/3211810344 shutdown_connections 2024-04-25T01:37:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.742+0000 7f0bea815640 1 --2- 172.21.15.112:0/3211810344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be40aa1a0 0x7f0be414ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.742+0000 7f0bea815640 1 --2- 172.21.15.112:0/3211810344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be406f0a0 0x7f0be4070d60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.742+0000 7f0bea815640 1 -- 172.21.15.112:0/3211810344 >> 172.21.15.112:0/3211810344 conn(0x7f0be406b160 msgr2=0x7f0be406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.743+0000 7f0bea815640 1 -- 172.21.15.112:0/3211810344 shutdown_connections 2024-04-25T01:37:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.743+0000 7f0bea815640 1 -- 172.21.15.112:0/3211810344 wait complete. 2024-04-25T01:37:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.745+0000 7f0bea815640 1 Processor -- start 2024-04-25T01:37:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.746+0000 7f0bea815640 1 -- start start 2024-04-25T01:37:26.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.747+0000 7f0bea815640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be406f0a0 0x7f0be4154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.747+0000 7f0bea815640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be40aa1a0 0x7f0be4153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.747+0000 7f0bea815640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0be40c3b10 0x7f0be414ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.747+0000 7f0bea815640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0be40c3f50 con 0x7f0be40aa1a0 2024-04-25T01:37:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.747+0000 7f0bea815640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0be414f1a0 con 0x7f0be40c3b10 2024-04-25T01:37:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.747+0000 7f0bea815640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0be40c40d0 con 0x7f0be406f0a0 2024-04-25T01:37:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.747+0000 7f0be8db2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0be40c3b10 0x7f0be414ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.747+0000 7f0be8db2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0be40c3b10 0x7f0be414ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:39832/0 (socket says 172.21.15.112:39832) 2024-04-25T01:37:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.747+0000 7f0be8db2640 1 -- 172.21.15.112:0/1016577254 learned_addr learned my addr 172.21.15.112:0/1016577254 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.747+0000 7f0be37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be40aa1a0 0x7f0be4153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.748+0000 7f0be8db2640 1 -- 172.21.15.112:0/1016577254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be406f0a0 msgr2=0x7f0be4154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.748+0000 7f0be8db2640 1 --2- 172.21.15.112:0/1016577254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be406f0a0 0x7f0be4154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.748+0000 7f0be8db2640 1 -- 172.21.15.112:0/1016577254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be40aa1a0 msgr2=0x7f0be4153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.748+0000 7f0be8db2640 1 --2- 172.21.15.112:0/1016577254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be40aa1a0 0x7f0be4153700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.748+0000 7f0be8db2640 1 -- 172.21.15.112:0/1016577254 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be40c4e80 con 0x7f0be40c3b10 2024-04-25T01:37:26.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.748+0000 7f0be37fe640 1 --2- 172.21.15.112:0/1016577254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be40aa1a0 0x7f0be4153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:37:26.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.748+0000 7f0be8db2640 1 --2- 172.21.15.112:0/1016577254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0be40c3b10 0x7f0be414ba80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00c9f0 tx=0x7f0bdc00ceb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:26.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.749+0000 7f0be0ff9640 1 -- 172.21.15.112:0/1016577254 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc017070 con 0x7f0be40c3b10 2024-04-25T01:37:26.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.749+0000 7f0be0ff9640 1 -- 172.21.15.112:0/1016577254 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bdc00ab70 con 0x7f0be40c3b10 2024-04-25T01:37:26.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.749+0000 7f0be0ff9640 1 -- 172.21.15.112:0/1016577254 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc012400 con 0x7f0be40c3b10 2024-04-25T01:37:26.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.749+0000 7f0bea815640 1 -- 172.21.15.112:0/1016577254 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be40c78d0 con 0x7f0be40c3b10 2024-04-25T01:37:26.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.749+0000 7f0bea815640 1 -- 172.21.15.112:0/1016577254 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be40c7e90 con 0x7f0be40c3b10 2024-04-25T01:37:26.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.752+0000 7f0be0ff9640 1 -- 172.21.15.112:0/1016577254 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0bdc01a070 con 0x7f0be40c3b10 2024-04-25T01:37:26.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.752+0000 7f0bea815640 1 --2- 172.21.15.112:0/1016577254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be414bfc0 0x7f0be4134c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.752+0000 7f0bea815640 1 -- 172.21.15.112:0/1016577254 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0be414c3a0 con 0x7f0be414bfc0 2024-04-25T01:37:26.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.753+0000 7f0be0ff9640 1 --2- 172.21.15.112:0/1016577254 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0bd006be60 0x7f0bd006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.753+0000 7f0be3fff640 1 --2- 172.21.15.112:0/1016577254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be414bfc0 0x7f0be4134c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.753+0000 7f0be0ff9640 1 -- 172.21.15.112:0/1016577254 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f0bdc08adb0 con 0x7f0be40c3b10 2024-04-25T01:37:26.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.753+0000 7f0be37fe640 1 --2- 172.21.15.112:0/1016577254 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0bd006be60 0x7f0bd006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.754+0000 7f0be37fe640 1 --2- 172.21.15.112:0/1016577254 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0bd006be60 0x7f0bd006e320 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0be406e790 tx=0x7f0bd40090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:26.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.759+0000 7f0be3fff640 1 -- 172.21.15.112:0/1016577254 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0be40c4c30 con 0x7f0be414bfc0 2024-04-25T01:37:26.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.762+0000 7f0be3fff640 1 --2- 172.21.15.112:0/1016577254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be414bfc0 0x7f0be4134c50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0be40c16c0 tx=0x7f0bd8032ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:26.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.762+0000 7f0be0ff9640 1 -- 172.21.15.112:0/1016577254 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8042020 con 0x7f0be414bfc0 2024-04-25T01:37:26.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.763+0000 7f0be0ff9640 1 -- 172.21.15.112:0/1016577254 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0bd803d670 con 0x7f0be414bfc0 2024-04-25T01:37:26.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.763+0000 7f0be0ff9640 1 -- 172.21.15.112:0/1016577254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be414bfc0 msgr2=0x7f0be4134c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.763+0000 7f0be0ff9640 1 --2- 172.21.15.112:0/1016577254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be414bfc0 0x7f0be4134c50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0be40c16c0 tx=0x7f0bd8032ee0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.763+0000 7f0be0ff9640 1 -- 172.21.15.112:0/1016577254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be414bfc0 msgr2=0x7f0be4134c50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:37:26.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.763+0000 7f0be0ff9640 1 --2- 172.21.15.112:0/1016577254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be414bfc0 0x7f0be4134c50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.780+0000 7f0bea815640 1 --2- 172.21.15.112:0/1016577254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be4135190 0x7f0be41355a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:26.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.780+0000 7f0bea815640 1 -- 172.21.15.112:0/1016577254 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0be4075980 con 0x7f0be4135190 2024-04-25T01:37:26.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.780+0000 7f0be3fff640 1 --2- 172.21.15.112:0/1016577254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be4135190 0x7f0be41355a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:26.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.782+0000 7f0be3fff640 1 -- 172.21.15.112:0/1016577254 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0be40682a0 con 0x7f0be4135190 2024-04-25T01:37:26.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.784+0000 7f0be3fff640 1 --2- 172.21.15.112:0/1016577254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be4135190 0x7f0be41355a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80097a0 tx=0x7f0bd8008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:26.766 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:37:26.766 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:37:26.766 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:37:26.766 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:37:26.767 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 94, 2024-04-25T01:37:26.767 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 16150, 2024-04-25T01:37:26.767 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:37:26.767 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:37:26.767 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:37:26.767 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:37:26.767 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:26.767 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:37:26.767 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:37:26.767 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:37:26.767 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:37:26.767 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:26.767 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:26.767 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:26.767 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:26.768 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:26.768 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:26.768 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:26.768 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:26.768 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:26.768 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:26.768 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:26.768 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:26.768 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:37:26.768 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:37:26.768 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:26.768 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:26.769 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:26.769 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:26.769 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:26.769 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:26.769 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:26.769 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:26.769 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:26.769 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:26.769 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:26.769 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:26.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.769 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:37:26.769 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:37:26.769 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:37:26.770 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:37:26.770 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:37:26.770 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:37:26.770 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:37:26.770 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:37:26.770 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:37:26.770 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:37:26.770 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:37:26.770 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:37:26.770 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:37:26.770 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:37:26.770 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:37:26.770 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:37:26.771 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:37:26.771 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:26.771 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:26.771 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:26.771 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:26.771 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:26.771 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:26.771 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:26.771 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:26.772 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:26.772 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:26.772 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:26.772 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:26.772 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:37:26.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.772 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:37:26.772 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.772 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:37:26.772 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:37:26.772 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:26.773 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:26.773 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.773 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:26.773 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:37:26.773 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:26.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.773 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.773 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:26.773 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:37:26.774 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:26.774 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.774 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:26.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.774 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:37:26.774 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:37:26.774 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:26.774 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:26.774 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:26.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.775 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:37:26.775 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:37:26.775 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:26.775 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:26.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.775 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:26.775 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:37:26.775 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:26.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.775 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:26.775 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:37:26.776 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:26.776 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.776 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:26.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.776 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:37:26.776 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:37:26.776 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:26.776 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:26.776 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:26.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.776 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.776 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:37:26.776 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:37:26.776 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:26.777 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:26.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.777 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:26.777 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:37:26.777 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:26.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.777 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:26.777 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:37:26.777 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:26.777 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.777 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:26.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.777 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:37:26.777 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:37:26.778 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:26.778 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:26.778 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:26.778 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.778 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:26.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.778 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:37:26.778 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:37:26.778 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.778 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:26.778 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:26.778 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:37:26.778 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.778 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:26.779 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:37:26.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:26.779 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:26.779 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:26.779 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:37:26.779 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:26.779 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:26.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:26.779 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:37:26.779 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:37:26.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.787+0000 7f0be0ff9640 1 -- 172.21.15.112:0/1016577254 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8042040 con 0x7f0be4135190 2024-04-25T01:37:26.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.787+0000 7f0be0ff9640 1 -- 172.21.15.112:0/1016577254 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4558 (secure 0 0 0) 0x7f0bd80492c0 con 0x7f0be4135190 2024-04-25T01:37:26.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.787+0000 7f0be0ff9640 1 -- 172.21.15.112:0/1016577254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be4135190 msgr2=0x7f0be41355a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.787+0000 7f0be0ff9640 1 --2- 172.21.15.112:0/1016577254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be4135190 0x7f0be41355a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80097a0 tx=0x7f0bd8008030 comp rx=0 tx=0).stop 2024-04-25T01:37:26.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.787+0000 7f0be0ff9640 1 -- 172.21.15.112:0/1016577254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be4135190 msgr2=0x7f0be41355a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:37:26.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.787+0000 7f0be0ff9640 1 --2- 172.21.15.112:0/1016577254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be4135190 0x7f0be41355a0 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80097a0 tx=0x7f0bd8008030 comp rx=0 tx=0).stop 2024-04-25T01:37:26.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.789+0000 7f0bea815640 1 -- 172.21.15.112:0/1016577254 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0bd006be60 msgr2=0x7f0bd006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.789+0000 7f0bea815640 1 --2- 172.21.15.112:0/1016577254 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0bd006be60 0x7f0bd006e320 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0be406e790 tx=0x7f0bd40090e0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.789+0000 7f0bea815640 1 -- 172.21.15.112:0/1016577254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0be40c3b10 msgr2=0x7f0be414ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:26.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.789+0000 7f0bea815640 1 --2- 172.21.15.112:0/1016577254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0be40c3b10 0x7f0be414ba80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00c9f0 tx=0x7f0bdc00ceb0 comp rx=0 tx=0).stop 2024-04-25T01:37:26.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.789+0000 7f0be8db2640 1 -- 172.21.15.112:0/1016577254 reap_dead start 2024-04-25T01:37:26.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.789+0000 7f0be8db2640 1 -- 172.21.15.112:0/1016577254 reap_dead start 2024-04-25T01:37:26.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.790+0000 7f0bea815640 1 -- 172.21.15.112:0/1016577254 shutdown_connections 2024-04-25T01:37:26.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.790+0000 7f0bea815640 1 -- 172.21.15.112:0/1016577254 >> 172.21.15.112:0/1016577254 conn(0x7f0be406b160 msgr2=0x7f0be414e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:26.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.791+0000 7f0bea815640 1 -- 172.21.15.112:0/1016577254 shutdown_connections 2024-04-25T01:37:26.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.791+0000 7f0bea815640 1 -- 172.21.15.112:0/1016577254 wait complete. 2024-04-25T01:37:26.788 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:37:26.788 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T01:37:26.788 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T01:37:27.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.958+0000 7f18e61b7640 1 Processor -- start 2024-04-25T01:37:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.959+0000 7f18e61b7640 1 -- start start 2024-04-25T01:37:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.959+0000 7f18e61b7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18e006f1f0 0x7f18e0070d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.959+0000 7f18e61b7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f18e0072d50 0x7f18e015f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.959+0000 7f18e61b7640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18e01634f0 con 0x7f18e00712a0 2024-04-25T01:37:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.959+0000 7f18e61b7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f18e0162fd0 con 0x7f18e0072d50 2024-04-25T01:37:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.959+0000 7f18e61b7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f18e00ab370 con 0x7f18e006f1f0 2024-04-25T01:37:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.959+0000 7f18dffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f18e0072d50 0x7f18e015f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.959+0000 7f18deffd640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f18e00712a0 0x7f18e00abb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:59212/0 (socket says 172.21.15.112:59212) 2024-04-25T01:37:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.960+0000 7f18df7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18e006f1f0 0x7f18e0070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.960+0000 7f18deffd640 1 -- 172.21.15.112:0/4228147922 learned_addr learned my addr 172.21.15.112:0/4228147922 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:37:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:26.960+0000 7f18de7fc640 1 -- 172.21.15.112:0/4228147922 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1721327018 0 0) 0x7f18e01634f0 con 0x7f18e00712a0 2024-04-25T01:37:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.055+0000 7f18de7fc640 1 -- 172.21.15.112:0/4228147922 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18c8003970 con 0x7f18e00712a0 2024-04-25T01:37:27.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.056+0000 7f18dffff640 1 -- 172.21.15.112:0/4228147922 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18e006f1f0 msgr2=0x7f18e0070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:37:27.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.056+0000 7f18dffff640 1 --2- 172.21.15.112:0/4228147922 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18e006f1f0 0x7f18e0070d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:27.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.056+0000 7f18dffff640 1 -- 172.21.15.112:0/4228147922 >> v1:172.21.15.187:6789/0 conn(0x7f18e00712a0 legacy=0x7f18e00abb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:27.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.056+0000 7f18dffff640 1 -- 172.21.15.112:0/4228147922 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e0163210 con 0x7f18e0072d50 2024-04-25T01:37:27.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.058+0000 7f18dffff640 1 --2- 172.21.15.112:0/4228147922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18e0072d50 0x7f18e015f3e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f18d8009b10 tx=0x7f18d802b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edb12970dd6e939d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:27.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.058+0000 7f18de7fc640 1 -- 172.21.15.112:0/4228147922 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8035020 con 0x7f18e0072d50 2024-04-25T01:37:27.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.058+0000 7f18de7fc640 1 -- 172.21.15.112:0/4228147922 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18d8031890 con 0x7f18e0072d50 2024-04-25T01:37:27.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.059+0000 7f18e61b7640 1 -- 172.21.15.112:0/4228147922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18e0072d50 msgr2=0x7f18e015f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:27.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.059+0000 7f18e61b7640 1 --2- 172.21.15.112:0/4228147922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18e0072d50 0x7f18e015f3e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f18d8009b10 tx=0x7f18d802b990 comp rx=0 tx=0).stop 2024-04-25T01:37:27.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.059+0000 7f18de7fc640 1 -- 172.21.15.112:0/4228147922 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8031c40 con 0x7f18e0072d50 2024-04-25T01:37:27.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.060+0000 7f18e61b7640 1 -- 172.21.15.112:0/4228147922 shutdown_connections 2024-04-25T01:37:27.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.060+0000 7f18e61b7640 1 --2- 172.21.15.112:0/4228147922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18e0072d50 0x7f18e015f3e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:27.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.060+0000 7f18e61b7640 1 --2- 172.21.15.112:0/4228147922 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18e006f1f0 0x7f18e0070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:27.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.060+0000 7f18e61b7640 1 -- 172.21.15.112:0/4228147922 >> 172.21.15.112:0/4228147922 conn(0x7f18e00c7080 msgr2=0x7f18e00c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:27.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.061+0000 7f18e61b7640 1 -- 172.21.15.112:0/4228147922 shutdown_connections 2024-04-25T01:37:27.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.061+0000 7f18e61b7640 1 -- 172.21.15.112:0/4228147922 wait complete. 2024-04-25T01:37:27.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.063+0000 7f18e61b7640 1 Processor -- start 2024-04-25T01:37:27.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.066+0000 7f18e61b7640 1 -- start start 2024-04-25T01:37:27.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.067+0000 7f18e61b7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18e006f1f0 0x7f18e00c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:27.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.067+0000 7f18e61b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18e0072d50 0x7f18e01f93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:27.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.067+0000 7f18e61b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18e01f9910 0x7f18e0164940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:27.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.067+0000 7f18df7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18e006f1f0 0x7f18e00c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:27.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.068+0000 7f18df7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18e006f1f0 0x7f18e00c5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:36324/0 (socket says 172.21.15.112:36324) 2024-04-25T01:37:27.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.068+0000 7f18dffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18e01f9910 0x7f18e0164940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:27.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.068+0000 7f18dffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18e01f9910 0x7f18e0164940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40786/0 (socket says 172.21.15.112:40786) 2024-04-25T01:37:27.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.068+0000 7f18e61b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f18e00aa7f0 con 0x7f18e01f9910 2024-04-25T01:37:27.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.068+0000 7f18e61b7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f18e00c1a80 con 0x7f18e0072d50 2024-04-25T01:37:27.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.068+0000 7f18e61b7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f18e006f5d0 con 0x7f18e006f1f0 2024-04-25T01:37:27.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.068+0000 7f18df7fe640 1 -- 172.21.15.112:0/1680267204 learned_addr learned my addr 172.21.15.112:0/1680267204 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:27.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.069+0000 7f18deffd640 1 --2- 172.21.15.112:0/1680267204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18e0072d50 0x7f18e01f93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:27.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.069+0000 7f18dffff640 1 -- 172.21.15.112:0/1680267204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18e006f1f0 msgr2=0x7f18e00c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:27.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.069+0000 7f18dffff640 1 --2- 172.21.15.112:0/1680267204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18e006f1f0 0x7f18e00c5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:27.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.069+0000 7f18dffff640 1 -- 172.21.15.112:0/1680267204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18e0072d50 msgr2=0x7f18e01f93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:27.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.069+0000 7f18dffff640 1 --2- 172.21.15.112:0/1680267204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18e0072d50 0x7f18e01f93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:27.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.069+0000 7f18dffff640 1 -- 172.21.15.112:0/1680267204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e00bc1b0 con 0x7f18e01f9910 2024-04-25T01:37:27.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.070+0000 7f18dffff640 1 --2- 172.21.15.112:0/1680267204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18e01f9910 0x7f18e0164940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f18e0073620 tx=0x7f18d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:27.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.070+0000 7f18c7fff640 1 -- 172.21.15.112:0/1680267204 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d804d020 con 0x7f18e01f9910 2024-04-25T01:37:27.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.071+0000 7f18c7fff640 1 -- 172.21.15.112:0/1680267204 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18d8031e30 con 0x7f18e01f9910 2024-04-25T01:37:27.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.071+0000 7f18c7fff640 1 -- 172.21.15.112:0/1680267204 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8003990 con 0x7f18e01f9910 2024-04-25T01:37:27.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.072+0000 7f18e61b7640 1 -- 172.21.15.112:0/1680267204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e01ebb00 con 0x7f18e01f9910 2024-04-25T01:37:27.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.072+0000 7f18e61b7640 1 -- 172.21.15.112:0/1680267204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e01ebdc0 con 0x7f18e01f9910 2024-04-25T01:37:27.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.073+0000 7f18e61b7640 1 -- 172.21.15.112:0/1680267204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18e00682a0 con 0x7f18e01f9910 2024-04-25T01:37:27.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.078+0000 7f18c7fff640 1 -- 172.21.15.112:0/1680267204 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f18d8050020 con 0x7f18e01f9910 2024-04-25T01:37:27.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.078+0000 7f18c7fff640 1 --2- 172.21.15.112:0/1680267204 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f18cc06bf30 0x7f18cc06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:27.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.079+0000 7f18c7fff640 1 -- 172.21.15.112:0/1680267204 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f18d80b47f0 con 0x7f18e01f9910 2024-04-25T01:37:27.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.079+0000 7f18df7fe640 1 --2- 172.21.15.112:0/1680267204 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f18cc06bf30 0x7f18cc06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:27.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.080+0000 7f18df7fe640 1 --2- 172.21.15.112:0/1680267204 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f18cc06bf30 0x7f18cc06e3f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009c00 tx=0x7f18d4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:27.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.085+0000 7f18c7fff640 1 -- 172.21.15.112:0/1680267204 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f18d8057030 con 0x7f18e01f9910 2024-04-25T01:37:27.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.594+0000 7f18e61b7640 1 -- 172.21.15.112:0/1680267204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f18e00aa370 con 0x7f18e01f9910 2024-04-25T01:37:27.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.597+0000 7f18c7fff640 1 -- 172.21.15.112:0/1680267204 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f18d802d040 con 0x7f18e01f9910 2024-04-25T01:37:27.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.604+0000 7f18e61b7640 1 -- 172.21.15.112:0/1680267204 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f18cc06bf30 msgr2=0x7f18cc06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:27.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.604+0000 7f18e61b7640 1 --2- 172.21.15.112:0/1680267204 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f18cc06bf30 0x7f18cc06e3f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009c00 tx=0x7f18d4009190 comp rx=0 tx=0).stop 2024-04-25T01:37:27.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.604+0000 7f18e61b7640 1 -- 172.21.15.112:0/1680267204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18e01f9910 msgr2=0x7f18e0164940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:27.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.604+0000 7f18e61b7640 1 --2- 172.21.15.112:0/1680267204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18e01f9910 0x7f18e0164940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f18e0073620 tx=0x7f18d8004a40 comp rx=0 tx=0).stop 2024-04-25T01:37:27.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.604+0000 7f18e61b7640 1 -- 172.21.15.112:0/1680267204 shutdown_connections 2024-04-25T01:37:27.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.604+0000 7f18e61b7640 1 --2- 172.21.15.112:0/1680267204 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f18cc06bf30 0x7f18cc06e3f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:27.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.604+0000 7f18e61b7640 1 --2- 172.21.15.112:0/1680267204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18e01f9910 0x7f18e0164940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:27.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.605+0000 7f18e61b7640 1 --2- 172.21.15.112:0/1680267204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18e0072d50 0x7f18e01f93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:27.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.605+0000 7f18e61b7640 1 --2- 172.21.15.112:0/1680267204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18e006f1f0 0x7f18e00c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:27.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.605+0000 7f18e61b7640 1 -- 172.21.15.112:0/1680267204 >> 172.21.15.112:0/1680267204 conn(0x7f18e00c7080 msgr2=0x7f18e0161e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:27.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.605+0000 7f18e61b7640 1 -- 172.21.15.112:0/1680267204 shutdown_connections 2024-04-25T01:37:27.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:27.605+0000 7f18e61b7640 1 -- 172.21.15.112:0/1680267204 wait complete. 2024-04-25T01:37:27.602 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T01:37:27.794 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:37:27.797 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:37:37.604 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:37:37.604 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:37:37.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.795+0000 7f0089942640 1 Processor -- start 2024-04-25T01:37:37.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.795+0000 7f0089942640 1 -- start start 2024-04-25T01:37:37.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.796+0000 7f0089942640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f00840c2090 0x7f00840ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:37.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.796+0000 7f0089942640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f00841522f0 0x7f008406ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:37.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.796+0000 7f0089942640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0084071aa0 con 0x7f008406f050 2024-04-25T01:37:37.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.796+0000 7f0089942640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f0084071d80 con 0x7f00840c2090 2024-04-25T01:37:37.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.796+0000 7f0089942640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0084072060 con 0x7f00841522f0 2024-04-25T01:37:37.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.796+0000 7f0082ffd640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f008406f050 0x7f00840c1980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:43870/0 (socket says 172.21.15.112:43870) 2024-04-25T01:37:37.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.796+0000 7f0082ffd640 1 -- 172.21.15.112:0/3912468883 learned_addr learned my addr 172.21.15.112:0/3912468883 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:37:37.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.797+0000 7f00827fc640 1 --2- 172.21.15.112:0/3912468883 >> v2:172.21.15.112:3300/0 conn(0x7f00840c2090 0x7f00840ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:37.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.797+0000 7f00837fe640 1 --2- 172.21.15.112:0/3912468883 >> v2:172.21.15.187:3300/0 conn(0x7f00841522f0 0x7f008406ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:37.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.797+0000 7f00827fc640 1 -- 172.21.15.112:0/3912468883 >> v2:172.21.15.187:3300/0 conn(0x7f00841522f0 msgr2=0x7f008406ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:37:37.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.797+0000 7f00827fc640 1 --2- 172.21.15.112:0/3912468883 >> v2:172.21.15.187:3300/0 conn(0x7f00841522f0 0x7f008406ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:37.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.797+0000 7f00827fc640 1 -- 172.21.15.112:0/3912468883 >> v1:172.21.15.187:6789/0 conn(0x7f008406f050 legacy=0x7f00840c1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:37.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.797+0000 7f00827fc640 1 -- 172.21.15.112:0/3912468883 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0084072340 con 0x7f00840c2090 2024-04-25T01:37:37.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.802+0000 7f00827fc640 1 --2- 172.21.15.112:0/3912468883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00840c2090 0x7f00840ac1a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f00840c37f0 tx=0x7f007802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7011d7f106aa79dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:37.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.803+0000 7f0081ffb640 1 -- 172.21.15.112:0/3912468883 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078037070 con 0x7f00840c2090 2024-04-25T01:37:37.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.803+0000 7f0081ffb640 1 -- 172.21.15.112:0/3912468883 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0078032440 con 0x7f00840c2090 2024-04-25T01:37:37.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.803+0000 7f0081ffb640 1 -- 172.21.15.112:0/3912468883 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078032810 con 0x7f00840c2090 2024-04-25T01:37:37.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.804+0000 7f0089942640 1 -- 172.21.15.112:0/3912468883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00840c2090 msgr2=0x7f00840ac1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:37.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.804+0000 7f0089942640 1 --2- 172.21.15.112:0/3912468883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00840c2090 0x7f00840ac1a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f00840c37f0 tx=0x7f007802a1a0 comp rx=0 tx=0).stop 2024-04-25T01:37:37.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.804+0000 7f0089942640 1 -- 172.21.15.112:0/3912468883 shutdown_connections 2024-04-25T01:37:37.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.804+0000 7f0089942640 1 --2- 172.21.15.112:0/3912468883 >> v2:172.21.15.187:3300/0 conn(0x7f00841522f0 0x7f008406ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:37.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.804+0000 7f0089942640 1 --2- 172.21.15.112:0/3912468883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00840c2090 0x7f00840ac1a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:37.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.804+0000 7f0089942640 1 -- 172.21.15.112:0/3912468883 >> 172.21.15.112:0/3912468883 conn(0x7f00840c6f80 msgr2=0x7f00840c7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:37.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.805+0000 7f0089942640 1 -- 172.21.15.112:0/3912468883 shutdown_connections 2024-04-25T01:37:37.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.805+0000 7f0089942640 1 -- 172.21.15.112:0/3912468883 wait complete. 2024-04-25T01:37:37.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.807+0000 7f0089942640 1 Processor -- start 2024-04-25T01:37:37.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.807+0000 7f0089942640 1 -- start start 2024-04-25T01:37:37.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.808+0000 7f0089942640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f008406f050 0x7f00840be940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:37.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.808+0000 7f0089942640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00840c2090 0x7f0084155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:37.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.808+0000 7f0089942640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00841522f0 0x7f00840bcd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:37.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.809+0000 7f0082ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f008406f050 0x7f00840be940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:37.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.809+0000 7f0082ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f008406f050 0x7f00840be940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43720/0 (socket says 172.21.15.112:43720) 2024-04-25T01:37:37.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.809+0000 7f0082ffd640 1 -- 172.21.15.112:0/3924680144 learned_addr learned my addr 172.21.15.112:0/3924680144 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:37.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.809+0000 7f0089942640 1 -- 172.21.15.112:0/3924680144 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0084072b40 con 0x7f00841522f0 2024-04-25T01:37:37.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.809+0000 7f0089942640 1 -- 172.21.15.112:0/3924680144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0084152700 con 0x7f008406f050 2024-04-25T01:37:37.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.809+0000 7f0089942640 1 -- 172.21.15.112:0/3924680144 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0084072cc0 con 0x7f00840c2090 2024-04-25T01:37:37.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.809+0000 7f00837fe640 1 --2- 172.21.15.112:0/3924680144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00841522f0 0x7f00840bcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:37.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.810+0000 7f0082ffd640 1 -- 172.21.15.112:0/3924680144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00840c2090 msgr2=0x7f0084155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:37.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.810+0000 7f0082ffd640 1 --2- 172.21.15.112:0/3924680144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00840c2090 0x7f0084155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:37.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.810+0000 7f0082ffd640 1 -- 172.21.15.112:0/3924680144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00841522f0 msgr2=0x7f00840bcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:37.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.810+0000 7f0082ffd640 1 --2- 172.21.15.112:0/3924680144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00841522f0 0x7f00840bcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:37.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.810+0000 7f0082ffd640 1 -- 172.21.15.112:0/3924680144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00840aa250 con 0x7f008406f050 2024-04-25T01:37:37.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.810+0000 7f00837fe640 1 --2- 172.21.15.112:0/3924680144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00841522f0 0x7f00840bcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:37:37.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.810+0000 7f0082ffd640 1 --2- 172.21.15.112:0/3924680144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f008406f050 0x7f00840be940 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f007c00cba0 tx=0x7f007c00a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:37.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.810+0000 7f006bfff640 1 -- 172.21.15.112:0/3924680144 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c017070 con 0x7f008406f050 2024-04-25T01:37:37.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.811+0000 7f006bfff640 1 -- 172.21.15.112:0/3924680144 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f007c00ad40 con 0x7f008406f050 2024-04-25T01:37:37.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.811+0000 7f006bfff640 1 -- 172.21.15.112:0/3924680144 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c012600 con 0x7f008406f050 2024-04-25T01:37:37.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.811+0000 7f0089942640 1 -- 172.21.15.112:0/3924680144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0084147f60 con 0x7f008406f050 2024-04-25T01:37:37.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.811+0000 7f0089942640 1 -- 172.21.15.112:0/3924680144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0084146f80 con 0x7f008406f050 2024-04-25T01:37:37.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.812+0000 7f0089942640 1 -- 172.21.15.112:0/3924680144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00841471c0 con 0x7f008406f050 2024-04-25T01:37:37.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.813+0000 7f006bfff640 1 -- 172.21.15.112:0/3924680144 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f007c01a070 con 0x7f008406f050 2024-04-25T01:37:37.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.813+0000 7f006bfff640 1 --2- 172.21.15.112:0/3924680144 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f007006bf00 0x7f007006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:37.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.814+0000 7f006bfff640 1 -- 172.21.15.112:0/3924680144 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f007c08aed0 con 0x7f008406f050 2024-04-25T01:37:37.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.814+0000 7f00827fc640 1 --2- 172.21.15.112:0/3924680144 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f007006bf00 0x7f007006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:37.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.815+0000 7f00827fc640 1 --2- 172.21.15.112:0/3924680144 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f007006bf00 0x7f007006e3c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0078034000 tx=0x7f007802c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:37.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:37.823+0000 7f006bfff640 1 -- 172.21.15.112:0/3924680144 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f00841471c0 con 0x7f008406f050 2024-04-25T01:37:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.342+0000 7f0089942640 1 -- 172.21.15.112:0/3924680144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f00840682a0 con 0x7f008406f050 2024-04-25T01:37:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.343+0000 7f006bfff640 1 -- 172.21.15.112:0/3924680144 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f007c058f00 con 0x7f008406f050 2024-04-25T01:37:38.321 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":94,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:37:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.347+0000 7f0089942640 1 -- 172.21.15.112:0/3924680144 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f007006bf00 msgr2=0x7f007006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.347+0000 7f0089942640 1 --2- 172.21.15.112:0/3924680144 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f007006bf00 0x7f007006e3c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0078034000 tx=0x7f007802c000 comp rx=0 tx=0).stop 2024-04-25T01:37:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.347+0000 7f0089942640 1 -- 172.21.15.112:0/3924680144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f008406f050 msgr2=0x7f00840be940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.347+0000 7f0089942640 1 --2- 172.21.15.112:0/3924680144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f008406f050 0x7f00840be940 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f007c00cba0 tx=0x7f007c00a430 comp rx=0 tx=0).stop 2024-04-25T01:37:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.347+0000 7f0089942640 1 -- 172.21.15.112:0/3924680144 shutdown_connections 2024-04-25T01:37:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.347+0000 7f0089942640 1 --2- 172.21.15.112:0/3924680144 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f007006bf00 0x7f007006e3c0 secure :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0078034000 tx=0x7f007802c000 comp rx=0 tx=0).stop 2024-04-25T01:37:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.347+0000 7f0089942640 1 --2- 172.21.15.112:0/3924680144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00841522f0 0x7f00840bcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.347+0000 7f0089942640 1 --2- 172.21.15.112:0/3924680144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00840c2090 0x7f0084155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.347+0000 7f0089942640 1 --2- 172.21.15.112:0/3924680144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f008406f050 0x7f00840be940 secure :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f007c00cba0 tx=0x7f007c00a430 comp rx=0 tx=0).stop 2024-04-25T01:37:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.347+0000 7f0089942640 1 -- 172.21.15.112:0/3924680144 >> 172.21.15.112:0/3924680144 conn(0x7f00840c6f80 msgr2=0x7f00840c7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.350+0000 7f0089942640 1 -- 172.21.15.112:0/3924680144 shutdown_connections 2024-04-25T01:37:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.352+0000 7f0089942640 1 -- 172.21.15.112:0/3924680144 wait complete. 2024-04-25T01:37:38.349 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:37:38.349 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T01:37:38.349 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:37:38.349 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:37:38.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.545+0000 7f9cfd3a2640 1 Processor -- start 2024-04-25T01:37:38.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.545+0000 7f9cfd3a2640 1 -- start start 2024-04-25T01:37:38.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.546+0000 7f9cfd3a2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cf806f510 0x7f9cf8071170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:38.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.546+0000 7f9cfd3a2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9cf80716b0 0x7f9cf814e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:38.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.546+0000 7f9cfd3a2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cf8155c70 con 0x7f9cf80730b0 2024-04-25T01:37:38.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.546+0000 7f9cfd3a2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f9cf80ac200 con 0x7f9cf80716b0 2024-04-25T01:37:38.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.546+0000 7f9cfd3a2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf81557e0 con 0x7f9cf806f510 2024-04-25T01:37:38.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.547+0000 7f9cf77fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9cf80716b0 0x7f9cf814e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:38.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.547+0000 7f9cf77fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9cf80716b0 0x7f9cf814e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41764/0 (socket says 172.21.15.112:41764) 2024-04-25T01:37:38.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.547+0000 7f9cf77fe640 1 -- 172.21.15.112:0/2162812915 learned_addr learned my addr 172.21.15.112:0/2162812915 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:38.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.547+0000 7f9cf77fe640 1 -- 172.21.15.112:0/2162812915 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cf806f510 msgr2=0x7f9cf8071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:37:38.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.547+0000 7f9cf77fe640 1 --2- 172.21.15.112:0/2162812915 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cf806f510 0x7f9cf8071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.547+0000 7f9cf77fe640 1 -- 172.21.15.112:0/2162812915 >> v1:172.21.15.187:6789/0 conn(0x7f9cf80730b0 legacy=0x7f9cf80c1a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:37:38.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.547+0000 7f9cf77fe640 1 -- 172.21.15.112:0/2162812915 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf80ac450 con 0x7f9cf80716b0 2024-04-25T01:37:38.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.549+0000 7f9cf77fe640 1 --2- 172.21.15.112:0/2162812915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf80716b0 0x7f9cf814e4e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9cec008870 tx=0x7f9cec02c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a87ff9c43465d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.551+0000 7f9cf5ffb640 1 -- 172.21.15.112:0/2162812915 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec03a070 con 0x7f9cf80716b0 2024-04-25T01:37:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.551+0000 7f9cf5ffb640 1 -- 172.21.15.112:0/2162812915 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cec032480 con 0x7f9cf80716b0 2024-04-25T01:37:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.551+0000 7f9cf5ffb640 1 -- 172.21.15.112:0/2162812915 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec032890 con 0x7f9cf80716b0 2024-04-25T01:37:38.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.552+0000 7f9cfd3a2640 1 -- 172.21.15.112:0/2162812915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf80716b0 msgr2=0x7f9cf814e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:38.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.552+0000 7f9cfd3a2640 1 --2- 172.21.15.112:0/2162812915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf80716b0 0x7f9cf814e4e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9cec008870 tx=0x7f9cec02c730 comp rx=0 tx=0).stop 2024-04-25T01:37:38.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.552+0000 7f9cfd3a2640 1 -- 172.21.15.112:0/2162812915 shutdown_connections 2024-04-25T01:37:38.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.552+0000 7f9cfd3a2640 1 --2- 172.21.15.112:0/2162812915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf80716b0 0x7f9cf814e4e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.552+0000 7f9cfd3a2640 1 --2- 172.21.15.112:0/2162812915 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cf806f510 0x7f9cf8071170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.552+0000 7f9cfd3a2640 1 -- 172.21.15.112:0/2162812915 >> 172.21.15.112:0/2162812915 conn(0x7f9cf80c7080 msgr2=0x7f9cf80c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:38.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.553+0000 7f9cfd3a2640 1 -- 172.21.15.112:0/2162812915 shutdown_connections 2024-04-25T01:37:38.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.553+0000 7f9cfd3a2640 1 -- 172.21.15.112:0/2162812915 wait complete. 2024-04-25T01:37:38.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.555+0000 7f9cfd3a2640 1 Processor -- start 2024-04-25T01:37:38.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.555+0000 7f9cfd3a2640 1 -- start start 2024-04-25T01:37:38.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.556+0000 7f9cfd3a2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf806f510 0x7f9cf814e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:38.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.557+0000 7f9cfd3a2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cf80716b0 0x7f9cf81dbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:38.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.557+0000 7f9cfd3a2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cf80730b0 0x7f9cf81518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:38.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.557+0000 7f9cfd3a2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf8155470 con 0x7f9cf806f510 2024-04-25T01:37:38.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.557+0000 7f9cfd3a2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf8155190 con 0x7f9cf80730b0 2024-04-25T01:37:38.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.557+0000 7f9cfd3a2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf80ab5a0 con 0x7f9cf80716b0 2024-04-25T01:37:38.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.557+0000 7f9cf6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf806f510 0x7f9cf814e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:38.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.557+0000 7f9cf6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf806f510 0x7f9cf814e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41778/0 (socket says 172.21.15.112:41778) 2024-04-25T01:37:38.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.557+0000 7f9cf6ffd640 1 -- 172.21.15.112:0/422875518 learned_addr learned my addr 172.21.15.112:0/422875518 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:38.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.557+0000 7f9cf77fe640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cf80730b0 0x7f9cf81518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:38.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.557+0000 7f9cf6ffd640 1 -- 172.21.15.112:0/422875518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cf80716b0 msgr2=0x7f9cf81dbe90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:37:38.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.557+0000 7f9cf6ffd640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cf80716b0 0x7f9cf81dbe90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.558+0000 7f9cf6ffd640 1 -- 172.21.15.112:0/422875518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cf80730b0 msgr2=0x7f9cf81518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:38.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.558+0000 7f9cf6ffd640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cf80730b0 0x7f9cf81518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.558+0000 7f9cf6ffd640 1 -- 172.21.15.112:0/422875518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf81e0620 con 0x7f9cf806f510 2024-04-25T01:37:38.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.558+0000 7f9cf77fe640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cf80730b0 0x7f9cf81518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:37:38.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.558+0000 7f9cf6ffd640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf806f510 0x7f9cf814e080 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000ba80 tx=0x7f9cf000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:38.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.560+0000 7f9cdf7fe640 1 -- 172.21.15.112:0/422875518 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0019070 con 0x7f9cf806f510 2024-04-25T01:37:38.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.560+0000 7f9cdf7fe640 1 -- 172.21.15.112:0/422875518 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf000cc00 con 0x7f9cf806f510 2024-04-25T01:37:38.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.560+0000 7f9cdf7fe640 1 -- 172.21.15.112:0/422875518 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0014440 con 0x7f9cf806f510 2024-04-25T01:37:38.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.560+0000 7f9cfd3a2640 1 -- 172.21.15.112:0/422875518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf81e0900 con 0x7f9cf806f510 2024-04-25T01:37:38.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.560+0000 7f9cfd3a2640 1 -- 172.21.15.112:0/422875518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf81e0ec0 con 0x7f9cf806f510 2024-04-25T01:37:38.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.561+0000 7f9cfd3a2640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd8002a70 0x7f9cd8004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:38.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.562+0000 7f9cfd3a2640 1 -- 172.21.15.112:0/422875518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd8007ec0 con 0x7f9cd8002a70 2024-04-25T01:37:38.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.563+0000 7f9cf67fc640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd8002a70 0x7f9cd8004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:38.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.564+0000 7f9cf67fc640 1 -- 172.21.15.112:0/422875518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9cd80080c0 con 0x7f9cd8002a70 2024-04-25T01:37:38.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.564+0000 7f9cdf7fe640 1 -- 172.21.15.112:0/422875518 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9cf001c070 con 0x7f9cf806f510 2024-04-25T01:37:38.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.565+0000 7f9cf67fc640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd8002a70 0x7f9cd8004f30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9cf81dce70 tx=0x7f9ce800b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:38.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.567+0000 7f9cdf7fe640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ce406bf30 0x7f9ce406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:38.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.567+0000 7f9cdf7fe640 1 -- 172.21.15.112:0/422875518 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f9cf008ade0 con 0x7f9cf806f510 2024-04-25T01:37:38.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.567+0000 7f9cdf7fe640 1 -- 172.21.15.112:0/422875518 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8015020 con 0x7f9cd8002a70 2024-04-25T01:37:38.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.567+0000 7f9cdf7fe640 1 -- 172.21.15.112:0/422875518 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9ce8011830 con 0x7f9cd8002a70 2024-04-25T01:37:38.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.567+0000 7f9cdf7fe640 1 -- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd8002a70 msgr2=0x7f9cd8004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:38.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.567+0000 7f9cdf7fe640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd8002a70 0x7f9cd8004f30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9cf81dce70 tx=0x7f9ce800b7a0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.567+0000 7f9cdf7fe640 1 -- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd8002a70 msgr2=0x7f9cd8004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:37:38.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.567+0000 7f9cdf7fe640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd8002a70 0x7f9cd8004f30 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9cf81dce70 tx=0x7f9ce800b7a0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.573+0000 7f9cf77fe640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ce406bf30 0x7f9ce406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:38.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.574+0000 7f9cf77fe640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ce406bf30 0x7f9ce406e3f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9cec037000 tx=0x7f9cec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:38.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.580+0000 7f9cfd3a2640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd8006f00 0x7f9cd8007390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:38.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.580+0000 7f9cfd3a2640 1 -- 172.21.15.112:0/422875518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd800a4a0 con 0x7f9cd8006f00 2024-04-25T01:37:38.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.580+0000 7f9cf67fc640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd8006f00 0x7f9cd8007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.581+0000 7f9cf67fc640 1 -- 172.21.15.112:0/422875518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9cd80078d0 con 0x7f9cd8006f00 2024-04-25T01:37:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.581+0000 7f9cf67fc640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd8006f00 0x7f9cd8007390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9cf81dce70 tx=0x7f9ce80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.582+0000 7f9cdf7fe640 1 -- 172.21.15.112:0/422875518 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8015050 con 0x7f9cd8006f00 2024-04-25T01:37:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.582+0000 7f9cdf7fe640 1 -- 172.21.15.112:0/422875518 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4562 (secure 0 0 0) 0x7f9ce801c2d0 con 0x7f9cd8006f00 2024-04-25T01:37:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.582+0000 7f9cdf7fe640 1 -- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd8006f00 msgr2=0x7f9cd8007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.582+0000 7f9cdf7fe640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd8006f00 0x7f9cd8007390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9cf81dce70 tx=0x7f9ce80046c0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.583+0000 7f9cdf7fe640 1 -- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd8006f00 msgr2=0x7f9cd8007390 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:37:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.583+0000 7f9cdf7fe640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd8006f00 0x7f9cd8007390 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9cf81dce70 tx=0x7f9ce80046c0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.561 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:37:38.561 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:37:38.561 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:37:38.561 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:37:38.561 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 94, 2024-04-25T01:37:38.562 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 109397, 2024-04-25T01:37:38.562 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:37:38.562 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:37:38.562 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:37:38.562 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:37:38.562 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:38.562 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 13, 2024-04-25T01:37:38.562 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:37:38.562 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:37:38.562 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:37:38.562 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:38.562 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:38.562 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:38.563 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:38.563 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:38.563 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:38.563 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:38.563 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:38.563 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:38.563 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:38.563 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:38.563 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:38.563 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:37:38.563 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:37:38.563 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:38.563 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:38.563 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:38.563 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:38.564 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:38.564 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:38.564 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:38.564 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:38.564 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:38.564 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:38.564 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:38.564 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:38.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.564 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:37:38.564 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:37:38.564 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:37:38.564 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:37:38.564 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:37:38.564 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:37:38.565 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:37:38.565 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:37:38.565 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:37:38.565 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:37:38.565 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:37:38.565 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:37:38.565 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:37:38.565 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:37:38.565 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:37:38.565 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:37:38.565 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:37:38.566 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:38.566 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:38.566 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:38.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:38.566 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:38.566 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:38.566 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:38.566 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:38.566 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:38.566 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:38.566 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:38.566 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:38.567 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:37:38.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.567 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:37:38.567 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.567 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:37:38.568 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:37:38.568 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:38.568 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:38.568 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.568 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:38.568 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:37:38.568 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:38.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.569 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.569 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:38.569 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:37:38.569 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:38.569 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:38.569 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:38.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.569 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:37:38.570 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:37:38.570 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:38.570 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:38.570 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:38.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.570 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.570 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:37:38.570 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:37:38.570 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:38.570 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:38.570 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.570 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:38.570 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:37:38.570 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:38.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.571 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.571 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:38.571 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:37:38.571 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:38.571 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:38.571 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:38.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.571 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:37:38.571 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:37:38.571 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:38.571 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:38.571 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:38.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.571 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.572 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:37:38.572 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:37:38.572 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:38.572 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:38.572 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.572 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:38.572 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:37:38.572 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:38.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.572 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.572 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:38.572 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:37:38.572 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:38.572 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:38.572 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:38.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.573 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:37:38.573 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:37:38.573 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:38.573 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:38.573 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:38.573 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:38.573 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:38.573 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.573 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:37:38.573 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:37:38.573 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.573 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:38.573 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:38.573 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:37:38.573 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:38.574 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:38.574 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:37:38.574 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.574 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:38.574 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:38.574 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:37:38.574 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:38.574 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:38.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.574 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:37:38.574 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:37:38.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.586+0000 7f9cfd3a2640 1 -- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ce406bf30 msgr2=0x7f9ce406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:38.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.586+0000 7f9cfd3a2640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ce406bf30 0x7f9ce406e3f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9cec037000 tx=0x7f9cec008040 comp rx=0 tx=0).stop 2024-04-25T01:37:38.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.586+0000 7f9cf77fe640 1 -- 172.21.15.112:0/422875518 reap_dead start 2024-04-25T01:37:38.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.586+0000 7f9cfd3a2640 1 -- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf806f510 msgr2=0x7f9cf814e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:38.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.586+0000 7f9cfd3a2640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf806f510 0x7f9cf814e080 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000ba80 tx=0x7f9cf000bf40 comp rx=0 tx=0).stop 2024-04-25T01:37:38.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.586+0000 7f9cfd3a2640 1 -- 172.21.15.112:0/422875518 shutdown_connections 2024-04-25T01:37:38.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.586+0000 7f9cfd3a2640 1 --2- 172.21.15.112:0/422875518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf806f510 0x7f9cf814e080 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.586+0000 7f9cfd3a2640 1 -- 172.21.15.112:0/422875518 >> 172.21.15.112:0/422875518 conn(0x7f9cf80c7080 msgr2=0x7f9cf806dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:38.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.587+0000 7f9cfd3a2640 1 -- 172.21.15.112:0/422875518 shutdown_connections 2024-04-25T01:37:38.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.587+0000 7f9cfd3a2640 1 -- 172.21.15.112:0/422875518 wait complete. 2024-04-25T01:37:38.586 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:37:38.586 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:37:38.586 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:37:38.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.783+0000 7fb607fff640 1 Processor -- start 2024-04-25T01:37:38.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.784+0000 7fb607fff640 1 -- start start 2024-04-25T01:37:38.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.785+0000 7fb607fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb6000dbce0 0x7fb6000474a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:38.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.785+0000 7fb607fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb6000df180 0x7fb600006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:38.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.785+0000 7fb607fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6000073e0 0x7fb60000a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:38.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.785+0000 7fb607fff640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fb60000a420 con 0x7fb6000df180 2024-04-25T01:37:38.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.785+0000 7fb607fff640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb60000bb50 con 0x7fb6000dbce0 2024-04-25T01:37:38.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.785+0000 7fb607fff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb600048060 con 0x7fb6000073e0 2024-04-25T01:37:38.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.785+0000 7fb6067fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb6000df180 0x7fb600006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:38.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.785+0000 7fb6067fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb6000df180 0x7fb600006ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41804/0 (socket says 172.21.15.112:41804) 2024-04-25T01:37:38.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.785+0000 7fb6067fc640 1 -- 172.21.15.112:0/2553916259 learned_addr learned my addr 172.21.15.112:0/2553916259 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:38.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.786+0000 7fb6067fc640 1 -- 172.21.15.112:0/2553916259 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6000073e0 msgr2=0x7fb60000a8b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:37:38.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.786+0000 7fb6067fc640 1 --2- 172.21.15.112:0/2553916259 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6000073e0 0x7fb60000a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.786+0000 7fb6067fc640 1 -- 172.21.15.112:0/2553916259 >> v2:172.21.15.187:3300/0 conn(0x7fb6000dbce0 msgr2=0x7fb6000474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:37:38.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.786+0000 7fb606ffd640 1 --2- 172.21.15.112:0/2553916259 >> v2:172.21.15.187:3300/0 conn(0x7fb6000dbce0 0x7fb6000474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:38.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.786+0000 7fb6067fc640 1 --2- 172.21.15.112:0/2553916259 >> v2:172.21.15.187:3300/0 conn(0x7fb6000dbce0 0x7fb6000474a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.786+0000 7fb6067fc640 1 -- 172.21.15.112:0/2553916259 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60000a0f0 con 0x7fb6000df180 2024-04-25T01:37:38.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.787+0000 7fb6067fc640 1 --2- 172.21.15.112:0/2553916259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6000df180 0x7fb600006ea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb60000a2c0 tx=0x7fb6080a5f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c16dc613163b247 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:38.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.787+0000 7fb605ffb640 1 -- 172.21.15.112:0/2553916259 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6080b3070 con 0x7fb6000df180 2024-04-25T01:37:38.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.787+0000 7fb605ffb640 1 -- 172.21.15.112:0/2553916259 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6080a6d10 con 0x7fb6000df180 2024-04-25T01:37:38.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.787+0000 7fb605ffb640 1 -- 172.21.15.112:0/2553916259 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6080ae640 con 0x7fb6000df180 2024-04-25T01:37:38.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.789+0000 7fb607fff640 1 -- 172.21.15.112:0/2553916259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6000df180 msgr2=0x7fb600006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:38.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.789+0000 7fb607fff640 1 --2- 172.21.15.112:0/2553916259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6000df180 0x7fb600006ea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb60000a2c0 tx=0x7fb6080a5f00 comp rx=0 tx=0).stop 2024-04-25T01:37:38.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.789+0000 7fb607fff640 1 -- 172.21.15.112:0/2553916259 shutdown_connections 2024-04-25T01:37:38.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.789+0000 7fb607fff640 1 --2- 172.21.15.112:0/2553916259 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6000073e0 0x7fb60000a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.789+0000 7fb607fff640 1 --2- 172.21.15.112:0/2553916259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6000df180 0x7fb600006ea0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.789+0000 7fb607fff640 1 --2- 172.21.15.112:0/2553916259 >> v2:172.21.15.187:3300/0 conn(0x7fb6000dbce0 0x7fb6000474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.789+0000 7fb607fff640 1 -- 172.21.15.112:0/2553916259 >> 172.21.15.112:0/2553916259 conn(0x7fb600048250 msgr2=0x7fb600048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:38.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.789+0000 7fb607fff640 1 -- 172.21.15.112:0/2553916259 shutdown_connections 2024-04-25T01:37:38.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.789+0000 7fb607fff640 1 -- 172.21.15.112:0/2553916259 wait complete. 2024-04-25T01:37:38.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.791+0000 7fb607fff640 1 Processor -- start 2024-04-25T01:37:38.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.791+0000 7fb607fff640 1 -- start start 2024-04-25T01:37:38.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.792+0000 7fb607fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6000073e0 0x7fb6000443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:38.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.796+0000 7fb606ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6000073e0 0x7fb6000443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:38.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.796+0000 7fb606ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6000073e0 0x7fb6000443a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41812/0 (socket says 172.21.15.112:41812) 2024-04-25T01:37:38.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.796+0000 7fb606ffd640 1 -- 172.21.15.112:0/331790348 learned_addr learned my addr 172.21.15.112:0/331790348 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:38.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.796+0000 7fb607fff640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6000dbce0 0x7fb6000dac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:38.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.796+0000 7fb607fff640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6000df180 0x7fb6000db140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:38.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.796+0000 7fb607fff640 1 -- 172.21.15.112:0/331790348 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb6000dc0c0 con 0x7fb6000073e0 2024-04-25T01:37:38.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.796+0000 7fb607fff640 1 -- 172.21.15.112:0/331790348 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb600047d80 con 0x7fb6000df180 2024-04-25T01:37:38.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.796+0000 7fb607fff640 1 -- 172.21.15.112:0/331790348 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb6000df5f0 con 0x7fb6000dbce0 2024-04-25T01:37:38.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.797+0000 7fb6067fc640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6000dbce0 0x7fb6000dac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:38.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.797+0000 7fb6077fe640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6000df180 0x7fb6000db140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:38.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.797+0000 7fb606ffd640 1 -- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6000dbce0 msgr2=0x7fb6000dac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:38.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.797+0000 7fb606ffd640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6000dbce0 0x7fb6000dac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.797+0000 7fb606ffd640 1 -- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6000df180 msgr2=0x7fb6000db140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:38.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.797+0000 7fb606ffd640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6000df180 0x7fb6000db140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.798+0000 7fb606ffd640 1 -- 172.21.15.112:0/331790348 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60002e520 con 0x7fb6000073e0 2024-04-25T01:37:38.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.798+0000 7fb606ffd640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6000073e0 0x7fb6000443a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800c820 tx=0x7fb5f800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:38.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.798+0000 7fb5effff640 1 -- 172.21.15.112:0/331790348 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8015070 con 0x7fb6000073e0 2024-04-25T01:37:38.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.798+0000 7fb5effff640 1 -- 172.21.15.112:0/331790348 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f8004020 con 0x7fb6000073e0 2024-04-25T01:37:38.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.798+0000 7fb5effff640 1 -- 172.21.15.112:0/331790348 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80043f0 con 0x7fb6000073e0 2024-04-25T01:37:38.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.799+0000 7fb607fff640 1 -- 172.21.15.112:0/331790348 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60002e800 con 0x7fb6000073e0 2024-04-25T01:37:38.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.800+0000 7fb607fff640 1 -- 172.21.15.112:0/331790348 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb60002edc0 con 0x7fb6000073e0 2024-04-25T01:37:38.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.801+0000 7fb5effff640 1 -- 172.21.15.112:0/331790348 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb5f8007830 con 0x7fb6000073e0 2024-04-25T01:37:38.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.803+0000 7fb5effff640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb5f406beb0 0x7fb5f406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:38.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.803+0000 7fb5effff640 1 -- 172.21.15.112:0/331790348 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fb5f8012e40 con 0x7fb6000073e0 2024-04-25T01:37:38.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.804+0000 7fb6067fc640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb5f406beb0 0x7fb5f406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:38.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.804+0000 7fb6067fc640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb5f406beb0 0x7fb5f406e370 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb60000c070 tx=0x7fb6080aa000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:38.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.805+0000 7fb607fff640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb600030fe0 0x7fb600003190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:38.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.805+0000 7fb607fff640 1 -- 172.21.15.112:0/331790348 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb600005390 con 0x7fb600030fe0 2024-04-25T01:37:38.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.806+0000 7fb6077fe640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb600030fe0 0x7fb600003190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:38.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.807+0000 7fb6077fe640 1 -- 172.21.15.112:0/331790348 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb60000b590 con 0x7fb600030fe0 2024-04-25T01:37:38.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.807+0000 7fb6077fe640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb600030fe0 0x7fb600003190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00d430 tx=0x7fb5fc00d8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:38.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.808+0000 7fb5effff640 1 -- 172.21.15.112:0/331790348 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc018020 con 0x7fb600030fe0 2024-04-25T01:37:38.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.808+0000 7fb5effff640 1 -- 172.21.15.112:0/331790348 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb5fc014620 con 0x7fb600030fe0 2024-04-25T01:37:38.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.808+0000 7fb5effff640 1 -- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb600030fe0 msgr2=0x7fb600003190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:38.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.808+0000 7fb5effff640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb600030fe0 0x7fb600003190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00d430 tx=0x7fb5fc00d8f0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.808+0000 7fb5effff640 1 -- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb600030fe0 msgr2=0x7fb600003190 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:37:38.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.808+0000 7fb5effff640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb600030fe0 0x7fb600003190 secure :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00d430 tx=0x7fb5fc00d8f0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.828+0000 7fb607fff640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb60016b050 0x7fb60003f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:38.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.828+0000 7fb607fff640 1 -- 172.21.15.112:0/331790348 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb600006e40 con 0x7fb60016b050 2024-04-25T01:37:38.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.831+0000 7fb6077fe640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb60016b050 0x7fb60003f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:38.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.835+0000 7fb6077fe640 1 -- 172.21.15.112:0/331790348 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb600006b60 con 0x7fb60016b050 2024-04-25T01:37:38.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.837+0000 7fb6077fe640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb60016b050 0x7fb60003f2b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb6000089d0 tx=0x7fb5fc0046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:38.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.838+0000 7fb5effff640 1 -- 172.21.15.112:0/331790348 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc018020 con 0x7fb60016b050 2024-04-25T01:37:38.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.838+0000 7fb5effff640 1 -- 172.21.15.112:0/331790348 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5070 (secure 0 0 0) 0x7fb5fc01f2c0 con 0x7fb60016b050 2024-04-25T01:37:38.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.838+0000 7fb5effff640 1 -- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb60016b050 msgr2=0x7fb60003f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:38.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.838+0000 7fb5effff640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb60016b050 0x7fb60003f2b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb6000089d0 tx=0x7fb5fc0046b0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.838+0000 7fb5effff640 1 -- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb60016b050 msgr2=0x7fb60003f2b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:37:38.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.838+0000 7fb5effff640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb60016b050 0x7fb60003f2b0 secure :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb6000089d0 tx=0x7fb5fc0046b0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.819 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:37:38.819 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:37:38.819 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:37:38.820 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:37:38.820 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 94, 2024-04-25T01:37:38.820 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 494527, 2024-04-25T01:37:38.820 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:37:38.820 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:37:38.820 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:37:38.820 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:37:38.820 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:38.821 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 13, 2024-04-25T01:37:38.821 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:37:38.821 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:37:38.821 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:37:38.821 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:38.821 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:38.821 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:38.822 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:38.822 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:38.822 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:38.822 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:38.822 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:38.822 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:38.822 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:38.822 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:38.822 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:38.822 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:37:38.822 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:37:38.823 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:38.823 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:38.823 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:38.823 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:38.823 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:38.823 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:38.823 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:38.823 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:38.823 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:38.823 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:38.823 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:38.823 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:38.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.824 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:37:38.824 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:37:38.824 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:37:38.824 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:37:38.824 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:37:38.824 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:37:38.824 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:37:38.824 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:37:38.824 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:37:38.824 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:37:38.824 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:37:38.825 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:37:38.825 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:37:38.825 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:37:38.825 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:37:38.825 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:37:38.825 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:37:38.825 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:38.825 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:38.826 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:38.826 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:38.826 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:38.826 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:38.826 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:38.826 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:38.826 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:38.826 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:38.826 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:38.826 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:38.826 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:37:38.826 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.827 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:37:38.827 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.827 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:37:38.827 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:37:38.827 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:38.827 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:38.827 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.827 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:38.827 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:37:38.827 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:38.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.827 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.828 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:38.828 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:37:38.828 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:38.828 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:38.828 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:38.828 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.828 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:37:38.828 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:37:38.828 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:38.828 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:38.828 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:38.828 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.828 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.829 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:37:38.829 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:37:38.829 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:38.829 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:38.829 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.829 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:38.829 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:37:38.829 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:38.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.829 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.829 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:38.829 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:37:38.830 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:38.830 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:38.830 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:38.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.830 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:37:38.830 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:37:38.830 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:38.830 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:38.830 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:38.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.830 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.831 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:37:38.831 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:37:38.831 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:38.831 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:38.831 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.831 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:38.831 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:37:38.831 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:38.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.831 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.831 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:38.831 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:37:38.831 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:38.832 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:38.832 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:38.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.832 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:37:38.832 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:37:38.832 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:38.832 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:38.832 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:38.832 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:38.832 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:38.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.832 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:37:38.833 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:37:38.833 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.833 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:38.833 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:38.833 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:37:38.833 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:38.833 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:38.833 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:37:38.833 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.833 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:38.833 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:38.833 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:37:38.833 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:38.833 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:38.834 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:37:38.834 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.834 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:38.834 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:38.834 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 8 2024-04-25T01:37:38.834 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:38.834 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:38.834 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:37:38.834 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.834 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:38.834 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:38.834 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 6 2024-04-25T01:37:38.835 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:38.835 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:38.835 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:37:38.835 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:38.835 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:38.835 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:38.835 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:37:38.835 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:38.835 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:38.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:38.835 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:37:38.835 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:37:38.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.843+0000 7fb607fff640 1 -- 172.21.15.112:0/331790348 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb5f406beb0 msgr2=0x7fb5f406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:38.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.843+0000 7fb607fff640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb5f406beb0 0x7fb5f406e370 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb60000c070 tx=0x7fb6080aa000 comp rx=0 tx=0).stop 2024-04-25T01:37:38.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.843+0000 7fb6077fe640 1 -- 172.21.15.112:0/331790348 reap_dead start 2024-04-25T01:37:38.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.843+0000 7fb607fff640 1 -- 172.21.15.112:0/331790348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6000073e0 msgr2=0x7fb6000443a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:38.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.843+0000 7fb607fff640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6000073e0 0x7fb6000443a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800c820 tx=0x7fb5f800cce0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.845+0000 7fb607fff640 1 -- 172.21.15.112:0/331790348 shutdown_connections 2024-04-25T01:37:38.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.845+0000 7fb607fff640 1 --2- 172.21.15.112:0/331790348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6000073e0 0x7fb6000443a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:38.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.845+0000 7fb607fff640 1 -- 172.21.15.112:0/331790348 >> 172.21.15.112:0/331790348 conn(0x7fb600048250 msgr2=0x7fb6000055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:38.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.846+0000 7fb607fff640 1 -- 172.21.15.112:0/331790348 shutdown_connections 2024-04-25T01:37:38.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:38.847+0000 7fb607fff640 1 -- 172.21.15.112:0/331790348 wait complete. 2024-04-25T01:37:38.843 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:37:38.843 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:37:38.843 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:37:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.035+0000 7fe67eb5a640 1 Processor -- start 2024-04-25T01:37:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.035+0000 7fe67eb5a640 1 -- start start 2024-04-25T01:37:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.036+0000 7fe67eb5a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe67806c3e0 0x7fe6780743d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.036+0000 7fe67eb5a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe6780c6930 0x7fe6780c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.036+0000 7fe67eb5a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6780c6400 con 0x7fe67814f170 2024-04-25T01:37:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.036+0000 7fe67eb5a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe6780c56d0 con 0x7fe6780c6930 2024-04-25T01:37:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.036+0000 7fe67eb5a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe6780c73f0 con 0x7fe67806c3e0 2024-04-25T01:37:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.036+0000 7fe677fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe6780c6930 0x7fe6780c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.036+0000 7fe677fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe6780c6930 0x7fe6780c6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43764/0 (socket says 172.21.15.112:43764) 2024-04-25T01:37:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.036+0000 7fe677fff640 1 -- 172.21.15.112:0/4074145635 learned_addr learned my addr 172.21.15.112:0/4074145635 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.037+0000 7fe67c8f6640 1 --2- 172.21.15.112:0/4074145635 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe67806c3e0 0x7fe6780743d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.037+0000 7fe677fff640 1 -- 172.21.15.112:0/4074145635 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe67806c3e0 msgr2=0x7fe6780743d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:37:39.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.037+0000 7fe677fff640 1 --2- 172.21.15.112:0/4074145635 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe67806c3e0 0x7fe6780743d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:39.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.037+0000 7fe677fff640 1 -- 172.21.15.112:0/4074145635 >> v1:172.21.15.187:6789/0 conn(0x7fe67814f170 legacy=0x7fe678151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:39.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.037+0000 7fe677fff640 1 -- 172.21.15.112:0/4074145635 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6780c5920 con 0x7fe6780c6930 2024-04-25T01:37:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.040+0000 7fe677fff640 1 --2- 172.21.15.112:0/4074145635 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6780c6930 0x7fe6780c6d90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe670009870 tx=0x7fe67002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e460867968ba865e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.040+0000 7fe6777fe640 1 -- 172.21.15.112:0/4074145635 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670037070 con 0x7fe6780c6930 2024-04-25T01:37:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.041+0000 7fe6777fe640 1 -- 172.21.15.112:0/4074145635 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe670032440 con 0x7fe6780c6930 2024-04-25T01:37:39.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.041+0000 7fe6777fe640 1 -- 172.21.15.112:0/4074145635 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670037070 con 0x7fe6780c6930 2024-04-25T01:37:39.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.042+0000 7fe67eb5a640 1 -- 172.21.15.112:0/4074145635 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6780c6930 msgr2=0x7fe6780c6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:39.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.042+0000 7fe67eb5a640 1 --2- 172.21.15.112:0/4074145635 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6780c6930 0x7fe6780c6d90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe670009870 tx=0x7fe67002a130 comp rx=0 tx=0).stop 2024-04-25T01:37:39.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.042+0000 7fe67eb5a640 1 -- 172.21.15.112:0/4074145635 shutdown_connections 2024-04-25T01:37:39.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.042+0000 7fe67eb5a640 1 --2- 172.21.15.112:0/4074145635 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6780c6930 0x7fe6780c6d90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:39.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.042+0000 7fe67eb5a640 1 --2- 172.21.15.112:0/4074145635 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe67806c3e0 0x7fe6780743d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:39.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.042+0000 7fe67eb5a640 1 -- 172.21.15.112:0/4074145635 >> 172.21.15.112:0/4074145635 conn(0x7fe6780c4260 msgr2=0x7fe6780c4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:39.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.042+0000 7fe67eb5a640 1 -- 172.21.15.112:0/4074145635 shutdown_connections 2024-04-25T01:37:39.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.042+0000 7fe67eb5a640 1 -- 172.21.15.112:0/4074145635 wait complete. 2024-04-25T01:37:39.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.043+0000 7fe67eb5a640 1 Processor -- start 2024-04-25T01:37:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.044+0000 7fe67eb5a640 1 -- start start 2024-04-25T01:37:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.045+0000 7fe67eb5a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe67806c3e0 0x7fe67814dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.045+0000 7fe67eb5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6780c6930 0x7fe678141840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.045+0000 7fe67eb5a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe67814f170 0x7fe678141d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.045+0000 7fe67eb5a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe6780c2c90 con 0x7fe6780c6930 2024-04-25T01:37:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.045+0000 7fe67eb5a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe6780c5ee0 con 0x7fe67814f170 2024-04-25T01:37:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.045+0000 7fe67eb5a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe678074910 con 0x7fe67806c3e0 2024-04-25T01:37:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.046+0000 7fe67d0f7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe67814f170 0x7fe678141d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.046+0000 7fe67d0f7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe67814f170 0x7fe678141d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43776/0 (socket says 172.21.15.112:43776) 2024-04-25T01:37:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.046+0000 7fe67d0f7640 1 -- 172.21.15.112:0/2488548428 learned_addr learned my addr 172.21.15.112:0/2488548428 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.046+0000 7fe677fff640 1 --2- 172.21.15.112:0/2488548428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6780c6930 0x7fe678141840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.046+0000 7fe67d0f7640 1 -- 172.21.15.112:0/2488548428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe67806c3e0 msgr2=0x7fe67814dec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:37:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.046+0000 7fe67d0f7640 1 --2- 172.21.15.112:0/2488548428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe67806c3e0 0x7fe67814dec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.046+0000 7fe67d0f7640 1 -- 172.21.15.112:0/2488548428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6780c6930 msgr2=0x7fe678141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.046+0000 7fe67d0f7640 1 --2- 172.21.15.112:0/2488548428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6780c6930 0x7fe678141840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.046+0000 7fe67d0f7640 1 -- 172.21.15.112:0/2488548428 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6780a8ee0 con 0x7fe67814f170 2024-04-25T01:37:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.047+0000 7fe67d0f7640 1 --2- 172.21.15.112:0/2488548428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe67814f170 0x7fe678141d80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe66000c3c0 tx=0x7fe66000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:39.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.048+0000 7fe6757fa640 1 -- 172.21.15.112:0/2488548428 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6600052d0 con 0x7fe67814f170 2024-04-25T01:37:39.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.048+0000 7fe67eb5a640 1 -- 172.21.15.112:0/2488548428 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6780a91c0 con 0x7fe67814f170 2024-04-25T01:37:39.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.048+0000 7fe67eb5a640 1 -- 172.21.15.112:0/2488548428 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe678148460 con 0x7fe67814f170 2024-04-25T01:37:39.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.048+0000 7fe6757fa640 1 -- 172.21.15.112:0/2488548428 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe660005470 con 0x7fe67814f170 2024-04-25T01:37:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.048+0000 7fe6757fa640 1 -- 172.21.15.112:0/2488548428 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660007220 con 0x7fe67814f170 2024-04-25T01:37:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.049+0000 7fe67eb5a640 1 --2- 172.21.15.112:0/2488548428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe65c002a70 0x7fe65c004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.049+0000 7fe67eb5a640 1 -- 172.21.15.112:0/2488548428 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe65c007ec0 con 0x7fe65c002a70 2024-04-25T01:37:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.049+0000 7fe67c8f6640 1 --2- 172.21.15.112:0/2488548428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe65c002a70 0x7fe65c004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.050+0000 7fe67c8f6640 1 -- 172.21.15.112:0/2488548428 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe65c0080c0 con 0x7fe65c002a70 2024-04-25T01:37:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.051+0000 7fe67c8f6640 1 --2- 172.21.15.112:0/2488548428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe65c002a70 0x7fe65c004f30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe66800eb20 tx=0x7fe66800c0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.051+0000 7fe6757fa640 1 -- 172.21.15.112:0/2488548428 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe6600073c0 con 0x7fe67814f170 2024-04-25T01:37:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.051+0000 7fe6757fa640 1 --2- 172.21.15.112:0/2488548428 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe65806be60 0x7fe65806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:39.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.051+0000 7fe6757fa640 1 -- 172.21.15.112:0/2488548428 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fe66008a470 con 0x7fe67814f170 2024-04-25T01:37:39.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.052+0000 7fe6757fa640 1 -- 172.21.15.112:0/2488548428 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668018020 con 0x7fe65c002a70 2024-04-25T01:37:39.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.052+0000 7fe677fff640 1 --2- 172.21.15.112:0/2488548428 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe65806be60 0x7fe65806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:39.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.052+0000 7fe6757fa640 1 -- 172.21.15.112:0/2488548428 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe66800cc70 con 0x7fe65c002a70 2024-04-25T01:37:39.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.052+0000 7fe6757fa640 1 -- 172.21.15.112:0/2488548428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe65c002a70 msgr2=0x7fe65c004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:39.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.052+0000 7fe6757fa640 1 --2- 172.21.15.112:0/2488548428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe65c002a70 0x7fe65c004f30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe66800eb20 tx=0x7fe66800c0c0 comp rx=0 tx=0).stop 2024-04-25T01:37:39.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.053+0000 7fe6757fa640 1 -- 172.21.15.112:0/2488548428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe65c002a70 msgr2=0x7fe65c004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:37:39.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.053+0000 7fe6757fa640 1 --2- 172.21.15.112:0/2488548428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe65c002a70 0x7fe65c004f30 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe66800eb20 tx=0x7fe66800c0c0 comp rx=0 tx=0).stop 2024-04-25T01:37:39.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.055+0000 7fe677fff640 1 --2- 172.21.15.112:0/2488548428 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe65806be60 0x7fe65806e320 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe670034000 tx=0x7fe67002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:39.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.075+0000 7fe67eb5a640 1 --2- 172.21.15.112:0/2488548428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe65c006f30 0x7fe65c0073a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:39.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.075+0000 7fe67eb5a640 1 -- 172.21.15.112:0/2488548428 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe65c00a4a0 con 0x7fe65c006f30 2024-04-25T01:37:39.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.076+0000 7fe67c8f6640 1 --2- 172.21.15.112:0/2488548428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe65c006f30 0x7fe65c0073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:39.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.076+0000 7fe67c8f6640 1 -- 172.21.15.112:0/2488548428 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe65c0078e0 con 0x7fe65c006f30 2024-04-25T01:37:39.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.077+0000 7fe67c8f6640 1 --2- 172.21.15.112:0/2488548428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe65c006f30 0x7fe65c0073a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe66800eb20 tx=0x7fe668004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:39.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.077+0000 7fe6757fa640 1 -- 172.21.15.112:0/2488548428 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668018030 con 0x7fe65c006f30 2024-04-25T01:37:39.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.078+0000 7fe6757fa640 1 -- 172.21.15.112:0/2488548428 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4559 (secure 0 0 0) 0x7fe66801f020 con 0x7fe65c006f30 2024-04-25T01:37:39.056 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:37:39.056 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:37:39.056 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:37:39.056 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:37:39.056 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 94, 2024-04-25T01:37:39.056 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 28442, 2024-04-25T01:37:39.056 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:37:39.056 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:37:39.056 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:37:39.057 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:37:39.057 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:39.057 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 13, 2024-04-25T01:37:39.057 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:37:39.057 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:37:39.057 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:37:39.057 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:39.057 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:39.057 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:39.057 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:39.057 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:39.057 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:39.057 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:39.058 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:39.058 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:39.058 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:39.058 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:39.058 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:39.058 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:37:39.058 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:37:39.058 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:39.058 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:39.058 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:39.058 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:39.058 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:39.058 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:39.059 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:39.059 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:39.059 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:39.059 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:39.059 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:39.059 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:39.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:39.059 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:37:39.059 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:37:39.059 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:37:39.059 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:37:39.060 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:37:39.060 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:37:39.060 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:37:39.060 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:37:39.060 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:37:39.060 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:37:39.060 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:37:39.060 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:37:39.060 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:37:39.060 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:37:39.060 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:37:39.060 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:37:39.060 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:37:39.060 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:39.061 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:39.061 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:39.061 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:39.061 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:39.061 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:39.061 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:39.061 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:39.061 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:39.061 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:39.061 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:39.061 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:39.061 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:37:39.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:39.061 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:37:39.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:39.062 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:37:39.062 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:37:39.062 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:39.062 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:39.062 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:39.062 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:39.062 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:37:39.062 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:39.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:39.062 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:39.062 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:39.062 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:37:39.062 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:39.063 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:39.063 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:39.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:39.063 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:37:39.063 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:37:39.063 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:39.063 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:39.063 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:39.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:39.063 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:39.063 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:37:39.063 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:37:39.063 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:39.063 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:37:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:37:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:37:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:37:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:39.065 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:39.065 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:39.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:39.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:39.065 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:37:39.065 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:37:39.065 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:39.065 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:39.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:39.065 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:39.065 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:37:39.065 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:39.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:39.066 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:39.066 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:39.066 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:37:39.066 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:39.066 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:39.066 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:39.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:39.066 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:37:39.066 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:37:39.066 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:39.066 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:39.066 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:39.067 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:39.067 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:39.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:39.067 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:37:39.067 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:37:39.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:39.067 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:39.067 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:39.067 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:37:39.067 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:39.067 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:39.067 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:37:39.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:39.067 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:39.068 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:39.068 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:37:39.068 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:39.068 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:39.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:39.068 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:37:39.068 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:37:39.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.078+0000 7fe6757fa640 1 -- 172.21.15.112:0/2488548428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe65c006f30 msgr2=0x7fe65c0073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:39.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.078+0000 7fe6757fa640 1 --2- 172.21.15.112:0/2488548428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe65c006f30 0x7fe65c0073a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe66800eb20 tx=0x7fe668004ab0 comp rx=0 tx=0).stop 2024-04-25T01:37:39.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.078+0000 7fe6757fa640 1 -- 172.21.15.112:0/2488548428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe65c006f30 msgr2=0x7fe65c0073a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:37:39.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.078+0000 7fe6757fa640 1 --2- 172.21.15.112:0/2488548428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe65c006f30 0x7fe65c0073a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:39.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.078+0000 7fe67eb5a640 1 -- 172.21.15.112:0/2488548428 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe65806be60 msgr2=0x7fe65806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:39.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.078+0000 7fe67eb5a640 1 --2- 172.21.15.112:0/2488548428 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe65806be60 0x7fe65806e320 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe670034000 tx=0x7fe67002e000 comp rx=0 tx=0).stop 2024-04-25T01:37:39.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.078+0000 7fe67eb5a640 1 -- 172.21.15.112:0/2488548428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe67814f170 msgr2=0x7fe678141d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:39.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.078+0000 7fe67eb5a640 1 --2- 172.21.15.112:0/2488548428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe67814f170 0x7fe678141d80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe66000c3c0 tx=0x7fe66000c880 comp rx=0 tx=0).stop 2024-04-25T01:37:39.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.079+0000 7fe67d0f7640 1 -- 172.21.15.112:0/2488548428 reap_dead start 2024-04-25T01:37:39.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.079+0000 7fe67d0f7640 1 -- 172.21.15.112:0/2488548428 reap_dead start 2024-04-25T01:37:39.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.079+0000 7fe67eb5a640 1 -- 172.21.15.112:0/2488548428 shutdown_connections 2024-04-25T01:37:39.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.079+0000 7fe67eb5a640 1 -- 172.21.15.112:0/2488548428 >> 172.21.15.112:0/2488548428 conn(0x7fe6780c4260 msgr2=0x7fe67814a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:39.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.080+0000 7fe67eb5a640 1 -- 172.21.15.112:0/2488548428 shutdown_connections 2024-04-25T01:37:39.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.080+0000 7fe67eb5a640 1 -- 172.21.15.112:0/2488548428 wait complete. 2024-04-25T01:37:39.075 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:37:39.076 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-04-25T01:37:39.076 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-25T01:37:39.076 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-04-25T01:37:39.076 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-04-25T01:37:39.076 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T01:37:39.076 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:37:39.099 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T01:37:39.099 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T01:37:39.100 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:37:39.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.292+0000 7feafcf97640 1 Processor -- start 2024-04-25T01:37:39.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.292+0000 7feafcf97640 1 -- start start 2024-04-25T01:37:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.293+0000 7feafcf97640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf00dbcd0 0x7feaf0047490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.293+0000 7feafcf97640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feaf00df170 0x7feaf0006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.293+0000 7feafcf97640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaf000bb40 con 0x7feaf00073d0 2024-04-25T01:37:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.293+0000 7feafcf97640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7feaf000a410 con 0x7feaf00df170 2024-04-25T01:37:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.293+0000 7feafcf97640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feaf0048050 con 0x7feaf00dbcd0 2024-04-25T01:37:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.294+0000 7feaf7fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7feaf00073d0 0x7feaf000a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:46868/0 (socket says 172.21.15.112:46868) 2024-04-25T01:37:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.294+0000 7feaf7fff640 1 -- 172.21.15.112:0/2247336418 learned_addr learned my addr 172.21.15.112:0/2247336418 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:37:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.294+0000 7feaf77fe640 1 --2- 172.21.15.112:0/2247336418 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf00dbcd0 0x7feaf0047490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.295+0000 7feaf6ffd640 1 --2- 172.21.15.112:0/2247336418 >> v2:172.21.15.112:3300/0 conn(0x7feaf00df170 0x7feaf0006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:39.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.295+0000 7feaf67fc640 1 -- 172.21.15.112:0/2247336418 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1497621823 0 0) 0x7feaf000bb40 con 0x7feaf00073d0 2024-04-25T01:37:39.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.295+0000 7feaf67fc640 1 -- 172.21.15.112:0/2247336418 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feae8003520 con 0x7feaf00073d0 2024-04-25T01:37:39.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.295+0000 7feaf6ffd640 1 -- 172.21.15.112:0/2247336418 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf00dbcd0 msgr2=0x7feaf0047490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:37:39.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.295+0000 7feaf6ffd640 1 --2- 172.21.15.112:0/2247336418 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf00dbcd0 0x7feaf0047490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:39.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.295+0000 7feaf6ffd640 1 -- 172.21.15.112:0/2247336418 >> v1:172.21.15.112:6789/0 conn(0x7feaf00073d0 legacy=0x7feaf000a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:39.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.295+0000 7feaf6ffd640 1 -- 172.21.15.112:0/2247336418 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf000a0e0 con 0x7feaf00df170 2024-04-25T01:37:39.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.297+0000 7feaf6ffd640 1 --2- 172.21.15.112:0/2247336418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaf00df170 0x7feaf0006e90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feaf80699b0 tx=0x7feaf80a5e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910722cb92e091a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:39.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.297+0000 7feaf67fc640 1 -- 172.21.15.112:0/2247336418 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf80b3070 con 0x7feaf00df170 2024-04-25T01:37:39.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.297+0000 7feaf67fc640 1 -- 172.21.15.112:0/2247336418 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf80a6c10 con 0x7feaf00df170 2024-04-25T01:37:39.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.299+0000 7feafcf97640 1 -- 172.21.15.112:0/2247336418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaf00df170 msgr2=0x7feaf0006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:39.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.299+0000 7feafcf97640 1 --2- 172.21.15.112:0/2247336418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaf00df170 0x7feaf0006e90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feaf80699b0 tx=0x7feaf80a5e00 comp rx=0 tx=0).stop 2024-04-25T01:37:39.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.299+0000 7feafcf97640 1 -- 172.21.15.112:0/2247336418 shutdown_connections 2024-04-25T01:37:39.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.299+0000 7feafcf97640 1 --2- 172.21.15.112:0/2247336418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaf00df170 0x7feaf0006e90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:39.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.299+0000 7feafcf97640 1 --2- 172.21.15.112:0/2247336418 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf00dbcd0 0x7feaf0047490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:39.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.299+0000 7feafcf97640 1 -- 172.21.15.112:0/2247336418 >> 172.21.15.112:0/2247336418 conn(0x7feaf0048240 msgr2=0x7feaf0048650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:39.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.303+0000 7feafcf97640 1 -- 172.21.15.112:0/2247336418 shutdown_connections 2024-04-25T01:37:39.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.306+0000 7feafcf97640 1 -- 172.21.15.112:0/2247336418 wait complete. 2024-04-25T01:37:39.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.308+0000 7feafcf97640 1 Processor -- start 2024-04-25T01:37:39.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.308+0000 7feafcf97640 1 -- start start 2024-04-25T01:37:39.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.309+0000 7feafcf97640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf00073d0 0x7feaf0042890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:39.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.310+0000 7feafcf97640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaf00dbcd0 0x7feaf002ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:39.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.310+0000 7feafcf97640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaf00df170 0x7feaf0049cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:39.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.310+0000 7feafcf97640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feaf00dc0b0 con 0x7feaf00df170 2024-04-25T01:37:39.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.310+0000 7feafcf97640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feaf0047d70 con 0x7feaf00dbcd0 2024-04-25T01:37:39.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.310+0000 7feafcf97640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feaf00df5b0 con 0x7feaf00073d0 2024-04-25T01:37:39.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.310+0000 7feaf77fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf00073d0 0x7feaf0042890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:39.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.310+0000 7feaf77fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf00073d0 0x7feaf0042890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:45698/0 (socket says 172.21.15.112:45698) 2024-04-25T01:37:39.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.310+0000 7feaf77fe640 1 -- 172.21.15.112:0/1344212223 learned_addr learned my addr 172.21.15.112:0/1344212223 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:39.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.310+0000 7feaf7fff640 1 --2- 172.21.15.112:0/1344212223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaf00df170 0x7feaf0049cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:39.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.310+0000 7feaf6ffd640 1 -- 172.21.15.112:0/1344212223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaf00dbcd0 msgr2=0x7feaf002ec40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:37:39.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.310+0000 7feaf6ffd640 1 --2- 172.21.15.112:0/1344212223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaf00dbcd0 0x7feaf002ec40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:37:39.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.311+0000 7feaf7fff640 1 -- 172.21.15.112:0/1344212223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf00073d0 msgr2=0x7feaf0042890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:39.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.311+0000 7feaf7fff640 1 --2- 172.21.15.112:0/1344212223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf00073d0 0x7feaf0042890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:39.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.311+0000 7feaf7fff640 1 -- 172.21.15.112:0/1344212223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaf00dbcd0 msgr2=0x7feaf002ec40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:37:39.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.311+0000 7feaf7fff640 1 --2- 172.21.15.112:0/1344212223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaf00dbcd0 0x7feaf002ec40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.311+0000 7feaf7fff640 1 -- 172.21.15.112:0/1344212223 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf0049510 con 0x7feaf00df170 2024-04-25T01:37:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.311+0000 7feaf7fff640 1 --2- 172.21.15.112:0/1344212223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaf00df170 0x7feaf0049cf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feaec00c750 tx=0x7feaec00cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.311+0000 7feae3fff640 1 -- 172.21.15.112:0/1344212223 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec017070 con 0x7feaf00df170 2024-04-25T01:37:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.311+0000 7feafcf97640 1 -- 172.21.15.112:0/1344212223 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf0174bb0 con 0x7feaf00df170 2024-04-25T01:37:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.311+0000 7feafcf97640 1 -- 172.21.15.112:0/1344212223 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf00497d0 con 0x7feaf00df170 2024-04-25T01:37:39.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.313+0000 7feae1ffb640 1 -- 172.21.15.112:0/1344212223 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf0005380 con 0x7feaf00df170 2024-04-25T01:37:39.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.314+0000 7feae3fff640 1 -- 172.21.15.112:0/1344212223 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaec0058b0 con 0x7feaf00df170 2024-04-25T01:37:39.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.314+0000 7feae3fff640 1 -- 172.21.15.112:0/1344212223 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec007da0 con 0x7feaf00df170 2024-04-25T01:37:39.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.315+0000 7feae3fff640 1 -- 172.21.15.112:0/1344212223 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feaec01a070 con 0x7feaf00df170 2024-04-25T01:37:39.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.316+0000 7feae3fff640 1 --2- 172.21.15.112:0/1344212223 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feadc06bde0 0x7feadc06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:39.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.316+0000 7feae3fff640 1 -- 172.21.15.112:0/1344212223 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7feaec08a2a0 con 0x7feaf00df170 2024-04-25T01:37:39.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.316+0000 7feaf77fe640 1 --2- 172.21.15.112:0/1344212223 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feadc06bde0 0x7feadc06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:39.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.317+0000 7feaf77fe640 1 --2- 172.21.15.112:0/1344212223 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feadc06bde0 0x7feadc06e2a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7feae4006fd0 tx=0x7feae4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:39.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.322+0000 7feae3fff640 1 -- 172.21.15.112:0/1344212223 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feaec0582f0 con 0x7feaf00df170 2024-04-25T01:37:39.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.821+0000 7feae1ffb640 1 -- 172.21.15.112:0/1344212223 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7feaf000b580 con 0x7feaf00df170 2024-04-25T01:37:39.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.821+0000 7feae3fff640 1 -- 172.21.15.112:0/1344212223 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7feaec05c5f0 con 0x7feaf00df170 2024-04-25T01:37:39.801 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":94,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:37:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.830+0000 7feae1ffb640 1 -- 172.21.15.112:0/1344212223 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feadc06bde0 msgr2=0x7feadc06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.830+0000 7feae1ffb640 1 --2- 172.21.15.112:0/1344212223 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feadc06bde0 0x7feadc06e2a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7feae4006fd0 tx=0x7feae4008040 comp rx=0 tx=0).stop 2024-04-25T01:37:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.830+0000 7feae1ffb640 1 -- 172.21.15.112:0/1344212223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaf00df170 msgr2=0x7feaf0049cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.830+0000 7feae1ffb640 1 --2- 172.21.15.112:0/1344212223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaf00df170 0x7feaf0049cf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feaec00c750 tx=0x7feaec00cb50 comp rx=0 tx=0).stop 2024-04-25T01:37:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.830+0000 7feae1ffb640 1 -- 172.21.15.112:0/1344212223 shutdown_connections 2024-04-25T01:37:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.830+0000 7feae1ffb640 1 --2- 172.21.15.112:0/1344212223 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feadc06bde0 0x7feadc06e2a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.830+0000 7feae1ffb640 1 --2- 172.21.15.112:0/1344212223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaf00df170 0x7feaf0049cf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.830+0000 7feae1ffb640 1 --2- 172.21.15.112:0/1344212223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaf00dbcd0 0x7feaf002ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.830+0000 7feae1ffb640 1 --2- 172.21.15.112:0/1344212223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaf00073d0 0x7feaf0042890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.830+0000 7feae1ffb640 1 -- 172.21.15.112:0/1344212223 >> 172.21.15.112:0/1344212223 conn(0x7feaf0048240 msgr2=0x7feaf0048620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.831+0000 7feae1ffb640 1 -- 172.21.15.112:0/1344212223 shutdown_connections 2024-04-25T01:37:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:39.831+0000 7feae1ffb640 1 -- 172.21.15.112:0/1344212223 wait complete. 2024-04-25T01:37:42.832 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:37:42.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.016+0000 7fb9ee363640 1 Processor -- start 2024-04-25T01:37:42.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.016+0000 7fb9ee363640 1 -- start start 2024-04-25T01:37:42.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.017+0000 7fb9ee363640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb9e80c6560 0x7fb9e8156530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:42.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.017+0000 7fb9ee363640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9e80c3140 0x7fb9e8156e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:42.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.017+0000 7fb9ee363640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9e8155a90 con 0x7fb9e80c70e0 2024-04-25T01:37:42.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.017+0000 7fb9ee363640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fb9e8070420 con 0x7fb9e80c6560 2024-04-25T01:37:42.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.017+0000 7fb9ec900640 1 -- >> v1:172.21.15.187:6789/0 conn(0x7fb9e80c70e0 legacy=0x7fb9e80aac50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.187:6789/0 2024-04-25T01:37:42.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.017+0000 7fb9ee363640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb9e8155d30 con 0x7fb9e80c3140 2024-04-25T01:37:42.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.018+0000 7fb9e7fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb9e80c6560 0x7fb9e8156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:42.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.018+0000 7fb9e7fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb9e80c6560 0x7fb9e8156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41952/0 (socket says 172.21.15.112:41952) 2024-04-25T01:37:42.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.018+0000 7fb9e7fff640 1 -- 172.21.15.112:0/1528260010 learned_addr learned my addr 172.21.15.112:0/1528260010 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:42.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.018+0000 7fb9e77fe640 1 --2- 172.21.15.112:0/1528260010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9e80c3140 0x7fb9e8156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:42.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.019+0000 7fb9e7fff640 1 -- 172.21.15.112:0/1528260010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9e80c3140 msgr2=0x7fb9e8156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:37:42.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.019+0000 7fb9e7fff640 1 --2- 172.21.15.112:0/1528260010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9e80c3140 0x7fb9e8156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:42.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.019+0000 7fb9e7fff640 1 -- 172.21.15.112:0/1528260010 >> v1:172.21.15.187:6789/0 conn(0x7fb9e80c70e0 legacy=0x7fb9e80aac50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:37:42.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.019+0000 7fb9e7fff640 1 -- 172.21.15.112:0/1528260010 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e8070700 con 0x7fb9e80c6560 2024-04-25T01:37:42.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.020+0000 7fb9e7fff640 1 --2- 172.21.15.112:0/1528260010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9e80c6560 0x7fb9e8156530 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8009870 tx=0x7fb9d802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c52960a72a71d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:42.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.021+0000 7fb9e6ffd640 1 -- 172.21.15.112:0/1528260010 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8037070 con 0x7fb9e80c6560 2024-04-25T01:37:42.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.021+0000 7fb9e6ffd640 1 -- 172.21.15.112:0/1528260010 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9d802ae20 con 0x7fb9e80c6560 2024-04-25T01:37:42.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.021+0000 7fb9e6ffd640 1 -- 172.21.15.112:0/1528260010 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8037070 con 0x7fb9e80c6560 2024-04-25T01:37:42.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.022+0000 7fb9ee363640 1 -- 172.21.15.112:0/1528260010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9e80c6560 msgr2=0x7fb9e8156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:43.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.022+0000 7fb9ee363640 1 --2- 172.21.15.112:0/1528260010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9e80c6560 0x7fb9e8156530 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8009870 tx=0x7fb9d802a110 comp rx=0 tx=0).stop 2024-04-25T01:37:43.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.024+0000 7fb9ee363640 1 -- 172.21.15.112:0/1528260010 shutdown_connections 2024-04-25T01:37:43.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.025+0000 7fb9ee363640 1 --2- 172.21.15.112:0/1528260010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9e80c3140 0x7fb9e8156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.025+0000 7fb9ee363640 1 --2- 172.21.15.112:0/1528260010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9e80c6560 0x7fb9e8156530 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.025+0000 7fb9ee363640 1 -- 172.21.15.112:0/1528260010 >> 172.21.15.112:0/1528260010 conn(0x7fb9e80743d0 msgr2=0x7fb9e80747b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:43.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.025+0000 7fb9ee363640 1 -- 172.21.15.112:0/1528260010 shutdown_connections 2024-04-25T01:37:43.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.025+0000 7fb9ee363640 1 -- 172.21.15.112:0/1528260010 wait complete. 2024-04-25T01:37:43.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.027+0000 7fb9ee363640 1 Processor -- start 2024-04-25T01:37:43.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.027+0000 7fb9ee363640 1 -- start start 2024-04-25T01:37:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.029+0000 7fb9ee363640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9e80c3140 0x7fb9e81efa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.029+0000 7fb9ee363640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9e80c6560 0x7fb9e81594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.029+0000 7fb9ee363640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9e80c70e0 0x7fb9e8152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.029+0000 7fb9ee363640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e80ab4e0 con 0x7fb9e80c6560 2024-04-25T01:37:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.029+0000 7fb9ee363640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e80ab360 con 0x7fb9e80c70e0 2024-04-25T01:37:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.029+0000 7fb9ee363640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb9e81573b0 con 0x7fb9e80c3140 2024-04-25T01:37:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.029+0000 7fb9ec900640 1 -- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9e80c70e0 msgr2=0x7fb9e8152270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:37:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.029+0000 7fb9ec900640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9e80c70e0 0x7fb9e8152270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:37:43.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.029+0000 7fb9e7fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9e80c3140 0x7fb9e81efa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:43.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.029+0000 7fb9e7fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9e80c3140 0x7fb9e81efa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:45718/0 (socket says 172.21.15.112:45718) 2024-04-25T01:37:43.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.029+0000 7fb9e7fff640 1 -- 172.21.15.112:0/3072975053 learned_addr learned my addr 172.21.15.112:0/3072975053 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:43.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.029+0000 7fb9e77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9e80c6560 0x7fb9e81594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:43.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.030+0000 7fb9e77fe640 1 -- 172.21.15.112:0/3072975053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9e80c3140 msgr2=0x7fb9e81efa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:43.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.030+0000 7fb9e77fe640 1 --2- 172.21.15.112:0/3072975053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9e80c3140 0x7fb9e81efa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.030+0000 7fb9e77fe640 1 -- 172.21.15.112:0/3072975053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9e80c70e0 msgr2=0x7fb9e8152270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:37:43.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.030+0000 7fb9e77fe640 1 --2- 172.21.15.112:0/3072975053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9e80c70e0 0x7fb9e8152270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.030+0000 7fb9e77fe640 1 -- 172.21.15.112:0/3072975053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e81f0060 con 0x7fb9e80c6560 2024-04-25T01:37:43.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.030+0000 7fb9e7fff640 1 --2- 172.21.15.112:0/3072975053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9e80c3140 0x7fb9e81efa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:37:43.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.031+0000 7fb9e77fe640 1 --2- 172.21.15.112:0/3072975053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9e80c6560 0x7fb9e81594d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00d550 tx=0x7fb9dc00da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:43.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.031+0000 7fb9e4ff9640 1 -- 172.21.15.112:0/3072975053 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc0125f0 con 0x7fb9e80c6560 2024-04-25T01:37:43.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.031+0000 7fb9ee363640 1 -- 172.21.15.112:0/3072975053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e81f0340 con 0x7fb9e80c6560 2024-04-25T01:37:43.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.031+0000 7fb9ee363640 1 -- 172.21.15.112:0/3072975053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e81f0900 con 0x7fb9e80c6560 2024-04-25T01:37:43.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.032+0000 7fb9e4ff9640 1 -- 172.21.15.112:0/3072975053 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9dc019070 con 0x7fb9e80c6560 2024-04-25T01:37:43.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.032+0000 7fb9e4ff9640 1 -- 172.21.15.112:0/3072975053 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc0129b0 con 0x7fb9e80c6560 2024-04-25T01:37:43.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.034+0000 7fb9ee363640 1 -- 172.21.15.112:0/3072975053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c8005290 con 0x7fb9e80c6560 2024-04-25T01:37:43.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.035+0000 7fb9e4ff9640 1 -- 172.21.15.112:0/3072975053 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb9dc012b50 con 0x7fb9e80c6560 2024-04-25T01:37:43.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.036+0000 7fb9e4ff9640 1 --2- 172.21.15.112:0/3072975053 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb9d406be60 0x7fb9d406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:43.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.036+0000 7fb9e7fff640 1 --2- 172.21.15.112:0/3072975053 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb9d406be60 0x7fb9d406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:43.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.037+0000 7fb9e7fff640 1 --2- 172.21.15.112:0/3072975053 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb9d406be60 0x7fb9d406e320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80090b0 tx=0x7fb9d802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:43.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.037+0000 7fb9e4ff9640 1 -- 172.21.15.112:0/3072975053 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fb9dc08b250 con 0x7fb9e80c6560 2024-04-25T01:37:43.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.044+0000 7fb9e4ff9640 1 -- 172.21.15.112:0/3072975053 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb9dc0599d0 con 0x7fb9e80c6560 2024-04-25T01:37:43.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.531+0000 7fb9ee363640 1 -- 172.21.15.112:0/3072975053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb9c8005850 con 0x7fb9e80c6560 2024-04-25T01:37:43.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.533+0000 7fb9e4ff9640 1 -- 172.21.15.112:0/3072975053 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7fb9dc0592d0 con 0x7fb9e80c6560 2024-04-25T01:37:43.512 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":94,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:37:43.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.537+0000 7fb9ee363640 1 -- 172.21.15.112:0/3072975053 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb9d406be60 msgr2=0x7fb9d406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:43.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.537+0000 7fb9ee363640 1 --2- 172.21.15.112:0/3072975053 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb9d406be60 0x7fb9d406e320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80090b0 tx=0x7fb9d802e000 comp rx=0 tx=0).stop 2024-04-25T01:37:43.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.537+0000 7fb9ee363640 1 -- 172.21.15.112:0/3072975053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9e80c6560 msgr2=0x7fb9e81594d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:43.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.537+0000 7fb9ee363640 1 --2- 172.21.15.112:0/3072975053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9e80c6560 0x7fb9e81594d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00d550 tx=0x7fb9dc00da10 comp rx=0 tx=0).stop 2024-04-25T01:37:43.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.538+0000 7fb9ee363640 1 -- 172.21.15.112:0/3072975053 shutdown_connections 2024-04-25T01:37:43.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.538+0000 7fb9ee363640 1 --2- 172.21.15.112:0/3072975053 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb9d406be60 0x7fb9d406e320 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.538+0000 7fb9ee363640 1 --2- 172.21.15.112:0/3072975053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9e80c70e0 0x7fb9e8152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.538+0000 7fb9ee363640 1 --2- 172.21.15.112:0/3072975053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9e80c6560 0x7fb9e81594d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.538+0000 7fb9ee363640 1 --2- 172.21.15.112:0/3072975053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9e80c3140 0x7fb9e81efa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.538+0000 7fb9ee363640 1 -- 172.21.15.112:0/3072975053 >> 172.21.15.112:0/3072975053 conn(0x7fb9e80743d0 msgr2=0x7fb9e806c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:43.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.538+0000 7fb9ee363640 1 -- 172.21.15.112:0/3072975053 shutdown_connections 2024-04-25T01:37:43.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:43.538+0000 7fb9ee363640 1 -- 172.21.15.112:0/3072975053 wait complete. 2024-04-25T01:37:46.537 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:37:46.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.723+0000 7f411064d640 1 Processor -- start 2024-04-25T01:37:46.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.723+0000 7f411064d640 1 -- start start 2024-04-25T01:37:46.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.724+0000 7f411064d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4108152ef0 0x7f41080ca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:46.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.724+0000 7f411064d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f41080c9570 0x7f41080c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:46.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.724+0000 7f411064d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41081532d0 con 0x7f41080c9060 2024-04-25T01:37:46.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.724+0000 7f411064d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f4108155a70 con 0x7f4108152ef0 2024-04-25T01:37:46.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.724+0000 7f411064d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f41080c9ba0 con 0x7f41080c9570 2024-04-25T01:37:46.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.724+0000 7f410ebea640 1 -- >> v1:172.21.15.187:6789/0 conn(0x7f41080c9060 legacy=0x7f4108158e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.187:6789/0 2024-04-25T01:37:46.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.724+0000 7f410dbe8640 1 -- >> v2:172.21.15.187:3300/0 conn(0x7f41080c9570 msgr2=0x7f41080c5ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:37:46.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.724+0000 7f410dbe8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f41080c9570 0x7f41080c5ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:37:46.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.725+0000 7f410e3e9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4108152ef0 0x7f41080ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:46.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.725+0000 7f410e3e9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4108152ef0 0x7f41080ca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41972/0 (socket says 172.21.15.112:41972) 2024-04-25T01:37:46.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.725+0000 7f410e3e9640 1 -- 172.21.15.112:0/4081438061 learned_addr learned my addr 172.21.15.112:0/4081438061 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:46.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.726+0000 7f410e3e9640 1 -- 172.21.15.112:0/4081438061 >> v2:172.21.15.187:3300/0 conn(0x7f41080c9570 msgr2=0x7f41080c5ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:37:46.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.726+0000 7f410e3e9640 1 --2- 172.21.15.112:0/4081438061 >> v2:172.21.15.187:3300/0 conn(0x7f41080c9570 0x7f41080c5ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:46.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.726+0000 7f410e3e9640 1 -- 172.21.15.112:0/4081438061 >> v1:172.21.15.187:6789/0 conn(0x7f41080c9060 legacy=0x7f4108158e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:37:46.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.726+0000 7f410e3e9640 1 -- 172.21.15.112:0/4081438061 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41080c9e80 con 0x7f4108152ef0 2024-04-25T01:37:46.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.727+0000 7f410e3e9640 1 --2- 172.21.15.112:0/4081438061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4108152ef0 0x7f41080ca680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f41080aabd0 tx=0x7f40fc02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c34758c1f5255b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:46.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.728+0000 7f410d3e7640 1 -- 172.21.15.112:0/4081438061 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc037070 con 0x7f4108152ef0 2024-04-25T01:37:46.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.728+0000 7f410d3e7640 1 -- 172.21.15.112:0/4081438061 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40fc032440 con 0x7f4108152ef0 2024-04-25T01:37:46.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.728+0000 7f410d3e7640 1 -- 172.21.15.112:0/4081438061 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc032810 con 0x7f4108152ef0 2024-04-25T01:37:46.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.729+0000 7f411064d640 1 -- 172.21.15.112:0/4081438061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4108152ef0 msgr2=0x7f41080ca680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:46.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.729+0000 7f411064d640 1 --2- 172.21.15.112:0/4081438061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4108152ef0 0x7f41080ca680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f41080aabd0 tx=0x7f40fc02a250 comp rx=0 tx=0).stop 2024-04-25T01:37:46.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.730+0000 7f411064d640 1 -- 172.21.15.112:0/4081438061 shutdown_connections 2024-04-25T01:37:46.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.730+0000 7f411064d640 1 --2- 172.21.15.112:0/4081438061 >> v2:172.21.15.187:3300/0 conn(0x7f41080c9570 0x7f41080c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:46.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.730+0000 7f411064d640 1 --2- 172.21.15.112:0/4081438061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4108152ef0 0x7f41080ca680 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:46.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.730+0000 7f411064d640 1 -- 172.21.15.112:0/4081438061 >> 172.21.15.112:0/4081438061 conn(0x7f41080cb160 msgr2=0x7f41080cb570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:46.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.730+0000 7f411064d640 1 -- 172.21.15.112:0/4081438061 shutdown_connections 2024-04-25T01:37:46.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.730+0000 7f411064d640 1 -- 172.21.15.112:0/4081438061 wait complete. 2024-04-25T01:37:46.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.732+0000 7f411064d640 1 Processor -- start 2024-04-25T01:37:46.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.732+0000 7f411064d640 1 -- start start 2024-04-25T01:37:46.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.733+0000 7f411064d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41080c9060 0x7f4108157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:46.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.734+0000 7f411064d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41080c9570 0x7f41080c04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:46.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.734+0000 7f411064d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4108152ef0 0x7f41081ea5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:46.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.734+0000 7f411064d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41080cad40 con 0x7f41080c9570 2024-04-25T01:37:46.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.734+0000 7f411064d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41080cabc0 con 0x7f4108152ef0 2024-04-25T01:37:46.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.734+0000 7f411064d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f41080caec0 con 0x7f41080c9060 2024-04-25T01:37:46.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.734+0000 7f410ebea640 1 -- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4108152ef0 msgr2=0x7f41081ea5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:37:46.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.734+0000 7f410e3e9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41080c9060 0x7f4108157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:46.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.734+0000 7f410ebea640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4108152ef0 0x7f41081ea5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:37:46.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.734+0000 7f410dbe8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41080c9570 0x7f41080c04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:46.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.734+0000 7f410dbe8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41080c9570 0x7f41080c04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41978/0 (socket says 172.21.15.112:41978) 2024-04-25T01:37:46.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.734+0000 7f410dbe8640 1 -- 172.21.15.112:0/1646898442 learned_addr learned my addr 172.21.15.112:0/1646898442 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:46.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.735+0000 7f410dbe8640 1 -- 172.21.15.112:0/1646898442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41080c9060 msgr2=0x7f4108157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:46.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.735+0000 7f410dbe8640 1 --2- 172.21.15.112:0/1646898442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41080c9060 0x7f4108157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:46.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.735+0000 7f410dbe8640 1 -- 172.21.15.112:0/1646898442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4108152ef0 msgr2=0x7f41081ea5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:37:46.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.735+0000 7f410dbe8640 1 --2- 172.21.15.112:0/1646898442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4108152ef0 0x7f41081ea5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:46.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.735+0000 7f410dbe8640 1 -- 172.21.15.112:0/1646898442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41081eb2a0 con 0x7f41080c9570 2024-04-25T01:37:46.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.735+0000 7f410e3e9640 1 --2- 172.21.15.112:0/1646898442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41080c9060 0x7f4108157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:37:46.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.735+0000 7f410dbe8640 1 --2- 172.21.15.112:0/1646898442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41080c9570 0x7f41080c04a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f40f800b710 tx=0x7f40f800bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:46.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.735+0000 7f4106ffd640 1 -- 172.21.15.112:0/1646898442 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8003ae0 con 0x7f41080c9570 2024-04-25T01:37:46.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.736+0000 7f4106ffd640 1 -- 172.21.15.112:0/1646898442 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f8010070 con 0x7f41080c9570 2024-04-25T01:37:46.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.736+0000 7f411064d640 1 -- 172.21.15.112:0/1646898442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4108146790 con 0x7f41080c9570 2024-04-25T01:37:46.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.736+0000 7f411064d640 1 -- 172.21.15.112:0/1646898442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41081eaec0 con 0x7f41080c9570 2024-04-25T01:37:46.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.736+0000 7f4106ffd640 1 -- 172.21.15.112:0/1646898442 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8005070 con 0x7f41080c9570 2024-04-25T01:37:46.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.737+0000 7f411064d640 1 -- 172.21.15.112:0/1646898442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41080689b0 con 0x7f41080c9570 2024-04-25T01:37:46.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.739+0000 7f4106ffd640 1 -- 172.21.15.112:0/1646898442 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f40f8005210 con 0x7f41080c9570 2024-04-25T01:37:46.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.740+0000 7f4106ffd640 1 --2- 172.21.15.112:0/1646898442 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f40f406bde0 0x7f40f406e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:46.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.740+0000 7f4106ffd640 1 -- 172.21.15.112:0/1646898442 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f40f808a9b0 con 0x7f41080c9570 2024-04-25T01:37:46.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.747+0000 7f410e3e9640 1 --2- 172.21.15.112:0/1646898442 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f40f406bde0 0x7f40f406e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:46.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.747+0000 7f410e3e9640 1 --2- 172.21.15.112:0/1646898442 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f40f406bde0 0x7f40f406e2a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f40fc034000 tx=0x7f40fc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:46.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:46.751+0000 7f4106ffd640 1 -- 172.21.15.112:0/1646898442 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f40f8058f30 con 0x7f41080c9570 2024-04-25T01:37:47.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:47.243+0000 7f411064d640 1 -- 172.21.15.112:0/1646898442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f41080cbd80 con 0x7f41080c9570 2024-04-25T01:37:47.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:47.244+0000 7f4106ffd640 1 -- 172.21.15.112:0/1646898442 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f40f8058830 con 0x7f41080c9570 2024-04-25T01:37:47.222 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":94,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:37:47.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:47.248+0000 7f411064d640 1 -- 172.21.15.112:0/1646898442 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f40f406bde0 msgr2=0x7f40f406e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:47.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:47.248+0000 7f411064d640 1 --2- 172.21.15.112:0/1646898442 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f40f406bde0 0x7f40f406e2a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f40fc034000 tx=0x7f40fc02e000 comp rx=0 tx=0).stop 2024-04-25T01:37:47.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:47.248+0000 7f411064d640 1 -- 172.21.15.112:0/1646898442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41080c9570 msgr2=0x7f41080c04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:47.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:47.248+0000 7f411064d640 1 --2- 172.21.15.112:0/1646898442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41080c9570 0x7f41080c04a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f40f800b710 tx=0x7f40f800bbd0 comp rx=0 tx=0).stop 2024-04-25T01:37:47.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:47.249+0000 7f411064d640 1 -- 172.21.15.112:0/1646898442 shutdown_connections 2024-04-25T01:37:47.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:47.249+0000 7f411064d640 1 --2- 172.21.15.112:0/1646898442 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f40f406bde0 0x7f40f406e2a0 secure :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f40fc034000 tx=0x7f40fc02e000 comp rx=0 tx=0).stop 2024-04-25T01:37:47.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:47.249+0000 7f411064d640 1 --2- 172.21.15.112:0/1646898442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4108152ef0 0x7f41081ea5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:47.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:47.249+0000 7f411064d640 1 --2- 172.21.15.112:0/1646898442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41080c9570 0x7f41080c04a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:47.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:47.249+0000 7f411064d640 1 --2- 172.21.15.112:0/1646898442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41080c9060 0x7f4108157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:47.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:47.249+0000 7f411064d640 1 -- 172.21.15.112:0/1646898442 >> 172.21.15.112:0/1646898442 conn(0x7f41080cb160 msgr2=0x7f4108151cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:47.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:47.249+0000 7f411064d640 1 -- 172.21.15.112:0/1646898442 shutdown_connections 2024-04-25T01:37:47.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:47.249+0000 7f411064d640 1 -- 172.21.15.112:0/1646898442 wait complete. 2024-04-25T01:37:50.248 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:37:50.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.434+0000 7fad351f6640 1 Processor -- start 2024-04-25T01:37:50.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.434+0000 7fad351f6640 1 -- start start 2024-04-25T01:37:50.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.436+0000 7fad351f6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fad280d7ae0 0x7fad280071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:50.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.436+0000 7fad351f6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 0x7fad2804e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:50.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.436+0000 7fad351f6640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad2800f9d0 con 0x7fad2804dae0 2024-04-25T01:37:50.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.436+0000 7fad351f6640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fad280da4b0 con 0x7fad2804dff0 2024-04-25T01:37:50.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.436+0000 7fad351f6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fad280da790 con 0x7fad280d7ae0 2024-04-25T01:37:50.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.436+0000 7fad2ffff640 1 -- >> v2:172.21.15.187:3300/0 conn(0x7fad280d7ae0 msgr2=0x7fad280071b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:37:50.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.436+0000 7fad2ffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fad280d7ae0 0x7fad280071b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:37:50.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.436+0000 7fad2f7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 0x7fad2804e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:50.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.436+0000 7fad349f5640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fad2804dae0 0x7fad2804ed10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57920/0 (socket says 172.21.15.112:57920) 2024-04-25T01:37:50.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.436+0000 7fad2f7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 0x7fad2804e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49350/0 (socket says 172.21.15.112:49350) 2024-04-25T01:37:50.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.436+0000 7fad349f5640 1 -- 172.21.15.112:0/2816119593 learned_addr learned my addr 172.21.15.112:0/2816119593 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:37:50.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.436+0000 7fad2f7fe640 1 -- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 msgr2=0x7fad2804e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:37:50.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.436+0000 7fad2f7fe640 1 -- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 msgr2=0x7fad2804e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:37:50.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.436+0000 7fad2f7fe640 1 --2- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 0x7fad2804e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:37:50.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.436+0000 7fad2f7fe640 1 --2- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 0x7fad2804e7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:37:50.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.637+0000 7fad2ffff640 1 -- 172.21.15.112:0/2816119593 >> v2:172.21.15.187:3300/0 conn(0x7fad280d7ae0 msgr2=0x7fad280071b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:37:50.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.637+0000 7fad2ffff640 1 --2- 172.21.15.112:0/2816119593 >> v2:172.21.15.187:3300/0 conn(0x7fad280d7ae0 0x7fad280071b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:37:50.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.637+0000 7fad2f7fe640 1 --2- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 0x7fad2804e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:50.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.637+0000 7fad2f7fe640 1 -- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 msgr2=0x7fad2804e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:37:50.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.637+0000 7fad2f7fe640 1 -- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 msgr2=0x7fad2804e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:37:50.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.637+0000 7fad2f7fe640 1 --2- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 0x7fad2804e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:37:50.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:50.637+0000 7fad2f7fe640 1 --2- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 0x7fad2804e7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:37:51.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:51.038+0000 7fad2ffff640 1 -- 172.21.15.112:0/2816119593 >> v2:172.21.15.187:3300/0 conn(0x7fad280d7ae0 msgr2=0x7fad280071b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:37:51.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:51.038+0000 7fad2ffff640 1 --2- 172.21.15.112:0/2816119593 >> v2:172.21.15.187:3300/0 conn(0x7fad280d7ae0 0x7fad280071b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:37:51.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:51.039+0000 7fad2f7fe640 1 --2- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 0x7fad2804e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:51.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:51.039+0000 7fad2f7fe640 1 -- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 msgr2=0x7fad2804e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:37:51.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:51.039+0000 7fad2f7fe640 1 -- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 msgr2=0x7fad2804e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:37:51.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:51.039+0000 7fad2f7fe640 1 --2- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 0x7fad2804e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:37:51.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:51.039+0000 7fad2f7fe640 1 --2- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 0x7fad2804e7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:37:51.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:51.839+0000 7fad2ffff640 1 -- 172.21.15.112:0/2816119593 >> v2:172.21.15.187:3300/0 conn(0x7fad280d7ae0 msgr2=0x7fad280071b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:37:51.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:51.839+0000 7fad2ffff640 1 --2- 172.21.15.112:0/2816119593 >> v2:172.21.15.187:3300/0 conn(0x7fad280d7ae0 0x7fad280071b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:37:51.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:51.840+0000 7fad2f7fe640 1 --2- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 0x7fad2804e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:51.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:51.840+0000 7fad2f7fe640 1 -- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 msgr2=0x7fad2804e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:37:51.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:51.840+0000 7fad2f7fe640 1 -- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 msgr2=0x7fad2804e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:37:51.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:51.840+0000 7fad2f7fe640 1 --2- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 0x7fad2804e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:37:51.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:51.840+0000 7fad2f7fe640 1 --2- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 0x7fad2804e7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:37:53.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:53.436+0000 7fad2dffb640 1 -- 172.21.15.112:0/2816119593 >> v2:172.21.15.187:3300/0 conn(0x7fad280d7ae0 msgr2=0x7fad280071b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:37:53.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:53.436+0000 7fad2dffb640 1 --2- 172.21.15.112:0/2816119593 >> v2:172.21.15.187:3300/0 conn(0x7fad280d7ae0 0x7fad280071b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:53.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:53.436+0000 7fad2dffb640 1 -- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 msgr2=0x7fad2804e7d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:37:53.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:53.436+0000 7fad2dffb640 1 --2- 172.21.15.112:0/2816119593 >> v2:172.21.15.112:3300/0 conn(0x7fad2804dff0 0x7fad2804e7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:53.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:53.436+0000 7fad2dffb640 1 -- 172.21.15.112:0/2816119593 >> v1:172.21.15.112:6789/0 conn(0x7fad2804dae0 legacy=0x7fad2804ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:53.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:53.436+0000 7fad2dffb640 1 --2- 172.21.15.112:0/2816119593 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad14003690 0x7fad14005ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:53.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:53.436+0000 7fad2dffb640 1 -- 172.21.15.112:0/2816119593 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad280daa70 con 0x7fad14006150 2024-04-25T01:37:53.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:53.436+0000 7fad2f7fe640 1 -- 172.21.15.112:0/2816119593 >> v1:172.21.15.187:6789/0 conn(0x7fad14006150 legacy=0x7fad14008610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.187:6789/0 2024-04-25T01:37:53.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:53.436+0000 7fad2dffb640 1 -- 172.21.15.112:0/2816119593 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fad280da4b0 con 0x7fad14003690 2024-04-25T01:37:53.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:53.436+0000 7fad2ffff640 1 --2- 172.21.15.112:0/2816119593 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad14003690 0x7fad14005ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:53.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:53.437+0000 7fad2ffff640 1 -- 172.21.15.112:0/2816119593 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad14003690 msgr2=0x7fad14005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:37:53.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:53.437+0000 7fad2ffff640 1 -- 172.21.15.112:0/2816119593 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad14003690 msgr2=0x7fad14005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:37:53.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:53.437+0000 7fad2ffff640 1 --2- 172.21.15.112:0/2816119593 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad14003690 0x7fad14005ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:37:53.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:53.437+0000 7fad2ffff640 1 --2- 172.21.15.112:0/2816119593 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad14003690 0x7fad14005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:37:53.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:53.638+0000 7fad2f7fe640 1 -- 172.21.15.112:0/2816119593 >> v1:172.21.15.187:6789/0 conn(0x7fad14006150 legacy=0x7fad14008610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.187:6789/0 2024-04-25T01:37:53.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:53.638+0000 7fad2ffff640 1 --2- 172.21.15.112:0/2816119593 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad14003690 0x7fad14005ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:53.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:53.638+0000 7fad2ffff640 1 -- 172.21.15.112:0/2816119593 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad14003690 msgr2=0x7fad14005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:37:53.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:53.638+0000 7fad2ffff640 1 -- 172.21.15.112:0/2816119593 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad14003690 msgr2=0x7fad14005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:37:53.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:53.638+0000 7fad2ffff640 1 --2- 172.21.15.112:0/2816119593 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad14003690 0x7fad14005ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:37:53.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:53.638+0000 7fad2ffff640 1 --2- 172.21.15.112:0/2816119593 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad14003690 0x7fad14005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:37:54.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.039+0000 7fad2f7fe640 1 -- 172.21.15.112:0/2816119593 >> v1:172.21.15.187:6789/0 conn(0x7fad14006150 legacy=0x7fad14008610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.187:6789/0 2024-04-25T01:37:54.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.039+0000 7fad2ffff640 1 --2- 172.21.15.112:0/2816119593 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad14003690 0x7fad14005ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:54.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.040+0000 7fad2ffff640 1 -- 172.21.15.112:0/2816119593 >> v1:172.21.15.187:6789/0 conn(0x7fad14006150 legacy=0x7fad14008610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:37:54.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.040+0000 7fad2ffff640 1 -- 172.21.15.112:0/2816119593 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1400afb0 con 0x7fad14003690 2024-04-25T01:37:54.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.041+0000 7fad2ffff640 1 --2- 172.21.15.112:0/2816119593 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad14003690 0x7fad14005ac0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fad24009dd0 tx=0x7fad2402b030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db942e35cc7e6f55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:54.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.041+0000 7fad2effd640 1 -- 172.21.15.112:0/2816119593 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24031070 con 0x7fad14003690 2024-04-25T01:37:54.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.041+0000 7fad2effd640 1 -- 172.21.15.112:0/2816119593 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad2402bd40 con 0x7fad14003690 2024-04-25T01:37:54.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.042+0000 7fad351f6640 1 -- 172.21.15.112:0/2816119593 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad14003690 msgr2=0x7fad14005ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:54.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.042+0000 7fad2effd640 1 -- 172.21.15.112:0/2816119593 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24031070 con 0x7fad14003690 2024-04-25T01:37:54.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.042+0000 7fad351f6640 1 --2- 172.21.15.112:0/2816119593 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad14003690 0x7fad14005ac0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fad24009dd0 tx=0x7fad2402b030 comp rx=0 tx=0).stop 2024-04-25T01:37:54.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.043+0000 7fad349f5640 1 -- 172.21.15.112:0/2816119593 reap_dead start 2024-04-25T01:37:54.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.043+0000 7fad351f6640 1 -- 172.21.15.112:0/2816119593 shutdown_connections 2024-04-25T01:37:54.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.043+0000 7fad351f6640 1 -- 172.21.15.112:0/2816119593 >> 172.21.15.112:0/2816119593 conn(0x7fad2804c660 msgr2=0x7fad2804ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:54.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.043+0000 7fad351f6640 1 -- 172.21.15.112:0/2816119593 shutdown_connections 2024-04-25T01:37:54.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.043+0000 7fad351f6640 1 -- 172.21.15.112:0/2816119593 wait complete. 2024-04-25T01:37:54.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.045+0000 7fad351f6640 1 Processor -- start 2024-04-25T01:37:54.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.045+0000 7fad351f6640 1 -- start start 2024-04-25T01:37:54.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.047+0000 7fad351f6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad280cd980 0x7fad280ce710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.047+0000 7fad351f6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad2816dfc0 0x7fad280d8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.047+0000 7fad351f6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad280cdd60 0x7fad28173550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.047+0000 7fad2f7fe640 1 -- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad2816dfc0 msgr2=0x7fad280d8f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:37:54.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.047+0000 7fad2f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad2816dfc0 0x7fad280d8f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:37:54.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.047+0000 7fad351f6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fad2804e4b0 con 0x7fad280cdd60 2024-04-25T01:37:54.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.047+0000 7fad351f6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fad280db270 con 0x7fad2816dfc0 2024-04-25T01:37:54.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.048+0000 7fad349f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad280cdd60 0x7fad28173550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:54.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.048+0000 7fad351f6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fad2800fdc0 con 0x7fad280cd980 2024-04-25T01:37:54.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.048+0000 7fad349f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad280cdd60 0x7fad28173550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49386/0 (socket says 172.21.15.112:49386) 2024-04-25T01:37:54.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.048+0000 7fad349f5640 1 -- 172.21.15.112:0/1782489446 learned_addr learned my addr 172.21.15.112:0/1782489446 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:54.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.048+0000 7fad2ffff640 1 --2- 172.21.15.112:0/1782489446 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad280cd980 0x7fad280ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:54.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.048+0000 7fad349f5640 1 -- 172.21.15.112:0/1782489446 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad280cd980 msgr2=0x7fad280ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:54.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.048+0000 7fad349f5640 1 --2- 172.21.15.112:0/1782489446 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad280cd980 0x7fad280ce710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.048+0000 7fad349f5640 1 -- 172.21.15.112:0/1782489446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad2816dfc0 msgr2=0x7fad280d8f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:37:54.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.048+0000 7fad349f5640 1 --2- 172.21.15.112:0/1782489446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad2816dfc0 0x7fad280d8f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.048+0000 7fad349f5640 1 -- 172.21.15.112:0/1782489446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad28174450 con 0x7fad280cdd60 2024-04-25T01:37:54.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.048+0000 7fad349f5640 1 --2- 172.21.15.112:0/1782489446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad280cdd60 0x7fad28173550 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fad30075bc0 tx=0x7fad300736d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:54.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.049+0000 7fad2cff9640 1 -- 172.21.15.112:0/1782489446 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30068d20 con 0x7fad280cdd60 2024-04-25T01:37:54.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.049+0000 7fad2cff9640 1 -- 172.21.15.112:0/1782489446 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad3008f030 con 0x7fad280cdd60 2024-04-25T01:37:54.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.049+0000 7fad351f6640 1 -- 172.21.15.112:0/1782489446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad280d8a40 con 0x7fad280cdd60 2024-04-25T01:37:54.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.049+0000 7fad2cff9640 1 -- 172.21.15.112:0/1782489446 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad300727e0 con 0x7fad280cdd60 2024-04-25T01:37:54.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.049+0000 7fad351f6640 1 -- 172.21.15.112:0/1782489446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad280ce430 con 0x7fad280cdd60 2024-04-25T01:37:54.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.050+0000 7fad351f6640 1 -- 172.21.15.112:0/1782489446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad280057e0 con 0x7fad280cdd60 2024-04-25T01:37:54.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.051+0000 7fad2cff9640 1 -- 172.21.15.112:0/1782489446 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad30072980 con 0x7fad280cdd60 2024-04-25T01:37:54.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.052+0000 7fad2cff9640 1 --2- 172.21.15.112:0/1782489446 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fad1406cb50 0x7fad1406f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.052+0000 7fad2cff9640 1 -- 172.21.15.112:0/1782489446 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fad301068e0 con 0x7fad280cdd60 2024-04-25T01:37:54.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.052+0000 7fad2ffff640 1 --2- 172.21.15.112:0/1782489446 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fad1406cb50 0x7fad1406f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:54.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.053+0000 7fad2ffff640 1 --2- 172.21.15.112:0/1782489446 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fad1406cb50 0x7fad1406f010 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fad24035000 tx=0x7fad24031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:54.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.060+0000 7fad2cff9640 1 -- 172.21.15.112:0/1782489446 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fad300d4ee0 con 0x7fad280cdd60 2024-04-25T01:37:54.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.567+0000 7fad351f6640 1 -- 172.21.15.112:0/1782489446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fad2800fba0 con 0x7fad280cdd60 2024-04-25T01:37:54.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.568+0000 7fad2cff9640 1 -- 172.21.15.112:0/1782489446 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7fad300d47e0 con 0x7fad280cdd60 2024-04-25T01:37:54.547 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":98,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:37:54.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.572+0000 7fad351f6640 1 -- 172.21.15.112:0/1782489446 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fad1406cb50 msgr2=0x7fad1406f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:54.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.572+0000 7fad351f6640 1 --2- 172.21.15.112:0/1782489446 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fad1406cb50 0x7fad1406f010 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fad24035000 tx=0x7fad24031000 comp rx=0 tx=0).stop 2024-04-25T01:37:54.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.572+0000 7fad351f6640 1 -- 172.21.15.112:0/1782489446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad280cdd60 msgr2=0x7fad28173550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:54.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.572+0000 7fad351f6640 1 --2- 172.21.15.112:0/1782489446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad280cdd60 0x7fad28173550 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fad30075bc0 tx=0x7fad300736d0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.572+0000 7fad351f6640 1 -- 172.21.15.112:0/1782489446 shutdown_connections 2024-04-25T01:37:54.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.572+0000 7fad351f6640 1 --2- 172.21.15.112:0/1782489446 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fad1406cb50 0x7fad1406f010 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.572+0000 7fad351f6640 1 --2- 172.21.15.112:0/1782489446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad280cdd60 0x7fad28173550 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.572+0000 7fad351f6640 1 --2- 172.21.15.112:0/1782489446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad2816dfc0 0x7fad280d8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.572+0000 7fad351f6640 1 --2- 172.21.15.112:0/1782489446 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad280cd980 0x7fad280ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.572+0000 7fad351f6640 1 -- 172.21.15.112:0/1782489446 >> 172.21.15.112:0/1782489446 conn(0x7fad2804c660 msgr2=0x7fad28006bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:54.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.572+0000 7fad351f6640 1 -- 172.21.15.112:0/1782489446 shutdown_connections 2024-04-25T01:37:54.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.573+0000 7fad351f6640 1 -- 172.21.15.112:0/1782489446 wait complete. 2024-04-25T01:37:54.571 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T01:37:54.571 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-04-25T01:37:54.571 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:37:54.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.758+0000 7f4b2c229640 1 Processor -- start 2024-04-25T01:37:54.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.759+0000 7f4b2c229640 1 -- start start 2024-04-25T01:37:54.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.760+0000 7f4b2c229640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b240ac000 0x7f4b24072ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.760+0000 7f4b2c229640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4b241571b0 0x7f4b24157640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.760+0000 7f4b2c229640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b240ac460 con 0x7f4b24070fd0 2024-04-25T01:37:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.760+0000 7f4b2c229640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f4b240c2190 con 0x7f4b241571b0 2024-04-25T01:37:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.760+0000 7f4b2c229640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4b240ab2d0 con 0x7f4b240ac000 2024-04-25T01:37:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.760+0000 7f4b2a7c6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4b241571b0 0x7f4b24157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.760+0000 7f4b2a7c6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4b241571b0 0x7f4b24157640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49398/0 (socket says 172.21.15.112:49398) 2024-04-25T01:37:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.760+0000 7f4b2a7c6640 1 -- 172.21.15.112:0/190860280 learned_addr learned my addr 172.21.15.112:0/190860280 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.760+0000 7f4b29fc5640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4b24070fd0 0x7f4b240c1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57936/0 (socket says 172.21.15.112:57936) 2024-04-25T01:37:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.761+0000 7f4b297c4640 1 --2- 172.21.15.112:0/190860280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b240ac000 0x7f4b24072ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.761+0000 7f4b2a7c6640 1 -- 172.21.15.112:0/190860280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b240ac000 msgr2=0x7f4b24072ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:37:54.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.761+0000 7f4b2a7c6640 1 --2- 172.21.15.112:0/190860280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b240ac000 0x7f4b24072ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.761+0000 7f4b2a7c6640 1 -- 172.21.15.112:0/190860280 >> v1:172.21.15.112:6789/0 conn(0x7f4b24070fd0 legacy=0x7f4b240c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:37:54.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.761+0000 7f4b2a7c6640 1 -- 172.21.15.112:0/190860280 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2415ac20 con 0x7f4b241571b0 2024-04-25T01:37:54.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.761+0000 7f4b297c4640 1 --2- 172.21.15.112:0/190860280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b240ac000 0x7f4b24072ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:37:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.762+0000 7f4b2a7c6640 1 --2- 172.21.15.112:0/190860280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b241571b0 0x7f4b24157640 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009e50 tx=0x7f4b1c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfdb417be80206b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.762+0000 7f4b28fc3640 1 -- 172.21.15.112:0/190860280 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c036070 con 0x7f4b241571b0 2024-04-25T01:37:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.762+0000 7f4b28fc3640 1 -- 172.21.15.112:0/190860280 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b1c031600 con 0x7f4b241571b0 2024-04-25T01:37:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.763+0000 7f4b28fc3640 1 -- 172.21.15.112:0/190860280 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c0319d0 con 0x7f4b241571b0 2024-04-25T01:37:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.763+0000 7f4b2c229640 1 -- 172.21.15.112:0/190860280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b241571b0 msgr2=0x7f4b24157640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.763+0000 7f4b2c229640 1 --2- 172.21.15.112:0/190860280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b241571b0 0x7f4b24157640 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009e50 tx=0x7f4b1c02a6c0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.763+0000 7f4b2c229640 1 -- 172.21.15.112:0/190860280 shutdown_connections 2024-04-25T01:37:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.764+0000 7f4b2c229640 1 --2- 172.21.15.112:0/190860280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b241571b0 0x7f4b24157640 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.764+0000 7f4b2c229640 1 --2- 172.21.15.112:0/190860280 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b240ac000 0x7f4b24072ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.764+0000 7f4b2c229640 1 -- 172.21.15.112:0/190860280 >> 172.21.15.112:0/190860280 conn(0x7f4b240c7080 msgr2=0x7f4b240c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.764+0000 7f4b2c229640 1 -- 172.21.15.112:0/190860280 shutdown_connections 2024-04-25T01:37:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.764+0000 7f4b2c229640 1 -- 172.21.15.112:0/190860280 wait complete. 2024-04-25T01:37:54.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.765+0000 7f4b2c229640 1 Processor -- start 2024-04-25T01:37:54.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.766+0000 7f4b2c229640 1 -- start start 2024-04-25T01:37:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.767+0000 7f4b2c229640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b24070fd0 0x7f4b24159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.767+0000 7f4b2c229640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b240ac000 0x7f4b24146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.767+0000 7f4b2c229640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b241571b0 0x7f4b24147360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.767+0000 7f4b2c229640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4b240ab800 con 0x7f4b24070fd0 2024-04-25T01:37:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.767+0000 7f4b2c229640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4b240abb70 con 0x7f4b241571b0 2024-04-25T01:37:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.767+0000 7f4b2c229640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4b2415a840 con 0x7f4b240ac000 2024-04-25T01:37:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.767+0000 7f4b2a7c6640 1 -- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b241571b0 msgr2=0x7f4b24147360 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:37:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.767+0000 7f4b2a7c6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b241571b0 0x7f4b24147360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:37:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.767+0000 7f4b29fc5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b24070fd0 0x7f4b24159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:54.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.767+0000 7f4b297c4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b240ac000 0x7f4b24146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:54.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.767+0000 7f4b297c4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b240ac000 0x7f4b24146e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:46796/0 (socket says 172.21.15.112:46796) 2024-04-25T01:37:54.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.767+0000 7f4b29fc5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b24070fd0 0x7f4b24159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49402/0 (socket says 172.21.15.112:49402) 2024-04-25T01:37:54.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.767+0000 7f4b297c4640 1 -- 172.21.15.112:0/159766357 learned_addr learned my addr 172.21.15.112:0/159766357 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:54.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.767+0000 7f4b29fc5640 1 -- 172.21.15.112:0/159766357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b240ac000 msgr2=0x7f4b24146e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:54.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.767+0000 7f4b29fc5640 1 --2- 172.21.15.112:0/159766357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b240ac000 0x7f4b24146e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.768+0000 7f4b29fc5640 1 -- 172.21.15.112:0/159766357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b241571b0 msgr2=0x7f4b24147360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:37:54.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.768+0000 7f4b29fc5640 1 --2- 172.21.15.112:0/159766357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b241571b0 0x7f4b24147360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.768+0000 7f4b29fc5640 1 -- 172.21.15.112:0/159766357 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2415da60 con 0x7f4b24070fd0 2024-04-25T01:37:54.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.768+0000 7f4b29fc5640 1 --2- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b24070fd0 0x7f4b24159330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400c780 tx=0x7f4b1400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:54.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.768+0000 7f4b1a7fc640 1 -- 172.21.15.112:0/159766357 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14017070 con 0x7f4b24070fd0 2024-04-25T01:37:54.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.768+0000 7f4b1a7fc640 1 -- 172.21.15.112:0/159766357 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b14007070 con 0x7f4b24070fd0 2024-04-25T01:37:54.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.768+0000 7f4b2c229640 1 -- 172.21.15.112:0/159766357 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b241478a0 con 0x7f4b24070fd0 2024-04-25T01:37:54.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.768+0000 7f4b2c229640 1 -- 172.21.15.112:0/159766357 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b241f3a30 con 0x7f4b24070fd0 2024-04-25T01:37:54.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.768+0000 7f4b1a7fc640 1 -- 172.21.15.112:0/159766357 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14007460 con 0x7f4b24070fd0 2024-04-25T01:37:54.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.770+0000 7f4b2c229640 1 --2- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c002a70 0x7f4b0c004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.770+0000 7f4b2c229640 1 -- 172.21.15.112:0/159766357 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c007ec0 con 0x7f4b0c002a70 2024-04-25T01:37:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.770+0000 7f4b297c4640 1 --2- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c002a70 0x7f4b0c004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.770+0000 7f4b297c4640 1 -- 172.21.15.112:0/159766357 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4b0c0080c0 con 0x7f4b0c002a70 2024-04-25T01:37:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.770+0000 7f4b1a7fc640 1 -- 172.21.15.112:0/159766357 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b1401a070 con 0x7f4b24070fd0 2024-04-25T01:37:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.770+0000 7f4b297c4640 1 --2- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c002a70 0x7f4b0c004f30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4b20006d40 tx=0x7f4b20011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.771+0000 7f4b1a7fc640 1 --2- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f4b0806be30 0x7f4b0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.771+0000 7f4b2a7c6640 1 --2- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f4b0806be30 0x7f4b0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.771+0000 7f4b1a7fc640 1 -- 172.21.15.112:0/159766357 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f4b1408a3b0 con 0x7f4b24070fd0 2024-04-25T01:37:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.772+0000 7f4b1a7fc640 1 -- 172.21.15.112:0/159766357 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20019020 con 0x7f4b0c002a70 2024-04-25T01:37:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.772+0000 7f4b1a7fc640 1 -- 172.21.15.112:0/159766357 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4b20004230 con 0x7f4b0c002a70 2024-04-25T01:37:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.772+0000 7f4b1a7fc640 1 -- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c002a70 msgr2=0x7f4b0c004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.772+0000 7f4b1a7fc640 1 --2- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c002a70 0x7f4b0c004f30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4b20006d40 tx=0x7f4b20011c90 comp rx=0 tx=0).stop 2024-04-25T01:37:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.772+0000 7f4b1a7fc640 1 -- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c002a70 msgr2=0x7f4b0c004f30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:37:54.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.772+0000 7f4b1a7fc640 1 --2- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c002a70 0x7f4b0c004f30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.772+0000 7f4b2a7c6640 1 --2- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f4b0806be30 0x7f4b0806e2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c033000 tx=0x7f4b1c009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.785+0000 7f4b2c229640 1 --2- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c006f30 0x7f4b0c0073c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.785+0000 7f4b2c229640 1 -- 172.21.15.112:0/159766357 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c00a4a0 con 0x7f4b0c006f30 2024-04-25T01:37:54.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.785+0000 7f4b297c4640 1 --2- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c006f30 0x7f4b0c0073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:54.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.788+0000 7f4b297c4640 1 -- 172.21.15.112:0/159766357 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4b0c007900 con 0x7f4b0c006f30 2024-04-25T01:37:54.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.788+0000 7f4b297c4640 1 --2- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c006f30 0x7f4b0c0073c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4b20006d40 tx=0x7f4b20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:54.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.789+0000 7f4b1a7fc640 1 -- 172.21.15.112:0/159766357 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20019070 con 0x7f4b0c006f30 2024-04-25T01:37:54.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.789+0000 7f4b1a7fc640 1 -- 172.21.15.112:0/159766357 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4890 (secure 0 0 0) 0x7f4b200202d0 con 0x7f4b0c006f30 2024-04-25T01:37:54.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.789+0000 7f4b1a7fc640 1 -- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c006f30 msgr2=0x7f4b0c0073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:54.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.789+0000 7f4b1a7fc640 1 --2- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c006f30 0x7f4b0c0073c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4b20006d40 tx=0x7f4b20004730 comp rx=0 tx=0).stop 2024-04-25T01:37:54.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.789+0000 7f4b1a7fc640 1 -- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c006f30 msgr2=0x7f4b0c0073c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:37:54.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.789+0000 7f4b1a7fc640 1 --2- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b0c006f30 0x7f4b0c0073c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.769 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:37:54.769 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:37:54.769 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:37:54.769 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:37:54.769 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 98, 2024-04-25T01:37:54.769 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 125602, 2024-04-25T01:37:54.769 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:37:54.770 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:37:54.770 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:37:54.770 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:54.770 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:37:54.770 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:37:54.770 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:37:54.770 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:37:54.770 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:54.770 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:54.770 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:54.770 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:54.770 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:54.771 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:54.771 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:54.771 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:54.771 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:54.771 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:54.771 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:54.771 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:54.771 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:37:54.771 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:37:54.771 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:54.771 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:54.771 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:54.772 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:54.772 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:54.772 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:54.772 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:54.772 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:54.772 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:54.772 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:54.772 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:54.772 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:54.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:54.772 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:37:54.772 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:37:54.772 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:37:54.773 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:37:54.773 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:37:54.773 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:37:54.773 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:37:54.773 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:37:54.773 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:37:54.773 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:37:54.773 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:37:54.773 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:37:54.773 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:37:54.773 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:37:54.773 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:37:54.774 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:37:54.774 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:37:54.774 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:54.774 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:54.774 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:54.774 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:54.774 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:54.774 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:54.774 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:54.774 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:54.774 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:54.775 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:54.775 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:54.775 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:54.775 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:37:54.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:54.775 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:37:54.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:54.775 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:37:54.775 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:37:54.775 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:54.775 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:54.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:54.776 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:54.776 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:37:54.776 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:54.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:54.776 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:54.776 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:54.776 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:37:54.776 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:54.776 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:54.776 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:54.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:54.776 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:37:54.777 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:37:54.777 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:54.777 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:54.777 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:54.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:54.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:54.777 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:37:54.777 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:37:54.777 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:54.777 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:54.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:54.777 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:54.777 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:37:54.778 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:54.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:54.778 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:54.778 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:54.778 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:37:54.778 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:54.778 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:54.778 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:54.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:54.778 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:37:54.778 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:37:54.779 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:54.779 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:54.779 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:54.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:54.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:54.779 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:37:54.779 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:37:54.779 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:54.779 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:54.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:54.779 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:54.779 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:37:54.779 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:54.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:54.780 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:54.780 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:54.780 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:37:54.780 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:54.780 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:54.780 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:54.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:54.780 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:37:54.780 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:37:54.780 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:54.780 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:54.780 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:54.781 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:54.781 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:54.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:54.781 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:37:54.781 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:37:54.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:54.781 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:54.781 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:54.781 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:37:54.781 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:54.781 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:54.781 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:37:54.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:54.782 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:54.782 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:54.782 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:37:54.782 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:54.782 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:54.782 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:37:54.782 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:54.782 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:54.782 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:54.782 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:37:54.782 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:54.783 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:54.783 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:37:54.783 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:54.783 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:54.783 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:54.783 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 6 2024-04-25T01:37:54.783 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:54.783 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:54.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:54.783 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:37:54.783 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:37:54.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.790+0000 7f4b2c229640 1 -- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f4b0806be30 msgr2=0x7f4b0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:54.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.790+0000 7f4b2c229640 1 --2- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f4b0806be30 0x7f4b0806e2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c033000 tx=0x7f4b1c009000 comp rx=0 tx=0).stop 2024-04-25T01:37:54.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.790+0000 7f4b2a7c6640 1 -- 172.21.15.112:0/159766357 reap_dead start 2024-04-25T01:37:54.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.790+0000 7f4b2c229640 1 -- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b24070fd0 msgr2=0x7f4b24159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:54.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.790+0000 7f4b2c229640 1 --2- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b24070fd0 0x7f4b24159330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400c780 tx=0x7f4b1400cc40 comp rx=0 tx=0).stop 2024-04-25T01:37:54.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.791+0000 7f4b2c229640 1 -- 172.21.15.112:0/159766357 shutdown_connections 2024-04-25T01:37:54.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.791+0000 7f4b2c229640 1 --2- 172.21.15.112:0/159766357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b24070fd0 0x7f4b24159330 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400c780 tx=0x7f4b1400cc40 comp rx=0 tx=0).stop 2024-04-25T01:37:54.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.791+0000 7f4b2c229640 1 -- 172.21.15.112:0/159766357 >> 172.21.15.112:0/159766357 conn(0x7f4b240c7080 msgr2=0x7f4b240c1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:54.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.791+0000 7f4b2c229640 1 -- 172.21.15.112:0/159766357 shutdown_connections 2024-04-25T01:37:54.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.791+0000 7f4b2c229640 1 -- 172.21.15.112:0/159766357 wait complete. 2024-04-25T01:37:54.788 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:37:54.789 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-04-25T01:37:54.789 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:37:54.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.985+0000 7ff5e0961640 1 Processor -- start 2024-04-25T01:37:54.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.985+0000 7ff5e0961640 1 -- start start 2024-04-25T01:37:54.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.986+0000 7ff5e0961640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff5dc1564f0 0x7ff5dc158da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.986+0000 7ff5e0961640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5dc06f2d0 con 0x7ff5dc152d60 2024-04-25T01:37:54.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.986+0000 7ff5e0961640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5dc159cd0 con 0x7ff5dc0c3400 2024-04-25T01:37:54.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.986+0000 7ff5e0961640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff5dc159430 con 0x7ff5dc1564f0 2024-04-25T01:37:54.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.986+0000 7ff5d9d9b640 1 -- >> v1:172.21.15.187:6789/0 conn(0x7ff5dc0c3400 legacy=0x7ff5dc155de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.187:6789/0 2024-04-25T01:37:54.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.986+0000 7ff5da59c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff5dc152d60 0x7ff5dc0c3b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57952/0 (socket says 172.21.15.112:57952) 2024-04-25T01:37:54.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.986+0000 7ff5dad9d640 1 -- >> v2:172.21.15.187:3300/0 conn(0x7ff5dc1564f0 msgr2=0x7ff5dc158da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:37:54.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.986+0000 7ff5da59c640 1 -- 172.21.15.112:0/2928793503 learned_addr learned my addr 172.21.15.112:0/2928793503 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:37:54.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.986+0000 7ff5dad9d640 1 --2- 172.21.15.112:0/2928793503 >> v2:172.21.15.187:3300/0 conn(0x7ff5dc1564f0 0x7ff5dc158da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:37:54.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.989+0000 7ff5d959a640 1 -- 172.21.15.112:0/2928793503 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2626098237 0 0) 0x7ff5dc06f2d0 con 0x7ff5dc152d60 2024-04-25T01:37:54.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.989+0000 7ff5d959a640 1 -- 172.21.15.112:0/2928793503 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5c4003520 con 0x7ff5dc152d60 2024-04-25T01:37:54.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.990+0000 7ff5d959a640 1 -- 172.21.15.112:0/2928793503 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1737922726 0 0) 0x7ff5c4003520 con 0x7ff5dc152d60 2024-04-25T01:37:54.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.990+0000 7ff5d959a640 1 -- 172.21.15.112:0/2928793503 >> v2:172.21.15.187:3300/0 conn(0x7ff5dc1564f0 msgr2=0x7ff5dc158da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:37:54.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.990+0000 7ff5d959a640 1 --2- 172.21.15.112:0/2928793503 >> v2:172.21.15.187:3300/0 conn(0x7ff5dc1564f0 0x7ff5dc158da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.990+0000 7ff5d959a640 1 -- 172.21.15.112:0/2928793503 >> v1:172.21.15.187:6789/0 conn(0x7ff5dc0c3400 legacy=0x7ff5dc155de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:37:54.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.990+0000 7ff5d959a640 1 -- 172.21.15.112:0/2928793503 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5dc159710 con 0x7ff5dc152d60 2024-04-25T01:37:54.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.990+0000 7ff5d959a640 1 -- 172.21.15.112:0/2928793503 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7ff5d4001a10 con 0x7ff5dc152d60 2024-04-25T01:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.990+0000 7ff5d959a640 1 -- 172.21.15.112:0/2928793503 >> v1:172.21.15.112:6789/0 conn(0x7ff5dc152d60 legacy=0x7ff5dc0c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.990+0000 7ff5d959a640 1 --2- 172.21.15.112:0/2928793503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5c40039b0 0x7ff5c4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.990+0000 7ff5d959a640 1 --2- 172.21.15.112:0/2928793503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5c4004800 0x7ff5c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.990+0000 7ff5da59c640 1 -- 172.21.15.112:0/2928793503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5c40039b0 msgr2=0x7ff5c4003db0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.990+0000 7ff5da59c640 1 --2- 172.21.15.112:0/2928793503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5c40039b0 0x7ff5c4003db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.990+0000 7ff5d959a640 1 --2- 172.21.15.112:0/2928793503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5c40053c0 0x7ff5c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.990+0000 7ff5d959a640 1 -- 172.21.15.112:0/2928793503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff5dc159430 con 0x7ff5c4004800 2024-04-25T01:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.990+0000 7ff5d959a640 1 -- 172.21.15.112:0/2928793503 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c400c260 con 0x7ff5c40039b0 2024-04-25T01:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.990+0000 7ff5d959a640 1 -- 172.21.15.112:0/2928793503 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c4014c70 con 0x7ff5c40053c0 2024-04-25T01:37:54.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.990+0000 7ff5dad9d640 1 --2- 172.21.15.112:0/2928793503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5c40053c0 0x7ff5c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:54.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.991+0000 7ff5d9d9b640 1 --2- 172.21.15.112:0/2928793503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5c4004800 0x7ff5c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:54.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.991+0000 7ff5d9d9b640 1 -- 172.21.15.112:0/2928793503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5c40053c0 msgr2=0x7ff5c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:37:54.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.991+0000 7ff5d9d9b640 1 --2- 172.21.15.112:0/2928793503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5c40053c0 0x7ff5c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.991+0000 7ff5d9d9b640 1 -- 172.21.15.112:0/2928793503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5c40039b0 msgr2=0x7ff5c4003db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:37:54.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.991+0000 7ff5d9d9b640 1 --2- 172.21.15.112:0/2928793503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5c40039b0 0x7ff5c4003db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.991+0000 7ff5d9d9b640 1 -- 172.21.15.112:0/2928793503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5dc159710 con 0x7ff5c4004800 2024-04-25T01:37:54.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.992+0000 7ff5dad9d640 1 -- 172.21.15.112:0/2928793503 reap_dead start 2024-04-25T01:37:54.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.993+0000 7ff5d9d9b640 1 --2- 172.21.15.112:0/2928793503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5c4004800 0x7ff5c4004c80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0009900 tx=0x7ff5c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c83f478e464760 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:54.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.994+0000 7ff5d959a640 1 -- 172.21.15.112:0/2928793503 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0009500 con 0x7ff5c4004800 2024-04-25T01:37:54.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.994+0000 7ff5d959a640 1 -- 172.21.15.112:0/2928793503 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d002e070 con 0x7ff5c4004800 2024-04-25T01:37:54.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.994+0000 7ff5d959a640 1 -- 172.21.15.112:0/2928793503 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0003df0 con 0x7ff5c4004800 2024-04-25T01:37:54.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.995+0000 7ff5e0961640 1 -- 172.21.15.112:0/2928793503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5c4004800 msgr2=0x7ff5c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:54.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.995+0000 7ff5e0961640 1 --2- 172.21.15.112:0/2928793503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5c4004800 0x7ff5c4004c80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0009900 tx=0x7ff5c4005830 comp rx=0 tx=0).stop 2024-04-25T01:37:54.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.995+0000 7ff5e0961640 1 -- 172.21.15.112:0/2928793503 shutdown_connections 2024-04-25T01:37:54.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.995+0000 7ff5e0961640 1 --2- 172.21.15.112:0/2928793503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5c4004800 0x7ff5c4004c80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.995+0000 7ff5e0961640 1 -- 172.21.15.112:0/2928793503 >> 172.21.15.112:0/2928793503 conn(0x7ff5dc06b160 msgr2=0x7ff5dc06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:54.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.996+0000 7ff5e0961640 1 -- 172.21.15.112:0/2928793503 shutdown_connections 2024-04-25T01:37:54.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.996+0000 7ff5e0961640 1 -- 172.21.15.112:0/2928793503 wait complete. 2024-04-25T01:37:54.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.998+0000 7ff5e0961640 1 Processor -- start 2024-04-25T01:37:54.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.998+0000 7ff5e0961640 1 -- start start 2024-04-25T01:37:54.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.999+0000 7ff5e0961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5c4004800 0x7ff5dc1eaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.999+0000 7ff5e0961640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5dc0c6b30 0x7ff5dc143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.999+0000 7ff5e0961640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5dc147060 0x7ff5dc1e6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.999+0000 7ff5e0961640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff5dc153140 con 0x7ff5c4004800 2024-04-25T01:37:54.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.999+0000 7ff5e0961640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff5dc1558c0 con 0x7ff5dc0c6b30 2024-04-25T01:37:54.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.999+0000 7ff5e0961640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff5dc0aa1a0 con 0x7ff5dc147060 2024-04-25T01:37:54.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.999+0000 7ff5d9d9b640 1 -- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5dc0c6b30 msgr2=0x7ff5dc143a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:37:54.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.999+0000 7ff5d9d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5dc0c6b30 0x7ff5dc143a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:37:54.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.999+0000 7ff5da59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5c4004800 0x7ff5dc1eaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.999+0000 7ff5da59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5c4004800 0x7ff5dc1eaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49426/0 (socket says 172.21.15.112:49426) 2024-04-25T01:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.999+0000 7ff5da59c640 1 -- 172.21.15.112:0/976275545 learned_addr learned my addr 172.21.15.112:0/976275545 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:54.999+0000 7ff5dad9d640 1 --2- 172.21.15.112:0/976275545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5dc147060 0x7ff5dc1e6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.000+0000 7ff5da59c640 1 -- 172.21.15.112:0/976275545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5dc147060 msgr2=0x7ff5dc1e6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.000+0000 7ff5da59c640 1 --2- 172.21.15.112:0/976275545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5dc147060 0x7ff5dc1e6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.000+0000 7ff5da59c640 1 -- 172.21.15.112:0/976275545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5dc0c6b30 msgr2=0x7ff5dc143a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.000+0000 7ff5da59c640 1 --2- 172.21.15.112:0/976275545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5dc0c6b30 0x7ff5dc143a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.000+0000 7ff5da59c640 1 -- 172.21.15.112:0/976275545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5dc0c65f0 con 0x7ff5c4004800 2024-04-25T01:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.000+0000 7ff5da59c640 1 --2- 172.21.15.112:0/976275545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5c4004800 0x7ff5dc1eaff0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400cb30 tx=0x7ff5d400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.000+0000 7ff5caffd640 1 -- 172.21.15.112:0/976275545 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4018070 con 0x7ff5c4004800 2024-04-25T01:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.000+0000 7ff5caffd640 1 -- 172.21.15.112:0/976275545 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d400a830 con 0x7ff5c4004800 2024-04-25T01:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.000+0000 7ff5caffd640 1 -- 172.21.15.112:0/976275545 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d400abe0 con 0x7ff5c4004800 2024-04-25T01:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.001+0000 7ff5e0961640 1 -- 172.21.15.112:0/976275545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5dc0c68d0 con 0x7ff5c4004800 2024-04-25T01:37:54.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.001+0000 7ff5e0961640 1 -- 172.21.15.112:0/976275545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5dc1ea7f0 con 0x7ff5c4004800 2024-04-25T01:37:54.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.002+0000 7ff5e0961640 1 --2- 172.21.15.112:0/976275545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5dc0682a0 0x7ff5dc0c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.002+0000 7ff5e0961640 1 -- 172.21.15.112:0/976275545 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff5dc143fa0 con 0x7ff5dc0682a0 2024-04-25T01:37:54.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.003+0000 7ff5d9d9b640 1 --2- 172.21.15.112:0/976275545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5dc0682a0 0x7ff5dc0c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:54.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.003+0000 7ff5caffd640 1 -- 172.21.15.112:0/976275545 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff5d40070d0 con 0x7ff5c4004800 2024-04-25T01:37:54.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.003+0000 7ff5caffd640 1 --2- 172.21.15.112:0/976275545 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff5bc06be10 0x7ff5bc06e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.004+0000 7ff5dad9d640 1 --2- 172.21.15.112:0/976275545 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff5bc06be10 0x7ff5bc06e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:54.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.004+0000 7ff5caffd640 1 -- 172.21.15.112:0/976275545 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7ff5d408a9f0 con 0x7ff5c4004800 2024-04-25T01:37:54.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.005+0000 7ff5dad9d640 1 --2- 172.21.15.112:0/976275545 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff5bc06be10 0x7ff5bc06e2d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc0c18c0 tx=0x7ff5cc0090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:54.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.005+0000 7ff5d9d9b640 1 -- 172.21.15.112:0/976275545 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff5dc152410 con 0x7ff5dc0682a0 2024-04-25T01:37:54.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.005+0000 7ff5d9d9b640 1 --2- 172.21.15.112:0/976275545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5dc0682a0 0x7ff5dc0c4c70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0003930 tx=0x7ff5d0005a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:54.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.006+0000 7ff5caffd640 1 -- 172.21.15.112:0/976275545 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0005270 con 0x7ff5dc0682a0 2024-04-25T01:37:54.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.006+0000 7ff5caffd640 1 -- 172.21.15.112:0/976275545 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff5d002e040 con 0x7ff5dc0682a0 2024-04-25T01:37:54.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.006+0000 7ff5caffd640 1 -- 172.21.15.112:0/976275545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5dc0682a0 msgr2=0x7ff5dc0c4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:54.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.006+0000 7ff5caffd640 1 --2- 172.21.15.112:0/976275545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5dc0682a0 0x7ff5dc0c4c70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0003930 tx=0x7ff5d0005a80 comp rx=0 tx=0).stop 2024-04-25T01:37:54.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.006+0000 7ff5caffd640 1 -- 172.21.15.112:0/976275545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5dc0682a0 msgr2=0x7ff5dc0c4c70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:37:54.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.006+0000 7ff5caffd640 1 --2- 172.21.15.112:0/976275545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5dc0682a0 0x7ff5dc0c4c70 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0003930 tx=0x7ff5d0005a80 comp rx=0 tx=0).stop 2024-04-25T01:37:54.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.020+0000 7ff5e0961640 1 --2- 172.21.15.112:0/976275545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5dc1e7440 0x7ff5dc0c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:54.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.021+0000 7ff5e0961640 1 -- 172.21.15.112:0/976275545 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff5dc075980 con 0x7ff5dc1e7440 2024-04-25T01:37:54.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.021+0000 7ff5d9d9b640 1 --2- 172.21.15.112:0/976275545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5dc1e7440 0x7ff5dc0c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:55.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.022+0000 7ff5d9d9b640 1 -- 172.21.15.112:0/976275545 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff5dc068b40 con 0x7ff5dc1e7440 2024-04-25T01:37:55.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.025+0000 7ff5d9d9b640 1 --2- 172.21.15.112:0/976275545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5dc1e7440 0x7ff5dc0c51b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc145740 tx=0x7ff5d0005a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:55.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.025+0000 7ff5caffd640 1 -- 172.21.15.112:0/976275545 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d00462b0 con 0x7ff5dc1e7440 2024-04-25T01:37:55.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.026+0000 7ff5caffd640 1 -- 172.21.15.112:0/976275545 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4887 (secure 0 0 0) 0x7ff5d0046880 con 0x7ff5dc1e7440 2024-04-25T01:37:55.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.026+0000 7ff5caffd640 1 -- 172.21.15.112:0/976275545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5dc1e7440 msgr2=0x7ff5dc0c51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:55.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.026+0000 7ff5caffd640 1 --2- 172.21.15.112:0/976275545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5dc1e7440 0x7ff5dc0c51b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc145740 tx=0x7ff5d0005a10 comp rx=0 tx=0).stop 2024-04-25T01:37:55.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.026+0000 7ff5caffd640 1 -- 172.21.15.112:0/976275545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5dc1e7440 msgr2=0x7ff5dc0c51b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:37:55.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.026+0000 7ff5caffd640 1 --2- 172.21.15.112:0/976275545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5dc1e7440 0x7ff5dc0c51b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:55.005 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:37:55.006 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:37:55.006 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:37:55.006 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:37:55.006 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 98, 2024-04-25T01:37:55.006 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 44390, 2024-04-25T01:37:55.006 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:37:55.006 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:37:55.006 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:37:55.006 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:55.006 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:37:55.006 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:37:55.006 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:37:55.006 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:37:55.006 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:37:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:37:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:55.007 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:37:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:37:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:37:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:37:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:37:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:37:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:37:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:37:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:37:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:55.010 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:37:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:37:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:37:55.011 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:37:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:37:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:37:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:37:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:55.012 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:37:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:37:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:37:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:55.013 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:37:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:37:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:37:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:37:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:37:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:37:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:37:55.014 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:37:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:37:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:37:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:37:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:37:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:37:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:37:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:37:55.015 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:37:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:37:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:37:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:37:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:55.016 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:37:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:37:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:37:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:37:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:37:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:37:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:37:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:37:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:37:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:37:55.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:37:55.018 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:37:55.018 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:37:55.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.027+0000 7ff5e0961640 1 -- 172.21.15.112:0/976275545 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff5bc06be10 msgr2=0x7ff5bc06e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.027+0000 7ff5e0961640 1 --2- 172.21.15.112:0/976275545 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff5bc06be10 0x7ff5bc06e2d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc0c18c0 tx=0x7ff5cc0090e0 comp rx=0 tx=0).stop 2024-04-25T01:37:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.027+0000 7ff5e0961640 1 -- 172.21.15.112:0/976275545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5c4004800 msgr2=0x7ff5dc1eaff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.027+0000 7ff5e0961640 1 --2- 172.21.15.112:0/976275545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5c4004800 0x7ff5dc1eaff0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400cb30 tx=0x7ff5d400a060 comp rx=0 tx=0).stop 2024-04-25T01:37:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.027+0000 7ff5dad9d640 1 -- 172.21.15.112:0/976275545 reap_dead start 2024-04-25T01:37:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.028+0000 7ff5e0961640 1 -- 172.21.15.112:0/976275545 shutdown_connections 2024-04-25T01:37:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.028+0000 7ff5e0961640 1 --2- 172.21.15.112:0/976275545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5c4004800 0x7ff5dc1eaff0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.028+0000 7ff5e0961640 1 -- 172.21.15.112:0/976275545 >> 172.21.15.112:0/976275545 conn(0x7ff5dc06b160 msgr2=0x7ff5dc0c24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.028+0000 7ff5e0961640 1 -- 172.21.15.112:0/976275545 shutdown_connections 2024-04-25T01:37:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:37:55.028+0000 7ff5e0961640 1 -- 172.21.15.112:0/976275545 wait complete. 2024-04-25T01:37:55.026 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:37:55.026 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T01:37:58.241 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:37:58.244 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:38:15.026 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-04-25T01:38:15.026 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-04-25T01:38:15.027 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T01:38:15.027 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T01:38:15.033 INFO:tasks.ceph.mon.b:Started 2024-04-25T01:38:15.034 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:38:15.034 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:38:15.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.222+0000 7fadbb52e640 1 Processor -- start 2024-04-25T01:38:15.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.222+0000 7fadbb52e640 1 -- start start 2024-04-25T01:38:15.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.224+0000 7fadbb52e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fadb414ee40 0x7fadb4071710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:15.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.224+0000 7fadbb52e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb406dc70 con 0x7fadb40682a0 2024-04-25T01:38:15.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.224+0000 7fadbb52e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb40714b0 con 0x7fadb4152400 2024-04-25T01:38:15.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.224+0000 7fadbb52e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fadb40733f0 con 0x7fadb414ee40 2024-04-25T01:38:15.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.224+0000 7fadbad2d640 1 -- >> v2:172.21.15.187:3300/0 conn(0x7fadb414ee40 msgr2=0x7fadb4071710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:38:15.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.224+0000 7fadbad2d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fadb414ee40 0x7fadb4071710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:38:15.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.224+0000 7fadb9d2b640 1 -- >> v1:172.21.15.187:6789/0 conn(0x7fadb4152400 legacy=0x7fadb40c1a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.187:6789/0 2024-04-25T01:38:15.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.228+0000 7fadba52c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fadb40682a0 0x7fadb406eeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59352/0 (socket says 172.21.15.112:59352) 2024-04-25T01:38:15.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.228+0000 7fadba52c640 1 -- 172.21.15.112:0/312107156 learned_addr learned my addr 172.21.15.112:0/312107156 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:38:15.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.229+0000 7fadb952a640 1 -- 172.21.15.112:0/312107156 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2413251602 0 0) 0x7fadb406dc70 con 0x7fadb40682a0 2024-04-25T01:38:15.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.229+0000 7fadb952a640 1 -- 172.21.15.112:0/312107156 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fada0003520 con 0x7fadb40682a0 2024-04-25T01:38:15.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.229+0000 7fadb952a640 1 -- 172.21.15.112:0/312107156 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1766869326 0 0) 0x7fada0003520 con 0x7fadb40682a0 2024-04-25T01:38:15.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.229+0000 7fadb952a640 1 -- 172.21.15.112:0/312107156 >> v2:172.21.15.187:3300/0 conn(0x7fadb414ee40 msgr2=0x7fadb4071710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:38:15.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.229+0000 7fadb952a640 1 --2- 172.21.15.112:0/312107156 >> v2:172.21.15.187:3300/0 conn(0x7fadb414ee40 0x7fadb4071710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.230+0000 7fadb952a640 1 -- 172.21.15.112:0/312107156 >> v1:172.21.15.187:6789/0 conn(0x7fadb4152400 legacy=0x7fadb40c1a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:38:15.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.230+0000 7fadb952a640 1 -- 172.21.15.112:0/312107156 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb406e1a0 con 0x7fadb40682a0 2024-04-25T01:38:15.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.230+0000 7fadb952a640 1 -- 172.21.15.112:0/312107156 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fadac0019d0 con 0x7fadb40682a0 2024-04-25T01:38:15.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.230+0000 7fadb952a640 1 -- 172.21.15.112:0/312107156 >> v1:172.21.15.112:6789/0 conn(0x7fadb40682a0 legacy=0x7fadb406eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:15.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.230+0000 7fadb952a640 1 --2- 172.21.15.112:0/312107156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fada00039f0 0x7fada0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:15.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.230+0000 7fadb952a640 1 --2- 172.21.15.112:0/312107156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fada0004800 0x7fada0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:15.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.230+0000 7fadb952a640 1 --2- 172.21.15.112:0/312107156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fada00052e0 0x7fada000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:15.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.230+0000 7fadb952a640 1 -- 172.21.15.112:0/312107156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fadb40733f0 con 0x7fada0004800 2024-04-25T01:38:15.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.230+0000 7fadb952a640 1 -- 172.21.15.112:0/312107156 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fada000c200 con 0x7fada00052e0 2024-04-25T01:38:15.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.230+0000 7fadb952a640 1 -- 172.21.15.112:0/312107156 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fada000c400 con 0x7fada00039f0 2024-04-25T01:38:15.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.230+0000 7fadbad2d640 1 -- 172.21.15.112:0/312107156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fada00052e0 msgr2=0x7fada000b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:38:15.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.230+0000 7fadbad2d640 1 --2- 172.21.15.112:0/312107156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fada00052e0 0x7fada000b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:38:15.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.230+0000 7fadb9d2b640 1 --2- 172.21.15.112:0/312107156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fada0004800 0x7fada0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:15.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.230+0000 7fadba52c640 1 --2- 172.21.15.112:0/312107156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fada00039f0 0x7fada0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:15.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.231+0000 7fadb9d2b640 1 -- 172.21.15.112:0/312107156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fada00039f0 msgr2=0x7fada0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:38:15.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.231+0000 7fadb9d2b640 1 --2- 172.21.15.112:0/312107156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fada00039f0 0x7fada0003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.231+0000 7fadb9d2b640 1 -- 172.21.15.112:0/312107156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fada00052e0 msgr2=0x7fada000b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:38:15.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.231+0000 7fadb9d2b640 1 --2- 172.21.15.112:0/312107156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fada00052e0 0x7fada000b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.231+0000 7fadbad2d640 1 -- 172.21.15.112:0/312107156 reap_dead start 2024-04-25T01:38:15.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.231+0000 7fadb9d2b640 1 -- 172.21.15.112:0/312107156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb406e1a0 con 0x7fada0004800 2024-04-25T01:38:15.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.232+0000 7fadb9d2b640 1 --2- 172.21.15.112:0/312107156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fada0004800 0x7fada0004c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fada0004780 tx=0x7fada0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a485fa7c1e0122e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:15.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.232+0000 7fadb952a640 1 -- 172.21.15.112:0/312107156 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4009520 con 0x7fada0004800 2024-04-25T01:38:15.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.232+0000 7fadb952a640 1 -- 172.21.15.112:0/312107156 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada402e070 con 0x7fada0004800 2024-04-25T01:38:15.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.233+0000 7fadb952a640 1 -- 172.21.15.112:0/312107156 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4003dd0 con 0x7fada0004800 2024-04-25T01:38:15.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.233+0000 7fadbb52e640 1 -- 172.21.15.112:0/312107156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fada0004800 msgr2=0x7fada0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:15.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.233+0000 7fadbb52e640 1 --2- 172.21.15.112:0/312107156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fada0004800 0x7fada0004c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fada0004780 tx=0x7fada0005830 comp rx=0 tx=0).stop 2024-04-25T01:38:15.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.233+0000 7fadbb52e640 1 -- 172.21.15.112:0/312107156 shutdown_connections 2024-04-25T01:38:15.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.233+0000 7fadbb52e640 1 --2- 172.21.15.112:0/312107156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fada0004800 0x7fada0004c80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.233+0000 7fadbb52e640 1 -- 172.21.15.112:0/312107156 >> 172.21.15.112:0/312107156 conn(0x7fadb40c3b80 msgr2=0x7fadb40c3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:15.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.233+0000 7fadbb52e640 1 -- 172.21.15.112:0/312107156 shutdown_connections 2024-04-25T01:38:15.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.234+0000 7fadbb52e640 1 -- 172.21.15.112:0/312107156 wait complete. 2024-04-25T01:38:15.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.235+0000 7fadbb52e640 1 Processor -- start 2024-04-25T01:38:15.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.235+0000 7fadbb52e640 1 -- start start 2024-04-25T01:38:15.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.236+0000 7fadbb52e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fada0004800 0x7fadb4150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:15.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.236+0000 7fadbb52e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadb4148d60 0x7fadb40ab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:15.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.236+0000 7fadbb52e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadb413fe10 0x7fadb40ab910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:15.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.236+0000 7fadba52c640 1 -- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fada0004800 msgr2=0x7fadb4150a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.187:3300/0 2024-04-25T01:38:15.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.236+0000 7fadba52c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fada0004800 0x7fadb4150a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:38:15.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.237+0000 7fadb9d2b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadb4148d60 0x7fadb40ab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:15.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.237+0000 7fadb9d2b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadb4148d60 0x7fadb40ab3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:60148/0 (socket says 172.21.15.112:60148) 2024-04-25T01:38:15.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.237+0000 7fadbad2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadb413fe10 0x7fadb40ab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:15.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.237+0000 7fadbad2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadb413fe10 0x7fadb40ab910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41516/0 (socket says 172.21.15.112:41516) 2024-04-25T01:38:15.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.237+0000 7fadbb52e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fadb40711c0 con 0x7fadb413fe10 2024-04-25T01:38:15.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.237+0000 7fadbb52e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fadb4073080 con 0x7fada0004800 2024-04-25T01:38:15.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.237+0000 7fadbb52e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fadb4070ea0 con 0x7fadb4148d60 2024-04-25T01:38:15.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.237+0000 7fadb9d2b640 1 -- 172.21.15.112:0/3511401131 learned_addr learned my addr 172.21.15.112:0/3511401131 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:15.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.238+0000 7fadbad2d640 1 -- 172.21.15.112:0/3511401131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadb4148d60 msgr2=0x7fadb40ab3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:15.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.238+0000 7fadbad2d640 1 --2- 172.21.15.112:0/3511401131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadb4148d60 0x7fadb40ab3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.238+0000 7fadbad2d640 1 -- 172.21.15.112:0/3511401131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fada0004800 msgr2=0x7fadb4150a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:38:15.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.238+0000 7fadbad2d640 1 --2- 172.21.15.112:0/3511401131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fada0004800 0x7fadb4150a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.238+0000 7fadbad2d640 1 -- 172.21.15.112:0/3511401131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb413f830 con 0x7fadb413fe10 2024-04-25T01:38:15.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.238+0000 7fadbad2d640 1 --2- 172.21.15.112:0/3511401131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadb413fe10 0x7fadb40ab910 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fadb406cb80 tx=0x7fadb0009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:15.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.238+0000 7fadb9d2b640 1 --2- 172.21.15.112:0/3511401131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadb4148d60 0x7fadb40ab3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-25T01:38:15.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.239+0000 7fadaaffd640 1 -- 172.21.15.112:0/3511401131 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0017070 con 0x7fadb413fe10 2024-04-25T01:38:15.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.239+0000 7fadaaffd640 1 -- 172.21.15.112:0/3511401131 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb0003c70 con 0x7fadb413fe10 2024-04-25T01:38:15.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.239+0000 7fadaaffd640 1 -- 172.21.15.112:0/3511401131 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0005310 con 0x7fadb413fe10 2024-04-25T01:38:15.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.239+0000 7fadbb52e640 1 -- 172.21.15.112:0/3511401131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb413d070 con 0x7fadb413fe10 2024-04-25T01:38:15.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.239+0000 7fadbb52e640 1 -- 172.21.15.112:0/3511401131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb40abe50 con 0x7fadb413fe10 2024-04-25T01:38:15.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.241+0000 7fadaaffd640 1 -- 172.21.15.112:0/3511401131 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fadb00054b0 con 0x7fadb413fe10 2024-04-25T01:38:15.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.241+0000 7fadbb52e640 1 -- 172.21.15.112:0/3511401131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb404f500 con 0x7fadb413fe10 2024-04-25T01:38:15.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.242+0000 7fadaaffd640 1 --2- 172.21.15.112:0/3511401131 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fad9806be30 0x7fad9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:15.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.242+0000 7fadaaffd640 1 -- 172.21.15.112:0/3511401131 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fadb008a090 con 0x7fadb413fe10 2024-04-25T01:38:15.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.242+0000 7fadba52c640 1 --2- 172.21.15.112:0/3511401131 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fad9806be30 0x7fad9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:15.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.243+0000 7fadba52c640 1 --2- 172.21.15.112:0/3511401131 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fad9806be30 0x7fad9806e2f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fadb406de40 tx=0x7fadac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:15.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.251+0000 7fadaaffd640 1 -- 172.21.15.112:0/3511401131 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fadb0058710 con 0x7fadb413fe10 2024-04-25T01:38:15.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.757+0000 7fadbb52e640 1 -- 172.21.15.112:0/3511401131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fadb40c3540 con 0x7fadb413fe10 2024-04-25T01:38:15.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.758+0000 7fadaaffd640 1 -- 172.21.15.112:0/3511401131 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1563 (secure 0 0 0) 0x7fadb0058010 con 0x7fadb413fe10 2024-04-25T01:38:15.737 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":98,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:38:15.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.766+0000 7fada8ff9640 1 -- 172.21.15.112:0/3511401131 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fad9806be30 msgr2=0x7fad9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:15.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.766+0000 7fada8ff9640 1 --2- 172.21.15.112:0/3511401131 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fad9806be30 0x7fad9806e2f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fadb406de40 tx=0x7fadac009040 comp rx=0 tx=0).stop 2024-04-25T01:38:15.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.766+0000 7fada8ff9640 1 -- 172.21.15.112:0/3511401131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadb413fe10 msgr2=0x7fadb40ab910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:15.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.766+0000 7fada8ff9640 1 --2- 172.21.15.112:0/3511401131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadb413fe10 0x7fadb40ab910 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fadb406cb80 tx=0x7fadb0009850 comp rx=0 tx=0).stop 2024-04-25T01:38:15.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.767+0000 7fada8ff9640 1 -- 172.21.15.112:0/3511401131 shutdown_connections 2024-04-25T01:38:15.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.767+0000 7fada8ff9640 1 --2- 172.21.15.112:0/3511401131 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fad9806be30 0x7fad9806e2f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.767+0000 7fada8ff9640 1 --2- 172.21.15.112:0/3511401131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadb413fe10 0x7fadb40ab910 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.767+0000 7fada8ff9640 1 --2- 172.21.15.112:0/3511401131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadb4148d60 0x7fadb40ab3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.767+0000 7fada8ff9640 1 --2- 172.21.15.112:0/3511401131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fada0004800 0x7fadb4150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.767+0000 7fada8ff9640 1 -- 172.21.15.112:0/3511401131 >> 172.21.15.112:0/3511401131 conn(0x7fadb40c3b80 msgr2=0x7fadb414d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:15.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.768+0000 7fada8ff9640 1 -- 172.21.15.112:0/3511401131 shutdown_connections 2024-04-25T01:38:15.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:15.768+0000 7fada8ff9640 1 -- 172.21.15.112:0/3511401131 wait complete. 2024-04-25T01:38:18.768 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:38:18.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.950+0000 7faf8bd09640 1 Processor -- start 2024-04-25T01:38:18.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.951+0000 7faf8bd09640 1 -- start start 2024-04-25T01:38:18.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.952+0000 7faf892a4640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7faf840c2090 0x7faf8406ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32864/0 (socket says 172.21.15.112:32864) 2024-04-25T01:38:18.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.952+0000 7faf892a4640 1 -- 172.21.15.112:0/2837064266 learned_addr learned my addr 172.21.15.112:0/2837064266 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:38:18.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.952+0000 7faf8bd09640 1 --2- 172.21.15.112:0/2837064266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf8406f050 0x7faf841522f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:18.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.952+0000 7faf8bd09640 1 -- 172.21.15.112:0/2837064266 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf84071aa0 con 0x7faf840c2090 2024-04-25T01:38:18.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.952+0000 7faf8bd09640 1 -- 172.21.15.112:0/2837064266 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf84071d80 con 0x7faf840ac230 2024-04-25T01:38:18.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.952+0000 7faf8bd09640 1 -- 172.21.15.112:0/2837064266 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faf84152830 con 0x7faf8406f050 2024-04-25T01:38:18.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.952+0000 7faf8a2a6640 1 --2- 172.21.15.112:0/2837064266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf8406f050 0x7faf841522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:18.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.952+0000 7faf88aa3640 1 -- 172.21.15.112:0/2837064266 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 437143869 0 0) 0x7faf84071aa0 con 0x7faf840c2090 2024-04-25T01:38:18.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.953+0000 7faf88aa3640 1 -- 172.21.15.112:0/2837064266 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf70003520 con 0x7faf840c2090 2024-04-25T01:38:18.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.953+0000 7faf88aa3640 1 -- 172.21.15.112:0/2837064266 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1110377776 0 0) 0x7faf70003520 con 0x7faf840c2090 2024-04-25T01:38:18.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.953+0000 7faf88aa3640 1 -- 172.21.15.112:0/2837064266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf8406f050 msgr2=0x7faf841522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:38:18.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.953+0000 7faf88aa3640 1 --2- 172.21.15.112:0/2837064266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf8406f050 0x7faf841522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:18.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.953+0000 7faf88aa3640 1 -- 172.21.15.112:0/2837064266 >> v1:172.21.15.187:6789/0 conn(0x7faf840ac230 legacy=0x7faf840c1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:18.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.953+0000 7faf88aa3640 1 -- 172.21.15.112:0/2837064266 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf84072060 con 0x7faf840c2090 2024-04-25T01:38:18.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.953+0000 7faf88aa3640 1 -- 172.21.15.112:0/2837064266 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7faf74004e60 con 0x7faf840c2090 2024-04-25T01:38:18.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.953+0000 7faf88aa3640 1 -- 172.21.15.112:0/2837064266 >> v1:172.21.15.112:6789/0 conn(0x7faf840c2090 legacy=0x7faf8406ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:18.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.953+0000 7faf88aa3640 1 --2- 172.21.15.112:0/2837064266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf700039e0 0x7faf70003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.953+0000 7faf88aa3640 1 --2- 172.21.15.112:0/2837064266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf70004800 0x7faf70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.953+0000 7faf88aa3640 1 --2- 172.21.15.112:0/2837064266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf8406f050 0x7faf7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.953+0000 7faf88aa3640 1 -- 172.21.15.112:0/2837064266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faf84152830 con 0x7faf70004800 2024-04-25T01:38:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.953+0000 7faf88aa3640 1 -- 172.21.15.112:0/2837064266 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faf70005520 con 0x7faf700039e0 2024-04-25T01:38:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.953+0000 7faf88aa3640 1 -- 172.21.15.112:0/2837064266 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faf7000bf00 con 0x7faf8406f050 2024-04-25T01:38:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.954+0000 7faf89aa5640 1 --2- 172.21.15.112:0/2837064266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf700039e0 0x7faf70003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.954+0000 7faf8a2a6640 1 --2- 172.21.15.112:0/2837064266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf8406f050 0x7faf7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.954+0000 7faf892a4640 1 --2- 172.21.15.112:0/2837064266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf70004800 0x7faf70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.956+0000 7faf89aa5640 1 -- 172.21.15.112:0/2837064266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf8406f050 msgr2=0x7faf7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:38:18.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.956+0000 7faf89aa5640 1 --2- 172.21.15.112:0/2837064266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf8406f050 0x7faf7000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:18.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.956+0000 7faf89aa5640 1 -- 172.21.15.112:0/2837064266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf70004800 msgr2=0x7faf70004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:38:18.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.956+0000 7faf89aa5640 1 --2- 172.21.15.112:0/2837064266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf70004800 0x7faf70004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:18.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.956+0000 7faf89aa5640 1 -- 172.21.15.112:0/2837064266 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf84072060 con 0x7faf700039e0 2024-04-25T01:38:18.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.960+0000 7faf89aa5640 1 --2- 172.21.15.112:0/2837064266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf700039e0 0x7faf70003e20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faf80001750 tx=0x7faf70005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=555e5a9b75a5219d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:18.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.961+0000 7faf88aa3640 1 -- 172.21.15.112:0/2837064266 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf80001370 con 0x7faf700039e0 2024-04-25T01:38:18.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.961+0000 7faf88aa3640 1 -- 172.21.15.112:0/2837064266 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf800052d0 con 0x7faf700039e0 2024-04-25T01:38:18.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.961+0000 7faf88aa3640 1 -- 172.21.15.112:0/2837064266 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8002abf0 con 0x7faf700039e0 2024-04-25T01:38:18.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.962+0000 7faf8bd09640 1 -- 172.21.15.112:0/2837064266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf700039e0 msgr2=0x7faf70003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:18.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.962+0000 7faf8bd09640 1 --2- 172.21.15.112:0/2837064266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf700039e0 0x7faf70003e20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faf80001750 tx=0x7faf70005830 comp rx=0 tx=0).stop 2024-04-25T01:38:18.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.962+0000 7faf8a2a6640 1 -- 172.21.15.112:0/2837064266 reap_dead start 2024-04-25T01:38:18.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.964+0000 7faf8bd09640 1 -- 172.21.15.112:0/2837064266 shutdown_connections 2024-04-25T01:38:18.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.964+0000 7faf8bd09640 1 -- 172.21.15.112:0/2837064266 >> 172.21.15.112:0/2837064266 conn(0x7faf840c6f80 msgr2=0x7faf840c7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:18.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.964+0000 7faf8bd09640 1 -- 172.21.15.112:0/2837064266 shutdown_connections 2024-04-25T01:38:18.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.965+0000 7faf8bd09640 1 -- 172.21.15.112:0/2837064266 wait complete. 2024-04-25T01:38:18.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.967+0000 7faf8bd09640 1 Processor -- start 2024-04-25T01:38:18.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.967+0000 7faf8bd09640 1 -- start start 2024-04-25T01:38:18.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.968+0000 7faf8bd09640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf8406d4f0 0x7faf841e76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:18.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.968+0000 7faf8bd09640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf841558c0 0x7faf8414ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:18.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.968+0000 7faf8bd09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf8414b080 0x7faf841ea820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:18.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.968+0000 7faf8bd09640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faf84072b40 con 0x7faf8414b080 2024-04-25T01:38:18.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.968+0000 7faf8bd09640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faf84072620 con 0x7faf8406d4f0 2024-04-25T01:38:18.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.968+0000 7faf8bd09640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faf84072cc0 con 0x7faf841558c0 2024-04-25T01:38:18.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.969+0000 7faf89aa5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf8406d4f0 0x7faf841e76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:18.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.969+0000 7faf8a2a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf8414b080 0x7faf841ea820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:18.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.970+0000 7faf892a4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf841558c0 0x7faf8414ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:18.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.970+0000 7faf89aa5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf8406d4f0 0x7faf841e76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:34512/0 (socket says 172.21.15.112:34512) 2024-04-25T01:38:18.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.970+0000 7faf89aa5640 1 -- 172.21.15.112:0/2717240095 learned_addr learned my addr 172.21.15.112:0/2717240095 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:18.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.971+0000 7faf892a4640 1 -- 172.21.15.112:0/2717240095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf8406d4f0 msgr2=0x7faf841e76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:18.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.971+0000 7faf892a4640 1 --2- 172.21.15.112:0/2717240095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf8406d4f0 0x7faf841e76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:18.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.971+0000 7faf892a4640 1 -- 172.21.15.112:0/2717240095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf8414b080 msgr2=0x7faf841ea820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:18.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.971+0000 7faf892a4640 1 --2- 172.21.15.112:0/2717240095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf8414b080 0x7faf841ea820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:18.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.971+0000 7faf892a4640 1 -- 172.21.15.112:0/2717240095 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf84153ce0 con 0x7faf841558c0 2024-04-25T01:38:18.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.971+0000 7faf8a2a6640 1 --2- 172.21.15.112:0/2717240095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf8414b080 0x7faf841ea820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:38:18.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.972+0000 7faf892a4640 1 --2- 172.21.15.112:0/2717240095 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf841558c0 0x7faf8414ab40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faf7400f9e0 tx=0x7faf7400fea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:18.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.973+0000 7faf7a7fc640 1 -- 172.21.15.112:0/2717240095 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74015070 con 0x7faf841558c0 2024-04-25T01:38:18.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.973+0000 7faf8bd09640 1 -- 172.21.15.112:0/2717240095 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf84154a30 con 0x7faf841558c0 2024-04-25T01:38:18.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.973+0000 7faf8bd09640 1 -- 172.21.15.112:0/2717240095 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faf84154ff0 con 0x7faf841558c0 2024-04-25T01:38:18.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.975+0000 7faf7a7fc640 1 -- 172.21.15.112:0/2717240095 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf74009040 con 0x7faf841558c0 2024-04-25T01:38:18.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.977+0000 7faf7a7fc640 1 -- 172.21.15.112:0/2717240095 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf740113f0 con 0x7faf841558c0 2024-04-25T01:38:18.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.978+0000 7faf8bd09640 1 -- 172.21.15.112:0/2717240095 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf6c005290 con 0x7faf841558c0 2024-04-25T01:38:18.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.979+0000 7faf7a7fc640 1 -- 172.21.15.112:0/2717240095 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faf74018070 con 0x7faf841558c0 2024-04-25T01:38:18.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.979+0000 7faf7a7fc640 1 --2- 172.21.15.112:0/2717240095 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faf6806bf60 0x7faf6806e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:18.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.980+0000 7faf7a7fc640 1 -- 172.21.15.112:0/2717240095 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7faf7408b9c0 con 0x7faf841558c0 2024-04-25T01:38:18.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.980+0000 7faf89aa5640 1 --2- 172.21.15.112:0/2717240095 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faf6806bf60 0x7faf6806e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:18.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.981+0000 7faf89aa5640 1 --2- 172.21.15.112:0/2717240095 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faf6806bf60 0x7faf6806e420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7faf80001750 tx=0x7faf8002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:18.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:18.988+0000 7faf7a7fc640 1 -- 172.21.15.112:0/2717240095 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faf74008050 con 0x7faf841558c0 2024-04-25T01:38:19.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:19.509+0000 7faf8bd09640 1 -- 172.21.15.112:0/2717240095 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7faf6c005580 con 0x7faf841558c0 2024-04-25T01:38:19.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:19.512+0000 7faf7a7fc640 1 -- 172.21.15.112:0/2717240095 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1563 (secure 0 0 0) 0x7faf7405a040 con 0x7faf841558c0 2024-04-25T01:38:19.492 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":98,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:38:19.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:19.516+0000 7faf8bd09640 1 -- 172.21.15.112:0/2717240095 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faf6806bf60 msgr2=0x7faf6806e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:19.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:19.516+0000 7faf8bd09640 1 --2- 172.21.15.112:0/2717240095 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faf6806bf60 0x7faf6806e420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7faf80001750 tx=0x7faf8002e000 comp rx=0 tx=0).stop 2024-04-25T01:38:19.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:19.516+0000 7faf8bd09640 1 -- 172.21.15.112:0/2717240095 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf841558c0 msgr2=0x7faf8414ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:19.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:19.516+0000 7faf8bd09640 1 --2- 172.21.15.112:0/2717240095 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf841558c0 0x7faf8414ab40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faf7400f9e0 tx=0x7faf7400fea0 comp rx=0 tx=0).stop 2024-04-25T01:38:19.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:19.516+0000 7faf8bd09640 1 -- 172.21.15.112:0/2717240095 shutdown_connections 2024-04-25T01:38:19.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:19.516+0000 7faf8bd09640 1 --2- 172.21.15.112:0/2717240095 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faf6806bf60 0x7faf6806e420 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:19.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:19.516+0000 7faf8bd09640 1 --2- 172.21.15.112:0/2717240095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf8414b080 0x7faf841ea820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:19.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:19.516+0000 7faf8bd09640 1 --2- 172.21.15.112:0/2717240095 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf841558c0 0x7faf8414ab40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:19.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:19.516+0000 7faf8bd09640 1 --2- 172.21.15.112:0/2717240095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf8406d4f0 0x7faf841e76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:19.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:19.516+0000 7faf8bd09640 1 -- 172.21.15.112:0/2717240095 >> 172.21.15.112:0/2717240095 conn(0x7faf840c6f80 msgr2=0x7faf840aa870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:19.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:19.516+0000 7faf8bd09640 1 -- 172.21.15.112:0/2717240095 shutdown_connections 2024-04-25T01:38:19.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:19.516+0000 7faf8bd09640 1 -- 172.21.15.112:0/2717240095 wait complete. 2024-04-25T01:38:22.515 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:38:22.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.706+0000 7fc03f59e640 1 Processor -- start 2024-04-25T01:38:22.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.706+0000 7fc03f59e640 1 -- start start 2024-04-25T01:38:22.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.707+0000 7fc03f59e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc0380dbce0 0x7fc0380474a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:22.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.707+0000 7fc03f59e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0380df180 0x7fc038006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:22.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.707+0000 7fc03f59e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc03800bb50 con 0x7fc0380073e0 2024-04-25T01:38:22.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.707+0000 7fc03f59e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fc03800a420 con 0x7fc0380dbce0 2024-04-25T01:38:22.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.707+0000 7fc03f59e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc038048060 con 0x7fc0380df180 2024-04-25T01:38:22.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.708+0000 7fc03ed9d640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fc0380073e0 0x7fc03800a8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:45408/0 (socket says 172.21.15.112:45408) 2024-04-25T01:38:22.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.708+0000 7fc03ed9d640 1 -- 172.21.15.112:0/3375745284 learned_addr learned my addr 172.21.15.112:0/3375745284 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:38:22.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.708+0000 7fc03dd9b640 1 --2- 172.21.15.112:0/3375745284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0380df180 0x7fc038006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:22.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.708+0000 7fc03e59c640 1 --2- 172.21.15.112:0/3375745284 >> v2:172.21.15.112:3300/0 conn(0x7fc0380dbce0 0x7fc0380474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:22.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.709+0000 7fc03dd9b640 1 -- 172.21.15.112:0/3375745284 >> v2:172.21.15.112:3300/0 conn(0x7fc0380dbce0 msgr2=0x7fc0380474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:38:22.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.709+0000 7fc03dd9b640 1 --2- 172.21.15.112:0/3375745284 >> v2:172.21.15.112:3300/0 conn(0x7fc0380dbce0 0x7fc0380474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:22.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.709+0000 7fc03dd9b640 1 -- 172.21.15.112:0/3375745284 >> v1:172.21.15.187:6789/0 conn(0x7fc0380073e0 legacy=0x7fc03800a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:22.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.709+0000 7fc03dd9b640 1 -- 172.21.15.112:0/3375745284 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03800a0f0 con 0x7fc0380df180 2024-04-25T01:38:22.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.710+0000 7fc03dd9b640 1 --2- 172.21.15.112:0/3375745284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0380df180 0x7fc038006ea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc0300097a0 tx=0x7fc03002a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c2b63d3d8a0b5d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:22.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.711+0000 7fc03d59a640 1 -- 172.21.15.112:0/3375745284 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030037070 con 0x7fc0380df180 2024-04-25T01:38:22.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.711+0000 7fc03d59a640 1 -- 172.21.15.112:0/3375745284 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc03002ae40 con 0x7fc0380df180 2024-04-25T01:38:22.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.711+0000 7fc03d59a640 1 -- 172.21.15.112:0/3375745284 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030032660 con 0x7fc0380df180 2024-04-25T01:38:22.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.712+0000 7fc03f59e640 1 -- 172.21.15.112:0/3375745284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0380df180 msgr2=0x7fc038006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:22.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.712+0000 7fc03f59e640 1 --2- 172.21.15.112:0/3375745284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0380df180 0x7fc038006ea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc0300097a0 tx=0x7fc03002a0f0 comp rx=0 tx=0).stop 2024-04-25T01:38:22.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.712+0000 7fc03f59e640 1 -- 172.21.15.112:0/3375745284 shutdown_connections 2024-04-25T01:38:22.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.712+0000 7fc03f59e640 1 --2- 172.21.15.112:0/3375745284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0380df180 0x7fc038006ea0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:22.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.712+0000 7fc03f59e640 1 --2- 172.21.15.112:0/3375745284 >> v2:172.21.15.112:3300/0 conn(0x7fc0380dbce0 0x7fc0380474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:22.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.712+0000 7fc03f59e640 1 -- 172.21.15.112:0/3375745284 >> 172.21.15.112:0/3375745284 conn(0x7fc038048250 msgr2=0x7fc038048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:22.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.712+0000 7fc03f59e640 1 -- 172.21.15.112:0/3375745284 shutdown_connections 2024-04-25T01:38:22.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.712+0000 7fc03f59e640 1 -- 172.21.15.112:0/3375745284 wait complete. 2024-04-25T01:38:22.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.714+0000 7fc03f59e640 1 Processor -- start 2024-04-25T01:38:22.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.714+0000 7fc03f59e640 1 -- start start 2024-04-25T01:38:22.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.715+0000 7fc03f59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0380073e0 0x7fc03804aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:22.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.715+0000 7fc03f59e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0380dbce0 0x7fc0380ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:22.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.715+0000 7fc03f59e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0380df180 0x7fc0380cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:22.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.715+0000 7fc03f59e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0380dc0c0 con 0x7fc0380073e0 2024-04-25T01:38:22.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.715+0000 7fc03f59e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc038047d80 con 0x7fc0380df180 2024-04-25T01:38:22.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.715+0000 7fc03f59e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc0380df5c0 con 0x7fc0380dbce0 2024-04-25T01:38:22.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.715+0000 7fc03e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0380073e0 0x7fc03804aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:22.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.715+0000 7fc03e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0380073e0 0x7fc03804aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45348/0 (socket says 172.21.15.112:45348) 2024-04-25T01:38:22.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.715+0000 7fc03dd9b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0380dbce0 0x7fc0380ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:22.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.716+0000 7fc03e59c640 1 -- 172.21.15.112:0/1451584464 learned_addr learned my addr 172.21.15.112:0/1451584464 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:22.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.716+0000 7fc03ed9d640 1 --2- 172.21.15.112:0/1451584464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0380df180 0x7fc0380cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:22.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.716+0000 7fc03dd9b640 1 -- 172.21.15.112:0/1451584464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0380df180 msgr2=0x7fc0380cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:22.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.716+0000 7fc03dd9b640 1 --2- 172.21.15.112:0/1451584464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0380df180 0x7fc0380cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:22.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.716+0000 7fc03dd9b640 1 -- 172.21.15.112:0/1451584464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0380073e0 msgr2=0x7fc03804aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:22.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.716+0000 7fc03dd9b640 1 --2- 172.21.15.112:0/1451584464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0380073e0 0x7fc03804aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:22.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.716+0000 7fc03dd9b640 1 -- 172.21.15.112:0/1451584464 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0380c8e10 con 0x7fc0380dbce0 2024-04-25T01:38:22.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.716+0000 7fc03dd9b640 1 --2- 172.21.15.112:0/1451584464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0380dbce0 0x7fc0380ccc80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc030031000 tx=0x7fc0300046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:22.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.717+0000 7fc02effd640 1 -- 172.21.15.112:0/1451584464 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030041020 con 0x7fc0380dbce0 2024-04-25T01:38:22.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.717+0000 7fc02effd640 1 -- 172.21.15.112:0/1451584464 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc03003c670 con 0x7fc0380dbce0 2024-04-25T01:38:22.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.717+0000 7fc02effd640 1 -- 172.21.15.112:0/1451584464 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030037040 con 0x7fc0380dbce0 2024-04-25T01:38:22.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.718+0000 7fc03f59e640 1 -- 172.21.15.112:0/1451584464 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0380c90f0 con 0x7fc0380dbce0 2024-04-25T01:38:22.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.718+0000 7fc03f59e640 1 -- 172.21.15.112:0/1451584464 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0380cb120 con 0x7fc0380dbce0 2024-04-25T01:38:22.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.721+0000 7fc03f59e640 1 -- 172.21.15.112:0/1451584464 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc024005290 con 0x7fc0380dbce0 2024-04-25T01:38:22.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.721+0000 7fc02effd640 1 -- 172.21.15.112:0/1451584464 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc030044020 con 0x7fc0380dbce0 2024-04-25T01:38:22.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.722+0000 7fc02effd640 1 --2- 172.21.15.112:0/1451584464 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc02006beb0 0x7fc02006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:22.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.722+0000 7fc02effd640 1 -- 172.21.15.112:0/1451584464 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fc0300ab640 con 0x7fc0380dbce0 2024-04-25T01:38:22.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.722+0000 7fc03e59c640 1 --2- 172.21.15.112:0/1451584464 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc02006beb0 0x7fc02006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:22.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.723+0000 7fc03e59c640 1 --2- 172.21.15.112:0/1451584464 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc02006beb0 0x7fc02006e370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc040068950 tx=0x7fc040076b30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:22.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:22.733+0000 7fc02effd640 1 -- 172.21.15.112:0/1451584464 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc03004b030 con 0x7fc0380dbce0 2024-04-25T01:38:23.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.242+0000 7fc03f59e640 1 -- 172.21.15.112:0/1451584464 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc024005850 con 0x7fc0380dbce0 2024-04-25T01:38:23.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.244+0000 7fc02effd640 1 -- 172.21.15.112:0/1451584464 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7fc030032e20 con 0x7fc0380dbce0 2024-04-25T01:38:23.225 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":100,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:38:23.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.248+0000 7fc03f59e640 1 -- 172.21.15.112:0/1451584464 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc02006beb0 msgr2=0x7fc02006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.248+0000 7fc03f59e640 1 --2- 172.21.15.112:0/1451584464 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc02006beb0 0x7fc02006e370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc040068950 tx=0x7fc040076b30 comp rx=0 tx=0).stop 2024-04-25T01:38:23.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.248+0000 7fc03f59e640 1 -- 172.21.15.112:0/1451584464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0380dbce0 msgr2=0x7fc0380ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.248+0000 7fc03f59e640 1 --2- 172.21.15.112:0/1451584464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0380dbce0 0x7fc0380ccc80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc030031000 tx=0x7fc0300046c0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.248+0000 7fc03f59e640 1 -- 172.21.15.112:0/1451584464 shutdown_connections 2024-04-25T01:38:23.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.248+0000 7fc03f59e640 1 --2- 172.21.15.112:0/1451584464 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc02006beb0 0x7fc02006e370 secure :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc040068950 tx=0x7fc040076b30 comp rx=0 tx=0).stop 2024-04-25T01:38:23.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.248+0000 7fc03f59e640 1 --2- 172.21.15.112:0/1451584464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0380df180 0x7fc0380cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.248+0000 7fc03f59e640 1 --2- 172.21.15.112:0/1451584464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0380dbce0 0x7fc0380ccc80 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc030031000 tx=0x7fc0300046c0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.248+0000 7fc03f59e640 1 --2- 172.21.15.112:0/1451584464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0380073e0 0x7fc03804aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.248+0000 7fc03f59e640 1 -- 172.21.15.112:0/1451584464 >> 172.21.15.112:0/1451584464 conn(0x7fc038048250 msgr2=0x7fc0380dfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:23.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.252+0000 7fc03f59e640 1 -- 172.21.15.112:0/1451584464 shutdown_connections 2024-04-25T01:38:23.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.252+0000 7fc03f59e640 1 -- 172.21.15.112:0/1451584464 wait complete. 2024-04-25T01:38:23.253 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:38:23.253 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:38:23.253 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:38:23.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.446+0000 7f9d01d09640 1 Processor -- start 2024-04-25T01:38:23.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.447+0000 7f9d01d09640 1 -- start start 2024-04-25T01:38:23.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.448+0000 7f9d01d09640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cfc070fd0 0x7f9cfc0ac000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.448+0000 7f9d01d09640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9cfc072ce0 0x7f9cfc0c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.448+0000 7f9d01d09640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cfc15ac20 con 0x7f9cfc1571b0 2024-04-25T01:38:23.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.448+0000 7f9d01d09640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f9cfc0ab2d0 con 0x7f9cfc072ce0 2024-04-25T01:38:23.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.448+0000 7f9d01d09640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9cfc0abb70 con 0x7f9cfc070fd0 2024-04-25T01:38:23.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.448+0000 7f9cfaffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9cfc072ce0 0x7f9cfc0c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:23.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.448+0000 7f9cfaffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9cfc072ce0 0x7f9cfc0c1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45360/0 (socket says 172.21.15.112:45360) 2024-04-25T01:38:23.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.448+0000 7f9cfaffd640 1 -- 172.21.15.112:0/3997697934 learned_addr learned my addr 172.21.15.112:0/3997697934 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:23.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.449+0000 7f9cfaffd640 1 -- 172.21.15.112:0/3997697934 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cfc070fd0 msgr2=0x7f9cfc0ac000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:38:23.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.449+0000 7f9cfaffd640 1 --2- 172.21.15.112:0/3997697934 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cfc070fd0 0x7f9cfc0ac000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.449+0000 7f9cfaffd640 1 -- 172.21.15.112:0/3997697934 >> v1:172.21.15.112:6789/0 conn(0x7f9cfc1571b0 legacy=0x7f9cfc15d3b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:38:23.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.449+0000 7f9cfaffd640 1 -- 172.21.15.112:0/3997697934 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cfc0ab520 con 0x7f9cfc072ce0 2024-04-25T01:38:23.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.450+0000 7f9cfaffd640 1 --2- 172.21.15.112:0/3997697934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cfc072ce0 0x7f9cfc0c1a80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400b1a0 tx=0x7f9cf402a940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f6e96fcad58785 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:23.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.453+0000 7f9cfa7fc640 1 -- 172.21.15.112:0/3997697934 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf40303f0 con 0x7f9cfc072ce0 2024-04-25T01:38:23.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.453+0000 7f9cfa7fc640 1 -- 172.21.15.112:0/3997697934 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf4035070 con 0x7f9cfc072ce0 2024-04-25T01:38:23.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.454+0000 7f9d01d09640 1 -- 172.21.15.112:0/3997697934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cfc072ce0 msgr2=0x7f9cfc0c1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.454+0000 7f9d01d09640 1 --2- 172.21.15.112:0/3997697934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cfc072ce0 0x7f9cfc0c1a80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400b1a0 tx=0x7f9cf402a940 comp rx=0 tx=0).stop 2024-04-25T01:38:23.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.454+0000 7f9d01d09640 1 -- 172.21.15.112:0/3997697934 shutdown_connections 2024-04-25T01:38:23.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.454+0000 7f9d01d09640 1 --2- 172.21.15.112:0/3997697934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cfc072ce0 0x7f9cfc0c1a80 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400b1a0 tx=0x7f9cf402a940 comp rx=0 tx=0).stop 2024-04-25T01:38:23.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.454+0000 7f9d01d09640 1 --2- 172.21.15.112:0/3997697934 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cfc070fd0 0x7f9cfc0ac000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.454+0000 7f9d01d09640 1 -- 172.21.15.112:0/3997697934 >> 172.21.15.112:0/3997697934 conn(0x7f9cfc0c7080 msgr2=0x7f9cfc0c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:23.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.461+0000 7f9d01d09640 1 -- 172.21.15.112:0/3997697934 shutdown_connections 2024-04-25T01:38:23.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.463+0000 7f9d01d09640 1 -- 172.21.15.112:0/3997697934 wait complete. 2024-04-25T01:38:23.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.464+0000 7f9d01d09640 1 Processor -- start 2024-04-25T01:38:23.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.465+0000 7f9d01d09640 1 -- start start 2024-04-25T01:38:23.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.466+0000 7f9d01d09640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cfc070fd0 0x7f9cfc0bc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.466+0000 7f9d01d09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cfc1571b0 0x7f9cfc209fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.466+0000 7f9d01d09640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cfc159220 0x7f9cfc176c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.466+0000 7f9d01d09640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc15a840 con 0x7f9cfc1571b0 2024-04-25T01:38:23.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.466+0000 7f9d01d09640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc073140 con 0x7f9cfc070fd0 2024-04-25T01:38:23.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.466+0000 7f9d01d09640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9cfc15a9c0 con 0x7f9cfc159220 2024-04-25T01:38:23.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.467+0000 7f9cfaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cfc1571b0 0x7f9cfc209fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:23.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.467+0000 7f9cfaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cfc1571b0 0x7f9cfc209fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45362/0 (socket says 172.21.15.112:45362) 2024-04-25T01:38:23.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.467+0000 7f9cfaffd640 1 -- 172.21.15.112:0/3035781892 learned_addr learned my addr 172.21.15.112:0/3035781892 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:23.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.467+0000 7f9cfaffd640 1 -- 172.21.15.112:0/3035781892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cfc159220 msgr2=0x7f9cfc176c20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:38:23.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.467+0000 7f9cfaffd640 1 --2- 172.21.15.112:0/3035781892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cfc159220 0x7f9cfc176c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.467+0000 7f9cfaffd640 1 -- 172.21.15.112:0/3035781892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cfc070fd0 msgr2=0x7f9cfc0bc0a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:38:23.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.467+0000 7f9cfaffd640 1 --2- 172.21.15.112:0/3035781892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cfc070fd0 0x7f9cfc0bc0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.467+0000 7f9cfaffd640 1 -- 172.21.15.112:0/3035781892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cfc1451a0 con 0x7f9cfc1571b0 2024-04-25T01:38:23.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.468+0000 7f9cfaffd640 1 --2- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cfc1571b0 0x7f9cfc209fa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4032040 tx=0x7f9cf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:23.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.469+0000 7f9ce3fff640 1 -- 172.21.15.112:0/3035781892 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4004240 con 0x7f9cfc1571b0 2024-04-25T01:38:23.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.469+0000 7f9ce3fff640 1 -- 172.21.15.112:0/3035781892 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf403e070 con 0x7f9cfc1571b0 2024-04-25T01:38:23.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.469+0000 7f9ce3fff640 1 -- 172.21.15.112:0/3035781892 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4004620 con 0x7f9cfc1571b0 2024-04-25T01:38:23.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.470+0000 7f9d01d09640 1 -- 172.21.15.112:0/3035781892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cfc144210 con 0x7f9cfc1571b0 2024-04-25T01:38:23.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.470+0000 7f9d01d09640 1 -- 172.21.15.112:0/3035781892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cfc1447d0 con 0x7f9cfc1571b0 2024-04-25T01:38:23.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.472+0000 7f9d01d09640 1 --2- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cdc002a70 0x7f9cdc004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.473+0000 7f9ce3fff640 1 -- 172.21.15.112:0/3035781892 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9cf4043470 con 0x7f9cfc1571b0 2024-04-25T01:38:23.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.475+0000 7f9cfb7fe640 1 --2- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cdc002a70 0x7f9cdc004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:23.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.475+0000 7f9d01d09640 1 -- 172.21.15.112:0/3035781892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc007ec0 con 0x7f9cdc002a70 2024-04-25T01:38:23.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.476+0000 7f9cfb7fe640 1 -- 172.21.15.112:0/3035781892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9cdc0080c0 con 0x7f9cdc002a70 2024-04-25T01:38:23.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.476+0000 7f9cfb7fe640 1 --2- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cdc002a70 0x7f9cdc004f30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00b680 tx=0x7f9cec00bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:23.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.477+0000 7f9ce3fff640 1 --2- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ce806bf30 0x7f9ce806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.477+0000 7f9cfbfff640 1 --2- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ce806bf30 0x7f9ce806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:23.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.478+0000 7f9cfbfff640 1 --2- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ce806bf30 0x7f9ce806e3f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009640 tx=0x7f9cf0006bd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:23.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.478+0000 7f9ce3fff640 1 -- 172.21.15.112:0/3035781892 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f9cf40ac1e0 con 0x7f9cfc1571b0 2024-04-25T01:38:23.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.478+0000 7f9ce3fff640 1 -- 172.21.15.112:0/3035781892 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec019070 con 0x7f9cdc002a70 2024-04-25T01:38:23.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.478+0000 7f9ce3fff640 1 -- 172.21.15.112:0/3035781892 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9cec00c720 con 0x7f9cdc002a70 2024-04-25T01:38:23.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.484+0000 7f9ce3fff640 1 -- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cdc002a70 msgr2=0x7f9cdc004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.484+0000 7f9ce3fff640 1 --2- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cdc002a70 0x7f9cdc004f30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00b680 tx=0x7f9cec00bb40 comp rx=0 tx=0).stop 2024-04-25T01:38:23.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.484+0000 7f9ce3fff640 1 -- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cdc002a70 msgr2=0x7f9cdc004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:38:23.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.484+0000 7f9ce3fff640 1 --2- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cdc002a70 0x7f9cdc004f30 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00b680 tx=0x7f9cec00bb40 comp rx=0 tx=0).stop 2024-04-25T01:38:23.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.491+0000 7f9d01d09640 1 --2- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cdc006f00 0x7f9cdc007390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.491+0000 7f9d01d09640 1 -- 172.21.15.112:0/3035781892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc00a4a0 con 0x7f9cdc006f00 2024-04-25T01:38:23.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.492+0000 7f9cfb7fe640 1 --2- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cdc006f00 0x7f9cdc007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:23.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.492+0000 7f9cfb7fe640 1 -- 172.21.15.112:0/3035781892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9cdc0078d0 con 0x7f9cdc006f00 2024-04-25T01:38:23.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.493+0000 7f9cfb7fe640 1 --2- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cdc006f00 0x7f9cdc007390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00b680 tx=0x7f9cec004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:23.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.494+0000 7f9ce3fff640 1 -- 172.21.15.112:0/3035781892 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec019040 con 0x7f9cdc006f00 2024-04-25T01:38:23.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.494+0000 7f9ce3fff640 1 -- 172.21.15.112:0/3035781892 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4902 (secure 0 0 0) 0x7f9cec01c2d0 con 0x7f9cdc006f00 2024-04-25T01:38:23.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.494+0000 7f9ce3fff640 1 -- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cdc006f00 msgr2=0x7f9cdc007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.475 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:38:23.475 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:38:23.475 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:38:23.475 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:38:23.475 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 100, 2024-04-25T01:38:23.475 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 154307, 2024-04-25T01:38:23.475 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:38:23.475 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:38:23.475 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:38:23.475 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:38:23.475 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:23.475 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 3, 2024-04-25T01:38:23.475 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:38:23.476 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:38:23.476 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:38:23.476 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:23.476 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:23.476 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:23.476 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:23.476 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:23.476 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:23.476 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:23.476 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:23.476 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:23.476 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:23.476 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:23.476 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:23.476 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:38:23.477 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:38:23.477 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:23.477 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:23.477 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:23.477 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:23.477 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:23.477 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:23.477 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:23.477 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:23.477 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:23.477 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:23.477 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:23.477 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:23.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.477 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:38:23.478 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:38:23.478 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:38:23.478 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:38:23.478 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:38:23.478 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:38:23.478 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:38:23.478 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:38:23.478 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:38:23.478 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:38:23.478 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:38:23.478 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:38:23.478 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:38:23.478 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:38:23.478 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:38:23.478 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:38:23.479 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:38:23.479 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:23.479 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:23.479 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:23.479 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:23.479 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:23.479 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:23.479 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:23.479 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:23.479 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:23.479 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:23.479 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:23.479 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:23.479 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:38:23.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.480 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:38:23.480 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.480 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:38:23.480 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:38:23.480 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:23.480 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:23.480 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.480 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:23.480 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:38:23.480 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:23.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.480 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.480 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:23.480 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:38:23.481 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:23.481 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.481 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:23.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.481 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:38:23.481 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:38:23.481 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:23.481 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:23.481 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:23.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.481 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.481 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:38:23.481 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:38:23.481 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:23.481 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:23.482 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.482 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:23.482 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:38:23.482 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:23.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.482 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.482 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:23.482 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:38:23.482 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:23.482 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.482 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:23.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.482 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:38:23.482 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:38:23.482 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:23.483 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:23.483 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:23.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.483 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.483 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:38:23.483 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:38:23.483 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:23.483 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:23.483 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.483 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:23.483 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:38:23.483 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:23.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.483 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.483 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:23.483 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:38:23.484 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:23.484 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.484 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:23.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.484 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:38:23.484 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:38:23.484 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:23.484 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:23.484 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:23.484 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.484 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:23.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.484 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:38:23.485 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:38:23.485 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.485 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:23.485 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:23.485 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:38:23.485 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.485 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:23.485 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:38:23.485 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.485 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:23.485 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:23.485 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:38:23.485 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.486 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:23.486 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:38:23.486 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.486 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:23.486 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:23.486 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:38:23.486 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.486 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:23.486 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:38:23.486 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.486 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:23.486 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:23.486 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 6 2024-04-25T01:38:23.486 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.486 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:23.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.487 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:38:23.487 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:38:23.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.494+0000 7f9ce3fff640 1 --2- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cdc006f00 0x7f9cdc007390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00b680 tx=0x7f9cec004700 comp rx=0 tx=0).stop 2024-04-25T01:38:23.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.494+0000 7f9ce3fff640 1 -- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cdc006f00 msgr2=0x7f9cdc007390 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:38:23.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.494+0000 7f9ce3fff640 1 --2- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cdc006f00 0x7f9cdc007390 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00b680 tx=0x7f9cec004700 comp rx=0 tx=0).stop 2024-04-25T01:38:23.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.495+0000 7f9d01d09640 1 -- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ce806bf30 msgr2=0x7f9ce806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.495+0000 7f9d01d09640 1 --2- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ce806bf30 0x7f9ce806e3f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009640 tx=0x7f9cf0006bd0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.495+0000 7f9d01d09640 1 -- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cfc1571b0 msgr2=0x7f9cfc209fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.495+0000 7f9d01d09640 1 --2- 172.21.15.112:0/3035781892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cfc1571b0 0x7f9cfc209fa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4032040 tx=0x7f9cf4004a40 comp rx=0 tx=0).stop 2024-04-25T01:38:23.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.495+0000 7f9cfbfff640 1 -- 172.21.15.112:0/3035781892 reap_dead start 2024-04-25T01:38:23.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.495+0000 7f9cfbfff640 1 -- 172.21.15.112:0/3035781892 reap_dead start 2024-04-25T01:38:23.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.495+0000 7f9d01d09640 1 -- 172.21.15.112:0/3035781892 shutdown_connections 2024-04-25T01:38:23.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.495+0000 7f9d01d09640 1 -- 172.21.15.112:0/3035781892 >> 172.21.15.112:0/3035781892 conn(0x7f9cfc0c7080 msgr2=0x7f9cfc156220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:23.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.501+0000 7f9d01d09640 1 -- 172.21.15.112:0/3035781892 shutdown_connections 2024-04-25T01:38:23.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.502+0000 7f9d01d09640 1 -- 172.21.15.112:0/3035781892 wait complete. 2024-04-25T01:38:23.503 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:38:23.503 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:38:23.503 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:38:23.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.688+0000 7f7ee51d5640 1 Processor -- start 2024-04-25T01:38:23.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.688+0000 7f7ee51d5640 1 -- start start 2024-04-25T01:38:23.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.689+0000 7f7ee51d5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ee0152d60 0x7f7ee0158da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.689+0000 7f7ee51d5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7ee0155de0 0x7f7ee01561f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.689+0000 7f7ee51d5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ee006f2d0 con 0x7f7ee00c3400 2024-04-25T01:38:23.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.689+0000 7f7ee51d5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7ee0156730 con 0x7f7ee0155de0 2024-04-25T01:38:23.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.689+0000 7f7ee51d5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee0159cd0 con 0x7f7ee0152d60 2024-04-25T01:38:23.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.689+0000 7f7edf59e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7ee0155de0 0x7f7ee01561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:23.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.689+0000 7f7eded9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ee0152d60 0x7f7ee0158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:23.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.689+0000 7f7eded9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ee0152d60 0x7f7ee0158da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:49948/0 (socket says 172.21.15.112:49948) 2024-04-25T01:38:23.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.689+0000 7f7ede59c640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f7ee00c3400 0x7f7ee00c3b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:45412/0 (socket says 172.21.15.112:45412) 2024-04-25T01:38:23.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.689+0000 7f7eded9d640 1 -- 172.21.15.112:0/2061644633 learned_addr learned my addr 172.21.15.112:0/2061644633 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:23.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.690+0000 7f7edf59e640 1 -- 172.21.15.112:0/2061644633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ee0152d60 msgr2=0x7f7ee0158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:38:23.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.690+0000 7f7edf59e640 1 --2- 172.21.15.112:0/2061644633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ee0152d60 0x7f7ee0158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.690+0000 7f7edf59e640 1 -- 172.21.15.112:0/2061644633 >> v1:172.21.15.187:6789/0 conn(0x7f7ee00c3400 legacy=0x7f7ee00c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.690+0000 7f7eddd9b640 1 -- 172.21.15.112:0/2061644633 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3605762106 0 0) 0x7f7ee006f2d0 con 0x7f7ee00c3400 2024-04-25T01:38:23.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.690+0000 7f7edf59e640 1 -- 172.21.15.112:0/2061644633 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee0159430 con 0x7f7ee0155de0 2024-04-25T01:38:23.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.691+0000 7f7eded9d640 1 --2- 172.21.15.112:0/2061644633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ee0152d60 0x7f7ee0158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:38:23.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.692+0000 7f7edf59e640 1 --2- 172.21.15.112:0/2061644633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ee0155de0 0x7f7ee01561f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80097a0 tx=0x7f7ec802a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e037403d15eebc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:23.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.692+0000 7f7eddd9b640 1 -- 172.21.15.112:0/2061644633 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8037070 con 0x7f7ee0155de0 2024-04-25T01:38:23.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.692+0000 7f7eddd9b640 1 -- 172.21.15.112:0/2061644633 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec802adb0 con 0x7f7ee0155de0 2024-04-25T01:38:23.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.692+0000 7f7eddd9b640 1 -- 172.21.15.112:0/2061644633 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8037070 con 0x7f7ee0155de0 2024-04-25T01:38:23.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.693+0000 7f7ee51d5640 1 -- 172.21.15.112:0/2061644633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ee0155de0 msgr2=0x7f7ee01561f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.693+0000 7f7ee51d5640 1 --2- 172.21.15.112:0/2061644633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ee0155de0 0x7f7ee01561f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80097a0 tx=0x7f7ec802a060 comp rx=0 tx=0).stop 2024-04-25T01:38:23.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.694+0000 7f7ee51d5640 1 -- 172.21.15.112:0/2061644633 shutdown_connections 2024-04-25T01:38:23.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.694+0000 7f7ee51d5640 1 --2- 172.21.15.112:0/2061644633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ee0155de0 0x7f7ee01561f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.694+0000 7f7ee51d5640 1 --2- 172.21.15.112:0/2061644633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ee0152d60 0x7f7ee0158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.694+0000 7f7ee51d5640 1 -- 172.21.15.112:0/2061644633 >> 172.21.15.112:0/2061644633 conn(0x7f7ee006b160 msgr2=0x7f7ee006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:23.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.694+0000 7f7ee51d5640 1 -- 172.21.15.112:0/2061644633 shutdown_connections 2024-04-25T01:38:23.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.694+0000 7f7ee51d5640 1 -- 172.21.15.112:0/2061644633 wait complete. 2024-04-25T01:38:23.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.696+0000 7f7ee51d5640 1 Processor -- start 2024-04-25T01:38:23.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.696+0000 7f7ee51d5640 1 -- start start 2024-04-25T01:38:23.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.697+0000 7f7ee51d5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ee00c3400 0x7f7ee0146bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.697+0000 7f7ee51d5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ee0152d60 0x7f7ee01eb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.697+0000 7f7ee51d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ee0155de0 0x7f7ee01eb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.697+0000 7f7ee51d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee01558c0 con 0x7f7ee0155de0 2024-04-25T01:38:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.697+0000 7f7ee51d5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee01599f0 con 0x7f7ee0152d60 2024-04-25T01:38:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.697+0000 7f7ee51d5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee0153140 con 0x7f7ee00c3400 2024-04-25T01:38:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.698+0000 7f7eded9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ee00c3400 0x7f7ee0146bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.698+0000 7f7eded9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ee00c3400 0x7f7ee0146bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:49962/0 (socket says 172.21.15.112:49962) 2024-04-25T01:38:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.698+0000 7f7eded9d640 1 -- 172.21.15.112:0/1890620507 learned_addr learned my addr 172.21.15.112:0/1890620507 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.698+0000 7f7ede59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ee0152d60 0x7f7ee01eb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.698+0000 7f7eded9d640 1 -- 172.21.15.112:0/1890620507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ee0152d60 msgr2=0x7f7ee01eb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.698+0000 7f7eded9d640 1 --2- 172.21.15.112:0/1890620507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ee0152d60 0x7f7ee01eb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.698+0000 7f7eded9d640 1 -- 172.21.15.112:0/1890620507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ee0155de0 msgr2=0x7f7ee01eb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.698+0000 7f7eded9d640 1 --2- 172.21.15.112:0/1890620507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ee0155de0 0x7f7ee01eb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.698+0000 7f7eded9d640 1 -- 172.21.15.112:0/1890620507 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee014d940 con 0x7f7ee00c3400 2024-04-25T01:38:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.698+0000 7f7ede59c640 1 --2- 172.21.15.112:0/1890620507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ee0152d60 0x7f7ee01eb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:38:23.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.702+0000 7f7eded9d640 1 --2- 172.21.15.112:0/1890620507 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ee00c3400 0x7f7ee0146bf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000e9e0 tx=0x7f7ed000eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:23.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.702+0000 7f7ed77fe640 1 -- 172.21.15.112:0/1890620507 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0017070 con 0x7f7ee00c3400 2024-04-25T01:38:23.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.703+0000 7f7ed77fe640 1 -- 172.21.15.112:0/1890620507 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed000c9d0 con 0x7f7ee00c3400 2024-04-25T01:38:23.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.703+0000 7f7ed77fe640 1 -- 172.21.15.112:0/1890620507 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed000cd80 con 0x7f7ee00c3400 2024-04-25T01:38:23.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.703+0000 7f7ee51d5640 1 -- 172.21.15.112:0/1890620507 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee01ee860 con 0x7f7ee00c3400 2024-04-25T01:38:23.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.703+0000 7f7ee51d5640 1 -- 172.21.15.112:0/1890620507 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee01eee20 con 0x7f7ee00c3400 2024-04-25T01:38:23.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.705+0000 7f7ed57fa640 1 --2- 172.21.15.112:0/1890620507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ec4002a70 0x7f7ec4004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.705+0000 7f7ed57fa640 1 -- 172.21.15.112:0/1890620507 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec4007ec0 con 0x7f7ec4002a70 2024-04-25T01:38:23.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.706+0000 7f7ede59c640 1 --2- 172.21.15.112:0/1890620507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ec4002a70 0x7f7ec4004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.707+0000 7f7ede59c640 1 -- 172.21.15.112:0/1890620507 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ec40080c0 con 0x7f7ec4002a70 2024-04-25T01:38:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.708+0000 7f7ed77fe640 1 -- 172.21.15.112:0/1890620507 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ed001a070 con 0x7f7ee00c3400 2024-04-25T01:38:23.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.708+0000 7f7ed77fe640 1 --2- 172.21.15.112:0/1890620507 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7ec006bf00 0x7f7ec006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.708+0000 7f7ed77fe640 1 -- 172.21.15.112:0/1890620507 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f7ed008afd0 con 0x7f7ee00c3400 2024-04-25T01:38:23.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.709+0000 7f7edf59e640 1 --2- 172.21.15.112:0/1890620507 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7ec006bf00 0x7f7ec006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:23.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.710+0000 7f7edf59e640 1 --2- 172.21.15.112:0/1890620507 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7ec006bf00 0x7f7ec006e3c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8034000 tx=0x7f7ec802c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:23.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.710+0000 7f7ede59c640 1 --2- 172.21.15.112:0/1890620507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ec4002a70 0x7f7ec4004f30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7ee01540a0 tx=0x7f7ed800f5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:23.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.710+0000 7f7ed77fe640 1 -- 172.21.15.112:0/1890620507 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8007a30 con 0x7f7ec4002a70 2024-04-25T01:38:23.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.711+0000 7f7ed77fe640 1 -- 172.21.15.112:0/1890620507 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7ed8018070 con 0x7f7ec4002a70 2024-04-25T01:38:23.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.711+0000 7f7ed77fe640 1 -- 172.21.15.112:0/1890620507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ec4002a70 msgr2=0x7f7ec4004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.711+0000 7f7ed77fe640 1 --2- 172.21.15.112:0/1890620507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ec4002a70 0x7f7ec4004f30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7ee01540a0 tx=0x7f7ed800f5a0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.711+0000 7f7ed77fe640 1 -- 172.21.15.112:0/1890620507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ec4002a70 msgr2=0x7f7ec4004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:38:23.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.711+0000 7f7ed77fe640 1 --2- 172.21.15.112:0/1890620507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ec4002a70 0x7f7ec4004f30 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7ee01540a0 tx=0x7f7ed800f5a0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.725+0000 7f7ed57fa640 1 --2- 172.21.15.112:0/1890620507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ec4006eb0 0x7f7ec4007340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.725+0000 7f7ed57fa640 1 -- 172.21.15.112:0/1890620507 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec400a4a0 con 0x7f7ec4006eb0 2024-04-25T01:38:23.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.725+0000 7f7ede59c640 1 --2- 172.21.15.112:0/1890620507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ec4006eb0 0x7f7ec4007340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:23.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.725+0000 7f7ede59c640 1 -- 172.21.15.112:0/1890620507 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7ec4007880 con 0x7f7ec4006eb0 2024-04-25T01:38:23.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.726+0000 7f7ede59c640 1 --2- 172.21.15.112:0/1890620507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ec4006eb0 0x7f7ec4007340 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0148e60 tx=0x7f7ed8001250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:23.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.726+0000 7f7ed77fe640 1 -- 172.21.15.112:0/1890620507 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed80137e0 con 0x7f7ec4006eb0 2024-04-25T01:38:23.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.727+0000 7f7ed77fe640 1 -- 172.21.15.112:0/1890620507 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4558 (secure 0 0 0) 0x7f7ed80137e0 con 0x7f7ec4006eb0 2024-04-25T01:38:23.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.727+0000 7f7ed77fe640 1 -- 172.21.15.112:0/1890620507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ec4006eb0 msgr2=0x7f7ec4007340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.727+0000 7f7ed77fe640 1 --2- 172.21.15.112:0/1890620507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ec4006eb0 0x7f7ec4007340 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0148e60 tx=0x7f7ed8001250 comp rx=0 tx=0).stop 2024-04-25T01:38:23.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.727+0000 7f7ed77fe640 1 -- 172.21.15.112:0/1890620507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ec4006eb0 msgr2=0x7f7ec4007340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:38:23.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.727+0000 7f7ed77fe640 1 --2- 172.21.15.112:0/1890620507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ec4006eb0 0x7f7ec4007340 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0148e60 tx=0x7f7ed8001250 comp rx=0 tx=0).stop 2024-04-25T01:38:23.709 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:38:23.709 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:38:23.709 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:38:23.709 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:38:23.709 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 100, 2024-04-25T01:38:23.709 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 8492, 2024-04-25T01:38:23.709 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:38:23.709 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:38:23.709 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:38:23.709 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:38:23.709 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:23.709 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 3, 2024-04-25T01:38:23.710 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:38:23.710 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:38:23.710 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:38:23.710 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:23.710 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:23.710 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:23.710 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:23.710 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:23.710 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:23.710 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:23.710 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:23.710 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:23.710 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:23.710 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:23.710 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:23.711 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:38:23.711 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:38:23.711 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:23.711 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:23.711 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:23.711 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:23.711 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:23.711 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:23.711 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:23.711 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:23.711 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:23.711 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:23.711 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:23.711 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:23.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.712 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:38:23.712 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:38:23.712 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:38:23.712 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:38:23.712 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:38:23.712 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:38:23.712 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:38:23.712 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:38:23.712 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:38:23.712 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:38:23.712 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:38:23.712 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:38:23.712 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:38:23.712 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:38:23.712 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:38:23.712 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:38:23.713 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:38:23.713 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:23.713 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:23.713 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:23.713 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:23.713 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:23.713 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:23.713 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:23.713 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:23.713 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:23.713 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:23.713 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:23.713 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:23.713 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:38:23.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.714 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:38:23.714 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.714 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:38:23.714 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:38:23.714 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:23.714 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:23.714 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.714 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:23.714 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:38:23.714 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:23.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.714 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.714 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:23.714 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:38:23.714 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:23.715 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.715 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:23.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.715 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:38:23.715 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:38:23.715 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:23.715 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:23.715 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:23.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.715 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.715 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:38:23.715 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:38:23.715 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:23.715 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:23.715 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.716 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:23.716 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:38:23.716 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:23.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.716 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.716 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:23.716 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:38:23.716 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:23.716 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.716 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:23.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.716 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:38:23.716 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:38:23.716 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:23.716 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:23.716 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:23.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.717 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.717 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:38:23.717 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:38:23.717 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:23.717 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:23.717 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.717 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:23.717 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:38:23.717 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:23.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.717 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.717 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:23.717 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:38:23.717 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:23.717 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.718 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:23.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.718 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:38:23.718 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:38:23.718 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:23.718 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:23.718 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:23.718 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.718 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:23.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.718 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:38:23.718 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:38:23.718 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.718 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:23.718 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:23.719 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:38:23.719 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.719 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:23.719 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:38:23.719 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.719 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:23.719 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:23.719 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:38:23.719 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.719 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:23.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.719 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:38:23.719 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:38:23.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.730+0000 7f7ed57fa640 1 -- 172.21.15.112:0/1890620507 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7ec006bf00 msgr2=0x7f7ec006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.730+0000 7f7ed57fa640 1 --2- 172.21.15.112:0/1890620507 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7ec006bf00 0x7f7ec006e3c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8034000 tx=0x7f7ec802c000 comp rx=0 tx=0).stop 2024-04-25T01:38:23.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.730+0000 7f7ed57fa640 1 -- 172.21.15.112:0/1890620507 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ee00c3400 msgr2=0x7f7ee0146bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.730+0000 7f7ed57fa640 1 --2- 172.21.15.112:0/1890620507 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ee00c3400 0x7f7ee0146bf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000e9e0 tx=0x7f7ed000eea0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.730+0000 7f7edf59e640 1 -- 172.21.15.112:0/1890620507 reap_dead start 2024-04-25T01:38:23.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.730+0000 7f7edf59e640 1 -- 172.21.15.112:0/1890620507 reap_dead start 2024-04-25T01:38:23.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.730+0000 7f7ed57fa640 1 -- 172.21.15.112:0/1890620507 shutdown_connections 2024-04-25T01:38:23.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.730+0000 7f7ed57fa640 1 -- 172.21.15.112:0/1890620507 >> 172.21.15.112:0/1890620507 conn(0x7f7ee006b160 msgr2=0x7f7ee0153550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:23.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.730+0000 7f7ed57fa640 1 -- 172.21.15.112:0/1890620507 shutdown_connections 2024-04-25T01:38:23.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.730+0000 7f7ed57fa640 1 -- 172.21.15.112:0/1890620507 wait complete. 2024-04-25T01:38:23.728 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:38:23.728 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:38:23.729 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:38:23.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.920+0000 7f130c161640 1 Processor -- start 2024-04-25T01:38:23.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.920+0000 7f130c161640 1 -- start start 2024-04-25T01:38:23.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.921+0000 7f130c161640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f13040ac1d0 0x7f1304070d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.921+0000 7f130c161640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f130415d370 0x7f130415d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.921+0000 7f130c161640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1304159bd0 con 0x7f13040aa210 2024-04-25T01:38:23.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.921+0000 7f130c161640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f1304159e90 con 0x7f130415d370 2024-04-25T01:38:23.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.921+0000 7f130c161640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f130415a170 con 0x7f13040ac1d0 2024-04-25T01:38:23.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.922+0000 7f130a6fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f130415d370 0x7f130415d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:23.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.922+0000 7f130a6fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f130415d370 0x7f130415d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45398/0 (socket says 172.21.15.112:45398) 2024-04-25T01:38:23.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.923+0000 7f130a6fe640 1 -- 172.21.15.112:0/2216879080 learned_addr learned my addr 172.21.15.112:0/2216879080 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:23.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.923+0000 7f1309efd640 1 --2- 172.21.15.112:0/2216879080 >> v2:172.21.15.187:3300/0 conn(0x7f13040ac1d0 0x7f1304070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:23.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.923+0000 7f130a6fe640 1 -- 172.21.15.112:0/2216879080 >> v2:172.21.15.187:3300/0 conn(0x7f13040ac1d0 msgr2=0x7f1304070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:38:23.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.923+0000 7f130a6fe640 1 --2- 172.21.15.112:0/2216879080 >> v2:172.21.15.187:3300/0 conn(0x7f13040ac1d0 0x7f1304070d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.923+0000 7f130a6fe640 1 -- 172.21.15.112:0/2216879080 >> v1:172.21.15.187:6789/0 conn(0x7f13040aa210 legacy=0x7f13040c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:38:23.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.923+0000 7f130a6fe640 1 -- 172.21.15.112:0/2216879080 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130415a450 con 0x7f130415d370 2024-04-25T01:38:23.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.924+0000 7f130a6fe640 1 --2- 172.21.15.112:0/2216879080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f130415d370 0x7f130415d7b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f130406c150 tx=0x7f12fc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa07b09b159b1967 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:23.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.925+0000 7f1308efb640 1 -- 172.21.15.112:0/2216879080 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc036070 con 0x7f130415d370 2024-04-25T01:38:23.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.925+0000 7f1308efb640 1 -- 172.21.15.112:0/2216879080 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12fc031600 con 0x7f130415d370 2024-04-25T01:38:23.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.925+0000 7f1308efb640 1 -- 172.21.15.112:0/2216879080 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc0319d0 con 0x7f130415d370 2024-04-25T01:38:23.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.926+0000 7f130c161640 1 -- 172.21.15.112:0/2216879080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f130415d370 msgr2=0x7f130415d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.926+0000 7f130c161640 1 --2- 172.21.15.112:0/2216879080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f130415d370 0x7f130415d7b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f130406c150 tx=0x7f12fc02a660 comp rx=0 tx=0).stop 2024-04-25T01:38:23.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.932+0000 7f130c161640 1 -- 172.21.15.112:0/2216879080 shutdown_connections 2024-04-25T01:38:23.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.932+0000 7f130c161640 1 --2- 172.21.15.112:0/2216879080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f130415d370 0x7f130415d7b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.932+0000 7f130c161640 1 --2- 172.21.15.112:0/2216879080 >> v2:172.21.15.187:3300/0 conn(0x7f13040ac1d0 0x7f1304070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.932+0000 7f130c161640 1 -- 172.21.15.112:0/2216879080 >> 172.21.15.112:0/2216879080 conn(0x7f130406b160 msgr2=0x7f130406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:23.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.932+0000 7f130c161640 1 -- 172.21.15.112:0/2216879080 shutdown_connections 2024-04-25T01:38:23.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.932+0000 7f130c161640 1 -- 172.21.15.112:0/2216879080 wait complete. 2024-04-25T01:38:23.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.934+0000 7f130c161640 1 Processor -- start 2024-04-25T01:38:23.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.934+0000 7f130c161640 1 -- start start 2024-04-25T01:38:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.935+0000 7f130c161640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13040aa210 0x7f13041448c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.935+0000 7f130c161640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13040ac1d0 0x7f13041420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.935+0000 7f130c161640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f130415d370 0x7f13041425f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.936+0000 7f1309efd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13040aa210 0x7f13041448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:23.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.936+0000 7f13096fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13040ac1d0 0x7f13041420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:23.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.936+0000 7f13096fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13040ac1d0 0x7f13041420b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:49968/0 (socket says 172.21.15.112:49968) 2024-04-25T01:38:23.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.936+0000 7f1309efd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13040aa210 0x7f13041448c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:34596/0 (socket says 172.21.15.112:34596) 2024-04-25T01:38:23.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.936+0000 7f13096fc640 1 -- 172.21.15.112:0/3285928237 learned_addr learned my addr 172.21.15.112:0/3285928237 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:23.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.936+0000 7f130c161640 1 -- 172.21.15.112:0/3285928237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f13040c4220 con 0x7f130415d370 2024-04-25T01:38:23.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.936+0000 7f130a6fe640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f130415d370 0x7f13041425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:23.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.937+0000 7f130c161640 1 -- 172.21.15.112:0/3285928237 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f13040ac5b0 con 0x7f13040aa210 2024-04-25T01:38:23.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.937+0000 7f130c161640 1 -- 172.21.15.112:0/3285928237 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f13040c43a0 con 0x7f13040ac1d0 2024-04-25T01:38:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.937+0000 7f130a6fe640 1 -- 172.21.15.112:0/3285928237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13040ac1d0 msgr2=0x7f13041420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.937+0000 7f130a6fe640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13040ac1d0 0x7f13041420b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.937+0000 7f130a6fe640 1 -- 172.21.15.112:0/3285928237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13040aa210 msgr2=0x7f13041448c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.937+0000 7f130a6fe640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13040aa210 0x7f13041448c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.937+0000 7f130a6fe640 1 -- 172.21.15.112:0/3285928237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13040c4ea0 con 0x7f130415d370 2024-04-25T01:38:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.938+0000 7f1309efd640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13040aa210 0x7f13041448c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:38:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.938+0000 7f130a6fe640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f130415d370 0x7f13041425f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f12fc030000 tx=0x7f12fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.939+0000 7f12fa7fc640 1 -- 172.21.15.112:0/3285928237 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc041020 con 0x7f130415d370 2024-04-25T01:38:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.939+0000 7f12fa7fc640 1 -- 172.21.15.112:0/3285928237 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12fc008710 con 0x7f130415d370 2024-04-25T01:38:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.939+0000 7f130c161640 1 -- 172.21.15.112:0/3285928237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13040c5180 con 0x7f130415d370 2024-04-25T01:38:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.939+0000 7f12fa7fc640 1 -- 172.21.15.112:0/3285928237 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc036040 con 0x7f130415d370 2024-04-25T01:38:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.939+0000 7f130c161640 1 -- 172.21.15.112:0/3285928237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13041594c0 con 0x7f130415d370 2024-04-25T01:38:23.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.941+0000 7f130c161640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1304142b30 0x7f1304143480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.941+0000 7f130c161640 1 -- 172.21.15.112:0/3285928237 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f13040c4c50 con 0x7f1304142b30 2024-04-25T01:38:23.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.941+0000 7f12fa7fc640 1 -- 172.21.15.112:0/3285928237 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f12fc044020 con 0x7f130415d370 2024-04-25T01:38:23.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.942+0000 7f12fa7fc640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f12e806be80 0x7f12e806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.942+0000 7f1309efd640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1304142b30 0x7f1304143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:23.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.942+0000 7f12fa7fc640 1 -- 172.21.15.112:0/3285928237 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f12fc0abef0 con 0x7f130415d370 2024-04-25T01:38:23.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.943+0000 7f13096fc640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f12e806be80 0x7f12e806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:23.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.944+0000 7f13096fc640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f12e806be80 0x7f12e806e340 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1300006fd0 tx=0x7f1300008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:23.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.947+0000 7f1309efd640 1 -- 172.21.15.112:0/3285928237 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1304075980 con 0x7f1304142b30 2024-04-25T01:38:23.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.948+0000 7f1309efd640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1304142b30 0x7f1304143480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f12f400d760 tx=0x7f12f400dc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:23.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.948+0000 7f12fa7fc640 1 -- 172.21.15.112:0/3285928237 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4016070 con 0x7f1304142b30 2024-04-25T01:38:23.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.949+0000 7f12fa7fc640 1 -- 172.21.15.112:0/3285928237 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f12f4008910 con 0x7f1304142b30 2024-04-25T01:38:23.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.949+0000 7f12fa7fc640 1 -- 172.21.15.112:0/3285928237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1304142b30 msgr2=0x7f1304143480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.949+0000 7f12fa7fc640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1304142b30 0x7f1304143480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f12f400d760 tx=0x7f12f400dc20 comp rx=0 tx=0).stop 2024-04-25T01:38:23.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.949+0000 7f12fa7fc640 1 -- 172.21.15.112:0/3285928237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1304142b30 msgr2=0x7f1304143480 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:38:23.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.949+0000 7f12fa7fc640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1304142b30 0x7f1304143480 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f12f400d760 tx=0x7f12f400dc20 comp rx=0 tx=0).stop 2024-04-25T01:38:23.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.963+0000 7f130c161640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13040c7320 0x7f13041439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:23.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.963+0000 7f130c161640 1 -- 172.21.15.112:0/3285928237 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f13040682a0 con 0x7f13040c7320 2024-04-25T01:38:23.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.963+0000 7f1309efd640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13040c7320 0x7f13041439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:23.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.966+0000 7f1309efd640 1 -- 172.21.15.112:0/3285928237 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1304165210 con 0x7f13040c7320 2024-04-25T01:38:23.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.966+0000 7f1309efd640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13040c7320 0x7f13041439c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f12f400e010 tx=0x7f12f40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:23.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.967+0000 7f12fa7fc640 1 -- 172.21.15.112:0/3285928237 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f401cd10 con 0x7f13040c7320 2024-04-25T01:38:23.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.967+0000 7f12fa7fc640 1 -- 172.21.15.112:0/3285928237 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4899 (secure 0 0 0) 0x7f12f4012460 con 0x7f13040c7320 2024-04-25T01:38:23.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.967+0000 7f12fa7fc640 1 -- 172.21.15.112:0/3285928237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13040c7320 msgr2=0x7f13041439c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.967+0000 7f12fa7fc640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13040c7320 0x7f13041439c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f12f400e010 tx=0x7f12f40046c0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.967+0000 7f12fa7fc640 1 -- 172.21.15.112:0/3285928237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13040c7320 msgr2=0x7f13041439c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:38:23.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.967+0000 7f12fa7fc640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13040c7320 0x7f13041439c0 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f12f400e010 tx=0x7f12f40046c0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.949 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:38:23.949 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:38:23.949 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:38:23.949 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:38:23.949 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 100, 2024-04-25T01:38:23.949 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 73332, 2024-04-25T01:38:23.950 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:38:23.950 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:38:23.950 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:38:23.950 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:38:23.950 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:23.950 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 4, 2024-04-25T01:38:23.950 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:38:23.950 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:38:23.950 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:38:23.950 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:23.950 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:23.950 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:23.950 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:23.951 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:23.951 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:23.951 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:23.951 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:23.951 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:23.951 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:23.951 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:23.951 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:23.951 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:38:23.951 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:38:23.951 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:23.951 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:23.951 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:23.951 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:23.952 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:23.952 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:23.952 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:23.952 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:23.952 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:23.952 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:23.952 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:23.952 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:23.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.952 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:38:23.952 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:38:23.952 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:38:23.952 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:38:23.952 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:38:23.952 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:38:23.953 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:38:23.953 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:38:23.953 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:38:23.953 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:38:23.953 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:38:23.953 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:38:23.953 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:38:23.953 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:38:23.953 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:38:23.953 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:38:23.953 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:38:23.953 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:23.953 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:23.953 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:23.953 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:23.954 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:23.954 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:23.954 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:23.954 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:23.954 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:23.954 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:23.954 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:23.954 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:23.954 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:38:23.954 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.954 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:38:23.954 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.954 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:38:23.954 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:38:23.954 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:23.955 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:23.955 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.955 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:23.955 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:38:23.955 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:23.955 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.955 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.955 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:23.955 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:38:23.955 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:23.955 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.955 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:23.955 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.955 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:38:23.955 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:38:23.956 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:23.956 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:23.956 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:23.956 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.956 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.956 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:38:23.956 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:38:23.956 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:23.956 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:23.956 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.956 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:23.956 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:38:23.956 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:23.956 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.956 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.957 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:23.957 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:38:23.957 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:23.957 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.957 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:23.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.957 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:38:23.957 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:38:23.957 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:23.957 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:23.957 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:23.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.957 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.957 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:38:23.957 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:38:23.958 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:23.958 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:23.958 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.958 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:23.958 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:38:23.958 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:23.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.958 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.958 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:23.958 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:38:23.958 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:23.958 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.958 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:23.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.958 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:38:23.959 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:38:23.959 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:23.959 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:23.959 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:23.959 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.959 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:23.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.959 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:38:23.959 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:38:23.959 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.959 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:23.959 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:23.959 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:38:23.959 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.959 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:23.960 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:38:23.960 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.960 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:23.960 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:23.960 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:38:23.960 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.960 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:23.960 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:38:23.960 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.960 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:23.960 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:23.960 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:38:23.960 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.960 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:23.960 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:38:23.961 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:23.961 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:23.961 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:23.961 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:38:23.961 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:23.961 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:23.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:23.961 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:38:23.961 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:38:23.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.968+0000 7f130c161640 1 -- 172.21.15.112:0/3285928237 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f12e806be80 msgr2=0x7f12e806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.969+0000 7f130c161640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f12e806be80 0x7f12e806e340 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1300006fd0 tx=0x7f1300008040 comp rx=0 tx=0).stop 2024-04-25T01:38:23.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.969+0000 7f130a6fe640 1 -- 172.21.15.112:0/3285928237 reap_dead start 2024-04-25T01:38:23.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.969+0000 7f130c161640 1 -- 172.21.15.112:0/3285928237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f130415d370 msgr2=0x7f13041425f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:23.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.969+0000 7f130c161640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f130415d370 0x7f13041425f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f12fc030000 tx=0x7f12fc004a40 comp rx=0 tx=0).stop 2024-04-25T01:38:23.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.969+0000 7f130c161640 1 -- 172.21.15.112:0/3285928237 shutdown_connections 2024-04-25T01:38:23.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.969+0000 7f130c161640 1 --2- 172.21.15.112:0/3285928237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f130415d370 0x7f13041425f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:23.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.969+0000 7f130c161640 1 -- 172.21.15.112:0/3285928237 >> 172.21.15.112:0/3285928237 conn(0x7f130406b160 msgr2=0x7f13040c3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:23.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.969+0000 7f130c161640 1 -- 172.21.15.112:0/3285928237 shutdown_connections 2024-04-25T01:38:23.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:23.969+0000 7f130c161640 1 -- 172.21.15.112:0/3285928237 wait complete. 2024-04-25T01:38:23.967 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:38:23.967 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T01:38:23.967 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T01:38:24.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.158+0000 7fd4384ee640 1 Processor -- start 2024-04-25T01:38:24.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.160+0000 7fd4384ee640 1 -- start start 2024-04-25T01:38:24.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.161+0000 7fd4384ee640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd4300cbb80 0x7fd43015d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:24.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.161+0000 7fd4384ee640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4300b0f70 0x7fd43015da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:24.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.161+0000 7fd4384ee640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4300c7840 con 0x7fd4300be250 2024-04-25T01:38:24.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.161+0000 7fd4384ee640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fd4300c75f0 con 0x7fd4300cbb80 2024-04-25T01:38:24.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.161+0000 7fd4384ee640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd4300ae840 con 0x7fd4300b0f70 2024-04-25T01:38:24.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.162+0000 7fd436a8b640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd4300be250 0x7fd43015a600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32868/0 (socket says 172.21.15.112:32868) 2024-04-25T01:38:24.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.162+0000 7fd436a8b640 1 -- 172.21.15.112:0/3645868993 learned_addr learned my addr 172.21.15.112:0/3645868993 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:38:24.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.162+0000 7fd43628a640 1 --2- 172.21.15.112:0/3645868993 >> v2:172.21.15.112:3300/0 conn(0x7fd4300cbb80 0x7fd43015d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:24.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.162+0000 7fd43628a640 1 -- 172.21.15.112:0/3645868993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4300b0f70 msgr2=0x7fd43015da10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:38:24.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.162+0000 7fd43628a640 1 --2- 172.21.15.112:0/3645868993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4300b0f70 0x7fd43015da10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:24.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.162+0000 7fd43628a640 1 -- 172.21.15.112:0/3645868993 >> v1:172.21.15.112:6789/0 conn(0x7fd4300be250 legacy=0x7fd43015a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:24.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.162+0000 7fd43628a640 1 -- 172.21.15.112:0/3645868993 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4300ae1f0 con 0x7fd4300cbb80 2024-04-25T01:38:24.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.163+0000 7fd43628a640 1 --2- 172.21.15.112:0/3645868993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4300cbb80 0x7fd43015d4d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0097a0 tx=0x7fd42c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0020e054dd2130 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:24.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.164+0000 7fd435288640 1 -- 172.21.15.112:0/3645868993 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c037070 con 0x7fd4300cbb80 2024-04-25T01:38:24.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.164+0000 7fd435288640 1 -- 172.21.15.112:0/3645868993 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd42c032440 con 0x7fd4300cbb80 2024-04-25T01:38:24.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.164+0000 7fd435288640 1 -- 172.21.15.112:0/3645868993 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c037070 con 0x7fd4300cbb80 2024-04-25T01:38:24.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.166+0000 7fd4384ee640 1 -- 172.21.15.112:0/3645868993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4300cbb80 msgr2=0x7fd43015d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:24.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.166+0000 7fd4384ee640 1 --2- 172.21.15.112:0/3645868993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4300cbb80 0x7fd43015d4d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0097a0 tx=0x7fd42c02a110 comp rx=0 tx=0).stop 2024-04-25T01:38:24.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.168+0000 7fd4384ee640 1 -- 172.21.15.112:0/3645868993 shutdown_connections 2024-04-25T01:38:24.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.168+0000 7fd4384ee640 1 --2- 172.21.15.112:0/3645868993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4300b0f70 0x7fd43015da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:24.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.168+0000 7fd4384ee640 1 --2- 172.21.15.112:0/3645868993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4300cbb80 0x7fd43015d4d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:24.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.168+0000 7fd4384ee640 1 -- 172.21.15.112:0/3645868993 >> 172.21.15.112:0/3645868993 conn(0x7fd4300ad360 msgr2=0x7fd4300cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:24.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.168+0000 7fd4384ee640 1 -- 172.21.15.112:0/3645868993 shutdown_connections 2024-04-25T01:38:24.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.168+0000 7fd4384ee640 1 -- 172.21.15.112:0/3645868993 wait complete. 2024-04-25T01:38:24.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.170+0000 7fd4384ee640 1 Processor -- start 2024-04-25T01:38:24.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.170+0000 7fd4384ee640 1 -- start start 2024-04-25T01:38:24.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.171+0000 7fd4384ee640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4300b0f70 0x7fd4300bb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:24.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.171+0000 7fd4384ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4300be250 0x7fd4301efb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:24.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.172+0000 7fd435a89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4300be250 0x7fd4301efb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:24.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.172+0000 7fd435a89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4300be250 0x7fd4301efb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45416/0 (socket says 172.21.15.112:45416) 2024-04-25T01:38:24.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.172+0000 7fd43628a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4300b0f70 0x7fd4300bb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:24.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.172+0000 7fd43628a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4300b0f70 0x7fd4300bb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:50000/0 (socket says 172.21.15.112:50000) 2024-04-25T01:38:24.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.172+0000 7fd435a89640 1 -- 172.21.15.112:0/2296481382 learned_addr learned my addr 172.21.15.112:0/2296481382 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:24.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.172+0000 7fd4384ee640 1 --2- 172.21.15.112:0/2296481382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4300cbb80 0x7fd4300bd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:24.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.172+0000 7fd4384ee640 1 -- 172.21.15.112:0/2296481382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd43015ad10 con 0x7fd4300be250 2024-04-25T01:38:24.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.172+0000 7fd4384ee640 1 -- 172.21.15.112:0/2296481382 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd430072640 con 0x7fd4300cbb80 2024-04-25T01:38:24.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.172+0000 7fd436a8b640 1 --2- 172.21.15.112:0/2296481382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4300cbb80 0x7fd4300bd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:24.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.173+0000 7fd4384ee640 1 -- 172.21.15.112:0/2296481382 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd43015ae90 con 0x7fd4300b0f70 2024-04-25T01:38:24.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.173+0000 7fd435a89640 1 -- 172.21.15.112:0/2296481382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4300b0f70 msgr2=0x7fd4300bb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:24.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.173+0000 7fd435a89640 1 --2- 172.21.15.112:0/2296481382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4300b0f70 0x7fd4300bb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:24.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.173+0000 7fd435a89640 1 -- 172.21.15.112:0/2296481382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4300cbb80 msgr2=0x7fd4300bd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:24.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.173+0000 7fd435a89640 1 --2- 172.21.15.112:0/2296481382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4300cbb80 0x7fd4300bd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:24.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.173+0000 7fd435a89640 1 -- 172.21.15.112:0/2296481382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4301f0090 con 0x7fd4300be250 2024-04-25T01:38:24.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.173+0000 7fd435a89640 1 --2- 172.21.15.112:0/2296481382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4300be250 0x7fd4301efb50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd42800ba50 tx=0x7fd42800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:24.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.174+0000 7fd436a8b640 1 --2- 172.21.15.112:0/2296481382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4300cbb80 0x7fd4300bd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:38:24.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.174+0000 7fd43628a640 1 --2- 172.21.15.112:0/2296481382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4300b0f70 0x7fd4300bb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:38:24.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.174+0000 7fd426ffd640 1 -- 172.21.15.112:0/2296481382 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428019070 con 0x7fd4300be250 2024-04-25T01:38:24.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.175+0000 7fd426ffd640 1 -- 172.21.15.112:0/2296481382 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd42800cc10 con 0x7fd4300be250 2024-04-25T01:38:24.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.175+0000 7fd4384ee640 1 -- 172.21.15.112:0/2296481382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4300c4580 con 0x7fd4300be250 2024-04-25T01:38:24.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.176+0000 7fd4384ee640 1 -- 172.21.15.112:0/2296481382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4300c4820 con 0x7fd4300be250 2024-04-25T01:38:24.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.177+0000 7fd426ffd640 1 -- 172.21.15.112:0/2296481382 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428014440 con 0x7fd4300be250 2024-04-25T01:38:24.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.178+0000 7fd426ffd640 1 -- 172.21.15.112:0/2296481382 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd42801c070 con 0x7fd4300be250 2024-04-25T01:38:24.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.178+0000 7fd426ffd640 1 --2- 172.21.15.112:0/2296481382 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd41406bf00 0x7fd41406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:24.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.178+0000 7fd426ffd640 1 -- 172.21.15.112:0/2296481382 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fd42808b040 con 0x7fd4300be250 2024-04-25T01:38:24.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.179+0000 7fd43628a640 1 --2- 172.21.15.112:0/2296481382 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd41406bf00 0x7fd41406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:24.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.179+0000 7fd43628a640 1 --2- 172.21.15.112:0/2296481382 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd41406bf00 0x7fd41406e3c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd42c034000 tx=0x7fd42c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:24.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.180+0000 7fd424ff9640 1 -- 172.21.15.112:0/2296481382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4300689b0 con 0x7fd4300be250 2024-04-25T01:38:24.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.188+0000 7fd426ffd640 1 -- 172.21.15.112:0/2296481382 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd4301f0090 con 0x7fd4300be250 2024-04-25T01:38:24.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.696+0000 7fd424ff9640 1 -- 172.21.15.112:0/2296481382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fd430156310 con 0x7fd4300be250 2024-04-25T01:38:24.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.698+0000 7fd426ffd640 1 -- 172.21.15.112:0/2296481382 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fd428059630 con 0x7fd4300be250 2024-04-25T01:38:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.702+0000 7fd424ff9640 1 -- 172.21.15.112:0/2296481382 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd41406bf00 msgr2=0x7fd41406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.703+0000 7fd424ff9640 1 --2- 172.21.15.112:0/2296481382 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd41406bf00 0x7fd41406e3c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd42c034000 tx=0x7fd42c02e000 comp rx=0 tx=0).stop 2024-04-25T01:38:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.703+0000 7fd424ff9640 1 -- 172.21.15.112:0/2296481382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4300be250 msgr2=0x7fd4301efb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.703+0000 7fd424ff9640 1 --2- 172.21.15.112:0/2296481382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4300be250 0x7fd4301efb50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd42800ba50 tx=0x7fd42800bf10 comp rx=0 tx=0).stop 2024-04-25T01:38:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.703+0000 7fd424ff9640 1 -- 172.21.15.112:0/2296481382 shutdown_connections 2024-04-25T01:38:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.703+0000 7fd424ff9640 1 --2- 172.21.15.112:0/2296481382 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd41406bf00 0x7fd41406e3c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.703+0000 7fd424ff9640 1 --2- 172.21.15.112:0/2296481382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4300cbb80 0x7fd4300bd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.703+0000 7fd424ff9640 1 --2- 172.21.15.112:0/2296481382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4300be250 0x7fd4301efb50 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd42800ba50 tx=0x7fd42800bf10 comp rx=0 tx=0).stop 2024-04-25T01:38:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.703+0000 7fd424ff9640 1 --2- 172.21.15.112:0/2296481382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4300b0f70 0x7fd4300bb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:24.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.703+0000 7fd424ff9640 1 -- 172.21.15.112:0/2296481382 >> 172.21.15.112:0/2296481382 conn(0x7fd4300ad360 msgr2=0x7fd430157cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:24.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.705+0000 7fd424ff9640 1 -- 172.21.15.112:0/2296481382 shutdown_connections 2024-04-25T01:38:24.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:24.705+0000 7fd424ff9640 1 -- 172.21.15.112:0/2296481382 wait complete. 2024-04-25T01:38:24.706 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T01:38:28.584 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:38:28.587 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:38:34.707 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:38:34.708 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:38:34.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.901+0000 7fb3222bc640 1 Processor -- start 2024-04-25T01:38:34.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.902+0000 7fb3222bc640 1 -- start start 2024-04-25T01:38:34.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.903+0000 7fb3222bc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb31c0725d0 0x7fb31c0c2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:34.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.903+0000 7fb3222bc640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb31c0c2ed0 0x7fb31c159f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:34.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.903+0000 7fb3222bc640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb31c06d500 con 0x7fb31c157280 2024-04-25T01:38:34.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.903+0000 7fb3222bc640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fb31c157660 con 0x7fb31c0725d0 2024-04-25T01:38:34.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.903+0000 7fb3222bc640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb31c06d7e0 con 0x7fb31c0c2ed0 2024-04-25T01:38:34.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.903+0000 7fb31bfff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb31c157280 0x7fb31c072d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49478/0 (socket says 172.21.15.112:49478) 2024-04-25T01:38:34.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.903+0000 7fb31bfff640 1 -- 172.21.15.112:0/2035652966 learned_addr learned my addr 172.21.15.112:0/2035652966 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:38:34.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.904+0000 7fb31b7fe640 1 --2- 172.21.15.112:0/2035652966 >> v2:172.21.15.112:3300/0 conn(0x7fb31c0725d0 0x7fb31c0c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:34.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.904+0000 7fb31b7fe640 1 -- 172.21.15.112:0/2035652966 >> v2:172.21.15.187:3300/0 conn(0x7fb31c0c2ed0 msgr2=0x7fb31c159f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:38:34.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.904+0000 7fb31b7fe640 1 --2- 172.21.15.112:0/2035652966 >> v2:172.21.15.187:3300/0 conn(0x7fb31c0c2ed0 0x7fb31c159f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:34.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.904+0000 7fb31b7fe640 1 -- 172.21.15.112:0/2035652966 >> v1:172.21.15.112:6789/0 conn(0x7fb31c157280 legacy=0x7fb31c072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:38:34.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.904+0000 7fb31b7fe640 1 -- 172.21.15.112:0/2035652966 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31c06dac0 con 0x7fb31c0725d0 2024-04-25T01:38:34.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.905+0000 7fb31b7fe640 1 --2- 172.21.15.112:0/2035652966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb31c0725d0 0x7fb31c0c2990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0c6f10 tx=0x7fb30c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c70fa11e8aa0f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:34.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.906+0000 7fb31affd640 1 -- 172.21.15.112:0/2035652966 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c037070 con 0x7fb31c0725d0 2024-04-25T01:38:34.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.906+0000 7fb31affd640 1 -- 172.21.15.112:0/2035652966 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb30c02adc0 con 0x7fb31c0725d0 2024-04-25T01:38:34.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.907+0000 7fb31affd640 1 -- 172.21.15.112:0/2035652966 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c032660 con 0x7fb31c0725d0 2024-04-25T01:38:34.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.907+0000 7fb3222bc640 1 -- 172.21.15.112:0/2035652966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb31c0725d0 msgr2=0x7fb31c0c2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:34.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.907+0000 7fb3222bc640 1 --2- 172.21.15.112:0/2035652966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb31c0725d0 0x7fb31c0c2990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0c6f10 tx=0x7fb30c02a2c0 comp rx=0 tx=0).stop 2024-04-25T01:38:34.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.912+0000 7fb3222bc640 1 -- 172.21.15.112:0/2035652966 shutdown_connections 2024-04-25T01:38:34.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.912+0000 7fb3222bc640 1 --2- 172.21.15.112:0/2035652966 >> v2:172.21.15.187:3300/0 conn(0x7fb31c0c2ed0 0x7fb31c159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:34.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.912+0000 7fb3222bc640 1 --2- 172.21.15.112:0/2035652966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb31c0725d0 0x7fb31c0c2990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:34.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.912+0000 7fb3222bc640 1 -- 172.21.15.112:0/2035652966 >> 172.21.15.112:0/2035652966 conn(0x7fb31c0c3a80 msgr2=0x7fb31c0c3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:34.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.912+0000 7fb3222bc640 1 -- 172.21.15.112:0/2035652966 shutdown_connections 2024-04-25T01:38:34.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.912+0000 7fb3222bc640 1 -- 172.21.15.112:0/2035652966 wait complete. 2024-04-25T01:38:34.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.914+0000 7fb3222bc640 1 Processor -- start 2024-04-25T01:38:34.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.914+0000 7fb3222bc640 1 -- start start 2024-04-25T01:38:34.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.915+0000 7fb3222bc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb31c0725d0 0x7fb31c138220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:34.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.915+0000 7fb3222bc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb31c0c2ed0 0x7fb31c152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:34.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.915+0000 7fb3222bc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb31c157280 0x7fb31c135a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:34.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.915+0000 7fb3222bc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c15a470 con 0x7fb31c0725d0 2024-04-25T01:38:34.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.915+0000 7fb3222bc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c06e080 con 0x7fb31c157280 2024-04-25T01:38:34.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.915+0000 7fb3222bc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb31c15a5f0 con 0x7fb31c0c2ed0 2024-04-25T01:38:34.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.916+0000 7fb31b7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb31c0c2ed0 0x7fb31c152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:34.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.916+0000 7fb31bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb31c0725d0 0x7fb31c138220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:34.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.916+0000 7fb31bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb31c0725d0 0x7fb31c138220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41114/0 (socket says 172.21.15.112:41114) 2024-04-25T01:38:34.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.916+0000 7fb31bfff640 1 -- 172.21.15.112:0/354912105 learned_addr learned my addr 172.21.15.112:0/354912105 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:34.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.916+0000 7fb320859640 1 --2- 172.21.15.112:0/354912105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb31c157280 0x7fb31c135a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.918+0000 7fb31bfff640 1 -- 172.21.15.112:0/354912105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb31c0c2ed0 msgr2=0x7fb31c152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.918+0000 7fb31bfff640 1 --2- 172.21.15.112:0/354912105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb31c0c2ed0 0x7fb31c152ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.918+0000 7fb31bfff640 1 -- 172.21.15.112:0/354912105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb31c157280 msgr2=0x7fb31c135a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.918+0000 7fb31bfff640 1 --2- 172.21.15.112:0/354912105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb31c157280 0x7fb31c135a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.918+0000 7fb31bfff640 1 -- 172.21.15.112:0/354912105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31c136e80 con 0x7fb31c0725d0 2024-04-25T01:38:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.919+0000 7fb31bfff640 1 --2- 172.21.15.112:0/354912105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb31c0725d0 0x7fb31c138220 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb31000b460 tx=0x7fb31000b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.919+0000 7fb318ff9640 1 -- 172.21.15.112:0/354912105 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310015020 con 0x7fb31c0725d0 2024-04-25T01:38:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.919+0000 7fb318ff9640 1 -- 172.21.15.112:0/354912105 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb310011830 con 0x7fb31c0725d0 2024-04-25T01:38:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.919+0000 7fb3222bc640 1 -- 172.21.15.112:0/354912105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb31c137160 con 0x7fb31c0725d0 2024-04-25T01:38:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.919+0000 7fb3222bc640 1 -- 172.21.15.112:0/354912105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb31c137720 con 0x7fb31c0725d0 2024-04-25T01:38:34.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.919+0000 7fb318ff9640 1 -- 172.21.15.112:0/354912105 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310015020 con 0x7fb31c0725d0 2024-04-25T01:38:34.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.922+0000 7fb3222bc640 1 -- 172.21.15.112:0/354912105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2fc005290 con 0x7fb31c0725d0 2024-04-25T01:38:34.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.922+0000 7fb318ff9640 1 -- 172.21.15.112:0/354912105 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb310018020 con 0x7fb31c0725d0 2024-04-25T01:38:34.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.923+0000 7fb318ff9640 1 --2- 172.21.15.112:0/354912105 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb30806c030 0x7fb30806e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:34.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.923+0000 7fb31b7fe640 1 --2- 172.21.15.112:0/354912105 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb30806c030 0x7fb30806e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:34.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.924+0000 7fb31b7fe640 1 --2- 172.21.15.112:0/354912105 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb30806c030 0x7fb30806e4f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb30c034000 tx=0x7fb30c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:34.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.924+0000 7fb318ff9640 1 -- 172.21.15.112:0/354912105 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fb310056c50 con 0x7fb31c0725d0 2024-04-25T01:38:34.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:34.934+0000 7fb318ff9640 1 -- 172.21.15.112:0/354912105 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb31001f020 con 0x7fb31c0725d0 2024-04-25T01:38:35.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.433+0000 7fb3222bc640 1 -- 172.21.15.112:0/354912105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb2fc005850 con 0x7fb31c0725d0 2024-04-25T01:38:35.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.434+0000 7fb318ff9640 1 -- 172.21.15.112:0/354912105 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1570 (secure 0 0 0) 0x7fb3100592b0 con 0x7fb31c0725d0 2024-04-25T01:38:35.417 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":100,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:38:35.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.441+0000 7fb3222bc640 1 -- 172.21.15.112:0/354912105 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb30806c030 msgr2=0x7fb30806e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.441+0000 7fb3222bc640 1 --2- 172.21.15.112:0/354912105 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb30806c030 0x7fb30806e4f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb30c034000 tx=0x7fb30c02e000 comp rx=0 tx=0).stop 2024-04-25T01:38:35.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.441+0000 7fb3222bc640 1 -- 172.21.15.112:0/354912105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb31c0725d0 msgr2=0x7fb31c138220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.441+0000 7fb3222bc640 1 --2- 172.21.15.112:0/354912105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb31c0725d0 0x7fb31c138220 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb31000b460 tx=0x7fb31000b920 comp rx=0 tx=0).stop 2024-04-25T01:38:35.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.444+0000 7fb3222bc640 1 -- 172.21.15.112:0/354912105 shutdown_connections 2024-04-25T01:38:35.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.444+0000 7fb3222bc640 1 --2- 172.21.15.112:0/354912105 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb30806c030 0x7fb30806e4f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.444+0000 7fb3222bc640 1 --2- 172.21.15.112:0/354912105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb31c157280 0x7fb31c135a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.444+0000 7fb3222bc640 1 --2- 172.21.15.112:0/354912105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb31c0c2ed0 0x7fb31c152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.444+0000 7fb3222bc640 1 --2- 172.21.15.112:0/354912105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb31c0725d0 0x7fb31c138220 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.444+0000 7fb3222bc640 1 -- 172.21.15.112:0/354912105 >> 172.21.15.112:0/354912105 conn(0x7fb31c0c3a80 msgr2=0x7fb31c071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:35.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.447+0000 7fb3222bc640 1 -- 172.21.15.112:0/354912105 shutdown_connections 2024-04-25T01:38:35.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.447+0000 7fb3222bc640 1 -- 172.21.15.112:0/354912105 wait complete. 2024-04-25T01:38:35.445 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:38:35.445 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T01:38:35.445 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:38:35.445 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:38:35.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.657+0000 7f301d3e2640 1 Processor -- start 2024-04-25T01:38:35.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.657+0000 7f301d3e2640 1 -- start start 2024-04-25T01:38:35.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.659+0000 7f301d3e2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f30180730b0 0x7f30180c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:35.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.659+0000 7f301d3e2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f30180c1fc0 0x7f301814e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:35.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.659+0000 7f301d3e2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3018155c70 con 0x7f301806f510 2024-04-25T01:38:35.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.659+0000 7f301d3e2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f30180ac200 con 0x7f30180c1fc0 2024-04-25T01:38:35.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.659+0000 7f300ffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f30180730b0 0x7f30180c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:35.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.659+0000 7f300ffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f30180730b0 0x7f30180c1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:35698/0 (socket says 172.21.15.112:35698) 2024-04-25T01:38:35.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.659+0000 7f300ffff640 1 -- 172.21.15.112:0/1380545271 learned_addr learned my addr 172.21.15.112:0/1380545271 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:35.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.659+0000 7f301d3e2640 1 -- 172.21.15.112:0/1380545271 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f30181557e0 con 0x7f30180730b0 2024-04-25T01:38:35.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.661+0000 7f30167fc640 1 -- 172.21.15.112:0/1380545271 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 115141678 0 0) 0x7f3018155c70 con 0x7f301806f510 2024-04-25T01:38:35.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.661+0000 7f30167fc640 1 -- 172.21.15.112:0/1380545271 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ff8003db0 con 0x7f301806f510 2024-04-25T01:38:35.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.661+0000 7f30177fe640 1 --2- 172.21.15.112:0/1380545271 >> v2:172.21.15.112:3300/0 conn(0x7f30180c1fc0 0x7f301814e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:35.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.661+0000 7f300ffff640 1 -- 172.21.15.112:0/1380545271 >> v2:172.21.15.112:3300/0 conn(0x7f30180c1fc0 msgr2=0x7f301814e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:38:35.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.661+0000 7f300ffff640 1 --2- 172.21.15.112:0/1380545271 >> v2:172.21.15.112:3300/0 conn(0x7f30180c1fc0 0x7f301814e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.661+0000 7f300ffff640 1 -- 172.21.15.112:0/1380545271 >> v1:172.21.15.187:6789/0 conn(0x7f301806f510 legacy=0x7f3018071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.662+0000 7f300ffff640 1 -- 172.21.15.112:0/1380545271 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30180ac450 con 0x7f30180730b0 2024-04-25T01:38:35.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.662+0000 7f30177fe640 1 --2- 172.21.15.112:0/1380545271 >> v2:172.21.15.112:3300/0 conn(0x7f30180c1fc0 0x7f301814e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:38:35.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.663+0000 7f300ffff640 1 --2- 172.21.15.112:0/1380545271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30180730b0 0x7f30180c1a80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3008009870 tx=0x7f300802a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3989c8baa38c7560 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:35.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.663+0000 7f30167fc640 1 -- 172.21.15.112:0/1380545271 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008037070 con 0x7f30180730b0 2024-04-25T01:38:35.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.663+0000 7f30167fc640 1 -- 172.21.15.112:0/1380545271 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3008032440 con 0x7f30180730b0 2024-04-25T01:38:35.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.663+0000 7f30167fc640 1 -- 172.21.15.112:0/1380545271 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008032810 con 0x7f30180730b0 2024-04-25T01:38:35.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.664+0000 7f301d3e2640 1 -- 172.21.15.112:0/1380545271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30180730b0 msgr2=0x7f30180c1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.664+0000 7f301d3e2640 1 --2- 172.21.15.112:0/1380545271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30180730b0 0x7f30180c1a80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3008009870 tx=0x7f300802a2a0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.664+0000 7f301d3e2640 1 -- 172.21.15.112:0/1380545271 shutdown_connections 2024-04-25T01:38:35.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.664+0000 7f301d3e2640 1 --2- 172.21.15.112:0/1380545271 >> v2:172.21.15.112:3300/0 conn(0x7f30180c1fc0 0x7f301814e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.664+0000 7f301d3e2640 1 --2- 172.21.15.112:0/1380545271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30180730b0 0x7f30180c1a80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.664+0000 7f301d3e2640 1 -- 172.21.15.112:0/1380545271 >> 172.21.15.112:0/1380545271 conn(0x7f30180c7080 msgr2=0x7f30180c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:35.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.665+0000 7f301d3e2640 1 -- 172.21.15.112:0/1380545271 shutdown_connections 2024-04-25T01:38:35.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.665+0000 7f301d3e2640 1 -- 172.21.15.112:0/1380545271 wait complete. 2024-04-25T01:38:35.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.667+0000 7f301d3e2640 1 Processor -- start 2024-04-25T01:38:35.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.667+0000 7f301d3e2640 1 -- start start 2024-04-25T01:38:35.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.668+0000 7f301d3e2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f301806f510 0x7f30181f00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:35.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.668+0000 7f301d3e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30180730b0 0x7f30181f3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:35.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.668+0000 7f301d3e2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30180c1fc0 0x7f30180bc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:35.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.668+0000 7f301d3e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3018155470 con 0x7f30180730b0 2024-04-25T01:38:35.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.668+0000 7f301d3e2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3018155190 con 0x7f301806f510 2024-04-25T01:38:35.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.668+0000 7f301d3e2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f30180c2430 con 0x7f30180c1fc0 2024-04-25T01:38:35.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.668+0000 7f30177fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30180c1fc0 0x7f30180bc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:35.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.668+0000 7f30177fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30180c1fc0 0x7f30180bc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:56230/0 (socket says 172.21.15.112:56230) 2024-04-25T01:38:35.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.668+0000 7f30177fe640 1 -- 172.21.15.112:0/1691301902 learned_addr learned my addr 172.21.15.112:0/1691301902 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:35.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.669+0000 7f300ffff640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30180730b0 0x7f30181f3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:35.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.669+0000 7f3016ffd640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f301806f510 0x7f30181f00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:35.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.669+0000 7f300ffff640 1 -- 172.21.15.112:0/1691301902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30180c1fc0 msgr2=0x7f30180bc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.669+0000 7f300ffff640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30180c1fc0 0x7f30180bc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.669+0000 7f300ffff640 1 -- 172.21.15.112:0/1691301902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f301806f510 msgr2=0x7f30181f00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.669+0000 7f300ffff640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f301806f510 0x7f30181f00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.669+0000 7f300ffff640 1 -- 172.21.15.112:0/1691301902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30181f3a80 con 0x7f30180730b0 2024-04-25T01:38:35.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.669+0000 7f3016ffd640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f301806f510 0x7f30181f00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:38:35.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.671+0000 7f30177fe640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30180c1fc0 0x7f30180bc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:38:35.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.671+0000 7f300ffff640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30180730b0 0x7f30181f3540 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f30180ac380 tx=0x7f30080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:35.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.672+0000 7f300f7fe640 1 -- 172.21.15.112:0/1691301902 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008041020 con 0x7f30180730b0 2024-04-25T01:38:35.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.673+0000 7f301d3e2640 1 -- 172.21.15.112:0/1691301902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3018152000 con 0x7f30180730b0 2024-04-25T01:38:35.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.673+0000 7f301d3e2640 1 -- 172.21.15.112:0/1691301902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30181f2d40 con 0x7f30180730b0 2024-04-25T01:38:35.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.673+0000 7f300f7fe640 1 -- 172.21.15.112:0/1691301902 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f300803d640 con 0x7f30180730b0 2024-04-25T01:38:35.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.674+0000 7f300f7fe640 1 -- 172.21.15.112:0/1691301902 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008037040 con 0x7f30180730b0 2024-04-25T01:38:35.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.677+0000 7f300f7fe640 1 -- 172.21.15.112:0/1691301902 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3008044020 con 0x7f30180730b0 2024-04-25T01:38:35.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.680+0000 7f301d3e2640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0002a70 0x7f2ff0004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:35.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.680+0000 7f301d3e2640 1 -- 172.21.15.112:0/1691301902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff0007ec0 con 0x7f2ff0002a70 2024-04-25T01:38:35.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.681+0000 7f300f7fe640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ffc06bf00 0x7f2ffc06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:35.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.682+0000 7f300f7fe640 1 -- 172.21.15.112:0/1691301902 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f30080ab6b0 con 0x7f30180730b0 2024-04-25T01:38:35.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.683+0000 7f3016ffd640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0002a70 0x7f2ff0004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:35.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.683+0000 7f30177fe640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ffc06bf00 0x7f2ffc06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:35.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.684+0000 7f3016ffd640 1 -- 172.21.15.112:0/1691301902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ff00080c0 con 0x7f2ff0002a70 2024-04-25T01:38:35.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.685+0000 7f30177fe640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ffc06bf00 0x7f2ffc06e3c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3010007520 tx=0x7f301000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:35.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.685+0000 7f3016ffd640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0002a70 0x7f2ff0004f30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f30000011e0 tx=0x7f300000ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:35.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.687+0000 7f300f7fe640 1 -- 172.21.15.112:0/1691301902 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30000052e0 con 0x7f2ff0002a70 2024-04-25T01:38:35.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.687+0000 7f300f7fe640 1 -- 172.21.15.112:0/1691301902 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3000005480 con 0x7f2ff0002a70 2024-04-25T01:38:35.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.687+0000 7f300f7fe640 1 -- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0002a70 msgr2=0x7f2ff0004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.687+0000 7f300f7fe640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0002a70 0x7f2ff0004f30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f30000011e0 tx=0x7f300000ff70 comp rx=0 tx=0).stop 2024-04-25T01:38:35.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.687+0000 7f300f7fe640 1 -- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0002a70 msgr2=0x7f2ff0004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:38:35.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.687+0000 7f300f7fe640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0002a70 0x7f2ff0004f30 secure :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f30000011e0 tx=0x7f300000ff70 comp rx=0 tx=0).stop 2024-04-25T01:38:35.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.715+0000 7f301d3e2640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0006eb0 0x7f2ff0007340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:35.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.716+0000 7f301d3e2640 1 -- 172.21.15.112:0/1691301902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff000a4a0 con 0x7f2ff0006eb0 2024-04-25T01:38:35.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.716+0000 7f3016ffd640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0006eb0 0x7f2ff0007340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:35.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.717+0000 7f3016ffd640 1 -- 172.21.15.112:0/1691301902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2ff0007880 con 0x7f2ff0006eb0 2024-04-25T01:38:35.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.717+0000 7f3016ffd640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0006eb0 0x7f2ff0007340 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3000016000 tx=0x7f3000001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:35.700 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:38:35.700 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:38:35.700 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:38:35.700 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:38:35.700 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 100, 2024-04-25T01:38:35.700 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 166532, 2024-04-25T01:38:35.700 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:38:35.700 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:38:35.700 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:38:35.700 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:38:35.700 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:35.701 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 15, 2024-04-25T01:38:35.701 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:38:35.701 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:38:35.701 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:38:35.701 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:35.701 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:35.701 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:35.701 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:35.701 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:35.701 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:35.701 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:35.701 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:35.702 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:35.702 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:35.702 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:35.702 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:35.702 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:38:35.702 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:38:35.702 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:35.702 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:35.702 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:35.702 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:35.702 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:35.702 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:35.703 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:35.703 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:35.703 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:35.703 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:35.703 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:35.703 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:35.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.703 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:38:35.703 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:38:35.703 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:38:35.703 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:38:35.703 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:38:35.703 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:38:35.704 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:38:35.704 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:38:35.704 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:38:35.704 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:38:35.704 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:38:35.704 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:38:35.704 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:38:35.704 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:38:35.704 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:38:35.704 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:38:35.704 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:38:35.704 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:35.705 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:35.705 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:35.705 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:35.705 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:35.705 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:35.705 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:35.705 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:35.705 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:35.705 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:35.705 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:35.705 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:35.705 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:38:35.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.706 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:38:35.706 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.706 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:38:35.706 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:38:35.706 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:35.706 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:35.706 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.706 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:35.706 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:38:35.706 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:35.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.706 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.706 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:35.706 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:38:35.707 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:35.707 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:35.707 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:35.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.707 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:38:35.707 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:38:35.707 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:35.707 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:35.707 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:35.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.707 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.707 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:38:35.707 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:38:35.708 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:35.708 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:35.708 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.708 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:35.708 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:38:35.708 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:35.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.708 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.708 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:35.708 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:38:35.708 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:35.708 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:35.708 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:35.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.709 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:38:35.709 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:38:35.709 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:35.709 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:35.709 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:35.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.709 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.709 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:38:35.709 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:38:35.709 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:35.709 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:35.709 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.710 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:35.710 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:38:35.710 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:35.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.710 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.710 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:35.710 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:38:35.710 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:35.710 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:35.710 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:35.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.710 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:38:35.710 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:38:35.711 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:35.711 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:35.711 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:35.711 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:35.711 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:35.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.711 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:38:35.711 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:38:35.711 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.712 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:35.712 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:35.712 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:38:35.712 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:35.712 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:35.712 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:38:35.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.712 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:35.712 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:35.712 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4 2024-04-25T01:38:35.712 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:35.712 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:35.712 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:38:35.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.713 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:35.713 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:35.713 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:38:35.713 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:35.713 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:35.713 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:38:35.713 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.713 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:35.713 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:35.713 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 6 2024-04-25T01:38:35.713 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:35.713 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:35.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.713 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:38:35.714 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:38:35.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.719+0000 7f300f7fe640 1 -- 172.21.15.112:0/1691301902 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000006c60 con 0x7f2ff0006eb0 2024-04-25T01:38:35.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.719+0000 7f300f7fe640 1 -- 172.21.15.112:0/1691301902 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4903 (secure 0 0 0) 0x7f3000006e00 con 0x7f2ff0006eb0 2024-04-25T01:38:35.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.719+0000 7f300f7fe640 1 -- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0006eb0 msgr2=0x7f2ff0007340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.719+0000 7f300f7fe640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0006eb0 0x7f2ff0007340 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3000016000 tx=0x7f3000001150 comp rx=0 tx=0).stop 2024-04-25T01:38:35.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.719+0000 7f300f7fe640 1 -- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0006eb0 msgr2=0x7f2ff0007340 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:38:35.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.719+0000 7f300f7fe640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0006eb0 0x7f2ff0007340 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.720+0000 7f301d3e2640 1 -- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ffc06bf00 msgr2=0x7f2ffc06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.720+0000 7f301d3e2640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ffc06bf00 0x7f2ffc06e3c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3010007520 tx=0x7f301000b040 comp rx=0 tx=0).stop 2024-04-25T01:38:35.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.721+0000 7f30177fe640 1 -- 172.21.15.112:0/1691301902 reap_dead start 2024-04-25T01:38:35.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.721+0000 7f301d3e2640 1 -- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30180730b0 msgr2=0x7f30181f3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.721+0000 7f301d3e2640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30180730b0 0x7f30181f3540 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f30180ac380 tx=0x7f30080046c0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.721+0000 7f301d3e2640 1 -- 172.21.15.112:0/1691301902 shutdown_connections 2024-04-25T01:38:35.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.721+0000 7f301d3e2640 1 --2- 172.21.15.112:0/1691301902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30180730b0 0x7f30181f3540 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f30180ac380 tx=0x7f30080046c0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.721+0000 7f301d3e2640 1 -- 172.21.15.112:0/1691301902 >> 172.21.15.112:0/1691301902 conn(0x7f30180c7080 msgr2=0x7f301806e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:35.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.721+0000 7f301d3e2640 1 -- 172.21.15.112:0/1691301902 shutdown_connections 2024-04-25T01:38:35.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.721+0000 7f301d3e2640 1 -- 172.21.15.112:0/1691301902 wait complete. 2024-04-25T01:38:35.721 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:38:35.721 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:38:35.721 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:38:35.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.922+0000 7fd4030df640 1 Processor -- start 2024-04-25T01:38:35.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.922+0000 7fd4030df640 1 -- start start 2024-04-25T01:38:35.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.923+0000 7fd4030df640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd3fc06c170 0x7fd3fc15a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:35.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.923+0000 7fd4030df640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3fc0c3fd0 con 0x7fd3fc0c3ad0 2024-04-25T01:38:35.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.923+0000 7fd4030df640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3fc15d410 con 0x7fd3fc071b20 2024-04-25T01:38:35.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.923+0000 7fd4030df640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fd3fc15dd40 con 0x7fd3fc06c170 2024-04-25T01:38:35.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.924+0000 7fd400e7b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd3fc06c170 0x7fd3fc15a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:35.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.924+0000 7fd3fbfff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd3fc0c3ad0 0x7fd3fc071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:49484/0 (socket says 172.21.15.112:49484) 2024-04-25T01:38:35.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.924+0000 7fd3fbfff640 1 -- 172.21.15.112:0/4155829226 learned_addr learned my addr 172.21.15.112:0/4155829226 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:38:35.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.924+0000 7fd3fb7fe640 1 -- 172.21.15.112:0/4155829226 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4209628012 0 0) 0x7fd3fc15d410 con 0x7fd3fc071b20 2024-04-25T01:38:35.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.924+0000 7fd3fb7fe640 1 -- 172.21.15.112:0/4155829226 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3ec003520 con 0x7fd3fc071b20 2024-04-25T01:38:35.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.925+0000 7fd3fb7fe640 1 -- 172.21.15.112:0/4155829226 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4029052737 0 0) 0x7fd3fc0c3fd0 con 0x7fd3fc0c3ad0 2024-04-25T01:38:35.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.925+0000 7fd400e7b640 1 -- 172.21.15.112:0/4155829226 >> v1:172.21.15.187:6789/0 conn(0x7fd3fc071b20 legacy=0x7fd3fc164700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.925+0000 7fd400e7b640 1 -- 172.21.15.112:0/4155829226 >> v1:172.21.15.112:6789/0 conn(0x7fd3fc0c3ad0 legacy=0x7fd3fc071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.925+0000 7fd400e7b640 1 -- 172.21.15.112:0/4155829226 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3fc15d6f0 con 0x7fd3fc06c170 2024-04-25T01:38:35.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.926+0000 7fd400e7b640 1 --2- 172.21.15.112:0/4155829226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3fc06c170 0x7fd3fc15a960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd3e80097a0 tx=0x7fd3e802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57cef4e149238e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:35.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.926+0000 7fd3fb7fe640 1 -- 172.21.15.112:0/4155829226 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8037070 con 0x7fd3fc06c170 2024-04-25T01:38:35.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.926+0000 7fd3fb7fe640 1 -- 172.21.15.112:0/4155829226 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e8032440 con 0x7fd3fc06c170 2024-04-25T01:38:35.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.927+0000 7fd3fb7fe640 1 -- 172.21.15.112:0/4155829226 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8032810 con 0x7fd3fc06c170 2024-04-25T01:38:35.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.927+0000 7fd4030df640 1 -- 172.21.15.112:0/4155829226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3fc06c170 msgr2=0x7fd3fc15a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.927+0000 7fd4030df640 1 --2- 172.21.15.112:0/4155829226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3fc06c170 0x7fd3fc15a960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd3e80097a0 tx=0x7fd3e802a1a0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.928+0000 7fd4030df640 1 -- 172.21.15.112:0/4155829226 shutdown_connections 2024-04-25T01:38:35.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.928+0000 7fd4030df640 1 --2- 172.21.15.112:0/4155829226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3fc06c170 0x7fd3fc15a960 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.928+0000 7fd4030df640 1 -- 172.21.15.112:0/4155829226 >> 172.21.15.112:0/4155829226 conn(0x7fd3fc06e450 msgr2=0x7fd3fc06e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:35.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.928+0000 7fd4030df640 1 -- 172.21.15.112:0/4155829226 shutdown_connections 2024-04-25T01:38:35.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.928+0000 7fd4030df640 1 -- 172.21.15.112:0/4155829226 wait complete. 2024-04-25T01:38:35.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.930+0000 7fd4030df640 1 Processor -- start 2024-04-25T01:38:35.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.930+0000 7fd4030df640 1 -- start start 2024-04-25T01:38:35.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.931+0000 7fd4030df640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3fc06c170 0x7fd3fc1fbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:35.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.931+0000 7fd4030df640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3fc071b20 0x7fd3fc1e5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:35.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.932+0000 7fd4030df640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3fc0c3ad0 0x7fd3fc0c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:35.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.932+0000 7fd4030df640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc072aa0 con 0x7fd3fc06c170 2024-04-25T01:38:35.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.932+0000 7fd4030df640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc15e1d0 con 0x7fd3fc071b20 2024-04-25T01:38:35.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.932+0000 7fd40167c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3fc0c3ad0 0x7fd3fc0c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:35.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.932+0000 7fd40167c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3fc0c3ad0 0x7fd3fc0c6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:56236/0 (socket says 172.21.15.112:56236) 2024-04-25T01:38:35.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.932+0000 7fd40167c640 1 -- 172.21.15.112:0/1635747980 learned_addr learned my addr 172.21.15.112:0/1635747980 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:35.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.932+0000 7fd3fbfff640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3fc071b20 0x7fd3fc1e5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:35.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.932+0000 7fd4030df640 1 -- 172.21.15.112:0/1635747980 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd3fc0748a0 con 0x7fd3fc0c3ad0 2024-04-25T01:38:35.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.933+0000 7fd400e7b640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3fc06c170 0x7fd3fc1fbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:35.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.933+0000 7fd400e7b640 1 -- 172.21.15.112:0/1635747980 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3fc0c3ad0 msgr2=0x7fd3fc0c6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.933+0000 7fd400e7b640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3fc0c3ad0 0x7fd3fc0c6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.933+0000 7fd400e7b640 1 -- 172.21.15.112:0/1635747980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3fc071b20 msgr2=0x7fd3fc1e5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.933+0000 7fd400e7b640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3fc071b20 0x7fd3fc1e5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.934+0000 7fd400e7b640 1 -- 172.21.15.112:0/1635747980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3fc1fc080 con 0x7fd3fc06c170 2024-04-25T01:38:35.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.934+0000 7fd400e7b640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3fc06c170 0x7fd3fc1fbb40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8031000 tx=0x7fd3e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:35.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.934+0000 7fd3f97fa640 1 -- 172.21.15.112:0/1635747980 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8041020 con 0x7fd3fc06c170 2024-04-25T01:38:35.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.935+0000 7fd4030df640 1 -- 172.21.15.112:0/1635747980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3fc1e5e60 con 0x7fd3fc06c170 2024-04-25T01:38:35.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.935+0000 7fd3f97fa640 1 -- 172.21.15.112:0/1635747980 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e803d6e0 con 0x7fd3fc06c170 2024-04-25T01:38:35.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.935+0000 7fd4030df640 1 -- 172.21.15.112:0/1635747980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3fc0a8cc0 con 0x7fd3fc06c170 2024-04-25T01:38:35.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.935+0000 7fd3f97fa640 1 -- 172.21.15.112:0/1635747980 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8037040 con 0x7fd3fc06c170 2024-04-25T01:38:35.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.935+0000 7fd3fbfff640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3fc071b20 0x7fd3fc1e5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:38:35.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.937+0000 7fd4030df640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3dc002a70 0x7fd3dc004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:35.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.937+0000 7fd4030df640 1 -- 172.21.15.112:0/1635747980 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc007ec0 con 0x7fd3dc002a70 2024-04-25T01:38:35.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.937+0000 7fd3fbfff640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3dc002a70 0x7fd3dc004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:35.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.938+0000 7fd3fbfff640 1 -- 172.21.15.112:0/1635747980 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd3dc0080c0 con 0x7fd3dc002a70 2024-04-25T01:38:35.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.938+0000 7fd3fbfff640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3dc002a70 0x7fd3dc004f30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0a7d10 tx=0x7fd3f0001cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:35.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.938+0000 7fd3f97fa640 1 -- 172.21.15.112:0/1635747980 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f000fa00 con 0x7fd3dc002a70 2024-04-25T01:38:35.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.939+0000 7fd3f97fa640 1 -- 172.21.15.112:0/1635747980 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd3e8044020 con 0x7fd3fc06c170 2024-04-25T01:38:35.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.940+0000 7fd3f97fa640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd3e006bd30 0x7fd3e006e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:35.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.940+0000 7fd3f97fa640 1 -- 172.21.15.112:0/1635747980 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fd3e80ab770 con 0x7fd3fc06c170 2024-04-25T01:38:35.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.940+0000 7fd40167c640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd3e006bd30 0x7fd3e006e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:35.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.940+0000 7fd3f97fa640 1 -- 172.21.15.112:0/1635747980 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd3f0018070 con 0x7fd3dc002a70 2024-04-25T01:38:35.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.940+0000 7fd3f97fa640 1 -- 172.21.15.112:0/1635747980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3dc002a70 msgr2=0x7fd3dc004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.940+0000 7fd3f97fa640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3dc002a70 0x7fd3dc004f30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0a7d10 tx=0x7fd3f0001cb0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.940+0000 7fd3f97fa640 1 -- 172.21.15.112:0/1635747980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3dc002a70 msgr2=0x7fd3dc004f30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:38:35.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.940+0000 7fd3f97fa640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3dc002a70 0x7fd3dc004f30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.941+0000 7fd40167c640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd3e006bd30 0x7fd3e006e1f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0c74f0 tx=0x7fd3f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:35.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.956+0000 7fd4030df640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3dc006f70 0x7fd3dc007400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:35.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.956+0000 7fd4030df640 1 -- 172.21.15.112:0/1635747980 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc00a4a0 con 0x7fd3dc006f70 2024-04-25T01:38:35.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.958+0000 7fd3fbfff640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3dc006f70 0x7fd3dc007400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:35.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.959+0000 7fd3fbfff640 1 -- 172.21.15.112:0/1635747980 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd3dc007940 con 0x7fd3dc006f70 2024-04-25T01:38:35.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.960+0000 7fd3fbfff640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3dc006f70 0x7fd3dc007400 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc008090 tx=0x7fd3f0000fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:35.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.962+0000 7fd3f97fa640 1 -- 172.21.15.112:0/1635747980 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f00138d0 con 0x7fd3dc006f70 2024-04-25T01:38:35.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.962+0000 7fd3f97fa640 1 -- 172.21.15.112:0/1635747980 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4560 (secure 0 0 0) 0x7fd3f0028470 con 0x7fd3dc006f70 2024-04-25T01:38:35.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.962+0000 7fd3f97fa640 1 -- 172.21.15.112:0/1635747980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3dc006f70 msgr2=0x7fd3dc007400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.962+0000 7fd3f97fa640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3dc006f70 0x7fd3dc007400 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc008090 tx=0x7fd3f0000fb0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.962+0000 7fd3f97fa640 1 -- 172.21.15.112:0/1635747980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3dc006f70 msgr2=0x7fd3dc007400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:38:35.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.962+0000 7fd3f97fa640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3dc006f70 0x7fd3dc007400 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc008090 tx=0x7fd3f0000fb0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.943 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:38:35.943 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:38:35.943 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:38:35.943 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:38:35.943 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 100, 2024-04-25T01:38:35.943 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 20725, 2024-04-25T01:38:35.944 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:38:35.944 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:38:35.944 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:38:35.944 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:38:35.944 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:35.944 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 16, 2024-04-25T01:38:35.944 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:38:35.944 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:38:35.944 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:38:35.944 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:35.944 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:35.944 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:35.945 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:35.945 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:35.945 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:35.945 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:35.945 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:35.945 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:35.945 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:35.945 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:35.945 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:35.945 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:38:35.945 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:38:35.945 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:35.945 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:35.945 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:35.945 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:35.946 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:35.946 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:35.946 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:35.946 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:35.946 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:35.946 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:35.946 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:35.946 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:35.946 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.946 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:38:35.946 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:38:35.946 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:38:35.946 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:38:35.947 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:38:35.947 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:38:35.947 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:38:35.947 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:38:35.947 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:38:35.947 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:38:35.947 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:38:35.947 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:38:35.947 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:38:35.947 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:38:35.947 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:38:35.947 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:38:35.947 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:38:35.947 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:35.947 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:35.948 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:35.948 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:35.948 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:35.948 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:35.948 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:35.948 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:35.948 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:35.948 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:35.948 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:35.948 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:35.948 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:38:35.948 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.948 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:38:35.948 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.949 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:38:35.949 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:38:35.949 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:35.949 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:35.949 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.949 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:35.949 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:38:35.949 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:35.949 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.949 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.949 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:35.949 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:38:35.949 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:35.949 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:38:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:38:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:38:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:38:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:35.951 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:38:35.951 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:35.951 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.951 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.951 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:35.951 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:38:35.951 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:35.951 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:35.951 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:35.951 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.951 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:38:35.951 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:38:35.951 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:35.951 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:35.952 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:35.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.952 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.952 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:38:35.952 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:38:35.952 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:35.952 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:35.952 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.952 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:35.952 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:38:35.952 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:35.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.952 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.952 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:35.952 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:38:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:38:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:38:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:38:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:38:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.954 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:35.954 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:35.954 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:38:35.954 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:35.954 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:35.954 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:38:35.954 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:35.954 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:35.954 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:35.954 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:38:35.954 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:35.954 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:35.954 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:35.954 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:38:35.955 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:38:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.963+0000 7fd4030df640 1 -- 172.21.15.112:0/1635747980 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd3e006bd30 msgr2=0x7fd3e006e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.963+0000 7fd4030df640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd3e006bd30 0x7fd3e006e1f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0c74f0 tx=0x7fd3f4009040 comp rx=0 tx=0).stop 2024-04-25T01:38:35.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.963+0000 7fd4030df640 1 -- 172.21.15.112:0/1635747980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3fc06c170 msgr2=0x7fd3fc1fbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.963+0000 7fd4030df640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3fc06c170 0x7fd3fc1fbb40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8031000 tx=0x7fd3e80046c0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.963+0000 7fd40167c640 1 -- 172.21.15.112:0/1635747980 reap_dead start 2024-04-25T01:38:35.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.963+0000 7fd4030df640 1 -- 172.21.15.112:0/1635747980 shutdown_connections 2024-04-25T01:38:35.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.963+0000 7fd4030df640 1 --2- 172.21.15.112:0/1635747980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3fc06c170 0x7fd3fc1fbb40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.963+0000 7fd4030df640 1 -- 172.21.15.112:0/1635747980 >> 172.21.15.112:0/1635747980 conn(0x7fd3fc06e450 msgr2=0x7fd3fc0c5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:35.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.964+0000 7fd4030df640 1 -- 172.21.15.112:0/1635747980 shutdown_connections 2024-04-25T01:38:35.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:35.965+0000 7fd4030df640 1 -- 172.21.15.112:0/1635747980 wait complete. 2024-04-25T01:38:35.970 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:38:35.971 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:38:35.971 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:38:36.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.166+0000 7f1481f2e640 1 Processor -- start 2024-04-25T01:38:36.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.166+0000 7f1481f2e640 1 -- start start 2024-04-25T01:38:36.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.167+0000 7f1481f2e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f14740d3c40 0x7f147404ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.167+0000 7f1481f2e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f147404b1b0 0x7f14740d6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.167+0000 7f1481f2e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14740468d0 con 0x7f14740d6fa0 2024-04-25T01:38:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.167+0000 7f1481f2e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1474047660 con 0x7f14740d3c40 2024-04-25T01:38:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.167+0000 7f1481f2e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f14740d65d0 con 0x7f147404b1b0 2024-04-25T01:38:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.168+0000 7f147bfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f147404b1b0 0x7f14740d6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.168+0000 7f147bfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f147404b1b0 0x7f14740d6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:56250/0 (socket says 172.21.15.112:56250) 2024-04-25T01:38:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.168+0000 7f147bfff640 1 -- 172.21.15.112:0/3109528354 learned_addr learned my addr 172.21.15.112:0/3109528354 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.168+0000 7f147bfff640 1 -- 172.21.15.112:0/3109528354 >> v2:172.21.15.187:3300/0 conn(0x7f14740d3c40 msgr2=0x7f147404ac70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:38:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.168+0000 7f147bfff640 1 --2- 172.21.15.112:0/3109528354 >> v2:172.21.15.187:3300/0 conn(0x7f14740d3c40 0x7f147404ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.168+0000 7f147bfff640 1 -- 172.21.15.112:0/3109528354 >> v1:172.21.15.112:6789/0 conn(0x7f14740d6fa0 legacy=0x7f14740d99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.168+0000 7f147bfff640 1 -- 172.21.15.112:0/3109528354 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1474046bb0 con 0x7f147404b1b0 2024-04-25T01:38:36.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.171+0000 7f147bfff640 1 --2- 172.21.15.112:0/3109528354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f147404b1b0 0x7f14740d6a60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f147c04fa20 tx=0x7f147c0a5f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5a1e7ac1a5d6757 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:36.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.172+0000 7f147b7fe640 1 -- 172.21.15.112:0/3109528354 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c0b3070 con 0x7f147404b1b0 2024-04-25T01:38:36.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.172+0000 7f147b7fe640 1 -- 172.21.15.112:0/3109528354 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f147c0a6cf0 con 0x7f147404b1b0 2024-04-25T01:38:36.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.172+0000 7f147b7fe640 1 -- 172.21.15.112:0/3109528354 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c0ae660 con 0x7f147404b1b0 2024-04-25T01:38:36.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.173+0000 7f1481f2e640 1 -- 172.21.15.112:0/3109528354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f147404b1b0 msgr2=0x7f14740d6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:36.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.173+0000 7f1481f2e640 1 --2- 172.21.15.112:0/3109528354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f147404b1b0 0x7f14740d6a60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f147c04fa20 tx=0x7f147c0a5f00 comp rx=0 tx=0).stop 2024-04-25T01:38:36.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.173+0000 7f1481f2e640 1 -- 172.21.15.112:0/3109528354 shutdown_connections 2024-04-25T01:38:36.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.173+0000 7f1481f2e640 1 --2- 172.21.15.112:0/3109528354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f147404b1b0 0x7f14740d6a60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:36.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.173+0000 7f1481f2e640 1 --2- 172.21.15.112:0/3109528354 >> v2:172.21.15.187:3300/0 conn(0x7f14740d3c40 0x7f147404ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:36.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.173+0000 7f1481f2e640 1 -- 172.21.15.112:0/3109528354 >> 172.21.15.112:0/3109528354 conn(0x7f1474005e90 msgr2=0x7f14740062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:36.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.175+0000 7f1481f2e640 1 -- 172.21.15.112:0/3109528354 shutdown_connections 2024-04-25T01:38:36.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.175+0000 7f1481f2e640 1 -- 172.21.15.112:0/3109528354 wait complete. 2024-04-25T01:38:36.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.177+0000 7f1481f2e640 1 Processor -- start 2024-04-25T01:38:36.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.177+0000 7f1481f2e640 1 -- start start 2024-04-25T01:38:36.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.178+0000 7f1481f2e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f147404b1b0 0x7f1474045400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:36.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.178+0000 7f1481f2e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14740d3c40 0x7f14740ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:36.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.178+0000 7f1481f2e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14740d6fa0 0x7f14740c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:36.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.178+0000 7f1481f2e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1474046360 con 0x7f14740d6fa0 2024-04-25T01:38:36.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.178+0000 7f1481f2e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f14740461e0 con 0x7f14740d3c40 2024-04-25T01:38:36.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.178+0000 7f1481f2e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1474046e90 con 0x7f147404b1b0 2024-04-25T01:38:36.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.179+0000 7f147bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14740d3c40 0x7f14740ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:36.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.179+0000 7f147bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14740d3c40 0x7f14740ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:35776/0 (socket says 172.21.15.112:35776) 2024-04-25T01:38:36.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.179+0000 7f147bfff640 1 -- 172.21.15.112:0/473062365 learned_addr learned my addr 172.21.15.112:0/473062365 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:36.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.179+0000 7f148172d640 1 --2- 172.21.15.112:0/473062365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14740d6fa0 0x7f14740c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:36.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.179+0000 7f147bfff640 1 -- 172.21.15.112:0/473062365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f147404b1b0 msgr2=0x7f1474045400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:38:36.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.179+0000 7f147bfff640 1 --2- 172.21.15.112:0/473062365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f147404b1b0 0x7f1474045400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:36.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.179+0000 7f147bfff640 1 -- 172.21.15.112:0/473062365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14740d6fa0 msgr2=0x7f14740c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:36.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.179+0000 7f147bfff640 1 --2- 172.21.15.112:0/473062365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14740d6fa0 0x7f14740c6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:36.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.179+0000 7f147bfff640 1 -- 172.21.15.112:0/473062365 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147416edb0 con 0x7f14740d3c40 2024-04-25T01:38:36.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.180+0000 7f147bfff640 1 --2- 172.21.15.112:0/473062365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14740d3c40 0x7f14740ca090 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f147c050770 tx=0x7f147c076a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:36.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.180+0000 7f14797fa640 1 -- 172.21.15.112:0/473062365 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c0bd070 con 0x7f14740d3c40 2024-04-25T01:38:36.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.180+0000 7f1481f2e640 1 -- 172.21.15.112:0/473062365 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f147416f090 con 0x7f14740d3c40 2024-04-25T01:38:36.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.180+0000 7f1481f2e640 1 -- 172.21.15.112:0/473062365 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f147402fd30 con 0x7f14740d3c40 2024-04-25T01:38:36.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.181+0000 7f1481f2e640 1 --2- 172.21.15.112:0/473062365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1474030c40 0x7f14740c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:36.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.182+0000 7f1481f2e640 1 -- 172.21.15.112:0/473062365 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f14740c7410 con 0x7f1474030c40 2024-04-25T01:38:36.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.182+0000 7f14797fa640 1 -- 172.21.15.112:0/473062365 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f147c0b77f0 con 0x7f14740d3c40 2024-04-25T01:38:36.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.182+0000 7f14797fa640 1 -- 172.21.15.112:0/473062365 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c0b3040 con 0x7f14740d3c40 2024-04-25T01:38:36.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.183+0000 7f14797fa640 1 -- 172.21.15.112:0/473062365 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f147c0c8060 con 0x7f14740d3c40 2024-04-25T01:38:36.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.184+0000 7f14797fa640 1 --2- 172.21.15.112:0/473062365 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f146006be30 0x7f146006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:36.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.184+0000 7f148172d640 1 --2- 172.21.15.112:0/473062365 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f146006be30 0x7f146006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:36.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.185+0000 7f14797fa640 1 -- 172.21.15.112:0/473062365 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f147c12ac40 con 0x7f14740d3c40 2024-04-25T01:38:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.185+0000 7f148172d640 1 --2- 172.21.15.112:0/473062365 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f146006be30 0x7f146006e2f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1470001660 tx=0x7f1470009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.186+0000 7f1480f2c640 1 --2- 172.21.15.112:0/473062365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1474030c40 0x7f14740c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:36.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.188+0000 7f1480f2c640 1 -- 172.21.15.112:0/473062365 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f147400b9c0 con 0x7f1474030c40 2024-04-25T01:38:36.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.188+0000 7f1480f2c640 1 --2- 172.21.15.112:0/473062365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1474030c40 0x7f14740c6ed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f146800b310 tx=0x7f146800b7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:36.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.191+0000 7f14797fa640 1 -- 172.21.15.112:0/473062365 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468015020 con 0x7f1474030c40 2024-04-25T01:38:36.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.191+0000 7f14797fa640 1 -- 172.21.15.112:0/473062365 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1468011830 con 0x7f1474030c40 2024-04-25T01:38:36.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.191+0000 7f14797fa640 1 -- 172.21.15.112:0/473062365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1474030c40 msgr2=0x7f14740c6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:36.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.191+0000 7f14797fa640 1 --2- 172.21.15.112:0/473062365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1474030c40 0x7f14740c6ed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f146800b310 tx=0x7f146800b7d0 comp rx=0 tx=0).stop 2024-04-25T01:38:36.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.191+0000 7f14797fa640 1 -- 172.21.15.112:0/473062365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1474030c40 msgr2=0x7f14740c6ed0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:38:36.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.191+0000 7f14797fa640 1 --2- 172.21.15.112:0/473062365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1474030c40 0x7f14740c6ed0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:36.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.208+0000 7f1481f2e640 1 --2- 172.21.15.112:0/473062365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1474171900 0x7f1474171d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:36.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.208+0000 7f1481f2e640 1 -- 172.21.15.112:0/473062365 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f147402ff70 con 0x7f1474171900 2024-04-25T01:38:36.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.210+0000 7f1480f2c640 1 --2- 172.21.15.112:0/473062365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1474171900 0x7f1474171d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:36.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.216+0000 7f1480f2c640 1 -- 172.21.15.112:0/473062365 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1474005390 con 0x7f1474171900 2024-04-25T01:38:36.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.216+0000 7f1480f2c640 1 --2- 172.21.15.112:0/473062365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1474171900 0x7f1474171d10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1468009fd0 tx=0x7f1468004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:36.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.219+0000 7f14797fa640 1 -- 172.21.15.112:0/473062365 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468015040 con 0x7f1474171900 2024-04-25T01:38:36.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.219+0000 7f14797fa640 1 -- 172.21.15.112:0/473062365 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4900 (secure 0 0 0) 0x7f1468011e40 con 0x7f1474171900 2024-04-25T01:38:36.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.219+0000 7f14797fa640 1 -- 172.21.15.112:0/473062365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1474171900 msgr2=0x7f1474171d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:36.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.219+0000 7f14797fa640 1 --2- 172.21.15.112:0/473062365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1474171900 0x7f1474171d10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1468009fd0 tx=0x7f1468004700 comp rx=0 tx=0).stop 2024-04-25T01:38:36.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.219+0000 7f14797fa640 1 -- 172.21.15.112:0/473062365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1474171900 msgr2=0x7f1474171d10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:38:36.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.219+0000 7f14797fa640 1 --2- 172.21.15.112:0/473062365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1474171900 0x7f1474171d10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:36.203 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:38:36.203 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:38:36.203 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:38:36.203 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:38:36.203 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 100, 2024-04-25T01:38:36.204 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 85582, 2024-04-25T01:38:36.204 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:38:36.204 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:38:36.204 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:38:36.204 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:38:36.204 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:36.204 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 16, 2024-04-25T01:38:36.204 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:38:36.204 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:38:36.204 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:38:36.204 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:36.204 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:36.204 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:36.204 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:36.205 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:36.205 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:36.205 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:36.205 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:36.205 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:36.205 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:36.205 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:36.205 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:36.205 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:38:36.205 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:38:36.205 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:36.205 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:36.205 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:36.205 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:36.205 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:36.206 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:36.206 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:36.206 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:36.206 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:36.206 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:36.206 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:36.206 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:36.206 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:36.206 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:38:36.206 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:38:36.207 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:38:36.207 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:38:36.207 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:38:36.207 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:38:36.207 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:38:36.207 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:38:36.207 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:38:36.207 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:38:36.207 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:38:36.207 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:38:36.207 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:38:36.207 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:38:36.207 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:38:36.207 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:38:36.207 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:38:36.208 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:36.208 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:36.208 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:36.208 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:36.208 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:36.208 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:36.208 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:36.208 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:36.209 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:36.209 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:36.209 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:36.209 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:36.209 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:38:36.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:36.209 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:38:36.209 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:36.209 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:38:36.209 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:38:36.209 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:36.209 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:36.209 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:36.209 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:36.210 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:38:36.210 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:36.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:36.210 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:36.210 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:36.210 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:38:36.210 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:36.210 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:36.210 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:36.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:36.210 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:38:36.210 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:38:36.210 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:36.210 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:36.211 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:36.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:36.211 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:36.211 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:38:36.211 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:38:36.211 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:36.211 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:36.211 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:36.211 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:36.211 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:38:36.211 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:36.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:36.212 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:36.212 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:36.212 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:38:36.212 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:36.212 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:36.212 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:36.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:36.212 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:38:36.212 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:38:36.212 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:36.212 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:36.212 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:36.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:36.213 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:36.213 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:38:36.213 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:38:36.213 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:36.213 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:36.213 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:36.213 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:36.213 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:38:36.213 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:36.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:36.213 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:36.213 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:36.213 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:38:36.213 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:36.214 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:36.214 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:36.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:36.214 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:38:36.214 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:38:36.214 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:36.214 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:36.214 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:36.214 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:36.214 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:36.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:36.214 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:38:36.214 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:38:36.214 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:36.214 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:36.215 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:36.215 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:38:36.215 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:36.215 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:36.215 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:38:36.215 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:36.215 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:36.215 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:36.215 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:38:36.215 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:36.215 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:36.215 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:38:36.215 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:36.215 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:36.216 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:36.216 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:38:36.216 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:36.216 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:36.216 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:38:36.216 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:36.216 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:36.216 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:36.216 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:38:36.216 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:36.216 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:36.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:36.216 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:38:36.216 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:38:36.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.221+0000 7f1481f2e640 1 -- 172.21.15.112:0/473062365 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f146006be30 msgr2=0x7f146006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:36.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.221+0000 7f1481f2e640 1 --2- 172.21.15.112:0/473062365 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f146006be30 0x7f146006e2f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1470001660 tx=0x7f1470009040 comp rx=0 tx=0).stop 2024-04-25T01:38:36.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.221+0000 7f148172d640 1 -- 172.21.15.112:0/473062365 reap_dead start 2024-04-25T01:38:36.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.222+0000 7f1481f2e640 1 -- 172.21.15.112:0/473062365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14740d3c40 msgr2=0x7f14740ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:36.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.222+0000 7f1481f2e640 1 --2- 172.21.15.112:0/473062365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14740d3c40 0x7f14740ca090 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f147c050770 tx=0x7f147c076a20 comp rx=0 tx=0).stop 2024-04-25T01:38:36.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.222+0000 7f1481f2e640 1 -- 172.21.15.112:0/473062365 shutdown_connections 2024-04-25T01:38:36.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.222+0000 7f1481f2e640 1 --2- 172.21.15.112:0/473062365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14740d3c40 0x7f14740ca090 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:36.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.222+0000 7f1481f2e640 1 -- 172.21.15.112:0/473062365 >> 172.21.15.112:0/473062365 conn(0x7f1474005e90 msgr2=0x7f1474041f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:36.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.223+0000 7f1481f2e640 1 -- 172.21.15.112:0/473062365 shutdown_connections 2024-04-25T01:38:36.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.223+0000 7f1481f2e640 1 -- 172.21.15.112:0/473062365 wait complete. 2024-04-25T01:38:36.229 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:38:36.229 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-04-25T01:38:36.230 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-25T01:38:36.230 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-04-25T01:38:36.230 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-04-25T01:38:36.230 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T01:38:36.230 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:38:36.252 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T01:38:36.252 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T01:38:36.252 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:38:36.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.491+0000 7fc89d909640 1 Processor -- start 2024-04-25T01:38:36.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.491+0000 7fc89d909640 1 -- start start 2024-04-25T01:38:36.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.495+0000 7fc89d909640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc898072140 0x7fc8981519a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:36.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.495+0000 7fc89d909640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc89806c9a0 0x7fc89806cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:36.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.495+0000 7fc89d909640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc898070cb0 con 0x7fc89813ab90 2024-04-25T01:38:36.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.495+0000 7fc89d909640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fc898070f90 con 0x7fc89806c9a0 2024-04-25T01:38:36.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.495+0000 7fc89d909640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc898071270 con 0x7fc898072140 2024-04-25T01:38:36.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.495+0000 7fc896ffd640 1 -- >> v1:172.21.15.112:6789/0 conn(0x7fc89813ab90 legacy=0x7fc8980c2990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:38:36.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.495+0000 7fc8967fc640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc898072140 0x7fc8981519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:36.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.495+0000 7fc8967fc640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc898072140 0x7fc8981519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:35842/0 (socket says 172.21.15.112:35842) 2024-04-25T01:38:36.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.495+0000 7fc8967fc640 1 -- 172.21.15.112:0/62073192 learned_addr learned my addr 172.21.15.112:0/62073192 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:36.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.496+0000 7fc8977fe640 1 -- 172.21.15.112:0/62073192 >> v2:172.21.15.112:3300/0 conn(0x7fc89806c9a0 msgr2=0x7fc89806cdb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:38:36.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.496+0000 7fc8977fe640 1 --2- 172.21.15.112:0/62073192 >> v2:172.21.15.112:3300/0 conn(0x7fc89806c9a0 0x7fc89806cdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:38:36.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.496+0000 7fc8967fc640 1 -- 172.21.15.112:0/62073192 >> v2:172.21.15.112:3300/0 conn(0x7fc89806c9a0 msgr2=0x7fc89806cdb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:38:36.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.496+0000 7fc8967fc640 1 --2- 172.21.15.112:0/62073192 >> v2:172.21.15.112:3300/0 conn(0x7fc89806c9a0 0x7fc89806cdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:36.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.496+0000 7fc8967fc640 1 -- 172.21.15.112:0/62073192 >> v1:172.21.15.112:6789/0 conn(0x7fc89813ab90 legacy=0x7fc8980c2990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:38:36.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.496+0000 7fc8967fc640 1 -- 172.21.15.112:0/62073192 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc898071550 con 0x7fc898072140 2024-04-25T01:38:36.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.497+0000 7fc8967fc640 1 --2- 172.21.15.112:0/62073192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc898072140 0x7fc8981519a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc89804fe10 tx=0x7fc89002ca50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4597da46d29879dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:36.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.499+0000 7fc895ffb640 1 -- 172.21.15.112:0/62073192 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890037020 con 0x7fc898072140 2024-04-25T01:38:36.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.499+0000 7fc895ffb640 1 -- 172.21.15.112:0/62073192 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc890033880 con 0x7fc898072140 2024-04-25T01:38:36.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.499+0000 7fc895ffb640 1 -- 172.21.15.112:0/62073192 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890033c50 con 0x7fc898072140 2024-04-25T01:38:36.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.500+0000 7fc89d909640 1 -- 172.21.15.112:0/62073192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc898072140 msgr2=0x7fc8981519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:36.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.500+0000 7fc89d909640 1 --2- 172.21.15.112:0/62073192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc898072140 0x7fc8981519a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc89804fe10 tx=0x7fc89002ca50 comp rx=0 tx=0).stop 2024-04-25T01:38:36.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.500+0000 7fc89d909640 1 -- 172.21.15.112:0/62073192 shutdown_connections 2024-04-25T01:38:36.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.500+0000 7fc89d909640 1 --2- 172.21.15.112:0/62073192 >> v2:172.21.15.112:3300/0 conn(0x7fc89806c9a0 0x7fc89806cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:36.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.500+0000 7fc89d909640 1 --2- 172.21.15.112:0/62073192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc898072140 0x7fc8981519a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:36.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.500+0000 7fc89d909640 1 -- 172.21.15.112:0/62073192 >> 172.21.15.112:0/62073192 conn(0x7fc8980c3a80 msgr2=0x7fc8980c3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:36.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.501+0000 7fc89d909640 1 -- 172.21.15.112:0/62073192 shutdown_connections 2024-04-25T01:38:36.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.501+0000 7fc89d909640 1 -- 172.21.15.112:0/62073192 wait complete. 2024-04-25T01:38:36.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.503+0000 7fc89d909640 1 Processor -- start 2024-04-25T01:38:36.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.503+0000 7fc89d909640 1 -- start start 2024-04-25T01:38:36.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.504+0000 7fc89d909640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc89806c9a0 0x7fc898151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:36.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.504+0000 7fc89d909640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc898072140 0x7fc89814eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:36.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.504+0000 7fc89d909640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc89813ab90 0x7fc8981f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:36.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.504+0000 7fc89d909640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8980c30a0 con 0x7fc89806c9a0 2024-04-25T01:38:36.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.504+0000 7fc89d909640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc898071b10 con 0x7fc898072140 2024-04-25T01:38:36.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.504+0000 7fc89d909640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc8980c3220 con 0x7fc89813ab90 2024-04-25T01:38:36.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.505+0000 7fc896ffd640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc89806c9a0 msgr2=0x7fc898151970 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:38:36.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.505+0000 7fc896ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc89806c9a0 0x7fc898151970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:38:36.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.506+0000 7fc8977fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc89813ab90 0x7fc8981f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:36.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.506+0000 7fc8977fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc89813ab90 0x7fc8981f7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:56320/0 (socket says 172.21.15.112:56320) 2024-04-25T01:38:36.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.506+0000 7fc8977fe640 1 -- 172.21.15.112:0/1042021006 learned_addr learned my addr 172.21.15.112:0/1042021006 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:36.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.507+0000 7fc8967fc640 1 --2- 172.21.15.112:0/1042021006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc898072140 0x7fc89814eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:36.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.507+0000 7fc8977fe640 1 -- 172.21.15.112:0/1042021006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc898072140 msgr2=0x7fc89814eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:36.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.507+0000 7fc8977fe640 1 --2- 172.21.15.112:0/1042021006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc898072140 0x7fc89814eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:36.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.507+0000 7fc8977fe640 1 -- 172.21.15.112:0/1042021006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc89806c9a0 msgr2=0x7fc898151970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:38:36.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.507+0000 7fc8977fe640 1 --2- 172.21.15.112:0/1042021006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc89806c9a0 0x7fc898151970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:36.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.507+0000 7fc8977fe640 1 -- 172.21.15.112:0/1042021006 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc898152430 con 0x7fc89813ab90 2024-04-25T01:38:36.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.508+0000 7fc8977fe640 1 --2- 172.21.15.112:0/1042021006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc89813ab90 0x7fc8981f7ff0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00b440 tx=0x7fc88c00b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:36.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.509+0000 7fc87ffff640 1 -- 172.21.15.112:0/1042021006 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c017070 con 0x7fc89813ab90 2024-04-25T01:38:36.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.509+0000 7fc87ffff640 1 -- 172.21.15.112:0/1042021006 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc88c003dc0 con 0x7fc89813ab90 2024-04-25T01:38:36.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.509+0000 7fc87ffff640 1 -- 172.21.15.112:0/1042021006 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c006310 con 0x7fc89813ab90 2024-04-25T01:38:36.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.511+0000 7fc89d909640 1 -- 172.21.15.112:0/1042021006 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8981fb690 con 0x7fc89813ab90 2024-04-25T01:38:36.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.511+0000 7fc89d909640 1 -- 172.21.15.112:0/1042021006 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc898152150 con 0x7fc89813ab90 2024-04-25T01:38:36.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.513+0000 7fc87ffff640 1 -- 172.21.15.112:0/1042021006 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc88c01a070 con 0x7fc89813ab90 2024-04-25T01:38:36.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.514+0000 7fc87ffff640 1 --2- 172.21.15.112:0/1042021006 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc88406be30 0x7fc88406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:36.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.514+0000 7fc896ffd640 1 --2- 172.21.15.112:0/1042021006 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc88406be30 0x7fc88406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:36.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.514+0000 7fc87ffff640 1 -- 172.21.15.112:0/1042021006 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fc88c08a4c0 con 0x7fc89813ab90 2024-04-25T01:38:36.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.514+0000 7fc89d909640 1 -- 172.21.15.112:0/1042021006 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8980682a0 con 0x7fc89813ab90 2024-04-25T01:38:36.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.515+0000 7fc896ffd640 1 --2- 172.21.15.112:0/1042021006 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc88406be30 0x7fc88406e2f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc888003980 tx=0x7fc888006d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:36.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:36.524+0000 7fc87ffff640 1 -- 172.21.15.112:0/1042021006 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc88c058b40 con 0x7fc89813ab90 2024-04-25T01:38:36.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:37.012+0000 7fc89d909640 1 -- 172.21.15.112:0/1042021006 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc898071f20 con 0x7fc89813ab90 2024-04-25T01:38:36.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:37.015+0000 7fc87ffff640 1 -- 172.21.15.112:0/1042021006 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1570 (secure 0 0 0) 0x7fc88c058440 con 0x7fc89813ab90 2024-04-25T01:38:37.000 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":100,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:38:37.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:37.025+0000 7fc89d909640 1 -- 172.21.15.112:0/1042021006 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc88406be30 msgr2=0x7fc88406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:37.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:37.025+0000 7fc89d909640 1 --2- 172.21.15.112:0/1042021006 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc88406be30 0x7fc88406e2f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc888003980 tx=0x7fc888006d30 comp rx=0 tx=0).stop 2024-04-25T01:38:37.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:37.026+0000 7fc89d909640 1 -- 172.21.15.112:0/1042021006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc89813ab90 msgr2=0x7fc8981f7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:37.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:37.026+0000 7fc89d909640 1 --2- 172.21.15.112:0/1042021006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc89813ab90 0x7fc8981f7ff0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00b440 tx=0x7fc88c00b900 comp rx=0 tx=0).stop 2024-04-25T01:38:37.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:37.026+0000 7fc89d909640 1 -- 172.21.15.112:0/1042021006 shutdown_connections 2024-04-25T01:38:37.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:37.026+0000 7fc89d909640 1 --2- 172.21.15.112:0/1042021006 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc88406be30 0x7fc88406e2f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:37.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:37.026+0000 7fc89d909640 1 --2- 172.21.15.112:0/1042021006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc89813ab90 0x7fc8981f7ff0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:37.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:37.026+0000 7fc89d909640 1 --2- 172.21.15.112:0/1042021006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc898072140 0x7fc89814eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:37.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:37.026+0000 7fc89d909640 1 --2- 172.21.15.112:0/1042021006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc89806c9a0 0x7fc898151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:37.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:37.026+0000 7fc89d909640 1 -- 172.21.15.112:0/1042021006 >> 172.21.15.112:0/1042021006 conn(0x7fc8980c3a80 msgr2=0x7fc8980abfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:37.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:37.026+0000 7fc89d909640 1 -- 172.21.15.112:0/1042021006 shutdown_connections 2024-04-25T01:38:37.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:37.026+0000 7fc89d909640 1 -- 172.21.15.112:0/1042021006 wait complete. 2024-04-25T01:38:40.027 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:38:40.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.217+0000 7fc2c2e61640 1 Processor -- start 2024-04-25T01:38:40.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.218+0000 7fc2c2e61640 1 -- start start 2024-04-25T01:38:40.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.219+0000 7fc2c2e61640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2bc071d80 0x7fc2bc153c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:40.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.219+0000 7fc2c2e61640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc2bc1541a0 0x7fc2bc15e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:40.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.219+0000 7fc2c2e61640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2bc153760 con 0x7fc2bc150d80 2024-04-25T01:38:40.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.219+0000 7fc2c2e61640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc2bc0a9c10 con 0x7fc2bc1541a0 2024-04-25T01:38:40.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.219+0000 7fc2c2e61640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc2bc0a9930 con 0x7fc2bc071d80 2024-04-25T01:38:40.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.219+0000 7fc2c0bfd640 1 -- >> v1:172.21.15.112:6789/0 conn(0x7fc2bc150d80 legacy=0x7fc2bc0729e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:38:40.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.220+0000 7fc2c13fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc2bc1541a0 0x7fc2bc15e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:40.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.220+0000 7fc2c13fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc2bc1541a0 0x7fc2bc15e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:53868/0 (socket says 172.21.15.112:53868) 2024-04-25T01:38:40.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.220+0000 7fc2c13fe640 1 -- 172.21.15.112:0/2510629379 learned_addr learned my addr 172.21.15.112:0/2510629379 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:40.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.220+0000 7fc2bbfff640 1 --2- 172.21.15.112:0/2510629379 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2bc071d80 0x7fc2bc153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:40.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.220+0000 7fc2bbfff640 1 -- 172.21.15.112:0/2510629379 >> v2:172.21.15.187:3300/0 conn(0x7fc2bc1541a0 msgr2=0x7fc2bc15e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:38:40.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.220+0000 7fc2bbfff640 1 --2- 172.21.15.112:0/2510629379 >> v2:172.21.15.187:3300/0 conn(0x7fc2bc1541a0 0x7fc2bc15e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:40.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.220+0000 7fc2bbfff640 1 -- 172.21.15.112:0/2510629379 >> v1:172.21.15.112:6789/0 conn(0x7fc2bc150d80 legacy=0x7fc2bc0729e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:38:40.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.220+0000 7fc2bbfff640 1 -- 172.21.15.112:0/2510629379 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2bc071720 con 0x7fc2bc071d80 2024-04-25T01:38:40.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.221+0000 7fc2bbfff640 1 --2- 172.21.15.112:0/2510629379 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2bc071d80 0x7fc2bc153c60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00dbc0 tx=0x7fc2ac02e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdff6ba69fef69f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:40.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.222+0000 7fc2bb7fe640 1 -- 172.21.15.112:0/2510629379 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac00d2d0 con 0x7fc2bc071d80 2024-04-25T01:38:40.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.222+0000 7fc2bb7fe640 1 -- 172.21.15.112:0/2510629379 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2ac00b450 con 0x7fc2bc071d80 2024-04-25T01:38:40.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.222+0000 7fc2bb7fe640 1 -- 172.21.15.112:0/2510629379 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac00b860 con 0x7fc2bc071d80 2024-04-25T01:38:40.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.223+0000 7fc2c2e61640 1 -- 172.21.15.112:0/2510629379 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2bc071d80 msgr2=0x7fc2bc153c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:40.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.223+0000 7fc2c2e61640 1 --2- 172.21.15.112:0/2510629379 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2bc071d80 0x7fc2bc153c60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00dbc0 tx=0x7fc2ac02e3c0 comp rx=0 tx=0).stop 2024-04-25T01:38:40.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.223+0000 7fc2c2e61640 1 -- 172.21.15.112:0/2510629379 shutdown_connections 2024-04-25T01:38:40.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.223+0000 7fc2c2e61640 1 --2- 172.21.15.112:0/2510629379 >> v2:172.21.15.187:3300/0 conn(0x7fc2bc1541a0 0x7fc2bc15e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:40.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.223+0000 7fc2c2e61640 1 --2- 172.21.15.112:0/2510629379 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2bc071d80 0x7fc2bc153c60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:40.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.223+0000 7fc2c2e61640 1 -- 172.21.15.112:0/2510629379 >> 172.21.15.112:0/2510629379 conn(0x7fc2bc0c3a80 msgr2=0x7fc2bc0c3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:40.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.224+0000 7fc2c2e61640 1 -- 172.21.15.112:0/2510629379 shutdown_connections 2024-04-25T01:38:40.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.225+0000 7fc2c2e61640 1 -- 172.21.15.112:0/2510629379 wait complete. 2024-04-25T01:38:40.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.227+0000 7fc2c2e61640 1 Processor -- start 2024-04-25T01:38:40.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.227+0000 7fc2c2e61640 1 -- start start 2024-04-25T01:38:40.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.228+0000 7fc2c2e61640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2bc071d80 0x7fc2bc14e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:40.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.228+0000 7fc2c2e61640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2bc150d80 0x7fc2bc0c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:40.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.228+0000 7fc2bbfff640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2bc150d80 msgr2=0x7fc2bc0c50c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:38:40.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.228+0000 7fc2bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2bc150d80 0x7fc2bc0c50c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:38:40.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.228+0000 7fc2c2e61640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2bc1541a0 0x7fc2bc0c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:40.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.228+0000 7fc2c2e61640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc151160 con 0x7fc2bc150d80 2024-04-25T01:38:40.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.228+0000 7fc2c2e61640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc156cd0 con 0x7fc2bc071d80 2024-04-25T01:38:40.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.228+0000 7fc2c2e61640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc2bc0a9fe0 con 0x7fc2bc1541a0 2024-04-25T01:38:40.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.229+0000 7fc2c0bfd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2bc071d80 0x7fc2bc14e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:40.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.229+0000 7fc2c0bfd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2bc071d80 0x7fc2bc14e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:53884/0 (socket says 172.21.15.112:53884) 2024-04-25T01:38:40.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.229+0000 7fc2c0bfd640 1 -- 172.21.15.112:0/4202450810 learned_addr learned my addr 172.21.15.112:0/4202450810 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:40.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.229+0000 7fc2c13fe640 1 --2- 172.21.15.112:0/4202450810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2bc1541a0 0x7fc2bc0c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:40.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.229+0000 7fc2c0bfd640 1 -- 172.21.15.112:0/4202450810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2bc1541a0 msgr2=0x7fc2bc0c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:40.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.229+0000 7fc2c0bfd640 1 --2- 172.21.15.112:0/4202450810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2bc1541a0 0x7fc2bc0c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:40.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.229+0000 7fc2c0bfd640 1 -- 172.21.15.112:0/4202450810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2bc150d80 msgr2=0x7fc2bc0c50c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:38:40.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.229+0000 7fc2c0bfd640 1 --2- 172.21.15.112:0/4202450810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2bc150d80 0x7fc2bc0c50c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:40.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.229+0000 7fc2c0bfd640 1 -- 172.21.15.112:0/4202450810 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2bc0c4c10 con 0x7fc2bc071d80 2024-04-25T01:38:40.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.230+0000 7fc2c0bfd640 1 --2- 172.21.15.112:0/4202450810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2bc071d80 0x7fc2bc14e840 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400ba10 tx=0x7fc2b400bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:40.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.231+0000 7fc2b97fa640 1 -- 172.21.15.112:0/4202450810 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b400cbd0 con 0x7fc2bc071d80 2024-04-25T01:38:40.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.231+0000 7fc2c2e61640 1 -- 172.21.15.112:0/4202450810 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2bc0c5b40 con 0x7fc2bc071d80 2024-04-25T01:38:40.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.231+0000 7fc2c2e61640 1 -- 172.21.15.112:0/4202450810 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2bc0c6180 con 0x7fc2bc071d80 2024-04-25T01:38:40.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.231+0000 7fc2b97fa640 1 -- 172.21.15.112:0/4202450810 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2b4010070 con 0x7fc2bc071d80 2024-04-25T01:38:40.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.231+0000 7fc2b97fa640 1 -- 172.21.15.112:0/4202450810 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4015470 con 0x7fc2bc071d80 2024-04-25T01:38:40.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.232+0000 7fc2c2e61640 1 -- 172.21.15.112:0/4202450810 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc29c005290 con 0x7fc2bc071d80 2024-04-25T01:38:40.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.233+0000 7fc2b97fa640 1 -- 172.21.15.112:0/4202450810 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc2b4015610 con 0x7fc2bc071d80 2024-04-25T01:38:40.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.234+0000 7fc2b97fa640 1 --2- 172.21.15.112:0/4202450810 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc2a806be30 0x7fc2a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:40.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.234+0000 7fc2b97fa640 1 -- 172.21.15.112:0/4202450810 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fc2b408b2f0 con 0x7fc2bc071d80 2024-04-25T01:38:40.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.235+0000 7fc2bbfff640 1 --2- 172.21.15.112:0/4202450810 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc2a806be30 0x7fc2a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:40.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.236+0000 7fc2bbfff640 1 --2- 172.21.15.112:0/4202450810 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc2a806be30 0x7fc2a806e2f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac037000 tx=0x7fc2ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:40.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.245+0000 7fc2b97fa640 1 -- 172.21.15.112:0/4202450810 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc2b40598e0 con 0x7fc2bc071d80 2024-04-25T01:38:40.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.740+0000 7fc2c2e61640 1 -- 172.21.15.112:0/4202450810 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc29c005580 con 0x7fc2bc071d80 2024-04-25T01:38:40.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.743+0000 7fc2b97fa640 1 -- 172.21.15.112:0/4202450810 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1570 (secure 0 0 0) 0x7fc2b40591e0 con 0x7fc2bc071d80 2024-04-25T01:38:40.726 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":100,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:38:40.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.749+0000 7fc2c2e61640 1 -- 172.21.15.112:0/4202450810 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc2a806be30 msgr2=0x7fc2a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:40.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.749+0000 7fc2c2e61640 1 --2- 172.21.15.112:0/4202450810 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc2a806be30 0x7fc2a806e2f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac037000 tx=0x7fc2ac008040 comp rx=0 tx=0).stop 2024-04-25T01:38:40.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.749+0000 7fc2c2e61640 1 -- 172.21.15.112:0/4202450810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2bc071d80 msgr2=0x7fc2bc14e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:40.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.749+0000 7fc2c2e61640 1 --2- 172.21.15.112:0/4202450810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2bc071d80 0x7fc2bc14e840 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400ba10 tx=0x7fc2b400bed0 comp rx=0 tx=0).stop 2024-04-25T01:38:40.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.749+0000 7fc2c2e61640 1 -- 172.21.15.112:0/4202450810 shutdown_connections 2024-04-25T01:38:40.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.749+0000 7fc2c2e61640 1 --2- 172.21.15.112:0/4202450810 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc2a806be30 0x7fc2a806e2f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:40.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.749+0000 7fc2c2e61640 1 --2- 172.21.15.112:0/4202450810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2bc1541a0 0x7fc2bc0c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:40.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.749+0000 7fc2c2e61640 1 --2- 172.21.15.112:0/4202450810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2bc150d80 0x7fc2bc0c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:40.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.749+0000 7fc2c2e61640 1 --2- 172.21.15.112:0/4202450810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2bc071d80 0x7fc2bc14e840 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400ba10 tx=0x7fc2b400bed0 comp rx=0 tx=0).stop 2024-04-25T01:38:40.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.750+0000 7fc2c2e61640 1 -- 172.21.15.112:0/4202450810 >> 172.21.15.112:0/4202450810 conn(0x7fc2bc0c3a80 msgr2=0x7fc2bc152fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:40.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.752+0000 7fc2c2e61640 1 -- 172.21.15.112:0/4202450810 shutdown_connections 2024-04-25T01:38:40.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:40.753+0000 7fc2c2e61640 1 -- 172.21.15.112:0/4202450810 wait complete. 2024-04-25T01:38:43.754 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:38:43.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.949+0000 7faddb858640 1 Processor -- start 2024-04-25T01:38:43.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.949+0000 7faddb858640 1 -- start start 2024-04-25T01:38:43.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.950+0000 7faddb858640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fadcc03e760 0x7fadcc03eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:43.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.950+0000 7faddb858640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadcc0dfd50 0x7fadcc0df2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:43.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.950+0000 7faddb858640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fadcc0e2510 0x7fadcc0e2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:43.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.950+0000 7faddb858640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fadcc046920 con 0x7fadcc0e2510 2024-04-25T01:38:43.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.950+0000 7faddb858640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fadcc0df820 con 0x7fadcc03e760 2024-04-25T01:38:43.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.950+0000 7faddb858640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fadcc0474a0 con 0x7fadcc0dfd50 2024-04-25T01:38:43.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.950+0000 7fadd9df5640 1 -- >> v2:172.21.15.112:3300/0 conn(0x7fadcc0e2510 msgr2=0x7fadcc0e2970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:38:43.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.950+0000 7fadd9df5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fadcc0e2510 0x7fadcc0e2970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:38:43.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.951+0000 7fadd8df3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadcc0dfd50 0x7fadcc0df2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:43.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.951+0000 7fadd8df3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadcc0dfd50 0x7fadcc0df2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:39734/0 (socket says 172.21.15.112:39734) 2024-04-25T01:38:43.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.951+0000 7fadd8df3640 1 -- 172.21.15.112:0/826334067 learned_addr learned my addr 172.21.15.112:0/826334067 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:43.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.953+0000 7fadd8df3640 1 -- 172.21.15.112:0/826334067 >> v2:172.21.15.187:3300/0 conn(0x7fadcc03e760 msgr2=0x7fadcc03eb40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:38:43.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.953+0000 7fadd8df3640 1 --2- 172.21.15.112:0/826334067 >> v2:172.21.15.187:3300/0 conn(0x7fadcc03e760 0x7fadcc03eb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:43.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.953+0000 7fadd8df3640 1 -- 172.21.15.112:0/826334067 >> v2:172.21.15.112:3300/0 conn(0x7fadcc0e2510 msgr2=0x7fadcc0e2970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:38:43.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.953+0000 7fadd8df3640 1 --2- 172.21.15.112:0/826334067 >> v2:172.21.15.112:3300/0 conn(0x7fadcc0e2510 0x7fadcc0e2970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:43.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.953+0000 7fadd8df3640 1 -- 172.21.15.112:0/826334067 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadcc047780 con 0x7fadcc0dfd50 2024-04-25T01:38:43.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.956+0000 7fadd8df3640 1 --2- 172.21.15.112:0/826334067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadcc0dfd50 0x7fadcc0df2e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fadcc02b030 tx=0x7fadc402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de8098629f2a2a57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:43.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.956+0000 7fadcbfff640 1 -- 172.21.15.112:0/826334067 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4037070 con 0x7fadcc0dfd50 2024-04-25T01:38:43.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.957+0000 7fadcbfff640 1 -- 172.21.15.112:0/826334067 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadc4032440 con 0x7fadcc0dfd50 2024-04-25T01:38:43.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.957+0000 7fadcbfff640 1 -- 172.21.15.112:0/826334067 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4032850 con 0x7fadcc0dfd50 2024-04-25T01:38:43.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.960+0000 7faddb858640 1 -- 172.21.15.112:0/826334067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadcc0dfd50 msgr2=0x7fadcc0df2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:43.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.960+0000 7faddb858640 1 --2- 172.21.15.112:0/826334067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadcc0dfd50 0x7fadcc0df2e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fadcc02b030 tx=0x7fadc402a210 comp rx=0 tx=0).stop 2024-04-25T01:38:43.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.962+0000 7faddb858640 1 -- 172.21.15.112:0/826334067 shutdown_connections 2024-04-25T01:38:43.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.962+0000 7faddb858640 1 --2- 172.21.15.112:0/826334067 >> v2:172.21.15.112:3300/0 conn(0x7fadcc0e2510 0x7fadcc0e2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:43.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.962+0000 7faddb858640 1 --2- 172.21.15.112:0/826334067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadcc0dfd50 0x7fadcc0df2e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:43.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.962+0000 7faddb858640 1 --2- 172.21.15.112:0/826334067 >> v2:172.21.15.187:3300/0 conn(0x7fadcc03e760 0x7fadcc03eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:43.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.962+0000 7faddb858640 1 -- 172.21.15.112:0/826334067 >> 172.21.15.112:0/826334067 conn(0x7fadcc048210 msgr2=0x7fadcc048620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:43.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.964+0000 7faddb858640 1 -- 172.21.15.112:0/826334067 shutdown_connections 2024-04-25T01:38:43.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.965+0000 7faddb858640 1 -- 172.21.15.112:0/826334067 wait complete. 2024-04-25T01:38:43.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.967+0000 7faddb858640 1 Processor -- start 2024-04-25T01:38:43.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.967+0000 7faddb858640 1 -- start start 2024-04-25T01:38:43.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.968+0000 7faddb858640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadcc03e760 0x7fadcc0d8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:43.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.968+0000 7faddb858640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadcc0dfd50 0x7fadcc178380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:43.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.968+0000 7faddb858640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadcc0e2510 0x7fadcc0dd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:43.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.968+0000 7faddb858640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fadcc0e1ff0 con 0x7fadcc03e760 2024-04-25T01:38:43.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.968+0000 7faddb858640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fadcc047d40 con 0x7fadcc0e2510 2024-04-25T01:38:43.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.968+0000 7faddb858640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fadcc00b960 con 0x7fadcc0dfd50 2024-04-25T01:38:43.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.971+0000 7fadd9df5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadcc0e2510 0x7fadcc0dd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:43.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.971+0000 7fadd9df5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadcc0e2510 0x7fadcc0dd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:53888/0 (socket says 172.21.15.112:53888) 2024-04-25T01:38:43.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.971+0000 7fadd9df5640 1 -- 172.21.15.112:0/833966346 learned_addr learned my addr 172.21.15.112:0/833966346 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:43.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.971+0000 7fadd9df5640 1 -- 172.21.15.112:0/833966346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadcc0dfd50 msgr2=0x7fadcc178380 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:38:43.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.971+0000 7fadd9df5640 1 --2- 172.21.15.112:0/833966346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadcc0dfd50 0x7fadcc178380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:43.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.971+0000 7fadd9df5640 1 -- 172.21.15.112:0/833966346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadcc03e760 msgr2=0x7fadcc0d8e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:38:43.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.971+0000 7fadd9df5640 1 --2- 172.21.15.112:0/833966346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadcc03e760 0x7fadcc0d8e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:43.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.971+0000 7fadd9df5640 1 -- 172.21.15.112:0/833966346 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadcc042b50 con 0x7fadcc0e2510 2024-04-25T01:38:43.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.974+0000 7fadd9df5640 1 --2- 172.21.15.112:0/833966346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadcc0e2510 0x7fadcc0dd370 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fadd4075bc0 tx=0x7fadd4068d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:43.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.976+0000 7fadc9ffb640 1 -- 172.21.15.112:0/833966346 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4070740 con 0x7fadcc0e2510 2024-04-25T01:38:43.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.976+0000 7fadc9ffb640 1 -- 172.21.15.112:0/833966346 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadd406e040 con 0x7fadcc0e2510 2024-04-25T01:38:43.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.976+0000 7fadc9ffb640 1 -- 172.21.15.112:0/833966346 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4070b80 con 0x7fadcc0e2510 2024-04-25T01:38:43.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.977+0000 7faddb858640 1 -- 172.21.15.112:0/833966346 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadcc0de370 con 0x7fadcc0e2510 2024-04-25T01:38:43.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.977+0000 7faddb858640 1 -- 172.21.15.112:0/833966346 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadcc042de0 con 0x7fadcc0e2510 2024-04-25T01:38:43.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.979+0000 7faddb858640 1 -- 172.21.15.112:0/833966346 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadcc005350 con 0x7fadcc0e2510 2024-04-25T01:38:43.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.980+0000 7fadc9ffb640 1 -- 172.21.15.112:0/833966346 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fadd4095430 con 0x7fadcc0e2510 2024-04-25T01:38:43.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.980+0000 7fadc9ffb640 1 --2- 172.21.15.112:0/833966346 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fadb806bf00 0x7fadb806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:43.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.980+0000 7fadc9ffb640 1 -- 172.21.15.112:0/833966346 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fadd4106ca0 con 0x7fadcc0e2510 2024-04-25T01:38:43.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.988+0000 7fadd95f4640 1 --2- 172.21.15.112:0/833966346 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fadb806bf00 0x7fadb806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:43.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.988+0000 7fadc9ffb640 1 -- 172.21.15.112:0/833966346 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fadd40d5320 con 0x7fadcc0e2510 2024-04-25T01:38:43.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:43.991+0000 7fadd95f4640 1 --2- 172.21.15.112:0/833966346 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fadb806bf00 0x7fadb806e3c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009670 tx=0x7fadd0006bd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:44.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:44.491+0000 7faddb858640 1 -- 172.21.15.112:0/833966346 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fadcc0425d0 con 0x7fadcc0e2510 2024-04-25T01:38:44.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:44.494+0000 7fadc9ffb640 1 -- 172.21.15.112:0/833966346 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1570 (secure 0 0 0) 0x7fadd40d4c20 con 0x7fadcc0e2510 2024-04-25T01:38:44.478 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":100,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:38:44.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:44.498+0000 7faddb858640 1 -- 172.21.15.112:0/833966346 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fadb806bf00 msgr2=0x7fadb806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:44.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:44.498+0000 7faddb858640 1 --2- 172.21.15.112:0/833966346 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fadb806bf00 0x7fadb806e3c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009670 tx=0x7fadd0006bd0 comp rx=0 tx=0).stop 2024-04-25T01:38:44.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:44.498+0000 7faddb858640 1 -- 172.21.15.112:0/833966346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadcc0e2510 msgr2=0x7fadcc0dd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:44.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:44.498+0000 7faddb858640 1 --2- 172.21.15.112:0/833966346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadcc0e2510 0x7fadcc0dd370 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fadd4075bc0 tx=0x7fadd4068d80 comp rx=0 tx=0).stop 2024-04-25T01:38:44.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:44.498+0000 7faddb858640 1 -- 172.21.15.112:0/833966346 shutdown_connections 2024-04-25T01:38:44.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:44.498+0000 7faddb858640 1 --2- 172.21.15.112:0/833966346 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fadb806bf00 0x7fadb806e3c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:44.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:44.498+0000 7faddb858640 1 --2- 172.21.15.112:0/833966346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadcc0e2510 0x7fadcc0dd370 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:44.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:44.498+0000 7faddb858640 1 --2- 172.21.15.112:0/833966346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadcc0dfd50 0x7fadcc178380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:44.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:44.498+0000 7faddb858640 1 --2- 172.21.15.112:0/833966346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadcc03e760 0x7fadcc0d8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:44.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:44.498+0000 7faddb858640 1 -- 172.21.15.112:0/833966346 >> 172.21.15.112:0/833966346 conn(0x7fadcc048210 msgr2=0x7fadcc02c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:44.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:44.498+0000 7faddb858640 1 -- 172.21.15.112:0/833966346 shutdown_connections 2024-04-25T01:38:44.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:44.498+0000 7faddb858640 1 -- 172.21.15.112:0/833966346 wait complete. 2024-04-25T01:38:47.500 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:38:47.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.687+0000 7f70c9d55640 1 Processor -- start 2024-04-25T01:38:47.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.687+0000 7f70c9d55640 1 -- start start 2024-04-25T01:38:47.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.688+0000 7f70c9d55640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 0x7f70bc04ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:47.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.688+0000 7f70c9d55640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 0x7f70bc0d6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:47.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.688+0000 7f70c9d55640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f70bc00b0d0 0x7f70bc0d6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:47.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.688+0000 7f70c9d55640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f70bc047660 con 0x7f70bc00b0d0 2024-04-25T01:38:47.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.688+0000 7f70c9d55640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f70bc0d65d0 con 0x7f70bc04b1b0 2024-04-25T01:38:47.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.688+0000 7f70c9d55640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f70bc0468d0 con 0x7f70bc0d3c40 2024-04-25T01:38:47.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.688+0000 7f70c9554640 1 -- >> v2:172.21.15.112:3300/0 conn(0x7f70bc00b0d0 msgr2=0x7f70bc0d6fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:38:47.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.688+0000 7f70c9554640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f70bc00b0d0 0x7f70bc0d6fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:38:47.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.689+0000 7f70c8d53640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 0x7f70bc04ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:47.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.689+0000 7f70c8d53640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 0x7f70bc04ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:54868/0 (socket says 172.21.15.112:54868) 2024-04-25T01:38:47.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.689+0000 7f70c8d53640 1 -- 172.21.15.112:0/3061825163 learned_addr learned my addr 172.21.15.112:0/3061825163 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:47.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.690+0000 7f70c8d53640 1 -- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 msgr2=0x7f70bc04ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:38:47.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.690+0000 7f70c8d53640 1 -- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 msgr2=0x7f70bc04ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:38:47.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.690+0000 7f70c8d53640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 0x7f70bc04ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:38:47.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.690+0000 7f70c8d53640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 0x7f70bc04ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:38:47.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.692+0000 7f70c3fff640 1 --2- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 0x7f70bc0d6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:47.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.694+0000 7f70c3fff640 1 -- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 msgr2=0x7f70bc0d6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:38:47.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.694+0000 7f70c3fff640 1 -- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 msgr2=0x7f70bc0d6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:38:47.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.694+0000 7f70c3fff640 1 --2- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 0x7f70bc0d6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:38:47.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.694+0000 7f70c3fff640 1 --2- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 0x7f70bc0d6a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:38:47.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.889+0000 7f70c9554640 1 -- 172.21.15.112:0/3061825163 >> v2:172.21.15.112:3300/0 conn(0x7f70bc00b0d0 msgr2=0x7f70bc0d6fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:38:47.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.889+0000 7f70c9554640 1 --2- 172.21.15.112:0/3061825163 >> v2:172.21.15.112:3300/0 conn(0x7f70bc00b0d0 0x7f70bc0d6fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:38:47.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.891+0000 7f70c8d53640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 0x7f70bc04ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:47.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.891+0000 7f70c8d53640 1 -- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 msgr2=0x7f70bc04ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:38:47.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.891+0000 7f70c8d53640 1 -- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 msgr2=0x7f70bc04ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:38:47.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.891+0000 7f70c8d53640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 0x7f70bc04ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:38:47.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.891+0000 7f70c8d53640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 0x7f70bc04ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:38:47.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.895+0000 7f70c3fff640 1 --2- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 0x7f70bc0d6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:47.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.895+0000 7f70c3fff640 1 -- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 msgr2=0x7f70bc0d6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:38:47.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.895+0000 7f70c3fff640 1 -- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 msgr2=0x7f70bc0d6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:38:47.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.895+0000 7f70c3fff640 1 --2- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 0x7f70bc0d6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:38:47.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:47.895+0000 7f70c3fff640 1 --2- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 0x7f70bc0d6a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:38:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:48.290+0000 7f70c9554640 1 -- 172.21.15.112:0/3061825163 >> v2:172.21.15.112:3300/0 conn(0x7f70bc00b0d0 msgr2=0x7f70bc0d6fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:38:48.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:48.290+0000 7f70c9554640 1 --2- 172.21.15.112:0/3061825163 >> v2:172.21.15.112:3300/0 conn(0x7f70bc00b0d0 0x7f70bc0d6fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:38:48.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:48.292+0000 7f70c8d53640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 0x7f70bc04ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:48.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:48.292+0000 7f70c8d53640 1 -- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 msgr2=0x7f70bc04ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:38:48.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:48.292+0000 7f70c8d53640 1 -- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 msgr2=0x7f70bc04ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:38:48.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:48.292+0000 7f70c8d53640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 0x7f70bc04ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:38:48.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:48.292+0000 7f70c8d53640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 0x7f70bc04ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:38:48.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:48.296+0000 7f70c3fff640 1 --2- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 0x7f70bc0d6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:48.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:48.296+0000 7f70c3fff640 1 -- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 msgr2=0x7f70bc0d6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:38:48.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:48.296+0000 7f70c3fff640 1 -- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 msgr2=0x7f70bc0d6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:38:48.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:48.296+0000 7f70c3fff640 1 --2- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 0x7f70bc0d6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:38:48.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:48.296+0000 7f70c3fff640 1 --2- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 0x7f70bc0d6a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:38:49.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:49.091+0000 7f70c9554640 1 -- 172.21.15.112:0/3061825163 >> v2:172.21.15.112:3300/0 conn(0x7f70bc00b0d0 msgr2=0x7f70bc0d6fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:38:49.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:49.091+0000 7f70c9554640 1 --2- 172.21.15.112:0/3061825163 >> v2:172.21.15.112:3300/0 conn(0x7f70bc00b0d0 0x7f70bc0d6fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:38:49.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:49.093+0000 7f70c8d53640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 0x7f70bc04ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:49.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:49.093+0000 7f70c8d53640 1 -- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 msgr2=0x7f70bc04ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:38:49.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:49.093+0000 7f70c8d53640 1 -- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 msgr2=0x7f70bc04ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:38:49.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:49.093+0000 7f70c8d53640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 0x7f70bc04ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:38:49.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:49.093+0000 7f70c8d53640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 0x7f70bc04ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:38:49.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:49.098+0000 7f70c3fff640 1 --2- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 0x7f70bc0d6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:49.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:49.098+0000 7f70c3fff640 1 -- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 msgr2=0x7f70bc0d6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:38:49.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:49.098+0000 7f70c3fff640 1 -- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 msgr2=0x7f70bc0d6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:38:49.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:49.098+0000 7f70c3fff640 1 --2- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 0x7f70bc0d6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:38:49.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:49.098+0000 7f70c3fff640 1 --2- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 0x7f70bc0d6a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:38:50.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.688+0000 7f70c27fc640 1 -- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 msgr2=0x7f70bc04ac70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:38:50.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.688+0000 7f70c27fc640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc0d3c40 0x7f70bc04ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:50.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.688+0000 7f70c27fc640 1 -- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 msgr2=0x7f70bc0d6a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:38:50.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.688+0000 7f70c27fc640 1 --2- 172.21.15.112:0/3061825163 >> v2:172.21.15.187:3300/0 conn(0x7f70bc04b1b0 0x7f70bc0d6a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:50.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.688+0000 7f70c27fc640 1 -- 172.21.15.112:0/3061825163 >> v2:172.21.15.112:3300/0 conn(0x7f70bc00b0d0 msgr2=0x7f70bc0d6fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:38:50.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.688+0000 7f70c27fc640 1 --2- 172.21.15.112:0/3061825163 >> v2:172.21.15.112:3300/0 conn(0x7f70bc00b0d0 0x7f70bc0d6fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:50.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.689+0000 7f70c3fff640 1 -- 172.21.15.112:0/3061825163 >> v1:172.21.15.112:6789/0 conn(0x7f70b0007320 legacy=0x7f70b00097e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:38:50.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.689+0000 7f70c27fc640 1 -- 172.21.15.112:0/3061825163 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70bc046bb0 con 0x7f70b0007320 2024-04-25T01:38:50.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.689+0000 7f70c27fc640 1 -- 172.21.15.112:0/3061825163 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70b000c0a0 con 0x7f70b0003670 2024-04-25T01:38:50.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.690+0000 7f70c37fe640 1 -- 172.21.15.112:0/3061825163 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1593387533 0 0) 0x7f70b000c0a0 con 0x7f70b0003670 2024-04-25T01:38:50.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.690+0000 7f70c37fe640 1 -- 172.21.15.112:0/3061825163 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70ac003520 con 0x7f70b0003670 2024-04-25T01:38:50.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.691+0000 7f70c37fe640 1 -- 172.21.15.112:0/3061825163 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1329127535 0 0) 0x7f70ac003520 con 0x7f70b0003670 2024-04-25T01:38:50.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.691+0000 7f70c37fe640 1 -- 172.21.15.112:0/3061825163 >> v1:172.21.15.112:6789/0 conn(0x7f70b0007320 legacy=0x7f70b00097e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:38:50.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.691+0000 7f70c37fe640 1 -- 172.21.15.112:0/3061825163 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b000d2e0 con 0x7f70b0003670 2024-04-25T01:38:50.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.691+0000 7f70c37fe640 1 -- 172.21.15.112:0/3061825163 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f70b400f040 con 0x7f70b0003670 2024-04-25T01:38:50.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.691+0000 7f70c37fe640 1 -- 172.21.15.112:0/3061825163 >> v1:172.21.15.187:6789/0 conn(0x7f70b0003670 legacy=0x7f70b0005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:50.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.691+0000 7f70c9554640 1 -- 172.21.15.112:0/3061825163 reap_dead start 2024-04-25T01:38:50.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.691+0000 7f70c37fe640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70ac0039b0 0x7f70ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:50.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.691+0000 7f70c37fe640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70ac004800 0x7f70ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:50.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.691+0000 7f70c37fe640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac0052e0 0x7f70ac00b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:50.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.691+0000 7f70c8d53640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70ac0039b0 0x7f70ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:50.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.691+0000 7f70c37fe640 1 -- 172.21.15.112:0/3061825163 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac00c140 con 0x7f70ac004800 2024-04-25T01:38:50.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.691+0000 7f70c37fe640 1 -- 172.21.15.112:0/3061825163 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac00c2f0 con 0x7f70ac0052e0 2024-04-25T01:38:50.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.691+0000 7f70c37fe640 1 -- 172.21.15.112:0/3061825163 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f70ac00c4f0 con 0x7f70ac0039b0 2024-04-25T01:38:50.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.691+0000 7f70c9554640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac0052e0 0x7f70ac00b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:50.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.692+0000 7f70c9554640 1 -- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70ac0039b0 msgr2=0x7f70ac003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:38:50.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.692+0000 7f70c9554640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70ac0039b0 0x7f70ac003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:50.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.692+0000 7f70c9554640 1 -- 172.21.15.112:0/3061825163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70ac004800 msgr2=0x7f70ac004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:38:50.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.692+0000 7f70c9554640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70ac004800 0x7f70ac004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:50.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.692+0000 7f70c9554640 1 -- 172.21.15.112:0/3061825163 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ac003520 con 0x7f70ac0052e0 2024-04-25T01:38:50.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.693+0000 7f70c9554640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac0052e0 0x7f70ac00b930 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f70c4068950 tx=0x7f70ac005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90d2e97cca74d53b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:50.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.693+0000 7f70c37fe640 1 -- 172.21.15.112:0/3061825163 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c404fa70 con 0x7f70ac0052e0 2024-04-25T01:38:50.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.693+0000 7f70c37fe640 1 -- 172.21.15.112:0/3061825163 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c40aa070 con 0x7f70ac0052e0 2024-04-25T01:38:50.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.694+0000 7f70c9d55640 1 -- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac0052e0 msgr2=0x7f70ac00b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:50.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.694+0000 7f70c9d55640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac0052e0 0x7f70ac00b930 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f70c4068950 tx=0x7f70ac005830 comp rx=0 tx=0).stop 2024-04-25T01:38:50.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.694+0000 7f70c9d55640 1 -- 172.21.15.112:0/3061825163 shutdown_connections 2024-04-25T01:38:50.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.694+0000 7f70c9d55640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac0052e0 0x7f70ac00b930 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:50.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.694+0000 7f70c9d55640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70ac004800 0x7f70ac004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:50.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.694+0000 7f70c9d55640 1 --2- 172.21.15.112:0/3061825163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70ac0039b0 0x7f70ac003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:50.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.694+0000 7f70c9d55640 1 -- 172.21.15.112:0/3061825163 >> 172.21.15.112:0/3061825163 conn(0x7f70bc005e90 msgr2=0x7f70bc0062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:50.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.695+0000 7f70c9d55640 1 -- 172.21.15.112:0/3061825163 shutdown_connections 2024-04-25T01:38:50.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.695+0000 7f70c9d55640 1 -- 172.21.15.112:0/3061825163 wait complete. 2024-04-25T01:38:50.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.696+0000 7f70c9d55640 1 Processor -- start 2024-04-25T01:38:50.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.697+0000 7f70c9d55640 1 -- start start 2024-04-25T01:38:50.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.698+0000 7f70c9d55640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac0039b0 0x7f70bc0d77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:50.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.698+0000 7f70c9d55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70ac004800 0x7f70bc0d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:50.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.698+0000 7f70c8d53640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac0039b0 0x7f70bc0d77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:50.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.698+0000 7f70c8d53640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac0039b0 0x7f70bc0d77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:46908/0 (socket says 172.21.15.112:46908) 2024-04-25T01:38:50.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.698+0000 7f70c8d53640 1 -- 172.21.15.112:0/1655825518 learned_addr learned my addr 172.21.15.112:0/1655825518 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:50.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.698+0000 7f70c9d55640 1 --2- 172.21.15.112:0/1655825518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc02eb00 0x7f70bc0cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:50.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.699+0000 7f70c9d55640 1 -- 172.21.15.112:0/1655825518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f70bc046360 con 0x7f70ac004800 2024-04-25T01:38:50.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.699+0000 7f70c9d55640 1 -- 172.21.15.112:0/1655825518 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f70bc0461e0 con 0x7f70ac0039b0 2024-04-25T01:38:50.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.699+0000 7f70c9d55640 1 -- 172.21.15.112:0/1655825518 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f70bc046e90 con 0x7f70bc02eb00 2024-04-25T01:38:50.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.699+0000 7f70c3fff640 1 -- 172.21.15.112:0/1655825518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70ac004800 msgr2=0x7f70bc0d2ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:38:50.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.699+0000 7f70c3fff640 1 --2- 172.21.15.112:0/1655825518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70ac004800 0x7f70bc0d2ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:38:50.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.699+0000 7f70c9554640 1 --2- 172.21.15.112:0/1655825518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc02eb00 0x7f70bc0cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:50.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.699+0000 7f70c8d53640 1 -- 172.21.15.112:0/1655825518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc02eb00 msgr2=0x7f70bc0cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:50.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.699+0000 7f70c8d53640 1 --2- 172.21.15.112:0/1655825518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc02eb00 0x7f70bc0cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:50.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.699+0000 7f70c8d53640 1 -- 172.21.15.112:0/1655825518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70ac004800 msgr2=0x7f70bc0d2ef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:38:50.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.699+0000 7f70c8d53640 1 --2- 172.21.15.112:0/1655825518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70ac004800 0x7f70bc0d2ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:50.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.699+0000 7f70c8d53640 1 -- 172.21.15.112:0/1655825518 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70bc0c8910 con 0x7f70ac0039b0 2024-04-25T01:38:50.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.700+0000 7f70c8d53640 1 --2- 172.21.15.112:0/1655825518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac0039b0 0x7f70bc0d77a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f70b40167e0 tx=0x7f70b4016ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:50.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.702+0000 7f70c17fa640 1 -- 172.21.15.112:0/1655825518 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b401c070 con 0x7f70ac0039b0 2024-04-25T01:38:50.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.703+0000 7f70c17fa640 1 -- 172.21.15.112:0/1655825518 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b4014d00 con 0x7f70ac0039b0 2024-04-25T01:38:50.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.704+0000 7f70c17fa640 1 -- 172.21.15.112:0/1655825518 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b400f040 con 0x7f70ac0039b0 2024-04-25T01:38:50.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.704+0000 7f70c9d55640 1 -- 172.21.15.112:0/1655825518 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70bc0c8bf0 con 0x7f70ac0039b0 2024-04-25T01:38:50.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.704+0000 7f70c9d55640 1 -- 172.21.15.112:0/1655825518 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70bc0c7de0 con 0x7f70ac0039b0 2024-04-25T01:38:50.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.706+0000 7f70c9d55640 1 -- 172.21.15.112:0/1655825518 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70bc0c7910 con 0x7f70ac0039b0 2024-04-25T01:38:50.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.711+0000 7f70c17fa640 1 -- 172.21.15.112:0/1655825518 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f70b4004270 con 0x7f70ac0039b0 2024-04-25T01:38:50.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.712+0000 7f70c17fa640 1 --2- 172.21.15.112:0/1655825518 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f70b006c4b0 0x7f70b006e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:50.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.712+0000 7f70c17fa640 1 -- 172.21.15.112:0/1655825518 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f70b408c9d0 con 0x7f70ac0039b0 2024-04-25T01:38:50.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.713+0000 7f70c3fff640 1 --2- 172.21.15.112:0/1655825518 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f70b006c4b0 0x7f70b006e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:50.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.714+0000 7f70c3fff640 1 --2- 172.21.15.112:0/1655825518 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f70b006c4b0 0x7f70b006e970 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0e3fc0 tx=0x7f70b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:50.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:50.722+0000 7f70c17fa640 1 -- 172.21.15.112:0/1655825518 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f70b405b050 con 0x7f70ac0039b0 2024-04-25T01:38:51.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.228+0000 7f70c9d55640 1 -- 172.21.15.112:0/1655825518 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f70bc0c7fe0 con 0x7f70ac0039b0 2024-04-25T01:38:51.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.232+0000 7f70c17fa640 1 -- 172.21.15.112:0/1655825518 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1563 (secure 0 0 0) 0x7f70b405a950 con 0x7f70ac0039b0 2024-04-25T01:38:51.216 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":102,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:38:51.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.238+0000 7f70c9d55640 1 -- 172.21.15.112:0/1655825518 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f70b006c4b0 msgr2=0x7f70b006e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:51.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.238+0000 7f70c9d55640 1 --2- 172.21.15.112:0/1655825518 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f70b006c4b0 0x7f70b006e970 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0e3fc0 tx=0x7f70b8008040 comp rx=0 tx=0).stop 2024-04-25T01:38:51.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.238+0000 7f70c9d55640 1 -- 172.21.15.112:0/1655825518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac0039b0 msgr2=0x7f70bc0d77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:51.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.238+0000 7f70c9d55640 1 --2- 172.21.15.112:0/1655825518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac0039b0 0x7f70bc0d77a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f70b40167e0 tx=0x7f70b4016ca0 comp rx=0 tx=0).stop 2024-04-25T01:38:51.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.239+0000 7f70c9d55640 1 -- 172.21.15.112:0/1655825518 shutdown_connections 2024-04-25T01:38:51.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.239+0000 7f70c9d55640 1 --2- 172.21.15.112:0/1655825518 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f70b006c4b0 0x7f70b006e970 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:51.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.239+0000 7f70c9d55640 1 --2- 172.21.15.112:0/1655825518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70bc02eb00 0x7f70bc0cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:51.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.239+0000 7f70c9d55640 1 --2- 172.21.15.112:0/1655825518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70ac004800 0x7f70bc0d2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:51.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.239+0000 7f70c9d55640 1 --2- 172.21.15.112:0/1655825518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac0039b0 0x7f70bc0d77a0 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f70b40167e0 tx=0x7f70b4016ca0 comp rx=0 tx=0).stop 2024-04-25T01:38:51.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.239+0000 7f70c9d55640 1 -- 172.21.15.112:0/1655825518 >> 172.21.15.112:0/1655825518 conn(0x7f70bc005e90 msgr2=0x7f70bc0d4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:51.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.254+0000 7f70c9d55640 1 -- 172.21.15.112:0/1655825518 shutdown_connections 2024-04-25T01:38:51.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.254+0000 7f70c9d55640 1 -- 172.21.15.112:0/1655825518 wait complete. 2024-04-25T01:38:51.258 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T01:38:51.258 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-04-25T01:38:51.258 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:38:51.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.461+0000 7fc42b59e640 1 Processor -- start 2024-04-25T01:38:51.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.461+0000 7fc42b59e640 1 -- start start 2024-04-25T01:38:51.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.463+0000 7fc42b59e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4240d0ff0 0x7fc42404dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:51.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.463+0000 7fc42b59e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc4240da420 0x7fc4240da860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:51.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.463+0000 7fc42b59e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4240daf50 con 0x7fc42400ec10 2024-04-25T01:38:51.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.463+0000 7fc42b59e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc42404e1e0 con 0x7fc4240da420 2024-04-25T01:38:51.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.463+0000 7fc42b59e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc4240db230 con 0x7fc4240d0ff0 2024-04-25T01:38:51.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.463+0000 7fc42ad9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc4240da420 0x7fc4240da860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:51.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.463+0000 7fc42ad9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc4240da420 0x7fc4240da860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:46922/0 (socket says 172.21.15.112:46922) 2024-04-25T01:38:51.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.463+0000 7fc42ad9d640 1 -- 172.21.15.112:0/1121248513 learned_addr learned my addr 172.21.15.112:0/1121248513 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:51.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.464+0000 7fc42a59c640 1 --2- 172.21.15.112:0/1121248513 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4240d0ff0 0x7fc42404dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:51.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.464+0000 7fc42a59c640 1 -- 172.21.15.112:0/1121248513 >> v2:172.21.15.187:3300/0 conn(0x7fc4240da420 msgr2=0x7fc4240da860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:38:51.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.464+0000 7fc42a59c640 1 --2- 172.21.15.112:0/1121248513 >> v2:172.21.15.187:3300/0 conn(0x7fc4240da420 0x7fc4240da860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:51.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.464+0000 7fc42a59c640 1 -- 172.21.15.112:0/1121248513 >> v1:172.21.15.187:6789/0 conn(0x7fc42400ec10 legacy=0x7fc424006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:51.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.464+0000 7fc42a59c640 1 -- 172.21.15.112:0/1121248513 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42400e410 con 0x7fc4240d0ff0 2024-04-25T01:38:51.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.468+0000 7fc42a59c640 1 --2- 172.21.15.112:0/1121248513 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4240d0ff0 0x7fc42404dca0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc4200097a0 tx=0x7fc42002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44f1ed943a82b619 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:51.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.472+0000 7fc42959a640 1 -- 172.21.15.112:0/1121248513 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc420037070 con 0x7fc4240d0ff0 2024-04-25T01:38:51.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.472+0000 7fc42959a640 1 -- 172.21.15.112:0/1121248513 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc420032440 con 0x7fc4240d0ff0 2024-04-25T01:38:51.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.472+0000 7fc42959a640 1 -- 172.21.15.112:0/1121248513 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc420037070 con 0x7fc4240d0ff0 2024-04-25T01:38:51.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.473+0000 7fc42b59e640 1 -- 172.21.15.112:0/1121248513 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4240d0ff0 msgr2=0x7fc42404dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:51.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.473+0000 7fc42b59e640 1 --2- 172.21.15.112:0/1121248513 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4240d0ff0 0x7fc42404dca0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc4200097a0 tx=0x7fc42002a130 comp rx=0 tx=0).stop 2024-04-25T01:38:51.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.474+0000 7fc42b59e640 1 -- 172.21.15.112:0/1121248513 shutdown_connections 2024-04-25T01:38:51.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.474+0000 7fc42b59e640 1 --2- 172.21.15.112:0/1121248513 >> v2:172.21.15.187:3300/0 conn(0x7fc4240da420 0x7fc4240da860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:51.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.474+0000 7fc42b59e640 1 --2- 172.21.15.112:0/1121248513 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4240d0ff0 0x7fc42404dca0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:51.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.474+0000 7fc42b59e640 1 -- 172.21.15.112:0/1121248513 >> 172.21.15.112:0/1121248513 conn(0x7fc42404c600 msgr2=0x7fc42404ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:51.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.474+0000 7fc42b59e640 1 -- 172.21.15.112:0/1121248513 shutdown_connections 2024-04-25T01:38:51.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.475+0000 7fc42b59e640 1 -- 172.21.15.112:0/1121248513 wait complete. 2024-04-25T01:38:51.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.476+0000 7fc42b59e640 1 Processor -- start 2024-04-25T01:38:51.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.477+0000 7fc42b59e640 1 -- start start 2024-04-25T01:38:51.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.478+0000 7fc42b59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc42400ec10 0x7fc4240d3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:51.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.478+0000 7fc42b59e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4240d0ff0 0x7fc4240d4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:51.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.478+0000 7fc42a59c640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc42400ec10 msgr2=0x7fc4240d3820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:38:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.478+0000 7fc42a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc42400ec10 0x7fc4240d3820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:38:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.478+0000 7fc42b59e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4240da420 0x7fc4240d25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.478+0000 7fc42b59e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc42400fba0 con 0x7fc42400ec10 2024-04-25T01:38:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.478+0000 7fc42b59e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc42400fa20 con 0x7fc4240d0ff0 2024-04-25T01:38:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.478+0000 7fc42b59e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc4240075b0 con 0x7fc4240da420 2024-04-25T01:38:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.478+0000 7fc429d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4240d0ff0 0x7fc4240d4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.478+0000 7fc429d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4240d0ff0 0x7fc4240d4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:46924/0 (socket says 172.21.15.112:46924) 2024-04-25T01:38:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.478+0000 7fc429d9b640 1 -- 172.21.15.112:0/1412997500 learned_addr learned my addr 172.21.15.112:0/1412997500 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.479+0000 7fc42ad9d640 1 --2- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4240da420 0x7fc4240d25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.479+0000 7fc42ad9d640 1 -- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4240d0ff0 msgr2=0x7fc4240d4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.479+0000 7fc42ad9d640 1 --2- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4240d0ff0 0x7fc4240d4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.479+0000 7fc42ad9d640 1 -- 172.21.15.112:0/1412997500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc42400ec10 msgr2=0x7fc4240d3820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:38:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.479+0000 7fc42ad9d640 1 --2- 172.21.15.112:0/1412997500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc42400ec10 0x7fc4240d3820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.479+0000 7fc42ad9d640 1 -- 172.21.15.112:0/1412997500 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc424033530 con 0x7fc4240da420 2024-04-25T01:38:51.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.480+0000 7fc42ad9d640 1 --2- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4240da420 0x7fc4240d25b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0756f0 tx=0x7fc42c068d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:51.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.484+0000 7fc41affd640 1 -- 172.21.15.112:0/1412997500 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c073510 con 0x7fc4240da420 2024-04-25T01:38:51.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.484+0000 7fc42b59e640 1 -- 172.21.15.112:0/1412997500 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc424033810 con 0x7fc4240da420 2024-04-25T01:38:51.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.485+0000 7fc42b59e640 1 -- 172.21.15.112:0/1412997500 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc424033dd0 con 0x7fc4240da420 2024-04-25T01:38:51.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.485+0000 7fc41affd640 1 -- 172.21.15.112:0/1412997500 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc42c0687a0 con 0x7fc4240da420 2024-04-25T01:38:51.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.486+0000 7fc41affd640 1 -- 172.21.15.112:0/1412997500 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c06f400 con 0x7fc4240da420 2024-04-25T01:38:51.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.487+0000 7fc42b59e640 1 --2- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc410002a70 0x7fc410004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:51.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.487+0000 7fc42b59e640 1 -- 172.21.15.112:0/1412997500 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc410007ec0 con 0x7fc410002a70 2024-04-25T01:38:51.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.487+0000 7fc42a59c640 1 --2- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc410002a70 0x7fc410004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:51.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.488+0000 7fc41affd640 1 -- 172.21.15.112:0/1412997500 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc42c06f5a0 con 0x7fc4240da420 2024-04-25T01:38:51.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.488+0000 7fc42a59c640 1 -- 172.21.15.112:0/1412997500 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc4100080c0 con 0x7fc410002a70 2024-04-25T01:38:51.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.488+0000 7fc42a59c640 1 --2- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc410002a70 0x7fc410004f30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc42404d700 tx=0x7fc4200046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:51.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.489+0000 7fc41affd640 1 --2- 172.21.15.112:0/1412997500 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc40c06bde0 0x7fc40c06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:51.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.489+0000 7fc41affd640 1 -- 172.21.15.112:0/1412997500 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fc42c106660 con 0x7fc4240da420 2024-04-25T01:38:51.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.489+0000 7fc429d9b640 1 --2- 172.21.15.112:0/1412997500 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc40c06bde0 0x7fc40c06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:51.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.489+0000 7fc41affd640 1 -- 172.21.15.112:0/1412997500 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc420037070 con 0x7fc410002a70 2024-04-25T01:38:51.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.489+0000 7fc41affd640 1 -- 172.21.15.112:0/1412997500 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc420004020 con 0x7fc410002a70 2024-04-25T01:38:51.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.489+0000 7fc41affd640 1 -- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc410002a70 msgr2=0x7fc410004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:51.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.489+0000 7fc41affd640 1 --2- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc410002a70 0x7fc410004f30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc42404d700 tx=0x7fc4200046c0 comp rx=0 tx=0).stop 2024-04-25T01:38:51.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.489+0000 7fc41affd640 1 -- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc410002a70 msgr2=0x7fc410004f30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:38:51.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.489+0000 7fc41affd640 1 --2- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc410002a70 0x7fc410004f30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:51.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.490+0000 7fc429d9b640 1 --2- 172.21.15.112:0/1412997500 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc40c06bde0 0x7fc40c06e2a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc42404f9b0 tx=0x7fc41c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:51.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.505+0000 7fc42b59e640 1 --2- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc410006f30 0x7fc4100073c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:51.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.505+0000 7fc42b59e640 1 -- 172.21.15.112:0/1412997500 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc41000a4a0 con 0x7fc410006f30 2024-04-25T01:38:51.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.505+0000 7fc42a59c640 1 --2- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc410006f30 0x7fc4100073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:51.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.508+0000 7fc42a59c640 1 -- 172.21.15.112:0/1412997500 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc410007900 con 0x7fc410006f30 2024-04-25T01:38:51.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.510+0000 7fc42a59c640 1 --2- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc410006f30 0x7fc4100073c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc4200097a0 tx=0x7fc420009190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:51.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.510+0000 7fc41affd640 1 -- 172.21.15.112:0/1412997500 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc420037040 con 0x7fc410006f30 2024-04-25T01:38:51.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.512+0000 7fc41affd640 1 -- 172.21.15.112:0/1412997500 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4720 (secure 0 0 0) 0x7fc420046950 con 0x7fc410006f30 2024-04-25T01:38:51.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.512+0000 7fc41affd640 1 -- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc410006f30 msgr2=0x7fc4100073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:51.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.512+0000 7fc41affd640 1 --2- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc410006f30 0x7fc4100073c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc4200097a0 tx=0x7fc420009190 comp rx=0 tx=0).stop 2024-04-25T01:38:51.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.513+0000 7fc41affd640 1 -- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc410006f30 msgr2=0x7fc4100073c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:38:51.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.513+0000 7fc41affd640 1 --2- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc410006f30 0x7fc4100073c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:51.495 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:38:51.495 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:38:51.495 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:38:51.495 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:38:51.495 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 102, 2024-04-25T01:38:51.495 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 36275, 2024-04-25T01:38:51.495 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:38:51.496 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:38:51.496 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:38:51.496 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:51.496 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:38:51.496 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:38:51.496 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:38:51.496 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:38:51.497 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:51.497 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:51.497 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:51.497 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:51.497 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:51.497 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:51.497 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:51.497 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:51.497 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:51.497 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:51.497 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:51.497 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:51.497 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:38:51.497 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:38:51.497 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:51.498 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:51.498 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:51.498 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:51.498 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:51.498 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:51.498 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:51.498 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:51.498 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:51.498 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:51.498 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:51.498 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:51.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.498 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:38:51.499 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:38:51.499 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:38:51.499 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:38:51.499 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:38:51.499 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:38:51.499 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:38:51.499 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:38:51.499 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:38:51.499 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:38:51.499 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:38:51.499 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:38:51.499 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:38:51.499 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:38:51.499 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:38:51.499 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:38:51.500 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:38:51.500 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:51.500 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:51.500 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:51.500 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:51.500 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:51.500 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:51.500 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:51.500 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:51.500 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:51.500 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:51.500 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:51.500 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:51.500 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:38:51.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.501 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:38:51.501 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.501 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:38:51.501 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:38:51.501 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:51.501 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:51.501 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.501 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:51.501 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:38:51.501 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:51.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.501 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.501 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:51.502 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:38:51.502 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:51.502 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:51.502 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:51.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.502 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:38:51.502 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:38:51.502 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:51.502 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:51.502 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:51.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.502 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.502 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:38:51.503 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:38:51.503 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:51.503 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:51.503 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.503 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:51.503 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:38:51.503 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:51.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.503 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.503 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:51.503 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:38:51.503 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:51.503 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:51.503 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:51.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.504 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:38:51.504 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:38:51.504 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:51.504 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:51.504 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:51.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.504 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.504 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:38:51.504 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:38:51.504 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:51.504 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:51.504 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.504 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:51.504 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:38:51.504 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:51.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.505 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.505 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:51.505 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:38:51.505 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:51.505 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:51.505 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:51.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.505 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:38:51.505 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:38:51.505 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:51.505 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:51.505 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:51.505 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:51.505 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:51.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.506 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:38:51.506 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:38:51.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.506 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:51.506 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:51.506 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:38:51.506 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:51.506 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:51.506 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:38:51.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.506 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:51.506 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:51.506 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:38:51.507 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:51.507 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:51.507 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:38:51.507 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.507 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:51.507 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:51.507 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:38:51.507 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:51.507 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:51.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.508 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:38:51.508 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:38:51.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.514+0000 7fc42b59e640 1 -- 172.21.15.112:0/1412997500 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc40c06bde0 msgr2=0x7fc40c06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:51.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.514+0000 7fc42b59e640 1 --2- 172.21.15.112:0/1412997500 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc40c06bde0 0x7fc40c06e2a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc42404f9b0 tx=0x7fc41c009040 comp rx=0 tx=0).stop 2024-04-25T01:38:51.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.514+0000 7fc42ad9d640 1 -- 172.21.15.112:0/1412997500 reap_dead start 2024-04-25T01:38:51.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.514+0000 7fc42b59e640 1 -- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4240da420 msgr2=0x7fc4240d25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:51.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.514+0000 7fc42b59e640 1 --2- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4240da420 0x7fc4240d25b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0756f0 tx=0x7fc42c068d80 comp rx=0 tx=0).stop 2024-04-25T01:38:51.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.514+0000 7fc42b59e640 1 -- 172.21.15.112:0/1412997500 shutdown_connections 2024-04-25T01:38:51.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.514+0000 7fc42b59e640 1 --2- 172.21.15.112:0/1412997500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4240da420 0x7fc4240d25b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:51.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.514+0000 7fc42b59e640 1 -- 172.21.15.112:0/1412997500 >> 172.21.15.112:0/1412997500 conn(0x7fc42404c600 msgr2=0x7fc42404c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:51.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.514+0000 7fc42b59e640 1 -- 172.21.15.112:0/1412997500 shutdown_connections 2024-04-25T01:38:51.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.514+0000 7fc42b59e640 1 -- 172.21.15.112:0/1412997500 wait complete. 2024-04-25T01:38:51.514 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:38:51.514 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-04-25T01:38:51.514 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:38:51.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.705+0000 7f99a23c4640 1 Processor -- start 2024-04-25T01:38:51.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.706+0000 7f99a23c4640 1 -- start start 2024-04-25T01:38:51.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.707+0000 7f99a23c4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f999c0c7b70 0x7f999c14e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:51.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.707+0000 7f99a23c4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c0730b0 0x7f999c14ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:51.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.707+0000 7f99a23c4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f999c155c70 con 0x7f999c06f510 2024-04-25T01:38:51.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.707+0000 7f99a23c4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f999c0ac200 con 0x7f999c0c7b70 2024-04-25T01:38:51.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.707+0000 7f99a23c4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f999c1557e0 con 0x7f999c0730b0 2024-04-25T01:38:51.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.707+0000 7f999bfff640 1 -- >> v1:172.21.15.112:6789/0 conn(0x7f999c06f510 legacy=0x7f999c071170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:38:51.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.708+0000 7f99a0961640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c0730b0 0x7f999c14ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:51.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.708+0000 7f99a0961640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c0730b0 0x7f999c14ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:54944/0 (socket says 172.21.15.112:54944) 2024-04-25T01:38:51.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.708+0000 7f99a0961640 1 -- 172.21.15.112:0/3013662947 learned_addr learned my addr 172.21.15.112:0/3013662947 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:51.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.708+0000 7f999b7fe640 1 --2- 172.21.15.112:0/3013662947 >> v2:172.21.15.187:3300/0 conn(0x7f999c0c7b70 0x7f999c14e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:51.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.709+0000 7f99a0961640 1 -- 172.21.15.112:0/3013662947 >> v2:172.21.15.187:3300/0 conn(0x7f999c0c7b70 msgr2=0x7f999c14e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:38:51.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.709+0000 7f99a0961640 1 --2- 172.21.15.112:0/3013662947 >> v2:172.21.15.187:3300/0 conn(0x7f999c0c7b70 0x7f999c14e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:51.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.709+0000 7f99a0961640 1 -- 172.21.15.112:0/3013662947 >> v1:172.21.15.112:6789/0 conn(0x7f999c06f510 legacy=0x7f999c071170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:38:51.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.709+0000 7f99a0961640 1 -- 172.21.15.112:0/3013662947 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c0ac450 con 0x7f999c0730b0 2024-04-25T01:38:51.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.709+0000 7f999b7fe640 1 --2- 172.21.15.112:0/3013662947 >> v2:172.21.15.187:3300/0 conn(0x7f999c0c7b70 0x7f999c14e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:38:51.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.715+0000 7f99a0961640 1 --2- 172.21.15.112:0/3013662947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c0730b0 0x7f999c14ea20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9994009e50 tx=0x7f999402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ff91676948180fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:51.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.717+0000 7f999affd640 1 -- 172.21.15.112:0/3013662947 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994036070 con 0x7f999c0730b0 2024-04-25T01:38:51.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.717+0000 7f999affd640 1 -- 172.21.15.112:0/3013662947 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9994031600 con 0x7f999c0730b0 2024-04-25T01:38:51.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.717+0000 7f999affd640 1 -- 172.21.15.112:0/3013662947 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99940319d0 con 0x7f999c0730b0 2024-04-25T01:38:51.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.718+0000 7f99a23c4640 1 -- 172.21.15.112:0/3013662947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c0730b0 msgr2=0x7f999c14ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:51.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.718+0000 7f99a23c4640 1 --2- 172.21.15.112:0/3013662947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c0730b0 0x7f999c14ea20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9994009e50 tx=0x7f999402a6d0 comp rx=0 tx=0).stop 2024-04-25T01:38:51.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.720+0000 7f99a23c4640 1 -- 172.21.15.112:0/3013662947 shutdown_connections 2024-04-25T01:38:51.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.720+0000 7f99a23c4640 1 --2- 172.21.15.112:0/3013662947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c0730b0 0x7f999c14ea20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:51.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.720+0000 7f99a23c4640 1 --2- 172.21.15.112:0/3013662947 >> v2:172.21.15.187:3300/0 conn(0x7f999c0c7b70 0x7f999c14e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:51.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.720+0000 7f99a23c4640 1 -- 172.21.15.112:0/3013662947 >> 172.21.15.112:0/3013662947 conn(0x7f999c0c7080 msgr2=0x7f999c0c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:51.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.724+0000 7f99a23c4640 1 -- 172.21.15.112:0/3013662947 shutdown_connections 2024-04-25T01:38:51.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.726+0000 7f99a23c4640 1 -- 172.21.15.112:0/3013662947 wait complete. 2024-04-25T01:38:51.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.727+0000 7f99a23c4640 1 Processor -- start 2024-04-25T01:38:51.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.728+0000 7f99a23c4640 1 -- start start 2024-04-25T01:38:51.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.729+0000 7f99a23c4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c06f510 0x7f999c13bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:51.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.729+0000 7f99a23c4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f999c0730b0 0x7f999c1515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:51.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.729+0000 7f99a23c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f999c0c7b70 0x7f999c151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:51.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.729+0000 7f99a23c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f999c155470 con 0x7f999c0c7b70 2024-04-25T01:38:51.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.729+0000 7f99a23c4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f999c155190 con 0x7f999c0730b0 2024-04-25T01:38:51.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.729+0000 7f99a23c4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f999c0ab5a0 con 0x7f999c06f510 2024-04-25T01:38:51.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.729+0000 7f99a0961640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f999c0c7b70 msgr2=0x7f999c151ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:38:51.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.729+0000 7f99a0961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f999c0c7b70 0x7f999c151ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:38:51.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.729+0000 7f999bfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c06f510 0x7f999c13bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:51.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.730+0000 7f999bfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c06f510 0x7f999c13bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:54946/0 (socket says 172.21.15.112:54946) 2024-04-25T01:38:51.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.730+0000 7f999bfff640 1 -- 172.21.15.112:0/3696009444 learned_addr learned my addr 172.21.15.112:0/3696009444 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:38:51.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.731+0000 7f999bfff640 1 -- 172.21.15.112:0/3696009444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f999c0730b0 msgr2=0x7f999c1515a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:38:51.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.731+0000 7f999bfff640 1 --2- 172.21.15.112:0/3696009444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f999c0730b0 0x7f999c1515a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:51.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.731+0000 7f999bfff640 1 -- 172.21.15.112:0/3696009444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f999c0c7b70 msgr2=0x7f999c151ae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:38:51.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.731+0000 7f999bfff640 1 --2- 172.21.15.112:0/3696009444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f999c0c7b70 0x7f999c151ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:51.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.731+0000 7f999bfff640 1 -- 172.21.15.112:0/3696009444 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c13c9f0 con 0x7f999c06f510 2024-04-25T01:38:51.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.736+0000 7f999bfff640 1 --2- 172.21.15.112:0/3696009444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c06f510 0x7f999c13bde0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f998c00b550 tx=0x7f998c00ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:51.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.737+0000 7f9998ff9640 1 -- 172.21.15.112:0/3696009444 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c003dc0 con 0x7f999c06f510 2024-04-25T01:38:51.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.737+0000 7f9998ff9640 1 -- 172.21.15.112:0/3696009444 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f998c015070 con 0x7f999c06f510 2024-04-25T01:38:51.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.737+0000 7f9998ff9640 1 -- 172.21.15.112:0/3696009444 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c0052b0 con 0x7f999c06f510 2024-04-25T01:38:51.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.737+0000 7f99a23c4640 1 -- 172.21.15.112:0/3696009444 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999c1f29a0 con 0x7f999c06f510 2024-04-25T01:38:51.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.738+0000 7f99a23c4640 1 -- 172.21.15.112:0/3696009444 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f999c13c320 con 0x7f999c06f510 2024-04-25T01:38:51.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.740+0000 7f99a23c4640 1 --2- 172.21.15.112:0/3696009444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c1f1bb0 0x7f999c1f1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:51.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.740+0000 7f9998ff9640 1 -- 172.21.15.112:0/3696009444 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f998c005450 con 0x7f999c06f510 2024-04-25T01:38:51.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.740+0000 7f99a23c4640 1 -- 172.21.15.112:0/3696009444 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f999c075980 con 0x7f999c1f1bb0 2024-04-25T01:38:51.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.741+0000 7f9998ff9640 1 --2- 172.21.15.112:0/3696009444 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f997c06be60 0x7f997c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:51.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.741+0000 7f99a0961640 1 --2- 172.21.15.112:0/3696009444 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f997c06be60 0x7f997c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:51.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.741+0000 7f9998ff9640 1 -- 172.21.15.112:0/3696009444 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f998c08aef0 con 0x7f999c06f510 2024-04-25T01:38:51.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.742+0000 7f99a0961640 1 --2- 172.21.15.112:0/3696009444 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f997c06be60 0x7f997c06e320 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9994033000 tx=0x7f9994009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:51.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.745+0000 7f999b7fe640 1 --2- 172.21.15.112:0/3696009444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c1f1bb0 0x7f999c1f1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:51.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.746+0000 7f999b7fe640 1 -- 172.21.15.112:0/3696009444 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f999c150be0 con 0x7f999c1f1bb0 2024-04-25T01:38:51.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.748+0000 7f999b7fe640 1 --2- 172.21.15.112:0/3696009444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c1f1bb0 0x7f999c1f1f90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f999c152680 tx=0x7f998400d890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:51.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.748+0000 7f9998ff9640 1 -- 172.21.15.112:0/3696009444 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9984018070 con 0x7f999c1f1bb0 2024-04-25T01:38:51.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.749+0000 7f9998ff9640 1 -- 172.21.15.112:0/3696009444 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9984004030 con 0x7f999c1f1bb0 2024-04-25T01:38:51.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.749+0000 7f9998ff9640 1 -- 172.21.15.112:0/3696009444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c1f1bb0 msgr2=0x7f999c1f1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:51.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.749+0000 7f9998ff9640 1 --2- 172.21.15.112:0/3696009444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c1f1bb0 0x7f999c1f1f90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f999c152680 tx=0x7f998400d890 comp rx=0 tx=0).stop 2024-04-25T01:38:51.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.749+0000 7f9998ff9640 1 -- 172.21.15.112:0/3696009444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c1f1bb0 msgr2=0x7f999c1f1f90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:38:51.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.749+0000 7f9998ff9640 1 --2- 172.21.15.112:0/3696009444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c1f1bb0 0x7f999c1f1f90 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f999c152680 tx=0x7f998400d890 comp rx=0 tx=0).stop 2024-04-25T01:38:51.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.776+0000 7f99a23c4640 1 --2- 172.21.15.112:0/3696009444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c14fc40 0x7f999c150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:51.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.777+0000 7f99a23c4640 1 -- 172.21.15.112:0/3696009444 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f999c0682a0 con 0x7f999c14fc40 2024-04-25T01:38:51.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.778+0000 7f999b7fe640 1 --2- 172.21.15.112:0/3696009444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c14fc40 0x7f999c150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:51.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.779+0000 7f999b7fe640 1 -- 172.21.15.112:0/3696009444 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f999c0aa6f0 con 0x7f999c14fc40 2024-04-25T01:38:51.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.780+0000 7f999b7fe640 1 --2- 172.21.15.112:0/3696009444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c14fc40 0x7f999c150050 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9984015000 tx=0x7f9984004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:51.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.782+0000 7f9998ff9640 1 -- 172.21.15.112:0/3696009444 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9984018030 con 0x7f999c14fc40 2024-04-25T01:38:51.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.783+0000 7f9998ff9640 1 -- 172.21.15.112:0/3696009444 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5059 (secure 0 0 0) 0x7f9984003610 con 0x7f999c14fc40 2024-04-25T01:38:51.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.783+0000 7f9998ff9640 1 -- 172.21.15.112:0/3696009444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c14fc40 msgr2=0x7f999c150050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:51.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.783+0000 7f9998ff9640 1 --2- 172.21.15.112:0/3696009444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c14fc40 0x7f999c150050 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9984015000 tx=0x7f9984004730 comp rx=0 tx=0).stop 2024-04-25T01:38:51.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.783+0000 7f9998ff9640 1 -- 172.21.15.112:0/3696009444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c14fc40 msgr2=0x7f999c150050 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:38:51.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.783+0000 7f9998ff9640 1 --2- 172.21.15.112:0/3696009444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c14fc40 0x7f999c150050 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9984015000 tx=0x7f9984004730 comp rx=0 tx=0).stop 2024-04-25T01:38:51.772 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:38:51.772 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:38:51.772 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:38:51.772 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:38:51.772 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 102, 2024-04-25T01:38:51.772 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 101147, 2024-04-25T01:38:51.772 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:38:51.772 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:38:51.772 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:38:51.772 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:51.773 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:38:51.773 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:38:51.773 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:38:51.773 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:38:51.773 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:51.773 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:51.773 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:51.773 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:51.773 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:51.773 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:51.773 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:51.773 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:51.773 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:51.773 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:51.774 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:51.774 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:51.774 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:38:51.774 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:38:51.774 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:51.774 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:51.774 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:51.774 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:51.774 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:51.774 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:51.774 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:51.774 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:51.774 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:51.774 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:51.775 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:51.775 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:51.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.775 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:38:51.775 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:38:51.775 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:38:51.775 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:38:51.775 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:38:51.775 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:38:51.775 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:38:51.775 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:38:51.775 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:38:51.775 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:38:51.775 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:38:51.775 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:38:51.776 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:38:51.776 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:38:51.776 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:38:51.776 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:38:51.776 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:38:51.776 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:38:51.776 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:38:51.776 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:38:51.776 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:38:51.776 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:38:51.776 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:38:51.776 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:38:51.776 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:38:51.776 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:38:51.777 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:38:51.777 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:38:51.777 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:51.777 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:38:51.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.777 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:38:51.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.777 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:38:51.777 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:38:51.777 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:51.777 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:51.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.777 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:51.777 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:38:51.777 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:51.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.778 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.778 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:51.778 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:38:51.778 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:51.778 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:51.778 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:51.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.778 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:38:51.778 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:38:51.778 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:51.778 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:51.778 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:51.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.779 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:38:51.779 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:38:51.779 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:51.779 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:51.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.779 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:51.779 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:38:51.779 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:51.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.779 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:51.779 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:38:51.779 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:51.780 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:51.780 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:51.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.780 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:38:51.780 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:38:51.780 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:51.780 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:51.780 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:51.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.780 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.780 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:38:51.780 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:38:51.780 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:38:51.780 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:38:51.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.781 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:38:51.781 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:38:51.781 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:51.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.781 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:38:51.781 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:38:51.781 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:38:51.781 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:51.781 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:51.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.781 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:38:51.781 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:38:51.781 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:38:51.782 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:38:51.782 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:38:51.782 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:51.782 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:51.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.782 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:38:51.782 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:38:51.782 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.782 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:51.782 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:51.782 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:38:51.782 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:51.782 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:51.783 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:38:51.783 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.783 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:51.783 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:51.783 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:38:51.783 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:51.783 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:51.783 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:38:51.783 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.783 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:51.783 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:51.783 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 8 2024-04-25T01:38:51.783 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:51.783 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:51.783 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:38:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 6 2024-04-25T01:38:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:38:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:38:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:38:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:38:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:38:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:38:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:38:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:38:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:38:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:38:51.784 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:38:51.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.791+0000 7f99927fc640 1 -- 172.21.15.112:0/3696009444 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f997c06be60 msgr2=0x7f997c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:51.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.791+0000 7f99927fc640 1 --2- 172.21.15.112:0/3696009444 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f997c06be60 0x7f997c06e320 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9994033000 tx=0x7f9994009000 comp rx=0 tx=0).stop 2024-04-25T01:38:51.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.791+0000 7f99927fc640 1 -- 172.21.15.112:0/3696009444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c06f510 msgr2=0x7f999c13bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:51.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.791+0000 7f99927fc640 1 --2- 172.21.15.112:0/3696009444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f999c06f510 0x7f999c13bde0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f998c00b550 tx=0x7f998c00ba10 comp rx=0 tx=0).stop 2024-04-25T01:38:51.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.793+0000 7f99a0961640 1 -- 172.21.15.112:0/3696009444 reap_dead start 2024-04-25T01:38:51.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.793+0000 7f99a0961640 1 -- 172.21.15.112:0/3696009444 reap_dead start 2024-04-25T01:38:51.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.795+0000 7f99927fc640 1 -- 172.21.15.112:0/3696009444 shutdown_connections 2024-04-25T01:38:51.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.795+0000 7f99927fc640 1 -- 172.21.15.112:0/3696009444 >> 172.21.15.112:0/3696009444 conn(0x7f999c0c7080 msgr2=0x7f999c0c2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:51.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.795+0000 7f99927fc640 1 -- 172.21.15.112:0/3696009444 shutdown_connections 2024-04-25T01:38:51.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:38:51.795+0000 7f99927fc640 1 -- 172.21.15.112:0/3696009444 wait complete. 2024-04-25T01:38:51.799 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:38:51.799 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T01:38:58.824 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:38:58.827 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:39:11.799 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-04-25T01:39:11.799 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-04-25T01:39:11.799 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T01:39:11.800 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T01:39:11.802 INFO:tasks.ceph.mon.a:Started 2024-04-25T01:39:11.802 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:39:11.802 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:39:11.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.003+0000 7f49728d4640 1 Processor -- start 2024-04-25T01:39:11.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.003+0000 7f49728d4640 1 -- start start 2024-04-25T01:39:11.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.004+0000 7f49728d4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f496c0c2bc0 0x7f496c157290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:11.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.004+0000 7f49728d4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f496c0c28e0 con 0x7f496c154620 2024-04-25T01:39:11.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.004+0000 7f49728d4640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f496c074560 con 0x7f496c06c3e0 2024-04-25T01:39:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.004+0000 7f49728d4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f496c0c3cd0 con 0x7f496c0c2bc0 2024-04-25T01:39:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.005+0000 7f496bfff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f496c06c3e0 0x7f496c153f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:59380/0 (socket says 172.21.15.112:59380) 2024-04-25T01:39:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.005+0000 7f496bfff640 1 -- 172.21.15.112:0/196084342 learned_addr learned my addr 172.21.15.112:0/196084342 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:39:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.010+0000 7f496affd640 1 -- 172.21.15.112:0/196084342 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 926604973 0 0) 0x7f496c074560 con 0x7f496c06c3e0 2024-04-25T01:39:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.010+0000 7f496affd640 1 -- 172.21.15.112:0/196084342 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4954003520 con 0x7f496c06c3e0 2024-04-25T01:39:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.010+0000 7f496b7fe640 1 --2- 172.21.15.112:0/196084342 >> v2:172.21.15.187:3300/0 conn(0x7f496c0c2bc0 0x7f496c157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:11.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.011+0000 7f496affd640 1 -- 172.21.15.112:0/196084342 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2409247635 0 0) 0x7f4954003520 con 0x7f496c06c3e0 2024-04-25T01:39:11.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.011+0000 7f496affd640 1 -- 172.21.15.112:0/196084342 >> v2:172.21.15.187:3300/0 conn(0x7f496c0c2bc0 msgr2=0x7f496c157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:39:11.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.011+0000 7f496affd640 1 --2- 172.21.15.112:0/196084342 >> v2:172.21.15.187:3300/0 conn(0x7f496c0c2bc0 0x7f496c157290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:11.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.011+0000 7f496affd640 1 -- 172.21.15.112:0/196084342 >> v1:172.21.15.112:6789/0 conn(0x7f496c154620 legacy=0x7f496c1577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:11.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.011+0000 7f496affd640 1 -- 172.21.15.112:0/196084342 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496c074840 con 0x7f496c06c3e0 2024-04-25T01:39:11.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.011+0000 7f496affd640 1 -- 172.21.15.112:0/196084342 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f49640019f0 con 0x7f496c06c3e0 2024-04-25T01:39:11.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.011+0000 7f496affd640 1 -- 172.21.15.112:0/196084342 >> v1:172.21.15.187:6789/0 conn(0x7f496c06c3e0 legacy=0x7f496c153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:11.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.011+0000 7f496affd640 1 --2- 172.21.15.112:0/196084342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49540039b0 0x7f4954003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:11.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.011+0000 7f496affd640 1 --2- 172.21.15.112:0/196084342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4954004800 0x7f4954004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:11.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.011+0000 7f496affd640 1 --2- 172.21.15.112:0/196084342 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49540053c0 0x7f495400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:11.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.011+0000 7f496affd640 1 -- 172.21.15.112:0/196084342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f495400c250 con 0x7f49540039b0 2024-04-25T01:39:11.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.011+0000 7f496affd640 1 -- 172.21.15.112:0/196084342 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f495400c400 con 0x7f4954004800 2024-04-25T01:39:11.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.011+0000 7f496affd640 1 -- 172.21.15.112:0/196084342 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f495400c600 con 0x7f49540053c0 2024-04-25T01:39:11.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.012+0000 7f496b7fe640 1 --2- 172.21.15.112:0/196084342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4954004800 0x7f4954004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:11.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.012+0000 7f4970e71640 1 --2- 172.21.15.112:0/196084342 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49540053c0 0x7f495400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:11.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.013+0000 7f4970e71640 1 -- 172.21.15.112:0/196084342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4954004800 msgr2=0x7f4954004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:39:11.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.013+0000 7f4970e71640 1 --2- 172.21.15.112:0/196084342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4954004800 0x7f4954004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:11.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.013+0000 7f4970e71640 1 -- 172.21.15.112:0/196084342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49540039b0 msgr2=0x7f4954003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:39:11.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.013+0000 7f4970e71640 1 --2- 172.21.15.112:0/196084342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49540039b0 0x7f4954003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:11.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.013+0000 7f4970e71640 1 -- 172.21.15.112:0/196084342 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496c074840 con 0x7f49540053c0 2024-04-25T01:39:11.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.013+0000 7f496b7fe640 1 --2- 172.21.15.112:0/196084342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4954004800 0x7f4954004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:39:11.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.014+0000 7f4970e71640 1 -- 172.21.15.112:0/196084342 reap_dead start 2024-04-25T01:39:11.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.017+0000 7f4970e71640 1 --2- 172.21.15.112:0/196084342 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49540053c0 0x7f495400b9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f495c001720 tx=0x7f4954005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3daf623193f2bb07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:11.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.018+0000 7f496affd640 1 -- 172.21.15.112:0/196084342 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c001370 con 0x7f49540053c0 2024-04-25T01:39:12.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.018+0000 7f496affd640 1 -- 172.21.15.112:0/196084342 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f495c0052d0 con 0x7f49540053c0 2024-04-25T01:39:12.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.018+0000 7f496affd640 1 -- 172.21.15.112:0/196084342 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c001370 con 0x7f49540053c0 2024-04-25T01:39:12.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.020+0000 7f49728d4640 1 -- 172.21.15.112:0/196084342 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49540053c0 msgr2=0x7f495400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:12.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.020+0000 7f49728d4640 1 --2- 172.21.15.112:0/196084342 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49540053c0 0x7f495400b9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f495c001720 tx=0x7f4954005830 comp rx=0 tx=0).stop 2024-04-25T01:39:12.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.020+0000 7f49728d4640 1 -- 172.21.15.112:0/196084342 shutdown_connections 2024-04-25T01:39:12.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.020+0000 7f49728d4640 1 --2- 172.21.15.112:0/196084342 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49540053c0 0x7f495400b9c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:12.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.020+0000 7f49728d4640 1 -- 172.21.15.112:0/196084342 >> 172.21.15.112:0/196084342 conn(0x7f496c0c42d0 msgr2=0x7f496c0c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:12.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.029+0000 7f49728d4640 1 -- 172.21.15.112:0/196084342 shutdown_connections 2024-04-25T01:39:12.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.029+0000 7f49728d4640 1 -- 172.21.15.112:0/196084342 wait complete. 2024-04-25T01:39:12.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.031+0000 7f49728d4640 1 Processor -- start 2024-04-25T01:39:12.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.031+0000 7f49728d4640 1 -- start start 2024-04-25T01:39:12.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.032+0000 7f49728d4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49540053c0 0x7f496c155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:12.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.032+0000 7f49728d4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f496c1ef190 0x7f496c0ab690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:12.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.032+0000 7f49728d4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f496c0a74d0 0x7f496c0a7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:12.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.032+0000 7f49728d4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f496c0c7350 con 0x7f496c0a74d0 2024-04-25T01:39:12.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.032+0000 7f49728d4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f496c0c2690 con 0x7f49540053c0 2024-04-25T01:39:12.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.032+0000 7f49728d4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f496c0a9190 con 0x7f496c1ef190 2024-04-25T01:39:12.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.032+0000 7f496b7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f496c1ef190 0x7f496c0ab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:12.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.032+0000 7f496b7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f496c1ef190 0x7f496c0ab690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:57042/0 (socket says 172.21.15.112:57042) 2024-04-25T01:39:12.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.032+0000 7f496b7fe640 1 -- 172.21.15.112:0/1270111806 learned_addr learned my addr 172.21.15.112:0/1270111806 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:12.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.033+0000 7f496b7fe640 1 -- 172.21.15.112:0/1270111806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49540053c0 msgr2=0x7f496c155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:12.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.033+0000 7f496b7fe640 1 --2- 172.21.15.112:0/1270111806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49540053c0 0x7f496c155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:12.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.033+0000 7f496b7fe640 1 -- 172.21.15.112:0/1270111806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f496c0a74d0 msgr2=0x7f496c0a7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:12.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.033+0000 7f496b7fe640 1 --2- 172.21.15.112:0/1270111806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f496c0a74d0 0x7f496c0a7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:12.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.033+0000 7f496b7fe640 1 -- 172.21.15.112:0/1270111806 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496c1ee6b0 con 0x7f496c1ef190 2024-04-25T01:39:12.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.033+0000 7f496b7fe640 1 --2- 172.21.15.112:0/1270111806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f496c1ef190 0x7f496c0ab690 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f496000e610 tx=0x7f496000ead0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:12.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.035+0000 7f4968ff9640 1 -- 172.21.15.112:0/1270111806 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960019020 con 0x7f496c1ef190 2024-04-25T01:39:12.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.035+0000 7f4968ff9640 1 -- 172.21.15.112:0/1270111806 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4960015690 con 0x7f496c1ef190 2024-04-25T01:39:12.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.035+0000 7f4968ff9640 1 -- 172.21.15.112:0/1270111806 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960015a40 con 0x7f496c1ef190 2024-04-25T01:39:12.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.038+0000 7f49728d4640 1 -- 172.21.15.112:0/1270111806 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496c1ee990 con 0x7f496c1ef190 2024-04-25T01:39:12.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.038+0000 7f49728d4640 1 -- 172.21.15.112:0/1270111806 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f496c0abbd0 con 0x7f496c1ef190 2024-04-25T01:39:12.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.039+0000 7f49728d4640 1 -- 172.21.15.112:0/1270111806 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f496c0689b0 con 0x7f496c1ef190 2024-04-25T01:39:12.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.041+0000 7f4968ff9640 1 -- 172.21.15.112:0/1270111806 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f496001c020 con 0x7f496c1ef190 2024-04-25T01:39:12.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.042+0000 7f4968ff9640 1 --2- 172.21.15.112:0/1270111806 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f495806be30 0x7f495806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:12.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.042+0000 7f4968ff9640 1 -- 172.21.15.112:0/1270111806 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f496008bcf0 con 0x7f496c1ef190 2024-04-25T01:39:12.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.042+0000 7f496bfff640 1 --2- 172.21.15.112:0/1270111806 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f495806be30 0x7f495806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:12.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.043+0000 7f496bfff640 1 --2- 172.21.15.112:0/1270111806 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f495806be30 0x7f495806e2f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f49640013d0 tx=0x7f4964007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:12.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.048+0000 7f4968ff9640 1 -- 172.21.15.112:0/1270111806 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4960023020 con 0x7f496c1ef190 2024-04-25T01:39:12.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.580+0000 7f49728d4640 1 -- 172.21.15.112:0/1270111806 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f496c0c66e0 con 0x7f496c1ef190 2024-04-25T01:39:12.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.583+0000 7f4968ff9640 1 -- 172.21.15.112:0/1270111806 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f4960060070 con 0x7f496c1ef190 2024-04-25T01:39:12.565 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":102,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:39:12.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.587+0000 7f49728d4640 1 -- 172.21.15.112:0/1270111806 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f495806be30 msgr2=0x7f495806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:12.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.587+0000 7f49728d4640 1 --2- 172.21.15.112:0/1270111806 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f495806be30 0x7f495806e2f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f49640013d0 tx=0x7f4964007ca0 comp rx=0 tx=0).stop 2024-04-25T01:39:12.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.587+0000 7f49728d4640 1 -- 172.21.15.112:0/1270111806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f496c1ef190 msgr2=0x7f496c0ab690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:12.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.587+0000 7f49728d4640 1 --2- 172.21.15.112:0/1270111806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f496c1ef190 0x7f496c0ab690 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f496000e610 tx=0x7f496000ead0 comp rx=0 tx=0).stop 2024-04-25T01:39:12.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.587+0000 7f49728d4640 1 -- 172.21.15.112:0/1270111806 shutdown_connections 2024-04-25T01:39:12.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.587+0000 7f49728d4640 1 --2- 172.21.15.112:0/1270111806 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f495806be30 0x7f495806e2f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:12.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.587+0000 7f49728d4640 1 --2- 172.21.15.112:0/1270111806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f496c0a74d0 0x7f496c0a7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:12.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.587+0000 7f49728d4640 1 --2- 172.21.15.112:0/1270111806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f496c1ef190 0x7f496c0ab690 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:12.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.587+0000 7f49728d4640 1 --2- 172.21.15.112:0/1270111806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49540053c0 0x7f496c155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:12.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.587+0000 7f49728d4640 1 -- 172.21.15.112:0/1270111806 >> 172.21.15.112:0/1270111806 conn(0x7f496c0c42d0 msgr2=0x7f496c072d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:12.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.590+0000 7f49728d4640 1 -- 172.21.15.112:0/1270111806 shutdown_connections 2024-04-25T01:39:12.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:12.590+0000 7f49728d4640 1 -- 172.21.15.112:0/1270111806 wait complete. 2024-04-25T01:39:15.596 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:39:15.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.783+0000 7fd1c81a9640 1 Processor -- start 2024-04-25T01:39:15.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.783+0000 7fd1c81a9640 1 -- start start 2024-04-25T01:39:15.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.784+0000 7fd1c81a9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd1b80dfd80 0x7fd1b803eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:15.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.784+0000 7fd1c81a9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1b80e2540 0x7fd1b80e29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:15.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.784+0000 7fd1c81a9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1b80474d0 con 0x7fd1b803e790 2024-04-25T01:39:15.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.784+0000 7fd1c81a9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fd1b8046950 con 0x7fd1b80dfd80 2024-04-25T01:39:15.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.784+0000 7fd1c81a9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b80477b0 con 0x7fd1b80e2540 2024-04-25T01:39:15.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.786+0000 7fd1c6746640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1b80e2540 0x7fd1b80e29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:15.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.787+0000 7fd1c6746640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1b80e2540 0x7fd1b80e29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:57100/0 (socket says 172.21.15.112:57100) 2024-04-25T01:39:15.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.787+0000 7fd1c6746640 1 -- 172.21.15.112:0/3482234824 learned_addr learned my addr 172.21.15.112:0/3482234824 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:15.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.788+0000 7fd1c6746640 1 -- 172.21.15.112:0/3482234824 >> v2:172.21.15.112:3300/0 conn(0x7fd1b80dfd80 msgr2=0x7fd1b803eb70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:39:15.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.788+0000 7fd1c6746640 1 --2- 172.21.15.112:0/3482234824 >> v2:172.21.15.112:3300/0 conn(0x7fd1b80dfd80 0x7fd1b803eb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:15.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.788+0000 7fd1c6746640 1 -- 172.21.15.112:0/3482234824 >> v1:172.21.15.112:6789/0 conn(0x7fd1b803e790 legacy=0x7fd1b80df310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:39:15.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.788+0000 7fd1c6746640 1 -- 172.21.15.112:0/3482234824 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b8047a90 con 0x7fd1b80e2540 2024-04-25T01:39:15.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.790+0000 7fd1c6746640 1 --2- 172.21.15.112:0/3482234824 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1b80e2540 0x7fd1b80e29d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0050ce0 tx=0x7fd1c00a5f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb864eefef3a6eeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:15.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.790+0000 7fd1c4f43640 1 -- 172.21.15.112:0/3482234824 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c00b3070 con 0x7fd1b80e2540 2024-04-25T01:39:15.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.790+0000 7fd1c4f43640 1 -- 172.21.15.112:0/3482234824 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c00a6d40 con 0x7fd1b80e2540 2024-04-25T01:39:15.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.791+0000 7fd1c81a9640 1 -- 172.21.15.112:0/3482234824 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1b80e2540 msgr2=0x7fd1b80e29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:15.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.791+0000 7fd1c81a9640 1 --2- 172.21.15.112:0/3482234824 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1b80e2540 0x7fd1b80e29d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0050ce0 tx=0x7fd1c00a5f30 comp rx=0 tx=0).stop 2024-04-25T01:39:15.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.793+0000 7fd1c81a9640 1 -- 172.21.15.112:0/3482234824 shutdown_connections 2024-04-25T01:39:15.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.793+0000 7fd1c81a9640 1 --2- 172.21.15.112:0/3482234824 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1b80e2540 0x7fd1b80e29d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:15.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.793+0000 7fd1c81a9640 1 --2- 172.21.15.112:0/3482234824 >> v2:172.21.15.112:3300/0 conn(0x7fd1b80dfd80 0x7fd1b803eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:15.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.793+0000 7fd1c81a9640 1 -- 172.21.15.112:0/3482234824 >> 172.21.15.112:0/3482234824 conn(0x7fd1b8048240 msgr2=0x7fd1b8048650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:15.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.793+0000 7fd1c81a9640 1 -- 172.21.15.112:0/3482234824 shutdown_connections 2024-04-25T01:39:15.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.798+0000 7fd1c81a9640 1 -- 172.21.15.112:0/3482234824 wait complete. 2024-04-25T01:39:15.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.800+0000 7fd1c81a9640 1 Processor -- start 2024-04-25T01:39:15.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.800+0000 7fd1c81a9640 1 -- start start 2024-04-25T01:39:15.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.801+0000 7fd1c81a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b803e790 0x7fd1b81a34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:15.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.801+0000 7fd1c81a9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1b80dfd80 0x7fd1b8041730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:15.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.801+0000 7fd1c81a9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1b80e2540 0x7fd1b819c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:15.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.801+0000 7fd1c81a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b800b990 con 0x7fd1b803e790 2024-04-25T01:39:15.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.801+0000 7fd1c81a9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b80e2020 con 0x7fd1b80e2540 2024-04-25T01:39:15.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.801+0000 7fd1c81a9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b800bb10 con 0x7fd1b80dfd80 2024-04-25T01:39:15.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.802+0000 7fd1c5f45640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b803e790 0x7fd1b81a34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.802+0000 7fd1c5f45640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b803e790 0x7fd1b81a34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59672/0 (socket says 172.21.15.112:59672) 2024-04-25T01:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.802+0000 7fd1c5f45640 1 -- 172.21.15.112:0/1746344429 learned_addr learned my addr 172.21.15.112:0/1746344429 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.802+0000 7fd1c5744640 1 --2- 172.21.15.112:0/1746344429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1b80dfd80 0x7fd1b8041730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.802+0000 7fd1c5f45640 1 -- 172.21.15.112:0/1746344429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b803e790 msgr2=0x7fd1b81a34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-25T01:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.802+0000 7fd1c5f45640 1 -- 172.21.15.112:0/1746344429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b803e790 msgr2=0x7fd1b81a34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.802+0000 7fd1c5f45640 1 --2- 172.21.15.112:0/1746344429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b803e790 0x7fd1b81a34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.802+0000 7fd1c5f45640 1 --2- 172.21.15.112:0/1746344429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b803e790 0x7fd1b81a34a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:39:15.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.804+0000 7fd1c6746640 1 --2- 172.21.15.112:0/1746344429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1b80e2540 0x7fd1b819c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:15.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.806+0000 7fd1c6746640 1 -- 172.21.15.112:0/1746344429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1b80dfd80 msgr2=0x7fd1b8041730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:15.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.806+0000 7fd1c6746640 1 --2- 172.21.15.112:0/1746344429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1b80dfd80 0x7fd1b8041730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:15.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.806+0000 7fd1c6746640 1 -- 172.21.15.112:0/1746344429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b803e790 msgr2=0x7fd1b81a34a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:39:15.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.806+0000 7fd1c6746640 1 --2- 172.21.15.112:0/1746344429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b803e790 0x7fd1b81a34a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:15.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.806+0000 7fd1c6746640 1 -- 172.21.15.112:0/1746344429 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b80d7670 con 0x7fd1b80e2540 2024-04-25T01:39:15.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.806+0000 7fd1c6746640 1 --2- 172.21.15.112:0/1746344429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1b80e2540 0x7fd1b819c780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0075bc0 tx=0x7fd1c0076a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:15.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.807+0000 7fd1b67fc640 1 -- 172.21.15.112:0/1746344429 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0087070 con 0x7fd1b80e2540 2024-04-25T01:39:15.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.807+0000 7fd1c81a9640 1 -- 172.21.15.112:0/1746344429 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b81a0990 con 0x7fd1b80e2540 2024-04-25T01:39:15.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.807+0000 7fd1c81a9640 1 -- 172.21.15.112:0/1746344429 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b80d5940 con 0x7fd1b80e2540 2024-04-25T01:39:15.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.807+0000 7fd1b67fc640 1 -- 172.21.15.112:0/1746344429 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c0082620 con 0x7fd1b80e2540 2024-04-25T01:39:15.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.807+0000 7fd1b67fc640 1 -- 172.21.15.112:0/1746344429 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c00829d0 con 0x7fd1b80e2540 2024-04-25T01:39:15.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.809+0000 7fd1b67fc640 1 -- 172.21.15.112:0/1746344429 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd1c00b6060 con 0x7fd1b80e2540 2024-04-25T01:39:15.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.810+0000 7fd1c81a9640 1 -- 172.21.15.112:0/1746344429 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1a8005290 con 0x7fd1b80e2540 2024-04-25T01:39:15.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.810+0000 7fd1b67fc640 1 --2- 172.21.15.112:0/1746344429 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd1a406bf30 0x7fd1a406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:15.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.811+0000 7fd1b67fc640 1 -- 172.21.15.112:0/1746344429 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fd1c00b4070 con 0x7fd1b80e2540 2024-04-25T01:39:15.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.811+0000 7fd1c5f45640 1 --2- 172.21.15.112:0/1746344429 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd1a406bf30 0x7fd1a406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:15.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.813+0000 7fd1c5f45640 1 --2- 172.21.15.112:0/1746344429 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd1a406bf30 0x7fd1a406e3f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0006f30 tx=0x7fd1b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:15.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:15.819+0000 7fd1b67fc640 1 -- 172.21.15.112:0/1746344429 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd1c00e3680 con 0x7fd1b80e2540 2024-04-25T01:39:16.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:16.350+0000 7fd1c81a9640 1 -- 172.21.15.112:0/1746344429 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd1a8005580 con 0x7fd1b80e2540 2024-04-25T01:39:16.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:16.361+0000 7fd1b67fc640 1 -- 172.21.15.112:0/1746344429 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7fd1c00e7980 con 0x7fd1b80e2540 2024-04-25T01:39:16.344 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":102,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:39:16.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:16.365+0000 7fd1c81a9640 1 -- 172.21.15.112:0/1746344429 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd1a406bf30 msgr2=0x7fd1a406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:16.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:16.365+0000 7fd1c81a9640 1 --2- 172.21.15.112:0/1746344429 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd1a406bf30 0x7fd1a406e3f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0006f30 tx=0x7fd1b0008040 comp rx=0 tx=0).stop 2024-04-25T01:39:16.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:16.365+0000 7fd1c81a9640 1 -- 172.21.15.112:0/1746344429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1b80e2540 msgr2=0x7fd1b819c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:16.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:16.365+0000 7fd1c81a9640 1 --2- 172.21.15.112:0/1746344429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1b80e2540 0x7fd1b819c780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0075bc0 tx=0x7fd1c0076a20 comp rx=0 tx=0).stop 2024-04-25T01:39:16.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:16.367+0000 7fd1c81a9640 1 -- 172.21.15.112:0/1746344429 shutdown_connections 2024-04-25T01:39:16.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:16.367+0000 7fd1c81a9640 1 --2- 172.21.15.112:0/1746344429 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd1a406bf30 0x7fd1a406e3f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:16.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:16.367+0000 7fd1c81a9640 1 --2- 172.21.15.112:0/1746344429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1b80e2540 0x7fd1b819c780 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:16.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:16.367+0000 7fd1c81a9640 1 --2- 172.21.15.112:0/1746344429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1b80dfd80 0x7fd1b8041730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:16.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:16.367+0000 7fd1c81a9640 1 --2- 172.21.15.112:0/1746344429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b803e790 0x7fd1b81a34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:16.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:16.367+0000 7fd1c81a9640 1 -- 172.21.15.112:0/1746344429 >> 172.21.15.112:0/1746344429 conn(0x7fd1b8048240 msgr2=0x7fd1b802b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:16.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:16.367+0000 7fd1c81a9640 1 -- 172.21.15.112:0/1746344429 shutdown_connections 2024-04-25T01:39:16.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:16.367+0000 7fd1c81a9640 1 -- 172.21.15.112:0/1746344429 wait complete. 2024-04-25T01:39:19.369 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:39:19.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.564+0000 7f51b74da640 1 Processor -- start 2024-04-25T01:39:19.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.564+0000 7f51b74da640 1 -- start start 2024-04-25T01:39:19.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.566+0000 7f51b5276640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f51b0072dc0 0x7f51b015fa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:35532/0 (socket says 172.21.15.112:35532) 2024-04-25T01:39:19.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.566+0000 7f51b5276640 1 -- 172.21.15.112:0/1427920975 learned_addr learned my addr 172.21.15.112:0/1427920975 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:39:19.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.566+0000 7f51b74da640 1 --2- 172.21.15.112:0/1427920975 >> v2:172.21.15.112:3300/0 conn(0x7f51b008b990 0x7f51b00c2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:19.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.566+0000 7f51b74da640 1 --2- 172.21.15.112:0/1427920975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51b00c2ed0 0x7f51b00715e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:19.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.566+0000 7f51b74da640 1 -- 172.21.15.112:0/1427920975 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51b01632f0 con 0x7f51b0072dc0 2024-04-25T01:39:19.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.566+0000 7f51b74da640 1 -- 172.21.15.112:0/1427920975 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f51b0160170 con 0x7f51b008b990 2024-04-25T01:39:19.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.566+0000 7f51b74da640 1 -- 172.21.15.112:0/1427920975 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f51b01630a0 con 0x7f51b00c2ed0 2024-04-25T01:39:19.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.567+0000 7f51b4a75640 1 --2- 172.21.15.112:0/1427920975 >> v2:172.21.15.112:3300/0 conn(0x7f51b008b990 0x7f51b00c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:19.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.567+0000 7f51b4a75640 1 -- 172.21.15.112:0/1427920975 >> v2:172.21.15.112:3300/0 conn(0x7f51b008b990 msgr2=0x7f51b00c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:39:19.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.567+0000 7f51b4a75640 1 -- 172.21.15.112:0/1427920975 >> v2:172.21.15.112:3300/0 conn(0x7f51b008b990 msgr2=0x7f51b00c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:19.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.567+0000 7f51b4a75640 1 --2- 172.21.15.112:0/1427920975 >> v2:172.21.15.112:3300/0 conn(0x7f51b008b990 0x7f51b00c2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:19.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.567+0000 7f51b4a75640 1 --2- 172.21.15.112:0/1427920975 >> v2:172.21.15.112:3300/0 conn(0x7f51b008b990 0x7f51b00c2990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:39:19.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.569+0000 7f51b5a77640 1 --2- 172.21.15.112:0/1427920975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51b00c2ed0 0x7f51b00715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:19.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.569+0000 7f51b5a77640 1 -- 172.21.15.112:0/1427920975 >> v2:172.21.15.112:3300/0 conn(0x7f51b008b990 msgr2=0x7f51b00c2990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:39:19.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.569+0000 7f51b5a77640 1 --2- 172.21.15.112:0/1427920975 >> v2:172.21.15.112:3300/0 conn(0x7f51b008b990 0x7f51b00c2990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.569+0000 7f51b5a77640 1 -- 172.21.15.112:0/1427920975 >> v1:172.21.15.112:6789/0 conn(0x7f51b0072dc0 legacy=0x7f51b015fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:19.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.569+0000 7f51b5a77640 1 -- 172.21.15.112:0/1427920975 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b0070de0 con 0x7f51b00c2ed0 2024-04-25T01:39:19.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.571+0000 7f51b5a77640 1 --2- 172.21.15.112:0/1427920975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51b00c2ed0 0x7f51b00715e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009dc0 tx=0x7f51a802a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56fb8d6b5e164365 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:19.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.571+0000 7f51a7fff640 1 -- 172.21.15.112:0/1427920975 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8007740 con 0x7f51b00c2ed0 2024-04-25T01:39:19.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.571+0000 7f51a7fff640 1 -- 172.21.15.112:0/1427920975 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a80078e0 con 0x7f51b00c2ed0 2024-04-25T01:39:19.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.571+0000 7f51a7fff640 1 -- 172.21.15.112:0/1427920975 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8007cb0 con 0x7f51b00c2ed0 2024-04-25T01:39:19.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.572+0000 7f51b74da640 1 -- 172.21.15.112:0/1427920975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51b00c2ed0 msgr2=0x7f51b00715e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:19.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.572+0000 7f51b74da640 1 --2- 172.21.15.112:0/1427920975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51b00c2ed0 0x7f51b00715e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009dc0 tx=0x7f51a802a5f0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.572+0000 7f51b74da640 1 -- 172.21.15.112:0/1427920975 shutdown_connections 2024-04-25T01:39:19.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.572+0000 7f51b74da640 1 --2- 172.21.15.112:0/1427920975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51b00c2ed0 0x7f51b00715e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.572+0000 7f51b74da640 1 --2- 172.21.15.112:0/1427920975 >> v2:172.21.15.112:3300/0 conn(0x7f51b008b990 0x7f51b00c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.572+0000 7f51b74da640 1 -- 172.21.15.112:0/1427920975 >> 172.21.15.112:0/1427920975 conn(0x7f51b00c3a80 msgr2=0x7f51b00c3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:19.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.572+0000 7f51b74da640 1 -- 172.21.15.112:0/1427920975 shutdown_connections 2024-04-25T01:39:19.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.572+0000 7f51b74da640 1 -- 172.21.15.112:0/1427920975 wait complete. 2024-04-25T01:39:19.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.574+0000 7f51b74da640 1 Processor -- start 2024-04-25T01:39:19.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.574+0000 7f51b74da640 1 -- start start 2024-04-25T01:39:19.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.575+0000 7f51b74da640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51b0072dc0 0x7f51b01ef660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:19.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.575+0000 7f51b74da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51b008b990 0x7f51b00a9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:19.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.576+0000 7f51b4a75640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51b008b990 0x7f51b00a9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:19.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.576+0000 7f51b4a75640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51b008b990 0x7f51b00a9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48384/0 (socket says 172.21.15.112:48384) 2024-04-25T01:39:19.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.576+0000 7f51b4a75640 1 -- 172.21.15.112:0/3400482561 learned_addr learned my addr 172.21.15.112:0/3400482561 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:19.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.576+0000 7f51b74da640 1 --2- 172.21.15.112:0/3400482561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51b00c2ed0 0x7f51b00a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:19.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.576+0000 7f51b74da640 1 -- 172.21.15.112:0/3400482561 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f51b0071150 con 0x7f51b008b990 2024-04-25T01:39:19.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.576+0000 7f51b74da640 1 -- 172.21.15.112:0/3400482561 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f51b01635d0 con 0x7f51b0072dc0 2024-04-25T01:39:19.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.576+0000 7f51b74da640 1 -- 172.21.15.112:0/3400482561 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f51b0071b20 con 0x7f51b00c2ed0 2024-04-25T01:39:19.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.576+0000 7f51b4a75640 1 -- 172.21.15.112:0/3400482561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51b008b990 msgr2=0x7f51b00a9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T01:39:19.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.576+0000 7f51b4a75640 1 -- 172.21.15.112:0/3400482561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51b008b990 msgr2=0x7f51b00a9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:39:19.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.577+0000 7f51b4a75640 1 --2- 172.21.15.112:0/3400482561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51b008b990 0x7f51b00a9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:19.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.577+0000 7f51b4a75640 1 --2- 172.21.15.112:0/3400482561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51b008b990 0x7f51b00a9770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:39:19.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.579+0000 7f51b5276640 1 --2- 172.21.15.112:0/3400482561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51b0072dc0 0x7f51b01ef660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:19.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.580+0000 7f51b5a77640 1 --2- 172.21.15.112:0/3400482561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51b00c2ed0 0x7f51b00a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:19.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.582+0000 7f51b5276640 1 -- 172.21.15.112:0/3400482561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51b00c2ed0 msgr2=0x7f51b00a9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:19.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.582+0000 7f51b5276640 1 --2- 172.21.15.112:0/3400482561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51b00c2ed0 0x7f51b00a9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.582+0000 7f51b5276640 1 -- 172.21.15.112:0/3400482561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51b008b990 msgr2=0x7f51b00a9770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:39:19.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.582+0000 7f51b5276640 1 --2- 172.21.15.112:0/3400482561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51b008b990 0x7f51b00a9770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.582+0000 7f51b5276640 1 -- 172.21.15.112:0/3400482561 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b01f98b0 con 0x7f51b0072dc0 2024-04-25T01:39:19.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.582+0000 7f51b5276640 1 --2- 172.21.15.112:0/3400482561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51b0072dc0 0x7f51b01ef660 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00e730 tx=0x7f51ac00ebf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:19.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.583+0000 7f51a5ffb640 1 -- 172.21.15.112:0/3400482561 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac019070 con 0x7f51b0072dc0 2024-04-25T01:39:19.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.583+0000 7f51b74da640 1 -- 172.21.15.112:0/3400482561 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b00abc10 con 0x7f51b0072dc0 2024-04-25T01:39:19.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.583+0000 7f51b74da640 1 -- 172.21.15.112:0/3400482561 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b01efba0 con 0x7f51b0072dc0 2024-04-25T01:39:19.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.583+0000 7f51a5ffb640 1 -- 172.21.15.112:0/3400482561 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51ac007cc0 con 0x7f51b0072dc0 2024-04-25T01:39:19.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.583+0000 7f51a5ffb640 1 -- 172.21.15.112:0/3400482561 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac00c840 con 0x7f51b0072dc0 2024-04-25T01:39:19.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.586+0000 7f51b74da640 1 -- 172.21.15.112:0/3400482561 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5190005290 con 0x7f51b0072dc0 2024-04-25T01:39:19.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.586+0000 7f51a5ffb640 1 -- 172.21.15.112:0/3400482561 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f51ac01c070 con 0x7f51b0072dc0 2024-04-25T01:39:19.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.587+0000 7f51a5ffb640 1 --2- 172.21.15.112:0/3400482561 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f519406be30 0x7f519406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:19.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.587+0000 7f51a5ffb640 1 -- 172.21.15.112:0/3400482561 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f51ac08c590 con 0x7f51b0072dc0 2024-04-25T01:39:19.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.587+0000 7f51b4a75640 1 --2- 172.21.15.112:0/3400482561 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f519406be30 0x7f519406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:19.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.588+0000 7f51b4a75640 1 --2- 172.21.15.112:0/3400482561 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f519406be30 0x7f519406e2f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f51b015f170 tx=0x7f51a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:19.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:19.598+0000 7f51a5ffb640 1 -- 172.21.15.112:0/3400482561 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f51ac05a580 con 0x7f51b0072dc0 2024-04-25T01:39:20.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:20.143+0000 7f51b74da640 1 -- 172.21.15.112:0/3400482561 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5190005580 con 0x7f51b0072dc0 2024-04-25T01:39:20.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:20.144+0000 7f51a5ffb640 1 -- 172.21.15.112:0/3400482561 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f51ac05e840 con 0x7f51b0072dc0 2024-04-25T01:39:20.130 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":102,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":29,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:39:20.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:20.152+0000 7f51b74da640 1 -- 172.21.15.112:0/3400482561 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f519406be30 msgr2=0x7f519406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:20.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:20.152+0000 7f51b74da640 1 --2- 172.21.15.112:0/3400482561 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f519406be30 0x7f519406e2f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f51b015f170 tx=0x7f51a0008040 comp rx=0 tx=0).stop 2024-04-25T01:39:20.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:20.152+0000 7f51b74da640 1 -- 172.21.15.112:0/3400482561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51b0072dc0 msgr2=0x7f51b01ef660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:20.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:20.152+0000 7f51b74da640 1 --2- 172.21.15.112:0/3400482561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51b0072dc0 0x7f51b01ef660 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00e730 tx=0x7f51ac00ebf0 comp rx=0 tx=0).stop 2024-04-25T01:39:20.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:20.153+0000 7f51b74da640 1 -- 172.21.15.112:0/3400482561 shutdown_connections 2024-04-25T01:39:20.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:20.153+0000 7f51b74da640 1 --2- 172.21.15.112:0/3400482561 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f519406be30 0x7f519406e2f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:20.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:20.153+0000 7f51b74da640 1 --2- 172.21.15.112:0/3400482561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51b00c2ed0 0x7f51b00a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:20.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:20.153+0000 7f51b74da640 1 --2- 172.21.15.112:0/3400482561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51b008b990 0x7f51b00a9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:20.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:20.153+0000 7f51b74da640 1 --2- 172.21.15.112:0/3400482561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51b0072dc0 0x7f51b01ef660 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:20.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:20.153+0000 7f51b74da640 1 -- 172.21.15.112:0/3400482561 >> 172.21.15.112:0/3400482561 conn(0x7f51b00c3a80 msgr2=0x7f51b00c5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:20.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:20.153+0000 7f51b74da640 1 -- 172.21.15.112:0/3400482561 shutdown_connections 2024-04-25T01:39:20.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:20.153+0000 7f51b74da640 1 -- 172.21.15.112:0/3400482561 wait complete. 2024-04-25T01:39:23.162 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:39:23.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.358+0000 7f3a2f8bd640 1 Processor -- start 2024-04-25T01:39:23.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.358+0000 7f3a2f8bd640 1 -- start start 2024-04-25T01:39:23.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.359+0000 7f3a2f8bd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3a2814ef00 0x7f3a280abd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:23.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.359+0000 7f3a2f8bd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3a280aa500 0x7f3a280c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:23.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.359+0000 7f3a2f8bd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a280c2650 0x7f3a2806f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:23.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.359+0000 7f3a2f8bd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f3a280ac9c0 con 0x7f3a280aa500 2024-04-25T01:39:23.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.359+0000 7f3a2f8bd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3a28152170 con 0x7f3a2814ef00 2024-04-25T01:39:23.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.359+0000 7f3a2f8bd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3a28151b60 con 0x7f3a280c2650 2024-04-25T01:39:23.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.360+0000 7f3a2de5a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a280c2650 0x7f3a2806f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:23.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.360+0000 7f3a2de5a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a280c2650 0x7f3a2806f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:51188/0 (socket says 172.21.15.112:51188) 2024-04-25T01:39:23.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.360+0000 7f3a2de5a640 1 -- 172.21.15.112:0/349765194 learned_addr learned my addr 172.21.15.112:0/349765194 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:23.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.360+0000 7f3a2de5a640 1 -- 172.21.15.112:0/349765194 >> v2:172.21.15.187:3300/0 conn(0x7f3a2814ef00 msgr2=0x7f3a280abd70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:39:23.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.360+0000 7f3a2de5a640 1 --2- 172.21.15.112:0/349765194 >> v2:172.21.15.187:3300/0 conn(0x7f3a2814ef00 0x7f3a280abd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:23.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.360+0000 7f3a2de5a640 1 -- 172.21.15.112:0/349765194 >> v2:172.21.15.112:3300/0 conn(0x7f3a280aa500 msgr2=0x7f3a280c2110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:39:23.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.360+0000 7f3a2de5a640 1 --2- 172.21.15.112:0/349765194 >> v2:172.21.15.112:3300/0 conn(0x7f3a280aa500 0x7f3a280c2110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:23.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.361+0000 7f3a2de5a640 1 -- 172.21.15.112:0/349765194 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a280ac6e0 con 0x7f3a280c2650 2024-04-25T01:39:23.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.362+0000 7f3a2de5a640 1 --2- 172.21.15.112:0/349765194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a280c2650 0x7f3a2806f360 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3a2806ecc0 tx=0x7f3a1802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=924882136387f5bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:23.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.366+0000 7f3a1ffff640 1 -- 172.21.15.112:0/349765194 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a18037070 con 0x7f3a280c2650 2024-04-25T01:39:23.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.366+0000 7f3a1ffff640 1 -- 172.21.15.112:0/349765194 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a18032440 con 0x7f3a280c2650 2024-04-25T01:39:23.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.366+0000 7f3a1ffff640 1 -- 172.21.15.112:0/349765194 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a18032810 con 0x7f3a280c2650 2024-04-25T01:39:23.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.367+0000 7f3a2f8bd640 1 -- 172.21.15.112:0/349765194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a280c2650 msgr2=0x7f3a2806f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:23.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.367+0000 7f3a2f8bd640 1 --2- 172.21.15.112:0/349765194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a280c2650 0x7f3a2806f360 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3a2806ecc0 tx=0x7f3a1802a210 comp rx=0 tx=0).stop 2024-04-25T01:39:23.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.367+0000 7f3a2f8bd640 1 -- 172.21.15.112:0/349765194 shutdown_connections 2024-04-25T01:39:23.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.367+0000 7f3a2f8bd640 1 --2- 172.21.15.112:0/349765194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a280c2650 0x7f3a2806f360 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:23.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.367+0000 7f3a2f8bd640 1 --2- 172.21.15.112:0/349765194 >> v2:172.21.15.112:3300/0 conn(0x7f3a280aa500 0x7f3a280c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:23.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.367+0000 7f3a2f8bd640 1 --2- 172.21.15.112:0/349765194 >> v2:172.21.15.187:3300/0 conn(0x7f3a2814ef00 0x7f3a280abd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:23.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.367+0000 7f3a2f8bd640 1 -- 172.21.15.112:0/349765194 >> 172.21.15.112:0/349765194 conn(0x7f3a280a9670 msgr2=0x7f3a280a9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:23.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.372+0000 7f3a2f8bd640 1 -- 172.21.15.112:0/349765194 shutdown_connections 2024-04-25T01:39:23.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.373+0000 7f3a2f8bd640 1 -- 172.21.15.112:0/349765194 wait complete. 2024-04-25T01:39:23.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.375+0000 7f3a2f8bd640 1 Processor -- start 2024-04-25T01:39:23.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.375+0000 7f3a2f8bd640 1 -- start start 2024-04-25T01:39:23.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.376+0000 7f3a2f8bd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a280aa500 0x7f3a280c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:23.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.376+0000 7f3a2f8bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a280c2650 0x7f3a281495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.376+0000 7f3a2ce58640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a280c2650 0x7f3a281495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.377+0000 7f3a2ce58640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a280c2650 0x7f3a281495b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48396/0 (socket says 172.21.15.112:48396) 2024-04-25T01:39:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.377+0000 7f3a2ce58640 1 -- 172.21.15.112:0/1751790568 learned_addr learned my addr 172.21.15.112:0/1751790568 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.377+0000 7f3a2f8bd640 1 --2- 172.21.15.112:0/1751790568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a2814ef00 0x7f3a280c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.377+0000 7f3a2f8bd640 1 -- 172.21.15.112:0/1751790568 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3a280ac2b0 con 0x7f3a280c2650 2024-04-25T01:39:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.377+0000 7f3a2f8bd640 1 -- 172.21.15.112:0/1751790568 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2814f2e0 con 0x7f3a280aa500 2024-04-25T01:39:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.377+0000 7f3a2d659640 1 --2- 172.21.15.112:0/1751790568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a280aa500 0x7f3a280c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.377+0000 7f3a2de5a640 1 --2- 172.21.15.112:0/1751790568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a2814ef00 0x7f3a280c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.377+0000 7f3a2f8bd640 1 -- 172.21.15.112:0/1751790568 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3a280ac430 con 0x7f3a2814ef00 2024-04-25T01:39:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.377+0000 7f3a2ce58640 1 -- 172.21.15.112:0/1751790568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a280c2650 msgr2=0x7f3a281495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T01:39:23.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.377+0000 7f3a2ce58640 1 -- 172.21.15.112:0/1751790568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a280c2650 msgr2=0x7f3a281495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:39:23.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.377+0000 7f3a2ce58640 1 --2- 172.21.15.112:0/1751790568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a280c2650 0x7f3a281495b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:23.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.378+0000 7f3a2ce58640 1 --2- 172.21.15.112:0/1751790568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a280c2650 0x7f3a281495b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:39:23.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.379+0000 7f3a2de5a640 1 -- 172.21.15.112:0/1751790568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a280aa500 msgr2=0x7f3a280c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:23.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.379+0000 7f3a2de5a640 1 --2- 172.21.15.112:0/1751790568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a280aa500 0x7f3a280c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:23.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.379+0000 7f3a2de5a640 1 -- 172.21.15.112:0/1751790568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a280c2650 msgr2=0x7f3a281495b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:39:23.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.379+0000 7f3a2de5a640 1 --2- 172.21.15.112:0/1751790568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a280c2650 0x7f3a281495b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:23.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.379+0000 7f3a2de5a640 1 -- 172.21.15.112:0/1751790568 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a281eaed0 con 0x7f3a2814ef00 2024-04-25T01:39:23.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.379+0000 7f3a2de5a640 1 --2- 172.21.15.112:0/1751790568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a2814ef00 0x7f3a280c5e40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3a18031000 tx=0x7f3a180046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:23.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.381+0000 7f3a1dffb640 1 -- 172.21.15.112:0/1751790568 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a18045020 con 0x7f3a2814ef00 2024-04-25T01:39:23.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.381+0000 7f3a1dffb640 1 -- 172.21.15.112:0/1751790568 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a1803b2f0 con 0x7f3a2814ef00 2024-04-25T01:39:23.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.381+0000 7f3a2f8bd640 1 -- 172.21.15.112:0/1751790568 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a281eb1b0 con 0x7f3a2814ef00 2024-04-25T01:39:23.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.381+0000 7f3a2f8bd640 1 -- 172.21.15.112:0/1751790568 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a281eb770 con 0x7f3a2814ef00 2024-04-25T01:39:23.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.382+0000 7f3a1dffb640 1 -- 172.21.15.112:0/1751790568 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a18037040 con 0x7f3a2814ef00 2024-04-25T01:39:23.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.387+0000 7f3a2f8bd640 1 -- 172.21.15.112:0/1751790568 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a08005290 con 0x7f3a2814ef00 2024-04-25T01:39:23.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.389+0000 7f3a1dffb640 1 -- 172.21.15.112:0/1751790568 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a18048020 con 0x7f3a2814ef00 2024-04-25T01:39:23.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.389+0000 7f3a1dffb640 1 --2- 172.21.15.112:0/1751790568 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a0c06be60 0x7f3a0c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:23.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.389+0000 7f3a1dffb640 1 -- 172.21.15.112:0/1751790568 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f3a180af540 con 0x7f3a2814ef00 2024-04-25T01:39:23.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.391+0000 7f3a2d659640 1 --2- 172.21.15.112:0/1751790568 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a0c06be60 0x7f3a0c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:23.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.392+0000 7f3a2d659640 1 --2- 172.21.15.112:0/1751790568 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a0c06be60 0x7f3a0c06e320 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3a2000b4a0 tx=0x7f3a2000d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:23.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.396+0000 7f3a1dffb640 1 -- 172.21.15.112:0/1751790568 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3a1804f030 con 0x7f3a2814ef00 2024-04-25T01:39:23.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.927+0000 7f3a2f8bd640 1 -- 172.21.15.112:0/1751790568 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3a08005580 con 0x7f3a2814ef00 2024-04-25T01:39:23.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.931+0000 7f3a1dffb640 1 -- 172.21.15.112:0/1751790568 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f3a1807dac0 con 0x7f3a2814ef00 2024-04-25T01:39:23.914 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":102,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":33,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:39:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.935+0000 7f3a2f8bd640 1 -- 172.21.15.112:0/1751790568 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a0c06be60 msgr2=0x7f3a0c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.935+0000 7f3a2f8bd640 1 --2- 172.21.15.112:0/1751790568 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a0c06be60 0x7f3a0c06e320 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3a2000b4a0 tx=0x7f3a2000d040 comp rx=0 tx=0).stop 2024-04-25T01:39:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.935+0000 7f3a2f8bd640 1 -- 172.21.15.112:0/1751790568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a2814ef00 msgr2=0x7f3a280c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.935+0000 7f3a2f8bd640 1 --2- 172.21.15.112:0/1751790568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a2814ef00 0x7f3a280c5e40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3a18031000 tx=0x7f3a180046c0 comp rx=0 tx=0).stop 2024-04-25T01:39:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.936+0000 7f3a2f8bd640 1 -- 172.21.15.112:0/1751790568 shutdown_connections 2024-04-25T01:39:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.936+0000 7f3a2f8bd640 1 --2- 172.21.15.112:0/1751790568 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a0c06be60 0x7f3a0c06e320 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.936+0000 7f3a2f8bd640 1 --2- 172.21.15.112:0/1751790568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a2814ef00 0x7f3a280c5e40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.936+0000 7f3a2f8bd640 1 --2- 172.21.15.112:0/1751790568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a280c2650 0x7f3a281495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.936+0000 7f3a2f8bd640 1 --2- 172.21.15.112:0/1751790568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a280aa500 0x7f3a280c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.936+0000 7f3a2f8bd640 1 -- 172.21.15.112:0/1751790568 >> 172.21.15.112:0/1751790568 conn(0x7f3a280a9670 msgr2=0x7f3a2814d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.936+0000 7f3a2f8bd640 1 -- 172.21.15.112:0/1751790568 shutdown_connections 2024-04-25T01:39:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:23.936+0000 7f3a2f8bd640 1 -- 172.21.15.112:0/1751790568 wait complete. 2024-04-25T01:39:26.938 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:39:27.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.124+0000 7fa788b3b640 1 Processor -- start 2024-04-25T01:39:27.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.124+0000 7fa788b3b640 1 -- start start 2024-04-25T01:39:27.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.125+0000 7fa788b3b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 0x7fa784071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:27.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.125+0000 7fa788b3b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 0x7fa78406b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:27.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.125+0000 7fa788b3b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 0x7fa784159e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:27.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.125+0000 7fa788b3b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa7840c3790 con 0x7fa78406bd90 2024-04-25T01:39:27.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.125+0000 7fa788b3b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa7840c3a70 con 0x7fa78406b6a0 2024-04-25T01:39:27.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.125+0000 7fa788b3b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa7840c3d50 con 0x7fa784157330 2024-04-25T01:39:27.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.125+0000 7fa78259c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 0x7fa784071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:27.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.126+0000 7fa781d9b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 0x7fa78406b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:27.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.126+0000 7fa78259c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 0x7fa784071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:51220/0 (socket says 172.21.15.112:51220) 2024-04-25T01:39:27.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.126+0000 7fa78259c640 1 -- 172.21.15.112:0/2438308081 learned_addr learned my addr 172.21.15.112:0/2438308081 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:27.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.126+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 0x7fa784159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:27.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.126+0000 7fa781d9b640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 msgr2=0x7fa78406b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:39:27.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.126+0000 7fa781d9b640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 msgr2=0x7fa78406b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:27.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.126+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 0x7fa78406b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:27.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.126+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 0x7fa78406b160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:39:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.127+0000 7fa78259c640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 msgr2=0x7fa784071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:39:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.127+0000 7fa78259c640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 msgr2=0x7fa784071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.127+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 0x7fa784071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.127+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 0x7fa784071e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:39:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.128+0000 7fa782d9d640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 msgr2=0x7fa784159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:39:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.128+0000 7fa782d9d640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 msgr2=0x7fa784159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.128+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 0x7fa784159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.128+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 0x7fa784159e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:39:27.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.327+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 0x7fa78406b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:27.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.327+0000 7fa781d9b640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 msgr2=0x7fa78406b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:39:27.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.327+0000 7fa781d9b640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 msgr2=0x7fa78406b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:27.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.327+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 0x7fa78406b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:27.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.327+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 0x7fa78406b160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:39:27.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.328+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 0x7fa784071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:27.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.330+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 0x7fa784159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:27.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.330+0000 7fa78259c640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 msgr2=0x7fa784071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:39:27.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.330+0000 7fa78259c640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 msgr2=0x7fa784071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:27.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.330+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 0x7fa784071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:27.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.330+0000 7fa782d9d640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 msgr2=0x7fa784159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:39:27.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.330+0000 7fa782d9d640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 msgr2=0x7fa784159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:27.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.330+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 0x7fa784159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:27.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.330+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 0x7fa784159e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:39:27.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.330+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 0x7fa784071e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:39:27.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.728+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 0x7fa78406b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:27.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.728+0000 7fa781d9b640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 msgr2=0x7fa78406b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:39:27.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.728+0000 7fa781d9b640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 msgr2=0x7fa78406b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:27.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.728+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 0x7fa78406b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:27.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.728+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 0x7fa78406b160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:39:27.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.731+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 0x7fa784159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:27.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.731+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 0x7fa784071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:27.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.731+0000 7fa782d9d640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 msgr2=0x7fa784159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:39:27.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.731+0000 7fa782d9d640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 msgr2=0x7fa784159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:27.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.731+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 0x7fa784159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:27.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.731+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 0x7fa784159e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:39:27.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.734+0000 7fa78259c640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 msgr2=0x7fa784071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:39:27.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.734+0000 7fa78259c640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 msgr2=0x7fa784071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:27.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.734+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 0x7fa784071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:27.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:27.734+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 0x7fa784071e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:39:28.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:28.529+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 0x7fa78406b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:28.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:28.529+0000 7fa781d9b640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 msgr2=0x7fa78406b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:39:28.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:28.529+0000 7fa781d9b640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 msgr2=0x7fa78406b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:28.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:28.529+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 0x7fa78406b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:28.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:28.529+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 0x7fa78406b160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:39:28.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:28.532+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 0x7fa784159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:28.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:28.532+0000 7fa782d9d640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 msgr2=0x7fa784159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:39:28.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:28.532+0000 7fa782d9d640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 msgr2=0x7fa784159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:28.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:28.532+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 0x7fa784159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:28.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:28.532+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 0x7fa784159e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:39:28.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:28.537+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 0x7fa784071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:28.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:28.537+0000 7fa78259c640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 msgr2=0x7fa784071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:39:28.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:28.537+0000 7fa78259c640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 msgr2=0x7fa784071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:28.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:28.537+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 0x7fa784071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:28.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:28.537+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 0x7fa784071e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:39:28.987 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:39:28.991 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:39:30.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:30.125+0000 7fa773fff640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 msgr2=0x7fa784071e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:39:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:30.125+0000 7fa773fff640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa784157330 0x7fa784071e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:30.125+0000 7fa773fff640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 msgr2=0x7fa784159e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:39:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:30.125+0000 7fa773fff640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa78406b6a0 0x7fa784159e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:30.125+0000 7fa773fff640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 msgr2=0x7fa78406b160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:39:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:30.125+0000 7fa773fff640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa78406bd90 0x7fa78406b160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:30.125+0000 7fa773fff640 1 -- 172.21.15.112:0/2438308081 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7840c4030 con 0x7fa76c007320 2024-04-25T01:39:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:30.126+0000 7fa773fff640 1 -- 172.21.15.112:0/2438308081 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa76c00c0a0 con 0x7fa76c003670 2024-04-25T01:39:33.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.126+0000 7fa773fff640 1 -- 172.21.15.112:0/2438308081 >> v1:172.21.15.187:6789/0 conn(0x7fa76c003670 legacy=0x7fa76c005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:33.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.126+0000 7fa773fff640 1 -- 172.21.15.112:0/2438308081 >> v1:172.21.15.112:6789/0 conn(0x7fa76c007320 legacy=0x7fa76c0097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:33.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.126+0000 7fa782d9d640 1 -- 172.21.15.112:0/2438308081 reap_dead start 2024-04-25T01:39:33.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.126+0000 7fa773fff640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 0x7fa76c005af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:33.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.126+0000 7fa773fff640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 0x7fa76c00cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:33.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.126+0000 7fa773fff640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 0x7fa76c00b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:33.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.126+0000 7fa773fff640 1 -- 172.21.15.112:0/2438308081 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa7840c3790 con 0x7fa76c00c3a0 2024-04-25T01:39:33.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.126+0000 7fa773fff640 1 -- 172.21.15.112:0/2438308081 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa7840c3a70 con 0x7fa76c00c780 2024-04-25T01:39:33.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.126+0000 7fa773fff640 1 -- 172.21.15.112:0/2438308081 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa7840c3d50 con 0x7fa76c00b050 2024-04-25T01:39:33.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.127+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 0x7fa76c00b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:33.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.127+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 0x7fa76c005af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:33.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.127+0000 7fa78259c640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 msgr2=0x7fa76c005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:39:33.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.127+0000 7fa782d9d640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 msgr2=0x7fa76c00b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:39:33.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.127+0000 7fa782d9d640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 msgr2=0x7fa76c00b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:33.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.127+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 0x7fa76c00b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:33.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.127+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 0x7fa76c00b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:39:33.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.127+0000 7fa78259c640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 msgr2=0x7fa76c005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:33.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.127+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 0x7fa76c005af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:33.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.127+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 0x7fa76c005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:39:33.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.127+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 0x7fa76c00cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:33.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.128+0000 7fa781d9b640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 msgr2=0x7fa76c00cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:39:33.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.128+0000 7fa781d9b640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 msgr2=0x7fa76c00cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:33.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.128+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 0x7fa76c00cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:33.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.128+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 0x7fa76c00cc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:39:33.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.328+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 0x7fa76c00b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:33.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.328+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 0x7fa76c005af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:33.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.328+0000 7fa782d9d640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 msgr2=0x7fa76c00b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:39:33.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.328+0000 7fa782d9d640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 msgr2=0x7fa76c00b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:33.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.328+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 0x7fa76c00b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:33.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.328+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 0x7fa76c00b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:39:33.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.328+0000 7fa78259c640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 msgr2=0x7fa76c005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:39:33.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.328+0000 7fa78259c640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 msgr2=0x7fa76c005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:33.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.328+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 0x7fa76c005af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:33.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.328+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 0x7fa76c005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:39:33.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.329+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 0x7fa76c00cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:33.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.329+0000 7fa781d9b640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 msgr2=0x7fa76c00cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:39:33.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.329+0000 7fa781d9b640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 msgr2=0x7fa76c00cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:33.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.329+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 0x7fa76c00cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:33.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.329+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 0x7fa76c00cc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:39:33.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.729+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 0x7fa76c005af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:33.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.729+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 0x7fa76c00b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:33.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.729+0000 7fa782d9d640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 msgr2=0x7fa76c00b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:39:33.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.729+0000 7fa782d9d640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 msgr2=0x7fa76c00b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:33.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.729+0000 7fa78259c640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 msgr2=0x7fa76c005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:39:33.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.729+0000 7fa78259c640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 msgr2=0x7fa76c005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:33.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.729+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 0x7fa76c00b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:33.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.729+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 0x7fa76c005af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:33.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.729+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 0x7fa76c00b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:39:33.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.729+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 0x7fa76c005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:39:33.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.730+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 0x7fa76c00cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:33.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.730+0000 7fa781d9b640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 msgr2=0x7fa76c00cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:39:33.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.730+0000 7fa781d9b640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 msgr2=0x7fa76c00cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:33.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.730+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 0x7fa76c00cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:33.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:33.730+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 0x7fa76c00cc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:39:34.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:34.530+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 0x7fa76c005af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:34.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:34.530+0000 7fa78259c640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 msgr2=0x7fa76c005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:39:34.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:34.530+0000 7fa78259c640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 msgr2=0x7fa76c005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:34.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:34.530+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 0x7fa76c005af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:34.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:34.530+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 0x7fa76c005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:39:34.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:34.531+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 0x7fa76c00b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:34.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:34.531+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 0x7fa76c00cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:34.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:34.531+0000 7fa782d9d640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 msgr2=0x7fa76c00b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:39:34.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:34.531+0000 7fa782d9d640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 msgr2=0x7fa76c00b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:34.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:34.531+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 0x7fa76c00b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:34.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:34.531+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 0x7fa76c00b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:39:34.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:34.531+0000 7fa781d9b640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 msgr2=0x7fa76c00cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:39:34.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:34.531+0000 7fa781d9b640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 msgr2=0x7fa76c00cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:39:34.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:34.531+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 0x7fa76c00cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:34.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:34.531+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 0x7fa76c00cc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:39:36.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.126+0000 7fa773fff640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 msgr2=0x7fa76c00b4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:39:36.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.126+0000 7fa773fff640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 0x7fa76c00b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:36.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.126+0000 7fa773fff640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 msgr2=0x7fa76c00cc30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:39:36.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.127+0000 7fa773fff640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.187:3300/0 conn(0x7fa76c00c780 0x7fa76c00cc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:36.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.127+0000 7fa773fff640 1 -- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 msgr2=0x7fa76c005af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:39:36.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.127+0000 7fa773fff640 1 --2- 172.21.15.112:0/2438308081 >> v2:172.21.15.112:3300/0 conn(0x7fa76c00c3a0 0x7fa76c005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:36.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.127+0000 7fa773fff640 1 -- 172.21.15.112:0/2438308081 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa76c00d340 con 0x7fa76c009fb0 2024-04-25T01:39:36.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.127+0000 7fa773fff640 1 -- 172.21.15.112:0/2438308081 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa76c0156c0 con 0x7fa76c00a3c0 2024-04-25T01:39:36.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.128+0000 7fa78159a640 1 -- 172.21.15.112:0/2438308081 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4144884937 0 0) 0x7fa76c0156c0 con 0x7fa76c00a3c0 2024-04-25T01:39:36.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.128+0000 7fa78159a640 1 -- 172.21.15.112:0/2438308081 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa764003520 con 0x7fa76c00a3c0 2024-04-25T01:39:36.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.129+0000 7fa78159a640 1 -- 172.21.15.112:0/2438308081 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1358302208 0 0) 0x7fa764003520 con 0x7fa76c00a3c0 2024-04-25T01:39:36.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.129+0000 7fa78159a640 1 -- 172.21.15.112:0/2438308081 >> v1:172.21.15.112:6789/0 conn(0x7fa76c009fb0 legacy=0x7fa76c009740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:36.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.129+0000 7fa78159a640 1 -- 172.21.15.112:0/2438308081 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76c0168e0 con 0x7fa76c00a3c0 2024-04-25T01:39:36.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.129+0000 7fa78159a640 1 -- 172.21.15.112:0/2438308081 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fa778015580 con 0x7fa76c00a3c0 2024-04-25T01:39:36.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.129+0000 7fa78159a640 1 -- 172.21.15.112:0/2438308081 >> v1:172.21.15.187:6789/0 conn(0x7fa76c00a3c0 legacy=0x7fa76c00a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:36.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.129+0000 7fa78159a640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7640039d0 0x7fa764003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:36.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.129+0000 7fa78159a640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 0x7fa764004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:36.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.129+0000 7fa78159a640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa764004e60 0x7fa76400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:36.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.129+0000 7fa78159a640 1 -- 172.21.15.112:0/2438308081 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa7640055f0 con 0x7fa764004e60 2024-04-25T01:39:36.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.129+0000 7fa78159a640 1 -- 172.21.15.112:0/2438308081 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa76400bf00 con 0x7fa7640039d0 2024-04-25T01:39:36.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.129+0000 7fa78159a640 1 -- 172.21.15.112:0/2438308081 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa76400c080 con 0x7fa76c00b050 2024-04-25T01:39:36.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.129+0000 7fa782d9d640 1 -- 172.21.15.112:0/2438308081 reap_dead start 2024-04-25T01:39:36.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.130+0000 7fa782d9d640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa764004e60 0x7fa76400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:36.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.130+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7640039d0 0x7fa764003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:36.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.130+0000 7fa781d9b640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 0x7fa764004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:36.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.130+0000 7fa78259c640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 msgr2=0x7fa764004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:39:36.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.130+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 0x7fa764004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:36.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.130+0000 7fa78259c640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa764004e60 msgr2=0x7fa76400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:39:36.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.130+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa764004e60 0x7fa76400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:36.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.130+0000 7fa78259c640 1 -- 172.21.15.112:0/2438308081 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76c0168e0 con 0x7fa7640039d0 2024-04-25T01:39:36.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.133+0000 7fa78259c640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7640039d0 0x7fa764003e10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa774009100 tx=0x7fa764005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f8209f626b16d52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:36.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.134+0000 7fa78159a640 1 -- 172.21.15.112:0/2438308081 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77400f040 con 0x7fa7640039d0 2024-04-25T01:39:36.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.134+0000 7fa78159a640 1 -- 172.21.15.112:0/2438308081 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa774004230 con 0x7fa7640039d0 2024-04-25T01:39:36.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.135+0000 7fa78159a640 1 -- 172.21.15.112:0/2438308081 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77400f040 con 0x7fa7640039d0 2024-04-25T01:39:36.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.135+0000 7fa788b3b640 1 -- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7640039d0 msgr2=0x7fa764003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:36.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.135+0000 7fa788b3b640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7640039d0 0x7fa764003e10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa774009100 tx=0x7fa764005830 comp rx=0 tx=0).stop 2024-04-25T01:39:36.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.135+0000 7fa788b3b640 1 -- 172.21.15.112:0/2438308081 shutdown_connections 2024-04-25T01:39:36.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.135+0000 7fa788b3b640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa764004e60 0x7fa76400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:36.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.135+0000 7fa788b3b640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7640039d0 0x7fa764003e10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:36.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.135+0000 7fa788b3b640 1 --2- 172.21.15.112:0/2438308081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa76c00b050 0x7fa764004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:36.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.135+0000 7fa788b3b640 1 -- 172.21.15.112:0/2438308081 >> 172.21.15.112:0/2438308081 conn(0x7fa7840c8130 msgr2=0x7fa7840c8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:36.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.135+0000 7fa788b3b640 1 -- 172.21.15.112:0/2438308081 shutdown_connections 2024-04-25T01:39:36.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.135+0000 7fa788b3b640 1 -- 172.21.15.112:0/2438308081 wait complete. 2024-04-25T01:39:36.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.141+0000 7fa788b3b640 1 Processor -- start 2024-04-25T01:39:36.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.141+0000 7fa788b3b640 1 -- start start 2024-04-25T01:39:36.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.142+0000 7fa788b3b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7640039d0 0x7fa7841eff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:36.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.142+0000 7fa788b3b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa764004e60 0x7fa7840c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:36.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.142+0000 7fa788b3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76c00b050 0x7fa7840c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:36.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.143+0000 7fa78259c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7640039d0 0x7fa7841eff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:36.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.143+0000 7fa78259c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7640039d0 0x7fa7841eff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:37580/0 (socket says 172.21.15.112:37580) 2024-04-25T01:39:36.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.143+0000 7fa781d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa764004e60 0x7fa7840c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:36.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.143+0000 7fa781d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa764004e60 0x7fa7840c71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:46424/0 (socket says 172.21.15.112:46424) 2024-04-25T01:39:36.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.143+0000 7fa782d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76c00b050 0x7fa7840c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:36.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.143+0000 7fa782d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76c00b050 0x7fa7840c76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39486/0 (socket says 172.21.15.112:39486) 2024-04-25T01:39:36.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.143+0000 7fa78259c640 1 -- 172.21.15.112:0/1906841384 learned_addr learned my addr 172.21.15.112:0/1906841384 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:36.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.143+0000 7fa788b3b640 1 -- 172.21.15.112:0/1906841384 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa78406b2e0 con 0x7fa76c00b050 2024-04-25T01:39:36.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.144+0000 7fa788b3b640 1 -- 172.21.15.112:0/1906841384 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa78406b160 con 0x7fa764004e60 2024-04-25T01:39:36.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.144+0000 7fa788b3b640 1 -- 172.21.15.112:0/1906841384 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa78406b460 con 0x7fa7640039d0 2024-04-25T01:39:36.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.144+0000 7fa782d9d640 1 -- 172.21.15.112:0/1906841384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76c00b050 msgr2=0x7fa7840c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-25T01:39:36.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.144+0000 7fa782d9d640 1 -- 172.21.15.112:0/1906841384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76c00b050 msgr2=0x7fa7840c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:39:36.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.144+0000 7fa782d9d640 1 --2- 172.21.15.112:0/1906841384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76c00b050 0x7fa7840c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:39:36.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.144+0000 7fa782d9d640 1 --2- 172.21.15.112:0/1906841384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76c00b050 0x7fa7840c76f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:39:36.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.145+0000 7fa78259c640 1 -- 172.21.15.112:0/1906841384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa764004e60 msgr2=0x7fa7840c71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:36.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.145+0000 7fa78259c640 1 --2- 172.21.15.112:0/1906841384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa764004e60 0x7fa7840c71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:36.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.145+0000 7fa78259c640 1 -- 172.21.15.112:0/1906841384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76c00b050 msgr2=0x7fa7840c76f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:39:36.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.145+0000 7fa78259c640 1 --2- 172.21.15.112:0/1906841384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76c00b050 0x7fa7840c76f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:36.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.145+0000 7fa78259c640 1 -- 172.21.15.112:0/1906841384 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa78415a6c0 con 0x7fa7640039d0 2024-04-25T01:39:36.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.145+0000 7fa781d9b640 1 --2- 172.21.15.112:0/1906841384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa764004e60 0x7fa7840c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:39:36.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.146+0000 7fa78259c640 1 --2- 172.21.15.112:0/1906841384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7640039d0 0x7fa7841eff90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa774036040 tx=0x7fa7740046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:36.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.146+0000 7fa772ffd640 1 -- 172.21.15.112:0/1906841384 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa774044020 con 0x7fa7640039d0 2024-04-25T01:39:36.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.146+0000 7fa772ffd640 1 -- 172.21.15.112:0/1906841384 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa774040470 con 0x7fa7640039d0 2024-04-25T01:39:36.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.146+0000 7fa772ffd640 1 -- 172.21.15.112:0/1906841384 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77400f040 con 0x7fa7640039d0 2024-04-25T01:39:36.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.147+0000 7fa788b3b640 1 -- 172.21.15.112:0/1906841384 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa78415a9a0 con 0x7fa7640039d0 2024-04-25T01:39:36.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.147+0000 7fa788b3b640 1 -- 172.21.15.112:0/1906841384 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa78415b9b0 con 0x7fa7640039d0 2024-04-25T01:39:36.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.149+0000 7fa788b3b640 1 -- 172.21.15.112:0/1906841384 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7840689b0 con 0x7fa7640039d0 2024-04-25T01:39:36.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.149+0000 7fa772ffd640 1 -- 172.21.15.112:0/1906841384 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa774047020 con 0x7fa7640039d0 2024-04-25T01:39:36.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.150+0000 7fa772ffd640 1 --2- 172.21.15.112:0/1906841384 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa76c06e690 0x7fa76c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:36.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.151+0000 7fa781d9b640 1 --2- 172.21.15.112:0/1906841384 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa76c06e690 0x7fa76c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:36.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.151+0000 7fa772ffd640 1 -- 172.21.15.112:0/1906841384 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fa7740ad4a0 con 0x7fa7640039d0 2024-04-25T01:39:36.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.152+0000 7fa781d9b640 1 --2- 172.21.15.112:0/1906841384 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa76c06e690 0x7fa76c070b50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00bf70 tx=0x7fa77801b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:36.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.159+0000 7fa772ffd640 1 -- 172.21.15.112:0/1906841384 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa77404e030 con 0x7fa7640039d0 2024-04-25T01:39:36.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.686+0000 7fa788b3b640 1 -- 172.21.15.112:0/1906841384 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa7840aa640 con 0x7fa7640039d0 2024-04-25T01:39:36.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.686+0000 7fa772ffd640 1 -- 172.21.15.112:0/1906841384 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1563 (secure 0 0 0) 0x7fa77407ba20 con 0x7fa7640039d0 2024-04-25T01:39:36.670 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":108,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:39:36.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.690+0000 7fa788b3b640 1 -- 172.21.15.112:0/1906841384 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa76c06e690 msgr2=0x7fa76c070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:36.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.690+0000 7fa788b3b640 1 --2- 172.21.15.112:0/1906841384 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa76c06e690 0x7fa76c070b50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00bf70 tx=0x7fa77801b040 comp rx=0 tx=0).stop 2024-04-25T01:39:36.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.690+0000 7fa788b3b640 1 -- 172.21.15.112:0/1906841384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7640039d0 msgr2=0x7fa7841eff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:36.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.690+0000 7fa788b3b640 1 --2- 172.21.15.112:0/1906841384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7640039d0 0x7fa7841eff90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa774036040 tx=0x7fa7740046c0 comp rx=0 tx=0).stop 2024-04-25T01:39:36.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.693+0000 7fa788b3b640 1 -- 172.21.15.112:0/1906841384 shutdown_connections 2024-04-25T01:39:36.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.693+0000 7fa788b3b640 1 --2- 172.21.15.112:0/1906841384 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa76c06e690 0x7fa76c070b50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:36.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.693+0000 7fa788b3b640 1 --2- 172.21.15.112:0/1906841384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76c00b050 0x7fa7840c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:36.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.693+0000 7fa788b3b640 1 --2- 172.21.15.112:0/1906841384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa764004e60 0x7fa7840c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:36.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.693+0000 7fa788b3b640 1 --2- 172.21.15.112:0/1906841384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7640039d0 0x7fa7841eff90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:36.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.693+0000 7fa788b3b640 1 -- 172.21.15.112:0/1906841384 >> 172.21.15.112:0/1906841384 conn(0x7fa7840c8130 msgr2=0x7fa784156590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:36.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.693+0000 7fa788b3b640 1 -- 172.21.15.112:0/1906841384 shutdown_connections 2024-04-25T01:39:36.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:36.693+0000 7fa788b3b640 1 -- 172.21.15.112:0/1906841384 wait complete. 2024-04-25T01:39:39.701 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:39:39.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.903+0000 7fe9778dd640 1 Processor -- start 2024-04-25T01:39:39.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.905+0000 7fe9778dd640 1 -- start start 2024-04-25T01:39:39.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.906+0000 7fe9778dd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe97014d6c0 0x7fe9700aac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:39.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.906+0000 7fe9778dd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9700ab190 0x7fe970153750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:39.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.906+0000 7fe9778dd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe970153c90 0x7fe9700c5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:39.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.906+0000 7fe9778dd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fe9700c4f20 con 0x7fe97014d6c0 2024-04-25T01:39:39.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.906+0000 7fe9778dd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe970154210 con 0x7fe970153c90 2024-04-25T01:39:39.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.906+0000 7fe9778dd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe9701544f0 con 0x7fe9700ab190 2024-04-25T01:39:39.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.906+0000 7fe975e7a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe970153c90 0x7fe9700c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:39.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.906+0000 7fe975e7a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe970153c90 0x7fe9700c5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:42646/0 (socket says 172.21.15.112:42646) 2024-04-25T01:39:39.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.906+0000 7fe975e7a640 1 -- 172.21.15.112:0/782609328 learned_addr learned my addr 172.21.15.112:0/782609328 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:39.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.907+0000 7fe975e7a640 1 -- 172.21.15.112:0/782609328 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9700ab190 msgr2=0x7fe970153750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:39:39.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.907+0000 7fe975e7a640 1 --2- 172.21.15.112:0/782609328 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9700ab190 0x7fe970153750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:39.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.907+0000 7fe975e7a640 1 -- 172.21.15.112:0/782609328 >> v2:172.21.15.112:3300/0 conn(0x7fe97014d6c0 msgr2=0x7fe9700aac50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:39:39.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.907+0000 7fe975e7a640 1 --2- 172.21.15.112:0/782609328 >> v2:172.21.15.112:3300/0 conn(0x7fe97014d6c0 0x7fe9700aac50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:39.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.907+0000 7fe975e7a640 1 -- 172.21.15.112:0/782609328 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9700c51e0 con 0x7fe970153c90 2024-04-25T01:39:39.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.908+0000 7fe975e7a640 1 --2- 172.21.15.112:0/782609328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe970153c90 0x7fe9700c5700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe9700c4a60 tx=0x7fe96802a6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca54a204cb6282d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:39.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.908+0000 7fe967fff640 1 -- 172.21.15.112:0/782609328 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968036070 con 0x7fe970153c90 2024-04-25T01:39:39.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.909+0000 7fe967fff640 1 -- 172.21.15.112:0/782609328 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe968031600 con 0x7fe970153c90 2024-04-25T01:39:39.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.909+0000 7fe967fff640 1 -- 172.21.15.112:0/782609328 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9680319d0 con 0x7fe970153c90 2024-04-25T01:39:39.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.909+0000 7fe9778dd640 1 -- 172.21.15.112:0/782609328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe970153c90 msgr2=0x7fe9700c5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:39.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.910+0000 7fe9778dd640 1 --2- 172.21.15.112:0/782609328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe970153c90 0x7fe9700c5700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe9700c4a60 tx=0x7fe96802a6d0 comp rx=0 tx=0).stop 2024-04-25T01:39:39.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.912+0000 7fe9778dd640 1 -- 172.21.15.112:0/782609328 shutdown_connections 2024-04-25T01:39:39.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.912+0000 7fe9778dd640 1 --2- 172.21.15.112:0/782609328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe970153c90 0x7fe9700c5700 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:39.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.912+0000 7fe9778dd640 1 --2- 172.21.15.112:0/782609328 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9700ab190 0x7fe970153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:39.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.912+0000 7fe9778dd640 1 --2- 172.21.15.112:0/782609328 >> v2:172.21.15.112:3300/0 conn(0x7fe97014d6c0 0x7fe9700aac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:39.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.912+0000 7fe9778dd640 1 -- 172.21.15.112:0/782609328 >> 172.21.15.112:0/782609328 conn(0x7fe9700743d0 msgr2=0x7fe9700747b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:39.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.912+0000 7fe9778dd640 1 -- 172.21.15.112:0/782609328 shutdown_connections 2024-04-25T01:39:39.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.912+0000 7fe9778dd640 1 -- 172.21.15.112:0/782609328 wait complete. 2024-04-25T01:39:39.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.914+0000 7fe9778dd640 1 Processor -- start 2024-04-25T01:39:39.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.918+0000 7fe9778dd640 1 -- start start 2024-04-25T01:39:39.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.919+0000 7fe9778dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9700ab190 0x7fe9700a87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:39.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.919+0000 7fe9778dd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe97014d6c0 0x7fe9700acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:39.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.919+0000 7fe9778dd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe970153c90 0x7fe9701500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:39.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.919+0000 7fe9778dd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe970070f90 con 0x7fe9700ab190 2024-04-25T01:39:39.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.919+0000 7fe9778dd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe970070e10 con 0x7fe970153c90 2024-04-25T01:39:39.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.919+0000 7fe9778dd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe9700c5c40 con 0x7fe97014d6c0 2024-04-25T01:39:39.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.922+0000 7fe975e7a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe970153c90 0x7fe9701500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:39.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.923+0000 7fe975e7a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe970153c90 0x7fe9701500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:42656/0 (socket says 172.21.15.112:42656) 2024-04-25T01:39:39.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.923+0000 7fe975e7a640 1 -- 172.21.15.112:0/267619196 learned_addr learned my addr 172.21.15.112:0/267619196 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:39.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.923+0000 7fe975679640 1 --2- 172.21.15.112:0/267619196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9700ab190 0x7fe9700a87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:39.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.923+0000 7fe975e7a640 1 -- 172.21.15.112:0/267619196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe97014d6c0 msgr2=0x7fe9700acfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:39.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.923+0000 7fe974e78640 1 --2- 172.21.15.112:0/267619196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe97014d6c0 0x7fe9700acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:39.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.923+0000 7fe975e7a640 1 --2- 172.21.15.112:0/267619196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe97014d6c0 0x7fe9700acfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:39.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.923+0000 7fe975e7a640 1 -- 172.21.15.112:0/267619196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9700ab190 msgr2=0x7fe9700a87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:39.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.923+0000 7fe975e7a640 1 --2- 172.21.15.112:0/267619196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9700ab190 0x7fe9700a87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:39.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.923+0000 7fe975e7a640 1 -- 172.21.15.112:0/267619196 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9700c8cc0 con 0x7fe970153c90 2024-04-25T01:39:39.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.924+0000 7fe975679640 1 --2- 172.21.15.112:0/267619196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9700ab190 0x7fe9700a87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:39:39.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.924+0000 7fe975e7a640 1 --2- 172.21.15.112:0/267619196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe970153c90 0x7fe9701500b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe968034040 tx=0x7fe968004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:39.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.925+0000 7fe974e78640 1 --2- 172.21.15.112:0/267619196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe97014d6c0 0x7fe9700acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:39:39.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.925+0000 7fe965ffb640 1 -- 172.21.15.112:0/267619196 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968041020 con 0x7fe970153c90 2024-04-25T01:39:39.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.925+0000 7fe965ffb640 1 -- 172.21.15.112:0/267619196 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe968007780 con 0x7fe970153c90 2024-04-25T01:39:39.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.925+0000 7fe965ffb640 1 -- 172.21.15.112:0/267619196 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968036040 con 0x7fe970153c90 2024-04-25T01:39:39.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.926+0000 7fe9778dd640 1 -- 172.21.15.112:0/267619196 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9700c94a0 con 0x7fe970153c90 2024-04-25T01:39:39.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.926+0000 7fe9778dd640 1 -- 172.21.15.112:0/267619196 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9700ad520 con 0x7fe970153c90 2024-04-25T01:39:39.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.930+0000 7fe965ffb640 1 -- 172.21.15.112:0/267619196 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe968044020 con 0x7fe970153c90 2024-04-25T01:39:39.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.932+0000 7fe965ffb640 1 --2- 172.21.15.112:0/267619196 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe95406bf00 0x7fe95406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:39.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.932+0000 7fe965ffb640 1 -- 172.21.15.112:0/267619196 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fe9680abfe0 con 0x7fe970153c90 2024-04-25T01:39:39.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.932+0000 7fe9778dd640 1 -- 172.21.15.112:0/267619196 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe950005290 con 0x7fe970153c90 2024-04-25T01:39:39.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.933+0000 7fe975679640 1 --2- 172.21.15.112:0/267619196 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe95406bf00 0x7fe95406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:39.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.933+0000 7fe975679640 1 --2- 172.21.15.112:0/267619196 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe95406bf00 0x7fe95406e3c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00a8b0 tx=0x7fe96c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:39.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:39.943+0000 7fe965ffb640 1 -- 172.21.15.112:0/267619196 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe96804b030 con 0x7fe970153c90 2024-04-25T01:39:40.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.459+0000 7fe9778dd640 1 -- 172.21.15.112:0/267619196 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe950005580 con 0x7fe970153c90 2024-04-25T01:39:40.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.460+0000 7fe965ffb640 1 -- 172.21.15.112:0/267619196 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7fe96807a690 con 0x7fe970153c90 2024-04-25T01:39:40.446 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":110,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:39:40.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.464+0000 7fe9778dd640 1 -- 172.21.15.112:0/267619196 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe95406bf00 msgr2=0x7fe95406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:40.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.464+0000 7fe9778dd640 1 --2- 172.21.15.112:0/267619196 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe95406bf00 0x7fe95406e3c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00a8b0 tx=0x7fe96c008040 comp rx=0 tx=0).stop 2024-04-25T01:39:40.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.464+0000 7fe9778dd640 1 -- 172.21.15.112:0/267619196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe970153c90 msgr2=0x7fe9701500b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:40.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.464+0000 7fe9778dd640 1 --2- 172.21.15.112:0/267619196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe970153c90 0x7fe9701500b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe968034040 tx=0x7fe968004a40 comp rx=0 tx=0).stop 2024-04-25T01:39:40.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.464+0000 7fe9778dd640 1 -- 172.21.15.112:0/267619196 shutdown_connections 2024-04-25T01:39:40.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.464+0000 7fe9778dd640 1 --2- 172.21.15.112:0/267619196 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe95406bf00 0x7fe95406e3c0 secure :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00a8b0 tx=0x7fe96c008040 comp rx=0 tx=0).stop 2024-04-25T01:39:40.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.464+0000 7fe9778dd640 1 --2- 172.21.15.112:0/267619196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe970153c90 0x7fe9701500b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:40.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.464+0000 7fe9778dd640 1 --2- 172.21.15.112:0/267619196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe97014d6c0 0x7fe9700acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:40.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.464+0000 7fe9778dd640 1 --2- 172.21.15.112:0/267619196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9700ab190 0x7fe9700a87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:40.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.464+0000 7fe9778dd640 1 -- 172.21.15.112:0/267619196 >> 172.21.15.112:0/267619196 conn(0x7fe9700743d0 msgr2=0x7fe9700a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:40.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.467+0000 7fe9778dd640 1 -- 172.21.15.112:0/267619196 shutdown_connections 2024-04-25T01:39:40.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.467+0000 7fe9778dd640 1 -- 172.21.15.112:0/267619196 wait complete. 2024-04-25T01:39:40.468 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:39:40.468 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:39:40.468 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:39:40.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.680+0000 7f5d292ae640 1 Processor -- start 2024-04-25T01:39:40.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.681+0000 7f5d292ae640 1 -- start start 2024-04-25T01:39:40.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.683+0000 7f5d292ae640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d24139f80 0x7f5d2406db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:40.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.683+0000 7f5d292ae640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5d2406e090 0x7f5d2415c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:40.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.683+0000 7f5d292ae640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d24155cf0 con 0x7f5d240c29d0 2024-04-25T01:39:40.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.683+0000 7f5d292ae640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f5d241557c0 con 0x7f5d2406e090 2024-04-25T01:39:40.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.683+0000 7f5d292ae640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5d2413a360 con 0x7f5d24139f80 2024-04-25T01:39:40.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.683+0000 7f5d23fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d24139f80 0x7f5d2406db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:40.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.683+0000 7f5d23fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d24139f80 0x7f5d2406db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:33220/0 (socket says 172.21.15.112:33220) 2024-04-25T01:39:40.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.683+0000 7f5d23fff640 1 -- 172.21.15.112:0/4087115605 learned_addr learned my addr 172.21.15.112:0/4087115605 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:40.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.684+0000 7f5d23fff640 1 -- 172.21.15.112:0/4087115605 >> v2:172.21.15.112:3300/0 conn(0x7f5d2406e090 msgr2=0x7f5d2415c120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:39:40.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.684+0000 7f5d23fff640 1 --2- 172.21.15.112:0/4087115605 >> v2:172.21.15.112:3300/0 conn(0x7f5d2406e090 0x7f5d2415c120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:40.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.684+0000 7f5d23fff640 1 -- 172.21.15.112:0/4087115605 >> v1:172.21.15.112:6789/0 conn(0x7f5d240c29d0 legacy=0x7f5d24151e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:39:40.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.684+0000 7f5d23fff640 1 -- 172.21.15.112:0/4087115605 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2413a710 con 0x7f5d24139f80 2024-04-25T01:39:40.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.686+0000 7f5d23fff640 1 --2- 172.21.15.112:0/4087115605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d24139f80 0x7f5d2406db50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009e50 tx=0x7f5d1c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=627b59d216a019f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:40.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.687+0000 7f5d22ffd640 1 -- 172.21.15.112:0/4087115605 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c036070 con 0x7f5d24139f80 2024-04-25T01:39:40.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.687+0000 7f5d22ffd640 1 -- 172.21.15.112:0/4087115605 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d1c031600 con 0x7f5d24139f80 2024-04-25T01:39:40.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.687+0000 7f5d22ffd640 1 -- 172.21.15.112:0/4087115605 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0319d0 con 0x7f5d24139f80 2024-04-25T01:39:40.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.688+0000 7f5d292ae640 1 -- 172.21.15.112:0/4087115605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d24139f80 msgr2=0x7f5d2406db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:40.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.688+0000 7f5d292ae640 1 --2- 172.21.15.112:0/4087115605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d24139f80 0x7f5d2406db50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009e50 tx=0x7f5d1c02a660 comp rx=0 tx=0).stop 2024-04-25T01:39:40.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.688+0000 7f5d292ae640 1 -- 172.21.15.112:0/4087115605 shutdown_connections 2024-04-25T01:39:40.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.688+0000 7f5d292ae640 1 --2- 172.21.15.112:0/4087115605 >> v2:172.21.15.112:3300/0 conn(0x7f5d2406e090 0x7f5d2415c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:40.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.688+0000 7f5d292ae640 1 --2- 172.21.15.112:0/4087115605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d24139f80 0x7f5d2406db50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:40.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.688+0000 7f5d292ae640 1 -- 172.21.15.112:0/4087115605 >> 172.21.15.112:0/4087115605 conn(0x7f5d240c6da0 msgr2=0x7f5d240c7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:40.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.689+0000 7f5d292ae640 1 -- 172.21.15.112:0/4087115605 shutdown_connections 2024-04-25T01:39:40.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.689+0000 7f5d292ae640 1 -- 172.21.15.112:0/4087115605 wait complete. 2024-04-25T01:39:40.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.690+0000 7f5d292ae640 1 Processor -- start 2024-04-25T01:39:40.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.691+0000 7f5d292ae640 1 -- start start 2024-04-25T01:39:40.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.692+0000 7f5d292ae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d2406e090 0x7f5d24138b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:40.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.692+0000 7f5d292ae640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d240c29d0 0x7f5d241e8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:40.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.692+0000 7f5d292ae640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d24139f80 0x7f5d240c5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:40.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.692+0000 7f5d292ae640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d240c35e0 con 0x7f5d2406e090 2024-04-25T01:39:40.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.692+0000 7f5d292ae640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5d24151940 con 0x7f5d24139f80 2024-04-25T01:39:40.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.692+0000 7f5d292ae640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5d24070b10 con 0x7f5d240c29d0 2024-04-25T01:39:40.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.692+0000 7f5d28aad640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d24139f80 0x7f5d240c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:40.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.692+0000 7f5d28aad640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d24139f80 0x7f5d240c5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:42660/0 (socket says 172.21.15.112:42660) 2024-04-25T01:39:40.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.692+0000 7f5d28aad640 1 -- 172.21.15.112:0/1115683437 learned_addr learned my addr 172.21.15.112:0/1115683437 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:40.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.692+0000 7f5d23fff640 1 --2- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d2406e090 0x7f5d24138b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:40.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.693+0000 7f5d23fff640 1 -- 172.21.15.112:0/1115683437 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d240c29d0 msgr2=0x7f5d241e8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:40.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.693+0000 7f5d23fff640 1 --2- 172.21.15.112:0/1115683437 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d240c29d0 0x7f5d241e8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:40.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.693+0000 7f5d23fff640 1 -- 172.21.15.112:0/1115683437 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d24139f80 msgr2=0x7f5d240c5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:40.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.693+0000 7f5d23fff640 1 --2- 172.21.15.112:0/1115683437 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d24139f80 0x7f5d240c5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:40.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.693+0000 7f5d23fff640 1 -- 172.21.15.112:0/1115683437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d24139180 con 0x7f5d2406e090 2024-04-25T01:39:40.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.694+0000 7f5d23fff640 1 --2- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d2406e090 0x7f5d24138b30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c030000 tx=0x7f5d1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:40.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.696+0000 7f5d20ff9640 1 -- 172.21.15.112:0/1115683437 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c041070 con 0x7f5d2406e090 2024-04-25T01:39:40.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.696+0000 7f5d20ff9640 1 -- 172.21.15.112:0/1115683437 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d1c031e30 con 0x7f5d2406e090 2024-04-25T01:39:40.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.696+0000 7f5d20ff9640 1 -- 172.21.15.112:0/1115683437 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c036040 con 0x7f5d2406e090 2024-04-25T01:39:40.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.696+0000 7f5d292ae640 1 -- 172.21.15.112:0/1115683437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d241377a0 con 0x7f5d2406e090 2024-04-25T01:39:40.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.696+0000 7f5d292ae640 1 -- 172.21.15.112:0/1115683437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d24137d60 con 0x7f5d2406e090 2024-04-25T01:39:40.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.698+0000 7f5d292ae640 1 --2- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d04002a70 0x7f5d04004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:40.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.698+0000 7f5d292ae640 1 -- 172.21.15.112:0/1115683437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d04007ec0 con 0x7f5d04002a70 2024-04-25T01:39:40.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.699+0000 7f5d237fe640 1 --2- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d04002a70 0x7f5d04004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.700+0000 7f5d20ff9640 1 -- 172.21.15.112:0/1115683437 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d1c044070 con 0x7f5d2406e090 2024-04-25T01:39:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.700+0000 7f5d237fe640 1 -- 172.21.15.112:0/1115683437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d040080c0 con 0x7f5d04002a70 2024-04-25T01:39:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.700+0000 7f5d237fe640 1 --2- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d04002a70 0x7f5d04004f30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5d240c60d0 tx=0x7f5d1800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.700+0000 7f5d20ff9640 1 --2- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5d1006e110 0x7f5d100705d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.701+0000 7f5d28aad640 1 --2- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5d1006e110 0x7f5d100705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.701+0000 7f5d20ff9640 1 -- 172.21.15.112:0/1115683437 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f5d1c0abdd0 con 0x7f5d2406e090 2024-04-25T01:39:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.701+0000 7f5d20ff9640 1 -- 172.21.15.112:0/1115683437 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18007020 con 0x7f5d04002a70 2024-04-25T01:39:40.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.702+0000 7f5d28aad640 1 --2- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5d1006e110 0x7f5d100705d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5d14006fd0 tx=0x7f5d14008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:40.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.702+0000 7f5d20ff9640 1 -- 172.21.15.112:0/1115683437 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5d18007020 con 0x7f5d04002a70 2024-04-25T01:39:40.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.702+0000 7f5d20ff9640 1 -- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d04002a70 msgr2=0x7f5d04004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:40.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.702+0000 7f5d20ff9640 1 --2- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d04002a70 0x7f5d04004f30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5d240c60d0 tx=0x7f5d1800c7a0 comp rx=0 tx=0).stop 2024-04-25T01:39:40.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.702+0000 7f5d20ff9640 1 -- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d04002a70 msgr2=0x7f5d04004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:39:40.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.702+0000 7f5d20ff9640 1 --2- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d04002a70 0x7f5d04004f30 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5d240c60d0 tx=0x7f5d1800c7a0 comp rx=0 tx=0).stop 2024-04-25T01:39:40.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.723+0000 7f5d292ae640 1 --2- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d04006eb0 0x7f5d04007340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:40.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.723+0000 7f5d292ae640 1 -- 172.21.15.112:0/1115683437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0400a4a0 con 0x7f5d04006eb0 2024-04-25T01:39:40.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.725+0000 7f5d237fe640 1 --2- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d04006eb0 0x7f5d04007340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:40.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.727+0000 7f5d237fe640 1 -- 172.21.15.112:0/1115683437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5d04007880 con 0x7f5d04006eb0 2024-04-25T01:39:40.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.727+0000 7f5d237fe640 1 --2- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d04006eb0 0x7f5d04007340 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5d240c60d0 tx=0x7f5d18001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:40.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.729+0000 7f5d20ff9640 1 -- 172.21.15.112:0/1115683437 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18006c70 con 0x7f5d04006eb0 2024-04-25T01:39:40.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.729+0000 7f5d20ff9640 1 -- 172.21.15.112:0/1115683437 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4561 (secure 0 0 0) 0x7f5d18006e10 con 0x7f5d04006eb0 2024-04-25T01:39:40.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.729+0000 7f5d20ff9640 1 -- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d04006eb0 msgr2=0x7f5d04007340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:40.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.729+0000 7f5d20ff9640 1 --2- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d04006eb0 0x7f5d04007340 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5d240c60d0 tx=0x7f5d18001020 comp rx=0 tx=0).stop 2024-04-25T01:39:40.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.729+0000 7f5d20ff9640 1 -- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d04006eb0 msgr2=0x7f5d04007340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:39:40.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.729+0000 7f5d20ff9640 1 --2- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d04006eb0 0x7f5d04007340 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5d240c60d0 tx=0x7f5d18001020 comp rx=0 tx=0).stop 2024-04-25T01:39:40.713 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:39:40.713 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:39:40.713 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:39:40.713 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:39:40.713 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 110, 2024-04-25T01:39:40.713 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 28774, 2024-04-25T01:39:40.713 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:39:40.713 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:39:40.714 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:39:40.714 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:39:40.714 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:40.714 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 3, 2024-04-25T01:39:40.714 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:39:40.714 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:39:40.714 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:39:40.714 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:39:40.714 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:39:40.714 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:39:40.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:39:40.714 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:39:40.714 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:39:40.715 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:39:40.715 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:39:40.715 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:39:40.715 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:39:40.715 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:39:40.715 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:40.715 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:39:40.715 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:39:40.715 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:39:40.715 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:39:40.715 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:39:40.715 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:39:40.715 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:39:40.715 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:39:40.715 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:39:40.716 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:39:40.716 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:39:40.716 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:39:40.716 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:39:40.716 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:40.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:40.716 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:39:40.716 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:39:40.716 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:39:40.716 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:39:40.716 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:39:40.717 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:39:40.717 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:39:40.717 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:39:40.717 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:39:40.717 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:39:40.717 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:39:40.717 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:39:40.717 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:39:40.717 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:39:40.717 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:39:40.717 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:39:40.717 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:39:40.717 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:39:40.717 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:39:40.718 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:39:40.718 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:39:40.718 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:39:40.718 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:39:40.718 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:39:40.718 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:39:40.718 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:39:40.718 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:39:40.718 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:39:40.718 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:40.718 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:39:40.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:40.718 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:39:40.718 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:40.718 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:39:40.719 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:39:40.719 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:39:40.719 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:39:40.719 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:40.719 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:39:40.719 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:39:40.719 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:40.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:40.719 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:40.719 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:39:40.719 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:39:40.719 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:40.719 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:40.719 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:40.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:40.720 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:39:40.720 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:39:40.720 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:39:40.720 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:39:40.720 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:39:40.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:40.720 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:40.720 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:39:40.720 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:39:40.720 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:39:40.720 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:39:40.720 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:40.720 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:39:40.720 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:39:40.720 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:40.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:40.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:40.721 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:39:40.721 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:39:40.721 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:40.721 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:40.721 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:40.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:40.721 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:39:40.721 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:39:40.721 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:39:40.721 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:39:40.721 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:39:40.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:40.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:40.721 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:39:40.722 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:39:40.722 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:39:40.722 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:39:40.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:40.722 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:39:40.722 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:39:40.722 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:40.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:40.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:40.722 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:39:40.722 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:39:40.722 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:40.722 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:40.722 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:40.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:40.723 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:39:40.723 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:39:40.723 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:39:40.723 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:39:40.725 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:39:40.726 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:40.726 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:40.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:40.726 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:39:40.726 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:39:40.726 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:40.726 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:39:40.726 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:39:40.726 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:39:40.726 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:40.726 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:40.726 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:39:40.727 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:40.727 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:39:40.727 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:39:40.727 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:39:40.727 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:40.727 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:40.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:40.727 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:39:40.727 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:39:40.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.731+0000 7f5d292ae640 1 -- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5d1006e110 msgr2=0x7f5d100705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:40.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.731+0000 7f5d292ae640 1 --2- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5d1006e110 0x7f5d100705d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5d14006fd0 tx=0x7f5d14008040 comp rx=0 tx=0).stop 2024-04-25T01:39:40.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.731+0000 7f5d292ae640 1 -- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d2406e090 msgr2=0x7f5d24138b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:40.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.731+0000 7f5d292ae640 1 --2- 172.21.15.112:0/1115683437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d2406e090 0x7f5d24138b30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c030000 tx=0x7f5d1c004a40 comp rx=0 tx=0).stop 2024-04-25T01:39:40.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.733+0000 7f5d28aad640 1 -- 172.21.15.112:0/1115683437 reap_dead start 2024-04-25T01:39:40.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.733+0000 7f5d28aad640 1 -- 172.21.15.112:0/1115683437 reap_dead start 2024-04-25T01:39:40.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.733+0000 7f5d292ae640 1 -- 172.21.15.112:0/1115683437 shutdown_connections 2024-04-25T01:39:40.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.733+0000 7f5d292ae640 1 -- 172.21.15.112:0/1115683437 >> 172.21.15.112:0/1115683437 conn(0x7f5d240c6da0 msgr2=0x7f5d24151f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:40.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.740+0000 7f5d292ae640 1 -- 172.21.15.112:0/1115683437 shutdown_connections 2024-04-25T01:39:40.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.740+0000 7f5d292ae640 1 -- 172.21.15.112:0/1115683437 wait complete. 2024-04-25T01:39:40.743 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:39:40.743 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:39:40.743 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:39:40.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.941+0000 7f9b972f8640 1 Processor -- start 2024-04-25T01:39:40.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.941+0000 7f9b972f8640 1 -- start start 2024-04-25T01:39:40.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.942+0000 7f9b972f8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b901573e0 0x7f9b9015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:40.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.942+0000 7f9b972f8640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b9015ab20 con 0x7f9b90071500 2024-04-25T01:39:40.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.942+0000 7f9b972f8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b90072330 con 0x7f9b90072990 2024-04-25T01:39:40.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.942+0000 7f9b972f8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9b9006f0d0 con 0x7f9b901573e0 2024-04-25T01:39:40.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.943+0000 7f9b96af7640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9b90071500 0x7f9b90164470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:42670/0 (socket says 172.21.15.112:42670) 2024-04-25T01:39:40.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.943+0000 7f9b96af7640 1 -- 172.21.15.112:0/2522162304 learned_addr learned my addr 172.21.15.112:0/2522162304 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:39:40.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.943+0000 7f9b962f6640 1 --2- 172.21.15.112:0/2522162304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b901573e0 0x7f9b9015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:40.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.944+0000 7f9b952f4640 1 -- 172.21.15.112:0/2522162304 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 685656290 0 0) 0x7f9b9015ab20 con 0x7f9b90071500 2024-04-25T01:39:40.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.944+0000 7f9b952f4640 1 -- 172.21.15.112:0/2522162304 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b7c003520 con 0x7f9b90071500 2024-04-25T01:39:40.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.944+0000 7f9b962f6640 1 -- 172.21.15.112:0/2522162304 >> v1:172.21.15.187:6789/0 conn(0x7f9b90072990 legacy=0x7f9b900c2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:39:40.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.944+0000 7f9b962f6640 1 -- 172.21.15.112:0/2522162304 >> v1:172.21.15.112:6789/0 conn(0x7f9b90071500 legacy=0x7f9b90164470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:40.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.944+0000 7f9b962f6640 1 -- 172.21.15.112:0/2522162304 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b90072610 con 0x7f9b901573e0 2024-04-25T01:39:40.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.946+0000 7f9b962f6640 1 --2- 172.21.15.112:0/2522162304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b901573e0 0x7f9b9015a5e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b800097d0 tx=0x7f9b8002a120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=611c974633ae3817 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:40.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.949+0000 7f9b952f4640 1 -- 172.21.15.112:0/2522162304 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80037070 con 0x7f9b901573e0 2024-04-25T01:39:40.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.949+0000 7f9b952f4640 1 -- 172.21.15.112:0/2522162304 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b8002ae30 con 0x7f9b901573e0 2024-04-25T01:39:40.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.949+0000 7f9b952f4640 1 -- 172.21.15.112:0/2522162304 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80032640 con 0x7f9b901573e0 2024-04-25T01:39:40.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.951+0000 7f9b972f8640 1 -- 172.21.15.112:0/2522162304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b901573e0 msgr2=0x7f9b9015a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:40.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.951+0000 7f9b972f8640 1 --2- 172.21.15.112:0/2522162304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b901573e0 0x7f9b9015a5e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b800097d0 tx=0x7f9b8002a120 comp rx=0 tx=0).stop 2024-04-25T01:39:40.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.952+0000 7f9b972f8640 1 -- 172.21.15.112:0/2522162304 shutdown_connections 2024-04-25T01:39:40.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.952+0000 7f9b972f8640 1 --2- 172.21.15.112:0/2522162304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b901573e0 0x7f9b9015a5e0 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b800097d0 tx=0x7f9b8002a120 comp rx=0 tx=0).stop 2024-04-25T01:39:40.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.952+0000 7f9b972f8640 1 -- 172.21.15.112:0/2522162304 >> 172.21.15.112:0/2522162304 conn(0x7f9b900c3930 msgr2=0x7f9b900c3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:40.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.956+0000 7f9b972f8640 1 -- 172.21.15.112:0/2522162304 shutdown_connections 2024-04-25T01:39:40.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.957+0000 7f9b972f8640 1 -- 172.21.15.112:0/2522162304 wait complete. 2024-04-25T01:39:40.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.959+0000 7f9b972f8640 1 Processor -- start 2024-04-25T01:39:40.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.959+0000 7f9b972f8640 1 -- start start 2024-04-25T01:39:40.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.960+0000 7f9b972f8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b90071500 0x7f9b90137d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:40.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.960+0000 7f9b972f8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b90072990 0x7f9b900a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:40.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.960+0000 7f9b972f8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b901231f0 0x7f9b900c2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:40.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.960+0000 7f9b972f8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9b900c2f50 con 0x7f9b90072990 2024-04-25T01:39:40.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.960+0000 7f9b972f8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9b90071070 con 0x7f9b901231f0 2024-04-25T01:39:40.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.960+0000 7f9b972f8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9b900c30d0 con 0x7f9b90071500 2024-04-25T01:39:40.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.961+0000 7f9b95af5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b90072990 0x7f9b900a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:40.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.961+0000 7f9b95af5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b90072990 0x7f9b900a9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40352/0 (socket says 172.21.15.112:40352) 2024-04-25T01:39:40.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.961+0000 7f9b95af5640 1 -- 172.21.15.112:0/590178929 learned_addr learned my addr 172.21.15.112:0/590178929 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:40.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.961+0000 7f9b962f6640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b90071500 0x7f9b90137d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:40.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.961+0000 7f9b96af7640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b901231f0 0x7f9b900c2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:40.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.961+0000 7f9b95af5640 1 -- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b90071500 msgr2=0x7f9b90137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:40.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.962+0000 7f9b95af5640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b90071500 0x7f9b90137d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:40.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.962+0000 7f9b95af5640 1 -- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b901231f0 msgr2=0x7f9b900c2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:40.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.962+0000 7f9b95af5640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b901231f0 0x7f9b900c2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:40.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.962+0000 7f9b95af5640 1 -- 172.21.15.112:0/590178929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b900aa2c0 con 0x7f9b90072990 2024-04-25T01:39:40.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.962+0000 7f9b96af7640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b901231f0 0x7f9b900c2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:39:40.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.965+0000 7f9b95af5640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b90072990 0x7f9b900a9d80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00b3f0 tx=0x7f9b8c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:40.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.966+0000 7f9b86ffd640 1 -- 172.21.15.112:0/590178929 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c015020 con 0x7f9b90072990 2024-04-25T01:39:40.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.966+0000 7f9b86ffd640 1 -- 172.21.15.112:0/590178929 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b8c011830 con 0x7f9b90072990 2024-04-25T01:39:40.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.966+0000 7f9b972f8640 1 -- 172.21.15.112:0/590178929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b900aaae0 con 0x7f9b90072990 2024-04-25T01:39:40.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.966+0000 7f9b972f8640 1 -- 172.21.15.112:0/590178929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b900ab0a0 con 0x7f9b90072990 2024-04-25T01:39:40.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.966+0000 7f9b86ffd640 1 -- 172.21.15.112:0/590178929 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c011c20 con 0x7f9b90072990 2024-04-25T01:39:40.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.967+0000 7f9b972f8640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b78002a70 0x7f9b78004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:40.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.967+0000 7f9b972f8640 1 -- 172.21.15.112:0/590178929 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9b78007ec0 con 0x7f9b78002a70 2024-04-25T01:39:40.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.968+0000 7f9b962f6640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b78002a70 0x7f9b78004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:40.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.970+0000 7f9b86ffd640 1 -- 172.21.15.112:0/590178929 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9b8c018020 con 0x7f9b90072990 2024-04-25T01:39:40.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.971+0000 7f9b86ffd640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9b7406bf30 0x7f9b7406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:40.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.971+0000 7f9b962f6640 1 -- 172.21.15.112:0/590178929 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b780080c0 con 0x7f9b78002a70 2024-04-25T01:39:40.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.971+0000 7f9b96af7640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9b7406bf30 0x7f9b7406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:40.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.971+0000 7f9b86ffd640 1 -- 172.21.15.112:0/590178929 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f9b8c08b080 con 0x7f9b90072990 2024-04-25T01:39:40.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.971+0000 7f9b962f6640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b78002a70 0x7f9b78004f30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9b90075550 tx=0x7f9b800046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:40.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.972+0000 7f9b96af7640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9b7406bf30 0x7f9b7406e3f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9b88005c00 tx=0x7f9b88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:40.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.979+0000 7f9b86ffd640 1 -- 172.21.15.112:0/590178929 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80037040 con 0x7f9b78002a70 2024-04-25T01:39:40.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.979+0000 7f9b86ffd640 1 -- 172.21.15.112:0/590178929 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9b8003c660 con 0x7f9b78002a70 2024-04-25T01:39:40.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.979+0000 7f9b86ffd640 1 -- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b78002a70 msgr2=0x7f9b78004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:40.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.979+0000 7f9b86ffd640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b78002a70 0x7f9b78004f30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9b90075550 tx=0x7f9b800046c0 comp rx=0 tx=0).stop 2024-04-25T01:39:40.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.979+0000 7f9b86ffd640 1 -- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b78002a70 msgr2=0x7f9b78004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:39:40.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.979+0000 7f9b86ffd640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b78002a70 0x7f9b78004f30 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9b90075550 tx=0x7f9b800046c0 comp rx=0 tx=0).stop 2024-04-25T01:39:40.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.996+0000 7f9b84ff9640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b6c002920 0x7f9b6c004dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:40.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.996+0000 7f9b84ff9640 1 -- 172.21.15.112:0/590178929 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c007d50 con 0x7f9b6c002920 2024-04-25T01:39:40.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.997+0000 7f9b962f6640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b6c002920 0x7f9b6c004dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:40.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:40.999+0000 7f9b962f6640 1 -- 172.21.15.112:0/590178929 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9b6c007f50 con 0x7f9b6c002920 2024-04-25T01:39:40.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.002+0000 7f9b962f6640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b6c002920 0x7f9b6c004dc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9b800097d0 tx=0x7f9b80008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:40.990 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:39:40.991 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:39:40.991 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:39:40.991 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:39:40.991 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 110, 2024-04-25T01:39:40.991 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 85768, 2024-04-25T01:39:40.991 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:39:40.991 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:39:40.991 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:39:40.991 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:39:40.991 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:40.991 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 3, 2024-04-25T01:39:40.991 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:39:40.991 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:39:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:39:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:39:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:39:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:39:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:39:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:39:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:39:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:39:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:39:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:39:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:39:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:39:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:39:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:39:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:39:40.993 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:39:40.993 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:39:40.993 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:39:40.993 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:39:40.993 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:39:40.993 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:39:40.993 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:39:40.993 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:39:40.993 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:39:40.993 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:39:40.993 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:40.993 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:40.993 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:39:40.993 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:39:40.994 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:39:40.994 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:39:40.994 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:39:40.994 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:39:40.994 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:39:40.994 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:39:40.994 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:39:40.994 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:39:40.994 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:39:40.994 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:39:40.994 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:39:40.994 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:39:40.994 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:39:40.994 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:39:40.995 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:39:40.995 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:39:40.995 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:39:40.995 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:39:40.995 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:39:40.995 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:39:40.995 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:39:40.995 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:39:40.995 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:39:40.995 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:39:40.995 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:39:40.995 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:39:40.996 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:40.996 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:39:40.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:40.996 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:39:40.996 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:40.996 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:39:40.996 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:39:40.996 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:39:40.996 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:39:40.996 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:40.996 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:39:40.996 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:39:40.997 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:40.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:40.997 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:40.997 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:39:40.997 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:39:40.997 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:40.997 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:40.997 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:40.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:40.997 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:39:40.997 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:39:40.997 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:39:40.997 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:39:40.998 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:39:40.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:40.998 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:40.998 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:39:40.998 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:39:40.998 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:39:40.998 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:39:40.998 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:40.998 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:39:40.998 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:39:40.998 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:40.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:40.998 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:40.998 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:39:40.999 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:39:40.999 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:40.999 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:40.999 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:40.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:40.999 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:39:40.999 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:39:40.999 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:39:40.999 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:39:40.999 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:39:40.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:40.999 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:40.999 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:39:40.999 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:39:40.999 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:39:41.000 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:39:41.000 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:41.000 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:39:41.000 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:39:41.000 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:41.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:41.000 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:41.000 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:39:41.000 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:39:41.000 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:41.000 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:41.000 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:41.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:41.000 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:39:41.001 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:39:41.001 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:39:41.001 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:39:41.001 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:39:41.001 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:41.001 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:41.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:41.001 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:39:41.001 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:39:41.001 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:41.001 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:39:41.001 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:39:41.001 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:39:41.001 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:41.002 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:41.002 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:39:41.002 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:41.002 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:39:41.002 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:39:41.002 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:39:41.002 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:41.002 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:41.002 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:39:41.002 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:41.002 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:39:41.002 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:39:41.002 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:39:41.002 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:41.003 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:41.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:41.003 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:39:41.003 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:39:41.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.007+0000 7f9b86ffd640 1 -- 172.21.15.112:0/590178929 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8004c070 con 0x7f9b6c002920 2024-04-25T01:39:41.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.007+0000 7f9b86ffd640 1 -- 172.21.15.112:0/590178929 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4729 (secure 0 0 0) 0x7f9b8004a070 con 0x7f9b6c002920 2024-04-25T01:39:41.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.007+0000 7f9b86ffd640 1 -- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b6c002920 msgr2=0x7f9b6c004dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:41.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.007+0000 7f9b86ffd640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b6c002920 0x7f9b6c004dc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9b800097d0 tx=0x7f9b80008030 comp rx=0 tx=0).stop 2024-04-25T01:39:41.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.007+0000 7f9b86ffd640 1 -- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b6c002920 msgr2=0x7f9b6c004dc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:39:41.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.007+0000 7f9b86ffd640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b6c002920 0x7f9b6c004dc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:41.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.009+0000 7f9b84ff9640 1 -- 172.21.15.112:0/590178929 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9b7406bf30 msgr2=0x7f9b7406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:41.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.009+0000 7f9b84ff9640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9b7406bf30 0x7f9b7406e3f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9b88005c00 tx=0x7f9b88009040 comp rx=0 tx=0).stop 2024-04-25T01:39:41.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.009+0000 7f9b96af7640 1 -- 172.21.15.112:0/590178929 reap_dead start 2024-04-25T01:39:41.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.010+0000 7f9b84ff9640 1 -- 172.21.15.112:0/590178929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b90072990 msgr2=0x7f9b900a9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:41.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.010+0000 7f9b84ff9640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b90072990 0x7f9b900a9d80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00b3f0 tx=0x7f9b8c00b8b0 comp rx=0 tx=0).stop 2024-04-25T01:39:41.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.010+0000 7f9b84ff9640 1 -- 172.21.15.112:0/590178929 shutdown_connections 2024-04-25T01:39:41.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.010+0000 7f9b84ff9640 1 --2- 172.21.15.112:0/590178929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b90072990 0x7f9b900a9d80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:41.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.011+0000 7f9b84ff9640 1 -- 172.21.15.112:0/590178929 >> 172.21.15.112:0/590178929 conn(0x7f9b900c3930 msgr2=0x7f9b90156140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:41.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.011+0000 7f9b84ff9640 1 -- 172.21.15.112:0/590178929 shutdown_connections 2024-04-25T01:39:41.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.013+0000 7f9b84ff9640 1 -- 172.21.15.112:0/590178929 wait complete. 2024-04-25T01:39:41.016 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:39:41.016 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:39:41.016 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:39:41.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.212+0000 7fcda759e640 1 Processor -- start 2024-04-25T01:39:41.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.213+0000 7fcda759e640 1 -- start start 2024-04-25T01:39:41.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.214+0000 7fcda759e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcda808ba50 0x7fcda806d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:41.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.214+0000 7fcda759e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcda808b500 0x7fcda8162540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:41.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.214+0000 7fcda759e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcda80723a0 con 0x7fcda815fcd0 2024-04-25T01:39:41.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.214+0000 7fcda759e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcda8072710 con 0x7fcda808b500 2024-04-25T01:39:41.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.214+0000 7fcda759e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcda806df70 con 0x7fcda808ba50 2024-04-25T01:39:41.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.214+0000 7fcda659c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcda808ba50 0x7fcda806d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.214+0000 7fcda659c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcda808ba50 0x7fcda806d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:33242/0 (socket says 172.21.15.112:33242) 2024-04-25T01:39:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.214+0000 7fcda659c640 1 -- 172.21.15.112:0/685996760 learned_addr learned my addr 172.21.15.112:0/685996760 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.215+0000 7fcda6d9d640 1 --2- 172.21.15.112:0/685996760 >> v2:172.21.15.187:3300/0 conn(0x7fcda808b500 0x7fcda8162540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.218+0000 7fcda6d9d640 1 -- 172.21.15.112:0/685996760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcda808ba50 msgr2=0x7fcda806d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:39:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.218+0000 7fcda6d9d640 1 --2- 172.21.15.112:0/685996760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcda808ba50 0x7fcda806d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.218+0000 7fcda6d9d640 1 -- 172.21.15.112:0/685996760 >> v1:172.21.15.187:6789/0 conn(0x7fcda815fcd0 legacy=0x7fcda806c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.218+0000 7fcda6d9d640 1 -- 172.21.15.112:0/685996760 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda806d060 con 0x7fcda808b500 2024-04-25T01:39:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.219+0000 7fcda6d9d640 1 --2- 172.21.15.112:0/685996760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcda808b500 0x7fcda8162540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009d80 tx=0x7fcda003b5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3716f248aaa133c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:41.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.219+0000 7fcda559a640 1 -- 172.21.15.112:0/685996760 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda0007540 con 0x7fcda808b500 2024-04-25T01:39:41.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.220+0000 7fcda559a640 1 -- 172.21.15.112:0/685996760 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda00076e0 con 0x7fcda808b500 2024-04-25T01:39:41.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.221+0000 7fcda759e640 1 -- 172.21.15.112:0/685996760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcda808b500 msgr2=0x7fcda8162540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:41.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.221+0000 7fcda759e640 1 --2- 172.21.15.112:0/685996760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcda808b500 0x7fcda8162540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009d80 tx=0x7fcda003b5e0 comp rx=0 tx=0).stop 2024-04-25T01:39:41.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.222+0000 7fcda759e640 1 -- 172.21.15.112:0/685996760 shutdown_connections 2024-04-25T01:39:41.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.222+0000 7fcda759e640 1 --2- 172.21.15.112:0/685996760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcda808b500 0x7fcda8162540 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:41.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.222+0000 7fcda759e640 1 --2- 172.21.15.112:0/685996760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcda808ba50 0x7fcda806d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:41.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.222+0000 7fcda759e640 1 -- 172.21.15.112:0/685996760 >> 172.21.15.112:0/685996760 conn(0x7fcda80c3930 msgr2=0x7fcda80c3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:41.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.234+0000 7fcda759e640 1 -- 172.21.15.112:0/685996760 shutdown_connections 2024-04-25T01:39:41.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.234+0000 7fcda759e640 1 -- 172.21.15.112:0/685996760 wait complete. 2024-04-25T01:39:41.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.236+0000 7fcda759e640 1 Processor -- start 2024-04-25T01:39:41.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.236+0000 7fcda759e640 1 -- start start 2024-04-25T01:39:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.237+0000 7fcda759e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcda808b500 0x7fcda80a9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.237+0000 7fcda759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcda815fcd0 0x7fcda81570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.237+0000 7fcda759e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcda8154b10 0x7fcda8154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.237+0000 7fcda759e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcda806eb10 con 0x7fcda815fcd0 2024-04-25T01:39:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.237+0000 7fcda759e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcda8072db0 con 0x7fcda8154b10 2024-04-25T01:39:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.237+0000 7fcda759e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcda806ec90 con 0x7fcda808b500 2024-04-25T01:39:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.237+0000 7fcda5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcda815fcd0 0x7fcda81570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.237+0000 7fcda5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcda815fcd0 0x7fcda81570f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40360/0 (socket says 172.21.15.112:40360) 2024-04-25T01:39:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.237+0000 7fcda5d9b640 1 -- 172.21.15.112:0/2906271749 learned_addr learned my addr 172.21.15.112:0/2906271749 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.238+0000 7fcda659c640 1 --2- 172.21.15.112:0/2906271749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcda808b500 0x7fcda80a9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.238+0000 7fcda5d9b640 1 -- 172.21.15.112:0/2906271749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcda808b500 msgr2=0x7fcda80a9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.238+0000 7fcda5d9b640 1 --2- 172.21.15.112:0/2906271749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcda808b500 0x7fcda80a9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.238+0000 7fcda5d9b640 1 -- 172.21.15.112:0/2906271749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcda8154b10 msgr2=0x7fcda8154f90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:39:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.238+0000 7fcda5d9b640 1 --2- 172.21.15.112:0/2906271749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcda8154b10 0x7fcda8154f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.238+0000 7fcda5d9b640 1 -- 172.21.15.112:0/2906271749 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda8157d00 con 0x7fcda815fcd0 2024-04-25T01:39:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.238+0000 7fcda5d9b640 1 --2- 172.21.15.112:0/2906271749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcda815fcd0 0x7fcda81570f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00c750 tx=0x7fcd9c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.239+0000 7fcd96ffd640 1 -- 172.21.15.112:0/2906271749 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c0050d0 con 0x7fcda815fcd0 2024-04-25T01:39:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.239+0000 7fcd96ffd640 1 -- 172.21.15.112:0/2906271749 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd9c005270 con 0x7fcda815fcd0 2024-04-25T01:39:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.239+0000 7fcd96ffd640 1 -- 172.21.15.112:0/2906271749 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c0035f0 con 0x7fcda815fcd0 2024-04-25T01:39:41.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.239+0000 7fcda759e640 1 -- 172.21.15.112:0/2906271749 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda81fb3e0 con 0x7fcda815fcd0 2024-04-25T01:39:41.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.239+0000 7fcda759e640 1 -- 172.21.15.112:0/2906271749 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda80aa770 con 0x7fcda815fcd0 2024-04-25T01:39:41.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.243+0000 7fcd96ffd640 1 -- 172.21.15.112:0/2906271749 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd9c0037b0 con 0x7fcda815fcd0 2024-04-25T01:39:41.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.243+0000 7fcd96ffd640 1 --2- 172.21.15.112:0/2906271749 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcd8806bf30 0x7fcd8806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:41.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.243+0000 7fcd96ffd640 1 -- 172.21.15.112:0/2906271749 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fcd9c08ac60 con 0x7fcda815fcd0 2024-04-25T01:39:41.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.243+0000 7fcda659c640 1 --2- 172.21.15.112:0/2906271749 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcd8806bf30 0x7fcd8806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:41.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.244+0000 7fcda759e640 1 --2- 172.21.15.112:0/2906271749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd8c002a70 0x7fcd8c004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:41.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.244+0000 7fcda759e640 1 -- 172.21.15.112:0/2906271749 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcd8c007ec0 con 0x7fcd8c002a70 2024-04-25T01:39:41.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.244+0000 7fcda6d9d640 1 --2- 172.21.15.112:0/2906271749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd8c002a70 0x7fcd8c004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:41.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.246+0000 7fcda659c640 1 --2- 172.21.15.112:0/2906271749 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcd8806bf30 0x7fcd8806e3f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcd98009bd0 tx=0x7fcd98009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:41.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.246+0000 7fcda6d9d640 1 -- 172.21.15.112:0/2906271749 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd8c0080c0 con 0x7fcd8c002a70 2024-04-25T01:39:41.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.247+0000 7fcda6d9d640 1 --2- 172.21.15.112:0/2906271749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd8c002a70 0x7fcd8c004f30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcda806c140 tx=0x7fcda0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:41.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.249+0000 7fcd96ffd640 1 -- 172.21.15.112:0/2906271749 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda0009450 con 0x7fcd8c002a70 2024-04-25T01:39:41.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.249+0000 7fcd96ffd640 1 -- 172.21.15.112:0/2906271749 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fcda0011070 con 0x7fcd8c002a70 2024-04-25T01:39:41.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.249+0000 7fcd96ffd640 1 -- 172.21.15.112:0/2906271749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd8c002a70 msgr2=0x7fcd8c004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:41.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.249+0000 7fcd96ffd640 1 --2- 172.21.15.112:0/2906271749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd8c002a70 0x7fcd8c004f30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcda806c140 tx=0x7fcda0004a40 comp rx=0 tx=0).stop 2024-04-25T01:39:41.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.249+0000 7fcd96ffd640 1 -- 172.21.15.112:0/2906271749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd8c002a70 msgr2=0x7fcd8c004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:39:41.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.249+0000 7fcd96ffd640 1 --2- 172.21.15.112:0/2906271749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd8c002a70 0x7fcd8c004f30 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcda806c140 tx=0x7fcda0004a40 comp rx=0 tx=0).stop 2024-04-25T01:39:41.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.270+0000 7fcda759e640 1 --2- 172.21.15.112:0/2906271749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd8c006ee0 0x7fcd8c007370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:41.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.270+0000 7fcda759e640 1 -- 172.21.15.112:0/2906271749 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcd8c00a4a0 con 0x7fcd8c006ee0 2024-04-25T01:39:41.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.271+0000 7fcda6d9d640 1 --2- 172.21.15.112:0/2906271749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd8c006ee0 0x7fcd8c007370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:41.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.276+0000 7fcda6d9d640 1 -- 172.21.15.112:0/2906271749 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fcd8c0078b0 con 0x7fcd8c006ee0 2024-04-25T01:39:41.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.277+0000 7fcda6d9d640 1 --2- 172.21.15.112:0/2906271749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd8c006ee0 0x7fcd8c007370 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009d80 tx=0x7fcda0009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:41.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.278+0000 7fcd96ffd640 1 -- 172.21.15.112:0/2906271749 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda0050070 con 0x7fcd8c006ee0 2024-04-25T01:39:41.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.278+0000 7fcd96ffd640 1 -- 172.21.15.112:0/2906271749 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5070 (secure 0 0 0) 0x7fcda001b040 con 0x7fcd8c006ee0 2024-04-25T01:39:41.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.278+0000 7fcd96ffd640 1 -- 172.21.15.112:0/2906271749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd8c006ee0 msgr2=0x7fcd8c007370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:41.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.278+0000 7fcd96ffd640 1 --2- 172.21.15.112:0/2906271749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd8c006ee0 0x7fcd8c007370 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009d80 tx=0x7fcda0009750 comp rx=0 tx=0).stop 2024-04-25T01:39:41.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.278+0000 7fcd96ffd640 1 -- 172.21.15.112:0/2906271749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd8c006ee0 msgr2=0x7fcd8c007370 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:39:41.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.278+0000 7fcd96ffd640 1 --2- 172.21.15.112:0/2906271749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd8c006ee0 0x7fcd8c007370 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009d80 tx=0x7fcda0009750 comp rx=0 tx=0).stop 2024-04-25T01:39:41.262 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:39:41.262 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:39:41.262 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:39:41.262 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:39:41.262 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 110, 2024-04-25T01:39:41.263 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 150642, 2024-04-25T01:39:41.263 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:39:41.263 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:39:41.263 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:39:41.263 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:39:41.263 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:41.263 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 3, 2024-04-25T01:39:41.263 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:39:41.263 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:39:41.263 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:39:41.263 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:39:41.263 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:39:41.263 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:39:41.264 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:39:41.264 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:39:41.264 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:39:41.264 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:39:41.264 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:39:41.264 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:39:41.264 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:39:41.264 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:39:41.264 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:41.264 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:39:41.264 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:39:41.264 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:39:41.264 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:39:41.264 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:39:41.264 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:39:41.264 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:39:41.265 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:39:41.265 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:39:41.265 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:39:41.265 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:39:41.265 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:39:41.265 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:39:41.265 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:41.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:41.265 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:39:41.265 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:39:41.265 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:39:41.265 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:39:41.265 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:39:41.265 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:39:41.265 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:39:41.265 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:39:41.266 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:39:41.266 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:39:41.266 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:39:41.266 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:39:41.266 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:39:41.266 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:39:41.266 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:39:41.266 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:39:41.266 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:39:41.266 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:39:41.266 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:39:41.266 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:39:41.266 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:39:41.267 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:39:41.267 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:39:41.267 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:39:41.267 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:39:41.267 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:39:41.267 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:39:41.267 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:39:41.267 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:41.267 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:39:41.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:41.267 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:39:41.267 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:41.267 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:39:41.267 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:39:41.268 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:39:41.268 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:39:41.268 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:41.268 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:39:41.268 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:39:41.268 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:41.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:41.268 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:41.268 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:39:41.268 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:39:41.268 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:41.268 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:41.268 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:41.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:41.268 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:39:41.268 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:39:41.269 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:39:41.269 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:39:41.269 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:39:41.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:41.269 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:41.269 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:39:41.269 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:39:41.269 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:39:41.269 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:39:41.269 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:41.269 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:39:41.270 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:39:41.270 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:41.270 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:41.270 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:41.270 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:39:41.270 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:39:41.270 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:41.270 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:41.270 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:41.270 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:41.270 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:39:41.270 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:39:41.270 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:39:41.270 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:39:41.271 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:39:41.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:41.271 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:41.271 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:39:41.271 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:39:41.271 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:39:41.271 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:39:41.271 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:41.271 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:39:41.271 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:39:41.272 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:41.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:41.272 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:41.272 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:39:41.272 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:39:41.272 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:41.272 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:41.272 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:41.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:41.272 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:39:41.272 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:39:41.272 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:39:41.272 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:39:41.272 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:39:41.273 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:41.273 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:41.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:41.273 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:39:41.273 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:39:41.273 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:41.273 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:39:41.273 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:39:41.273 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:39:41.273 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:41.273 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:41.273 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:39:41.273 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:41.273 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:39:41.273 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:39:41.274 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:39:41.274 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:41.274 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:41.274 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:39:41.274 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:41.274 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:39:41.274 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:39:41.274 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 8 2024-04-25T01:39:41.274 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:41.274 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:41.274 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:39:41.275 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:41.275 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:39:41.275 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:39:41.275 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:39:41.275 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:41.275 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:41.275 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:39:41.275 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:41.275 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:39:41.275 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:39:41.275 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:39:41.276 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:41.276 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:41.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:41.276 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:39:41.276 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:39:41.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.279+0000 7fcda759e640 1 -- 172.21.15.112:0/2906271749 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcd8806bf30 msgr2=0x7fcd8806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:41.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.279+0000 7fcda759e640 1 --2- 172.21.15.112:0/2906271749 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcd8806bf30 0x7fcd8806e3f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcd98009bd0 tx=0x7fcd98009240 comp rx=0 tx=0).stop 2024-04-25T01:39:41.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.279+0000 7fcda6d9d640 1 -- 172.21.15.112:0/2906271749 reap_dead start 2024-04-25T01:39:41.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.279+0000 7fcda759e640 1 -- 172.21.15.112:0/2906271749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcda815fcd0 msgr2=0x7fcda81570f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:41.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.279+0000 7fcda759e640 1 --2- 172.21.15.112:0/2906271749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcda815fcd0 0x7fcda81570f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00c750 tx=0x7fcd9c00cc10 comp rx=0 tx=0).stop 2024-04-25T01:39:41.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.280+0000 7fcda759e640 1 -- 172.21.15.112:0/2906271749 shutdown_connections 2024-04-25T01:39:41.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.280+0000 7fcda759e640 1 --2- 172.21.15.112:0/2906271749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcda815fcd0 0x7fcda81570f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:41.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.280+0000 7fcda759e640 1 -- 172.21.15.112:0/2906271749 >> 172.21.15.112:0/2906271749 conn(0x7fcda80c3930 msgr2=0x7fcda815b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:41.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.280+0000 7fcda759e640 1 -- 172.21.15.112:0/2906271749 shutdown_connections 2024-04-25T01:39:41.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.280+0000 7fcda759e640 1 -- 172.21.15.112:0/2906271749 wait complete. 2024-04-25T01:39:41.290 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:39:41.290 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T01:39:41.290 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T01:39:41.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.534+0000 7feec6ca7640 1 Processor -- start 2024-04-25T01:39:41.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.535+0000 7feec6ca7640 1 -- start start 2024-04-25T01:39:41.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.536+0000 7feec6ca7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feec006bb70 0x7feec006ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:41.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.536+0000 7feec6ca7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feec0159b10 0x7feec0159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:41.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.536+0000 7feec6ca7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feec006dfb0 con 0x7feec0157140 2024-04-25T01:39:41.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.536+0000 7feec6ca7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7feec006e290 con 0x7feec0159b10 2024-04-25T01:39:41.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.536+0000 7feec6ca7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feec006e570 con 0x7feec006bb70 2024-04-25T01:39:41.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.536+0000 7feec64a6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feec0159b10 0x7feec0159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:41.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.536+0000 7feec64a6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feec0159b10 0x7feec0159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:42728/0 (socket says 172.21.15.112:42728) 2024-04-25T01:39:41.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.536+0000 7feec64a6640 1 -- 172.21.15.112:0/2688105922 learned_addr learned my addr 172.21.15.112:0/2688105922 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:41.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.537+0000 7feec54a4640 1 --2- 172.21.15.112:0/2688105922 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feec006bb70 0x7feec006ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:41.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.537+0000 7feec64a6640 1 -- 172.21.15.112:0/2688105922 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feec006bb70 msgr2=0x7feec006ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:39:41.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.537+0000 7feec64a6640 1 --2- 172.21.15.112:0/2688105922 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feec006bb70 0x7feec006ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:41.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.537+0000 7feec64a6640 1 -- 172.21.15.112:0/2688105922 >> v1:172.21.15.112:6789/0 conn(0x7feec0157140 legacy=0x7feec00c28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:39:41.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.538+0000 7feec64a6640 1 -- 172.21.15.112:0/2688105922 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec006e850 con 0x7feec0159b10 2024-04-25T01:39:41.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.539+0000 7feec64a6640 1 --2- 172.21.15.112:0/2688105922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feec0159b10 0x7feec0159fa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009870 tx=0x7feeb002a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=512038bf705bbfab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:41.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.544+0000 7feec4ca3640 1 -- 172.21.15.112:0/2688105922 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb0037070 con 0x7feec0159b10 2024-04-25T01:39:41.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.544+0000 7feec4ca3640 1 -- 172.21.15.112:0/2688105922 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb0032440 con 0x7feec0159b10 2024-04-25T01:39:41.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.544+0000 7feec4ca3640 1 -- 172.21.15.112:0/2688105922 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb0032810 con 0x7feec0159b10 2024-04-25T01:39:41.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.545+0000 7feec6ca7640 1 -- 172.21.15.112:0/2688105922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feec0159b10 msgr2=0x7feec0159fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:41.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.545+0000 7feec6ca7640 1 --2- 172.21.15.112:0/2688105922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feec0159b10 0x7feec0159fa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009870 tx=0x7feeb002a1e0 comp rx=0 tx=0).stop 2024-04-25T01:39:41.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.551+0000 7feec6ca7640 1 -- 172.21.15.112:0/2688105922 shutdown_connections 2024-04-25T01:39:41.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.551+0000 7feec6ca7640 1 --2- 172.21.15.112:0/2688105922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feec0159b10 0x7feec0159fa0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:41.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.551+0000 7feec6ca7640 1 --2- 172.21.15.112:0/2688105922 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feec006bb70 0x7feec006ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:41.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.551+0000 7feec6ca7640 1 -- 172.21.15.112:0/2688105922 >> 172.21.15.112:0/2688105922 conn(0x7feec00c49b0 msgr2=0x7feec00c4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:41.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.554+0000 7feec6ca7640 1 -- 172.21.15.112:0/2688105922 shutdown_connections 2024-04-25T01:39:41.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.554+0000 7feec6ca7640 1 -- 172.21.15.112:0/2688105922 wait complete. 2024-04-25T01:39:41.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.556+0000 7feec6ca7640 1 Processor -- start 2024-04-25T01:39:41.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.556+0000 7feec6ca7640 1 -- start start 2024-04-25T01:39:41.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.557+0000 7feec6ca7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feec006bb70 0x7feec015db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:41.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.557+0000 7feec6ca7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feec0157140 0x7feec00a9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:41.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.557+0000 7feec6ca7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feec0159b10 0x7feec01f2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:41.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.557+0000 7feec6ca7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feec006f2b0 con 0x7feec0159b10 2024-04-25T01:39:41.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.557+0000 7feec6ca7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feec00730f0 con 0x7feec0157140 2024-04-25T01:39:41.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.557+0000 7feec6ca7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feec015a4e0 con 0x7feec006bb70 2024-04-25T01:39:41.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.558+0000 7feec5ca5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feec006bb70 0x7feec015db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:41.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.558+0000 7feec64a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feec0159b10 0x7feec01f2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:41.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.558+0000 7feec54a4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feec0157140 0x7feec00a9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:41.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.558+0000 7feec54a4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feec0157140 0x7feec00a9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:42730/0 (socket says 172.21.15.112:42730) 2024-04-25T01:39:41.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.558+0000 7feec54a4640 1 -- 172.21.15.112:0/1666396228 learned_addr learned my addr 172.21.15.112:0/1666396228 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:41.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.559+0000 7feec54a4640 1 -- 172.21.15.112:0/1666396228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feec006bb70 msgr2=0x7feec015db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:41.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.559+0000 7feec54a4640 1 --2- 172.21.15.112:0/1666396228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feec006bb70 0x7feec015db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:41.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.559+0000 7feec54a4640 1 -- 172.21.15.112:0/1666396228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feec0159b10 msgr2=0x7feec01f2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:41.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.559+0000 7feec54a4640 1 --2- 172.21.15.112:0/1666396228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feec0159b10 0x7feec01f2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:41.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.559+0000 7feec54a4640 1 -- 172.21.15.112:0/1666396228 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec01f3330 con 0x7feec0157140 2024-04-25T01:39:41.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.560+0000 7feec64a6640 1 --2- 172.21.15.112:0/1666396228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feec0159b10 0x7feec01f2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:39:41.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.560+0000 7feec54a4640 1 --2- 172.21.15.112:0/1666396228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feec0157140 0x7feec00a9da0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feeb800ce10 tx=0x7feeb800a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:41.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.560+0000 7feeb67fc640 1 -- 172.21.15.112:0/1666396228 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8003b90 con 0x7feec0157140 2024-04-25T01:39:41.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.561+0000 7feeb67fc640 1 -- 172.21.15.112:0/1666396228 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb8003d30 con 0x7feec0157140 2024-04-25T01:39:41.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.561+0000 7feeb67fc640 1 -- 172.21.15.112:0/1666396228 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8004220 con 0x7feec0157140 2024-04-25T01:39:41.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.561+0000 7feec6ca7640 1 -- 172.21.15.112:0/1666396228 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec01e6a40 con 0x7feec0157140 2024-04-25T01:39:41.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.562+0000 7feec6ca7640 1 -- 172.21.15.112:0/1666396228 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec01585c0 con 0x7feec0157140 2024-04-25T01:39:41.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.563+0000 7feec6ca7640 1 -- 172.21.15.112:0/1666396228 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec004f500 con 0x7feec0157140 2024-04-25T01:39:41.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.565+0000 7feeb67fc640 1 -- 172.21.15.112:0/1666396228 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feeb80043c0 con 0x7feec0157140 2024-04-25T01:39:41.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.565+0000 7feeb67fc640 1 --2- 172.21.15.112:0/1666396228 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feea406bf00 0x7feea406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:41.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.565+0000 7feeb67fc640 1 -- 172.21.15.112:0/1666396228 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7feeb808bbe0 con 0x7feec0157140 2024-04-25T01:39:41.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.567+0000 7feec5ca5640 1 --2- 172.21.15.112:0/1666396228 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feea406bf00 0x7feea406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:41.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.568+0000 7feec5ca5640 1 --2- 172.21.15.112:0/1666396228 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feea406bf00 0x7feea406e3c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7feec015ced0 tx=0x7feebc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:41.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:41.573+0000 7feeb67fc640 1 -- 172.21.15.112:0/1666396228 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feeb8059c10 con 0x7feec0157140 2024-04-25T01:39:42.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:42.097+0000 7feec6ca7640 1 -- 172.21.15.112:0/1666396228 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7feec00c5800 con 0x7feec0157140 2024-04-25T01:39:42.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:42.101+0000 7feeb67fc640 1 -- 172.21.15.112:0/1666396228 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7feeb805df10 con 0x7feec0157140 2024-04-25T01:39:42.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:42.106+0000 7feec6ca7640 1 -- 172.21.15.112:0/1666396228 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feea406bf00 msgr2=0x7feea406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:42.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:42.106+0000 7feec6ca7640 1 --2- 172.21.15.112:0/1666396228 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feea406bf00 0x7feea406e3c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7feec015ced0 tx=0x7feebc009040 comp rx=0 tx=0).stop 2024-04-25T01:39:42.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:42.106+0000 7feec6ca7640 1 -- 172.21.15.112:0/1666396228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feec0157140 msgr2=0x7feec00a9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:42.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:42.106+0000 7feec6ca7640 1 --2- 172.21.15.112:0/1666396228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feec0157140 0x7feec00a9da0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feeb800ce10 tx=0x7feeb800a660 comp rx=0 tx=0).stop 2024-04-25T01:39:42.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:42.109+0000 7feec6ca7640 1 -- 172.21.15.112:0/1666396228 shutdown_connections 2024-04-25T01:39:42.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:42.109+0000 7feec6ca7640 1 --2- 172.21.15.112:0/1666396228 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7feea406bf00 0x7feea406e3c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:42.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:42.109+0000 7feec6ca7640 1 --2- 172.21.15.112:0/1666396228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feec0159b10 0x7feec01f2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:42.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:42.109+0000 7feec6ca7640 1 --2- 172.21.15.112:0/1666396228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feec0157140 0x7feec00a9da0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:42.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:42.109+0000 7feec6ca7640 1 --2- 172.21.15.112:0/1666396228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feec006bb70 0x7feec015db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:42.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:42.109+0000 7feec6ca7640 1 -- 172.21.15.112:0/1666396228 >> 172.21.15.112:0/1666396228 conn(0x7feec00c49b0 msgr2=0x7feec0158b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:42.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:42.109+0000 7feec6ca7640 1 -- 172.21.15.112:0/1666396228 shutdown_connections 2024-04-25T01:39:42.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:42.110+0000 7feec6ca7640 1 -- 172.21.15.112:0/1666396228 wait complete. 2024-04-25T01:39:42.123 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T01:39:52.124 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:39:52.124 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:39:52.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.291+0000 7f6ef4a2a640 1 Processor -- start 2024-04-25T01:39:52.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.291+0000 7f6ef4a2a640 1 -- start start 2024-04-25T01:39:52.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.291+0000 7f6ef4a2a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6ef015fcd0 0x7f6ef0072dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:52.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.291+0000 7f6ef4a2a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6ef008b990 0x7f6ef00c2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:52.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.291+0000 7f6ef4a2a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ef006c8a0 con 0x7f6ef00c2ed0 2024-04-25T01:39:52.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.291+0000 7f6ef4a2a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f6ef01630a0 con 0x7f6ef015fcd0 2024-04-25T01:39:52.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.291+0000 7f6ef4a2a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6ef01635d0 con 0x7f6ef008b990 2024-04-25T01:39:52.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.291+0000 7f6eee59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6ef015fcd0 0x7f6ef0072dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:52.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.291+0000 7f6eee59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6ef015fcd0 0x7f6ef0072dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49902/0 (socket says 172.21.15.112:49902) 2024-04-25T01:39:52.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.291+0000 7f6eee59c640 1 -- 172.21.15.112:0/1274601214 learned_addr learned my addr 172.21.15.112:0/1274601214 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:52.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.292+0000 7f6ee5d9b640 1 --2- 172.21.15.112:0/1274601214 >> v2:172.21.15.187:3300/0 conn(0x7f6ef008b990 0x7f6ef00c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:52.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.292+0000 7f6eedd9b640 1 -- 172.21.15.112:0/1274601214 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3394381412 0 0) 0x7f6ef006c8a0 con 0x7f6ef00c2ed0 2024-04-25T01:39:52.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.299+0000 7f6eee59c640 1 -- 172.21.15.112:0/1274601214 >> v2:172.21.15.187:3300/0 conn(0x7f6ef008b990 msgr2=0x7f6ef00c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:39:52.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.299+0000 7f6eee59c640 1 --2- 172.21.15.112:0/1274601214 >> v2:172.21.15.187:3300/0 conn(0x7f6ef008b990 0x7f6ef00c2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:52.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.299+0000 7f6eee59c640 1 -- 172.21.15.112:0/1274601214 >> v1:172.21.15.187:6789/0 conn(0x7f6ef00c2ed0 legacy=0x7f6ef00715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:52.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.299+0000 7f6eee59c640 1 -- 172.21.15.112:0/1274601214 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef01600b0 con 0x7f6ef015fcd0 2024-04-25T01:39:52.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.300+0000 7f6eee59c640 1 --2- 172.21.15.112:0/1274601214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef015fcd0 0x7f6ef0072dc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0163220 tx=0x7f6ee002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeedaece312dc60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:52.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.301+0000 7f6ee5d9b640 1 --2- 172.21.15.112:0/1274601214 >> v2:172.21.15.187:3300/0 conn(0x7f6ef008b990 0x7f6ef00c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:39:52.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.303+0000 7f6eedd9b640 1 -- 172.21.15.112:0/1274601214 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0037070 con 0x7f6ef015fcd0 2024-04-25T01:39:52.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.303+0000 7f6eedd9b640 1 -- 172.21.15.112:0/1274601214 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee0032440 con 0x7f6ef015fcd0 2024-04-25T01:39:52.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.303+0000 7f6eedd9b640 1 -- 172.21.15.112:0/1274601214 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00327f0 con 0x7f6ef015fcd0 2024-04-25T01:39:52.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.312+0000 7f6ef4a2a640 1 -- 172.21.15.112:0/1274601214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef015fcd0 msgr2=0x7f6ef0072dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:52.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.312+0000 7f6ef4a2a640 1 --2- 172.21.15.112:0/1274601214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef015fcd0 0x7f6ef0072dc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0163220 tx=0x7f6ee002a180 comp rx=0 tx=0).stop 2024-04-25T01:39:52.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.314+0000 7f6ef4a2a640 1 -- 172.21.15.112:0/1274601214 shutdown_connections 2024-04-25T01:39:52.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.314+0000 7f6ef4a2a640 1 --2- 172.21.15.112:0/1274601214 >> v2:172.21.15.187:3300/0 conn(0x7f6ef008b990 0x7f6ef00c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:52.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.314+0000 7f6ef4a2a640 1 --2- 172.21.15.112:0/1274601214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef015fcd0 0x7f6ef0072dc0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:52.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.314+0000 7f6ef4a2a640 1 -- 172.21.15.112:0/1274601214 >> 172.21.15.112:0/1274601214 conn(0x7f6ef00c3a80 msgr2=0x7f6ef00c3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:52.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.317+0000 7f6ef4a2a640 1 -- 172.21.15.112:0/1274601214 shutdown_connections 2024-04-25T01:39:52.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.318+0000 7f6ef4a2a640 1 -- 172.21.15.112:0/1274601214 wait complete. 2024-04-25T01:39:52.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.320+0000 7f6ef4a2a640 1 Processor -- start 2024-04-25T01:39:52.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.321+0000 7f6ef4a2a640 1 -- start start 2024-04-25T01:39:52.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.322+0000 7f6ef4a2a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ef008b990 0x7f6ef0165330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:52.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.322+0000 7f6ef4a2a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ef00c2ed0 0x7f6ef0157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:52.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.322+0000 7f6ef4a2a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef015fcd0 0x7f6ef0157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:52.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.322+0000 7f6ef4a2a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef0070de0 con 0x7f6ef015fcd0 2024-04-25T01:39:52.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.323+0000 7f6ef4a2a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef0071150 con 0x7f6ef008b990 2024-04-25T01:39:52.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.323+0000 7f6ef4a2a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef0072a10 con 0x7f6ef00c2ed0 2024-04-25T01:39:52.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.323+0000 7f6eeed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef015fcd0 0x7f6ef0157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:52.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.323+0000 7f6eeed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef015fcd0 0x7f6ef0157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49904/0 (socket says 172.21.15.112:49904) 2024-04-25T01:39:52.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.323+0000 7f6eeed9d640 1 -- 172.21.15.112:0/1615376943 learned_addr learned my addr 172.21.15.112:0/1615376943 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:52.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.324+0000 7f6ee5d9b640 1 --2- 172.21.15.112:0/1615376943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ef00c2ed0 0x7f6ef0157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:52.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.324+0000 7f6eeed9d640 1 -- 172.21.15.112:0/1615376943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ef00c2ed0 msgr2=0x7f6ef0157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:52.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.324+0000 7f6eeed9d640 1 --2- 172.21.15.112:0/1615376943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ef00c2ed0 0x7f6ef0157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:52.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.324+0000 7f6eeed9d640 1 -- 172.21.15.112:0/1615376943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ef008b990 msgr2=0x7f6ef0165330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:52.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.324+0000 7f6eeed9d640 1 --2- 172.21.15.112:0/1615376943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ef008b990 0x7f6ef0165330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:52.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.324+0000 7f6eeed9d640 1 -- 172.21.15.112:0/1615376943 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef0165e50 con 0x7f6ef015fcd0 2024-04-25T01:39:52.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.325+0000 7f6eeed9d640 1 --2- 172.21.15.112:0/1615376943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef015fcd0 0x7f6ef0157b80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800c7b0 tx=0x7f6ee800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:52.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.325+0000 7f6ee77fe640 1 -- 172.21.15.112:0/1615376943 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8003ba0 con 0x7f6ef015fcd0 2024-04-25T01:39:52.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.325+0000 7f6ee77fe640 1 -- 172.21.15.112:0/1615376943 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee8003d40 con 0x7f6ef015fcd0 2024-04-25T01:39:52.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.325+0000 7f6ef4a2a640 1 -- 172.21.15.112:0/1615376943 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef0165b70 con 0x7f6ef015fcd0 2024-04-25T01:39:52.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.327+0000 7f6ef4a2a640 1 -- 172.21.15.112:0/1615376943 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef00a9960 con 0x7f6ef015fcd0 2024-04-25T01:39:52.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.328+0000 7f6ee77fe640 1 -- 172.21.15.112:0/1615376943 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8005270 con 0x7f6ef015fcd0 2024-04-25T01:39:52.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.328+0000 7f6ee77fe640 1 -- 172.21.15.112:0/1615376943 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ee80054d0 con 0x7f6ef015fcd0 2024-04-25T01:39:52.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.329+0000 7f6ee77fe640 1 --2- 172.21.15.112:0/1615376943 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6ed006e0c0 0x7f6ed0070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:52.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.329+0000 7f6ee77fe640 1 -- 172.21.15.112:0/1615376943 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f6ee808f060 con 0x7f6ef015fcd0 2024-04-25T01:39:52.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.329+0000 7f6eee59c640 1 --2- 172.21.15.112:0/1615376943 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6ed006e0c0 0x7f6ed0070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:52.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.330+0000 7f6ef4a2a640 1 -- 172.21.15.112:0/1615376943 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed4005290 con 0x7f6ef015fcd0 2024-04-25T01:39:52.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.344+0000 7f6ee77fe640 1 -- 172.21.15.112:0/1615376943 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6ee8058f60 con 0x7f6ef015fcd0 2024-04-25T01:39:52.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.348+0000 7f6eee59c640 1 --2- 172.21.15.112:0/1615376943 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6ed006e0c0 0x7f6ed0070580 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0034000 tx=0x7f6ee002c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:52.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.932+0000 7f6ef4a2a640 1 -- 172.21.15.112:0/1615376943 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6ed4005850 con 0x7f6ef015fcd0 2024-04-25T01:39:52.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.933+0000 7f6ee77fe640 1 -- 172.21.15.112:0/1615376943 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1570 (secure 0 0 0) 0x7f6ee8058860 con 0x7f6ef015fcd0 2024-04-25T01:39:52.924 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":110,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:39:52.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.941+0000 7f6ef4a2a640 1 -- 172.21.15.112:0/1615376943 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6ed006e0c0 msgr2=0x7f6ed0070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:52.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.941+0000 7f6ef4a2a640 1 --2- 172.21.15.112:0/1615376943 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6ed006e0c0 0x7f6ed0070580 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0034000 tx=0x7f6ee002c000 comp rx=0 tx=0).stop 2024-04-25T01:39:52.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.941+0000 7f6ef4a2a640 1 -- 172.21.15.112:0/1615376943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef015fcd0 msgr2=0x7f6ef0157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:52.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.941+0000 7f6ef4a2a640 1 --2- 172.21.15.112:0/1615376943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef015fcd0 0x7f6ef0157b80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800c7b0 tx=0x7f6ee800cbb0 comp rx=0 tx=0).stop 2024-04-25T01:39:52.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.942+0000 7f6ef4a2a640 1 -- 172.21.15.112:0/1615376943 shutdown_connections 2024-04-25T01:39:52.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.942+0000 7f6ef4a2a640 1 --2- 172.21.15.112:0/1615376943 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6ed006e0c0 0x7f6ed0070580 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:52.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.942+0000 7f6ef4a2a640 1 --2- 172.21.15.112:0/1615376943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef015fcd0 0x7f6ef0157b80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:52.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.942+0000 7f6ef4a2a640 1 --2- 172.21.15.112:0/1615376943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ef00c2ed0 0x7f6ef0157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:52.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.942+0000 7f6ef4a2a640 1 --2- 172.21.15.112:0/1615376943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ef008b990 0x7f6ef0165330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:52.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.942+0000 7f6ef4a2a640 1 -- 172.21.15.112:0/1615376943 >> 172.21.15.112:0/1615376943 conn(0x7f6ef00c3a80 msgr2=0x7f6ef0162690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:52.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.942+0000 7f6ef4a2a640 1 -- 172.21.15.112:0/1615376943 shutdown_connections 2024-04-25T01:39:52.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:52.942+0000 7f6ef4a2a640 1 -- 172.21.15.112:0/1615376943 wait complete. 2024-04-25T01:39:52.948 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:39:52.948 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T01:39:52.948 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:39:52.948 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:39:53.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.154+0000 7f5ca4d9e640 1 Processor -- start 2024-04-25T01:39:53.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.154+0000 7f5ca4d9e640 1 -- start start 2024-04-25T01:39:53.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.155+0000 7f5ca4d9e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5ca015ab30 0x7f5ca00b0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.156+0000 7f5ca4d9e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5ca00c5e10 0x7f5ca00b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.156+0000 7f5ca4d9e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ca00aea10 con 0x7f5ca00be180 2024-04-25T01:39:53.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.156+0000 7f5ca4d9e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f5ca00aecf0 con 0x7f5ca00c5e10 2024-04-25T01:39:53.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.156+0000 7f5ca4d9e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5ca00ae0a0 con 0x7f5ca015ab30 2024-04-25T01:39:53.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.156+0000 7f5c9e59c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5ca015ab30 0x7f5ca00b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.156+0000 7f5c9ed9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5ca00c5e10 0x7f5ca00b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.156+0000 7f5c9dd9b640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f5ca00be180 0x7f5ca00c6d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:36922/0 (socket says 172.21.15.112:36922) 2024-04-25T01:39:53.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.156+0000 7f5c9dd9b640 1 -- 172.21.15.112:0/600549317 learned_addr learned my addr 172.21.15.112:0/600549317 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:39:53.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.156+0000 7f5c9ed9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5ca00c5e10 0x7f5ca00b0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49914/0 (socket says 172.21.15.112:49914) 2024-04-25T01:39:53.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.157+0000 7f5c9e59c640 1 -- 172.21.15.112:0/600549317 >> v2:172.21.15.112:3300/0 conn(0x7f5ca00c5e10 msgr2=0x7f5ca00b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:39:53.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.157+0000 7f5c9e59c640 1 --2- 172.21.15.112:0/600549317 >> v2:172.21.15.112:3300/0 conn(0x7f5ca00c5e10 0x7f5ca00b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.157+0000 7f5c9e59c640 1 -- 172.21.15.112:0/600549317 >> v1:172.21.15.187:6789/0 conn(0x7f5ca00be180 legacy=0x7f5ca00c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.157+0000 7f5c9e59c640 1 -- 172.21.15.112:0/600549317 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca00ae380 con 0x7f5ca015ab30 2024-04-25T01:39:53.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.159+0000 7f5c9e59c640 1 --2- 172.21.15.112:0/600549317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ca015ab30 0x7f5ca00b0590 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00753b0 tx=0x7f5c9002a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16c0b7b9b1b9a2c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:53.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.162+0000 7f5c9d59a640 1 -- 172.21.15.112:0/600549317 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90007740 con 0x7f5ca015ab30 2024-04-25T01:39:53.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.163+0000 7f5c9d59a640 1 -- 172.21.15.112:0/600549317 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c900078e0 con 0x7f5ca015ab30 2024-04-25T01:39:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.163+0000 7f5c9d59a640 1 -- 172.21.15.112:0/600549317 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90007cf0 con 0x7f5ca015ab30 2024-04-25T01:39:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.163+0000 7f5ca4d9e640 1 -- 172.21.15.112:0/600549317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ca015ab30 msgr2=0x7f5ca00b0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.163+0000 7f5ca4d9e640 1 --2- 172.21.15.112:0/600549317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ca015ab30 0x7f5ca00b0590 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00753b0 tx=0x7f5c9002a610 comp rx=0 tx=0).stop 2024-04-25T01:39:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.164+0000 7f5ca4d9e640 1 -- 172.21.15.112:0/600549317 shutdown_connections 2024-04-25T01:39:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.164+0000 7f5ca4d9e640 1 --2- 172.21.15.112:0/600549317 >> v2:172.21.15.112:3300/0 conn(0x7f5ca00c5e10 0x7f5ca00b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.164+0000 7f5ca4d9e640 1 --2- 172.21.15.112:0/600549317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ca015ab30 0x7f5ca00b0590 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.164+0000 7f5ca4d9e640 1 -- 172.21.15.112:0/600549317 >> 172.21.15.112:0/600549317 conn(0x7f5ca00cb170 msgr2=0x7f5ca00cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:53.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.168+0000 7f5ca4d9e640 1 -- 172.21.15.112:0/600549317 shutdown_connections 2024-04-25T01:39:53.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.168+0000 7f5ca4d9e640 1 -- 172.21.15.112:0/600549317 wait complete. 2024-04-25T01:39:53.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.170+0000 7f5ca4d9e640 1 Processor -- start 2024-04-25T01:39:53.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.172+0000 7f5ca4d9e640 1 -- start start 2024-04-25T01:39:53.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.173+0000 7f5ca4d9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca00be180 0x7f5ca00c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.173+0000 7f5ca4d9e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ca00c5e10 0x7f5ca00c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.173+0000 7f5ca4d9e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ca015ab30 0x7f5ca0154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.173+0000 7f5ca4d9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca00c7480 con 0x7f5ca00be180 2024-04-25T01:39:53.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.173+0000 7f5ca4d9e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca0072580 con 0x7f5ca00c5e10 2024-04-25T01:39:53.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.173+0000 7f5ca4d9e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca00c7600 con 0x7f5ca015ab30 2024-04-25T01:39:53.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.174+0000 7f5c9e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca00be180 0x7f5ca00c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.174+0000 7f5c9e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca00be180 0x7f5ca00c24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49930/0 (socket says 172.21.15.112:49930) 2024-04-25T01:39:53.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.174+0000 7f5c9dd9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ca00c5e10 0x7f5ca00c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.174+0000 7f5c9e59c640 1 -- 172.21.15.112:0/3408738678 learned_addr learned my addr 172.21.15.112:0/3408738678 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:53.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.175+0000 7f5c9e59c640 1 -- 172.21.15.112:0/3408738678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ca015ab30 msgr2=0x7f5ca0154640 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:39:53.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.175+0000 7f5c9e59c640 1 --2- 172.21.15.112:0/3408738678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ca015ab30 0x7f5ca0154640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.175+0000 7f5c9e59c640 1 -- 172.21.15.112:0/3408738678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ca00c5e10 msgr2=0x7f5ca00c29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.175+0000 7f5c9e59c640 1 --2- 172.21.15.112:0/3408738678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ca00c5e10 0x7f5ca00c29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.175+0000 7f5c9e59c640 1 -- 172.21.15.112:0/3408738678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca00c3eb0 con 0x7f5ca00be180 2024-04-25T01:39:53.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.176+0000 7f5c9e59c640 1 --2- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca00be180 0x7f5ca00c24b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5c90000f80 tx=0x7f5c90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:53.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.178+0000 7f5c8effd640 1 -- 172.21.15.112:0/3408738678 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9003f070 con 0x7f5ca00be180 2024-04-25T01:39:53.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.178+0000 7f5c8effd640 1 -- 172.21.15.112:0/3408738678 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c90031bc0 con 0x7f5ca00be180 2024-04-25T01:39:53.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.178+0000 7f5c8effd640 1 -- 172.21.15.112:0/3408738678 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90044460 con 0x7f5ca00be180 2024-04-25T01:39:53.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.182+0000 7f5ca4d9e640 1 -- 172.21.15.112:0/3408738678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca015de30 con 0x7f5ca00be180 2024-04-25T01:39:53.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.182+0000 7f5ca4d9e640 1 -- 172.21.15.112:0/3408738678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca015e0d0 con 0x7f5ca00be180 2024-04-25T01:39:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.184+0000 7f5ca4d9e640 1 --2- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca0154b80 0x7f5ca00c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.184+0000 7f5ca4d9e640 1 -- 172.21.15.112:0/3408738678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca00af910 con 0x7f5ca0154b80 2024-04-25T01:39:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.186+0000 7f5c9dd9b640 1 --2- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca0154b80 0x7f5ca00c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.186+0000 7f5c9dd9b640 1 -- 172.21.15.112:0/3408738678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ca0075980 con 0x7f5ca0154b80 2024-04-25T01:39:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.187+0000 7f5c9dd9b640 1 --2- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca0154b80 0x7f5ca00c4460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5ca015a030 tx=0x7f5c9400fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.187+0000 7f5c8effd640 1 -- 172.21.15.112:0/3408738678 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c940058b0 con 0x7f5ca0154b80 2024-04-25T01:39:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.187+0000 7f5c8effd640 1 -- 172.21.15.112:0/3408738678 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5c94018070 con 0x7f5ca0154b80 2024-04-25T01:39:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.187+0000 7f5c8effd640 1 -- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca0154b80 msgr2=0x7f5ca00c4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.187+0000 7f5c8effd640 1 --2- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca0154b80 0x7f5ca00c4460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5ca015a030 tx=0x7f5c9400fc00 comp rx=0 tx=0).stop 2024-04-25T01:39:53.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.187+0000 7f5c8effd640 1 -- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca0154b80 msgr2=0x7f5ca00c4460 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:39:53.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.187+0000 7f5c8effd640 1 --2- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca0154b80 0x7f5ca00c4460 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.188+0000 7f5c8effd640 1 -- 172.21.15.112:0/3408738678 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c9003b410 con 0x7f5ca00be180 2024-04-25T01:39:53.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.196+0000 7f5c8effd640 1 --2- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c8007c900 0x7f5c8007edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.196+0000 7f5c8effd640 1 -- 172.21.15.112:0/3408738678 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f5c900ac860 con 0x7f5ca00be180 2024-04-25T01:39:53.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.198+0000 7f5c9dd9b640 1 --2- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c8007c900 0x7f5c8007edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.201+0000 7f5c9dd9b640 1 --2- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c8007c900 0x7f5c8007edc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5c94017000 tx=0x7f5c94007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.212+0000 7f5ca4d9e640 1 --2- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca00c49a0 0x7f5ca00c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.212+0000 7f5c9ed9d640 1 --2- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca00c49a0 0x7f5ca00c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.212+0000 7f5ca4d9e640 1 -- 172.21.15.112:0/3408738678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca00679a0 con 0x7f5ca00c49a0 2024-04-25T01:39:53.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.213+0000 7f5c9ed9d640 1 -- 172.21.15.112:0/3408738678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5ca00682a0 con 0x7f5ca00c49a0 2024-04-25T01:39:53.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.213+0000 7f5c9ed9d640 1 --2- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca00c49a0 0x7f5ca00c4d80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800eae0 tx=0x7f5c98009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:53.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.214+0000 7f5c8effd640 1 -- 172.21.15.112:0/3408738678 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98017070 con 0x7f5ca00c49a0 2024-04-25T01:39:53.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.215+0000 7f5c8effd640 1 -- 172.21.15.112:0/3408738678 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4562 (secure 0 0 0) 0x7f5c98004020 con 0x7f5ca00c49a0 2024-04-25T01:39:53.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.215+0000 7f5c8effd640 1 -- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca00c49a0 msgr2=0x7f5ca00c4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.215+0000 7f5c8effd640 1 --2- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca00c49a0 0x7f5ca00c4d80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800eae0 tx=0x7f5c98009eb0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.215+0000 7f5c8effd640 1 -- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca00c49a0 msgr2=0x7f5ca00c4d80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:39:53.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.215+0000 7f5c8effd640 1 --2- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca00c49a0 0x7f5ca00c4d80 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800eae0 tx=0x7f5c98009eb0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.199 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:39:53.199 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:39:53.199 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:39:53.199 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:39:53.199 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 110, 2024-04-25T01:39:53.199 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 41260, 2024-04-25T01:39:53.199 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:39:53.199 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:39:53.199 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:39:53.200 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:39:53.200 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:53.200 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 15, 2024-04-25T01:39:53.200 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:39:53.200 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:39:53.200 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:39:53.200 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:39:53.200 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:39:53.200 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:39:53.201 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:39:53.201 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:39:53.201 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:39:53.201 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:39:53.201 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:39:53.201 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:39:53.201 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:39:53.201 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:39:53.201 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:53.201 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:39:53.201 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:39:53.201 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:39:53.201 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:39:53.201 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:39:53.201 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:39:53.202 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:39:53.202 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:39:53.202 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:39:53.202 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:39:53.202 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:39:53.202 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:39:53.202 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:39:53.202 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:53.202 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.202 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:39:53.202 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:39:53.202 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:39:53.202 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:39:53.202 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:39:53.203 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:39:53.203 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:39:53.203 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:39:53.203 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:39:53.203 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:39:53.203 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:39:53.203 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:39:53.203 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:39:53.203 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:39:53.203 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:39:53.203 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:39:53.203 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:39:53.203 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:39:53.203 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:39:53.204 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:39:53.204 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:39:53.204 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:39:53.204 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:39:53.204 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:39:53.204 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:39:53.204 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:39:53.204 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:39:53.204 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:39:53.204 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:53.205 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:39:53.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.205 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:39:53.205 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.205 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:39:53.205 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:39:53.205 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:39:53.205 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:39:53.205 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.205 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:39:53.205 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:39:53.205 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:53.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.205 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.206 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:39:53.206 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:39:53.206 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:53.206 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.206 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:53.206 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.206 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:39:53.206 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:39:53.206 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:39:53.206 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:39:53.206 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:39:53.206 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.206 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.206 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:39:53.207 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:39:53.207 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:39:53.207 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:39:53.207 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.207 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:39:53.207 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:39:53.207 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:53.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.207 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.207 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:39:53.207 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:39:53.207 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:53.207 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.207 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:53.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.208 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:39:53.208 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:39:53.208 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:39:53.208 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:39:53.208 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:39:53.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.208 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.208 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:39:53.208 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:39:53.208 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:39:53.208 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:39:53.208 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.208 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:39:53.208 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:39:53.209 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:53.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.209 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.209 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:39:53.209 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:39:53.209 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:53.209 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.209 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:53.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.209 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:39:53.209 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:39:53.209 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:39:53.209 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:39:53.209 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:39:53.209 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.209 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:53.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.210 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:39:53.210 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:39:53.210 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.210 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:39:53.210 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:39:53.210 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:39:53.210 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.210 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:53.210 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:39:53.210 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.210 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:39:53.210 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:39:53.210 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:39:53.211 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.211 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:53.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.211 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:39:53.211 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:39:53.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.217+0000 7f5c8cff9640 1 -- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c8007c900 msgr2=0x7f5c8007edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.217+0000 7f5c8cff9640 1 --2- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5c8007c900 0x7f5c8007edc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5c94017000 tx=0x7f5c94007000 comp rx=0 tx=0).stop 2024-04-25T01:39:53.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.217+0000 7f5c9ed9d640 1 -- 172.21.15.112:0/3408738678 reap_dead start 2024-04-25T01:39:53.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.217+0000 7f5c8cff9640 1 -- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca00be180 msgr2=0x7f5ca00c24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.217+0000 7f5c8cff9640 1 --2- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca00be180 0x7f5ca00c24b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5c90000f80 tx=0x7f5c90004a40 comp rx=0 tx=0).stop 2024-04-25T01:39:53.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.221+0000 7f5c8cff9640 1 -- 172.21.15.112:0/3408738678 shutdown_connections 2024-04-25T01:39:53.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.221+0000 7f5c8cff9640 1 --2- 172.21.15.112:0/3408738678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca00be180 0x7f5ca00c24b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.221+0000 7f5c8cff9640 1 -- 172.21.15.112:0/3408738678 >> 172.21.15.112:0/3408738678 conn(0x7f5ca00cb170 msgr2=0x7f5ca00ab6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:53.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.221+0000 7f5c8cff9640 1 -- 172.21.15.112:0/3408738678 shutdown_connections 2024-04-25T01:39:53.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.221+0000 7f5c8cff9640 1 -- 172.21.15.112:0/3408738678 wait complete. 2024-04-25T01:39:53.227 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:39:53.227 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:39:53.227 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:39:53.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.449+0000 7f23156b3640 1 Processor -- start 2024-04-25T01:39:53.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.449+0000 7f23156b3640 1 -- start start 2024-04-25T01:39:53.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.450+0000 7f23156b3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23100c2420 0x7f23100c3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.451+0000 7f23156b3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f23100c5660 0x7f2310159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.451+0000 7f23156b3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f231006f110 con 0x7f2310157460 2024-04-25T01:39:53.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.451+0000 7f230effd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2310157460 0x7f231006c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50776/0 (socket says 172.21.15.112:50776) 2024-04-25T01:39:53.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.451+0000 7f230effd640 1 -- 172.21.15.112:0/3077307170 learned_addr learned my addr 172.21.15.112:0/3077307170 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:39:53.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.451+0000 7f23156b3640 1 -- 172.21.15.112:0/3077307170 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f231006fc40 con 0x7f23100c5660 2024-04-25T01:39:53.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.451+0000 7f23156b3640 1 -- 172.21.15.112:0/3077307170 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f231006f8f0 con 0x7f23100c2420 2024-04-25T01:39:53.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.451+0000 7f230f7fe640 1 --2- 172.21.15.112:0/3077307170 >> v2:172.21.15.187:3300/0 conn(0x7f23100c5660 0x7f2310159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.451+0000 7f230dffb640 1 -- 172.21.15.112:0/3077307170 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3872640296 0 0) 0x7f231006f110 con 0x7f2310157460 2024-04-25T01:39:53.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.451+0000 7f230dffb640 1 -- 172.21.15.112:0/3077307170 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22f8003520 con 0x7f2310157460 2024-04-25T01:39:53.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.452+0000 7f230e7fc640 1 --2- 172.21.15.112:0/3077307170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23100c2420 0x7f23100c3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.452+0000 7f230f7fe640 1 -- 172.21.15.112:0/3077307170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23100c2420 msgr2=0x7f23100c3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:39:53.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.452+0000 7f230f7fe640 1 --2- 172.21.15.112:0/3077307170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23100c2420 0x7f23100c3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.452+0000 7f230f7fe640 1 -- 172.21.15.112:0/3077307170 >> v1:172.21.15.112:6789/0 conn(0x7f2310157460 legacy=0x7f231006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.452+0000 7f230f7fe640 1 -- 172.21.15.112:0/3077307170 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231006f3d0 con 0x7f23100c5660 2024-04-25T01:39:53.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.459+0000 7f230f7fe640 1 --2- 172.21.15.112:0/3077307170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23100c5660 0x7f2310159cc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f231006fdc0 tx=0x7f230402a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67cf94013abefe04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:53.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.461+0000 7f230dffb640 1 -- 172.21.15.112:0/3077307170 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304037070 con 0x7f23100c5660 2024-04-25T01:39:53.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.461+0000 7f230dffb640 1 -- 172.21.15.112:0/3077307170 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f230402add0 con 0x7f23100c5660 2024-04-25T01:39:53.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.461+0000 7f230dffb640 1 -- 172.21.15.112:0/3077307170 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304032660 con 0x7f23100c5660 2024-04-25T01:39:53.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.462+0000 7f23156b3640 1 -- 172.21.15.112:0/3077307170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23100c5660 msgr2=0x7f2310159cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.462+0000 7f23156b3640 1 --2- 172.21.15.112:0/3077307170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23100c5660 0x7f2310159cc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f231006fdc0 tx=0x7f230402a080 comp rx=0 tx=0).stop 2024-04-25T01:39:53.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.462+0000 7f23156b3640 1 -- 172.21.15.112:0/3077307170 shutdown_connections 2024-04-25T01:39:53.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.462+0000 7f23156b3640 1 --2- 172.21.15.112:0/3077307170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23100c5660 0x7f2310159cc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.462+0000 7f23156b3640 1 --2- 172.21.15.112:0/3077307170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23100c2420 0x7f23100c3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.462+0000 7f23156b3640 1 -- 172.21.15.112:0/3077307170 >> 172.21.15.112:0/3077307170 conn(0x7f23100c4260 msgr2=0x7f23100c4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:53.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.468+0000 7f23156b3640 1 -- 172.21.15.112:0/3077307170 shutdown_connections 2024-04-25T01:39:53.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.470+0000 7f23156b3640 1 -- 172.21.15.112:0/3077307170 wait complete. 2024-04-25T01:39:53.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.473+0000 7f23156b3640 1 Processor -- start 2024-04-25T01:39:53.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.476+0000 7f23156b3640 1 -- start start 2024-04-25T01:39:53.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.477+0000 7f23156b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23100c2420 0x7f23101f0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.477+0000 7f23156b3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23100c5660 0x7f231015da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.477+0000 7f23156b3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2310157460 0x7f2310158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.477+0000 7f23156b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f231015a380 con 0x7f23100c2420 2024-04-25T01:39:53.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.477+0000 7f23156b3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f231015a200 con 0x7f2310157460 2024-04-25T01:39:53.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.477+0000 7f23156b3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f231015a500 con 0x7f23100c5660 2024-04-25T01:39:53.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.478+0000 7f230effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23100c2420 0x7f23101f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.478+0000 7f230effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23100c2420 0x7f23101f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49956/0 (socket says 172.21.15.112:49956) 2024-04-25T01:39:53.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.478+0000 7f230effd640 1 -- 172.21.15.112:0/4080626800 learned_addr learned my addr 172.21.15.112:0/4080626800 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:53.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.479+0000 7f230f7fe640 1 --2- 172.21.15.112:0/4080626800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2310157460 0x7f2310158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.479+0000 7f230e7fc640 1 --2- 172.21.15.112:0/4080626800 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23100c5660 0x7f231015da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.479+0000 7f230effd640 1 -- 172.21.15.112:0/4080626800 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23100c5660 msgr2=0x7f231015da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.479+0000 7f230effd640 1 --2- 172.21.15.112:0/4080626800 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23100c5660 0x7f231015da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.479+0000 7f230effd640 1 -- 172.21.15.112:0/4080626800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2310157460 msgr2=0x7f2310158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.479+0000 7f230effd640 1 --2- 172.21.15.112:0/4080626800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2310157460 0x7f2310158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.479+0000 7f230effd640 1 -- 172.21.15.112:0/4080626800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23101583d0 con 0x7f23100c2420 2024-04-25T01:39:53.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.479+0000 7f230effd640 1 --2- 172.21.15.112:0/4080626800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23100c2420 0x7f23101f0430 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f230000c3f0 tx=0x7f230000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:53.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.480+0000 7f22f77fe640 1 -- 172.21.15.112:0/4080626800 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23000074e0 con 0x7f23100c2420 2024-04-25T01:39:53.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.480+0000 7f22f77fe640 1 -- 172.21.15.112:0/4080626800 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23000047e0 con 0x7f23100c2420 2024-04-25T01:39:53.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.480+0000 7f22f77fe640 1 -- 172.21.15.112:0/4080626800 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23000074e0 con 0x7f23100c2420 2024-04-25T01:39:53.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.483+0000 7f23156b3640 1 -- 172.21.15.112:0/4080626800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231015bb60 con 0x7f23100c2420 2024-04-25T01:39:53.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.483+0000 7f23156b3640 1 -- 172.21.15.112:0/4080626800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2310158670 con 0x7f23100c2420 2024-04-25T01:39:53.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.484+0000 7f23156b3640 1 --2- 172.21.15.112:0/4080626800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23101590d0 0x7f23101594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.484+0000 7f23156b3640 1 -- 172.21.15.112:0/4080626800 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2310075980 con 0x7f23101590d0 2024-04-25T01:39:53.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.487+0000 7f230e7fc640 1 --2- 172.21.15.112:0/4080626800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23101590d0 0x7f23101594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.488+0000 7f230e7fc640 1 -- 172.21.15.112:0/4080626800 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2310156af0 con 0x7f23101590d0 2024-04-25T01:39:53.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.488+0000 7f230e7fc640 1 --2- 172.21.15.112:0/4080626800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23101590d0 0x7f23101594e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f23100736f0 tx=0x7f230800c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:53.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.488+0000 7f22f77fe640 1 -- 172.21.15.112:0/4080626800 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308003da0 con 0x7f23101590d0 2024-04-25T01:39:53.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.489+0000 7f22f77fe640 1 -- 172.21.15.112:0/4080626800 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2308013070 con 0x7f23101590d0 2024-04-25T01:39:53.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.489+0000 7f22f77fe640 1 -- 172.21.15.112:0/4080626800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23101590d0 msgr2=0x7f23101594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.489+0000 7f22f77fe640 1 --2- 172.21.15.112:0/4080626800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23101590d0 0x7f23101594e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f23100736f0 tx=0x7f230800c6e0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.489+0000 7f22f77fe640 1 -- 172.21.15.112:0/4080626800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23101590d0 msgr2=0x7f23101594e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:39:53.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.489+0000 7f22f77fe640 1 --2- 172.21.15.112:0/4080626800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23101590d0 0x7f23101594e0 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f23100736f0 tx=0x7f230800c6e0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.517+0000 7f22f77fe640 1 -- 172.21.15.112:0/4080626800 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2300011070 con 0x7f23100c2420 2024-04-25T01:39:53.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.518+0000 7f22f77fe640 1 --2- 172.21.15.112:0/4080626800 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f22fc06bdd0 0x7f22fc06e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.518+0000 7f22f77fe640 1 -- 172.21.15.112:0/4080626800 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f23000072e0 con 0x7f23100c2420 2024-04-25T01:39:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.518+0000 7f230e7fc640 1 --2- 172.21.15.112:0/4080626800 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f22fc06bdd0 0x7f22fc06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.519+0000 7f230e7fc640 1 --2- 172.21.15.112:0/4080626800 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f22fc06bdd0 0x7f22fc06e290 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f230800c720 tx=0x7f2308014000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:53.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.527+0000 7f23156b3640 1 --2- 172.21.15.112:0/4080626800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f231014c740 0x7f231014cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.527+0000 7f23156b3640 1 -- 172.21.15.112:0/4080626800 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f23100689b0 con 0x7f231014c740 2024-04-25T01:39:53.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.531+0000 7f230f7fe640 1 --2- 172.21.15.112:0/4080626800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f231014c740 0x7f231014cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.537+0000 7f230f7fe640 1 -- 172.21.15.112:0/4080626800 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f23100c1b00 con 0x7f231014c740 2024-04-25T01:39:53.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.539+0000 7f230f7fe640 1 --2- 172.21.15.112:0/4080626800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f231014c740 0x7f231014cb20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2310071100 tx=0x7f23040046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:53.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.539+0000 7f22f77fe640 1 -- 172.21.15.112:0/4080626800 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304037040 con 0x7f231014c740 2024-04-25T01:39:53.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.541+0000 7f22f77fe640 1 -- 172.21.15.112:0/4080626800 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4730 (secure 0 0 0) 0x7f230403d440 con 0x7f231014c740 2024-04-25T01:39:53.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.541+0000 7f22f77fe640 1 -- 172.21.15.112:0/4080626800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f231014c740 msgr2=0x7f231014cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.541+0000 7f22f77fe640 1 --2- 172.21.15.112:0/4080626800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f231014c740 0x7f231014cb20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2310071100 tx=0x7f23040046c0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.541+0000 7f22f77fe640 1 -- 172.21.15.112:0/4080626800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f231014c740 msgr2=0x7f231014cb20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:39:53.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.541+0000 7f22f77fe640 1 --2- 172.21.15.112:0/4080626800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f231014c740 0x7f231014cb20 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2310071100 tx=0x7f23040046c0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.528 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:39:53.528 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:39:53.528 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:39:53.528 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:39:53.528 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 110, 2024-04-25T01:39:53.528 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 98304, 2024-04-25T01:39:53.528 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:39:53.528 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:39:53.528 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:39:53.528 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:39:53.528 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:53.528 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 16, 2024-04-25T01:39:53.528 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:39:53.528 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:39:53.529 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:39:53.529 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:39:53.529 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:39:53.529 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:39:53.529 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:39:53.529 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:39:53.529 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:39:53.529 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:39:53.529 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:39:53.529 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:39:53.529 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:39:53.529 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:39:53.529 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:53.529 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:39:53.530 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:39:53.530 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:39:53.530 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:39:53.530 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:39:53.530 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:39:53.530 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:39:53.530 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:39:53.530 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:39:53.530 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:39:53.530 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:39:53.530 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:39:53.530 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:39:53.530 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:53.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.531 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:39:53.531 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:39:53.531 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:39:53.531 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:39:53.531 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:39:53.531 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:39:53.531 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:39:53.531 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:39:53.531 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:39:53.531 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:39:53.531 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:39:53.531 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:39:53.531 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:39:53.531 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:39:53.531 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:39:53.532 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:39:53.532 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:39:53.532 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:39:53.532 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:39:53.532 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:39:53.532 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:39:53.532 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:39:53.532 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:39:53.532 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:39:53.532 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:39:53.532 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:39:53.532 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:39:53.532 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:39:53.532 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:53.532 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:39:53.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.533 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:39:53.533 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.533 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:39:53.533 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:39:53.533 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:39:53.533 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:39:53.533 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.533 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:39:53.533 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:39:53.533 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:53.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.533 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.533 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:39:53.534 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:39:53.534 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:53.534 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.534 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:53.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.534 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:39:53.534 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:39:53.534 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:39:53.534 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:39:53.534 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:39:53.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.534 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.534 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:39:53.534 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:39:53.534 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:39:53.535 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:39:53.535 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.535 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:39:53.535 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:39:53.535 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:53.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.535 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.535 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:39:53.535 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:39:53.535 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:53.535 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.535 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:53.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.536 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:39:53.536 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:39:53.536 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:39:53.536 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:39:53.536 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:39:53.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.536 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.536 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:39:53.536 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:39:53.536 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:39:53.536 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:39:53.536 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.536 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:39:53.536 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:39:53.537 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:53.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.537 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.537 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:39:53.537 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:39:53.537 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:53.537 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.537 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:53.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.537 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:39:53.537 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:39:53.537 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:39:53.537 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:39:53.537 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:39:53.538 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.538 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:53.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.538 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:39:53.538 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:39:53.538 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.538 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:39:53.538 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:39:53.538 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:39:53.538 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.538 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:53.538 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:39:53.538 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.538 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:39:53.539 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:39:53.539 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:39:53.539 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.539 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:53.539 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:39:53.539 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.539 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:39:53.539 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:39:53.539 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:39:53.539 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.539 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:53.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.539 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:39:53.539 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:39:53.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.545+0000 7f23156b3640 1 -- 172.21.15.112:0/4080626800 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f22fc06bdd0 msgr2=0x7f22fc06e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.545+0000 7f23156b3640 1 --2- 172.21.15.112:0/4080626800 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f22fc06bdd0 0x7f22fc06e290 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f230800c720 tx=0x7f2308014000 comp rx=0 tx=0).stop 2024-04-25T01:39:53.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.545+0000 7f23156b3640 1 -- 172.21.15.112:0/4080626800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23100c2420 msgr2=0x7f23101f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.545+0000 7f23156b3640 1 --2- 172.21.15.112:0/4080626800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23100c2420 0x7f23101f0430 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f230000c3f0 tx=0x7f230000c8b0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.547+0000 7f230f7fe640 1 -- 172.21.15.112:0/4080626800 reap_dead start 2024-04-25T01:39:53.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.547+0000 7f23156b3640 1 -- 172.21.15.112:0/4080626800 shutdown_connections 2024-04-25T01:39:53.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.547+0000 7f23156b3640 1 -- 172.21.15.112:0/4080626800 >> 172.21.15.112:0/4080626800 conn(0x7f23100c4260 msgr2=0x7f2310153150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:53.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.548+0000 7f23156b3640 1 -- 172.21.15.112:0/4080626800 shutdown_connections 2024-04-25T01:39:53.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.548+0000 7f23156b3640 1 -- 172.21.15.112:0/4080626800 wait complete. 2024-04-25T01:39:53.556 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:39:53.556 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:39:53.557 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:39:53.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.779+0000 7f9cc697e640 1 Processor -- start 2024-04-25T01:39:53.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.783+0000 7f9cc697e640 1 -- start start 2024-04-25T01:39:53.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.784+0000 7f9cc697e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cc0070fd0 0x7f9cc00ac000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.784+0000 7f9cc697e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9cc0072ce0 0x7f9cc00c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.784+0000 7f9cc697e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9cc01571b0 0x7f9cc0157640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.785+0000 7f9cbf7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9cc0072ce0 0x7f9cc00c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.785+0000 7f9cbf7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9cc0072ce0 0x7f9cc00c1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:58772/0 (socket says 172.21.15.112:58772) 2024-04-25T01:39:53.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.785+0000 7f9cbf7fe640 1 -- 172.21.15.112:0/442078013 learned_addr learned my addr 172.21.15.112:0/442078013 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:53.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.785+0000 7f9cc4f1b640 1 --2- 172.21.15.112:0/442078013 >> v2:172.21.15.112:3300/0 conn(0x7f9cc01571b0 0x7f9cc0157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.785+0000 7f9cbffff640 1 --2- 172.21.15.112:0/442078013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cc0070fd0 0x7f9cc00ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.784+0000 7f9cc697e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f9cc00ab2d0 con 0x7f9cc01571b0 2024-04-25T01:39:53.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.785+0000 7f9cc697e640 1 -- 172.21.15.112:0/442078013 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9cc00abb70 con 0x7f9cc0072ce0 2024-04-25T01:39:53.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.785+0000 7f9cc697e640 1 -- 172.21.15.112:0/442078013 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc015ac20 con 0x7f9cc0070fd0 2024-04-25T01:39:53.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.786+0000 7f9cbf7fe640 1 -- 172.21.15.112:0/442078013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cc0070fd0 msgr2=0x7f9cc00ac000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:39:53.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.786+0000 7f9cbf7fe640 1 --2- 172.21.15.112:0/442078013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cc0070fd0 0x7f9cc00ac000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.786+0000 7f9cbf7fe640 1 -- 172.21.15.112:0/442078013 >> v2:172.21.15.112:3300/0 conn(0x7f9cc01571b0 msgr2=0x7f9cc0157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:39:53.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.786+0000 7f9cbf7fe640 1 --2- 172.21.15.112:0/442078013 >> v2:172.21.15.112:3300/0 conn(0x7f9cc01571b0 0x7f9cc0157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.786+0000 7f9cbf7fe640 1 -- 172.21.15.112:0/442078013 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc00ab520 con 0x7f9cc0072ce0 2024-04-25T01:39:53.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.787+0000 7f9cc4f1b640 1 --2- 172.21.15.112:0/442078013 >> v2:172.21.15.112:3300/0 conn(0x7f9cc01571b0 0x7f9cc0157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:39:53.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.787+0000 7f9cbf7fe640 1 --2- 172.21.15.112:0/442078013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cc0072ce0 0x7f9cc00c1a80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00735a0 tx=0x7f9cb402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=184f37d510787d28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:53.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.788+0000 7f9cbeffd640 1 -- 172.21.15.112:0/442078013 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4037070 con 0x7f9cc0072ce0 2024-04-25T01:39:53.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.788+0000 7f9cbeffd640 1 -- 172.21.15.112:0/442078013 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb4032440 con 0x7f9cc0072ce0 2024-04-25T01:39:53.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.788+0000 7f9cbeffd640 1 -- 172.21.15.112:0/442078013 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb40327f0 con 0x7f9cc0072ce0 2024-04-25T01:39:53.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.789+0000 7f9cc697e640 1 -- 172.21.15.112:0/442078013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cc0072ce0 msgr2=0x7f9cc00c1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.789+0000 7f9cc697e640 1 --2- 172.21.15.112:0/442078013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cc0072ce0 0x7f9cc00c1a80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00735a0 tx=0x7f9cb402a210 comp rx=0 tx=0).stop 2024-04-25T01:39:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.790+0000 7f9cc697e640 1 -- 172.21.15.112:0/442078013 shutdown_connections 2024-04-25T01:39:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.790+0000 7f9cc697e640 1 --2- 172.21.15.112:0/442078013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cc0072ce0 0x7f9cc00c1a80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.790+0000 7f9cc697e640 1 --2- 172.21.15.112:0/442078013 >> v2:172.21.15.112:3300/0 conn(0x7f9cc01571b0 0x7f9cc0157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.790+0000 7f9cc697e640 1 --2- 172.21.15.112:0/442078013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cc0070fd0 0x7f9cc00ac000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.790+0000 7f9cc697e640 1 -- 172.21.15.112:0/442078013 >> 172.21.15.112:0/442078013 conn(0x7f9cc00c7080 msgr2=0x7f9cc00c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.790+0000 7f9cc697e640 1 -- 172.21.15.112:0/442078013 shutdown_connections 2024-04-25T01:39:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.790+0000 7f9cc697e640 1 -- 172.21.15.112:0/442078013 wait complete. 2024-04-25T01:39:53.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.792+0000 7f9cc697e640 1 Processor -- start 2024-04-25T01:39:53.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.793+0000 7f9cc697e640 1 -- start start 2024-04-25T01:39:53.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.794+0000 7f9cc697e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cc0070fd0 0x7f9cc015b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.795+0000 7f9cc697e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cc0072ce0 0x7f9cc01512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.795+0000 7f9cc697e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cc01571b0 0x7f9cc0146fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.795+0000 7f9cc697e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc015a840 con 0x7f9cc0072ce0 2024-04-25T01:39:53.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.795+0000 7f9cc697e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc0073140 con 0x7f9cc0070fd0 2024-04-25T01:39:53.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.795+0000 7f9cc697e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc015a9c0 con 0x7f9cc01571b0 2024-04-25T01:39:53.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.796+0000 7f9cbf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cc0072ce0 0x7f9cc01512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.796+0000 7f9cbf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cc0072ce0 0x7f9cc01512f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49986/0 (socket says 172.21.15.112:49986) 2024-04-25T01:39:53.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.796+0000 7f9cbf7fe640 1 -- 172.21.15.112:0/2626854676 learned_addr learned my addr 172.21.15.112:0/2626854676 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:53.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.796+0000 7f9cbf7fe640 1 -- 172.21.15.112:0/2626854676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cc01571b0 msgr2=0x7f9cc0146fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:39:53.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.796+0000 7f9cbf7fe640 1 --2- 172.21.15.112:0/2626854676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9cc01571b0 0x7f9cc0146fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.796+0000 7f9cbf7fe640 1 -- 172.21.15.112:0/2626854676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cc0070fd0 msgr2=0x7f9cc015b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:39:53.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.797+0000 7f9cbf7fe640 1 --2- 172.21.15.112:0/2626854676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cc0070fd0 0x7f9cc015b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.797+0000 7f9cbf7fe640 1 -- 172.21.15.112:0/2626854676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc01f36c0 con 0x7f9cc0072ce0 2024-04-25T01:39:53.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.797+0000 7f9cbf7fe640 1 --2- 172.21.15.112:0/2626854676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cc0072ce0 0x7f9cc01512f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9cb402a250 tx=0x7f9cb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:53.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.798+0000 7f9cbcff9640 1 -- 172.21.15.112:0/2626854676 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4037040 con 0x7f9cc0072ce0 2024-04-25T01:39:53.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.798+0000 7f9cc697e640 1 -- 172.21.15.112:0/2626854676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc01f39a0 con 0x7f9cc0072ce0 2024-04-25T01:39:53.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.798+0000 7f9cc697e640 1 -- 172.21.15.112:0/2626854676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc006ccd0 con 0x7f9cc0072ce0 2024-04-25T01:39:53.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.799+0000 7f9cbcff9640 1 -- 172.21.15.112:0/2626854676 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb4042070 con 0x7f9cc0072ce0 2024-04-25T01:39:53.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.800+0000 7f9cbcff9640 1 -- 172.21.15.112:0/2626854676 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4003e40 con 0x7f9cc0072ce0 2024-04-25T01:39:53.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.801+0000 7f9cbcff9640 1 -- 172.21.15.112:0/2626854676 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9cb4047460 con 0x7f9cc0072ce0 2024-04-25T01:39:53.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.802+0000 7f9cbcff9640 1 --2- 172.21.15.112:0/2626854676 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9cac06be60 0x7f9cac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.804+0000 7f9cbcff9640 1 -- 172.21.15.112:0/2626854676 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f9cb40abac0 con 0x7f9cc0072ce0 2024-04-25T01:39:53.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.805+0000 7f9ca67fc640 1 --2- 172.21.15.112:0/2626854676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ca0002a70 0x7f9ca0004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.805+0000 7f9ca67fc640 1 -- 172.21.15.112:0/2626854676 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca0007ec0 con 0x7f9ca0002a70 2024-04-25T01:39:53.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.805+0000 7f9cc4f1b640 1 --2- 172.21.15.112:0/2626854676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ca0002a70 0x7f9ca0004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.807+0000 7f9cbffff640 1 --2- 172.21.15.112:0/2626854676 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9cac06be60 0x7f9cac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.808+0000 7f9cbffff640 1 --2- 172.21.15.112:0/2626854676 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9cac06be60 0x7f9cac06e320 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0006b10 tx=0x7f9cb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:53.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.811+0000 7f9cc4f1b640 1 -- 172.21.15.112:0/2626854676 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9ca00080c0 con 0x7f9ca0002a70 2024-04-25T01:39:53.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.814+0000 7f9cc4f1b640 1 --2- 172.21.15.112:0/2626854676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ca0002a70 0x7f9ca0004f30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800eb50 tx=0x7f9cb800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:53.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.814+0000 7f9cbcff9640 1 -- 172.21.15.112:0/2626854676 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8018030 con 0x7f9ca0002a70 2024-04-25T01:39:53.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.815+0000 7f9cbcff9640 1 -- 172.21.15.112:0/2626854676 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9cb800cb10 con 0x7f9ca0002a70 2024-04-25T01:39:53.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.815+0000 7f9cbcff9640 1 -- 172.21.15.112:0/2626854676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ca0002a70 msgr2=0x7f9ca0004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.815+0000 7f9cbcff9640 1 --2- 172.21.15.112:0/2626854676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ca0002a70 0x7f9ca0004f30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800eb50 tx=0x7f9cb800c490 comp rx=0 tx=0).stop 2024-04-25T01:39:53.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.815+0000 7f9cbcff9640 1 -- 172.21.15.112:0/2626854676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ca0002a70 msgr2=0x7f9ca0004f30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:39:53.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.815+0000 7f9cbcff9640 1 --2- 172.21.15.112:0/2626854676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ca0002a70 0x7f9ca0004f30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.832+0000 7f9ca67fc640 1 --2- 172.21.15.112:0/2626854676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ca0006ee0 0x7f9ca0007370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.832+0000 7f9ca67fc640 1 -- 172.21.15.112:0/2626854676 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca000a4a0 con 0x7f9ca0006ee0 2024-04-25T01:39:53.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.833+0000 7f9cc4f1b640 1 --2- 172.21.15.112:0/2626854676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ca0006ee0 0x7f9ca0007370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.840+0000 7f9cc4f1b640 1 -- 172.21.15.112:0/2626854676 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9ca00078b0 con 0x7f9ca0006ee0 2024-04-25T01:39:53.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.841+0000 7f9cc4f1b640 1 --2- 172.21.15.112:0/2626854676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ca0006ee0 0x7f9ca0007370 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8018000 tx=0x7f9cb8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:53.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.842+0000 7f9cbcff9640 1 -- 172.21.15.112:0/2626854676 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8003b90 con 0x7f9ca0006ee0 2024-04-25T01:39:53.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.842+0000 7f9cbcff9640 1 -- 172.21.15.112:0/2626854676 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5071 (secure 0 0 0) 0x7f9cb8003d30 con 0x7f9ca0006ee0 2024-04-25T01:39:53.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.842+0000 7f9cbcff9640 1 -- 172.21.15.112:0/2626854676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ca0006ee0 msgr2=0x7f9ca0007370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.842+0000 7f9cbcff9640 1 --2- 172.21.15.112:0/2626854676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ca0006ee0 0x7f9ca0007370 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8018000 tx=0x7f9cb8004ab0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.842+0000 7f9cbcff9640 1 -- 172.21.15.112:0/2626854676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ca0006ee0 msgr2=0x7f9ca0007370 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:39:53.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.842+0000 7f9cbcff9640 1 --2- 172.21.15.112:0/2626854676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ca0006ee0 0x7f9ca0007370 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.826 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:39:53.826 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:39:53.826 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:39:53.826 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:39:53.826 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 110, 2024-04-25T01:39:53.826 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 163207, 2024-04-25T01:39:53.827 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:39:53.827 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:39:53.827 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:39:53.827 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:39:53.827 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:53.827 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 16, 2024-04-25T01:39:53.827 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:39:53.827 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:39:53.827 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:39:53.827 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:39:53.827 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:39:53.827 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:39:53.827 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:39:53.828 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:39:53.828 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:39:53.828 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:39:53.828 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:39:53.828 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:39:53.828 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:39:53.828 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:39:53.828 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:53.828 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:39:53.828 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:39:53.828 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:39:53.828 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:39:53.828 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:39:53.828 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:39:53.828 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:39:53.829 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:39:53.829 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:39:53.829 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:39:53.829 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:39:53.829 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:39:53.829 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:39:53.829 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:53.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.829 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:39:53.829 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:39:53.829 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:39:53.829 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:39:53.829 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:39:53.830 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:39:53.830 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:39:53.830 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:39:53.830 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:39:53.830 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:39:53.830 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:39:53.830 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:39:53.830 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:39:53.830 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:39:53.830 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:39:53.830 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:39:53.830 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:39:53.830 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.832 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:39:53.832 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:39:53.832 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:39:53.832 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:39:53.832 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.832 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:39:53.832 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:39:53.832 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:53.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.832 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.832 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:39:53.832 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:39:53.832 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:53.832 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.833 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:53.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.833 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:39:53.833 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:39:53.833 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:39:53.833 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:39:53.833 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:39:53.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.833 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.833 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:39:53.833 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:39:53.833 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:39:53.833 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:39:53.833 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.833 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:39:53.834 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:39:53.834 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:53.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.834 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.834 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:39:53.834 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:39:53.834 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:53.834 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.834 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:53.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.834 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:39:53.835 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:39:53.835 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:39:53.835 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:39:53.835 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:39:53.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.835 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.835 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:39:53.835 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:39:53.835 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:39:53.835 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:39:53.835 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.835 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:39:53.835 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:39:53.835 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:53.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.836 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.836 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:39:53.836 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:39:53.836 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:39:53.836 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.836 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:53.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.836 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:39:53.836 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:39:53.836 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:39:53.836 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:39:53.836 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:39:53.836 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.837 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:53.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.837 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:39:53.837 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:39:53.837 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.837 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:39:53.837 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:39:53.837 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:39:53.837 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.837 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:53.837 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:39:53.837 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.837 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:39:53.837 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:39:53.837 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:39:53.838 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.838 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:53.838 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:39:53.838 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.838 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:39:53.838 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:39:53.838 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 8 2024-04-25T01:39:53.838 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.838 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:53.838 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:39:53.838 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.838 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:39:53.838 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:39:53.838 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:39:53.839 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.839 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:39:53.839 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:39:53.839 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:39:53.839 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:39:53.839 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:39:53.839 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:39:53.839 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:39:53.839 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:39:53.839 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:39:53.839 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:39:53.839 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:39:53.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.844+0000 7f9cc697e640 1 -- 172.21.15.112:0/2626854676 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9cac06be60 msgr2=0x7f9cac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.844+0000 7f9cc697e640 1 --2- 172.21.15.112:0/2626854676 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9cac06be60 0x7f9cac06e320 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0006b10 tx=0x7f9cb0008040 comp rx=0 tx=0).stop 2024-04-25T01:39:53.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.844+0000 7f9cc4f1b640 1 -- 172.21.15.112:0/2626854676 reap_dead start 2024-04-25T01:39:53.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.844+0000 7f9cc697e640 1 -- 172.21.15.112:0/2626854676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cc0072ce0 msgr2=0x7f9cc01512f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.844+0000 7f9cc697e640 1 --2- 172.21.15.112:0/2626854676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cc0072ce0 0x7f9cc01512f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9cb402a250 tx=0x7f9cb40046c0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.844+0000 7f9cc697e640 1 -- 172.21.15.112:0/2626854676 shutdown_connections 2024-04-25T01:39:53.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.844+0000 7f9cc697e640 1 --2- 172.21.15.112:0/2626854676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cc0072ce0 0x7f9cc01512f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.844+0000 7f9cc697e640 1 -- 172.21.15.112:0/2626854676 >> 172.21.15.112:0/2626854676 conn(0x7f9cc00c7080 msgr2=0x7f9cc0156650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:53.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.848+0000 7f9cc697e640 1 -- 172.21.15.112:0/2626854676 shutdown_connections 2024-04-25T01:39:53.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:53.849+0000 7f9cc697e640 1 -- 172.21.15.112:0/2626854676 wait complete. 2024-04-25T01:39:53.853 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:39:53.853 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-04-25T01:39:53.853 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-25T01:39:53.853 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-04-25T01:39:53.853 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-04-25T01:39:53.853 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T01:39:53.853 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:39:53.872 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T01:39:53.872 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T01:39:53.872 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:39:54.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.143+0000 7f3315b4f640 1 Processor -- start 2024-04-25T01:39:54.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.143+0000 7f3315b4f640 1 -- start start 2024-04-25T01:39:54.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.145+0000 7f3315b4f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3310152600 0x7f331006f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:54.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.145+0000 7f3315b4f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3310151b60 con 0x7f331014ef00 2024-04-25T01:39:54.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.145+0000 7f3315b4f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33100ac6e0 con 0x7f33100aa500 2024-04-25T01:39:54.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.145+0000 7f3315b4f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f33100ac9c0 con 0x7f3310152600 2024-04-25T01:39:54.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.145+0000 7f330f7fe640 1 -- >> v1:172.21.15.112:6789/0 conn(0x7f331014ef00 legacy=0x7f33100abd70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:39:54.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.147+0000 7f330ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3310152600 0x7f331006f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:54.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.147+0000 7f330ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3310152600 0x7f331006f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:33518/0 (socket says 172.21.15.112:33518) 2024-04-25T01:39:54.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.147+0000 7f330ffff640 1 -- 172.21.15.112:0/1165333514 learned_addr learned my addr 172.21.15.112:0/1165333514 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:54.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.147+0000 7f330ffff640 1 -- 172.21.15.112:0/1165333514 >> v1:172.21.15.187:6789/0 conn(0x7f33100aa500 legacy=0x7f33100c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:39:54.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.148+0000 7f330e7fc640 1 -- 172.21.15.112:0/1165333514 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3600126751 0 0) 0x7f33100ac6e0 con 0x7f33100aa500 2024-04-25T01:39:54.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.151+0000 7f330ffff640 1 -- 172.21.15.112:0/1165333514 >> v1:172.21.15.112:6789/0 conn(0x7f331014ef00 legacy=0x7f33100abd70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:39:54.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.151+0000 7f330ffff640 1 -- 172.21.15.112:0/1165333514 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3310151e00 con 0x7f3310152600 2024-04-25T01:39:54.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.153+0000 7f330ffff640 1 --2- 172.21.15.112:0/1165333514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3310152600 0x7f331006f360 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f330400a940 tx=0x7f330402e990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb583aaead8c057b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:54.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.154+0000 7f330e7fc640 1 -- 172.21.15.112:0/1165333514 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304030040 con 0x7f3310152600 2024-04-25T01:39:54.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.158+0000 7f330e7fc640 1 -- 172.21.15.112:0/1165333514 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f330403b070 con 0x7f3310152600 2024-04-25T01:39:54.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.159+0000 7f3315b4f640 1 -- 172.21.15.112:0/1165333514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3310152600 msgr2=0x7f331006f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:54.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.159+0000 7f3315b4f640 1 --2- 172.21.15.112:0/1165333514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3310152600 0x7f331006f360 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f330400a940 tx=0x7f330402e990 comp rx=0 tx=0).stop 2024-04-25T01:39:54.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.162+0000 7f3315b4f640 1 -- 172.21.15.112:0/1165333514 shutdown_connections 2024-04-25T01:39:54.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.162+0000 7f3315b4f640 1 --2- 172.21.15.112:0/1165333514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3310152600 0x7f331006f360 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:54.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.162+0000 7f3315b4f640 1 -- 172.21.15.112:0/1165333514 >> 172.21.15.112:0/1165333514 conn(0x7f33100a9670 msgr2=0x7f33100a9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:54.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.162+0000 7f3315b4f640 1 -- 172.21.15.112:0/1165333514 shutdown_connections 2024-04-25T01:39:54.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.162+0000 7f3315b4f640 1 -- 172.21.15.112:0/1165333514 wait complete. 2024-04-25T01:39:54.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.163+0000 7f3315b4f640 1 Processor -- start 2024-04-25T01:39:54.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.164+0000 7f3315b4f640 1 -- start start 2024-04-25T01:39:54.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.164+0000 7f3315b4f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33100aa500 0x7f3310140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:54.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.165+0000 7f3315b4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331014ef00 0x7f331013ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:54.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.165+0000 7f3315b4f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3310152600 0x7f3310149540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:54.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.165+0000 7f3315b4f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f33100ac480 con 0x7f331014ef00 2024-04-25T01:39:54.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.165+0000 7f3315b4f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3310152170 con 0x7f33100aa500 2024-04-25T01:39:54.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.165+0000 7f3315b4f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f331014f2e0 con 0x7f3310152600 2024-04-25T01:39:54.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.165+0000 7f330effd640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331014ef00 msgr2=0x7f331013ca70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:39:54.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.165+0000 7f330effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331014ef00 0x7f331013ca70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:39:54.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.165+0000 7f330f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33100aa500 0x7f3310140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:54.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.165+0000 7f330f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33100aa500 0x7f3310140ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:58792/0 (socket says 172.21.15.112:58792) 2024-04-25T01:39:54.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.165+0000 7f330f7fe640 1 -- 172.21.15.112:0/1668612699 learned_addr learned my addr 172.21.15.112:0/1668612699 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:54.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.165+0000 7f330ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3310152600 0x7f3310149540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:54.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.166+0000 7f330f7fe640 1 -- 172.21.15.112:0/1668612699 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3310152600 msgr2=0x7f3310149540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:54.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.166+0000 7f330f7fe640 1 --2- 172.21.15.112:0/1668612699 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3310152600 0x7f3310149540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:54.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.166+0000 7f330f7fe640 1 -- 172.21.15.112:0/1668612699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331014ef00 msgr2=0x7f331013ca70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:39:54.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.166+0000 7f330f7fe640 1 --2- 172.21.15.112:0/1668612699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331014ef00 0x7f331013ca70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:54.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.166+0000 7f330f7fe640 1 -- 172.21.15.112:0/1668612699 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3310143080 con 0x7f33100aa500 2024-04-25T01:39:54.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.170+0000 7f330f7fe640 1 --2- 172.21.15.112:0/1668612699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33100aa500 0x7f3310140ab0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f330000b580 tx=0x7f330000ba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.171+0000 7f32f7fff640 1 -- 172.21.15.112:0/1668612699 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300003bc0 con 0x7f33100aa500 2024-04-25T01:39:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.171+0000 7f32f7fff640 1 -- 172.21.15.112:0/1668612699 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3300015070 con 0x7f33100aa500 2024-04-25T01:39:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.171+0000 7f3315b4f640 1 -- 172.21.15.112:0/1668612699 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3310143340 con 0x7f33100aa500 2024-04-25T01:39:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.171+0000 7f3315b4f640 1 -- 172.21.15.112:0/1668612699 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3310140590 con 0x7f33100aa500 2024-04-25T01:39:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.172+0000 7f32f7fff640 1 -- 172.21.15.112:0/1668612699 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33000050f0 con 0x7f33100aa500 2024-04-25T01:39:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.174+0000 7f32f7fff640 1 -- 172.21.15.112:0/1668612699 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3300005290 con 0x7f33100aa500 2024-04-25T01:39:54.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.175+0000 7f32f7fff640 1 --2- 172.21.15.112:0/1668612699 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f32fc06be30 0x7f32fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:54.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.175+0000 7f32f7fff640 1 -- 172.21.15.112:0/1668612699 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f330008ab10 con 0x7f33100aa500 2024-04-25T01:39:54.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.175+0000 7f330effd640 1 --2- 172.21.15.112:0/1668612699 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f32fc06be30 0x7f32fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:54.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.176+0000 7f330effd640 1 --2- 172.21.15.112:0/1668612699 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f32fc06be30 0x7f32fc06e2f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3308001560 tx=0x7f3308007cd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:54.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.176+0000 7f3315b4f640 1 -- 172.21.15.112:0/1668612699 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f0005290 con 0x7f33100aa500 2024-04-25T01:39:54.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.185+0000 7f32f7fff640 1 -- 172.21.15.112:0/1668612699 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3300059190 con 0x7f33100aa500 2024-04-25T01:39:54.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.772+0000 7f3315b4f640 1 -- 172.21.15.112:0/1668612699 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f32f0005580 con 0x7f33100aa500 2024-04-25T01:39:54.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.775+0000 7f32f7fff640 1 -- 172.21.15.112:0/1668612699 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1570 (secure 0 0 0) 0x7f3300058a90 con 0x7f33100aa500 2024-04-25T01:39:54.759 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":110,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:39:54.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.779+0000 7f3315b4f640 1 -- 172.21.15.112:0/1668612699 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f32fc06be30 msgr2=0x7f32fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:54.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.779+0000 7f3315b4f640 1 --2- 172.21.15.112:0/1668612699 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f32fc06be30 0x7f32fc06e2f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3308001560 tx=0x7f3308007cd0 comp rx=0 tx=0).stop 2024-04-25T01:39:54.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.779+0000 7f3315b4f640 1 -- 172.21.15.112:0/1668612699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33100aa500 msgr2=0x7f3310140ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:54.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.779+0000 7f3315b4f640 1 --2- 172.21.15.112:0/1668612699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33100aa500 0x7f3310140ab0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f330000b580 tx=0x7f330000ba40 comp rx=0 tx=0).stop 2024-04-25T01:39:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.779+0000 7f3315b4f640 1 -- 172.21.15.112:0/1668612699 shutdown_connections 2024-04-25T01:39:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.779+0000 7f3315b4f640 1 --2- 172.21.15.112:0/1668612699 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f32fc06be30 0x7f32fc06e2f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.779+0000 7f3315b4f640 1 --2- 172.21.15.112:0/1668612699 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3310152600 0x7f3310149540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.779+0000 7f3315b4f640 1 --2- 172.21.15.112:0/1668612699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331014ef00 0x7f331013ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.779+0000 7f3315b4f640 1 --2- 172.21.15.112:0/1668612699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33100aa500 0x7f3310140ab0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.779+0000 7f3315b4f640 1 -- 172.21.15.112:0/1668612699 >> 172.21.15.112:0/1668612699 conn(0x7f33100a9670 msgr2=0x7f33100673f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:54.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.783+0000 7f3315b4f640 1 -- 172.21.15.112:0/1668612699 shutdown_connections 2024-04-25T01:39:54.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:54.783+0000 7f3315b4f640 1 -- 172.21.15.112:0/1668612699 wait complete. 2024-04-25T01:39:57.785 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:39:57.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.982+0000 7f9ad3ab3640 1 Processor -- start 2024-04-25T01:39:57.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.982+0000 7f9ad3ab3640 1 -- start start 2024-04-25T01:39:57.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.984+0000 7f9ad3ab3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9acc0c2840 0x7f9acc0c2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:57.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.984+0000 7f9ad3ab3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9acc163700 0x7f9acc163b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:57.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.984+0000 7f9ad2050640 1 -- >> v2:172.21.15.112:3300/0 conn(0x7f9acc163700 msgr2=0x7f9acc163b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:39:57.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.984+0000 7f9ad2050640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9acc163700 0x7f9acc163b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:39:57.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.984+0000 7f9ad3ab3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9acc070ec0 con 0x7f9acc162c90 2024-04-25T01:39:57.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.984+0000 7f9ad184f640 1 -- >> v1:172.21.15.112:6789/0 conn(0x7f9acc162c90 legacy=0x7f9acc15f570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:39:57.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.984+0000 7f9ad3ab3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f9acc0717f0 con 0x7f9acc163700 2024-04-25T01:39:57.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.984+0000 7f9ad3ab3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9acc15fc80 con 0x7f9acc0c2840 2024-04-25T01:39:57.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.986+0000 7f9ad104e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9acc0c2840 0x7f9acc0c2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:57.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.986+0000 7f9ad104e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9acc0c2840 0x7f9acc0c2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:54760/0 (socket says 172.21.15.112:54760) 2024-04-25T01:39:57.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.986+0000 7f9ad104e640 1 -- 172.21.15.112:0/2572617717 learned_addr learned my addr 172.21.15.112:0/2572617717 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:57.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.987+0000 7f9ad104e640 1 -- 172.21.15.112:0/2572617717 >> v2:172.21.15.112:3300/0 conn(0x7f9acc163700 msgr2=0x7f9acc163b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:39:57.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.987+0000 7f9ad104e640 1 --2- 172.21.15.112:0/2572617717 >> v2:172.21.15.112:3300/0 conn(0x7f9acc163700 0x7f9acc163b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:57.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.987+0000 7f9ad104e640 1 -- 172.21.15.112:0/2572617717 >> v1:172.21.15.112:6789/0 conn(0x7f9acc162c90 legacy=0x7f9acc15f570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:39:57.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.987+0000 7f9ad104e640 1 -- 172.21.15.112:0/2572617717 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9acc0711a0 con 0x7f9acc0c2840 2024-04-25T01:39:57.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.990+0000 7f9ad104e640 1 --2- 172.21.15.112:0/2572617717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9acc0c2840 0x7f9acc0c2cd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009e50 tx=0x7f9ac402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=919b6be19f23f495 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:57.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.990+0000 7f9ad084d640 1 -- 172.21.15.112:0/2572617717 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4036070 con 0x7f9acc0c2840 2024-04-25T01:39:57.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.992+0000 7f9ad084d640 1 -- 172.21.15.112:0/2572617717 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ac4031600 con 0x7f9acc0c2840 2024-04-25T01:39:57.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.992+0000 7f9ad084d640 1 -- 172.21.15.112:0/2572617717 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac40319d0 con 0x7f9acc0c2840 2024-04-25T01:39:57.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.993+0000 7f9ad3ab3640 1 -- 172.21.15.112:0/2572617717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9acc0c2840 msgr2=0x7f9acc0c2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:57.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.993+0000 7f9ad3ab3640 1 --2- 172.21.15.112:0/2572617717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9acc0c2840 0x7f9acc0c2cd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009e50 tx=0x7f9ac402a660 comp rx=0 tx=0).stop 2024-04-25T01:39:57.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.994+0000 7f9ad3ab3640 1 -- 172.21.15.112:0/2572617717 shutdown_connections 2024-04-25T01:39:57.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.994+0000 7f9ad3ab3640 1 --2- 172.21.15.112:0/2572617717 >> v2:172.21.15.112:3300/0 conn(0x7f9acc163700 0x7f9acc163b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:57.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.994+0000 7f9ad3ab3640 1 --2- 172.21.15.112:0/2572617717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9acc0c2840 0x7f9acc0c2cd0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009e50 tx=0x7f9ac402a660 comp rx=0 tx=0).stop 2024-04-25T01:39:57.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.994+0000 7f9ad3ab3640 1 -- 172.21.15.112:0/2572617717 >> 172.21.15.112:0/2572617717 conn(0x7f9acc0c3930 msgr2=0x7f9acc0c3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:57.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.998+0000 7f9ad3ab3640 1 -- 172.21.15.112:0/2572617717 shutdown_connections 2024-04-25T01:39:57.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:57.999+0000 7f9ad3ab3640 1 -- 172.21.15.112:0/2572617717 wait complete. 2024-04-25T01:39:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.001+0000 7f9ad3ab3640 1 Processor -- start 2024-04-25T01:39:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.001+0000 7f9ad3ab3640 1 -- start start 2024-04-25T01:39:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.002+0000 7f9ad3ab3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9acc162c90 0x7f9acc15c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.002+0000 7f9ad3ab3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9acc163700 0x7f9acc1ed2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.002+0000 7f9ad3ab3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9acc1ee470 0x7f9acc1ed7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.002+0000 7f9ad3ab3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc1631f0 con 0x7f9acc162c90 2024-04-25T01:39:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.002+0000 7f9ad3ab3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc163070 con 0x7f9acc163700 2024-04-25T01:39:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.002+0000 7f9ad2050640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9acc1ee470 0x7f9acc1ed7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.002+0000 7f9ad2050640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9acc1ee470 0x7f9acc1ed7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:54772/0 (socket says 172.21.15.112:54772) 2024-04-25T01:39:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.002+0000 7f9ad2050640 1 -- 172.21.15.112:0/1345993942 learned_addr learned my addr 172.21.15.112:0/1345993942 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:39:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.002+0000 7f9ad184f640 1 -- 172.21.15.112:0/1345993942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9acc162c90 msgr2=0x7f9acc15c850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:39:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.002+0000 7f9ad184f640 1 --2- 172.21.15.112:0/1345993942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9acc162c90 0x7f9acc15c850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:39:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.003+0000 7f9ad3ab3640 1 -- 172.21.15.112:0/1345993942 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9acc072cb0 con 0x7f9acc1ee470 2024-04-25T01:39:57.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.005+0000 7f9ad2050640 1 -- 172.21.15.112:0/1345993942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9acc163700 msgr2=0x7f9acc1ed2b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:39:57.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.005+0000 7f9ad2050640 1 --2- 172.21.15.112:0/1345993942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9acc163700 0x7f9acc1ed2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:57.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.005+0000 7f9ad2050640 1 -- 172.21.15.112:0/1345993942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9acc162c90 msgr2=0x7f9acc15c850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:39:57.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.005+0000 7f9ad2050640 1 --2- 172.21.15.112:0/1345993942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9acc162c90 0x7f9acc15c850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:57.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.005+0000 7f9ad2050640 1 -- 172.21.15.112:0/1345993942 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9acc1f8360 con 0x7f9acc1ee470 2024-04-25T01:39:57.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.005+0000 7f9ad2050640 1 --2- 172.21.15.112:0/1345993942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9acc1ee470 0x7f9acc1ed7f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00d820 tx=0x7f9abc00dce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:57.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.006+0000 7f9ac27fc640 1 -- 172.21.15.112:0/1345993942 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc00ad50 con 0x7f9acc1ee470 2024-04-25T01:39:57.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.006+0000 7f9ad3ab3640 1 -- 172.21.15.112:0/1345993942 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9acc1f8640 con 0x7f9acc1ee470 2024-04-25T01:39:57.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.006+0000 7f9ad3ab3640 1 -- 172.21.15.112:0/1345993942 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9acc15cd90 con 0x7f9acc1ee470 2024-04-25T01:39:57.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.006+0000 7f9ac27fc640 1 -- 172.21.15.112:0/1345993942 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9abc0163f0 con 0x7f9acc1ee470 2024-04-25T01:39:57.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.006+0000 7f9ac27fc640 1 -- 172.21.15.112:0/1345993942 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc0167e0 con 0x7f9acc1ee470 2024-04-25T01:39:57.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.007+0000 7f9ad3ab3640 1 -- 172.21.15.112:0/1345993942 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab4005290 con 0x7f9acc1ee470 2024-04-25T01:39:57.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.009+0000 7f9ac27fc640 1 -- 172.21.15.112:0/1345993942 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9abc01d020 con 0x7f9acc1ee470 2024-04-25T01:39:57.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.009+0000 7f9ac27fc640 1 --2- 172.21.15.112:0/1345993942 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ab006e020 0x7f9ab00704e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:57.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.009+0000 7f9ac27fc640 1 -- 172.21.15.112:0/1345993942 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f9abc08b850 con 0x7f9acc1ee470 2024-04-25T01:39:57.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.010+0000 7f9ad184f640 1 --2- 172.21.15.112:0/1345993942 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ab006e020 0x7f9ab00704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:57.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.011+0000 7f9ad184f640 1 --2- 172.21.15.112:0/1345993942 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ab006e020 0x7f9ab00704e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8003980 tx=0x7f9ac8006d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:58.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.023+0000 7f9ac27fc640 1 -- 172.21.15.112:0/1345993942 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9abc059e50 con 0x7f9acc1ee470 2024-04-25T01:39:58.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.560+0000 7f9ad3ab3640 1 -- 172.21.15.112:0/1345993942 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9ab4005580 con 0x7f9acc1ee470 2024-04-25T01:39:58.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.561+0000 7f9ac27fc640 1 -- 172.21.15.112:0/1345993942 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1570 (secure 0 0 0) 0x7f9abc059750 con 0x7f9acc1ee470 2024-04-25T01:39:58.553 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":110,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:39:58.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.573+0000 7f9aaffff640 1 -- 172.21.15.112:0/1345993942 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ab006e020 msgr2=0x7f9ab00704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:58.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.573+0000 7f9aaffff640 1 --2- 172.21.15.112:0/1345993942 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ab006e020 0x7f9ab00704e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8003980 tx=0x7f9ac8006d30 comp rx=0 tx=0).stop 2024-04-25T01:39:58.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.573+0000 7f9aaffff640 1 -- 172.21.15.112:0/1345993942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9acc1ee470 msgr2=0x7f9acc1ed7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:58.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.573+0000 7f9aaffff640 1 --2- 172.21.15.112:0/1345993942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9acc1ee470 0x7f9acc1ed7f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00d820 tx=0x7f9abc00dce0 comp rx=0 tx=0).stop 2024-04-25T01:39:58.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.575+0000 7f9aaffff640 1 -- 172.21.15.112:0/1345993942 shutdown_connections 2024-04-25T01:39:58.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.575+0000 7f9aaffff640 1 --2- 172.21.15.112:0/1345993942 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9ab006e020 0x7f9ab00704e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:58.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.575+0000 7f9aaffff640 1 --2- 172.21.15.112:0/1345993942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9acc1ee470 0x7f9acc1ed7f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:58.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.575+0000 7f9aaffff640 1 --2- 172.21.15.112:0/1345993942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9acc163700 0x7f9acc1ed2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:58.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.575+0000 7f9aaffff640 1 --2- 172.21.15.112:0/1345993942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9acc162c90 0x7f9acc15c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:58.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.575+0000 7f9aaffff640 1 -- 172.21.15.112:0/1345993942 >> 172.21.15.112:0/1345993942 conn(0x7f9acc0c3930 msgr2=0x7f9acc165790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:58.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.580+0000 7f9aaffff640 1 -- 172.21.15.112:0/1345993942 shutdown_connections 2024-04-25T01:39:58.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:39:58.580+0000 7f9aaffff640 1 -- 172.21.15.112:0/1345993942 wait complete. 2024-04-25T01:39:59.448 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:39:59.451 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:40:01.587 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:40:01.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.785+0000 7fd704d90640 1 Processor -- start 2024-04-25T01:40:01.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.785+0000 7fd704d90640 1 -- start start 2024-04-25T01:40:01.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.786+0000 7fd704d90640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd7000c3980 0x7fd700153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:01.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.786+0000 7fd704d90640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7000aeac0 con 0x7fd70006f010 2024-04-25T01:40:01.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.786+0000 7fd704d90640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7001578c0 con 0x7fd7000c5f40 2024-04-25T01:40:01.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.786+0000 7fd704d90640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd700071090 con 0x7fd7000c3980 2024-04-25T01:40:01.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.786+0000 7fd6fed9d640 1 -- >> v1:172.21.15.112:6789/0 conn(0x7fd70006f010 legacy=0x7fd70006f440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:40:01.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.787+0000 7fd6fe59c640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fd7000c5f40 0x7fd7001535e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:57298/0 (socket says 172.21.15.112:57298) 2024-04-25T01:40:01.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.787+0000 7fd6fe59c640 1 -- 172.21.15.112:0/3533709251 learned_addr learned my addr 172.21.15.112:0/3533709251 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:40:01.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.787+0000 7fd6fdd9b640 1 --2- 172.21.15.112:0/3533709251 >> v2:172.21.15.187:3300/0 conn(0x7fd7000c3980 0x7fd700153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:01.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.787+0000 7fd6fdd9b640 1 -- 172.21.15.112:0/3533709251 >> v1:172.21.15.187:6789/0 conn(0x7fd7000c5f40 legacy=0x7fd7001535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:01.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.787+0000 7fd6fdd9b640 1 -- 172.21.15.112:0/3533709251 >> v1:172.21.15.112:6789/0 conn(0x7fd70006f010 legacy=0x7fd70006f440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:40:01.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.787+0000 7fd6fdd9b640 1 -- 172.21.15.112:0/3533709251 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7000712c0 con 0x7fd7000c3980 2024-04-25T01:40:01.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.789+0000 7fd6fdd9b640 1 --2- 172.21.15.112:0/3533709251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7000c3980 0x7fd700153cf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd6f40097a0 tx=0x7fd6f402a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffdbf2058270e052 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:01.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.789+0000 7fd6fd59a640 1 -- 172.21.15.112:0/3533709251 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4037070 con 0x7fd7000c3980 2024-04-25T01:40:01.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.789+0000 7fd6fd59a640 1 -- 172.21.15.112:0/3533709251 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f4032440 con 0x7fd7000c3980 2024-04-25T01:40:01.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.789+0000 7fd6fd59a640 1 -- 172.21.15.112:0/3533709251 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4032810 con 0x7fd7000c3980 2024-04-25T01:40:01.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.791+0000 7fd704d90640 1 -- 172.21.15.112:0/3533709251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7000c3980 msgr2=0x7fd700153cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:01.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.791+0000 7fd704d90640 1 --2- 172.21.15.112:0/3533709251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7000c3980 0x7fd700153cf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd6f40097a0 tx=0x7fd6f402a1e0 comp rx=0 tx=0).stop 2024-04-25T01:40:01.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.792+0000 7fd704d90640 1 -- 172.21.15.112:0/3533709251 shutdown_connections 2024-04-25T01:40:01.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.792+0000 7fd704d90640 1 --2- 172.21.15.112:0/3533709251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7000c3980 0x7fd700153cf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:01.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.792+0000 7fd704d90640 1 -- 172.21.15.112:0/3533709251 >> 172.21.15.112:0/3533709251 conn(0x7fd7000c8f80 msgr2=0x7fd7000c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:01.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.793+0000 7fd704d90640 1 -- 172.21.15.112:0/3533709251 shutdown_connections 2024-04-25T01:40:01.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.793+0000 7fd704d90640 1 -- 172.21.15.112:0/3533709251 wait complete. 2024-04-25T01:40:01.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.795+0000 7fd704d90640 1 Processor -- start 2024-04-25T01:40:01.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.795+0000 7fd704d90640 1 -- start start 2024-04-25T01:40:01.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.796+0000 7fd704d90640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd70006f010 0x7fd70014dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:01.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.796+0000 7fd704d90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7000c3980 0x7fd7001f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:01.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.796+0000 7fd704d90640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7000c5f40 0x7fd7001ef6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:01.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.796+0000 7fd704d90640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd7000717e0 con 0x7fd7000c3980 2024-04-25T01:40:01.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.796+0000 7fd704d90640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd700157590 con 0x7fd70006f010 2024-04-25T01:40:01.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.796+0000 7fd704d90640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd7000aef30 con 0x7fd7000c5f40 2024-04-25T01:40:01.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.796+0000 7fd6fdd9b640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7000c3980 msgr2=0x7fd7001f0b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:40:01.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.796+0000 7fd6fdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7000c3980 0x7fd7001f0b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:01.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.796+0000 7fd6fe59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd70006f010 0x7fd70014dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:01.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.797+0000 7fd6fe59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd70006f010 0x7fd70014dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:37428/0 (socket says 172.21.15.112:37428) 2024-04-25T01:40:01.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.797+0000 7fd6fe59c640 1 -- 172.21.15.112:0/3889833144 learned_addr learned my addr 172.21.15.112:0/3889833144 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:01.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.797+0000 7fd6fed9d640 1 --2- 172.21.15.112:0/3889833144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7000c5f40 0x7fd7001ef6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:01.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.801+0000 7fd6fe59c640 1 -- 172.21.15.112:0/3889833144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7000c5f40 msgr2=0x7fd7001ef6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:01.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.801+0000 7fd6fe59c640 1 --2- 172.21.15.112:0/3889833144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7000c5f40 0x7fd7001ef6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:01.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.801+0000 7fd6fe59c640 1 -- 172.21.15.112:0/3889833144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7000c3980 msgr2=0x7fd7001f0b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:40:01.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.801+0000 7fd6fe59c640 1 --2- 172.21.15.112:0/3889833144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7000c3980 0x7fd7001f0b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:01.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.801+0000 7fd6fe59c640 1 -- 172.21.15.112:0/3889833144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd700151bc0 con 0x7fd70006f010 2024-04-25T01:40:01.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.803+0000 7fd6fe59c640 1 --2- 172.21.15.112:0/3889833144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd70006f010 0x7fd70014dd50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c3f0 tx=0x7fd6f000c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:01.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.803+0000 7fd6eeffd640 1 -- 172.21.15.112:0/3889833144 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0017070 con 0x7fd70006f010 2024-04-25T01:40:01.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.803+0000 7fd704d90640 1 -- 172.21.15.112:0/3889833144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7001518e0 con 0x7fd70006f010 2024-04-25T01:40:01.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.804+0000 7fd704d90640 1 -- 172.21.15.112:0/3889833144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd70006d000 con 0x7fd70006f010 2024-04-25T01:40:01.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.804+0000 7fd6eeffd640 1 -- 172.21.15.112:0/3889833144 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f0005ce0 con 0x7fd70006f010 2024-04-25T01:40:01.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.804+0000 7fd6eeffd640 1 -- 172.21.15.112:0/3889833144 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00072d0 con 0x7fd70006f010 2024-04-25T01:40:01.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.807+0000 7fd6eeffd640 1 -- 172.21.15.112:0/3889833144 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd6f001a070 con 0x7fd70006f010 2024-04-25T01:40:01.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.808+0000 7fd6eeffd640 1 --2- 172.21.15.112:0/3889833144 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd6e006bde0 0x7fd6e006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:01.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.808+0000 7fd6eeffd640 1 -- 172.21.15.112:0/3889833144 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fd6f008a440 con 0x7fd70006f010 2024-04-25T01:40:01.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.811+0000 7fd704d90640 1 -- 172.21.15.112:0/3889833144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7000682a0 con 0x7fd70006f010 2024-04-25T01:40:01.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.812+0000 7fd6fdd9b640 1 --2- 172.21.15.112:0/3889833144 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd6e006bde0 0x7fd6e006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:01.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.813+0000 7fd6fdd9b640 1 --2- 172.21.15.112:0/3889833144 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd6e006bde0 0x7fd6e006e2a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4034000 tx=0x7fd6f402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:01.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:01.826+0000 7fd6eeffd640 1 -- 172.21.15.112:0/3889833144 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd6f00583f0 con 0x7fd70006f010 2024-04-25T01:40:02.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:02.352+0000 7fd704d90640 1 -- 172.21.15.112:0/3889833144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd70006ff60 con 0x7fd70006f010 2024-04-25T01:40:02.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:02.353+0000 7fd6eeffd640 1 -- 172.21.15.112:0/3889833144 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1570 (secure 0 0 0) 0x7fd6f005c6f0 con 0x7fd70006f010 2024-04-25T01:40:02.338 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":110,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:40:02.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:02.360+0000 7fd6ecff9640 1 -- 172.21.15.112:0/3889833144 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd6e006bde0 msgr2=0x7fd6e006e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:02.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:02.360+0000 7fd6ecff9640 1 --2- 172.21.15.112:0/3889833144 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd6e006bde0 0x7fd6e006e2a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4034000 tx=0x7fd6f402e000 comp rx=0 tx=0).stop 2024-04-25T01:40:02.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:02.360+0000 7fd6ecff9640 1 -- 172.21.15.112:0/3889833144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd70006f010 msgr2=0x7fd70014dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:02.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:02.360+0000 7fd6ecff9640 1 --2- 172.21.15.112:0/3889833144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd70006f010 0x7fd70014dd50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c3f0 tx=0x7fd6f000c8b0 comp rx=0 tx=0).stop 2024-04-25T01:40:02.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:02.361+0000 7fd6ecff9640 1 -- 172.21.15.112:0/3889833144 shutdown_connections 2024-04-25T01:40:02.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:02.361+0000 7fd6ecff9640 1 --2- 172.21.15.112:0/3889833144 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd6e006bde0 0x7fd6e006e2a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:02.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:02.361+0000 7fd6ecff9640 1 --2- 172.21.15.112:0/3889833144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7000c5f40 0x7fd7001ef6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:02.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:02.362+0000 7fd6ecff9640 1 --2- 172.21.15.112:0/3889833144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7000c3980 0x7fd7001f0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:02.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:02.362+0000 7fd6ecff9640 1 --2- 172.21.15.112:0/3889833144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd70006f010 0x7fd70014dd50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:02.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:02.362+0000 7fd6ecff9640 1 -- 172.21.15.112:0/3889833144 >> 172.21.15.112:0/3889833144 conn(0x7fd7000c8f80 msgr2=0x7fd7000a9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:02.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:02.364+0000 7fd6ecff9640 1 -- 172.21.15.112:0/3889833144 shutdown_connections 2024-04-25T01:40:02.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:02.366+0000 7fd6ecff9640 1 -- 172.21.15.112:0/3889833144 wait complete. 2024-04-25T01:40:05.374 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:40:05.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:05.557+0000 7f48a7fff640 1 Processor -- start 2024-04-25T01:40:05.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:05.557+0000 7f48a7fff640 1 -- start start 2024-04-25T01:40:05.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:05.558+0000 7f48a7fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 0x7f48a004dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:05.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:05.558+0000 7f48a7fff640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48a00da530 con 0x7f48a000e2d0 2024-04-25T01:40:05.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:05.558+0000 7f48a7fff640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48a00da810 con 0x7f48a004dff0 2024-04-25T01:40:05.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:05.558+0000 7f48a7fff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f48a00daaf0 con 0x7f48a00351c0 2024-04-25T01:40:05.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:05.559+0000 7f48a77fe640 1 -- >> v1:172.21.15.112:6789/0 conn(0x7f48a000e2d0 legacy=0x7f48a000e740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:40:05.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:05.559+0000 7f48a6ffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 0x7f48a004dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:05.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:05.560+0000 7f48a6ffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 0x7f48a004dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:54788/0 (socket says 172.21.15.112:54788) 2024-04-25T01:40:05.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:05.560+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 learned_addr learned my addr 172.21.15.112:0/2905102890 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:05.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:05.560+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 msgr2=0x7f48a004dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:40:05.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:05.560+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 msgr2=0x7f48a004dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:40:05.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:05.560+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 0x7f48a004dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:40:05.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:05.560+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 0x7f48a004dab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:05.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:05.763+0000 7f48a77fe640 1 -- 172.21.15.112:0/2905102890 >> v1:172.21.15.112:6789/0 conn(0x7f48a000e2d0 legacy=0x7f48a000e740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:40:05.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:05.763+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 0x7f48a004dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:05.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:05.763+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 msgr2=0x7f48a004dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:40:05.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:05.763+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 msgr2=0x7f48a004dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:40:05.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:05.763+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 0x7f48a004dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:40:05.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:05.763+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 0x7f48a004dab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:40:06.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:06.164+0000 7f48a77fe640 1 -- 172.21.15.112:0/2905102890 >> v1:172.21.15.112:6789/0 conn(0x7f48a000e2d0 legacy=0x7f48a000e740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:40:06.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:06.164+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 0x7f48a004dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:06.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:06.164+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 msgr2=0x7f48a004dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:40:06.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:06.164+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 msgr2=0x7f48a004dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:40:06.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:06.164+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 0x7f48a004dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:40:06.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:06.164+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 0x7f48a004dab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:40:06.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:06.965+0000 7f48a77fe640 1 -- 172.21.15.112:0/2905102890 >> v1:172.21.15.112:6789/0 conn(0x7f48a000e2d0 legacy=0x7f48a000e740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:40:06.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:06.965+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 0x7f48a004dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:06.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:06.969+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 msgr2=0x7f48a004dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:40:06.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:06.969+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 msgr2=0x7f48a004dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:40:06.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:06.969+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 0x7f48a004dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:40:06.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:06.969+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 0x7f48a004dab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:40:08.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.558+0000 7f48a4ff9640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 msgr2=0x7f48a004dab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:40:08.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.558+0000 7f48a4ff9640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00351c0 0x7f48a004dab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:08.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.558+0000 7f48a4ff9640 1 -- 172.21.15.112:0/2905102890 >> v1:172.21.15.187:6789/0 conn(0x7f48a004dff0 legacy=0x7f48a0007200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:08.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.558+0000 7f48a4ff9640 1 -- 172.21.15.112:0/2905102890 >> v1:172.21.15.112:6789/0 conn(0x7f48a000e2d0 legacy=0x7f48a000e740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:40:08.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.559+0000 7f48a4ff9640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.112:3300/0 conn(0x7f48940036f0 0x7f4894005b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:08.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.559+0000 7f48a4ff9640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 0x7f4894008660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:08.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.559+0000 7f48a4ff9640 1 -- 172.21.15.112:0/2905102890 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f48a00daaf0 con 0x7f48940036f0 2024-04-25T01:40:08.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.559+0000 7f48a4ff9640 1 -- 172.21.15.112:0/2905102890 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4894008d20 con 0x7f48940061a0 2024-04-25T01:40:08.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.559+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.112:3300/0 conn(0x7f48940036f0 msgr2=0x7f4894005b70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:40:08.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.559+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.112:3300/0 conn(0x7f48940036f0 0x7f4894005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:08.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.559+0000 7f48a67fc640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 0x7f4894008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:08.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.559+0000 7f48a67fc640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 msgr2=0x7f4894008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:40:08.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.560+0000 7f48a67fc640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 msgr2=0x7f4894008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:40:08.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.560+0000 7f48a67fc640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 0x7f4894008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:40:08.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.560+0000 7f48a67fc640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 0x7f4894008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:08.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.760+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.112:3300/0 conn(0x7f48940036f0 msgr2=0x7f4894005b70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:40:08.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.760+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.112:3300/0 conn(0x7f48940036f0 0x7f4894005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:40:08.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.761+0000 7f48a67fc640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 0x7f4894008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:08.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.761+0000 7f48a67fc640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 msgr2=0x7f4894008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:40:08.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.761+0000 7f48a67fc640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 msgr2=0x7f4894008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:40:08.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.761+0000 7f48a67fc640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 0x7f4894008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:40:08.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:08.761+0000 7f48a67fc640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 0x7f4894008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:40:09.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:09.161+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.112:3300/0 conn(0x7f48940036f0 msgr2=0x7f4894005b70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:40:09.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:09.161+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.112:3300/0 conn(0x7f48940036f0 0x7f4894005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:40:09.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:09.162+0000 7f48a67fc640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 0x7f4894008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:09.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:09.162+0000 7f48a67fc640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 msgr2=0x7f4894008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:40:09.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:09.162+0000 7f48a67fc640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 msgr2=0x7f4894008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:40:09.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:09.162+0000 7f48a67fc640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 0x7f4894008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:40:09.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:09.162+0000 7f48a67fc640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 0x7f4894008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:40:09.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:09.962+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.112:3300/0 conn(0x7f48940036f0 msgr2=0x7f4894005b70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:40:09.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:09.962+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.112:3300/0 conn(0x7f48940036f0 0x7f4894005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:40:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:09.963+0000 7f48a67fc640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 0x7f4894008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:09.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:09.973+0000 7f48a67fc640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 msgr2=0x7f4894008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:40:09.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:09.973+0000 7f48a67fc640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 msgr2=0x7f4894008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:40:09.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:09.973+0000 7f48a67fc640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 0x7f4894008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:40:09.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:09.973+0000 7f48a67fc640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 0x7f4894008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:40:11.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.559+0000 7f48a4ff9640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 msgr2=0x7f4894008660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:40:11.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.559+0000 7f48a4ff9640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f48940061a0 0x7f4894008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:11.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.559+0000 7f48a4ff9640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.112:3300/0 conn(0x7f48940036f0 msgr2=0x7f4894005b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:40:11.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.559+0000 7f48a4ff9640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.112:3300/0 conn(0x7f48940036f0 0x7f4894005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:11.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.559+0000 7f48a77fe640 1 -- 172.21.15.112:0/2905102890 reap_dead start 2024-04-25T01:40:11.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.559+0000 7f48a4ff9640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 0x7f4894009150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:11.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.559+0000 7f48a4ff9640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 0x7f489400f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:11.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.560+0000 7f48a4ff9640 1 -- 172.21.15.112:0/2905102890 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48a00dadd0 con 0x7f4894009690 2024-04-25T01:40:11.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.560+0000 7f48a4ff9640 1 -- 172.21.15.112:0/2905102890 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f48a00daaf0 con 0x7f489400d230 2024-04-25T01:40:11.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.560+0000 7f48a4ff9640 1 -- 172.21.15.112:0/2905102890 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4894008d20 con 0x7f4894003b60 2024-04-25T01:40:11.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.560+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 0x7f4894009150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:11.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.560+0000 7f48a77fe640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 0x7f489400f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:11.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.560+0000 7f48a77fe640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 msgr2=0x7f489400f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:40:11.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.560+0000 7f48a77fe640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 msgr2=0x7f489400f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:40:11.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.560+0000 7f48a77fe640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 0x7f489400f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:40:11.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.560+0000 7f48a77fe640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 0x7f489400f6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:11.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.561+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 msgr2=0x7f4894009150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:40:11.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.561+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 msgr2=0x7f4894009150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:40:11.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.561+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 0x7f4894009150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:40:11.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.561+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 0x7f4894009150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:11.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.761+0000 7f48a77fe640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 0x7f489400f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:11.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.762+0000 7f48a77fe640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 msgr2=0x7f489400f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:40:11.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.762+0000 7f48a77fe640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 msgr2=0x7f489400f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:40:11.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.762+0000 7f48a77fe640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 0x7f489400f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:40:11.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.762+0000 7f48a77fe640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 0x7f489400f6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:40:11.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.762+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 0x7f4894009150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:11.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.762+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 msgr2=0x7f4894009150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:40:11.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.762+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 msgr2=0x7f4894009150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:40:11.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.762+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 0x7f4894009150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:40:11.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:11.762+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 0x7f4894009150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:40:12.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:12.163+0000 7f48a77fe640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 0x7f489400f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:12.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:12.163+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 0x7f4894009150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:12.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:12.163+0000 7f48a77fe640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 msgr2=0x7f489400f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:40:12.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:12.163+0000 7f48a77fe640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 msgr2=0x7f489400f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:40:12.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:12.163+0000 7f48a77fe640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 0x7f489400f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:40:12.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:12.163+0000 7f48a77fe640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 0x7f489400f6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:40:12.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:12.163+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 msgr2=0x7f4894009150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:40:12.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:12.163+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 msgr2=0x7f4894009150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:40:12.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:12.163+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 0x7f4894009150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:40:12.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:12.163+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 0x7f4894009150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:40:12.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:12.964+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 0x7f4894009150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:12.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:12.964+0000 7f48a77fe640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 0x7f489400f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:12.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:12.964+0000 7f48a77fe640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 msgr2=0x7f489400f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:40:12.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:12.964+0000 7f48a77fe640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 msgr2=0x7f489400f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:40:12.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:12.964+0000 7f48a77fe640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 0x7f489400f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:40:12.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:12.964+0000 7f48a77fe640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 0x7f489400f6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:40:12.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:12.966+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 msgr2=0x7f4894009150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:40:12.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:12.966+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 msgr2=0x7f4894009150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:40:12.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:12.966+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 0x7f4894009150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:40:12.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:12.966+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 0x7f4894009150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:40:13.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.657+0000 7f48a5ffb640 1 -- 172.21.15.112:0/2905102890 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1234618962 0 0) 0x7f48a00dadd0 con 0x7f4894009690 2024-04-25T01:40:13.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.657+0000 7f48a5ffb640 1 -- 172.21.15.112:0/2905102890 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4890003520 con 0x7f4894009690 2024-04-25T01:40:13.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.696+0000 7f48a5ffb640 1 -- 172.21.15.112:0/2905102890 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1958411111 0 0) 0x7f4890003520 con 0x7f4894009690 2024-04-25T01:40:13.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.696+0000 7f48a5ffb640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 msgr2=0x7f4894009150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:40:13.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.696+0000 7f48a5ffb640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4894003b60 0x7f4894009150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:13.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.696+0000 7f48a5ffb640 1 -- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 msgr2=0x7f489400f6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:40:13.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.696+0000 7f48a5ffb640 1 --2- 172.21.15.112:0/2905102890 >> v2:172.21.15.187:3300/0 conn(0x7f489400d230 0x7f489400f6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:13.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.696+0000 7f48a5ffb640 1 -- 172.21.15.112:0/2905102890 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a00da530 con 0x7f4894009690 2024-04-25T01:40:13.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.696+0000 7f48a5ffb640 1 -- 172.21.15.112:0/2905102890 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f489c0013d0 con 0x7f4894009690 2024-04-25T01:40:13.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.696+0000 7f48a5ffb640 1 -- 172.21.15.112:0/2905102890 >> v1:172.21.15.187:6789/0 conn(0x7f4894009690 legacy=0x7f489400ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:13.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.696+0000 7f48a5ffb640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48900039b0 0x7f4890003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:13.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.696+0000 7f48a5ffb640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4890004800 0x7f4890004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:13.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.696+0000 7f48a5ffb640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48900051c0 0x7f489000b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:13.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.696+0000 7f48a5ffb640 1 -- 172.21.15.112:0/2905102890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f48a00daaf0 con 0x7f48900039b0 2024-04-25T01:40:13.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.696+0000 7f48a5ffb640 1 -- 172.21.15.112:0/2905102890 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4894008d20 con 0x7f4890004800 2024-04-25T01:40:13.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.696+0000 7f48a5ffb640 1 -- 172.21.15.112:0/2905102890 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f489000c000 con 0x7f48900051c0 2024-04-25T01:40:13.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.697+0000 7f48a77fe640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48900051c0 0x7f489000b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:13.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.697+0000 7f48a6ffd640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48900039b0 msgr2=0x7f4890003dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:40:13.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.697+0000 7f48a77fe640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4890004800 msgr2=0x7f4890004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:13.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.697+0000 7f48a77fe640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4890004800 0x7f4890004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:13.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.697+0000 7f48a77fe640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48900039b0 msgr2=0x7f4890003dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:40:13.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.697+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48900039b0 0x7f4890003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:13.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.697+0000 7f48a77fe640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48900039b0 0x7f4890003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:13.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.697+0000 7f48a77fe640 1 -- 172.21.15.112:0/2905102890 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4890003520 con 0x7f48900051c0 2024-04-25T01:40:13.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.698+0000 7f48a77fe640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48900051c0 0x7f489000b930 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f48a8068950 tx=0x7f4890005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed6227ebd8d113da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:13.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.699+0000 7f48a5ffb640 1 -- 172.21.15.112:0/2905102890 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a80b4070 con 0x7f48900051c0 2024-04-25T01:40:13.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.699+0000 7f48a5ffb640 1 -- 172.21.15.112:0/2905102890 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48a806e040 con 0x7f48900051c0 2024-04-25T01:40:13.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.699+0000 7f48a5ffb640 1 -- 172.21.15.112:0/2905102890 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a806d040 con 0x7f48900051c0 2024-04-25T01:40:13.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.700+0000 7f48a7fff640 1 -- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48900051c0 msgr2=0x7f489000b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:13.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.700+0000 7f48a7fff640 1 --2- 172.21.15.112:0/2905102890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48900051c0 0x7f489000b930 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f48a8068950 tx=0x7f4890005830 comp rx=0 tx=0).stop 2024-04-25T01:40:13.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.700+0000 7f48a77fe640 1 -- 172.21.15.112:0/2905102890 reap_dead start 2024-04-25T01:40:13.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.700+0000 7f48a7fff640 1 -- 172.21.15.112:0/2905102890 shutdown_connections 2024-04-25T01:40:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.700+0000 7f48a7fff640 1 -- 172.21.15.112:0/2905102890 >> 172.21.15.112:0/2905102890 conn(0x7f48a0031720 msgr2=0x7f48a0031b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.700+0000 7f48a7fff640 1 -- 172.21.15.112:0/2905102890 shutdown_connections 2024-04-25T01:40:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.700+0000 7f48a7fff640 1 -- 172.21.15.112:0/2905102890 wait complete. 2024-04-25T01:40:13.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.702+0000 7f48a7fff640 1 Processor -- start 2024-04-25T01:40:13.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.702+0000 7f48a7fff640 1 -- start start 2024-04-25T01:40:13.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.703+0000 7f48a7fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00d1a40 0x7f48a00dbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:13.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.703+0000 7f48a7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48a00de460 0x7f48a00d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.703+0000 7f48a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48a016f610 0x7f48a00cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.703+0000 7f48a7fff640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f48a00355a0 con 0x7f48a016f610 2024-04-25T01:40:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.703+0000 7f48a7fff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f48a0032360 con 0x7f48a00de460 2024-04-25T01:40:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.703+0000 7f48a7fff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f48a0035720 con 0x7f48a00d1a40 2024-04-25T01:40:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.703+0000 7f48a77fe640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48a016f610 msgr2=0x7f48a00cf200 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:40:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.703+0000 7f48a77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48a016f610 0x7f48a00cf200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.703+0000 7f48a6ffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00d1a40 0x7f48a00dbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.703+0000 7f48a6ffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00d1a40 0x7f48a00dbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:46644/0 (socket says 172.21.15.112:46644) 2024-04-25T01:40:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.703+0000 7f48a6ffd640 1 -- 172.21.15.112:0/1352265435 learned_addr learned my addr 172.21.15.112:0/1352265435 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.704+0000 7f48a6ffd640 1 -- 172.21.15.112:0/1352265435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48a00de460 msgr2=0x7f48a00d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.704+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/1352265435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48a00de460 0x7f48a00d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.704+0000 7f48a6ffd640 1 -- 172.21.15.112:0/1352265435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48a016f610 msgr2=0x7f48a00cf200 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:40:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.704+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/1352265435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48a016f610 0x7f48a00cf200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.704+0000 7f48a6ffd640 1 -- 172.21.15.112:0/1352265435 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a0007a70 con 0x7f48a00d1a40 2024-04-25T01:40:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.705+0000 7f48a6ffd640 1 --2- 172.21.15.112:0/1352265435 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00d1a40 0x7f48a00dbbe0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4898016420 tx=0x7f48980168e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.705+0000 7f488f7fe640 1 -- 172.21.15.112:0/1352265435 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489802a070 con 0x7f48a00d1a40 2024-04-25T01:40:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.706+0000 7f48a7fff640 1 -- 172.21.15.112:0/1352265435 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a00cfec0 con 0x7f48a00d1a40 2024-04-25T01:40:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.706+0000 7f488f7fe640 1 -- 172.21.15.112:0/1352265435 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4898004220 con 0x7f48a00d1a40 2024-04-25T01:40:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.706+0000 7f488f7fe640 1 -- 172.21.15.112:0/1352265435 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48980146d0 con 0x7f48a00d1a40 2024-04-25T01:40:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.706+0000 7f48a7fff640 1 -- 172.21.15.112:0/1352265435 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a016ee10 con 0x7f48a00d1a40 2024-04-25T01:40:13.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.708+0000 7f48a7fff640 1 -- 172.21.15.112:0/1352265435 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4888005290 con 0x7f48a00d1a40 2024-04-25T01:40:13.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.708+0000 7f488f7fe640 1 -- 172.21.15.112:0/1352265435 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4898013040 con 0x7f48a00d1a40 2024-04-25T01:40:13.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.708+0000 7f488f7fe640 1 --2- 172.21.15.112:0/1352265435 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f489406d880 0x7f489406fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:13.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.709+0000 7f488f7fe640 1 -- 172.21.15.112:0/1352265435 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f4898095860 con 0x7f48a00d1a40 2024-04-25T01:40:13.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.709+0000 7f48a67fc640 1 --2- 172.21.15.112:0/1352265435 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f489406d880 0x7f489406fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:13.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.709+0000 7f48a67fc640 1 --2- 172.21.15.112:0/1352265435 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f489406d880 0x7f489406fd40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4894003590 tx=0x7f489c0014d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:13.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:13.720+0000 7f488f7fe640 1 -- 172.21.15.112:0/1352265435 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4898063e80 con 0x7f48a00d1a40 2024-04-25T01:40:14.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.222+0000 7f48a7fff640 1 -- 172.21.15.112:0/1352265435 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4888005850 con 0x7f48a00d1a40 2024-04-25T01:40:14.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.224+0000 7f488f7fe640 1 -- 172.21.15.112:0/1352265435 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1563 (secure 0 0 0) 0x7f4898063780 con 0x7f48a00d1a40 2024-04-25T01:40:14.209 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":114,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:40:14.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.228+0000 7f48a7fff640 1 -- 172.21.15.112:0/1352265435 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f489406d880 msgr2=0x7f489406fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:14.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.229+0000 7f48a7fff640 1 --2- 172.21.15.112:0/1352265435 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f489406d880 0x7f489406fd40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4894003590 tx=0x7f489c0014d0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.229+0000 7f48a7fff640 1 -- 172.21.15.112:0/1352265435 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00d1a40 msgr2=0x7f48a00dbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:14.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.229+0000 7f48a7fff640 1 --2- 172.21.15.112:0/1352265435 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00d1a40 0x7f48a00dbbe0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4898016420 tx=0x7f48980168e0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.229+0000 7f48a7fff640 1 -- 172.21.15.112:0/1352265435 shutdown_connections 2024-04-25T01:40:14.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.229+0000 7f48a7fff640 1 --2- 172.21.15.112:0/1352265435 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f489406d880 0x7f489406fd40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.229+0000 7f48a7fff640 1 --2- 172.21.15.112:0/1352265435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48a016f610 0x7f48a00cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.229+0000 7f48a7fff640 1 --2- 172.21.15.112:0/1352265435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48a00de460 0x7f48a00d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.229+0000 7f48a7fff640 1 --2- 172.21.15.112:0/1352265435 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48a00d1a40 0x7f48a00dbbe0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.229+0000 7f48a7fff640 1 -- 172.21.15.112:0/1352265435 >> 172.21.15.112:0/1352265435 conn(0x7f48a0031720 msgr2=0x7f48a00d8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:14.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.230+0000 7f48a7fff640 1 -- 172.21.15.112:0/1352265435 shutdown_connections 2024-04-25T01:40:14.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.230+0000 7f48a7fff640 1 -- 172.21.15.112:0/1352265435 wait complete. 2024-04-25T01:40:14.236 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T01:40:14.237 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-04-25T01:40:14.237 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:40:14.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.432+0000 7fd13f71b640 1 Processor -- start 2024-04-25T01:40:14.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.433+0000 7fd13f71b640 1 -- start start 2024-04-25T01:40:14.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.434+0000 7fd13f71b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd1380c2190 0x7fd13814e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:14.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.434+0000 7fd13f71b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd138155c70 con 0x7fd13806f510 2024-04-25T01:40:14.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.434+0000 7fd13f71b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1380ac450 con 0x7fd1380730b0 2024-04-25T01:40:14.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.434+0000 7fd13f71b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd1380ac200 con 0x7fd1380c2190 2024-04-25T01:40:14.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.436+0000 7fd13ccb6640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fd1380730b0 0x7fd1380c1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:57804/0 (socket says 172.21.15.112:57804) 2024-04-25T01:40:14.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.436+0000 7fd13ccb6640 1 -- 172.21.15.112:0/939833646 learned_addr learned my addr 172.21.15.112:0/939833646 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:40:14.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.436+0000 7fd13dcb8640 1 --2- 172.21.15.112:0/939833646 >> v2:172.21.15.187:3300/0 conn(0x7fd1380c2190 0x7fd13814e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:14.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.436+0000 7fd13d4b7640 1 -- 172.21.15.112:0/939833646 >> v1:172.21.15.112:6789/0 conn(0x7fd13806f510 legacy=0x7fd138071170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:40:14.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.437+0000 7fd12ffff640 1 -- 172.21.15.112:0/939833646 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2961241300 0 0) 0x7fd1380ac450 con 0x7fd1380730b0 2024-04-25T01:40:14.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.437+0000 7fd12ffff640 1 -- 172.21.15.112:0/939833646 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd124003520 con 0x7fd1380730b0 2024-04-25T01:40:14.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.438+0000 7fd12ffff640 1 -- 172.21.15.112:0/939833646 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3707050715 0 0) 0x7fd124003520 con 0x7fd1380730b0 2024-04-25T01:40:14.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.438+0000 7fd12ffff640 1 -- 172.21.15.112:0/939833646 >> v2:172.21.15.187:3300/0 conn(0x7fd1380c2190 msgr2=0x7fd13814e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:14.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.438+0000 7fd12ffff640 1 --2- 172.21.15.112:0/939833646 >> v2:172.21.15.187:3300/0 conn(0x7fd1380c2190 0x7fd13814e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.438+0000 7fd12ffff640 1 -- 172.21.15.112:0/939833646 >> v1:172.21.15.112:6789/0 conn(0x7fd13806f510 legacy=0x7fd138071170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:40:14.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.438+0000 7fd12ffff640 1 -- 172.21.15.112:0/939833646 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd138154eb0 con 0x7fd1380730b0 2024-04-25T01:40:14.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.438+0000 7fd12ffff640 1 -- 172.21.15.112:0/939833646 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fd128001670 con 0x7fd1380730b0 2024-04-25T01:40:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.438+0000 7fd12ffff640 1 -- 172.21.15.112:0/939833646 >> v1:172.21.15.187:6789/0 conn(0x7fd1380730b0 legacy=0x7fd1380c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.438+0000 7fd12ffff640 1 --2- 172.21.15.112:0/939833646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1240039b0 0x7fd124003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.439+0000 7fd13d4b7640 1 -- 172.21.15.112:0/939833646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1240039b0 msgr2=0x7fd124003db0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:40:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.439+0000 7fd13d4b7640 1 --2- 172.21.15.112:0/939833646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1240039b0 0x7fd124003db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.439+0000 7fd12ffff640 1 --2- 172.21.15.112:0/939833646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd124004800 0x7fd124004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.439+0000 7fd12ffff640 1 --2- 172.21.15.112:0/939833646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1240052e0 0x7fd12400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.439+0000 7fd12ffff640 1 -- 172.21.15.112:0/939833646 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd1380ac200 con 0x7fd1240039b0 2024-04-25T01:40:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.439+0000 7fd12ffff640 1 -- 172.21.15.112:0/939833646 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd12400c190 con 0x7fd1240052e0 2024-04-25T01:40:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.439+0000 7fd12ffff640 1 -- 172.21.15.112:0/939833646 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd12400c390 con 0x7fd124004800 2024-04-25T01:40:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.439+0000 7fd13ccb6640 1 --2- 172.21.15.112:0/939833646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd124004800 0x7fd124004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:14.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.441+0000 7fd13ccb6640 1 -- 172.21.15.112:0/939833646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1240052e0 msgr2=0x7fd12400b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:40:14.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.441+0000 7fd13ccb6640 1 --2- 172.21.15.112:0/939833646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1240052e0 0x7fd12400b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.441+0000 7fd13ccb6640 1 -- 172.21.15.112:0/939833646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1240039b0 msgr2=0x7fd124003db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:40:14.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.441+0000 7fd13ccb6640 1 --2- 172.21.15.112:0/939833646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1240039b0 0x7fd124003db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.441+0000 7fd13ccb6640 1 -- 172.21.15.112:0/939833646 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd124003520 con 0x7fd124004800 2024-04-25T01:40:14.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.442+0000 7fd13dcb8640 1 -- 172.21.15.112:0/939833646 reap_dead start 2024-04-25T01:40:14.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.443+0000 7fd13ccb6640 1 --2- 172.21.15.112:0/939833646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd124004800 0x7fd124004c80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd138073520 tx=0x7fd124005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35b96e9d6b354c6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:14.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.443+0000 7fd12ffff640 1 -- 172.21.15.112:0/939833646 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128037070 con 0x7fd124004800 2024-04-25T01:40:14.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.443+0000 7fd12ffff640 1 -- 172.21.15.112:0/939833646 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd128009040 con 0x7fd124004800 2024-04-25T01:40:14.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.443+0000 7fd12ffff640 1 -- 172.21.15.112:0/939833646 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12800b610 con 0x7fd124004800 2024-04-25T01:40:14.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.445+0000 7fd13f71b640 1 -- 172.21.15.112:0/939833646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd124004800 msgr2=0x7fd124004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:14.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.445+0000 7fd13f71b640 1 --2- 172.21.15.112:0/939833646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd124004800 0x7fd124004c80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd138073520 tx=0x7fd124005830 comp rx=0 tx=0).stop 2024-04-25T01:40:14.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.446+0000 7fd13f71b640 1 -- 172.21.15.112:0/939833646 shutdown_connections 2024-04-25T01:40:14.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.446+0000 7fd13f71b640 1 --2- 172.21.15.112:0/939833646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd124004800 0x7fd124004c80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.446+0000 7fd13f71b640 1 -- 172.21.15.112:0/939833646 >> 172.21.15.112:0/939833646 conn(0x7fd1380c7080 msgr2=0x7fd1380c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:14.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.448+0000 7fd13f71b640 1 -- 172.21.15.112:0/939833646 shutdown_connections 2024-04-25T01:40:14.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.448+0000 7fd13f71b640 1 -- 172.21.15.112:0/939833646 wait complete. 2024-04-25T01:40:14.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.450+0000 7fd13f71b640 1 Processor -- start 2024-04-25T01:40:14.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.450+0000 7fd13f71b640 1 -- start start 2024-04-25T01:40:14.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.451+0000 7fd13f71b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd124004800 0x7fd1381eb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:14.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.451+0000 7fd13f71b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd138149da0 0x7fd1381ee320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:14.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.451+0000 7fd13f71b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd13806d960 0x7fd13806dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:14.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.451+0000 7fd13f71b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd138155470 con 0x7fd13806d960 2024-04-25T01:40:14.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.451+0000 7fd13f71b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd1381557e0 con 0x7fd124004800 2024-04-25T01:40:14.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.451+0000 7fd13f71b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd1380ab5a0 con 0x7fd138149da0 2024-04-25T01:40:14.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.451+0000 7fd13dcb8640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd13806d960 msgr2=0x7fd13806dde0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:40:14.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.451+0000 7fd13dcb8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd13806d960 0x7fd13806dde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.453+0000 7fd13ccb6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd138149da0 0x7fd1381ee320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.453+0000 7fd13ccb6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd138149da0 0x7fd1381ee320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:46662/0 (socket says 172.21.15.112:46662) 2024-04-25T01:40:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.453+0000 7fd13ccb6640 1 -- 172.21.15.112:0/2876574919 learned_addr learned my addr 172.21.15.112:0/2876574919 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:14.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.455+0000 7fd13d4b7640 1 --2- 172.21.15.112:0/2876574919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd124004800 0x7fd1381eb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:14.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.456+0000 7fd13ccb6640 1 -- 172.21.15.112:0/2876574919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd124004800 msgr2=0x7fd1381eb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:14.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.456+0000 7fd13ccb6640 1 --2- 172.21.15.112:0/2876574919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd124004800 0x7fd1381eb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.456+0000 7fd13ccb6640 1 -- 172.21.15.112:0/2876574919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd13806d960 msgr2=0x7fd13806dde0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:40:14.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.456+0000 7fd13ccb6640 1 --2- 172.21.15.112:0/2876574919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd13806d960 0x7fd13806dde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.456+0000 7fd13ccb6640 1 -- 172.21.15.112:0/2876574919 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1381ead70 con 0x7fd138149da0 2024-04-25T01:40:14.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.457+0000 7fd13ccb6640 1 --2- 172.21.15.112:0/2876574919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd138149da0 0x7fd1381ee320 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd12800ba80 tx=0x7fd1380c3aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:14.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.460+0000 7fd12dffb640 1 -- 172.21.15.112:0/2876574919 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128037070 con 0x7fd138149da0 2024-04-25T01:40:14.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.460+0000 7fd13f71b640 1 -- 172.21.15.112:0/2876574919 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd138149b50 con 0x7fd138149da0 2024-04-25T01:40:14.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.461+0000 7fd13f71b640 1 -- 172.21.15.112:0/2876574919 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd13815a120 con 0x7fd138149da0 2024-04-25T01:40:14.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.461+0000 7fd12dffb640 1 -- 172.21.15.112:0/2876574919 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd128004bf0 con 0x7fd138149da0 2024-04-25T01:40:14.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.461+0000 7fd12dffb640 1 -- 172.21.15.112:0/2876574919 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128032420 con 0x7fd138149da0 2024-04-25T01:40:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.462+0000 7fd13f71b640 1 --2- 172.21.15.112:0/2876574919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1381eb3a0 0x7fd138151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.462+0000 7fd13f71b640 1 -- 172.21.15.112:0/2876574919 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd138159bf0 con 0x7fd1381eb3a0 2024-04-25T01:40:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.463+0000 7fd12dffb640 1 -- 172.21.15.112:0/2876574919 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd128047400 con 0x7fd138149da0 2024-04-25T01:40:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.463+0000 7fd13d4b7640 1 --2- 172.21.15.112:0/2876574919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1381eb3a0 0x7fd138151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.464+0000 7fd12dffb640 1 --2- 172.21.15.112:0/2876574919 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd11c06be60 0x7fd11c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.464+0000 7fd12dffb640 1 -- 172.21.15.112:0/2876574919 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fd1280aba40 con 0x7fd138149da0 2024-04-25T01:40:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.464+0000 7fd13dcb8640 1 --2- 172.21.15.112:0/2876574919 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd11c06be60 0x7fd11c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.465+0000 7fd13d4b7640 1 -- 172.21.15.112:0/2876574919 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd138075980 con 0x7fd1381eb3a0 2024-04-25T01:40:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.466+0000 7fd13d4b7640 1 --2- 172.21.15.112:0/2876574919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1381eb3a0 0x7fd138151360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd13400d700 tx=0x7fd13400dbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.467+0000 7fd12dffb640 1 -- 172.21.15.112:0/2876574919 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1340039b0 con 0x7fd1381eb3a0 2024-04-25T01:40:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.468+0000 7fd13dcb8640 1 --2- 172.21.15.112:0/2876574919 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd11c06be60 0x7fd11c06e320 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd138073490 tx=0x7fd130009830 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.468+0000 7fd12dffb640 1 -- 172.21.15.112:0/2876574919 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd1340039b0 con 0x7fd1381eb3a0 2024-04-25T01:40:14.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.468+0000 7fd12dffb640 1 -- 172.21.15.112:0/2876574919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1381eb3a0 msgr2=0x7fd138151360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:14.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.469+0000 7fd12dffb640 1 --2- 172.21.15.112:0/2876574919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1381eb3a0 0x7fd138151360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd13400d700 tx=0x7fd13400dbc0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.469+0000 7fd12dffb640 1 -- 172.21.15.112:0/2876574919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1381eb3a0 msgr2=0x7fd138151360 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:40:14.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.469+0000 7fd12dffb640 1 --2- 172.21.15.112:0/2876574919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1381eb3a0 0x7fd138151360 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd13400d700 tx=0x7fd13400dbc0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.486+0000 7fd13f71b640 1 --2- 172.21.15.112:0/2876574919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1381eaf90 0x7fd138158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:14.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.487+0000 7fd13f71b640 1 -- 172.21.15.112:0/2876574919 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd1380aa710 con 0x7fd1381eaf90 2024-04-25T01:40:14.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.489+0000 7fd13d4b7640 1 --2- 172.21.15.112:0/2876574919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1381eaf90 0x7fd138158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:14.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.492+0000 7fd13d4b7640 1 -- 172.21.15.112:0/2876574919 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd13814d4c0 con 0x7fd1381eaf90 2024-04-25T01:40:14.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.493+0000 7fd13d4b7640 1 --2- 172.21.15.112:0/2876574919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1381eaf90 0x7fd138158a30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd134000c00 tx=0x7fd1340059d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:14.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.493+0000 7fd12dffb640 1 -- 172.21.15.112:0/2876574919 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13400bb40 con 0x7fd1381eaf90 2024-04-25T01:40:14.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.494+0000 7fd12dffb640 1 -- 172.21.15.112:0/2876574919 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4721 (secure 0 0 0) 0x7fd13400bce0 con 0x7fd1381eaf90 2024-04-25T01:40:14.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.494+0000 7fd12dffb640 1 -- 172.21.15.112:0/2876574919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1381eaf90 msgr2=0x7fd138158a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:14.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.494+0000 7fd12dffb640 1 --2- 172.21.15.112:0/2876574919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1381eaf90 0x7fd138158a30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd134000c00 tx=0x7fd1340059d0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.494+0000 7fd12dffb640 1 -- 172.21.15.112:0/2876574919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1381eaf90 msgr2=0x7fd138158a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:40:14.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.494+0000 7fd12dffb640 1 --2- 172.21.15.112:0/2876574919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1381eaf90 0x7fd138158a30 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd134000c00 tx=0x7fd1340059d0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.479 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:40:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:40:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:40:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:40:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 114, 2024-04-25T01:40:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 119258, 2024-04-25T01:40:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:40:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:40:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:40:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 0, 2024-04-25T01:40:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:40:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:40:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:40:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:40:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:40:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:40:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:40:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:40:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:40:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:40:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:40:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:40:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:40:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:40:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:40:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:40:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:40:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:40:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:40:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:40:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:40:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:40:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:40:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:40:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:40:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:40:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:40:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:40:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:40:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:40:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:40:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:40:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:40:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:40:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:40:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:40:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:40:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:40:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:40:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:40:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:40:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:40:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:40:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:40:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:40:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:40:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:40:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:40:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:40:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:40:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:40:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:40:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:40:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:40:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:40:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:40:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:40:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:40:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:40:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:40:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:40:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:40:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:40:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:40:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:40:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:40:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:40:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:40:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:40:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:40:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:40:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:40:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:40:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:40:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:40:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:40:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:40:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:40:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:40:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:40:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:40:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:40:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:40:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:40:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:40:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:40:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:40:14.491 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.491 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:40:14.491 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:40:14.491 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:14.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.491 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.491 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:40:14.491 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:40:14.491 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:14.491 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:14.491 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:14.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.491 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:40:14.491 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:40:14.492 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:40:14.492 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:40:14.492 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:40:14.492 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:14.492 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:14.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.492 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:40:14.492 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:40:14.492 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.492 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:40:14.492 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:40:14.492 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:40:14.492 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:14.492 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:14.492 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:40:14.493 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.493 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:40:14.493 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:40:14.493 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:40:14.493 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:14.493 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:14.493 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:40:14.493 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.493 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:40:14.493 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:40:14.493 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:40:14.493 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:14.493 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:14.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.494 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:40:14.494 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:40:14.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.495+0000 7fd13f71b640 1 -- 172.21.15.112:0/2876574919 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd11c06be60 msgr2=0x7fd11c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:14.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.495+0000 7fd13f71b640 1 --2- 172.21.15.112:0/2876574919 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd11c06be60 0x7fd11c06e320 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd138073490 tx=0x7fd130009830 comp rx=0 tx=0).stop 2024-04-25T01:40:14.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.495+0000 7fd13f71b640 1 -- 172.21.15.112:0/2876574919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd138149da0 msgr2=0x7fd1381ee320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:14.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.495+0000 7fd13f71b640 1 --2- 172.21.15.112:0/2876574919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd138149da0 0x7fd1381ee320 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd12800ba80 tx=0x7fd1380c3aa0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.497+0000 7fd13dcb8640 1 -- 172.21.15.112:0/2876574919 reap_dead start 2024-04-25T01:40:14.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.497+0000 7fd13dcb8640 1 -- 172.21.15.112:0/2876574919 reap_dead start 2024-04-25T01:40:14.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.497+0000 7fd13f71b640 1 -- 172.21.15.112:0/2876574919 shutdown_connections 2024-04-25T01:40:14.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.497+0000 7fd13f71b640 1 -- 172.21.15.112:0/2876574919 >> 172.21.15.112:0/2876574919 conn(0x7fd1380c7080 msgr2=0x7fd13814dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:14.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.498+0000 7fd13f71b640 1 -- 172.21.15.112:0/2876574919 shutdown_connections 2024-04-25T01:40:14.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.498+0000 7fd13f71b640 1 -- 172.21.15.112:0/2876574919 wait complete. 2024-04-25T01:40:14.522 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:40:14.523 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-04-25T01:40:14.523 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:40:14.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.742+0000 7f7d10ab9640 1 Processor -- start 2024-04-25T01:40:14.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.742+0000 7f7d10ab9640 1 -- start start 2024-04-25T01:40:14.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.744+0000 7f7d10ab9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04006b70 0x7f7d040db790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:14.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.744+0000 7f7d10ab9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7d040dbcd0 0x7f7d040e2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:14.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.744+0000 7f7d10ab9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d040072c0 con 0x7f7d0404a980 2024-04-25T01:40:14.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.744+0000 7f7d10ab9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f7d040dc180 con 0x7f7d040dbcd0 2024-04-25T01:40:14.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.744+0000 7f7d10ab9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7d04030ad0 con 0x7f7d04006b70 2024-04-25T01:40:14.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.744+0000 7f7d0b7fe640 1 -- >> v1:172.21.15.112:6789/0 conn(0x7f7d0404a980 legacy=0x7f7d040d5580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.112:6789/0 2024-04-25T01:40:14.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.744+0000 7f7d0bfff640 1 -- >> v2:172.21.15.112:3300/0 conn(0x7f7d040dbcd0 msgr2=0x7f7d040e2b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:40:14.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.744+0000 7f7d0bfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7d040dbcd0 0x7f7d040e2b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:14.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.746+0000 7f7d0affd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04006b70 0x7f7d040db790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:14.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.746+0000 7f7d0affd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04006b70 0x7f7d040db790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:46676/0 (socket says 172.21.15.112:46676) 2024-04-25T01:40:14.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.746+0000 7f7d0affd640 1 -- 172.21.15.112:0/4243186743 learned_addr learned my addr 172.21.15.112:0/4243186743 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:14.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.748+0000 7f7d0affd640 1 -- 172.21.15.112:0/4243186743 >> v2:172.21.15.112:3300/0 conn(0x7f7d040dbcd0 msgr2=0x7f7d040e2b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:40:14.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.748+0000 7f7d0affd640 1 --2- 172.21.15.112:0/4243186743 >> v2:172.21.15.112:3300/0 conn(0x7f7d040dbcd0 0x7f7d040e2b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.748+0000 7f7d0affd640 1 -- 172.21.15.112:0/4243186743 >> v1:172.21.15.112:6789/0 conn(0x7f7d0404a980 legacy=0x7f7d040d5580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:40:14.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.748+0000 7f7d0affd640 1 -- 172.21.15.112:0/4243186743 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d04030db0 con 0x7f7d04006b70 2024-04-25T01:40:14.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.749+0000 7f7d0affd640 1 --2- 172.21.15.112:0/4243186743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04006b70 0x7f7d040db790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009870 tx=0x7f7cf802a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fc88e0cb0aacd8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:14.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.751+0000 7f7d0a7fc640 1 -- 172.21.15.112:0/4243186743 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8037070 con 0x7f7d04006b70 2024-04-25T01:40:14.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.751+0000 7f7d0a7fc640 1 -- 172.21.15.112:0/4243186743 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf8032440 con 0x7f7d04006b70 2024-04-25T01:40:14.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.751+0000 7f7d0a7fc640 1 -- 172.21.15.112:0/4243186743 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8032810 con 0x7f7d04006b70 2024-04-25T01:40:14.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.752+0000 7f7d10ab9640 1 -- 172.21.15.112:0/4243186743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04006b70 msgr2=0x7f7d040db790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:14.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.752+0000 7f7d10ab9640 1 --2- 172.21.15.112:0/4243186743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04006b70 0x7f7d040db790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009870 tx=0x7f7cf802a2c0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.752+0000 7f7d10ab9640 1 -- 172.21.15.112:0/4243186743 shutdown_connections 2024-04-25T01:40:14.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.752+0000 7f7d10ab9640 1 --2- 172.21.15.112:0/4243186743 >> v2:172.21.15.112:3300/0 conn(0x7f7d040dbcd0 0x7f7d040e2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.752+0000 7f7d10ab9640 1 --2- 172.21.15.112:0/4243186743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04006b70 0x7f7d040db790 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.752+0000 7f7d10ab9640 1 -- 172.21.15.112:0/4243186743 >> 172.21.15.112:0/4243186743 conn(0x7f7d04005e90 msgr2=0x7f7d040062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:14.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.754+0000 7f7d10ab9640 1 -- 172.21.15.112:0/4243186743 shutdown_connections 2024-04-25T01:40:14.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.758+0000 7f7d10ab9640 1 -- 172.21.15.112:0/4243186743 wait complete. 2024-04-25T01:40:14.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.760+0000 7f7d10ab9640 1 Processor -- start 2024-04-25T01:40:14.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.760+0000 7f7d10ab9640 1 -- start start 2024-04-25T01:40:14.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.761+0000 7f7d10ab9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04006b70 0x7f7d041749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:14.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.761+0000 7f7d10ab9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d0404a980 0x7f7d04177d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:14.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.761+0000 7f7d10ab9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d040dbcd0 0x7f7d040da960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:14.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.761+0000 7f7d10ab9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d04009680 con 0x7f7d0404a980 2024-04-25T01:40:14.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.761+0000 7f7d10ab9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7d04006f90 con 0x7f7d040dbcd0 2024-04-25T01:40:14.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.761+0000 7f7d10ab9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7d040315b0 con 0x7f7d04006b70 2024-04-25T01:40:14.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.762+0000 7f7d0affd640 1 -- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d0404a980 msgr2=0x7f7d04177d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:3300/0 2024-04-25T01:40:14.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.762+0000 7f7d0affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d0404a980 0x7f7d04177d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:14.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.764+0000 7f7d0b7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04006b70 0x7f7d041749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:14.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.765+0000 7f7d0b7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04006b70 0x7f7d041749a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:46686/0 (socket says 172.21.15.112:46686) 2024-04-25T01:40:14.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.765+0000 7f7d0b7fe640 1 -- 172.21.15.112:0/1634194508 learned_addr learned my addr 172.21.15.112:0/1634194508 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:14.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.765+0000 7f7d0bfff640 1 --2- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d040dbcd0 0x7f7d040da960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:14.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.768+0000 7f7d0b7fe640 1 -- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d040dbcd0 msgr2=0x7f7d040da960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:14.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.768+0000 7f7d0b7fe640 1 --2- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d040dbcd0 0x7f7d040da960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.768+0000 7f7d0b7fe640 1 -- 172.21.15.112:0/1634194508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d0404a980 msgr2=0x7f7d04177d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:40:14.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.768+0000 7f7d0b7fe640 1 --2- 172.21.15.112:0/1634194508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d0404a980 0x7f7d04177d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.768+0000 7f7d0b7fe640 1 -- 172.21.15.112:0/1634194508 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d04173f00 con 0x7f7d04006b70 2024-04-25T01:40:14.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.768+0000 7f7d0b7fe640 1 --2- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04006b70 0x7f7d041749a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c02d700 tx=0x7f7d0c068d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:14.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.769+0000 7f7cf7fff640 1 -- 172.21.15.112:0/1634194508 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c070800 con 0x7f7d04006b70 2024-04-25T01:40:14.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.769+0000 7f7cf7fff640 1 -- 172.21.15.112:0/1634194508 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d0c06e040 con 0x7f7d04006b70 2024-04-25T01:40:14.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.769+0000 7f7cf7fff640 1 -- 172.21.15.112:0/1634194508 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c070ca0 con 0x7f7d04006b70 2024-04-25T01:40:14.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.770+0000 7f7d10ab9640 1 -- 172.21.15.112:0/1634194508 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d040d0040 con 0x7f7d04006b70 2024-04-25T01:40:14.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.770+0000 7f7d10ab9640 1 -- 172.21.15.112:0/1634194508 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d040daea0 con 0x7f7d04006b70 2024-04-25T01:40:14.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.771+0000 7f7d10ab9640 1 --2- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d040d74d0 0x7f7d040d78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:14.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.771+0000 7f7d10ab9640 1 -- 172.21.15.112:0/1634194508 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7d041782b0 con 0x7f7d040d74d0 2024-04-25T01:40:14.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.772+0000 7f7d0affd640 1 --2- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d040d74d0 0x7f7d040d78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:14.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.772+0000 7f7d0affd640 1 -- 172.21.15.112:0/1634194508 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7d04005390 con 0x7f7d040d74d0 2024-04-25T01:40:14.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.772+0000 7f7d0affd640 1 --2- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d040d74d0 0x7f7d040d78b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009930 tx=0x7f7cf80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:14.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.773+0000 7f7cf7fff640 1 -- 172.21.15.112:0/1634194508 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8037040 con 0x7f7d040d74d0 2024-04-25T01:40:14.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.774+0000 7f7cf7fff640 1 -- 172.21.15.112:0/1634194508 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7cf8004030 con 0x7f7d040d74d0 2024-04-25T01:40:14.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.774+0000 7f7cf7fff640 1 -- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d040d74d0 msgr2=0x7f7d040d78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:14.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.774+0000 7f7cf7fff640 1 --2- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d040d74d0 0x7f7d040d78b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009930 tx=0x7f7cf80046c0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.774+0000 7f7cf7fff640 1 -- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d040d74d0 msgr2=0x7f7d040d78b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:40:14.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.774+0000 7f7cf7fff640 1 --2- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d040d74d0 0x7f7d040d78b0 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009930 tx=0x7f7cf80046c0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.774+0000 7f7cf7fff640 1 -- 172.21.15.112:0/1634194508 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d0c095430 con 0x7f7d04006b70 2024-04-25T01:40:14.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.775+0000 7f7cf7fff640 1 --2- 172.21.15.112:0/1634194508 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7cfc06b8f0 0x7f7cfc06ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:14.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.775+0000 7f7cf7fff640 1 -- 172.21.15.112:0/1634194508 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f7d0c106e10 con 0x7f7d04006b70 2024-04-25T01:40:14.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.775+0000 7f7d0affd640 1 --2- 172.21.15.112:0/1634194508 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7cfc06b8f0 0x7f7cfc06ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:14.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.776+0000 7f7d0affd640 1 --2- 172.21.15.112:0/1634194508 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7cfc06b8f0 0x7f7cfc06ddb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8004000 tx=0x7f7cf804b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:14.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.790+0000 7f7d10ab9640 1 --2- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04003190 0x7f7d040cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:14.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.790+0000 7f7d10ab9640 1 -- 172.21.15.112:0/1634194508 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7d04049a20 con 0x7f7d04003190 2024-04-25T01:40:14.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.792+0000 7f7d0bfff640 1 --2- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04003190 0x7f7d040cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:14.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.793+0000 7f7d0bfff640 1 -- 172.21.15.112:0/1634194508 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7d040d0320 con 0x7f7d04003190 2024-04-25T01:40:14.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.796+0000 7f7d0bfff640 1 --2- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04003190 0x7f7d040cca80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7d00003980 tx=0x7f7d0000c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:14.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.796+0000 7f7cf7fff640 1 -- 172.21.15.112:0/1634194508 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00003dc0 con 0x7f7d04003190 2024-04-25T01:40:14.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.799+0000 7f7cf7fff640 1 -- 172.21.15.112:0/1634194508 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5059 (secure 0 0 0) 0x7f7d00003dc0 con 0x7f7d04003190 2024-04-25T01:40:14.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.799+0000 7f7cf7fff640 1 -- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04003190 msgr2=0x7f7d040cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:14.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.799+0000 7f7cf7fff640 1 --2- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04003190 0x7f7d040cca80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7d00003980 tx=0x7f7d0000c810 comp rx=0 tx=0).stop 2024-04-25T01:40:14.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.799+0000 7f7cf7fff640 1 -- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04003190 msgr2=0x7f7d040cca80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:40:14.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.799+0000 7f7cf7fff640 1 --2- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04003190 0x7f7d040cca80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.787 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:40:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:40:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:40:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:40:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 114, 2024-04-25T01:40:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 184161, 2024-04-25T01:40:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:40:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:40:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:40:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:40:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:40:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:40:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:40:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:40:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:40:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:40:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:40:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:40:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:40:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:40:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:40:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:40:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:40:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:40:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:40:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:40:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:40:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:40:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:40:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:40:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:40:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:40:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:40:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:40:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:40:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:40:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:40:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:40:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:40:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:40:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:40:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:40:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:40:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:40:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:40:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:40:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:40:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:40:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:40:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:40:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:40:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:40:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:40:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:40:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:40:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:40:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:40:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:40:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:40:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:40:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:40:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:40:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:40:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:40:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:40:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:40:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:40:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:40:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:40:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:40:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:40:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:40:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:40:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:40:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:40:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:40:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:40:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:40:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:40:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:40:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:40:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:40:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:40:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:40:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:40:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:40:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:40:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:40:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:40:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:40:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:40:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:40:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:40:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:40:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:40:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:40:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:40:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:40:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:40:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:40:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:40:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:40:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:40:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:40:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:40:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:40:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:40:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:40:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:40:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:40:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:40:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:40:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:40:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:40:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:40:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:40:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:40:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:40:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 8 2024-04-25T01:40:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:40:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:40:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:40:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 6 2024-04-25T01:40:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:40:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:40:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:40:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:40:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:40:14.800 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:40:14.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.809+0000 7f7d10ab9640 1 -- 172.21.15.112:0/1634194508 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7cfc06b8f0 msgr2=0x7f7cfc06ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:14.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.809+0000 7f7d10ab9640 1 --2- 172.21.15.112:0/1634194508 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7cfc06b8f0 0x7f7cfc06ddb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8004000 tx=0x7f7cf804b040 comp rx=0 tx=0).stop 2024-04-25T01:40:14.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.809+0000 7f7d0bfff640 1 -- 172.21.15.112:0/1634194508 reap_dead start 2024-04-25T01:40:14.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.809+0000 7f7d10ab9640 1 -- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04006b70 msgr2=0x7f7d041749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:14.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.809+0000 7f7d10ab9640 1 --2- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04006b70 0x7f7d041749a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c02d700 tx=0x7f7d0c068d80 comp rx=0 tx=0).stop 2024-04-25T01:40:14.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.811+0000 7f7d10ab9640 1 -- 172.21.15.112:0/1634194508 shutdown_connections 2024-04-25T01:40:14.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.811+0000 7f7d10ab9640 1 --2- 172.21.15.112:0/1634194508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d04006b70 0x7f7d041749a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:14.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.811+0000 7f7d10ab9640 1 -- 172.21.15.112:0/1634194508 >> 172.21.15.112:0/1634194508 conn(0x7f7d04005e90 msgr2=0x7f7d040cee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:14.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.813+0000 7f7d10ab9640 1 -- 172.21.15.112:0/1634194508 shutdown_connections 2024-04-25T01:40:14.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:14.813+0000 7f7d10ab9640 1 -- 172.21.15.112:0/1634194508 wait complete. 2024-04-25T01:40:14.819 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T01:40:14.819 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T01:40:25.612 INFO:tasks.workunit.client.1.smithi187.stdout:Syncing()...3 sec 2024-04-25T01:40:25.613 INFO:tasks.workunit.client.1.smithi187.stdout:FFSB benchmark finished at: Thu Apr 25 01:40:25 2024 2024-04-25T01:40:25.613 INFO:tasks.workunit.client.1.smithi187.stdout: 2024-04-25T01:40:25.613 INFO:tasks.workunit.client.1.smithi187.stdout:Results: 2024-04-25T01:40:25.613 INFO:tasks.workunit.client.1.smithi187.stdout:Benchmark took 311.95 sec 2024-04-25T01:40:25.613 INFO:tasks.workunit.client.1.smithi187.stdout: 2024-04-25T01:40:25.613 INFO:tasks.workunit.client.1.smithi187.stdout:Total Results 2024-04-25T01:40:25.613 INFO:tasks.workunit.client.1.smithi187.stdout:=============== 2024-04-25T01:40:25.613 INFO:tasks.workunit.client.1.smithi187.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-04-25T01:40:25.613 INFO:tasks.workunit.client.1.smithi187.stdout: ======= ============ ========= ======= =========== ========== 2024-04-25T01:40:25.613 INFO:tasks.workunit.client.1.smithi187.stdout: write : 1541120 4940.32 100.000% 100.000% 19.3MB/sec 2024-04-25T01:40:25.613 INFO:tasks.workunit.client.1.smithi187.stdout:- 2024-04-25T01:40:25.614 INFO:tasks.workunit.client.1.smithi187.stdout:4940.32 Transactions per Second 2024-04-25T01:40:25.614 INFO:tasks.workunit.client.1.smithi187.stdout: 2024-04-25T01:40:25.614 INFO:tasks.workunit.client.1.smithi187.stdout:Throughput Results 2024-04-25T01:40:25.614 INFO:tasks.workunit.client.1.smithi187.stdout:=================== 2024-04-25T01:40:25.614 INFO:tasks.workunit.client.1.smithi187.stdout:Write Throughput: 19.3MB/sec 2024-04-25T01:40:25.621 INFO:tasks.workunit.client.1.smithi187.stdout: 2024-04-25T01:40:25.621 INFO:tasks.workunit.client.1.smithi187.stdout:System Call Latency statistics in millisecs 2024-04-25T01:40:25.622 INFO:tasks.workunit.client.1.smithi187.stdout:===== 2024-04-25T01:40:25.622 INFO:tasks.workunit.client.1.smithi187.stdout: Min Avg Max Total Calls 2024-04-25T01:40:25.622 INFO:tasks.workunit.client.1.smithi187.stdout: ======== ======== ======== ============ 2024-04-25T01:40:25.622 INFO:tasks.workunit.client.1.smithi187.stdout:[ open] 0.438000 224.928634 8925.208984 1204 2024-04-25T01:40:25.622 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[0] 5000.000000 - 10000.000000 : 26 2024-04-25T01:40:25.622 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[1] 2000.000000 - 5000.000000 : 14 2024-04-25T01:40:25.622 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[2] 1000.000000 - 2000.000000 : 8 2024-04-25T01:40:25.622 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[3] 500.000000 - 1000.000000 : 2 2024-04-25T01:40:25.622 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[4] 200.000000 - 500.000000 : 4 2024-04-25T01:40:25.623 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[5] 100.000000 - 200.000000 : 1 2024-04-25T01:40:25.623 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[6] 50.000000 - 100.000000 : 39 2024-04-25T01:40:25.623 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[7] 20.000000 - 50.000000 : 429 2024-04-25T01:40:25.623 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[8] 10.000000 - 20.000000 : 455 2024-04-25T01:40:25.623 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[9] 5.000000 - 10.000000 : 146 2024-04-25T01:40:25.623 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[10] 2.000000 - 5.000000 : 55 2024-04-25T01:40:25.623 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[11] 1.000000 - 2.000000 : 20 2024-04-25T01:40:25.623 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[12] 0.500000 - 1.000000 : 4 2024-04-25T01:40:25.623 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[13] 0.200000 - 0.500000 : 1 2024-04-25T01:40:25.623 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[14] 0.100000 - 0.200000 : 0 2024-04-25T01:40:25.623 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2024-04-25T01:40:25.623 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-04-25T01:40:25.624 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-04-25T01:40:25.624 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-04-25T01:40:25.624 INFO:tasks.workunit.client.1.smithi187.stdout: 2024-04-25T01:40:25.624 INFO:tasks.workunit.client.1.smithi187.stdout:[ write] 0.087000 6.175222 196.212997 1541120 2024-04-25T01:40:25.624 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-04-25T01:40:25.624 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-04-25T01:40:25.624 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-04-25T01:40:25.624 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-04-25T01:40:25.624 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-04-25T01:40:25.624 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[5] 100.000000 - 200.000000 : 371 2024-04-25T01:40:25.624 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[6] 50.000000 - 100.000000 : 7469 2024-04-25T01:40:25.624 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[7] 20.000000 - 50.000000 : 84520 2024-04-25T01:40:25.625 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[8] 10.000000 - 20.000000 : 198495 2024-04-25T01:40:25.625 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[9] 5.000000 - 10.000000 : 311729 2024-04-25T01:40:25.625 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[10] 2.000000 - 5.000000 : 365076 2024-04-25T01:40:25.625 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[11] 1.000000 - 2.000000 : 138724 2024-04-25T01:40:25.625 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[12] 0.500000 - 1.000000 : 123337 2024-04-25T01:40:25.625 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[13] 0.200000 - 0.500000 : 196062 2024-04-25T01:40:25.625 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[14] 0.100000 - 0.200000 : 115098 2024-04-25T01:40:25.625 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[15] 0.050000 - 0.100000 : 239 2024-04-25T01:40:25.625 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-04-25T01:40:25.625 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-04-25T01:40:25.625 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-04-25T01:40:25.626 INFO:tasks.workunit.client.1.smithi187.stdout: 2024-04-25T01:40:25.626 INFO:tasks.workunit.client.1.smithi187.stdout:[ lseek] 0.000000 0.000859 3.221000 1541120 2024-04-25T01:40:25.626 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-04-25T01:40:25.626 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-04-25T01:40:25.626 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-04-25T01:40:25.626 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-04-25T01:40:25.626 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-04-25T01:40:25.626 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-04-25T01:40:25.626 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-04-25T01:40:25.626 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2024-04-25T01:40:25.626 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-04-25T01:40:25.626 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[9] 5.000000 - 10.000000 : 0 2024-04-25T01:40:25.627 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[10] 2.000000 - 5.000000 : 2 2024-04-25T01:40:25.627 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[11] 1.000000 - 2.000000 : 1 2024-04-25T01:40:25.627 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[12] 0.500000 - 1.000000 : 4 2024-04-25T01:40:25.627 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[13] 0.200000 - 0.500000 : 6 2024-04-25T01:40:25.627 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[14] 0.100000 - 0.200000 : 19 2024-04-25T01:40:25.627 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[15] 0.050000 - 0.100000 : 24 2024-04-25T01:40:25.627 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[16] 0.020000 - 0.050000 : 185 2024-04-25T01:40:25.627 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[17] 0.010000 - 0.020000 : 977 2024-04-25T01:40:25.627 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[18] 0.000000 - 0.010000 : 1539902 2024-04-25T01:40:25.627 INFO:tasks.workunit.client.1.smithi187.stdout: 2024-04-25T01:40:25.627 INFO:tasks.workunit.client.1.smithi187.stdout:[ close] 0.019000 3.210262 81.091003 1204 2024-04-25T01:40:25.627 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-04-25T01:40:25.627 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-04-25T01:40:25.627 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-04-25T01:40:25.628 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-04-25T01:40:25.628 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-04-25T01:40:25.628 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-04-25T01:40:25.628 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[6] 50.000000 - 100.000000 : 3 2024-04-25T01:40:25.628 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[7] 20.000000 - 50.000000 : 41 2024-04-25T01:40:25.628 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[8] 10.000000 - 20.000000 : 65 2024-04-25T01:40:25.628 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[9] 5.000000 - 10.000000 : 89 2024-04-25T01:40:25.628 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[10] 2.000000 - 5.000000 : 146 2024-04-25T01:40:25.628 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[11] 1.000000 - 2.000000 : 129 2024-04-25T01:40:25.628 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[12] 0.500000 - 1.000000 : 89 2024-04-25T01:40:25.628 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[13] 0.200000 - 0.500000 : 149 2024-04-25T01:40:25.628 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[14] 0.100000 - 0.200000 : 157 2024-04-25T01:40:25.628 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[15] 0.050000 - 0.100000 : 81 2024-04-25T01:40:25.628 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[16] 0.020000 - 0.050000 : 254 2024-04-25T01:40:25.629 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[17] 0.010000 - 0.020000 : 1 2024-04-25T01:40:25.629 INFO:tasks.workunit.client.1.smithi187.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-04-25T01:40:25.629 INFO:tasks.workunit.client.1.smithi187.stdout: 2024-04-25T01:40:25.629 INFO:tasks.workunit.client.1.smithi187.stdout: 2024-04-25T01:40:25.629 INFO:tasks.workunit.client.1.smithi187.stdout: 2024-04-25T01:40:25.629 INFO:tasks.workunit.client.1.smithi187.stdout: 2024-04-25T01:40:25.629 INFO:tasks.workunit.client.1.smithi187.stdout:0.8% User Time 2024-04-25T01:40:25.629 INFO:tasks.workunit.client.1.smithi187.stdout:33.4% System Time 2024-04-25T01:40:25.629 INFO:tasks.workunit.client.1.smithi187.stdout:34.2% CPU Utilization 2024-04-25T01:40:25.636 INFO:tasks.workunit.client.1.smithi187.stderr:+ cd .. 2024-04-25T01:40:25.642 INFO:tasks.workunit.client.1.smithi187.stderr:+ rm -r tmp ffsb 2024-04-25T01:40:29.042 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T01:40:29.158 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-25T01:40:29.249 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.1... 2024-04-25T01:40:29.249 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-25T01:40:29.691 DEBUG:teuthology.parallel:result is None 2024-04-25T01:40:29.741 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:40:29.748 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:40:34.820 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-04-25T01:40:34.820 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-04-25T01:40:34.820 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T01:40:34.820 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T01:40:34.823 INFO:tasks.ceph.mon.a:Started 2024-04-25T01:40:34.823 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:40:34.823 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:40:35.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.029+0000 7fd2d5b29640 1 Processor -- start 2024-04-25T01:40:35.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.029+0000 7fd2d5b29640 1 -- start start 2024-04-25T01:40:35.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.030+0000 7fd2d5b29640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd2d013a080 0x7fd2d0071d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:35.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.030+0000 7fd2d5b29640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2d013aac0 0x7fd2d0071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:35.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.030+0000 7fd2d5b29640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2d00c28b0 con 0x7fd2d00722b0 2024-04-25T01:40:35.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.030+0000 7fd2d5b29640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fd2d01526d0 con 0x7fd2d013a080 2024-04-25T01:40:35.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.030+0000 7fd2d5b29640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d006dbe0 con 0x7fd2d013aac0 2024-04-25T01:40:35.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.031+0000 7fd2cffff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fd2d00722b0 0x7fd2d0071030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:52328/0 (socket says 172.21.15.112:52328) 2024-04-25T01:40:35.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.031+0000 7fd2cffff640 1 -- 172.21.15.112:0/830036820 learned_addr learned my addr 172.21.15.112:0/830036820 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:40:35.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.031+0000 7fd2d5328640 1 --2- 172.21.15.112:0/830036820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2d013aac0 0x7fd2d0071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:35.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.031+0000 7fd2cf7fe640 1 -- 172.21.15.112:0/830036820 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4187587375 0 0) 0x7fd2d00c28b0 con 0x7fd2d00722b0 2024-04-25T01:40:35.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.031+0000 7fd2cf7fe640 1 -- 172.21.15.112:0/830036820 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2bc003db0 con 0x7fd2d00722b0 2024-04-25T01:40:35.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.032+0000 7fd2d4b27640 1 --2- 172.21.15.112:0/830036820 >> v2:172.21.15.112:3300/0 conn(0x7fd2d013a080 0x7fd2d0071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:35.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.032+0000 7fd2d5328640 1 -- 172.21.15.112:0/830036820 >> v2:172.21.15.112:3300/0 conn(0x7fd2d013a080 msgr2=0x7fd2d0071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:35.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.032+0000 7fd2cf7fe640 1 -- 172.21.15.112:0/830036820 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3986462161 0 0) 0x7fd2bc003db0 con 0x7fd2d00722b0 2024-04-25T01:40:35.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.032+0000 7fd2d4b27640 1 -- 172.21.15.112:0/830036820 >> v2:172.21.15.112:3300/0 conn(0x7fd2d013a080 msgr2=0x7fd2d0071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:40:35.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.032+0000 7fd2d4b27640 1 -- 172.21.15.112:0/830036820 >> v2:172.21.15.112:3300/0 conn(0x7fd2d013a080 msgr2=0x7fd2d0071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:40:35.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.032+0000 7fd2d4b27640 1 --2- 172.21.15.112:0/830036820 >> v2:172.21.15.112:3300/0 conn(0x7fd2d013a080 0x7fd2d0071d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:40:35.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.032+0000 7fd2d4b27640 1 --2- 172.21.15.112:0/830036820 >> v2:172.21.15.112:3300/0 conn(0x7fd2d013a080 0x7fd2d0071d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:35.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.032+0000 7fd2d5328640 1 --2- 172.21.15.112:0/830036820 >> v2:172.21.15.112:3300/0 conn(0x7fd2d013a080 0x7fd2d0071d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:35.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.032+0000 7fd2d5328640 1 -- 172.21.15.112:0/830036820 >> v1:172.21.15.187:6789/0 conn(0x7fd2d00722b0 legacy=0x7fd2d0071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:35.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.032+0000 7fd2d5328640 1 -- 172.21.15.112:0/830036820 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d0152110 con 0x7fd2d013aac0 2024-04-25T01:40:35.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.038+0000 7fd2d5328640 1 --2- 172.21.15.112:0/830036820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2d013aac0 0x7fd2d0071740 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80097a0 tx=0x7fd2c802a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c4707cb058686ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:35.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.041+0000 7fd2cf7fe640 1 -- 172.21.15.112:0/830036820 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8037070 con 0x7fd2d013aac0 2024-04-25T01:40:35.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.041+0000 7fd2cf7fe640 1 -- 172.21.15.112:0/830036820 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c802ab30 con 0x7fd2d013aac0 2024-04-25T01:40:35.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.042+0000 7fd2d5b29640 1 -- 172.21.15.112:0/830036820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2d013aac0 msgr2=0x7fd2d0071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:35.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.042+0000 7fd2d5b29640 1 --2- 172.21.15.112:0/830036820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2d013aac0 0x7fd2d0071740 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80097a0 tx=0x7fd2c802a0a0 comp rx=0 tx=0).stop 2024-04-25T01:40:35.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.043+0000 7fd2d5b29640 1 -- 172.21.15.112:0/830036820 shutdown_connections 2024-04-25T01:40:35.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.043+0000 7fd2d5b29640 1 --2- 172.21.15.112:0/830036820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2d013aac0 0x7fd2d0071740 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:35.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.043+0000 7fd2d5b29640 1 --2- 172.21.15.112:0/830036820 >> v2:172.21.15.112:3300/0 conn(0x7fd2d013a080 0x7fd2d0071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:35.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.043+0000 7fd2d5b29640 1 -- 172.21.15.112:0/830036820 >> 172.21.15.112:0/830036820 conn(0x7fd2d006ca70 msgr2=0x7fd2d006ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:35.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.047+0000 7fd2d5b29640 1 -- 172.21.15.112:0/830036820 shutdown_connections 2024-04-25T01:40:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.047+0000 7fd2d5b29640 1 -- 172.21.15.112:0/830036820 wait complete. 2024-04-25T01:40:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.049+0000 7fd2d5b29640 1 Processor -- start 2024-04-25T01:40:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.049+0000 7fd2d5b29640 1 -- start start 2024-04-25T01:40:35.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.050+0000 7fd2d5b29640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2d00722b0 0x7fd2d014c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:35.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.050+0000 7fd2d5b29640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2d013a080 0x7fd2d01fb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:35.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.050+0000 7fd2d5b29640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d013aac0 0x7fd2d01fb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:35.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.050+0000 7fd2d5b29640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d00c2b50 con 0x7fd2d013aac0 2024-04-25T01:40:35.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.050+0000 7fd2d5b29640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d0070ba0 con 0x7fd2d013a080 2024-04-25T01:40:35.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.050+0000 7fd2d5b29640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d006d590 con 0x7fd2d00722b0 2024-04-25T01:40:35.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.053+0000 7fd2d5328640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d013aac0 0x7fd2d01fb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:35.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.053+0000 7fd2d5328640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d013aac0 0x7fd2d01fb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45216/0 (socket says 172.21.15.112:45216) 2024-04-25T01:40:35.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.053+0000 7fd2d5328640 1 -- 172.21.15.112:0/116469810 learned_addr learned my addr 172.21.15.112:0/116469810 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:35.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.053+0000 7fd2cffff640 1 --2- 172.21.15.112:0/116469810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2d013a080 0x7fd2d01fb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:35.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.053+0000 7fd2d5328640 1 -- 172.21.15.112:0/116469810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d013aac0 msgr2=0x7fd2d01fb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-25T01:40:35.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.053+0000 7fd2d5328640 1 -- 172.21.15.112:0/116469810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d013aac0 msgr2=0x7fd2d01fb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:40:35.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.053+0000 7fd2d5328640 1 --2- 172.21.15.112:0/116469810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d013aac0 0x7fd2d01fb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:40:35.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.053+0000 7fd2d5328640 1 --2- 172.21.15.112:0/116469810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d013aac0 0x7fd2d01fb9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:35.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.053+0000 7fd2d4b27640 1 --2- 172.21.15.112:0/116469810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2d00722b0 0x7fd2d014c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:35.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.054+0000 7fd2d4b27640 1 -- 172.21.15.112:0/116469810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2d013a080 msgr2=0x7fd2d01fb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:35.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.054+0000 7fd2d4b27640 1 --2- 172.21.15.112:0/116469810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2d013a080 0x7fd2d01fb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:35.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.054+0000 7fd2d4b27640 1 -- 172.21.15.112:0/116469810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d013aac0 msgr2=0x7fd2d01fb9c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:40:35.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.054+0000 7fd2d4b27640 1 --2- 172.21.15.112:0/116469810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d013aac0 0x7fd2d01fb9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:35.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.054+0000 7fd2d4b27640 1 -- 172.21.15.112:0/116469810 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d014e180 con 0x7fd2d00722b0 2024-04-25T01:40:35.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.055+0000 7fd2d4b27640 1 --2- 172.21.15.112:0/116469810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2d00722b0 0x7fd2d014c000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000a720 tx=0x7fd2c000d490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:35.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.055+0000 7fd2cd7fa640 1 -- 172.21.15.112:0/116469810 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c000dda0 con 0x7fd2d00722b0 2024-04-25T01:40:35.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.056+0000 7fd2d5b29640 1 -- 172.21.15.112:0/116469810 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d0133ff0 con 0x7fd2d00722b0 2024-04-25T01:40:35.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.056+0000 7fd2d5b29640 1 -- 172.21.15.112:0/116469810 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d00c65c0 con 0x7fd2d00722b0 2024-04-25T01:40:35.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.056+0000 7fd2cd7fa640 1 -- 172.21.15.112:0/116469810 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c0018070 con 0x7fd2d00722b0 2024-04-25T01:40:35.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.056+0000 7fd2cd7fa640 1 -- 172.21.15.112:0/116469810 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0008680 con 0x7fd2d00722b0 2024-04-25T01:40:35.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.059+0000 7fd2d5b29640 1 -- 172.21.15.112:0/116469810 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2b4005290 con 0x7fd2d00722b0 2024-04-25T01:40:35.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.063+0000 7fd2cd7fa640 1 -- 172.21.15.112:0/116469810 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd2c0005d10 con 0x7fd2d00722b0 2024-04-25T01:40:35.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.063+0000 7fd2cd7fa640 1 --2- 172.21.15.112:0/116469810 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd2b006bf00 0x7fd2b006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:35.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.063+0000 7fd2cffff640 1 --2- 172.21.15.112:0/116469810 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd2b006bf00 0x7fd2b006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:35.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.064+0000 7fd2cffff640 1 --2- 172.21.15.112:0/116469810 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd2b006bf00 0x7fd2b006e3c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00718b0 tx=0x7fd2b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:35.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.064+0000 7fd2cd7fa640 1 -- 172.21.15.112:0/116469810 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fd2c008b8c0 con 0x7fd2d00722b0 2024-04-25T01:40:35.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.069+0000 7fd2cd7fa640 1 -- 172.21.15.112:0/116469810 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd2c0059f40 con 0x7fd2d00722b0 2024-04-25T01:40:35.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.607+0000 7fd2d5b29640 1 -- 172.21.15.112:0/116469810 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd2b4005580 con 0x7fd2d00722b0 2024-04-25T01:40:35.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.607+0000 7fd2cd7fa640 1 -- 172.21.15.112:0/116469810 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7fd2c0059840 con 0x7fd2d00722b0 2024-04-25T01:40:35.596 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":114,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:40:35.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.615+0000 7fd2d5b29640 1 -- 172.21.15.112:0/116469810 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd2b006bf00 msgr2=0x7fd2b006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:35.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.615+0000 7fd2d5b29640 1 --2- 172.21.15.112:0/116469810 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd2b006bf00 0x7fd2b006e3c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00718b0 tx=0x7fd2b8009040 comp rx=0 tx=0).stop 2024-04-25T01:40:35.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.615+0000 7fd2d5b29640 1 -- 172.21.15.112:0/116469810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2d00722b0 msgr2=0x7fd2d014c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:35.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.615+0000 7fd2d5b29640 1 --2- 172.21.15.112:0/116469810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2d00722b0 0x7fd2d014c000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000a720 tx=0x7fd2c000d490 comp rx=0 tx=0).stop 2024-04-25T01:40:35.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.615+0000 7fd2d5b29640 1 -- 172.21.15.112:0/116469810 shutdown_connections 2024-04-25T01:40:35.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.615+0000 7fd2d5b29640 1 --2- 172.21.15.112:0/116469810 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd2b006bf00 0x7fd2b006e3c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:35.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.615+0000 7fd2d5b29640 1 --2- 172.21.15.112:0/116469810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d013aac0 0x7fd2d01fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:35.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.615+0000 7fd2d5b29640 1 --2- 172.21.15.112:0/116469810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2d013a080 0x7fd2d01fb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:35.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.615+0000 7fd2d5b29640 1 --2- 172.21.15.112:0/116469810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2d00722b0 0x7fd2d014c000 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000a720 tx=0x7fd2c000d490 comp rx=0 tx=0).stop 2024-04-25T01:40:35.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.615+0000 7fd2d5b29640 1 -- 172.21.15.112:0/116469810 >> 172.21.15.112:0/116469810 conn(0x7fd2d006ca70 msgr2=0x7fd2d0070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:35.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.615+0000 7fd2d5b29640 1 -- 172.21.15.112:0/116469810 shutdown_connections 2024-04-25T01:40:35.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:35.615+0000 7fd2d5b29640 1 -- 172.21.15.112:0/116469810 wait complete. 2024-04-25T01:40:38.629 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:40:38.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.801+0000 7f9e09970640 1 Processor -- start 2024-04-25T01:40:38.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.802+0000 7f9e09970640 1 -- start start 2024-04-25T01:40:38.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.803+0000 7f9e09970640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9e040ac230 0x7f9e041522f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:38.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.803+0000 7f9e09970640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9e040c1980 0x7f9e040c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:38.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.803+0000 7f9e09970640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e0406f050 0x7f9e0406d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:38.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.803+0000 7f9e09970640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f9e04152830 con 0x7f9e040c1980 2024-04-25T01:40:38.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.803+0000 7f9e09970640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9e04071aa0 con 0x7f9e040ac230 2024-04-25T01:40:38.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.803+0000 7f9e09970640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9e04071d80 con 0x7f9e0406f050 2024-04-25T01:40:38.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.803+0000 7f9e027fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9e040c1980 0x7f9e040c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:38.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.804+0000 7f9e027fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9e040c1980 0x7f9e040c1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42302/0 (socket says 172.21.15.112:42302) 2024-04-25T01:40:38.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.804+0000 7f9e027fc640 1 -- 172.21.15.112:0/1976536530 learned_addr learned my addr 172.21.15.112:0/1976536530 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:38.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.804+0000 7f9e037fe640 1 --2- 172.21.15.112:0/1976536530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e0406f050 0x7f9e0406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:38.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.804+0000 7f9e027fc640 1 -- 172.21.15.112:0/1976536530 >> v2:172.21.15.112:3300/0 conn(0x7f9e040c1980 msgr2=0x7f9e040c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:40:38.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.804+0000 7f9e027fc640 1 -- 172.21.15.112:0/1976536530 >> v2:172.21.15.112:3300/0 conn(0x7f9e040c1980 msgr2=0x7f9e040c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:40:38.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.804+0000 7f9e027fc640 1 --2- 172.21.15.112:0/1976536530 >> v2:172.21.15.112:3300/0 conn(0x7f9e040c1980 0x7f9e040c1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:40:38.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.804+0000 7f9e02ffd640 1 --2- 172.21.15.112:0/1976536530 >> v2:172.21.15.187:3300/0 conn(0x7f9e040ac230 0x7f9e041522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:38.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.804+0000 7f9e027fc640 1 --2- 172.21.15.112:0/1976536530 >> v2:172.21.15.112:3300/0 conn(0x7f9e040c1980 0x7f9e040c1e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:38.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.804+0000 7f9e037fe640 1 -- 172.21.15.112:0/1976536530 >> v2:172.21.15.187:3300/0 conn(0x7f9e040ac230 msgr2=0x7f9e041522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:38.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.804+0000 7f9e037fe640 1 --2- 172.21.15.112:0/1976536530 >> v2:172.21.15.187:3300/0 conn(0x7f9e040ac230 0x7f9e041522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:38.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.804+0000 7f9e037fe640 1 -- 172.21.15.112:0/1976536530 >> v2:172.21.15.112:3300/0 conn(0x7f9e040c1980 msgr2=0x7f9e040c1e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:40:38.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.804+0000 7f9e037fe640 1 --2- 172.21.15.112:0/1976536530 >> v2:172.21.15.112:3300/0 conn(0x7f9e040c1980 0x7f9e040c1e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:38.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.804+0000 7f9e037fe640 1 -- 172.21.15.112:0/1976536530 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e04072060 con 0x7f9e0406f050 2024-04-25T01:40:38.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.805+0000 7f9e037fe640 1 --2- 172.21.15.112:0/1976536530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e0406f050 0x7f9e0406d760 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9e04073450 tx=0x7f9dfc02a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96f6e509fa94b949 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:38.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.806+0000 7f9e01ffb640 1 -- 172.21.15.112:0/1976536530 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc037070 con 0x7f9e0406f050 2024-04-25T01:40:38.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.806+0000 7f9e01ffb640 1 -- 172.21.15.112:0/1976536530 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dfc02add0 con 0x7f9e0406f050 2024-04-25T01:40:38.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.806+0000 7f9e01ffb640 1 -- 172.21.15.112:0/1976536530 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc032660 con 0x7f9e0406f050 2024-04-25T01:40:38.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.807+0000 7f9e09970640 1 -- 172.21.15.112:0/1976536530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e0406f050 msgr2=0x7f9e0406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:38.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.807+0000 7f9e09970640 1 --2- 172.21.15.112:0/1976536530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e0406f050 0x7f9e0406d760 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9e04073450 tx=0x7f9dfc02a080 comp rx=0 tx=0).stop 2024-04-25T01:40:38.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.807+0000 7f9e09970640 1 -- 172.21.15.112:0/1976536530 shutdown_connections 2024-04-25T01:40:38.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.807+0000 7f9e09970640 1 --2- 172.21.15.112:0/1976536530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e0406f050 0x7f9e0406d760 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:38.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.807+0000 7f9e09970640 1 --2- 172.21.15.112:0/1976536530 >> v2:172.21.15.112:3300/0 conn(0x7f9e040c1980 0x7f9e040c1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:38.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.807+0000 7f9e09970640 1 --2- 172.21.15.112:0/1976536530 >> v2:172.21.15.187:3300/0 conn(0x7f9e040ac230 0x7f9e041522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:38.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.807+0000 7f9e09970640 1 -- 172.21.15.112:0/1976536530 >> 172.21.15.112:0/1976536530 conn(0x7f9e040c6f80 msgr2=0x7f9e040c7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:38.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.807+0000 7f9e09970640 1 -- 172.21.15.112:0/1976536530 shutdown_connections 2024-04-25T01:40:38.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.807+0000 7f9e09970640 1 -- 172.21.15.112:0/1976536530 wait complete. 2024-04-25T01:40:38.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.809+0000 7f9e09970640 1 Processor -- start 2024-04-25T01:40:38.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.809+0000 7f9e09970640 1 -- start start 2024-04-25T01:40:38.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.810+0000 7f9e09970640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e0406f050 0x7f9e040a9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:38.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.810+0000 7f9e09970640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e040ac230 0x7f9e04154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:38.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.810+0000 7f9e09970640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e040c1980 0x7f9e0414bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:38.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.810+0000 7f9e09970640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9e040c2350 con 0x7f9e040c1980 2024-04-25T01:40:38.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.810+0000 7f9e09970640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9e04072620 con 0x7f9e0406f050 2024-04-25T01:40:38.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.810+0000 7f9e09970640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9e04072b40 con 0x7f9e040ac230 2024-04-25T01:40:38.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.811+0000 7f9e037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e040c1980 0x7f9e0414bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:38.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.811+0000 7f9e037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e040c1980 0x7f9e0414bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42306/0 (socket says 172.21.15.112:42306) 2024-04-25T01:40:38.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.811+0000 7f9e037fe640 1 -- 172.21.15.112:0/2153849810 learned_addr learned my addr 172.21.15.112:0/2153849810 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:38.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.811+0000 7f9e027fc640 1 --2- 172.21.15.112:0/2153849810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e040ac230 0x7f9e04154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:38.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.811+0000 7f9e037fe640 1 -- 172.21.15.112:0/2153849810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e040c1980 msgr2=0x7f9e0414bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T01:40:38.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.811+0000 7f9e037fe640 1 -- 172.21.15.112:0/2153849810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e040c1980 msgr2=0x7f9e0414bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:40:38.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.811+0000 7f9e037fe640 1 --2- 172.21.15.112:0/2153849810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e040c1980 0x7f9e0414bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:40:38.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.811+0000 7f9e037fe640 1 --2- 172.21.15.112:0/2153849810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e040c1980 0x7f9e0414bdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:38.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.811+0000 7f9e027fc640 1 -- 172.21.15.112:0/2153849810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e0406f050 msgr2=0x7f9e040a9ec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:40:38.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.811+0000 7f9e027fc640 1 --2- 172.21.15.112:0/2153849810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e0406f050 0x7f9e040a9ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:38.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.811+0000 7f9e027fc640 1 -- 172.21.15.112:0/2153849810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e040c1980 msgr2=0x7f9e0414bdb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:40:38.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.811+0000 7f9e027fc640 1 --2- 172.21.15.112:0/2153849810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e040c1980 0x7f9e0414bdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:38.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.811+0000 7f9e027fc640 1 -- 172.21.15.112:0/2153849810 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e041ecf90 con 0x7f9e040ac230 2024-04-25T01:40:38.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.811+0000 7f9e027fc640 1 --2- 172.21.15.112:0/2153849810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e040ac230 0x7f9e04154d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9dec00e7e0 tx=0x7f9dec00eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:38.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.812+0000 7f9dfbfff640 1 -- 172.21.15.112:0/2153849810 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec017070 con 0x7f9e040ac230 2024-04-25T01:40:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.812+0000 7f9e09970640 1 -- 172.21.15.112:0/2153849810 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e041ed270 con 0x7f9e040ac230 2024-04-25T01:40:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.812+0000 7f9dfbfff640 1 -- 172.21.15.112:0/2153849810 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dec004020 con 0x7f9e040ac230 2024-04-25T01:40:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.812+0000 7f9e09970640 1 -- 172.21.15.112:0/2153849810 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e041ed830 con 0x7f9e040ac230 2024-04-25T01:40:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.812+0000 7f9dfbfff640 1 -- 172.21.15.112:0/2153849810 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec00f040 con 0x7f9e040ac230 2024-04-25T01:40:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.814+0000 7f9dfbfff640 1 -- 172.21.15.112:0/2153849810 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9dec00c6d0 con 0x7f9e040ac230 2024-04-25T01:40:38.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.815+0000 7f9dfbfff640 1 --2- 172.21.15.112:0/2153849810 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9de406be30 0x7f9de406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:38.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.815+0000 7f9dfbfff640 1 -- 172.21.15.112:0/2153849810 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f9dec08b050 con 0x7f9e040ac230 2024-04-25T01:40:38.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.815+0000 7f9e02ffd640 1 --2- 172.21.15.112:0/2153849810 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9de406be30 0x7f9de406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:38.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.816+0000 7f9e09970640 1 -- 172.21.15.112:0/2153849810 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de8005290 con 0x7f9e040ac230 2024-04-25T01:40:38.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.819+0000 7f9e02ffd640 1 --2- 172.21.15.112:0/2153849810 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9de406be30 0x7f9de406e2f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9df400e190 tx=0x7f9df400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:38.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:38.823+0000 7f9dfbfff640 1 -- 172.21.15.112:0/2153849810 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9dec059690 con 0x7f9e040ac230 2024-04-25T01:40:39.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:39.297+0000 7f9e09970640 1 -- 172.21.15.112:0/2153849810 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9de8005580 con 0x7f9e040ac230 2024-04-25T01:40:39.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:39.299+0000 7f9dfbfff640 1 -- 172.21.15.112:0/2153849810 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f9dec058f90 con 0x7f9e040ac230 2024-04-25T01:40:39.286 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":114,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:40:39.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:39.303+0000 7f9e09970640 1 -- 172.21.15.112:0/2153849810 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9de406be30 msgr2=0x7f9de406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:39.303+0000 7f9e09970640 1 --2- 172.21.15.112:0/2153849810 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9de406be30 0x7f9de406e2f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9df400e190 tx=0x7f9df400b040 comp rx=0 tx=0).stop 2024-04-25T01:40:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:39.303+0000 7f9e09970640 1 -- 172.21.15.112:0/2153849810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e040ac230 msgr2=0x7f9e04154d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:39.303+0000 7f9e09970640 1 --2- 172.21.15.112:0/2153849810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e040ac230 0x7f9e04154d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9dec00e7e0 tx=0x7f9dec00eca0 comp rx=0 tx=0).stop 2024-04-25T01:40:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:39.303+0000 7f9e09970640 1 -- 172.21.15.112:0/2153849810 shutdown_connections 2024-04-25T01:40:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:39.303+0000 7f9e09970640 1 --2- 172.21.15.112:0/2153849810 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9de406be30 0x7f9de406e2f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:39.303+0000 7f9e09970640 1 --2- 172.21.15.112:0/2153849810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e040c1980 0x7f9e0414bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:39.303+0000 7f9e09970640 1 --2- 172.21.15.112:0/2153849810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e040ac230 0x7f9e04154d90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:39.303+0000 7f9e09970640 1 --2- 172.21.15.112:0/2153849810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e0406f050 0x7f9e040a9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:39.303+0000 7f9e09970640 1 -- 172.21.15.112:0/2153849810 >> 172.21.15.112:0/2153849810 conn(0x7f9e040c6f80 msgr2=0x7f9e040ab170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:39.303+0000 7f9e09970640 1 -- 172.21.15.112:0/2153849810 shutdown_connections 2024-04-25T01:40:39.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:39.303+0000 7f9e09970640 1 -- 172.21.15.112:0/2153849810 wait complete. 2024-04-25T01:40:42.307 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:40:42.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.431+0000 7f108d7c8640 1 Processor -- start 2024-04-25T01:40:42.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.431+0000 7f108d7c8640 1 -- start start 2024-04-25T01:40:42.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.431+0000 7f108d7c8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1088155d80 0x7f1088156160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.431+0000 7f108d7c8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f108814c790 0x7f1088153a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.431+0000 7f108d7c8640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f108806dfb0 con 0x7f108806dab0 2024-04-25T01:40:42.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.431+0000 7f108d7c8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f108806ebc0 con 0x7f108814c790 2024-04-25T01:40:42.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.431+0000 7f108d7c8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f108806e290 con 0x7f1088155d80 2024-04-25T01:40:42.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.431+0000 7f10867fc640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f108806dab0 0x7f108814c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:41326/0 (socket says 172.21.15.112:41326) 2024-04-25T01:40:42.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.431+0000 7f10867fc640 1 -- 172.21.15.112:0/3354805306 learned_addr learned my addr 172.21.15.112:0/3354805306 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:40:42.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.432+0000 7f10877fe640 1 --2- 172.21.15.112:0/3354805306 >> v2:172.21.15.187:3300/0 conn(0x7f108814c790 0x7f1088153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.432+0000 7f1086ffd640 1 --2- 172.21.15.112:0/3354805306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1088155d80 0x7f1088156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.432+0000 7f1085ffb640 1 -- 172.21.15.112:0/3354805306 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1883132026 0 0) 0x7f108806dfb0 con 0x7f108806dab0 2024-04-25T01:40:42.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.432+0000 7f1085ffb640 1 -- 172.21.15.112:0/3354805306 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1070003520 con 0x7f108806dab0 2024-04-25T01:40:42.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.432+0000 7f10877fe640 1 -- 172.21.15.112:0/3354805306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1088155d80 msgr2=0x7f1088156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:42.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.432+0000 7f10877fe640 1 --2- 172.21.15.112:0/3354805306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1088155d80 0x7f1088156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.432+0000 7f10877fe640 1 -- 172.21.15.112:0/3354805306 >> v1:172.21.15.112:6789/0 conn(0x7f108806dab0 legacy=0x7f108814c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.432+0000 7f10877fe640 1 -- 172.21.15.112:0/3354805306 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108806e570 con 0x7f108814c790 2024-04-25T01:40:42.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.433+0000 7f10877fe640 1 --2- 172.21.15.112:0/3354805306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f108814c790 0x7f1088153a80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1078009d90 tx=0x7f107802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9674719b1404b02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:42.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.433+0000 7f1085ffb640 1 -- 172.21.15.112:0/3354805306 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078031600 con 0x7f108814c790 2024-04-25T01:40:42.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.433+0000 7f1085ffb640 1 -- 172.21.15.112:0/3354805306 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10780317a0 con 0x7f108814c790 2024-04-25T01:40:42.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.433+0000 7f1085ffb640 1 -- 172.21.15.112:0/3354805306 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078031b70 con 0x7f108814c790 2024-04-25T01:40:42.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.433+0000 7f108d7c8640 1 -- 172.21.15.112:0/3354805306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f108814c790 msgr2=0x7f1088153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.433+0000 7f108d7c8640 1 --2- 172.21.15.112:0/3354805306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f108814c790 0x7f1088153a80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1078009d90 tx=0x7f107802a5d0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.433+0000 7f108d7c8640 1 -- 172.21.15.112:0/3354805306 shutdown_connections 2024-04-25T01:40:42.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.433+0000 7f108d7c8640 1 --2- 172.21.15.112:0/3354805306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f108814c790 0x7f1088153a80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.433+0000 7f108d7c8640 1 --2- 172.21.15.112:0/3354805306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1088155d80 0x7f1088156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.433+0000 7f108d7c8640 1 -- 172.21.15.112:0/3354805306 >> 172.21.15.112:0/3354805306 conn(0x7f10880c6f80 msgr2=0x7f10880c7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:42.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.434+0000 7f108d7c8640 1 -- 172.21.15.112:0/3354805306 shutdown_connections 2024-04-25T01:40:42.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.434+0000 7f108d7c8640 1 -- 172.21.15.112:0/3354805306 wait complete. 2024-04-25T01:40:42.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.435+0000 7f108d7c8640 1 Processor -- start 2024-04-25T01:40:42.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.435+0000 7f108d7c8640 1 -- start start 2024-04-25T01:40:42.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.435+0000 7f108d7c8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f108806dab0 0x7f1088152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f108d7c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f108814c790 0x7f10880723d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f108d7c8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1088155d80 0x7f1088142800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f108d7c8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1088156820 con 0x7f108814c790 2024-04-25T01:40:42.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f108d7c8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f10881566a0 con 0x7f108806dab0 2024-04-25T01:40:42.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f108d7c8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f108806c910 con 0x7f1088155d80 2024-04-25T01:40:42.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f10867fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f108814c790 0x7f10880723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f1086ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f108806dab0 0x7f1088152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f1086ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f108806dab0 0x7f1088152df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43998/0 (socket says 172.21.15.112:43998) 2024-04-25T01:40:42.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f1086ffd640 1 -- 172.21.15.112:0/3881914290 learned_addr learned my addr 172.21.15.112:0/3881914290 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:42.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f10877fe640 1 --2- 172.21.15.112:0/3881914290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1088155d80 0x7f1088142800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f10867fc640 1 -- 172.21.15.112:0/3881914290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1088155d80 msgr2=0x7f1088142800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f10867fc640 1 --2- 172.21.15.112:0/3881914290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1088155d80 0x7f1088142800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f10867fc640 1 -- 172.21.15.112:0/3881914290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f108806dab0 msgr2=0x7f1088152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f10867fc640 1 --2- 172.21.15.112:0/3881914290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f108806dab0 0x7f1088152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f10867fc640 1 -- 172.21.15.112:0/3881914290 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10881551a0 con 0x7f108814c790 2024-04-25T01:40:42.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f10867fc640 1 --2- 172.21.15.112:0/3881914290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f108814c790 0x7f10880723d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f107c00c400 tx=0x7f107c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:42.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f106f7fe640 1 -- 172.21.15.112:0/3881914290 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c017070 con 0x7f108814c790 2024-04-25T01:40:42.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f106f7fe640 1 -- 172.21.15.112:0/3881914290 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f107c005ce0 con 0x7f108814c790 2024-04-25T01:40:42.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f108d7c8640 1 -- 172.21.15.112:0/3881914290 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10881505e0 con 0x7f108814c790 2024-04-25T01:40:42.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f106f7fe640 1 -- 172.21.15.112:0/3881914290 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c0072f0 con 0x7f108814c790 2024-04-25T01:40:42.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.436+0000 7f108d7c8640 1 -- 172.21.15.112:0/3881914290 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1088150eb0 con 0x7f108814c790 2024-04-25T01:40:42.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.437+0000 7f108d7c8640 1 -- 172.21.15.112:0/3881914290 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10880682a0 con 0x7f108814c790 2024-04-25T01:40:42.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.438+0000 7f106f7fe640 1 -- 172.21.15.112:0/3881914290 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f107c01a070 con 0x7f108814c790 2024-04-25T01:40:42.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.438+0000 7f106f7fe640 1 --2- 172.21.15.112:0/3881914290 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f107406be30 0x7f107406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.438+0000 7f106f7fe640 1 -- 172.21.15.112:0/3881914290 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f107c08a440 con 0x7f108814c790 2024-04-25T01:40:42.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.438+0000 7f1086ffd640 1 --2- 172.21.15.112:0/3881914290 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f107406be30 0x7f107406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.439+0000 7f1086ffd640 1 --2- 172.21.15.112:0/3881914290 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f107406be30 0x7f107406e2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f10881490e0 tx=0x7f1080009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:42.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.443+0000 7f106f7fe640 1 -- 172.21.15.112:0/3881914290 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f107c007e20 con 0x7f108814c790 2024-04-25T01:40:42.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.698+0000 7f108d7c8640 1 -- 172.21.15.112:0/3881914290 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f10880c6d20 con 0x7f108814c790 2024-04-25T01:40:42.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.699+0000 7f106f7fe640 1 -- 172.21.15.112:0/3881914290 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f107c058490 con 0x7f108814c790 2024-04-25T01:40:42.684 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":118,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:40:42.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.701+0000 7f108d7c8640 1 -- 172.21.15.112:0/3881914290 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f107406be30 msgr2=0x7f107406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.701+0000 7f108d7c8640 1 --2- 172.21.15.112:0/3881914290 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f107406be30 0x7f107406e2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f10881490e0 tx=0x7f1080009190 comp rx=0 tx=0).stop 2024-04-25T01:40:42.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.701+0000 7f108d7c8640 1 -- 172.21.15.112:0/3881914290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f108814c790 msgr2=0x7f10880723d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.701+0000 7f108d7c8640 1 --2- 172.21.15.112:0/3881914290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f108814c790 0x7f10880723d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f107c00c400 tx=0x7f107c00c8c0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.701+0000 7f108d7c8640 1 -- 172.21.15.112:0/3881914290 shutdown_connections 2024-04-25T01:40:42.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.701+0000 7f108d7c8640 1 --2- 172.21.15.112:0/3881914290 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f107406be30 0x7f107406e2f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.701+0000 7f108d7c8640 1 --2- 172.21.15.112:0/3881914290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1088155d80 0x7f1088142800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.701+0000 7f108d7c8640 1 --2- 172.21.15.112:0/3881914290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f108814c790 0x7f10880723d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.701+0000 7f108d7c8640 1 --2- 172.21.15.112:0/3881914290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f108806dab0 0x7f1088152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.701+0000 7f108d7c8640 1 -- 172.21.15.112:0/3881914290 >> 172.21.15.112:0/3881914290 conn(0x7f10880c6f80 msgr2=0x7f10880c7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:42.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.701+0000 7f108d7c8640 1 -- 172.21.15.112:0/3881914290 shutdown_connections 2024-04-25T01:40:42.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.701+0000 7f108d7c8640 1 -- 172.21.15.112:0/3881914290 wait complete. 2024-04-25T01:40:42.697 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:40:42.697 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T01:40:42.697 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T01:40:42.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.809+0000 7fd951a10640 1 Processor -- start 2024-04-25T01:40:42.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.809+0000 7fd951a10640 1 -- start start 2024-04-25T01:40:42.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.810+0000 7fd951a10640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd94c1596a0 0x7fd94c152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.810+0000 7fd951a10640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd94c0c72b0 0x7fd94c160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.810+0000 7fd951a10640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd94c159b90 con 0x7fd94c1592c0 2024-04-25T01:40:42.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.810+0000 7fd951a10640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fd94c0c6e20 con 0x7fd94c0c72b0 2024-04-25T01:40:42.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.810+0000 7fd951a10640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd94c06c970 con 0x7fd94c1596a0 2024-04-25T01:40:42.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.810+0000 7fd94b7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd94c0c72b0 0x7fd94c160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.810+0000 7fd94affd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd94c1592c0 0x7fd94c0c9070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:41328/0 (socket says 172.21.15.112:41328) 2024-04-25T01:40:42.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.810+0000 7fd94affd640 1 -- 172.21.15.112:0/316765982 learned_addr learned my addr 172.21.15.112:0/316765982 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:40:42.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.811+0000 7fd94b7fe640 1 -- 172.21.15.112:0/316765982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd94c1596a0 msgr2=0x7fd94c152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:42.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.811+0000 7fd94b7fe640 1 --2- 172.21.15.112:0/316765982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd94c1596a0 0x7fd94c152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.811+0000 7fd94b7fe640 1 -- 172.21.15.112:0/316765982 >> v1:172.21.15.112:6789/0 conn(0x7fd94c1592c0 legacy=0x7fd94c0c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.811+0000 7fd94a7fc640 1 -- 172.21.15.112:0/316765982 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3626353653 0 0) 0x7fd94c159b90 con 0x7fd94c1592c0 2024-04-25T01:40:42.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.811+0000 7fd94b7fe640 1 -- 172.21.15.112:0/316765982 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c06cc50 con 0x7fd94c0c72b0 2024-04-25T01:40:42.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.811+0000 7fd94b7fe640 1 --2- 172.21.15.112:0/316765982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c0c72b0 0x7fd94c160380 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd944009d90 tx=0x7fd94402a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e318d45e59dd3840 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:42.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.811+0000 7fd94a7fc640 1 -- 172.21.15.112:0/316765982 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944007740 con 0x7fd94c0c72b0 2024-04-25T01:40:42.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.811+0000 7fd94a7fc640 1 -- 172.21.15.112:0/316765982 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9440078e0 con 0x7fd94c0c72b0 2024-04-25T01:40:42.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.811+0000 7fd94a7fc640 1 -- 172.21.15.112:0/316765982 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944007cb0 con 0x7fd94c0c72b0 2024-04-25T01:40:42.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.812+0000 7fd951a10640 1 -- 172.21.15.112:0/316765982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c0c72b0 msgr2=0x7fd94c160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.812+0000 7fd951a10640 1 --2- 172.21.15.112:0/316765982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c0c72b0 0x7fd94c160380 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd944009d90 tx=0x7fd94402a530 comp rx=0 tx=0).stop 2024-04-25T01:40:42.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.812+0000 7fd951a10640 1 -- 172.21.15.112:0/316765982 shutdown_connections 2024-04-25T01:40:42.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.812+0000 7fd951a10640 1 --2- 172.21.15.112:0/316765982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c0c72b0 0x7fd94c160380 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.812+0000 7fd951a10640 1 --2- 172.21.15.112:0/316765982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd94c1596a0 0x7fd94c152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.812+0000 7fd951a10640 1 -- 172.21.15.112:0/316765982 >> 172.21.15.112:0/316765982 conn(0x7fd94c0cb170 msgr2=0x7fd94c0cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.812+0000 7fd951a10640 1 -- 172.21.15.112:0/316765982 shutdown_connections 2024-04-25T01:40:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.812+0000 7fd951a10640 1 -- 172.21.15.112:0/316765982 wait complete. 2024-04-25T01:40:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.813+0000 7fd951a10640 1 Processor -- start 2024-04-25T01:40:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.813+0000 7fd951a10640 1 -- start start 2024-04-25T01:40:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd951a10640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c0c72b0 0x7fd94c14ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd951a10640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd94c1592c0 0x7fd94c1eeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd951a10640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd94c1596a0 0x7fd94c1ec810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd951a10640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c06c510 con 0x7fd94c0c72b0 2024-04-25T01:40:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd951a10640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c06c390 con 0x7fd94c1592c0 2024-04-25T01:40:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd951a10640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd94c06db20 con 0x7fd94c1596a0 2024-04-25T01:40:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd94affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c0c72b0 0x7fd94c14ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd94affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c0c72b0 0x7fd94c14ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42318/0 (socket says 172.21.15.112:42318) 2024-04-25T01:40:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd94affd640 1 -- 172.21.15.112:0/812447735 learned_addr learned my addr 172.21.15.112:0/812447735 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd943fff640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd94c1592c0 0x7fd94c1eeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd94b7fe640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd94c1596a0 0x7fd94c1ec810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd94affd640 1 -- 172.21.15.112:0/812447735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd94c1596a0 msgr2=0x7fd94c1ec810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd94affd640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd94c1596a0 0x7fd94c1ec810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd94affd640 1 -- 172.21.15.112:0/812447735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd94c1592c0 msgr2=0x7fd94c1eeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd94affd640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd94c1592c0 0x7fd94c1eeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd94affd640 1 -- 172.21.15.112:0/812447735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c14e170 con 0x7fd94c0c72b0 2024-04-25T01:40:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd943fff640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd94c1592c0 0x7fd94c1eeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:40:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd94affd640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c0c72b0 0x7fd94c14ec10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00c4a0 tx=0x7fd93c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd948ff9640 1 -- 172.21.15.112:0/812447735 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c017070 con 0x7fd94c0c72b0 2024-04-25T01:40:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd948ff9640 1 -- 172.21.15.112:0/812447735 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd93c005ce0 con 0x7fd94c0c72b0 2024-04-25T01:40:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd951a10640 1 -- 172.21.15.112:0/812447735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94c0b1060 con 0x7fd94c0c72b0 2024-04-25T01:40:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.814+0000 7fd948ff9640 1 -- 172.21.15.112:0/812447735 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0072b0 con 0x7fd94c0c72b0 2024-04-25T01:40:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.815+0000 7fd951a10640 1 -- 172.21.15.112:0/812447735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd94c1ef6a0 con 0x7fd94c0c72b0 2024-04-25T01:40:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.815+0000 7fd951a10640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c14e340 0x7fd94c148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.815+0000 7fd951a10640 1 -- 172.21.15.112:0/812447735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c075980 con 0x7fd94c14e340 2024-04-25T01:40:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.815+0000 7fd943fff640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c14e340 0x7fd94c148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.816+0000 7fd943fff640 1 -- 172.21.15.112:0/812447735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd94c0679a0 con 0x7fd94c14e340 2024-04-25T01:40:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.816+0000 7fd943fff640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c14e340 0x7fd94c148dd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd94c159ac0 tx=0x7fd93400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.816+0000 7fd948ff9640 1 -- 172.21.15.112:0/812447735 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934015070 con 0x7fd94c14e340 2024-04-25T01:40:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.816+0000 7fd948ff9640 1 -- 172.21.15.112:0/812447735 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd93c01a070 con 0x7fd94c0c72b0 2024-04-25T01:40:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.816+0000 7fd948ff9640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd92c06bd30 0x7fd92c06e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.817+0000 7fd94b7fe640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd92c06bd30 0x7fd92c06e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.817+0000 7fd948ff9640 1 -- 172.21.15.112:0/812447735 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fd93c08a530 con 0x7fd94c0c72b0 2024-04-25T01:40:42.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.817+0000 7fd948ff9640 1 -- 172.21.15.112:0/812447735 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd934011800 con 0x7fd94c14e340 2024-04-25T01:40:42.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.817+0000 7fd948ff9640 1 -- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c14e340 msgr2=0x7fd94c148dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.817+0000 7fd948ff9640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c14e340 0x7fd94c148dd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd94c159ac0 tx=0x7fd93400c950 comp rx=0 tx=0).stop 2024-04-25T01:40:42.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.817+0000 7fd948ff9640 1 -- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c14e340 msgr2=0x7fd94c148dd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:40:42.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.817+0000 7fd948ff9640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c14e340 0x7fd94c148dd0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.817+0000 7fd94b7fe640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd92c06bd30 0x7fd92c06e1f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd944034000 tx=0x7fd94402f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:42.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.823+0000 7fd951a10640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c149e50 0x7fd94c149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.823+0000 7fd951a10640 1 -- 172.21.15.112:0/812447735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c0c79d0 con 0x7fd94c149e50 2024-04-25T01:40:42.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.823+0000 7fd943fff640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c149e50 0x7fd94c149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.824+0000 7fd943fff640 1 -- 172.21.15.112:0/812447735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd94c0c5ca0 con 0x7fd94c149e50 2024-04-25T01:40:42.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.824+0000 7fd943fff640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c149e50 0x7fd94c149310 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd934017000 tx=0x7fd934004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:42.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.824+0000 7fd948ff9640 1 -- 172.21.15.112:0/812447735 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9340243f0 con 0x7fd94c149e50 2024-04-25T01:40:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.824+0000 7fd948ff9640 1 -- 172.21.15.112:0/812447735 <== mon.0 v2:172.21.15.112:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4560 (secure 0 0 0) 0x7fd9340243f0 con 0x7fd94c149e50 2024-04-25T01:40:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.824+0000 7fd948ff9640 1 -- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c149e50 msgr2=0x7fd94c149310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.824+0000 7fd948ff9640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c149e50 0x7fd94c149310 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd934017000 tx=0x7fd934004730 comp rx=0 tx=0).stop 2024-04-25T01:40:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.824+0000 7fd948ff9640 1 -- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c149e50 msgr2=0x7fd94c149310 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:40:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.824+0000 7fd948ff9640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c149e50 0x7fd94c149310 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.817 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:40:42.817 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:40:42.817 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:40:42.817 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "leader", 2024-04-25T01:40:42.817 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 118, 2024-04-25T01:40:42.817 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 7851, 2024-04-25T01:40:42.817 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:40:42.818 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:40:42.818 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:40:42.818 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:40:42.818 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:42.818 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:40:42.818 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:40:42.818 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:40:42.819 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:40:42.819 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:40:42.819 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:40:42.819 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:40:42.819 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:40:42.819 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:40:42.819 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:40:42.819 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:40:42.819 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:40:42.819 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:40:42.820 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:40:42.820 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:40:42.820 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:42.820 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:40:42.820 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:40:42.820 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:40:42.820 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:40:42.820 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:40:42.820 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:40:42.821 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:40:42.821 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:40:42.821 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:40:42.821 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:40:42.821 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:40:42.821 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:40:42.821 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:40:42.821 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:42.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.822 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:40:42.822 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:40:42.822 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:40:42.822 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:40:42.822 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:40:42.822 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:40:42.822 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:40:42.822 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:40:42.823 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:40:42.823 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:40:42.823 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:40:42.823 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:40:42.823 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:40:42.823 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:40:42.823 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:40:42.823 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:40:42.823 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:40:42.823 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:40:42.823 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:40:42.823 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:40:42.823 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:40:42.823 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:40:42.824 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:40:42.824 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:40:42.824 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:40:42.824 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:40:42.824 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:40:42.824 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:40:42.824 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:42.824 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:40:42.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.824 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:40:42.824 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.824 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:40:42.824 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:40:42.824 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:40:42.825 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:40:42.825 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.825 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:40:42.825 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:40:42.825 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:42.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.825 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.825 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:40:42.825 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:40:42.825 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:42.825 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:42.825 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:42.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.825 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:40:42.826 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:40:42.826 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:40:42.826 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:40:42.826 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:40:42.826 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.826 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.826 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:40:42.826 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:40:42.826 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:40:42.826 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:40:42.826 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.826 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:40:42.826 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:40:42.827 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:42.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.827 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.827 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:40:42.827 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:40:42.827 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:42.827 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:42.827 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:42.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.827 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:40:42.827 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:40:42.828 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:40:42.828 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:40:42.828 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:40:42.828 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.828 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.828 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:40:42.828 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:40:42.828 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:40:42.828 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:40:42.828 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.828 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:40:42.829 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:40:42.829 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:42.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.829 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.829 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:40:42.829 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:40:42.829 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:42.830 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:42.830 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:42.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.830 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:40:42.830 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:40:42.830 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:40:42.830 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:40:42.830 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:40:42.830 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:42.830 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:42.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.830 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:40:42.830 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:40:42.830 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.831 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:40:42.831 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:40:42.831 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:40:42.831 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:42.831 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:42.831 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:40:42.831 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.831 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:40:42.831 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:40:42.831 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:40:42.831 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:42.831 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:42.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.831 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:40:42.832 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:40:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.824+0000 7fd951a10640 1 -- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd92c06bd30 msgr2=0x7fd92c06e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.824+0000 7fd951a10640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd92c06bd30 0x7fd92c06e1f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd944034000 tx=0x7fd94402f000 comp rx=0 tx=0).stop 2024-04-25T01:40:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.824+0000 7fd951a10640 1 -- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c0c72b0 msgr2=0x7fd94c14ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.824+0000 7fd951a10640 1 --2- 172.21.15.112:0/812447735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd94c0c72b0 0x7fd94c14ec10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00c4a0 tx=0x7fd93c00c960 comp rx=0 tx=0).stop 2024-04-25T01:40:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.824+0000 7fd94b7fe640 1 -- 172.21.15.112:0/812447735 reap_dead start 2024-04-25T01:40:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.824+0000 7fd94b7fe640 1 -- 172.21.15.112:0/812447735 reap_dead start 2024-04-25T01:40:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.824+0000 7fd951a10640 1 -- 172.21.15.112:0/812447735 shutdown_connections 2024-04-25T01:40:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.824+0000 7fd951a10640 1 -- 172.21.15.112:0/812447735 >> 172.21.15.112:0/812447735 conn(0x7fd94c0cb170 msgr2=0x7fd94c0cb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.825+0000 7fd951a10640 1 -- 172.21.15.112:0/812447735 shutdown_connections 2024-04-25T01:40:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.825+0000 7fd951a10640 1 -- 172.21.15.112:0/812447735 wait complete. 2024-04-25T01:40:42.834 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:40:42.834 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T01:40:42.834 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T01:40:42.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.947+0000 7ff03fa6b640 1 Processor -- start 2024-04-25T01:40:42.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.948+0000 7ff03fa6b640 1 -- start start 2024-04-25T01:40:42.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.948+0000 7ff03fa6b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff0381592c0 0x7ff0380c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.948+0000 7ff03fa6b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff0380c72b0 0x7ff038160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.948+0000 7ff03fa6b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff038159b90 con 0x7ff0381596a0 2024-04-25T01:40:42.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.948+0000 7ff03fa6b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7ff0380c6e20 con 0x7ff0381592c0 2024-04-25T01:40:42.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.948+0000 7ff03fa6b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff03806c970 con 0x7ff0380c72b0 2024-04-25T01:40:42.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.949+0000 7ff03d807640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff0381592c0 0x7ff0380c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.949+0000 7ff03d006640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff0381596a0 0x7ff038152d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:41340/0 (socket says 172.21.15.112:41340) 2024-04-25T01:40:42.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.949+0000 7ff03d807640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff0381592c0 0x7ff0380c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42348/0 (socket says 172.21.15.112:42348) 2024-04-25T01:40:42.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.949+0000 7ff03d006640 1 -- 172.21.15.112:0/1244648821 learned_addr learned my addr 172.21.15.112:0/1244648821 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:40:42.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.949+0000 7ff03e008640 1 --2- 172.21.15.112:0/1244648821 >> v2:172.21.15.187:3300/0 conn(0x7ff0380c72b0 0x7ff038160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.949+0000 7ff03d807640 1 -- 172.21.15.112:0/1244648821 >> v2:172.21.15.187:3300/0 conn(0x7ff0380c72b0 msgr2=0x7ff038160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:42.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.949+0000 7ff03d807640 1 --2- 172.21.15.112:0/1244648821 >> v2:172.21.15.187:3300/0 conn(0x7ff0380c72b0 0x7ff038160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.949+0000 7ff03d807640 1 -- 172.21.15.112:0/1244648821 >> v1:172.21.15.112:6789/0 conn(0x7ff0381596a0 legacy=0x7ff038152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.949+0000 7ff03d807640 1 -- 172.21.15.112:0/1244648821 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03806cc50 con 0x7ff0381592c0 2024-04-25T01:40:42.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.949+0000 7ff03d807640 1 --2- 172.21.15.112:0/1244648821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0381592c0 0x7ff0380c9070 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff0380c21d0 tx=0x7ff03002a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d2c2cf689d38fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:42.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.950+0000 7ff03c805640 1 -- 172.21.15.112:0/1244648821 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030007740 con 0x7ff0381592c0 2024-04-25T01:40:42.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.950+0000 7ff03c805640 1 -- 172.21.15.112:0/1244648821 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0300078e0 con 0x7ff0381592c0 2024-04-25T01:40:42.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.950+0000 7ff03c805640 1 -- 172.21.15.112:0/1244648821 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030007cb0 con 0x7ff0381592c0 2024-04-25T01:40:42.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.950+0000 7ff03fa6b640 1 -- 172.21.15.112:0/1244648821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0381592c0 msgr2=0x7ff0380c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.950+0000 7ff03fa6b640 1 --2- 172.21.15.112:0/1244648821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0381592c0 0x7ff0380c9070 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff0380c21d0 tx=0x7ff03002a560 comp rx=0 tx=0).stop 2024-04-25T01:40:42.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.950+0000 7ff03fa6b640 1 -- 172.21.15.112:0/1244648821 shutdown_connections 2024-04-25T01:40:42.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.950+0000 7ff03fa6b640 1 --2- 172.21.15.112:0/1244648821 >> v2:172.21.15.187:3300/0 conn(0x7ff0380c72b0 0x7ff038160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.950+0000 7ff03fa6b640 1 --2- 172.21.15.112:0/1244648821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0381592c0 0x7ff0380c9070 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.950+0000 7ff03fa6b640 1 -- 172.21.15.112:0/1244648821 >> 172.21.15.112:0/1244648821 conn(0x7ff0380cb170 msgr2=0x7ff0380cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:42.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.950+0000 7ff03fa6b640 1 -- 172.21.15.112:0/1244648821 shutdown_connections 2024-04-25T01:40:42.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.950+0000 7ff03fa6b640 1 -- 172.21.15.112:0/1244648821 wait complete. 2024-04-25T01:40:42.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.951+0000 7ff03fa6b640 1 Processor -- start 2024-04-25T01:40:42.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.951+0000 7ff03fa6b640 1 -- start start 2024-04-25T01:40:42.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.952+0000 7ff03fa6b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0380c72b0 0x7ff038158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.952+0000 7ff03fa6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0381592c0 0x7ff038154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.952+0000 7ff03fa6b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0381596a0 0x7ff0381553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.952+0000 7ff03fa6b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff038153590 con 0x7ff0381592c0 2024-04-25T01:40:42.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.952+0000 7ff03fa6b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff038153410 con 0x7ff0380c72b0 2024-04-25T01:40:42.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.952+0000 7ff03fa6b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff03806db20 con 0x7ff0381596a0 2024-04-25T01:40:42.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.952+0000 7ff03d006640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0381592c0 0x7ff038154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.952+0000 7ff03d006640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0381592c0 0x7ff038154e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42356/0 (socket says 172.21.15.112:42356) 2024-04-25T01:40:42.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.952+0000 7ff03d006640 1 -- 172.21.15.112:0/4032418762 learned_addr learned my addr 172.21.15.112:0/4032418762 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:42.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.952+0000 7ff03e008640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0381596a0 0x7ff0381553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.952+0000 7ff03d807640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0380c72b0 0x7ff038158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.952+0000 7ff03d006640 1 -- 172.21.15.112:0/4032418762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0381596a0 msgr2=0x7ff0381553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.952+0000 7ff03d006640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0381596a0 0x7ff0381553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.952+0000 7ff03d006640 1 -- 172.21.15.112:0/4032418762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0380c72b0 msgr2=0x7ff038158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.952+0000 7ff03d006640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0380c72b0 0x7ff038158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.952+0000 7ff03d006640 1 -- 172.21.15.112:0/4032418762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03814bc90 con 0x7ff0381592c0 2024-04-25T01:40:42.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.953+0000 7ff03d006640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0381592c0 0x7ff038154e60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff02800c470 tx=0x7ff02800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:42.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.953+0000 7ff02e7fc640 1 -- 172.21.15.112:0/4032418762 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff028017070 con 0x7ff0381592c0 2024-04-25T01:40:42.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.953+0000 7ff02e7fc640 1 -- 172.21.15.112:0/4032418762 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff028005ce0 con 0x7ff0381592c0 2024-04-25T01:40:42.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.953+0000 7ff02e7fc640 1 -- 172.21.15.112:0/4032418762 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0280072d0 con 0x7ff0381592c0 2024-04-25T01:40:42.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.953+0000 7ff03fa6b640 1 -- 172.21.15.112:0/4032418762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03814bf30 con 0x7ff0381592c0 2024-04-25T01:40:42.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.953+0000 7ff03fa6b640 1 -- 172.21.15.112:0/4032418762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03814c4f0 con 0x7ff0381592c0 2024-04-25T01:40:42.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.954+0000 7ff03fa6b640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff03814a0b0 0x7ff038153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.954+0000 7ff03fa6b640 1 -- 172.21.15.112:0/4032418762 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff038075980 con 0x7ff03814a0b0 2024-04-25T01:40:42.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.954+0000 7ff03d807640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff03814a0b0 0x7ff038153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.954+0000 7ff03d807640 1 -- 172.21.15.112:0/4032418762 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff0380679a0 con 0x7ff03814a0b0 2024-04-25T01:40:42.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.954+0000 7ff02e7fc640 1 -- 172.21.15.112:0/4032418762 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff02801a070 con 0x7ff0381592c0 2024-04-25T01:40:42.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.954+0000 7ff03d807640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff03814a0b0 0x7ff038153c80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff0380bea40 tx=0x7ff030004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:42.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.954+0000 7ff02e7fc640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff01c06be60 0x7ff01c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.955+0000 7ff03e008640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff01c06be60 0x7ff01c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.955+0000 7ff02e7fc640 1 -- 172.21.15.112:0/4032418762 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7ff02808a360 con 0x7ff0381592c0 2024-04-25T01:40:42.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.955+0000 7ff02e7fc640 1 -- 172.21.15.112:0/4032418762 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03003c8f0 con 0x7ff03814a0b0 2024-04-25T01:40:42.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.955+0000 7ff02e7fc640 1 -- 172.21.15.112:0/4032418762 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff030045420 con 0x7ff03814a0b0 2024-04-25T01:40:42.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.955+0000 7ff02e7fc640 1 -- 172.21.15.112:0/4032418762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff03814a0b0 msgr2=0x7ff038153c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.955+0000 7ff02e7fc640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff03814a0b0 0x7ff038153c80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff0380bea40 tx=0x7ff030004a40 comp rx=0 tx=0).stop 2024-04-25T01:40:42.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.955+0000 7ff02e7fc640 1 -- 172.21.15.112:0/4032418762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff03814a0b0 msgr2=0x7ff038153c80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:40:42.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.955+0000 7ff02e7fc640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff03814a0b0 0x7ff038153c80 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff0380bea40 tx=0x7ff030004a40 comp rx=0 tx=0).stop 2024-04-25T01:40:42.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.955+0000 7ff03e008640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff01c06be60 0x7ff01c06e320 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff034006fd0 tx=0x7ff034008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:42.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.961+0000 7ff03fa6b640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0381541c0 0x7ff0381545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:42.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.961+0000 7ff03fa6b640 1 -- 172.21.15.112:0/4032418762 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0380682a0 con 0x7ff0381541c0 2024-04-25T01:40:42.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.962+0000 7ff03d807640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0381541c0 0x7ff0381545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:42.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.962+0000 7ff03d807640 1 -- 172.21.15.112:0/4032418762 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff038150520 con 0x7ff0381541c0 2024-04-25T01:40:42.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.962+0000 7ff03d807640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0381541c0 0x7ff0381545d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff030000f80 tx=0x7ff030030000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:42.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.962+0000 7ff02e7fc640 1 -- 172.21.15.112:0/4032418762 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030004250 con 0x7ff0381541c0 2024-04-25T01:40:42.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.962+0000 7ff02e7fc640 1 -- 172.21.15.112:0/4032418762 <== mon.1 v2:172.21.15.187:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4730 (secure 0 0 0) 0x7ff030004250 con 0x7ff0381541c0 2024-04-25T01:40:42.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.962+0000 7ff02e7fc640 1 -- 172.21.15.112:0/4032418762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0381541c0 msgr2=0x7ff0381545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.962+0000 7ff02e7fc640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0381541c0 0x7ff0381545d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff030000f80 tx=0x7ff030030000 comp rx=0 tx=0).stop 2024-04-25T01:40:42.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.962+0000 7ff02e7fc640 1 -- 172.21.15.112:0/4032418762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0381541c0 msgr2=0x7ff0381545d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:40:42.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.962+0000 7ff02e7fc640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0381541c0 0x7ff0381545d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.948 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:40:42.949 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:40:42.949 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:40:42.949 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:40:42.949 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 118, 2024-04-25T01:40:42.949 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 147727, 2024-04-25T01:40:42.949 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:40:42.949 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:40:42.949 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:40:42.949 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:40:42.949 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:42.949 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:40:42.949 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:40:42.949 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:40:42.949 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:40:42.950 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:40:42.950 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:40:42.950 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:40:42.950 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:40:42.950 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:40:42.950 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:40:42.950 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:40:42.950 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:40:42.950 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:40:42.950 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:40:42.950 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:40:42.950 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:42.950 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:40:42.950 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:40:42.951 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:40:42.951 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:40:42.951 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:40:42.951 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:40:42.951 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:40:42.951 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:40:42.951 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:40:42.951 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:40:42.951 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:40:42.951 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:40:42.951 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:40:42.951 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:42.951 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.951 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:40:42.951 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:40:42.952 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:40:42.952 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:40:42.952 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:40:42.952 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:40:42.952 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:40:42.952 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:40:42.952 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:40:42.952 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:40:42.952 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:40:42.952 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:40:42.952 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:40:42.952 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:40:42.952 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:40:42.952 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:40:42.952 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:40:42.953 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:40:42.953 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:40:42.953 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:40:42.953 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:40:42.953 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:40:42.953 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:40:42.953 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:40:42.953 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:40:42.953 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:40:42.953 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:40:42.953 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:40:42.953 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:42.953 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:40:42.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.953 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:40:42.953 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.954 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:40:42.954 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:40:42.954 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:40:42.954 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:40:42.954 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.954 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:40:42.954 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:40:42.954 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:42.954 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.954 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.954 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:40:42.954 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:40:42.954 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:42.954 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:42.955 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:42.955 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.955 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:40:42.955 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:40:42.955 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:40:42.955 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:40:42.955 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:40:42.955 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.955 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.955 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:40:42.955 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:40:42.955 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:40:42.955 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:40:42.955 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.956 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:40:42.956 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:40:42.956 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:42.956 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.956 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.956 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:40:42.956 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:40:42.956 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:42.956 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:42.956 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:42.956 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.956 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:40:42.956 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:40:42.956 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:40:42.956 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:40:42.957 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:40:42.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.957 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.957 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:40:42.957 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:40:42.957 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:40:42.957 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:40:42.957 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.957 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:40:42.957 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:40:42.957 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:42.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.957 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.957 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:40:42.957 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:40:42.958 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:42.958 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:42.958 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:42.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.958 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:40:42.958 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:40:42.958 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:40:42.958 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:40:42.958 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:40:42.958 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:42.958 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:42.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.958 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:40:42.958 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:40:42.959 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.959 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:40:42.959 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:40:42.959 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:40:42.959 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:42.959 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:42.959 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:40:42.959 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.959 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:40:42.959 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:40:42.959 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:40:42.959 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:42.959 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:42.959 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:40:42.959 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:42.960 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:40:42.960 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:40:42.960 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:40:42.960 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:42.960 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:42.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:42.960 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:40:42.960 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:40:42.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.963+0000 7ff03fa6b640 1 -- 172.21.15.112:0/4032418762 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff01c06be60 msgr2=0x7ff01c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.963+0000 7ff03fa6b640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff01c06be60 0x7ff01c06e320 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff034006fd0 tx=0x7ff034008040 comp rx=0 tx=0).stop 2024-04-25T01:40:42.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.963+0000 7ff03fa6b640 1 -- 172.21.15.112:0/4032418762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0381592c0 msgr2=0x7ff038154e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:42.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.963+0000 7ff03fa6b640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0381592c0 0x7ff038154e60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff02800c470 tx=0x7ff02800c930 comp rx=0 tx=0).stop 2024-04-25T01:40:42.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.963+0000 7ff03e008640 1 -- 172.21.15.112:0/4032418762 reap_dead start 2024-04-25T01:40:42.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.963+0000 7ff03fa6b640 1 -- 172.21.15.112:0/4032418762 shutdown_connections 2024-04-25T01:40:42.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.963+0000 7ff03fa6b640 1 --2- 172.21.15.112:0/4032418762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0381592c0 0x7ff038154e60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:42.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.963+0000 7ff03fa6b640 1 -- 172.21.15.112:0/4032418762 >> 172.21.15.112:0/4032418762 conn(0x7ff0380cb170 msgr2=0x7ff0380ab5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:42.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.963+0000 7ff03fa6b640 1 -- 172.21.15.112:0/4032418762 shutdown_connections 2024-04-25T01:40:42.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:42.963+0000 7ff03fa6b640 1 -- 172.21.15.112:0/4032418762 wait complete. 2024-04-25T01:40:42.962 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:40:42.962 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T01:40:42.963 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T01:40:43.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.075+0000 7ff86ae7d640 1 Processor -- start 2024-04-25T01:40:43.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.075+0000 7ff86ae7d640 1 -- start start 2024-04-25T01:40:43.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.076+0000 7ff86ae7d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff86406f1f0 0x7ff864070d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:43.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.076+0000 7ff86ae7d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8641634f0 con 0x7ff8640712a0 2024-04-25T01:40:43.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.076+0000 7ff86ae7d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8640ab370 con 0x7ff864072d50 2024-04-25T01:40:43.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.076+0000 7ff86ae7d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7ff864162fd0 con 0x7ff86406f1f0 2024-04-25T01:40:43.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.076+0000 7ff868c19640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff86406f1f0 0x7ff864070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:43.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.076+0000 7ff863fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff8640712a0 0x7ff8640abb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:41356/0 (socket says 172.21.15.112:41356) 2024-04-25T01:40:43.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.076+0000 7ff863fff640 1 -- 172.21.15.112:0/3606924744 learned_addr learned my addr 172.21.15.112:0/3606924744 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:40:43.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.076+0000 7ff868c19640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff86406f1f0 0x7ff864070d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42364/0 (socket says 172.21.15.112:42364) 2024-04-25T01:40:43.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.076+0000 7ff868c19640 1 -- 172.21.15.112:0/3606924744 >> v1:172.21.15.187:6789/0 conn(0x7ff864072d50 legacy=0x7ff86415f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:43.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.076+0000 7ff868c19640 1 -- 172.21.15.112:0/3606924744 >> v1:172.21.15.112:6789/0 conn(0x7ff8640712a0 legacy=0x7ff8640abb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:43.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.076+0000 7ff8637fe640 1 -- 172.21.15.112:0/3606924744 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1446400774 0 0) 0x7ff8641634f0 con 0x7ff8640712a0 2024-04-25T01:40:43.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.076+0000 7ff868c19640 1 -- 172.21.15.112:0/3606924744 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff864163210 con 0x7ff86406f1f0 2024-04-25T01:40:43.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.077+0000 7ff868c19640 1 --2- 172.21.15.112:0/3606924744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86406f1f0 0x7ff864070d60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff8580097a0 tx=0x7ff85802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c736e2b4ae781b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:43.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.077+0000 7ff8637fe640 1 -- 172.21.15.112:0/3606924744 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858037070 con 0x7ff86406f1f0 2024-04-25T01:40:43.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.077+0000 7ff8637fe640 1 -- 172.21.15.112:0/3606924744 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff85802adc0 con 0x7ff86406f1f0 2024-04-25T01:40:43.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.077+0000 7ff8637fe640 1 -- 172.21.15.112:0/3606924744 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858032660 con 0x7ff86406f1f0 2024-04-25T01:40:43.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.077+0000 7ff86ae7d640 1 -- 172.21.15.112:0/3606924744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86406f1f0 msgr2=0x7ff864070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:43.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.077+0000 7ff86ae7d640 1 --2- 172.21.15.112:0/3606924744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86406f1f0 0x7ff864070d60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff8580097a0 tx=0x7ff85802a0b0 comp rx=0 tx=0).stop 2024-04-25T01:40:43.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.078+0000 7ff86ae7d640 1 -- 172.21.15.112:0/3606924744 shutdown_connections 2024-04-25T01:40:43.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.078+0000 7ff86ae7d640 1 --2- 172.21.15.112:0/3606924744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86406f1f0 0x7ff864070d60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:43.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.078+0000 7ff86ae7d640 1 -- 172.21.15.112:0/3606924744 >> 172.21.15.112:0/3606924744 conn(0x7ff8640c7080 msgr2=0x7ff8640c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:43.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.078+0000 7ff86ae7d640 1 -- 172.21.15.112:0/3606924744 shutdown_connections 2024-04-25T01:40:43.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.078+0000 7ff86ae7d640 1 -- 172.21.15.112:0/3606924744 wait complete. 2024-04-25T01:40:43.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.079+0000 7ff86ae7d640 1 Processor -- start 2024-04-25T01:40:43.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.079+0000 7ff86ae7d640 1 -- start start 2024-04-25T01:40:43.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.079+0000 7ff86ae7d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86406f1f0 0x7ff8640bf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:43.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.079+0000 7ff86ae7d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8640712a0 0x7ff8641676c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:43.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.079+0000 7ff86ae7d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff864072d50 0x7ff8640bd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:43.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.079+0000 7ff86ae7d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff8640aa7f0 con 0x7ff86406f1f0 2024-04-25T01:40:43.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.079+0000 7ff86ae7d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff8640c1a80 con 0x7ff864072d50 2024-04-25T01:40:43.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.079+0000 7ff86ae7d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff86406f5d0 con 0x7ff8640712a0 2024-04-25T01:40:43.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.080+0000 7ff868c19640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86406f1f0 0x7ff8640bf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:43.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.080+0000 7ff868c19640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86406f1f0 0x7ff8640bf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42380/0 (socket says 172.21.15.112:42380) 2024-04-25T01:40:43.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.080+0000 7ff868c19640 1 -- 172.21.15.112:0/2124458133 learned_addr learned my addr 172.21.15.112:0/2124458133 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:43.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.080+0000 7ff86941a640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff864072d50 0x7ff8640bd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:43.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.080+0000 7ff863fff640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8640712a0 0x7ff8641676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:43.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.080+0000 7ff868c19640 1 -- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8640712a0 msgr2=0x7ff8641676c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:43.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.080+0000 7ff868c19640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8640712a0 0x7ff8641676c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:43.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.080+0000 7ff868c19640 1 -- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff864072d50 msgr2=0x7ff8640bd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:43.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.080+0000 7ff868c19640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff864072d50 0x7ff8640bd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:43.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.080+0000 7ff868c19640 1 -- 172.21.15.112:0/2124458133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff864167c00 con 0x7ff86406f1f0 2024-04-25T01:40:43.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.080+0000 7ff86941a640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff864072d50 0x7ff8640bd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:40:43.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.080+0000 7ff868c19640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86406f1f0 0x7ff8640bf7e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff8580097a0 tx=0x7ff8580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:43.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.080+0000 7ff8617fa640 1 -- 172.21.15.112:0/2124458133 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858037040 con 0x7ff86406f1f0 2024-04-25T01:40:43.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.080+0000 7ff8617fa640 1 -- 172.21.15.112:0/2124458133 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff858004030 con 0x7ff86406f1f0 2024-04-25T01:40:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.080+0000 7ff86ae7d640 1 -- 172.21.15.112:0/2124458133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8640be790 con 0x7ff86406f1f0 2024-04-25T01:40:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.080+0000 7ff8617fa640 1 -- 172.21.15.112:0/2124458133 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858004400 con 0x7ff86406f1f0 2024-04-25T01:40:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.080+0000 7ff86ae7d640 1 -- 172.21.15.112:0/2124458133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8640bed50 con 0x7ff86406f1f0 2024-04-25T01:40:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.081+0000 7ff86ae7d640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff86415f710 0x7ff86415faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.081+0000 7ff86ae7d640 1 -- 172.21.15.112:0/2124458133 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff8640c17f0 con 0x7ff86415f710 2024-04-25T01:40:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.081+0000 7ff863fff640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff86415f710 0x7ff86415faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.082+0000 7ff8617fa640 1 -- 172.21.15.112:0/2124458133 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff858003610 con 0x7ff86406f1f0 2024-04-25T01:40:43.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.082+0000 7ff863fff640 1 -- 172.21.15.112:0/2124458133 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff864075980 con 0x7ff86415f710 2024-04-25T01:40:43.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.082+0000 7ff8617fa640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff84806be30 0x7ff84806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:43.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.082+0000 7ff8617fa640 1 -- 172.21.15.112:0/2124458133 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7ff8580ac030 con 0x7ff86406f1f0 2024-04-25T01:40:43.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.082+0000 7ff86941a640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff84806be30 0x7ff84806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:43.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.082+0000 7ff863fff640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff86415f710 0x7ff86415faf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff8641626c0 tx=0x7ff8500018b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:43.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.082+0000 7ff8617fa640 1 -- 172.21.15.112:0/2124458133 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850012590 con 0x7ff86415f710 2024-04-25T01:40:43.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.082+0000 7ff86941a640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff84806be30 0x7ff84806e2f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff864160580 tx=0x7ff85c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:43.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.083+0000 7ff8617fa640 1 -- 172.21.15.112:0/2124458133 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff850018070 con 0x7ff86415f710 2024-04-25T01:40:43.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.083+0000 7ff8617fa640 1 -- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff86415f710 msgr2=0x7ff86415faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:43.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.083+0000 7ff8617fa640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff86415f710 0x7ff86415faf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff8641626c0 tx=0x7ff8500018b0 comp rx=0 tx=0).stop 2024-04-25T01:40:43.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.083+0000 7ff8617fa640 1 -- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff86415f710 msgr2=0x7ff86415faf0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:40:43.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.083+0000 7ff8617fa640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff86415f710 0x7ff86415faf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:43.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.089+0000 7ff86ae7d640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8641fdb50 0x7ff8641fdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:43.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.089+0000 7ff86ae7d640 1 -- 172.21.15.112:0/2124458133 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff8640c3430 con 0x7ff8641fdb50 2024-04-25T01:40:43.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.089+0000 7ff863fff640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8641fdb50 0x7ff8641fdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:43.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.090+0000 7ff863fff640 1 -- 172.21.15.112:0/2124458133 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff8640682a0 con 0x7ff8641fdb50 2024-04-25T01:40:43.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.090+0000 7ff863fff640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8641fdb50 0x7ff8641fdf30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff86415f370 tx=0x7ff850001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:43.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.090+0000 7ff8617fa640 1 -- 172.21.15.112:0/2124458133 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850012d70 con 0x7ff8641fdb50 2024-04-25T01:40:43.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.090+0000 7ff8617fa640 1 -- 172.21.15.112:0/2124458133 <== mon.2 v2:172.21.15.187:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5070 (secure 0 0 0) 0x7ff850012d70 con 0x7ff8641fdb50 2024-04-25T01:40:43.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.090+0000 7ff8617fa640 1 -- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8641fdb50 msgr2=0x7ff8641fdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:43.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.090+0000 7ff8617fa640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8641fdb50 0x7ff8641fdf30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff86415f370 tx=0x7ff850001080 comp rx=0 tx=0).stop 2024-04-25T01:40:43.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.090+0000 7ff8617fa640 1 -- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8641fdb50 msgr2=0x7ff8641fdf30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T01:40:43.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.090+0000 7ff8617fa640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8641fdb50 0x7ff8641fdf30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:43.076 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:40:43.076 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:40:43.076 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:40:43.077 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "peon", 2024-04-25T01:40:43.077 INFO:teuthology.orchestra.run.smithi112.stdout: "election_epoch": 118, 2024-04-25T01:40:43.077 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 212456, 2024-04-25T01:40:43.077 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum": [ 2024-04-25T01:40:43.077 INFO:teuthology.orchestra.run.smithi112.stdout: 0, 2024-04-25T01:40:43.077 INFO:teuthology.orchestra.run.smithi112.stdout: 1, 2024-04-25T01:40:43.077 INFO:teuthology.orchestra.run.smithi112.stdout: 2 2024-04-25T01:40:43.077 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:43.077 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_age": 1, 2024-04-25T01:40:43.077 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:40:43.077 INFO:teuthology.orchestra.run.smithi112.stdout: "required_con": "2450521708007866372", 2024-04-25T01:40:43.077 INFO:teuthology.orchestra.run.smithi112.stdout: "required_mon": [ 2024-04-25T01:40:43.077 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:40:43.078 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:40:43.078 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:40:43.078 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:40:43.078 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:40:43.078 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:40:43.078 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:40:43.078 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:40:43.078 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:40:43.078 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:40:43.079 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:40:43.079 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:43.079 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_con": "4540701547738038271", 2024-04-25T01:40:43.079 INFO:teuthology.orchestra.run.smithi112.stdout: "quorum_mon": [ 2024-04-25T01:40:43.079 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:40:43.079 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:40:43.079 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:40:43.079 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:40:43.079 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:40:43.079 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:40:43.079 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:40:43.079 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:40:43.079 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:40:43.079 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:40:43.079 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:40:43.080 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:43.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:43.080 INFO:teuthology.orchestra.run.smithi112.stdout: "outside_quorum": [], 2024-04-25T01:40:43.080 INFO:teuthology.orchestra.run.smithi112.stdout: "extra_probe_peers": [], 2024-04-25T01:40:43.080 INFO:teuthology.orchestra.run.smithi112.stdout: "sync_provider": [], 2024-04-25T01:40:43.080 INFO:teuthology.orchestra.run.smithi112.stdout: "monmap": { 2024-04-25T01:40:43.080 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 1, 2024-04-25T01:40:43.080 INFO:teuthology.orchestra.run.smithi112.stdout: "fsid": "a88d8313-513a-4f1d-8e1c-e81e1847e3c2", 2024-04-25T01:40:43.080 INFO:teuthology.orchestra.run.smithi112.stdout: "modified": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:40:43.080 INFO:teuthology.orchestra.run.smithi112.stdout: "created": "2024-04-25T01:24:54.928910Z", 2024-04-25T01:40:43.080 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release": 19, 2024-04-25T01:40:43.080 INFO:teuthology.orchestra.run.smithi112.stdout: "min_mon_release_name": "squid", 2024-04-25T01:40:43.080 INFO:teuthology.orchestra.run.smithi112.stdout: "election_strategy": 1, 2024-04-25T01:40:43.080 INFO:teuthology.orchestra.run.smithi112.stdout: "disallowed_leaders: ": "", 2024-04-25T01:40:43.081 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false, 2024-04-25T01:40:43.081 INFO:teuthology.orchestra.run.smithi112.stdout: "tiebreaker_mon": "", 2024-04-25T01:40:43.081 INFO:teuthology.orchestra.run.smithi112.stdout: "removed_ranks: ": "", 2024-04-25T01:40:43.081 INFO:teuthology.orchestra.run.smithi112.stdout: "features": { 2024-04-25T01:40:43.081 INFO:teuthology.orchestra.run.smithi112.stdout: "persistent": [ 2024-04-25T01:40:43.081 INFO:teuthology.orchestra.run.smithi112.stdout: "kraken", 2024-04-25T01:40:43.081 INFO:teuthology.orchestra.run.smithi112.stdout: "luminous", 2024-04-25T01:40:43.081 INFO:teuthology.orchestra.run.smithi112.stdout: "mimic", 2024-04-25T01:40:43.081 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap-prune", 2024-04-25T01:40:43.081 INFO:teuthology.orchestra.run.smithi112.stdout: "nautilus", 2024-04-25T01:40:43.081 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus", 2024-04-25T01:40:43.081 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific", 2024-04-25T01:40:43.081 INFO:teuthology.orchestra.run.smithi112.stdout: "elector-pinging", 2024-04-25T01:40:43.081 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy", 2024-04-25T01:40:43.081 INFO:teuthology.orchestra.run.smithi112.stdout: "reef", 2024-04-25T01:40:43.082 INFO:teuthology.orchestra.run.smithi112.stdout: "squid" 2024-04-25T01:40:43.082 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:43.082 INFO:teuthology.orchestra.run.smithi112.stdout: "optional": [] 2024-04-25T01:40:43.082 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:43.082 INFO:teuthology.orchestra.run.smithi112.stdout: "mons": [ 2024-04-25T01:40:43.082 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:43.082 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 0, 2024-04-25T01:40:43.082 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "a", 2024-04-25T01:40:43.082 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:40:43.082 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:40:43.082 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:43.082 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:40:43.082 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:3300", 2024-04-25T01:40:43.082 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:43.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:43.083 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:43.083 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:40:43.083 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789", 2024-04-25T01:40:43.083 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:43.083 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:43.083 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:43.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:43.083 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6789/0", 2024-04-25T01:40:43.083 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.112:6789/0", 2024-04-25T01:40:43.083 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:40:43.083 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:40:43.083 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:40:43.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:43.084 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:43.084 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 1, 2024-04-25T01:40:43.084 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "b", 2024-04-25T01:40:43.084 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:40:43.084 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:40:43.084 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:43.084 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:40:43.084 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3300", 2024-04-25T01:40:43.084 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:43.084 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:43.084 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:43.084 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:40:43.084 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789", 2024-04-25T01:40:43.084 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:43.084 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:43.085 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:43.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:43.085 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6789/0", 2024-04-25T01:40:43.085 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6789/0", 2024-04-25T01:40:43.085 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:40:43.085 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:40:43.085 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:40:43.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:43.085 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:43.085 INFO:teuthology.orchestra.run.smithi112.stdout: "rank": 2, 2024-04-25T01:40:43.085 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "c", 2024-04-25T01:40:43.085 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addrs": { 2024-04-25T01:40:43.085 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T01:40:43.085 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:43.086 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T01:40:43.086 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:3301", 2024-04-25T01:40:43.086 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:43.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:43.086 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:43.086 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T01:40:43.086 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790", 2024-04-25T01:40:43.086 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 0 2024-04-25T01:40:43.086 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:43.086 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:43.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:43.086 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:6790/0", 2024-04-25T01:40:43.086 INFO:teuthology.orchestra.run.smithi112.stdout: "public_addr": "172.21.15.187:6790/0", 2024-04-25T01:40:43.086 INFO:teuthology.orchestra.run.smithi112.stdout: "priority": 0, 2024-04-25T01:40:43.087 INFO:teuthology.orchestra.run.smithi112.stdout: "weight": 0, 2024-04-25T01:40:43.087 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_location": "{}" 2024-04-25T01:40:43.087 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:43.087 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:43.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:43.087 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_map": { 2024-04-25T01:40:43.087 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": [ 2024-04-25T01:40:43.087 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:43.087 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:40:43.087 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:40:43.087 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 1 2024-04-25T01:40:43.087 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:43.087 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:43.087 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": [ 2024-04-25T01:40:43.087 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:43.087 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:40:43.088 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:40:43.088 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 3 2024-04-25T01:40:43.088 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:43.088 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:43.088 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": [ 2024-04-25T01:40:43.088 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:43.088 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:40:43.088 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:40:43.088 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 8 2024-04-25T01:40:43.088 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:43.088 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:43.088 INFO:teuthology.orchestra.run.smithi112.stdout: "client": [ 2024-04-25T01:40:43.088 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:43.088 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:40:43.088 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:40:43.089 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5 2024-04-25T01:40:43.089 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:43.089 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T01:40:43.089 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": [ 2024-04-25T01:40:43.089 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T01:40:43.089 INFO:teuthology.orchestra.run.smithi112.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T01:40:43.089 INFO:teuthology.orchestra.run.smithi112.stdout: "release": "luminous", 2024-04-25T01:40:43.089 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 2 2024-04-25T01:40:43.089 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T01:40:43.089 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T01:40:43.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T01:40:43.089 INFO:teuthology.orchestra.run.smithi112.stdout: "stretch_mode": false 2024-04-25T01:40:43.089 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:40:43.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.090+0000 7ff86ae7d640 1 -- 172.21.15.112:0/2124458133 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff84806be30 msgr2=0x7ff84806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:43.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.090+0000 7ff86ae7d640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff84806be30 0x7ff84806e2f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff864160580 tx=0x7ff85c009040 comp rx=0 tx=0).stop 2024-04-25T01:40:43.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.091+0000 7ff86ae7d640 1 -- 172.21.15.112:0/2124458133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86406f1f0 msgr2=0x7ff8640bf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:43.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.091+0000 7ff86ae7d640 1 --2- 172.21.15.112:0/2124458133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86406f1f0 0x7ff8640bf7e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff8580097a0 tx=0x7ff8580046c0 comp rx=0 tx=0).stop 2024-04-25T01:40:43.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.091+0000 7ff86941a640 1 -- 172.21.15.112:0/2124458133 reap_dead start 2024-04-25T01:40:43.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.091+0000 7ff86941a640 1 -- 172.21.15.112:0/2124458133 reap_dead start 2024-04-25T01:40:43.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.091+0000 7ff86ae7d640 1 -- 172.21.15.112:0/2124458133 shutdown_connections 2024-04-25T01:40:43.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.091+0000 7ff86ae7d640 1 -- 172.21.15.112:0/2124458133 >> 172.21.15.112:0/2124458133 conn(0x7ff8640c7080 msgr2=0x7ff8641619b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:43.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.091+0000 7ff86ae7d640 1 -- 172.21.15.112:0/2124458133 shutdown_connections 2024-04-25T01:40:43.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.091+0000 7ff86ae7d640 1 -- 172.21.15.112:0/2124458133 wait complete. 2024-04-25T01:40:43.092 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T01:40:43.092 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T01:40:43.092 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T01:40:43.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.203+0000 7f3cda42a640 1 Processor -- start 2024-04-25T01:40:43.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.203+0000 7f3cda42a640 1 -- start start 2024-04-25T01:40:43.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.203+0000 7f3cda42a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3cd41596a0 0x7f3cd4152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:43.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.203+0000 7f3cda42a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cd4159b90 con 0x7f3cd40c72b0 2024-04-25T01:40:43.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.203+0000 7f3cda42a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cd406c970 con 0x7f3cd41592c0 2024-04-25T01:40:43.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.203+0000 7f3cda42a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f3cd40c6e20 con 0x7f3cd41596a0 2024-04-25T01:40:43.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.203+0000 7f3cd37fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3cd41596a0 0x7f3cd4152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:43.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.203+0000 7f3cd37fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3cd41596a0 0x7f3cd4152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42394/0 (socket says 172.21.15.112:42394) 2024-04-25T01:40:43.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.203+0000 7f3cd37fe640 1 -- 172.21.15.112:0/3039087405 learned_addr learned my addr 172.21.15.112:0/3039087405 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:43.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.203+0000 7f3cd89c7640 1 --1- 172.21.15.112:0/3039087405 >> v1:172.21.15.112:6789/0 conn(0x7f3cd40c72b0 0x7f3cd4160380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:41368/0 (socket says 172.21.15.112:41368) 2024-04-25T01:40:43.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.204+0000 7f3cd37fe640 1 -- 172.21.15.112:0/3039087405 >> v1:172.21.15.187:6789/0 conn(0x7f3cd41592c0 legacy=0x7f3cd40c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:43.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.204+0000 7f3cd37fe640 1 -- 172.21.15.112:0/3039087405 >> v1:172.21.15.112:6789/0 conn(0x7f3cd40c72b0 legacy=0x7f3cd4160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:43.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.204+0000 7f3cd37fe640 1 -- 172.21.15.112:0/3039087405 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd406cc50 con 0x7f3cd41596a0 2024-04-25T01:40:43.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.204+0000 7f3cd37fe640 1 --2- 172.21.15.112:0/3039087405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cd41596a0 0x7f3cd4152d00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009870 tx=0x7f3cc802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aeffb2656e40c62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:43.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.205+0000 7f3cd2ffd640 1 -- 172.21.15.112:0/3039087405 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8037070 con 0x7f3cd41596a0 2024-04-25T01:40:43.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.205+0000 7f3cd2ffd640 1 -- 172.21.15.112:0/3039087405 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc8032440 con 0x7f3cd41596a0 2024-04-25T01:40:43.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.205+0000 7f3cd2ffd640 1 -- 172.21.15.112:0/3039087405 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8032810 con 0x7f3cd41596a0 2024-04-25T01:40:43.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.205+0000 7f3cda42a640 1 -- 172.21.15.112:0/3039087405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cd41596a0 msgr2=0x7f3cd4152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:43.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.205+0000 7f3cda42a640 1 --2- 172.21.15.112:0/3039087405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cd41596a0 0x7f3cd4152d00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009870 tx=0x7f3cc802a180 comp rx=0 tx=0).stop 2024-04-25T01:40:43.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.205+0000 7f3cda42a640 1 -- 172.21.15.112:0/3039087405 shutdown_connections 2024-04-25T01:40:43.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.205+0000 7f3cda42a640 1 --2- 172.21.15.112:0/3039087405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cd41596a0 0x7f3cd4152d00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:43.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.205+0000 7f3cda42a640 1 -- 172.21.15.112:0/3039087405 >> 172.21.15.112:0/3039087405 conn(0x7f3cd40cb170 msgr2=0x7f3cd40cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:43.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.205+0000 7f3cda42a640 1 -- 172.21.15.112:0/3039087405 shutdown_connections 2024-04-25T01:40:43.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.205+0000 7f3cda42a640 1 -- 172.21.15.112:0/3039087405 wait complete. 2024-04-25T01:40:43.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.206+0000 7f3cda42a640 1 Processor -- start 2024-04-25T01:40:43.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.206+0000 7f3cda42a640 1 -- start start 2024-04-25T01:40:43.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.207+0000 7f3cda42a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3cd40c72b0 0x7f3cd41ef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:43.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.207+0000 7f3cda42a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cd41592c0 0x7f3cd414a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:43.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.207+0000 7f3cda42a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cd41596a0 0x7f3cd414f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:43.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.207+0000 7f3cda42a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd406c510 con 0x7f3cd41592c0 2024-04-25T01:40:43.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.207+0000 7f3cda42a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd406c390 con 0x7f3cd41596a0 2024-04-25T01:40:43.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.207+0000 7f3cda42a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd406db20 con 0x7f3cd40c72b0 2024-04-25T01:40:43.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.207+0000 7f3cd37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cd41592c0 0x7f3cd414a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:43.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.207+0000 7f3cd37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cd41592c0 0x7f3cd414a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42406/0 (socket says 172.21.15.112:42406) 2024-04-25T01:40:43.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.207+0000 7f3cd37fe640 1 -- 172.21.15.112:0/1694471678 learned_addr learned my addr 172.21.15.112:0/1694471678 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:43.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.207+0000 7f3cd89c7640 1 --2- 172.21.15.112:0/1694471678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cd41596a0 0x7f3cd414f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:43.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.207+0000 7f3cd37fe640 1 -- 172.21.15.112:0/1694471678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3cd40c72b0 msgr2=0x7f3cd41ef6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:43.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.207+0000 7f3cd3fff640 1 --2- 172.21.15.112:0/1694471678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3cd40c72b0 0x7f3cd41ef6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:43.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.207+0000 7f3cd37fe640 1 --2- 172.21.15.112:0/1694471678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3cd40c72b0 0x7f3cd41ef6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:43.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.207+0000 7f3cd37fe640 1 -- 172.21.15.112:0/1694471678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cd41596a0 msgr2=0x7f3cd414f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:43.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.207+0000 7f3cd37fe640 1 --2- 172.21.15.112:0/1694471678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cd41596a0 0x7f3cd414f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:43.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.207+0000 7f3cd37fe640 1 -- 172.21.15.112:0/1694471678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd41eec00 con 0x7f3cd41592c0 2024-04-25T01:40:43.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.207+0000 7f3cd89c7640 1 --2- 172.21.15.112:0/1694471678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cd41596a0 0x7f3cd414f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:40:43.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.208+0000 7f3cd37fe640 1 --2- 172.21.15.112:0/1694471678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cd41592c0 0x7f3cd414a400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009870 tx=0x7f3cc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:43.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.208+0000 7f3cd0ff9640 1 -- 172.21.15.112:0/1694471678 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8041020 con 0x7f3cd41592c0 2024-04-25T01:40:43.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.208+0000 7f3cd0ff9640 1 -- 172.21.15.112:0/1694471678 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc8030990 con 0x7f3cd41592c0 2024-04-25T01:40:43.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.208+0000 7f3cda42a640 1 -- 172.21.15.112:0/1694471678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd41ec210 con 0x7f3cd41592c0 2024-04-25T01:40:43.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.208+0000 7f3cd0ff9640 1 -- 172.21.15.112:0/1694471678 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8037040 con 0x7f3cd41592c0 2024-04-25T01:40:43.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.208+0000 7f3cda42a640 1 -- 172.21.15.112:0/1694471678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd41ec4f0 con 0x7f3cd41592c0 2024-04-25T01:40:43.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.209+0000 7f3cda42a640 1 -- 172.21.15.112:0/1694471678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cd40679a0 con 0x7f3cd41592c0 2024-04-25T01:40:43.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.209+0000 7f3cd0ff9640 1 -- 172.21.15.112:0/1694471678 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3cc8044020 con 0x7f3cd41592c0 2024-04-25T01:40:43.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.209+0000 7f3cd0ff9640 1 --2- 172.21.15.112:0/1694471678 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3cc006be60 0x7f3cc006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:43.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.210+0000 7f3cd0ff9640 1 -- 172.21.15.112:0/1694471678 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f3cc80abbc0 con 0x7f3cd41592c0 2024-04-25T01:40:43.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.210+0000 7f3cd3fff640 1 --2- 172.21.15.112:0/1694471678 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3cc006be60 0x7f3cc006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:43.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.210+0000 7f3cd3fff640 1 --2- 172.21.15.112:0/1694471678 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3cc006be60 0x7f3cc006e320 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4003d90 tx=0x7f3cc4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:43.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.214+0000 7f3cd0ff9640 1 -- 172.21.15.112:0/1694471678 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3cc804b030 con 0x7f3cd41592c0 2024-04-25T01:40:43.353 INFO:tasks.workunit.client.0.smithi112.stdout:Syncing()...2 sec 2024-04-25T01:40:43.353 INFO:tasks.workunit.client.0.smithi112.stdout:FFSB benchmark finished at: Thu Apr 25 01:40:43 2024 2024-04-25T01:40:43.353 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T01:40:43.353 INFO:tasks.workunit.client.0.smithi112.stdout:Results: 2024-04-25T01:40:43.353 INFO:tasks.workunit.client.0.smithi112.stdout:Benchmark took 306.03 sec 2024-04-25T01:40:43.353 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T01:40:43.353 INFO:tasks.workunit.client.0.smithi112.stdout:Total Results 2024-04-25T01:40:43.354 INFO:tasks.workunit.client.0.smithi112.stdout:=============== 2024-04-25T01:40:43.354 INFO:tasks.workunit.client.0.smithi112.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-04-25T01:40:43.354 INFO:tasks.workunit.client.0.smithi112.stdout: ======= ============ ========= ======= =========== ========== 2024-04-25T01:40:43.354 INFO:tasks.workunit.client.0.smithi112.stdout: write : 1605120 5244.97 100.000% 100.000% 20.5MB/sec 2024-04-25T01:40:43.354 INFO:tasks.workunit.client.0.smithi112.stdout:- 2024-04-25T01:40:43.354 INFO:tasks.workunit.client.0.smithi112.stdout:5244.97 Transactions per Second 2024-04-25T01:40:43.354 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T01:40:43.354 INFO:tasks.workunit.client.0.smithi112.stdout:Throughput Results 2024-04-25T01:40:43.354 INFO:tasks.workunit.client.0.smithi112.stdout:=================== 2024-04-25T01:40:43.354 INFO:tasks.workunit.client.0.smithi112.stdout:Write Throughput: 20.5MB/sec 2024-04-25T01:40:43.359 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T01:40:43.360 INFO:tasks.workunit.client.0.smithi112.stdout:System Call Latency statistics in millisecs 2024-04-25T01:40:43.360 INFO:tasks.workunit.client.0.smithi112.stdout:===== 2024-04-25T01:40:43.360 INFO:tasks.workunit.client.0.smithi112.stdout: Min Avg Max Total Calls 2024-04-25T01:40:43.360 INFO:tasks.workunit.client.0.smithi112.stdout: ======== ======== ======== ============ 2024-04-25T01:40:43.360 INFO:tasks.workunit.client.0.smithi112.stdout:[ open] 0.417000 151.201979 6146.803223 1254 2024-04-25T01:40:43.360 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[0] 5000.000000 - 10000.000000 : 10 2024-04-25T01:40:43.360 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[1] 2000.000000 - 5000.000000 : 25 2024-04-25T01:40:43.360 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[2] 1000.000000 - 2000.000000 : 17 2024-04-25T01:40:43.360 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[3] 500.000000 - 1000.000000 : 6 2024-04-25T01:40:43.360 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[4] 200.000000 - 500.000000 : 6 2024-04-25T01:40:43.360 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[5] 100.000000 - 200.000000 : 4 2024-04-25T01:40:43.360 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[6] 50.000000 - 100.000000 : 13 2024-04-25T01:40:43.361 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[7] 20.000000 - 50.000000 : 429 2024-04-25T01:40:43.361 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[8] 10.000000 - 20.000000 : 499 2024-04-25T01:40:43.361 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[9] 5.000000 - 10.000000 : 162 2024-04-25T01:40:43.361 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[10] 2.000000 - 5.000000 : 54 2024-04-25T01:40:43.361 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[11] 1.000000 - 2.000000 : 19 2024-04-25T01:40:43.361 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[12] 0.500000 - 1.000000 : 9 2024-04-25T01:40:43.361 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[13] 0.200000 - 0.500000 : 1 2024-04-25T01:40:43.361 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[14] 0.100000 - 0.200000 : 0 2024-04-25T01:40:43.361 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2024-04-25T01:40:43.361 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-04-25T01:40:43.361 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-04-25T01:40:43.361 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-04-25T01:40:43.361 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T01:40:43.361 INFO:tasks.workunit.client.0.smithi112.stdout:[ write] 0.087000 5.909946 439.851013 1605120 2024-04-25T01:40:43.362 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-04-25T01:40:43.362 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-04-25T01:40:43.362 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-04-25T01:40:43.362 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-04-25T01:40:43.362 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[4] 200.000000 - 500.000000 : 7 2024-04-25T01:40:43.362 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[5] 100.000000 - 200.000000 : 176 2024-04-25T01:40:43.362 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[6] 50.000000 - 100.000000 : 4460 2024-04-25T01:40:43.362 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[7] 20.000000 - 50.000000 : 79097 2024-04-25T01:40:43.362 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[8] 10.000000 - 20.000000 : 203037 2024-04-25T01:40:43.362 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[9] 5.000000 - 10.000000 : 347761 2024-04-25T01:40:43.363 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[10] 2.000000 - 5.000000 : 394597 2024-04-25T01:40:43.363 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[11] 1.000000 - 2.000000 : 139336 2024-04-25T01:40:43.363 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[12] 0.500000 - 1.000000 : 85668 2024-04-25T01:40:43.363 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[13] 0.200000 - 0.500000 : 198321 2024-04-25T01:40:43.363 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[14] 0.100000 - 0.200000 : 152047 2024-04-25T01:40:43.363 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[15] 0.050000 - 0.100000 : 613 2024-04-25T01:40:43.363 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-04-25T01:40:43.363 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-04-25T01:40:43.363 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-04-25T01:40:43.363 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T01:40:43.363 INFO:tasks.workunit.client.0.smithi112.stdout:[ lseek] 0.000000 0.000891 4.565000 1605120 2024-04-25T01:40:43.363 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-04-25T01:40:43.363 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-04-25T01:40:43.363 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-04-25T01:40:43.364 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-04-25T01:40:43.364 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-04-25T01:40:43.364 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-04-25T01:40:43.364 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-04-25T01:40:43.364 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2024-04-25T01:40:43.364 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-04-25T01:40:43.364 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[9] 5.000000 - 10.000000 : 0 2024-04-25T01:40:43.364 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[10] 2.000000 - 5.000000 : 1 2024-04-25T01:40:43.364 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[11] 1.000000 - 2.000000 : 0 2024-04-25T01:40:43.364 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[12] 0.500000 - 1.000000 : 0 2024-04-25T01:40:43.364 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[13] 0.200000 - 0.500000 : 6 2024-04-25T01:40:43.364 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[14] 0.100000 - 0.200000 : 15 2024-04-25T01:40:43.364 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[15] 0.050000 - 0.100000 : 11 2024-04-25T01:40:43.364 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[16] 0.020000 - 0.050000 : 179 2024-04-25T01:40:43.365 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[17] 0.010000 - 0.020000 : 1074 2024-04-25T01:40:43.365 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[18] 0.000000 - 0.010000 : 1603834 2024-04-25T01:40:43.365 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T01:40:43.365 INFO:tasks.workunit.client.0.smithi112.stdout:[ close] 0.017000 2.814107 81.513000 1254 2024-04-25T01:40:43.365 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-04-25T01:40:43.365 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-04-25T01:40:43.365 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-04-25T01:40:43.365 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-04-25T01:40:43.365 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-04-25T01:40:43.365 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-04-25T01:40:43.365 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[6] 50.000000 - 100.000000 : 1 2024-04-25T01:40:43.365 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[7] 20.000000 - 50.000000 : 31 2024-04-25T01:40:43.365 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[8] 10.000000 - 20.000000 : 61 2024-04-25T01:40:43.365 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[9] 5.000000 - 10.000000 : 98 2024-04-25T01:40:43.366 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[10] 2.000000 - 5.000000 : 176 2024-04-25T01:40:43.366 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[11] 1.000000 - 2.000000 : 147 2024-04-25T01:40:43.366 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[12] 0.500000 - 1.000000 : 63 2024-04-25T01:40:43.366 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[13] 0.200000 - 0.500000 : 94 2024-04-25T01:40:43.366 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[14] 0.100000 - 0.200000 : 129 2024-04-25T01:40:43.366 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[15] 0.050000 - 0.100000 : 108 2024-04-25T01:40:43.367 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[16] 0.020000 - 0.050000 : 345 2024-04-25T01:40:43.367 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[17] 0.010000 - 0.020000 : 1 2024-04-25T01:40:43.367 INFO:tasks.workunit.client.0.smithi112.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-04-25T01:40:43.367 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T01:40:43.367 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T01:40:43.367 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T01:40:43.367 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T01:40:43.367 INFO:tasks.workunit.client.0.smithi112.stdout:1.0% User Time 2024-04-25T01:40:43.367 INFO:tasks.workunit.client.0.smithi112.stdout:27.7% System Time 2024-04-25T01:40:43.367 INFO:tasks.workunit.client.0.smithi112.stdout:28.7% CPU Utilization 2024-04-25T01:40:43.368 INFO:tasks.workunit.client.0.smithi112.stderr:+ cd .. 2024-04-25T01:40:43.368 INFO:tasks.workunit.client.0.smithi112.stderr:+ rm -r tmp ffsb 2024-04-25T01:40:43.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.481+0000 7f3cda42a640 1 -- 172.21.15.112:0/1694471678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f3cd4156a50 con 0x7f3cd41592c0 2024-04-25T01:40:43.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.482+0000 7f3cd0ff9640 1 -- 172.21.15.112:0/1694471678 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f3cc807a260 con 0x7f3cd41592c0 2024-04-25T01:40:43.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.484+0000 7f3cda42a640 1 -- 172.21.15.112:0/1694471678 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3cc006be60 msgr2=0x7f3cc006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:43.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.484+0000 7f3cda42a640 1 --2- 172.21.15.112:0/1694471678 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3cc006be60 0x7f3cc006e320 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4003d90 tx=0x7f3cc4009040 comp rx=0 tx=0).stop 2024-04-25T01:40:43.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.484+0000 7f3cda42a640 1 -- 172.21.15.112:0/1694471678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cd41592c0 msgr2=0x7f3cd414a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:43.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.484+0000 7f3cda42a640 1 --2- 172.21.15.112:0/1694471678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cd41592c0 0x7f3cd414a400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009870 tx=0x7f3cc80046c0 comp rx=0 tx=0).stop 2024-04-25T01:40:43.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.484+0000 7f3cda42a640 1 -- 172.21.15.112:0/1694471678 shutdown_connections 2024-04-25T01:40:43.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.484+0000 7f3cda42a640 1 --2- 172.21.15.112:0/1694471678 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3cc006be60 0x7f3cc006e320 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:43.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.484+0000 7f3cda42a640 1 --2- 172.21.15.112:0/1694471678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cd41596a0 0x7f3cd414f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:43.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.484+0000 7f3cda42a640 1 --2- 172.21.15.112:0/1694471678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cd41592c0 0x7f3cd414a400 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:43.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.485+0000 7f3cda42a640 1 --2- 172.21.15.112:0/1694471678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3cd40c72b0 0x7f3cd41ef6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:43.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.485+0000 7f3cda42a640 1 -- 172.21.15.112:0/1694471678 >> 172.21.15.112:0/1694471678 conn(0x7f3cd40cb170 msgr2=0x7f3cd4149820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:43.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.485+0000 7f3cda42a640 1 -- 172.21.15.112:0/1694471678 shutdown_connections 2024-04-25T01:40:43.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:43.485+0000 7f3cda42a640 1 -- 172.21.15.112:0/1694471678 wait complete. 2024-04-25T01:40:43.483 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T01:40:45.300 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T01:40:45.300 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T01:40:45.331 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-04-25T01:40:45.332 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T01:40:45.667 DEBUG:teuthology.parallel:result is None 2024-04-25T01:40:45.668 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T01:40:45.699 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T01:40:45.700 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-25T01:40:45.734 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-25T01:40:45.734 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2024-04-25T01:40:45.745 INFO:tasks.mon_thrash:joining mon_thrasher 2024-04-25T01:40:53.484 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:40:53.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.607+0000 7fe379210640 1 Processor -- start 2024-04-25T01:40:53.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.607+0000 7fe379210640 1 -- start start 2024-04-25T01:40:53.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.608+0000 7fe379210640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe3741592c0 0x7fe3740c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:53.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.608+0000 7fe379210640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe374159b90 con 0x7fe3741596a0 2024-04-25T01:40:53.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.608+0000 7fe379210640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe37406c970 con 0x7fe3740c72b0 2024-04-25T01:40:53.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.608+0000 7fe379210640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe3740c6e20 con 0x7fe3741592c0 2024-04-25T01:40:53.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.608+0000 7fe37259c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe3741596a0 0x7fe374152d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:48414/0 (socket says 172.21.15.112:48414) 2024-04-25T01:40:53.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.608+0000 7fe37259c640 1 -- 172.21.15.112:0/2206607616 learned_addr learned my addr 172.21.15.112:0/2206607616 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:40:53.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.608+0000 7fe372d9d640 1 --2- 172.21.15.112:0/2206607616 >> v2:172.21.15.187:3300/0 conn(0x7fe3741592c0 0x7fe3740c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:53.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe371d9b640 1 -- 172.21.15.112:0/2206607616 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1008945759 0 0) 0x7fe374159b90 con 0x7fe3741596a0 2024-04-25T01:40:53.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe371d9b640 1 -- 172.21.15.112:0/2206607616 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe35c003520 con 0x7fe3741596a0 2024-04-25T01:40:53.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe371d9b640 1 -- 172.21.15.112:0/2206607616 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 12184778 0 0) 0x7fe35c003520 con 0x7fe3741596a0 2024-04-25T01:40:53.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe371d9b640 1 -- 172.21.15.112:0/2206607616 >> v2:172.21.15.187:3300/0 conn(0x7fe3741592c0 msgr2=0x7fe3740c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:53.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe371d9b640 1 --2- 172.21.15.112:0/2206607616 >> v2:172.21.15.187:3300/0 conn(0x7fe3741592c0 0x7fe3740c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:53.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe371d9b640 1 -- 172.21.15.112:0/2206607616 >> v1:172.21.15.187:6789/0 conn(0x7fe3740c72b0 legacy=0x7fe374160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:53.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe371d9b640 1 -- 172.21.15.112:0/2206607616 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37406cc50 con 0x7fe3741596a0 2024-04-25T01:40:53.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe371d9b640 1 -- 172.21.15.112:0/2206607616 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fe368001670 con 0x7fe3741596a0 2024-04-25T01:40:53.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe371d9b640 1 -- 172.21.15.112:0/2206607616 >> v1:172.21.15.112:6789/0 conn(0x7fe3741596a0 legacy=0x7fe374152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:53.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe371d9b640 1 --2- 172.21.15.112:0/2206607616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe35c0039b0 0x7fe35c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:53.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe371d9b640 1 --2- 172.21.15.112:0/2206607616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe35c004800 0x7fe35c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:53.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe371d9b640 1 --2- 172.21.15.112:0/2206607616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe35c0052e0 0x7fe35c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:53.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe371d9b640 1 -- 172.21.15.112:0/2206607616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c00c180 con 0x7fe35c0039b0 2024-04-25T01:40:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe371d9b640 1 -- 172.21.15.112:0/2206607616 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c00c330 con 0x7fe35c004800 2024-04-25T01:40:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe371d9b640 1 -- 172.21.15.112:0/2206607616 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe35c00c530 con 0x7fe35c0052e0 2024-04-25T01:40:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe372d9d640 1 --2- 172.21.15.112:0/2206607616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe35c0039b0 0x7fe35c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe37259c640 1 --2- 172.21.15.112:0/2206607616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe35c004800 0x7fe35c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe37359e640 1 --2- 172.21.15.112:0/2206607616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe35c0052e0 0x7fe35c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe372d9d640 1 -- 172.21.15.112:0/2206607616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe35c0052e0 msgr2=0x7fe35c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe372d9d640 1 --2- 172.21.15.112:0/2206607616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe35c0052e0 0x7fe35c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe372d9d640 1 -- 172.21.15.112:0/2206607616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe35c004800 msgr2=0x7fe35c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe372d9d640 1 --2- 172.21.15.112:0/2206607616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe35c004800 0x7fe35c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe372d9d640 1 -- 172.21.15.112:0/2206607616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe35c003520 con 0x7fe35c0039b0 2024-04-25T01:40:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe37359e640 1 --2- 172.21.15.112:0/2206607616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe35c0052e0 0x7fe35c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:40:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.609+0000 7fe37359e640 1 -- 172.21.15.112:0/2206607616 reap_dead start 2024-04-25T01:40:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.610+0000 7fe372d9d640 1 --2- 172.21.15.112:0/2206607616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe35c0039b0 0x7fe35c003dd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe364009f90 tx=0x7fe35c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260277863fe0c905 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.610+0000 7fe371d9b640 1 -- 172.21.15.112:0/2206607616 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364009c10 con 0x7fe35c0039b0 2024-04-25T01:40:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.610+0000 7fe371d9b640 1 -- 172.21.15.112:0/2206607616 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe364036070 con 0x7fe35c0039b0 2024-04-25T01:40:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.610+0000 7fe371d9b640 1 -- 172.21.15.112:0/2206607616 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364031830 con 0x7fe35c0039b0 2024-04-25T01:40:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.611+0000 7fe379210640 1 -- 172.21.15.112:0/2206607616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe35c0039b0 msgr2=0x7fe35c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.611+0000 7fe379210640 1 --2- 172.21.15.112:0/2206607616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe35c0039b0 0x7fe35c003dd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe364009f90 tx=0x7fe35c005830 comp rx=0 tx=0).stop 2024-04-25T01:40:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.611+0000 7fe379210640 1 -- 172.21.15.112:0/2206607616 shutdown_connections 2024-04-25T01:40:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.611+0000 7fe379210640 1 --2- 172.21.15.112:0/2206607616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe35c0039b0 0x7fe35c003dd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.611+0000 7fe379210640 1 -- 172.21.15.112:0/2206607616 >> 172.21.15.112:0/2206607616 conn(0x7fe3740cb170 msgr2=0x7fe3740cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.611+0000 7fe379210640 1 -- 172.21.15.112:0/2206607616 shutdown_connections 2024-04-25T01:40:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.611+0000 7fe379210640 1 -- 172.21.15.112:0/2206607616 wait complete. 2024-04-25T01:40:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.612+0000 7fe379210640 1 Processor -- start 2024-04-25T01:40:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.612+0000 7fe379210640 1 -- start start 2024-04-25T01:40:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.613+0000 7fe379210640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe35c0039b0 0x7fe37414b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.613+0000 7fe379210640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3741588b0 0x7fe3741482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.613+0000 7fe379210640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe374148800 0x7fe3741eba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.613+0000 7fe379210640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe37406dca0 con 0x7fe374148800 2024-04-25T01:40:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.613+0000 7fe379210640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe37406db20 con 0x7fe3741588b0 2024-04-25T01:40:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.613+0000 7fe379210640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe3740c5ec0 con 0x7fe35c0039b0 2024-04-25T01:40:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.613+0000 7fe37359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe374148800 0x7fe3741eba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.613+0000 7fe37359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe374148800 0x7fe3741eba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47112/0 (socket says 172.21.15.112:47112) 2024-04-25T01:40:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.613+0000 7fe37359e640 1 -- 172.21.15.112:0/2864819673 learned_addr learned my addr 172.21.15.112:0/2864819673 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.613+0000 7fe37259c640 1 --2- 172.21.15.112:0/2864819673 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3741588b0 0x7fe3741482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.613+0000 7fe372d9d640 1 --2- 172.21.15.112:0/2864819673 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe35c0039b0 0x7fe37414b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.614+0000 7fe37359e640 1 -- 172.21.15.112:0/2864819673 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe35c0039b0 msgr2=0x7fe37414b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:53.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.614+0000 7fe37359e640 1 --2- 172.21.15.112:0/2864819673 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe35c0039b0 0x7fe37414b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:53.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.614+0000 7fe37359e640 1 -- 172.21.15.112:0/2864819673 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3741588b0 msgr2=0x7fe3741482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:53.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.614+0000 7fe37359e640 1 --2- 172.21.15.112:0/2864819673 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3741588b0 0x7fe3741482c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:53.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.614+0000 7fe37359e640 1 -- 172.21.15.112:0/2864819673 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3740c0c20 con 0x7fe374148800 2024-04-25T01:40:53.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.614+0000 7fe37359e640 1 --2- 172.21.15.112:0/2864819673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe374148800 0x7fe3741eba40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe37408d3b0 tx=0x7fe36c00e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:53.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.614+0000 7fe35b7fe640 1 -- 172.21.15.112:0/2864819673 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c019070 con 0x7fe374148800 2024-04-25T01:40:53.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.614+0000 7fe35b7fe640 1 -- 172.21.15.112:0/2864819673 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe36c0052e0 con 0x7fe374148800 2024-04-25T01:40:53.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.614+0000 7fe379210640 1 -- 172.21.15.112:0/2864819673 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3740c0f00 con 0x7fe374148800 2024-04-25T01:40:53.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.614+0000 7fe35b7fe640 1 -- 172.21.15.112:0/2864819673 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c005490 con 0x7fe374148800 2024-04-25T01:40:53.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.614+0000 7fe379210640 1 -- 172.21.15.112:0/2864819673 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3740c14c0 con 0x7fe374148800 2024-04-25T01:40:53.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.615+0000 7fe379210640 1 -- 172.21.15.112:0/2864819673 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3740679a0 con 0x7fe374148800 2024-04-25T01:40:53.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.615+0000 7fe35b7fe640 1 -- 172.21.15.112:0/2864819673 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe36c003810 con 0x7fe374148800 2024-04-25T01:40:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.616+0000 7fe35b7fe640 1 --2- 172.21.15.112:0/2864819673 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe36006be60 0x7fe36006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.616+0000 7fe35b7fe640 1 -- 172.21.15.112:0/2864819673 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fe36c08a2a0 con 0x7fe374148800 2024-04-25T01:40:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.616+0000 7fe372d9d640 1 --2- 172.21.15.112:0/2864819673 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe36006be60 0x7fe36006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.616+0000 7fe372d9d640 1 --2- 172.21.15.112:0/2864819673 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe36006be60 0x7fe36006e320 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe364035000 tx=0x7fe36402f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.621+0000 7fe35b7fe640 1 -- 172.21.15.112:0/2864819673 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe36c058260 con 0x7fe374148800 2024-04-25T01:40:53.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.831+0000 7fe379210640 1 -- 172.21.15.112:0/2864819673 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3740682a0 con 0x7fe374148800 2024-04-25T01:40:53.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.836+0000 7fe35b7fe640 1 -- 172.21.15.112:0/2864819673 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+6491 (secure 0 0 0) 0x7fe36c05c560 con 0x7fe374148800 2024-04-25T01:40:53.822 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:40:53.822 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/4047219598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4047219598},{"type":"v1","addr":"172.21.15.187:6834","nonce":4047219598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2204640907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2204640907},{"type":"v1","addr":"172.21.15.187:6835","nonce":2204640907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:25:52.687260+0000","modified":"2024-04-25T01:25:55.702719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/2115984531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2115984531},{"type":"v1","addr":"172.21.15.112:6837","nonce":2115984531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1},{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:26:01.740521+0000","modified":"2024-04-25T01:26:04.746955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":8,"state":"up:active","state_seq":6,"addr":"172.21.15.112:6835/278686477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":278686477},{"type":"v1","addr":"172.21.15.112:6835","nonce":278686477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":2}]} 2024-04-25T01:40:53.822 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 11 2024-04-25T01:40:53.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.838+0000 7fe379210640 1 -- 172.21.15.112:0/2864819673 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe36006be60 msgr2=0x7fe36006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:53.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.838+0000 7fe379210640 1 --2- 172.21.15.112:0/2864819673 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe36006be60 0x7fe36006e320 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe364035000 tx=0x7fe36402f000 comp rx=0 tx=0).stop 2024-04-25T01:40:53.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.838+0000 7fe379210640 1 -- 172.21.15.112:0/2864819673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe374148800 msgr2=0x7fe3741eba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:53.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.838+0000 7fe379210640 1 --2- 172.21.15.112:0/2864819673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe374148800 0x7fe3741eba40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe37408d3b0 tx=0x7fe36c00e820 comp rx=0 tx=0).stop 2024-04-25T01:40:53.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.838+0000 7fe379210640 1 -- 172.21.15.112:0/2864819673 shutdown_connections 2024-04-25T01:40:53.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.838+0000 7fe379210640 1 --2- 172.21.15.112:0/2864819673 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe36006be60 0x7fe36006e320 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:53.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.838+0000 7fe379210640 1 --2- 172.21.15.112:0/2864819673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe374148800 0x7fe3741eba40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:53.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.838+0000 7fe379210640 1 --2- 172.21.15.112:0/2864819673 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3741588b0 0x7fe3741482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:53.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.838+0000 7fe379210640 1 --2- 172.21.15.112:0/2864819673 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe35c0039b0 0x7fe37414b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:53.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.838+0000 7fe379210640 1 -- 172.21.15.112:0/2864819673 >> 172.21.15.112:0/2864819673 conn(0x7fe3740cb170 msgr2=0x7fe3740ac050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:53.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.839+0000 7fe379210640 1 -- 172.21.15.112:0/2864819673 shutdown_connections 2024-04-25T01:40:53.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.839+0000 7fe379210640 1 -- 172.21.15.112:0/2864819673 wait complete. 2024-04-25T01:40:53.836 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T01:40:53.837 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T01:40:53.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.945+0000 7fdc31229640 1 Processor -- start 2024-04-25T01:40:53.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.945+0000 7fdc31229640 1 -- start start 2024-04-25T01:40:53.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.946+0000 7fdc31229640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc2c155d80 0x7fdc2c156160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:53.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.946+0000 7fdc31229640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdc2c06dab0 0x7fdc2c14c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:53.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.946+0000 7fdc31229640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc2c06dfb0 con 0x7fdc2c14c5c0 2024-04-25T01:40:53.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.946+0000 7fdc31229640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fdc2c06ebc0 con 0x7fdc2c06dab0 2024-04-25T01:40:53.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.946+0000 7fdc31229640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdc2c06e290 con 0x7fdc2c155d80 2024-04-25T01:40:53.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.946+0000 7fdc2a59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdc2c06dab0 0x7fdc2c14c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:53.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.946+0000 7fdc2a59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdc2c06dab0 0x7fdc2c14c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47116/0 (socket says 172.21.15.112:47116) 2024-04-25T01:40:53.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.946+0000 7fdc2a59c640 1 -- 172.21.15.112:0/2000891094 learned_addr learned my addr 172.21.15.112:0/2000891094 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:53.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.946+0000 7fdc2ad9d640 1 --2- 172.21.15.112:0/2000891094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc2c155d80 0x7fdc2c156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:53.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.947+0000 7fdc2a59c640 1 -- 172.21.15.112:0/2000891094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc2c155d80 msgr2=0x7fdc2c156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:53.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.947+0000 7fdc2a59c640 1 --2- 172.21.15.112:0/2000891094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc2c155d80 0x7fdc2c156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:53.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.947+0000 7fdc2a59c640 1 -- 172.21.15.112:0/2000891094 >> v1:172.21.15.187:6789/0 conn(0x7fdc2c14c5c0 legacy=0x7fdc2c153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:53.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.947+0000 7fdc2a59c640 1 -- 172.21.15.112:0/2000891094 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2c06e570 con 0x7fdc2c06dab0 2024-04-25T01:40:53.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.947+0000 7fdc2ad9d640 1 --2- 172.21.15.112:0/2000891094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc2c155d80 0x7fdc2c156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:40:53.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.947+0000 7fdc2a59c640 1 --2- 172.21.15.112:0/2000891094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc2c06dab0 0x7fdc2c14c080 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009870 tx=0x7fdc1c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6620422c0adfc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:53.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.947+0000 7fdc29d9b640 1 -- 172.21.15.112:0/2000891094 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c037070 con 0x7fdc2c06dab0 2024-04-25T01:40:53.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.947+0000 7fdc29d9b640 1 -- 172.21.15.112:0/2000891094 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc1c032440 con 0x7fdc2c06dab0 2024-04-25T01:40:53.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.947+0000 7fdc29d9b640 1 -- 172.21.15.112:0/2000891094 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c032810 con 0x7fdc2c06dab0 2024-04-25T01:40:53.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.948+0000 7fdc31229640 1 -- 172.21.15.112:0/2000891094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc2c06dab0 msgr2=0x7fdc2c14c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:53.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.948+0000 7fdc31229640 1 --2- 172.21.15.112:0/2000891094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc2c06dab0 0x7fdc2c14c080 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009870 tx=0x7fdc1c02a1c0 comp rx=0 tx=0).stop 2024-04-25T01:40:53.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.948+0000 7fdc31229640 1 -- 172.21.15.112:0/2000891094 shutdown_connections 2024-04-25T01:40:53.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.948+0000 7fdc31229640 1 --2- 172.21.15.112:0/2000891094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc2c06dab0 0x7fdc2c14c080 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:53.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.948+0000 7fdc31229640 1 --2- 172.21.15.112:0/2000891094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc2c155d80 0x7fdc2c156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:53.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.948+0000 7fdc31229640 1 -- 172.21.15.112:0/2000891094 >> 172.21.15.112:0/2000891094 conn(0x7fdc2c0c6f80 msgr2=0x7fdc2c0c7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:53.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.948+0000 7fdc31229640 1 -- 172.21.15.112:0/2000891094 shutdown_connections 2024-04-25T01:40:53.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.948+0000 7fdc31229640 1 -- 172.21.15.112:0/2000891094 wait complete. 2024-04-25T01:40:53.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.949+0000 7fdc31229640 1 Processor -- start 2024-04-25T01:40:53.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.949+0000 7fdc31229640 1 -- start start 2024-04-25T01:40:53.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.950+0000 7fdc31229640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc2c06dab0 0x7fdc2c152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:53.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.950+0000 7fdc31229640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc2c14c5c0 0x7fdc2c1eca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:53.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.950+0000 7fdc31229640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc2c155d80 0x7fdc2c14de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:53.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.950+0000 7fdc31229640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c156820 con 0x7fdc2c155d80 2024-04-25T01:40:53.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.950+0000 7fdc31229640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c1566a0 con 0x7fdc2c14c5c0 2024-04-25T01:40:53.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.950+0000 7fdc31229640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdc2c06c910 con 0x7fdc2c06dab0 2024-04-25T01:40:53.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.950+0000 7fdc2b59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc2c155d80 0x7fdc2c14de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:53.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.950+0000 7fdc2b59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc2c155d80 0x7fdc2c14de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47128/0 (socket says 172.21.15.112:47128) 2024-04-25T01:40:53.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.950+0000 7fdc2b59e640 1 -- 172.21.15.112:0/1964284585 learned_addr learned my addr 172.21.15.112:0/1964284585 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:53.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.950+0000 7fdc2ad9d640 1 --2- 172.21.15.112:0/1964284585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc2c06dab0 0x7fdc2c152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:53.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.950+0000 7fdc2a59c640 1 --2- 172.21.15.112:0/1964284585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc2c14c5c0 0x7fdc2c1eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:53.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.950+0000 7fdc2b59e640 1 -- 172.21.15.112:0/1964284585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc2c06dab0 msgr2=0x7fdc2c152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:53.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.950+0000 7fdc2b59e640 1 --2- 172.21.15.112:0/1964284585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc2c06dab0 0x7fdc2c152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:53.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.950+0000 7fdc2b59e640 1 -- 172.21.15.112:0/1964284585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc2c14c5c0 msgr2=0x7fdc2c1eca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:53.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.950+0000 7fdc2b59e640 1 --2- 172.21.15.112:0/1964284585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc2c14c5c0 0x7fdc2c1eca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:53.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.950+0000 7fdc2b59e640 1 -- 172.21.15.112:0/1964284585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2c0c6090 con 0x7fdc2c155d80 2024-04-25T01:40:53.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.951+0000 7fdc2b59e640 1 --2- 172.21.15.112:0/1964284585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc2c155d80 0x7fdc2c14de40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400c7a0 tx=0x7fdc2400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:53.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.951+0000 7fdc137fe640 1 -- 172.21.15.112:0/1964284585 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc24003b70 con 0x7fdc2c155d80 2024-04-25T01:40:53.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.951+0000 7fdc137fe640 1 -- 172.21.15.112:0/1964284585 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc24003d30 con 0x7fdc2c155d80 2024-04-25T01:40:53.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.951+0000 7fdc137fe640 1 -- 172.21.15.112:0/1964284585 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2400aa90 con 0x7fdc2c155d80 2024-04-25T01:40:53.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.951+0000 7fdc31229640 1 -- 172.21.15.112:0/1964284585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2c0c6370 con 0x7fdc2c155d80 2024-04-25T01:40:53.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.951+0000 7fdc31229640 1 -- 172.21.15.112:0/1964284585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc2c0c6930 con 0x7fdc2c155d80 2024-04-25T01:40:53.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.952+0000 7fdc31229640 1 -- 172.21.15.112:0/1964284585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc2c154c00 con 0x7fdc2c155d80 2024-04-25T01:40:53.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.952+0000 7fdc137fe640 1 -- 172.21.15.112:0/1964284585 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc2400ac30 con 0x7fdc2c155d80 2024-04-25T01:40:53.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.953+0000 7fdc137fe640 1 --2- 172.21.15.112:0/1964284585 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fdc1806be30 0x7fdc1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:53.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.953+0000 7fdc137fe640 1 -- 172.21.15.112:0/1964284585 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fdc2408afc0 con 0x7fdc2c155d80 2024-04-25T01:40:53.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.953+0000 7fdc2ad9d640 1 --2- 172.21.15.112:0/1964284585 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fdc1806be30 0x7fdc1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:53.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.953+0000 7fdc2ad9d640 1 --2- 172.21.15.112:0/1964284585 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fdc1806be30 0x7fdc1806e2f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c06ee20 tx=0x7fdc20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:53.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:53.957+0000 7fdc137fe640 1 -- 172.21.15.112:0/1964284585 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdc2c154c00 con 0x7fdc2c155d80 2024-04-25T01:40:54.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.195+0000 7fdc31229640 1 -- 172.21.15.112:0/1964284585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fdc2c0682a0 con 0x7fdc2c155d80 2024-04-25T01:40:54.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.196+0000 7fdc137fe640 1 -- 172.21.15.112:0/1964284585 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1570 (secure 0 0 0) 0x7fdc24059010 con 0x7fdc2c155d80 2024-04-25T01:40:54.181 INFO:teuthology.orchestra.run.smithi112.stdout:{"election_epoch":118,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","modified":"2024-04-25T01:24:54.928910Z","created":"2024-04-25T01:24:54.928910Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3301","nonce":0},{"type":"v1","addr":"172.21.15.187:6790","nonce":0}]},"addr":"172.21.15.187:6790/0","public_addr":"172.21.15.187:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T01:40:54.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.198+0000 7fdc31229640 1 -- 172.21.15.112:0/1964284585 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fdc1806be30 msgr2=0x7fdc1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:54.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.198+0000 7fdc31229640 1 --2- 172.21.15.112:0/1964284585 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fdc1806be30 0x7fdc1806e2f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c06ee20 tx=0x7fdc20008040 comp rx=0 tx=0).stop 2024-04-25T01:40:54.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.198+0000 7fdc31229640 1 -- 172.21.15.112:0/1964284585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc2c155d80 msgr2=0x7fdc2c14de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:54.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.198+0000 7fdc31229640 1 --2- 172.21.15.112:0/1964284585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc2c155d80 0x7fdc2c14de40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400c7a0 tx=0x7fdc2400cc60 comp rx=0 tx=0).stop 2024-04-25T01:40:54.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.198+0000 7fdc31229640 1 -- 172.21.15.112:0/1964284585 shutdown_connections 2024-04-25T01:40:54.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.198+0000 7fdc31229640 1 --2- 172.21.15.112:0/1964284585 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fdc1806be30 0x7fdc1806e2f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.198+0000 7fdc31229640 1 --2- 172.21.15.112:0/1964284585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc2c155d80 0x7fdc2c14de40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.198+0000 7fdc31229640 1 --2- 172.21.15.112:0/1964284585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc2c14c5c0 0x7fdc2c1eca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.198+0000 7fdc31229640 1 --2- 172.21.15.112:0/1964284585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc2c06dab0 0x7fdc2c152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.198+0000 7fdc31229640 1 -- 172.21.15.112:0/1964284585 >> 172.21.15.112:0/1964284585 conn(0x7fdc2c0c6f80 msgr2=0x7fdc2c153f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:54.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.198+0000 7fdc31229640 1 -- 172.21.15.112:0/1964284585 shutdown_connections 2024-04-25T01:40:54.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.198+0000 7fdc31229640 1 -- 172.21.15.112:0/1964284585 wait complete. 2024-04-25T01:40:54.195 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T01:40:54.195 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T01:40:54.205 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T01:40:54.205 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:40:54.205 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:40:54.223 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:40:54.223 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:40:54.279 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:40:54.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.428+0000 7fd8d77b7640 1 Processor -- start 2024-04-25T01:40:54.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.429+0000 7fd8d77b7640 1 -- start start 2024-04-25T01:40:54.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.429+0000 7fd8d77b7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd8d01592c0 0x7fd8d00c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:54.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.429+0000 7fd8d77b7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8d01596a0 0x7fd8d0152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:54.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.429+0000 7fd8d77b7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd8d00c72b0 0x7fd8d0155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:54.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.429+0000 7fd8d77b7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fd8d00c6e20 con 0x7fd8d01592c0 2024-04-25T01:40:54.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.429+0000 7fd8d77b7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd8d0159b90 con 0x7fd8d00c72b0 2024-04-25T01:40:54.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.429+0000 7fd8d77b7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd8d006c970 con 0x7fd8d01596a0 2024-04-25T01:40:54.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.429+0000 7fd8d5553640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd8d01592c0 0x7fd8d00c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:54.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.429+0000 7fd8d5553640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd8d01592c0 0x7fd8d00c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47132/0 (socket says 172.21.15.112:47132) 2024-04-25T01:40:54.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.429+0000 7fd8d5553640 1 -- 172.21.15.112:0/745822447 learned_addr learned my addr 172.21.15.112:0/745822447 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:54.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.430+0000 7fd8d5d54640 1 --2- 172.21.15.112:0/745822447 >> v2:172.21.15.187:3300/0 conn(0x7fd8d00c72b0 0x7fd8d0155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:54.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.430+0000 7fd8d4d52640 1 --2- 172.21.15.112:0/745822447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8d01596a0 0x7fd8d0152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:54.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.430+0000 7fd8d5553640 1 -- 172.21.15.112:0/745822447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8d01596a0 msgr2=0x7fd8d0152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:54.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.430+0000 7fd8d5553640 1 --2- 172.21.15.112:0/745822447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8d01596a0 0x7fd8d0152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.430+0000 7fd8d5553640 1 -- 172.21.15.112:0/745822447 >> v2:172.21.15.187:3300/0 conn(0x7fd8d00c72b0 msgr2=0x7fd8d0155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:54.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.430+0000 7fd8d5553640 1 --2- 172.21.15.112:0/745822447 >> v2:172.21.15.187:3300/0 conn(0x7fd8d00c72b0 0x7fd8d0155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.430+0000 7fd8d5553640 1 -- 172.21.15.112:0/745822447 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d006cc50 con 0x7fd8d01592c0 2024-04-25T01:40:54.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.430+0000 7fd8d4d52640 1 --2- 172.21.15.112:0/745822447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8d01596a0 0x7fd8d0152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:40:54.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.430+0000 7fd8d5553640 1 --2- 172.21.15.112:0/745822447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d01592c0 0x7fd8d00c9070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd8d00aac40 tx=0x7fd8cc02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6be5d421cda4ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:54.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.431+0000 7fd8c7fff640 1 -- 172.21.15.112:0/745822447 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc037070 con 0x7fd8d01592c0 2024-04-25T01:40:54.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.431+0000 7fd8c7fff640 1 -- 172.21.15.112:0/745822447 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8cc02ae00 con 0x7fd8d01592c0 2024-04-25T01:40:54.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.431+0000 7fd8c7fff640 1 -- 172.21.15.112:0/745822447 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc032660 con 0x7fd8d01592c0 2024-04-25T01:40:54.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.431+0000 7fd8d77b7640 1 -- 172.21.15.112:0/745822447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d01592c0 msgr2=0x7fd8d00c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:54.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.431+0000 7fd8d77b7640 1 --2- 172.21.15.112:0/745822447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d01592c0 0x7fd8d00c9070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd8d00aac40 tx=0x7fd8cc02a0b0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.431+0000 7fd8d77b7640 1 -- 172.21.15.112:0/745822447 shutdown_connections 2024-04-25T01:40:54.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.431+0000 7fd8d77b7640 1 --2- 172.21.15.112:0/745822447 >> v2:172.21.15.187:3300/0 conn(0x7fd8d00c72b0 0x7fd8d0155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.431+0000 7fd8d77b7640 1 --2- 172.21.15.112:0/745822447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8d01596a0 0x7fd8d0152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.431+0000 7fd8d77b7640 1 --2- 172.21.15.112:0/745822447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d01592c0 0x7fd8d00c9070 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.431+0000 7fd8d77b7640 1 -- 172.21.15.112:0/745822447 >> 172.21.15.112:0/745822447 conn(0x7fd8d00cb170 msgr2=0x7fd8d00cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:54.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.431+0000 7fd8d77b7640 1 -- 172.21.15.112:0/745822447 shutdown_connections 2024-04-25T01:40:54.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.431+0000 7fd8d77b7640 1 -- 172.21.15.112:0/745822447 wait complete. 2024-04-25T01:40:54.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.432+0000 7fd8d77b7640 1 Processor -- start 2024-04-25T01:40:54.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.432+0000 7fd8d77b7640 1 -- start start 2024-04-25T01:40:54.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.433+0000 7fd8d77b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d00c72b0 0x7fd8d01484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:54.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.433+0000 7fd8d77b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8d01592c0 0x7fd8d014d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:54.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.433+0000 7fd8d77b7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8d01596a0 0x7fd8d00aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:54.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.433+0000 7fd8d77b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d006dca0 con 0x7fd8d00c72b0 2024-04-25T01:40:54.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.433+0000 7fd8d77b7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d006db20 con 0x7fd8d01592c0 2024-04-25T01:40:54.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.433+0000 7fd8d77b7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd8d00c95b0 con 0x7fd8d01596a0 2024-04-25T01:40:54.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.433+0000 7fd8d5553640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d00c72b0 0x7fd8d01484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:54.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.433+0000 7fd8d5553640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d00c72b0 0x7fd8d01484e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47136/0 (socket says 172.21.15.112:47136) 2024-04-25T01:40:54.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.433+0000 7fd8d5553640 1 -- 172.21.15.112:0/67953388 learned_addr learned my addr 172.21.15.112:0/67953388 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:54.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.433+0000 7fd8d4d52640 1 --2- 172.21.15.112:0/67953388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8d01592c0 0x7fd8d014d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:54.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.433+0000 7fd8d5d54640 1 --2- 172.21.15.112:0/67953388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8d01596a0 0x7fd8d00aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:54.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.433+0000 7fd8d5553640 1 -- 172.21.15.112:0/67953388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8d01596a0 msgr2=0x7fd8d00aaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:54.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.433+0000 7fd8d5553640 1 --2- 172.21.15.112:0/67953388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8d01596a0 0x7fd8d00aaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.433+0000 7fd8d5553640 1 -- 172.21.15.112:0/67953388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8d01592c0 msgr2=0x7fd8d014d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:54.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.433+0000 7fd8d5553640 1 --2- 172.21.15.112:0/67953388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8d01592c0 0x7fd8d014d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.433+0000 7fd8d5553640 1 -- 172.21.15.112:0/67953388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d014b160 con 0x7fd8d00c72b0 2024-04-25T01:40:54.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.433+0000 7fd8d4d52640 1 --2- 172.21.15.112:0/67953388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8d01592c0 0x7fd8d014d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:40:54.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.434+0000 7fd8d5553640 1 --2- 172.21.15.112:0/67953388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d00c72b0 0x7fd8d01484e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc02a0f0 tx=0x7fd8cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:54.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.434+0000 7fd8c5ffb640 1 -- 172.21.15.112:0/67953388 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc041070 con 0x7fd8d00c72b0 2024-04-25T01:40:54.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.434+0000 7fd8c5ffb640 1 -- 172.21.15.112:0/67953388 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8cc03c670 con 0x7fd8d00c72b0 2024-04-25T01:40:54.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.434+0000 7fd8c5ffb640 1 -- 172.21.15.112:0/67953388 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc037040 con 0x7fd8d00c72b0 2024-04-25T01:40:54.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.434+0000 7fd8d77b7640 1 -- 172.21.15.112:0/67953388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8d014b440 con 0x7fd8d00c72b0 2024-04-25T01:40:54.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.434+0000 7fd8d77b7640 1 -- 172.21.15.112:0/67953388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8d014a360 con 0x7fd8d00c72b0 2024-04-25T01:40:54.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.435+0000 7fd8d77b7640 1 -- 172.21.15.112:0/67953388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8d00679a0 con 0x7fd8d00c72b0 2024-04-25T01:40:54.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.435+0000 7fd8c5ffb640 1 -- 172.21.15.112:0/67953388 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd8cc03a070 con 0x7fd8d00c72b0 2024-04-25T01:40:54.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.436+0000 7fd8c5ffb640 1 --2- 172.21.15.112:0/67953388 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd8b406be30 0x7fd8b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:54.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.436+0000 7fd8c5ffb640 1 -- 172.21.15.112:0/67953388 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fd8cc0ab0d0 con 0x7fd8d00c72b0 2024-04-25T01:40:54.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.436+0000 7fd8d4d52640 1 --2- 172.21.15.112:0/67953388 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd8b406be30 0x7fd8b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:54.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.436+0000 7fd8d4d52640 1 --2- 172.21.15.112:0/67953388 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd8b406be30 0x7fd8b406e2f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0159ee0 tx=0x7fd8c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:54.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.440+0000 7fd8c5ffb640 1 -- 172.21.15.112:0/67953388 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd8cc079770 con 0x7fd8d00c72b0 2024-04-25T01:40:54.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.613+0000 7fd8d77b7640 1 -- 172.21.15.112:0/67953388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8d00682a0 con 0x7fd8d00c72b0 2024-04-25T01:40:54.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.614+0000 7fd8c5ffb640 1 -- 172.21.15.112:0/67953388 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+20616 (secure 0 0 0) 0x7fd8cc079070 con 0x7fd8d00c72b0 2024-04-25T01:40:54.600 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:40:54.601 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:26:52.110548+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-25T01:25:59.478275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-25T01:26:00.719358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:40:54.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.616+0000 7fd8d77b7640 1 -- 172.21.15.112:0/67953388 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd8b406be30 msgr2=0x7fd8b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:54.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.616+0000 7fd8d77b7640 1 --2- 172.21.15.112:0/67953388 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd8b406be30 0x7fd8b406e2f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0159ee0 tx=0x7fd8c0008040 comp rx=0 tx=0).stop 2024-04-25T01:40:54.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.616+0000 7fd8d77b7640 1 -- 172.21.15.112:0/67953388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d00c72b0 msgr2=0x7fd8d01484e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:54.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.616+0000 7fd8d77b7640 1 --2- 172.21.15.112:0/67953388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d00c72b0 0x7fd8d01484e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc02a0f0 tx=0x7fd8cc0046c0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.616+0000 7fd8d77b7640 1 -- 172.21.15.112:0/67953388 shutdown_connections 2024-04-25T01:40:54.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.616+0000 7fd8d77b7640 1 --2- 172.21.15.112:0/67953388 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd8b406be30 0x7fd8b406e2f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.616+0000 7fd8d77b7640 1 --2- 172.21.15.112:0/67953388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8d01596a0 0x7fd8d00aaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.616+0000 7fd8d77b7640 1 --2- 172.21.15.112:0/67953388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8d01592c0 0x7fd8d014d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.616+0000 7fd8d77b7640 1 --2- 172.21.15.112:0/67953388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8d00c72b0 0x7fd8d01484e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.616+0000 7fd8d77b7640 1 -- 172.21.15.112:0/67953388 >> 172.21.15.112:0/67953388 conn(0x7fd8d00cb170 msgr2=0x7fd8d00cb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:54.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.617+0000 7fd8d77b7640 1 -- 172.21.15.112:0/67953388 shutdown_connections 2024-04-25T01:40:54.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.617+0000 7fd8d77b7640 1 -- 172.21.15.112:0/67953388 wait complete. 2024-04-25T01:40:54.614 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:25:40.679119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-04-25T01:25:50.593045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-04-25T01:25:51.664930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-04-25T01:25:59.478275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-04-25T01:26:00.719358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:40:54.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:40:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.729+0000 7f7cc57ed640 1 Processor -- start 2024-04-25T01:40:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.729+0000 7f7cc57ed640 1 -- start start 2024-04-25T01:40:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.729+0000 7f7cc57ed640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7cc0149f40 0x7f7cc0157660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.729+0000 7f7cc57ed640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cc0154400 con 0x7f7cc0153f70 2024-04-25T01:40:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.729+0000 7f7cc57ed640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cc00c87c0 con 0x7f7cc00c8330 2024-04-25T01:40:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.729+0000 7f7cc57ed640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f7cc00c8aa0 con 0x7f7cc0149f40 2024-04-25T01:40:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.729+0000 7f7cbeffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7cc0149f40 0x7f7cc0157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.729+0000 7f7cbf7fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7cc0153f70 0x7f7cc00ad740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:48422/0 (socket says 172.21.15.112:48422) 2024-04-25T01:40:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.729+0000 7f7cbf7fe640 1 -- 172.21.15.112:0/2891984510 learned_addr learned my addr 172.21.15.112:0/2891984510 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:40:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.730+0000 7f7cbeffd640 1 -- 172.21.15.112:0/2891984510 >> v1:172.21.15.187:6789/0 conn(0x7f7cc00c8330 legacy=0x7f7cc01549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.730+0000 7f7cbeffd640 1 -- 172.21.15.112:0/2891984510 >> v1:172.21.15.112:6789/0 conn(0x7f7cc0153f70 legacy=0x7f7cc00ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.730+0000 7f7cbeffd640 1 -- 172.21.15.112:0/2891984510 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc0156b80 con 0x7f7cc0149f40 2024-04-25T01:40:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.730+0000 7f7cbdffb640 1 -- 172.21.15.112:0/2891984510 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1741014710 0 0) 0x7f7cc0154400 con 0x7f7cc0153f70 2024-04-25T01:40:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.730+0000 7f7cbeffd640 1 --2- 172.21.15.112:0/2891984510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc0149f40 0x7f7cc0157660 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80097a0 tx=0x7f7cb802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbaad3b8a9be3cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.731+0000 7f7cbdffb640 1 -- 172.21.15.112:0/2891984510 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8037070 con 0x7f7cc0149f40 2024-04-25T01:40:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.731+0000 7f7cbdffb640 1 -- 172.21.15.112:0/2891984510 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb8032440 con 0x7f7cc0149f40 2024-04-25T01:40:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.731+0000 7f7cbdffb640 1 -- 172.21.15.112:0/2891984510 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8032810 con 0x7f7cc0149f40 2024-04-25T01:40:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.731+0000 7f7cc57ed640 1 -- 172.21.15.112:0/2891984510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc0149f40 msgr2=0x7f7cc0157660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.731+0000 7f7cc57ed640 1 --2- 172.21.15.112:0/2891984510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc0149f40 0x7f7cc0157660 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80097a0 tx=0x7f7cb802a230 comp rx=0 tx=0).stop 2024-04-25T01:40:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.731+0000 7f7cc57ed640 1 -- 172.21.15.112:0/2891984510 shutdown_connections 2024-04-25T01:40:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.731+0000 7f7cc57ed640 1 --2- 172.21.15.112:0/2891984510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc0149f40 0x7f7cc0157660 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.731+0000 7f7cc57ed640 1 -- 172.21.15.112:0/2891984510 >> 172.21.15.112:0/2891984510 conn(0x7f7cc00c8f60 msgr2=0x7f7cc00c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:54.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.731+0000 7f7cc57ed640 1 -- 172.21.15.112:0/2891984510 shutdown_connections 2024-04-25T01:40:54.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.731+0000 7f7cc57ed640 1 -- 172.21.15.112:0/2891984510 wait complete. 2024-04-25T01:40:54.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.732+0000 7f7cc57ed640 1 Processor -- start 2024-04-25T01:40:54.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.732+0000 7f7cc57ed640 1 -- start start 2024-04-25T01:40:54.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.733+0000 7f7cc57ed640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cc0149f40 0x7f7cc00c2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:54.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.733+0000 7f7cc57ed640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc0153f70 0x7f7cc01e7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:54.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.733+0000 7f7cc57ed640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7cc014f5b0 0x7f7cc01e83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:54.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.733+0000 7f7cc57ed640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc014a320 con 0x7f7cc0153f70 2024-04-25T01:40:54.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.733+0000 7f7cc57ed640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc0157140 con 0x7f7cc0149f40 2024-04-25T01:40:54.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.733+0000 7f7cc57ed640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc01550f0 con 0x7f7cc014f5b0 2024-04-25T01:40:54.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.733+0000 7f7cbe7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc0153f70 0x7f7cc01e7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:54.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.733+0000 7f7cbe7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc0153f70 0x7f7cc01e7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47140/0 (socket says 172.21.15.112:47140) 2024-04-25T01:40:54.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.733+0000 7f7cbe7fc640 1 -- 172.21.15.112:0/2565769078 learned_addr learned my addr 172.21.15.112:0/2565769078 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:54.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.733+0000 7f7cbeffd640 1 --2- 172.21.15.112:0/2565769078 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cc0149f40 0x7f7cc00c2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:54.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.733+0000 7f7cbe7fc640 1 -- 172.21.15.112:0/2565769078 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7cc014f5b0 msgr2=0x7f7cc01e83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:54.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.733+0000 7f7cbe7fc640 1 --2- 172.21.15.112:0/2565769078 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7cc014f5b0 0x7f7cc01e83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.733+0000 7f7cbe7fc640 1 -- 172.21.15.112:0/2565769078 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cc0149f40 msgr2=0x7f7cc00c2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:54.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.733+0000 7f7cbe7fc640 1 --2- 172.21.15.112:0/2565769078 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cc0149f40 0x7f7cc00c2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.733+0000 7f7cbe7fc640 1 -- 172.21.15.112:0/2565769078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc014a540 con 0x7f7cc0153f70 2024-04-25T01:40:54.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.733+0000 7f7cbeffd640 1 --2- 172.21.15.112:0/2565769078 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cc0149f40 0x7f7cc00c2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:40:54.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.734+0000 7f7cbe7fc640 1 --2- 172.21.15.112:0/2565769078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc0153f70 0x7f7cc01e7e80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400c7a0 tx=0x7f7cb400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:54.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.734+0000 7f7ca77fe640 1 -- 172.21.15.112:0/2565769078 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4003b70 con 0x7f7cc0153f70 2024-04-25T01:40:54.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.734+0000 7f7ca77fe640 1 -- 172.21.15.112:0/2565769078 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb4003d10 con 0x7f7cc0153f70 2024-04-25T01:40:54.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.734+0000 7f7cc57ed640 1 -- 172.21.15.112:0/2565769078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc01e8900 con 0x7f7cc0153f70 2024-04-25T01:40:54.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.734+0000 7f7ca77fe640 1 -- 172.21.15.112:0/2565769078 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb400aa30 con 0x7f7cc0153f70 2024-04-25T01:40:54.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.734+0000 7f7cc57ed640 1 -- 172.21.15.112:0/2565769078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc014a7e0 con 0x7f7cc0153f70 2024-04-25T01:40:54.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.735+0000 7f7cc57ed640 1 -- 172.21.15.112:0/2565769078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc014a9e0 con 0x7f7cc0153f70 2024-04-25T01:40:54.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.735+0000 7f7ca77fe640 1 -- 172.21.15.112:0/2565769078 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7cb400abd0 con 0x7f7cc0153f70 2024-04-25T01:40:54.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.736+0000 7f7ca77fe640 1 --2- 172.21.15.112:0/2565769078 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7cac06be60 0x7f7cac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:54.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.736+0000 7f7ca77fe640 1 -- 172.21.15.112:0/2565769078 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f7cb408b080 con 0x7f7cc0153f70 2024-04-25T01:40:54.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.736+0000 7f7cbeffd640 1 --2- 172.21.15.112:0/2565769078 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7cac06be60 0x7f7cac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:54.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.736+0000 7f7cbeffd640 1 --2- 172.21.15.112:0/2565769078 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7cac06be60 0x7f7cac06e320 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8034000 tx=0x7f7cb802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:54.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.740+0000 7f7ca77fe640 1 -- 172.21.15.112:0/2565769078 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7cc014a9e0 con 0x7f7cc0153f70 2024-04-25T01:40:54.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.916+0000 7f7cc57ed640 1 -- 172.21.15.112:0/2565769078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7cc006e480 con 0x7f7cc0153f70 2024-04-25T01:40:54.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.916+0000 7f7ca77fe640 1 -- 172.21.15.112:0/2565769078 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f7cb4059720 con 0x7f7cc0153f70 2024-04-25T01:40:54.903 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T01:40:54.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.918+0000 7f7cc57ed640 1 -- 172.21.15.112:0/2565769078 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7cac06be60 msgr2=0x7f7cac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:54.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.918+0000 7f7cc57ed640 1 --2- 172.21.15.112:0/2565769078 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7cac06be60 0x7f7cac06e320 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8034000 tx=0x7f7cb802e000 comp rx=0 tx=0).stop 2024-04-25T01:40:54.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.918+0000 7f7cc57ed640 1 -- 172.21.15.112:0/2565769078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc0153f70 msgr2=0x7f7cc01e7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:54.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.918+0000 7f7cc57ed640 1 --2- 172.21.15.112:0/2565769078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc0153f70 0x7f7cc01e7e80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400c7a0 tx=0x7f7cb400cc60 comp rx=0 tx=0).stop 2024-04-25T01:40:54.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.918+0000 7f7cc57ed640 1 -- 172.21.15.112:0/2565769078 shutdown_connections 2024-04-25T01:40:54.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.918+0000 7f7cc57ed640 1 --2- 172.21.15.112:0/2565769078 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f7cac06be60 0x7f7cac06e320 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.918+0000 7f7cc57ed640 1 --2- 172.21.15.112:0/2565769078 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7cc014f5b0 0x7f7cc01e83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.918+0000 7f7cc57ed640 1 --2- 172.21.15.112:0/2565769078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc0153f70 0x7f7cc01e7e80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.918+0000 7f7cc57ed640 1 --2- 172.21.15.112:0/2565769078 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cc0149f40 0x7f7cc00c2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:54.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.918+0000 7f7cc57ed640 1 -- 172.21.15.112:0/2565769078 >> 172.21.15.112:0/2565769078 conn(0x7f7cc00c8f60 msgr2=0x7f7cc00c54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:54.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.918+0000 7f7cc57ed640 1 -- 172.21.15.112:0/2565769078 shutdown_connections 2024-04-25T01:40:54.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:54.918+0000 7f7cc57ed640 1 -- 172.21.15.112:0/2565769078 wait complete. 2024-04-25T01:40:54.915 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-04-25T01:40:55.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.030+0000 7f64b559e640 1 Processor -- start 2024-04-25T01:40:55.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.030+0000 7f64b559e640 1 -- start start 2024-04-25T01:40:55.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.031+0000 7f64b559e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f64b00c4f30 0x7f64b006db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.031+0000 7f64b559e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f64b006e050 0x7f64b01558a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.031+0000 7f64b559e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64b0158f90 con 0x7f64b0152db0 2024-04-25T01:40:55.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.031+0000 7f64b559e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f64b0159d50 con 0x7f64b00c4f30 2024-04-25T01:40:55.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.031+0000 7f64b559e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f64b0159270 con 0x7f64b006e050 2024-04-25T01:40:55.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.031+0000 7f64af7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f64b00c4f30 0x7f64b006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.031+0000 7f64af7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f64b00c4f30 0x7f64b006db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47148/0 (socket says 172.21.15.112:47148) 2024-04-25T01:40:55.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.031+0000 7f64af7fe640 1 -- 172.21.15.112:0/263816674 learned_addr learned my addr 172.21.15.112:0/263816674 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:55.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.031+0000 7f64b4d9d640 1 --2- 172.21.15.112:0/263816674 >> v2:172.21.15.187:3300/0 conn(0x7f64b006e050 0x7f64b01558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.031+0000 7f64af7fe640 1 -- 172.21.15.112:0/263816674 >> v2:172.21.15.187:3300/0 conn(0x7f64b006e050 msgr2=0x7f64b01558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.031+0000 7f64af7fe640 1 --2- 172.21.15.112:0/263816674 >> v2:172.21.15.187:3300/0 conn(0x7f64b006e050 0x7f64b01558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.031+0000 7f64af7fe640 1 -- 172.21.15.112:0/263816674 >> v1:172.21.15.187:6789/0 conn(0x7f64b0152db0 legacy=0x7f64b00c2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.031+0000 7f64af7fe640 1 -- 172.21.15.112:0/263816674 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b0159550 con 0x7f64b00c4f30 2024-04-25T01:40:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.032+0000 7f64af7fe640 1 --2- 172.21.15.112:0/263816674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64b00c4f30 0x7f64b006db10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f64a4009870 tx=0x7f64a402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29eac42f404170e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.032+0000 7f64aeffd640 1 -- 172.21.15.112:0/263816674 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a4037070 con 0x7f64b00c4f30 2024-04-25T01:40:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.032+0000 7f64aeffd640 1 -- 172.21.15.112:0/263816674 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64a4032440 con 0x7f64b00c4f30 2024-04-25T01:40:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.032+0000 7f64aeffd640 1 -- 172.21.15.112:0/263816674 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a4032810 con 0x7f64b00c4f30 2024-04-25T01:40:55.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.033+0000 7f64b559e640 1 -- 172.21.15.112:0/263816674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64b00c4f30 msgr2=0x7f64b006db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.033+0000 7f64b559e640 1 --2- 172.21.15.112:0/263816674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64b00c4f30 0x7f64b006db10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f64a4009870 tx=0x7f64a402a2a0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.033+0000 7f64b559e640 1 -- 172.21.15.112:0/263816674 shutdown_connections 2024-04-25T01:40:55.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.033+0000 7f64b559e640 1 --2- 172.21.15.112:0/263816674 >> v2:172.21.15.187:3300/0 conn(0x7f64b006e050 0x7f64b01558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.033+0000 7f64b559e640 1 --2- 172.21.15.112:0/263816674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64b00c4f30 0x7f64b006db10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.033+0000 7f64b559e640 1 -- 172.21.15.112:0/263816674 >> 172.21.15.112:0/263816674 conn(0x7f64b00743d0 msgr2=0x7f64b00747b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:55.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.033+0000 7f64b559e640 1 -- 172.21.15.112:0/263816674 shutdown_connections 2024-04-25T01:40:55.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.033+0000 7f64b559e640 1 -- 172.21.15.112:0/263816674 wait complete. 2024-04-25T01:40:55.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.034+0000 7f64b559e640 1 Processor -- start 2024-04-25T01:40:55.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.034+0000 7f64b559e640 1 -- start start 2024-04-25T01:40:55.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.035+0000 7f64b559e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64b006e050 0x7f64b01549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.035+0000 7f64b559e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64b00c4f30 0x7f64b01eba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.035+0000 7f64b559e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64b0152db0 0x7f64b014bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.035+0000 7f64b559e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f64b0153310 con 0x7f64b0152db0 2024-04-25T01:40:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.035+0000 7f64b559e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f64b0153190 con 0x7f64b00c4f30 2024-04-25T01:40:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.035+0000 7f64b559e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f64b0155de0 con 0x7f64b006e050 2024-04-25T01:40:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.035+0000 7f64b4d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64b0152db0 0x7f64b014bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.035+0000 7f64b4d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64b0152db0 0x7f64b014bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47160/0 (socket says 172.21.15.112:47160) 2024-04-25T01:40:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.035+0000 7f64b4d9d640 1 -- 172.21.15.112:0/2486236626 learned_addr learned my addr 172.21.15.112:0/2486236626 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.035+0000 7f64af7fe640 1 --2- 172.21.15.112:0/2486236626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64b00c4f30 0x7f64b01eba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.035+0000 7f64b4d9d640 1 -- 172.21.15.112:0/2486236626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64b006e050 msgr2=0x7f64b01549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.035+0000 7f64b4d9d640 1 --2- 172.21.15.112:0/2486236626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64b006e050 0x7f64b01549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.035+0000 7f64b4d9d640 1 -- 172.21.15.112:0/2486236626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64b00c4f30 msgr2=0x7f64b01eba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.035+0000 7f64b4d9d640 1 --2- 172.21.15.112:0/2486236626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64b00c4f30 0x7f64b01eba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.035+0000 7f64b4d9d640 1 -- 172.21.15.112:0/2486236626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b01df7f0 con 0x7f64b0152db0 2024-04-25T01:40:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.035+0000 7f64b4d9d640 1 --2- 172.21.15.112:0/2486236626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64b0152db0 0x7f64b014bbf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009dd0 tx=0x7f64a800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.035+0000 7f64acff9640 1 -- 172.21.15.112:0/2486236626 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8018020 con 0x7f64b0152db0 2024-04-25T01:40:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.035+0000 7f64acff9640 1 -- 172.21.15.112:0/2486236626 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64a8014480 con 0x7f64b0152db0 2024-04-25T01:40:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.036+0000 7f64b559e640 1 -- 172.21.15.112:0/2486236626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b01ef320 con 0x7f64b0152db0 2024-04-25T01:40:55.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.036+0000 7f64acff9640 1 -- 172.21.15.112:0/2486236626 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8014870 con 0x7f64b0152db0 2024-04-25T01:40:55.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.036+0000 7f64b559e640 1 -- 172.21.15.112:0/2486236626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b0154f10 con 0x7f64b0152db0 2024-04-25T01:40:55.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.036+0000 7f64b559e640 1 -- 172.21.15.112:0/2486236626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64b01523b0 con 0x7f64b0152db0 2024-04-25T01:40:55.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.037+0000 7f64acff9640 1 -- 172.21.15.112:0/2486236626 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f64a8005d10 con 0x7f64b0152db0 2024-04-25T01:40:55.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.037+0000 7f64acff9640 1 --2- 172.21.15.112:0/2486236626 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f649c06be30 0x7f649c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.037+0000 7f64acff9640 1 -- 172.21.15.112:0/2486236626 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f64a808bca0 con 0x7f64b0152db0 2024-04-25T01:40:55.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.037+0000 7f64affff640 1 --2- 172.21.15.112:0/2486236626 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f649c06be30 0x7f649c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.038+0000 7f64affff640 1 --2- 172.21.15.112:0/2486236626 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f649c06be30 0x7f649c06e2f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f64a0000f60 tx=0x7f64a00079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:55.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.042+0000 7f64acff9640 1 -- 172.21.15.112:0/2486236626 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f64b01523b0 con 0x7f64b0152db0 2024-04-25T01:40:55.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.213+0000 7f64b559e640 1 -- 172.21.15.112:0/2486236626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) -- 0x7f64b00758c0 con 0x7f64b0152db0 2024-04-25T01:40:55.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.213+0000 7f64acff9640 1 -- 172.21.15.112:0/2486236626 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v26) ==== 99+0+11 (secure 0 0 0) 0x7f64a8022070 con 0x7f64b0152db0 2024-04-25T01:40:55.199 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T01:40:55.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.215+0000 7f64b559e640 1 -- 172.21.15.112:0/2486236626 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f649c06be30 msgr2=0x7f649c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.215+0000 7f64b559e640 1 --2- 172.21.15.112:0/2486236626 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f649c06be30 0x7f649c06e2f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f64a0000f60 tx=0x7f64a00079d0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.215+0000 7f64b559e640 1 -- 172.21.15.112:0/2486236626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64b0152db0 msgr2=0x7f64b014bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.215+0000 7f64b559e640 1 --2- 172.21.15.112:0/2486236626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64b0152db0 0x7f64b014bbf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009dd0 tx=0x7f64a800c490 comp rx=0 tx=0).stop 2024-04-25T01:40:55.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.215+0000 7f64b559e640 1 -- 172.21.15.112:0/2486236626 shutdown_connections 2024-04-25T01:40:55.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.215+0000 7f64b559e640 1 --2- 172.21.15.112:0/2486236626 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f649c06be30 0x7f649c06e2f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.215+0000 7f64b559e640 1 --2- 172.21.15.112:0/2486236626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64b0152db0 0x7f64b014bbf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.215+0000 7f64b559e640 1 --2- 172.21.15.112:0/2486236626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64b00c4f30 0x7f64b01eba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.215+0000 7f64b559e640 1 --2- 172.21.15.112:0/2486236626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64b006e050 0x7f64b01549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.215+0000 7f64b559e640 1 -- 172.21.15.112:0/2486236626 >> 172.21.15.112:0/2486236626 conn(0x7f64b00743d0 msgr2=0x7f64b00c2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:55.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.215+0000 7f64b559e640 1 -- 172.21.15.112:0/2486236626 shutdown_connections 2024-04-25T01:40:55.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.215+0000 7f64b559e640 1 -- 172.21.15.112:0/2486236626 wait complete. 2024-04-25T01:40:55.212 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-04-25T01:40:55.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.327+0000 7f5dde58b640 1 Processor -- start 2024-04-25T01:40:55.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.327+0000 7f5dde58b640 1 -- start start 2024-04-25T01:40:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.327+0000 7f5dde58b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5dd80c9250 0x7f5dd80c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.327+0000 7f5dde58b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5dd80ca8a0 0x7f5dd806c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.327+0000 7f5dde58b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dd81534a0 con 0x7f5dd80ca370 2024-04-25T01:40:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.327+0000 7f5dde58b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f5dd8156990 con 0x7f5dd80ca8a0 2024-04-25T01:40:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.327+0000 7f5dde58b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5dd8156500 con 0x7f5dd80c9250 2024-04-25T01:40:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.327+0000 7f5ddcb28640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5dd80ca8a0 0x7f5dd806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.327+0000 7f5dd77fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5dd80ca370 0x7f5dd806bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:48436/0 (socket says 172.21.15.112:48436) 2024-04-25T01:40:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.328+0000 7f5dd77fe640 1 -- 172.21.15.112:0/1679640972 learned_addr learned my addr 172.21.15.112:0/1679640972 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:40:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.328+0000 7f5dd7fff640 1 --2- 172.21.15.112:0/1679640972 >> v2:172.21.15.187:3300/0 conn(0x7f5dd80c9250 0x7f5dd80c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.328+0000 7f5ddcb28640 1 -- 172.21.15.112:0/1679640972 >> v2:172.21.15.187:3300/0 conn(0x7f5dd80c9250 msgr2=0x7f5dd80c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:55.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.328+0000 7f5ddcb28640 1 --2- 172.21.15.112:0/1679640972 >> v2:172.21.15.187:3300/0 conn(0x7f5dd80c9250 0x7f5dd80c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.328+0000 7f5ddcb28640 1 -- 172.21.15.112:0/1679640972 >> v1:172.21.15.112:6789/0 conn(0x7f5dd80ca370 legacy=0x7f5dd806bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.328+0000 7f5dd6ffd640 1 -- 172.21.15.112:0/1679640972 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2252252998 0 0) 0x7f5dd81534a0 con 0x7f5dd80ca370 2024-04-25T01:40:55.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.328+0000 7f5ddcb28640 1 -- 172.21.15.112:0/1679640972 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd80cae30 con 0x7f5dd80ca8a0 2024-04-25T01:40:55.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.328+0000 7f5dd7fff640 1 --2- 172.21.15.112:0/1679640972 >> v2:172.21.15.187:3300/0 conn(0x7f5dd80c9250 0x7f5dd80c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:40:55.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.328+0000 7f5ddcb28640 1 --2- 172.21.15.112:0/1679640972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd80ca8a0 0x7f5dd806c420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8066f10 tx=0x7f5dd002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c650ccbbac227c9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:55.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.329+0000 7f5dd6ffd640 1 -- 172.21.15.112:0/1679640972 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0007740 con 0x7f5dd80ca8a0 2024-04-25T01:40:55.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.329+0000 7f5dd6ffd640 1 -- 172.21.15.112:0/1679640972 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd00078e0 con 0x7f5dd80ca8a0 2024-04-25T01:40:55.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.329+0000 7f5dd6ffd640 1 -- 172.21.15.112:0/1679640972 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0007cb0 con 0x7f5dd80ca8a0 2024-04-25T01:40:55.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.329+0000 7f5dde58b640 1 -- 172.21.15.112:0/1679640972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd80ca8a0 msgr2=0x7f5dd806c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.329+0000 7f5dde58b640 1 --2- 172.21.15.112:0/1679640972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd80ca8a0 0x7f5dd806c420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8066f10 tx=0x7f5dd002a5f0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.329+0000 7f5dde58b640 1 -- 172.21.15.112:0/1679640972 shutdown_connections 2024-04-25T01:40:55.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.329+0000 7f5dde58b640 1 --2- 172.21.15.112:0/1679640972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd80ca8a0 0x7f5dd806c420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.329+0000 7f5dde58b640 1 --2- 172.21.15.112:0/1679640972 >> v2:172.21.15.187:3300/0 conn(0x7f5dd80c9250 0x7f5dd80c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.329+0000 7f5dde58b640 1 -- 172.21.15.112:0/1679640972 >> 172.21.15.112:0/1679640972 conn(0x7f5dd80cb310 msgr2=0x7f5dd80cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:55.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.329+0000 7f5dde58b640 1 -- 172.21.15.112:0/1679640972 shutdown_connections 2024-04-25T01:40:55.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.329+0000 7f5dde58b640 1 -- 172.21.15.112:0/1679640972 wait complete. 2024-04-25T01:40:55.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.330+0000 7f5dde58b640 1 Processor -- start 2024-04-25T01:40:55.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.330+0000 7f5dde58b640 1 -- start start 2024-04-25T01:40:55.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.331+0000 7f5dde58b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dd80c9250 0x7f5dd81eb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.331+0000 7f5dde58b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd80ca370 0x7f5dd8158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.331+0000 7f5dde58b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd80ca8a0 0x7f5dd814e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.331+0000 7f5dde58b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd8155eb0 con 0x7f5dd80ca8a0 2024-04-25T01:40:55.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.331+0000 7f5dde58b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd8155bd0 con 0x7f5dd80ca370 2024-04-25T01:40:55.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.331+0000 7f5dde58b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5dd8156190 con 0x7f5dd80c9250 2024-04-25T01:40:55.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.331+0000 7f5ddcb28640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd80ca8a0 0x7f5dd814e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.331+0000 7f5ddcb28640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd80ca8a0 0x7f5dd814e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47180/0 (socket says 172.21.15.112:47180) 2024-04-25T01:40:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.331+0000 7f5ddcb28640 1 -- 172.21.15.112:0/1043588906 learned_addr learned my addr 172.21.15.112:0/1043588906 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.331+0000 7f5ddcb28640 1 -- 172.21.15.112:0/1043588906 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dd80c9250 msgr2=0x7f5dd81eb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.331+0000 7f5dd77fe640 1 --2- 172.21.15.112:0/1043588906 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd80ca370 0x7f5dd8158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.331+0000 7f5dd7fff640 1 --2- 172.21.15.112:0/1043588906 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dd80c9250 0x7f5dd81eb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.331+0000 7f5ddcb28640 1 --2- 172.21.15.112:0/1043588906 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dd80c9250 0x7f5dd81eb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.332+0000 7f5ddcb28640 1 -- 172.21.15.112:0/1043588906 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd80ca370 msgr2=0x7f5dd8158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.332+0000 7f5ddcb28640 1 --2- 172.21.15.112:0/1043588906 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd80ca370 0x7f5dd8158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.332+0000 7f5ddcb28640 1 -- 172.21.15.112:0/1043588906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd80c35f0 con 0x7f5dd80ca8a0 2024-04-25T01:40:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.332+0000 7f5ddcb28640 1 --2- 172.21.15.112:0/1043588906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd80ca8a0 0x7f5dd814e3e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009e50 tx=0x7f5dd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.332+0000 7f5dd4ff9640 1 -- 172.21.15.112:0/1043588906 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0040070 con 0x7f5dd80ca8a0 2024-04-25T01:40:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.332+0000 7f5dd4ff9640 1 -- 172.21.15.112:0/1043588906 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd0031cc0 con 0x7f5dd80ca8a0 2024-04-25T01:40:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.332+0000 7f5dde58b640 1 -- 172.21.15.112:0/1043588906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd80c38d0 con 0x7f5dd80ca8a0 2024-04-25T01:40:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.332+0000 7f5dd4ff9640 1 -- 172.21.15.112:0/1043588906 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0045680 con 0x7f5dd80ca8a0 2024-04-25T01:40:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.332+0000 7f5dde58b640 1 -- 172.21.15.112:0/1043588906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd80bf260 con 0x7f5dd80ca8a0 2024-04-25T01:40:55.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.333+0000 7f5dde58b640 1 -- 172.21.15.112:0/1043588906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dd804f500 con 0x7f5dd80ca8a0 2024-04-25T01:40:55.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.333+0000 7f5dd4ff9640 1 -- 172.21.15.112:0/1043588906 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5dd0039040 con 0x7f5dd80ca8a0 2024-04-25T01:40:55.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.333+0000 7f5dd4ff9640 1 --2- 172.21.15.112:0/1043588906 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5dc406be30 0x7f5dc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.334+0000 7f5dd4ff9640 1 -- 172.21.15.112:0/1043588906 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f5dd00acc30 con 0x7f5dd80ca8a0 2024-04-25T01:40:55.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.334+0000 7f5dd7fff640 1 --2- 172.21.15.112:0/1043588906 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5dc406be30 0x7f5dc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.334+0000 7f5dd7fff640 1 --2- 172.21.15.112:0/1043588906 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5dc406be30 0x7f5dc406e2f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8008850 tx=0x7f5dc8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:55.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.338+0000 7f5dd4ff9640 1 -- 172.21.15.112:0/1043588906 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5dd00443c0 con 0x7f5dd80ca8a0 2024-04-25T01:40:55.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.510+0000 7f5dde58b640 1 -- 172.21.15.112:0/1043588906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) -- 0x7f5dd80bf430 con 0x7f5dd80ca8a0 2024-04-25T01:40:55.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.511+0000 7f5dd4ff9640 1 -- 172.21.15.112:0/1043588906 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v26) ==== 95+0+11 (secure 0 0 0) 0x7f5dd007b310 con 0x7f5dd80ca8a0 2024-04-25T01:40:55.497 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T01:40:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.513+0000 7f5dde58b640 1 -- 172.21.15.112:0/1043588906 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5dc406be30 msgr2=0x7f5dc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.513+0000 7f5dde58b640 1 --2- 172.21.15.112:0/1043588906 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5dc406be30 0x7f5dc406e2f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8008850 tx=0x7f5dc8008040 comp rx=0 tx=0).stop 2024-04-25T01:40:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.513+0000 7f5dde58b640 1 -- 172.21.15.112:0/1043588906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd80ca8a0 msgr2=0x7f5dd814e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.513+0000 7f5dde58b640 1 --2- 172.21.15.112:0/1043588906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd80ca8a0 0x7f5dd814e3e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009e50 tx=0x7f5dd0004a40 comp rx=0 tx=0).stop 2024-04-25T01:40:55.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.513+0000 7f5dde58b640 1 -- 172.21.15.112:0/1043588906 shutdown_connections 2024-04-25T01:40:55.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.513+0000 7f5dde58b640 1 --2- 172.21.15.112:0/1043588906 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f5dc406be30 0x7f5dc406e2f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.513+0000 7f5dde58b640 1 --2- 172.21.15.112:0/1043588906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd80ca8a0 0x7f5dd814e3e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.513+0000 7f5dde58b640 1 --2- 172.21.15.112:0/1043588906 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd80ca370 0x7f5dd8158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.513+0000 7f5dde58b640 1 --2- 172.21.15.112:0/1043588906 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dd80c9250 0x7f5dd81eb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.513+0000 7f5dde58b640 1 -- 172.21.15.112:0/1043588906 >> 172.21.15.112:0/1043588906 conn(0x7f5dd80cb310 msgr2=0x7f5dd81538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:55.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.513+0000 7f5dde58b640 1 -- 172.21.15.112:0/1043588906 shutdown_connections 2024-04-25T01:40:55.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.513+0000 7f5dde58b640 1 -- 172.21.15.112:0/1043588906 wait complete. 2024-04-25T01:40:55.510 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-04-25T01:40:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.623+0000 7f6592361640 1 Processor -- start 2024-04-25T01:40:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.623+0000 7f6592361640 1 -- start start 2024-04-25T01:40:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.624+0000 7f6592361640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f658c14fbe0 0x7f658c0c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.624+0000 7f6592361640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f658c0c2e60 0x7f658c0726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.624+0000 7f6592361640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f658c06f290 con 0x7f658c06d460 2024-04-25T01:40:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.624+0000 7f6592361640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f658c14ffc0 con 0x7f658c0c2e60 2024-04-25T01:40:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.624+0000 7f6592361640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f658c1533a0 con 0x7f658c14fbe0 2024-04-25T01:40:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.624+0000 7f658bfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f658c14fbe0 0x7f658c0c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.624+0000 7f65908fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f658c0c2e60 0x7f658c0726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.624+0000 7f658bfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f658c14fbe0 0x7f658c0c2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:35484/0 (socket says 172.21.15.112:35484) 2024-04-25T01:40:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.624+0000 7f658bfff640 1 -- 172.21.15.112:0/3057057888 learned_addr learned my addr 172.21.15.112:0/3057057888 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.624+0000 7f658bfff640 1 -- 172.21.15.112:0/3057057888 >> v2:172.21.15.187:3300/0 conn(0x7f658c0c2e60 msgr2=0x7f658c0726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.624+0000 7f658bfff640 1 --2- 172.21.15.112:0/3057057888 >> v2:172.21.15.187:3300/0 conn(0x7f658c0c2e60 0x7f658c0726c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.624+0000 7f658bfff640 1 -- 172.21.15.112:0/3057057888 >> v1:172.21.15.187:6789/0 conn(0x7f658c06d460 legacy=0x7f658c071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.624+0000 7f658bfff640 1 -- 172.21.15.112:0/3057057888 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658c06efb0 con 0x7f658c14fbe0 2024-04-25T01:40:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.624+0000 7f65908fe640 1 --2- 172.21.15.112:0/3057057888 >> v2:172.21.15.187:3300/0 conn(0x7f658c0c2e60 0x7f658c0726c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:40:55.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.625+0000 7f658bfff640 1 --2- 172.21.15.112:0/3057057888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f658c14fbe0 0x7f658c0c2920 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f657c00bb10 tx=0x7f657c02c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f267b09221578afc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:55.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.625+0000 7f658affd640 1 -- 172.21.15.112:0/3057057888 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c032790 con 0x7f658c14fbe0 2024-04-25T01:40:55.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.625+0000 7f658affd640 1 -- 172.21.15.112:0/3057057888 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f657c038070 con 0x7f658c14fbe0 2024-04-25T01:40:55.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.625+0000 7f658affd640 1 -- 172.21.15.112:0/3057057888 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c032b50 con 0x7f658c14fbe0 2024-04-25T01:40:55.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.626+0000 7f6592361640 1 -- 172.21.15.112:0/3057057888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f658c14fbe0 msgr2=0x7f658c0c2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.626+0000 7f6592361640 1 --2- 172.21.15.112:0/3057057888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f658c14fbe0 0x7f658c0c2920 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f657c00bb10 tx=0x7f657c02c890 comp rx=0 tx=0).stop 2024-04-25T01:40:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.626+0000 7f6592361640 1 -- 172.21.15.112:0/3057057888 shutdown_connections 2024-04-25T01:40:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.626+0000 7f6592361640 1 --2- 172.21.15.112:0/3057057888 >> v2:172.21.15.187:3300/0 conn(0x7f658c0c2e60 0x7f658c0726c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.626+0000 7f6592361640 1 --2- 172.21.15.112:0/3057057888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f658c14fbe0 0x7f658c0c2920 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.626+0000 7f6592361640 1 -- 172.21.15.112:0/3057057888 >> 172.21.15.112:0/3057057888 conn(0x7f658c0c3a10 msgr2=0x7f658c0c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.626+0000 7f6592361640 1 -- 172.21.15.112:0/3057057888 shutdown_connections 2024-04-25T01:40:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.626+0000 7f6592361640 1 -- 172.21.15.112:0/3057057888 wait complete. 2024-04-25T01:40:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.627+0000 7f6592361640 1 Processor -- start 2024-04-25T01:40:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.627+0000 7f6592361640 1 -- start start 2024-04-25T01:40:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.628+0000 7f6592361640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f658c06d460 0x7f658c145930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.628+0000 7f6592361640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f658c0c2e60 0x7f658c151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.628+0000 7f6592361640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f658c14fbe0 0x7f658c14bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.628+0000 7f6592361640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f658c072c00 con 0x7f658c0c2e60 2024-04-25T01:40:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.628+0000 7f6592361640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f658c153030 con 0x7f658c14fbe0 2024-04-25T01:40:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.628+0000 7f6592361640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f658c0c3700 con 0x7f658c06d460 2024-04-25T01:40:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.628+0000 7f658b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f658c0c2e60 0x7f658c151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.628+0000 7f658b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f658c0c2e60 0x7f658c151750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47192/0 (socket says 172.21.15.112:47192) 2024-04-25T01:40:55.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.628+0000 7f658b7fe640 1 -- 172.21.15.112:0/2452148335 learned_addr learned my addr 172.21.15.112:0/2452148335 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:55.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.628+0000 7f65908fe640 1 --2- 172.21.15.112:0/2452148335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f658c14fbe0 0x7f658c14bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.628+0000 7f658bfff640 1 --2- 172.21.15.112:0/2452148335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f658c06d460 0x7f658c145930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.628+0000 7f658b7fe640 1 -- 172.21.15.112:0/2452148335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f658c06d460 msgr2=0x7f658c145930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.628+0000 7f658b7fe640 1 --2- 172.21.15.112:0/2452148335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f658c06d460 0x7f658c145930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.628+0000 7f658b7fe640 1 -- 172.21.15.112:0/2452148335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f658c14fbe0 msgr2=0x7f658c14bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.628+0000 7f658b7fe640 1 --2- 172.21.15.112:0/2452148335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f658c14fbe0 0x7f658c14bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.628+0000 7f658b7fe640 1 -- 172.21.15.112:0/2452148335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658c0c20f0 con 0x7f658c0c2e60 2024-04-25T01:40:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.628+0000 7f658bfff640 1 --2- 172.21.15.112:0/2452148335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f658c06d460 0x7f658c145930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:40:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.628+0000 7f658b7fe640 1 --2- 172.21.15.112:0/2452148335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f658c0c2e60 0x7f658c151750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f658000c3c0 tx=0x7f658000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.628+0000 7f6588ff9640 1 -- 172.21.15.112:0/2452148335 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65800052d0 con 0x7f658c0c2e60 2024-04-25T01:40:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.629+0000 7f6588ff9640 1 -- 172.21.15.112:0/2452148335 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6580005490 con 0x7f658c0c2e60 2024-04-25T01:40:55.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.629+0000 7f6592361640 1 -- 172.21.15.112:0/2452148335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f658c0ab290 con 0x7f658c0c2e60 2024-04-25T01:40:55.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.629+0000 7f6588ff9640 1 -- 172.21.15.112:0/2452148335 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580007260 con 0x7f658c0c2e60 2024-04-25T01:40:55.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.629+0000 7f6592361640 1 -- 172.21.15.112:0/2452148335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f658c1eebc0 con 0x7f658c0c2e60 2024-04-25T01:40:55.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.630+0000 7f6592361640 1 -- 172.21.15.112:0/2452148335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f658c04f500 con 0x7f658c0c2e60 2024-04-25T01:40:55.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.630+0000 7f6588ff9640 1 -- 172.21.15.112:0/2452148335 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6580007400 con 0x7f658c0c2e60 2024-04-25T01:40:55.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.630+0000 7f6588ff9640 1 --2- 172.21.15.112:0/2452148335 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f657806be30 0x7f657806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.630+0000 7f6588ff9640 1 -- 172.21.15.112:0/2452148335 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f658008a6c0 con 0x7f658c0c2e60 2024-04-25T01:40:55.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.630+0000 7f658bfff640 1 --2- 172.21.15.112:0/2452148335 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f657806be30 0x7f657806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.631+0000 7f658bfff640 1 --2- 172.21.15.112:0/2452148335 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f657806be30 0x7f657806e2f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f657c037000 tx=0x7f657c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:55.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.635+0000 7f6588ff9640 1 -- 172.21.15.112:0/2452148335 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f658c0c20f0 con 0x7f658c0c2e60 2024-04-25T01:40:55.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.809+0000 7f6592361640 1 -- 172.21.15.112:0/2452148335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) -- 0x7f658c0c3470 con 0x7f658c0c2e60 2024-04-25T01:40:55.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.809+0000 7f6588ff9640 1 -- 172.21.15.112:0/2452148335 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v26) ==== 99+0+11 (secure 0 0 0) 0x7f6580058cd0 con 0x7f658c0c2e60 2024-04-25T01:40:55.795 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T01:40:55.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.811+0000 7f6592361640 1 -- 172.21.15.112:0/2452148335 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f657806be30 msgr2=0x7f657806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.811+0000 7f6592361640 1 --2- 172.21.15.112:0/2452148335 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f657806be30 0x7f657806e2f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f657c037000 tx=0x7f657c008040 comp rx=0 tx=0).stop 2024-04-25T01:40:55.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.811+0000 7f6592361640 1 -- 172.21.15.112:0/2452148335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f658c0c2e60 msgr2=0x7f658c151750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.811+0000 7f6592361640 1 --2- 172.21.15.112:0/2452148335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f658c0c2e60 0x7f658c151750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f658000c3c0 tx=0x7f658000c880 comp rx=0 tx=0).stop 2024-04-25T01:40:55.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.811+0000 7f6592361640 1 -- 172.21.15.112:0/2452148335 shutdown_connections 2024-04-25T01:40:55.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.811+0000 7f6592361640 1 --2- 172.21.15.112:0/2452148335 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f657806be30 0x7f657806e2f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.811+0000 7f6592361640 1 --2- 172.21.15.112:0/2452148335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f658c14fbe0 0x7f658c14bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.811+0000 7f6592361640 1 --2- 172.21.15.112:0/2452148335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f658c0c2e60 0x7f658c151750 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.811+0000 7f6592361640 1 --2- 172.21.15.112:0/2452148335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f658c06d460 0x7f658c145930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.811+0000 7f6592361640 1 -- 172.21.15.112:0/2452148335 >> 172.21.15.112:0/2452148335 conn(0x7f658c0c3a10 msgr2=0x7f658c0c0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:55.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.811+0000 7f6592361640 1 -- 172.21.15.112:0/2452148335 shutdown_connections 2024-04-25T01:40:55.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.811+0000 7f6592361640 1 -- 172.21.15.112:0/2452148335 wait complete. 2024-04-25T01:40:55.808 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-04-25T01:40:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.922+0000 7f07f034f640 1 Processor -- start 2024-04-25T01:40:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.922+0000 7f07f034f640 1 -- start start 2024-04-25T01:40:55.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.923+0000 7f07f034f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f07e814fbe0 0x7f07e80c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.923+0000 7f07f034f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f07e80c2e60 0x7f07e80726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.923+0000 7f07f034f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07e806f290 con 0x7f07e806d460 2024-04-25T01:40:55.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.923+0000 7f07f034f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f07e814ffc0 con 0x7f07e80c2e60 2024-04-25T01:40:55.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.923+0000 7f07f034f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f07e81533a0 con 0x7f07e814fbe0 2024-04-25T01:40:55.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.923+0000 7f07ee8ec640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f07e80c2e60 0x7f07e80726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.923+0000 7f07ee8ec640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f07e80c2e60 0x7f07e80726c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47196/0 (socket says 172.21.15.112:47196) 2024-04-25T01:40:55.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.923+0000 7f07ee8ec640 1 -- 172.21.15.112:0/1289029922 learned_addr learned my addr 172.21.15.112:0/1289029922 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:55.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.923+0000 7f07ee8ec640 1 -- 172.21.15.112:0/1289029922 >> v2:172.21.15.187:3300/0 conn(0x7f07e814fbe0 msgr2=0x7f07e80c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:55.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.923+0000 7f07ee8ec640 1 --2- 172.21.15.112:0/1289029922 >> v2:172.21.15.187:3300/0 conn(0x7f07e814fbe0 0x7f07e80c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.923+0000 7f07ee8ec640 1 -- 172.21.15.112:0/1289029922 >> v1:172.21.15.187:6789/0 conn(0x7f07e806d460 legacy=0x7f07e8071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:55.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.923+0000 7f07ee8ec640 1 -- 172.21.15.112:0/1289029922 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e806efb0 con 0x7f07e80c2e60 2024-04-25T01:40:55.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.924+0000 7f07ee8ec640 1 --2- 172.21.15.112:0/1289029922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e80c2e60 0x7f07e80726c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f07e8068020 tx=0x7f07e002c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebd08f57e3d9c04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:55.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.924+0000 7f07ed0e9640 1 -- 172.21.15.112:0/1289029922 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0032710 con 0x7f07e80c2e60 2024-04-25T01:40:55.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.924+0000 7f07ed0e9640 1 -- 172.21.15.112:0/1289029922 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e00328b0 con 0x7f07e80c2e60 2024-04-25T01:40:55.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.924+0000 7f07ed0e9640 1 -- 172.21.15.112:0/1289029922 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0032ca0 con 0x7f07e80c2e60 2024-04-25T01:40:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.925+0000 7f07f034f640 1 -- 172.21.15.112:0/1289029922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e80c2e60 msgr2=0x7f07e80726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.925+0000 7f07f034f640 1 --2- 172.21.15.112:0/1289029922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e80c2e60 0x7f07e80726c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f07e8068020 tx=0x7f07e002c850 comp rx=0 tx=0).stop 2024-04-25T01:40:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.925+0000 7f07f034f640 1 -- 172.21.15.112:0/1289029922 shutdown_connections 2024-04-25T01:40:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.925+0000 7f07f034f640 1 --2- 172.21.15.112:0/1289029922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e80c2e60 0x7f07e80726c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.925+0000 7f07f034f640 1 --2- 172.21.15.112:0/1289029922 >> v2:172.21.15.187:3300/0 conn(0x7f07e814fbe0 0x7f07e80c2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.925+0000 7f07f034f640 1 -- 172.21.15.112:0/1289029922 >> 172.21.15.112:0/1289029922 conn(0x7f07e80c3a10 msgr2=0x7f07e80c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.925+0000 7f07f034f640 1 -- 172.21.15.112:0/1289029922 shutdown_connections 2024-04-25T01:40:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.925+0000 7f07f034f640 1 -- 172.21.15.112:0/1289029922 wait complete. 2024-04-25T01:40:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.926+0000 7f07f034f640 1 Processor -- start 2024-04-25T01:40:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.926+0000 7f07f034f640 1 -- start start 2024-04-25T01:40:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.927+0000 7f07f034f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07e806d460 0x7f07e80ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.927+0000 7f07f034f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07e80c2e60 0x7f07e81e77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.927+0000 7f07f034f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e814fbe0 0x7f07e81e7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.927+0000 7f07f034f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f07e8072c00 con 0x7f07e814fbe0 2024-04-25T01:40:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.927+0000 7f07f034f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f07e8153030 con 0x7f07e80c2e60 2024-04-25T01:40:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.927+0000 7f07f034f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f07e80c3700 con 0x7f07e806d460 2024-04-25T01:40:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.927+0000 7f07ee8ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e814fbe0 0x7f07e81e7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.927+0000 7f07ee8ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e814fbe0 0x7f07e81e7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47200/0 (socket says 172.21.15.112:47200) 2024-04-25T01:40:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.927+0000 7f07ee8ec640 1 -- 172.21.15.112:0/2716636795 learned_addr learned my addr 172.21.15.112:0/2716636795 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.927+0000 7f07ed8ea640 1 --2- 172.21.15.112:0/2716636795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07e80c2e60 0x7f07e81e77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.927+0000 7f07ee0eb640 1 --2- 172.21.15.112:0/2716636795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07e806d460 0x7f07e80ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.927+0000 7f07ee8ec640 1 -- 172.21.15.112:0/2716636795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07e806d460 msgr2=0x7f07e80ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.927+0000 7f07ee8ec640 1 --2- 172.21.15.112:0/2716636795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07e806d460 0x7f07e80ac5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.927+0000 7f07ee8ec640 1 -- 172.21.15.112:0/2716636795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07e80c2e60 msgr2=0x7f07e81e77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.927+0000 7f07ee8ec640 1 --2- 172.21.15.112:0/2716636795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07e80c2e60 0x7f07e81e77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:55.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.927+0000 7f07ee8ec640 1 -- 172.21.15.112:0/2716636795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e8146970 con 0x7f07e814fbe0 2024-04-25T01:40:55.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.927+0000 7f07ee8ec640 1 --2- 172.21.15.112:0/2716636795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e814fbe0 0x7f07e81e7d20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f07e000bc00 tx=0x7f07e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:55.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.927+0000 7f07deffd640 1 -- 172.21.15.112:0/2716636795 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0042030 con 0x7f07e814fbe0 2024-04-25T01:40:55.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.928+0000 7f07deffd640 1 -- 172.21.15.112:0/2716636795 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e0003610 con 0x7f07e814fbe0 2024-04-25T01:40:55.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.928+0000 7f07deffd640 1 -- 172.21.15.112:0/2716636795 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e00039e0 con 0x7f07e814fbe0 2024-04-25T01:40:55.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.928+0000 7f07f034f640 1 -- 172.21.15.112:0/2716636795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e81ead30 con 0x7f07e814fbe0 2024-04-25T01:40:55.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.928+0000 7f07f034f640 1 -- 172.21.15.112:0/2716636795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e81eb2f0 con 0x7f07e814fbe0 2024-04-25T01:40:55.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.929+0000 7f07f034f640 1 -- 172.21.15.112:0/2716636795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e8068480 con 0x7f07e814fbe0 2024-04-25T01:40:55.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.929+0000 7f07deffd640 1 -- 172.21.15.112:0/2716636795 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f07e0045030 con 0x7f07e814fbe0 2024-04-25T01:40:55.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.929+0000 7f07deffd640 1 --2- 172.21.15.112:0/2716636795 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f07cc06be30 0x7f07cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:55.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.929+0000 7f07ee0eb640 1 --2- 172.21.15.112:0/2716636795 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f07cc06be30 0x7f07cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:55.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.929+0000 7f07deffd640 1 -- 172.21.15.112:0/2716636795 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f07e00ae050 con 0x7f07e814fbe0 2024-04-25T01:40:55.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.930+0000 7f07ee0eb640 1 --2- 172.21.15.112:0/2716636795 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f07cc06be30 0x7f07cc06e2f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f07e80c6d40 tx=0x7f07e4007310 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:55.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:55.934+0000 7f07deffd640 1 -- 172.21.15.112:0/2716636795 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f07e8068480 con 0x7f07e814fbe0 2024-04-25T01:40:56.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.105+0000 7f07f034f640 1 -- 172.21.15.112:0/2716636795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) -- 0x7f07e804f500 con 0x7f07e814fbe0 2024-04-25T01:40:56.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.105+0000 7f07deffd640 1 -- 172.21.15.112:0/2716636795 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v26) ==== 95+0+11 (secure 0 0 0) 0x7f07e007c6f0 con 0x7f07e814fbe0 2024-04-25T01:40:56.091 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T01:40:56.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.107+0000 7f07f034f640 1 -- 172.21.15.112:0/2716636795 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f07cc06be30 msgr2=0x7f07cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:56.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.107+0000 7f07f034f640 1 --2- 172.21.15.112:0/2716636795 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f07cc06be30 0x7f07cc06e2f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f07e80c6d40 tx=0x7f07e4007310 comp rx=0 tx=0).stop 2024-04-25T01:40:56.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.107+0000 7f07f034f640 1 -- 172.21.15.112:0/2716636795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e814fbe0 msgr2=0x7f07e81e7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:56.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.107+0000 7f07f034f640 1 --2- 172.21.15.112:0/2716636795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e814fbe0 0x7f07e81e7d20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f07e000bc00 tx=0x7f07e00046c0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.108+0000 7f07f034f640 1 -- 172.21.15.112:0/2716636795 shutdown_connections 2024-04-25T01:40:56.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.108+0000 7f07f034f640 1 --2- 172.21.15.112:0/2716636795 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f07cc06be30 0x7f07cc06e2f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.108+0000 7f07f034f640 1 --2- 172.21.15.112:0/2716636795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e814fbe0 0x7f07e81e7d20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.108+0000 7f07f034f640 1 --2- 172.21.15.112:0/2716636795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07e80c2e60 0x7f07e81e77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.108+0000 7f07f034f640 1 --2- 172.21.15.112:0/2716636795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07e806d460 0x7f07e80ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.108+0000 7f07f034f640 1 -- 172.21.15.112:0/2716636795 >> 172.21.15.112:0/2716636795 conn(0x7f07e80c3a10 msgr2=0x7f07e814d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:56.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.108+0000 7f07f034f640 1 -- 172.21.15.112:0/2716636795 shutdown_connections 2024-04-25T01:40:56.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.108+0000 7f07f034f640 1 -- 172.21.15.112:0/2716636795 wait complete. 2024-04-25T01:40:56.105 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:40:56.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.217+0000 7fcbc2fc1640 1 Processor -- start 2024-04-25T01:40:56.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.217+0000 7fcbc2fc1640 1 -- start start 2024-04-25T01:40:56.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.218+0000 7fcbc2fc1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbbc1592c0 0x7fcbbc0c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:56.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.218+0000 7fcbc2fc1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcbbc0c72b0 0x7fcbbc160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:56.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.218+0000 7fcbc2fc1640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbbc159b90 con 0x7fcbbc1596a0 2024-04-25T01:40:56.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.218+0000 7fcbc2fc1640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcbbc0c6e20 con 0x7fcbbc0c72b0 2024-04-25T01:40:56.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.218+0000 7fcbc2fc1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcbbc06c970 con 0x7fcbbc1592c0 2024-04-25T01:40:56.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.218+0000 7fcbbbfff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fcbbc1596a0 0x7fcbbc152d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:54074/0 (socket says 172.21.15.112:54074) 2024-04-25T01:40:56.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.218+0000 7fcbbbfff640 1 -- 172.21.15.112:0/3706604612 learned_addr learned my addr 172.21.15.112:0/3706604612 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:40:56.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.218+0000 7fcbc0d5d640 1 --2- 172.21.15.112:0/3706604612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbbc1592c0 0x7fcbbc0c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:56.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.218+0000 7fcbc155e640 1 --2- 172.21.15.112:0/3706604612 >> v2:172.21.15.187:3300/0 conn(0x7fcbbc0c72b0 0x7fcbbc160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:56.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.219+0000 7fcbc0d5d640 1 -- 172.21.15.112:0/3706604612 >> v2:172.21.15.187:3300/0 conn(0x7fcbbc0c72b0 msgr2=0x7fcbbc160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:56.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.219+0000 7fcbc0d5d640 1 --2- 172.21.15.112:0/3706604612 >> v2:172.21.15.187:3300/0 conn(0x7fcbbc0c72b0 0x7fcbbc160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.219+0000 7fcbc0d5d640 1 -- 172.21.15.112:0/3706604612 >> v1:172.21.15.187:6789/0 conn(0x7fcbbc1596a0 legacy=0x7fcbbc152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:56.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.219+0000 7fcbc0d5d640 1 -- 172.21.15.112:0/3706604612 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbbc06cc50 con 0x7fcbbc1592c0 2024-04-25T01:40:56.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.219+0000 7fcbc0d5d640 1 --2- 172.21.15.112:0/3706604612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbbc1592c0 0x7fcbbc0c9070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00097a0 tx=0x7fcbb002a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a428688c2130920b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:56.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.219+0000 7fcbbb7fe640 1 -- 172.21.15.112:0/3706604612 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0037070 con 0x7fcbbc1592c0 2024-04-25T01:40:56.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.219+0000 7fcbbb7fe640 1 -- 172.21.15.112:0/3706604612 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb002adb0 con 0x7fcbbc1592c0 2024-04-25T01:40:56.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.220+0000 7fcbbb7fe640 1 -- 172.21.15.112:0/3706604612 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0032660 con 0x7fcbbc1592c0 2024-04-25T01:40:56.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.220+0000 7fcbc2fc1640 1 -- 172.21.15.112:0/3706604612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbbc1592c0 msgr2=0x7fcbbc0c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:56.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.220+0000 7fcbc2fc1640 1 --2- 172.21.15.112:0/3706604612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbbc1592c0 0x7fcbbc0c9070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00097a0 tx=0x7fcbb002a060 comp rx=0 tx=0).stop 2024-04-25T01:40:56.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.220+0000 7fcbc2fc1640 1 -- 172.21.15.112:0/3706604612 shutdown_connections 2024-04-25T01:40:56.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.220+0000 7fcbc2fc1640 1 --2- 172.21.15.112:0/3706604612 >> v2:172.21.15.187:3300/0 conn(0x7fcbbc0c72b0 0x7fcbbc160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.220+0000 7fcbc2fc1640 1 --2- 172.21.15.112:0/3706604612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbbc1592c0 0x7fcbbc0c9070 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.220+0000 7fcbc2fc1640 1 -- 172.21.15.112:0/3706604612 >> 172.21.15.112:0/3706604612 conn(0x7fcbbc0cb170 msgr2=0x7fcbbc0cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:56.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.220+0000 7fcbc2fc1640 1 -- 172.21.15.112:0/3706604612 shutdown_connections 2024-04-25T01:40:56.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.220+0000 7fcbc2fc1640 1 -- 172.21.15.112:0/3706604612 wait complete. 2024-04-25T01:40:56.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.221+0000 7fcbc2fc1640 1 Processor -- start 2024-04-25T01:40:56.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.221+0000 7fcbc2fc1640 1 -- start start 2024-04-25T01:40:56.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.222+0000 7fcbc2fc1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbbc0c72b0 0x7fcbbc0ac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:56.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.222+0000 7fcbc2fc1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbbc1592c0 0x7fcbbc14d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:56.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.222+0000 7fcbc2fc1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbbc1596a0 0x7fcbbc1ee980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:56.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.222+0000 7fcbc2fc1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc153590 con 0x7fcbbc0c72b0 2024-04-25T01:40:56.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.222+0000 7fcbc2fc1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc153410 con 0x7fcbbc1592c0 2024-04-25T01:40:56.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.222+0000 7fcbc2fc1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcbbc06db20 con 0x7fcbbc1596a0 2024-04-25T01:40:56.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.222+0000 7fcbc0d5d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbbc0c72b0 0x7fcbbc0ac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:56.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.222+0000 7fcbc0d5d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbbc0c72b0 0x7fcbbc0ac4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47208/0 (socket says 172.21.15.112:47208) 2024-04-25T01:40:56.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.222+0000 7fcbc0d5d640 1 -- 172.21.15.112:0/2313773694 learned_addr learned my addr 172.21.15.112:0/2313773694 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:56.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.222+0000 7fcbbbfff640 1 --2- 172.21.15.112:0/2313773694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbbc1592c0 0x7fcbbc14d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:56.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.222+0000 7fcbc155e640 1 --2- 172.21.15.112:0/2313773694 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbbc1596a0 0x7fcbbc1ee980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:56.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.222+0000 7fcbc0d5d640 1 -- 172.21.15.112:0/2313773694 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbbc1596a0 msgr2=0x7fcbbc1ee980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:56.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.222+0000 7fcbc0d5d640 1 --2- 172.21.15.112:0/2313773694 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbbc1596a0 0x7fcbbc1ee980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.222+0000 7fcbc0d5d640 1 -- 172.21.15.112:0/2313773694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbbc1592c0 msgr2=0x7fcbbc14d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:56.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.222+0000 7fcbc0d5d640 1 --2- 172.21.15.112:0/2313773694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbbc1592c0 0x7fcbbc14d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.222+0000 7fcbc0d5d640 1 -- 172.21.15.112:0/2313773694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbbc156310 con 0x7fcbbc0c72b0 2024-04-25T01:40:56.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.222+0000 7fcbbbfff640 1 --2- 172.21.15.112:0/2313773694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbbc1592c0 0x7fcbbc14d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:40:56.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.222+0000 7fcbc155e640 1 --2- 172.21.15.112:0/2313773694 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbbc1596a0 0x7fcbbc1ee980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:40:56.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.223+0000 7fcbc0d5d640 1 --2- 172.21.15.112:0/2313773694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbbc0c72b0 0x7fcbbc0ac4b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0000c00 tx=0x7fcbb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:56.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.223+0000 7fcbb97fa640 1 -- 172.21.15.112:0/2313773694 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0041070 con 0x7fcbbc0c72b0 2024-04-25T01:40:56.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.223+0000 7fcbb97fa640 1 -- 172.21.15.112:0/2313773694 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb003c460 con 0x7fcbbc0c72b0 2024-04-25T01:40:56.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.223+0000 7fcbc2fc1640 1 -- 172.21.15.112:0/2313773694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbbc1565f0 con 0x7fcbbc0c72b0 2024-04-25T01:40:56.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.223+0000 7fcbb97fa640 1 -- 172.21.15.112:0/2313773694 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0037040 con 0x7fcbbc0c72b0 2024-04-25T01:40:56.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.223+0000 7fcbc2fc1640 1 -- 172.21.15.112:0/2313773694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbbc0abf90 con 0x7fcbbc0c72b0 2024-04-25T01:40:56.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.224+0000 7fcbc2fc1640 1 -- 172.21.15.112:0/2313773694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbbc0679a0 con 0x7fcbbc0c72b0 2024-04-25T01:40:56.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.224+0000 7fcbb97fa640 1 -- 172.21.15.112:0/2313773694 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcbb0044070 con 0x7fcbbc0c72b0 2024-04-25T01:40:56.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.224+0000 7fcbb97fa640 1 --2- 172.21.15.112:0/2313773694 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcba006be30 0x7fcba006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:56.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.225+0000 7fcbb97fa640 1 -- 172.21.15.112:0/2313773694 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fcbb00ab820 con 0x7fcbbc0c72b0 2024-04-25T01:40:56.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.225+0000 7fcbbbfff640 1 --2- 172.21.15.112:0/2313773694 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcba006be30 0x7fcba006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:56.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.225+0000 7fcbbbfff640 1 --2- 172.21.15.112:0/2313773694 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcba006be30 0x7fcba006e2f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc0c5a10 tx=0x7fcba8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:56.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.229+0000 7fcbb97fa640 1 -- 172.21.15.112:0/2313773694 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcbb0079fe0 con 0x7fcbbc0c72b0 2024-04-25T01:40:56.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.416+0000 7fcbc2fc1640 1 -- 172.21.15.112:0/2313773694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcbbc0c9f10 con 0x7fcbbc0c72b0 2024-04-25T01:40:56.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.417+0000 7fcbb97fa640 1 -- 172.21.15.112:0/2313773694 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fcbbc0c9f10 con 0x7fcbbc0c72b0 2024-04-25T01:40:56.403 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-04-25T01:40:56.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.419+0000 7fcbc2fc1640 1 -- 172.21.15.112:0/2313773694 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcba006be30 msgr2=0x7fcba006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:56.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.419+0000 7fcbc2fc1640 1 --2- 172.21.15.112:0/2313773694 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcba006be30 0x7fcba006e2f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc0c5a10 tx=0x7fcba8009040 comp rx=0 tx=0).stop 2024-04-25T01:40:56.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.419+0000 7fcbc2fc1640 1 -- 172.21.15.112:0/2313773694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbbc0c72b0 msgr2=0x7fcbbc0ac4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:56.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.419+0000 7fcbc2fc1640 1 --2- 172.21.15.112:0/2313773694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbbc0c72b0 0x7fcbbc0ac4b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0000c00 tx=0x7fcbb00046c0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.419+0000 7fcbc2fc1640 1 -- 172.21.15.112:0/2313773694 shutdown_connections 2024-04-25T01:40:56.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.419+0000 7fcbc2fc1640 1 --2- 172.21.15.112:0/2313773694 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcba006be30 0x7fcba006e2f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.419+0000 7fcbc2fc1640 1 --2- 172.21.15.112:0/2313773694 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbbc1596a0 0x7fcbbc1ee980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.419+0000 7fcbc2fc1640 1 --2- 172.21.15.112:0/2313773694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbbc1592c0 0x7fcbbc14d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.419+0000 7fcbc2fc1640 1 --2- 172.21.15.112:0/2313773694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbbc0c72b0 0x7fcbbc0ac4b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.419+0000 7fcbc2fc1640 1 -- 172.21.15.112:0/2313773694 >> 172.21.15.112:0/2313773694 conn(0x7fcbbc0cb170 msgr2=0x7fcbbc0c9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:56.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.419+0000 7fcbc2fc1640 1 -- 172.21.15.112:0/2313773694 shutdown_connections 2024-04-25T01:40:56.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.419+0000 7fcbc2fc1640 1 -- 172.21.15.112:0/2313773694 wait complete. 2024-04-25T01:40:56.416 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T01:40:56.416 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:40:56.432 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:40:56.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.582+0000 7f3f9844c640 1 Processor -- start 2024-04-25T01:40:56.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.582+0000 7f3f9844c640 1 -- start start 2024-04-25T01:40:56.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.583+0000 7f3f9844c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3f900c3050 0x7f3f900c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:56.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.583+0000 7f3f9844c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3f90071770 0x7f3f900c1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:56.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.583+0000 7f3f9844c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f901552d0 0x7f3f90155740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:56.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.583+0000 7f3f9844c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f3f900aa5b0 con 0x7f3f900c3050 2024-04-25T01:40:56.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.583+0000 7f3f9844c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3f900c2500 con 0x7f3f90071770 2024-04-25T01:40:56.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.583+0000 7f3f9844c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f900abd70 con 0x7f3f901552d0 2024-04-25T01:40:56.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.583+0000 7f3f961e8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3f900c3050 0x7f3f900c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:56.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.583+0000 7f3f961e8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3f900c3050 0x7f3f900c1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47222/0 (socket says 172.21.15.112:47222) 2024-04-25T01:40:56.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.583+0000 7f3f961e8640 1 -- 172.21.15.112:0/3010664554 learned_addr learned my addr 172.21.15.112:0/3010664554 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:56.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.583+0000 7f3f961e8640 1 -- 172.21.15.112:0/3010664554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f901552d0 msgr2=0x7f3f90155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:56.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.583+0000 7f3f961e8640 1 --2- 172.21.15.112:0/3010664554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f901552d0 0x7f3f90155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.583+0000 7f3f961e8640 1 -- 172.21.15.112:0/3010664554 >> v2:172.21.15.187:3300/0 conn(0x7f3f90071770 msgr2=0x7f3f900c1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:56.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.583+0000 7f3f959e7640 1 --2- 172.21.15.112:0/3010664554 >> v2:172.21.15.187:3300/0 conn(0x7f3f90071770 0x7f3f900c1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:56.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.583+0000 7f3f961e8640 1 --2- 172.21.15.112:0/3010664554 >> v2:172.21.15.187:3300/0 conn(0x7f3f90071770 0x7f3f900c1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.583+0000 7f3f961e8640 1 -- 172.21.15.112:0/3010664554 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f900ac100 con 0x7f3f900c3050 2024-04-25T01:40:56.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.584+0000 7f3f961e8640 1 --2- 172.21.15.112:0/3010664554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f900c3050 0x7f3f900c1a80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3f90073390 tx=0x7f3f8802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bdbc10ce161746 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:56.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.584+0000 7f3f951e6640 1 -- 172.21.15.112:0/3010664554 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f88036070 con 0x7f3f900c3050 2024-04-25T01:40:56.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.584+0000 7f3f951e6640 1 -- 172.21.15.112:0/3010664554 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f88031600 con 0x7f3f900c3050 2024-04-25T01:40:56.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.584+0000 7f3f951e6640 1 -- 172.21.15.112:0/3010664554 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f880319d0 con 0x7f3f900c3050 2024-04-25T01:40:56.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.585+0000 7f3f9844c640 1 -- 172.21.15.112:0/3010664554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f900c3050 msgr2=0x7f3f900c1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:56.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.585+0000 7f3f9844c640 1 --2- 172.21.15.112:0/3010664554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f900c3050 0x7f3f900c1a80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3f90073390 tx=0x7f3f8802a610 comp rx=0 tx=0).stop 2024-04-25T01:40:56.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.585+0000 7f3f9844c640 1 -- 172.21.15.112:0/3010664554 shutdown_connections 2024-04-25T01:40:56.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.585+0000 7f3f9844c640 1 --2- 172.21.15.112:0/3010664554 >> v2:172.21.15.187:3300/0 conn(0x7f3f90071770 0x7f3f900c1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.585+0000 7f3f9844c640 1 --2- 172.21.15.112:0/3010664554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f901552d0 0x7f3f90155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.585+0000 7f3f9844c640 1 --2- 172.21.15.112:0/3010664554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f900c3050 0x7f3f900c1a80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.585+0000 7f3f9844c640 1 -- 172.21.15.112:0/3010664554 >> 172.21.15.112:0/3010664554 conn(0x7f3f9006b160 msgr2=0x7f3f9006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:56.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.585+0000 7f3f9844c640 1 -- 172.21.15.112:0/3010664554 shutdown_connections 2024-04-25T01:40:56.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.585+0000 7f3f9844c640 1 -- 172.21.15.112:0/3010664554 wait complete. 2024-04-25T01:40:56.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.586+0000 7f3f9844c640 1 Processor -- start 2024-04-25T01:40:56.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.586+0000 7f3f9844c640 1 -- start start 2024-04-25T01:40:56.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.586+0000 7f3f9844c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f90071770 0x7f3f9014b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:56.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f9844c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f900c3050 0x7f3f90146a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:56.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f9844c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f901552d0 0x7f3f90146fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:56.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f9844c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f900ab850 con 0x7f3f901552d0 2024-04-25T01:40:56.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f9844c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3f900ab570 con 0x7f3f900c3050 2024-04-25T01:40:56.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f9844c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f90155c80 con 0x7f3f90071770 2024-04-25T01:40:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f969e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f901552d0 0x7f3f90146fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f969e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f901552d0 0x7f3f90146fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47232/0 (socket says 172.21.15.112:47232) 2024-04-25T01:40:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f969e9640 1 -- 172.21.15.112:0/3178168209 learned_addr learned my addr 172.21.15.112:0/3178168209 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f959e7640 1 --2- 172.21.15.112:0/3178168209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f900c3050 0x7f3f90146a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f961e8640 1 --2- 172.21.15.112:0/3178168209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f90071770 0x7f3f9014b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f969e9640 1 -- 172.21.15.112:0/3178168209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f90071770 msgr2=0x7f3f9014b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f969e9640 1 --2- 172.21.15.112:0/3178168209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f90071770 0x7f3f9014b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f969e9640 1 -- 172.21.15.112:0/3178168209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f900c3050 msgr2=0x7f3f90146a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f969e9640 1 --2- 172.21.15.112:0/3178168209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f900c3050 0x7f3f90146a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f969e9640 1 -- 172.21.15.112:0/3178168209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f901474f0 con 0x7f3f901552d0 2024-04-25T01:40:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f969e9640 1 --2- 172.21.15.112:0/3178168209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f901552d0 0x7f3f90146fb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000b440 tx=0x7f3f8000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f8effd640 1 -- 172.21.15.112:0/3178168209 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80015020 con 0x7f3f901552d0 2024-04-25T01:40:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f8effd640 1 -- 172.21.15.112:0/3178168209 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f80011830 con 0x7f3f901552d0 2024-04-25T01:40:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f9844c640 1 -- 172.21.15.112:0/3178168209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f901ea010 con 0x7f3f901552d0 2024-04-25T01:40:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f8effd640 1 -- 172.21.15.112:0/3178168209 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80011c20 con 0x7f3f901552d0 2024-04-25T01:40:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.587+0000 7f3f9844c640 1 -- 172.21.15.112:0/3178168209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f901ea2d0 con 0x7f3f901552d0 2024-04-25T01:40:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.588+0000 7f3f9844c640 1 -- 172.21.15.112:0/3178168209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f900682a0 con 0x7f3f901552d0 2024-04-25T01:40:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.589+0000 7f3f8effd640 1 -- 172.21.15.112:0/3178168209 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f80018020 con 0x7f3f901552d0 2024-04-25T01:40:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.589+0000 7f3f8effd640 1 --2- 172.21.15.112:0/3178168209 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3f7c06be60 0x7f3f7c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.589+0000 7f3f8effd640 1 -- 172.21.15.112:0/3178168209 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f3f8008ae60 con 0x7f3f901552d0 2024-04-25T01:40:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.589+0000 7f3f961e8640 1 --2- 172.21.15.112:0/3178168209 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3f7c06be60 0x7f3f7c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.589+0000 7f3f961e8640 1 --2- 172.21.15.112:0/3178168209 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3f7c06be60 0x7f3f7c06e320 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3f88033000 tx=0x7f3f88009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:56.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.593+0000 7f3f8effd640 1 -- 172.21.15.112:0/3178168209 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3f8001f020 con 0x7f3f901552d0 2024-04-25T01:40:56.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.766+0000 7f3f9844c640 1 -- 172.21.15.112:0/3178168209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f90072c40 con 0x7f3f901552d0 2024-04-25T01:40:56.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.768+0000 7f3f8effd640 1 -- 172.21.15.112:0/3178168209 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+20616 (secure 0 0 0) 0x7f3f80058eb0 con 0x7f3f901552d0 2024-04-25T01:40:56.754 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:40:56.754 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:26:52.110548+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-25T01:25:59.478275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-25T01:26:00.719358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:40:56.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.770+0000 7f3f9844c640 1 -- 172.21.15.112:0/3178168209 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3f7c06be60 msgr2=0x7f3f7c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:56.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.770+0000 7f3f9844c640 1 --2- 172.21.15.112:0/3178168209 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3f7c06be60 0x7f3f7c06e320 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3f88033000 tx=0x7f3f88009000 comp rx=0 tx=0).stop 2024-04-25T01:40:56.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.770+0000 7f3f9844c640 1 -- 172.21.15.112:0/3178168209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f901552d0 msgr2=0x7f3f90146fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:56.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.770+0000 7f3f9844c640 1 --2- 172.21.15.112:0/3178168209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f901552d0 0x7f3f90146fb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000b440 tx=0x7f3f8000b900 comp rx=0 tx=0).stop 2024-04-25T01:40:56.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.770+0000 7f3f9844c640 1 -- 172.21.15.112:0/3178168209 shutdown_connections 2024-04-25T01:40:56.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.770+0000 7f3f9844c640 1 --2- 172.21.15.112:0/3178168209 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3f7c06be60 0x7f3f7c06e320 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.770+0000 7f3f9844c640 1 --2- 172.21.15.112:0/3178168209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f901552d0 0x7f3f90146fb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.770+0000 7f3f9844c640 1 --2- 172.21.15.112:0/3178168209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f900c3050 0x7f3f90146a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.770+0000 7f3f9844c640 1 --2- 172.21.15.112:0/3178168209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f90071770 0x7f3f9014b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.770+0000 7f3f9844c640 1 -- 172.21.15.112:0/3178168209 >> 172.21.15.112:0/3178168209 conn(0x7f3f9006b160 msgr2=0x7f3f900aa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:56.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.770+0000 7f3f9844c640 1 -- 172.21.15.112:0/3178168209 shutdown_connections 2024-04-25T01:40:56.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.770+0000 7f3f9844c640 1 -- 172.21.15.112:0/3178168209 wait complete. 2024-04-25T01:40:56.767 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:25:40.679119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-04-25T01:25:50.593045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-04-25T01:25:51.664930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-04-25T01:25:59.478275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-04-25T01:26:00.719358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:40:56.768 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:40:56.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.882+0000 7f8fe883f640 1 Processor -- start 2024-04-25T01:40:56.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.882+0000 7f8fe883f640 1 -- start start 2024-04-25T01:40:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.882+0000 7f8fe883f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8fe4072c20 0x7f8fe40aa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.882+0000 7f8fe883f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fe4155d40 con 0x7f8fe414f220 2024-04-25T01:40:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.882+0000 7f8fe883f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fe4156020 con 0x7f8fe40ac830 2024-04-25T01:40:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.883+0000 7f8fe883f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f8fe4071700 con 0x7f8fe4072c20 2024-04-25T01:40:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.883+0000 7f8fe2d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8fe4072c20 0x7f8fe40aa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.883+0000 7f8fe2d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8fe4072c20 0x7f8fe40aa640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47234/0 (socket says 172.21.15.112:47234) 2024-04-25T01:40:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.883+0000 7f8fe259c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8fe414f220 0x7f8fe4072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:48442/0 (socket says 172.21.15.112:48442) 2024-04-25T01:40:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.883+0000 7f8fe2d9d640 1 -- 172.21.15.112:0/348490532 learned_addr learned my addr 172.21.15.112:0/348490532 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.883+0000 7f8fe2d9d640 1 -- 172.21.15.112:0/348490532 >> v1:172.21.15.187:6789/0 conn(0x7f8fe40ac830 legacy=0x7f8fe4155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.883+0000 7f8fe2d9d640 1 -- 172.21.15.112:0/348490532 >> v1:172.21.15.112:6789/0 conn(0x7f8fe414f220 legacy=0x7f8fe4072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.883+0000 7f8fe159a640 1 -- 172.21.15.112:0/348490532 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1008906301 0 0) 0x7f8fe4155d40 con 0x7f8fe414f220 2024-04-25T01:40:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.883+0000 7f8fe2d9d640 1 -- 172.21.15.112:0/348490532 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe4071950 con 0x7f8fe4072c20 2024-04-25T01:40:56.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.884+0000 7f8fe2d9d640 1 --2- 172.21.15.112:0/348490532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fe4072c20 0x7f8fe40aa640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009e50 tx=0x7f8fdc02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfd2561614f7a1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:56.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.884+0000 7f8fe159a640 1 -- 172.21.15.112:0/348490532 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc036070 con 0x7f8fe4072c20 2024-04-25T01:40:56.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.884+0000 7f8fe159a640 1 -- 172.21.15.112:0/348490532 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fdc031600 con 0x7f8fe4072c20 2024-04-25T01:40:56.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.884+0000 7f8fe159a640 1 -- 172.21.15.112:0/348490532 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc0319d0 con 0x7f8fe4072c20 2024-04-25T01:40:56.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.884+0000 7f8fe883f640 1 -- 172.21.15.112:0/348490532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fe4072c20 msgr2=0x7f8fe40aa640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:56.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.884+0000 7f8fe883f640 1 --2- 172.21.15.112:0/348490532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fe4072c20 0x7f8fe40aa640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009e50 tx=0x7f8fdc02a6d0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.884+0000 7f8fe883f640 1 -- 172.21.15.112:0/348490532 shutdown_connections 2024-04-25T01:40:56.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.884+0000 7f8fe883f640 1 --2- 172.21.15.112:0/348490532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fe4072c20 0x7f8fe40aa640 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.884+0000 7f8fe883f640 1 -- 172.21.15.112:0/348490532 >> 172.21.15.112:0/348490532 conn(0x7f8fe40c3a10 msgr2=0x7f8fe40c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:56.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.884+0000 7f8fe883f640 1 -- 172.21.15.112:0/348490532 shutdown_connections 2024-04-25T01:40:56.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.884+0000 7f8fe883f640 1 -- 172.21.15.112:0/348490532 wait complete. 2024-04-25T01:40:56.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.886+0000 7f8fe883f640 1 Processor -- start 2024-04-25T01:40:56.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.886+0000 7f8fe883f640 1 -- start start 2024-04-25T01:40:56.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.886+0000 7f8fe883f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fe4072c20 0x7f8fe406dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:56.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.886+0000 7f8fe883f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fe40ac830 0x7f8fe40c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:56.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.886+0000 7f8fe883f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fe414f220 0x7f8fe40c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:56.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.886+0000 7f8fe883f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe414f600 con 0x7f8fe4072c20 2024-04-25T01:40:56.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.886+0000 7f8fe883f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe4071cc0 con 0x7f8fe40ac830 2024-04-25T01:40:56.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.886+0000 7f8fe883f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe41559a0 con 0x7f8fe414f220 2024-04-25T01:40:56.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.886+0000 7f8fe259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fe4072c20 0x7f8fe406dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:56.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.886+0000 7f8fe259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fe4072c20 0x7f8fe406dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47244/0 (socket says 172.21.15.112:47244) 2024-04-25T01:40:56.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.886+0000 7f8fe259c640 1 -- 172.21.15.112:0/4060696907 learned_addr learned my addr 172.21.15.112:0/4060696907 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:56.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.887+0000 7f8fe1d9b640 1 --2- 172.21.15.112:0/4060696907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fe40ac830 0x7f8fe40c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:56.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.887+0000 7f8fe259c640 1 -- 172.21.15.112:0/4060696907 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fe414f220 msgr2=0x7f8fe40c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:56.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.887+0000 7f8fe259c640 1 --2- 172.21.15.112:0/4060696907 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fe414f220 0x7f8fe40c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.887+0000 7f8fe259c640 1 -- 172.21.15.112:0/4060696907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fe40ac830 msgr2=0x7f8fe40c4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:56.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.887+0000 7f8fe259c640 1 --2- 172.21.15.112:0/4060696907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fe40ac830 0x7f8fe40c4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:56.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.887+0000 7f8fe259c640 1 -- 172.21.15.112:0/4060696907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe4152ac0 con 0x7f8fe4072c20 2024-04-25T01:40:56.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.887+0000 7f8fe1d9b640 1 --2- 172.21.15.112:0/4060696907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fe40ac830 0x7f8fe40c4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:40:56.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.887+0000 7f8fe259c640 1 --2- 172.21.15.112:0/4060696907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fe4072c20 0x7f8fe406dc20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800c430 tx=0x7f8fd800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:56.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.887+0000 7f8fd2ffd640 1 -- 172.21.15.112:0/4060696907 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd80074e0 con 0x7f8fe4072c20 2024-04-25T01:40:56.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.887+0000 7f8fd2ffd640 1 -- 172.21.15.112:0/4060696907 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fd80047e0 con 0x7f8fe4072c20 2024-04-25T01:40:56.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.887+0000 7f8fd2ffd640 1 -- 172.21.15.112:0/4060696907 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8004bd0 con 0x7f8fe4072c20 2024-04-25T01:40:56.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.887+0000 7f8fe883f640 1 -- 172.21.15.112:0/4060696907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe41542f0 con 0x7f8fe4072c20 2024-04-25T01:40:56.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.887+0000 7f8fe883f640 1 -- 172.21.15.112:0/4060696907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe4148a80 con 0x7f8fe4072c20 2024-04-25T01:40:56.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.888+0000 7f8fe883f640 1 -- 172.21.15.112:0/4060696907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe404f500 con 0x7f8fe4072c20 2024-04-25T01:40:56.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.888+0000 7f8fd2ffd640 1 -- 172.21.15.112:0/4060696907 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8fd8011070 con 0x7f8fe4072c20 2024-04-25T01:40:56.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.889+0000 7f8fd2ffd640 1 --2- 172.21.15.112:0/4060696907 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8fc406be30 0x7f8fc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:56.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.889+0000 7f8fd2ffd640 1 -- 172.21.15.112:0/4060696907 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f8fd808ade0 con 0x7f8fe4072c20 2024-04-25T01:40:56.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.889+0000 7f8fe1d9b640 1 --2- 172.21.15.112:0/4060696907 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8fc406be30 0x7f8fc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:56.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.889+0000 7f8fe1d9b640 1 --2- 172.21.15.112:0/4060696907 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8fc406be30 0x7f8fc406e2f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40c12f0 tx=0x7f8fd4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:56.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:56.893+0000 7f8fd2ffd640 1 -- 172.21.15.112:0/4060696907 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8fd8059480 con 0x7f8fe4072c20 2024-04-25T01:40:57.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.066+0000 7f8fe883f640 1 -- 172.21.15.112:0/4060696907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8fe40c36e0 con 0x7f8fe4072c20 2024-04-25T01:40:57.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.067+0000 7f8fd2ffd640 1 -- 172.21.15.112:0/4060696907 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f8fd8058d80 con 0x7f8fe4072c20 2024-04-25T01:40:57.053 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T01:40:57.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.069+0000 7f8fe883f640 1 -- 172.21.15.112:0/4060696907 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8fc406be30 msgr2=0x7f8fc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.069+0000 7f8fe883f640 1 --2- 172.21.15.112:0/4060696907 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8fc406be30 0x7f8fc406e2f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40c12f0 tx=0x7f8fd4009040 comp rx=0 tx=0).stop 2024-04-25T01:40:57.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.069+0000 7f8fe883f640 1 -- 172.21.15.112:0/4060696907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fe4072c20 msgr2=0x7f8fe406dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.069+0000 7f8fe883f640 1 --2- 172.21.15.112:0/4060696907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fe4072c20 0x7f8fe406dc20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800c430 tx=0x7f8fd800c8f0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.069+0000 7f8fe883f640 1 -- 172.21.15.112:0/4060696907 shutdown_connections 2024-04-25T01:40:57.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.069+0000 7f8fe883f640 1 --2- 172.21.15.112:0/4060696907 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f8fc406be30 0x7f8fc406e2f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.069+0000 7f8fe883f640 1 --2- 172.21.15.112:0/4060696907 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fe414f220 0x7f8fe40c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.069+0000 7f8fe883f640 1 --2- 172.21.15.112:0/4060696907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fe40ac830 0x7f8fe40c4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.069+0000 7f8fe883f640 1 --2- 172.21.15.112:0/4060696907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fe4072c20 0x7f8fe406dc20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.069+0000 7f8fe883f640 1 -- 172.21.15.112:0/4060696907 >> 172.21.15.112:0/4060696907 conn(0x7f8fe40c3a10 msgr2=0x7f8fe40aafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:57.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.069+0000 7f8fe883f640 1 -- 172.21.15.112:0/4060696907 shutdown_connections 2024-04-25T01:40:57.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.069+0000 7f8fe883f640 1 -- 172.21.15.112:0/4060696907 wait complete. 2024-04-25T01:40:57.067 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-04-25T01:40:57.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.181+0000 7fd7712da640 1 Processor -- start 2024-04-25T01:40:57.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.181+0000 7fd7712da640 1 -- start start 2024-04-25T01:40:57.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.182+0000 7fd7712da640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd76c06bfe0 0x7fd76c0ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.182+0000 7fd7712da640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd76c0c6490 con 0x7fd76c152fe0 2024-04-25T01:40:57.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.182+0000 7fd7712da640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd76c159a70 con 0x7fd76c06c4c0 2024-04-25T01:40:57.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.182+0000 7fd7712da640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fd76c0c9a60 con 0x7fd76c06bfe0 2024-04-25T01:40:57.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.182+0000 7fd763fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd76c06bfe0 0x7fd76c0ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.182+0000 7fd76affd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd76c152fe0 0x7fd76c0c5d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:48450/0 (socket says 172.21.15.112:48450) 2024-04-25T01:40:57.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.182+0000 7fd76affd640 1 -- 172.21.15.112:0/1838037535 learned_addr learned my addr 172.21.15.112:0/1838037535 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:40:57.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.182+0000 7fd763fff640 1 -- 172.21.15.112:0/1838037535 >> v1:172.21.15.187:6789/0 conn(0x7fd76c06c4c0 legacy=0x7fd76c158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:57.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.182+0000 7fd763fff640 1 -- 172.21.15.112:0/1838037535 >> v1:172.21.15.112:6789/0 conn(0x7fd76c152fe0 legacy=0x7fd76c0c5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.182+0000 7fd76a7fc640 1 -- 172.21.15.112:0/1838037535 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2738934507 0 0) 0x7fd76c0c6490 con 0x7fd76c152fe0 2024-04-25T01:40:57.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.182+0000 7fd763fff640 1 -- 172.21.15.112:0/1838037535 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76c159d50 con 0x7fd76c06bfe0 2024-04-25T01:40:57.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.183+0000 7fd763fff640 1 --2- 172.21.15.112:0/1838037535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd76c06bfe0 0x7fd76c0ca4b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd7540097a0 tx=0x7fd75402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b8390a42744739 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:57.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.183+0000 7fd76a7fc640 1 -- 172.21.15.112:0/1838037535 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd754037070 con 0x7fd76c06bfe0 2024-04-25T01:40:57.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.183+0000 7fd76a7fc640 1 -- 172.21.15.112:0/1838037535 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd75402ae50 con 0x7fd76c06bfe0 2024-04-25T01:40:57.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.183+0000 7fd76a7fc640 1 -- 172.21.15.112:0/1838037535 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd754032660 con 0x7fd76c06bfe0 2024-04-25T01:40:57.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.183+0000 7fd7712da640 1 -- 172.21.15.112:0/1838037535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd76c06bfe0 msgr2=0x7fd76c0ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.183+0000 7fd7712da640 1 --2- 172.21.15.112:0/1838037535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd76c06bfe0 0x7fd76c0ca4b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd7540097a0 tx=0x7fd75402a140 comp rx=0 tx=0).stop 2024-04-25T01:40:57.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.184+0000 7fd7712da640 1 -- 172.21.15.112:0/1838037535 shutdown_connections 2024-04-25T01:40:57.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.184+0000 7fd7712da640 1 --2- 172.21.15.112:0/1838037535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd76c06bfe0 0x7fd76c0ca4b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.184+0000 7fd7712da640 1 -- 172.21.15.112:0/1838037535 >> 172.21.15.112:0/1838037535 conn(0x7fd76c0cb330 msgr2=0x7fd76c0cb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:57.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.184+0000 7fd7712da640 1 -- 172.21.15.112:0/1838037535 shutdown_connections 2024-04-25T01:40:57.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.184+0000 7fd7712da640 1 -- 172.21.15.112:0/1838037535 wait complete. 2024-04-25T01:40:57.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.185+0000 7fd7712da640 1 Processor -- start 2024-04-25T01:40:57.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.185+0000 7fd7712da640 1 -- start start 2024-04-25T01:40:57.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.185+0000 7fd7712da640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd76c06bfe0 0x7fd76c153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.185+0000 7fd7712da640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd76c06c4c0 0x7fd76c1581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.185+0000 7fd7712da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd76c152fe0 0x7fd76c1eea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.185+0000 7fd7712da640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c0c7890 con 0x7fd76c152fe0 2024-04-25T01:40:57.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.185+0000 7fd7712da640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c0ca020 con 0x7fd76c06c4c0 2024-04-25T01:40:57.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.185+0000 7fd7712da640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd76c1533c0 con 0x7fd76c06bfe0 2024-04-25T01:40:57.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.185+0000 7fd76b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd76c152fe0 0x7fd76c1eea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.185+0000 7fd76b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd76c152fe0 0x7fd76c1eea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47268/0 (socket says 172.21.15.112:47268) 2024-04-25T01:40:57.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.185+0000 7fd76b7fe640 1 -- 172.21.15.112:0/3330752315 learned_addr learned my addr 172.21.15.112:0/3330752315 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:57.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.186+0000 7fd76affd640 1 --2- 172.21.15.112:0/3330752315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd76c06bfe0 0x7fd76c153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.186+0000 7fd763fff640 1 --2- 172.21.15.112:0/3330752315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd76c06c4c0 0x7fd76c1581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.186+0000 7fd76b7fe640 1 -- 172.21.15.112:0/3330752315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd76c06bfe0 msgr2=0x7fd76c153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.186+0000 7fd76b7fe640 1 --2- 172.21.15.112:0/3330752315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd76c06bfe0 0x7fd76c153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.186+0000 7fd76b7fe640 1 -- 172.21.15.112:0/3330752315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd76c06c4c0 msgr2=0x7fd76c1581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.186+0000 7fd76b7fe640 1 --2- 172.21.15.112:0/3330752315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd76c06c4c0 0x7fd76c1581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.186+0000 7fd76b7fe640 1 -- 172.21.15.112:0/3330752315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76c1ee580 con 0x7fd76c152fe0 2024-04-25T01:40:57.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.186+0000 7fd76affd640 1 --2- 172.21.15.112:0/3330752315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd76c06bfe0 0x7fd76c153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:40:57.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.186+0000 7fd76b7fe640 1 --2- 172.21.15.112:0/3330752315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd76c152fe0 0x7fd76c1eea30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd76400c7b0 tx=0x7fd76400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:57.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.186+0000 7fd7637fe640 1 -- 172.21.15.112:0/3330752315 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764004b60 con 0x7fd76c152fe0 2024-04-25T01:40:57.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.186+0000 7fd7637fe640 1 -- 172.21.15.112:0/3330752315 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd764004d00 con 0x7fd76c152fe0 2024-04-25T01:40:57.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.186+0000 7fd7637fe640 1 -- 172.21.15.112:0/3330752315 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76400aa90 con 0x7fd76c152fe0 2024-04-25T01:40:57.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.186+0000 7fd7712da640 1 -- 172.21.15.112:0/3330752315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76c151dd0 con 0x7fd76c152fe0 2024-04-25T01:40:57.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.186+0000 7fd7712da640 1 -- 172.21.15.112:0/3330752315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd76c1eba70 con 0x7fd76c152fe0 2024-04-25T01:40:57.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.187+0000 7fd7712da640 1 -- 172.21.15.112:0/3330752315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd76c04f500 con 0x7fd76c152fe0 2024-04-25T01:40:57.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.187+0000 7fd7637fe640 1 -- 172.21.15.112:0/3330752315 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd76400ac30 con 0x7fd76c152fe0 2024-04-25T01:40:57.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.188+0000 7fd7637fe640 1 --2- 172.21.15.112:0/3330752315 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd74c06be30 0x7fd74c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.188+0000 7fd7637fe640 1 -- 172.21.15.112:0/3330752315 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fd76408b530 con 0x7fd76c152fe0 2024-04-25T01:40:57.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.188+0000 7fd76affd640 1 --2- 172.21.15.112:0/3330752315 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd74c06be30 0x7fd74c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.188+0000 7fd76affd640 1 --2- 172.21.15.112:0/3330752315 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd74c06be30 0x7fd74c06e2f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd75c000f60 tx=0x7fd75c007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:57.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.192+0000 7fd7637fe640 1 -- 172.21.15.112:0/3330752315 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd764020070 con 0x7fd76c152fe0 2024-04-25T01:40:57.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.367+0000 7fd7712da640 1 -- 172.21.15.112:0/3330752315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) -- 0x7fd76c0c5b60 con 0x7fd76c152fe0 2024-04-25T01:40:57.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.367+0000 7fd7637fe640 1 -- 172.21.15.112:0/3330752315 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v26) ==== 99+0+11 (secure 0 0 0) 0x7fd764059b40 con 0x7fd76c152fe0 2024-04-25T01:40:57.353 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T01:40:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.369+0000 7fd7712da640 1 -- 172.21.15.112:0/3330752315 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd74c06be30 msgr2=0x7fd74c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.369+0000 7fd7712da640 1 --2- 172.21.15.112:0/3330752315 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd74c06be30 0x7fd74c06e2f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd75c000f60 tx=0x7fd75c007ca0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.369+0000 7fd7712da640 1 -- 172.21.15.112:0/3330752315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd76c152fe0 msgr2=0x7fd76c1eea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.369+0000 7fd7712da640 1 --2- 172.21.15.112:0/3330752315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd76c152fe0 0x7fd76c1eea30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd76400c7b0 tx=0x7fd76400cc70 comp rx=0 tx=0).stop 2024-04-25T01:40:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.369+0000 7fd7712da640 1 -- 172.21.15.112:0/3330752315 shutdown_connections 2024-04-25T01:40:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.369+0000 7fd7712da640 1 --2- 172.21.15.112:0/3330752315 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd74c06be30 0x7fd74c06e2f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.369+0000 7fd7712da640 1 --2- 172.21.15.112:0/3330752315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd76c152fe0 0x7fd76c1eea30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.369+0000 7fd7712da640 1 --2- 172.21.15.112:0/3330752315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd76c06c4c0 0x7fd76c1581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.369+0000 7fd7712da640 1 --2- 172.21.15.112:0/3330752315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd76c06bfe0 0x7fd76c153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.369+0000 7fd7712da640 1 -- 172.21.15.112:0/3330752315 >> 172.21.15.112:0/3330752315 conn(0x7fd76c0cb330 msgr2=0x7fd76c0cb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.369+0000 7fd7712da640 1 -- 172.21.15.112:0/3330752315 shutdown_connections 2024-04-25T01:40:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.369+0000 7fd7712da640 1 -- 172.21.15.112:0/3330752315 wait complete. 2024-04-25T01:40:57.366 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-04-25T01:40:57.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.483+0000 7fd687109640 1 Processor -- start 2024-04-25T01:40:57.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.483+0000 7fd687109640 1 -- start start 2024-04-25T01:40:57.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.484+0000 7fd687109640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd680159680 0x7fd6801612b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.484+0000 7fd687109640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6800c3970 con 0x7fd6800c3d90 2024-04-25T01:40:57.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.484+0000 7fd687109640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6800c6070 con 0x7fd680153210 2024-04-25T01:40:57.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.484+0000 7fd687109640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fd6800c5660 con 0x7fd680159680 2024-04-25T01:40:57.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.484+0000 7fd67ffff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd6800c3d90 0x7fd680158f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37894/0 (socket says 172.21.15.112:37894) 2024-04-25T01:40:57.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.484+0000 7fd6856a6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd680159680 0x7fd6801612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.484+0000 7fd67ffff640 1 -- 172.21.15.112:0/330044065 learned_addr learned my addr 172.21.15.112:0/330044065 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:40:57.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.484+0000 7fd67f7fe640 1 -- 172.21.15.112:0/330044065 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3824502645 0 0) 0x7fd6800c3970 con 0x7fd6800c3d90 2024-04-25T01:40:57.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.484+0000 7fd67f7fe640 1 -- 172.21.15.112:0/330044065 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd668003520 con 0x7fd6800c3d90 2024-04-25T01:40:57.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.484+0000 7fd6856a6640 1 -- 172.21.15.112:0/330044065 >> v1:172.21.15.187:6789/0 conn(0x7fd680153210 legacy=0x7fd68006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:57.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.484+0000 7fd6856a6640 1 -- 172.21.15.112:0/330044065 >> v1:172.21.15.112:6789/0 conn(0x7fd6800c3d90 legacy=0x7fd680158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.485+0000 7fd6856a6640 1 -- 172.21.15.112:0/330044065 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6800c6350 con 0x7fd680159680 2024-04-25T01:40:57.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.485+0000 7fd6856a6640 1 --2- 172.21.15.112:0/330044065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd680159680 0x7fd6801612b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd678009d90 tx=0x7fd67802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f437ef133b3305 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:57.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.485+0000 7fd67f7fe640 1 -- 172.21.15.112:0/330044065 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678007740 con 0x7fd680159680 2024-04-25T01:40:57.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.485+0000 7fd67f7fe640 1 -- 172.21.15.112:0/330044065 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6780078e0 con 0x7fd680159680 2024-04-25T01:40:57.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.485+0000 7fd67f7fe640 1 -- 172.21.15.112:0/330044065 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678007c90 con 0x7fd680159680 2024-04-25T01:40:57.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.486+0000 7fd687109640 1 -- 172.21.15.112:0/330044065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd680159680 msgr2=0x7fd6801612b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.486+0000 7fd687109640 1 --2- 172.21.15.112:0/330044065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd680159680 0x7fd6801612b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd678009d90 tx=0x7fd67802a660 comp rx=0 tx=0).stop 2024-04-25T01:40:57.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.486+0000 7fd687109640 1 -- 172.21.15.112:0/330044065 shutdown_connections 2024-04-25T01:40:57.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.486+0000 7fd687109640 1 --2- 172.21.15.112:0/330044065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd680159680 0x7fd6801612b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.486+0000 7fd687109640 1 -- 172.21.15.112:0/330044065 >> 172.21.15.112:0/330044065 conn(0x7fd6800c4260 msgr2=0x7fd6800c4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:57.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.486+0000 7fd687109640 1 -- 172.21.15.112:0/330044065 shutdown_connections 2024-04-25T01:40:57.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.486+0000 7fd687109640 1 -- 172.21.15.112:0/330044065 wait complete. 2024-04-25T01:40:57.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.487+0000 7fd687109640 1 Processor -- start 2024-04-25T01:40:57.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.487+0000 7fd687109640 1 -- start start 2024-04-25T01:40:57.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.488+0000 7fd687109640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6800c3d90 0x7fd680072720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.488+0000 7fd687109640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd680153210 0x7fd680156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.488+0000 7fd687109640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6801573c0 0x7fd6801ee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.488+0000 7fd687109640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd68006ca00 con 0x7fd6801573c0 2024-04-25T01:40:57.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.488+0000 7fd687109640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd68006c880 con 0x7fd680153210 2024-04-25T01:40:57.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.488+0000 7fd687109640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd6800c5af0 con 0x7fd6800c3d90 2024-04-25T01:40:57.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.488+0000 7fd6856a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6801573c0 0x7fd6801ee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.488+0000 7fd6856a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6801573c0 0x7fd6801ee7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34584/0 (socket says 172.21.15.112:34584) 2024-04-25T01:40:57.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.488+0000 7fd6856a6640 1 -- 172.21.15.112:0/707039998 learned_addr learned my addr 172.21.15.112:0/707039998 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:57.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.488+0000 7fd67ffff640 1 --2- 172.21.15.112:0/707039998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd680153210 0x7fd680156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.488+0000 7fd684ea5640 1 --2- 172.21.15.112:0/707039998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6800c3d90 0x7fd680072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.488+0000 7fd6856a6640 1 -- 172.21.15.112:0/707039998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6800c3d90 msgr2=0x7fd680072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.488+0000 7fd6856a6640 1 --2- 172.21.15.112:0/707039998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6800c3d90 0x7fd680072720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.488+0000 7fd6856a6640 1 -- 172.21.15.112:0/707039998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd680153210 msgr2=0x7fd680156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.488+0000 7fd6856a6640 1 --2- 172.21.15.112:0/707039998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd680153210 0x7fd680156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.488+0000 7fd6856a6640 1 -- 172.21.15.112:0/707039998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd680151430 con 0x7fd6801573c0 2024-04-25T01:40:57.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.488+0000 7fd6856a6640 1 --2- 172.21.15.112:0/707039998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6801573c0 0x7fd6801ee7c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd678009d90 tx=0x7fd678004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:57.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.489+0000 7fd67d7fa640 1 -- 172.21.15.112:0/707039998 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67803f070 con 0x7fd6801573c0 2024-04-25T01:40:57.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.489+0000 7fd67d7fa640 1 -- 172.21.15.112:0/707039998 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd678031c40 con 0x7fd6801573c0 2024-04-25T01:40:57.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.489+0000 7fd687109640 1 -- 172.21.15.112:0/707039998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd680156b40 con 0x7fd6801573c0 2024-04-25T01:40:57.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.489+0000 7fd67d7fa640 1 -- 172.21.15.112:0/707039998 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678009450 con 0x7fd6801573c0 2024-04-25T01:40:57.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.489+0000 7fd687109640 1 -- 172.21.15.112:0/707039998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd680150950 con 0x7fd6801573c0 2024-04-25T01:40:57.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.490+0000 7fd687109640 1 -- 172.21.15.112:0/707039998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd68004f500 con 0x7fd6801573c0 2024-04-25T01:40:57.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.490+0000 7fd67d7fa640 1 -- 172.21.15.112:0/707039998 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd678003990 con 0x7fd6801573c0 2024-04-25T01:40:57.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.490+0000 7fd67d7fa640 1 --2- 172.21.15.112:0/707039998 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd66c06be30 0x7fd66c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.490+0000 7fd67d7fa640 1 -- 172.21.15.112:0/707039998 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fd6780ac2a0 con 0x7fd6801573c0 2024-04-25T01:40:57.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.490+0000 7fd684ea5640 1 --2- 172.21.15.112:0/707039998 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd66c06be30 0x7fd66c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.491+0000 7fd684ea5640 1 --2- 172.21.15.112:0/707039998 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd66c06be30 0x7fd66c06e2f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd670007fd0 tx=0x7fd670009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:57.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.495+0000 7fd67d7fa640 1 -- 172.21.15.112:0/707039998 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd67807a940 con 0x7fd6801573c0 2024-04-25T01:40:57.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.671+0000 7fd687109640 1 -- 172.21.15.112:0/707039998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) -- 0x7fd680074950 con 0x7fd6801573c0 2024-04-25T01:40:57.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.671+0000 7fd67d7fa640 1 -- 172.21.15.112:0/707039998 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v26) ==== 95+0+11 (secure 0 0 0) 0x7fd67807a240 con 0x7fd6801573c0 2024-04-25T01:40:57.657 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T01:40:57.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.673+0000 7fd687109640 1 -- 172.21.15.112:0/707039998 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd66c06be30 msgr2=0x7fd66c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.673+0000 7fd687109640 1 --2- 172.21.15.112:0/707039998 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd66c06be30 0x7fd66c06e2f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd670007fd0 tx=0x7fd670009040 comp rx=0 tx=0).stop 2024-04-25T01:40:57.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.673+0000 7fd687109640 1 -- 172.21.15.112:0/707039998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6801573c0 msgr2=0x7fd6801ee7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.673+0000 7fd687109640 1 --2- 172.21.15.112:0/707039998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6801573c0 0x7fd6801ee7c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd678009d90 tx=0x7fd678004a40 comp rx=0 tx=0).stop 2024-04-25T01:40:57.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.673+0000 7fd687109640 1 -- 172.21.15.112:0/707039998 shutdown_connections 2024-04-25T01:40:57.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.673+0000 7fd687109640 1 --2- 172.21.15.112:0/707039998 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd66c06be30 0x7fd66c06e2f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.673+0000 7fd687109640 1 --2- 172.21.15.112:0/707039998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6801573c0 0x7fd6801ee7c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.673+0000 7fd687109640 1 --2- 172.21.15.112:0/707039998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd680153210 0x7fd680156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.673+0000 7fd687109640 1 --2- 172.21.15.112:0/707039998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6800c3d90 0x7fd680072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.673+0000 7fd687109640 1 -- 172.21.15.112:0/707039998 >> 172.21.15.112:0/707039998 conn(0x7fd6800c4260 msgr2=0x7fd68014e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:57.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.673+0000 7fd687109640 1 -- 172.21.15.112:0/707039998 shutdown_connections 2024-04-25T01:40:57.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.673+0000 7fd687109640 1 -- 172.21.15.112:0/707039998 wait complete. 2024-04-25T01:40:57.670 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-04-25T01:40:57.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.784+0000 7f741fb5a640 1 Processor -- start 2024-04-25T01:40:57.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.784+0000 7f741fb5a640 1 -- start start 2024-04-25T01:40:57.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.785+0000 7f741fb5a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f741814fbe0 0x7f74180c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.785+0000 7f741fb5a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74180c2e60 0x7f74180726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.785+0000 7f741fb5a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f741806f290 con 0x7f741806d460 2024-04-25T01:40:57.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.785+0000 7f741fb5a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f741814ffc0 con 0x7f741814fbe0 2024-04-25T01:40:57.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.785+0000 7f741fb5a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f74181533a0 con 0x7f74180c2e60 2024-04-25T01:40:57.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.785+0000 7f741d0f5640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f741806d460 0x7f7418071fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37900/0 (socket says 172.21.15.112:37900) 2024-04-25T01:40:57.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.785+0000 7f741d0f5640 1 -- 172.21.15.112:0/40094424 learned_addr learned my addr 172.21.15.112:0/40094424 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:40:57.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.785+0000 7f741d8f6640 1 --2- 172.21.15.112:0/40094424 >> v2:172.21.15.187:3300/0 conn(0x7f741814fbe0 0x7f74180c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.785+0000 7f741e0f7640 1 --2- 172.21.15.112:0/40094424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74180c2e60 0x7f74180726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741c8f4640 1 -- 172.21.15.112:0/40094424 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3936251952 0 0) 0x7f741806f290 con 0x7f741806d460 2024-04-25T01:40:57.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741c8f4640 1 -- 172.21.15.112:0/40094424 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7404003520 con 0x7f741806d460 2024-04-25T01:40:57.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741c8f4640 1 -- 172.21.15.112:0/40094424 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3272692349 0 0) 0x7f7404003520 con 0x7f741806d460 2024-04-25T01:40:57.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741c8f4640 1 -- 172.21.15.112:0/40094424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74180c2e60 msgr2=0x7f74180726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:57.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741c8f4640 1 --2- 172.21.15.112:0/40094424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74180c2e60 0x7f74180726c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741c8f4640 1 -- 172.21.15.112:0/40094424 >> v2:172.21.15.187:3300/0 conn(0x7f741814fbe0 msgr2=0x7f74180c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:57.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741c8f4640 1 --2- 172.21.15.112:0/40094424 >> v2:172.21.15.187:3300/0 conn(0x7f741814fbe0 0x7f74180c2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741c8f4640 1 -- 172.21.15.112:0/40094424 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741806efb0 con 0x7f741806d460 2024-04-25T01:40:57.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741c8f4640 1 -- 172.21.15.112:0/40094424 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f7408001670 con 0x7f741806d460 2024-04-25T01:40:57.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741c8f4640 1 -- 172.21.15.112:0/40094424 >> v1:172.21.15.112:6789/0 conn(0x7f741806d460 legacy=0x7f7418071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741c8f4640 1 --2- 172.21.15.112:0/40094424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74040039b0 0x7f7404003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741c8f4640 1 --2- 172.21.15.112:0/40094424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74180c2e60 0x7f7404004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741c8f4640 1 --2- 172.21.15.112:0/40094424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7404004d40 0x7f74040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741c8f4640 1 -- 172.21.15.112:0/40094424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74181533a0 con 0x7f7404004d40 2024-04-25T01:40:57.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741c8f4640 1 -- 172.21.15.112:0/40094424 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f740400b9c0 con 0x7f74040039b0 2024-04-25T01:40:57.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741c8f4640 1 -- 172.21.15.112:0/40094424 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f740400bb40 con 0x7f74180c2e60 2024-04-25T01:40:57.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741e0f7640 1 --2- 172.21.15.112:0/40094424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7404004d40 0x7f74040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741d8f6640 1 --2- 172.21.15.112:0/40094424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74040039b0 0x7f7404003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741d0f5640 1 --2- 172.21.15.112:0/40094424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74180c2e60 0x7f7404004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741e0f7640 1 -- 172.21.15.112:0/40094424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74180c2e60 msgr2=0x7f7404004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:57.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741e0f7640 1 --2- 172.21.15.112:0/40094424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74180c2e60 0x7f7404004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741e0f7640 1 -- 172.21.15.112:0/40094424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74040039b0 msgr2=0x7f7404003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:57.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741e0f7640 1 --2- 172.21.15.112:0/40094424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74040039b0 0x7f7404003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741e0f7640 1 -- 172.21.15.112:0/40094424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7404003520 con 0x7f7404004d40 2024-04-25T01:40:57.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741d0f5640 1 --2- 172.21.15.112:0/40094424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74180c2e60 0x7f7404004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:40:57.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.786+0000 7f741d8f6640 1 --2- 172.21.15.112:0/40094424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74040039b0 0x7f7404003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:40:57.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.787+0000 7f741e0f7640 1 --2- 172.21.15.112:0/40094424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7404004d40 0x7f74040051c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7410009ac0 tx=0x7f7404005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1188fa35a9a2079a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:57.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.787+0000 7f741c8f4640 1 -- 172.21.15.112:0/40094424 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410036070 con 0x7f7404004d40 2024-04-25T01:40:57.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.787+0000 7f741c8f4640 1 -- 172.21.15.112:0/40094424 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7410031400 con 0x7f7404004d40 2024-04-25T01:40:57.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.787+0000 7f741c8f4640 1 -- 172.21.15.112:0/40094424 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410009690 con 0x7f7404004d40 2024-04-25T01:40:57.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.788+0000 7f741fb5a640 1 -- 172.21.15.112:0/40094424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7404004d40 msgr2=0x7f74040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.788+0000 7f741fb5a640 1 --2- 172.21.15.112:0/40094424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7404004d40 0x7f74040051c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7410009ac0 tx=0x7f7404005830 comp rx=0 tx=0).stop 2024-04-25T01:40:57.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.788+0000 7f741e0f7640 1 -- 172.21.15.112:0/40094424 reap_dead start 2024-04-25T01:40:57.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.788+0000 7f741fb5a640 1 -- 172.21.15.112:0/40094424 shutdown_connections 2024-04-25T01:40:57.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.788+0000 7f741fb5a640 1 -- 172.21.15.112:0/40094424 >> 172.21.15.112:0/40094424 conn(0x7f74180c3a10 msgr2=0x7f74180c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:57.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.788+0000 7f741fb5a640 1 -- 172.21.15.112:0/40094424 shutdown_connections 2024-04-25T01:40:57.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.788+0000 7f741fb5a640 1 -- 172.21.15.112:0/40094424 wait complete. 2024-04-25T01:40:57.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.789+0000 7f741fb5a640 1 Processor -- start 2024-04-25T01:40:57.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.789+0000 7f741fb5a640 1 -- start start 2024-04-25T01:40:57.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.789+0000 7f741fb5a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741814b6a0 0x7f74181e7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.789+0000 7f741fb5a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74181e8410 0x7f74180ab640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.789+0000 7f741fb5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74180abb80 0x7f74181ea510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.789+0000 7f741fb5a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74180c3700 con 0x7f74180abb80 2024-04-25T01:40:57.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.789+0000 7f741fb5a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7418153030 con 0x7f741814b6a0 2024-04-25T01:40:57.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.789+0000 7f741fb5a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f74180aaed0 con 0x7f74181e8410 2024-04-25T01:40:57.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.790+0000 7f741e0f7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74180abb80 0x7f74181ea510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.790+0000 7f741e0f7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74180abb80 0x7f74181ea510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34596/0 (socket says 172.21.15.112:34596) 2024-04-25T01:40:57.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.790+0000 7f741e0f7640 1 -- 172.21.15.112:0/2929804157 learned_addr learned my addr 172.21.15.112:0/2929804157 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:57.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.790+0000 7f741d0f5640 1 --2- 172.21.15.112:0/2929804157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74181e8410 0x7f74180ab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.790+0000 7f741d8f6640 1 --2- 172.21.15.112:0/2929804157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741814b6a0 0x7f74181e7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.790+0000 7f741e0f7640 1 -- 172.21.15.112:0/2929804157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74181e8410 msgr2=0x7f74180ab640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.790+0000 7f741e0f7640 1 --2- 172.21.15.112:0/2929804157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74181e8410 0x7f74180ab640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.790+0000 7f741e0f7640 1 -- 172.21.15.112:0/2929804157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741814b6a0 msgr2=0x7f74181e7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.790+0000 7f741e0f7640 1 --2- 172.21.15.112:0/2929804157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741814b6a0 0x7f74181e7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.790+0000 7f741e0f7640 1 -- 172.21.15.112:0/2929804157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7418156910 con 0x7f74180abb80 2024-04-25T01:40:57.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.790+0000 7f741d8f6640 1 --2- 172.21.15.112:0/2929804157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741814b6a0 0x7f74181e7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:40:57.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.790+0000 7f741e0f7640 1 --2- 172.21.15.112:0/2929804157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74180abb80 0x7f74181ea510 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7410009be0 tx=0x7f7410004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:57.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.790+0000 7f740e7fc640 1 -- 172.21.15.112:0/2929804157 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410041070 con 0x7f74180abb80 2024-04-25T01:40:57.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.790+0000 7f740e7fc640 1 -- 172.21.15.112:0/2929804157 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7410037040 con 0x7f74180abb80 2024-04-25T01:40:57.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.790+0000 7f740e7fc640 1 -- 172.21.15.112:0/2929804157 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410036070 con 0x7f74180abb80 2024-04-25T01:40:57.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.790+0000 7f741fb5a640 1 -- 172.21.15.112:0/2929804157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74180ac4d0 con 0x7f74180abb80 2024-04-25T01:40:57.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.790+0000 7f741fb5a640 1 -- 172.21.15.112:0/2929804157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74180aca90 con 0x7f74180abb80 2024-04-25T01:40:57.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.791+0000 7f741fb5a640 1 -- 172.21.15.112:0/2929804157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f741804f500 con 0x7f74180abb80 2024-04-25T01:40:57.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.792+0000 7f740e7fc640 1 -- 172.21.15.112:0/2929804157 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7410044070 con 0x7f74180abb80 2024-04-25T01:40:57.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.792+0000 7f740e7fc640 1 --2- 172.21.15.112:0/2929804157 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f73fc06bda0 0x7f73fc06e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.792+0000 7f741d8f6640 1 --2- 172.21.15.112:0/2929804157 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f73fc06bda0 0x7f73fc06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.792+0000 7f740e7fc640 1 -- 172.21.15.112:0/2929804157 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f74100aa740 con 0x7f74180abb80 2024-04-25T01:40:57.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.792+0000 7f741d8f6640 1 --2- 172.21.15.112:0/2929804157 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f73fc06bda0 0x7f73fc06e260 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f74180c48e0 tx=0x7f741400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:57.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.796+0000 7f740e7fc640 1 -- 172.21.15.112:0/2929804157 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7410078790 con 0x7f74180abb80 2024-04-25T01:40:57.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.969+0000 7f741fb5a640 1 -- 172.21.15.112:0/2929804157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) -- 0x7f74180a9e20 con 0x7f74180abb80 2024-04-25T01:40:57.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.970+0000 7f740e7fc640 1 -- 172.21.15.112:0/2929804157 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v26) ==== 99+0+11 (secure 0 0 0) 0x7f741007ca90 con 0x7f74180abb80 2024-04-25T01:40:57.956 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T01:40:57.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.972+0000 7f741fb5a640 1 -- 172.21.15.112:0/2929804157 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f73fc06bda0 msgr2=0x7f73fc06e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.972+0000 7f741fb5a640 1 --2- 172.21.15.112:0/2929804157 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f73fc06bda0 0x7f73fc06e260 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f74180c48e0 tx=0x7f741400b040 comp rx=0 tx=0).stop 2024-04-25T01:40:57.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.972+0000 7f741fb5a640 1 -- 172.21.15.112:0/2929804157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74180abb80 msgr2=0x7f74181ea510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.972+0000 7f741fb5a640 1 --2- 172.21.15.112:0/2929804157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74180abb80 0x7f74181ea510 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7410009be0 tx=0x7f7410004730 comp rx=0 tx=0).stop 2024-04-25T01:40:57.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.972+0000 7f741fb5a640 1 -- 172.21.15.112:0/2929804157 shutdown_connections 2024-04-25T01:40:57.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.972+0000 7f741fb5a640 1 --2- 172.21.15.112:0/2929804157 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f73fc06bda0 0x7f73fc06e260 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.972+0000 7f741fb5a640 1 --2- 172.21.15.112:0/2929804157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74180abb80 0x7f74181ea510 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.972+0000 7f741fb5a640 1 --2- 172.21.15.112:0/2929804157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74181e8410 0x7f74180ab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.972+0000 7f741fb5a640 1 --2- 172.21.15.112:0/2929804157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741814b6a0 0x7f74181e7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.972+0000 7f741fb5a640 1 -- 172.21.15.112:0/2929804157 >> 172.21.15.112:0/2929804157 conn(0x7f74180c3a10 msgr2=0x7f74181515e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:57.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.972+0000 7f741fb5a640 1 -- 172.21.15.112:0/2929804157 shutdown_connections 2024-04-25T01:40:57.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:57.972+0000 7f741fb5a640 1 -- 172.21.15.112:0/2929804157 wait complete. 2024-04-25T01:40:57.969 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-04-25T01:40:58.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.083+0000 7f559646d640 1 Processor -- start 2024-04-25T01:40:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.083+0000 7f559646d640 1 -- start start 2024-04-25T01:40:58.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.083+0000 7f559646d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55900c9250 0x7f55900c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.083+0000 7f559646d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f55900ca370 0x7f559006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.083+0000 7f559646d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55901534a0 con 0x7f55900ca8a0 2024-04-25T01:40:58.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.083+0000 7f559646d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5590156990 con 0x7f55900ca370 2024-04-25T01:40:58.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.083+0000 7f559646d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5590156500 con 0x7f55900c9250 2024-04-25T01:40:58.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.083+0000 7f5594a0a640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f55900ca8a0 0x7f559006c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37916/0 (socket says 172.21.15.112:37916) 2024-04-25T01:40:58.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.083+0000 7f5594a0a640 1 -- 172.21.15.112:0/1610256964 learned_addr learned my addr 172.21.15.112:0/1610256964 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:40:58.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558f7fe640 1 --2- 172.21.15.112:0/1610256964 >> v2:172.21.15.187:3300/0 conn(0x7f55900ca370 0x7f559006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558ffff640 1 --2- 172.21.15.112:0/1610256964 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55900c9250 0x7f55900c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558effd640 1 -- 172.21.15.112:0/1610256964 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1442159957 0 0) 0x7f55901534a0 con 0x7f55900ca8a0 2024-04-25T01:40:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558effd640 1 -- 172.21.15.112:0/1610256964 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5578003520 con 0x7f55900ca8a0 2024-04-25T01:40:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558effd640 1 -- 172.21.15.112:0/1610256964 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 302268154 0 0) 0x7f5578003520 con 0x7f55900ca8a0 2024-04-25T01:40:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558effd640 1 -- 172.21.15.112:0/1610256964 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55900c9250 msgr2=0x7f55900c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558effd640 1 --2- 172.21.15.112:0/1610256964 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55900c9250 0x7f55900c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558effd640 1 -- 172.21.15.112:0/1610256964 >> v2:172.21.15.187:3300/0 conn(0x7f55900ca370 msgr2=0x7f559006bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558effd640 1 --2- 172.21.15.112:0/1610256964 >> v2:172.21.15.187:3300/0 conn(0x7f55900ca370 0x7f559006bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558effd640 1 -- 172.21.15.112:0/1610256964 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55900cae30 con 0x7f55900ca8a0 2024-04-25T01:40:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558effd640 1 -- 172.21.15.112:0/1610256964 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f55880019f0 con 0x7f55900ca8a0 2024-04-25T01:40:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558effd640 1 -- 172.21.15.112:0/1610256964 >> v1:172.21.15.112:6789/0 conn(0x7f55900ca8a0 legacy=0x7f559006c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558effd640 1 --2- 172.21.15.112:0/1610256964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55780039b0 0x7f5578003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558effd640 1 --2- 172.21.15.112:0/1610256964 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55900c9250 0x7f5578004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558effd640 1 --2- 172.21.15.112:0/1610256964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5578004d40 0x7f55780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558effd640 1 -- 172.21.15.112:0/1610256964 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5590156990 con 0x7f55780039b0 2024-04-25T01:40:58.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558effd640 1 -- 172.21.15.112:0/1610256964 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5590156500 con 0x7f5578004d40 2024-04-25T01:40:58.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558effd640 1 -- 172.21.15.112:0/1610256964 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f557800ba40 con 0x7f55900c9250 2024-04-25T01:40:58.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558ffff640 1 --2- 172.21.15.112:0/1610256964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55780039b0 0x7f5578003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558f7fe640 1 --2- 172.21.15.112:0/1610256964 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55900c9250 0x7f5578004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f5594a0a640 1 --2- 172.21.15.112:0/1610256964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5578004d40 0x7f55780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558ffff640 1 -- 172.21.15.112:0/1610256964 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55900c9250 msgr2=0x7f5578004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:58.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558ffff640 1 --2- 172.21.15.112:0/1610256964 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55900c9250 0x7f5578004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558ffff640 1 -- 172.21.15.112:0/1610256964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5578004d40 msgr2=0x7f55780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:58.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558ffff640 1 --2- 172.21.15.112:0/1610256964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5578004d40 0x7f55780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f558ffff640 1 -- 172.21.15.112:0/1610256964 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5578003520 con 0x7f55780039b0 2024-04-25T01:40:58.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.084+0000 7f5594a0a640 1 --2- 172.21.15.112:0/1610256964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5578004d40 0x7f55780051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:40:58.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.085+0000 7f558ffff640 1 --2- 172.21.15.112:0/1610256964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55780039b0 0x7f5578003dd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5584009ac0 tx=0x7f5578005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75bedf7eb5b6a8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:58.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.085+0000 7f558effd640 1 -- 172.21.15.112:0/1610256964 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584036070 con 0x7f55780039b0 2024-04-25T01:40:58.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.085+0000 7f558effd640 1 -- 172.21.15.112:0/1610256964 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5584031400 con 0x7f55780039b0 2024-04-25T01:40:58.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.085+0000 7f558effd640 1 -- 172.21.15.112:0/1610256964 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584009690 con 0x7f55780039b0 2024-04-25T01:40:58.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.086+0000 7f559646d640 1 -- 172.21.15.112:0/1610256964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55780039b0 msgr2=0x7f5578003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:58.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.086+0000 7f559646d640 1 --2- 172.21.15.112:0/1610256964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55780039b0 0x7f5578003dd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5584009ac0 tx=0x7f5578005830 comp rx=0 tx=0).stop 2024-04-25T01:40:58.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.086+0000 7f5594a0a640 1 -- 172.21.15.112:0/1610256964 reap_dead start 2024-04-25T01:40:58.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.086+0000 7f559646d640 1 -- 172.21.15.112:0/1610256964 shutdown_connections 2024-04-25T01:40:58.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.086+0000 7f559646d640 1 -- 172.21.15.112:0/1610256964 >> 172.21.15.112:0/1610256964 conn(0x7f55900cb310 msgr2=0x7f55900cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:58.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.086+0000 7f559646d640 1 -- 172.21.15.112:0/1610256964 shutdown_connections 2024-04-25T01:40:58.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.086+0000 7f559646d640 1 -- 172.21.15.112:0/1610256964 wait complete. 2024-04-25T01:40:58.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.087+0000 7f559646d640 1 Processor -- start 2024-04-25T01:40:58.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.087+0000 7f559646d640 1 -- start start 2024-04-25T01:40:58.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f559646d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55780039b0 0x7f5590158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f559646d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55900bfb60 0x7f55901eb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f559646d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55901e28e0 0x7f55901ebd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f559646d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5590155eb0 con 0x7f55780039b0 2024-04-25T01:40:58.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f559646d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5590155bd0 con 0x7f55900bfb60 2024-04-25T01:40:58.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f559646d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5590156190 con 0x7f55901e28e0 2024-04-25T01:40:58.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f558ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55780039b0 0x7f5590158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f558ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55780039b0 0x7f5590158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34608/0 (socket says 172.21.15.112:34608) 2024-04-25T01:40:58.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f558ffff640 1 -- 172.21.15.112:0/1216300567 learned_addr learned my addr 172.21.15.112:0/1216300567 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:58.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f558f7fe640 1 --2- 172.21.15.112:0/1216300567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55900bfb60 0x7f55901eb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f5594a0a640 1 --2- 172.21.15.112:0/1216300567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55901e28e0 0x7f55901ebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f558ffff640 1 -- 172.21.15.112:0/1216300567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55901e28e0 msgr2=0x7f55901ebd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:58.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f558ffff640 1 --2- 172.21.15.112:0/1216300567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55901e28e0 0x7f55901ebd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f558ffff640 1 -- 172.21.15.112:0/1216300567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55900bfb60 msgr2=0x7f55901eb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:58.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f558ffff640 1 --2- 172.21.15.112:0/1216300567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55900bfb60 0x7f55901eb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f558ffff640 1 -- 172.21.15.112:0/1216300567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55900c1410 con 0x7f55780039b0 2024-04-25T01:40:58.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f558f7fe640 1 --2- 172.21.15.112:0/1216300567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55900bfb60 0x7f55901eb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:40:58.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f558ffff640 1 --2- 172.21.15.112:0/1216300567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55780039b0 0x7f5590158060 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5584009ac0 tx=0x7f5584004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:58.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f558cff9640 1 -- 172.21.15.112:0/1216300567 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584040020 con 0x7f55780039b0 2024-04-25T01:40:58.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f558cff9640 1 -- 172.21.15.112:0/1216300567 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5584003bc0 con 0x7f55780039b0 2024-04-25T01:40:58.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f559646d640 1 -- 172.21.15.112:0/1216300567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55900c16f0 con 0x7f55780039b0 2024-04-25T01:40:58.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f558cff9640 1 -- 172.21.15.112:0/1216300567 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584036040 con 0x7f55780039b0 2024-04-25T01:40:58.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.088+0000 7f559646d640 1 -- 172.21.15.112:0/1216300567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55901585a0 con 0x7f55780039b0 2024-04-25T01:40:58.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.089+0000 7f559646d640 1 -- 172.21.15.112:0/1216300567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f559004f500 con 0x7f55780039b0 2024-04-25T01:40:58.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.090+0000 7f558cff9640 1 -- 172.21.15.112:0/1216300567 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5584043020 con 0x7f55780039b0 2024-04-25T01:40:58.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.090+0000 7f558cff9640 1 --2- 172.21.15.112:0/1216300567 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f557c06be30 0x7f557c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.090+0000 7f558cff9640 1 -- 172.21.15.112:0/1216300567 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f55840ab350 con 0x7f55780039b0 2024-04-25T01:40:58.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.090+0000 7f558f7fe640 1 --2- 172.21.15.112:0/1216300567 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f557c06be30 0x7f557c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.091+0000 7f558f7fe640 1 --2- 172.21.15.112:0/1216300567 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f557c06be30 0x7f557c06e2f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f55901554b0 tx=0x7f5580008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:58.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.095+0000 7f558cff9640 1 -- 172.21.15.112:0/1216300567 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f558404a070 con 0x7f55780039b0 2024-04-25T01:40:58.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.266+0000 7f559646d640 1 -- 172.21.15.112:0/1216300567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) -- 0x7f55900be970 con 0x7f55780039b0 2024-04-25T01:40:58.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.266+0000 7f558cff9640 1 -- 172.21.15.112:0/1216300567 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v26) ==== 95+0+11 (secure 0 0 0) 0x7f55840044e0 con 0x7f55780039b0 2024-04-25T01:40:58.253 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T01:40:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.268+0000 7f559646d640 1 -- 172.21.15.112:0/1216300567 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f557c06be30 msgr2=0x7f557c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.268+0000 7f559646d640 1 --2- 172.21.15.112:0/1216300567 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f557c06be30 0x7f557c06e2f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f55901554b0 tx=0x7f5580008040 comp rx=0 tx=0).stop 2024-04-25T01:40:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.268+0000 7f559646d640 1 -- 172.21.15.112:0/1216300567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55780039b0 msgr2=0x7f5590158060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.268+0000 7f559646d640 1 --2- 172.21.15.112:0/1216300567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55780039b0 0x7f5590158060 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5584009ac0 tx=0x7f5584004730 comp rx=0 tx=0).stop 2024-04-25T01:40:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.269+0000 7f559646d640 1 -- 172.21.15.112:0/1216300567 shutdown_connections 2024-04-25T01:40:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.269+0000 7f559646d640 1 --2- 172.21.15.112:0/1216300567 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f557c06be30 0x7f557c06e2f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.269+0000 7f559646d640 1 --2- 172.21.15.112:0/1216300567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55901e28e0 0x7f55901ebd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.269+0000 7f559646d640 1 --2- 172.21.15.112:0/1216300567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55900bfb60 0x7f55901eb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.269+0000 7f559646d640 1 --2- 172.21.15.112:0/1216300567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55780039b0 0x7f5590158060 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.269+0000 7f559646d640 1 -- 172.21.15.112:0/1216300567 >> 172.21.15.112:0/1216300567 conn(0x7f55900cb310 msgr2=0x7f559006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:58.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.269+0000 7f559646d640 1 -- 172.21.15.112:0/1216300567 shutdown_connections 2024-04-25T01:40:58.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.269+0000 7f559646d640 1 -- 172.21.15.112:0/1216300567 wait complete. 2024-04-25T01:40:58.266 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:40:58.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.378+0000 7fafa6b4f640 1 Processor -- start 2024-04-25T01:40:58.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.378+0000 7fafa6b4f640 1 -- start start 2024-04-25T01:40:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7fafa6b4f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fafa00724c0 0x7fafa00c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7fafa6b4f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafa006e0f0 con 0x7fafa0072d80 2024-04-25T01:40:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7fafa6b4f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafa00c2180 con 0x7fafa01536c0 2024-04-25T01:40:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7fafa6b4f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fafa00c2460 con 0x7fafa00724c0 2024-04-25T01:40:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7fafa48eb640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fafa0072d80 0x7fafa014d340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37930/0 (socket says 172.21.15.112:37930) 2024-04-25T01:40:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7fafa48eb640 1 -- 172.21.15.112:0/2314948604 learned_addr learned my addr 172.21.15.112:0/2314948604 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:40:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7fafa50ec640 1 --2- 172.21.15.112:0/2314948604 >> v2:172.21.15.187:3300/0 conn(0x7fafa00724c0 0x7fafa00c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7faf9f7fe640 1 -- 172.21.15.112:0/2314948604 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4147906330 0 0) 0x7faf94001110 con 0x7fafa0072d80 2024-04-25T01:40:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7faf9f7fe640 1 -- 172.21.15.112:0/2314948604 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf90003520 con 0x7fafa0072d80 2024-04-25T01:40:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7faf9f7fe640 1 -- 172.21.15.112:0/2314948604 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2154280636 0 0) 0x7faf90003520 con 0x7fafa0072d80 2024-04-25T01:40:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7faf9f7fe640 1 -- 172.21.15.112:0/2314948604 >> v2:172.21.15.187:3300/0 conn(0x7fafa00724c0 msgr2=0x7fafa00c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7faf9f7fe640 1 --2- 172.21.15.112:0/2314948604 >> v2:172.21.15.187:3300/0 conn(0x7fafa00724c0 0x7fafa00c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7faf9f7fe640 1 -- 172.21.15.112:0/2314948604 >> v1:172.21.15.187:6789/0 conn(0x7fafa01536c0 legacy=0x7fafa0071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7faf9f7fe640 1 -- 172.21.15.112:0/2314948604 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa00c2740 con 0x7fafa0072d80 2024-04-25T01:40:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7faf9f7fe640 1 -- 172.21.15.112:0/2314948604 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7faf94001840 con 0x7fafa0072d80 2024-04-25T01:40:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7faf9f7fe640 1 -- 172.21.15.112:0/2314948604 >> v1:172.21.15.112:6789/0 conn(0x7fafa0072d80 legacy=0x7fafa014d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7faf9f7fe640 1 --2- 172.21.15.112:0/2314948604 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf900039b0 0x7faf90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7faf9f7fe640 1 --2- 172.21.15.112:0/2314948604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf90004800 0x7faf90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7faf9f7fe640 1 --2- 172.21.15.112:0/2314948604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf900051c0 0x7faf9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7faf9f7fe640 1 -- 172.21.15.112:0/2314948604 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fafa00c2460 con 0x7faf90004800 2024-04-25T01:40:58.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7faf9f7fe640 1 -- 172.21.15.112:0/2314948604 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faf9000c060 con 0x7faf900051c0 2024-04-25T01:40:58.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7faf9f7fe640 1 -- 172.21.15.112:0/2314948604 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faf9000c260 con 0x7faf900039b0 2024-04-25T01:40:58.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7faf9ffff640 1 --2- 172.21.15.112:0/2314948604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf90004800 0x7faf90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7fafa50ec640 1 --2- 172.21.15.112:0/2314948604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf900051c0 0x7faf9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.379+0000 7fafa48eb640 1 --2- 172.21.15.112:0/2314948604 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf900039b0 0x7faf90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.380+0000 7faf9ffff640 1 -- 172.21.15.112:0/2314948604 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf900039b0 msgr2=0x7faf90003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:58.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.380+0000 7faf9ffff640 1 --2- 172.21.15.112:0/2314948604 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf900039b0 0x7faf90003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.380+0000 7faf9ffff640 1 -- 172.21.15.112:0/2314948604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf900051c0 msgr2=0x7faf9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:40:58.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.380+0000 7faf9ffff640 1 --2- 172.21.15.112:0/2314948604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf900051c0 0x7faf9000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.380+0000 7faf9ffff640 1 -- 172.21.15.112:0/2314948604 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf90003520 con 0x7faf90004800 2024-04-25T01:40:58.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.380+0000 7fafa50ec640 1 -- 172.21.15.112:0/2314948604 reap_dead start 2024-04-25T01:40:58.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.380+0000 7faf9ffff640 1 --2- 172.21.15.112:0/2314948604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf90004800 0x7faf90004c80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faf8c001750 tx=0x7faf90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73304b7f02c3de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:58.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.380+0000 7faf9f7fe640 1 -- 172.21.15.112:0/2314948604 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c001370 con 0x7faf90004800 2024-04-25T01:40:58.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.380+0000 7faf9f7fe640 1 -- 172.21.15.112:0/2314948604 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf8c0052d0 con 0x7faf90004800 2024-04-25T01:40:58.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.380+0000 7faf9f7fe640 1 -- 172.21.15.112:0/2314948604 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c02abc0 con 0x7faf90004800 2024-04-25T01:40:58.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.381+0000 7fafa6b4f640 1 -- 172.21.15.112:0/2314948604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf90004800 msgr2=0x7faf90004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:58.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.381+0000 7fafa6b4f640 1 --2- 172.21.15.112:0/2314948604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf90004800 0x7faf90004c80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faf8c001750 tx=0x7faf90005830 comp rx=0 tx=0).stop 2024-04-25T01:40:58.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.381+0000 7fafa6b4f640 1 -- 172.21.15.112:0/2314948604 shutdown_connections 2024-04-25T01:40:58.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.381+0000 7fafa6b4f640 1 --2- 172.21.15.112:0/2314948604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf90004800 0x7faf90004c80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.381+0000 7fafa6b4f640 1 -- 172.21.15.112:0/2314948604 >> 172.21.15.112:0/2314948604 conn(0x7fafa006b160 msgr2=0x7fafa006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:58.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.381+0000 7fafa6b4f640 1 -- 172.21.15.112:0/2314948604 shutdown_connections 2024-04-25T01:40:58.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.381+0000 7fafa6b4f640 1 -- 172.21.15.112:0/2314948604 wait complete. 2024-04-25T01:40:58.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.382+0000 7fafa6b4f640 1 Processor -- start 2024-04-25T01:40:58.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.382+0000 7fafa6b4f640 1 -- start start 2024-04-25T01:40:58.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.383+0000 7fafa6b4f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf90004800 0x7fafa01ece90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.383+0000 7fafa6b4f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafa01e8f10 0x7fafa014c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.383+0000 7fafa6b4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafa014a200 0x7fafa01458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.383+0000 7fafa6b4f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fafa00728d0 con 0x7fafa014a200 2024-04-25T01:40:58.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.383+0000 7fafa6b4f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fafa0071890 con 0x7faf90004800 2024-04-25T01:40:58.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.383+0000 7fafa6b4f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fafa0073160 con 0x7fafa01e8f10 2024-04-25T01:40:58.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.383+0000 7fafa50ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafa014a200 0x7fafa01458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.383+0000 7fafa50ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafa014a200 0x7fafa01458a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34616/0 (socket says 172.21.15.112:34616) 2024-04-25T01:40:58.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.383+0000 7fafa50ec640 1 -- 172.21.15.112:0/2602509395 learned_addr learned my addr 172.21.15.112:0/2602509395 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:40:58.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.383+0000 7fafa48eb640 1 --2- 172.21.15.112:0/2602509395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf90004800 0x7fafa01ece90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.383+0000 7faf9ffff640 1 --2- 172.21.15.112:0/2602509395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafa01e8f10 0x7fafa014c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.383+0000 7fafa50ec640 1 -- 172.21.15.112:0/2602509395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafa01e8f10 msgr2=0x7fafa014c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:58.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.383+0000 7fafa50ec640 1 --2- 172.21.15.112:0/2602509395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafa01e8f10 0x7fafa014c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.383+0000 7fafa50ec640 1 -- 172.21.15.112:0/2602509395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf90004800 msgr2=0x7fafa01ece90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:58.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.383+0000 7fafa50ec640 1 --2- 172.21.15.112:0/2602509395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf90004800 0x7fafa01ece90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.383+0000 7fafa50ec640 1 -- 172.21.15.112:0/2602509395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa01dd180 con 0x7fafa014a200 2024-04-25T01:40:58.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.383+0000 7faf9ffff640 1 --2- 172.21.15.112:0/2602509395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafa01e8f10 0x7fafa014c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:40:58.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.384+0000 7fafa50ec640 1 --2- 172.21.15.112:0/2602509395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafa014a200 0x7fafa01458a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faf98011b40 tx=0x7faf9800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:58.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.384+0000 7faf9d7fa640 1 -- 172.21.15.112:0/2602509395 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9800fcb0 con 0x7fafa014a200 2024-04-25T01:40:58.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.384+0000 7faf9d7fa640 1 -- 172.21.15.112:0/2602509395 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf9800fe50 con 0x7fafa014a200 2024-04-25T01:40:58.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.384+0000 7fafa6b4f640 1 -- 172.21.15.112:0/2602509395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa01db390 con 0x7fafa014a200 2024-04-25T01:40:58.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.384+0000 7faf9d7fa640 1 -- 172.21.15.112:0/2602509395 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf98015670 con 0x7fafa014a200 2024-04-25T01:40:58.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.384+0000 7fafa6b4f640 1 -- 172.21.15.112:0/2602509395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa014cc80 con 0x7fafa014a200 2024-04-25T01:40:58.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.385+0000 7fafa6b4f640 1 -- 172.21.15.112:0/2602509395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa00682a0 con 0x7fafa014a200 2024-04-25T01:40:58.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.385+0000 7faf9d7fa640 1 -- 172.21.15.112:0/2602509395 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faf98015810 con 0x7fafa014a200 2024-04-25T01:40:58.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.386+0000 7faf9d7fa640 1 --2- 172.21.15.112:0/2602509395 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faf8406be30 0x7faf8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.386+0000 7faf9d7fa640 1 -- 172.21.15.112:0/2602509395 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7faf9808bf70 con 0x7fafa014a200 2024-04-25T01:40:58.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.386+0000 7fafa48eb640 1 --2- 172.21.15.112:0/2602509395 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faf8406be30 0x7faf8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.386+0000 7fafa48eb640 1 --2- 172.21.15.112:0/2602509395 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faf8406be30 0x7faf8406e2f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7faf94007fd0 tx=0x7faf94007a70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:58.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.390+0000 7faf9d7fa640 1 -- 172.21.15.112:0/2602509395 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fafa01db390 con 0x7fafa014a200 2024-04-25T01:40:58.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.578+0000 7fafa6b4f640 1 -- 172.21.15.112:0/2602509395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fafa01e92f0 con 0x7fafa014a200 2024-04-25T01:40:58.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.579+0000 7faf9d7fa640 1 -- 172.21.15.112:0/2602509395 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7faf98019030 con 0x7fafa014a200 2024-04-25T01:40:58.565 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-04-25T01:40:58.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.581+0000 7fafa6b4f640 1 -- 172.21.15.112:0/2602509395 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faf8406be30 msgr2=0x7faf8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:58.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.581+0000 7fafa6b4f640 1 --2- 172.21.15.112:0/2602509395 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faf8406be30 0x7faf8406e2f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7faf94007fd0 tx=0x7faf94007a70 comp rx=0 tx=0).stop 2024-04-25T01:40:58.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.581+0000 7fafa6b4f640 1 -- 172.21.15.112:0/2602509395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafa014a200 msgr2=0x7fafa01458a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:58.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.581+0000 7fafa6b4f640 1 --2- 172.21.15.112:0/2602509395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafa014a200 0x7fafa01458a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faf98011b40 tx=0x7faf9800f1b0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.581+0000 7fafa6b4f640 1 -- 172.21.15.112:0/2602509395 shutdown_connections 2024-04-25T01:40:58.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.581+0000 7fafa6b4f640 1 --2- 172.21.15.112:0/2602509395 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7faf8406be30 0x7faf8406e2f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.581+0000 7fafa6b4f640 1 --2- 172.21.15.112:0/2602509395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafa014a200 0x7fafa01458a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.581+0000 7fafa6b4f640 1 --2- 172.21.15.112:0/2602509395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafa01e8f10 0x7fafa014c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.581+0000 7fafa6b4f640 1 --2- 172.21.15.112:0/2602509395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf90004800 0x7fafa01ece90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.581+0000 7fafa6b4f640 1 -- 172.21.15.112:0/2602509395 >> 172.21.15.112:0/2602509395 conn(0x7fafa006b160 msgr2=0x7fafa01526d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:58.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.581+0000 7fafa6b4f640 1 -- 172.21.15.112:0/2602509395 shutdown_connections 2024-04-25T01:40:58.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:40:58.581+0000 7fafa6b4f640 1 -- 172.21.15.112:0/2602509395 wait complete. 2024-04-25T01:40:58.578 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi112.front.sepia.ceph.com... 2024-04-25T01:40:58.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:40:58.579 DEBUG:teuthology.orchestra.run.smithi112:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T01:40:58.620 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:40:58.622 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi112.stderr:ceph-fuse[45496]: fuse finished with error 0 and tester_r 0 2024-04-25T01:41:00.049 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:41:00.052 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:41:04.722 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T01:41:04.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:41:04.722 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T01:41:04.741 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:41:04.741 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:41:04.778 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:41:04.778 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:41:04.837 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:41:04.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.956+0000 7f2aab3e7640 1 Processor -- start 2024-04-25T01:41:04.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.957+0000 7f2aab3e7640 1 -- start start 2024-04-25T01:41:04.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.957+0000 7f2aab3e7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2aa41596a0 0x7f2aa4152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:04.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.957+0000 7f2aab3e7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aa4159b90 con 0x7f2aa41592c0 2024-04-25T01:41:04.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.957+0000 7f2aab3e7640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aa406c970 con 0x7f2aa40c72b0 2024-04-25T01:41:04.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.957+0000 7f2aab3e7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2aa40c6e20 con 0x7f2aa41596a0 2024-04-25T01:41:04.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2aa9183640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2aa41592c0 0x7f2aa40c9070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37934/0 (socket says 172.21.15.112:37934) 2024-04-25T01:41:04.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2aa9183640 1 -- 172.21.15.112:0/1266898321 learned_addr learned my addr 172.21.15.112:0/1266898321 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:04.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2aa8982640 1 --2- 172.21.15.112:0/1266898321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2aa41596a0 0x7f2aa4152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:04.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2a9bfff640 1 -- 172.21.15.112:0/1266898321 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1724567887 0 0) 0x7f2aa4159b90 con 0x7f2aa41592c0 2024-04-25T01:41:04.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2a9bfff640 1 -- 172.21.15.112:0/1266898321 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a90003520 con 0x7f2aa41592c0 2024-04-25T01:41:04.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2a9bfff640 1 -- 172.21.15.112:0/1266898321 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 865979772 0 0) 0x7f2a90003520 con 0x7f2aa41592c0 2024-04-25T01:41:04.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2a9bfff640 1 -- 172.21.15.112:0/1266898321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2aa41596a0 msgr2=0x7f2aa4152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:04.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2a9bfff640 1 --2- 172.21.15.112:0/1266898321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2aa41596a0 0x7f2aa4152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:04.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2a9bfff640 1 -- 172.21.15.112:0/1266898321 >> v1:172.21.15.187:6789/0 conn(0x7f2aa40c72b0 legacy=0x7f2aa4160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:04.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2a9bfff640 1 -- 172.21.15.112:0/1266898321 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa406cc50 con 0x7f2aa41592c0 2024-04-25T01:41:04.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2a9bfff640 1 -- 172.21.15.112:0/1266898321 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f2aa0001690 con 0x7f2aa41592c0 2024-04-25T01:41:04.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2a9bfff640 1 -- 172.21.15.112:0/1266898321 >> v1:172.21.15.112:6789/0 conn(0x7f2aa41592c0 legacy=0x7f2aa40c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:04.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2a9bfff640 1 --2- 172.21.15.112:0/1266898321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a900039e0 0x7f2a90003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:04.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2a9bfff640 1 --2- 172.21.15.112:0/1266898321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2aa41596a0 0x7f2a90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:04.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2a9bfff640 1 --2- 172.21.15.112:0/1266898321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a90004d40 0x7f2a900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:04.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2a9bfff640 1 -- 172.21.15.112:0/1266898321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9000bb50 con 0x7f2a90004d40 2024-04-25T01:41:04.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2a9bfff640 1 -- 172.21.15.112:0/1266898321 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9000bcd0 con 0x7f2a900039e0 2024-04-25T01:41:04.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2a9bfff640 1 -- 172.21.15.112:0/1266898321 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2a9000bea0 con 0x7f2aa41596a0 2024-04-25T01:41:04.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2aa9984640 1 --2- 172.21.15.112:0/1266898321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a90004d40 0x7f2a900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:04.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2aa8982640 1 --2- 172.21.15.112:0/1266898321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2aa41596a0 0x7f2a90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:04.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.958+0000 7f2aa9183640 1 --2- 172.21.15.112:0/1266898321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a900039e0 0x7f2a90003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:04.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.959+0000 7f2aa9984640 1 -- 172.21.15.112:0/1266898321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2aa41596a0 msgr2=0x7f2a90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:04.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.959+0000 7f2aa9984640 1 --2- 172.21.15.112:0/1266898321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2aa41596a0 0x7f2a90004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:04.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.959+0000 7f2aa9984640 1 -- 172.21.15.112:0/1266898321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a900039e0 msgr2=0x7f2a90003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:04.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.959+0000 7f2aa9984640 1 --2- 172.21.15.112:0/1266898321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a900039e0 0x7f2a90003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:04.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.959+0000 7f2aa9984640 1 -- 172.21.15.112:0/1266898321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a90003520 con 0x7f2a90004d40 2024-04-25T01:41:04.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.959+0000 7f2aa9984640 1 --2- 172.21.15.112:0/1266898321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a90004d40 0x7f2a900051c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c001f90 tx=0x7f2a90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c439d57452b95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:04.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.959+0000 7f2a9bfff640 1 -- 172.21.15.112:0/1266898321 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c001b30 con 0x7f2a90004d40 2024-04-25T01:41:04.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.959+0000 7f2a9bfff640 1 -- 172.21.15.112:0/1266898321 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a9c031620 con 0x7f2a90004d40 2024-04-25T01:41:04.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.959+0000 7f2a9bfff640 1 -- 172.21.15.112:0/1266898321 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c031a10 con 0x7f2a90004d40 2024-04-25T01:41:04.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.960+0000 7f2aab3e7640 1 -- 172.21.15.112:0/1266898321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a90004d40 msgr2=0x7f2a900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:04.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.960+0000 7f2aab3e7640 1 --2- 172.21.15.112:0/1266898321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a90004d40 0x7f2a900051c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c001f90 tx=0x7f2a90005830 comp rx=0 tx=0).stop 2024-04-25T01:41:04.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.960+0000 7f2aa9984640 1 -- 172.21.15.112:0/1266898321 reap_dead start 2024-04-25T01:41:04.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.960+0000 7f2aab3e7640 1 -- 172.21.15.112:0/1266898321 shutdown_connections 2024-04-25T01:41:04.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.960+0000 7f2aab3e7640 1 -- 172.21.15.112:0/1266898321 >> 172.21.15.112:0/1266898321 conn(0x7f2aa40cb170 msgr2=0x7f2aa40cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:04.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.960+0000 7f2aab3e7640 1 -- 172.21.15.112:0/1266898321 shutdown_connections 2024-04-25T01:41:04.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.960+0000 7f2aab3e7640 1 -- 172.21.15.112:0/1266898321 wait complete. 2024-04-25T01:41:04.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.961+0000 7f2aab3e7640 1 Processor -- start 2024-04-25T01:41:04.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.961+0000 7f2aab3e7640 1 -- start start 2024-04-25T01:41:04.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.962+0000 7f2aab3e7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a90004d40 0x7f2aa40aad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:04.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.962+0000 7f2aab3e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa41ec4d0 0x7f2aa4146230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:04.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.962+0000 7f2aab3e7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2aa4146770 0x7f2aa414b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:04.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.962+0000 7f2aab3e7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa406c510 con 0x7f2aa41ec4d0 2024-04-25T01:41:04.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.962+0000 7f2aab3e7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa406c390 con 0x7f2aa4146770 2024-04-25T01:41:04.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.962+0000 7f2aab3e7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2aa406db20 con 0x7f2a90004d40 2024-04-25T01:41:04.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.962+0000 7f2aa8982640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa41ec4d0 0x7f2aa4146230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:04.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.962+0000 7f2aa8982640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa41ec4d0 0x7f2aa4146230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34642/0 (socket says 172.21.15.112:34642) 2024-04-25T01:41:04.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.962+0000 7f2aa8982640 1 -- 172.21.15.112:0/2635185196 learned_addr learned my addr 172.21.15.112:0/2635185196 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:04.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.962+0000 7f2aa9984640 1 --2- 172.21.15.112:0/2635185196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2aa4146770 0x7f2aa414b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:04.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.962+0000 7f2aa9183640 1 --2- 172.21.15.112:0/2635185196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a90004d40 0x7f2aa40aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:04.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.962+0000 7f2aa8982640 1 -- 172.21.15.112:0/2635185196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a90004d40 msgr2=0x7f2aa40aad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:04.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.962+0000 7f2aa8982640 1 --2- 172.21.15.112:0/2635185196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a90004d40 0x7f2aa40aad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:04.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.962+0000 7f2aa8982640 1 -- 172.21.15.112:0/2635185196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2aa4146770 msgr2=0x7f2aa414b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:04.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.962+0000 7f2aa8982640 1 --2- 172.21.15.112:0/2635185196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2aa4146770 0x7f2aa414b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:04.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.962+0000 7f2aa8982640 1 -- 172.21.15.112:0/2635185196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa40ab860 con 0x7f2aa41ec4d0 2024-04-25T01:41:04.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.962+0000 7f2aa9183640 1 --2- 172.21.15.112:0/2635185196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a90004d40 0x7f2aa40aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:41:04.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.962+0000 7f2aa8982640 1 --2- 172.21.15.112:0/2635185196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa41ec4d0 0x7f2aa4146230 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2a94011910 tx=0x7f2a94011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:04.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.963+0000 7f2a99ffb640 1 -- 172.21.15.112:0/2635185196 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a94019070 con 0x7f2aa41ec4d0 2024-04-25T01:41:04.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.963+0000 7f2a99ffb640 1 -- 172.21.15.112:0/2635185196 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a94004240 con 0x7f2aa41ec4d0 2024-04-25T01:41:04.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.963+0000 7f2a99ffb640 1 -- 172.21.15.112:0/2635185196 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9400f7c0 con 0x7f2aa41ec4d0 2024-04-25T01:41:04.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.963+0000 7f2aab3e7640 1 -- 172.21.15.112:0/2635185196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa40abb40 con 0x7f2aa41ec4d0 2024-04-25T01:41:04.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.963+0000 7f2aab3e7640 1 -- 172.21.15.112:0/2635185196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa4146f80 con 0x7f2aa41ec4d0 2024-04-25T01:41:04.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.964+0000 7f2aab3e7640 1 -- 172.21.15.112:0/2635185196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aa40679a0 con 0x7f2aa41ec4d0 2024-04-25T01:41:04.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.964+0000 7f2a99ffb640 1 -- 172.21.15.112:0/2635185196 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a9401c070 con 0x7f2aa41ec4d0 2024-04-25T01:41:04.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.964+0000 7f2a99ffb640 1 --2- 172.21.15.112:0/2635185196 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2a8806be30 0x7f2a8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:04.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.965+0000 7f2a99ffb640 1 -- 172.21.15.112:0/2635185196 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f2a9408b520 con 0x7f2aa41ec4d0 2024-04-25T01:41:04.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.965+0000 7f2aa9183640 1 --2- 172.21.15.112:0/2635185196 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2a8806be30 0x7f2a8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:04.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.965+0000 7f2aa9183640 1 --2- 172.21.15.112:0/2635185196 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2a8806be30 0x7f2a8806e2f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4159d60 tx=0x7f2aa0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:04.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:04.969+0000 7f2a99ffb640 1 -- 172.21.15.112:0/2635185196 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2a94059570 con 0x7f2aa41ec4d0 2024-04-25T01:41:05.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.145+0000 7f2aab3e7640 1 -- 172.21.15.112:0/2635185196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2aa40682a0 con 0x7f2aa41ec4d0 2024-04-25T01:41:05.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.146+0000 7f2a99ffb640 1 -- 172.21.15.112:0/2635185196 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+20616 (secure 0 0 0) 0x7f2a9405d870 con 0x7f2aa41ec4d0 2024-04-25T01:41:05.132 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:41:05.132 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:26:52.110548+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-25T01:25:59.478275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-25T01:26:00.719358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:41:05.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.148+0000 7f2aab3e7640 1 -- 172.21.15.112:0/2635185196 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2a8806be30 msgr2=0x7f2a8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:05.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.148+0000 7f2aab3e7640 1 --2- 172.21.15.112:0/2635185196 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2a8806be30 0x7f2a8806e2f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4159d60 tx=0x7f2aa0009040 comp rx=0 tx=0).stop 2024-04-25T01:41:05.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.148+0000 7f2aab3e7640 1 -- 172.21.15.112:0/2635185196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa41ec4d0 msgr2=0x7f2aa4146230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:05.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.148+0000 7f2aab3e7640 1 --2- 172.21.15.112:0/2635185196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa41ec4d0 0x7f2aa4146230 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2a94011910 tx=0x7f2a94011dd0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.148+0000 7f2aab3e7640 1 -- 172.21.15.112:0/2635185196 shutdown_connections 2024-04-25T01:41:05.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.148+0000 7f2aab3e7640 1 --2- 172.21.15.112:0/2635185196 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2a8806be30 0x7f2a8806e2f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.148+0000 7f2aab3e7640 1 --2- 172.21.15.112:0/2635185196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2aa4146770 0x7f2aa414b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.148+0000 7f2aab3e7640 1 --2- 172.21.15.112:0/2635185196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa41ec4d0 0x7f2aa4146230 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.148+0000 7f2aab3e7640 1 --2- 172.21.15.112:0/2635185196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a90004d40 0x7f2aa40aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.148+0000 7f2aab3e7640 1 -- 172.21.15.112:0/2635185196 >> 172.21.15.112:0/2635185196 conn(0x7f2aa40cb170 msgr2=0x7f2aa40cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:05.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.148+0000 7f2aab3e7640 1 -- 172.21.15.112:0/2635185196 shutdown_connections 2024-04-25T01:41:05.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.148+0000 7f2aab3e7640 1 -- 172.21.15.112:0/2635185196 wait complete. 2024-04-25T01:41:05.145 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:25:40.679119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-04-25T01:25:50.593045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-04-25T01:25:51.664930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-04-25T01:25:59.478275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-04-25T01:26:00.719358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:41:05.146 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:41:05.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.259+0000 7f6940963640 1 Processor -- start 2024-04-25T01:41:05.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.259+0000 7f6940963640 1 -- start start 2024-04-25T01:41:05.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.260+0000 7f6940963640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f693c06ebb0 0x7f693c071200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.260+0000 7f6940963640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f693c06cf50 0x7f693c13dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.260+0000 7f6940963640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f693c158fe0 con 0x7f693c13e1b0 2024-04-25T01:41:05.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.260+0000 7f6940963640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f693c158d90 con 0x7f693c06cf50 2024-04-25T01:41:05.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.260+0000 7f6940963640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f693c159bf0 con 0x7f693c06ebb0 2024-04-25T01:41:05.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.260+0000 7f6933fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f693c06cf50 0x7f693c13dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:05.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.260+0000 7f693ad9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f693c13e1b0 0x7f693c1559c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37950/0 (socket says 172.21.15.112:37950) 2024-04-25T01:41:05.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.260+0000 7f693ad9d640 1 -- 172.21.15.112:0/1385678368 learned_addr learned my addr 172.21.15.112:0/1385678368 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:05.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.260+0000 7f6933fff640 1 -- 172.21.15.112:0/1385678368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f693c06ebb0 msgr2=0x7f693c071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:05.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.260+0000 7f693a59c640 1 --2- 172.21.15.112:0/1385678368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f693c06ebb0 0x7f693c071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:05.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.260+0000 7f6939d9b640 1 -- 172.21.15.112:0/1385678368 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 162559563 0 0) 0x7f693c158fe0 con 0x7f693c13e1b0 2024-04-25T01:41:05.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.260+0000 7f6933fff640 1 --2- 172.21.15.112:0/1385678368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f693c06ebb0 0x7f693c071200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.260+0000 7f6933fff640 1 -- 172.21.15.112:0/1385678368 >> v1:172.21.15.112:6789/0 conn(0x7f693c13e1b0 legacy=0x7f693c1559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:05.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.260+0000 7f6933fff640 1 -- 172.21.15.112:0/1385678368 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c1592c0 con 0x7f693c06cf50 2024-04-25T01:41:05.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.261+0000 7f6933fff640 1 --2- 172.21.15.112:0/1385678368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f693c06cf50 0x7f693c13dc70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f69240097a0 tx=0x7f692402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253324806c76f81a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:05.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.261+0000 7f6939d9b640 1 -- 172.21.15.112:0/1385678368 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924037070 con 0x7f693c06cf50 2024-04-25T01:41:05.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.261+0000 7f6939d9b640 1 -- 172.21.15.112:0/1385678368 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f692402ae50 con 0x7f693c06cf50 2024-04-25T01:41:05.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.261+0000 7f6939d9b640 1 -- 172.21.15.112:0/1385678368 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924032660 con 0x7f693c06cf50 2024-04-25T01:41:05.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.261+0000 7f6940963640 1 -- 172.21.15.112:0/1385678368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f693c06cf50 msgr2=0x7f693c13dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:05.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.261+0000 7f6940963640 1 --2- 172.21.15.112:0/1385678368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f693c06cf50 0x7f693c13dc70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f69240097a0 tx=0x7f692402a100 comp rx=0 tx=0).stop 2024-04-25T01:41:05.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.261+0000 7f6940963640 1 -- 172.21.15.112:0/1385678368 shutdown_connections 2024-04-25T01:41:05.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.261+0000 7f6940963640 1 --2- 172.21.15.112:0/1385678368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f693c06cf50 0x7f693c13dc70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.261+0000 7f6940963640 1 --2- 172.21.15.112:0/1385678368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f693c06ebb0 0x7f693c071200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.261+0000 7f6940963640 1 -- 172.21.15.112:0/1385678368 >> 172.21.15.112:0/1385678368 conn(0x7f693c0c3a10 msgr2=0x7f693c0c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:05.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.261+0000 7f6940963640 1 -- 172.21.15.112:0/1385678368 shutdown_connections 2024-04-25T01:41:05.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.261+0000 7f6940963640 1 -- 172.21.15.112:0/1385678368 wait complete. 2024-04-25T01:41:05.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.263+0000 7f6940963640 1 Processor -- start 2024-04-25T01:41:05.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.263+0000 7f6940963640 1 -- start start 2024-04-25T01:41:05.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.263+0000 7f6940963640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f693c06cf50 0x7f693c157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.263+0000 7f6940963640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f693c06ebb0 0x7f693c158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.263+0000 7f6940963640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f693c13e1b0 0x7f693c0c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.263+0000 7f6940963640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f693c06ef90 con 0x7f693c06ebb0 2024-04-25T01:41:05.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.263+0000 7f6940963640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f693c159880 con 0x7f693c13e1b0 2024-04-25T01:41:05.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.263+0000 7f6940963640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f693c06f110 con 0x7f693c06cf50 2024-04-25T01:41:05.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.263+0000 7f6933fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f693c06ebb0 0x7f693c158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:05.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.263+0000 7f6933fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f693c06ebb0 0x7f693c158480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34660/0 (socket says 172.21.15.112:34660) 2024-04-25T01:41:05.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.263+0000 7f6933fff640 1 -- 172.21.15.112:0/149582184 learned_addr learned my addr 172.21.15.112:0/149582184 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:05.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.264+0000 7f6933fff640 1 -- 172.21.15.112:0/149582184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f693c06cf50 msgr2=0x7f693c157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:05.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.264+0000 7f6933fff640 1 --2- 172.21.15.112:0/149582184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f693c06cf50 0x7f693c157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.264+0000 7f6933fff640 1 -- 172.21.15.112:0/149582184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f693c13e1b0 msgr2=0x7f693c0c4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:05.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.264+0000 7f6933fff640 1 --2- 172.21.15.112:0/149582184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f693c13e1b0 0x7f693c0c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.264+0000 7f6933fff640 1 -- 172.21.15.112:0/149582184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c1ec3d0 con 0x7f693c06ebb0 2024-04-25T01:41:05.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.264+0000 7f6933fff640 1 --2- 172.21.15.112:0/149582184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f693c06ebb0 0x7f693c158480 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f69240098c0 tx=0x7f69240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:05.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.264+0000 7f6932ffd640 1 -- 172.21.15.112:0/149582184 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924041030 con 0x7f693c06ebb0 2024-04-25T01:41:05.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.264+0000 7f6932ffd640 1 -- 172.21.15.112:0/149582184 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f692402ac50 con 0x7f693c06ebb0 2024-04-25T01:41:05.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.264+0000 7f6940963640 1 -- 172.21.15.112:0/149582184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693c1ec6b0 con 0x7f693c06ebb0 2024-04-25T01:41:05.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.264+0000 7f6932ffd640 1 -- 172.21.15.112:0/149582184 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924037040 con 0x7f693c06ebb0 2024-04-25T01:41:05.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.264+0000 7f6940963640 1 -- 172.21.15.112:0/149582184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693c139020 con 0x7f693c06ebb0 2024-04-25T01:41:05.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.265+0000 7f6940963640 1 -- 172.21.15.112:0/149582184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f693c04f500 con 0x7f693c06ebb0 2024-04-25T01:41:05.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.265+0000 7f6932ffd640 1 -- 172.21.15.112:0/149582184 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6924044070 con 0x7f693c06ebb0 2024-04-25T01:41:05.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.266+0000 7f6932ffd640 1 --2- 172.21.15.112:0/149582184 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f691c06be30 0x7f691c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.266+0000 7f6932ffd640 1 -- 172.21.15.112:0/149582184 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f69240ab830 con 0x7f693c06ebb0 2024-04-25T01:41:05.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.266+0000 7f693a59c640 1 --2- 172.21.15.112:0/149582184 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f691c06be30 0x7f691c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:05.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.266+0000 7f693a59c640 1 --2- 172.21.15.112:0/149582184 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f691c06be30 0x7f691c06e2f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f692c006fd0 tx=0x7f692c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:05.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.270+0000 7f6932ffd640 1 -- 172.21.15.112:0/149582184 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6924044e40 con 0x7f693c06ebb0 2024-04-25T01:41:05.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.449+0000 7f6940963640 1 -- 172.21.15.112:0/149582184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f693c13df40 con 0x7f693c06ebb0 2024-04-25T01:41:05.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.449+0000 7f6932ffd640 1 -- 172.21.15.112:0/149582184 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f6924079ed0 con 0x7f693c06ebb0 2024-04-25T01:41:05.435 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T01:41:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.451+0000 7f6940963640 1 -- 172.21.15.112:0/149582184 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f691c06be30 msgr2=0x7f691c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:05.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.451+0000 7f6940963640 1 --2- 172.21.15.112:0/149582184 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f691c06be30 0x7f691c06e2f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f692c006fd0 tx=0x7f692c008040 comp rx=0 tx=0).stop 2024-04-25T01:41:05.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.451+0000 7f6940963640 1 -- 172.21.15.112:0/149582184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f693c06ebb0 msgr2=0x7f693c158480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:05.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.451+0000 7f6940963640 1 --2- 172.21.15.112:0/149582184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f693c06ebb0 0x7f693c158480 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f69240098c0 tx=0x7f69240046c0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.451+0000 7f6940963640 1 -- 172.21.15.112:0/149582184 shutdown_connections 2024-04-25T01:41:05.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.451+0000 7f6940963640 1 --2- 172.21.15.112:0/149582184 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f691c06be30 0x7f691c06e2f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.451+0000 7f6940963640 1 --2- 172.21.15.112:0/149582184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f693c13e1b0 0x7f693c0c4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.451+0000 7f6940963640 1 --2- 172.21.15.112:0/149582184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f693c06ebb0 0x7f693c158480 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.451+0000 7f6940963640 1 --2- 172.21.15.112:0/149582184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f693c06cf50 0x7f693c157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.451+0000 7f6940963640 1 -- 172.21.15.112:0/149582184 >> 172.21.15.112:0/149582184 conn(0x7f693c0c3a10 msgr2=0x7f693c072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:05.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.451+0000 7f6940963640 1 -- 172.21.15.112:0/149582184 shutdown_connections 2024-04-25T01:41:05.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.451+0000 7f6940963640 1 -- 172.21.15.112:0/149582184 wait complete. 2024-04-25T01:41:05.448 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-04-25T01:41:05.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.563+0000 7fe111dd5640 1 Processor -- start 2024-04-25T01:41:05.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.563+0000 7fe111dd5640 1 -- start start 2024-04-25T01:41:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.564+0000 7fe111dd5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe10c06d460 0x7fe10c071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.564+0000 7fe111dd5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe10c06f290 con 0x7fe10c14fbe0 2024-04-25T01:41:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.564+0000 7fe111dd5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe10c06efb0 con 0x7fe10c0724f0 2024-04-25T01:41:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.564+0000 7fe111dd5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe10c14ffc0 con 0x7fe10c06d460 2024-04-25T01:41:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.564+0000 7fe10b7fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe10c14fbe0 0x7fe10c0c2920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37952/0 (socket says 172.21.15.112:37952) 2024-04-25T01:41:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.564+0000 7fe10b7fe640 1 -- 172.21.15.112:0/3658311336 learned_addr learned my addr 172.21.15.112:0/3658311336 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:05.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.564+0000 7fe10affd640 1 --2- 172.21.15.112:0/3658311336 >> v2:172.21.15.187:3300/0 conn(0x7fe10c06d460 0x7fe10c071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:05.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.564+0000 7fe10a7fc640 1 -- 172.21.15.112:0/3658311336 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1424356859 0 0) 0x7fe10c06f290 con 0x7fe10c14fbe0 2024-04-25T01:41:05.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.564+0000 7fe10a7fc640 1 -- 172.21.15.112:0/3658311336 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0f4003520 con 0x7fe10c14fbe0 2024-04-25T01:41:05.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10a7fc640 1 -- 172.21.15.112:0/3658311336 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1293758584 0 0) 0x7fe0f4003520 con 0x7fe10c14fbe0 2024-04-25T01:41:05.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10a7fc640 1 -- 172.21.15.112:0/3658311336 >> v2:172.21.15.187:3300/0 conn(0x7fe10c06d460 msgr2=0x7fe10c071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:05.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10a7fc640 1 --2- 172.21.15.112:0/3658311336 >> v2:172.21.15.187:3300/0 conn(0x7fe10c06d460 0x7fe10c071fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10a7fc640 1 -- 172.21.15.112:0/3658311336 >> v1:172.21.15.187:6789/0 conn(0x7fe10c0724f0 legacy=0x7fe10c15ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:05.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10a7fc640 1 -- 172.21.15.112:0/3658311336 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10c0a9530 con 0x7fe10c14fbe0 2024-04-25T01:41:05.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10a7fc640 1 -- 172.21.15.112:0/3658311336 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fe100001670 con 0x7fe10c14fbe0 2024-04-25T01:41:05.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10a7fc640 1 -- 172.21.15.112:0/3658311336 >> v1:172.21.15.112:6789/0 conn(0x7fe10c14fbe0 legacy=0x7fe10c0c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:05.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10a7fc640 1 --2- 172.21.15.112:0/3658311336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0f40039b0 0x7fe0f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10a7fc640 1 --2- 172.21.15.112:0/3658311336 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0f4004800 0x7fe0f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10a7fc640 1 --2- 172.21.15.112:0/3658311336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0f40051c0 0x7fe0f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10a7fc640 1 -- 172.21.15.112:0/3658311336 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f400c130 con 0x7fe0f40039b0 2024-04-25T01:41:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10a7fc640 1 -- 172.21.15.112:0/3658311336 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f400c2e0 con 0x7fe0f40051c0 2024-04-25T01:41:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10a7fc640 1 -- 172.21.15.112:0/3658311336 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe0f400c4e0 con 0x7fe0f4004800 2024-04-25T01:41:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10b7fe640 1 --2- 172.21.15.112:0/3658311336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0f40039b0 0x7fe0f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10bfff640 1 --2- 172.21.15.112:0/3658311336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0f40051c0 0x7fe0f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10affd640 1 --2- 172.21.15.112:0/3658311336 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0f4004800 0x7fe0f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10b7fe640 1 -- 172.21.15.112:0/3658311336 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0f4004800 msgr2=0x7fe0f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10b7fe640 1 --2- 172.21.15.112:0/3658311336 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0f4004800 0x7fe0f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10b7fe640 1 -- 172.21.15.112:0/3658311336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0f40051c0 msgr2=0x7fe0f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10b7fe640 1 --2- 172.21.15.112:0/3658311336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0f40051c0 0x7fe0f400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10b7fe640 1 -- 172.21.15.112:0/3658311336 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f4003520 con 0x7fe0f40039b0 2024-04-25T01:41:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.565+0000 7fe10bfff640 1 -- 172.21.15.112:0/3658311336 reap_dead start 2024-04-25T01:41:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.566+0000 7fe10b7fe640 1 --2- 172.21.15.112:0/3658311336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0f40039b0 0x7fe0f4003dd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe10000c240 tx=0x7fe0f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbb786a9acff222 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:05.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.566+0000 7fe10a7fc640 1 -- 172.21.15.112:0/3658311336 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe100007cf0 con 0x7fe0f40039b0 2024-04-25T01:41:05.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.566+0000 7fe10a7fc640 1 -- 172.21.15.112:0/3658311336 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1000058d0 con 0x7fe0f40039b0 2024-04-25T01:41:05.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.566+0000 7fe10a7fc640 1 -- 172.21.15.112:0/3658311336 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe100005d20 con 0x7fe0f40039b0 2024-04-25T01:41:05.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.566+0000 7fe111dd5640 1 -- 172.21.15.112:0/3658311336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0f40039b0 msgr2=0x7fe0f4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:05.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.566+0000 7fe111dd5640 1 --2- 172.21.15.112:0/3658311336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0f40039b0 0x7fe0f4003dd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe10000c240 tx=0x7fe0f4005830 comp rx=0 tx=0).stop 2024-04-25T01:41:05.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.566+0000 7fe111dd5640 1 -- 172.21.15.112:0/3658311336 shutdown_connections 2024-04-25T01:41:05.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.566+0000 7fe111dd5640 1 --2- 172.21.15.112:0/3658311336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0f40039b0 0x7fe0f4003dd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.566+0000 7fe111dd5640 1 -- 172.21.15.112:0/3658311336 >> 172.21.15.112:0/3658311336 conn(0x7fe10c0c3a10 msgr2=0x7fe10c0c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:05.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.566+0000 7fe111dd5640 1 -- 172.21.15.112:0/3658311336 shutdown_connections 2024-04-25T01:41:05.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.566+0000 7fe111dd5640 1 -- 172.21.15.112:0/3658311336 wait complete. 2024-04-25T01:41:05.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.567+0000 7fe111dd5640 1 Processor -- start 2024-04-25T01:41:05.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.568+0000 7fe111dd5640 1 -- start start 2024-04-25T01:41:05.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.568+0000 7fe111dd5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0f40039b0 0x7fe10c0abc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.568+0000 7fe111dd5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe10c0c6870 0x7fe10c1422b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.568+0000 7fe111dd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10c1ef4a0 0x7fe10c1ebd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.568+0000 7fe111dd5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c0c3700 con 0x7fe10c1ef4a0 2024-04-25T01:41:05.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.568+0000 7fe111dd5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c1533a0 con 0x7fe0f40039b0 2024-04-25T01:41:05.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.568+0000 7fe111dd5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe10c1427f0 con 0x7fe10c0c6870 2024-04-25T01:41:05.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.568+0000 7fe10bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10c1ef4a0 0x7fe10c1ebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:05.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.568+0000 7fe10bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10c1ef4a0 0x7fe10c1ebd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34674/0 (socket says 172.21.15.112:34674) 2024-04-25T01:41:05.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.568+0000 7fe10bfff640 1 -- 172.21.15.112:0/2993469006 learned_addr learned my addr 172.21.15.112:0/2993469006 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:05.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.569+0000 7fe10affd640 1 --2- 172.21.15.112:0/2993469006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe10c0c6870 0x7fe10c1422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:05.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.569+0000 7fe10b7fe640 1 --2- 172.21.15.112:0/2993469006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0f40039b0 0x7fe10c0abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:05.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.569+0000 7fe10bfff640 1 -- 172.21.15.112:0/2993469006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe10c0c6870 msgr2=0x7fe10c1422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:05.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.569+0000 7fe10bfff640 1 --2- 172.21.15.112:0/2993469006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe10c0c6870 0x7fe10c1422b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.569+0000 7fe10bfff640 1 -- 172.21.15.112:0/2993469006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0f40039b0 msgr2=0x7fe10c0abc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:05.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.569+0000 7fe10bfff640 1 --2- 172.21.15.112:0/2993469006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0f40039b0 0x7fe10c0abc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.569+0000 7fe10bfff640 1 -- 172.21.15.112:0/2993469006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10c0c5c60 con 0x7fe10c1ef4a0 2024-04-25T01:41:05.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.569+0000 7fe10bfff640 1 --2- 172.21.15.112:0/2993469006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10c1ef4a0 0x7fe10c1ebd00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe10c06bfd0 tx=0x7fe10400e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:05.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.569+0000 7fe0f3fff640 1 -- 172.21.15.112:0/2993469006 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104019070 con 0x7fe10c1ef4a0 2024-04-25T01:41:05.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.569+0000 7fe0f3fff640 1 -- 172.21.15.112:0/2993469006 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe104006e20 con 0x7fe10c1ef4a0 2024-04-25T01:41:05.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.569+0000 7fe111dd5640 1 -- 172.21.15.112:0/2993469006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe10c1eb520 con 0x7fe10c1ef4a0 2024-04-25T01:41:05.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.569+0000 7fe0f3fff640 1 -- 172.21.15.112:0/2993469006 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104003b90 con 0x7fe10c1ef4a0 2024-04-25T01:41:05.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.569+0000 7fe111dd5640 1 -- 172.21.15.112:0/2993469006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe10c1eb7e0 con 0x7fe10c1ef4a0 2024-04-25T01:41:05.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.570+0000 7fe111dd5640 1 -- 172.21.15.112:0/2993469006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe10c0c13d0 con 0x7fe10c1ef4a0 2024-04-25T01:41:05.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.571+0000 7fe0f3fff640 1 -- 172.21.15.112:0/2993469006 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe10401c070 con 0x7fe10c1ef4a0 2024-04-25T01:41:05.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.571+0000 7fe0f3fff640 1 --2- 172.21.15.112:0/2993469006 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe0f806be30 0x7fe0f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.571+0000 7fe0f3fff640 1 -- 172.21.15.112:0/2993469006 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fe10408a560 con 0x7fe10c1ef4a0 2024-04-25T01:41:05.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.571+0000 7fe10b7fe640 1 --2- 172.21.15.112:0/2993469006 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe0f806be30 0x7fe0f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:05.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.571+0000 7fe10b7fe640 1 --2- 172.21.15.112:0/2993469006 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe0f806be30 0x7fe0f806e2f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe100033000 tx=0x7fe100009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:05.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.575+0000 7fe0f3fff640 1 -- 172.21.15.112:0/2993469006 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe10c0c13d0 con 0x7fe10c1ef4a0 2024-04-25T01:41:05.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.751+0000 7fe111dd5640 1 -- 172.21.15.112:0/2993469006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) -- 0x7fe10c04f500 con 0x7fe10c1ef4a0 2024-04-25T01:41:05.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.751+0000 7fe0f3fff640 1 -- 172.21.15.112:0/2993469006 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v26) ==== 99+0+11 (secure 0 0 0) 0x7fe104058c00 con 0x7fe10c1ef4a0 2024-04-25T01:41:05.737 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T01:41:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.753+0000 7fe111dd5640 1 -- 172.21.15.112:0/2993469006 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe0f806be30 msgr2=0x7fe0f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.753+0000 7fe111dd5640 1 --2- 172.21.15.112:0/2993469006 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe0f806be30 0x7fe0f806e2f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe100033000 tx=0x7fe100009030 comp rx=0 tx=0).stop 2024-04-25T01:41:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.753+0000 7fe111dd5640 1 -- 172.21.15.112:0/2993469006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10c1ef4a0 msgr2=0x7fe10c1ebd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.753+0000 7fe111dd5640 1 --2- 172.21.15.112:0/2993469006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10c1ef4a0 0x7fe10c1ebd00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe10c06bfd0 tx=0x7fe10400e960 comp rx=0 tx=0).stop 2024-04-25T01:41:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.753+0000 7fe111dd5640 1 -- 172.21.15.112:0/2993469006 shutdown_connections 2024-04-25T01:41:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.753+0000 7fe111dd5640 1 --2- 172.21.15.112:0/2993469006 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe0f806be30 0x7fe0f806e2f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.753+0000 7fe111dd5640 1 --2- 172.21.15.112:0/2993469006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10c1ef4a0 0x7fe10c1ebd00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.753+0000 7fe111dd5640 1 --2- 172.21.15.112:0/2993469006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe10c0c6870 0x7fe10c1422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.753+0000 7fe111dd5640 1 --2- 172.21.15.112:0/2993469006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0f40039b0 0x7fe10c0abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.753+0000 7fe111dd5640 1 -- 172.21.15.112:0/2993469006 >> 172.21.15.112:0/2993469006 conn(0x7fe10c0c3a10 msgr2=0x7fe10c0c3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.753+0000 7fe111dd5640 1 -- 172.21.15.112:0/2993469006 shutdown_connections 2024-04-25T01:41:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.753+0000 7fe111dd5640 1 -- 172.21.15.112:0/2993469006 wait complete. 2024-04-25T01:41:05.750 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-04-25T01:41:05.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.865+0000 7f487dc9e640 1 Processor -- start 2024-04-25T01:41:05.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.865+0000 7f487dc9e640 1 -- start start 2024-04-25T01:41:05.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f487dc9e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f487806d460 0x7f4878071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f487dc9e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f487806f290 con 0x7f48780724f0 2024-04-25T01:41:05.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f487dc9e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f487806efb0 con 0x7f487814fbe0 2024-04-25T01:41:05.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f487dc9e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f487814ffc0 con 0x7f487806d460 2024-04-25T01:41:05.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f4877fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f48780724f0 0x7f487815ce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37956/0 (socket says 172.21.15.112:37956) 2024-04-25T01:41:05.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f4877fff640 1 -- 172.21.15.112:0/3231007674 learned_addr learned my addr 172.21.15.112:0/3231007674 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:05.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f4876ffd640 1 --2- 172.21.15.112:0/3231007674 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f487806d460 0x7f4878071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:05.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f48767fc640 1 -- 172.21.15.112:0/3231007674 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 728174814 0 0) 0x7f487806f290 con 0x7f48780724f0 2024-04-25T01:41:05.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f48767fc640 1 -- 172.21.15.112:0/3231007674 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4860003520 con 0x7f48780724f0 2024-04-25T01:41:05.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f48767fc640 1 -- 172.21.15.112:0/3231007674 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 66824503 0 0) 0x7f487806efb0 con 0x7f487814fbe0 2024-04-25T01:41:05.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f48767fc640 1 -- 172.21.15.112:0/3231007674 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f487806f290 con 0x7f487814fbe0 2024-04-25T01:41:05.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f48767fc640 1 -- 172.21.15.112:0/3231007674 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4084847461 0 0) 0x7f4860003520 con 0x7f48780724f0 2024-04-25T01:41:05.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f48767fc640 1 -- 172.21.15.112:0/3231007674 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f487806d460 msgr2=0x7f4878071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:05.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f48767fc640 1 --2- 172.21.15.112:0/3231007674 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f487806d460 0x7f4878071fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f48767fc640 1 -- 172.21.15.112:0/3231007674 >> v1:172.21.15.187:6789/0 conn(0x7f487814fbe0 legacy=0x7f48780c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:05.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f48767fc640 1 -- 172.21.15.112:0/3231007674 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48780a9530 con 0x7f48780724f0 2024-04-25T01:41:05.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f48767fc640 1 -- 172.21.15.112:0/3231007674 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f48700019f0 con 0x7f48780724f0 2024-04-25T01:41:05.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f48767fc640 1 -- 172.21.15.112:0/3231007674 >> v1:172.21.15.112:6789/0 conn(0x7f48780724f0 legacy=0x7f487815ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:05.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f48767fc640 1 --2- 172.21.15.112:0/3231007674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4860003b70 0x7f4860003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f48767fc640 1 --2- 172.21.15.112:0/3231007674 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f487806d460 0x7f4860004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f48767fc640 1 --2- 172.21.15.112:0/3231007674 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4860004d40 0x7f48600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f48767fc640 1 -- 172.21.15.112:0/3231007674 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f486000ba90 con 0x7f4860003b70 2024-04-25T01:41:05.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f48767fc640 1 -- 172.21.15.112:0/3231007674 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f486000bc40 con 0x7f4860004d40 2024-04-25T01:41:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f48767fc640 1 -- 172.21.15.112:0/3231007674 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f486000be40 con 0x7f487806d460 2024-04-25T01:41:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.866+0000 7f48777fe640 1 --2- 172.21.15.112:0/3231007674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4860003b70 0x7f4860003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.867+0000 7f4876ffd640 1 --2- 172.21.15.112:0/3231007674 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f487806d460 0x7f4860004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.867+0000 7f4877fff640 1 --2- 172.21.15.112:0/3231007674 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4860004d40 0x7f48600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.867+0000 7f48777fe640 1 -- 172.21.15.112:0/3231007674 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f487806d460 msgr2=0x7f4860004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.867+0000 7f48777fe640 1 --2- 172.21.15.112:0/3231007674 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f487806d460 0x7f4860004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.867+0000 7f48777fe640 1 -- 172.21.15.112:0/3231007674 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4860004d40 msgr2=0x7f48600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.867+0000 7f48777fe640 1 --2- 172.21.15.112:0/3231007674 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4860004d40 0x7f48600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.867+0000 7f48777fe640 1 -- 172.21.15.112:0/3231007674 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4860003520 con 0x7f4860003b70 2024-04-25T01:41:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.867+0000 7f48777fe640 1 --2- 172.21.15.112:0/3231007674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4860003b70 0x7f4860003f70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f48680018d0 tx=0x7f4860005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebcb57d136d26f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.867+0000 7f48767fc640 1 -- 172.21.15.112:0/3231007674 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48680014f0 con 0x7f4860003b70 2024-04-25T01:41:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.868+0000 7f48767fc640 1 -- 172.21.15.112:0/3231007674 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48680052d0 con 0x7f4860003b70 2024-04-25T01:41:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.868+0000 7f48767fc640 1 -- 172.21.15.112:0/3231007674 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486802aca0 con 0x7f4860003b70 2024-04-25T01:41:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.868+0000 7f487dc9e640 1 -- 172.21.15.112:0/3231007674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4860003b70 msgr2=0x7f4860003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.868+0000 7f487dc9e640 1 --2- 172.21.15.112:0/3231007674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4860003b70 0x7f4860003f70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f48680018d0 tx=0x7f4860005830 comp rx=0 tx=0).stop 2024-04-25T01:41:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.868+0000 7f4877fff640 1 -- 172.21.15.112:0/3231007674 reap_dead start 2024-04-25T01:41:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.868+0000 7f487dc9e640 1 -- 172.21.15.112:0/3231007674 shutdown_connections 2024-04-25T01:41:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.868+0000 7f487dc9e640 1 -- 172.21.15.112:0/3231007674 >> 172.21.15.112:0/3231007674 conn(0x7f48780c3a10 msgr2=0x7f48780c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.868+0000 7f487dc9e640 1 -- 172.21.15.112:0/3231007674 shutdown_connections 2024-04-25T01:41:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.868+0000 7f487dc9e640 1 -- 172.21.15.112:0/3231007674 wait complete. 2024-04-25T01:41:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.869+0000 7f487dc9e640 1 Processor -- start 2024-04-25T01:41:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.869+0000 7f487dc9e640 1 -- start start 2024-04-25T01:41:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.870+0000 7f487dc9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4860003b70 0x7f4878149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.870+0000 7f487dc9e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48780c05b0 0x7f48781eb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.870+0000 7f487dc9e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48781ebd40 0x7f48781ee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.870+0000 7f487dc9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f48780c3030 con 0x7f4860003b70 2024-04-25T01:41:05.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.870+0000 7f487dc9e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f48781533a0 con 0x7f48780c05b0 2024-04-25T01:41:05.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.870+0000 7f487dc9e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f48780c31b0 con 0x7f48781ebd40 2024-04-25T01:41:05.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.870+0000 7f48777fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4860003b70 0x7f4878149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:05.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.870+0000 7f48777fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4860003b70 0x7f4878149d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34688/0 (socket says 172.21.15.112:34688) 2024-04-25T01:41:05.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.870+0000 7f48777fe640 1 -- 172.21.15.112:0/1084422535 learned_addr learned my addr 172.21.15.112:0/1084422535 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:05.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.870+0000 7f48777fe640 1 -- 172.21.15.112:0/1084422535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48781ebd40 msgr2=0x7f48781ee7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:05.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.870+0000 7f4876ffd640 1 --2- 172.21.15.112:0/1084422535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48780c05b0 0x7f48781eb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:05.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.870+0000 7f48777fe640 1 --2- 172.21.15.112:0/1084422535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48781ebd40 0x7f48781ee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.870+0000 7f48777fe640 1 -- 172.21.15.112:0/1084422535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48780c05b0 msgr2=0x7f48781eb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:05.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.870+0000 7f48777fe640 1 --2- 172.21.15.112:0/1084422535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48780c05b0 0x7f48781eb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:05.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.870+0000 7f48777fe640 1 -- 172.21.15.112:0/1084422535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487814a340 con 0x7f4860003b70 2024-04-25T01:41:05.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.871+0000 7f48777fe640 1 --2- 172.21.15.112:0/1084422535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4860003b70 0x7f4878149d60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4868005040 tx=0x7f48680012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:05.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.871+0000 7f485ffff640 1 -- 172.21.15.112:0/1084422535 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486803f020 con 0x7f4860003b70 2024-04-25T01:41:05.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.871+0000 7f485ffff640 1 -- 172.21.15.112:0/1084422535 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4868006e50 con 0x7f4860003b70 2024-04-25T01:41:05.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.871+0000 7f487dc9e640 1 -- 172.21.15.112:0/1084422535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487814a780 con 0x7f4860003b70 2024-04-25T01:41:05.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.871+0000 7f485ffff640 1 -- 172.21.15.112:0/1084422535 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868037040 con 0x7f4860003b70 2024-04-25T01:41:05.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.871+0000 7f487dc9e640 1 -- 172.21.15.112:0/1084422535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f487814cda0 con 0x7f4860003b70 2024-04-25T01:41:05.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.872+0000 7f487dc9e640 1 -- 172.21.15.112:0/1084422535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4878068a10 con 0x7f4860003b70 2024-04-25T01:41:05.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.872+0000 7f485ffff640 1 -- 172.21.15.112:0/1084422535 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4868042020 con 0x7f4860003b70 2024-04-25T01:41:05.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.873+0000 7f485ffff640 1 --2- 172.21.15.112:0/1084422535 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f486406be30 0x7f486406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:05.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.873+0000 7f4876ffd640 1 --2- 172.21.15.112:0/1084422535 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f486406be30 0x7f486406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:05.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.873+0000 7f485ffff640 1 -- 172.21.15.112:0/1084422535 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f48680ab3c0 con 0x7f4860003b70 2024-04-25T01:41:05.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.873+0000 7f4876ffd640 1 --2- 172.21.15.112:0/1084422535 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f486406be30 0x7f486406e2f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4878151de0 tx=0x7f486c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:05.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:05.877+0000 7f485ffff640 1 -- 172.21.15.112:0/1084422535 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4878068a10 con 0x7f4860003b70 2024-04-25T01:41:06.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.048+0000 7f487dc9e640 1 -- 172.21.15.112:0/1084422535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) -- 0x7f487804f500 con 0x7f4860003b70 2024-04-25T01:41:06.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.049+0000 7f485ffff640 1 -- 172.21.15.112:0/1084422535 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v26) ==== 95+0+11 (secure 0 0 0) 0x7f4868049020 con 0x7f4860003b70 2024-04-25T01:41:06.035 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T01:41:06.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.050+0000 7f487dc9e640 1 -- 172.21.15.112:0/1084422535 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f486406be30 msgr2=0x7f486406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.050+0000 7f487dc9e640 1 --2- 172.21.15.112:0/1084422535 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f486406be30 0x7f486406e2f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4878151de0 tx=0x7f486c008040 comp rx=0 tx=0).stop 2024-04-25T01:41:06.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.050+0000 7f487dc9e640 1 -- 172.21.15.112:0/1084422535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4860003b70 msgr2=0x7f4878149d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.050+0000 7f487dc9e640 1 --2- 172.21.15.112:0/1084422535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4860003b70 0x7f4878149d60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4868005040 tx=0x7f48680012d0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.051+0000 7f487dc9e640 1 -- 172.21.15.112:0/1084422535 shutdown_connections 2024-04-25T01:41:06.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.051+0000 7f487dc9e640 1 --2- 172.21.15.112:0/1084422535 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f486406be30 0x7f486406e2f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.051+0000 7f487dc9e640 1 --2- 172.21.15.112:0/1084422535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48781ebd40 0x7f48781ee7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.051+0000 7f487dc9e640 1 --2- 172.21.15.112:0/1084422535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48780c05b0 0x7f48781eb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.051+0000 7f487dc9e640 1 --2- 172.21.15.112:0/1084422535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4860003b70 0x7f4878149d60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.051+0000 7f487dc9e640 1 -- 172.21.15.112:0/1084422535 >> 172.21.15.112:0/1084422535 conn(0x7f48780c3a10 msgr2=0x7f48780c3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:06.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.051+0000 7f487dc9e640 1 -- 172.21.15.112:0/1084422535 shutdown_connections 2024-04-25T01:41:06.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.051+0000 7f487dc9e640 1 -- 172.21.15.112:0/1084422535 wait complete. 2024-04-25T01:41:06.048 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-04-25T01:41:06.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.162+0000 7f9db4072640 1 Processor -- start 2024-04-25T01:41:06.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.162+0000 7f9db4072640 1 -- start start 2024-04-25T01:41:06.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.162+0000 7f9db4072640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9dac14fbe0 0x7f9dac0c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.163+0000 7f9db4072640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9dac06d460 0x7f9dac071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.163+0000 7f9db4072640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dac06f290 con 0x7f9dac0c2e60 2024-04-25T01:41:06.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.163+0000 7f9db4072640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9dac14ffc0 con 0x7f9dac06d460 2024-04-25T01:41:06.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.163+0000 7f9db4072640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9dac1533a0 con 0x7f9dac14fbe0 2024-04-25T01:41:06.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.163+0000 7f9db1e0e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9dac14fbe0 0x7f9dac0c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:06.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.163+0000 7f9db160d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9dac06d460 0x7f9dac071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:06.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.163+0000 7f9db1e0e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9dac14fbe0 0x7f9dac0c2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:41970/0 (socket says 172.21.15.112:41970) 2024-04-25T01:41:06.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.163+0000 7f9db1e0e640 1 -- 172.21.15.112:0/524261220 learned_addr learned my addr 172.21.15.112:0/524261220 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:06.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.163+0000 7f9db160d640 1 -- 172.21.15.112:0/524261220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9dac14fbe0 msgr2=0x7f9dac0c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:06.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.163+0000 7f9db160d640 1 --2- 172.21.15.112:0/524261220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9dac14fbe0 0x7f9dac0c2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.163+0000 7f9db160d640 1 -- 172.21.15.112:0/524261220 >> v1:172.21.15.187:6789/0 conn(0x7f9dac0c2e60 legacy=0x7f9dac0724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.163+0000 7f9db160d640 1 -- 172.21.15.112:0/524261220 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dac06efb0 con 0x7f9dac06d460 2024-04-25T01:41:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.164+0000 7f9db160d640 1 --2- 172.21.15.112:0/524261220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9dac06d460 0x7f9dac071fb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9da80097a0 tx=0x7f9da802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e9e08713801ecf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.164+0000 7f9db0e0c640 1 -- 172.21.15.112:0/524261220 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8037070 con 0x7f9dac06d460 2024-04-25T01:41:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.164+0000 7f9db0e0c640 1 -- 172.21.15.112:0/524261220 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da802ae50 con 0x7f9dac06d460 2024-04-25T01:41:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.164+0000 7f9db0e0c640 1 -- 172.21.15.112:0/524261220 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8032660 con 0x7f9dac06d460 2024-04-25T01:41:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.165+0000 7f9db4072640 1 -- 172.21.15.112:0/524261220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9dac06d460 msgr2=0x7f9dac071fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.165+0000 7f9db4072640 1 --2- 172.21.15.112:0/524261220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9dac06d460 0x7f9dac071fb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9da80097a0 tx=0x7f9da802a100 comp rx=0 tx=0).stop 2024-04-25T01:41:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.165+0000 7f9db4072640 1 -- 172.21.15.112:0/524261220 shutdown_connections 2024-04-25T01:41:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.165+0000 7f9db4072640 1 --2- 172.21.15.112:0/524261220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9dac06d460 0x7f9dac071fb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.165+0000 7f9db4072640 1 --2- 172.21.15.112:0/524261220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9dac14fbe0 0x7f9dac0c2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.165+0000 7f9db4072640 1 -- 172.21.15.112:0/524261220 >> 172.21.15.112:0/524261220 conn(0x7f9dac0c3a10 msgr2=0x7f9dac0c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:06.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.165+0000 7f9db4072640 1 -- 172.21.15.112:0/524261220 shutdown_connections 2024-04-25T01:41:06.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.165+0000 7f9db4072640 1 -- 172.21.15.112:0/524261220 wait complete. 2024-04-25T01:41:06.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.166+0000 7f9db4072640 1 Processor -- start 2024-04-25T01:41:06.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.166+0000 7f9db4072640 1 -- start start 2024-04-25T01:41:06.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.166+0000 7f9db4072640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9dac06d460 0x7f9dac1515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.166+0000 7f9db4072640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9dac0c2e60 0x7f9dac145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.166+0000 7f9db4072640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dac14fbe0 0x7f9dac145570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.166+0000 7f9db4072640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac072c00 con 0x7f9dac14fbe0 2024-04-25T01:41:06.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.166+0000 7f9db4072640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac153030 con 0x7f9dac06d460 2024-04-25T01:41:06.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.166+0000 7f9db4072640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9dac0c3700 con 0x7f9dac0c2e60 2024-04-25T01:41:06.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.166+0000 7f9db260f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dac14fbe0 0x7f9dac145570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:06.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.167+0000 7f9db260f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dac14fbe0 0x7f9dac145570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34700/0 (socket says 172.21.15.112:34700) 2024-04-25T01:41:06.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.167+0000 7f9db260f640 1 -- 172.21.15.112:0/3213511672 learned_addr learned my addr 172.21.15.112:0/3213511672 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:06.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.167+0000 7f9db1e0e640 1 --2- 172.21.15.112:0/3213511672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9dac06d460 0x7f9dac1515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:06.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.167+0000 7f9db160d640 1 --2- 172.21.15.112:0/3213511672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9dac0c2e60 0x7f9dac145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:06.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.167+0000 7f9db260f640 1 -- 172.21.15.112:0/3213511672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9dac0c2e60 msgr2=0x7f9dac145030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.167+0000 7f9db260f640 1 --2- 172.21.15.112:0/3213511672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9dac0c2e60 0x7f9dac145030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.167+0000 7f9db260f640 1 -- 172.21.15.112:0/3213511672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9dac06d460 msgr2=0x7f9dac1515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.167+0000 7f9db260f640 1 --2- 172.21.15.112:0/3213511672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9dac06d460 0x7f9dac1515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.167+0000 7f9db260f640 1 -- 172.21.15.112:0/3213511672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dac152490 con 0x7f9dac14fbe0 2024-04-25T01:41:06.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.167+0000 7f9db260f640 1 --2- 172.21.15.112:0/3213511672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dac14fbe0 0x7f9dac145570 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9da400c7a0 tx=0x7f9da400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:06.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.167+0000 7f9da27fc640 1 -- 172.21.15.112:0/3213511672 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da4003b70 con 0x7f9dac14fbe0 2024-04-25T01:41:06.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.167+0000 7f9da27fc640 1 -- 172.21.15.112:0/3213511672 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da4003d30 con 0x7f9dac14fbe0 2024-04-25T01:41:06.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.167+0000 7f9db4072640 1 -- 172.21.15.112:0/3213511672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dac152770 con 0x7f9dac14fbe0 2024-04-25T01:41:06.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.167+0000 7f9da27fc640 1 -- 172.21.15.112:0/3213511672 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da400aa30 con 0x7f9dac14fbe0 2024-04-25T01:41:06.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.167+0000 7f9db4072640 1 -- 172.21.15.112:0/3213511672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dac1490a0 con 0x7f9dac14fbe0 2024-04-25T01:41:06.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.168+0000 7f9db4072640 1 -- 172.21.15.112:0/3213511672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dac04f500 con 0x7f9dac14fbe0 2024-04-25T01:41:06.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.169+0000 7f9da27fc640 1 -- 172.21.15.112:0/3213511672 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9da400abd0 con 0x7f9dac14fbe0 2024-04-25T01:41:06.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.169+0000 7f9da27fc640 1 --2- 172.21.15.112:0/3213511672 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9d9006be30 0x7f9d9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.169+0000 7f9da27fc640 1 -- 172.21.15.112:0/3213511672 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f9da408af90 con 0x7f9dac14fbe0 2024-04-25T01:41:06.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.169+0000 7f9db1e0e640 1 --2- 172.21.15.112:0/3213511672 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9d9006be30 0x7f9d9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:06.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.169+0000 7f9db1e0e640 1 --2- 172.21.15.112:0/3213511672 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9d9006be30 0x7f9d9006e2f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009bd0 tx=0x7f9d9c009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:06.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.173+0000 7f9da27fc640 1 -- 172.21.15.112:0/3213511672 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9da4058f60 con 0x7f9dac14fbe0 2024-04-25T01:41:06.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.346+0000 7f9db4072640 1 -- 172.21.15.112:0/3213511672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) -- 0x7f9dac156980 con 0x7f9dac14fbe0 2024-04-25T01:41:06.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.347+0000 7f9da27fc640 1 -- 172.21.15.112:0/3213511672 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v26) ==== 99+0+11 (secure 0 0 0) 0x7f9da405d260 con 0x7f9dac14fbe0 2024-04-25T01:41:06.333 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T01:41:06.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.348+0000 7f9db4072640 1 -- 172.21.15.112:0/3213511672 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9d9006be30 msgr2=0x7f9d9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.348+0000 7f9db4072640 1 --2- 172.21.15.112:0/3213511672 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9d9006be30 0x7f9d9006e2f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009bd0 tx=0x7f9d9c009240 comp rx=0 tx=0).stop 2024-04-25T01:41:06.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.348+0000 7f9db4072640 1 -- 172.21.15.112:0/3213511672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dac14fbe0 msgr2=0x7f9dac145570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.348+0000 7f9db4072640 1 --2- 172.21.15.112:0/3213511672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dac14fbe0 0x7f9dac145570 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9da400c7a0 tx=0x7f9da400cc60 comp rx=0 tx=0).stop 2024-04-25T01:41:06.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.349+0000 7f9db4072640 1 -- 172.21.15.112:0/3213511672 shutdown_connections 2024-04-25T01:41:06.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.349+0000 7f9db4072640 1 --2- 172.21.15.112:0/3213511672 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9d9006be30 0x7f9d9006e2f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.349+0000 7f9db4072640 1 --2- 172.21.15.112:0/3213511672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dac14fbe0 0x7f9dac145570 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.349+0000 7f9db4072640 1 --2- 172.21.15.112:0/3213511672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9dac0c2e60 0x7f9dac145030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.349+0000 7f9db4072640 1 --2- 172.21.15.112:0/3213511672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9dac06d460 0x7f9dac1515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.349+0000 7f9db4072640 1 -- 172.21.15.112:0/3213511672 >> 172.21.15.112:0/3213511672 conn(0x7f9dac0c3a10 msgr2=0x7f9dac0c3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:06.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.349+0000 7f9db4072640 1 -- 172.21.15.112:0/3213511672 shutdown_connections 2024-04-25T01:41:06.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.349+0000 7f9db4072640 1 -- 172.21.15.112:0/3213511672 wait complete. 2024-04-25T01:41:06.346 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-04-25T01:41:06.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.460+0000 7f563bcb3640 1 Processor -- start 2024-04-25T01:41:06.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.460+0000 7f563bcb3640 1 -- start start 2024-04-25T01:41:06.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.461+0000 7f563bcb3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f56340c9250 0x7f56340c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.461+0000 7f563bcb3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56340ca8a0 0x7f563406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.461+0000 7f563bcb3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56341534a0 con 0x7f56340ca370 2024-04-25T01:41:06.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.461+0000 7f563bcb3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f5634156990 con 0x7f56340c9250 2024-04-25T01:41:06.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.461+0000 7f563bcb3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5634156500 con 0x7f56340ca8a0 2024-04-25T01:41:06.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.461+0000 7f5639a4f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f56340c9250 0x7f56340c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:06.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.461+0000 7f563924e640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f56340ca370 0x7f563406bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37964/0 (socket says 172.21.15.112:37964) 2024-04-25T01:41:06.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.461+0000 7f5639a4f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f56340c9250 0x7f56340c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34704/0 (socket says 172.21.15.112:34704) 2024-04-25T01:41:06.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.461+0000 7f5639a4f640 1 -- 172.21.15.112:0/1005501892 learned_addr learned my addr 172.21.15.112:0/1005501892 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:06.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.461+0000 7f5639a4f640 1 -- 172.21.15.112:0/1005501892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56340ca8a0 msgr2=0x7f563406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:06.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.461+0000 7f5639a4f640 1 --2- 172.21.15.112:0/1005501892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56340ca8a0 0x7f563406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.461+0000 7f5639a4f640 1 -- 172.21.15.112:0/1005501892 >> v1:172.21.15.112:6789/0 conn(0x7f56340ca370 legacy=0x7f563406bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.461+0000 7f5639a4f640 1 -- 172.21.15.112:0/1005501892 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56340cae30 con 0x7f56340c9250 2024-04-25T01:41:06.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.461+0000 7f5638a4d640 1 -- 172.21.15.112:0/1005501892 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 789481861 0 0) 0x7f56341534a0 con 0x7f56340ca370 2024-04-25T01:41:06.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.462+0000 7f5639a4f640 1 --2- 172.21.15.112:0/1005501892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56340c9250 0x7f56340c9630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5630009870 tx=0x7f563002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b73c1748b874d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:06.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.462+0000 7f5638a4d640 1 -- 172.21.15.112:0/1005501892 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630037070 con 0x7f56340c9250 2024-04-25T01:41:06.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.462+0000 7f5638a4d640 1 -- 172.21.15.112:0/1005501892 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5630032440 con 0x7f56340c9250 2024-04-25T01:41:06.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.462+0000 7f5638a4d640 1 -- 172.21.15.112:0/1005501892 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630032810 con 0x7f56340c9250 2024-04-25T01:41:06.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.462+0000 7f563bcb3640 1 -- 172.21.15.112:0/1005501892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56340c9250 msgr2=0x7f56340c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.462+0000 7f563bcb3640 1 --2- 172.21.15.112:0/1005501892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56340c9250 0x7f56340c9630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5630009870 tx=0x7f563002a1c0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.463+0000 7f563bcb3640 1 -- 172.21.15.112:0/1005501892 shutdown_connections 2024-04-25T01:41:06.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.463+0000 7f563bcb3640 1 --2- 172.21.15.112:0/1005501892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56340ca8a0 0x7f563406c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.463+0000 7f563bcb3640 1 --2- 172.21.15.112:0/1005501892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56340c9250 0x7f56340c9630 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.463+0000 7f563bcb3640 1 -- 172.21.15.112:0/1005501892 >> 172.21.15.112:0/1005501892 conn(0x7f56340cb310 msgr2=0x7f56340cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:06.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.463+0000 7f563bcb3640 1 -- 172.21.15.112:0/1005501892 shutdown_connections 2024-04-25T01:41:06.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.463+0000 7f563bcb3640 1 -- 172.21.15.112:0/1005501892 wait complete. 2024-04-25T01:41:06.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.464+0000 7f563bcb3640 1 Processor -- start 2024-04-25T01:41:06.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.464+0000 7f563bcb3640 1 -- start start 2024-04-25T01:41:06.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.464+0000 7f563bcb3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56340c9250 0x7f5634158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.464+0000 7f563bcb3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56340ca370 0x7f56340c3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.464+0000 7f563bcb3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56340ca8a0 0x7f56340c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.464+0000 7f563bcb3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5634155eb0 con 0x7f56340ca8a0 2024-04-25T01:41:06.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.464+0000 7f563bcb3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5634155bd0 con 0x7f56340ca370 2024-04-25T01:41:06.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.464+0000 7f563bcb3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5634156190 con 0x7f56340c9250 2024-04-25T01:41:06.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.464+0000 7f563a250640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56340ca8a0 0x7f56340c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:06.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.465+0000 7f563a250640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56340ca8a0 0x7f56340c3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34706/0 (socket says 172.21.15.112:34706) 2024-04-25T01:41:06.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.465+0000 7f563a250640 1 -- 172.21.15.112:0/3781110884 learned_addr learned my addr 172.21.15.112:0/3781110884 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:06.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.465+0000 7f5639a4f640 1 --2- 172.21.15.112:0/3781110884 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56340c9250 0x7f5634158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:06.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.465+0000 7f563924e640 1 --2- 172.21.15.112:0/3781110884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56340ca370 0x7f56340c3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:06.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.465+0000 7f563a250640 1 -- 172.21.15.112:0/3781110884 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56340c9250 msgr2=0x7f5634158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.465+0000 7f563a250640 1 --2- 172.21.15.112:0/3781110884 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56340c9250 0x7f5634158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.465+0000 7f563a250640 1 -- 172.21.15.112:0/3781110884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56340ca370 msgr2=0x7f56340c3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.465+0000 7f563a250640 1 --2- 172.21.15.112:0/3781110884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56340ca370 0x7f56340c3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.465+0000 7f563a250640 1 -- 172.21.15.112:0/3781110884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56340c1360 con 0x7f56340ca8a0 2024-04-25T01:41:06.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.465+0000 7f563a250640 1 --2- 172.21.15.112:0/3781110884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56340ca8a0 0x7f56340c3fb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f562c00b9e0 tx=0x7f562c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:06.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.465+0000 7f562a7fc640 1 -- 172.21.15.112:0/3781110884 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c019070 con 0x7f56340ca8a0 2024-04-25T01:41:06.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.465+0000 7f562a7fc640 1 -- 172.21.15.112:0/3781110884 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f562c00cb20 con 0x7f56340ca8a0 2024-04-25T01:41:06.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.465+0000 7f562a7fc640 1 -- 172.21.15.112:0/3781110884 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c014440 con 0x7f56340ca8a0 2024-04-25T01:41:06.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.465+0000 7f563bcb3640 1 -- 172.21.15.112:0/3781110884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56340c15d0 con 0x7f56340ca8a0 2024-04-25T01:41:06.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.465+0000 7f563bcb3640 1 -- 172.21.15.112:0/3781110884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56340c1b90 con 0x7f56340ca8a0 2024-04-25T01:41:06.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.466+0000 7f563bcb3640 1 -- 172.21.15.112:0/3781110884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f563404f500 con 0x7f56340ca8a0 2024-04-25T01:41:06.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.467+0000 7f562a7fc640 1 -- 172.21.15.112:0/3781110884 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f562c01c070 con 0x7f56340ca8a0 2024-04-25T01:41:06.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.467+0000 7f562a7fc640 1 --2- 172.21.15.112:0/3781110884 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f561806be30 0x7f561806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.467+0000 7f562a7fc640 1 -- 172.21.15.112:0/3781110884 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f562c08aea0 con 0x7f56340ca8a0 2024-04-25T01:41:06.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.467+0000 7f5639a4f640 1 --2- 172.21.15.112:0/3781110884 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f561806be30 0x7f561806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:06.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.467+0000 7f5639a4f640 1 --2- 172.21.15.112:0/3781110884 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f561806be30 0x7f561806e2f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5630034000 tx=0x7f563002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:06.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.471+0000 7f562a7fc640 1 -- 172.21.15.112:0/3781110884 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f562c059540 con 0x7f56340ca8a0 2024-04-25T01:41:06.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.648+0000 7f563bcb3640 1 -- 172.21.15.112:0/3781110884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) -- 0x7f563406d970 con 0x7f56340ca8a0 2024-04-25T01:41:06.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.649+0000 7f562a7fc640 1 -- 172.21.15.112:0/3781110884 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v26) ==== 95+0+11 (secure 0 0 0) 0x7f562c058e40 con 0x7f56340ca8a0 2024-04-25T01:41:06.635 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T01:41:06.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.650+0000 7f563bcb3640 1 -- 172.21.15.112:0/3781110884 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f561806be30 msgr2=0x7f561806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.650+0000 7f563bcb3640 1 --2- 172.21.15.112:0/3781110884 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f561806be30 0x7f561806e2f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5630034000 tx=0x7f563002e000 comp rx=0 tx=0).stop 2024-04-25T01:41:06.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.650+0000 7f563bcb3640 1 -- 172.21.15.112:0/3781110884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56340ca8a0 msgr2=0x7f56340c3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.650+0000 7f563bcb3640 1 --2- 172.21.15.112:0/3781110884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56340ca8a0 0x7f56340c3fb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f562c00b9e0 tx=0x7f562c00bea0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.651+0000 7f563bcb3640 1 -- 172.21.15.112:0/3781110884 shutdown_connections 2024-04-25T01:41:06.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.651+0000 7f563bcb3640 1 --2- 172.21.15.112:0/3781110884 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f561806be30 0x7f561806e2f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.651+0000 7f563bcb3640 1 --2- 172.21.15.112:0/3781110884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56340ca8a0 0x7f56340c3fb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.651+0000 7f563bcb3640 1 --2- 172.21.15.112:0/3781110884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56340ca370 0x7f56340c3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.651+0000 7f563bcb3640 1 --2- 172.21.15.112:0/3781110884 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56340c9250 0x7f5634158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.651+0000 7f563bcb3640 1 -- 172.21.15.112:0/3781110884 >> 172.21.15.112:0/3781110884 conn(0x7f56340cb310 msgr2=0x7f56341538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:06.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.651+0000 7f563bcb3640 1 -- 172.21.15.112:0/3781110884 shutdown_connections 2024-04-25T01:41:06.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.651+0000 7f563bcb3640 1 -- 172.21.15.112:0/3781110884 wait complete. 2024-04-25T01:41:06.649 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:41:06.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.759+0000 7fcc18638640 1 Processor -- start 2024-04-25T01:41:06.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.759+0000 7fcc18638640 1 -- start start 2024-04-25T01:41:06.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.760+0000 7fcc18638640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcc101596a0 0x7fcc10152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.760+0000 7fcc18638640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc100c72b0 0x7fcc10160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.760+0000 7fcc18638640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc10159b90 con 0x7fcc101592c0 2024-04-25T01:41:06.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.760+0000 7fcc18638640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcc100c6e20 con 0x7fcc101596a0 2024-04-25T01:41:06.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.760+0000 7fcc18638640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcc1006c970 con 0x7fcc100c72b0 2024-04-25T01:41:06.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.760+0000 7fcc163d4640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fcc101592c0 0x7fcc100c9070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37966/0 (socket says 172.21.15.112:37966) 2024-04-25T01:41:06.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.760+0000 7fcc163d4640 1 -- 172.21.15.112:0/2880899861 learned_addr learned my addr 172.21.15.112:0/2880899861 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc15bd3640 1 --2- 172.21.15.112:0/2880899861 >> v2:172.21.15.187:3300/0 conn(0x7fcc101596a0 0x7fcc10152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc16bd5640 1 --2- 172.21.15.112:0/2880899861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc100c72b0 0x7fcc10160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc153d2640 1 -- 172.21.15.112:0/2880899861 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 173534188 0 0) 0x7fcc10159b90 con 0x7fcc101592c0 2024-04-25T01:41:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc153d2640 1 -- 172.21.15.112:0/2880899861 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcbfc003520 con 0x7fcc101592c0 2024-04-25T01:41:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc153d2640 1 -- 172.21.15.112:0/2880899861 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2353519366 0 0) 0x7fcbfc003520 con 0x7fcc101592c0 2024-04-25T01:41:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc153d2640 1 -- 172.21.15.112:0/2880899861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc100c72b0 msgr2=0x7fcc10160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc153d2640 1 --2- 172.21.15.112:0/2880899861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc100c72b0 0x7fcc10160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc153d2640 1 -- 172.21.15.112:0/2880899861 >> v2:172.21.15.187:3300/0 conn(0x7fcc101596a0 msgr2=0x7fcc10152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc153d2640 1 --2- 172.21.15.112:0/2880899861 >> v2:172.21.15.187:3300/0 conn(0x7fcc101596a0 0x7fcc10152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc153d2640 1 -- 172.21.15.112:0/2880899861 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1006cc50 con 0x7fcc101592c0 2024-04-25T01:41:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc153d2640 1 -- 172.21.15.112:0/2880899861 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fcc0c001a10 con 0x7fcc101592c0 2024-04-25T01:41:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc153d2640 1 -- 172.21.15.112:0/2880899861 >> v1:172.21.15.112:6789/0 conn(0x7fcc101592c0 legacy=0x7fcc100c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc153d2640 1 --2- 172.21.15.112:0/2880899861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc0039b0 0x7fcbfc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc153d2640 1 --2- 172.21.15.112:0/2880899861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbfc004800 0x7fcbfc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc153d2640 1 --2- 172.21.15.112:0/2880899861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc100c72b0 0x7fcbfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc153d2640 1 -- 172.21.15.112:0/2880899861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1006c970 con 0x7fcbfc0039b0 2024-04-25T01:41:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc153d2640 1 -- 172.21.15.112:0/2880899861 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc00b9f0 con 0x7fcbfc004800 2024-04-25T01:41:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc153d2640 1 -- 172.21.15.112:0/2880899861 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcbfc00bbf0 con 0x7fcc100c72b0 2024-04-25T01:41:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc163d4640 1 --2- 172.21.15.112:0/2880899861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc0039b0 0x7fcbfc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc15bd3640 1 --2- 172.21.15.112:0/2880899861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbfc004800 0x7fcbfc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc16bd5640 1 --2- 172.21.15.112:0/2880899861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc100c72b0 0x7fcbfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc163d4640 1 -- 172.21.15.112:0/2880899861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc100c72b0 msgr2=0x7fcbfc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc163d4640 1 --2- 172.21.15.112:0/2880899861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc100c72b0 0x7fcbfc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc163d4640 1 -- 172.21.15.112:0/2880899861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbfc004800 msgr2=0x7fcbfc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc163d4640 1 --2- 172.21.15.112:0/2880899861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbfc004800 0x7fcbfc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.761+0000 7fcc163d4640 1 -- 172.21.15.112:0/2880899861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbfc003520 con 0x7fcbfc0039b0 2024-04-25T01:41:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.762+0000 7fcc163d4640 1 --2- 172.21.15.112:0/2880899861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc0039b0 0x7fcbfc003dd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00a880 tx=0x7fcbfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96029172ad2e49a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.762+0000 7fcc153d2640 1 -- 172.21.15.112:0/2880899861 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c00a4d0 con 0x7fcbfc0039b0 2024-04-25T01:41:06.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.762+0000 7fcc153d2640 1 -- 172.21.15.112:0/2880899861 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc0c02f070 con 0x7fcbfc0039b0 2024-04-25T01:41:06.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.762+0000 7fcc153d2640 1 -- 172.21.15.112:0/2880899861 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c02bcc0 con 0x7fcbfc0039b0 2024-04-25T01:41:06.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.763+0000 7fcc18638640 1 -- 172.21.15.112:0/2880899861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc0039b0 msgr2=0x7fcbfc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.763+0000 7fcc18638640 1 --2- 172.21.15.112:0/2880899861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc0039b0 0x7fcbfc003dd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00a880 tx=0x7fcbfc005830 comp rx=0 tx=0).stop 2024-04-25T01:41:06.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.763+0000 7fcc16bd5640 1 -- 172.21.15.112:0/2880899861 reap_dead start 2024-04-25T01:41:06.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.763+0000 7fcc18638640 1 -- 172.21.15.112:0/2880899861 shutdown_connections 2024-04-25T01:41:06.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.763+0000 7fcc18638640 1 -- 172.21.15.112:0/2880899861 >> 172.21.15.112:0/2880899861 conn(0x7fcc100cb170 msgr2=0x7fcc100cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:06.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.763+0000 7fcc18638640 1 -- 172.21.15.112:0/2880899861 shutdown_connections 2024-04-25T01:41:06.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.763+0000 7fcc18638640 1 -- 172.21.15.112:0/2880899861 wait complete. 2024-04-25T01:41:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.764+0000 7fcc18638640 1 Processor -- start 2024-04-25T01:41:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.764+0000 7fcc18638640 1 -- start start 2024-04-25T01:41:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.765+0000 7fcc18638640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc0039b0 0x7fcc1014d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.765+0000 7fcc18638640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc100c52e0 0x7fcc100c2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.765+0000 7fcc18638640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc100c2780 0x7fcc10148450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.765+0000 7fcc18638640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1006c510 con 0x7fcbfc0039b0 2024-04-25T01:41:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.765+0000 7fcc18638640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1006c390 con 0x7fcc100c52e0 2024-04-25T01:41:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.765+0000 7fcc18638640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcc1006db20 con 0x7fcc100c2780 2024-04-25T01:41:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.765+0000 7fcc163d4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc0039b0 0x7fcc1014d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.765+0000 7fcc163d4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc0039b0 0x7fcc1014d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34730/0 (socket says 172.21.15.112:34730) 2024-04-25T01:41:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.765+0000 7fcc163d4640 1 -- 172.21.15.112:0/460772151 learned_addr learned my addr 172.21.15.112:0/460772151 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.765+0000 7fcc15bd3640 1 --2- 172.21.15.112:0/460772151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc100c52e0 0x7fcc100c2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.765+0000 7fcc16bd5640 1 --2- 172.21.15.112:0/460772151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc100c2780 0x7fcc10148450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.765+0000 7fcc163d4640 1 -- 172.21.15.112:0/460772151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc100c2780 msgr2=0x7fcc10148450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.765+0000 7fcc163d4640 1 --2- 172.21.15.112:0/460772151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc100c2780 0x7fcc10148450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.765+0000 7fcc163d4640 1 -- 172.21.15.112:0/460772151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc100c52e0 msgr2=0x7fcc100c2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.765+0000 7fcc163d4640 1 --2- 172.21.15.112:0/460772151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc100c52e0 0x7fcc100c2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.765+0000 7fcc163d4640 1 -- 172.21.15.112:0/460772151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc100be420 con 0x7fcbfc0039b0 2024-04-25T01:41:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.765+0000 7fcc163d4640 1 --2- 172.21.15.112:0/460772151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc0039b0 0x7fcc1014d540 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c000f80 tx=0x7fcc1006e210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.765+0000 7fcc06ffd640 1 -- 172.21.15.112:0/460772151 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c044070 con 0x7fcbfc0039b0 2024-04-25T01:41:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.765+0000 7fcc06ffd640 1 -- 172.21.15.112:0/460772151 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc0c02be40 con 0x7fcbfc0039b0 2024-04-25T01:41:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.765+0000 7fcc06ffd640 1 -- 172.21.15.112:0/460772151 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c002020 con 0x7fcbfc0039b0 2024-04-25T01:41:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.766+0000 7fcc18638640 1 -- 172.21.15.112:0/460772151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1014e050 con 0x7fcbfc0039b0 2024-04-25T01:41:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.766+0000 7fcc18638640 1 -- 172.21.15.112:0/460772151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc1014abf0 con 0x7fcbfc0039b0 2024-04-25T01:41:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.766+0000 7fcc18638640 1 -- 172.21.15.112:0/460772151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc100c8a80 con 0x7fcbfc0039b0 2024-04-25T01:41:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.767+0000 7fcc06ffd640 1 -- 172.21.15.112:0/460772151 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcc0c047070 con 0x7fcbfc0039b0 2024-04-25T01:41:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.767+0000 7fcc06ffd640 1 --2- 172.21.15.112:0/460772151 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcbf406be60 0x7fcbf406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.767+0000 7fcc06ffd640 1 -- 172.21.15.112:0/460772151 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fcc0c0aade0 con 0x7fcbfc0039b0 2024-04-25T01:41:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.767+0000 7fcc15bd3640 1 --2- 172.21.15.112:0/460772151 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcbf406be60 0x7fcbf406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:06.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.768+0000 7fcc15bd3640 1 --2- 172.21.15.112:0/460772151 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcbf406be60 0x7fcbf406e320 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcc10147050 tx=0x7fcc00009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:06.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.772+0000 7fcc06ffd640 1 -- 172.21.15.112:0/460772151 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcc100c8a80 con 0x7fcbfc0039b0 2024-04-25T01:41:06.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.960+0000 7fcc18638640 1 -- 172.21.15.112:0/460772151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcc100682a0 con 0x7fcbfc0039b0 2024-04-25T01:41:06.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.960+0000 7fcc06ffd640 1 -- 172.21.15.112:0/460772151 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fcc0c0793f0 con 0x7fcbfc0039b0 2024-04-25T01:41:06.946 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-04-25T01:41:06.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.962+0000 7fcc18638640 1 -- 172.21.15.112:0/460772151 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcbf406be60 msgr2=0x7fcbf406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.962+0000 7fcc18638640 1 --2- 172.21.15.112:0/460772151 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcbf406be60 0x7fcbf406e320 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcc10147050 tx=0x7fcc00009190 comp rx=0 tx=0).stop 2024-04-25T01:41:06.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.962+0000 7fcc18638640 1 -- 172.21.15.112:0/460772151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc0039b0 msgr2=0x7fcc1014d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.962+0000 7fcc18638640 1 --2- 172.21.15.112:0/460772151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc0039b0 0x7fcc1014d540 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c000f80 tx=0x7fcc1006e210 comp rx=0 tx=0).stop 2024-04-25T01:41:06.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.962+0000 7fcc18638640 1 -- 172.21.15.112:0/460772151 shutdown_connections 2024-04-25T01:41:06.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.962+0000 7fcc18638640 1 --2- 172.21.15.112:0/460772151 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcbf406be60 0x7fcbf406e320 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.962+0000 7fcc18638640 1 --2- 172.21.15.112:0/460772151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc100c2780 0x7fcc10148450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.962+0000 7fcc18638640 1 --2- 172.21.15.112:0/460772151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc100c52e0 0x7fcc100c2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.962+0000 7fcc18638640 1 --2- 172.21.15.112:0/460772151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc0039b0 0x7fcc1014d540 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.962+0000 7fcc18638640 1 -- 172.21.15.112:0/460772151 >> 172.21.15.112:0/460772151 conn(0x7fcc100cb170 msgr2=0x7fcc100aae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:06.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.962+0000 7fcc18638640 1 -- 172.21.15.112:0/460772151 shutdown_connections 2024-04-25T01:41:06.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:06.962+0000 7fcc18638640 1 -- 172.21.15.112:0/460772151 wait complete. 2024-04-25T01:41:06.959 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T01:41:06.959 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:41:06.979 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:41:07.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.090+0000 7f0dd1df9640 1 Processor -- start 2024-04-25T01:41:07.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.090+0000 7f0dd1df9640 1 -- start start 2024-04-25T01:41:07.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.090+0000 7f0dd1df9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0dcc1592c0 0x7f0dcc0c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.090+0000 7f0dd1df9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dcc1596a0 0x7f0dcc152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.090+0000 7f0dd1df9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dcc159b90 con 0x7f0dcc0c72b0 2024-04-25T01:41:07.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.090+0000 7f0dd1df9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f0dcc0c6e20 con 0x7f0dcc1592c0 2024-04-25T01:41:07.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.090+0000 7f0dd1df9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0dcc06c970 con 0x7f0dcc1596a0 2024-04-25T01:41:07.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.090+0000 7f0dcb7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0dcc1592c0 0x7f0dcc0c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:07.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.090+0000 7f0dcb7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0dcc1592c0 0x7f0dcc0c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34736/0 (socket says 172.21.15.112:34736) 2024-04-25T01:41:07.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.090+0000 7f0dcb7fe640 1 -- 172.21.15.112:0/2228815544 learned_addr learned my addr 172.21.15.112:0/2228815544 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:07.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.091+0000 7f0dcb7fe640 1 -- 172.21.15.112:0/2228815544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dcc1596a0 msgr2=0x7f0dcc152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:07.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.091+0000 7f0dcb7fe640 1 --2- 172.21.15.112:0/2228815544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dcc1596a0 0x7f0dcc152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.091+0000 7f0dcb7fe640 1 -- 172.21.15.112:0/2228815544 >> v1:172.21.15.187:6789/0 conn(0x7f0dcc0c72b0 legacy=0x7f0dcc155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:07.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.091+0000 7f0dcb7fe640 1 -- 172.21.15.112:0/2228815544 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc06cc50 con 0x7f0dcc1592c0 2024-04-25T01:41:07.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.091+0000 7f0dcb7fe640 1 --2- 172.21.15.112:0/2228815544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dcc1592c0 0x7f0dcc0c9070 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc08cb00 tx=0x7f0dbc02a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0778a8aae76b46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:07.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.092+0000 7f0dcaffd640 1 -- 172.21.15.112:0/2228815544 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc037070 con 0x7f0dcc1592c0 2024-04-25T01:41:07.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.092+0000 7f0dcaffd640 1 -- 172.21.15.112:0/2228815544 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dbc02ade0 con 0x7f0dcc1592c0 2024-04-25T01:41:07.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.092+0000 7f0dcaffd640 1 -- 172.21.15.112:0/2228815544 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc032660 con 0x7f0dcc1592c0 2024-04-25T01:41:07.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.092+0000 7f0dd1df9640 1 -- 172.21.15.112:0/2228815544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dcc1592c0 msgr2=0x7f0dcc0c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:07.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.092+0000 7f0dd1df9640 1 --2- 172.21.15.112:0/2228815544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dcc1592c0 0x7f0dcc0c9070 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc08cb00 tx=0x7f0dbc02a0d0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.092+0000 7f0dd1df9640 1 -- 172.21.15.112:0/2228815544 shutdown_connections 2024-04-25T01:41:07.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.092+0000 7f0dd1df9640 1 --2- 172.21.15.112:0/2228815544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dcc1596a0 0x7f0dcc152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.092+0000 7f0dd1df9640 1 --2- 172.21.15.112:0/2228815544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dcc1592c0 0x7f0dcc0c9070 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.092+0000 7f0dd1df9640 1 -- 172.21.15.112:0/2228815544 >> 172.21.15.112:0/2228815544 conn(0x7f0dcc0cb170 msgr2=0x7f0dcc0cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:07.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.092+0000 7f0dd1df9640 1 -- 172.21.15.112:0/2228815544 shutdown_connections 2024-04-25T01:41:07.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.092+0000 7f0dd1df9640 1 -- 172.21.15.112:0/2228815544 wait complete. 2024-04-25T01:41:07.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.093+0000 7f0dd1df9640 1 Processor -- start 2024-04-25T01:41:07.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.093+0000 7f0dd1df9640 1 -- start start 2024-04-25T01:41:07.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.094+0000 7f0dd1df9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0dcc0c72b0 0x7f0dcc1483a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.094+0000 7f0dd1df9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dcc1592c0 0x7f0dcc0c0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.094+0000 7f0dd1df9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dcc1596a0 0x7f0dcc14a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.094+0000 7f0dd1df9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc06dca0 con 0x7f0dcc1596a0 2024-04-25T01:41:07.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.094+0000 7f0dd1df9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc06db20 con 0x7f0dcc0c72b0 2024-04-25T01:41:07.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.094+0000 7f0dd1df9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0dcc0c95b0 con 0x7f0dcc1592c0 2024-04-25T01:41:07.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.094+0000 7f0dcbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dcc1596a0 0x7f0dcc14a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:07.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.094+0000 7f0dcbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dcc1596a0 0x7f0dcc14a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34742/0 (socket says 172.21.15.112:34742) 2024-04-25T01:41:07.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.094+0000 7f0dcbfff640 1 -- 172.21.15.112:0/4087388154 learned_addr learned my addr 172.21.15.112:0/4087388154 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:07.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.094+0000 7f0dcb7fe640 1 --2- 172.21.15.112:0/4087388154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0dcc0c72b0 0x7f0dcc1483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:07.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.094+0000 7f0dc2ffd640 1 --2- 172.21.15.112:0/4087388154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dcc1592c0 0x7f0dcc0c0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:07.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.094+0000 7f0dcbfff640 1 -- 172.21.15.112:0/4087388154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dcc1592c0 msgr2=0x7f0dcc0c0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:07.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.094+0000 7f0dcbfff640 1 --2- 172.21.15.112:0/4087388154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dcc1592c0 0x7f0dcc0c0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.094+0000 7f0dcbfff640 1 -- 172.21.15.112:0/4087388154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0dcc0c72b0 msgr2=0x7f0dcc1483a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:07.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.094+0000 7f0dcbfff640 1 --2- 172.21.15.112:0/4087388154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0dcc0c72b0 0x7f0dcc1483a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.094+0000 7f0dcbfff640 1 -- 172.21.15.112:0/4087388154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc14b4d0 con 0x7f0dcc1596a0 2024-04-25T01:41:07.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.095+0000 7f0dcbfff640 1 --2- 172.21.15.112:0/4087388154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dcc1596a0 0x7f0dcc14a050 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400c7b0 tx=0x7f0dc400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:07.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.095+0000 7f0dc8ff9640 1 -- 172.21.15.112:0/4087388154 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4017070 con 0x7f0dcc1596a0 2024-04-25T01:41:07.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.095+0000 7f0dc8ff9640 1 -- 172.21.15.112:0/4087388154 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc4003da0 con 0x7f0dcc1596a0 2024-04-25T01:41:07.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.095+0000 7f0dd1df9640 1 -- 172.21.15.112:0/4087388154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dcc0bf120 con 0x7f0dcc1596a0 2024-04-25T01:41:07.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.095+0000 7f0dc8ff9640 1 -- 172.21.15.112:0/4087388154 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc40052b0 con 0x7f0dcc1596a0 2024-04-25T01:41:07.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.095+0000 7f0dd1df9640 1 -- 172.21.15.112:0/4087388154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dcc149850 con 0x7f0dcc1596a0 2024-04-25T01:41:07.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.096+0000 7f0dd1df9640 1 -- 172.21.15.112:0/4087388154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dcc0679a0 con 0x7f0dcc1596a0 2024-04-25T01:41:07.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.096+0000 7f0dc8ff9640 1 -- 172.21.15.112:0/4087388154 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0dc401a070 con 0x7f0dcc1596a0 2024-04-25T01:41:07.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.097+0000 7f0dc8ff9640 1 --2- 172.21.15.112:0/4087388154 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0dac06be30 0x7f0dac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.097+0000 7f0dc8ff9640 1 -- 172.21.15.112:0/4087388154 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f0dc408a8a0 con 0x7f0dcc1596a0 2024-04-25T01:41:07.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.097+0000 7f0dcb7fe640 1 --2- 172.21.15.112:0/4087388154 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0dac06be30 0x7f0dac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:07.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.097+0000 7f0dcb7fe640 1 --2- 172.21.15.112:0/4087388154 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0dac06be30 0x7f0dac06e2f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc034000 tx=0x7f0dbc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:07.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.101+0000 7f0dc8ff9640 1 -- 172.21.15.112:0/4087388154 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0dcc14b4d0 con 0x7f0dcc1596a0 2024-04-25T01:41:07.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.274+0000 7f0dd1df9640 1 -- 172.21.15.112:0/4087388154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0dcc0682a0 con 0x7f0dcc1596a0 2024-04-25T01:41:07.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.276+0000 7f0dc8ff9640 1 -- 172.21.15.112:0/4087388154 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+20616 (secure 0 0 0) 0x7f0dc4058eb0 con 0x7f0dcc1596a0 2024-04-25T01:41:07.262 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:41:07.262 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:26:52.110548+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-25T01:25:59.478275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-25T01:26:00.719358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:41:07.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.278+0000 7f0dd1df9640 1 -- 172.21.15.112:0/4087388154 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0dac06be30 msgr2=0x7f0dac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:07.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.278+0000 7f0dd1df9640 1 --2- 172.21.15.112:0/4087388154 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0dac06be30 0x7f0dac06e2f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc034000 tx=0x7f0dbc02e000 comp rx=0 tx=0).stop 2024-04-25T01:41:07.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.278+0000 7f0dd1df9640 1 -- 172.21.15.112:0/4087388154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dcc1596a0 msgr2=0x7f0dcc14a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:07.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.278+0000 7f0dd1df9640 1 --2- 172.21.15.112:0/4087388154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dcc1596a0 0x7f0dcc14a050 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400c7b0 tx=0x7f0dc400cc70 comp rx=0 tx=0).stop 2024-04-25T01:41:07.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.278+0000 7f0dd1df9640 1 -- 172.21.15.112:0/4087388154 shutdown_connections 2024-04-25T01:41:07.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.278+0000 7f0dd1df9640 1 --2- 172.21.15.112:0/4087388154 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f0dac06be30 0x7f0dac06e2f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.278+0000 7f0dd1df9640 1 --2- 172.21.15.112:0/4087388154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dcc1596a0 0x7f0dcc14a050 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.278+0000 7f0dd1df9640 1 --2- 172.21.15.112:0/4087388154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dcc1592c0 0x7f0dcc0c0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.278+0000 7f0dd1df9640 1 --2- 172.21.15.112:0/4087388154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0dcc0c72b0 0x7f0dcc1483a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.278+0000 7f0dd1df9640 1 -- 172.21.15.112:0/4087388154 >> 172.21.15.112:0/4087388154 conn(0x7f0dcc0cb170 msgr2=0x7f0dcc0ab660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:07.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.278+0000 7f0dd1df9640 1 -- 172.21.15.112:0/4087388154 shutdown_connections 2024-04-25T01:41:07.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.278+0000 7f0dd1df9640 1 -- 172.21.15.112:0/4087388154 wait complete. 2024-04-25T01:41:07.276 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:25:40.679119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-04-25T01:25:50.593045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-04-25T01:25:51.664930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-04-25T01:25:59.478275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-04-25T01:26:00.719358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:41:07.276 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:41:07.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.390+0000 7f618f5d5640 1 Processor -- start 2024-04-25T01:41:07.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.390+0000 7f618f5d5640 1 -- start start 2024-04-25T01:41:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.391+0000 7f618f5d5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f61880c9250 0x7f61880c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.391+0000 7f618f5d5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f61880ca8a0 0x7f618806c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.391+0000 7f618f5d5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61881534a0 con 0x7f61880ca370 2024-04-25T01:41:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.391+0000 7f618f5d5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f6188156990 con 0x7f61880c9250 2024-04-25T01:41:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.391+0000 7f618f5d5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6188156500 con 0x7f61880ca8a0 2024-04-25T01:41:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.391+0000 7f618d371640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f61880c9250 0x7f61880c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.391+0000 7f618cb70640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f61880ca370 0x7f618806bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:37974/0 (socket says 172.21.15.112:37974) 2024-04-25T01:41:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.391+0000 7f618cb70640 1 -- 172.21.15.112:0/68414837 learned_addr learned my addr 172.21.15.112:0/68414837 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.392+0000 7f618d371640 1 -- 172.21.15.112:0/68414837 >> v2:172.21.15.187:3300/0 conn(0x7f61880ca8a0 msgr2=0x7f618806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.392+0000 7f618d371640 1 --2- 172.21.15.112:0/68414837 >> v2:172.21.15.187:3300/0 conn(0x7f61880ca8a0 0x7f618806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.392+0000 7f618d371640 1 -- 172.21.15.112:0/68414837 >> v1:172.21.15.112:6789/0 conn(0x7f61880ca370 legacy=0x7f618806bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.392+0000 7f617ffff640 1 -- 172.21.15.112:0/68414837 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2812612166 0 0) 0x7f61881534a0 con 0x7f61880ca370 2024-04-25T01:41:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.392+0000 7f618d371640 1 -- 172.21.15.112:0/68414837 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61880cae30 con 0x7f61880c9250 2024-04-25T01:41:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.392+0000 7f618d371640 1 --2- 172.21.15.112:0/68414837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61880c9250 0x7f61880c9630 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f618806dc20 tx=0x7f618402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edf6d1f216dbf76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.392+0000 7f617ffff640 1 -- 172.21.15.112:0/68414837 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6184007740 con 0x7f61880c9250 2024-04-25T01:41:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.392+0000 7f617ffff640 1 -- 172.21.15.112:0/68414837 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61840078e0 con 0x7f61880c9250 2024-04-25T01:41:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.392+0000 7f617ffff640 1 -- 172.21.15.112:0/68414837 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6184007cb0 con 0x7f61880c9250 2024-04-25T01:41:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.393+0000 7f618f5d5640 1 -- 172.21.15.112:0/68414837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61880c9250 msgr2=0x7f61880c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.393+0000 7f618f5d5640 1 --2- 172.21.15.112:0/68414837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61880c9250 0x7f61880c9630 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f618806dc20 tx=0x7f618402a5f0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.393+0000 7f618f5d5640 1 -- 172.21.15.112:0/68414837 shutdown_connections 2024-04-25T01:41:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.393+0000 7f618f5d5640 1 --2- 172.21.15.112:0/68414837 >> v2:172.21.15.187:3300/0 conn(0x7f61880ca8a0 0x7f618806c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.393+0000 7f618f5d5640 1 --2- 172.21.15.112:0/68414837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61880c9250 0x7f61880c9630 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.393+0000 7f618f5d5640 1 -- 172.21.15.112:0/68414837 >> 172.21.15.112:0/68414837 conn(0x7f61880cb310 msgr2=0x7f61880cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.393+0000 7f618f5d5640 1 -- 172.21.15.112:0/68414837 shutdown_connections 2024-04-25T01:41:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.393+0000 7f618f5d5640 1 -- 172.21.15.112:0/68414837 wait complete. 2024-04-25T01:41:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.394+0000 7f618f5d5640 1 Processor -- start 2024-04-25T01:41:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.394+0000 7f618f5d5640 1 -- start start 2024-04-25T01:41:07.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.395+0000 7f618f5d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61880c9250 0x7f61881ec340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.395+0000 7f618f5d5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61880ca370 0x7f61880c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.395+0000 7f618f5d5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61880ca8a0 0x7f6188157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.395+0000 7f618f5d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6188155eb0 con 0x7f61880c9250 2024-04-25T01:41:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.395+0000 7f618f5d5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6188155bd0 con 0x7f61880ca8a0 2024-04-25T01:41:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.395+0000 7f618f5d5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6188156190 con 0x7f61880ca370 2024-04-25T01:41:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.395+0000 7f618d371640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61880c9250 0x7f61881ec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.395+0000 7f618d371640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61880c9250 0x7f61881ec340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34760/0 (socket says 172.21.15.112:34760) 2024-04-25T01:41:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.395+0000 7f618d371640 1 -- 172.21.15.112:0/2429741375 learned_addr learned my addr 172.21.15.112:0/2429741375 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.395+0000 7f618d371640 1 -- 172.21.15.112:0/2429741375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61880ca370 msgr2=0x7f61880c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.395+0000 7f618d371640 1 --2- 172.21.15.112:0/2429741375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61880ca370 0x7f61880c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.395+0000 7f618d371640 1 -- 172.21.15.112:0/2429741375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61880ca8a0 msgr2=0x7f6188157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.395+0000 7f618d371640 1 --2- 172.21.15.112:0/2429741375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61880ca8a0 0x7f6188157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.395+0000 7f618d371640 1 -- 172.21.15.112:0/2429741375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61880c1960 con 0x7f61880c9250 2024-04-25T01:41:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.395+0000 7f618d371640 1 --2- 172.21.15.112:0/2429741375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61880c9250 0x7f61881ec340 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6184009e80 tx=0x7f6184004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.395+0000 7f617dffb640 1 -- 172.21.15.112:0/2429741375 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6184040070 con 0x7f61880c9250 2024-04-25T01:41:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.395+0000 7f617dffb640 1 -- 172.21.15.112:0/2429741375 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6184031c40 con 0x7f61880c9250 2024-04-25T01:41:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.395+0000 7f618f5d5640 1 -- 172.21.15.112:0/2429741375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61880c1c40 con 0x7f61880c9250 2024-04-25T01:41:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.395+0000 7f617dffb640 1 -- 172.21.15.112:0/2429741375 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618403c420 con 0x7f61880c9250 2024-04-25T01:41:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.396+0000 7f618f5d5640 1 -- 172.21.15.112:0/2429741375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61880c4b80 con 0x7f61880c9250 2024-04-25T01:41:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.397+0000 7f618f5d5640 1 -- 172.21.15.112:0/2429741375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61880cbab0 con 0x7f61880c9250 2024-04-25T01:41:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.397+0000 7f617dffb640 1 -- 172.21.15.112:0/2429741375 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6184039040 con 0x7f61880c9250 2024-04-25T01:41:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.397+0000 7f617dffb640 1 --2- 172.21.15.112:0/2429741375 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f616c06be30 0x7f616c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.398+0000 7f617dffb640 1 -- 172.21.15.112:0/2429741375 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f61840ac910 con 0x7f61880c9250 2024-04-25T01:41:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.398+0000 7f618cb70640 1 --2- 172.21.15.112:0/2429741375 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f616c06be30 0x7f616c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.398+0000 7f618cb70640 1 --2- 172.21.15.112:0/2429741375 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f616c06be30 0x7f616c06e2f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6178007fd0 tx=0x7f6178007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:07.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.402+0000 7f617dffb640 1 -- 172.21.15.112:0/2429741375 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f61880cbab0 con 0x7f61880c9250 2024-04-25T01:41:07.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.574+0000 7f618f5d5640 1 -- 172.21.15.112:0/2429741375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f618804f500 con 0x7f61880c9250 2024-04-25T01:41:07.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.574+0000 7f617dffb640 1 -- 172.21.15.112:0/2429741375 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f618407afb0 con 0x7f61880c9250 2024-04-25T01:41:07.560 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T01:41:07.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.576+0000 7f618f5d5640 1 -- 172.21.15.112:0/2429741375 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f616c06be30 msgr2=0x7f616c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:07.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.576+0000 7f618f5d5640 1 --2- 172.21.15.112:0/2429741375 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f616c06be30 0x7f616c06e2f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6178007fd0 tx=0x7f6178007920 comp rx=0 tx=0).stop 2024-04-25T01:41:07.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.576+0000 7f618f5d5640 1 -- 172.21.15.112:0/2429741375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61880c9250 msgr2=0x7f61881ec340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:07.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.576+0000 7f618f5d5640 1 --2- 172.21.15.112:0/2429741375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61880c9250 0x7f61881ec340 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6184009e80 tx=0x7f6184004a40 comp rx=0 tx=0).stop 2024-04-25T01:41:07.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.576+0000 7f618f5d5640 1 -- 172.21.15.112:0/2429741375 shutdown_connections 2024-04-25T01:41:07.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.576+0000 7f618f5d5640 1 --2- 172.21.15.112:0/2429741375 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f616c06be30 0x7f616c06e2f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.576+0000 7f618f5d5640 1 --2- 172.21.15.112:0/2429741375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61880ca8a0 0x7f6188157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.576+0000 7f618f5d5640 1 --2- 172.21.15.112:0/2429741375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61880ca370 0x7f61880c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.576+0000 7f618f5d5640 1 --2- 172.21.15.112:0/2429741375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61880c9250 0x7f61881ec340 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.576+0000 7f618f5d5640 1 -- 172.21.15.112:0/2429741375 >> 172.21.15.112:0/2429741375 conn(0x7f61880cb310 msgr2=0x7f61881553d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:07.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.576+0000 7f618f5d5640 1 -- 172.21.15.112:0/2429741375 shutdown_connections 2024-04-25T01:41:07.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.576+0000 7f618f5d5640 1 -- 172.21.15.112:0/2429741375 wait complete. 2024-04-25T01:41:07.574 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-04-25T01:41:07.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.687+0000 7fcb88ebd640 1 Processor -- start 2024-04-25T01:41:07.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.687+0000 7fcb88ebd640 1 -- start start 2024-04-25T01:41:07.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.688+0000 7fcb88ebd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb840726d0 0x7fcb84070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.688+0000 7fcb88ebd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcb84157600 0x7fcb840c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.688+0000 7fcb88ebd640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb84157ab0 con 0x7fcb84150c80 2024-04-25T01:41:07.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.688+0000 7fcb88ebd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcb84072b60 con 0x7fcb84157600 2024-04-25T01:41:07.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.688+0000 7fcb88ebd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcb84072e40 con 0x7fcb840726d0 2024-04-25T01:41:07.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.688+0000 7fcb81d9b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb840726d0 0x7fcb84070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:07.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.688+0000 7fcb82d9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcb84157600 0x7fcb840c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:07.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.688+0000 7fcb82d9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcb84157600 0x7fcb840c2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43682/0 (socket says 172.21.15.112:43682) 2024-04-25T01:41:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.688+0000 7fcb81d9b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb840726d0 0x7fcb84070cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:34492/0 (socket says 172.21.15.112:34492) 2024-04-25T01:41:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.688+0000 7fcb82d9d640 1 -- 172.21.15.112:0/3658869215 learned_addr learned my addr 172.21.15.112:0/3658869215 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.688+0000 7fcb82d9d640 1 -- 172.21.15.112:0/3658869215 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb840726d0 msgr2=0x7fcb84070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.688+0000 7fcb82d9d640 1 --2- 172.21.15.112:0/3658869215 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb840726d0 0x7fcb84070cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.688+0000 7fcb82d9d640 1 -- 172.21.15.112:0/3658869215 >> v1:172.21.15.187:6789/0 conn(0x7fcb84150c80 legacy=0x7fcb840ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.688+0000 7fcb82d9d640 1 -- 172.21.15.112:0/3658869215 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb84073120 con 0x7fcb84157600 2024-04-25T01:41:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.688+0000 7fcb81d9b640 1 --2- 172.21.15.112:0/3658869215 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb840726d0 0x7fcb84070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:41:07.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.689+0000 7fcb82d9d640 1 --2- 172.21.15.112:0/3658869215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb84157600 0x7fcb840c2920 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009870 tx=0x7fcb7c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d35e81e875d13ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:07.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.689+0000 7fcb8159a640 1 -- 172.21.15.112:0/3658869215 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c037070 con 0x7fcb84157600 2024-04-25T01:41:07.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.689+0000 7fcb8159a640 1 -- 172.21.15.112:0/3658869215 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb7c032440 con 0x7fcb84157600 2024-04-25T01:41:07.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.689+0000 7fcb8159a640 1 -- 172.21.15.112:0/3658869215 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c032810 con 0x7fcb84157600 2024-04-25T01:41:07.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.690+0000 7fcb88ebd640 1 -- 172.21.15.112:0/3658869215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb84157600 msgr2=0x7fcb840c2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:07.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.690+0000 7fcb88ebd640 1 --2- 172.21.15.112:0/3658869215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb84157600 0x7fcb840c2920 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009870 tx=0x7fcb7c02a1c0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.690+0000 7fcb88ebd640 1 -- 172.21.15.112:0/3658869215 shutdown_connections 2024-04-25T01:41:07.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.690+0000 7fcb88ebd640 1 --2- 172.21.15.112:0/3658869215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb84157600 0x7fcb840c2920 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.690+0000 7fcb88ebd640 1 --2- 172.21.15.112:0/3658869215 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb840726d0 0x7fcb84070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.690+0000 7fcb88ebd640 1 -- 172.21.15.112:0/3658869215 >> 172.21.15.112:0/3658869215 conn(0x7fcb840c3a10 msgr2=0x7fcb840c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:07.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.690+0000 7fcb88ebd640 1 -- 172.21.15.112:0/3658869215 shutdown_connections 2024-04-25T01:41:07.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.690+0000 7fcb88ebd640 1 -- 172.21.15.112:0/3658869215 wait complete. 2024-04-25T01:41:07.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.691+0000 7fcb88ebd640 1 Processor -- start 2024-04-25T01:41:07.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.691+0000 7fcb88ebd640 1 -- start start 2024-04-25T01:41:07.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.692+0000 7fcb88ebd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb84150c80 0x7fcb841ecc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.692+0000 7fcb88ebd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb84157600 0x7fcb841514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.692+0000 7fcb88ebd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb840c2c50 0x7fcb84151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.692+0000 7fcb88ebd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb840ab8f0 con 0x7fcb840c2c50 2024-04-25T01:41:07.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.692+0000 7fcb88ebd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcb840ac400 con 0x7fcb84157600 2024-04-25T01:41:07.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.692+0000 7fcb88ebd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcb84151060 con 0x7fcb84150c80 2024-04-25T01:41:07.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.692+0000 7fcb82d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb840c2c50 0x7fcb84151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:07.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.692+0000 7fcb82d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb840c2c50 0x7fcb84151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56356/0 (socket says 172.21.15.112:56356) 2024-04-25T01:41:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.692+0000 7fcb82d9d640 1 -- 172.21.15.112:0/1579438726 learned_addr learned my addr 172.21.15.112:0/1579438726 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.692+0000 7fcb8259c640 1 --2- 172.21.15.112:0/1579438726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb84150c80 0x7fcb841ecc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.692+0000 7fcb81d9b640 1 --2- 172.21.15.112:0/1579438726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb84157600 0x7fcb841514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.692+0000 7fcb82d9d640 1 -- 172.21.15.112:0/1579438726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb84150c80 msgr2=0x7fcb841ecc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.692+0000 7fcb82d9d640 1 --2- 172.21.15.112:0/1579438726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb84150c80 0x7fcb841ecc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.692+0000 7fcb82d9d640 1 -- 172.21.15.112:0/1579438726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb84157600 msgr2=0x7fcb841514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.692+0000 7fcb82d9d640 1 --2- 172.21.15.112:0/1579438726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb84157600 0x7fcb841514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.692+0000 7fcb82d9d640 1 -- 172.21.15.112:0/1579438726 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb841dd650 con 0x7fcb840c2c50 2024-04-25T01:41:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.692+0000 7fcb8259c640 1 --2- 172.21.15.112:0/1579438726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb84150c80 0x7fcb841ecc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:41:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.692+0000 7fcb81d9b640 1 --2- 172.21.15.112:0/1579438726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb84157600 0x7fcb841514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:41:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.693+0000 7fcb82d9d640 1 --2- 172.21.15.112:0/1579438726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb840c2c50 0x7fcb84151a10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c02a200 tx=0x7fcb7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.693+0000 7fcb72ffd640 1 -- 172.21.15.112:0/1579438726 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c041020 con 0x7fcb840c2c50 2024-04-25T01:41:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.693+0000 7fcb72ffd640 1 -- 172.21.15.112:0/1579438726 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb7c03d640 con 0x7fcb840c2c50 2024-04-25T01:41:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.693+0000 7fcb72ffd640 1 -- 172.21.15.112:0/1579438726 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c037040 con 0x7fcb840c2c50 2024-04-25T01:41:07.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.693+0000 7fcb88ebd640 1 -- 172.21.15.112:0/1579438726 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb841ed5e0 con 0x7fcb840c2c50 2024-04-25T01:41:07.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.693+0000 7fcb88ebd640 1 -- 172.21.15.112:0/1579438726 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb841ddbf0 con 0x7fcb840c2c50 2024-04-25T01:41:07.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.694+0000 7fcb88ebd640 1 -- 172.21.15.112:0/1579438726 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb8404f500 con 0x7fcb840c2c50 2024-04-25T01:41:07.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.694+0000 7fcb72ffd640 1 -- 172.21.15.112:0/1579438726 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcb7c044020 con 0x7fcb840c2c50 2024-04-25T01:41:07.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.695+0000 7fcb72ffd640 1 --2- 172.21.15.112:0/1579438726 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcb6406be30 0x7fcb6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.695+0000 7fcb72ffd640 1 -- 172.21.15.112:0/1579438726 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fcb7c0ab5b0 con 0x7fcb840c2c50 2024-04-25T01:41:07.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.695+0000 7fcb8259c640 1 --2- 172.21.15.112:0/1579438726 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcb6406be30 0x7fcb6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:07.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.695+0000 7fcb8259c640 1 --2- 172.21.15.112:0/1579438726 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcb6406be30 0x7fcb6406e2f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcb74001d70 tx=0x7fcb74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:07.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.699+0000 7fcb72ffd640 1 -- 172.21.15.112:0/1579438726 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcb7c04b070 con 0x7fcb840c2c50 2024-04-25T01:41:07.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.872+0000 7fcb88ebd640 1 -- 172.21.15.112:0/1579438726 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) -- 0x7fcb840a9500 con 0x7fcb840c2c50 2024-04-25T01:41:07.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.872+0000 7fcb72ffd640 1 -- 172.21.15.112:0/1579438726 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v26) ==== 99+0+11 (secure 0 0 0) 0x7fcb7c079bd0 con 0x7fcb840c2c50 2024-04-25T01:41:07.859 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T01:41:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.874+0000 7fcb88ebd640 1 -- 172.21.15.112:0/1579438726 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcb6406be30 msgr2=0x7fcb6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.874+0000 7fcb88ebd640 1 --2- 172.21.15.112:0/1579438726 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcb6406be30 0x7fcb6406e2f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcb74001d70 tx=0x7fcb74009040 comp rx=0 tx=0).stop 2024-04-25T01:41:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.874+0000 7fcb88ebd640 1 -- 172.21.15.112:0/1579438726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb840c2c50 msgr2=0x7fcb84151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.874+0000 7fcb88ebd640 1 --2- 172.21.15.112:0/1579438726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb840c2c50 0x7fcb84151a10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c02a200 tx=0x7fcb7c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.874+0000 7fcb88ebd640 1 -- 172.21.15.112:0/1579438726 shutdown_connections 2024-04-25T01:41:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.874+0000 7fcb88ebd640 1 --2- 172.21.15.112:0/1579438726 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fcb6406be30 0x7fcb6406e2f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.874+0000 7fcb88ebd640 1 --2- 172.21.15.112:0/1579438726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb840c2c50 0x7fcb84151a10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.874+0000 7fcb88ebd640 1 --2- 172.21.15.112:0/1579438726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb84157600 0x7fcb841514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.874+0000 7fcb88ebd640 1 --2- 172.21.15.112:0/1579438726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb84150c80 0x7fcb841ecc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.874+0000 7fcb88ebd640 1 -- 172.21.15.112:0/1579438726 >> 172.21.15.112:0/1579438726 conn(0x7fcb840c3a10 msgr2=0x7fcb840c6490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.874+0000 7fcb88ebd640 1 -- 172.21.15.112:0/1579438726 shutdown_connections 2024-04-25T01:41:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.874+0000 7fcb88ebd640 1 -- 172.21.15.112:0/1579438726 wait complete. 2024-04-25T01:41:07.872 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-04-25T01:41:07.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.986+0000 7fc467695640 1 Processor -- start 2024-04-25T01:41:07.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.986+0000 7fc467695640 1 -- start start 2024-04-25T01:41:07.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.987+0000 7fc467695640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc4600c9250 0x7fc4600c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.987+0000 7fc467695640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4600ca370 0x7fc46006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.987+0000 7fc467695640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4601534a0 con 0x7fc4600ca8a0 2024-04-25T01:41:07.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.987+0000 7fc467695640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fc460156990 con 0x7fc4600c9250 2024-04-25T01:41:07.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.987+0000 7fc467695640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc460156500 con 0x7fc4600ca370 2024-04-25T01:41:07.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.987+0000 7fc465431640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc4600c9250 0x7fc4600c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:07.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.987+0000 7fc465431640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc4600c9250 0x7fc4600c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56366/0 (socket says 172.21.15.112:56366) 2024-04-25T01:41:07.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.987+0000 7fc465431640 1 -- 172.21.15.112:0/2043056607 learned_addr learned my addr 172.21.15.112:0/2043056607 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:07.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.988+0000 7fc465431640 1 -- 172.21.15.112:0/2043056607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4600ca370 msgr2=0x7fc46006bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:07.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.988+0000 7fc465431640 1 --2- 172.21.15.112:0/2043056607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4600ca370 0x7fc46006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.988+0000 7fc465431640 1 -- 172.21.15.112:0/2043056607 >> v1:172.21.15.187:6789/0 conn(0x7fc4600ca8a0 legacy=0x7fc46006c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:07.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.988+0000 7fc465431640 1 -- 172.21.15.112:0/2043056607 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4600cae30 con 0x7fc4600c9250 2024-04-25T01:41:07.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.988+0000 7fc465431640 1 --2- 172.21.15.112:0/2043056607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4600c9250 0x7fc4600c9630 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009e50 tx=0x7fc45c02a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42f8e024b6cc82e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:07.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.988+0000 7fc457fff640 1 -- 172.21.15.112:0/2043056607 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c036070 con 0x7fc4600c9250 2024-04-25T01:41:07.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.988+0000 7fc457fff640 1 -- 172.21.15.112:0/2043056607 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc45c031600 con 0x7fc4600c9250 2024-04-25T01:41:07.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.988+0000 7fc457fff640 1 -- 172.21.15.112:0/2043056607 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0319d0 con 0x7fc4600c9250 2024-04-25T01:41:07.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.989+0000 7fc467695640 1 -- 172.21.15.112:0/2043056607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4600c9250 msgr2=0x7fc4600c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:07.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.989+0000 7fc467695640 1 --2- 172.21.15.112:0/2043056607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4600c9250 0x7fc4600c9630 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009e50 tx=0x7fc45c02a5b0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.989+0000 7fc467695640 1 -- 172.21.15.112:0/2043056607 shutdown_connections 2024-04-25T01:41:07.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.989+0000 7fc467695640 1 --2- 172.21.15.112:0/2043056607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4600ca370 0x7fc46006bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.989+0000 7fc467695640 1 --2- 172.21.15.112:0/2043056607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4600c9250 0x7fc4600c9630 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.989+0000 7fc467695640 1 -- 172.21.15.112:0/2043056607 >> 172.21.15.112:0/2043056607 conn(0x7fc4600cb310 msgr2=0x7fc4600cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:07.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.989+0000 7fc467695640 1 -- 172.21.15.112:0/2043056607 shutdown_connections 2024-04-25T01:41:07.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.989+0000 7fc467695640 1 -- 172.21.15.112:0/2043056607 wait complete. 2024-04-25T01:41:07.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.990+0000 7fc467695640 1 Processor -- start 2024-04-25T01:41:07.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.990+0000 7fc467695640 1 -- start start 2024-04-25T01:41:07.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.991+0000 7fc467695640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4600c9250 0x7fc460150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.991+0000 7fc467695640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4600ca370 0x7fc4600c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.991+0000 7fc467695640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4600ca8a0 0x7fc4600c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.991+0000 7fc467695640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc460155eb0 con 0x7fc4600c9250 2024-04-25T01:41:07.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.991+0000 7fc467695640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc460155bd0 con 0x7fc4600ca8a0 2024-04-25T01:41:07.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.991+0000 7fc467695640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc460156190 con 0x7fc4600ca370 2024-04-25T01:41:07.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.991+0000 7fc465431640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4600c9250 0x7fc460150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:07.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.991+0000 7fc465431640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4600c9250 0x7fc460150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56380/0 (socket says 172.21.15.112:56380) 2024-04-25T01:41:07.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.991+0000 7fc465431640 1 -- 172.21.15.112:0/2660459853 learned_addr learned my addr 172.21.15.112:0/2660459853 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:07.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.991+0000 7fc465431640 1 -- 172.21.15.112:0/2660459853 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4600ca370 msgr2=0x7fc4600c37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:07.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.991+0000 7fc465431640 1 --2- 172.21.15.112:0/2660459853 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4600ca370 0x7fc4600c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.991+0000 7fc465431640 1 -- 172.21.15.112:0/2660459853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4600ca8a0 msgr2=0x7fc4600c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:07.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.991+0000 7fc465431640 1 --2- 172.21.15.112:0/2660459853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4600ca8a0 0x7fc4600c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:07.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.991+0000 7fc465431640 1 -- 172.21.15.112:0/2660459853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4600c4260 con 0x7fc4600c9250 2024-04-25T01:41:07.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.991+0000 7fc465431640 1 --2- 172.21.15.112:0/2660459853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4600c9250 0x7fc460150a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc45c000f80 tx=0x7fc45c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:07.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.991+0000 7fc455ffb640 1 -- 172.21.15.112:0/2660459853 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c041020 con 0x7fc4600c9250 2024-04-25T01:41:07.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.992+0000 7fc455ffb640 1 -- 172.21.15.112:0/2660459853 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc45c007800 con 0x7fc4600c9250 2024-04-25T01:41:07.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.992+0000 7fc467695640 1 -- 172.21.15.112:0/2660459853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4601505b0 con 0x7fc4600c9250 2024-04-25T01:41:07.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.992+0000 7fc455ffb640 1 -- 172.21.15.112:0/2660459853 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c036040 con 0x7fc4600c9250 2024-04-25T01:41:07.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.992+0000 7fc467695640 1 -- 172.21.15.112:0/2660459853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc46014f7f0 con 0x7fc4600c9250 2024-04-25T01:41:07.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.993+0000 7fc467695640 1 -- 172.21.15.112:0/2660459853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc46004f500 con 0x7fc4600c9250 2024-04-25T01:41:07.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.993+0000 7fc455ffb640 1 -- 172.21.15.112:0/2660459853 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc45c044020 con 0x7fc4600c9250 2024-04-25T01:41:07.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.993+0000 7fc455ffb640 1 --2- 172.21.15.112:0/2660459853 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc44406be30 0x7fc44406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:07.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.993+0000 7fc455ffb640 1 -- 172.21.15.112:0/2660459853 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fc45c0abfa0 con 0x7fc4600c9250 2024-04-25T01:41:07.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.994+0000 7fc464c30640 1 --2- 172.21.15.112:0/2660459853 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc44406be30 0x7fc44406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:07.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.994+0000 7fc464c30640 1 --2- 172.21.15.112:0/2660459853 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc44406be30 0x7fc44406e2f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc46014ee30 tx=0x7fc450006c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:07.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:07.998+0000 7fc455ffb640 1 -- 172.21.15.112:0/2660459853 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc45c04b030 con 0x7fc4600c9250 2024-04-25T01:41:08.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.169+0000 7fc467695640 1 -- 172.21.15.112:0/2660459853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) -- 0x7fc4600c9030 con 0x7fc4600c9250 2024-04-25T01:41:08.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.169+0000 7fc455ffb640 1 -- 172.21.15.112:0/2660459853 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v26) ==== 95+0+11 (secure 0 0 0) 0x7fc45c07a640 con 0x7fc4600c9250 2024-04-25T01:41:08.156 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T01:41:08.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.171+0000 7fc467695640 1 -- 172.21.15.112:0/2660459853 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc44406be30 msgr2=0x7fc44406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:08.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.171+0000 7fc467695640 1 --2- 172.21.15.112:0/2660459853 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc44406be30 0x7fc44406e2f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc46014ee30 tx=0x7fc450006c90 comp rx=0 tx=0).stop 2024-04-25T01:41:08.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.171+0000 7fc467695640 1 -- 172.21.15.112:0/2660459853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4600c9250 msgr2=0x7fc460150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:08.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.171+0000 7fc467695640 1 --2- 172.21.15.112:0/2660459853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4600c9250 0x7fc460150a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc45c000f80 tx=0x7fc45c004a40 comp rx=0 tx=0).stop 2024-04-25T01:41:08.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.171+0000 7fc467695640 1 -- 172.21.15.112:0/2660459853 shutdown_connections 2024-04-25T01:41:08.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.171+0000 7fc467695640 1 --2- 172.21.15.112:0/2660459853 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fc44406be30 0x7fc44406e2f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.171+0000 7fc467695640 1 --2- 172.21.15.112:0/2660459853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4600ca8a0 0x7fc4600c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.171+0000 7fc467695640 1 --2- 172.21.15.112:0/2660459853 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4600ca370 0x7fc4600c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.171+0000 7fc467695640 1 --2- 172.21.15.112:0/2660459853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4600c9250 0x7fc460150a60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.171+0000 7fc467695640 1 -- 172.21.15.112:0/2660459853 >> 172.21.15.112:0/2660459853 conn(0x7fc4600cb310 msgr2=0x7fc4600c59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:08.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.171+0000 7fc467695640 1 -- 172.21.15.112:0/2660459853 shutdown_connections 2024-04-25T01:41:08.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.171+0000 7fc467695640 1 -- 172.21.15.112:0/2660459853 wait complete. 2024-04-25T01:41:08.169 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-04-25T01:41:08.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.282+0000 7f1e78763640 1 Processor -- start 2024-04-25T01:41:08.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.282+0000 7f1e78763640 1 -- start start 2024-04-25T01:41:08.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.283+0000 7f1e78763640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e7014fbe0 0x7f1e700c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.283+0000 7f1e78763640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1e700c2e60 0x7f1e700726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.283+0000 7f1e78763640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e7006f290 con 0x7f1e7006d460 2024-04-25T01:41:08.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.283+0000 7f1e78763640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1e7014ffc0 con 0x7f1e700c2e60 2024-04-25T01:41:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.283+0000 7f1e78763640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1e701533a0 con 0x7f1e7014fbe0 2024-04-25T01:41:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.283+0000 7f1e75cfe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1e7006d460 0x7f1e70071fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:39314/0 (socket says 172.21.15.112:39314) 2024-04-25T01:41:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.283+0000 7f1e75cfe640 1 -- 172.21.15.112:0/3866325504 learned_addr learned my addr 172.21.15.112:0/3866325504 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.283+0000 7f1e764ff640 1 --2- 172.21.15.112:0/3866325504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e7014fbe0 0x7f1e700c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.283+0000 7f1e76d00640 1 --2- 172.21.15.112:0/3866325504 >> v2:172.21.15.187:3300/0 conn(0x7f1e700c2e60 0x7f1e700726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.283+0000 7f1e754fd640 1 -- 172.21.15.112:0/3866325504 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3384995465 0 0) 0x7f1e7006f290 con 0x7f1e7006d460 2024-04-25T01:41:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.283+0000 7f1e754fd640 1 -- 172.21.15.112:0/3866325504 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e5c003520 con 0x7f1e7006d460 2024-04-25T01:41:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.283+0000 7f1e754fd640 1 -- 172.21.15.112:0/3866325504 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1667855118 0 0) 0x7f1e5c003520 con 0x7f1e7006d460 2024-04-25T01:41:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.283+0000 7f1e754fd640 1 -- 172.21.15.112:0/3866325504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e7014fbe0 msgr2=0x7f1e700c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.283+0000 7f1e754fd640 1 --2- 172.21.15.112:0/3866325504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e7014fbe0 0x7f1e700c2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.283+0000 7f1e754fd640 1 -- 172.21.15.112:0/3866325504 >> v2:172.21.15.187:3300/0 conn(0x7f1e700c2e60 msgr2=0x7f1e700726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.283+0000 7f1e754fd640 1 --2- 172.21.15.112:0/3866325504 >> v2:172.21.15.187:3300/0 conn(0x7f1e700c2e60 0x7f1e700726c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.283+0000 7f1e754fd640 1 -- 172.21.15.112:0/3866325504 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7006efb0 con 0x7f1e7006d460 2024-04-25T01:41:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.283+0000 7f1e754fd640 1 -- 172.21.15.112:0/3866325504 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f1e60001670 con 0x7f1e7006d460 2024-04-25T01:41:08.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.284+0000 7f1e754fd640 1 -- 172.21.15.112:0/3866325504 >> v1:172.21.15.112:6789/0 conn(0x7f1e7006d460 legacy=0x7f1e70071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:08.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.284+0000 7f1e754fd640 1 --2- 172.21.15.112:0/3866325504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c0039b0 0x7f1e5c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.284+0000 7f1e754fd640 1 --2- 172.21.15.112:0/3866325504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e7014fbe0 0x7f1e5c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.284+0000 7f1e754fd640 1 --2- 172.21.15.112:0/3866325504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e5c004d40 0x7f1e5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.284+0000 7f1e754fd640 1 -- 172.21.15.112:0/3866325504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e701533a0 con 0x7f1e5c0039b0 2024-04-25T01:41:08.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.284+0000 7f1e754fd640 1 -- 172.21.15.112:0/3866325504 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c00b9c0 con 0x7f1e5c004d40 2024-04-25T01:41:08.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.284+0000 7f1e754fd640 1 -- 172.21.15.112:0/3866325504 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5c00bb40 con 0x7f1e7014fbe0 2024-04-25T01:41:08.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.284+0000 7f1e764ff640 1 --2- 172.21.15.112:0/3866325504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c0039b0 0x7f1e5c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:08.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.284+0000 7f1e75cfe640 1 --2- 172.21.15.112:0/3866325504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e7014fbe0 0x7f1e5c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:08.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.284+0000 7f1e76d00640 1 --2- 172.21.15.112:0/3866325504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e5c004d40 0x7f1e5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:08.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.284+0000 7f1e764ff640 1 -- 172.21.15.112:0/3866325504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e7014fbe0 msgr2=0x7f1e5c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:08.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.284+0000 7f1e764ff640 1 --2- 172.21.15.112:0/3866325504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e7014fbe0 0x7f1e5c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.284+0000 7f1e764ff640 1 -- 172.21.15.112:0/3866325504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e5c004d40 msgr2=0x7f1e5c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:08.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.284+0000 7f1e764ff640 1 --2- 172.21.15.112:0/3866325504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e5c004d40 0x7f1e5c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.284+0000 7f1e764ff640 1 -- 172.21.15.112:0/3866325504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5c003520 con 0x7f1e5c0039b0 2024-04-25T01:41:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.285+0000 7f1e764ff640 1 --2- 172.21.15.112:0/3866325504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c0039b0 0x7f1e5c003dd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009fc0 tx=0x7f1e5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62624421b9468358 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.285+0000 7f1e754fd640 1 -- 172.21.15.112:0/3866325504 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c009c10 con 0x7f1e5c0039b0 2024-04-25T01:41:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.285+0000 7f1e754fd640 1 -- 172.21.15.112:0/3866325504 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e6c036070 con 0x7f1e5c0039b0 2024-04-25T01:41:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.285+0000 7f1e754fd640 1 -- 172.21.15.112:0/3866325504 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c031880 con 0x7f1e5c0039b0 2024-04-25T01:41:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.285+0000 7f1e78763640 1 -- 172.21.15.112:0/3866325504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c0039b0 msgr2=0x7f1e5c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.285+0000 7f1e78763640 1 --2- 172.21.15.112:0/3866325504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c0039b0 0x7f1e5c003dd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009fc0 tx=0x7f1e5c005830 comp rx=0 tx=0).stop 2024-04-25T01:41:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.285+0000 7f1e76d00640 1 -- 172.21.15.112:0/3866325504 reap_dead start 2024-04-25T01:41:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.285+0000 7f1e78763640 1 -- 172.21.15.112:0/3866325504 shutdown_connections 2024-04-25T01:41:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.285+0000 7f1e78763640 1 -- 172.21.15.112:0/3866325504 >> 172.21.15.112:0/3866325504 conn(0x7f1e700c3a10 msgr2=0x7f1e700c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.285+0000 7f1e78763640 1 -- 172.21.15.112:0/3866325504 shutdown_connections 2024-04-25T01:41:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.285+0000 7f1e78763640 1 -- 172.21.15.112:0/3866325504 wait complete. 2024-04-25T01:41:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.286+0000 7f1e78763640 1 Processor -- start 2024-04-25T01:41:08.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.286+0000 7f1e78763640 1 -- start start 2024-04-25T01:41:08.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.287+0000 7f1e78763640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c0039b0 0x7f1e700c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.287+0000 7f1e78763640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e700ac4d0 0x7f1e701438b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.287+0000 7f1e78763640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e700ac950 0x7f1e70143df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.287+0000 7f1e78763640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e70072c00 con 0x7f1e5c0039b0 2024-04-25T01:41:08.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.287+0000 7f1e78763640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1e70153030 con 0x7f1e700ac4d0 2024-04-25T01:41:08.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.287+0000 7f1e78763640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1e700c3700 con 0x7f1e700ac950 2024-04-25T01:41:08.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.287+0000 7f1e764ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c0039b0 0x7f1e700c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:08.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.287+0000 7f1e764ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c0039b0 0x7f1e700c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56402/0 (socket says 172.21.15.112:56402) 2024-04-25T01:41:08.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.287+0000 7f1e764ff640 1 -- 172.21.15.112:0/1346820775 learned_addr learned my addr 172.21.15.112:0/1346820775 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:08.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.287+0000 7f1e75cfe640 1 --2- 172.21.15.112:0/1346820775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e700ac4d0 0x7f1e701438b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:08.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.287+0000 7f1e764ff640 1 -- 172.21.15.112:0/1346820775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e700ac950 msgr2=0x7f1e70143df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:08.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.287+0000 7f1e764ff640 1 --2- 172.21.15.112:0/1346820775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e700ac950 0x7f1e70143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.287+0000 7f1e764ff640 1 -- 172.21.15.112:0/1346820775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e700ac4d0 msgr2=0x7f1e701438b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:08.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.287+0000 7f1e764ff640 1 --2- 172.21.15.112:0/1346820775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e700ac4d0 0x7f1e701438b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.287+0000 7f1e764ff640 1 -- 172.21.15.112:0/1346820775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e70144330 con 0x7f1e5c0039b0 2024-04-25T01:41:08.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.288+0000 7f1e764ff640 1 --2- 172.21.15.112:0/1346820775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c0039b0 0x7f1e700c5ef0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c02aee0 tx=0x7f1e6c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:08.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.288+0000 7f1e66ffd640 1 -- 172.21.15.112:0/1346820775 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c009c10 con 0x7f1e5c0039b0 2024-04-25T01:41:08.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.288+0000 7f1e66ffd640 1 -- 172.21.15.112:0/1346820775 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e6c031e50 con 0x7f1e5c0039b0 2024-04-25T01:41:08.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.288+0000 7f1e78763640 1 -- 172.21.15.112:0/1346820775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e700ab640 con 0x7f1e5c0039b0 2024-04-25T01:41:08.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.288+0000 7f1e66ffd640 1 -- 172.21.15.112:0/1346820775 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c003bb0 con 0x7f1e5c0039b0 2024-04-25T01:41:08.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.288+0000 7f1e78763640 1 -- 172.21.15.112:0/1346820775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e700abc00 con 0x7f1e5c0039b0 2024-04-25T01:41:08.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.289+0000 7f1e78763640 1 -- 172.21.15.112:0/1346820775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e7004f500 con 0x7f1e5c0039b0 2024-04-25T01:41:08.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.289+0000 7f1e66ffd640 1 -- 172.21.15.112:0/1346820775 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e6c005d10 con 0x7f1e5c0039b0 2024-04-25T01:41:08.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.290+0000 7f1e66ffd640 1 --2- 172.21.15.112:0/1346820775 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e5406be30 0x7f1e5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.290+0000 7f1e66ffd640 1 -- 172.21.15.112:0/1346820775 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f1e6c0ac450 con 0x7f1e5c0039b0 2024-04-25T01:41:08.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.290+0000 7f1e75cfe640 1 --2- 172.21.15.112:0/1346820775 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e5406be30 0x7f1e5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:08.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.290+0000 7f1e75cfe640 1 --2- 172.21.15.112:0/1346820775 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e5406be30 0x7f1e5406e2f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1e60001eb0 tx=0x7f1e60001470 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:08.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.294+0000 7f1e66ffd640 1 -- 172.21.15.112:0/1346820775 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1e6c07aaf0 con 0x7f1e5c0039b0 2024-04-25T01:41:08.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.467+0000 7f1e78763640 1 -- 172.21.15.112:0/1346820775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) -- 0x7f1e700735c0 con 0x7f1e5c0039b0 2024-04-25T01:41:08.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.467+0000 7f1e66ffd640 1 -- 172.21.15.112:0/1346820775 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v26) ==== 99+0+11 (secure 0 0 0) 0x7f1e6c07a3f0 con 0x7f1e5c0039b0 2024-04-25T01:41:08.454 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T01:41:08.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.470+0000 7f1e78763640 1 -- 172.21.15.112:0/1346820775 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e5406be30 msgr2=0x7f1e5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:08.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.470+0000 7f1e78763640 1 --2- 172.21.15.112:0/1346820775 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e5406be30 0x7f1e5406e2f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1e60001eb0 tx=0x7f1e60001470 comp rx=0 tx=0).stop 2024-04-25T01:41:08.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.470+0000 7f1e78763640 1 -- 172.21.15.112:0/1346820775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c0039b0 msgr2=0x7f1e700c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:08.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.470+0000 7f1e78763640 1 --2- 172.21.15.112:0/1346820775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c0039b0 0x7f1e700c5ef0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c02aee0 tx=0x7f1e6c009750 comp rx=0 tx=0).stop 2024-04-25T01:41:08.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.470+0000 7f1e78763640 1 -- 172.21.15.112:0/1346820775 shutdown_connections 2024-04-25T01:41:08.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.470+0000 7f1e78763640 1 --2- 172.21.15.112:0/1346820775 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e5406be30 0x7f1e5406e2f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.470+0000 7f1e78763640 1 --2- 172.21.15.112:0/1346820775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e700ac950 0x7f1e70143df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.470+0000 7f1e78763640 1 --2- 172.21.15.112:0/1346820775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e700ac4d0 0x7f1e701438b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.470+0000 7f1e78763640 1 --2- 172.21.15.112:0/1346820775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c0039b0 0x7f1e700c5ef0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.470+0000 7f1e78763640 1 -- 172.21.15.112:0/1346820775 >> 172.21.15.112:0/1346820775 conn(0x7f1e700c3a10 msgr2=0x7f1e700c22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:08.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.470+0000 7f1e78763640 1 -- 172.21.15.112:0/1346820775 shutdown_connections 2024-04-25T01:41:08.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.470+0000 7f1e78763640 1 -- 172.21.15.112:0/1346820775 wait complete. 2024-04-25T01:41:08.467 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-04-25T01:41:08.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.580+0000 7fd7343e7640 1 Processor -- start 2024-04-25T01:41:08.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.580+0000 7fd7343e7640 1 -- start start 2024-04-25T01:41:08.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.581+0000 7fd7343e7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd72c14fbe0 0x7fd72c0c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.581+0000 7fd7343e7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd72c06f290 con 0x7fd72c06d460 2024-04-25T01:41:08.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.581+0000 7fd7343e7640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd72c06efb0 con 0x7fd72c0c2e60 2024-04-25T01:41:08.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.581+0000 7fd7343e7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd72c14ffc0 con 0x7fd72c14fbe0 2024-04-25T01:41:08.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.581+0000 7fd731982640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd72c06d460 0x7fd72c071fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:39320/0 (socket says 172.21.15.112:39320) 2024-04-25T01:41:08.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.581+0000 7fd731982640 1 -- 172.21.15.112:0/3103935564 learned_addr learned my addr 172.21.15.112:0/3103935564 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:08.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.581+0000 7fd732183640 1 --2- 172.21.15.112:0/3103935564 >> v2:172.21.15.187:3300/0 conn(0x7fd72c14fbe0 0x7fd72c0c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:08.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.581+0000 7fd731181640 1 -- 172.21.15.112:0/3103935564 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4140012617 0 0) 0x7fd72c06f290 con 0x7fd72c06d460 2024-04-25T01:41:08.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.581+0000 7fd731181640 1 -- 172.21.15.112:0/3103935564 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd718003520 con 0x7fd72c06d460 2024-04-25T01:41:08.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd731181640 1 -- 172.21.15.112:0/3103935564 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2623152339 0 0) 0x7fd718003520 con 0x7fd72c06d460 2024-04-25T01:41:08.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd731181640 1 -- 172.21.15.112:0/3103935564 >> v2:172.21.15.187:3300/0 conn(0x7fd72c14fbe0 msgr2=0x7fd72c0c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:08.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd731181640 1 --2- 172.21.15.112:0/3103935564 >> v2:172.21.15.187:3300/0 conn(0x7fd72c14fbe0 0x7fd72c0c2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd731181640 1 -- 172.21.15.112:0/3103935564 >> v1:172.21.15.187:6789/0 conn(0x7fd72c0c2e60 legacy=0x7fd72c0726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:08.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd731181640 1 -- 172.21.15.112:0/3103935564 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c0a9530 con 0x7fd72c06d460 2024-04-25T01:41:08.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd731181640 1 -- 172.21.15.112:0/3103935564 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fd71c001670 con 0x7fd72c06d460 2024-04-25T01:41:08.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd731181640 1 -- 172.21.15.112:0/3103935564 >> v1:172.21.15.112:6789/0 conn(0x7fd72c06d460 legacy=0x7fd72c071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:08.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd731181640 1 --2- 172.21.15.112:0/3103935564 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7180039b0 0x7fd718003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd731181640 1 --2- 172.21.15.112:0/3103935564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd718004800 0x7fd718004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd731181640 1 --2- 172.21.15.112:0/3103935564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7180052e0 0x7fd71800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd731181640 1 -- 172.21.15.112:0/3103935564 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd71800c180 con 0x7fd718004800 2024-04-25T01:41:08.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd731181640 1 -- 172.21.15.112:0/3103935564 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd71800c330 con 0x7fd7180052e0 2024-04-25T01:41:08.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd731181640 1 -- 172.21.15.112:0/3103935564 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd71800c530 con 0x7fd7180039b0 2024-04-25T01:41:08.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd731982640 1 --2- 172.21.15.112:0/3103935564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd718004800 0x7fd718004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:08.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd732984640 1 --2- 172.21.15.112:0/3103935564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7180052e0 0x7fd71800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:08.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd732183640 1 --2- 172.21.15.112:0/3103935564 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7180039b0 0x7fd718003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:08.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd731982640 1 -- 172.21.15.112:0/3103935564 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7180039b0 msgr2=0x7fd718003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:08.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd731982640 1 --2- 172.21.15.112:0/3103935564 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7180039b0 0x7fd718003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd731982640 1 -- 172.21.15.112:0/3103935564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7180052e0 msgr2=0x7fd71800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:08.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd731982640 1 --2- 172.21.15.112:0/3103935564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7180052e0 0x7fd71800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd732984640 1 -- 172.21.15.112:0/3103935564 reap_dead start 2024-04-25T01:41:08.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.582+0000 7fd731982640 1 -- 172.21.15.112:0/3103935564 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd718003520 con 0x7fd718004800 2024-04-25T01:41:08.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.583+0000 7fd731982640 1 --2- 172.21.15.112:0/3103935564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd718004800 0x7fd718004c80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00c210 tx=0x7fd718005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18af72506a3697be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:08.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.583+0000 7fd731181640 1 -- 172.21.15.112:0/3103935564 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c007cf0 con 0x7fd718004800 2024-04-25T01:41:08.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.583+0000 7fd731181640 1 -- 172.21.15.112:0/3103935564 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd71c005af0 con 0x7fd718004800 2024-04-25T01:41:08.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.583+0000 7fd731181640 1 -- 172.21.15.112:0/3103935564 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c007070 con 0x7fd718004800 2024-04-25T01:41:08.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.583+0000 7fd7343e7640 1 -- 172.21.15.112:0/3103935564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd718004800 msgr2=0x7fd718004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:08.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.583+0000 7fd7343e7640 1 --2- 172.21.15.112:0/3103935564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd718004800 0x7fd718004c80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00c210 tx=0x7fd718005830 comp rx=0 tx=0).stop 2024-04-25T01:41:08.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.584+0000 7fd7343e7640 1 -- 172.21.15.112:0/3103935564 shutdown_connections 2024-04-25T01:41:08.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.584+0000 7fd7343e7640 1 --2- 172.21.15.112:0/3103935564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd718004800 0x7fd718004c80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.584+0000 7fd7343e7640 1 -- 172.21.15.112:0/3103935564 >> 172.21.15.112:0/3103935564 conn(0x7fd72c0c3a10 msgr2=0x7fd72c0c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:08.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.584+0000 7fd7343e7640 1 -- 172.21.15.112:0/3103935564 shutdown_connections 2024-04-25T01:41:08.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.584+0000 7fd7343e7640 1 -- 172.21.15.112:0/3103935564 wait complete. 2024-04-25T01:41:08.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.585+0000 7fd7343e7640 1 Processor -- start 2024-04-25T01:41:08.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.585+0000 7fd7343e7640 1 -- start start 2024-04-25T01:41:08.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.585+0000 7fd7343e7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd718004800 0x7fd72c144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.585+0000 7fd7343e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd72c153900 0x7fd72c14c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.585+0000 7fd7343e7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd72c145280 0x7fd72c14cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.585+0000 7fd7343e7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c0c3700 con 0x7fd72c153900 2024-04-25T01:41:08.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.585+0000 7fd7343e7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c1533a0 con 0x7fd718004800 2024-04-25T01:41:08.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.585+0000 7fd7343e7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd72c1ec9a0 con 0x7fd72c145280 2024-04-25T01:41:08.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.585+0000 7fd731982640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd72c153900 0x7fd72c14c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:08.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.585+0000 7fd731982640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd72c153900 0x7fd72c14c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56416/0 (socket says 172.21.15.112:56416) 2024-04-25T01:41:08.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.585+0000 7fd731982640 1 -- 172.21.15.112:0/709459164 learned_addr learned my addr 172.21.15.112:0/709459164 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:08.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.586+0000 7fd732183640 1 --2- 172.21.15.112:0/709459164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd718004800 0x7fd72c144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:08.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.586+0000 7fd732984640 1 --2- 172.21.15.112:0/709459164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd72c145280 0x7fd72c14cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:08.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.586+0000 7fd731982640 1 -- 172.21.15.112:0/709459164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd72c145280 msgr2=0x7fd72c14cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:08.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.586+0000 7fd731982640 1 --2- 172.21.15.112:0/709459164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd72c145280 0x7fd72c14cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.586+0000 7fd731982640 1 -- 172.21.15.112:0/709459164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd718004800 msgr2=0x7fd72c144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:08.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.586+0000 7fd731982640 1 --2- 172.21.15.112:0/709459164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd718004800 0x7fd72c144d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.586+0000 7fd731982640 1 -- 172.21.15.112:0/709459164 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c1ecc80 con 0x7fd72c153900 2024-04-25T01:41:08.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.586+0000 7fd732183640 1 --2- 172.21.15.112:0/709459164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd718004800 0x7fd72c144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:41:08.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.586+0000 7fd731982640 1 --2- 172.21.15.112:0/709459164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd72c153900 0x7fd72c14c670 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd71c001970 tx=0x7fd72c153310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:08.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.586+0000 7fd722ffd640 1 -- 172.21.15.112:0/709459164 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c041020 con 0x7fd72c153900 2024-04-25T01:41:08.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.586+0000 7fd722ffd640 1 -- 172.21.15.112:0/709459164 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd71c038040 con 0x7fd72c153900 2024-04-25T01:41:08.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.586+0000 7fd7343e7640 1 -- 172.21.15.112:0/709459164 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72c1ecf60 con 0x7fd72c153900 2024-04-25T01:41:08.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.586+0000 7fd722ffd640 1 -- 172.21.15.112:0/709459164 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c007070 con 0x7fd72c153900 2024-04-25T01:41:08.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.586+0000 7fd7343e7640 1 -- 172.21.15.112:0/709459164 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd72c1ed520 con 0x7fd72c153900 2024-04-25T01:41:08.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.587+0000 7fd7343e7640 1 -- 172.21.15.112:0/709459164 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd72c04f500 con 0x7fd72c153900 2024-04-25T01:41:08.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.587+0000 7fd722ffd640 1 -- 172.21.15.112:0/709459164 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd71c044020 con 0x7fd72c153900 2024-04-25T01:41:08.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.588+0000 7fd722ffd640 1 --2- 172.21.15.112:0/709459164 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd71006be60 0x7fd71006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.588+0000 7fd722ffd640 1 -- 172.21.15.112:0/709459164 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fd71c0ab430 con 0x7fd72c153900 2024-04-25T01:41:08.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.588+0000 7fd732183640 1 --2- 172.21.15.112:0/709459164 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd71006be60 0x7fd71006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:08.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.588+0000 7fd732183640 1 --2- 172.21.15.112:0/709459164 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd71006be60 0x7fd71006e320 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd7280074d0 tx=0x7fd72800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:08.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.592+0000 7fd722ffd640 1 -- 172.21.15.112:0/709459164 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd71c079ad0 con 0x7fd72c153900 2024-04-25T01:41:08.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.765+0000 7fd7343e7640 1 -- 172.21.15.112:0/709459164 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) -- 0x7fd72c145b60 con 0x7fd72c153900 2024-04-25T01:41:08.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.766+0000 7fd722ffd640 1 -- 172.21.15.112:0/709459164 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v26) ==== 95+0+11 (secure 0 0 0) 0x7fd71c0793d0 con 0x7fd72c153900 2024-04-25T01:41:08.752 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T01:41:08.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.768+0000 7fd7343e7640 1 -- 172.21.15.112:0/709459164 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd71006be60 msgr2=0x7fd71006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:08.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.768+0000 7fd7343e7640 1 --2- 172.21.15.112:0/709459164 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd71006be60 0x7fd71006e320 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd7280074d0 tx=0x7fd72800b040 comp rx=0 tx=0).stop 2024-04-25T01:41:08.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.768+0000 7fd7343e7640 1 -- 172.21.15.112:0/709459164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd72c153900 msgr2=0x7fd72c14c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:08.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.768+0000 7fd7343e7640 1 --2- 172.21.15.112:0/709459164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd72c153900 0x7fd72c14c670 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd71c001970 tx=0x7fd72c153310 comp rx=0 tx=0).stop 2024-04-25T01:41:08.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.768+0000 7fd7343e7640 1 -- 172.21.15.112:0/709459164 shutdown_connections 2024-04-25T01:41:08.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.768+0000 7fd7343e7640 1 --2- 172.21.15.112:0/709459164 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd71006be60 0x7fd71006e320 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.768+0000 7fd7343e7640 1 --2- 172.21.15.112:0/709459164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd72c145280 0x7fd72c14cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.768+0000 7fd7343e7640 1 --2- 172.21.15.112:0/709459164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd72c153900 0x7fd72c14c670 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.768+0000 7fd7343e7640 1 --2- 172.21.15.112:0/709459164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd718004800 0x7fd72c144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.768+0000 7fd7343e7640 1 -- 172.21.15.112:0/709459164 >> 172.21.15.112:0/709459164 conn(0x7fd72c0c3a10 msgr2=0x7fd72c06daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:08.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.768+0000 7fd7343e7640 1 -- 172.21.15.112:0/709459164 shutdown_connections 2024-04-25T01:41:08.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.768+0000 7fd7343e7640 1 -- 172.21.15.112:0/709459164 wait complete. 2024-04-25T01:41:08.766 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:41:08.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.878+0000 7f2855f9e640 1 Processor -- start 2024-04-25T01:41:08.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.878+0000 7f2855f9e640 1 -- start start 2024-04-25T01:41:08.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.879+0000 7f2855f9e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f28500ac3b0 0x7f2850155f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.879+0000 7f2855f9e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28501529b0 con 0x7f285014c3f0 2024-04-25T01:41:08.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.879+0000 7f2855f9e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2850152c70 con 0x7f28501524a0 2024-04-25T01:41:08.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.879+0000 7f2855f9e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2850155a90 con 0x7f28500ac3b0 2024-04-25T01:41:08.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.879+0000 7f284f7fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f285014c3f0 0x7f28500c2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:39336/0 (socket says 172.21.15.112:39336) 2024-04-25T01:41:08.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.879+0000 7f284f7fe640 1 -- 172.21.15.112:0/3326728459 learned_addr learned my addr 172.21.15.112:0/3326728459 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:08.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.879+0000 7f284effd640 1 --2- 172.21.15.112:0/3326728459 >> v2:172.21.15.187:3300/0 conn(0x7f28500ac3b0 0x7f2850155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:08.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.879+0000 7f284e7fc640 1 -- 172.21.15.112:0/3326728459 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2194374382 0 0) 0x7f28501529b0 con 0x7f285014c3f0 2024-04-25T01:41:08.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.879+0000 7f284e7fc640 1 -- 172.21.15.112:0/3326728459 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2838005730 con 0x7f285014c3f0 2024-04-25T01:41:08.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.879+0000 7f284e7fc640 1 -- 172.21.15.112:0/3326728459 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1244881537 0 0) 0x7f2838005730 con 0x7f285014c3f0 2024-04-25T01:41:08.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.879+0000 7f284effd640 1 -- 172.21.15.112:0/3326728459 >> v1:172.21.15.187:6789/0 conn(0x7f28501524a0 legacy=0x7f285015ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:08.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.879+0000 7f284effd640 1 -- 172.21.15.112:0/3326728459 >> v1:172.21.15.112:6789/0 conn(0x7f285014c3f0 legacy=0x7f28500c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:08.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.879+0000 7f284effd640 1 -- 172.21.15.112:0/3326728459 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2850152f50 con 0x7f28500ac3b0 2024-04-25T01:41:08.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.880+0000 7f284effd640 1 --2- 172.21.15.112:0/3326728459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28500ac3b0 0x7f2850155f20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f28440097a0 tx=0x7f284402a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11be48c6003eaf16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:08.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.880+0000 7f284e7fc640 1 -- 172.21.15.112:0/3326728459 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844037070 con 0x7f28500ac3b0 2024-04-25T01:41:08.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.880+0000 7f284e7fc640 1 -- 172.21.15.112:0/3326728459 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f284402abc0 con 0x7f28500ac3b0 2024-04-25T01:41:08.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.880+0000 7f284e7fc640 1 -- 172.21.15.112:0/3326728459 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840001670 con 0x7f28500ac3b0 2024-04-25T01:41:08.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.881+0000 7f2855f9e640 1 -- 172.21.15.112:0/3326728459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28500ac3b0 msgr2=0x7f2850155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:08.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.881+0000 7f2855f9e640 1 --2- 172.21.15.112:0/3326728459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28500ac3b0 0x7f2850155f20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f28440097a0 tx=0x7f284402a130 comp rx=0 tx=0).stop 2024-04-25T01:41:08.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.881+0000 7f2855f9e640 1 -- 172.21.15.112:0/3326728459 shutdown_connections 2024-04-25T01:41:08.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.881+0000 7f2855f9e640 1 --2- 172.21.15.112:0/3326728459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28500ac3b0 0x7f2850155f20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.881+0000 7f2855f9e640 1 -- 172.21.15.112:0/3326728459 >> 172.21.15.112:0/3326728459 conn(0x7f285006b160 msgr2=0x7f285006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:08.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.881+0000 7f2855f9e640 1 -- 172.21.15.112:0/3326728459 shutdown_connections 2024-04-25T01:41:08.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.881+0000 7f2855f9e640 1 -- 172.21.15.112:0/3326728459 wait complete. 2024-04-25T01:41:08.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.882+0000 7f2855f9e640 1 Processor -- start 2024-04-25T01:41:08.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.882+0000 7f2855f9e640 1 -- start start 2024-04-25T01:41:08.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f2855f9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28500ac3b0 0x7f2850154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f2855f9e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f285014c3f0 0x7f28501ee920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f2855f9e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28501524a0 0x7f28501eee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f2855f9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f28500c32a0 con 0x7f28500ac3b0 2024-04-25T01:41:08.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f2855f9e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2850070d60 con 0x7f28501524a0 2024-04-25T01:41:08.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f2855f9e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f28500c3420 con 0x7f285014c3f0 2024-04-25T01:41:08.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f284f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28500ac3b0 0x7f2850154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:08.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f284f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28500ac3b0 0x7f2850154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56420/0 (socket says 172.21.15.112:56420) 2024-04-25T01:41:08.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f284f7fe640 1 -- 172.21.15.112:0/3450728641 learned_addr learned my addr 172.21.15.112:0/3450728641 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:08.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f284f7fe640 1 -- 172.21.15.112:0/3450728641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f285014c3f0 msgr2=0x7f28501ee920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:08.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f284ffff640 1 --2- 172.21.15.112:0/3450728641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28501524a0 0x7f28501eee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:08.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f284f7fe640 1 --2- 172.21.15.112:0/3450728641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f285014c3f0 0x7f28501ee920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f284f7fe640 1 -- 172.21.15.112:0/3450728641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28501524a0 msgr2=0x7f28501eee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:08.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f284f7fe640 1 --2- 172.21.15.112:0/3450728641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28501524a0 0x7f28501eee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:08.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f284f7fe640 1 -- 172.21.15.112:0/3450728641 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2850145950 con 0x7f28500ac3b0 2024-04-25T01:41:08.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f284ffff640 1 --2- 172.21.15.112:0/3450728641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28501524a0 0x7f28501eee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:41:08.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f284f7fe640 1 --2- 172.21.15.112:0/3450728641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28500ac3b0 0x7f2850154c80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f284000c780 tx=0x7f284000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:08.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f2837fff640 1 -- 172.21.15.112:0/3450728641 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840017070 con 0x7f28500ac3b0 2024-04-25T01:41:08.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f2837fff640 1 -- 172.21.15.112:0/3450728641 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2840005ad0 con 0x7f28500ac3b0 2024-04-25T01:41:08.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f2837fff640 1 -- 172.21.15.112:0/3450728641 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840007070 con 0x7f28500ac3b0 2024-04-25T01:41:08.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.883+0000 7f2855f9e640 1 -- 172.21.15.112:0/3450728641 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2850145c30 con 0x7f28500ac3b0 2024-04-25T01:41:08.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.884+0000 7f2855f9e640 1 -- 172.21.15.112:0/3450728641 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f285006ecf0 con 0x7f28500ac3b0 2024-04-25T01:41:08.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.884+0000 7f2855f9e640 1 -- 172.21.15.112:0/3450728641 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28500682a0 con 0x7f28500ac3b0 2024-04-25T01:41:08.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.885+0000 7f2837fff640 1 -- 172.21.15.112:0/3450728641 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f284001a070 con 0x7f28500ac3b0 2024-04-25T01:41:08.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.885+0000 7f2837fff640 1 --2- 172.21.15.112:0/3450728641 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f283c06be30 0x7f283c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:08.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.885+0000 7f2837fff640 1 -- 172.21.15.112:0/3450728641 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f284008a7a0 con 0x7f28500ac3b0 2024-04-25T01:41:08.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.885+0000 7f284effd640 1 --2- 172.21.15.112:0/3450728641 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f283c06be30 0x7f283c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:08.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.886+0000 7f284effd640 1 --2- 172.21.15.112:0/3450728641 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f283c06be30 0x7f283c06e2f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2844034000 tx=0x7f284402c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:08.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:08.890+0000 7f2837fff640 1 -- 172.21.15.112:0/3450728641 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2840058d90 con 0x7f28500ac3b0 2024-04-25T01:41:09.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:09.077+0000 7f2855f9e640 1 -- 172.21.15.112:0/3450728641 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2850145e00 con 0x7f28500ac3b0 2024-04-25T01:41:09.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:09.077+0000 7f2837fff640 1 -- 172.21.15.112:0/3450728641 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f2840058690 con 0x7f28500ac3b0 2024-04-25T01:41:09.063 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-04-25T01:41:09.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:09.079+0000 7f2855f9e640 1 -- 172.21.15.112:0/3450728641 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f283c06be30 msgr2=0x7f283c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:09.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:09.079+0000 7f2855f9e640 1 --2- 172.21.15.112:0/3450728641 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f283c06be30 0x7f283c06e2f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2844034000 tx=0x7f284402c000 comp rx=0 tx=0).stop 2024-04-25T01:41:09.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:09.079+0000 7f2855f9e640 1 -- 172.21.15.112:0/3450728641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28500ac3b0 msgr2=0x7f2850154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:09.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:09.079+0000 7f2855f9e640 1 --2- 172.21.15.112:0/3450728641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28500ac3b0 0x7f2850154c80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f284000c780 tx=0x7f284000cc40 comp rx=0 tx=0).stop 2024-04-25T01:41:09.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:09.079+0000 7f2855f9e640 1 -- 172.21.15.112:0/3450728641 shutdown_connections 2024-04-25T01:41:09.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:09.079+0000 7f2855f9e640 1 --2- 172.21.15.112:0/3450728641 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f283c06be30 0x7f283c06e2f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:09.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:09.079+0000 7f2855f9e640 1 --2- 172.21.15.112:0/3450728641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28501524a0 0x7f28501eee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:09.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:09.079+0000 7f2855f9e640 1 --2- 172.21.15.112:0/3450728641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f285014c3f0 0x7f28501ee920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:09.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:09.079+0000 7f2855f9e640 1 --2- 172.21.15.112:0/3450728641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28500ac3b0 0x7f2850154c80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:09.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:09.079+0000 7f2855f9e640 1 -- 172.21.15.112:0/3450728641 >> 172.21.15.112:0/3450728641 conn(0x7f285006b160 msgr2=0x7f2850150ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:09.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:09.079+0000 7f2855f9e640 1 -- 172.21.15.112:0/3450728641 shutdown_connections 2024-04-25T01:41:09.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:09.079+0000 7f2855f9e640 1 -- 172.21.15.112:0/3450728641 wait complete. 2024-04-25T01:41:09.076 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi187.front.sepia.ceph.com... 2024-04-25T01:41:09.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:41:09.077 DEBUG:teuthology.orchestra.run.smithi187:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T01:41:09.126 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:41:09.132 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi187.stderr:ceph-fuse[41261]: fuse finished with error 0 and tester_r 0 2024-04-25T01:41:15.228 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T01:41:15.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:41:15.229 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T01:41:15.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:41:15.247 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-04-25T01:41:15.305 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T01:41:15.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:41:15.305 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:41:15.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:41:15.373 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2024-04-25T01:41:15.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:41:15.482 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-25T01:41:15.511 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T01:41:15.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:41:15.512 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:41:15.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:41:15.581 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-04-25T01:41:15.674 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T01:41:15.699 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T01:41:15.699 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:41:15.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.815+0000 7f556c8d2640 1 Processor -- start 2024-04-25T01:41:15.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.815+0000 7f556c8d2640 1 -- start start 2024-04-25T01:41:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.816+0000 7f556c8d2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55681592c0 0x7f55680c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.816+0000 7f556c8d2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f55680c72b0 0x7f5568160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.816+0000 7f556c8d2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5568159b90 con 0x7f55681596a0 2024-04-25T01:41:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.816+0000 7f556c8d2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f55680c6e20 con 0x7f55680c72b0 2024-04-25T01:41:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.816+0000 7f556c8d2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f556806c970 con 0x7f55681592c0 2024-04-25T01:41:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.816+0000 7f5566d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f55680c72b0 0x7f5568160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.816+0000 7f5566d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f55680c72b0 0x7f5568160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56436/0 (socket says 172.21.15.112:56436) 2024-04-25T01:41:15.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.816+0000 7f5566d9d640 1 -- 172.21.15.112:0/421263955 learned_addr learned my addr 172.21.15.112:0/421263955 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:15.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.817+0000 7f556659c640 1 --2- 172.21.15.112:0/421263955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55681592c0 0x7f55680c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:15.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.817+0000 7f5566d9d640 1 -- 172.21.15.112:0/421263955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55681592c0 msgr2=0x7f55680c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:15.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.817+0000 7f5566d9d640 1 --2- 172.21.15.112:0/421263955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55681592c0 0x7f55680c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:15.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.817+0000 7f5566d9d640 1 -- 172.21.15.112:0/421263955 >> v1:172.21.15.187:6789/0 conn(0x7f55681596a0 legacy=0x7f5568152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:15.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.817+0000 7f5566d9d640 1 -- 172.21.15.112:0/421263955 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556806cc50 con 0x7f55680c72b0 2024-04-25T01:41:15.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.817+0000 7f5566d9d640 1 --2- 172.21.15.112:0/421263955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55680c72b0 0x7f5568160380 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f55680aac40 tx=0x7f556002a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5cf52a005387985 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:15.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.817+0000 7f5565d9b640 1 -- 172.21.15.112:0/421263955 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560036070 con 0x7f55680c72b0 2024-04-25T01:41:15.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.818+0000 7f5565d9b640 1 -- 172.21.15.112:0/421263955 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5560031600 con 0x7f55680c72b0 2024-04-25T01:41:15.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.818+0000 7f5565d9b640 1 -- 172.21.15.112:0/421263955 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55600319d0 con 0x7f55680c72b0 2024-04-25T01:41:15.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.818+0000 7f556c8d2640 1 -- 172.21.15.112:0/421263955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55680c72b0 msgr2=0x7f5568160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:15.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.818+0000 7f556c8d2640 1 --2- 172.21.15.112:0/421263955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55680c72b0 0x7f5568160380 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f55680aac40 tx=0x7f556002a590 comp rx=0 tx=0).stop 2024-04-25T01:41:15.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.818+0000 7f556c8d2640 1 -- 172.21.15.112:0/421263955 shutdown_connections 2024-04-25T01:41:15.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.818+0000 7f556c8d2640 1 --2- 172.21.15.112:0/421263955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55680c72b0 0x7f5568160380 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:15.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.818+0000 7f556c8d2640 1 --2- 172.21.15.112:0/421263955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55681592c0 0x7f55680c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:15.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.818+0000 7f556c8d2640 1 -- 172.21.15.112:0/421263955 >> 172.21.15.112:0/421263955 conn(0x7f55680cb170 msgr2=0x7f55680cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:15.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.818+0000 7f556c8d2640 1 -- 172.21.15.112:0/421263955 shutdown_connections 2024-04-25T01:41:15.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.818+0000 7f556c8d2640 1 -- 172.21.15.112:0/421263955 wait complete. 2024-04-25T01:41:15.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.819+0000 7f556c8d2640 1 Processor -- start 2024-04-25T01:41:15.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.819+0000 7f556c8d2640 1 -- start start 2024-04-25T01:41:15.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.820+0000 7f556c8d2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55680c72b0 0x7f55680c0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:15.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.820+0000 7f556c8d2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55681592c0 0x7f55680bef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:15.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.820+0000 7f556c8d2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55681596a0 0x7f55680bf450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:15.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.820+0000 7f556c8d2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5568153590 con 0x7f55681596a0 2024-04-25T01:41:15.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.820+0000 7f556c8d2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5568153410 con 0x7f55680c72b0 2024-04-25T01:41:15.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.820+0000 7f556c8d2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f556806db20 con 0x7f55681592c0 2024-04-25T01:41:15.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.820+0000 7f5566d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55681596a0 0x7f55680bf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:15.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.820+0000 7f5566d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55681596a0 0x7f55680bf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56446/0 (socket says 172.21.15.112:56446) 2024-04-25T01:41:15.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.820+0000 7f5566d9d640 1 -- 172.21.15.112:0/2293368770 learned_addr learned my addr 172.21.15.112:0/2293368770 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:15.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.820+0000 7f5566d9d640 1 -- 172.21.15.112:0/2293368770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55681592c0 msgr2=0x7f55680bef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:15.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.820+0000 7f5566d9d640 1 --2- 172.21.15.112:0/2293368770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55681592c0 0x7f55680bef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:15.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.820+0000 7f5566d9d640 1 -- 172.21.15.112:0/2293368770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55680c72b0 msgr2=0x7f55680c0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:15.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.820+0000 7f556659c640 1 --2- 172.21.15.112:0/2293368770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55680c72b0 0x7f55680c0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:15.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.820+0000 7f5566d9d640 1 --2- 172.21.15.112:0/2293368770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55680c72b0 0x7f55680c0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:15.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.820+0000 7f5566d9d640 1 -- 172.21.15.112:0/2293368770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55680c0ea0 con 0x7f55681596a0 2024-04-25T01:41:15.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.820+0000 7f556659c640 1 --2- 172.21.15.112:0/2293368770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55680c72b0 0x7f55680c0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:41:15.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.821+0000 7f5566d9d640 1 --2- 172.21.15.112:0/2293368770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55681596a0 0x7f55680bf450 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5560009f40 tx=0x7f5560004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.821+0000 7f555effd640 1 -- 172.21.15.112:0/2293368770 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560036050 con 0x7f55681596a0 2024-04-25T01:41:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.821+0000 7f555effd640 1 -- 172.21.15.112:0/2293368770 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5560042070 con 0x7f55681596a0 2024-04-25T01:41:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.821+0000 7f556c8d2640 1 -- 172.21.15.112:0/2293368770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5568155760 con 0x7f55681596a0 2024-04-25T01:41:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.821+0000 7f555effd640 1 -- 172.21.15.112:0/2293368770 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560007b90 con 0x7f55681596a0 2024-04-25T01:41:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.821+0000 7f556c8d2640 1 -- 172.21.15.112:0/2293368770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55680c1dc0 con 0x7f55681596a0 2024-04-25T01:41:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.822+0000 7f556c8d2640 1 -- 172.21.15.112:0/2293368770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55680c1070 con 0x7f55681596a0 2024-04-25T01:41:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.822+0000 7f555effd640 1 -- 172.21.15.112:0/2293368770 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5560047460 con 0x7f55681596a0 2024-04-25T01:41:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.823+0000 7f555effd640 1 --2- 172.21.15.112:0/2293368770 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f554806be30 0x7f554806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.823+0000 7f556659c640 1 --2- 172.21.15.112:0/2293368770 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f554806be30 0x7f554806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.823+0000 7f555effd640 1 -- 172.21.15.112:0/2293368770 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f55600ac140 con 0x7f55681596a0 2024-04-25T01:41:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.823+0000 7f556659c640 1 --2- 172.21.15.112:0/2293368770 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f554806be30 0x7f554806e2f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f55681e1d30 tx=0x7f5558008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:15.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.827+0000 7f555effd640 1 -- 172.21.15.112:0/2293368770 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5560045360 con 0x7f55681596a0 2024-04-25T01:41:15.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:15.997+0000 7f556c8d2640 1 -- 172.21.15.112:0/2293368770 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5568075980 con 0x7f554806be30 2024-04-25T01:41:15.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.009+0000 7f555effd640 1 -- 172.21.15.112:0/2293368770 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+493703 (secure 0 0 0) 0x7f55580085a0 con 0x7f554806be30 2024-04-25T01:41:16.003 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:41:16.006 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T01:41:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.013+0000 7f556c8d2640 1 -- 172.21.15.112:0/2293368770 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f554806be30 msgr2=0x7f554806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.013+0000 7f556c8d2640 1 --2- 172.21.15.112:0/2293368770 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f554806be30 0x7f554806e2f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f55681e1d30 tx=0x7f5558008040 comp rx=0 tx=0).stop 2024-04-25T01:41:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.013+0000 7f556c8d2640 1 -- 172.21.15.112:0/2293368770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55681596a0 msgr2=0x7f55680bf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.013+0000 7f556c8d2640 1 --2- 172.21.15.112:0/2293368770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55681596a0 0x7f55680bf450 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5560009f40 tx=0x7f5560004a40 comp rx=0 tx=0).stop 2024-04-25T01:41:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.013+0000 7f556c8d2640 1 -- 172.21.15.112:0/2293368770 shutdown_connections 2024-04-25T01:41:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.013+0000 7f556c8d2640 1 --2- 172.21.15.112:0/2293368770 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f554806be30 0x7f554806e2f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.013+0000 7f556c8d2640 1 --2- 172.21.15.112:0/2293368770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55681596a0 0x7f55680bf450 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.013+0000 7f556c8d2640 1 --2- 172.21.15.112:0/2293368770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55681592c0 0x7f55680bef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.013+0000 7f556c8d2640 1 --2- 172.21.15.112:0/2293368770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55680c72b0 0x7f55680c0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.013+0000 7f556c8d2640 1 -- 172.21.15.112:0/2293368770 >> 172.21.15.112:0/2293368770 conn(0x7f55680cb170 msgr2=0x7f5568152010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.013+0000 7f556c8d2640 1 -- 172.21.15.112:0/2293368770 shutdown_connections 2024-04-25T01:41:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.013+0000 7f556c8d2640 1 -- 172.21.15.112:0/2293368770 wait complete. 2024-04-25T01:41:16.012 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":486,"stamp":"2024-04-25T01:41:14.915187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":36043200,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":196,"num_read_kb":327,"num_write":3138327,"num_write_kb":38804753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":607268,"ondisk_log_size":607268,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":7846528,"kb_used_data":72704,"kb_used_omap":12,"kb_used_meta":7773747,"kb_avail":747128192,"statfs":{"total":773094113280,"available":765059268608,"internally_reserved":0,"allocated":74448896,"data_stored":72990936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":7960317488},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":70,"apply_latency_ms":70,"commit_latency_ns":70000000,"apply_latency_ns":70000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003261"},"pg_stats":[{"pgid":"5.18","version":"26'4099","reported_seq":4121,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.267400+0000","last_change":"2024-04-25T01:26:02.412272+0000","last_active":"2024-04-25T01:41:09.267400+0000","last_peered":"2024-04-25T01:41:09.267400+0000","last_clean":"2024-04-25T01:41:09.267400+0000","last_became_active":"2024-04-25T01:26:02.411176+0000","last_became_peered":"2024-04-25T01:26:02.411176+0000","last_unstale":"2024-04-25T01:41:09.267400+0000","last_undegraded":"2024-04-25T01:41:09.267400+0000","last_fullsized":"2024-04-25T01:41:09.267400+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4099,"log_dups_size":0,"ondisk_log_size":4099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:49.498275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21931,"num_write_kb":288545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"26'658","reported_seq":691,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:44.444563+0000","last_change":"2024-04-25T01:25:52.353978+0000","last_active":"2024-04-25T01:40:44.444563+0000","last_peered":"2024-04-25T01:40:44.444563+0000","last_clean":"2024-04-25T01:40:44.444563+0000","last_became_active":"2024-04-25T01:25:52.353649+0000","last_became_peered":"2024-04-25T01:25:52.353649+0000","last_unstale":"2024-04-25T01:40:44.444563+0000","last_undegraded":"2024-04-25T01:40:44.444563+0000","last_fullsized":"2024-04-25T01:40:44.444563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":4468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"26'5333","reported_seq":5364,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.730049+0000","last_change":"2024-04-25T01:25:53.364257+0000","last_active":"2024-04-25T01:40:58.730049+0000","last_peered":"2024-04-25T01:40:58.730049+0000","last_clean":"2024-04-25T01:40:58.730049+0000","last_became_active":"2024-04-25T01:25:53.363409+0000","last_became_peered":"2024-04-25T01:25:53.363409+0000","last_unstale":"2024-04-25T01:40:58.730049+0000","last_undegraded":"2024-04-25T01:40:58.730049+0000","last_fullsized":"2024-04-25T01:40:58.730049+0000","mapping_epoch":18,"log_start":"26'100","ondisk_log_start":"26'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28405,"num_write_kb":331018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"26'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:31.329698+0000","last_change":"2024-04-25T01:26:01.407136+0000","last_active":"2024-04-25T01:40:31.329698+0000","last_peered":"2024-04-25T01:40:31.329698+0000","last_clean":"2024-04-25T01:40:31.329698+0000","last_became_active":"2024-04-25T01:26:01.406855+0000","last_became_peered":"2024-04-25T01:26:01.406855+0000","last_unstale":"2024-04-25T01:40:31.329698+0000","last_undegraded":"2024-04-25T01:40:31.329698+0000","last_fullsized":"2024-04-25T01:40:31.329698+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:02.180826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"26'5328","reported_seq":5350,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.321101+0000","last_change":"2024-04-25T01:26:02.425367+0000","last_active":"2024-04-25T01:41:09.321101+0000","last_peered":"2024-04-25T01:41:09.321101+0000","last_clean":"2024-04-25T01:41:09.321101+0000","last_became_active":"2024-04-25T01:26:02.425071+0000","last_became_peered":"2024-04-25T01:26:02.425071+0000","last_unstale":"2024-04-25T01:41:09.321101+0000","last_undegraded":"2024-04-25T01:41:09.321101+0000","last_fullsized":"2024-04-25T01:41:09.321101+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5328,"log_dups_size":0,"ondisk_log_size":5328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:47.810943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27477,"num_write_kb":347746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"26'8","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.218358+0000","last_change":"2024-04-25T01:25:52.349185+0000","last_active":"2024-04-25T01:40:49.218358+0000","last_peered":"2024-04-25T01:40:49.218358+0000","last_clean":"2024-04-25T01:40:49.218358+0000","last_became_active":"2024-04-25T01:25:52.348931+0000","last_became_peered":"2024-04-25T01:25:52.348931+0000","last_unstale":"2024-04-25T01:40:49.218358+0000","last_undegraded":"2024-04-25T01:40:49.218358+0000","last_fullsized":"2024-04-25T01:40:49.218358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"26'5837","reported_seq":5868,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.789129+0000","last_change":"2024-04-25T01:25:53.374150+0000","last_active":"2024-04-25T01:40:58.789129+0000","last_peered":"2024-04-25T01:40:58.789129+0000","last_clean":"2024-04-25T01:40:58.789129+0000","last_became_active":"2024-04-25T01:25:53.373755+0000","last_became_peered":"2024-04-25T01:25:53.373755+0000","last_unstale":"2024-04-25T01:40:58.789129+0000","last_undegraded":"2024-04-25T01:40:58.789129+0000","last_fullsized":"2024-04-25T01:40:58.789129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5837,"log_dups_size":0,"ondisk_log_size":5837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30942,"num_write_kb":369940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"26'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:32.791998+0000","last_change":"2024-04-25T01:26:01.406334+0000","last_active":"2024-04-25T01:40:32.791998+0000","last_peered":"2024-04-25T01:40:32.791998+0000","last_clean":"2024-04-25T01:40:32.791998+0000","last_became_active":"2024-04-25T01:26:01.405994+0000","last_became_peered":"2024-04-25T01:26:01.405994+0000","last_unstale":"2024-04-25T01:40:32.791998+0000","last_undegraded":"2024-04-25T01:40:32.791998+0000","last_fullsized":"2024-04-25T01:40:32.791998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:43.953437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"26'4188","reported_seq":4210,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.336545+0000","last_change":"2024-04-25T01:26:02.427695+0000","last_active":"2024-04-25T01:41:09.336545+0000","last_peered":"2024-04-25T01:41:09.336545+0000","last_clean":"2024-04-25T01:41:09.336545+0000","last_became_active":"2024-04-25T01:26:02.427443+0000","last_became_peered":"2024-04-25T01:26:02.427443+0000","last_unstale":"2024-04-25T01:41:09.336545+0000","last_undegraded":"2024-04-25T01:41:09.336545+0000","last_fullsized":"2024-04-25T01:41:09.336545+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4188,"log_dups_size":0,"ondisk_log_size":4188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20869,"num_write_kb":259707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"26'11","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.299916+0000","last_change":"2024-04-25T01:25:52.356403+0000","last_active":"2024-04-25T01:40:49.299916+0000","last_peered":"2024-04-25T01:40:49.299916+0000","last_clean":"2024-04-25T01:40:49.299916+0000","last_became_active":"2024-04-25T01:25:52.355822+0000","last_became_peered":"2024-04-25T01:25:52.355822+0000","last_unstale":"2024-04-25T01:40:49.299916+0000","last_undegraded":"2024-04-25T01:40:49.299916+0000","last_fullsized":"2024-04-25T01:40:49.299916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"26'3747","reported_seq":3777,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.846633+0000","last_change":"2024-04-25T01:25:53.370061+0000","last_active":"2024-04-25T01:40:58.846633+0000","last_peered":"2024-04-25T01:40:58.846633+0000","last_clean":"2024-04-25T01:40:58.846633+0000","last_became_active":"2024-04-25T01:25:53.369721+0000","last_became_peered":"2024-04-25T01:25:53.369721+0000","last_unstale":"2024-04-25T01:40:58.846633+0000","last_undegraded":"2024-04-25T01:40:58.846633+0000","last_fullsized":"2024-04-25T01:40:58.846633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3747,"log_dups_size":0,"ondisk_log_size":3747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18978,"num_write_kb":223593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"26'627","reported_seq":651,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.103722+0000","last_change":"2024-04-25T01:26:01.408179+0000","last_active":"2024-04-25T01:40:37.103722+0000","last_peered":"2024-04-25T01:40:37.103722+0000","last_clean":"2024-04-25T01:40:37.103722+0000","last_became_active":"2024-04-25T01:26:01.407909+0000","last_became_peered":"2024-04-25T01:26:01.407909+0000","last_unstale":"2024-04-25T01:40:37.103722+0000","last_undegraded":"2024-04-25T01:40:37.103722+0000","last_fullsized":"2024-04-25T01:40:37.103722+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:48.865940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":625,"num_write_kb":4235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"26'3441","reported_seq":3463,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.298394+0000","last_change":"2024-04-25T01:26:02.424615+0000","last_active":"2024-04-25T01:41:09.298394+0000","last_peered":"2024-04-25T01:41:09.298394+0000","last_clean":"2024-04-25T01:41:09.298394+0000","last_became_active":"2024-04-25T01:26:02.424325+0000","last_became_peered":"2024-04-25T01:26:02.424325+0000","last_unstale":"2024-04-25T01:41:09.298394+0000","last_undegraded":"2024-04-25T01:41:09.298394+0000","last_fullsized":"2024-04-25T01:41:09.298394+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3441,"log_dups_size":0,"ondisk_log_size":3441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17582,"num_write_kb":230191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"26'3","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.256836+0000","last_change":"2024-04-25T01:25:52.359139+0000","last_active":"2024-04-25T01:40:49.256836+0000","last_peered":"2024-04-25T01:40:49.256836+0000","last_clean":"2024-04-25T01:40:49.256836+0000","last_became_active":"2024-04-25T01:25:52.358841+0000","last_became_peered":"2024-04-25T01:25:52.358841+0000","last_unstale":"2024-04-25T01:40:49.256836+0000","last_undegraded":"2024-04-25T01:40:49.256836+0000","last_fullsized":"2024-04-25T01:40:49.256836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"26'5828","reported_seq":5858,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.878112+0000","last_change":"2024-04-25T01:25:53.368739+0000","last_active":"2024-04-25T01:40:58.878112+0000","last_peered":"2024-04-25T01:40:58.878112+0000","last_clean":"2024-04-25T01:40:58.878112+0000","last_became_active":"2024-04-25T01:25:53.368470+0000","last_became_peered":"2024-04-25T01:25:53.368470+0000","last_unstale":"2024-04-25T01:40:58.878112+0000","last_undegraded":"2024-04-25T01:40:58.878112+0000","last_fullsized":"2024-04-25T01:40:58.878112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5828,"log_dups_size":0,"ondisk_log_size":5828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:29.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30463,"num_write_kb":367857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"26'65","reported_seq":89,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.150373+0000","last_change":"2024-04-25T01:26:01.407969+0000","last_active":"2024-04-25T01:41:09.150373+0000","last_peered":"2024-04-25T01:41:09.150373+0000","last_clean":"2024-04-25T01:41:09.150373+0000","last_became_active":"2024-04-25T01:26:01.407655+0000","last_became_peered":"2024-04-25T01:26:01.407655+0000","last_unstale":"2024-04-25T01:41:09.150373+0000","last_undegraded":"2024-04-25T01:41:09.150373+0000","last_fullsized":"2024-04-25T01:41:09.150373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:58.485864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":738477,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61,"num_write_kb":764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"26'4794","reported_seq":4816,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.329809+0000","last_change":"2024-04-25T01:26:02.417776+0000","last_active":"2024-04-25T01:41:09.329809+0000","last_peered":"2024-04-25T01:41:09.329809+0000","last_clean":"2024-04-25T01:41:09.329809+0000","last_became_active":"2024-04-25T01:26:02.417524+0000","last_became_peered":"2024-04-25T01:26:02.417524+0000","last_unstale":"2024-04-25T01:41:09.329809+0000","last_undegraded":"2024-04-25T01:41:09.329809+0000","last_fullsized":"2024-04-25T01:41:09.329809+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4794,"log_dups_size":0,"ondisk_log_size":4794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.688469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24624,"num_write_kb":311668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"26'4532","reported_seq":4563,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.725104+0000","last_change":"2024-04-25T01:25:53.372385+0000","last_active":"2024-04-25T01:40:58.725104+0000","last_peered":"2024-04-25T01:40:58.725104+0000","last_clean":"2024-04-25T01:40:58.725104+0000","last_became_active":"2024-04-25T01:25:53.372064+0000","last_became_peered":"2024-04-25T01:25:53.372064+0000","last_unstale":"2024-04-25T01:40:58.725104+0000","last_undegraded":"2024-04-25T01:40:58.725104+0000","last_fullsized":"2024-04-25T01:40:58.725104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4532,"log_dups_size":0,"ondisk_log_size":4532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23642,"num_write_kb":279226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"26'1100","reported_seq":1132,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.233609+0000","last_change":"2024-04-25T01:25:52.358616+0000","last_active":"2024-04-25T01:40:49.233609+0000","last_peered":"2024-04-25T01:40:49.233609+0000","last_clean":"2024-04-25T01:40:49.233609+0000","last_became_active":"2024-04-25T01:25:52.358318+0000","last_became_peered":"2024-04-25T01:25:52.358318+0000","last_unstale":"2024-04-25T01:40:49.233609+0000","last_undegraded":"2024-04-25T01:40:49.233609+0000","last_fullsized":"2024-04-25T01:40:49.233609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1100,"log_dups_size":0,"ondisk_log_size":1100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1091,"num_write_kb":4419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"26'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:39:31.750022+0000","last_change":"2024-04-25T01:26:01.400599+0000","last_active":"2024-04-25T01:39:31.750022+0000","last_peered":"2024-04-25T01:39:31.750022+0000","last_clean":"2024-04-25T01:39:31.750022+0000","last_became_active":"2024-04-25T01:26:01.400266+0000","last_became_peered":"2024-04-25T01:26:01.400266+0000","last_unstale":"2024-04-25T01:39:31.750022+0000","last_undegraded":"2024-04-25T01:39:31.750022+0000","last_fullsized":"2024-04-25T01:39:31.750022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"26'4554","reported_seq":4576,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.270037+0000","last_change":"2024-04-25T01:26:02.420269+0000","last_active":"2024-04-25T01:41:09.270037+0000","last_peered":"2024-04-25T01:41:09.270037+0000","last_clean":"2024-04-25T01:41:09.270037+0000","last_became_active":"2024-04-25T01:26:02.420001+0000","last_became_peered":"2024-04-25T01:26:02.420001+0000","last_unstale":"2024-04-25T01:41:09.270037+0000","last_undegraded":"2024-04-25T01:41:09.270037+0000","last_fullsized":"2024-04-25T01:41:09.270037+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4554,"log_dups_size":0,"ondisk_log_size":4554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:38.149069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23431,"num_write_kb":298735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"26'3793","reported_seq":3823,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.792694+0000","last_change":"2024-04-25T01:25:53.361074+0000","last_active":"2024-04-25T01:40:58.792694+0000","last_peered":"2024-04-25T01:40:58.792694+0000","last_clean":"2024-04-25T01:40:58.792694+0000","last_became_active":"2024-04-25T01:25:53.360802+0000","last_became_peered":"2024-04-25T01:25:53.360802+0000","last_unstale":"2024-04-25T01:40:58.792694+0000","last_undegraded":"2024-04-25T01:40:58.792694+0000","last_fullsized":"2024-04-25T01:40:58.792694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3793,"log_dups_size":0,"ondisk_log_size":3793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19299,"num_write_kb":249413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"26'3","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.225267+0000","last_change":"2024-04-25T01:25:52.348841+0000","last_active":"2024-04-25T01:40:49.225267+0000","last_peered":"2024-04-25T01:40:49.225267+0000","last_clean":"2024-04-25T01:40:49.225267+0000","last_became_active":"2024-04-25T01:25:52.348550+0000","last_became_peered":"2024-04-25T01:25:52.348550+0000","last_unstale":"2024-04-25T01:40:49.225267+0000","last_undegraded":"2024-04-25T01:40:49.225267+0000","last_fullsized":"2024-04-25T01:40:49.225267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"26'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:36.992419+0000","last_change":"2024-04-25T01:26:01.408529+0000","last_active":"2024-04-25T01:40:36.992419+0000","last_peered":"2024-04-25T01:40:36.992419+0000","last_clean":"2024-04-25T01:40:36.992419+0000","last_became_active":"2024-04-25T01:26:01.407738+0000","last_became_peered":"2024-04-25T01:26:01.407738+0000","last_unstale":"2024-04-25T01:40:36.992419+0000","last_undegraded":"2024-04-25T01:40:36.992419+0000","last_fullsized":"2024-04-25T01:40:36.992419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:37.142255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"26'3863","reported_seq":3885,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.299717+0000","last_change":"2024-04-25T01:26:02.417731+0000","last_active":"2024-04-25T01:41:09.299717+0000","last_peered":"2024-04-25T01:41:09.299717+0000","last_clean":"2024-04-25T01:41:09.299717+0000","last_became_active":"2024-04-25T01:26:02.417438+0000","last_became_peered":"2024-04-25T01:26:02.417438+0000","last_unstale":"2024-04-25T01:41:09.299717+0000","last_undegraded":"2024-04-25T01:41:09.299717+0000","last_fullsized":"2024-04-25T01:41:09.299717+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3863,"log_dups_size":0,"ondisk_log_size":3863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19753,"num_write_kb":275789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"26'4400","reported_seq":4431,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.812159+0000","last_change":"2024-04-25T01:25:53.361790+0000","last_active":"2024-04-25T01:40:58.812159+0000","last_peered":"2024-04-25T01:40:58.812159+0000","last_clean":"2024-04-25T01:40:58.812159+0000","last_became_active":"2024-04-25T01:25:53.361542+0000","last_became_peered":"2024-04-25T01:25:53.361542+0000","last_unstale":"2024-04-25T01:40:58.812159+0000","last_undegraded":"2024-04-25T01:40:58.812159+0000","last_fullsized":"2024-04-25T01:40:58.812159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4400,"log_dups_size":0,"ondisk_log_size":4400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22906,"num_write_kb":296647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"26'3","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.235744+0000","last_change":"2024-04-25T01:25:52.353926+0000","last_active":"2024-04-25T01:40:49.235744+0000","last_peered":"2024-04-25T01:40:49.235744+0000","last_clean":"2024-04-25T01:40:49.235744+0000","last_became_active":"2024-04-25T01:25:52.353656+0000","last_became_peered":"2024-04-25T01:25:52.353656+0000","last_unstale":"2024-04-25T01:40:49.235744+0000","last_undegraded":"2024-04-25T01:40:49.235744+0000","last_fullsized":"2024-04-25T01:40:49.235744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"26'5","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.165411+0000","last_change":"2024-04-25T01:26:01.418556+0000","last_active":"2024-04-25T01:40:37.165411+0000","last_peered":"2024-04-25T01:40:37.165411+0000","last_clean":"2024-04-25T01:40:37.165411+0000","last_became_active":"2024-04-25T01:26:01.418263+0000","last_became_peered":"2024-04-25T01:26:01.418263+0000","last_unstale":"2024-04-25T01:40:37.165411+0000","last_undegraded":"2024-04-25T01:40:37.165411+0000","last_fullsized":"2024-04-25T01:40:37.165411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:28.139183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"26'5570","reported_seq":5592,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.349577+0000","last_change":"2024-04-25T01:26:02.413429+0000","last_active":"2024-04-25T01:41:09.349577+0000","last_peered":"2024-04-25T01:41:09.349577+0000","last_clean":"2024-04-25T01:41:09.349577+0000","last_became_active":"2024-04-25T01:26:02.412632+0000","last_became_peered":"2024-04-25T01:26:02.412632+0000","last_unstale":"2024-04-25T01:41:09.349577+0000","last_undegraded":"2024-04-25T01:41:09.349577+0000","last_fullsized":"2024-04-25T01:41:09.349577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5570,"log_dups_size":0,"ondisk_log_size":5570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:10.049310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27971,"num_write_kb":349705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"26'4715","reported_seq":4746,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.811959+0000","last_change":"2024-04-25T01:25:53.361653+0000","last_active":"2024-04-25T01:40:58.811959+0000","last_peered":"2024-04-25T01:40:58.811959+0000","last_clean":"2024-04-25T01:40:58.811959+0000","last_became_active":"2024-04-25T01:25:53.361265+0000","last_became_peered":"2024-04-25T01:25:53.361265+0000","last_unstale":"2024-04-25T01:40:58.811959+0000","last_undegraded":"2024-04-25T01:40:58.811959+0000","last_fullsized":"2024-04-25T01:40:58.811959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4715,"log_dups_size":0,"ondisk_log_size":4715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23886,"num_write_kb":288267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'3","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.248825+0000","last_change":"2024-04-25T01:25:52.353836+0000","last_active":"2024-04-25T01:40:49.248825+0000","last_peered":"2024-04-25T01:40:49.248825+0000","last_clean":"2024-04-25T01:40:49.248825+0000","last_became_active":"2024-04-25T01:25:52.353552+0000","last_became_peered":"2024-04-25T01:25:52.353552+0000","last_unstale":"2024-04-25T01:40:49.248825+0000","last_undegraded":"2024-04-25T01:40:49.248825+0000","last_fullsized":"2024-04-25T01:40:49.248825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"26'6","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.134073+0000","last_change":"2024-04-25T01:26:01.411323+0000","last_active":"2024-04-25T01:40:37.134073+0000","last_peered":"2024-04-25T01:40:37.134073+0000","last_clean":"2024-04-25T01:40:37.134073+0000","last_became_active":"2024-04-25T01:26:01.410378+0000","last_became_peered":"2024-04-25T01:26:01.410378+0000","last_unstale":"2024-04-25T01:40:37.134073+0000","last_undegraded":"2024-04-25T01:40:37.134073+0000","last_fullsized":"2024-04-25T01:40:37.134073+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"26'4403","reported_seq":4426,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.251745+0000","last_change":"2024-04-25T01:26:02.416646+0000","last_active":"2024-04-25T01:41:09.251745+0000","last_peered":"2024-04-25T01:41:09.251745+0000","last_clean":"2024-04-25T01:41:09.251745+0000","last_became_active":"2024-04-25T01:26:02.416395+0000","last_became_peered":"2024-04-25T01:26:02.416395+0000","last_unstale":"2024-04-25T01:41:09.251745+0000","last_undegraded":"2024-04-25T01:41:09.251745+0000","last_fullsized":"2024-04-25T01:41:09.251745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4403,"log_dups_size":0,"ondisk_log_size":4403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23642,"num_write_kb":287389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"26'3766","reported_seq":3797,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.767490+0000","last_change":"2024-04-25T01:25:53.370111+0000","last_active":"2024-04-25T01:40:58.767490+0000","last_peered":"2024-04-25T01:40:58.767490+0000","last_clean":"2024-04-25T01:40:58.767490+0000","last_became_active":"2024-04-25T01:25:53.369837+0000","last_became_peered":"2024-04-25T01:25:53.369837+0000","last_unstale":"2024-04-25T01:40:58.767490+0000","last_undegraded":"2024-04-25T01:40:58.767490+0000","last_fullsized":"2024-04-25T01:40:58.767490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3766,"log_dups_size":0,"ondisk_log_size":3766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19293,"num_write_kb":249241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"26'5","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.300730+0000","last_change":"2024-04-25T01:25:52.353269+0000","last_active":"2024-04-25T01:40:49.300730+0000","last_peered":"2024-04-25T01:40:49.300730+0000","last_clean":"2024-04-25T01:40:49.300730+0000","last_became_active":"2024-04-25T01:25:52.352991+0000","last_became_peered":"2024-04-25T01:25:52.352991+0000","last_unstale":"2024-04-25T01:40:49.300730+0000","last_undegraded":"2024-04-25T01:40:49.300730+0000","last_fullsized":"2024-04-25T01:40:49.300730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"26'41","reported_seq":105,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:47.845985+0000","last_change":"2024-04-25T01:26:01.410360+0000","last_active":"2024-04-25T01:40:47.845985+0000","last_peered":"2024-04-25T01:40:47.845985+0000","last_clean":"2024-04-25T01:40:47.845985+0000","last_became_active":"2024-04-25T01:26:01.410076+0000","last_became_peered":"2024-04-25T01:26:01.410076+0000","last_unstale":"2024-04-25T01:40:47.845985+0000","last_undegraded":"2024-04-25T01:40:47.845985+0000","last_fullsized":"2024-04-25T01:40:47.845985+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:52.561251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":112,"num_write":39,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"26'4289","reported_seq":4312,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.299549+0000","last_change":"2024-04-25T01:26:02.422182+0000","last_active":"2024-04-25T01:41:09.299549+0000","last_peered":"2024-04-25T01:41:09.299549+0000","last_clean":"2024-04-25T01:41:09.299549+0000","last_became_active":"2024-04-25T01:26:02.421885+0000","last_became_peered":"2024-04-25T01:26:02.421885+0000","last_unstale":"2024-04-25T01:41:09.299549+0000","last_undegraded":"2024-04-25T01:41:09.299549+0000","last_fullsized":"2024-04-25T01:41:09.299549+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4289,"log_dups_size":0,"ondisk_log_size":4289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:03.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22172,"num_write_kb":285532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"26'4466","reported_seq":4497,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.806516+0000","last_change":"2024-04-25T01:25:53.364967+0000","last_active":"2024-04-25T01:40:58.806516+0000","last_peered":"2024-04-25T01:40:58.806516+0000","last_clean":"2024-04-25T01:40:58.806516+0000","last_became_active":"2024-04-25T01:25:53.364694+0000","last_became_peered":"2024-04-25T01:25:53.364694+0000","last_unstale":"2024-04-25T01:40:58.806516+0000","last_undegraded":"2024-04-25T01:40:58.806516+0000","last_fullsized":"2024-04-25T01:40:58.806516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4466,"log_dups_size":0,"ondisk_log_size":4466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22859,"num_write_kb":292275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'571","reported_seq":604,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:51.521378+0000","last_change":"2024-04-25T01:25:52.355395+0000","last_active":"2024-04-25T01:40:51.521378+0000","last_peered":"2024-04-25T01:40:51.521378+0000","last_clean":"2024-04-25T01:40:51.521378+0000","last_became_active":"2024-04-25T01:25:52.355137+0000","last_became_peered":"2024-04-25T01:25:52.355137+0000","last_unstale":"2024-04-25T01:40:51.521378+0000","last_undegraded":"2024-04-25T01:40:51.521378+0000","last_fullsized":"2024-04-25T01:40:51.521378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:26.467968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"26'5","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.173222+0000","last_change":"2024-04-25T01:26:01.416108+0000","last_active":"2024-04-25T01:40:37.173222+0000","last_peered":"2024-04-25T01:40:37.173222+0000","last_clean":"2024-04-25T01:40:37.173222+0000","last_became_active":"2024-04-25T01:26:01.415824+0000","last_became_peered":"2024-04-25T01:26:01.415824+0000","last_unstale":"2024-04-25T01:40:37.173222+0000","last_undegraded":"2024-04-25T01:40:37.173222+0000","last_fullsized":"2024-04-25T01:40:37.173222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:28.139183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"26'4989","reported_seq":5012,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.273942+0000","last_change":"2024-04-25T01:26:02.416208+0000","last_active":"2024-04-25T01:41:09.273942+0000","last_peered":"2024-04-25T01:41:09.273942+0000","last_clean":"2024-04-25T01:41:09.273942+0000","last_became_active":"2024-04-25T01:26:02.415949+0000","last_became_peered":"2024-04-25T01:26:02.415949+0000","last_unstale":"2024-04-25T01:41:09.273942+0000","last_undegraded":"2024-04-25T01:41:09.273942+0000","last_fullsized":"2024-04-25T01:41:09.273942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4989,"log_dups_size":0,"ondisk_log_size":4989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:21.097225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25442,"num_write_kb":314904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"26'4796","reported_seq":4827,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.753012+0000","last_change":"2024-04-25T01:25:53.366944+0000","last_active":"2024-04-25T01:40:58.753012+0000","last_peered":"2024-04-25T01:40:58.753012+0000","last_clean":"2024-04-25T01:40:58.753012+0000","last_became_active":"2024-04-25T01:25:53.366693+0000","last_became_peered":"2024-04-25T01:25:53.366693+0000","last_unstale":"2024-04-25T01:40:58.753012+0000","last_undegraded":"2024-04-25T01:40:58.753012+0000","last_fullsized":"2024-04-25T01:40:58.753012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25026,"num_write_kb":309186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"26'4","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.236233+0000","last_change":"2024-04-25T01:25:52.356842+0000","last_active":"2024-04-25T01:40:49.236233+0000","last_peered":"2024-04-25T01:40:49.236233+0000","last_clean":"2024-04-25T01:40:49.236233+0000","last_became_active":"2024-04-25T01:25:52.356396+0000","last_became_peered":"2024-04-25T01:25:52.356396+0000","last_unstale":"2024-04-25T01:40:49.236233+0000","last_undegraded":"2024-04-25T01:40:49.236233+0000","last_fullsized":"2024-04-25T01:40:49.236233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"26'1","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:38:30.810691+0000","last_change":"2024-04-25T01:26:01.407934+0000","last_active":"2024-04-25T01:38:30.810691+0000","last_peered":"2024-04-25T01:38:30.810691+0000","last_clean":"2024-04-25T01:38:30.810691+0000","last_became_active":"2024-04-25T01:26:01.407659+0000","last_became_peered":"2024-04-25T01:26:01.407659+0000","last_unstale":"2024-04-25T01:38:30.810691+0000","last_undegraded":"2024-04-25T01:38:30.810691+0000","last_fullsized":"2024-04-25T01:38:30.810691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:19.629335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"26'4196","reported_seq":4219,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.266753+0000","last_change":"2024-04-25T01:26:02.418087+0000","last_active":"2024-04-25T01:41:09.266753+0000","last_peered":"2024-04-25T01:41:09.266753+0000","last_clean":"2024-04-25T01:41:09.266753+0000","last_became_active":"2024-04-25T01:26:02.417622+0000","last_became_peered":"2024-04-25T01:26:02.417622+0000","last_unstale":"2024-04-25T01:41:09.266753+0000","last_undegraded":"2024-04-25T01:41:09.266753+0000","last_fullsized":"2024-04-25T01:41:09.266753+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4196,"log_dups_size":0,"ondisk_log_size":4196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:03.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22617,"num_write_kb":283190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"26'4472","reported_seq":4503,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.865649+0000","last_change":"2024-04-25T01:25:53.368164+0000","last_active":"2024-04-25T01:40:58.865649+0000","last_peered":"2024-04-25T01:40:58.865649+0000","last_clean":"2024-04-25T01:40:58.865649+0000","last_became_active":"2024-04-25T01:25:53.367854+0000","last_became_peered":"2024-04-25T01:25:53.367854+0000","last_unstale":"2024-04-25T01:40:58.865649+0000","last_undegraded":"2024-04-25T01:40:58.865649+0000","last_fullsized":"2024-04-25T01:40:58.865649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4472,"log_dups_size":0,"ondisk_log_size":4472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23917,"num_write_kb":276200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'5","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.237276+0000","last_change":"2024-04-25T01:25:52.350128+0000","last_active":"2024-04-25T01:40:49.237276+0000","last_peered":"2024-04-25T01:40:49.237276+0000","last_clean":"2024-04-25T01:40:49.237276+0000","last_became_active":"2024-04-25T01:25:52.349874+0000","last_became_peered":"2024-04-25T01:25:52.349874+0000","last_unstale":"2024-04-25T01:40:49.237276+0000","last_undegraded":"2024-04-25T01:40:49.237276+0000","last_fullsized":"2024-04-25T01:40:49.237276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"26'5","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.136409+0000","last_change":"2024-04-25T01:26:01.418196+0000","last_active":"2024-04-25T01:40:37.136409+0000","last_peered":"2024-04-25T01:40:37.136409+0000","last_clean":"2024-04-25T01:40:37.136409+0000","last_became_active":"2024-04-25T01:26:01.417820+0000","last_became_peered":"2024-04-25T01:26:01.417820+0000","last_unstale":"2024-04-25T01:40:37.136409+0000","last_undegraded":"2024-04-25T01:40:37.136409+0000","last_fullsized":"2024-04-25T01:40:37.136409+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"26'4093","reported_seq":4116,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.276141+0000","last_change":"2024-04-25T01:26:02.422761+0000","last_active":"2024-04-25T01:41:09.276141+0000","last_peered":"2024-04-25T01:41:09.276141+0000","last_clean":"2024-04-25T01:41:09.276141+0000","last_became_active":"2024-04-25T01:26:02.422503+0000","last_became_peered":"2024-04-25T01:26:02.422503+0000","last_unstale":"2024-04-25T01:41:09.276141+0000","last_undegraded":"2024-04-25T01:41:09.276141+0000","last_fullsized":"2024-04-25T01:41:09.276141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4093,"log_dups_size":0,"ondisk_log_size":4093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:55.040946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21550,"num_write_kb":266696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"26'4812","reported_seq":4843,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.791228+0000","last_change":"2024-04-25T01:25:53.361696+0000","last_active":"2024-04-25T01:40:58.791228+0000","last_peered":"2024-04-25T01:40:58.791228+0000","last_clean":"2024-04-25T01:40:58.791228+0000","last_became_active":"2024-04-25T01:25:53.361156+0000","last_became_peered":"2024-04-25T01:25:53.361156+0000","last_unstale":"2024-04-25T01:40:58.791228+0000","last_undegraded":"2024-04-25T01:40:58.791228+0000","last_fullsized":"2024-04-25T01:40:58.791228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4812,"log_dups_size":0,"ondisk_log_size":4812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:27.472315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25777,"num_write_kb":295962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"26'5","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.233531+0000","last_change":"2024-04-25T01:25:52.354499+0000","last_active":"2024-04-25T01:40:49.233531+0000","last_peered":"2024-04-25T01:40:49.233531+0000","last_clean":"2024-04-25T01:40:49.233531+0000","last_became_active":"2024-04-25T01:25:52.354193+0000","last_became_peered":"2024-04-25T01:25:52.354193+0000","last_unstale":"2024-04-25T01:40:49.233531+0000","last_undegraded":"2024-04-25T01:40:49.233531+0000","last_fullsized":"2024-04-25T01:40:49.233531+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"26'5","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.016444+0000","last_change":"2024-04-25T01:26:01.418186+0000","last_active":"2024-04-25T01:40:37.016444+0000","last_peered":"2024-04-25T01:40:37.016444+0000","last_clean":"2024-04-25T01:40:37.016444+0000","last_became_active":"2024-04-25T01:26:01.417805+0000","last_became_peered":"2024-04-25T01:26:01.417805+0000","last_unstale":"2024-04-25T01:40:37.016444+0000","last_undegraded":"2024-04-25T01:40:37.016444+0000","last_fullsized":"2024-04-25T01:40:37.016444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:19.629335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"26'3951","reported_seq":3973,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.320321+0000","last_change":"2024-04-25T01:26:02.415877+0000","last_active":"2024-04-25T01:41:09.320321+0000","last_peered":"2024-04-25T01:41:09.320321+0000","last_clean":"2024-04-25T01:41:09.320321+0000","last_became_active":"2024-04-25T01:26:02.415619+0000","last_became_peered":"2024-04-25T01:26:02.415619+0000","last_unstale":"2024-04-25T01:41:09.320321+0000","last_undegraded":"2024-04-25T01:41:09.320321+0000","last_fullsized":"2024-04-25T01:41:09.320321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3951,"log_dups_size":0,"ondisk_log_size":3951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:14.647083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20033,"num_write_kb":252185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"26'4985","reported_seq":5016,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.752192+0000","last_change":"2024-04-25T01:25:53.362922+0000","last_active":"2024-04-25T01:40:58.752192+0000","last_peered":"2024-04-25T01:40:58.752192+0000","last_clean":"2024-04-25T01:40:58.752192+0000","last_became_active":"2024-04-25T01:25:53.362533+0000","last_became_peered":"2024-04-25T01:25:53.362533+0000","last_unstale":"2024-04-25T01:40:58.752192+0000","last_undegraded":"2024-04-25T01:40:58.752192+0000","last_fullsized":"2024-04-25T01:40:58.752192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4985,"log_dups_size":0,"ondisk_log_size":4985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26332,"num_write_kb":298165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"26'43","reported_seq":117,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:47.854708+0000","last_change":"2024-04-25T01:25:52.345321+0000","last_active":"2024-04-25T01:40:47.854708+0000","last_peered":"2024-04-25T01:40:47.854708+0000","last_clean":"2024-04-25T01:40:47.854708+0000","last_became_active":"2024-04-25T01:25:52.345061+0000","last_became_peered":"2024-04-25T01:25:52.345061+0000","last_unstale":"2024-04-25T01:40:47.854708+0000","last_undegraded":"2024-04-25T01:40:47.854708+0000","last_fullsized":"2024-04-25T01:40:47.854708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":41,"num_read_kb":115,"num_write":41,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"26'4","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:36.977064+0000","last_change":"2024-04-25T01:26:01.418516+0000","last_active":"2024-04-25T01:40:36.977064+0000","last_peered":"2024-04-25T01:40:36.977064+0000","last_clean":"2024-04-25T01:40:36.977064+0000","last_became_active":"2024-04-25T01:26:01.418173+0000","last_became_peered":"2024-04-25T01:26:01.418173+0000","last_unstale":"2024-04-25T01:40:36.977064+0000","last_undegraded":"2024-04-25T01:40:36.977064+0000","last_fullsized":"2024-04-25T01:40:36.977064+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:02.568472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"26'4134","reported_seq":4156,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.333895+0000","last_change":"2024-04-25T01:26:02.418115+0000","last_active":"2024-04-25T01:41:09.333895+0000","last_peered":"2024-04-25T01:41:09.333895+0000","last_clean":"2024-04-25T01:41:09.333895+0000","last_became_active":"2024-04-25T01:26:02.417355+0000","last_became_peered":"2024-04-25T01:26:02.417355+0000","last_unstale":"2024-04-25T01:41:09.333895+0000","last_undegraded":"2024-04-25T01:41:09.333895+0000","last_fullsized":"2024-04-25T01:41:09.333895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4134,"log_dups_size":0,"ondisk_log_size":4134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:19.644468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21715,"num_write_kb":287743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"26'4344","reported_seq":4375,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.752413+0000","last_change":"2024-04-25T01:25:53.355950+0000","last_active":"2024-04-25T01:40:58.752413+0000","last_peered":"2024-04-25T01:40:58.752413+0000","last_clean":"2024-04-25T01:40:58.752413+0000","last_became_active":"2024-04-25T01:25:53.355648+0000","last_became_peered":"2024-04-25T01:25:53.355648+0000","last_unstale":"2024-04-25T01:40:58.752413+0000","last_undegraded":"2024-04-25T01:40:58.752413+0000","last_fullsized":"2024-04-25T01:40:58.752413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4344,"log_dups_size":0,"ondisk_log_size":4344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23132,"num_write_kb":289328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"26'5","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.249850+0000","last_change":"2024-04-25T01:25:52.357683+0000","last_active":"2024-04-25T01:40:49.249850+0000","last_peered":"2024-04-25T01:40:49.249850+0000","last_clean":"2024-04-25T01:40:49.249850+0000","last_became_active":"2024-04-25T01:25:52.356924+0000","last_became_peered":"2024-04-25T01:25:52.356924+0000","last_unstale":"2024-04-25T01:40:49.249850+0000","last_undegraded":"2024-04-25T01:40:49.249850+0000","last_fullsized":"2024-04-25T01:40:49.249850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"26'570","reported_seq":595,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:47.577546+0000","last_change":"2024-04-25T01:26:01.421260+0000","last_active":"2024-04-25T01:40:47.577546+0000","last_peered":"2024-04-25T01:40:47.577546+0000","last_clean":"2024-04-25T01:40:47.577546+0000","last_became_active":"2024-04-25T01:26:01.420831+0000","last_became_peered":"2024-04-25T01:26:01.420831+0000","last_unstale":"2024-04-25T01:40:47.577546+0000","last_undegraded":"2024-04-25T01:40:47.577546+0000","last_fullsized":"2024-04-25T01:40:47.577546+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1330,"num_write_kb":427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"26'4947","reported_seq":4970,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.324019+0000","last_change":"2024-04-25T01:26:02.421386+0000","last_active":"2024-04-25T01:41:09.324019+0000","last_peered":"2024-04-25T01:41:09.324019+0000","last_clean":"2024-04-25T01:41:09.324019+0000","last_became_active":"2024-04-25T01:26:02.421079+0000","last_became_peered":"2024-04-25T01:26:02.421079+0000","last_unstale":"2024-04-25T01:41:09.324019+0000","last_undegraded":"2024-04-25T01:41:09.324019+0000","last_fullsized":"2024-04-25T01:41:09.324019+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4947,"log_dups_size":0,"ondisk_log_size":4947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25964,"num_write_kb":313022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"26'4754","reported_seq":4770,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.812843+0000","last_change":"2024-04-25T01:26:52.515572+0000","last_active":"2024-04-25T01:40:58.812843+0000","last_peered":"2024-04-25T01:40:58.812843+0000","last_clean":"2024-04-25T01:40:58.812843+0000","last_became_active":"2024-04-25T01:26:52.515254+0000","last_became_peered":"2024-04-25T01:26:52.515254+0000","last_unstale":"2024-04-25T01:40:58.812843+0000","last_undegraded":"2024-04-25T01:40:58.812843+0000","last_fullsized":"2024-04-25T01:40:58.812843+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4754,"log_dups_size":0,"ondisk_log_size":4754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:08.975868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24235,"num_write_kb":310192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:38:25.919823+0000","last_change":"2024-04-25T01:25:52.353155+0000","last_active":"2024-04-25T01:38:25.919823+0000","last_peered":"2024-04-25T01:38:25.919823+0000","last_clean":"2024-04-25T01:38:25.919823+0000","last_became_active":"2024-04-25T01:25:52.352826+0000","last_became_peered":"2024-04-25T01:25:52.352826+0000","last_unstale":"2024-04-25T01:38:25.919823+0000","last_undegraded":"2024-04-25T01:38:25.919823+0000","last_fullsized":"2024-04-25T01:38:25.919823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":189,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:37:45.823358+0000","last_change":"2024-04-25T01:25:41.719670+0000","last_active":"2024-04-25T01:37:45.823358+0000","last_peered":"2024-04-25T01:37:45.823358+0000","last_clean":"2024-04-25T01:37:45.823358+0000","last_became_active":"2024-04-25T01:25:41.718721+0000","last_became_peered":"2024-04-25T01:25:41.718721+0000","last_unstale":"2024-04-25T01:37:45.823358+0000","last_undegraded":"2024-04-25T01:37:45.823358+0000","last_fullsized":"2024-04-25T01:37:45.823358+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_clean_scrub_stamp":"2024-04-25T01:25:40.688097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:27.456746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"26'4","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.023240+0000","last_change":"2024-04-25T01:26:01.409339+0000","last_active":"2024-04-25T01:40:37.023240+0000","last_peered":"2024-04-25T01:40:37.023240+0000","last_clean":"2024-04-25T01:40:37.023240+0000","last_became_active":"2024-04-25T01:26:01.409062+0000","last_became_peered":"2024-04-25T01:26:01.409062+0000","last_unstale":"2024-04-25T01:40:37.023240+0000","last_undegraded":"2024-04-25T01:40:37.023240+0000","last_fullsized":"2024-04-25T01:40:37.023240+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:52.561251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"26'3833","reported_seq":3855,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.306093+0000","last_change":"2024-04-25T01:26:02.418574+0000","last_active":"2024-04-25T01:41:09.306093+0000","last_peered":"2024-04-25T01:41:09.306093+0000","last_clean":"2024-04-25T01:41:09.306093+0000","last_became_active":"2024-04-25T01:26:02.416546+0000","last_became_peered":"2024-04-25T01:26:02.416546+0000","last_unstale":"2024-04-25T01:41:09.306093+0000","last_undegraded":"2024-04-25T01:41:09.306093+0000","last_fullsized":"2024-04-25T01:41:09.306093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3833,"log_dups_size":0,"ondisk_log_size":3833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:43.994141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19504,"num_write_kb":250193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"26'5320","reported_seq":5350,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.811822+0000","last_change":"2024-04-25T01:25:53.375523+0000","last_active":"2024-04-25T01:40:58.811822+0000","last_peered":"2024-04-25T01:40:58.811822+0000","last_clean":"2024-04-25T01:40:58.811822+0000","last_became_active":"2024-04-25T01:25:53.375219+0000","last_became_peered":"2024-04-25T01:25:53.375219+0000","last_unstale":"2024-04-25T01:40:58.811822+0000","last_undegraded":"2024-04-25T01:40:58.811822+0000","last_fullsized":"2024-04-25T01:40:58.811822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5320,"log_dups_size":0,"ondisk_log_size":5320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27324,"num_write_kb":314375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'3","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.181066+0000","last_change":"2024-04-25T01:25:52.352994+0000","last_active":"2024-04-25T01:40:49.181066+0000","last_peered":"2024-04-25T01:40:49.181066+0000","last_clean":"2024-04-25T01:40:49.181066+0000","last_became_active":"2024-04-25T01:25:52.352644+0000","last_became_peered":"2024-04-25T01:25:52.352644+0000","last_unstale":"2024-04-25T01:40:49.181066+0000","last_undegraded":"2024-04-25T01:40:49.181066+0000","last_fullsized":"2024-04-25T01:40:49.181066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"26'3","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.140689+0000","last_change":"2024-04-25T01:26:01.420891+0000","last_active":"2024-04-25T01:40:37.140689+0000","last_peered":"2024-04-25T01:40:37.140689+0000","last_clean":"2024-04-25T01:40:37.140689+0000","last_became_active":"2024-04-25T01:26:01.420664+0000","last_became_peered":"2024-04-25T01:26:01.420664+0000","last_unstale":"2024-04-25T01:40:37.140689+0000","last_undegraded":"2024-04-25T01:40:37.140689+0000","last_fullsized":"2024-04-25T01:40:37.140689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"26'4000","reported_seq":4022,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.297815+0000","last_change":"2024-04-25T01:26:02.403489+0000","last_active":"2024-04-25T01:41:09.297815+0000","last_peered":"2024-04-25T01:41:09.297815+0000","last_clean":"2024-04-25T01:41:09.297815+0000","last_became_active":"2024-04-25T01:26:02.403206+0000","last_became_peered":"2024-04-25T01:26:02.403206+0000","last_unstale":"2024-04-25T01:41:09.297815+0000","last_undegraded":"2024-04-25T01:41:09.297815+0000","last_fullsized":"2024-04-25T01:41:09.297815+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4000,"log_dups_size":0,"ondisk_log_size":4000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:49.872754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21231,"num_write_kb":269526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"26'4796","reported_seq":4826,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.770361+0000","last_change":"2024-04-25T01:25:53.371018+0000","last_active":"2024-04-25T01:40:58.770361+0000","last_peered":"2024-04-25T01:40:58.770361+0000","last_clean":"2024-04-25T01:40:58.770361+0000","last_became_active":"2024-04-25T01:25:53.370539+0000","last_became_peered":"2024-04-25T01:25:53.370539+0000","last_unstale":"2024-04-25T01:40:58.770361+0000","last_undegraded":"2024-04-25T01:40:58.770361+0000","last_fullsized":"2024-04-25T01:40:58.770361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25182,"num_write_kb":313860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"26'2","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:39:25.987518+0000","last_change":"2024-04-25T01:25:52.355133+0000","last_active":"2024-04-25T01:39:25.987518+0000","last_peered":"2024-04-25T01:39:25.987518+0000","last_clean":"2024-04-25T01:39:25.987518+0000","last_became_active":"2024-04-25T01:25:52.354836+0000","last_became_peered":"2024-04-25T01:25:52.354836+0000","last_unstale":"2024-04-25T01:39:25.987518+0000","last_undegraded":"2024-04-25T01:39:25.987518+0000","last_fullsized":"2024-04-25T01:39:25.987518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:10.331704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"26'1112","reported_seq":1136,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.176654+0000","last_change":"2024-04-25T01:26:01.419285+0000","last_active":"2024-04-25T01:40:37.176654+0000","last_peered":"2024-04-25T01:40:37.176654+0000","last_clean":"2024-04-25T01:40:37.176654+0000","last_became_active":"2024-04-25T01:26:01.418987+0000","last_became_peered":"2024-04-25T01:26:01.418987+0000","last_unstale":"2024-04-25T01:40:37.176654+0000","last_undegraded":"2024-04-25T01:40:37.176654+0000","last_fullsized":"2024-04-25T01:40:37.176654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1112,"log_dups_size":0,"ondisk_log_size":1112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1103,"num_write_kb":4430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"26'4082","reported_seq":4104,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.262432+0000","last_change":"2024-04-25T01:26:02.416544+0000","last_active":"2024-04-25T01:41:09.262432+0000","last_peered":"2024-04-25T01:41:09.262432+0000","last_clean":"2024-04-25T01:41:09.262432+0000","last_became_active":"2024-04-25T01:26:02.416189+0000","last_became_peered":"2024-04-25T01:26:02.416189+0000","last_unstale":"2024-04-25T01:41:09.262432+0000","last_undegraded":"2024-04-25T01:41:09.262432+0000","last_fullsized":"2024-04-25T01:41:09.262432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4082,"log_dups_size":0,"ondisk_log_size":4082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.688469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20562,"num_write_kb":274915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"26'5666","reported_seq":5696,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.889157+0000","last_change":"2024-04-25T01:25:53.370754+0000","last_active":"2024-04-25T01:40:58.889157+0000","last_peered":"2024-04-25T01:40:58.889157+0000","last_clean":"2024-04-25T01:40:58.889157+0000","last_became_active":"2024-04-25T01:25:53.370074+0000","last_became_peered":"2024-04-25T01:25:53.370074+0000","last_unstale":"2024-04-25T01:40:58.889157+0000","last_undegraded":"2024-04-25T01:40:58.889157+0000","last_fullsized":"2024-04-25T01:40:58.889157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5666,"log_dups_size":0,"ondisk_log_size":5666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29161,"num_write_kb":354533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"26'6","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.237424+0000","last_change":"2024-04-25T01:25:52.345943+0000","last_active":"2024-04-25T01:40:49.237424+0000","last_peered":"2024-04-25T01:40:49.237424+0000","last_clean":"2024-04-25T01:40:49.237424+0000","last_became_active":"2024-04-25T01:25:52.345717+0000","last_became_peered":"2024-04-25T01:25:52.345717+0000","last_unstale":"2024-04-25T01:40:49.237424+0000","last_undegraded":"2024-04-25T01:40:49.237424+0000","last_fullsized":"2024-04-25T01:40:49.237424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"26'3","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.011634+0000","last_change":"2024-04-25T01:26:01.416027+0000","last_active":"2024-04-25T01:40:37.011634+0000","last_peered":"2024-04-25T01:40:37.011634+0000","last_clean":"2024-04-25T01:40:37.011634+0000","last_became_active":"2024-04-25T01:26:01.415750+0000","last_became_peered":"2024-04-25T01:26:01.415750+0000","last_unstale":"2024-04-25T01:40:37.011634+0000","last_undegraded":"2024-04-25T01:40:37.011634+0000","last_fullsized":"2024-04-25T01:40:37.011634+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:27.042376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"26'4748","reported_seq":4770,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.276555+0000","last_change":"2024-04-25T01:26:02.414929+0000","last_active":"2024-04-25T01:41:09.276555+0000","last_peered":"2024-04-25T01:41:09.276555+0000","last_clean":"2024-04-25T01:41:09.276555+0000","last_became_active":"2024-04-25T01:26:02.414316+0000","last_became_peered":"2024-04-25T01:26:02.414316+0000","last_unstale":"2024-04-25T01:41:09.276555+0000","last_undegraded":"2024-04-25T01:41:09.276555+0000","last_fullsized":"2024-04-25T01:41:09.276555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4748,"log_dups_size":0,"ondisk_log_size":4748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:53.568065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24613,"num_write_kb":303491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"26'4546","reported_seq":4576,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.712213+0000","last_change":"2024-04-25T01:25:53.365171+0000","last_active":"2024-04-25T01:40:58.712213+0000","last_peered":"2024-04-25T01:40:58.712213+0000","last_clean":"2024-04-25T01:40:58.712213+0000","last_became_active":"2024-04-25T01:25:53.364049+0000","last_became_peered":"2024-04-25T01:25:53.364049+0000","last_unstale":"2024-04-25T01:40:58.712213+0000","last_undegraded":"2024-04-25T01:40:58.712213+0000","last_fullsized":"2024-04-25T01:40:58.712213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4546,"log_dups_size":0,"ondisk_log_size":4546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22740,"num_write_kb":287925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'5","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.243400+0000","last_change":"2024-04-25T01:25:52.350688+0000","last_active":"2024-04-25T01:40:49.243400+0000","last_peered":"2024-04-25T01:40:49.243400+0000","last_clean":"2024-04-25T01:40:49.243400+0000","last_became_active":"2024-04-25T01:25:52.350357+0000","last_became_peered":"2024-04-25T01:25:52.350357+0000","last_unstale":"2024-04-25T01:40:49.243400+0000","last_undegraded":"2024-04-25T01:40:49.243400+0000","last_fullsized":"2024-04-25T01:40:49.243400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"26'3","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.002855+0000","last_change":"2024-04-25T01:26:01.418580+0000","last_active":"2024-04-25T01:40:37.002855+0000","last_peered":"2024-04-25T01:40:37.002855+0000","last_clean":"2024-04-25T01:40:37.002855+0000","last_became_active":"2024-04-25T01:26:01.418179+0000","last_became_peered":"2024-04-25T01:26:01.418179+0000","last_unstale":"2024-04-25T01:40:37.002855+0000","last_undegraded":"2024-04-25T01:40:37.002855+0000","last_fullsized":"2024-04-25T01:40:37.002855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:42.987327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"26'3833","reported_seq":3855,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.301162+0000","last_change":"2024-04-25T01:26:02.418424+0000","last_active":"2024-04-25T01:41:09.301162+0000","last_peered":"2024-04-25T01:41:09.301162+0000","last_clean":"2024-04-25T01:41:09.301162+0000","last_became_active":"2024-04-25T01:26:02.418090+0000","last_became_peered":"2024-04-25T01:26:02.418090+0000","last_unstale":"2024-04-25T01:41:09.301162+0000","last_undegraded":"2024-04-25T01:41:09.301162+0000","last_fullsized":"2024-04-25T01:41:09.301162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3833,"log_dups_size":0,"ondisk_log_size":3833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:10.049310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19507,"num_write_kb":254331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"26'4627","reported_seq":4657,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.811217+0000","last_change":"2024-04-25T01:25:53.366376+0000","last_active":"2024-04-25T01:40:58.811217+0000","last_peered":"2024-04-25T01:40:58.811217+0000","last_clean":"2024-04-25T01:40:58.811217+0000","last_became_active":"2024-04-25T01:25:53.365715+0000","last_became_peered":"2024-04-25T01:25:53.365715+0000","last_unstale":"2024-04-25T01:40:58.811217+0000","last_undegraded":"2024-04-25T01:40:58.811217+0000","last_fullsized":"2024-04-25T01:40:58.811217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4627,"log_dups_size":0,"ondisk_log_size":4627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24780,"num_write_kb":300200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"26'11","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.216654+0000","last_change":"2024-04-25T01:25:52.347413+0000","last_active":"2024-04-25T01:40:49.216654+0000","last_peered":"2024-04-25T01:40:49.216654+0000","last_clean":"2024-04-25T01:40:49.216654+0000","last_became_active":"2024-04-25T01:25:52.347144+0000","last_became_peered":"2024-04-25T01:25:52.347144+0000","last_unstale":"2024-04-25T01:40:49.216654+0000","last_undegraded":"2024-04-25T01:40:49.216654+0000","last_fullsized":"2024-04-25T01:40:49.216654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"26'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.101937+0000","last_change":"2024-04-25T01:26:01.409917+0000","last_active":"2024-04-25T01:40:37.101937+0000","last_peered":"2024-04-25T01:40:37.101937+0000","last_clean":"2024-04-25T01:40:37.101937+0000","last_became_active":"2024-04-25T01:26:01.409654+0000","last_became_peered":"2024-04-25T01:26:01.409654+0000","last_unstale":"2024-04-25T01:40:37.101937+0000","last_undegraded":"2024-04-25T01:40:37.101937+0000","last_fullsized":"2024-04-25T01:40:37.101937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:37.142255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"26'4668","reported_seq":4690,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.342103+0000","last_change":"2024-04-25T01:26:02.425103+0000","last_active":"2024-04-25T01:41:09.342103+0000","last_peered":"2024-04-25T01:41:09.342103+0000","last_clean":"2024-04-25T01:41:09.342103+0000","last_became_active":"2024-04-25T01:26:02.424819+0000","last_became_peered":"2024-04-25T01:26:02.424819+0000","last_unstale":"2024-04-25T01:41:09.342103+0000","last_undegraded":"2024-04-25T01:41:09.342103+0000","last_fullsized":"2024-04-25T01:41:09.342103+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4668,"log_dups_size":0,"ondisk_log_size":4668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23987,"num_write_kb":301090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"26'5240","reported_seq":5270,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.796905+0000","last_change":"2024-04-25T01:25:53.364987+0000","last_active":"2024-04-25T01:40:58.796905+0000","last_peered":"2024-04-25T01:40:58.796905+0000","last_clean":"2024-04-25T01:40:58.796905+0000","last_became_active":"2024-04-25T01:25:53.364431+0000","last_became_peered":"2024-04-25T01:25:53.364431+0000","last_unstale":"2024-04-25T01:40:58.796905+0000","last_undegraded":"2024-04-25T01:40:58.796905+0000","last_fullsized":"2024-04-25T01:40:58.796905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5240,"log_dups_size":0,"ondisk_log_size":5240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27403,"num_write_kb":326994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'6","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.234588+0000","last_change":"2024-04-25T01:25:52.351205+0000","last_active":"2024-04-25T01:40:49.234588+0000","last_peered":"2024-04-25T01:40:49.234588+0000","last_clean":"2024-04-25T01:40:49.234588+0000","last_became_active":"2024-04-25T01:25:52.350946+0000","last_became_peered":"2024-04-25T01:25:52.350946+0000","last_unstale":"2024-04-25T01:40:49.234588+0000","last_undegraded":"2024-04-25T01:40:49.234588+0000","last_fullsized":"2024-04-25T01:40:49.234588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"26'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:31.329770+0000","last_change":"2024-04-25T01:26:01.415799+0000","last_active":"2024-04-25T01:40:31.329770+0000","last_peered":"2024-04-25T01:40:31.329770+0000","last_clean":"2024-04-25T01:40:31.329770+0000","last_became_active":"2024-04-25T01:26:01.415572+0000","last_became_peered":"2024-04-25T01:26:01.415572+0000","last_unstale":"2024-04-25T01:40:31.329770+0000","last_undegraded":"2024-04-25T01:40:31.329770+0000","last_fullsized":"2024-04-25T01:40:31.329770+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:54.034132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"26'4341","reported_seq":4363,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.334259+0000","last_change":"2024-04-25T01:26:02.422959+0000","last_active":"2024-04-25T01:41:09.334259+0000","last_peered":"2024-04-25T01:41:09.334259+0000","last_clean":"2024-04-25T01:41:09.334259+0000","last_became_active":"2024-04-25T01:26:02.422210+0000","last_became_peered":"2024-04-25T01:26:02.422210+0000","last_unstale":"2024-04-25T01:41:09.334259+0000","last_undegraded":"2024-04-25T01:41:09.334259+0000","last_fullsized":"2024-04-25T01:41:09.334259+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4341,"log_dups_size":0,"ondisk_log_size":4341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22600,"num_write_kb":283130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"26'4919","reported_seq":4949,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.871808+0000","last_change":"2024-04-25T01:25:53.368187+0000","last_active":"2024-04-25T01:40:58.871808+0000","last_peered":"2024-04-25T01:40:58.871808+0000","last_clean":"2024-04-25T01:40:58.871808+0000","last_became_active":"2024-04-25T01:25:53.367664+0000","last_became_peered":"2024-04-25T01:25:53.367664+0000","last_unstale":"2024-04-25T01:40:58.871808+0000","last_undegraded":"2024-04-25T01:40:58.871808+0000","last_fullsized":"2024-04-25T01:40:58.871808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4919,"log_dups_size":0,"ondisk_log_size":4919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25274,"num_write_kb":330818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"26'5","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.184872+0000","last_change":"2024-04-25T01:25:52.357397+0000","last_active":"2024-04-25T01:40:49.184872+0000","last_peered":"2024-04-25T01:40:49.184872+0000","last_clean":"2024-04-25T01:40:49.184872+0000","last_became_active":"2024-04-25T01:25:52.356881+0000","last_became_peered":"2024-04-25T01:25:52.356881+0000","last_unstale":"2024-04-25T01:40:49.184872+0000","last_undegraded":"2024-04-25T01:40:49.184872+0000","last_fullsized":"2024-04-25T01:40:49.184872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"26'314","reported_seq":338,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:46.230895+0000","last_change":"2024-04-25T01:26:01.420103+0000","last_active":"2024-04-25T01:40:46.230895+0000","last_peered":"2024-04-25T01:40:46.230895+0000","last_clean":"2024-04-25T01:40:46.230895+0000","last_became_active":"2024-04-25T01:26:01.419526+0000","last_became_peered":"2024-04-25T01:26:01.419526+0000","last_unstale":"2024-04-25T01:40:46.230895+0000","last_undegraded":"2024-04-25T01:40:46.230895+0000","last_fullsized":"2024-04-25T01:40:46.230895+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:27.042376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":310,"num_write_kb":4240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"26'6253","reported_seq":6275,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.317329+0000","last_change":"2024-04-25T01:26:02.421488+0000","last_active":"2024-04-25T01:41:09.317329+0000","last_peered":"2024-04-25T01:41:09.317329+0000","last_clean":"2024-04-25T01:41:09.317329+0000","last_became_active":"2024-04-25T01:26:02.419308+0000","last_became_peered":"2024-04-25T01:26:02.419308+0000","last_unstale":"2024-04-25T01:41:09.317329+0000","last_undegraded":"2024-04-25T01:41:09.317329+0000","last_fullsized":"2024-04-25T01:41:09.317329+0000","mapping_epoch":22,"log_start":"26'1000","ondisk_log_start":"26'1000","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5253,"log_dups_size":0,"ondisk_log_size":5253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32276,"num_write_kb":400144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"26'4645","reported_seq":4675,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.788191+0000","last_change":"2024-04-25T01:25:53.370720+0000","last_active":"2024-04-25T01:40:58.788191+0000","last_peered":"2024-04-25T01:40:58.788191+0000","last_clean":"2024-04-25T01:40:58.788191+0000","last_became_active":"2024-04-25T01:25:53.370142+0000","last_became_peered":"2024-04-25T01:25:53.370142+0000","last_unstale":"2024-04-25T01:40:58.788191+0000","last_undegraded":"2024-04-25T01:40:58.788191+0000","last_fullsized":"2024-04-25T01:40:58.788191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4645,"log_dups_size":0,"ondisk_log_size":4645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:52.833379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24130,"num_write_kb":289357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"26'8","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.232990+0000","last_change":"2024-04-25T01:25:52.349092+0000","last_active":"2024-04-25T01:40:49.232990+0000","last_peered":"2024-04-25T01:40:49.232990+0000","last_clean":"2024-04-25T01:40:49.232990+0000","last_became_active":"2024-04-25T01:25:52.348846+0000","last_became_peered":"2024-04-25T01:25:52.348846+0000","last_unstale":"2024-04-25T01:40:49.232990+0000","last_undegraded":"2024-04-25T01:40:49.232990+0000","last_fullsized":"2024-04-25T01:40:49.232990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:21.982440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"26'26","reported_seq":50,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.278610+0000","last_change":"2024-04-25T01:26:01.414517+0000","last_active":"2024-04-25T01:40:37.278610+0000","last_peered":"2024-04-25T01:40:37.278610+0000","last_clean":"2024-04-25T01:40:37.278610+0000","last_became_active":"2024-04-25T01:26:01.414243+0000","last_became_peered":"2024-04-25T01:26:01.414243+0000","last_unstale":"2024-04-25T01:40:37.278610+0000","last_undegraded":"2024-04-25T01:40:37.278610+0000","last_fullsized":"2024-04-25T01:40:37.278610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:48.491461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"26'5295","reported_seq":5317,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.307012+0000","last_change":"2024-04-25T01:26:02.415893+0000","last_active":"2024-04-25T01:41:09.307012+0000","last_peered":"2024-04-25T01:41:09.307012+0000","last_clean":"2024-04-25T01:41:09.307012+0000","last_became_active":"2024-04-25T01:26:02.415596+0000","last_became_peered":"2024-04-25T01:26:02.415596+0000","last_unstale":"2024-04-25T01:41:09.307012+0000","last_undegraded":"2024-04-25T01:41:09.307012+0000","last_fullsized":"2024-04-25T01:41:09.307012+0000","mapping_epoch":22,"log_start":"26'100","ondisk_log_start":"26'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5195,"log_dups_size":0,"ondisk_log_size":5195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26962,"num_write_kb":337426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"26'4577","reported_seq":4607,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.793341+0000","last_change":"2024-04-25T01:25:53.373572+0000","last_active":"2024-04-25T01:40:58.793341+0000","last_peered":"2024-04-25T01:40:58.793341+0000","last_clean":"2024-04-25T01:40:58.793341+0000","last_became_active":"2024-04-25T01:25:53.373129+0000","last_became_peered":"2024-04-25T01:25:53.373129+0000","last_unstale":"2024-04-25T01:40:58.793341+0000","last_undegraded":"2024-04-25T01:40:58.793341+0000","last_fullsized":"2024-04-25T01:40:58.793341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23671,"num_write_kb":287382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'335","reported_seq":367,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.269952+0000","last_change":"2024-04-25T01:25:52.350446+0000","last_active":"2024-04-25T01:40:49.269952+0000","last_peered":"2024-04-25T01:40:49.269952+0000","last_clean":"2024-04-25T01:40:49.269952+0000","last_became_active":"2024-04-25T01:25:52.349448+0000","last_became_peered":"2024-04-25T01:25:52.349448+0000","last_unstale":"2024-04-25T01:40:49.269952+0000","last_undegraded":"2024-04-25T01:40:49.269952+0000","last_fullsized":"2024-04-25T01:40:49.269952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":331,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"26'5","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.111146+0000","last_change":"2024-04-25T01:26:01.418794+0000","last_active":"2024-04-25T01:40:37.111146+0000","last_peered":"2024-04-25T01:40:37.111146+0000","last_clean":"2024-04-25T01:40:37.111146+0000","last_became_active":"2024-04-25T01:26:01.418541+0000","last_became_peered":"2024-04-25T01:26:01.418541+0000","last_unstale":"2024-04-25T01:40:37.111146+0000","last_undegraded":"2024-04-25T01:40:37.111146+0000","last_fullsized":"2024-04-25T01:40:37.111146+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"26'4257","reported_seq":4279,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.301321+0000","last_change":"2024-04-25T01:26:02.425391+0000","last_active":"2024-04-25T01:41:09.301321+0000","last_peered":"2024-04-25T01:41:09.301321+0000","last_clean":"2024-04-25T01:41:09.301321+0000","last_became_active":"2024-04-25T01:26:02.425106+0000","last_became_peered":"2024-04-25T01:26:02.425106+0000","last_unstale":"2024-04-25T01:41:09.301321+0000","last_undegraded":"2024-04-25T01:41:09.301321+0000","last_fullsized":"2024-04-25T01:41:09.301321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4257,"log_dups_size":0,"ondisk_log_size":4257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22152,"num_write_kb":281377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"26'6968","reported_seq":6998,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.878933+0000","last_change":"2024-04-25T01:25:53.366416+0000","last_active":"2024-04-25T01:40:58.878933+0000","last_peered":"2024-04-25T01:40:58.878933+0000","last_clean":"2024-04-25T01:40:58.878933+0000","last_became_active":"2024-04-25T01:25:53.366157+0000","last_became_peered":"2024-04-25T01:25:53.366157+0000","last_unstale":"2024-04-25T01:40:58.878933+0000","last_undegraded":"2024-04-25T01:40:58.878933+0000","last_fullsized":"2024-04-25T01:40:58.878933+0000","mapping_epoch":18,"log_start":"26'1500","ondisk_log_start":"26'1500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5468,"log_dups_size":0,"ondisk_log_size":5468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:26.277566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35763,"num_write_kb":414188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"26'25","reported_seq":57,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.254786+0000","last_change":"2024-04-25T01:25:52.357610+0000","last_active":"2024-04-25T01:40:49.254786+0000","last_peered":"2024-04-25T01:40:49.254786+0000","last_clean":"2024-04-25T01:40:49.254786+0000","last_became_active":"2024-04-25T01:25:52.357355+0000","last_became_peered":"2024-04-25T01:25:52.357355+0000","last_unstale":"2024-04-25T01:40:49.254786+0000","last_undegraded":"2024-04-25T01:40:49.254786+0000","last_fullsized":"2024-04-25T01:40:49.254786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"26'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.122870+0000","last_change":"2024-04-25T01:26:01.412666+0000","last_active":"2024-04-25T01:40:37.122870+0000","last_peered":"2024-04-25T01:40:37.122870+0000","last_clean":"2024-04-25T01:40:37.122870+0000","last_became_active":"2024-04-25T01:26:01.412132+0000","last_became_peered":"2024-04-25T01:26:01.412132+0000","last_unstale":"2024-04-25T01:40:37.122870+0000","last_undegraded":"2024-04-25T01:40:37.122870+0000","last_fullsized":"2024-04-25T01:40:37.122870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"26'5141","reported_seq":5163,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.286915+0000","last_change":"2024-04-25T01:26:02.413106+0000","last_active":"2024-04-25T01:41:09.286915+0000","last_peered":"2024-04-25T01:41:09.286915+0000","last_clean":"2024-04-25T01:41:09.286915+0000","last_became_active":"2024-04-25T01:26:02.412759+0000","last_became_peered":"2024-04-25T01:26:02.412759+0000","last_unstale":"2024-04-25T01:41:09.286915+0000","last_undegraded":"2024-04-25T01:41:09.286915+0000","last_fullsized":"2024-04-25T01:41:09.286915+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5141,"log_dups_size":0,"ondisk_log_size":5141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:28.049190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27471,"num_write_kb":310956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"26'3595","reported_seq":3625,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.782418+0000","last_change":"2024-04-25T01:25:53.374311+0000","last_active":"2024-04-25T01:40:58.782418+0000","last_peered":"2024-04-25T01:40:58.782418+0000","last_clean":"2024-04-25T01:40:58.782418+0000","last_became_active":"2024-04-25T01:25:53.373890+0000","last_became_peered":"2024-04-25T01:25:53.373890+0000","last_unstale":"2024-04-25T01:40:58.782418+0000","last_undegraded":"2024-04-25T01:40:58.782418+0000","last_fullsized":"2024-04-25T01:40:58.782418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3595,"log_dups_size":0,"ondisk_log_size":3595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18396,"num_write_kb":249899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"26'2","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.190934+0000","last_change":"2024-04-25T01:25:52.356231+0000","last_active":"2024-04-25T01:40:49.190934+0000","last_peered":"2024-04-25T01:40:49.190934+0000","last_clean":"2024-04-25T01:40:49.190934+0000","last_became_active":"2024-04-25T01:25:52.355913+0000","last_became_peered":"2024-04-25T01:25:52.355913+0000","last_unstale":"2024-04-25T01:40:49.190934+0000","last_undegraded":"2024-04-25T01:40:49.190934+0000","last_fullsized":"2024-04-25T01:40:49.190934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"26'12","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.017092+0000","last_change":"2024-04-25T01:26:01.419513+0000","last_active":"2024-04-25T01:40:37.017092+0000","last_peered":"2024-04-25T01:40:37.017092+0000","last_clean":"2024-04-25T01:40:37.017092+0000","last_became_active":"2024-04-25T01:26:01.419015+0000","last_became_peered":"2024-04-25T01:26:01.419015+0000","last_unstale":"2024-04-25T01:40:37.017092+0000","last_undegraded":"2024-04-25T01:40:37.017092+0000","last_fullsized":"2024-04-25T01:40:37.017092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:10.486367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"26'5013","reported_seq":5035,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.306814+0000","last_change":"2024-04-25T01:26:02.425241+0000","last_active":"2024-04-25T01:41:09.306814+0000","last_peered":"2024-04-25T01:41:09.306814+0000","last_clean":"2024-04-25T01:41:09.306814+0000","last_became_active":"2024-04-25T01:26:02.424960+0000","last_became_peered":"2024-04-25T01:26:02.424960+0000","last_unstale":"2024-04-25T01:41:09.306814+0000","last_undegraded":"2024-04-25T01:41:09.306814+0000","last_fullsized":"2024-04-25T01:41:09.306814+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5013,"log_dups_size":0,"ondisk_log_size":5013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25777,"num_write_kb":320486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"26'4876","reported_seq":4906,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.800754+0000","last_change":"2024-04-25T01:25:53.371445+0000","last_active":"2024-04-25T01:40:58.800754+0000","last_peered":"2024-04-25T01:40:58.800754+0000","last_clean":"2024-04-25T01:40:58.800754+0000","last_became_active":"2024-04-25T01:25:53.371086+0000","last_became_peered":"2024-04-25T01:25:53.371086+0000","last_unstale":"2024-04-25T01:40:58.800754+0000","last_undegraded":"2024-04-25T01:40:58.800754+0000","last_fullsized":"2024-04-25T01:40:58.800754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4876,"log_dups_size":0,"ondisk_log_size":4876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24656,"num_write_kb":303790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"26'2","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:26.547207+0000","last_change":"2024-04-25T01:25:52.357459+0000","last_active":"2024-04-25T01:40:26.547207+0000","last_peered":"2024-04-25T01:40:26.547207+0000","last_clean":"2024-04-25T01:40:26.547207+0000","last_became_active":"2024-04-25T01:25:52.357182+0000","last_became_peered":"2024-04-25T01:25:52.357182+0000","last_unstale":"2024-04-25T01:40:26.547207+0000","last_undegraded":"2024-04-25T01:40:26.547207+0000","last_fullsized":"2024-04-25T01:40:26.547207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"26'6","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.088116+0000","last_change":"2024-04-25T01:26:01.415321+0000","last_active":"2024-04-25T01:40:37.088116+0000","last_peered":"2024-04-25T01:40:37.088116+0000","last_clean":"2024-04-25T01:40:37.088116+0000","last_became_active":"2024-04-25T01:26:01.415020+0000","last_became_peered":"2024-04-25T01:26:01.415020+0000","last_unstale":"2024-04-25T01:40:37.088116+0000","last_undegraded":"2024-04-25T01:40:37.088116+0000","last_fullsized":"2024-04-25T01:40:37.088116+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:20.090411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"26'5316","reported_seq":5339,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.331286+0000","last_change":"2024-04-25T01:26:02.416917+0000","last_active":"2024-04-25T01:41:09.331286+0000","last_peered":"2024-04-25T01:41:09.331286+0000","last_clean":"2024-04-25T01:41:09.331286+0000","last_became_active":"2024-04-25T01:26:02.416623+0000","last_became_peered":"2024-04-25T01:26:02.416623+0000","last_unstale":"2024-04-25T01:41:09.331286+0000","last_undegraded":"2024-04-25T01:41:09.331286+0000","last_fullsized":"2024-04-25T01:41:09.331286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5316,"log_dups_size":0,"ondisk_log_size":5316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:55.040946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27734,"num_write_kb":356956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"26'8","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.028842+0000","last_change":"2024-04-25T01:26:01.415472+0000","last_active":"2024-04-25T01:40:37.028842+0000","last_peered":"2024-04-25T01:40:37.028842+0000","last_clean":"2024-04-25T01:40:37.028842+0000","last_became_active":"2024-04-25T01:26:01.415184+0000","last_became_peered":"2024-04-25T01:26:01.415184+0000","last_unstale":"2024-04-25T01:40:37.028842+0000","last_undegraded":"2024-04-25T01:40:37.028842+0000","last_fullsized":"2024-04-25T01:40:37.028842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"26'5623","reported_seq":5653,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.800035+0000","last_change":"2024-04-25T01:25:53.363043+0000","last_active":"2024-04-25T01:40:58.800035+0000","last_peered":"2024-04-25T01:40:58.800035+0000","last_clean":"2024-04-25T01:40:58.800035+0000","last_became_active":"2024-04-25T01:25:53.362601+0000","last_became_peered":"2024-04-25T01:25:53.362601+0000","last_unstale":"2024-04-25T01:40:58.800035+0000","last_undegraded":"2024-04-25T01:40:58.800035+0000","last_fullsized":"2024-04-25T01:40:58.800035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5623,"log_dups_size":0,"ondisk_log_size":5623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29125,"num_write_kb":354366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"26'3","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:46.309270+0000","last_change":"2024-04-25T01:25:52.358081+0000","last_active":"2024-04-25T01:40:46.309270+0000","last_peered":"2024-04-25T01:40:46.309270+0000","last_clean":"2024-04-25T01:40:46.309270+0000","last_became_active":"2024-04-25T01:25:52.357365+0000","last_became_peered":"2024-04-25T01:25:52.357365+0000","last_unstale":"2024-04-25T01:40:46.309270+0000","last_undegraded":"2024-04-25T01:40:46.309270+0000","last_fullsized":"2024-04-25T01:40:46.309270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"26'4779","reported_seq":4802,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.260694+0000","last_change":"2024-04-25T01:26:02.418872+0000","last_active":"2024-04-25T01:41:09.260694+0000","last_peered":"2024-04-25T01:41:09.260694+0000","last_clean":"2024-04-25T01:41:09.260694+0000","last_became_active":"2024-04-25T01:26:02.418614+0000","last_became_peered":"2024-04-25T01:26:02.418614+0000","last_unstale":"2024-04-25T01:41:09.260694+0000","last_undegraded":"2024-04-25T01:41:09.260694+0000","last_fullsized":"2024-04-25T01:41:09.260694+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4779,"log_dups_size":0,"ondisk_log_size":4779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:49.498275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25207,"num_write_kb":318198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"26'632","reported_seq":657,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:27.924243+0000","last_change":"2024-04-25T01:26:01.410258+0000","last_active":"2024-04-25T01:40:27.924243+0000","last_peered":"2024-04-25T01:40:27.924243+0000","last_clean":"2024-04-25T01:40:27.924243+0000","last_became_active":"2024-04-25T01:26:01.410004+0000","last_became_peered":"2024-04-25T01:26:01.410004+0000","last_unstale":"2024-04-25T01:40:27.924243+0000","last_undegraded":"2024-04-25T01:40:27.924243+0000","last_fullsized":"2024-04-25T01:40:27.924243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:48.865940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":630,"num_write_kb":4461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"26'4416","reported_seq":4446,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.812059+0000","last_change":"2024-04-25T01:25:53.363794+0000","last_active":"2024-04-25T01:40:58.812059+0000","last_peered":"2024-04-25T01:40:58.812059+0000","last_clean":"2024-04-25T01:40:58.812059+0000","last_became_active":"2024-04-25T01:25:53.363509+0000","last_became_peered":"2024-04-25T01:25:53.363509+0000","last_unstale":"2024-04-25T01:40:58.812059+0000","last_undegraded":"2024-04-25T01:40:58.812059+0000","last_fullsized":"2024-04-25T01:40:58.812059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4416,"log_dups_size":0,"ondisk_log_size":4416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23348,"num_write_kb":294385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"26'3","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:26.190581+0000","last_change":"2024-04-25T01:25:52.347464+0000","last_active":"2024-04-25T01:40:26.190581+0000","last_peered":"2024-04-25T01:40:26.190581+0000","last_clean":"2024-04-25T01:40:26.190581+0000","last_became_active":"2024-04-25T01:25:52.347138+0000","last_became_peered":"2024-04-25T01:25:52.347138+0000","last_unstale":"2024-04-25T01:40:26.190581+0000","last_undegraded":"2024-04-25T01:40:26.190581+0000","last_fullsized":"2024-04-25T01:40:26.190581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"26'5575","reported_seq":5597,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.362996+0000","last_change":"2024-04-25T01:26:02.425948+0000","last_active":"2024-04-25T01:41:09.362996+0000","last_peered":"2024-04-25T01:41:09.362996+0000","last_clean":"2024-04-25T01:41:09.362996+0000","last_became_active":"2024-04-25T01:26:02.424947+0000","last_became_peered":"2024-04-25T01:26:02.424947+0000","last_unstale":"2024-04-25T01:41:09.362996+0000","last_undegraded":"2024-04-25T01:41:09.362996+0000","last_fullsized":"2024-04-25T01:41:09.362996+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5575,"log_dups_size":0,"ondisk_log_size":5575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28821,"num_write_kb":361321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"26'87","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.639358+0000","last_change":"2024-04-25T01:25:52.349989+0000","last_active":"2024-04-25T01:40:58.639358+0000","last_peered":"2024-04-25T01:40:58.639358+0000","last_clean":"2024-04-25T01:40:58.639358+0000","last_became_active":"2024-04-25T01:25:52.349672+0000","last_became_peered":"2024-04-25T01:25:52.349672+0000","last_unstale":"2024-04-25T01:40:58.639358+0000","last_undegraded":"2024-04-25T01:40:58.639358+0000","last_fullsized":"2024-04-25T01:40:58.639358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":980823,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83,"num_write_kb":1015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"26'3832","reported_seq":3862,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.794598+0000","last_change":"2024-04-25T01:25:53.356610+0000","last_active":"2024-04-25T01:40:58.794598+0000","last_peered":"2024-04-25T01:40:58.794598+0000","last_clean":"2024-04-25T01:40:58.794598+0000","last_became_active":"2024-04-25T01:25:53.356015+0000","last_became_peered":"2024-04-25T01:25:53.356015+0000","last_unstale":"2024-04-25T01:40:58.794598+0000","last_undegraded":"2024-04-25T01:40:58.794598+0000","last_fullsized":"2024-04-25T01:40:58.794598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3832,"log_dups_size":0,"ondisk_log_size":3832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19386,"num_write_kb":237359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"26'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.169856+0000","last_change":"2024-04-25T01:26:01.420261+0000","last_active":"2024-04-25T01:40:37.169856+0000","last_peered":"2024-04-25T01:40:37.169856+0000","last_clean":"2024-04-25T01:40:37.169856+0000","last_became_active":"2024-04-25T01:26:01.420003+0000","last_became_peered":"2024-04-25T01:26:01.420003+0000","last_unstale":"2024-04-25T01:40:37.169856+0000","last_undegraded":"2024-04-25T01:40:37.169856+0000","last_fullsized":"2024-04-25T01:40:37.169856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:28.695853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"26'3175","reported_seq":3197,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.349312+0000","last_change":"2024-04-25T01:26:02.426327+0000","last_active":"2024-04-25T01:41:09.349312+0000","last_peered":"2024-04-25T01:41:09.349312+0000","last_clean":"2024-04-25T01:41:09.349312+0000","last_became_active":"2024-04-25T01:26:02.426054+0000","last_became_peered":"2024-04-25T01:26:02.426054+0000","last_unstale":"2024-04-25T01:41:09.349312+0000","last_undegraded":"2024-04-25T01:41:09.349312+0000","last_fullsized":"2024-04-25T01:41:09.349312+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3175,"log_dups_size":0,"ondisk_log_size":3175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:17.809738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15854,"num_write_kb":211025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"26'605","reported_seq":637,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.248248+0000","last_change":"2024-04-25T01:25:52.353955+0000","last_active":"2024-04-25T01:40:49.248248+0000","last_peered":"2024-04-25T01:40:49.248248+0000","last_clean":"2024-04-25T01:40:49.248248+0000","last_became_active":"2024-04-25T01:25:52.353659+0000","last_became_peered":"2024-04-25T01:25:52.353659+0000","last_unstale":"2024-04-25T01:40:49.248248+0000","last_undegraded":"2024-04-25T01:40:49.248248+0000","last_fullsized":"2024-04-25T01:40:49.248248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":603,"num_write_kb":4239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"26'4191","reported_seq":4221,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.749061+0000","last_change":"2024-04-25T01:25:53.366586+0000","last_active":"2024-04-25T01:40:58.749061+0000","last_peered":"2024-04-25T01:40:58.749061+0000","last_clean":"2024-04-25T01:40:58.749061+0000","last_became_active":"2024-04-25T01:25:53.366302+0000","last_became_peered":"2024-04-25T01:25:53.366302+0000","last_unstale":"2024-04-25T01:40:58.749061+0000","last_undegraded":"2024-04-25T01:40:58.749061+0000","last_fullsized":"2024-04-25T01:40:58.749061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4191,"log_dups_size":0,"ondisk_log_size":4191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21716,"num_write_kb":263159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"26'11","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.173685+0000","last_change":"2024-04-25T01:26:01.420960+0000","last_active":"2024-04-25T01:40:37.173685+0000","last_peered":"2024-04-25T01:40:37.173685+0000","last_clean":"2024-04-25T01:40:37.173685+0000","last_became_active":"2024-04-25T01:26:01.420713+0000","last_became_peered":"2024-04-25T01:26:01.420713+0000","last_unstale":"2024-04-25T01:40:37.173685+0000","last_undegraded":"2024-04-25T01:40:37.173685+0000","last_fullsized":"2024-04-25T01:40:37.173685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:29.489275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"26'4569","reported_seq":4591,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.350310+0000","last_change":"2024-04-25T01:26:02.419282+0000","last_active":"2024-04-25T01:41:09.350310+0000","last_peered":"2024-04-25T01:41:09.350310+0000","last_clean":"2024-04-25T01:41:09.350310+0000","last_became_active":"2024-04-25T01:26:02.418802+0000","last_became_peered":"2024-04-25T01:26:02.418802+0000","last_unstale":"2024-04-25T01:41:09.350310+0000","last_undegraded":"2024-04-25T01:41:09.350310+0000","last_fullsized":"2024-04-25T01:41:09.350310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4569,"log_dups_size":0,"ondisk_log_size":4569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:46.807175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23161,"num_write_kb":277521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"26'6240","reported_seq":6271,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.802380+0000","last_change":"2024-04-25T01:25:53.367143+0000","last_active":"2024-04-25T01:40:58.802380+0000","last_peered":"2024-04-25T01:40:58.802380+0000","last_clean":"2024-04-25T01:40:58.802380+0000","last_became_active":"2024-04-25T01:25:53.366847+0000","last_became_peered":"2024-04-25T01:25:53.366847+0000","last_unstale":"2024-04-25T01:40:58.802380+0000","last_undegraded":"2024-04-25T01:40:58.802380+0000","last_fullsized":"2024-04-25T01:40:58.802380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":6240,"log_dups_size":0,"ondisk_log_size":6240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32134,"num_write_kb":382834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"26'5302","reported_seq":5324,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.305354+0000","last_change":"2024-04-25T01:26:02.416438+0000","last_active":"2024-04-25T01:41:09.305354+0000","last_peered":"2024-04-25T01:41:09.305354+0000","last_clean":"2024-04-25T01:41:09.305354+0000","last_became_active":"2024-04-25T01:26:02.416170+0000","last_became_peered":"2024-04-25T01:26:02.416170+0000","last_unstale":"2024-04-25T01:41:09.305354+0000","last_undegraded":"2024-04-25T01:41:09.305354+0000","last_fullsized":"2024-04-25T01:41:09.305354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5302,"log_dups_size":0,"ondisk_log_size":5302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27000,"num_write_kb":345927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"26'4854","reported_seq":4885,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.807931+0000","last_change":"2024-04-25T01:25:53.371345+0000","last_active":"2024-04-25T01:40:58.807931+0000","last_peered":"2024-04-25T01:40:58.807931+0000","last_clean":"2024-04-25T01:40:58.807931+0000","last_became_active":"2024-04-25T01:25:53.371080+0000","last_became_peered":"2024-04-25T01:25:53.371080+0000","last_unstale":"2024-04-25T01:40:58.807931+0000","last_undegraded":"2024-04-25T01:40:58.807931+0000","last_fullsized":"2024-04-25T01:40:58.807931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4854,"log_dups_size":0,"ondisk_log_size":4854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:09.855763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24922,"num_write_kb":316898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"26'4098","reported_seq":4121,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.322931+0000","last_change":"2024-04-25T01:26:02.419688+0000","last_active":"2024-04-25T01:41:09.322931+0000","last_peered":"2024-04-25T01:41:09.322931+0000","last_clean":"2024-04-25T01:41:09.322931+0000","last_became_active":"2024-04-25T01:26:02.419417+0000","last_became_peered":"2024-04-25T01:26:02.419417+0000","last_unstale":"2024-04-25T01:41:09.322931+0000","last_undegraded":"2024-04-25T01:41:09.322931+0000","last_fullsized":"2024-04-25T01:41:09.322931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4098,"log_dups_size":0,"ondisk_log_size":4098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21368,"num_write_kb":269972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"26'5565","reported_seq":5596,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.839128+0000","last_change":"2024-04-25T01:25:53.369850+0000","last_active":"2024-04-25T01:40:58.839128+0000","last_peered":"2024-04-25T01:40:58.839128+0000","last_clean":"2024-04-25T01:40:58.839128+0000","last_became_active":"2024-04-25T01:25:53.369350+0000","last_became_peered":"2024-04-25T01:25:53.369350+0000","last_unstale":"2024-04-25T01:40:58.839128+0000","last_undegraded":"2024-04-25T01:40:58.839128+0000","last_fullsized":"2024-04-25T01:40:58.839128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5565,"log_dups_size":0,"ondisk_log_size":5565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29144,"num_write_kb":362518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"26'5067","reported_seq":5089,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.356968+0000","last_change":"2024-04-25T01:26:02.425528+0000","last_active":"2024-04-25T01:41:09.356968+0000","last_peered":"2024-04-25T01:41:09.356968+0000","last_clean":"2024-04-25T01:41:09.356968+0000","last_became_active":"2024-04-25T01:26:02.425160+0000","last_became_peered":"2024-04-25T01:26:02.425160+0000","last_unstale":"2024-04-25T01:41:09.356968+0000","last_undegraded":"2024-04-25T01:41:09.356968+0000","last_fullsized":"2024-04-25T01:41:09.356968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26737,"num_write_kb":324333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"26'3787","reported_seq":3818,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.775758+0000","last_change":"2024-04-25T01:25:53.357142+0000","last_active":"2024-04-25T01:40:58.775758+0000","last_peered":"2024-04-25T01:40:58.775758+0000","last_clean":"2024-04-25T01:40:58.775758+0000","last_became_active":"2024-04-25T01:25:53.356854+0000","last_became_peered":"2024-04-25T01:25:53.356854+0000","last_unstale":"2024-04-25T01:40:58.775758+0000","last_undegraded":"2024-04-25T01:40:58.775758+0000","last_fullsized":"2024-04-25T01:40:58.775758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3787,"log_dups_size":0,"ondisk_log_size":3787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19900,"num_write_kb":255902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"26'5481","reported_seq":5504,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.289690+0000","last_change":"2024-04-25T01:26:02.417492+0000","last_active":"2024-04-25T01:41:09.289690+0000","last_peered":"2024-04-25T01:41:09.289690+0000","last_clean":"2024-04-25T01:41:09.289690+0000","last_became_active":"2024-04-25T01:26:02.417234+0000","last_became_peered":"2024-04-25T01:26:02.417234+0000","last_unstale":"2024-04-25T01:41:09.289690+0000","last_undegraded":"2024-04-25T01:41:09.289690+0000","last_fullsized":"2024-04-25T01:41:09.289690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5481,"log_dups_size":0,"ondisk_log_size":5481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28430,"num_write_kb":359662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"26'4160","reported_seq":4191,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.791055+0000","last_change":"2024-04-25T01:25:53.364575+0000","last_active":"2024-04-25T01:40:58.791055+0000","last_peered":"2024-04-25T01:40:58.791055+0000","last_clean":"2024-04-25T01:40:58.791055+0000","last_became_active":"2024-04-25T01:25:53.364311+0000","last_became_peered":"2024-04-25T01:25:53.364311+0000","last_unstale":"2024-04-25T01:40:58.791055+0000","last_undegraded":"2024-04-25T01:40:58.791055+0000","last_fullsized":"2024-04-25T01:40:58.791055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4160,"log_dups_size":0,"ondisk_log_size":4160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21579,"num_write_kb":270844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"26'4125","reported_seq":4148,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.329055+0000","last_change":"2024-04-25T01:26:02.426813+0000","last_active":"2024-04-25T01:41:09.329055+0000","last_peered":"2024-04-25T01:41:09.329055+0000","last_clean":"2024-04-25T01:41:09.329055+0000","last_became_active":"2024-04-25T01:26:02.426552+0000","last_became_peered":"2024-04-25T01:26:02.426552+0000","last_unstale":"2024-04-25T01:41:09.329055+0000","last_undegraded":"2024-04-25T01:41:09.329055+0000","last_fullsized":"2024-04-25T01:41:09.329055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4125,"log_dups_size":0,"ondisk_log_size":4125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:03.687174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21785,"num_write_kb":263458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"26'5101","reported_seq":5132,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.740912+0000","last_change":"2024-04-25T01:25:53.368413+0000","last_active":"2024-04-25T01:40:58.740912+0000","last_peered":"2024-04-25T01:40:58.740912+0000","last_clean":"2024-04-25T01:40:58.740912+0000","last_became_active":"2024-04-25T01:25:53.368176+0000","last_became_peered":"2024-04-25T01:25:53.368176+0000","last_unstale":"2024-04-25T01:40:58.740912+0000","last_undegraded":"2024-04-25T01:40:58.740912+0000","last_fullsized":"2024-04-25T01:40:58.740912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5101,"log_dups_size":0,"ondisk_log_size":5101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26850,"num_write_kb":324721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"26'6020","reported_seq":6043,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.350276+0000","last_change":"2024-04-25T01:26:02.425185+0000","last_active":"2024-04-25T01:41:09.350276+0000","last_peered":"2024-04-25T01:41:09.350276+0000","last_clean":"2024-04-25T01:41:09.350276+0000","last_became_active":"2024-04-25T01:26:02.424903+0000","last_became_peered":"2024-04-25T01:26:02.424903+0000","last_unstale":"2024-04-25T01:41:09.350276+0000","last_undegraded":"2024-04-25T01:41:09.350276+0000","last_fullsized":"2024-04-25T01:41:09.350276+0000","mapping_epoch":22,"log_start":"26'300","ondisk_log_start":"26'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5720,"log_dups_size":0,"ondisk_log_size":5720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31332,"num_write_kb":379614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"26'4058","reported_seq":4089,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.867006+0000","last_change":"2024-04-25T01:25:53.372638+0000","last_active":"2024-04-25T01:40:58.867006+0000","last_peered":"2024-04-25T01:40:58.867006+0000","last_clean":"2024-04-25T01:40:58.867006+0000","last_became_active":"2024-04-25T01:25:53.372001+0000","last_became_peered":"2024-04-25T01:25:53.372001+0000","last_unstale":"2024-04-25T01:40:58.867006+0000","last_undegraded":"2024-04-25T01:40:58.867006+0000","last_fullsized":"2024-04-25T01:40:58.867006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4058,"log_dups_size":0,"ondisk_log_size":4058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20502,"num_write_kb":266785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"26'4843","reported_seq":4866,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.307262+0000","last_change":"2024-04-25T01:26:02.420867+0000","last_active":"2024-04-25T01:41:09.307262+0000","last_peered":"2024-04-25T01:41:09.307262+0000","last_clean":"2024-04-25T01:41:09.307262+0000","last_became_active":"2024-04-25T01:26:02.420621+0000","last_became_peered":"2024-04-25T01:26:02.420621+0000","last_unstale":"2024-04-25T01:41:09.307262+0000","last_undegraded":"2024-04-25T01:41:09.307262+0000","last_fullsized":"2024-04-25T01:41:09.307262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4843,"log_dups_size":0,"ondisk_log_size":4843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:11.493181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24777,"num_write_kb":316390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"26'5268","reported_seq":5299,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.743440+0000","last_change":"2024-04-25T01:25:53.365509+0000","last_active":"2024-04-25T01:40:58.743440+0000","last_peered":"2024-04-25T01:40:58.743440+0000","last_clean":"2024-04-25T01:40:58.743440+0000","last_became_active":"2024-04-25T01:25:53.365245+0000","last_became_peered":"2024-04-25T01:25:53.365245+0000","last_unstale":"2024-04-25T01:40:58.743440+0000","last_undegraded":"2024-04-25T01:40:58.743440+0000","last_fullsized":"2024-04-25T01:40:58.743440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5268,"log_dups_size":0,"ondisk_log_size":5268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26981,"num_write_kb":345791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"26'3558","reported_seq":3580,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.356542+0000","last_change":"2024-04-25T01:26:02.415672+0000","last_active":"2024-04-25T01:41:09.356542+0000","last_peered":"2024-04-25T01:41:09.356542+0000","last_clean":"2024-04-25T01:41:09.356542+0000","last_became_active":"2024-04-25T01:26:02.415389+0000","last_became_peered":"2024-04-25T01:26:02.415389+0000","last_unstale":"2024-04-25T01:41:09.356542+0000","last_undegraded":"2024-04-25T01:41:09.356542+0000","last_fullsized":"2024-04-25T01:41:09.356542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3558,"log_dups_size":0,"ondisk_log_size":3558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:52.342053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18097,"num_write_kb":236200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"26'4040","reported_seq":4071,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.818105+0000","last_change":"2024-04-25T01:25:53.365391+0000","last_active":"2024-04-25T01:40:58.818105+0000","last_peered":"2024-04-25T01:40:58.818105+0000","last_clean":"2024-04-25T01:40:58.818105+0000","last_became_active":"2024-04-25T01:25:53.364829+0000","last_became_peered":"2024-04-25T01:25:53.364829+0000","last_unstale":"2024-04-25T01:40:58.818105+0000","last_undegraded":"2024-04-25T01:40:58.818105+0000","last_fullsized":"2024-04-25T01:40:58.818105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4040,"log_dups_size":0,"ondisk_log_size":4040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20607,"num_write_kb":262822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"26'5431","reported_seq":5453,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.318622+0000","last_change":"2024-04-25T01:26:02.418292+0000","last_active":"2024-04-25T01:41:09.318622+0000","last_peered":"2024-04-25T01:41:09.318622+0000","last_clean":"2024-04-25T01:41:09.318622+0000","last_became_active":"2024-04-25T01:26:02.418036+0000","last_became_peered":"2024-04-25T01:26:02.418036+0000","last_unstale":"2024-04-25T01:41:09.318622+0000","last_undegraded":"2024-04-25T01:41:09.318622+0000","last_fullsized":"2024-04-25T01:41:09.318622+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5431,"log_dups_size":0,"ondisk_log_size":5431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28059,"num_write_kb":358266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"26'4709","reported_seq":4739,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.812514+0000","last_change":"2024-04-25T01:25:53.355667+0000","last_active":"2024-04-25T01:40:58.812514+0000","last_peered":"2024-04-25T01:40:58.812514+0000","last_clean":"2024-04-25T01:40:58.812514+0000","last_became_active":"2024-04-25T01:25:53.355417+0000","last_became_peered":"2024-04-25T01:25:53.355417+0000","last_unstale":"2024-04-25T01:40:58.812514+0000","last_undegraded":"2024-04-25T01:40:58.812514+0000","last_fullsized":"2024-04-25T01:40:58.812514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4709,"log_dups_size":0,"ondisk_log_size":4709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24394,"num_write_kb":298399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"26'4741","reported_seq":4763,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.305268+0000","last_change":"2024-04-25T01:26:02.414994+0000","last_active":"2024-04-25T01:41:09.305268+0000","last_peered":"2024-04-25T01:41:09.305268+0000","last_clean":"2024-04-25T01:41:09.305268+0000","last_became_active":"2024-04-25T01:26:02.414735+0000","last_became_peered":"2024-04-25T01:26:02.414735+0000","last_unstale":"2024-04-25T01:41:09.305268+0000","last_undegraded":"2024-04-25T01:41:09.305268+0000","last_fullsized":"2024-04-25T01:41:09.305268+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4741,"log_dups_size":0,"ondisk_log_size":4741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:44.960251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24824,"num_write_kb":300285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"26'5111","reported_seq":5141,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.869446+0000","last_change":"2024-04-25T01:25:53.371240+0000","last_active":"2024-04-25T01:40:58.869446+0000","last_peered":"2024-04-25T01:40:58.869446+0000","last_clean":"2024-04-25T01:40:58.869446+0000","last_became_active":"2024-04-25T01:25:53.370164+0000","last_became_peered":"2024-04-25T01:25:53.370164+0000","last_unstale":"2024-04-25T01:40:58.869446+0000","last_undegraded":"2024-04-25T01:40:58.869446+0000","last_fullsized":"2024-04-25T01:40:58.869446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26542,"num_write_kb":327744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"26'4367","reported_seq":4389,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.310155+0000","last_change":"2024-04-25T01:26:02.418272+0000","last_active":"2024-04-25T01:41:09.310155+0000","last_peered":"2024-04-25T01:41:09.310155+0000","last_clean":"2024-04-25T01:41:09.310155+0000","last_became_active":"2024-04-25T01:26:02.417977+0000","last_became_peered":"2024-04-25T01:26:02.417977+0000","last_unstale":"2024-04-25T01:41:09.310155+0000","last_undegraded":"2024-04-25T01:41:09.310155+0000","last_fullsized":"2024-04-25T01:41:09.310155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4367,"log_dups_size":0,"ondisk_log_size":4367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22024,"num_write_kb":293008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"26'5285","reported_seq":5315,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.820507+0000","last_change":"2024-04-25T01:25:53.376415+0000","last_active":"2024-04-25T01:40:58.820507+0000","last_peered":"2024-04-25T01:40:58.820507+0000","last_clean":"2024-04-25T01:40:58.820507+0000","last_became_active":"2024-04-25T01:25:53.376104+0000","last_became_peered":"2024-04-25T01:25:53.376104+0000","last_unstale":"2024-04-25T01:40:58.820507+0000","last_undegraded":"2024-04-25T01:40:58.820507+0000","last_fullsized":"2024-04-25T01:40:58.820507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5285,"log_dups_size":0,"ondisk_log_size":5285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27331,"num_write_kb":318531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"26'5020","reported_seq":5042,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.344122+0000","last_change":"2024-04-25T01:26:02.426333+0000","last_active":"2024-04-25T01:41:09.344122+0000","last_peered":"2024-04-25T01:41:09.344122+0000","last_clean":"2024-04-25T01:41:09.344122+0000","last_became_active":"2024-04-25T01:26:02.426069+0000","last_became_peered":"2024-04-25T01:26:02.426069+0000","last_unstale":"2024-04-25T01:41:09.344122+0000","last_undegraded":"2024-04-25T01:41:09.344122+0000","last_fullsized":"2024-04-25T01:41:09.344122+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5020,"log_dups_size":0,"ondisk_log_size":5020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26121,"num_write_kb":325976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"26'4910","reported_seq":4940,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.774391+0000","last_change":"2024-04-25T01:25:53.370314+0000","last_active":"2024-04-25T01:40:58.774391+0000","last_peered":"2024-04-25T01:40:58.774391+0000","last_clean":"2024-04-25T01:40:58.774391+0000","last_became_active":"2024-04-25T01:25:53.370049+0000","last_became_peered":"2024-04-25T01:25:53.370049+0000","last_unstale":"2024-04-25T01:40:58.774391+0000","last_undegraded":"2024-04-25T01:40:58.774391+0000","last_fullsized":"2024-04-25T01:40:58.774391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4910,"log_dups_size":0,"ondisk_log_size":4910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25833,"num_write_kb":304449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"26'4775","reported_seq":4797,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.332568+0000","last_change":"2024-04-25T01:26:02.415464+0000","last_active":"2024-04-25T01:41:09.332568+0000","last_peered":"2024-04-25T01:41:09.332568+0000","last_clean":"2024-04-25T01:41:09.332568+0000","last_became_active":"2024-04-25T01:26:02.415173+0000","last_became_peered":"2024-04-25T01:26:02.415173+0000","last_unstale":"2024-04-25T01:41:09.332568+0000","last_undegraded":"2024-04-25T01:41:09.332568+0000","last_fullsized":"2024-04-25T01:41:09.332568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4775,"log_dups_size":0,"ondisk_log_size":4775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.343169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24075,"num_write_kb":305295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"26'4891","reported_seq":4921,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.797939+0000","last_change":"2024-04-25T01:25:53.369585+0000","last_active":"2024-04-25T01:40:58.797939+0000","last_peered":"2024-04-25T01:40:58.797939+0000","last_clean":"2024-04-25T01:40:58.797939+0000","last_became_active":"2024-04-25T01:25:53.368499+0000","last_became_peered":"2024-04-25T01:25:53.368499+0000","last_unstale":"2024-04-25T01:40:58.797939+0000","last_undegraded":"2024-04-25T01:40:58.797939+0000","last_fullsized":"2024-04-25T01:40:58.797939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4891,"log_dups_size":0,"ondisk_log_size":4891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24867,"num_write_kb":304528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"26'4199","reported_seq":4221,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.286525+0000","last_change":"2024-04-25T01:26:02.421975+0000","last_active":"2024-04-25T01:41:09.286525+0000","last_peered":"2024-04-25T01:41:09.286525+0000","last_clean":"2024-04-25T01:41:09.286525+0000","last_became_active":"2024-04-25T01:26:02.421698+0000","last_became_peered":"2024-04-25T01:26:02.421698+0000","last_unstale":"2024-04-25T01:41:09.286525+0000","last_undegraded":"2024-04-25T01:41:09.286525+0000","last_fullsized":"2024-04-25T01:41:09.286525+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4199,"log_dups_size":0,"ondisk_log_size":4199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21738,"num_write_kb":267270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"26'3844","reported_seq":3874,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.786269+0000","last_change":"2024-04-25T01:25:53.370666+0000","last_active":"2024-04-25T01:40:58.786269+0000","last_peered":"2024-04-25T01:40:58.786269+0000","last_clean":"2024-04-25T01:40:58.786269+0000","last_became_active":"2024-04-25T01:25:53.369733+0000","last_became_peered":"2024-04-25T01:25:53.369733+0000","last_unstale":"2024-04-25T01:40:58.786269+0000","last_undegraded":"2024-04-25T01:40:58.786269+0000","last_fullsized":"2024-04-25T01:40:58.786269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3844,"log_dups_size":0,"ondisk_log_size":3844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19367,"num_write_kb":241344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"26'4316","reported_seq":4338,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.253195+0000","last_change":"2024-04-25T01:26:02.418114+0000","last_active":"2024-04-25T01:41:09.253195+0000","last_peered":"2024-04-25T01:41:09.253195+0000","last_clean":"2024-04-25T01:41:09.253195+0000","last_became_active":"2024-04-25T01:26:02.417810+0000","last_became_peered":"2024-04-25T01:26:02.417810+0000","last_unstale":"2024-04-25T01:41:09.253195+0000","last_undegraded":"2024-04-25T01:41:09.253195+0000","last_fullsized":"2024-04-25T01:41:09.253195+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4316,"log_dups_size":0,"ondisk_log_size":4316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:30.496089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22517,"num_write_kb":290939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"26'5814","reported_seq":5830,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.796893+0000","last_change":"2024-04-25T01:26:52.120824+0000","last_active":"2024-04-25T01:40:58.796893+0000","last_peered":"2024-04-25T01:40:58.796893+0000","last_clean":"2024-04-25T01:40:58.796893+0000","last_became_active":"2024-04-25T01:26:52.120588+0000","last_became_peered":"2024-04-25T01:26:52.120588+0000","last_unstale":"2024-04-25T01:40:58.796893+0000","last_undegraded":"2024-04-25T01:40:58.796893+0000","last_fullsized":"2024-04-25T01:40:58.796893+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5814,"log_dups_size":0,"ondisk_log_size":5814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:14.466644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29929,"num_write_kb":365779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.36","version":"26'4743","reported_seq":4765,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.335933+0000","last_change":"2024-04-25T01:26:02.426289+0000","last_active":"2024-04-25T01:41:09.335933+0000","last_peered":"2024-04-25T01:41:09.335933+0000","last_clean":"2024-04-25T01:41:09.335933+0000","last_became_active":"2024-04-25T01:26:02.426015+0000","last_became_peered":"2024-04-25T01:26:02.426015+0000","last_unstale":"2024-04-25T01:41:09.335933+0000","last_undegraded":"2024-04-25T01:41:09.335933+0000","last_fullsized":"2024-04-25T01:41:09.335933+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4743,"log_dups_size":0,"ondisk_log_size":4743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25470,"num_write_kb":331505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"26'4366","reported_seq":4396,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.821883+0000","last_change":"2024-04-25T01:25:53.374186+0000","last_active":"2024-04-25T01:40:58.821883+0000","last_peered":"2024-04-25T01:40:58.821883+0000","last_clean":"2024-04-25T01:40:58.821883+0000","last_became_active":"2024-04-25T01:25:53.373766+0000","last_became_peered":"2024-04-25T01:25:53.373766+0000","last_unstale":"2024-04-25T01:40:58.821883+0000","last_undegraded":"2024-04-25T01:40:58.821883+0000","last_fullsized":"2024-04-25T01:40:58.821883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4366,"log_dups_size":0,"ondisk_log_size":4366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23328,"num_write_kb":273725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"26'5875","reported_seq":5897,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.356318+0000","last_change":"2024-04-25T01:26:02.416345+0000","last_active":"2024-04-25T01:41:09.356318+0000","last_peered":"2024-04-25T01:41:09.356318+0000","last_clean":"2024-04-25T01:41:09.356318+0000","last_became_active":"2024-04-25T01:26:02.416082+0000","last_became_peered":"2024-04-25T01:26:02.416082+0000","last_unstale":"2024-04-25T01:41:09.356318+0000","last_undegraded":"2024-04-25T01:41:09.356318+0000","last_fullsized":"2024-04-25T01:41:09.356318+0000","mapping_epoch":22,"log_start":"26'400","ondisk_log_start":"26'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5475,"log_dups_size":0,"ondisk_log_size":5475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:47.810943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31009,"num_write_kb":370059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"26'4898","reported_seq":4928,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.812217+0000","last_change":"2024-04-25T01:25:53.371602+0000","last_active":"2024-04-25T01:40:58.812217+0000","last_peered":"2024-04-25T01:40:58.812217+0000","last_clean":"2024-04-25T01:40:58.812217+0000","last_became_active":"2024-04-25T01:25:53.371362+0000","last_became_peered":"2024-04-25T01:25:53.371362+0000","last_unstale":"2024-04-25T01:40:58.812217+0000","last_undegraded":"2024-04-25T01:40:58.812217+0000","last_fullsized":"2024-04-25T01:40:58.812217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4898,"log_dups_size":0,"ondisk_log_size":4898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24919,"num_write_kb":308746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"26'3961","reported_seq":3983,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.316956+0000","last_change":"2024-04-25T01:26:02.425888+0000","last_active":"2024-04-25T01:41:09.316956+0000","last_peered":"2024-04-25T01:41:09.316956+0000","last_clean":"2024-04-25T01:41:09.316956+0000","last_became_active":"2024-04-25T01:26:02.425577+0000","last_became_peered":"2024-04-25T01:26:02.425577+0000","last_unstale":"2024-04-25T01:41:09.316956+0000","last_undegraded":"2024-04-25T01:41:09.316956+0000","last_fullsized":"2024-04-25T01:41:09.316956+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3961,"log_dups_size":0,"ondisk_log_size":3961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20296,"num_write_kb":253395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"26'4670","reported_seq":4700,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.788790+0000","last_change":"2024-04-25T01:25:53.370996+0000","last_active":"2024-04-25T01:40:58.788790+0000","last_peered":"2024-04-25T01:40:58.788790+0000","last_clean":"2024-04-25T01:40:58.788790+0000","last_became_active":"2024-04-25T01:25:53.370479+0000","last_became_peered":"2024-04-25T01:25:53.370479+0000","last_unstale":"2024-04-25T01:40:58.788790+0000","last_undegraded":"2024-04-25T01:40:58.788790+0000","last_fullsized":"2024-04-25T01:40:58.788790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4670,"log_dups_size":0,"ondisk_log_size":4670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:57.636067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24421,"num_write_kb":306681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"26'4733","reported_seq":4755,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.275948+0000","last_change":"2024-04-25T01:26:02.423420+0000","last_active":"2024-04-25T01:41:09.275948+0000","last_peered":"2024-04-25T01:41:09.275948+0000","last_clean":"2024-04-25T01:41:09.275948+0000","last_became_active":"2024-04-25T01:26:02.423146+0000","last_became_peered":"2024-04-25T01:26:02.423146+0000","last_unstale":"2024-04-25T01:41:09.275948+0000","last_undegraded":"2024-04-25T01:41:09.275948+0000","last_fullsized":"2024-04-25T01:41:09.275948+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4733,"log_dups_size":0,"ondisk_log_size":4733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23928,"num_write_kb":292515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"26'4513","reported_seq":4543,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.807710+0000","last_change":"2024-04-25T01:25:53.363512+0000","last_active":"2024-04-25T01:40:58.807710+0000","last_peered":"2024-04-25T01:40:58.807710+0000","last_clean":"2024-04-25T01:40:58.807710+0000","last_became_active":"2024-04-25T01:25:53.363246+0000","last_became_peered":"2024-04-25T01:25:53.363246+0000","last_unstale":"2024-04-25T01:40:58.807710+0000","last_undegraded":"2024-04-25T01:40:58.807710+0000","last_fullsized":"2024-04-25T01:40:58.807710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4513,"log_dups_size":0,"ondisk_log_size":4513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23705,"num_write_kb":279339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"26'4587","reported_seq":4609,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.306205+0000","last_change":"2024-04-25T01:26:02.413346+0000","last_active":"2024-04-25T01:41:09.306205+0000","last_peered":"2024-04-25T01:41:09.306205+0000","last_clean":"2024-04-25T01:41:09.306205+0000","last_became_active":"2024-04-25T01:26:02.413036+0000","last_became_peered":"2024-04-25T01:26:02.413036+0000","last_unstale":"2024-04-25T01:41:09.306205+0000","last_undegraded":"2024-04-25T01:41:09.306205+0000","last_fullsized":"2024-04-25T01:41:09.306205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4587,"log_dups_size":0,"ondisk_log_size":4587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24042,"num_write_kb":305325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"26'3998","reported_seq":4028,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.876422+0000","last_change":"2024-04-25T01:25:53.364457+0000","last_active":"2024-04-25T01:40:58.876422+0000","last_peered":"2024-04-25T01:40:58.876422+0000","last_clean":"2024-04-25T01:40:58.876422+0000","last_became_active":"2024-04-25T01:25:53.364103+0000","last_became_peered":"2024-04-25T01:25:53.364103+0000","last_unstale":"2024-04-25T01:40:58.876422+0000","last_undegraded":"2024-04-25T01:40:58.876422+0000","last_fullsized":"2024-04-25T01:40:58.876422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3998,"log_dups_size":0,"ondisk_log_size":3998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:11.336752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20343,"num_write_kb":253659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"26'4194","reported_seq":4216,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.298446+0000","last_change":"2024-04-25T01:26:02.418792+0000","last_active":"2024-04-25T01:41:09.298446+0000","last_peered":"2024-04-25T01:41:09.298446+0000","last_clean":"2024-04-25T01:41:09.298446+0000","last_became_active":"2024-04-25T01:26:02.416942+0000","last_became_peered":"2024-04-25T01:26:02.416942+0000","last_unstale":"2024-04-25T01:41:09.298446+0000","last_undegraded":"2024-04-25T01:41:09.298446+0000","last_fullsized":"2024-04-25T01:41:09.298446+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4194,"log_dups_size":0,"ondisk_log_size":4194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:59.492678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21868,"num_write_kb":272015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"26'4800","reported_seq":4830,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.771199+0000","last_change":"2024-04-25T01:25:53.371647+0000","last_active":"2024-04-25T01:40:58.771199+0000","last_peered":"2024-04-25T01:40:58.771199+0000","last_clean":"2024-04-25T01:40:58.771199+0000","last_became_active":"2024-04-25T01:25:53.371353+0000","last_became_peered":"2024-04-25T01:25:53.371353+0000","last_unstale":"2024-04-25T01:40:58.771199+0000","last_undegraded":"2024-04-25T01:40:58.771199+0000","last_fullsized":"2024-04-25T01:40:58.771199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4800,"log_dups_size":0,"ondisk_log_size":4800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24494,"num_write_kb":294847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"26'4139","reported_seq":4161,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.342741+0000","last_change":"2024-04-25T01:26:02.426335+0000","last_active":"2024-04-25T01:41:09.342741+0000","last_peered":"2024-04-25T01:41:09.342741+0000","last_clean":"2024-04-25T01:41:09.342741+0000","last_became_active":"2024-04-25T01:26:02.426024+0000","last_became_peered":"2024-04-25T01:26:02.426024+0000","last_unstale":"2024-04-25T01:41:09.342741+0000","last_undegraded":"2024-04-25T01:41:09.342741+0000","last_fullsized":"2024-04-25T01:41:09.342741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4139,"log_dups_size":0,"ondisk_log_size":4139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21818,"num_write_kb":267665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"26'5798","reported_seq":5828,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.869849+0000","last_change":"2024-04-25T01:25:53.372842+0000","last_active":"2024-04-25T01:40:58.869849+0000","last_peered":"2024-04-25T01:40:58.869849+0000","last_clean":"2024-04-25T01:40:58.869849+0000","last_became_active":"2024-04-25T01:25:53.372580+0000","last_became_peered":"2024-04-25T01:25:53.372580+0000","last_unstale":"2024-04-25T01:40:58.869849+0000","last_undegraded":"2024-04-25T01:40:58.869849+0000","last_fullsized":"2024-04-25T01:40:58.869849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5798,"log_dups_size":0,"ondisk_log_size":5798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30839,"num_write_kb":352989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"26'4687","reported_seq":4709,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.302280+0000","last_change":"2024-04-25T01:26:02.414632+0000","last_active":"2024-04-25T01:41:09.302280+0000","last_peered":"2024-04-25T01:41:09.302280+0000","last_clean":"2024-04-25T01:41:09.302280+0000","last_became_active":"2024-04-25T01:26:02.414388+0000","last_became_peered":"2024-04-25T01:26:02.414388+0000","last_unstale":"2024-04-25T01:41:09.302280+0000","last_undegraded":"2024-04-25T01:41:09.302280+0000","last_fullsized":"2024-04-25T01:41:09.302280+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4687,"log_dups_size":0,"ondisk_log_size":4687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23866,"num_write_kb":304526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"26'5655","reported_seq":5690,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.888926+0000","last_change":"2024-04-25T01:26:52.515807+0000","last_active":"2024-04-25T01:40:58.888926+0000","last_peered":"2024-04-25T01:40:58.888926+0000","last_clean":"2024-04-25T01:40:58.888926+0000","last_became_active":"2024-04-25T01:26:52.515504+0000","last_became_peered":"2024-04-25T01:26:52.515504+0000","last_unstale":"2024-04-25T01:40:58.888926+0000","last_undegraded":"2024-04-25T01:40:58.888926+0000","last_fullsized":"2024-04-25T01:40:58.888926+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5655,"log_dups_size":0,"ondisk_log_size":5655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:48.624287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29809,"num_write_kb":365295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"26'5769","reported_seq":5792,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.330796+0000","last_change":"2024-04-25T01:26:02.414366+0000","last_active":"2024-04-25T01:41:09.330796+0000","last_peered":"2024-04-25T01:41:09.330796+0000","last_clean":"2024-04-25T01:41:09.330796+0000","last_became_active":"2024-04-25T01:26:02.414061+0000","last_became_peered":"2024-04-25T01:26:02.414061+0000","last_unstale":"2024-04-25T01:41:09.330796+0000","last_undegraded":"2024-04-25T01:41:09.330796+0000","last_fullsized":"2024-04-25T01:41:09.330796+0000","mapping_epoch":22,"log_start":"26'500","ondisk_log_start":"26'500","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5269,"log_dups_size":0,"ondisk_log_size":5269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:19.644468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30404,"num_write_kb":379952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"26'5055","reported_seq":5085,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.844665+0000","last_change":"2024-04-25T01:25:53.374061+0000","last_active":"2024-04-25T01:40:58.844665+0000","last_peered":"2024-04-25T01:40:58.844665+0000","last_clean":"2024-04-25T01:40:58.844665+0000","last_became_active":"2024-04-25T01:25:53.373848+0000","last_became_peered":"2024-04-25T01:25:53.373848+0000","last_unstale":"2024-04-25T01:40:58.844665+0000","last_undegraded":"2024-04-25T01:40:58.844665+0000","last_fullsized":"2024-04-25T01:40:58.844665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5055,"log_dups_size":0,"ondisk_log_size":5055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25773,"num_write_kb":324544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"26'4195","reported_seq":4218,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.317465+0000","last_change":"2024-04-25T01:26:02.414620+0000","last_active":"2024-04-25T01:41:09.317465+0000","last_peered":"2024-04-25T01:41:09.317465+0000","last_clean":"2024-04-25T01:41:09.317465+0000","last_became_active":"2024-04-25T01:26:02.414326+0000","last_became_peered":"2024-04-25T01:26:02.414326+0000","last_unstale":"2024-04-25T01:41:09.317465+0000","last_undegraded":"2024-04-25T01:41:09.317465+0000","last_fullsized":"2024-04-25T01:41:09.317465+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4195,"log_dups_size":0,"ondisk_log_size":4195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:30.496089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21944,"num_write_kb":264053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"26'4639","reported_seq":4669,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.755355+0000","last_change":"2024-04-25T01:25:53.364698+0000","last_active":"2024-04-25T01:40:58.755355+0000","last_peered":"2024-04-25T01:40:58.755355+0000","last_clean":"2024-04-25T01:40:58.755355+0000","last_became_active":"2024-04-25T01:25:53.364435+0000","last_became_peered":"2024-04-25T01:25:53.364435+0000","last_unstale":"2024-04-25T01:40:58.755355+0000","last_undegraded":"2024-04-25T01:40:58.755355+0000","last_fullsized":"2024-04-25T01:40:58.755355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4639,"log_dups_size":0,"ondisk_log_size":4639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24873,"num_write_kb":296467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"26'4393","reported_seq":4416,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.341040+0000","last_change":"2024-04-25T01:26:02.418331+0000","last_active":"2024-04-25T01:41:09.341040+0000","last_peered":"2024-04-25T01:41:09.341040+0000","last_clean":"2024-04-25T01:41:09.341040+0000","last_became_active":"2024-04-25T01:26:02.418057+0000","last_became_peered":"2024-04-25T01:26:02.418057+0000","last_unstale":"2024-04-25T01:41:09.341040+0000","last_undegraded":"2024-04-25T01:41:09.341040+0000","last_fullsized":"2024-04-25T01:41:09.341040+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4393,"log_dups_size":0,"ondisk_log_size":4393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:17.809738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22950,"num_write_kb":276425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"26'3744","reported_seq":3759,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.790360+0000","last_change":"2024-04-25T01:26:52.514832+0000","last_active":"2024-04-25T01:40:58.790360+0000","last_peered":"2024-04-25T01:40:58.790360+0000","last_clean":"2024-04-25T01:40:58.790360+0000","last_became_active":"2024-04-25T01:26:52.514260+0000","last_became_peered":"2024-04-25T01:26:52.514260+0000","last_unstale":"2024-04-25T01:40:58.790360+0000","last_undegraded":"2024-04-25T01:40:58.790360+0000","last_fullsized":"2024-04-25T01:40:58.790360+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3744,"log_dups_size":0,"ondisk_log_size":3744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:51.631130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19457,"num_write_kb":254077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3d","version":"26'4696","reported_seq":4719,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.334457+0000","last_change":"2024-04-25T01:26:02.422720+0000","last_active":"2024-04-25T01:41:09.334457+0000","last_peered":"2024-04-25T01:41:09.334457+0000","last_clean":"2024-04-25T01:41:09.334457+0000","last_became_active":"2024-04-25T01:26:02.421848+0000","last_became_peered":"2024-04-25T01:26:02.421848+0000","last_unstale":"2024-04-25T01:41:09.334457+0000","last_undegraded":"2024-04-25T01:41:09.334457+0000","last_fullsized":"2024-04-25T01:41:09.334457+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4696,"log_dups_size":0,"ondisk_log_size":4696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24119,"num_write_kb":305642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"26'5605","reported_seq":5635,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.863402+0000","last_change":"2024-04-25T01:25:53.367953+0000","last_active":"2024-04-25T01:40:58.863402+0000","last_peered":"2024-04-25T01:40:58.863402+0000","last_clean":"2024-04-25T01:40:58.863402+0000","last_became_active":"2024-04-25T01:25:53.367358+0000","last_became_peered":"2024-04-25T01:25:53.367358+0000","last_unstale":"2024-04-25T01:40:58.863402+0000","last_undegraded":"2024-04-25T01:40:58.863402+0000","last_fullsized":"2024-04-25T01:40:58.863402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5605,"log_dups_size":0,"ondisk_log_size":5605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27762,"num_write_kb":352787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"26'4305","reported_seq":4327,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.310344+0000","last_change":"2024-04-25T01:26:02.422300+0000","last_active":"2024-04-25T01:41:09.310344+0000","last_peered":"2024-04-25T01:41:09.310344+0000","last_clean":"2024-04-25T01:41:09.310344+0000","last_became_active":"2024-04-25T01:26:02.421702+0000","last_became_peered":"2024-04-25T01:26:02.421702+0000","last_unstale":"2024-04-25T01:41:09.310344+0000","last_undegraded":"2024-04-25T01:41:09.310344+0000","last_fullsized":"2024-04-25T01:41:09.310344+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4305,"log_dups_size":0,"ondisk_log_size":4305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21701,"num_write_kb":263037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"26'4577","reported_seq":4608,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.784384+0000","last_change":"2024-04-25T01:25:53.371513+0000","last_active":"2024-04-25T01:40:58.784384+0000","last_peered":"2024-04-25T01:40:58.784384+0000","last_clean":"2024-04-25T01:40:58.784384+0000","last_became_active":"2024-04-25T01:25:53.370421+0000","last_became_peered":"2024-04-25T01:25:53.370421+0000","last_unstale":"2024-04-25T01:40:58.784384+0000","last_undegraded":"2024-04-25T01:40:58.784384+0000","last_fullsized":"2024-04-25T01:40:58.784384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24195,"num_write_kb":281461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"26'5562","reported_seq":5584,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.298700+0000","last_change":"2024-04-25T01:26:02.411784+0000","last_active":"2024-04-25T01:41:09.298700+0000","last_peered":"2024-04-25T01:41:09.298700+0000","last_clean":"2024-04-25T01:41:09.298700+0000","last_became_active":"2024-04-25T01:26:02.411478+0000","last_became_peered":"2024-04-25T01:26:02.411478+0000","last_unstale":"2024-04-25T01:41:09.298700+0000","last_undegraded":"2024-04-25T01:41:09.298700+0000","last_fullsized":"2024-04-25T01:41:09.298700+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5562,"log_dups_size":0,"ondisk_log_size":5562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:20.194816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27685,"num_write_kb":352571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"26'5001","reported_seq":5032,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.762988+0000","last_change":"2024-04-25T01:25:53.363829+0000","last_active":"2024-04-25T01:40:58.762988+0000","last_peered":"2024-04-25T01:40:58.762988+0000","last_clean":"2024-04-25T01:40:58.762988+0000","last_became_active":"2024-04-25T01:25:53.363525+0000","last_became_peered":"2024-04-25T01:25:53.363525+0000","last_unstale":"2024-04-25T01:40:58.762988+0000","last_undegraded":"2024-04-25T01:40:58.762988+0000","last_fullsized":"2024-04-25T01:40:58.762988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5001,"log_dups_size":0,"ondisk_log_size":5001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25760,"num_write_kb":312110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"26'5076","reported_seq":5098,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.350154+0000","last_change":"2024-04-25T01:26:02.421208+0000","last_active":"2024-04-25T01:41:09.350154+0000","last_peered":"2024-04-25T01:41:09.350154+0000","last_clean":"2024-04-25T01:41:09.350154+0000","last_became_active":"2024-04-25T01:26:02.418796+0000","last_became_peered":"2024-04-25T01:26:02.418796+0000","last_unstale":"2024-04-25T01:41:09.350154+0000","last_undegraded":"2024-04-25T01:41:09.350154+0000","last_fullsized":"2024-04-25T01:41:09.350154+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5076,"log_dups_size":0,"ondisk_log_size":5076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25596,"num_write_kb":323780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"26'6025","reported_seq":6056,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.748285+0000","last_change":"2024-04-25T01:25:53.365057+0000","last_active":"2024-04-25T01:40:58.748285+0000","last_peered":"2024-04-25T01:40:58.748285+0000","last_clean":"2024-04-25T01:40:58.748285+0000","last_became_active":"2024-04-25T01:25:53.364793+0000","last_became_peered":"2024-04-25T01:25:53.364793+0000","last_unstale":"2024-04-25T01:40:58.748285+0000","last_undegraded":"2024-04-25T01:40:58.748285+0000","last_fullsized":"2024-04-25T01:40:58.748285+0000","mapping_epoch":18,"log_start":"26'200","ondisk_log_start":"26'200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5825,"log_dups_size":0,"ondisk_log_size":5825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31759,"num_write_kb":385508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"26'4493","reported_seq":4515,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.263776+0000","last_change":"2024-04-25T01:26:02.417606+0000","last_active":"2024-04-25T01:41:09.263776+0000","last_peered":"2024-04-25T01:41:09.263776+0000","last_clean":"2024-04-25T01:41:09.263776+0000","last_became_active":"2024-04-25T01:26:02.417350+0000","last_became_peered":"2024-04-25T01:26:02.417350+0000","last_unstale":"2024-04-25T01:41:09.263776+0000","last_undegraded":"2024-04-25T01:41:09.263776+0000","last_fullsized":"2024-04-25T01:41:09.263776+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4493,"log_dups_size":0,"ondisk_log_size":4493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23464,"num_write_kb":290699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"26'4350","reported_seq":4381,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.880219+0000","last_change":"2024-04-25T01:25:53.367762+0000","last_active":"2024-04-25T01:40:58.880219+0000","last_peered":"2024-04-25T01:40:58.880219+0000","last_clean":"2024-04-25T01:40:58.880219+0000","last_became_active":"2024-04-25T01:25:53.367383+0000","last_became_peered":"2024-04-25T01:25:53.367383+0000","last_unstale":"2024-04-25T01:40:58.880219+0000","last_undegraded":"2024-04-25T01:40:58.880219+0000","last_fullsized":"2024-04-25T01:40:58.880219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4350,"log_dups_size":0,"ondisk_log_size":4350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:05.589744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23070,"num_write_kb":268669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1533236,"num_write_kb":19254592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":294168,"ondisk_log_size":294168,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":17605243,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":39,"num_read_kb":112,"num_write":4172,"num_write_kb":18793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35274752,"data_stored":35210486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3499,"ondisk_log_size":3499,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1596536,"num_write_kb":19509893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305948,"ondisk_log_size":305948,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":17847589,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":41,"num_read_kb":115,"num_write":4224,"num_write_kb":19059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35758080,"data_stored":35695178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3536,"ondisk_log_size":3536,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575039,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1217764,"kb_used_data":13148,"kb_used_omap":1,"kb_used_meta":1204606,"kb_avail":93154076,"statfs":{"total":96636764160,"available":95389773824,"internally_reserved":0,"allocated":13463552,"data_stored":13287019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":1233517002},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575037,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":780520,"kb_used_data":9440,"kb_used_omap":1,"kb_used_meta":771070,"kb_avail":93591320,"statfs":{"total":96636764160,"available":95837511680,"internally_reserved":0,"allocated":9666560,"data_stored":9483786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":789576134},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575038,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1059708,"kb_used_data":4468,"kb_used_omap":1,"kb_used_meta":1055230,"kb_avail":93312132,"statfs":{"total":96636764160,"available":95551623168,"internally_reserved":0,"allocated":4575232,"data_stored":4396213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1080555973},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575038,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1089176,"kb_used_data":10256,"kb_used_omap":1,"kb_used_meta":1078910,"kb_avail":93282664,"statfs":{"total":96636764160,"available":95521447936,"internally_reserved":0,"allocated":10502144,"data_stored":10309285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1104804293},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575038,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1161176,"kb_used_data":17488,"kb_used_omap":1,"kb_used_meta":1143678,"kb_avail":93210664,"statfs":{"total":96636764160,"available":95447719936,"internally_reserved":0,"allocated":17907712,"data_stored":17718206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1171126727},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575038,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":729908,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":729534,"kb_avail":93641932,"statfs":{"total":96636764160,"available":95889338368,"internally_reserved":0,"allocated":372736,"data_stored":201217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":747043269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575038,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":888620,"kb_used_data":8484,"kb_used_omap":1,"kb_used_meta":880126,"kb_avail":93483220,"statfs":{"total":96636764160,"available":95726817280,"internally_reserved":0,"allocated":8687616,"data_stored":8502529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":901249477},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575037,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":919656,"kb_used_data":9056,"kb_used_omap":1,"kb_used_meta":910590,"kb_avail":93452184,"statfs":{"total":96636764160,"available":95695036416,"internally_reserved":0,"allocated":9273344,"data_stored":9092681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":932444613},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":88080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8486912,"data_stored":8477360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5189632,"data_stored":5175253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5185536,"data_stored":5175799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8389302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9142272,"data_stored":9127779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5033984,"data_stored":5020965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4292608,"data_stored":4283056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:41:16.020 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:41:16.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.162+0000 7f25be163640 1 Processor -- start 2024-04-25T01:41:16.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.162+0000 7f25be163640 1 -- start start 2024-04-25T01:41:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25be163640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f25b806bc50 0x7f25b8159700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25be163640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25b8156460 con 0x7f25b80a9490 2024-04-25T01:41:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25be163640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25b8156740 con 0x7f25b806b160 2024-04-25T01:41:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25be163640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f25b806c0c0 con 0x7f25b806bc50 2024-04-25T01:41:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25b6ffd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f25b80a9490 0x7f25b8158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:39344/0 (socket says 172.21.15.112:39344) 2024-04-25T01:41:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25b6ffd640 1 -- 172.21.15.112:0/3899987507 learned_addr learned my addr 172.21.15.112:0/3899987507 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25b7fff640 1 --2- 172.21.15.112:0/3899987507 >> v2:172.21.15.187:3300/0 conn(0x7f25b806bc50 0x7f25b8159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25b67fc640 1 -- 172.21.15.112:0/3899987507 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 153760965 0 0) 0x7f25b8156460 con 0x7f25b80a9490 2024-04-25T01:41:16.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25b67fc640 1 -- 172.21.15.112:0/3899987507 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25a4003520 con 0x7f25b80a9490 2024-04-25T01:41:16.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25b67fc640 1 -- 172.21.15.112:0/3899987507 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2067924155 0 0) 0x7f25a4003520 con 0x7f25b80a9490 2024-04-25T01:41:16.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25b67fc640 1 -- 172.21.15.112:0/3899987507 >> v2:172.21.15.187:3300/0 conn(0x7f25b806bc50 msgr2=0x7f25b8159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:16.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25b67fc640 1 --2- 172.21.15.112:0/3899987507 >> v2:172.21.15.187:3300/0 conn(0x7f25b806bc50 0x7f25b8159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25b67fc640 1 -- 172.21.15.112:0/3899987507 >> v1:172.21.15.187:6789/0 conn(0x7f25b806b160 legacy=0x7f25b806b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:16.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25b67fc640 1 -- 172.21.15.112:0/3899987507 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b80c2c80 con 0x7f25b80a9490 2024-04-25T01:41:16.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25b67fc640 1 -- 172.21.15.112:0/3899987507 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f25a8001670 con 0x7f25b80a9490 2024-04-25T01:41:16.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25b67fc640 1 -- 172.21.15.112:0/3899987507 >> v1:172.21.15.112:6789/0 conn(0x7f25b80a9490 legacy=0x7f25b8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:16.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25b67fc640 1 --2- 172.21.15.112:0/3899987507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25a40039b0 0x7f25a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25b67fc640 1 --2- 172.21.15.112:0/3899987507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25a4004800 0x7f25a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25b67fc640 1 --2- 172.21.15.112:0/3899987507 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25a40051c0 0x7f25a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25b67fc640 1 -- 172.21.15.112:0/3899987507 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25b806c0c0 con 0x7f25a40039b0 2024-04-25T01:41:16.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25b67fc640 1 -- 172.21.15.112:0/3899987507 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f25a400c160 con 0x7f25a4004800 2024-04-25T01:41:16.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25b67fc640 1 -- 172.21.15.112:0/3899987507 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f25a400c360 con 0x7f25a40051c0 2024-04-25T01:41:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.163+0000 7f25b77fe640 1 --2- 172.21.15.112:0/3899987507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25a40039b0 0x7f25a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.164+0000 7f25b6ffd640 1 --2- 172.21.15.112:0/3899987507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25a4004800 0x7f25a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.164+0000 7f25b7fff640 1 --2- 172.21.15.112:0/3899987507 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25a40051c0 0x7f25a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.164+0000 7f25b77fe640 1 -- 172.21.15.112:0/3899987507 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25a40051c0 msgr2=0x7f25a400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.164+0000 7f25b77fe640 1 --2- 172.21.15.112:0/3899987507 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25a40051c0 0x7f25a400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.164+0000 7f25b77fe640 1 -- 172.21.15.112:0/3899987507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25a4004800 msgr2=0x7f25a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.164+0000 7f25b77fe640 1 --2- 172.21.15.112:0/3899987507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25a4004800 0x7f25a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.164+0000 7f25b77fe640 1 -- 172.21.15.112:0/3899987507 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a4003520 con 0x7f25a40039b0 2024-04-25T01:41:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.164+0000 7f25b7fff640 1 -- 172.21.15.112:0/3899987507 reap_dead start 2024-04-25T01:41:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.164+0000 7f25b6ffd640 1 --2- 172.21.15.112:0/3899987507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25a4004800 0x7f25a4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:41:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.164+0000 7f25b77fe640 1 --2- 172.21.15.112:0/3899987507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25a40039b0 0x7f25a4003dd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f25a0001750 tx=0x7f25a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23c56c845dc216a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.164+0000 7f25b67fc640 1 -- 172.21.15.112:0/3899987507 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0001370 con 0x7f25a40039b0 2024-04-25T01:41:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.165+0000 7f25b67fc640 1 -- 172.21.15.112:0/3899987507 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25a00052d0 con 0x7f25a40039b0 2024-04-25T01:41:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.165+0000 7f25b67fc640 1 -- 172.21.15.112:0/3899987507 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a002abc0 con 0x7f25a40039b0 2024-04-25T01:41:16.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.165+0000 7f25be163640 1 -- 172.21.15.112:0/3899987507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25a40039b0 msgr2=0x7f25a4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:16.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.165+0000 7f25be163640 1 --2- 172.21.15.112:0/3899987507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25a40039b0 0x7f25a4003dd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f25a0001750 tx=0x7f25a4005830 comp rx=0 tx=0).stop 2024-04-25T01:41:16.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.165+0000 7f25be163640 1 -- 172.21.15.112:0/3899987507 shutdown_connections 2024-04-25T01:41:16.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.165+0000 7f25be163640 1 --2- 172.21.15.112:0/3899987507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25a40039b0 0x7f25a4003dd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.165+0000 7f25be163640 1 -- 172.21.15.112:0/3899987507 >> 172.21.15.112:0/3899987507 conn(0x7f25b80c4c00 msgr2=0x7f25b80c4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:16.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.165+0000 7f25be163640 1 -- 172.21.15.112:0/3899987507 shutdown_connections 2024-04-25T01:41:16.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.165+0000 7f25be163640 1 -- 172.21.15.112:0/3899987507 wait complete. 2024-04-25T01:41:16.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.166+0000 7f25be163640 1 Processor -- start 2024-04-25T01:41:16.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.166+0000 7f25be163640 1 -- start start 2024-04-25T01:41:16.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.167+0000 7f25be163640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25a40039b0 0x7f25b8139db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.167+0000 7f25be163640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25b81545f0 0x7f25b8156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.167+0000 7f25be163640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25b8157410 0x7f25b8137780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.167+0000 7f25be163640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25b813efd0 con 0x7f25b8157410 2024-04-25T01:41:16.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.167+0000 7f25be163640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f25b80c4a50 con 0x7f25a40039b0 2024-04-25T01:41:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.167+0000 7f25be163640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f25b813f150 con 0x7f25b81545f0 2024-04-25T01:41:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.167+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25b8157410 0x7f25b8137780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.167+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25b8157410 0x7f25b8137780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56462/0 (socket says 172.21.15.112:56462) 2024-04-25T01:41:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.167+0000 7f25b7fff640 1 -- 172.21.15.112:0/3733393511 learned_addr learned my addr 172.21.15.112:0/3733393511 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.167+0000 7f25b77fe640 1 --2- 172.21.15.112:0/3733393511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25a40039b0 0x7f25b8139db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.167+0000 7f25b6ffd640 1 --2- 172.21.15.112:0/3733393511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25b81545f0 0x7f25b8156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.167+0000 7f25b7fff640 1 -- 172.21.15.112:0/3733393511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25b81545f0 msgr2=0x7f25b8156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.167+0000 7f25b7fff640 1 --2- 172.21.15.112:0/3733393511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25b81545f0 0x7f25b8156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.167+0000 7f25b7fff640 1 -- 172.21.15.112:0/3733393511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25a40039b0 msgr2=0x7f25b8139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.167+0000 7f25b7fff640 1 --2- 172.21.15.112:0/3733393511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25a40039b0 0x7f25b8139db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.167+0000 7f25b7fff640 1 -- 172.21.15.112:0/3733393511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b813e2b0 con 0x7f25b8157410 2024-04-25T01:41:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.168+0000 7f25b7fff640 1 --2- 172.21.15.112:0/3733393511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25b8157410 0x7f25b8137780 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f25b80c5c30 tx=0x7f25b00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.168+0000 7f25affff640 1 -- 172.21.15.112:0/3733393511 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0011db0 con 0x7f25b8157410 2024-04-25T01:41:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.168+0000 7f25affff640 1 -- 172.21.15.112:0/3733393511 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25b001b070 con 0x7f25b8157410 2024-04-25T01:41:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.168+0000 7f25be163640 1 -- 172.21.15.112:0/3733393511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b8154d80 con 0x7f25b8157410 2024-04-25T01:41:16.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.168+0000 7f25affff640 1 -- 172.21.15.112:0/3733393511 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0003990 con 0x7f25b8157410 2024-04-25T01:41:16.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.168+0000 7f25be163640 1 -- 172.21.15.112:0/3733393511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b8155360 con 0x7f25b8157410 2024-04-25T01:41:16.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.169+0000 7f25be163640 1 -- 172.21.15.112:0/3733393511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b80682a0 con 0x7f25b8157410 2024-04-25T01:41:16.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.169+0000 7f25affff640 1 -- 172.21.15.112:0/3733393511 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f25b0003b30 con 0x7f25b8157410 2024-04-25T01:41:16.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.170+0000 7f25affff640 1 --2- 172.21.15.112:0/3733393511 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f259806be60 0x7f259806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.170+0000 7f25affff640 1 -- 172.21.15.112:0/3733393511 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f25b008b940 con 0x7f25b8157410 2024-04-25T01:41:16.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.170+0000 7f25b77fe640 1 --2- 172.21.15.112:0/3733393511 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f259806be60 0x7f259806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.170+0000 7f25b77fe640 1 --2- 172.21.15.112:0/3733393511 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f259806be60 0x7f259806e320 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f25a0032000 tx=0x7f25a002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:16.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.174+0000 7f25affff640 1 -- 172.21.15.112:0/3733393511 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f25b0003e20 con 0x7f25b8157410 2024-04-25T01:41:16.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.344+0000 7f25be163640 1 -- 172.21.15.112:0/3733393511 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25b8158a60 con 0x7f259806be60 2024-04-25T01:41:16.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.354+0000 7f25affff640 1 -- 172.21.15.112:0/3733393511 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+493703 (secure 0 0 0) 0x7f25b8158a60 con 0x7f259806be60 2024-04-25T01:41:16.343 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:41:16.350 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T01:41:16.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.358+0000 7f25be163640 1 -- 172.21.15.112:0/3733393511 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f259806be60 msgr2=0x7f259806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:16.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.358+0000 7f25be163640 1 --2- 172.21.15.112:0/3733393511 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f259806be60 0x7f259806e320 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f25a0032000 tx=0x7f25a002e000 comp rx=0 tx=0).stop 2024-04-25T01:41:16.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.358+0000 7f25be163640 1 -- 172.21.15.112:0/3733393511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25b8157410 msgr2=0x7f25b8137780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:16.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.358+0000 7f25be163640 1 --2- 172.21.15.112:0/3733393511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25b8157410 0x7f25b8137780 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f25b80c5c30 tx=0x7f25b00115a0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.359+0000 7f25be163640 1 -- 172.21.15.112:0/3733393511 shutdown_connections 2024-04-25T01:41:16.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.359+0000 7f25be163640 1 --2- 172.21.15.112:0/3733393511 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f259806be60 0x7f259806e320 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.359+0000 7f25be163640 1 --2- 172.21.15.112:0/3733393511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25b8157410 0x7f25b8137780 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.359+0000 7f25be163640 1 --2- 172.21.15.112:0/3733393511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25b81545f0 0x7f25b8156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.359+0000 7f25be163640 1 --2- 172.21.15.112:0/3733393511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25a40039b0 0x7f25b8139db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.359+0000 7f25be163640 1 -- 172.21.15.112:0/3733393511 >> 172.21.15.112:0/3733393511 conn(0x7f25b80c4c00 msgr2=0x7f25b80aa070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:16.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.359+0000 7f25be163640 1 -- 172.21.15.112:0/3733393511 shutdown_connections 2024-04-25T01:41:16.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.359+0000 7f25be163640 1 -- 172.21.15.112:0/3733393511 wait complete. 2024-04-25T01:41:16.357 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":486,"stamp":"2024-04-25T01:41:14.915187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":36043200,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":196,"num_read_kb":327,"num_write":3138327,"num_write_kb":38804753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":607268,"ondisk_log_size":607268,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":7846528,"kb_used_data":72704,"kb_used_omap":12,"kb_used_meta":7773747,"kb_avail":747128192,"statfs":{"total":773094113280,"available":765059268608,"internally_reserved":0,"allocated":74448896,"data_stored":72990936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":7960317488},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":70,"apply_latency_ms":70,"commit_latency_ns":70000000,"apply_latency_ns":70000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003261"},"pg_stats":[{"pgid":"5.18","version":"26'4099","reported_seq":4121,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.267400+0000","last_change":"2024-04-25T01:26:02.412272+0000","last_active":"2024-04-25T01:41:09.267400+0000","last_peered":"2024-04-25T01:41:09.267400+0000","last_clean":"2024-04-25T01:41:09.267400+0000","last_became_active":"2024-04-25T01:26:02.411176+0000","last_became_peered":"2024-04-25T01:26:02.411176+0000","last_unstale":"2024-04-25T01:41:09.267400+0000","last_undegraded":"2024-04-25T01:41:09.267400+0000","last_fullsized":"2024-04-25T01:41:09.267400+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4099,"log_dups_size":0,"ondisk_log_size":4099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:49.498275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21931,"num_write_kb":288545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"26'658","reported_seq":691,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:44.444563+0000","last_change":"2024-04-25T01:25:52.353978+0000","last_active":"2024-04-25T01:40:44.444563+0000","last_peered":"2024-04-25T01:40:44.444563+0000","last_clean":"2024-04-25T01:40:44.444563+0000","last_became_active":"2024-04-25T01:25:52.353649+0000","last_became_peered":"2024-04-25T01:25:52.353649+0000","last_unstale":"2024-04-25T01:40:44.444563+0000","last_undegraded":"2024-04-25T01:40:44.444563+0000","last_fullsized":"2024-04-25T01:40:44.444563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":4468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"26'5333","reported_seq":5364,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.730049+0000","last_change":"2024-04-25T01:25:53.364257+0000","last_active":"2024-04-25T01:40:58.730049+0000","last_peered":"2024-04-25T01:40:58.730049+0000","last_clean":"2024-04-25T01:40:58.730049+0000","last_became_active":"2024-04-25T01:25:53.363409+0000","last_became_peered":"2024-04-25T01:25:53.363409+0000","last_unstale":"2024-04-25T01:40:58.730049+0000","last_undegraded":"2024-04-25T01:40:58.730049+0000","last_fullsized":"2024-04-25T01:40:58.730049+0000","mapping_epoch":18,"log_start":"26'100","ondisk_log_start":"26'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28405,"num_write_kb":331018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"26'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:31.329698+0000","last_change":"2024-04-25T01:26:01.407136+0000","last_active":"2024-04-25T01:40:31.329698+0000","last_peered":"2024-04-25T01:40:31.329698+0000","last_clean":"2024-04-25T01:40:31.329698+0000","last_became_active":"2024-04-25T01:26:01.406855+0000","last_became_peered":"2024-04-25T01:26:01.406855+0000","last_unstale":"2024-04-25T01:40:31.329698+0000","last_undegraded":"2024-04-25T01:40:31.329698+0000","last_fullsized":"2024-04-25T01:40:31.329698+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:02.180826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"26'5328","reported_seq":5350,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.321101+0000","last_change":"2024-04-25T01:26:02.425367+0000","last_active":"2024-04-25T01:41:09.321101+0000","last_peered":"2024-04-25T01:41:09.321101+0000","last_clean":"2024-04-25T01:41:09.321101+0000","last_became_active":"2024-04-25T01:26:02.425071+0000","last_became_peered":"2024-04-25T01:26:02.425071+0000","last_unstale":"2024-04-25T01:41:09.321101+0000","last_undegraded":"2024-04-25T01:41:09.321101+0000","last_fullsized":"2024-04-25T01:41:09.321101+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5328,"log_dups_size":0,"ondisk_log_size":5328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:47.810943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27477,"num_write_kb":347746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"26'8","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.218358+0000","last_change":"2024-04-25T01:25:52.349185+0000","last_active":"2024-04-25T01:40:49.218358+0000","last_peered":"2024-04-25T01:40:49.218358+0000","last_clean":"2024-04-25T01:40:49.218358+0000","last_became_active":"2024-04-25T01:25:52.348931+0000","last_became_peered":"2024-04-25T01:25:52.348931+0000","last_unstale":"2024-04-25T01:40:49.218358+0000","last_undegraded":"2024-04-25T01:40:49.218358+0000","last_fullsized":"2024-04-25T01:40:49.218358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"26'5837","reported_seq":5868,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.789129+0000","last_change":"2024-04-25T01:25:53.374150+0000","last_active":"2024-04-25T01:40:58.789129+0000","last_peered":"2024-04-25T01:40:58.789129+0000","last_clean":"2024-04-25T01:40:58.789129+0000","last_became_active":"2024-04-25T01:25:53.373755+0000","last_became_peered":"2024-04-25T01:25:53.373755+0000","last_unstale":"2024-04-25T01:40:58.789129+0000","last_undegraded":"2024-04-25T01:40:58.789129+0000","last_fullsized":"2024-04-25T01:40:58.789129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5837,"log_dups_size":0,"ondisk_log_size":5837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30942,"num_write_kb":369940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"26'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:32.791998+0000","last_change":"2024-04-25T01:26:01.406334+0000","last_active":"2024-04-25T01:40:32.791998+0000","last_peered":"2024-04-25T01:40:32.791998+0000","last_clean":"2024-04-25T01:40:32.791998+0000","last_became_active":"2024-04-25T01:26:01.405994+0000","last_became_peered":"2024-04-25T01:26:01.405994+0000","last_unstale":"2024-04-25T01:40:32.791998+0000","last_undegraded":"2024-04-25T01:40:32.791998+0000","last_fullsized":"2024-04-25T01:40:32.791998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:43.953437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"26'4188","reported_seq":4210,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.336545+0000","last_change":"2024-04-25T01:26:02.427695+0000","last_active":"2024-04-25T01:41:09.336545+0000","last_peered":"2024-04-25T01:41:09.336545+0000","last_clean":"2024-04-25T01:41:09.336545+0000","last_became_active":"2024-04-25T01:26:02.427443+0000","last_became_peered":"2024-04-25T01:26:02.427443+0000","last_unstale":"2024-04-25T01:41:09.336545+0000","last_undegraded":"2024-04-25T01:41:09.336545+0000","last_fullsized":"2024-04-25T01:41:09.336545+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4188,"log_dups_size":0,"ondisk_log_size":4188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20869,"num_write_kb":259707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"26'11","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.299916+0000","last_change":"2024-04-25T01:25:52.356403+0000","last_active":"2024-04-25T01:40:49.299916+0000","last_peered":"2024-04-25T01:40:49.299916+0000","last_clean":"2024-04-25T01:40:49.299916+0000","last_became_active":"2024-04-25T01:25:52.355822+0000","last_became_peered":"2024-04-25T01:25:52.355822+0000","last_unstale":"2024-04-25T01:40:49.299916+0000","last_undegraded":"2024-04-25T01:40:49.299916+0000","last_fullsized":"2024-04-25T01:40:49.299916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"26'3747","reported_seq":3777,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.846633+0000","last_change":"2024-04-25T01:25:53.370061+0000","last_active":"2024-04-25T01:40:58.846633+0000","last_peered":"2024-04-25T01:40:58.846633+0000","last_clean":"2024-04-25T01:40:58.846633+0000","last_became_active":"2024-04-25T01:25:53.369721+0000","last_became_peered":"2024-04-25T01:25:53.369721+0000","last_unstale":"2024-04-25T01:40:58.846633+0000","last_undegraded":"2024-04-25T01:40:58.846633+0000","last_fullsized":"2024-04-25T01:40:58.846633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3747,"log_dups_size":0,"ondisk_log_size":3747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18978,"num_write_kb":223593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"26'627","reported_seq":651,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.103722+0000","last_change":"2024-04-25T01:26:01.408179+0000","last_active":"2024-04-25T01:40:37.103722+0000","last_peered":"2024-04-25T01:40:37.103722+0000","last_clean":"2024-04-25T01:40:37.103722+0000","last_became_active":"2024-04-25T01:26:01.407909+0000","last_became_peered":"2024-04-25T01:26:01.407909+0000","last_unstale":"2024-04-25T01:40:37.103722+0000","last_undegraded":"2024-04-25T01:40:37.103722+0000","last_fullsized":"2024-04-25T01:40:37.103722+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:48.865940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":625,"num_write_kb":4235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"26'3441","reported_seq":3463,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.298394+0000","last_change":"2024-04-25T01:26:02.424615+0000","last_active":"2024-04-25T01:41:09.298394+0000","last_peered":"2024-04-25T01:41:09.298394+0000","last_clean":"2024-04-25T01:41:09.298394+0000","last_became_active":"2024-04-25T01:26:02.424325+0000","last_became_peered":"2024-04-25T01:26:02.424325+0000","last_unstale":"2024-04-25T01:41:09.298394+0000","last_undegraded":"2024-04-25T01:41:09.298394+0000","last_fullsized":"2024-04-25T01:41:09.298394+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3441,"log_dups_size":0,"ondisk_log_size":3441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17582,"num_write_kb":230191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"26'3","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.256836+0000","last_change":"2024-04-25T01:25:52.359139+0000","last_active":"2024-04-25T01:40:49.256836+0000","last_peered":"2024-04-25T01:40:49.256836+0000","last_clean":"2024-04-25T01:40:49.256836+0000","last_became_active":"2024-04-25T01:25:52.358841+0000","last_became_peered":"2024-04-25T01:25:52.358841+0000","last_unstale":"2024-04-25T01:40:49.256836+0000","last_undegraded":"2024-04-25T01:40:49.256836+0000","last_fullsized":"2024-04-25T01:40:49.256836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"26'5828","reported_seq":5858,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.878112+0000","last_change":"2024-04-25T01:25:53.368739+0000","last_active":"2024-04-25T01:40:58.878112+0000","last_peered":"2024-04-25T01:40:58.878112+0000","last_clean":"2024-04-25T01:40:58.878112+0000","last_became_active":"2024-04-25T01:25:53.368470+0000","last_became_peered":"2024-04-25T01:25:53.368470+0000","last_unstale":"2024-04-25T01:40:58.878112+0000","last_undegraded":"2024-04-25T01:40:58.878112+0000","last_fullsized":"2024-04-25T01:40:58.878112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5828,"log_dups_size":0,"ondisk_log_size":5828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:29.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30463,"num_write_kb":367857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"26'65","reported_seq":89,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.150373+0000","last_change":"2024-04-25T01:26:01.407969+0000","last_active":"2024-04-25T01:41:09.150373+0000","last_peered":"2024-04-25T01:41:09.150373+0000","last_clean":"2024-04-25T01:41:09.150373+0000","last_became_active":"2024-04-25T01:26:01.407655+0000","last_became_peered":"2024-04-25T01:26:01.407655+0000","last_unstale":"2024-04-25T01:41:09.150373+0000","last_undegraded":"2024-04-25T01:41:09.150373+0000","last_fullsized":"2024-04-25T01:41:09.150373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:58.485864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":738477,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61,"num_write_kb":764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"26'4794","reported_seq":4816,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.329809+0000","last_change":"2024-04-25T01:26:02.417776+0000","last_active":"2024-04-25T01:41:09.329809+0000","last_peered":"2024-04-25T01:41:09.329809+0000","last_clean":"2024-04-25T01:41:09.329809+0000","last_became_active":"2024-04-25T01:26:02.417524+0000","last_became_peered":"2024-04-25T01:26:02.417524+0000","last_unstale":"2024-04-25T01:41:09.329809+0000","last_undegraded":"2024-04-25T01:41:09.329809+0000","last_fullsized":"2024-04-25T01:41:09.329809+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4794,"log_dups_size":0,"ondisk_log_size":4794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.688469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24624,"num_write_kb":311668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"26'4532","reported_seq":4563,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.725104+0000","last_change":"2024-04-25T01:25:53.372385+0000","last_active":"2024-04-25T01:40:58.725104+0000","last_peered":"2024-04-25T01:40:58.725104+0000","last_clean":"2024-04-25T01:40:58.725104+0000","last_became_active":"2024-04-25T01:25:53.372064+0000","last_became_peered":"2024-04-25T01:25:53.372064+0000","last_unstale":"2024-04-25T01:40:58.725104+0000","last_undegraded":"2024-04-25T01:40:58.725104+0000","last_fullsized":"2024-04-25T01:40:58.725104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4532,"log_dups_size":0,"ondisk_log_size":4532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23642,"num_write_kb":279226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"26'1100","reported_seq":1132,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.233609+0000","last_change":"2024-04-25T01:25:52.358616+0000","last_active":"2024-04-25T01:40:49.233609+0000","last_peered":"2024-04-25T01:40:49.233609+0000","last_clean":"2024-04-25T01:40:49.233609+0000","last_became_active":"2024-04-25T01:25:52.358318+0000","last_became_peered":"2024-04-25T01:25:52.358318+0000","last_unstale":"2024-04-25T01:40:49.233609+0000","last_undegraded":"2024-04-25T01:40:49.233609+0000","last_fullsized":"2024-04-25T01:40:49.233609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1100,"log_dups_size":0,"ondisk_log_size":1100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1091,"num_write_kb":4419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"26'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:39:31.750022+0000","last_change":"2024-04-25T01:26:01.400599+0000","last_active":"2024-04-25T01:39:31.750022+0000","last_peered":"2024-04-25T01:39:31.750022+0000","last_clean":"2024-04-25T01:39:31.750022+0000","last_became_active":"2024-04-25T01:26:01.400266+0000","last_became_peered":"2024-04-25T01:26:01.400266+0000","last_unstale":"2024-04-25T01:39:31.750022+0000","last_undegraded":"2024-04-25T01:39:31.750022+0000","last_fullsized":"2024-04-25T01:39:31.750022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"26'4554","reported_seq":4576,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.270037+0000","last_change":"2024-04-25T01:26:02.420269+0000","last_active":"2024-04-25T01:41:09.270037+0000","last_peered":"2024-04-25T01:41:09.270037+0000","last_clean":"2024-04-25T01:41:09.270037+0000","last_became_active":"2024-04-25T01:26:02.420001+0000","last_became_peered":"2024-04-25T01:26:02.420001+0000","last_unstale":"2024-04-25T01:41:09.270037+0000","last_undegraded":"2024-04-25T01:41:09.270037+0000","last_fullsized":"2024-04-25T01:41:09.270037+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4554,"log_dups_size":0,"ondisk_log_size":4554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:38.149069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23431,"num_write_kb":298735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"26'3793","reported_seq":3823,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.792694+0000","last_change":"2024-04-25T01:25:53.361074+0000","last_active":"2024-04-25T01:40:58.792694+0000","last_peered":"2024-04-25T01:40:58.792694+0000","last_clean":"2024-04-25T01:40:58.792694+0000","last_became_active":"2024-04-25T01:25:53.360802+0000","last_became_peered":"2024-04-25T01:25:53.360802+0000","last_unstale":"2024-04-25T01:40:58.792694+0000","last_undegraded":"2024-04-25T01:40:58.792694+0000","last_fullsized":"2024-04-25T01:40:58.792694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3793,"log_dups_size":0,"ondisk_log_size":3793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19299,"num_write_kb":249413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"26'3","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.225267+0000","last_change":"2024-04-25T01:25:52.348841+0000","last_active":"2024-04-25T01:40:49.225267+0000","last_peered":"2024-04-25T01:40:49.225267+0000","last_clean":"2024-04-25T01:40:49.225267+0000","last_became_active":"2024-04-25T01:25:52.348550+0000","last_became_peered":"2024-04-25T01:25:52.348550+0000","last_unstale":"2024-04-25T01:40:49.225267+0000","last_undegraded":"2024-04-25T01:40:49.225267+0000","last_fullsized":"2024-04-25T01:40:49.225267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"26'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:36.992419+0000","last_change":"2024-04-25T01:26:01.408529+0000","last_active":"2024-04-25T01:40:36.992419+0000","last_peered":"2024-04-25T01:40:36.992419+0000","last_clean":"2024-04-25T01:40:36.992419+0000","last_became_active":"2024-04-25T01:26:01.407738+0000","last_became_peered":"2024-04-25T01:26:01.407738+0000","last_unstale":"2024-04-25T01:40:36.992419+0000","last_undegraded":"2024-04-25T01:40:36.992419+0000","last_fullsized":"2024-04-25T01:40:36.992419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:37.142255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"26'3863","reported_seq":3885,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.299717+0000","last_change":"2024-04-25T01:26:02.417731+0000","last_active":"2024-04-25T01:41:09.299717+0000","last_peered":"2024-04-25T01:41:09.299717+0000","last_clean":"2024-04-25T01:41:09.299717+0000","last_became_active":"2024-04-25T01:26:02.417438+0000","last_became_peered":"2024-04-25T01:26:02.417438+0000","last_unstale":"2024-04-25T01:41:09.299717+0000","last_undegraded":"2024-04-25T01:41:09.299717+0000","last_fullsized":"2024-04-25T01:41:09.299717+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3863,"log_dups_size":0,"ondisk_log_size":3863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19753,"num_write_kb":275789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"26'4400","reported_seq":4431,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.812159+0000","last_change":"2024-04-25T01:25:53.361790+0000","last_active":"2024-04-25T01:40:58.812159+0000","last_peered":"2024-04-25T01:40:58.812159+0000","last_clean":"2024-04-25T01:40:58.812159+0000","last_became_active":"2024-04-25T01:25:53.361542+0000","last_became_peered":"2024-04-25T01:25:53.361542+0000","last_unstale":"2024-04-25T01:40:58.812159+0000","last_undegraded":"2024-04-25T01:40:58.812159+0000","last_fullsized":"2024-04-25T01:40:58.812159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4400,"log_dups_size":0,"ondisk_log_size":4400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22906,"num_write_kb":296647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"26'3","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.235744+0000","last_change":"2024-04-25T01:25:52.353926+0000","last_active":"2024-04-25T01:40:49.235744+0000","last_peered":"2024-04-25T01:40:49.235744+0000","last_clean":"2024-04-25T01:40:49.235744+0000","last_became_active":"2024-04-25T01:25:52.353656+0000","last_became_peered":"2024-04-25T01:25:52.353656+0000","last_unstale":"2024-04-25T01:40:49.235744+0000","last_undegraded":"2024-04-25T01:40:49.235744+0000","last_fullsized":"2024-04-25T01:40:49.235744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"26'5","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.165411+0000","last_change":"2024-04-25T01:26:01.418556+0000","last_active":"2024-04-25T01:40:37.165411+0000","last_peered":"2024-04-25T01:40:37.165411+0000","last_clean":"2024-04-25T01:40:37.165411+0000","last_became_active":"2024-04-25T01:26:01.418263+0000","last_became_peered":"2024-04-25T01:26:01.418263+0000","last_unstale":"2024-04-25T01:40:37.165411+0000","last_undegraded":"2024-04-25T01:40:37.165411+0000","last_fullsized":"2024-04-25T01:40:37.165411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:28.139183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"26'5570","reported_seq":5592,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.349577+0000","last_change":"2024-04-25T01:26:02.413429+0000","last_active":"2024-04-25T01:41:09.349577+0000","last_peered":"2024-04-25T01:41:09.349577+0000","last_clean":"2024-04-25T01:41:09.349577+0000","last_became_active":"2024-04-25T01:26:02.412632+0000","last_became_peered":"2024-04-25T01:26:02.412632+0000","last_unstale":"2024-04-25T01:41:09.349577+0000","last_undegraded":"2024-04-25T01:41:09.349577+0000","last_fullsized":"2024-04-25T01:41:09.349577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5570,"log_dups_size":0,"ondisk_log_size":5570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:10.049310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27971,"num_write_kb":349705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"26'4715","reported_seq":4746,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.811959+0000","last_change":"2024-04-25T01:25:53.361653+0000","last_active":"2024-04-25T01:40:58.811959+0000","last_peered":"2024-04-25T01:40:58.811959+0000","last_clean":"2024-04-25T01:40:58.811959+0000","last_became_active":"2024-04-25T01:25:53.361265+0000","last_became_peered":"2024-04-25T01:25:53.361265+0000","last_unstale":"2024-04-25T01:40:58.811959+0000","last_undegraded":"2024-04-25T01:40:58.811959+0000","last_fullsized":"2024-04-25T01:40:58.811959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4715,"log_dups_size":0,"ondisk_log_size":4715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23886,"num_write_kb":288267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'3","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.248825+0000","last_change":"2024-04-25T01:25:52.353836+0000","last_active":"2024-04-25T01:40:49.248825+0000","last_peered":"2024-04-25T01:40:49.248825+0000","last_clean":"2024-04-25T01:40:49.248825+0000","last_became_active":"2024-04-25T01:25:52.353552+0000","last_became_peered":"2024-04-25T01:25:52.353552+0000","last_unstale":"2024-04-25T01:40:49.248825+0000","last_undegraded":"2024-04-25T01:40:49.248825+0000","last_fullsized":"2024-04-25T01:40:49.248825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"26'6","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.134073+0000","last_change":"2024-04-25T01:26:01.411323+0000","last_active":"2024-04-25T01:40:37.134073+0000","last_peered":"2024-04-25T01:40:37.134073+0000","last_clean":"2024-04-25T01:40:37.134073+0000","last_became_active":"2024-04-25T01:26:01.410378+0000","last_became_peered":"2024-04-25T01:26:01.410378+0000","last_unstale":"2024-04-25T01:40:37.134073+0000","last_undegraded":"2024-04-25T01:40:37.134073+0000","last_fullsized":"2024-04-25T01:40:37.134073+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"26'4403","reported_seq":4426,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.251745+0000","last_change":"2024-04-25T01:26:02.416646+0000","last_active":"2024-04-25T01:41:09.251745+0000","last_peered":"2024-04-25T01:41:09.251745+0000","last_clean":"2024-04-25T01:41:09.251745+0000","last_became_active":"2024-04-25T01:26:02.416395+0000","last_became_peered":"2024-04-25T01:26:02.416395+0000","last_unstale":"2024-04-25T01:41:09.251745+0000","last_undegraded":"2024-04-25T01:41:09.251745+0000","last_fullsized":"2024-04-25T01:41:09.251745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4403,"log_dups_size":0,"ondisk_log_size":4403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23642,"num_write_kb":287389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"26'3766","reported_seq":3797,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.767490+0000","last_change":"2024-04-25T01:25:53.370111+0000","last_active":"2024-04-25T01:40:58.767490+0000","last_peered":"2024-04-25T01:40:58.767490+0000","last_clean":"2024-04-25T01:40:58.767490+0000","last_became_active":"2024-04-25T01:25:53.369837+0000","last_became_peered":"2024-04-25T01:25:53.369837+0000","last_unstale":"2024-04-25T01:40:58.767490+0000","last_undegraded":"2024-04-25T01:40:58.767490+0000","last_fullsized":"2024-04-25T01:40:58.767490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3766,"log_dups_size":0,"ondisk_log_size":3766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19293,"num_write_kb":249241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"26'5","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.300730+0000","last_change":"2024-04-25T01:25:52.353269+0000","last_active":"2024-04-25T01:40:49.300730+0000","last_peered":"2024-04-25T01:40:49.300730+0000","last_clean":"2024-04-25T01:40:49.300730+0000","last_became_active":"2024-04-25T01:25:52.352991+0000","last_became_peered":"2024-04-25T01:25:52.352991+0000","last_unstale":"2024-04-25T01:40:49.300730+0000","last_undegraded":"2024-04-25T01:40:49.300730+0000","last_fullsized":"2024-04-25T01:40:49.300730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"26'41","reported_seq":105,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:47.845985+0000","last_change":"2024-04-25T01:26:01.410360+0000","last_active":"2024-04-25T01:40:47.845985+0000","last_peered":"2024-04-25T01:40:47.845985+0000","last_clean":"2024-04-25T01:40:47.845985+0000","last_became_active":"2024-04-25T01:26:01.410076+0000","last_became_peered":"2024-04-25T01:26:01.410076+0000","last_unstale":"2024-04-25T01:40:47.845985+0000","last_undegraded":"2024-04-25T01:40:47.845985+0000","last_fullsized":"2024-04-25T01:40:47.845985+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:52.561251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":112,"num_write":39,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"26'4289","reported_seq":4312,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.299549+0000","last_change":"2024-04-25T01:26:02.422182+0000","last_active":"2024-04-25T01:41:09.299549+0000","last_peered":"2024-04-25T01:41:09.299549+0000","last_clean":"2024-04-25T01:41:09.299549+0000","last_became_active":"2024-04-25T01:26:02.421885+0000","last_became_peered":"2024-04-25T01:26:02.421885+0000","last_unstale":"2024-04-25T01:41:09.299549+0000","last_undegraded":"2024-04-25T01:41:09.299549+0000","last_fullsized":"2024-04-25T01:41:09.299549+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4289,"log_dups_size":0,"ondisk_log_size":4289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:03.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22172,"num_write_kb":285532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"26'4466","reported_seq":4497,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.806516+0000","last_change":"2024-04-25T01:25:53.364967+0000","last_active":"2024-04-25T01:40:58.806516+0000","last_peered":"2024-04-25T01:40:58.806516+0000","last_clean":"2024-04-25T01:40:58.806516+0000","last_became_active":"2024-04-25T01:25:53.364694+0000","last_became_peered":"2024-04-25T01:25:53.364694+0000","last_unstale":"2024-04-25T01:40:58.806516+0000","last_undegraded":"2024-04-25T01:40:58.806516+0000","last_fullsized":"2024-04-25T01:40:58.806516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4466,"log_dups_size":0,"ondisk_log_size":4466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22859,"num_write_kb":292275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'571","reported_seq":604,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:51.521378+0000","last_change":"2024-04-25T01:25:52.355395+0000","last_active":"2024-04-25T01:40:51.521378+0000","last_peered":"2024-04-25T01:40:51.521378+0000","last_clean":"2024-04-25T01:40:51.521378+0000","last_became_active":"2024-04-25T01:25:52.355137+0000","last_became_peered":"2024-04-25T01:25:52.355137+0000","last_unstale":"2024-04-25T01:40:51.521378+0000","last_undegraded":"2024-04-25T01:40:51.521378+0000","last_fullsized":"2024-04-25T01:40:51.521378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:26.467968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"26'5","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.173222+0000","last_change":"2024-04-25T01:26:01.416108+0000","last_active":"2024-04-25T01:40:37.173222+0000","last_peered":"2024-04-25T01:40:37.173222+0000","last_clean":"2024-04-25T01:40:37.173222+0000","last_became_active":"2024-04-25T01:26:01.415824+0000","last_became_peered":"2024-04-25T01:26:01.415824+0000","last_unstale":"2024-04-25T01:40:37.173222+0000","last_undegraded":"2024-04-25T01:40:37.173222+0000","last_fullsized":"2024-04-25T01:40:37.173222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:28.139183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"26'4989","reported_seq":5012,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.273942+0000","last_change":"2024-04-25T01:26:02.416208+0000","last_active":"2024-04-25T01:41:09.273942+0000","last_peered":"2024-04-25T01:41:09.273942+0000","last_clean":"2024-04-25T01:41:09.273942+0000","last_became_active":"2024-04-25T01:26:02.415949+0000","last_became_peered":"2024-04-25T01:26:02.415949+0000","last_unstale":"2024-04-25T01:41:09.273942+0000","last_undegraded":"2024-04-25T01:41:09.273942+0000","last_fullsized":"2024-04-25T01:41:09.273942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4989,"log_dups_size":0,"ondisk_log_size":4989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:21.097225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25442,"num_write_kb":314904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"26'4796","reported_seq":4827,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.753012+0000","last_change":"2024-04-25T01:25:53.366944+0000","last_active":"2024-04-25T01:40:58.753012+0000","last_peered":"2024-04-25T01:40:58.753012+0000","last_clean":"2024-04-25T01:40:58.753012+0000","last_became_active":"2024-04-25T01:25:53.366693+0000","last_became_peered":"2024-04-25T01:25:53.366693+0000","last_unstale":"2024-04-25T01:40:58.753012+0000","last_undegraded":"2024-04-25T01:40:58.753012+0000","last_fullsized":"2024-04-25T01:40:58.753012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25026,"num_write_kb":309186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"26'4","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.236233+0000","last_change":"2024-04-25T01:25:52.356842+0000","last_active":"2024-04-25T01:40:49.236233+0000","last_peered":"2024-04-25T01:40:49.236233+0000","last_clean":"2024-04-25T01:40:49.236233+0000","last_became_active":"2024-04-25T01:25:52.356396+0000","last_became_peered":"2024-04-25T01:25:52.356396+0000","last_unstale":"2024-04-25T01:40:49.236233+0000","last_undegraded":"2024-04-25T01:40:49.236233+0000","last_fullsized":"2024-04-25T01:40:49.236233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"26'1","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:38:30.810691+0000","last_change":"2024-04-25T01:26:01.407934+0000","last_active":"2024-04-25T01:38:30.810691+0000","last_peered":"2024-04-25T01:38:30.810691+0000","last_clean":"2024-04-25T01:38:30.810691+0000","last_became_active":"2024-04-25T01:26:01.407659+0000","last_became_peered":"2024-04-25T01:26:01.407659+0000","last_unstale":"2024-04-25T01:38:30.810691+0000","last_undegraded":"2024-04-25T01:38:30.810691+0000","last_fullsized":"2024-04-25T01:38:30.810691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:19.629335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"26'4196","reported_seq":4219,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.266753+0000","last_change":"2024-04-25T01:26:02.418087+0000","last_active":"2024-04-25T01:41:09.266753+0000","last_peered":"2024-04-25T01:41:09.266753+0000","last_clean":"2024-04-25T01:41:09.266753+0000","last_became_active":"2024-04-25T01:26:02.417622+0000","last_became_peered":"2024-04-25T01:26:02.417622+0000","last_unstale":"2024-04-25T01:41:09.266753+0000","last_undegraded":"2024-04-25T01:41:09.266753+0000","last_fullsized":"2024-04-25T01:41:09.266753+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4196,"log_dups_size":0,"ondisk_log_size":4196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:03.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22617,"num_write_kb":283190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"26'4472","reported_seq":4503,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.865649+0000","last_change":"2024-04-25T01:25:53.368164+0000","last_active":"2024-04-25T01:40:58.865649+0000","last_peered":"2024-04-25T01:40:58.865649+0000","last_clean":"2024-04-25T01:40:58.865649+0000","last_became_active":"2024-04-25T01:25:53.367854+0000","last_became_peered":"2024-04-25T01:25:53.367854+0000","last_unstale":"2024-04-25T01:40:58.865649+0000","last_undegraded":"2024-04-25T01:40:58.865649+0000","last_fullsized":"2024-04-25T01:40:58.865649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4472,"log_dups_size":0,"ondisk_log_size":4472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23917,"num_write_kb":276200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'5","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.237276+0000","last_change":"2024-04-25T01:25:52.350128+0000","last_active":"2024-04-25T01:40:49.237276+0000","last_peered":"2024-04-25T01:40:49.237276+0000","last_clean":"2024-04-25T01:40:49.237276+0000","last_became_active":"2024-04-25T01:25:52.349874+0000","last_became_peered":"2024-04-25T01:25:52.349874+0000","last_unstale":"2024-04-25T01:40:49.237276+0000","last_undegraded":"2024-04-25T01:40:49.237276+0000","last_fullsized":"2024-04-25T01:40:49.237276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"26'5","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.136409+0000","last_change":"2024-04-25T01:26:01.418196+0000","last_active":"2024-04-25T01:40:37.136409+0000","last_peered":"2024-04-25T01:40:37.136409+0000","last_clean":"2024-04-25T01:40:37.136409+0000","last_became_active":"2024-04-25T01:26:01.417820+0000","last_became_peered":"2024-04-25T01:26:01.417820+0000","last_unstale":"2024-04-25T01:40:37.136409+0000","last_undegraded":"2024-04-25T01:40:37.136409+0000","last_fullsized":"2024-04-25T01:40:37.136409+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"26'4093","reported_seq":4116,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.276141+0000","last_change":"2024-04-25T01:26:02.422761+0000","last_active":"2024-04-25T01:41:09.276141+0000","last_peered":"2024-04-25T01:41:09.276141+0000","last_clean":"2024-04-25T01:41:09.276141+0000","last_became_active":"2024-04-25T01:26:02.422503+0000","last_became_peered":"2024-04-25T01:26:02.422503+0000","last_unstale":"2024-04-25T01:41:09.276141+0000","last_undegraded":"2024-04-25T01:41:09.276141+0000","last_fullsized":"2024-04-25T01:41:09.276141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4093,"log_dups_size":0,"ondisk_log_size":4093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:55.040946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21550,"num_write_kb":266696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"26'4812","reported_seq":4843,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.791228+0000","last_change":"2024-04-25T01:25:53.361696+0000","last_active":"2024-04-25T01:40:58.791228+0000","last_peered":"2024-04-25T01:40:58.791228+0000","last_clean":"2024-04-25T01:40:58.791228+0000","last_became_active":"2024-04-25T01:25:53.361156+0000","last_became_peered":"2024-04-25T01:25:53.361156+0000","last_unstale":"2024-04-25T01:40:58.791228+0000","last_undegraded":"2024-04-25T01:40:58.791228+0000","last_fullsized":"2024-04-25T01:40:58.791228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4812,"log_dups_size":0,"ondisk_log_size":4812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:27.472315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25777,"num_write_kb":295962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"26'5","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.233531+0000","last_change":"2024-04-25T01:25:52.354499+0000","last_active":"2024-04-25T01:40:49.233531+0000","last_peered":"2024-04-25T01:40:49.233531+0000","last_clean":"2024-04-25T01:40:49.233531+0000","last_became_active":"2024-04-25T01:25:52.354193+0000","last_became_peered":"2024-04-25T01:25:52.354193+0000","last_unstale":"2024-04-25T01:40:49.233531+0000","last_undegraded":"2024-04-25T01:40:49.233531+0000","last_fullsized":"2024-04-25T01:40:49.233531+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"26'5","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.016444+0000","last_change":"2024-04-25T01:26:01.418186+0000","last_active":"2024-04-25T01:40:37.016444+0000","last_peered":"2024-04-25T01:40:37.016444+0000","last_clean":"2024-04-25T01:40:37.016444+0000","last_became_active":"2024-04-25T01:26:01.417805+0000","last_became_peered":"2024-04-25T01:26:01.417805+0000","last_unstale":"2024-04-25T01:40:37.016444+0000","last_undegraded":"2024-04-25T01:40:37.016444+0000","last_fullsized":"2024-04-25T01:40:37.016444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:19.629335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"26'3951","reported_seq":3973,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.320321+0000","last_change":"2024-04-25T01:26:02.415877+0000","last_active":"2024-04-25T01:41:09.320321+0000","last_peered":"2024-04-25T01:41:09.320321+0000","last_clean":"2024-04-25T01:41:09.320321+0000","last_became_active":"2024-04-25T01:26:02.415619+0000","last_became_peered":"2024-04-25T01:26:02.415619+0000","last_unstale":"2024-04-25T01:41:09.320321+0000","last_undegraded":"2024-04-25T01:41:09.320321+0000","last_fullsized":"2024-04-25T01:41:09.320321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3951,"log_dups_size":0,"ondisk_log_size":3951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:14.647083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20033,"num_write_kb":252185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"26'4985","reported_seq":5016,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.752192+0000","last_change":"2024-04-25T01:25:53.362922+0000","last_active":"2024-04-25T01:40:58.752192+0000","last_peered":"2024-04-25T01:40:58.752192+0000","last_clean":"2024-04-25T01:40:58.752192+0000","last_became_active":"2024-04-25T01:25:53.362533+0000","last_became_peered":"2024-04-25T01:25:53.362533+0000","last_unstale":"2024-04-25T01:40:58.752192+0000","last_undegraded":"2024-04-25T01:40:58.752192+0000","last_fullsized":"2024-04-25T01:40:58.752192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4985,"log_dups_size":0,"ondisk_log_size":4985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26332,"num_write_kb":298165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"26'43","reported_seq":117,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:47.854708+0000","last_change":"2024-04-25T01:25:52.345321+0000","last_active":"2024-04-25T01:40:47.854708+0000","last_peered":"2024-04-25T01:40:47.854708+0000","last_clean":"2024-04-25T01:40:47.854708+0000","last_became_active":"2024-04-25T01:25:52.345061+0000","last_became_peered":"2024-04-25T01:25:52.345061+0000","last_unstale":"2024-04-25T01:40:47.854708+0000","last_undegraded":"2024-04-25T01:40:47.854708+0000","last_fullsized":"2024-04-25T01:40:47.854708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":41,"num_read_kb":115,"num_write":41,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"26'4","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:36.977064+0000","last_change":"2024-04-25T01:26:01.418516+0000","last_active":"2024-04-25T01:40:36.977064+0000","last_peered":"2024-04-25T01:40:36.977064+0000","last_clean":"2024-04-25T01:40:36.977064+0000","last_became_active":"2024-04-25T01:26:01.418173+0000","last_became_peered":"2024-04-25T01:26:01.418173+0000","last_unstale":"2024-04-25T01:40:36.977064+0000","last_undegraded":"2024-04-25T01:40:36.977064+0000","last_fullsized":"2024-04-25T01:40:36.977064+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:02.568472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"26'4134","reported_seq":4156,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.333895+0000","last_change":"2024-04-25T01:26:02.418115+0000","last_active":"2024-04-25T01:41:09.333895+0000","last_peered":"2024-04-25T01:41:09.333895+0000","last_clean":"2024-04-25T01:41:09.333895+0000","last_became_active":"2024-04-25T01:26:02.417355+0000","last_became_peered":"2024-04-25T01:26:02.417355+0000","last_unstale":"2024-04-25T01:41:09.333895+0000","last_undegraded":"2024-04-25T01:41:09.333895+0000","last_fullsized":"2024-04-25T01:41:09.333895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4134,"log_dups_size":0,"ondisk_log_size":4134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:19.644468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21715,"num_write_kb":287743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"26'4344","reported_seq":4375,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.752413+0000","last_change":"2024-04-25T01:25:53.355950+0000","last_active":"2024-04-25T01:40:58.752413+0000","last_peered":"2024-04-25T01:40:58.752413+0000","last_clean":"2024-04-25T01:40:58.752413+0000","last_became_active":"2024-04-25T01:25:53.355648+0000","last_became_peered":"2024-04-25T01:25:53.355648+0000","last_unstale":"2024-04-25T01:40:58.752413+0000","last_undegraded":"2024-04-25T01:40:58.752413+0000","last_fullsized":"2024-04-25T01:40:58.752413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4344,"log_dups_size":0,"ondisk_log_size":4344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23132,"num_write_kb":289328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"26'5","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.249850+0000","last_change":"2024-04-25T01:25:52.357683+0000","last_active":"2024-04-25T01:40:49.249850+0000","last_peered":"2024-04-25T01:40:49.249850+0000","last_clean":"2024-04-25T01:40:49.249850+0000","last_became_active":"2024-04-25T01:25:52.356924+0000","last_became_peered":"2024-04-25T01:25:52.356924+0000","last_unstale":"2024-04-25T01:40:49.249850+0000","last_undegraded":"2024-04-25T01:40:49.249850+0000","last_fullsized":"2024-04-25T01:40:49.249850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"26'570","reported_seq":595,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:47.577546+0000","last_change":"2024-04-25T01:26:01.421260+0000","last_active":"2024-04-25T01:40:47.577546+0000","last_peered":"2024-04-25T01:40:47.577546+0000","last_clean":"2024-04-25T01:40:47.577546+0000","last_became_active":"2024-04-25T01:26:01.420831+0000","last_became_peered":"2024-04-25T01:26:01.420831+0000","last_unstale":"2024-04-25T01:40:47.577546+0000","last_undegraded":"2024-04-25T01:40:47.577546+0000","last_fullsized":"2024-04-25T01:40:47.577546+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1330,"num_write_kb":427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"26'4947","reported_seq":4970,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.324019+0000","last_change":"2024-04-25T01:26:02.421386+0000","last_active":"2024-04-25T01:41:09.324019+0000","last_peered":"2024-04-25T01:41:09.324019+0000","last_clean":"2024-04-25T01:41:09.324019+0000","last_became_active":"2024-04-25T01:26:02.421079+0000","last_became_peered":"2024-04-25T01:26:02.421079+0000","last_unstale":"2024-04-25T01:41:09.324019+0000","last_undegraded":"2024-04-25T01:41:09.324019+0000","last_fullsized":"2024-04-25T01:41:09.324019+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4947,"log_dups_size":0,"ondisk_log_size":4947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25964,"num_write_kb":313022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"26'4754","reported_seq":4770,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.812843+0000","last_change":"2024-04-25T01:26:52.515572+0000","last_active":"2024-04-25T01:40:58.812843+0000","last_peered":"2024-04-25T01:40:58.812843+0000","last_clean":"2024-04-25T01:40:58.812843+0000","last_became_active":"2024-04-25T01:26:52.515254+0000","last_became_peered":"2024-04-25T01:26:52.515254+0000","last_unstale":"2024-04-25T01:40:58.812843+0000","last_undegraded":"2024-04-25T01:40:58.812843+0000","last_fullsized":"2024-04-25T01:40:58.812843+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4754,"log_dups_size":0,"ondisk_log_size":4754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:08.975868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24235,"num_write_kb":310192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:38:25.919823+0000","last_change":"2024-04-25T01:25:52.353155+0000","last_active":"2024-04-25T01:38:25.919823+0000","last_peered":"2024-04-25T01:38:25.919823+0000","last_clean":"2024-04-25T01:38:25.919823+0000","last_became_active":"2024-04-25T01:25:52.352826+0000","last_became_peered":"2024-04-25T01:25:52.352826+0000","last_unstale":"2024-04-25T01:38:25.919823+0000","last_undegraded":"2024-04-25T01:38:25.919823+0000","last_fullsized":"2024-04-25T01:38:25.919823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":189,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:37:45.823358+0000","last_change":"2024-04-25T01:25:41.719670+0000","last_active":"2024-04-25T01:37:45.823358+0000","last_peered":"2024-04-25T01:37:45.823358+0000","last_clean":"2024-04-25T01:37:45.823358+0000","last_became_active":"2024-04-25T01:25:41.718721+0000","last_became_peered":"2024-04-25T01:25:41.718721+0000","last_unstale":"2024-04-25T01:37:45.823358+0000","last_undegraded":"2024-04-25T01:37:45.823358+0000","last_fullsized":"2024-04-25T01:37:45.823358+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_clean_scrub_stamp":"2024-04-25T01:25:40.688097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:27.456746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"26'4","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.023240+0000","last_change":"2024-04-25T01:26:01.409339+0000","last_active":"2024-04-25T01:40:37.023240+0000","last_peered":"2024-04-25T01:40:37.023240+0000","last_clean":"2024-04-25T01:40:37.023240+0000","last_became_active":"2024-04-25T01:26:01.409062+0000","last_became_peered":"2024-04-25T01:26:01.409062+0000","last_unstale":"2024-04-25T01:40:37.023240+0000","last_undegraded":"2024-04-25T01:40:37.023240+0000","last_fullsized":"2024-04-25T01:40:37.023240+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:52.561251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"26'3833","reported_seq":3855,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.306093+0000","last_change":"2024-04-25T01:26:02.418574+0000","last_active":"2024-04-25T01:41:09.306093+0000","last_peered":"2024-04-25T01:41:09.306093+0000","last_clean":"2024-04-25T01:41:09.306093+0000","last_became_active":"2024-04-25T01:26:02.416546+0000","last_became_peered":"2024-04-25T01:26:02.416546+0000","last_unstale":"2024-04-25T01:41:09.306093+0000","last_undegraded":"2024-04-25T01:41:09.306093+0000","last_fullsized":"2024-04-25T01:41:09.306093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3833,"log_dups_size":0,"ondisk_log_size":3833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:43.994141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19504,"num_write_kb":250193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"26'5320","reported_seq":5350,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.811822+0000","last_change":"2024-04-25T01:25:53.375523+0000","last_active":"2024-04-25T01:40:58.811822+0000","last_peered":"2024-04-25T01:40:58.811822+0000","last_clean":"2024-04-25T01:40:58.811822+0000","last_became_active":"2024-04-25T01:25:53.375219+0000","last_became_peered":"2024-04-25T01:25:53.375219+0000","last_unstale":"2024-04-25T01:40:58.811822+0000","last_undegraded":"2024-04-25T01:40:58.811822+0000","last_fullsized":"2024-04-25T01:40:58.811822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5320,"log_dups_size":0,"ondisk_log_size":5320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27324,"num_write_kb":314375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'3","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.181066+0000","last_change":"2024-04-25T01:25:52.352994+0000","last_active":"2024-04-25T01:40:49.181066+0000","last_peered":"2024-04-25T01:40:49.181066+0000","last_clean":"2024-04-25T01:40:49.181066+0000","last_became_active":"2024-04-25T01:25:52.352644+0000","last_became_peered":"2024-04-25T01:25:52.352644+0000","last_unstale":"2024-04-25T01:40:49.181066+0000","last_undegraded":"2024-04-25T01:40:49.181066+0000","last_fullsized":"2024-04-25T01:40:49.181066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"26'3","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.140689+0000","last_change":"2024-04-25T01:26:01.420891+0000","last_active":"2024-04-25T01:40:37.140689+0000","last_peered":"2024-04-25T01:40:37.140689+0000","last_clean":"2024-04-25T01:40:37.140689+0000","last_became_active":"2024-04-25T01:26:01.420664+0000","last_became_peered":"2024-04-25T01:26:01.420664+0000","last_unstale":"2024-04-25T01:40:37.140689+0000","last_undegraded":"2024-04-25T01:40:37.140689+0000","last_fullsized":"2024-04-25T01:40:37.140689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"26'4000","reported_seq":4022,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.297815+0000","last_change":"2024-04-25T01:26:02.403489+0000","last_active":"2024-04-25T01:41:09.297815+0000","last_peered":"2024-04-25T01:41:09.297815+0000","last_clean":"2024-04-25T01:41:09.297815+0000","last_became_active":"2024-04-25T01:26:02.403206+0000","last_became_peered":"2024-04-25T01:26:02.403206+0000","last_unstale":"2024-04-25T01:41:09.297815+0000","last_undegraded":"2024-04-25T01:41:09.297815+0000","last_fullsized":"2024-04-25T01:41:09.297815+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4000,"log_dups_size":0,"ondisk_log_size":4000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:49.872754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21231,"num_write_kb":269526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"26'4796","reported_seq":4826,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.770361+0000","last_change":"2024-04-25T01:25:53.371018+0000","last_active":"2024-04-25T01:40:58.770361+0000","last_peered":"2024-04-25T01:40:58.770361+0000","last_clean":"2024-04-25T01:40:58.770361+0000","last_became_active":"2024-04-25T01:25:53.370539+0000","last_became_peered":"2024-04-25T01:25:53.370539+0000","last_unstale":"2024-04-25T01:40:58.770361+0000","last_undegraded":"2024-04-25T01:40:58.770361+0000","last_fullsized":"2024-04-25T01:40:58.770361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25182,"num_write_kb":313860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"26'2","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:39:25.987518+0000","last_change":"2024-04-25T01:25:52.355133+0000","last_active":"2024-04-25T01:39:25.987518+0000","last_peered":"2024-04-25T01:39:25.987518+0000","last_clean":"2024-04-25T01:39:25.987518+0000","last_became_active":"2024-04-25T01:25:52.354836+0000","last_became_peered":"2024-04-25T01:25:52.354836+0000","last_unstale":"2024-04-25T01:39:25.987518+0000","last_undegraded":"2024-04-25T01:39:25.987518+0000","last_fullsized":"2024-04-25T01:39:25.987518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:10.331704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"26'1112","reported_seq":1136,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.176654+0000","last_change":"2024-04-25T01:26:01.419285+0000","last_active":"2024-04-25T01:40:37.176654+0000","last_peered":"2024-04-25T01:40:37.176654+0000","last_clean":"2024-04-25T01:40:37.176654+0000","last_became_active":"2024-04-25T01:26:01.418987+0000","last_became_peered":"2024-04-25T01:26:01.418987+0000","last_unstale":"2024-04-25T01:40:37.176654+0000","last_undegraded":"2024-04-25T01:40:37.176654+0000","last_fullsized":"2024-04-25T01:40:37.176654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1112,"log_dups_size":0,"ondisk_log_size":1112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1103,"num_write_kb":4430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"26'4082","reported_seq":4104,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.262432+0000","last_change":"2024-04-25T01:26:02.416544+0000","last_active":"2024-04-25T01:41:09.262432+0000","last_peered":"2024-04-25T01:41:09.262432+0000","last_clean":"2024-04-25T01:41:09.262432+0000","last_became_active":"2024-04-25T01:26:02.416189+0000","last_became_peered":"2024-04-25T01:26:02.416189+0000","last_unstale":"2024-04-25T01:41:09.262432+0000","last_undegraded":"2024-04-25T01:41:09.262432+0000","last_fullsized":"2024-04-25T01:41:09.262432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4082,"log_dups_size":0,"ondisk_log_size":4082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.688469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20562,"num_write_kb":274915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"26'5666","reported_seq":5696,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.889157+0000","last_change":"2024-04-25T01:25:53.370754+0000","last_active":"2024-04-25T01:40:58.889157+0000","last_peered":"2024-04-25T01:40:58.889157+0000","last_clean":"2024-04-25T01:40:58.889157+0000","last_became_active":"2024-04-25T01:25:53.370074+0000","last_became_peered":"2024-04-25T01:25:53.370074+0000","last_unstale":"2024-04-25T01:40:58.889157+0000","last_undegraded":"2024-04-25T01:40:58.889157+0000","last_fullsized":"2024-04-25T01:40:58.889157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5666,"log_dups_size":0,"ondisk_log_size":5666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29161,"num_write_kb":354533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"26'6","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.237424+0000","last_change":"2024-04-25T01:25:52.345943+0000","last_active":"2024-04-25T01:40:49.237424+0000","last_peered":"2024-04-25T01:40:49.237424+0000","last_clean":"2024-04-25T01:40:49.237424+0000","last_became_active":"2024-04-25T01:25:52.345717+0000","last_became_peered":"2024-04-25T01:25:52.345717+0000","last_unstale":"2024-04-25T01:40:49.237424+0000","last_undegraded":"2024-04-25T01:40:49.237424+0000","last_fullsized":"2024-04-25T01:40:49.237424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"26'3","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.011634+0000","last_change":"2024-04-25T01:26:01.416027+0000","last_active":"2024-04-25T01:40:37.011634+0000","last_peered":"2024-04-25T01:40:37.011634+0000","last_clean":"2024-04-25T01:40:37.011634+0000","last_became_active":"2024-04-25T01:26:01.415750+0000","last_became_peered":"2024-04-25T01:26:01.415750+0000","last_unstale":"2024-04-25T01:40:37.011634+0000","last_undegraded":"2024-04-25T01:40:37.011634+0000","last_fullsized":"2024-04-25T01:40:37.011634+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:27.042376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"26'4748","reported_seq":4770,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.276555+0000","last_change":"2024-04-25T01:26:02.414929+0000","last_active":"2024-04-25T01:41:09.276555+0000","last_peered":"2024-04-25T01:41:09.276555+0000","last_clean":"2024-04-25T01:41:09.276555+0000","last_became_active":"2024-04-25T01:26:02.414316+0000","last_became_peered":"2024-04-25T01:26:02.414316+0000","last_unstale":"2024-04-25T01:41:09.276555+0000","last_undegraded":"2024-04-25T01:41:09.276555+0000","last_fullsized":"2024-04-25T01:41:09.276555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4748,"log_dups_size":0,"ondisk_log_size":4748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:53.568065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24613,"num_write_kb":303491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"26'4546","reported_seq":4576,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.712213+0000","last_change":"2024-04-25T01:25:53.365171+0000","last_active":"2024-04-25T01:40:58.712213+0000","last_peered":"2024-04-25T01:40:58.712213+0000","last_clean":"2024-04-25T01:40:58.712213+0000","last_became_active":"2024-04-25T01:25:53.364049+0000","last_became_peered":"2024-04-25T01:25:53.364049+0000","last_unstale":"2024-04-25T01:40:58.712213+0000","last_undegraded":"2024-04-25T01:40:58.712213+0000","last_fullsized":"2024-04-25T01:40:58.712213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4546,"log_dups_size":0,"ondisk_log_size":4546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22740,"num_write_kb":287925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'5","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.243400+0000","last_change":"2024-04-25T01:25:52.350688+0000","last_active":"2024-04-25T01:40:49.243400+0000","last_peered":"2024-04-25T01:40:49.243400+0000","last_clean":"2024-04-25T01:40:49.243400+0000","last_became_active":"2024-04-25T01:25:52.350357+0000","last_became_peered":"2024-04-25T01:25:52.350357+0000","last_unstale":"2024-04-25T01:40:49.243400+0000","last_undegraded":"2024-04-25T01:40:49.243400+0000","last_fullsized":"2024-04-25T01:40:49.243400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"26'3","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.002855+0000","last_change":"2024-04-25T01:26:01.418580+0000","last_active":"2024-04-25T01:40:37.002855+0000","last_peered":"2024-04-25T01:40:37.002855+0000","last_clean":"2024-04-25T01:40:37.002855+0000","last_became_active":"2024-04-25T01:26:01.418179+0000","last_became_peered":"2024-04-25T01:26:01.418179+0000","last_unstale":"2024-04-25T01:40:37.002855+0000","last_undegraded":"2024-04-25T01:40:37.002855+0000","last_fullsized":"2024-04-25T01:40:37.002855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:42.987327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"26'3833","reported_seq":3855,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.301162+0000","last_change":"2024-04-25T01:26:02.418424+0000","last_active":"2024-04-25T01:41:09.301162+0000","last_peered":"2024-04-25T01:41:09.301162+0000","last_clean":"2024-04-25T01:41:09.301162+0000","last_became_active":"2024-04-25T01:26:02.418090+0000","last_became_peered":"2024-04-25T01:26:02.418090+0000","last_unstale":"2024-04-25T01:41:09.301162+0000","last_undegraded":"2024-04-25T01:41:09.301162+0000","last_fullsized":"2024-04-25T01:41:09.301162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3833,"log_dups_size":0,"ondisk_log_size":3833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:10.049310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19507,"num_write_kb":254331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"26'4627","reported_seq":4657,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.811217+0000","last_change":"2024-04-25T01:25:53.366376+0000","last_active":"2024-04-25T01:40:58.811217+0000","last_peered":"2024-04-25T01:40:58.811217+0000","last_clean":"2024-04-25T01:40:58.811217+0000","last_became_active":"2024-04-25T01:25:53.365715+0000","last_became_peered":"2024-04-25T01:25:53.365715+0000","last_unstale":"2024-04-25T01:40:58.811217+0000","last_undegraded":"2024-04-25T01:40:58.811217+0000","last_fullsized":"2024-04-25T01:40:58.811217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4627,"log_dups_size":0,"ondisk_log_size":4627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24780,"num_write_kb":300200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"26'11","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.216654+0000","last_change":"2024-04-25T01:25:52.347413+0000","last_active":"2024-04-25T01:40:49.216654+0000","last_peered":"2024-04-25T01:40:49.216654+0000","last_clean":"2024-04-25T01:40:49.216654+0000","last_became_active":"2024-04-25T01:25:52.347144+0000","last_became_peered":"2024-04-25T01:25:52.347144+0000","last_unstale":"2024-04-25T01:40:49.216654+0000","last_undegraded":"2024-04-25T01:40:49.216654+0000","last_fullsized":"2024-04-25T01:40:49.216654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"26'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.101937+0000","last_change":"2024-04-25T01:26:01.409917+0000","last_active":"2024-04-25T01:40:37.101937+0000","last_peered":"2024-04-25T01:40:37.101937+0000","last_clean":"2024-04-25T01:40:37.101937+0000","last_became_active":"2024-04-25T01:26:01.409654+0000","last_became_peered":"2024-04-25T01:26:01.409654+0000","last_unstale":"2024-04-25T01:40:37.101937+0000","last_undegraded":"2024-04-25T01:40:37.101937+0000","last_fullsized":"2024-04-25T01:40:37.101937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:37.142255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"26'4668","reported_seq":4690,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.342103+0000","last_change":"2024-04-25T01:26:02.425103+0000","last_active":"2024-04-25T01:41:09.342103+0000","last_peered":"2024-04-25T01:41:09.342103+0000","last_clean":"2024-04-25T01:41:09.342103+0000","last_became_active":"2024-04-25T01:26:02.424819+0000","last_became_peered":"2024-04-25T01:26:02.424819+0000","last_unstale":"2024-04-25T01:41:09.342103+0000","last_undegraded":"2024-04-25T01:41:09.342103+0000","last_fullsized":"2024-04-25T01:41:09.342103+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4668,"log_dups_size":0,"ondisk_log_size":4668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23987,"num_write_kb":301090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"26'5240","reported_seq":5270,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.796905+0000","last_change":"2024-04-25T01:25:53.364987+0000","last_active":"2024-04-25T01:40:58.796905+0000","last_peered":"2024-04-25T01:40:58.796905+0000","last_clean":"2024-04-25T01:40:58.796905+0000","last_became_active":"2024-04-25T01:25:53.364431+0000","last_became_peered":"2024-04-25T01:25:53.364431+0000","last_unstale":"2024-04-25T01:40:58.796905+0000","last_undegraded":"2024-04-25T01:40:58.796905+0000","last_fullsized":"2024-04-25T01:40:58.796905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5240,"log_dups_size":0,"ondisk_log_size":5240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27403,"num_write_kb":326994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'6","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.234588+0000","last_change":"2024-04-25T01:25:52.351205+0000","last_active":"2024-04-25T01:40:49.234588+0000","last_peered":"2024-04-25T01:40:49.234588+0000","last_clean":"2024-04-25T01:40:49.234588+0000","last_became_active":"2024-04-25T01:25:52.350946+0000","last_became_peered":"2024-04-25T01:25:52.350946+0000","last_unstale":"2024-04-25T01:40:49.234588+0000","last_undegraded":"2024-04-25T01:40:49.234588+0000","last_fullsized":"2024-04-25T01:40:49.234588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"26'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:31.329770+0000","last_change":"2024-04-25T01:26:01.415799+0000","last_active":"2024-04-25T01:40:31.329770+0000","last_peered":"2024-04-25T01:40:31.329770+0000","last_clean":"2024-04-25T01:40:31.329770+0000","last_became_active":"2024-04-25T01:26:01.415572+0000","last_became_peered":"2024-04-25T01:26:01.415572+0000","last_unstale":"2024-04-25T01:40:31.329770+0000","last_undegraded":"2024-04-25T01:40:31.329770+0000","last_fullsized":"2024-04-25T01:40:31.329770+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:54.034132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"26'4341","reported_seq":4363,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.334259+0000","last_change":"2024-04-25T01:26:02.422959+0000","last_active":"2024-04-25T01:41:09.334259+0000","last_peered":"2024-04-25T01:41:09.334259+0000","last_clean":"2024-04-25T01:41:09.334259+0000","last_became_active":"2024-04-25T01:26:02.422210+0000","last_became_peered":"2024-04-25T01:26:02.422210+0000","last_unstale":"2024-04-25T01:41:09.334259+0000","last_undegraded":"2024-04-25T01:41:09.334259+0000","last_fullsized":"2024-04-25T01:41:09.334259+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4341,"log_dups_size":0,"ondisk_log_size":4341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22600,"num_write_kb":283130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"26'4919","reported_seq":4949,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.871808+0000","last_change":"2024-04-25T01:25:53.368187+0000","last_active":"2024-04-25T01:40:58.871808+0000","last_peered":"2024-04-25T01:40:58.871808+0000","last_clean":"2024-04-25T01:40:58.871808+0000","last_became_active":"2024-04-25T01:25:53.367664+0000","last_became_peered":"2024-04-25T01:25:53.367664+0000","last_unstale":"2024-04-25T01:40:58.871808+0000","last_undegraded":"2024-04-25T01:40:58.871808+0000","last_fullsized":"2024-04-25T01:40:58.871808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4919,"log_dups_size":0,"ondisk_log_size":4919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25274,"num_write_kb":330818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"26'5","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.184872+0000","last_change":"2024-04-25T01:25:52.357397+0000","last_active":"2024-04-25T01:40:49.184872+0000","last_peered":"2024-04-25T01:40:49.184872+0000","last_clean":"2024-04-25T01:40:49.184872+0000","last_became_active":"2024-04-25T01:25:52.356881+0000","last_became_peered":"2024-04-25T01:25:52.356881+0000","last_unstale":"2024-04-25T01:40:49.184872+0000","last_undegraded":"2024-04-25T01:40:49.184872+0000","last_fullsized":"2024-04-25T01:40:49.184872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"26'314","reported_seq":338,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:46.230895+0000","last_change":"2024-04-25T01:26:01.420103+0000","last_active":"2024-04-25T01:40:46.230895+0000","last_peered":"2024-04-25T01:40:46.230895+0000","last_clean":"2024-04-25T01:40:46.230895+0000","last_became_active":"2024-04-25T01:26:01.419526+0000","last_became_peered":"2024-04-25T01:26:01.419526+0000","last_unstale":"2024-04-25T01:40:46.230895+0000","last_undegraded":"2024-04-25T01:40:46.230895+0000","last_fullsized":"2024-04-25T01:40:46.230895+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:27.042376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":310,"num_write_kb":4240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"26'6253","reported_seq":6275,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.317329+0000","last_change":"2024-04-25T01:26:02.421488+0000","last_active":"2024-04-25T01:41:09.317329+0000","last_peered":"2024-04-25T01:41:09.317329+0000","last_clean":"2024-04-25T01:41:09.317329+0000","last_became_active":"2024-04-25T01:26:02.419308+0000","last_became_peered":"2024-04-25T01:26:02.419308+0000","last_unstale":"2024-04-25T01:41:09.317329+0000","last_undegraded":"2024-04-25T01:41:09.317329+0000","last_fullsized":"2024-04-25T01:41:09.317329+0000","mapping_epoch":22,"log_start":"26'1000","ondisk_log_start":"26'1000","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5253,"log_dups_size":0,"ondisk_log_size":5253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32276,"num_write_kb":400144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"26'4645","reported_seq":4675,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.788191+0000","last_change":"2024-04-25T01:25:53.370720+0000","last_active":"2024-04-25T01:40:58.788191+0000","last_peered":"2024-04-25T01:40:58.788191+0000","last_clean":"2024-04-25T01:40:58.788191+0000","last_became_active":"2024-04-25T01:25:53.370142+0000","last_became_peered":"2024-04-25T01:25:53.370142+0000","last_unstale":"2024-04-25T01:40:58.788191+0000","last_undegraded":"2024-04-25T01:40:58.788191+0000","last_fullsized":"2024-04-25T01:40:58.788191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4645,"log_dups_size":0,"ondisk_log_size":4645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:52.833379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24130,"num_write_kb":289357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"26'8","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.232990+0000","last_change":"2024-04-25T01:25:52.349092+0000","last_active":"2024-04-25T01:40:49.232990+0000","last_peered":"2024-04-25T01:40:49.232990+0000","last_clean":"2024-04-25T01:40:49.232990+0000","last_became_active":"2024-04-25T01:25:52.348846+0000","last_became_peered":"2024-04-25T01:25:52.348846+0000","last_unstale":"2024-04-25T01:40:49.232990+0000","last_undegraded":"2024-04-25T01:40:49.232990+0000","last_fullsized":"2024-04-25T01:40:49.232990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:21.982440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"26'26","reported_seq":50,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.278610+0000","last_change":"2024-04-25T01:26:01.414517+0000","last_active":"2024-04-25T01:40:37.278610+0000","last_peered":"2024-04-25T01:40:37.278610+0000","last_clean":"2024-04-25T01:40:37.278610+0000","last_became_active":"2024-04-25T01:26:01.414243+0000","last_became_peered":"2024-04-25T01:26:01.414243+0000","last_unstale":"2024-04-25T01:40:37.278610+0000","last_undegraded":"2024-04-25T01:40:37.278610+0000","last_fullsized":"2024-04-25T01:40:37.278610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:48.491461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"26'5295","reported_seq":5317,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.307012+0000","last_change":"2024-04-25T01:26:02.415893+0000","last_active":"2024-04-25T01:41:09.307012+0000","last_peered":"2024-04-25T01:41:09.307012+0000","last_clean":"2024-04-25T01:41:09.307012+0000","last_became_active":"2024-04-25T01:26:02.415596+0000","last_became_peered":"2024-04-25T01:26:02.415596+0000","last_unstale":"2024-04-25T01:41:09.307012+0000","last_undegraded":"2024-04-25T01:41:09.307012+0000","last_fullsized":"2024-04-25T01:41:09.307012+0000","mapping_epoch":22,"log_start":"26'100","ondisk_log_start":"26'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5195,"log_dups_size":0,"ondisk_log_size":5195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26962,"num_write_kb":337426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"26'4577","reported_seq":4607,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.793341+0000","last_change":"2024-04-25T01:25:53.373572+0000","last_active":"2024-04-25T01:40:58.793341+0000","last_peered":"2024-04-25T01:40:58.793341+0000","last_clean":"2024-04-25T01:40:58.793341+0000","last_became_active":"2024-04-25T01:25:53.373129+0000","last_became_peered":"2024-04-25T01:25:53.373129+0000","last_unstale":"2024-04-25T01:40:58.793341+0000","last_undegraded":"2024-04-25T01:40:58.793341+0000","last_fullsized":"2024-04-25T01:40:58.793341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23671,"num_write_kb":287382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'335","reported_seq":367,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.269952+0000","last_change":"2024-04-25T01:25:52.350446+0000","last_active":"2024-04-25T01:40:49.269952+0000","last_peered":"2024-04-25T01:40:49.269952+0000","last_clean":"2024-04-25T01:40:49.269952+0000","last_became_active":"2024-04-25T01:25:52.349448+0000","last_became_peered":"2024-04-25T01:25:52.349448+0000","last_unstale":"2024-04-25T01:40:49.269952+0000","last_undegraded":"2024-04-25T01:40:49.269952+0000","last_fullsized":"2024-04-25T01:40:49.269952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":331,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"26'5","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.111146+0000","last_change":"2024-04-25T01:26:01.418794+0000","last_active":"2024-04-25T01:40:37.111146+0000","last_peered":"2024-04-25T01:40:37.111146+0000","last_clean":"2024-04-25T01:40:37.111146+0000","last_became_active":"2024-04-25T01:26:01.418541+0000","last_became_peered":"2024-04-25T01:26:01.418541+0000","last_unstale":"2024-04-25T01:40:37.111146+0000","last_undegraded":"2024-04-25T01:40:37.111146+0000","last_fullsized":"2024-04-25T01:40:37.111146+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"26'4257","reported_seq":4279,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.301321+0000","last_change":"2024-04-25T01:26:02.425391+0000","last_active":"2024-04-25T01:41:09.301321+0000","last_peered":"2024-04-25T01:41:09.301321+0000","last_clean":"2024-04-25T01:41:09.301321+0000","last_became_active":"2024-04-25T01:26:02.425106+0000","last_became_peered":"2024-04-25T01:26:02.425106+0000","last_unstale":"2024-04-25T01:41:09.301321+0000","last_undegraded":"2024-04-25T01:41:09.301321+0000","last_fullsized":"2024-04-25T01:41:09.301321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4257,"log_dups_size":0,"ondisk_log_size":4257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22152,"num_write_kb":281377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"26'6968","reported_seq":6998,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.878933+0000","last_change":"2024-04-25T01:25:53.366416+0000","last_active":"2024-04-25T01:40:58.878933+0000","last_peered":"2024-04-25T01:40:58.878933+0000","last_clean":"2024-04-25T01:40:58.878933+0000","last_became_active":"2024-04-25T01:25:53.366157+0000","last_became_peered":"2024-04-25T01:25:53.366157+0000","last_unstale":"2024-04-25T01:40:58.878933+0000","last_undegraded":"2024-04-25T01:40:58.878933+0000","last_fullsized":"2024-04-25T01:40:58.878933+0000","mapping_epoch":18,"log_start":"26'1500","ondisk_log_start":"26'1500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5468,"log_dups_size":0,"ondisk_log_size":5468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:26.277566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35763,"num_write_kb":414188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"26'25","reported_seq":57,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.254786+0000","last_change":"2024-04-25T01:25:52.357610+0000","last_active":"2024-04-25T01:40:49.254786+0000","last_peered":"2024-04-25T01:40:49.254786+0000","last_clean":"2024-04-25T01:40:49.254786+0000","last_became_active":"2024-04-25T01:25:52.357355+0000","last_became_peered":"2024-04-25T01:25:52.357355+0000","last_unstale":"2024-04-25T01:40:49.254786+0000","last_undegraded":"2024-04-25T01:40:49.254786+0000","last_fullsized":"2024-04-25T01:40:49.254786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"26'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.122870+0000","last_change":"2024-04-25T01:26:01.412666+0000","last_active":"2024-04-25T01:40:37.122870+0000","last_peered":"2024-04-25T01:40:37.122870+0000","last_clean":"2024-04-25T01:40:37.122870+0000","last_became_active":"2024-04-25T01:26:01.412132+0000","last_became_peered":"2024-04-25T01:26:01.412132+0000","last_unstale":"2024-04-25T01:40:37.122870+0000","last_undegraded":"2024-04-25T01:40:37.122870+0000","last_fullsized":"2024-04-25T01:40:37.122870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"26'5141","reported_seq":5163,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.286915+0000","last_change":"2024-04-25T01:26:02.413106+0000","last_active":"2024-04-25T01:41:09.286915+0000","last_peered":"2024-04-25T01:41:09.286915+0000","last_clean":"2024-04-25T01:41:09.286915+0000","last_became_active":"2024-04-25T01:26:02.412759+0000","last_became_peered":"2024-04-25T01:26:02.412759+0000","last_unstale":"2024-04-25T01:41:09.286915+0000","last_undegraded":"2024-04-25T01:41:09.286915+0000","last_fullsized":"2024-04-25T01:41:09.286915+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5141,"log_dups_size":0,"ondisk_log_size":5141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:28.049190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27471,"num_write_kb":310956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"26'3595","reported_seq":3625,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.782418+0000","last_change":"2024-04-25T01:25:53.374311+0000","last_active":"2024-04-25T01:40:58.782418+0000","last_peered":"2024-04-25T01:40:58.782418+0000","last_clean":"2024-04-25T01:40:58.782418+0000","last_became_active":"2024-04-25T01:25:53.373890+0000","last_became_peered":"2024-04-25T01:25:53.373890+0000","last_unstale":"2024-04-25T01:40:58.782418+0000","last_undegraded":"2024-04-25T01:40:58.782418+0000","last_fullsized":"2024-04-25T01:40:58.782418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3595,"log_dups_size":0,"ondisk_log_size":3595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18396,"num_write_kb":249899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"26'2","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.190934+0000","last_change":"2024-04-25T01:25:52.356231+0000","last_active":"2024-04-25T01:40:49.190934+0000","last_peered":"2024-04-25T01:40:49.190934+0000","last_clean":"2024-04-25T01:40:49.190934+0000","last_became_active":"2024-04-25T01:25:52.355913+0000","last_became_peered":"2024-04-25T01:25:52.355913+0000","last_unstale":"2024-04-25T01:40:49.190934+0000","last_undegraded":"2024-04-25T01:40:49.190934+0000","last_fullsized":"2024-04-25T01:40:49.190934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"26'12","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.017092+0000","last_change":"2024-04-25T01:26:01.419513+0000","last_active":"2024-04-25T01:40:37.017092+0000","last_peered":"2024-04-25T01:40:37.017092+0000","last_clean":"2024-04-25T01:40:37.017092+0000","last_became_active":"2024-04-25T01:26:01.419015+0000","last_became_peered":"2024-04-25T01:26:01.419015+0000","last_unstale":"2024-04-25T01:40:37.017092+0000","last_undegraded":"2024-04-25T01:40:37.017092+0000","last_fullsized":"2024-04-25T01:40:37.017092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:10.486367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"26'5013","reported_seq":5035,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.306814+0000","last_change":"2024-04-25T01:26:02.425241+0000","last_active":"2024-04-25T01:41:09.306814+0000","last_peered":"2024-04-25T01:41:09.306814+0000","last_clean":"2024-04-25T01:41:09.306814+0000","last_became_active":"2024-04-25T01:26:02.424960+0000","last_became_peered":"2024-04-25T01:26:02.424960+0000","last_unstale":"2024-04-25T01:41:09.306814+0000","last_undegraded":"2024-04-25T01:41:09.306814+0000","last_fullsized":"2024-04-25T01:41:09.306814+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5013,"log_dups_size":0,"ondisk_log_size":5013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25777,"num_write_kb":320486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"26'4876","reported_seq":4906,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.800754+0000","last_change":"2024-04-25T01:25:53.371445+0000","last_active":"2024-04-25T01:40:58.800754+0000","last_peered":"2024-04-25T01:40:58.800754+0000","last_clean":"2024-04-25T01:40:58.800754+0000","last_became_active":"2024-04-25T01:25:53.371086+0000","last_became_peered":"2024-04-25T01:25:53.371086+0000","last_unstale":"2024-04-25T01:40:58.800754+0000","last_undegraded":"2024-04-25T01:40:58.800754+0000","last_fullsized":"2024-04-25T01:40:58.800754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4876,"log_dups_size":0,"ondisk_log_size":4876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24656,"num_write_kb":303790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"26'2","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:26.547207+0000","last_change":"2024-04-25T01:25:52.357459+0000","last_active":"2024-04-25T01:40:26.547207+0000","last_peered":"2024-04-25T01:40:26.547207+0000","last_clean":"2024-04-25T01:40:26.547207+0000","last_became_active":"2024-04-25T01:25:52.357182+0000","last_became_peered":"2024-04-25T01:25:52.357182+0000","last_unstale":"2024-04-25T01:40:26.547207+0000","last_undegraded":"2024-04-25T01:40:26.547207+0000","last_fullsized":"2024-04-25T01:40:26.547207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"26'6","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.088116+0000","last_change":"2024-04-25T01:26:01.415321+0000","last_active":"2024-04-25T01:40:37.088116+0000","last_peered":"2024-04-25T01:40:37.088116+0000","last_clean":"2024-04-25T01:40:37.088116+0000","last_became_active":"2024-04-25T01:26:01.415020+0000","last_became_peered":"2024-04-25T01:26:01.415020+0000","last_unstale":"2024-04-25T01:40:37.088116+0000","last_undegraded":"2024-04-25T01:40:37.088116+0000","last_fullsized":"2024-04-25T01:40:37.088116+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:20.090411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"26'5316","reported_seq":5339,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.331286+0000","last_change":"2024-04-25T01:26:02.416917+0000","last_active":"2024-04-25T01:41:09.331286+0000","last_peered":"2024-04-25T01:41:09.331286+0000","last_clean":"2024-04-25T01:41:09.331286+0000","last_became_active":"2024-04-25T01:26:02.416623+0000","last_became_peered":"2024-04-25T01:26:02.416623+0000","last_unstale":"2024-04-25T01:41:09.331286+0000","last_undegraded":"2024-04-25T01:41:09.331286+0000","last_fullsized":"2024-04-25T01:41:09.331286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5316,"log_dups_size":0,"ondisk_log_size":5316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:55.040946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27734,"num_write_kb":356956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"26'8","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.028842+0000","last_change":"2024-04-25T01:26:01.415472+0000","last_active":"2024-04-25T01:40:37.028842+0000","last_peered":"2024-04-25T01:40:37.028842+0000","last_clean":"2024-04-25T01:40:37.028842+0000","last_became_active":"2024-04-25T01:26:01.415184+0000","last_became_peered":"2024-04-25T01:26:01.415184+0000","last_unstale":"2024-04-25T01:40:37.028842+0000","last_undegraded":"2024-04-25T01:40:37.028842+0000","last_fullsized":"2024-04-25T01:40:37.028842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"26'5623","reported_seq":5653,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.800035+0000","last_change":"2024-04-25T01:25:53.363043+0000","last_active":"2024-04-25T01:40:58.800035+0000","last_peered":"2024-04-25T01:40:58.800035+0000","last_clean":"2024-04-25T01:40:58.800035+0000","last_became_active":"2024-04-25T01:25:53.362601+0000","last_became_peered":"2024-04-25T01:25:53.362601+0000","last_unstale":"2024-04-25T01:40:58.800035+0000","last_undegraded":"2024-04-25T01:40:58.800035+0000","last_fullsized":"2024-04-25T01:40:58.800035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5623,"log_dups_size":0,"ondisk_log_size":5623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29125,"num_write_kb":354366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"26'3","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:46.309270+0000","last_change":"2024-04-25T01:25:52.358081+0000","last_active":"2024-04-25T01:40:46.309270+0000","last_peered":"2024-04-25T01:40:46.309270+0000","last_clean":"2024-04-25T01:40:46.309270+0000","last_became_active":"2024-04-25T01:25:52.357365+0000","last_became_peered":"2024-04-25T01:25:52.357365+0000","last_unstale":"2024-04-25T01:40:46.309270+0000","last_undegraded":"2024-04-25T01:40:46.309270+0000","last_fullsized":"2024-04-25T01:40:46.309270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"26'4779","reported_seq":4802,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.260694+0000","last_change":"2024-04-25T01:26:02.418872+0000","last_active":"2024-04-25T01:41:09.260694+0000","last_peered":"2024-04-25T01:41:09.260694+0000","last_clean":"2024-04-25T01:41:09.260694+0000","last_became_active":"2024-04-25T01:26:02.418614+0000","last_became_peered":"2024-04-25T01:26:02.418614+0000","last_unstale":"2024-04-25T01:41:09.260694+0000","last_undegraded":"2024-04-25T01:41:09.260694+0000","last_fullsized":"2024-04-25T01:41:09.260694+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4779,"log_dups_size":0,"ondisk_log_size":4779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:49.498275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25207,"num_write_kb":318198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"26'632","reported_seq":657,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:27.924243+0000","last_change":"2024-04-25T01:26:01.410258+0000","last_active":"2024-04-25T01:40:27.924243+0000","last_peered":"2024-04-25T01:40:27.924243+0000","last_clean":"2024-04-25T01:40:27.924243+0000","last_became_active":"2024-04-25T01:26:01.410004+0000","last_became_peered":"2024-04-25T01:26:01.410004+0000","last_unstale":"2024-04-25T01:40:27.924243+0000","last_undegraded":"2024-04-25T01:40:27.924243+0000","last_fullsized":"2024-04-25T01:40:27.924243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:48.865940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":630,"num_write_kb":4461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"26'4416","reported_seq":4446,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.812059+0000","last_change":"2024-04-25T01:25:53.363794+0000","last_active":"2024-04-25T01:40:58.812059+0000","last_peered":"2024-04-25T01:40:58.812059+0000","last_clean":"2024-04-25T01:40:58.812059+0000","last_became_active":"2024-04-25T01:25:53.363509+0000","last_became_peered":"2024-04-25T01:25:53.363509+0000","last_unstale":"2024-04-25T01:40:58.812059+0000","last_undegraded":"2024-04-25T01:40:58.812059+0000","last_fullsized":"2024-04-25T01:40:58.812059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4416,"log_dups_size":0,"ondisk_log_size":4416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23348,"num_write_kb":294385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"26'3","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:26.190581+0000","last_change":"2024-04-25T01:25:52.347464+0000","last_active":"2024-04-25T01:40:26.190581+0000","last_peered":"2024-04-25T01:40:26.190581+0000","last_clean":"2024-04-25T01:40:26.190581+0000","last_became_active":"2024-04-25T01:25:52.347138+0000","last_became_peered":"2024-04-25T01:25:52.347138+0000","last_unstale":"2024-04-25T01:40:26.190581+0000","last_undegraded":"2024-04-25T01:40:26.190581+0000","last_fullsized":"2024-04-25T01:40:26.190581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"26'5575","reported_seq":5597,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.362996+0000","last_change":"2024-04-25T01:26:02.425948+0000","last_active":"2024-04-25T01:41:09.362996+0000","last_peered":"2024-04-25T01:41:09.362996+0000","last_clean":"2024-04-25T01:41:09.362996+0000","last_became_active":"2024-04-25T01:26:02.424947+0000","last_became_peered":"2024-04-25T01:26:02.424947+0000","last_unstale":"2024-04-25T01:41:09.362996+0000","last_undegraded":"2024-04-25T01:41:09.362996+0000","last_fullsized":"2024-04-25T01:41:09.362996+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5575,"log_dups_size":0,"ondisk_log_size":5575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28821,"num_write_kb":361321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"26'87","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.639358+0000","last_change":"2024-04-25T01:25:52.349989+0000","last_active":"2024-04-25T01:40:58.639358+0000","last_peered":"2024-04-25T01:40:58.639358+0000","last_clean":"2024-04-25T01:40:58.639358+0000","last_became_active":"2024-04-25T01:25:52.349672+0000","last_became_peered":"2024-04-25T01:25:52.349672+0000","last_unstale":"2024-04-25T01:40:58.639358+0000","last_undegraded":"2024-04-25T01:40:58.639358+0000","last_fullsized":"2024-04-25T01:40:58.639358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":980823,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83,"num_write_kb":1015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"26'3832","reported_seq":3862,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.794598+0000","last_change":"2024-04-25T01:25:53.356610+0000","last_active":"2024-04-25T01:40:58.794598+0000","last_peered":"2024-04-25T01:40:58.794598+0000","last_clean":"2024-04-25T01:40:58.794598+0000","last_became_active":"2024-04-25T01:25:53.356015+0000","last_became_peered":"2024-04-25T01:25:53.356015+0000","last_unstale":"2024-04-25T01:40:58.794598+0000","last_undegraded":"2024-04-25T01:40:58.794598+0000","last_fullsized":"2024-04-25T01:40:58.794598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3832,"log_dups_size":0,"ondisk_log_size":3832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19386,"num_write_kb":237359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"26'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.169856+0000","last_change":"2024-04-25T01:26:01.420261+0000","last_active":"2024-04-25T01:40:37.169856+0000","last_peered":"2024-04-25T01:40:37.169856+0000","last_clean":"2024-04-25T01:40:37.169856+0000","last_became_active":"2024-04-25T01:26:01.420003+0000","last_became_peered":"2024-04-25T01:26:01.420003+0000","last_unstale":"2024-04-25T01:40:37.169856+0000","last_undegraded":"2024-04-25T01:40:37.169856+0000","last_fullsized":"2024-04-25T01:40:37.169856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:28.695853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"26'3175","reported_seq":3197,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.349312+0000","last_change":"2024-04-25T01:26:02.426327+0000","last_active":"2024-04-25T01:41:09.349312+0000","last_peered":"2024-04-25T01:41:09.349312+0000","last_clean":"2024-04-25T01:41:09.349312+0000","last_became_active":"2024-04-25T01:26:02.426054+0000","last_became_peered":"2024-04-25T01:26:02.426054+0000","last_unstale":"2024-04-25T01:41:09.349312+0000","last_undegraded":"2024-04-25T01:41:09.349312+0000","last_fullsized":"2024-04-25T01:41:09.349312+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3175,"log_dups_size":0,"ondisk_log_size":3175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:17.809738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15854,"num_write_kb":211025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"26'605","reported_seq":637,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.248248+0000","last_change":"2024-04-25T01:25:52.353955+0000","last_active":"2024-04-25T01:40:49.248248+0000","last_peered":"2024-04-25T01:40:49.248248+0000","last_clean":"2024-04-25T01:40:49.248248+0000","last_became_active":"2024-04-25T01:25:52.353659+0000","last_became_peered":"2024-04-25T01:25:52.353659+0000","last_unstale":"2024-04-25T01:40:49.248248+0000","last_undegraded":"2024-04-25T01:40:49.248248+0000","last_fullsized":"2024-04-25T01:40:49.248248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":603,"num_write_kb":4239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"26'4191","reported_seq":4221,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.749061+0000","last_change":"2024-04-25T01:25:53.366586+0000","last_active":"2024-04-25T01:40:58.749061+0000","last_peered":"2024-04-25T01:40:58.749061+0000","last_clean":"2024-04-25T01:40:58.749061+0000","last_became_active":"2024-04-25T01:25:53.366302+0000","last_became_peered":"2024-04-25T01:25:53.366302+0000","last_unstale":"2024-04-25T01:40:58.749061+0000","last_undegraded":"2024-04-25T01:40:58.749061+0000","last_fullsized":"2024-04-25T01:40:58.749061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4191,"log_dups_size":0,"ondisk_log_size":4191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21716,"num_write_kb":263159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"26'11","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.173685+0000","last_change":"2024-04-25T01:26:01.420960+0000","last_active":"2024-04-25T01:40:37.173685+0000","last_peered":"2024-04-25T01:40:37.173685+0000","last_clean":"2024-04-25T01:40:37.173685+0000","last_became_active":"2024-04-25T01:26:01.420713+0000","last_became_peered":"2024-04-25T01:26:01.420713+0000","last_unstale":"2024-04-25T01:40:37.173685+0000","last_undegraded":"2024-04-25T01:40:37.173685+0000","last_fullsized":"2024-04-25T01:40:37.173685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:29.489275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"26'4569","reported_seq":4591,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.350310+0000","last_change":"2024-04-25T01:26:02.419282+0000","last_active":"2024-04-25T01:41:09.350310+0000","last_peered":"2024-04-25T01:41:09.350310+0000","last_clean":"2024-04-25T01:41:09.350310+0000","last_became_active":"2024-04-25T01:26:02.418802+0000","last_became_peered":"2024-04-25T01:26:02.418802+0000","last_unstale":"2024-04-25T01:41:09.350310+0000","last_undegraded":"2024-04-25T01:41:09.350310+0000","last_fullsized":"2024-04-25T01:41:09.350310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4569,"log_dups_size":0,"ondisk_log_size":4569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:46.807175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23161,"num_write_kb":277521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"26'6240","reported_seq":6271,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.802380+0000","last_change":"2024-04-25T01:25:53.367143+0000","last_active":"2024-04-25T01:40:58.802380+0000","last_peered":"2024-04-25T01:40:58.802380+0000","last_clean":"2024-04-25T01:40:58.802380+0000","last_became_active":"2024-04-25T01:25:53.366847+0000","last_became_peered":"2024-04-25T01:25:53.366847+0000","last_unstale":"2024-04-25T01:40:58.802380+0000","last_undegraded":"2024-04-25T01:40:58.802380+0000","last_fullsized":"2024-04-25T01:40:58.802380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":6240,"log_dups_size":0,"ondisk_log_size":6240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32134,"num_write_kb":382834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"26'5302","reported_seq":5324,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.305354+0000","last_change":"2024-04-25T01:26:02.416438+0000","last_active":"2024-04-25T01:41:09.305354+0000","last_peered":"2024-04-25T01:41:09.305354+0000","last_clean":"2024-04-25T01:41:09.305354+0000","last_became_active":"2024-04-25T01:26:02.416170+0000","last_became_peered":"2024-04-25T01:26:02.416170+0000","last_unstale":"2024-04-25T01:41:09.305354+0000","last_undegraded":"2024-04-25T01:41:09.305354+0000","last_fullsized":"2024-04-25T01:41:09.305354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5302,"log_dups_size":0,"ondisk_log_size":5302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27000,"num_write_kb":345927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"26'4854","reported_seq":4885,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.807931+0000","last_change":"2024-04-25T01:25:53.371345+0000","last_active":"2024-04-25T01:40:58.807931+0000","last_peered":"2024-04-25T01:40:58.807931+0000","last_clean":"2024-04-25T01:40:58.807931+0000","last_became_active":"2024-04-25T01:25:53.371080+0000","last_became_peered":"2024-04-25T01:25:53.371080+0000","last_unstale":"2024-04-25T01:40:58.807931+0000","last_undegraded":"2024-04-25T01:40:58.807931+0000","last_fullsized":"2024-04-25T01:40:58.807931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4854,"log_dups_size":0,"ondisk_log_size":4854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:09.855763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24922,"num_write_kb":316898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"26'4098","reported_seq":4121,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.322931+0000","last_change":"2024-04-25T01:26:02.419688+0000","last_active":"2024-04-25T01:41:09.322931+0000","last_peered":"2024-04-25T01:41:09.322931+0000","last_clean":"2024-04-25T01:41:09.322931+0000","last_became_active":"2024-04-25T01:26:02.419417+0000","last_became_peered":"2024-04-25T01:26:02.419417+0000","last_unstale":"2024-04-25T01:41:09.322931+0000","last_undegraded":"2024-04-25T01:41:09.322931+0000","last_fullsized":"2024-04-25T01:41:09.322931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4098,"log_dups_size":0,"ondisk_log_size":4098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21368,"num_write_kb":269972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"26'5565","reported_seq":5596,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.839128+0000","last_change":"2024-04-25T01:25:53.369850+0000","last_active":"2024-04-25T01:40:58.839128+0000","last_peered":"2024-04-25T01:40:58.839128+0000","last_clean":"2024-04-25T01:40:58.839128+0000","last_became_active":"2024-04-25T01:25:53.369350+0000","last_became_peered":"2024-04-25T01:25:53.369350+0000","last_unstale":"2024-04-25T01:40:58.839128+0000","last_undegraded":"2024-04-25T01:40:58.839128+0000","last_fullsized":"2024-04-25T01:40:58.839128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5565,"log_dups_size":0,"ondisk_log_size":5565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29144,"num_write_kb":362518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"26'5067","reported_seq":5089,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.356968+0000","last_change":"2024-04-25T01:26:02.425528+0000","last_active":"2024-04-25T01:41:09.356968+0000","last_peered":"2024-04-25T01:41:09.356968+0000","last_clean":"2024-04-25T01:41:09.356968+0000","last_became_active":"2024-04-25T01:26:02.425160+0000","last_became_peered":"2024-04-25T01:26:02.425160+0000","last_unstale":"2024-04-25T01:41:09.356968+0000","last_undegraded":"2024-04-25T01:41:09.356968+0000","last_fullsized":"2024-04-25T01:41:09.356968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26737,"num_write_kb":324333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"26'3787","reported_seq":3818,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.775758+0000","last_change":"2024-04-25T01:25:53.357142+0000","last_active":"2024-04-25T01:40:58.775758+0000","last_peered":"2024-04-25T01:40:58.775758+0000","last_clean":"2024-04-25T01:40:58.775758+0000","last_became_active":"2024-04-25T01:25:53.356854+0000","last_became_peered":"2024-04-25T01:25:53.356854+0000","last_unstale":"2024-04-25T01:40:58.775758+0000","last_undegraded":"2024-04-25T01:40:58.775758+0000","last_fullsized":"2024-04-25T01:40:58.775758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3787,"log_dups_size":0,"ondisk_log_size":3787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19900,"num_write_kb":255902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"26'5481","reported_seq":5504,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.289690+0000","last_change":"2024-04-25T01:26:02.417492+0000","last_active":"2024-04-25T01:41:09.289690+0000","last_peered":"2024-04-25T01:41:09.289690+0000","last_clean":"2024-04-25T01:41:09.289690+0000","last_became_active":"2024-04-25T01:26:02.417234+0000","last_became_peered":"2024-04-25T01:26:02.417234+0000","last_unstale":"2024-04-25T01:41:09.289690+0000","last_undegraded":"2024-04-25T01:41:09.289690+0000","last_fullsized":"2024-04-25T01:41:09.289690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5481,"log_dups_size":0,"ondisk_log_size":5481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28430,"num_write_kb":359662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"26'4160","reported_seq":4191,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.791055+0000","last_change":"2024-04-25T01:25:53.364575+0000","last_active":"2024-04-25T01:40:58.791055+0000","last_peered":"2024-04-25T01:40:58.791055+0000","last_clean":"2024-04-25T01:40:58.791055+0000","last_became_active":"2024-04-25T01:25:53.364311+0000","last_became_peered":"2024-04-25T01:25:53.364311+0000","last_unstale":"2024-04-25T01:40:58.791055+0000","last_undegraded":"2024-04-25T01:40:58.791055+0000","last_fullsized":"2024-04-25T01:40:58.791055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4160,"log_dups_size":0,"ondisk_log_size":4160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21579,"num_write_kb":270844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"26'4125","reported_seq":4148,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.329055+0000","last_change":"2024-04-25T01:26:02.426813+0000","last_active":"2024-04-25T01:41:09.329055+0000","last_peered":"2024-04-25T01:41:09.329055+0000","last_clean":"2024-04-25T01:41:09.329055+0000","last_became_active":"2024-04-25T01:26:02.426552+0000","last_became_peered":"2024-04-25T01:26:02.426552+0000","last_unstale":"2024-04-25T01:41:09.329055+0000","last_undegraded":"2024-04-25T01:41:09.329055+0000","last_fullsized":"2024-04-25T01:41:09.329055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4125,"log_dups_size":0,"ondisk_log_size":4125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:03.687174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21785,"num_write_kb":263458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"26'5101","reported_seq":5132,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.740912+0000","last_change":"2024-04-25T01:25:53.368413+0000","last_active":"2024-04-25T01:40:58.740912+0000","last_peered":"2024-04-25T01:40:58.740912+0000","last_clean":"2024-04-25T01:40:58.740912+0000","last_became_active":"2024-04-25T01:25:53.368176+0000","last_became_peered":"2024-04-25T01:25:53.368176+0000","last_unstale":"2024-04-25T01:40:58.740912+0000","last_undegraded":"2024-04-25T01:40:58.740912+0000","last_fullsized":"2024-04-25T01:40:58.740912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5101,"log_dups_size":0,"ondisk_log_size":5101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26850,"num_write_kb":324721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"26'6020","reported_seq":6043,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.350276+0000","last_change":"2024-04-25T01:26:02.425185+0000","last_active":"2024-04-25T01:41:09.350276+0000","last_peered":"2024-04-25T01:41:09.350276+0000","last_clean":"2024-04-25T01:41:09.350276+0000","last_became_active":"2024-04-25T01:26:02.424903+0000","last_became_peered":"2024-04-25T01:26:02.424903+0000","last_unstale":"2024-04-25T01:41:09.350276+0000","last_undegraded":"2024-04-25T01:41:09.350276+0000","last_fullsized":"2024-04-25T01:41:09.350276+0000","mapping_epoch":22,"log_start":"26'300","ondisk_log_start":"26'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5720,"log_dups_size":0,"ondisk_log_size":5720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31332,"num_write_kb":379614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"26'4058","reported_seq":4089,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.867006+0000","last_change":"2024-04-25T01:25:53.372638+0000","last_active":"2024-04-25T01:40:58.867006+0000","last_peered":"2024-04-25T01:40:58.867006+0000","last_clean":"2024-04-25T01:40:58.867006+0000","last_became_active":"2024-04-25T01:25:53.372001+0000","last_became_peered":"2024-04-25T01:25:53.372001+0000","last_unstale":"2024-04-25T01:40:58.867006+0000","last_undegraded":"2024-04-25T01:40:58.867006+0000","last_fullsized":"2024-04-25T01:40:58.867006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4058,"log_dups_size":0,"ondisk_log_size":4058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20502,"num_write_kb":266785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"26'4843","reported_seq":4866,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.307262+0000","last_change":"2024-04-25T01:26:02.420867+0000","last_active":"2024-04-25T01:41:09.307262+0000","last_peered":"2024-04-25T01:41:09.307262+0000","last_clean":"2024-04-25T01:41:09.307262+0000","last_became_active":"2024-04-25T01:26:02.420621+0000","last_became_peered":"2024-04-25T01:26:02.420621+0000","last_unstale":"2024-04-25T01:41:09.307262+0000","last_undegraded":"2024-04-25T01:41:09.307262+0000","last_fullsized":"2024-04-25T01:41:09.307262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4843,"log_dups_size":0,"ondisk_log_size":4843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:11.493181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24777,"num_write_kb":316390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"26'5268","reported_seq":5299,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.743440+0000","last_change":"2024-04-25T01:25:53.365509+0000","last_active":"2024-04-25T01:40:58.743440+0000","last_peered":"2024-04-25T01:40:58.743440+0000","last_clean":"2024-04-25T01:40:58.743440+0000","last_became_active":"2024-04-25T01:25:53.365245+0000","last_became_peered":"2024-04-25T01:25:53.365245+0000","last_unstale":"2024-04-25T01:40:58.743440+0000","last_undegraded":"2024-04-25T01:40:58.743440+0000","last_fullsized":"2024-04-25T01:40:58.743440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5268,"log_dups_size":0,"ondisk_log_size":5268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26981,"num_write_kb":345791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"26'3558","reported_seq":3580,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.356542+0000","last_change":"2024-04-25T01:26:02.415672+0000","last_active":"2024-04-25T01:41:09.356542+0000","last_peered":"2024-04-25T01:41:09.356542+0000","last_clean":"2024-04-25T01:41:09.356542+0000","last_became_active":"2024-04-25T01:26:02.415389+0000","last_became_peered":"2024-04-25T01:26:02.415389+0000","last_unstale":"2024-04-25T01:41:09.356542+0000","last_undegraded":"2024-04-25T01:41:09.356542+0000","last_fullsized":"2024-04-25T01:41:09.356542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3558,"log_dups_size":0,"ondisk_log_size":3558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:52.342053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18097,"num_write_kb":236200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"26'4040","reported_seq":4071,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.818105+0000","last_change":"2024-04-25T01:25:53.365391+0000","last_active":"2024-04-25T01:40:58.818105+0000","last_peered":"2024-04-25T01:40:58.818105+0000","last_clean":"2024-04-25T01:40:58.818105+0000","last_became_active":"2024-04-25T01:25:53.364829+0000","last_became_peered":"2024-04-25T01:25:53.364829+0000","last_unstale":"2024-04-25T01:40:58.818105+0000","last_undegraded":"2024-04-25T01:40:58.818105+0000","last_fullsized":"2024-04-25T01:40:58.818105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4040,"log_dups_size":0,"ondisk_log_size":4040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20607,"num_write_kb":262822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"26'5431","reported_seq":5453,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.318622+0000","last_change":"2024-04-25T01:26:02.418292+0000","last_active":"2024-04-25T01:41:09.318622+0000","last_peered":"2024-04-25T01:41:09.318622+0000","last_clean":"2024-04-25T01:41:09.318622+0000","last_became_active":"2024-04-25T01:26:02.418036+0000","last_became_peered":"2024-04-25T01:26:02.418036+0000","last_unstale":"2024-04-25T01:41:09.318622+0000","last_undegraded":"2024-04-25T01:41:09.318622+0000","last_fullsized":"2024-04-25T01:41:09.318622+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5431,"log_dups_size":0,"ondisk_log_size":5431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28059,"num_write_kb":358266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"26'4709","reported_seq":4739,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.812514+0000","last_change":"2024-04-25T01:25:53.355667+0000","last_active":"2024-04-25T01:40:58.812514+0000","last_peered":"2024-04-25T01:40:58.812514+0000","last_clean":"2024-04-25T01:40:58.812514+0000","last_became_active":"2024-04-25T01:25:53.355417+0000","last_became_peered":"2024-04-25T01:25:53.355417+0000","last_unstale":"2024-04-25T01:40:58.812514+0000","last_undegraded":"2024-04-25T01:40:58.812514+0000","last_fullsized":"2024-04-25T01:40:58.812514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4709,"log_dups_size":0,"ondisk_log_size":4709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24394,"num_write_kb":298399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"26'4741","reported_seq":4763,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.305268+0000","last_change":"2024-04-25T01:26:02.414994+0000","last_active":"2024-04-25T01:41:09.305268+0000","last_peered":"2024-04-25T01:41:09.305268+0000","last_clean":"2024-04-25T01:41:09.305268+0000","last_became_active":"2024-04-25T01:26:02.414735+0000","last_became_peered":"2024-04-25T01:26:02.414735+0000","last_unstale":"2024-04-25T01:41:09.305268+0000","last_undegraded":"2024-04-25T01:41:09.305268+0000","last_fullsized":"2024-04-25T01:41:09.305268+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4741,"log_dups_size":0,"ondisk_log_size":4741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:44.960251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24824,"num_write_kb":300285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"26'5111","reported_seq":5141,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.869446+0000","last_change":"2024-04-25T01:25:53.371240+0000","last_active":"2024-04-25T01:40:58.869446+0000","last_peered":"2024-04-25T01:40:58.869446+0000","last_clean":"2024-04-25T01:40:58.869446+0000","last_became_active":"2024-04-25T01:25:53.370164+0000","last_became_peered":"2024-04-25T01:25:53.370164+0000","last_unstale":"2024-04-25T01:40:58.869446+0000","last_undegraded":"2024-04-25T01:40:58.869446+0000","last_fullsized":"2024-04-25T01:40:58.869446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26542,"num_write_kb":327744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"26'4367","reported_seq":4389,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.310155+0000","last_change":"2024-04-25T01:26:02.418272+0000","last_active":"2024-04-25T01:41:09.310155+0000","last_peered":"2024-04-25T01:41:09.310155+0000","last_clean":"2024-04-25T01:41:09.310155+0000","last_became_active":"2024-04-25T01:26:02.417977+0000","last_became_peered":"2024-04-25T01:26:02.417977+0000","last_unstale":"2024-04-25T01:41:09.310155+0000","last_undegraded":"2024-04-25T01:41:09.310155+0000","last_fullsized":"2024-04-25T01:41:09.310155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4367,"log_dups_size":0,"ondisk_log_size":4367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22024,"num_write_kb":293008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"26'5285","reported_seq":5315,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.820507+0000","last_change":"2024-04-25T01:25:53.376415+0000","last_active":"2024-04-25T01:40:58.820507+0000","last_peered":"2024-04-25T01:40:58.820507+0000","last_clean":"2024-04-25T01:40:58.820507+0000","last_became_active":"2024-04-25T01:25:53.376104+0000","last_became_peered":"2024-04-25T01:25:53.376104+0000","last_unstale":"2024-04-25T01:40:58.820507+0000","last_undegraded":"2024-04-25T01:40:58.820507+0000","last_fullsized":"2024-04-25T01:40:58.820507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5285,"log_dups_size":0,"ondisk_log_size":5285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27331,"num_write_kb":318531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"26'5020","reported_seq":5042,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.344122+0000","last_change":"2024-04-25T01:26:02.426333+0000","last_active":"2024-04-25T01:41:09.344122+0000","last_peered":"2024-04-25T01:41:09.344122+0000","last_clean":"2024-04-25T01:41:09.344122+0000","last_became_active":"2024-04-25T01:26:02.426069+0000","last_became_peered":"2024-04-25T01:26:02.426069+0000","last_unstale":"2024-04-25T01:41:09.344122+0000","last_undegraded":"2024-04-25T01:41:09.344122+0000","last_fullsized":"2024-04-25T01:41:09.344122+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5020,"log_dups_size":0,"ondisk_log_size":5020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26121,"num_write_kb":325976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"26'4910","reported_seq":4940,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.774391+0000","last_change":"2024-04-25T01:25:53.370314+0000","last_active":"2024-04-25T01:40:58.774391+0000","last_peered":"2024-04-25T01:40:58.774391+0000","last_clean":"2024-04-25T01:40:58.774391+0000","last_became_active":"2024-04-25T01:25:53.370049+0000","last_became_peered":"2024-04-25T01:25:53.370049+0000","last_unstale":"2024-04-25T01:40:58.774391+0000","last_undegraded":"2024-04-25T01:40:58.774391+0000","last_fullsized":"2024-04-25T01:40:58.774391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4910,"log_dups_size":0,"ondisk_log_size":4910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25833,"num_write_kb":304449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"26'4775","reported_seq":4797,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.332568+0000","last_change":"2024-04-25T01:26:02.415464+0000","last_active":"2024-04-25T01:41:09.332568+0000","last_peered":"2024-04-25T01:41:09.332568+0000","last_clean":"2024-04-25T01:41:09.332568+0000","last_became_active":"2024-04-25T01:26:02.415173+0000","last_became_peered":"2024-04-25T01:26:02.415173+0000","last_unstale":"2024-04-25T01:41:09.332568+0000","last_undegraded":"2024-04-25T01:41:09.332568+0000","last_fullsized":"2024-04-25T01:41:09.332568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4775,"log_dups_size":0,"ondisk_log_size":4775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.343169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24075,"num_write_kb":305295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"26'4891","reported_seq":4921,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.797939+0000","last_change":"2024-04-25T01:25:53.369585+0000","last_active":"2024-04-25T01:40:58.797939+0000","last_peered":"2024-04-25T01:40:58.797939+0000","last_clean":"2024-04-25T01:40:58.797939+0000","last_became_active":"2024-04-25T01:25:53.368499+0000","last_became_peered":"2024-04-25T01:25:53.368499+0000","last_unstale":"2024-04-25T01:40:58.797939+0000","last_undegraded":"2024-04-25T01:40:58.797939+0000","last_fullsized":"2024-04-25T01:40:58.797939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4891,"log_dups_size":0,"ondisk_log_size":4891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24867,"num_write_kb":304528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"26'4199","reported_seq":4221,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.286525+0000","last_change":"2024-04-25T01:26:02.421975+0000","last_active":"2024-04-25T01:41:09.286525+0000","last_peered":"2024-04-25T01:41:09.286525+0000","last_clean":"2024-04-25T01:41:09.286525+0000","last_became_active":"2024-04-25T01:26:02.421698+0000","last_became_peered":"2024-04-25T01:26:02.421698+0000","last_unstale":"2024-04-25T01:41:09.286525+0000","last_undegraded":"2024-04-25T01:41:09.286525+0000","last_fullsized":"2024-04-25T01:41:09.286525+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4199,"log_dups_size":0,"ondisk_log_size":4199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21738,"num_write_kb":267270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"26'3844","reported_seq":3874,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.786269+0000","last_change":"2024-04-25T01:25:53.370666+0000","last_active":"2024-04-25T01:40:58.786269+0000","last_peered":"2024-04-25T01:40:58.786269+0000","last_clean":"2024-04-25T01:40:58.786269+0000","last_became_active":"2024-04-25T01:25:53.369733+0000","last_became_peered":"2024-04-25T01:25:53.369733+0000","last_unstale":"2024-04-25T01:40:58.786269+0000","last_undegraded":"2024-04-25T01:40:58.786269+0000","last_fullsized":"2024-04-25T01:40:58.786269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3844,"log_dups_size":0,"ondisk_log_size":3844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19367,"num_write_kb":241344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"26'4316","reported_seq":4338,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.253195+0000","last_change":"2024-04-25T01:26:02.418114+0000","last_active":"2024-04-25T01:41:09.253195+0000","last_peered":"2024-04-25T01:41:09.253195+0000","last_clean":"2024-04-25T01:41:09.253195+0000","last_became_active":"2024-04-25T01:26:02.417810+0000","last_became_peered":"2024-04-25T01:26:02.417810+0000","last_unstale":"2024-04-25T01:41:09.253195+0000","last_undegraded":"2024-04-25T01:41:09.253195+0000","last_fullsized":"2024-04-25T01:41:09.253195+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4316,"log_dups_size":0,"ondisk_log_size":4316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:30.496089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22517,"num_write_kb":290939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"26'5814","reported_seq":5830,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.796893+0000","last_change":"2024-04-25T01:26:52.120824+0000","last_active":"2024-04-25T01:40:58.796893+0000","last_peered":"2024-04-25T01:40:58.796893+0000","last_clean":"2024-04-25T01:40:58.796893+0000","last_became_active":"2024-04-25T01:26:52.120588+0000","last_became_peered":"2024-04-25T01:26:52.120588+0000","last_unstale":"2024-04-25T01:40:58.796893+0000","last_undegraded":"2024-04-25T01:40:58.796893+0000","last_fullsized":"2024-04-25T01:40:58.796893+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5814,"log_dups_size":0,"ondisk_log_size":5814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:14.466644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29929,"num_write_kb":365779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.36","version":"26'4743","reported_seq":4765,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.335933+0000","last_change":"2024-04-25T01:26:02.426289+0000","last_active":"2024-04-25T01:41:09.335933+0000","last_peered":"2024-04-25T01:41:09.335933+0000","last_clean":"2024-04-25T01:41:09.335933+0000","last_became_active":"2024-04-25T01:26:02.426015+0000","last_became_peered":"2024-04-25T01:26:02.426015+0000","last_unstale":"2024-04-25T01:41:09.335933+0000","last_undegraded":"2024-04-25T01:41:09.335933+0000","last_fullsized":"2024-04-25T01:41:09.335933+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4743,"log_dups_size":0,"ondisk_log_size":4743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25470,"num_write_kb":331505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"26'4366","reported_seq":4396,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.821883+0000","last_change":"2024-04-25T01:25:53.374186+0000","last_active":"2024-04-25T01:40:58.821883+0000","last_peered":"2024-04-25T01:40:58.821883+0000","last_clean":"2024-04-25T01:40:58.821883+0000","last_became_active":"2024-04-25T01:25:53.373766+0000","last_became_peered":"2024-04-25T01:25:53.373766+0000","last_unstale":"2024-04-25T01:40:58.821883+0000","last_undegraded":"2024-04-25T01:40:58.821883+0000","last_fullsized":"2024-04-25T01:40:58.821883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4366,"log_dups_size":0,"ondisk_log_size":4366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23328,"num_write_kb":273725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"26'5875","reported_seq":5897,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.356318+0000","last_change":"2024-04-25T01:26:02.416345+0000","last_active":"2024-04-25T01:41:09.356318+0000","last_peered":"2024-04-25T01:41:09.356318+0000","last_clean":"2024-04-25T01:41:09.356318+0000","last_became_active":"2024-04-25T01:26:02.416082+0000","last_became_peered":"2024-04-25T01:26:02.416082+0000","last_unstale":"2024-04-25T01:41:09.356318+0000","last_undegraded":"2024-04-25T01:41:09.356318+0000","last_fullsized":"2024-04-25T01:41:09.356318+0000","mapping_epoch":22,"log_start":"26'400","ondisk_log_start":"26'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5475,"log_dups_size":0,"ondisk_log_size":5475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:47.810943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31009,"num_write_kb":370059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"26'4898","reported_seq":4928,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.812217+0000","last_change":"2024-04-25T01:25:53.371602+0000","last_active":"2024-04-25T01:40:58.812217+0000","last_peered":"2024-04-25T01:40:58.812217+0000","last_clean":"2024-04-25T01:40:58.812217+0000","last_became_active":"2024-04-25T01:25:53.371362+0000","last_became_peered":"2024-04-25T01:25:53.371362+0000","last_unstale":"2024-04-25T01:40:58.812217+0000","last_undegraded":"2024-04-25T01:40:58.812217+0000","last_fullsized":"2024-04-25T01:40:58.812217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4898,"log_dups_size":0,"ondisk_log_size":4898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24919,"num_write_kb":308746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"26'3961","reported_seq":3983,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.316956+0000","last_change":"2024-04-25T01:26:02.425888+0000","last_active":"2024-04-25T01:41:09.316956+0000","last_peered":"2024-04-25T01:41:09.316956+0000","last_clean":"2024-04-25T01:41:09.316956+0000","last_became_active":"2024-04-25T01:26:02.425577+0000","last_became_peered":"2024-04-25T01:26:02.425577+0000","last_unstale":"2024-04-25T01:41:09.316956+0000","last_undegraded":"2024-04-25T01:41:09.316956+0000","last_fullsized":"2024-04-25T01:41:09.316956+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3961,"log_dups_size":0,"ondisk_log_size":3961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20296,"num_write_kb":253395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"26'4670","reported_seq":4700,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.788790+0000","last_change":"2024-04-25T01:25:53.370996+0000","last_active":"2024-04-25T01:40:58.788790+0000","last_peered":"2024-04-25T01:40:58.788790+0000","last_clean":"2024-04-25T01:40:58.788790+0000","last_became_active":"2024-04-25T01:25:53.370479+0000","last_became_peered":"2024-04-25T01:25:53.370479+0000","last_unstale":"2024-04-25T01:40:58.788790+0000","last_undegraded":"2024-04-25T01:40:58.788790+0000","last_fullsized":"2024-04-25T01:40:58.788790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4670,"log_dups_size":0,"ondisk_log_size":4670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:57.636067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24421,"num_write_kb":306681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"26'4733","reported_seq":4755,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.275948+0000","last_change":"2024-04-25T01:26:02.423420+0000","last_active":"2024-04-25T01:41:09.275948+0000","last_peered":"2024-04-25T01:41:09.275948+0000","last_clean":"2024-04-25T01:41:09.275948+0000","last_became_active":"2024-04-25T01:26:02.423146+0000","last_became_peered":"2024-04-25T01:26:02.423146+0000","last_unstale":"2024-04-25T01:41:09.275948+0000","last_undegraded":"2024-04-25T01:41:09.275948+0000","last_fullsized":"2024-04-25T01:41:09.275948+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4733,"log_dups_size":0,"ondisk_log_size":4733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23928,"num_write_kb":292515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"26'4513","reported_seq":4543,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.807710+0000","last_change":"2024-04-25T01:25:53.363512+0000","last_active":"2024-04-25T01:40:58.807710+0000","last_peered":"2024-04-25T01:40:58.807710+0000","last_clean":"2024-04-25T01:40:58.807710+0000","last_became_active":"2024-04-25T01:25:53.363246+0000","last_became_peered":"2024-04-25T01:25:53.363246+0000","last_unstale":"2024-04-25T01:40:58.807710+0000","last_undegraded":"2024-04-25T01:40:58.807710+0000","last_fullsized":"2024-04-25T01:40:58.807710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4513,"log_dups_size":0,"ondisk_log_size":4513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23705,"num_write_kb":279339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"26'4587","reported_seq":4609,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.306205+0000","last_change":"2024-04-25T01:26:02.413346+0000","last_active":"2024-04-25T01:41:09.306205+0000","last_peered":"2024-04-25T01:41:09.306205+0000","last_clean":"2024-04-25T01:41:09.306205+0000","last_became_active":"2024-04-25T01:26:02.413036+0000","last_became_peered":"2024-04-25T01:26:02.413036+0000","last_unstale":"2024-04-25T01:41:09.306205+0000","last_undegraded":"2024-04-25T01:41:09.306205+0000","last_fullsized":"2024-04-25T01:41:09.306205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4587,"log_dups_size":0,"ondisk_log_size":4587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24042,"num_write_kb":305325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"26'3998","reported_seq":4028,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.876422+0000","last_change":"2024-04-25T01:25:53.364457+0000","last_active":"2024-04-25T01:40:58.876422+0000","last_peered":"2024-04-25T01:40:58.876422+0000","last_clean":"2024-04-25T01:40:58.876422+0000","last_became_active":"2024-04-25T01:25:53.364103+0000","last_became_peered":"2024-04-25T01:25:53.364103+0000","last_unstale":"2024-04-25T01:40:58.876422+0000","last_undegraded":"2024-04-25T01:40:58.876422+0000","last_fullsized":"2024-04-25T01:40:58.876422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3998,"log_dups_size":0,"ondisk_log_size":3998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:11.336752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20343,"num_write_kb":253659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"26'4194","reported_seq":4216,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.298446+0000","last_change":"2024-04-25T01:26:02.418792+0000","last_active":"2024-04-25T01:41:09.298446+0000","last_peered":"2024-04-25T01:41:09.298446+0000","last_clean":"2024-04-25T01:41:09.298446+0000","last_became_active":"2024-04-25T01:26:02.416942+0000","last_became_peered":"2024-04-25T01:26:02.416942+0000","last_unstale":"2024-04-25T01:41:09.298446+0000","last_undegraded":"2024-04-25T01:41:09.298446+0000","last_fullsized":"2024-04-25T01:41:09.298446+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4194,"log_dups_size":0,"ondisk_log_size":4194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:59.492678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21868,"num_write_kb":272015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"26'4800","reported_seq":4830,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.771199+0000","last_change":"2024-04-25T01:25:53.371647+0000","last_active":"2024-04-25T01:40:58.771199+0000","last_peered":"2024-04-25T01:40:58.771199+0000","last_clean":"2024-04-25T01:40:58.771199+0000","last_became_active":"2024-04-25T01:25:53.371353+0000","last_became_peered":"2024-04-25T01:25:53.371353+0000","last_unstale":"2024-04-25T01:40:58.771199+0000","last_undegraded":"2024-04-25T01:40:58.771199+0000","last_fullsized":"2024-04-25T01:40:58.771199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4800,"log_dups_size":0,"ondisk_log_size":4800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24494,"num_write_kb":294847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"26'4139","reported_seq":4161,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.342741+0000","last_change":"2024-04-25T01:26:02.426335+0000","last_active":"2024-04-25T01:41:09.342741+0000","last_peered":"2024-04-25T01:41:09.342741+0000","last_clean":"2024-04-25T01:41:09.342741+0000","last_became_active":"2024-04-25T01:26:02.426024+0000","last_became_peered":"2024-04-25T01:26:02.426024+0000","last_unstale":"2024-04-25T01:41:09.342741+0000","last_undegraded":"2024-04-25T01:41:09.342741+0000","last_fullsized":"2024-04-25T01:41:09.342741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4139,"log_dups_size":0,"ondisk_log_size":4139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21818,"num_write_kb":267665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"26'5798","reported_seq":5828,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.869849+0000","last_change":"2024-04-25T01:25:53.372842+0000","last_active":"2024-04-25T01:40:58.869849+0000","last_peered":"2024-04-25T01:40:58.869849+0000","last_clean":"2024-04-25T01:40:58.869849+0000","last_became_active":"2024-04-25T01:25:53.372580+0000","last_became_peered":"2024-04-25T01:25:53.372580+0000","last_unstale":"2024-04-25T01:40:58.869849+0000","last_undegraded":"2024-04-25T01:40:58.869849+0000","last_fullsized":"2024-04-25T01:40:58.869849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5798,"log_dups_size":0,"ondisk_log_size":5798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30839,"num_write_kb":352989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"26'4687","reported_seq":4709,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.302280+0000","last_change":"2024-04-25T01:26:02.414632+0000","last_active":"2024-04-25T01:41:09.302280+0000","last_peered":"2024-04-25T01:41:09.302280+0000","last_clean":"2024-04-25T01:41:09.302280+0000","last_became_active":"2024-04-25T01:26:02.414388+0000","last_became_peered":"2024-04-25T01:26:02.414388+0000","last_unstale":"2024-04-25T01:41:09.302280+0000","last_undegraded":"2024-04-25T01:41:09.302280+0000","last_fullsized":"2024-04-25T01:41:09.302280+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4687,"log_dups_size":0,"ondisk_log_size":4687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23866,"num_write_kb":304526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"26'5655","reported_seq":5690,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.888926+0000","last_change":"2024-04-25T01:26:52.515807+0000","last_active":"2024-04-25T01:40:58.888926+0000","last_peered":"2024-04-25T01:40:58.888926+0000","last_clean":"2024-04-25T01:40:58.888926+0000","last_became_active":"2024-04-25T01:26:52.515504+0000","last_became_peered":"2024-04-25T01:26:52.515504+0000","last_unstale":"2024-04-25T01:40:58.888926+0000","last_undegraded":"2024-04-25T01:40:58.888926+0000","last_fullsized":"2024-04-25T01:40:58.888926+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5655,"log_dups_size":0,"ondisk_log_size":5655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:48.624287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29809,"num_write_kb":365295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"26'5769","reported_seq":5792,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.330796+0000","last_change":"2024-04-25T01:26:02.414366+0000","last_active":"2024-04-25T01:41:09.330796+0000","last_peered":"2024-04-25T01:41:09.330796+0000","last_clean":"2024-04-25T01:41:09.330796+0000","last_became_active":"2024-04-25T01:26:02.414061+0000","last_became_peered":"2024-04-25T01:26:02.414061+0000","last_unstale":"2024-04-25T01:41:09.330796+0000","last_undegraded":"2024-04-25T01:41:09.330796+0000","last_fullsized":"2024-04-25T01:41:09.330796+0000","mapping_epoch":22,"log_start":"26'500","ondisk_log_start":"26'500","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5269,"log_dups_size":0,"ondisk_log_size":5269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:19.644468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30404,"num_write_kb":379952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"26'5055","reported_seq":5085,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.844665+0000","last_change":"2024-04-25T01:25:53.374061+0000","last_active":"2024-04-25T01:40:58.844665+0000","last_peered":"2024-04-25T01:40:58.844665+0000","last_clean":"2024-04-25T01:40:58.844665+0000","last_became_active":"2024-04-25T01:25:53.373848+0000","last_became_peered":"2024-04-25T01:25:53.373848+0000","last_unstale":"2024-04-25T01:40:58.844665+0000","last_undegraded":"2024-04-25T01:40:58.844665+0000","last_fullsized":"2024-04-25T01:40:58.844665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5055,"log_dups_size":0,"ondisk_log_size":5055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25773,"num_write_kb":324544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"26'4195","reported_seq":4218,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.317465+0000","last_change":"2024-04-25T01:26:02.414620+0000","last_active":"2024-04-25T01:41:09.317465+0000","last_peered":"2024-04-25T01:41:09.317465+0000","last_clean":"2024-04-25T01:41:09.317465+0000","last_became_active":"2024-04-25T01:26:02.414326+0000","last_became_peered":"2024-04-25T01:26:02.414326+0000","last_unstale":"2024-04-25T01:41:09.317465+0000","last_undegraded":"2024-04-25T01:41:09.317465+0000","last_fullsized":"2024-04-25T01:41:09.317465+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4195,"log_dups_size":0,"ondisk_log_size":4195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:30.496089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21944,"num_write_kb":264053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"26'4639","reported_seq":4669,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.755355+0000","last_change":"2024-04-25T01:25:53.364698+0000","last_active":"2024-04-25T01:40:58.755355+0000","last_peered":"2024-04-25T01:40:58.755355+0000","last_clean":"2024-04-25T01:40:58.755355+0000","last_became_active":"2024-04-25T01:25:53.364435+0000","last_became_peered":"2024-04-25T01:25:53.364435+0000","last_unstale":"2024-04-25T01:40:58.755355+0000","last_undegraded":"2024-04-25T01:40:58.755355+0000","last_fullsized":"2024-04-25T01:40:58.755355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4639,"log_dups_size":0,"ondisk_log_size":4639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24873,"num_write_kb":296467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"26'4393","reported_seq":4416,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.341040+0000","last_change":"2024-04-25T01:26:02.418331+0000","last_active":"2024-04-25T01:41:09.341040+0000","last_peered":"2024-04-25T01:41:09.341040+0000","last_clean":"2024-04-25T01:41:09.341040+0000","last_became_active":"2024-04-25T01:26:02.418057+0000","last_became_peered":"2024-04-25T01:26:02.418057+0000","last_unstale":"2024-04-25T01:41:09.341040+0000","last_undegraded":"2024-04-25T01:41:09.341040+0000","last_fullsized":"2024-04-25T01:41:09.341040+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4393,"log_dups_size":0,"ondisk_log_size":4393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:17.809738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22950,"num_write_kb":276425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"26'3744","reported_seq":3759,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.790360+0000","last_change":"2024-04-25T01:26:52.514832+0000","last_active":"2024-04-25T01:40:58.790360+0000","last_peered":"2024-04-25T01:40:58.790360+0000","last_clean":"2024-04-25T01:40:58.790360+0000","last_became_active":"2024-04-25T01:26:52.514260+0000","last_became_peered":"2024-04-25T01:26:52.514260+0000","last_unstale":"2024-04-25T01:40:58.790360+0000","last_undegraded":"2024-04-25T01:40:58.790360+0000","last_fullsized":"2024-04-25T01:40:58.790360+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3744,"log_dups_size":0,"ondisk_log_size":3744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:51.631130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19457,"num_write_kb":254077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3d","version":"26'4696","reported_seq":4719,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.334457+0000","last_change":"2024-04-25T01:26:02.422720+0000","last_active":"2024-04-25T01:41:09.334457+0000","last_peered":"2024-04-25T01:41:09.334457+0000","last_clean":"2024-04-25T01:41:09.334457+0000","last_became_active":"2024-04-25T01:26:02.421848+0000","last_became_peered":"2024-04-25T01:26:02.421848+0000","last_unstale":"2024-04-25T01:41:09.334457+0000","last_undegraded":"2024-04-25T01:41:09.334457+0000","last_fullsized":"2024-04-25T01:41:09.334457+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4696,"log_dups_size":0,"ondisk_log_size":4696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24119,"num_write_kb":305642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"26'5605","reported_seq":5635,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.863402+0000","last_change":"2024-04-25T01:25:53.367953+0000","last_active":"2024-04-25T01:40:58.863402+0000","last_peered":"2024-04-25T01:40:58.863402+0000","last_clean":"2024-04-25T01:40:58.863402+0000","last_became_active":"2024-04-25T01:25:53.367358+0000","last_became_peered":"2024-04-25T01:25:53.367358+0000","last_unstale":"2024-04-25T01:40:58.863402+0000","last_undegraded":"2024-04-25T01:40:58.863402+0000","last_fullsized":"2024-04-25T01:40:58.863402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5605,"log_dups_size":0,"ondisk_log_size":5605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27762,"num_write_kb":352787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"26'4305","reported_seq":4327,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.310344+0000","last_change":"2024-04-25T01:26:02.422300+0000","last_active":"2024-04-25T01:41:09.310344+0000","last_peered":"2024-04-25T01:41:09.310344+0000","last_clean":"2024-04-25T01:41:09.310344+0000","last_became_active":"2024-04-25T01:26:02.421702+0000","last_became_peered":"2024-04-25T01:26:02.421702+0000","last_unstale":"2024-04-25T01:41:09.310344+0000","last_undegraded":"2024-04-25T01:41:09.310344+0000","last_fullsized":"2024-04-25T01:41:09.310344+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4305,"log_dups_size":0,"ondisk_log_size":4305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21701,"num_write_kb":263037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"26'4577","reported_seq":4608,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.784384+0000","last_change":"2024-04-25T01:25:53.371513+0000","last_active":"2024-04-25T01:40:58.784384+0000","last_peered":"2024-04-25T01:40:58.784384+0000","last_clean":"2024-04-25T01:40:58.784384+0000","last_became_active":"2024-04-25T01:25:53.370421+0000","last_became_peered":"2024-04-25T01:25:53.370421+0000","last_unstale":"2024-04-25T01:40:58.784384+0000","last_undegraded":"2024-04-25T01:40:58.784384+0000","last_fullsized":"2024-04-25T01:40:58.784384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24195,"num_write_kb":281461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"26'5562","reported_seq":5584,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.298700+0000","last_change":"2024-04-25T01:26:02.411784+0000","last_active":"2024-04-25T01:41:09.298700+0000","last_peered":"2024-04-25T01:41:09.298700+0000","last_clean":"2024-04-25T01:41:09.298700+0000","last_became_active":"2024-04-25T01:26:02.411478+0000","last_became_peered":"2024-04-25T01:26:02.411478+0000","last_unstale":"2024-04-25T01:41:09.298700+0000","last_undegraded":"2024-04-25T01:41:09.298700+0000","last_fullsized":"2024-04-25T01:41:09.298700+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5562,"log_dups_size":0,"ondisk_log_size":5562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:20.194816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27685,"num_write_kb":352571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"26'5001","reported_seq":5032,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.762988+0000","last_change":"2024-04-25T01:25:53.363829+0000","last_active":"2024-04-25T01:40:58.762988+0000","last_peered":"2024-04-25T01:40:58.762988+0000","last_clean":"2024-04-25T01:40:58.762988+0000","last_became_active":"2024-04-25T01:25:53.363525+0000","last_became_peered":"2024-04-25T01:25:53.363525+0000","last_unstale":"2024-04-25T01:40:58.762988+0000","last_undegraded":"2024-04-25T01:40:58.762988+0000","last_fullsized":"2024-04-25T01:40:58.762988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5001,"log_dups_size":0,"ondisk_log_size":5001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25760,"num_write_kb":312110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"26'5076","reported_seq":5098,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.350154+0000","last_change":"2024-04-25T01:26:02.421208+0000","last_active":"2024-04-25T01:41:09.350154+0000","last_peered":"2024-04-25T01:41:09.350154+0000","last_clean":"2024-04-25T01:41:09.350154+0000","last_became_active":"2024-04-25T01:26:02.418796+0000","last_became_peered":"2024-04-25T01:26:02.418796+0000","last_unstale":"2024-04-25T01:41:09.350154+0000","last_undegraded":"2024-04-25T01:41:09.350154+0000","last_fullsized":"2024-04-25T01:41:09.350154+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5076,"log_dups_size":0,"ondisk_log_size":5076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25596,"num_write_kb":323780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"26'6025","reported_seq":6056,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.748285+0000","last_change":"2024-04-25T01:25:53.365057+0000","last_active":"2024-04-25T01:40:58.748285+0000","last_peered":"2024-04-25T01:40:58.748285+0000","last_clean":"2024-04-25T01:40:58.748285+0000","last_became_active":"2024-04-25T01:25:53.364793+0000","last_became_peered":"2024-04-25T01:25:53.364793+0000","last_unstale":"2024-04-25T01:40:58.748285+0000","last_undegraded":"2024-04-25T01:40:58.748285+0000","last_fullsized":"2024-04-25T01:40:58.748285+0000","mapping_epoch":18,"log_start":"26'200","ondisk_log_start":"26'200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5825,"log_dups_size":0,"ondisk_log_size":5825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31759,"num_write_kb":385508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"26'4493","reported_seq":4515,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.263776+0000","last_change":"2024-04-25T01:26:02.417606+0000","last_active":"2024-04-25T01:41:09.263776+0000","last_peered":"2024-04-25T01:41:09.263776+0000","last_clean":"2024-04-25T01:41:09.263776+0000","last_became_active":"2024-04-25T01:26:02.417350+0000","last_became_peered":"2024-04-25T01:26:02.417350+0000","last_unstale":"2024-04-25T01:41:09.263776+0000","last_undegraded":"2024-04-25T01:41:09.263776+0000","last_fullsized":"2024-04-25T01:41:09.263776+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4493,"log_dups_size":0,"ondisk_log_size":4493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23464,"num_write_kb":290699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"26'4350","reported_seq":4381,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.880219+0000","last_change":"2024-04-25T01:25:53.367762+0000","last_active":"2024-04-25T01:40:58.880219+0000","last_peered":"2024-04-25T01:40:58.880219+0000","last_clean":"2024-04-25T01:40:58.880219+0000","last_became_active":"2024-04-25T01:25:53.367383+0000","last_became_peered":"2024-04-25T01:25:53.367383+0000","last_unstale":"2024-04-25T01:40:58.880219+0000","last_undegraded":"2024-04-25T01:40:58.880219+0000","last_fullsized":"2024-04-25T01:40:58.880219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4350,"log_dups_size":0,"ondisk_log_size":4350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:05.589744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23070,"num_write_kb":268669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1533236,"num_write_kb":19254592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":294168,"ondisk_log_size":294168,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":17605243,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":39,"num_read_kb":112,"num_write":4172,"num_write_kb":18793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35274752,"data_stored":35210486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3499,"ondisk_log_size":3499,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1596536,"num_write_kb":19509893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305948,"ondisk_log_size":305948,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":17847589,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":41,"num_read_kb":115,"num_write":4224,"num_write_kb":19059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35758080,"data_stored":35695178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3536,"ondisk_log_size":3536,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575039,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1217764,"kb_used_data":13148,"kb_used_omap":1,"kb_used_meta":1204606,"kb_avail":93154076,"statfs":{"total":96636764160,"available":95389773824,"internally_reserved":0,"allocated":13463552,"data_stored":13287019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":1233517002},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575037,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":780520,"kb_used_data":9440,"kb_used_omap":1,"kb_used_meta":771070,"kb_avail":93591320,"statfs":{"total":96636764160,"available":95837511680,"internally_reserved":0,"allocated":9666560,"data_stored":9483786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":789576134},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575038,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1059708,"kb_used_data":4468,"kb_used_omap":1,"kb_used_meta":1055230,"kb_avail":93312132,"statfs":{"total":96636764160,"available":95551623168,"internally_reserved":0,"allocated":4575232,"data_stored":4396213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1080555973},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575038,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1089176,"kb_used_data":10256,"kb_used_omap":1,"kb_used_meta":1078910,"kb_avail":93282664,"statfs":{"total":96636764160,"available":95521447936,"internally_reserved":0,"allocated":10502144,"data_stored":10309285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1104804293},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575038,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1161176,"kb_used_data":17488,"kb_used_omap":1,"kb_used_meta":1143678,"kb_avail":93210664,"statfs":{"total":96636764160,"available":95447719936,"internally_reserved":0,"allocated":17907712,"data_stored":17718206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1171126727},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575038,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":729908,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":729534,"kb_avail":93641932,"statfs":{"total":96636764160,"available":95889338368,"internally_reserved":0,"allocated":372736,"data_stored":201217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":747043269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575038,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":888620,"kb_used_data":8484,"kb_used_omap":1,"kb_used_meta":880126,"kb_avail":93483220,"statfs":{"total":96636764160,"available":95726817280,"internally_reserved":0,"allocated":8687616,"data_stored":8502529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":901249477},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575037,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":919656,"kb_used_data":9056,"kb_used_omap":1,"kb_used_meta":910590,"kb_avail":93452184,"statfs":{"total":96636764160,"available":95695036416,"internally_reserved":0,"allocated":9273344,"data_stored":9092681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":932444613},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":88080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8486912,"data_stored":8477360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5189632,"data_stored":5175253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5185536,"data_stored":5175799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8389302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9142272,"data_stored":9127779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5033984,"data_stored":5020965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4292608,"data_stored":4283056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:41:16.364 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T01:41:16.364 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:41:16.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.507+0000 7f1301010640 1 Processor -- start 2024-04-25T01:41:16.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.507+0000 7f1301010640 1 -- start start 2024-04-25T01:41:16.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f1301010640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12fc072d80 0x7f12fc1536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f1301010640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12fc06e0f0 con 0x7f12fc071db0 2024-04-25T01:41:16.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f1301010640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12fc0c1f00 con 0x7f12fc0c1a80 2024-04-25T01:41:16.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f1301010640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f12fc072950 con 0x7f12fc072d80 2024-04-25T01:41:16.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fb59e640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f12fc071db0 0x7f12fc072240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:39360/0 (socket says 172.21.15.112:39360) 2024-04-25T01:41:16.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fb59e640 1 -- 172.21.15.112:0/3897076173 learned_addr learned my addr 172.21.15.112:0/3897076173 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:16.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fad9d640 1 --2- 172.21.15.112:0/3897076173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12fc072d80 0x7f12fc1536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fa59c640 1 -- 172.21.15.112:0/3897076173 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2092583936 0 0) 0x7f12f4001490 con 0x7f12fc071db0 2024-04-25T01:41:16.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fa59c640 1 -- 172.21.15.112:0/3897076173 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12e8003520 con 0x7f12fc071db0 2024-04-25T01:41:16.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fa59c640 1 -- 172.21.15.112:0/3897076173 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1594484205 0 0) 0x7f12e4001110 con 0x7f12fc0c1a80 2024-04-25T01:41:16.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fa59c640 1 -- 172.21.15.112:0/3897076173 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12f4001490 con 0x7f12fc0c1a80 2024-04-25T01:41:16.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fa59c640 1 -- 172.21.15.112:0/3897076173 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2386594151 0 0) 0x7f12e8003520 con 0x7f12fc071db0 2024-04-25T01:41:16.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fa59c640 1 -- 172.21.15.112:0/3897076173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12fc072d80 msgr2=0x7f12fc1536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:16.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fa59c640 1 --2- 172.21.15.112:0/3897076173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12fc072d80 0x7f12fc1536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fa59c640 1 -- 172.21.15.112:0/3897076173 >> v1:172.21.15.187:6789/0 conn(0x7f12fc0c1a80 legacy=0x7f12fc14d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:16.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fa59c640 1 -- 172.21.15.112:0/3897076173 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12fc0c2180 con 0x7f12fc071db0 2024-04-25T01:41:16.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fa59c640 1 -- 172.21.15.112:0/3897076173 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f12f4001bc0 con 0x7f12fc071db0 2024-04-25T01:41:16.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fa59c640 1 -- 172.21.15.112:0/3897076173 >> v1:172.21.15.112:6789/0 conn(0x7f12fc071db0 legacy=0x7f12fc072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:16.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fa59c640 1 --2- 172.21.15.112:0/3897076173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12e8003b70 0x7f12e8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fa59c640 1 --2- 172.21.15.112:0/3897076173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12e8004800 0x7f12e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fa59c640 1 --2- 172.21.15.112:0/3897076173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12fc072d80 0x7f12e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fa59c640 1 -- 172.21.15.112:0/3897076173 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc072950 con 0x7f12e8003b70 2024-04-25T01:41:16.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fa59c640 1 -- 172.21.15.112:0/3897076173 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f12e800ba70 con 0x7f12e8004800 2024-04-25T01:41:16.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fa59c640 1 -- 172.21.15.112:0/3897076173 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f12e800bc70 con 0x7f12fc072d80 2024-04-25T01:41:16.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.508+0000 7f12fad9d640 1 --2- 172.21.15.112:0/3897076173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12e8003b70 0x7f12e8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.509+0000 7f12f3fff640 1 --2- 172.21.15.112:0/3897076173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12e8004800 0x7f12e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.509+0000 7f12fb59e640 1 --2- 172.21.15.112:0/3897076173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12fc072d80 0x7f12e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.509+0000 7f12fad9d640 1 -- 172.21.15.112:0/3897076173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12fc072d80 msgr2=0x7f12e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:16.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.509+0000 7f12fad9d640 1 --2- 172.21.15.112:0/3897076173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12fc072d80 0x7f12e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.509+0000 7f12fad9d640 1 -- 172.21.15.112:0/3897076173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12e8004800 msgr2=0x7f12e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:16.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.509+0000 7f12fad9d640 1 --2- 172.21.15.112:0/3897076173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12e8004800 0x7f12e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.509+0000 7f12fad9d640 1 -- 172.21.15.112:0/3897076173 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e8003520 con 0x7f12e8003b70 2024-04-25T01:41:16.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.509+0000 7f12fad9d640 1 --2- 172.21.15.112:0/3897076173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12e8003b70 0x7f12e8003f70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009ac0 tx=0x7f12e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728d1f78d3e7c1b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:16.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.510+0000 7f12fa59c640 1 -- 172.21.15.112:0/3897076173 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec036070 con 0x7f12e8003b70 2024-04-25T01:41:16.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.510+0000 7f12fa59c640 1 -- 172.21.15.112:0/3897076173 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12ec031400 con 0x7f12e8003b70 2024-04-25T01:41:16.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.510+0000 7f12fa59c640 1 -- 172.21.15.112:0/3897076173 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec009690 con 0x7f12e8003b70 2024-04-25T01:41:16.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.510+0000 7f1301010640 1 -- 172.21.15.112:0/3897076173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12e8003b70 msgr2=0x7f12e8003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:16.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.510+0000 7f1301010640 1 --2- 172.21.15.112:0/3897076173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12e8003b70 0x7f12e8003f70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009ac0 tx=0x7f12e8005830 comp rx=0 tx=0).stop 2024-04-25T01:41:16.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.510+0000 7f12fb59e640 1 -- 172.21.15.112:0/3897076173 reap_dead start 2024-04-25T01:41:16.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.510+0000 7f1301010640 1 -- 172.21.15.112:0/3897076173 shutdown_connections 2024-04-25T01:41:16.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.510+0000 7f1301010640 1 -- 172.21.15.112:0/3897076173 >> 172.21.15.112:0/3897076173 conn(0x7f12fc06b160 msgr2=0x7f12fc06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:16.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.510+0000 7f1301010640 1 -- 172.21.15.112:0/3897076173 shutdown_connections 2024-04-25T01:41:16.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.510+0000 7f1301010640 1 -- 172.21.15.112:0/3897076173 wait complete. 2024-04-25T01:41:16.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.511+0000 7f1301010640 1 Processor -- start 2024-04-25T01:41:16.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.511+0000 7f1301010640 1 -- start start 2024-04-25T01:41:16.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.512+0000 7f1301010640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12e8003b70 0x7f12fc1e9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.512+0000 7f1301010640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12fc1521b0 0x7f12fc155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.512+0000 7f1301010640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12fc142b10 0x7f12fc142fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.512+0000 7f1301010640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc0c2a20 con 0x7f12fc142b10 2024-04-25T01:41:16.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.512+0000 7f1301010640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc0c2740 con 0x7f12fc1521b0 2024-04-25T01:41:16.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.512+0000 7f1301010640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f12fc071890 con 0x7f12e8003b70 2024-04-25T01:41:16.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.512+0000 7f12fb59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12fc142b10 0x7f12fc142fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.512+0000 7f12fb59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12fc142b10 0x7f12fc142fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56472/0 (socket says 172.21.15.112:56472) 2024-04-25T01:41:16.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.512+0000 7f12fb59e640 1 -- 172.21.15.112:0/2686476670 learned_addr learned my addr 172.21.15.112:0/2686476670 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:16.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.512+0000 7f12fad9d640 1 --2- 172.21.15.112:0/2686476670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12e8003b70 0x7f12fc1e9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.512+0000 7f12f3fff640 1 --2- 172.21.15.112:0/2686476670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12fc1521b0 0x7f12fc155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.512+0000 7f12fb59e640 1 -- 172.21.15.112:0/2686476670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12e8003b70 msgr2=0x7f12fc1e9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:16.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.512+0000 7f12fb59e640 1 --2- 172.21.15.112:0/2686476670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12e8003b70 0x7f12fc1e9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.512+0000 7f12fb59e640 1 -- 172.21.15.112:0/2686476670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12fc1521b0 msgr2=0x7f12fc155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:16.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.512+0000 7f12fb59e640 1 --2- 172.21.15.112:0/2686476670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12fc1521b0 0x7f12fc155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.512+0000 7f12fb59e640 1 -- 172.21.15.112:0/2686476670 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12fc1e0390 con 0x7f12fc142b10 2024-04-25T01:41:16.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.512+0000 7f12fad9d640 1 --2- 172.21.15.112:0/2686476670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12e8003b70 0x7f12fc1e9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:41:16.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.513+0000 7f12fb59e640 1 --2- 172.21.15.112:0/2686476670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12fc142b10 0x7f12fc142fc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0c4c40 tx=0x7f12f4007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:16.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.513+0000 7f12f37fe640 1 -- 172.21.15.112:0/2686476670 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4007a30 con 0x7f12fc142b10 2024-04-25T01:41:16.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.513+0000 7f12f37fe640 1 -- 172.21.15.112:0/2686476670 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12f4018070 con 0x7f12fc142b10 2024-04-25T01:41:16.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.513+0000 7f12f37fe640 1 -- 172.21.15.112:0/2686476670 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4013630 con 0x7f12fc142b10 2024-04-25T01:41:16.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.513+0000 7f1301010640 1 -- 172.21.15.112:0/2686476670 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12fc1e0670 con 0x7f12fc142b10 2024-04-25T01:41:16.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.513+0000 7f1301010640 1 -- 172.21.15.112:0/2686476670 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12fc156100 con 0x7f12fc142b10 2024-04-25T01:41:16.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.514+0000 7f1301010640 1 -- 172.21.15.112:0/2686476670 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12fc0682a0 con 0x7f12fc142b10 2024-04-25T01:41:16.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.514+0000 7f12f37fe640 1 -- 172.21.15.112:0/2686476670 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f12f40137d0 con 0x7f12fc142b10 2024-04-25T01:41:16.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.515+0000 7f12f37fe640 1 --2- 172.21.15.112:0/2686476670 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f12dc06be60 0x7f12dc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.515+0000 7f12f37fe640 1 -- 172.21.15.112:0/2686476670 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f12f408ba30 con 0x7f12fc142b10 2024-04-25T01:41:16.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.515+0000 7f12fad9d640 1 --2- 172.21.15.112:0/2686476670 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f12dc06be60 0x7f12dc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.515+0000 7f12fad9d640 1 --2- 172.21.15.112:0/2686476670 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f12dc06be60 0x7f12dc06e320 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f12ec033000 tx=0x7f12ec008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:16.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.519+0000 7f12f37fe640 1 -- 172.21.15.112:0/2686476670 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f12fc06e0f0 con 0x7f12fc142b10 2024-04-25T01:41:16.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.689+0000 7f1301010640 1 -- 172.21.15.112:0/2686476670 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f12fc153f40 con 0x7f12dc06be60 2024-04-25T01:41:16.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.700+0000 7f12f37fe640 1 -- 172.21.15.112:0/2686476670 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+493703 (secure 0 0 0) 0x7f12ec037040 con 0x7f12dc06be60 2024-04-25T01:41:16.689 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:41:16.698 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T01:41:16.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.704+0000 7f1301010640 1 -- 172.21.15.112:0/2686476670 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f12dc06be60 msgr2=0x7f12dc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:16.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.704+0000 7f1301010640 1 --2- 172.21.15.112:0/2686476670 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f12dc06be60 0x7f12dc06e320 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f12ec033000 tx=0x7f12ec008030 comp rx=0 tx=0).stop 2024-04-25T01:41:16.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.704+0000 7f1301010640 1 -- 172.21.15.112:0/2686476670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12fc142b10 msgr2=0x7f12fc142fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:16.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.704+0000 7f1301010640 1 --2- 172.21.15.112:0/2686476670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12fc142b10 0x7f12fc142fc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0c4c40 tx=0x7f12f4007430 comp rx=0 tx=0).stop 2024-04-25T01:41:16.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.704+0000 7f1301010640 1 -- 172.21.15.112:0/2686476670 shutdown_connections 2024-04-25T01:41:16.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.704+0000 7f1301010640 1 --2- 172.21.15.112:0/2686476670 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f12dc06be60 0x7f12dc06e320 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.704+0000 7f1301010640 1 --2- 172.21.15.112:0/2686476670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12fc142b10 0x7f12fc142fc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.704+0000 7f1301010640 1 --2- 172.21.15.112:0/2686476670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12fc1521b0 0x7f12fc155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.704+0000 7f1301010640 1 --2- 172.21.15.112:0/2686476670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12e8003b70 0x7f12fc1e9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.704+0000 7f1301010640 1 -- 172.21.15.112:0/2686476670 >> 172.21.15.112:0/2686476670 conn(0x7f12fc06b160 msgr2=0x7f12fc147ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:16.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.704+0000 7f1301010640 1 -- 172.21.15.112:0/2686476670 shutdown_connections 2024-04-25T01:41:16.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.704+0000 7f1301010640 1 -- 172.21.15.112:0/2686476670 wait complete. 2024-04-25T01:41:16.703 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":486,"stamp":"2024-04-25T01:41:14.915187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":36043200,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":196,"num_read_kb":327,"num_write":3138327,"num_write_kb":38804753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":607268,"ondisk_log_size":607268,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":7846528,"kb_used_data":72704,"kb_used_omap":12,"kb_used_meta":7773747,"kb_avail":747128192,"statfs":{"total":773094113280,"available":765059268608,"internally_reserved":0,"allocated":74448896,"data_stored":72990936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":7960317488},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":70,"apply_latency_ms":70,"commit_latency_ns":70000000,"apply_latency_ns":70000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003261"},"pg_stats":[{"pgid":"5.18","version":"26'4099","reported_seq":4121,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.267400+0000","last_change":"2024-04-25T01:26:02.412272+0000","last_active":"2024-04-25T01:41:09.267400+0000","last_peered":"2024-04-25T01:41:09.267400+0000","last_clean":"2024-04-25T01:41:09.267400+0000","last_became_active":"2024-04-25T01:26:02.411176+0000","last_became_peered":"2024-04-25T01:26:02.411176+0000","last_unstale":"2024-04-25T01:41:09.267400+0000","last_undegraded":"2024-04-25T01:41:09.267400+0000","last_fullsized":"2024-04-25T01:41:09.267400+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4099,"log_dups_size":0,"ondisk_log_size":4099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:49.498275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21931,"num_write_kb":288545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"26'658","reported_seq":691,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:44.444563+0000","last_change":"2024-04-25T01:25:52.353978+0000","last_active":"2024-04-25T01:40:44.444563+0000","last_peered":"2024-04-25T01:40:44.444563+0000","last_clean":"2024-04-25T01:40:44.444563+0000","last_became_active":"2024-04-25T01:25:52.353649+0000","last_became_peered":"2024-04-25T01:25:52.353649+0000","last_unstale":"2024-04-25T01:40:44.444563+0000","last_undegraded":"2024-04-25T01:40:44.444563+0000","last_fullsized":"2024-04-25T01:40:44.444563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":4468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"26'5333","reported_seq":5364,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.730049+0000","last_change":"2024-04-25T01:25:53.364257+0000","last_active":"2024-04-25T01:40:58.730049+0000","last_peered":"2024-04-25T01:40:58.730049+0000","last_clean":"2024-04-25T01:40:58.730049+0000","last_became_active":"2024-04-25T01:25:53.363409+0000","last_became_peered":"2024-04-25T01:25:53.363409+0000","last_unstale":"2024-04-25T01:40:58.730049+0000","last_undegraded":"2024-04-25T01:40:58.730049+0000","last_fullsized":"2024-04-25T01:40:58.730049+0000","mapping_epoch":18,"log_start":"26'100","ondisk_log_start":"26'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28405,"num_write_kb":331018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"26'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:31.329698+0000","last_change":"2024-04-25T01:26:01.407136+0000","last_active":"2024-04-25T01:40:31.329698+0000","last_peered":"2024-04-25T01:40:31.329698+0000","last_clean":"2024-04-25T01:40:31.329698+0000","last_became_active":"2024-04-25T01:26:01.406855+0000","last_became_peered":"2024-04-25T01:26:01.406855+0000","last_unstale":"2024-04-25T01:40:31.329698+0000","last_undegraded":"2024-04-25T01:40:31.329698+0000","last_fullsized":"2024-04-25T01:40:31.329698+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:02.180826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"26'5328","reported_seq":5350,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.321101+0000","last_change":"2024-04-25T01:26:02.425367+0000","last_active":"2024-04-25T01:41:09.321101+0000","last_peered":"2024-04-25T01:41:09.321101+0000","last_clean":"2024-04-25T01:41:09.321101+0000","last_became_active":"2024-04-25T01:26:02.425071+0000","last_became_peered":"2024-04-25T01:26:02.425071+0000","last_unstale":"2024-04-25T01:41:09.321101+0000","last_undegraded":"2024-04-25T01:41:09.321101+0000","last_fullsized":"2024-04-25T01:41:09.321101+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5328,"log_dups_size":0,"ondisk_log_size":5328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:47.810943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27477,"num_write_kb":347746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"26'8","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.218358+0000","last_change":"2024-04-25T01:25:52.349185+0000","last_active":"2024-04-25T01:40:49.218358+0000","last_peered":"2024-04-25T01:40:49.218358+0000","last_clean":"2024-04-25T01:40:49.218358+0000","last_became_active":"2024-04-25T01:25:52.348931+0000","last_became_peered":"2024-04-25T01:25:52.348931+0000","last_unstale":"2024-04-25T01:40:49.218358+0000","last_undegraded":"2024-04-25T01:40:49.218358+0000","last_fullsized":"2024-04-25T01:40:49.218358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"26'5837","reported_seq":5868,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.789129+0000","last_change":"2024-04-25T01:25:53.374150+0000","last_active":"2024-04-25T01:40:58.789129+0000","last_peered":"2024-04-25T01:40:58.789129+0000","last_clean":"2024-04-25T01:40:58.789129+0000","last_became_active":"2024-04-25T01:25:53.373755+0000","last_became_peered":"2024-04-25T01:25:53.373755+0000","last_unstale":"2024-04-25T01:40:58.789129+0000","last_undegraded":"2024-04-25T01:40:58.789129+0000","last_fullsized":"2024-04-25T01:40:58.789129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5837,"log_dups_size":0,"ondisk_log_size":5837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30942,"num_write_kb":369940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"26'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:32.791998+0000","last_change":"2024-04-25T01:26:01.406334+0000","last_active":"2024-04-25T01:40:32.791998+0000","last_peered":"2024-04-25T01:40:32.791998+0000","last_clean":"2024-04-25T01:40:32.791998+0000","last_became_active":"2024-04-25T01:26:01.405994+0000","last_became_peered":"2024-04-25T01:26:01.405994+0000","last_unstale":"2024-04-25T01:40:32.791998+0000","last_undegraded":"2024-04-25T01:40:32.791998+0000","last_fullsized":"2024-04-25T01:40:32.791998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:43.953437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"26'4188","reported_seq":4210,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.336545+0000","last_change":"2024-04-25T01:26:02.427695+0000","last_active":"2024-04-25T01:41:09.336545+0000","last_peered":"2024-04-25T01:41:09.336545+0000","last_clean":"2024-04-25T01:41:09.336545+0000","last_became_active":"2024-04-25T01:26:02.427443+0000","last_became_peered":"2024-04-25T01:26:02.427443+0000","last_unstale":"2024-04-25T01:41:09.336545+0000","last_undegraded":"2024-04-25T01:41:09.336545+0000","last_fullsized":"2024-04-25T01:41:09.336545+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4188,"log_dups_size":0,"ondisk_log_size":4188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20869,"num_write_kb":259707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"26'11","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.299916+0000","last_change":"2024-04-25T01:25:52.356403+0000","last_active":"2024-04-25T01:40:49.299916+0000","last_peered":"2024-04-25T01:40:49.299916+0000","last_clean":"2024-04-25T01:40:49.299916+0000","last_became_active":"2024-04-25T01:25:52.355822+0000","last_became_peered":"2024-04-25T01:25:52.355822+0000","last_unstale":"2024-04-25T01:40:49.299916+0000","last_undegraded":"2024-04-25T01:40:49.299916+0000","last_fullsized":"2024-04-25T01:40:49.299916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"26'3747","reported_seq":3777,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.846633+0000","last_change":"2024-04-25T01:25:53.370061+0000","last_active":"2024-04-25T01:40:58.846633+0000","last_peered":"2024-04-25T01:40:58.846633+0000","last_clean":"2024-04-25T01:40:58.846633+0000","last_became_active":"2024-04-25T01:25:53.369721+0000","last_became_peered":"2024-04-25T01:25:53.369721+0000","last_unstale":"2024-04-25T01:40:58.846633+0000","last_undegraded":"2024-04-25T01:40:58.846633+0000","last_fullsized":"2024-04-25T01:40:58.846633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3747,"log_dups_size":0,"ondisk_log_size":3747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18978,"num_write_kb":223593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"26'627","reported_seq":651,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.103722+0000","last_change":"2024-04-25T01:26:01.408179+0000","last_active":"2024-04-25T01:40:37.103722+0000","last_peered":"2024-04-25T01:40:37.103722+0000","last_clean":"2024-04-25T01:40:37.103722+0000","last_became_active":"2024-04-25T01:26:01.407909+0000","last_became_peered":"2024-04-25T01:26:01.407909+0000","last_unstale":"2024-04-25T01:40:37.103722+0000","last_undegraded":"2024-04-25T01:40:37.103722+0000","last_fullsized":"2024-04-25T01:40:37.103722+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:48.865940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":625,"num_write_kb":4235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"26'3441","reported_seq":3463,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.298394+0000","last_change":"2024-04-25T01:26:02.424615+0000","last_active":"2024-04-25T01:41:09.298394+0000","last_peered":"2024-04-25T01:41:09.298394+0000","last_clean":"2024-04-25T01:41:09.298394+0000","last_became_active":"2024-04-25T01:26:02.424325+0000","last_became_peered":"2024-04-25T01:26:02.424325+0000","last_unstale":"2024-04-25T01:41:09.298394+0000","last_undegraded":"2024-04-25T01:41:09.298394+0000","last_fullsized":"2024-04-25T01:41:09.298394+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3441,"log_dups_size":0,"ondisk_log_size":3441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17582,"num_write_kb":230191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"26'3","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.256836+0000","last_change":"2024-04-25T01:25:52.359139+0000","last_active":"2024-04-25T01:40:49.256836+0000","last_peered":"2024-04-25T01:40:49.256836+0000","last_clean":"2024-04-25T01:40:49.256836+0000","last_became_active":"2024-04-25T01:25:52.358841+0000","last_became_peered":"2024-04-25T01:25:52.358841+0000","last_unstale":"2024-04-25T01:40:49.256836+0000","last_undegraded":"2024-04-25T01:40:49.256836+0000","last_fullsized":"2024-04-25T01:40:49.256836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"26'5828","reported_seq":5858,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.878112+0000","last_change":"2024-04-25T01:25:53.368739+0000","last_active":"2024-04-25T01:40:58.878112+0000","last_peered":"2024-04-25T01:40:58.878112+0000","last_clean":"2024-04-25T01:40:58.878112+0000","last_became_active":"2024-04-25T01:25:53.368470+0000","last_became_peered":"2024-04-25T01:25:53.368470+0000","last_unstale":"2024-04-25T01:40:58.878112+0000","last_undegraded":"2024-04-25T01:40:58.878112+0000","last_fullsized":"2024-04-25T01:40:58.878112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5828,"log_dups_size":0,"ondisk_log_size":5828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:29.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30463,"num_write_kb":367857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"26'65","reported_seq":89,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.150373+0000","last_change":"2024-04-25T01:26:01.407969+0000","last_active":"2024-04-25T01:41:09.150373+0000","last_peered":"2024-04-25T01:41:09.150373+0000","last_clean":"2024-04-25T01:41:09.150373+0000","last_became_active":"2024-04-25T01:26:01.407655+0000","last_became_peered":"2024-04-25T01:26:01.407655+0000","last_unstale":"2024-04-25T01:41:09.150373+0000","last_undegraded":"2024-04-25T01:41:09.150373+0000","last_fullsized":"2024-04-25T01:41:09.150373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:58.485864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":738477,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61,"num_write_kb":764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"26'4794","reported_seq":4816,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.329809+0000","last_change":"2024-04-25T01:26:02.417776+0000","last_active":"2024-04-25T01:41:09.329809+0000","last_peered":"2024-04-25T01:41:09.329809+0000","last_clean":"2024-04-25T01:41:09.329809+0000","last_became_active":"2024-04-25T01:26:02.417524+0000","last_became_peered":"2024-04-25T01:26:02.417524+0000","last_unstale":"2024-04-25T01:41:09.329809+0000","last_undegraded":"2024-04-25T01:41:09.329809+0000","last_fullsized":"2024-04-25T01:41:09.329809+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4794,"log_dups_size":0,"ondisk_log_size":4794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.688469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24624,"num_write_kb":311668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"26'4532","reported_seq":4563,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.725104+0000","last_change":"2024-04-25T01:25:53.372385+0000","last_active":"2024-04-25T01:40:58.725104+0000","last_peered":"2024-04-25T01:40:58.725104+0000","last_clean":"2024-04-25T01:40:58.725104+0000","last_became_active":"2024-04-25T01:25:53.372064+0000","last_became_peered":"2024-04-25T01:25:53.372064+0000","last_unstale":"2024-04-25T01:40:58.725104+0000","last_undegraded":"2024-04-25T01:40:58.725104+0000","last_fullsized":"2024-04-25T01:40:58.725104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4532,"log_dups_size":0,"ondisk_log_size":4532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23642,"num_write_kb":279226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"26'1100","reported_seq":1132,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.233609+0000","last_change":"2024-04-25T01:25:52.358616+0000","last_active":"2024-04-25T01:40:49.233609+0000","last_peered":"2024-04-25T01:40:49.233609+0000","last_clean":"2024-04-25T01:40:49.233609+0000","last_became_active":"2024-04-25T01:25:52.358318+0000","last_became_peered":"2024-04-25T01:25:52.358318+0000","last_unstale":"2024-04-25T01:40:49.233609+0000","last_undegraded":"2024-04-25T01:40:49.233609+0000","last_fullsized":"2024-04-25T01:40:49.233609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1100,"log_dups_size":0,"ondisk_log_size":1100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1091,"num_write_kb":4419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"26'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:39:31.750022+0000","last_change":"2024-04-25T01:26:01.400599+0000","last_active":"2024-04-25T01:39:31.750022+0000","last_peered":"2024-04-25T01:39:31.750022+0000","last_clean":"2024-04-25T01:39:31.750022+0000","last_became_active":"2024-04-25T01:26:01.400266+0000","last_became_peered":"2024-04-25T01:26:01.400266+0000","last_unstale":"2024-04-25T01:39:31.750022+0000","last_undegraded":"2024-04-25T01:39:31.750022+0000","last_fullsized":"2024-04-25T01:39:31.750022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"26'4554","reported_seq":4576,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.270037+0000","last_change":"2024-04-25T01:26:02.420269+0000","last_active":"2024-04-25T01:41:09.270037+0000","last_peered":"2024-04-25T01:41:09.270037+0000","last_clean":"2024-04-25T01:41:09.270037+0000","last_became_active":"2024-04-25T01:26:02.420001+0000","last_became_peered":"2024-04-25T01:26:02.420001+0000","last_unstale":"2024-04-25T01:41:09.270037+0000","last_undegraded":"2024-04-25T01:41:09.270037+0000","last_fullsized":"2024-04-25T01:41:09.270037+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4554,"log_dups_size":0,"ondisk_log_size":4554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:38.149069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23431,"num_write_kb":298735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"26'3793","reported_seq":3823,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.792694+0000","last_change":"2024-04-25T01:25:53.361074+0000","last_active":"2024-04-25T01:40:58.792694+0000","last_peered":"2024-04-25T01:40:58.792694+0000","last_clean":"2024-04-25T01:40:58.792694+0000","last_became_active":"2024-04-25T01:25:53.360802+0000","last_became_peered":"2024-04-25T01:25:53.360802+0000","last_unstale":"2024-04-25T01:40:58.792694+0000","last_undegraded":"2024-04-25T01:40:58.792694+0000","last_fullsized":"2024-04-25T01:40:58.792694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3793,"log_dups_size":0,"ondisk_log_size":3793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19299,"num_write_kb":249413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"26'3","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.225267+0000","last_change":"2024-04-25T01:25:52.348841+0000","last_active":"2024-04-25T01:40:49.225267+0000","last_peered":"2024-04-25T01:40:49.225267+0000","last_clean":"2024-04-25T01:40:49.225267+0000","last_became_active":"2024-04-25T01:25:52.348550+0000","last_became_peered":"2024-04-25T01:25:52.348550+0000","last_unstale":"2024-04-25T01:40:49.225267+0000","last_undegraded":"2024-04-25T01:40:49.225267+0000","last_fullsized":"2024-04-25T01:40:49.225267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"26'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:36.992419+0000","last_change":"2024-04-25T01:26:01.408529+0000","last_active":"2024-04-25T01:40:36.992419+0000","last_peered":"2024-04-25T01:40:36.992419+0000","last_clean":"2024-04-25T01:40:36.992419+0000","last_became_active":"2024-04-25T01:26:01.407738+0000","last_became_peered":"2024-04-25T01:26:01.407738+0000","last_unstale":"2024-04-25T01:40:36.992419+0000","last_undegraded":"2024-04-25T01:40:36.992419+0000","last_fullsized":"2024-04-25T01:40:36.992419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:37.142255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"26'3863","reported_seq":3885,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.299717+0000","last_change":"2024-04-25T01:26:02.417731+0000","last_active":"2024-04-25T01:41:09.299717+0000","last_peered":"2024-04-25T01:41:09.299717+0000","last_clean":"2024-04-25T01:41:09.299717+0000","last_became_active":"2024-04-25T01:26:02.417438+0000","last_became_peered":"2024-04-25T01:26:02.417438+0000","last_unstale":"2024-04-25T01:41:09.299717+0000","last_undegraded":"2024-04-25T01:41:09.299717+0000","last_fullsized":"2024-04-25T01:41:09.299717+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3863,"log_dups_size":0,"ondisk_log_size":3863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19753,"num_write_kb":275789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"26'4400","reported_seq":4431,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.812159+0000","last_change":"2024-04-25T01:25:53.361790+0000","last_active":"2024-04-25T01:40:58.812159+0000","last_peered":"2024-04-25T01:40:58.812159+0000","last_clean":"2024-04-25T01:40:58.812159+0000","last_became_active":"2024-04-25T01:25:53.361542+0000","last_became_peered":"2024-04-25T01:25:53.361542+0000","last_unstale":"2024-04-25T01:40:58.812159+0000","last_undegraded":"2024-04-25T01:40:58.812159+0000","last_fullsized":"2024-04-25T01:40:58.812159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4400,"log_dups_size":0,"ondisk_log_size":4400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22906,"num_write_kb":296647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"26'3","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.235744+0000","last_change":"2024-04-25T01:25:52.353926+0000","last_active":"2024-04-25T01:40:49.235744+0000","last_peered":"2024-04-25T01:40:49.235744+0000","last_clean":"2024-04-25T01:40:49.235744+0000","last_became_active":"2024-04-25T01:25:52.353656+0000","last_became_peered":"2024-04-25T01:25:52.353656+0000","last_unstale":"2024-04-25T01:40:49.235744+0000","last_undegraded":"2024-04-25T01:40:49.235744+0000","last_fullsized":"2024-04-25T01:40:49.235744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"26'5","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.165411+0000","last_change":"2024-04-25T01:26:01.418556+0000","last_active":"2024-04-25T01:40:37.165411+0000","last_peered":"2024-04-25T01:40:37.165411+0000","last_clean":"2024-04-25T01:40:37.165411+0000","last_became_active":"2024-04-25T01:26:01.418263+0000","last_became_peered":"2024-04-25T01:26:01.418263+0000","last_unstale":"2024-04-25T01:40:37.165411+0000","last_undegraded":"2024-04-25T01:40:37.165411+0000","last_fullsized":"2024-04-25T01:40:37.165411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:28.139183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"26'5570","reported_seq":5592,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.349577+0000","last_change":"2024-04-25T01:26:02.413429+0000","last_active":"2024-04-25T01:41:09.349577+0000","last_peered":"2024-04-25T01:41:09.349577+0000","last_clean":"2024-04-25T01:41:09.349577+0000","last_became_active":"2024-04-25T01:26:02.412632+0000","last_became_peered":"2024-04-25T01:26:02.412632+0000","last_unstale":"2024-04-25T01:41:09.349577+0000","last_undegraded":"2024-04-25T01:41:09.349577+0000","last_fullsized":"2024-04-25T01:41:09.349577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5570,"log_dups_size":0,"ondisk_log_size":5570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:10.049310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27971,"num_write_kb":349705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"26'4715","reported_seq":4746,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.811959+0000","last_change":"2024-04-25T01:25:53.361653+0000","last_active":"2024-04-25T01:40:58.811959+0000","last_peered":"2024-04-25T01:40:58.811959+0000","last_clean":"2024-04-25T01:40:58.811959+0000","last_became_active":"2024-04-25T01:25:53.361265+0000","last_became_peered":"2024-04-25T01:25:53.361265+0000","last_unstale":"2024-04-25T01:40:58.811959+0000","last_undegraded":"2024-04-25T01:40:58.811959+0000","last_fullsized":"2024-04-25T01:40:58.811959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4715,"log_dups_size":0,"ondisk_log_size":4715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23886,"num_write_kb":288267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'3","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.248825+0000","last_change":"2024-04-25T01:25:52.353836+0000","last_active":"2024-04-25T01:40:49.248825+0000","last_peered":"2024-04-25T01:40:49.248825+0000","last_clean":"2024-04-25T01:40:49.248825+0000","last_became_active":"2024-04-25T01:25:52.353552+0000","last_became_peered":"2024-04-25T01:25:52.353552+0000","last_unstale":"2024-04-25T01:40:49.248825+0000","last_undegraded":"2024-04-25T01:40:49.248825+0000","last_fullsized":"2024-04-25T01:40:49.248825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"26'6","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.134073+0000","last_change":"2024-04-25T01:26:01.411323+0000","last_active":"2024-04-25T01:40:37.134073+0000","last_peered":"2024-04-25T01:40:37.134073+0000","last_clean":"2024-04-25T01:40:37.134073+0000","last_became_active":"2024-04-25T01:26:01.410378+0000","last_became_peered":"2024-04-25T01:26:01.410378+0000","last_unstale":"2024-04-25T01:40:37.134073+0000","last_undegraded":"2024-04-25T01:40:37.134073+0000","last_fullsized":"2024-04-25T01:40:37.134073+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"26'4403","reported_seq":4426,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.251745+0000","last_change":"2024-04-25T01:26:02.416646+0000","last_active":"2024-04-25T01:41:09.251745+0000","last_peered":"2024-04-25T01:41:09.251745+0000","last_clean":"2024-04-25T01:41:09.251745+0000","last_became_active":"2024-04-25T01:26:02.416395+0000","last_became_peered":"2024-04-25T01:26:02.416395+0000","last_unstale":"2024-04-25T01:41:09.251745+0000","last_undegraded":"2024-04-25T01:41:09.251745+0000","last_fullsized":"2024-04-25T01:41:09.251745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4403,"log_dups_size":0,"ondisk_log_size":4403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23642,"num_write_kb":287389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"26'3766","reported_seq":3797,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.767490+0000","last_change":"2024-04-25T01:25:53.370111+0000","last_active":"2024-04-25T01:40:58.767490+0000","last_peered":"2024-04-25T01:40:58.767490+0000","last_clean":"2024-04-25T01:40:58.767490+0000","last_became_active":"2024-04-25T01:25:53.369837+0000","last_became_peered":"2024-04-25T01:25:53.369837+0000","last_unstale":"2024-04-25T01:40:58.767490+0000","last_undegraded":"2024-04-25T01:40:58.767490+0000","last_fullsized":"2024-04-25T01:40:58.767490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3766,"log_dups_size":0,"ondisk_log_size":3766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19293,"num_write_kb":249241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"26'5","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.300730+0000","last_change":"2024-04-25T01:25:52.353269+0000","last_active":"2024-04-25T01:40:49.300730+0000","last_peered":"2024-04-25T01:40:49.300730+0000","last_clean":"2024-04-25T01:40:49.300730+0000","last_became_active":"2024-04-25T01:25:52.352991+0000","last_became_peered":"2024-04-25T01:25:52.352991+0000","last_unstale":"2024-04-25T01:40:49.300730+0000","last_undegraded":"2024-04-25T01:40:49.300730+0000","last_fullsized":"2024-04-25T01:40:49.300730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"26'41","reported_seq":105,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:47.845985+0000","last_change":"2024-04-25T01:26:01.410360+0000","last_active":"2024-04-25T01:40:47.845985+0000","last_peered":"2024-04-25T01:40:47.845985+0000","last_clean":"2024-04-25T01:40:47.845985+0000","last_became_active":"2024-04-25T01:26:01.410076+0000","last_became_peered":"2024-04-25T01:26:01.410076+0000","last_unstale":"2024-04-25T01:40:47.845985+0000","last_undegraded":"2024-04-25T01:40:47.845985+0000","last_fullsized":"2024-04-25T01:40:47.845985+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:52.561251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":112,"num_write":39,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"26'4289","reported_seq":4312,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.299549+0000","last_change":"2024-04-25T01:26:02.422182+0000","last_active":"2024-04-25T01:41:09.299549+0000","last_peered":"2024-04-25T01:41:09.299549+0000","last_clean":"2024-04-25T01:41:09.299549+0000","last_became_active":"2024-04-25T01:26:02.421885+0000","last_became_peered":"2024-04-25T01:26:02.421885+0000","last_unstale":"2024-04-25T01:41:09.299549+0000","last_undegraded":"2024-04-25T01:41:09.299549+0000","last_fullsized":"2024-04-25T01:41:09.299549+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4289,"log_dups_size":0,"ondisk_log_size":4289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:03.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22172,"num_write_kb":285532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"26'4466","reported_seq":4497,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.806516+0000","last_change":"2024-04-25T01:25:53.364967+0000","last_active":"2024-04-25T01:40:58.806516+0000","last_peered":"2024-04-25T01:40:58.806516+0000","last_clean":"2024-04-25T01:40:58.806516+0000","last_became_active":"2024-04-25T01:25:53.364694+0000","last_became_peered":"2024-04-25T01:25:53.364694+0000","last_unstale":"2024-04-25T01:40:58.806516+0000","last_undegraded":"2024-04-25T01:40:58.806516+0000","last_fullsized":"2024-04-25T01:40:58.806516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4466,"log_dups_size":0,"ondisk_log_size":4466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22859,"num_write_kb":292275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'571","reported_seq":604,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:51.521378+0000","last_change":"2024-04-25T01:25:52.355395+0000","last_active":"2024-04-25T01:40:51.521378+0000","last_peered":"2024-04-25T01:40:51.521378+0000","last_clean":"2024-04-25T01:40:51.521378+0000","last_became_active":"2024-04-25T01:25:52.355137+0000","last_became_peered":"2024-04-25T01:25:52.355137+0000","last_unstale":"2024-04-25T01:40:51.521378+0000","last_undegraded":"2024-04-25T01:40:51.521378+0000","last_fullsized":"2024-04-25T01:40:51.521378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:26.467968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"26'5","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.173222+0000","last_change":"2024-04-25T01:26:01.416108+0000","last_active":"2024-04-25T01:40:37.173222+0000","last_peered":"2024-04-25T01:40:37.173222+0000","last_clean":"2024-04-25T01:40:37.173222+0000","last_became_active":"2024-04-25T01:26:01.415824+0000","last_became_peered":"2024-04-25T01:26:01.415824+0000","last_unstale":"2024-04-25T01:40:37.173222+0000","last_undegraded":"2024-04-25T01:40:37.173222+0000","last_fullsized":"2024-04-25T01:40:37.173222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:28.139183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"26'4989","reported_seq":5012,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.273942+0000","last_change":"2024-04-25T01:26:02.416208+0000","last_active":"2024-04-25T01:41:09.273942+0000","last_peered":"2024-04-25T01:41:09.273942+0000","last_clean":"2024-04-25T01:41:09.273942+0000","last_became_active":"2024-04-25T01:26:02.415949+0000","last_became_peered":"2024-04-25T01:26:02.415949+0000","last_unstale":"2024-04-25T01:41:09.273942+0000","last_undegraded":"2024-04-25T01:41:09.273942+0000","last_fullsized":"2024-04-25T01:41:09.273942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4989,"log_dups_size":0,"ondisk_log_size":4989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:21.097225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25442,"num_write_kb":314904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"26'4796","reported_seq":4827,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.753012+0000","last_change":"2024-04-25T01:25:53.366944+0000","last_active":"2024-04-25T01:40:58.753012+0000","last_peered":"2024-04-25T01:40:58.753012+0000","last_clean":"2024-04-25T01:40:58.753012+0000","last_became_active":"2024-04-25T01:25:53.366693+0000","last_became_peered":"2024-04-25T01:25:53.366693+0000","last_unstale":"2024-04-25T01:40:58.753012+0000","last_undegraded":"2024-04-25T01:40:58.753012+0000","last_fullsized":"2024-04-25T01:40:58.753012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25026,"num_write_kb":309186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"26'4","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.236233+0000","last_change":"2024-04-25T01:25:52.356842+0000","last_active":"2024-04-25T01:40:49.236233+0000","last_peered":"2024-04-25T01:40:49.236233+0000","last_clean":"2024-04-25T01:40:49.236233+0000","last_became_active":"2024-04-25T01:25:52.356396+0000","last_became_peered":"2024-04-25T01:25:52.356396+0000","last_unstale":"2024-04-25T01:40:49.236233+0000","last_undegraded":"2024-04-25T01:40:49.236233+0000","last_fullsized":"2024-04-25T01:40:49.236233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"26'1","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:38:30.810691+0000","last_change":"2024-04-25T01:26:01.407934+0000","last_active":"2024-04-25T01:38:30.810691+0000","last_peered":"2024-04-25T01:38:30.810691+0000","last_clean":"2024-04-25T01:38:30.810691+0000","last_became_active":"2024-04-25T01:26:01.407659+0000","last_became_peered":"2024-04-25T01:26:01.407659+0000","last_unstale":"2024-04-25T01:38:30.810691+0000","last_undegraded":"2024-04-25T01:38:30.810691+0000","last_fullsized":"2024-04-25T01:38:30.810691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:19.629335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"26'4196","reported_seq":4219,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.266753+0000","last_change":"2024-04-25T01:26:02.418087+0000","last_active":"2024-04-25T01:41:09.266753+0000","last_peered":"2024-04-25T01:41:09.266753+0000","last_clean":"2024-04-25T01:41:09.266753+0000","last_became_active":"2024-04-25T01:26:02.417622+0000","last_became_peered":"2024-04-25T01:26:02.417622+0000","last_unstale":"2024-04-25T01:41:09.266753+0000","last_undegraded":"2024-04-25T01:41:09.266753+0000","last_fullsized":"2024-04-25T01:41:09.266753+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4196,"log_dups_size":0,"ondisk_log_size":4196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:03.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22617,"num_write_kb":283190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"26'4472","reported_seq":4503,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.865649+0000","last_change":"2024-04-25T01:25:53.368164+0000","last_active":"2024-04-25T01:40:58.865649+0000","last_peered":"2024-04-25T01:40:58.865649+0000","last_clean":"2024-04-25T01:40:58.865649+0000","last_became_active":"2024-04-25T01:25:53.367854+0000","last_became_peered":"2024-04-25T01:25:53.367854+0000","last_unstale":"2024-04-25T01:40:58.865649+0000","last_undegraded":"2024-04-25T01:40:58.865649+0000","last_fullsized":"2024-04-25T01:40:58.865649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4472,"log_dups_size":0,"ondisk_log_size":4472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23917,"num_write_kb":276200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'5","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.237276+0000","last_change":"2024-04-25T01:25:52.350128+0000","last_active":"2024-04-25T01:40:49.237276+0000","last_peered":"2024-04-25T01:40:49.237276+0000","last_clean":"2024-04-25T01:40:49.237276+0000","last_became_active":"2024-04-25T01:25:52.349874+0000","last_became_peered":"2024-04-25T01:25:52.349874+0000","last_unstale":"2024-04-25T01:40:49.237276+0000","last_undegraded":"2024-04-25T01:40:49.237276+0000","last_fullsized":"2024-04-25T01:40:49.237276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"26'5","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.136409+0000","last_change":"2024-04-25T01:26:01.418196+0000","last_active":"2024-04-25T01:40:37.136409+0000","last_peered":"2024-04-25T01:40:37.136409+0000","last_clean":"2024-04-25T01:40:37.136409+0000","last_became_active":"2024-04-25T01:26:01.417820+0000","last_became_peered":"2024-04-25T01:26:01.417820+0000","last_unstale":"2024-04-25T01:40:37.136409+0000","last_undegraded":"2024-04-25T01:40:37.136409+0000","last_fullsized":"2024-04-25T01:40:37.136409+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"26'4093","reported_seq":4116,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.276141+0000","last_change":"2024-04-25T01:26:02.422761+0000","last_active":"2024-04-25T01:41:09.276141+0000","last_peered":"2024-04-25T01:41:09.276141+0000","last_clean":"2024-04-25T01:41:09.276141+0000","last_became_active":"2024-04-25T01:26:02.422503+0000","last_became_peered":"2024-04-25T01:26:02.422503+0000","last_unstale":"2024-04-25T01:41:09.276141+0000","last_undegraded":"2024-04-25T01:41:09.276141+0000","last_fullsized":"2024-04-25T01:41:09.276141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4093,"log_dups_size":0,"ondisk_log_size":4093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:55.040946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21550,"num_write_kb":266696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"26'4812","reported_seq":4843,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.791228+0000","last_change":"2024-04-25T01:25:53.361696+0000","last_active":"2024-04-25T01:40:58.791228+0000","last_peered":"2024-04-25T01:40:58.791228+0000","last_clean":"2024-04-25T01:40:58.791228+0000","last_became_active":"2024-04-25T01:25:53.361156+0000","last_became_peered":"2024-04-25T01:25:53.361156+0000","last_unstale":"2024-04-25T01:40:58.791228+0000","last_undegraded":"2024-04-25T01:40:58.791228+0000","last_fullsized":"2024-04-25T01:40:58.791228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4812,"log_dups_size":0,"ondisk_log_size":4812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:27.472315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25777,"num_write_kb":295962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"26'5","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.233531+0000","last_change":"2024-04-25T01:25:52.354499+0000","last_active":"2024-04-25T01:40:49.233531+0000","last_peered":"2024-04-25T01:40:49.233531+0000","last_clean":"2024-04-25T01:40:49.233531+0000","last_became_active":"2024-04-25T01:25:52.354193+0000","last_became_peered":"2024-04-25T01:25:52.354193+0000","last_unstale":"2024-04-25T01:40:49.233531+0000","last_undegraded":"2024-04-25T01:40:49.233531+0000","last_fullsized":"2024-04-25T01:40:49.233531+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"26'5","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.016444+0000","last_change":"2024-04-25T01:26:01.418186+0000","last_active":"2024-04-25T01:40:37.016444+0000","last_peered":"2024-04-25T01:40:37.016444+0000","last_clean":"2024-04-25T01:40:37.016444+0000","last_became_active":"2024-04-25T01:26:01.417805+0000","last_became_peered":"2024-04-25T01:26:01.417805+0000","last_unstale":"2024-04-25T01:40:37.016444+0000","last_undegraded":"2024-04-25T01:40:37.016444+0000","last_fullsized":"2024-04-25T01:40:37.016444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:19.629335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"26'3951","reported_seq":3973,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.320321+0000","last_change":"2024-04-25T01:26:02.415877+0000","last_active":"2024-04-25T01:41:09.320321+0000","last_peered":"2024-04-25T01:41:09.320321+0000","last_clean":"2024-04-25T01:41:09.320321+0000","last_became_active":"2024-04-25T01:26:02.415619+0000","last_became_peered":"2024-04-25T01:26:02.415619+0000","last_unstale":"2024-04-25T01:41:09.320321+0000","last_undegraded":"2024-04-25T01:41:09.320321+0000","last_fullsized":"2024-04-25T01:41:09.320321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3951,"log_dups_size":0,"ondisk_log_size":3951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:14.647083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20033,"num_write_kb":252185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"26'4985","reported_seq":5016,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.752192+0000","last_change":"2024-04-25T01:25:53.362922+0000","last_active":"2024-04-25T01:40:58.752192+0000","last_peered":"2024-04-25T01:40:58.752192+0000","last_clean":"2024-04-25T01:40:58.752192+0000","last_became_active":"2024-04-25T01:25:53.362533+0000","last_became_peered":"2024-04-25T01:25:53.362533+0000","last_unstale":"2024-04-25T01:40:58.752192+0000","last_undegraded":"2024-04-25T01:40:58.752192+0000","last_fullsized":"2024-04-25T01:40:58.752192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4985,"log_dups_size":0,"ondisk_log_size":4985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26332,"num_write_kb":298165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"26'43","reported_seq":117,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:47.854708+0000","last_change":"2024-04-25T01:25:52.345321+0000","last_active":"2024-04-25T01:40:47.854708+0000","last_peered":"2024-04-25T01:40:47.854708+0000","last_clean":"2024-04-25T01:40:47.854708+0000","last_became_active":"2024-04-25T01:25:52.345061+0000","last_became_peered":"2024-04-25T01:25:52.345061+0000","last_unstale":"2024-04-25T01:40:47.854708+0000","last_undegraded":"2024-04-25T01:40:47.854708+0000","last_fullsized":"2024-04-25T01:40:47.854708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":41,"num_read_kb":115,"num_write":41,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"26'4","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:36.977064+0000","last_change":"2024-04-25T01:26:01.418516+0000","last_active":"2024-04-25T01:40:36.977064+0000","last_peered":"2024-04-25T01:40:36.977064+0000","last_clean":"2024-04-25T01:40:36.977064+0000","last_became_active":"2024-04-25T01:26:01.418173+0000","last_became_peered":"2024-04-25T01:26:01.418173+0000","last_unstale":"2024-04-25T01:40:36.977064+0000","last_undegraded":"2024-04-25T01:40:36.977064+0000","last_fullsized":"2024-04-25T01:40:36.977064+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:02.568472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"26'4134","reported_seq":4156,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.333895+0000","last_change":"2024-04-25T01:26:02.418115+0000","last_active":"2024-04-25T01:41:09.333895+0000","last_peered":"2024-04-25T01:41:09.333895+0000","last_clean":"2024-04-25T01:41:09.333895+0000","last_became_active":"2024-04-25T01:26:02.417355+0000","last_became_peered":"2024-04-25T01:26:02.417355+0000","last_unstale":"2024-04-25T01:41:09.333895+0000","last_undegraded":"2024-04-25T01:41:09.333895+0000","last_fullsized":"2024-04-25T01:41:09.333895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4134,"log_dups_size":0,"ondisk_log_size":4134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:19.644468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21715,"num_write_kb":287743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"26'4344","reported_seq":4375,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.752413+0000","last_change":"2024-04-25T01:25:53.355950+0000","last_active":"2024-04-25T01:40:58.752413+0000","last_peered":"2024-04-25T01:40:58.752413+0000","last_clean":"2024-04-25T01:40:58.752413+0000","last_became_active":"2024-04-25T01:25:53.355648+0000","last_became_peered":"2024-04-25T01:25:53.355648+0000","last_unstale":"2024-04-25T01:40:58.752413+0000","last_undegraded":"2024-04-25T01:40:58.752413+0000","last_fullsized":"2024-04-25T01:40:58.752413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4344,"log_dups_size":0,"ondisk_log_size":4344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23132,"num_write_kb":289328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"26'5","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.249850+0000","last_change":"2024-04-25T01:25:52.357683+0000","last_active":"2024-04-25T01:40:49.249850+0000","last_peered":"2024-04-25T01:40:49.249850+0000","last_clean":"2024-04-25T01:40:49.249850+0000","last_became_active":"2024-04-25T01:25:52.356924+0000","last_became_peered":"2024-04-25T01:25:52.356924+0000","last_unstale":"2024-04-25T01:40:49.249850+0000","last_undegraded":"2024-04-25T01:40:49.249850+0000","last_fullsized":"2024-04-25T01:40:49.249850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"26'570","reported_seq":595,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:47.577546+0000","last_change":"2024-04-25T01:26:01.421260+0000","last_active":"2024-04-25T01:40:47.577546+0000","last_peered":"2024-04-25T01:40:47.577546+0000","last_clean":"2024-04-25T01:40:47.577546+0000","last_became_active":"2024-04-25T01:26:01.420831+0000","last_became_peered":"2024-04-25T01:26:01.420831+0000","last_unstale":"2024-04-25T01:40:47.577546+0000","last_undegraded":"2024-04-25T01:40:47.577546+0000","last_fullsized":"2024-04-25T01:40:47.577546+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1330,"num_write_kb":427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"26'4947","reported_seq":4970,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.324019+0000","last_change":"2024-04-25T01:26:02.421386+0000","last_active":"2024-04-25T01:41:09.324019+0000","last_peered":"2024-04-25T01:41:09.324019+0000","last_clean":"2024-04-25T01:41:09.324019+0000","last_became_active":"2024-04-25T01:26:02.421079+0000","last_became_peered":"2024-04-25T01:26:02.421079+0000","last_unstale":"2024-04-25T01:41:09.324019+0000","last_undegraded":"2024-04-25T01:41:09.324019+0000","last_fullsized":"2024-04-25T01:41:09.324019+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4947,"log_dups_size":0,"ondisk_log_size":4947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25964,"num_write_kb":313022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"26'4754","reported_seq":4770,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.812843+0000","last_change":"2024-04-25T01:26:52.515572+0000","last_active":"2024-04-25T01:40:58.812843+0000","last_peered":"2024-04-25T01:40:58.812843+0000","last_clean":"2024-04-25T01:40:58.812843+0000","last_became_active":"2024-04-25T01:26:52.515254+0000","last_became_peered":"2024-04-25T01:26:52.515254+0000","last_unstale":"2024-04-25T01:40:58.812843+0000","last_undegraded":"2024-04-25T01:40:58.812843+0000","last_fullsized":"2024-04-25T01:40:58.812843+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4754,"log_dups_size":0,"ondisk_log_size":4754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:08.975868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24235,"num_write_kb":310192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:38:25.919823+0000","last_change":"2024-04-25T01:25:52.353155+0000","last_active":"2024-04-25T01:38:25.919823+0000","last_peered":"2024-04-25T01:38:25.919823+0000","last_clean":"2024-04-25T01:38:25.919823+0000","last_became_active":"2024-04-25T01:25:52.352826+0000","last_became_peered":"2024-04-25T01:25:52.352826+0000","last_unstale":"2024-04-25T01:38:25.919823+0000","last_undegraded":"2024-04-25T01:38:25.919823+0000","last_fullsized":"2024-04-25T01:38:25.919823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":189,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:37:45.823358+0000","last_change":"2024-04-25T01:25:41.719670+0000","last_active":"2024-04-25T01:37:45.823358+0000","last_peered":"2024-04-25T01:37:45.823358+0000","last_clean":"2024-04-25T01:37:45.823358+0000","last_became_active":"2024-04-25T01:25:41.718721+0000","last_became_peered":"2024-04-25T01:25:41.718721+0000","last_unstale":"2024-04-25T01:37:45.823358+0000","last_undegraded":"2024-04-25T01:37:45.823358+0000","last_fullsized":"2024-04-25T01:37:45.823358+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_clean_scrub_stamp":"2024-04-25T01:25:40.688097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:27.456746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"26'4","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.023240+0000","last_change":"2024-04-25T01:26:01.409339+0000","last_active":"2024-04-25T01:40:37.023240+0000","last_peered":"2024-04-25T01:40:37.023240+0000","last_clean":"2024-04-25T01:40:37.023240+0000","last_became_active":"2024-04-25T01:26:01.409062+0000","last_became_peered":"2024-04-25T01:26:01.409062+0000","last_unstale":"2024-04-25T01:40:37.023240+0000","last_undegraded":"2024-04-25T01:40:37.023240+0000","last_fullsized":"2024-04-25T01:40:37.023240+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:52.561251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"26'3833","reported_seq":3855,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.306093+0000","last_change":"2024-04-25T01:26:02.418574+0000","last_active":"2024-04-25T01:41:09.306093+0000","last_peered":"2024-04-25T01:41:09.306093+0000","last_clean":"2024-04-25T01:41:09.306093+0000","last_became_active":"2024-04-25T01:26:02.416546+0000","last_became_peered":"2024-04-25T01:26:02.416546+0000","last_unstale":"2024-04-25T01:41:09.306093+0000","last_undegraded":"2024-04-25T01:41:09.306093+0000","last_fullsized":"2024-04-25T01:41:09.306093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3833,"log_dups_size":0,"ondisk_log_size":3833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:43.994141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19504,"num_write_kb":250193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"26'5320","reported_seq":5350,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.811822+0000","last_change":"2024-04-25T01:25:53.375523+0000","last_active":"2024-04-25T01:40:58.811822+0000","last_peered":"2024-04-25T01:40:58.811822+0000","last_clean":"2024-04-25T01:40:58.811822+0000","last_became_active":"2024-04-25T01:25:53.375219+0000","last_became_peered":"2024-04-25T01:25:53.375219+0000","last_unstale":"2024-04-25T01:40:58.811822+0000","last_undegraded":"2024-04-25T01:40:58.811822+0000","last_fullsized":"2024-04-25T01:40:58.811822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5320,"log_dups_size":0,"ondisk_log_size":5320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27324,"num_write_kb":314375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'3","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.181066+0000","last_change":"2024-04-25T01:25:52.352994+0000","last_active":"2024-04-25T01:40:49.181066+0000","last_peered":"2024-04-25T01:40:49.181066+0000","last_clean":"2024-04-25T01:40:49.181066+0000","last_became_active":"2024-04-25T01:25:52.352644+0000","last_became_peered":"2024-04-25T01:25:52.352644+0000","last_unstale":"2024-04-25T01:40:49.181066+0000","last_undegraded":"2024-04-25T01:40:49.181066+0000","last_fullsized":"2024-04-25T01:40:49.181066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"26'3","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.140689+0000","last_change":"2024-04-25T01:26:01.420891+0000","last_active":"2024-04-25T01:40:37.140689+0000","last_peered":"2024-04-25T01:40:37.140689+0000","last_clean":"2024-04-25T01:40:37.140689+0000","last_became_active":"2024-04-25T01:26:01.420664+0000","last_became_peered":"2024-04-25T01:26:01.420664+0000","last_unstale":"2024-04-25T01:40:37.140689+0000","last_undegraded":"2024-04-25T01:40:37.140689+0000","last_fullsized":"2024-04-25T01:40:37.140689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"26'4000","reported_seq":4022,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.297815+0000","last_change":"2024-04-25T01:26:02.403489+0000","last_active":"2024-04-25T01:41:09.297815+0000","last_peered":"2024-04-25T01:41:09.297815+0000","last_clean":"2024-04-25T01:41:09.297815+0000","last_became_active":"2024-04-25T01:26:02.403206+0000","last_became_peered":"2024-04-25T01:26:02.403206+0000","last_unstale":"2024-04-25T01:41:09.297815+0000","last_undegraded":"2024-04-25T01:41:09.297815+0000","last_fullsized":"2024-04-25T01:41:09.297815+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4000,"log_dups_size":0,"ondisk_log_size":4000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:49.872754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21231,"num_write_kb":269526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"26'4796","reported_seq":4826,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.770361+0000","last_change":"2024-04-25T01:25:53.371018+0000","last_active":"2024-04-25T01:40:58.770361+0000","last_peered":"2024-04-25T01:40:58.770361+0000","last_clean":"2024-04-25T01:40:58.770361+0000","last_became_active":"2024-04-25T01:25:53.370539+0000","last_became_peered":"2024-04-25T01:25:53.370539+0000","last_unstale":"2024-04-25T01:40:58.770361+0000","last_undegraded":"2024-04-25T01:40:58.770361+0000","last_fullsized":"2024-04-25T01:40:58.770361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25182,"num_write_kb":313860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"26'2","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:39:25.987518+0000","last_change":"2024-04-25T01:25:52.355133+0000","last_active":"2024-04-25T01:39:25.987518+0000","last_peered":"2024-04-25T01:39:25.987518+0000","last_clean":"2024-04-25T01:39:25.987518+0000","last_became_active":"2024-04-25T01:25:52.354836+0000","last_became_peered":"2024-04-25T01:25:52.354836+0000","last_unstale":"2024-04-25T01:39:25.987518+0000","last_undegraded":"2024-04-25T01:39:25.987518+0000","last_fullsized":"2024-04-25T01:39:25.987518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:10.331704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"26'1112","reported_seq":1136,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.176654+0000","last_change":"2024-04-25T01:26:01.419285+0000","last_active":"2024-04-25T01:40:37.176654+0000","last_peered":"2024-04-25T01:40:37.176654+0000","last_clean":"2024-04-25T01:40:37.176654+0000","last_became_active":"2024-04-25T01:26:01.418987+0000","last_became_peered":"2024-04-25T01:26:01.418987+0000","last_unstale":"2024-04-25T01:40:37.176654+0000","last_undegraded":"2024-04-25T01:40:37.176654+0000","last_fullsized":"2024-04-25T01:40:37.176654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1112,"log_dups_size":0,"ondisk_log_size":1112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1103,"num_write_kb":4430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"26'4082","reported_seq":4104,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.262432+0000","last_change":"2024-04-25T01:26:02.416544+0000","last_active":"2024-04-25T01:41:09.262432+0000","last_peered":"2024-04-25T01:41:09.262432+0000","last_clean":"2024-04-25T01:41:09.262432+0000","last_became_active":"2024-04-25T01:26:02.416189+0000","last_became_peered":"2024-04-25T01:26:02.416189+0000","last_unstale":"2024-04-25T01:41:09.262432+0000","last_undegraded":"2024-04-25T01:41:09.262432+0000","last_fullsized":"2024-04-25T01:41:09.262432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4082,"log_dups_size":0,"ondisk_log_size":4082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.688469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20562,"num_write_kb":274915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"26'5666","reported_seq":5696,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.889157+0000","last_change":"2024-04-25T01:25:53.370754+0000","last_active":"2024-04-25T01:40:58.889157+0000","last_peered":"2024-04-25T01:40:58.889157+0000","last_clean":"2024-04-25T01:40:58.889157+0000","last_became_active":"2024-04-25T01:25:53.370074+0000","last_became_peered":"2024-04-25T01:25:53.370074+0000","last_unstale":"2024-04-25T01:40:58.889157+0000","last_undegraded":"2024-04-25T01:40:58.889157+0000","last_fullsized":"2024-04-25T01:40:58.889157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5666,"log_dups_size":0,"ondisk_log_size":5666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29161,"num_write_kb":354533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"26'6","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.237424+0000","last_change":"2024-04-25T01:25:52.345943+0000","last_active":"2024-04-25T01:40:49.237424+0000","last_peered":"2024-04-25T01:40:49.237424+0000","last_clean":"2024-04-25T01:40:49.237424+0000","last_became_active":"2024-04-25T01:25:52.345717+0000","last_became_peered":"2024-04-25T01:25:52.345717+0000","last_unstale":"2024-04-25T01:40:49.237424+0000","last_undegraded":"2024-04-25T01:40:49.237424+0000","last_fullsized":"2024-04-25T01:40:49.237424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"26'3","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.011634+0000","last_change":"2024-04-25T01:26:01.416027+0000","last_active":"2024-04-25T01:40:37.011634+0000","last_peered":"2024-04-25T01:40:37.011634+0000","last_clean":"2024-04-25T01:40:37.011634+0000","last_became_active":"2024-04-25T01:26:01.415750+0000","last_became_peered":"2024-04-25T01:26:01.415750+0000","last_unstale":"2024-04-25T01:40:37.011634+0000","last_undegraded":"2024-04-25T01:40:37.011634+0000","last_fullsized":"2024-04-25T01:40:37.011634+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:27.042376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"26'4748","reported_seq":4770,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.276555+0000","last_change":"2024-04-25T01:26:02.414929+0000","last_active":"2024-04-25T01:41:09.276555+0000","last_peered":"2024-04-25T01:41:09.276555+0000","last_clean":"2024-04-25T01:41:09.276555+0000","last_became_active":"2024-04-25T01:26:02.414316+0000","last_became_peered":"2024-04-25T01:26:02.414316+0000","last_unstale":"2024-04-25T01:41:09.276555+0000","last_undegraded":"2024-04-25T01:41:09.276555+0000","last_fullsized":"2024-04-25T01:41:09.276555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4748,"log_dups_size":0,"ondisk_log_size":4748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:53.568065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24613,"num_write_kb":303491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"26'4546","reported_seq":4576,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.712213+0000","last_change":"2024-04-25T01:25:53.365171+0000","last_active":"2024-04-25T01:40:58.712213+0000","last_peered":"2024-04-25T01:40:58.712213+0000","last_clean":"2024-04-25T01:40:58.712213+0000","last_became_active":"2024-04-25T01:25:53.364049+0000","last_became_peered":"2024-04-25T01:25:53.364049+0000","last_unstale":"2024-04-25T01:40:58.712213+0000","last_undegraded":"2024-04-25T01:40:58.712213+0000","last_fullsized":"2024-04-25T01:40:58.712213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4546,"log_dups_size":0,"ondisk_log_size":4546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22740,"num_write_kb":287925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'5","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.243400+0000","last_change":"2024-04-25T01:25:52.350688+0000","last_active":"2024-04-25T01:40:49.243400+0000","last_peered":"2024-04-25T01:40:49.243400+0000","last_clean":"2024-04-25T01:40:49.243400+0000","last_became_active":"2024-04-25T01:25:52.350357+0000","last_became_peered":"2024-04-25T01:25:52.350357+0000","last_unstale":"2024-04-25T01:40:49.243400+0000","last_undegraded":"2024-04-25T01:40:49.243400+0000","last_fullsized":"2024-04-25T01:40:49.243400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"26'3","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.002855+0000","last_change":"2024-04-25T01:26:01.418580+0000","last_active":"2024-04-25T01:40:37.002855+0000","last_peered":"2024-04-25T01:40:37.002855+0000","last_clean":"2024-04-25T01:40:37.002855+0000","last_became_active":"2024-04-25T01:26:01.418179+0000","last_became_peered":"2024-04-25T01:26:01.418179+0000","last_unstale":"2024-04-25T01:40:37.002855+0000","last_undegraded":"2024-04-25T01:40:37.002855+0000","last_fullsized":"2024-04-25T01:40:37.002855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:42.987327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"26'3833","reported_seq":3855,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.301162+0000","last_change":"2024-04-25T01:26:02.418424+0000","last_active":"2024-04-25T01:41:09.301162+0000","last_peered":"2024-04-25T01:41:09.301162+0000","last_clean":"2024-04-25T01:41:09.301162+0000","last_became_active":"2024-04-25T01:26:02.418090+0000","last_became_peered":"2024-04-25T01:26:02.418090+0000","last_unstale":"2024-04-25T01:41:09.301162+0000","last_undegraded":"2024-04-25T01:41:09.301162+0000","last_fullsized":"2024-04-25T01:41:09.301162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3833,"log_dups_size":0,"ondisk_log_size":3833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:10.049310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19507,"num_write_kb":254331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"26'4627","reported_seq":4657,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.811217+0000","last_change":"2024-04-25T01:25:53.366376+0000","last_active":"2024-04-25T01:40:58.811217+0000","last_peered":"2024-04-25T01:40:58.811217+0000","last_clean":"2024-04-25T01:40:58.811217+0000","last_became_active":"2024-04-25T01:25:53.365715+0000","last_became_peered":"2024-04-25T01:25:53.365715+0000","last_unstale":"2024-04-25T01:40:58.811217+0000","last_undegraded":"2024-04-25T01:40:58.811217+0000","last_fullsized":"2024-04-25T01:40:58.811217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4627,"log_dups_size":0,"ondisk_log_size":4627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24780,"num_write_kb":300200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"26'11","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.216654+0000","last_change":"2024-04-25T01:25:52.347413+0000","last_active":"2024-04-25T01:40:49.216654+0000","last_peered":"2024-04-25T01:40:49.216654+0000","last_clean":"2024-04-25T01:40:49.216654+0000","last_became_active":"2024-04-25T01:25:52.347144+0000","last_became_peered":"2024-04-25T01:25:52.347144+0000","last_unstale":"2024-04-25T01:40:49.216654+0000","last_undegraded":"2024-04-25T01:40:49.216654+0000","last_fullsized":"2024-04-25T01:40:49.216654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"26'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.101937+0000","last_change":"2024-04-25T01:26:01.409917+0000","last_active":"2024-04-25T01:40:37.101937+0000","last_peered":"2024-04-25T01:40:37.101937+0000","last_clean":"2024-04-25T01:40:37.101937+0000","last_became_active":"2024-04-25T01:26:01.409654+0000","last_became_peered":"2024-04-25T01:26:01.409654+0000","last_unstale":"2024-04-25T01:40:37.101937+0000","last_undegraded":"2024-04-25T01:40:37.101937+0000","last_fullsized":"2024-04-25T01:40:37.101937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:37.142255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"26'4668","reported_seq":4690,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.342103+0000","last_change":"2024-04-25T01:26:02.425103+0000","last_active":"2024-04-25T01:41:09.342103+0000","last_peered":"2024-04-25T01:41:09.342103+0000","last_clean":"2024-04-25T01:41:09.342103+0000","last_became_active":"2024-04-25T01:26:02.424819+0000","last_became_peered":"2024-04-25T01:26:02.424819+0000","last_unstale":"2024-04-25T01:41:09.342103+0000","last_undegraded":"2024-04-25T01:41:09.342103+0000","last_fullsized":"2024-04-25T01:41:09.342103+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4668,"log_dups_size":0,"ondisk_log_size":4668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23987,"num_write_kb":301090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"26'5240","reported_seq":5270,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.796905+0000","last_change":"2024-04-25T01:25:53.364987+0000","last_active":"2024-04-25T01:40:58.796905+0000","last_peered":"2024-04-25T01:40:58.796905+0000","last_clean":"2024-04-25T01:40:58.796905+0000","last_became_active":"2024-04-25T01:25:53.364431+0000","last_became_peered":"2024-04-25T01:25:53.364431+0000","last_unstale":"2024-04-25T01:40:58.796905+0000","last_undegraded":"2024-04-25T01:40:58.796905+0000","last_fullsized":"2024-04-25T01:40:58.796905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5240,"log_dups_size":0,"ondisk_log_size":5240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27403,"num_write_kb":326994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'6","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.234588+0000","last_change":"2024-04-25T01:25:52.351205+0000","last_active":"2024-04-25T01:40:49.234588+0000","last_peered":"2024-04-25T01:40:49.234588+0000","last_clean":"2024-04-25T01:40:49.234588+0000","last_became_active":"2024-04-25T01:25:52.350946+0000","last_became_peered":"2024-04-25T01:25:52.350946+0000","last_unstale":"2024-04-25T01:40:49.234588+0000","last_undegraded":"2024-04-25T01:40:49.234588+0000","last_fullsized":"2024-04-25T01:40:49.234588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"26'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:31.329770+0000","last_change":"2024-04-25T01:26:01.415799+0000","last_active":"2024-04-25T01:40:31.329770+0000","last_peered":"2024-04-25T01:40:31.329770+0000","last_clean":"2024-04-25T01:40:31.329770+0000","last_became_active":"2024-04-25T01:26:01.415572+0000","last_became_peered":"2024-04-25T01:26:01.415572+0000","last_unstale":"2024-04-25T01:40:31.329770+0000","last_undegraded":"2024-04-25T01:40:31.329770+0000","last_fullsized":"2024-04-25T01:40:31.329770+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:54.034132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"26'4341","reported_seq":4363,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.334259+0000","last_change":"2024-04-25T01:26:02.422959+0000","last_active":"2024-04-25T01:41:09.334259+0000","last_peered":"2024-04-25T01:41:09.334259+0000","last_clean":"2024-04-25T01:41:09.334259+0000","last_became_active":"2024-04-25T01:26:02.422210+0000","last_became_peered":"2024-04-25T01:26:02.422210+0000","last_unstale":"2024-04-25T01:41:09.334259+0000","last_undegraded":"2024-04-25T01:41:09.334259+0000","last_fullsized":"2024-04-25T01:41:09.334259+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4341,"log_dups_size":0,"ondisk_log_size":4341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22600,"num_write_kb":283130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"26'4919","reported_seq":4949,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.871808+0000","last_change":"2024-04-25T01:25:53.368187+0000","last_active":"2024-04-25T01:40:58.871808+0000","last_peered":"2024-04-25T01:40:58.871808+0000","last_clean":"2024-04-25T01:40:58.871808+0000","last_became_active":"2024-04-25T01:25:53.367664+0000","last_became_peered":"2024-04-25T01:25:53.367664+0000","last_unstale":"2024-04-25T01:40:58.871808+0000","last_undegraded":"2024-04-25T01:40:58.871808+0000","last_fullsized":"2024-04-25T01:40:58.871808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4919,"log_dups_size":0,"ondisk_log_size":4919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25274,"num_write_kb":330818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"26'5","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.184872+0000","last_change":"2024-04-25T01:25:52.357397+0000","last_active":"2024-04-25T01:40:49.184872+0000","last_peered":"2024-04-25T01:40:49.184872+0000","last_clean":"2024-04-25T01:40:49.184872+0000","last_became_active":"2024-04-25T01:25:52.356881+0000","last_became_peered":"2024-04-25T01:25:52.356881+0000","last_unstale":"2024-04-25T01:40:49.184872+0000","last_undegraded":"2024-04-25T01:40:49.184872+0000","last_fullsized":"2024-04-25T01:40:49.184872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"26'314","reported_seq":338,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:46.230895+0000","last_change":"2024-04-25T01:26:01.420103+0000","last_active":"2024-04-25T01:40:46.230895+0000","last_peered":"2024-04-25T01:40:46.230895+0000","last_clean":"2024-04-25T01:40:46.230895+0000","last_became_active":"2024-04-25T01:26:01.419526+0000","last_became_peered":"2024-04-25T01:26:01.419526+0000","last_unstale":"2024-04-25T01:40:46.230895+0000","last_undegraded":"2024-04-25T01:40:46.230895+0000","last_fullsized":"2024-04-25T01:40:46.230895+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:27.042376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":310,"num_write_kb":4240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"26'6253","reported_seq":6275,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.317329+0000","last_change":"2024-04-25T01:26:02.421488+0000","last_active":"2024-04-25T01:41:09.317329+0000","last_peered":"2024-04-25T01:41:09.317329+0000","last_clean":"2024-04-25T01:41:09.317329+0000","last_became_active":"2024-04-25T01:26:02.419308+0000","last_became_peered":"2024-04-25T01:26:02.419308+0000","last_unstale":"2024-04-25T01:41:09.317329+0000","last_undegraded":"2024-04-25T01:41:09.317329+0000","last_fullsized":"2024-04-25T01:41:09.317329+0000","mapping_epoch":22,"log_start":"26'1000","ondisk_log_start":"26'1000","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5253,"log_dups_size":0,"ondisk_log_size":5253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32276,"num_write_kb":400144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"26'4645","reported_seq":4675,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.788191+0000","last_change":"2024-04-25T01:25:53.370720+0000","last_active":"2024-04-25T01:40:58.788191+0000","last_peered":"2024-04-25T01:40:58.788191+0000","last_clean":"2024-04-25T01:40:58.788191+0000","last_became_active":"2024-04-25T01:25:53.370142+0000","last_became_peered":"2024-04-25T01:25:53.370142+0000","last_unstale":"2024-04-25T01:40:58.788191+0000","last_undegraded":"2024-04-25T01:40:58.788191+0000","last_fullsized":"2024-04-25T01:40:58.788191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4645,"log_dups_size":0,"ondisk_log_size":4645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:52.833379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24130,"num_write_kb":289357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"26'8","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.232990+0000","last_change":"2024-04-25T01:25:52.349092+0000","last_active":"2024-04-25T01:40:49.232990+0000","last_peered":"2024-04-25T01:40:49.232990+0000","last_clean":"2024-04-25T01:40:49.232990+0000","last_became_active":"2024-04-25T01:25:52.348846+0000","last_became_peered":"2024-04-25T01:25:52.348846+0000","last_unstale":"2024-04-25T01:40:49.232990+0000","last_undegraded":"2024-04-25T01:40:49.232990+0000","last_fullsized":"2024-04-25T01:40:49.232990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:21.982440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"26'26","reported_seq":50,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.278610+0000","last_change":"2024-04-25T01:26:01.414517+0000","last_active":"2024-04-25T01:40:37.278610+0000","last_peered":"2024-04-25T01:40:37.278610+0000","last_clean":"2024-04-25T01:40:37.278610+0000","last_became_active":"2024-04-25T01:26:01.414243+0000","last_became_peered":"2024-04-25T01:26:01.414243+0000","last_unstale":"2024-04-25T01:40:37.278610+0000","last_undegraded":"2024-04-25T01:40:37.278610+0000","last_fullsized":"2024-04-25T01:40:37.278610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:48.491461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"26'5295","reported_seq":5317,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.307012+0000","last_change":"2024-04-25T01:26:02.415893+0000","last_active":"2024-04-25T01:41:09.307012+0000","last_peered":"2024-04-25T01:41:09.307012+0000","last_clean":"2024-04-25T01:41:09.307012+0000","last_became_active":"2024-04-25T01:26:02.415596+0000","last_became_peered":"2024-04-25T01:26:02.415596+0000","last_unstale":"2024-04-25T01:41:09.307012+0000","last_undegraded":"2024-04-25T01:41:09.307012+0000","last_fullsized":"2024-04-25T01:41:09.307012+0000","mapping_epoch":22,"log_start":"26'100","ondisk_log_start":"26'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5195,"log_dups_size":0,"ondisk_log_size":5195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26962,"num_write_kb":337426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"26'4577","reported_seq":4607,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.793341+0000","last_change":"2024-04-25T01:25:53.373572+0000","last_active":"2024-04-25T01:40:58.793341+0000","last_peered":"2024-04-25T01:40:58.793341+0000","last_clean":"2024-04-25T01:40:58.793341+0000","last_became_active":"2024-04-25T01:25:53.373129+0000","last_became_peered":"2024-04-25T01:25:53.373129+0000","last_unstale":"2024-04-25T01:40:58.793341+0000","last_undegraded":"2024-04-25T01:40:58.793341+0000","last_fullsized":"2024-04-25T01:40:58.793341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23671,"num_write_kb":287382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'335","reported_seq":367,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.269952+0000","last_change":"2024-04-25T01:25:52.350446+0000","last_active":"2024-04-25T01:40:49.269952+0000","last_peered":"2024-04-25T01:40:49.269952+0000","last_clean":"2024-04-25T01:40:49.269952+0000","last_became_active":"2024-04-25T01:25:52.349448+0000","last_became_peered":"2024-04-25T01:25:52.349448+0000","last_unstale":"2024-04-25T01:40:49.269952+0000","last_undegraded":"2024-04-25T01:40:49.269952+0000","last_fullsized":"2024-04-25T01:40:49.269952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":331,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"26'5","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.111146+0000","last_change":"2024-04-25T01:26:01.418794+0000","last_active":"2024-04-25T01:40:37.111146+0000","last_peered":"2024-04-25T01:40:37.111146+0000","last_clean":"2024-04-25T01:40:37.111146+0000","last_became_active":"2024-04-25T01:26:01.418541+0000","last_became_peered":"2024-04-25T01:26:01.418541+0000","last_unstale":"2024-04-25T01:40:37.111146+0000","last_undegraded":"2024-04-25T01:40:37.111146+0000","last_fullsized":"2024-04-25T01:40:37.111146+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"26'4257","reported_seq":4279,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.301321+0000","last_change":"2024-04-25T01:26:02.425391+0000","last_active":"2024-04-25T01:41:09.301321+0000","last_peered":"2024-04-25T01:41:09.301321+0000","last_clean":"2024-04-25T01:41:09.301321+0000","last_became_active":"2024-04-25T01:26:02.425106+0000","last_became_peered":"2024-04-25T01:26:02.425106+0000","last_unstale":"2024-04-25T01:41:09.301321+0000","last_undegraded":"2024-04-25T01:41:09.301321+0000","last_fullsized":"2024-04-25T01:41:09.301321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4257,"log_dups_size":0,"ondisk_log_size":4257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22152,"num_write_kb":281377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"26'6968","reported_seq":6998,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.878933+0000","last_change":"2024-04-25T01:25:53.366416+0000","last_active":"2024-04-25T01:40:58.878933+0000","last_peered":"2024-04-25T01:40:58.878933+0000","last_clean":"2024-04-25T01:40:58.878933+0000","last_became_active":"2024-04-25T01:25:53.366157+0000","last_became_peered":"2024-04-25T01:25:53.366157+0000","last_unstale":"2024-04-25T01:40:58.878933+0000","last_undegraded":"2024-04-25T01:40:58.878933+0000","last_fullsized":"2024-04-25T01:40:58.878933+0000","mapping_epoch":18,"log_start":"26'1500","ondisk_log_start":"26'1500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5468,"log_dups_size":0,"ondisk_log_size":5468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:26.277566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35763,"num_write_kb":414188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"26'25","reported_seq":57,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.254786+0000","last_change":"2024-04-25T01:25:52.357610+0000","last_active":"2024-04-25T01:40:49.254786+0000","last_peered":"2024-04-25T01:40:49.254786+0000","last_clean":"2024-04-25T01:40:49.254786+0000","last_became_active":"2024-04-25T01:25:52.357355+0000","last_became_peered":"2024-04-25T01:25:52.357355+0000","last_unstale":"2024-04-25T01:40:49.254786+0000","last_undegraded":"2024-04-25T01:40:49.254786+0000","last_fullsized":"2024-04-25T01:40:49.254786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"26'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.122870+0000","last_change":"2024-04-25T01:26:01.412666+0000","last_active":"2024-04-25T01:40:37.122870+0000","last_peered":"2024-04-25T01:40:37.122870+0000","last_clean":"2024-04-25T01:40:37.122870+0000","last_became_active":"2024-04-25T01:26:01.412132+0000","last_became_peered":"2024-04-25T01:26:01.412132+0000","last_unstale":"2024-04-25T01:40:37.122870+0000","last_undegraded":"2024-04-25T01:40:37.122870+0000","last_fullsized":"2024-04-25T01:40:37.122870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"26'5141","reported_seq":5163,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.286915+0000","last_change":"2024-04-25T01:26:02.413106+0000","last_active":"2024-04-25T01:41:09.286915+0000","last_peered":"2024-04-25T01:41:09.286915+0000","last_clean":"2024-04-25T01:41:09.286915+0000","last_became_active":"2024-04-25T01:26:02.412759+0000","last_became_peered":"2024-04-25T01:26:02.412759+0000","last_unstale":"2024-04-25T01:41:09.286915+0000","last_undegraded":"2024-04-25T01:41:09.286915+0000","last_fullsized":"2024-04-25T01:41:09.286915+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5141,"log_dups_size":0,"ondisk_log_size":5141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:28.049190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27471,"num_write_kb":310956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"26'3595","reported_seq":3625,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.782418+0000","last_change":"2024-04-25T01:25:53.374311+0000","last_active":"2024-04-25T01:40:58.782418+0000","last_peered":"2024-04-25T01:40:58.782418+0000","last_clean":"2024-04-25T01:40:58.782418+0000","last_became_active":"2024-04-25T01:25:53.373890+0000","last_became_peered":"2024-04-25T01:25:53.373890+0000","last_unstale":"2024-04-25T01:40:58.782418+0000","last_undegraded":"2024-04-25T01:40:58.782418+0000","last_fullsized":"2024-04-25T01:40:58.782418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3595,"log_dups_size":0,"ondisk_log_size":3595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18396,"num_write_kb":249899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"26'2","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.190934+0000","last_change":"2024-04-25T01:25:52.356231+0000","last_active":"2024-04-25T01:40:49.190934+0000","last_peered":"2024-04-25T01:40:49.190934+0000","last_clean":"2024-04-25T01:40:49.190934+0000","last_became_active":"2024-04-25T01:25:52.355913+0000","last_became_peered":"2024-04-25T01:25:52.355913+0000","last_unstale":"2024-04-25T01:40:49.190934+0000","last_undegraded":"2024-04-25T01:40:49.190934+0000","last_fullsized":"2024-04-25T01:40:49.190934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"26'12","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.017092+0000","last_change":"2024-04-25T01:26:01.419513+0000","last_active":"2024-04-25T01:40:37.017092+0000","last_peered":"2024-04-25T01:40:37.017092+0000","last_clean":"2024-04-25T01:40:37.017092+0000","last_became_active":"2024-04-25T01:26:01.419015+0000","last_became_peered":"2024-04-25T01:26:01.419015+0000","last_unstale":"2024-04-25T01:40:37.017092+0000","last_undegraded":"2024-04-25T01:40:37.017092+0000","last_fullsized":"2024-04-25T01:40:37.017092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:10.486367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"26'5013","reported_seq":5035,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.306814+0000","last_change":"2024-04-25T01:26:02.425241+0000","last_active":"2024-04-25T01:41:09.306814+0000","last_peered":"2024-04-25T01:41:09.306814+0000","last_clean":"2024-04-25T01:41:09.306814+0000","last_became_active":"2024-04-25T01:26:02.424960+0000","last_became_peered":"2024-04-25T01:26:02.424960+0000","last_unstale":"2024-04-25T01:41:09.306814+0000","last_undegraded":"2024-04-25T01:41:09.306814+0000","last_fullsized":"2024-04-25T01:41:09.306814+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5013,"log_dups_size":0,"ondisk_log_size":5013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25777,"num_write_kb":320486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"26'4876","reported_seq":4906,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.800754+0000","last_change":"2024-04-25T01:25:53.371445+0000","last_active":"2024-04-25T01:40:58.800754+0000","last_peered":"2024-04-25T01:40:58.800754+0000","last_clean":"2024-04-25T01:40:58.800754+0000","last_became_active":"2024-04-25T01:25:53.371086+0000","last_became_peered":"2024-04-25T01:25:53.371086+0000","last_unstale":"2024-04-25T01:40:58.800754+0000","last_undegraded":"2024-04-25T01:40:58.800754+0000","last_fullsized":"2024-04-25T01:40:58.800754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4876,"log_dups_size":0,"ondisk_log_size":4876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24656,"num_write_kb":303790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"26'2","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:26.547207+0000","last_change":"2024-04-25T01:25:52.357459+0000","last_active":"2024-04-25T01:40:26.547207+0000","last_peered":"2024-04-25T01:40:26.547207+0000","last_clean":"2024-04-25T01:40:26.547207+0000","last_became_active":"2024-04-25T01:25:52.357182+0000","last_became_peered":"2024-04-25T01:25:52.357182+0000","last_unstale":"2024-04-25T01:40:26.547207+0000","last_undegraded":"2024-04-25T01:40:26.547207+0000","last_fullsized":"2024-04-25T01:40:26.547207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"26'6","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.088116+0000","last_change":"2024-04-25T01:26:01.415321+0000","last_active":"2024-04-25T01:40:37.088116+0000","last_peered":"2024-04-25T01:40:37.088116+0000","last_clean":"2024-04-25T01:40:37.088116+0000","last_became_active":"2024-04-25T01:26:01.415020+0000","last_became_peered":"2024-04-25T01:26:01.415020+0000","last_unstale":"2024-04-25T01:40:37.088116+0000","last_undegraded":"2024-04-25T01:40:37.088116+0000","last_fullsized":"2024-04-25T01:40:37.088116+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:20.090411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"26'5316","reported_seq":5339,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.331286+0000","last_change":"2024-04-25T01:26:02.416917+0000","last_active":"2024-04-25T01:41:09.331286+0000","last_peered":"2024-04-25T01:41:09.331286+0000","last_clean":"2024-04-25T01:41:09.331286+0000","last_became_active":"2024-04-25T01:26:02.416623+0000","last_became_peered":"2024-04-25T01:26:02.416623+0000","last_unstale":"2024-04-25T01:41:09.331286+0000","last_undegraded":"2024-04-25T01:41:09.331286+0000","last_fullsized":"2024-04-25T01:41:09.331286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5316,"log_dups_size":0,"ondisk_log_size":5316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:55.040946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27734,"num_write_kb":356956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"26'8","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.028842+0000","last_change":"2024-04-25T01:26:01.415472+0000","last_active":"2024-04-25T01:40:37.028842+0000","last_peered":"2024-04-25T01:40:37.028842+0000","last_clean":"2024-04-25T01:40:37.028842+0000","last_became_active":"2024-04-25T01:26:01.415184+0000","last_became_peered":"2024-04-25T01:26:01.415184+0000","last_unstale":"2024-04-25T01:40:37.028842+0000","last_undegraded":"2024-04-25T01:40:37.028842+0000","last_fullsized":"2024-04-25T01:40:37.028842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"26'5623","reported_seq":5653,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.800035+0000","last_change":"2024-04-25T01:25:53.363043+0000","last_active":"2024-04-25T01:40:58.800035+0000","last_peered":"2024-04-25T01:40:58.800035+0000","last_clean":"2024-04-25T01:40:58.800035+0000","last_became_active":"2024-04-25T01:25:53.362601+0000","last_became_peered":"2024-04-25T01:25:53.362601+0000","last_unstale":"2024-04-25T01:40:58.800035+0000","last_undegraded":"2024-04-25T01:40:58.800035+0000","last_fullsized":"2024-04-25T01:40:58.800035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5623,"log_dups_size":0,"ondisk_log_size":5623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29125,"num_write_kb":354366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"26'3","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:46.309270+0000","last_change":"2024-04-25T01:25:52.358081+0000","last_active":"2024-04-25T01:40:46.309270+0000","last_peered":"2024-04-25T01:40:46.309270+0000","last_clean":"2024-04-25T01:40:46.309270+0000","last_became_active":"2024-04-25T01:25:52.357365+0000","last_became_peered":"2024-04-25T01:25:52.357365+0000","last_unstale":"2024-04-25T01:40:46.309270+0000","last_undegraded":"2024-04-25T01:40:46.309270+0000","last_fullsized":"2024-04-25T01:40:46.309270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"26'4779","reported_seq":4802,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.260694+0000","last_change":"2024-04-25T01:26:02.418872+0000","last_active":"2024-04-25T01:41:09.260694+0000","last_peered":"2024-04-25T01:41:09.260694+0000","last_clean":"2024-04-25T01:41:09.260694+0000","last_became_active":"2024-04-25T01:26:02.418614+0000","last_became_peered":"2024-04-25T01:26:02.418614+0000","last_unstale":"2024-04-25T01:41:09.260694+0000","last_undegraded":"2024-04-25T01:41:09.260694+0000","last_fullsized":"2024-04-25T01:41:09.260694+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4779,"log_dups_size":0,"ondisk_log_size":4779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:49.498275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25207,"num_write_kb":318198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"26'632","reported_seq":657,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:27.924243+0000","last_change":"2024-04-25T01:26:01.410258+0000","last_active":"2024-04-25T01:40:27.924243+0000","last_peered":"2024-04-25T01:40:27.924243+0000","last_clean":"2024-04-25T01:40:27.924243+0000","last_became_active":"2024-04-25T01:26:01.410004+0000","last_became_peered":"2024-04-25T01:26:01.410004+0000","last_unstale":"2024-04-25T01:40:27.924243+0000","last_undegraded":"2024-04-25T01:40:27.924243+0000","last_fullsized":"2024-04-25T01:40:27.924243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:48.865940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":630,"num_write_kb":4461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"26'4416","reported_seq":4446,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.812059+0000","last_change":"2024-04-25T01:25:53.363794+0000","last_active":"2024-04-25T01:40:58.812059+0000","last_peered":"2024-04-25T01:40:58.812059+0000","last_clean":"2024-04-25T01:40:58.812059+0000","last_became_active":"2024-04-25T01:25:53.363509+0000","last_became_peered":"2024-04-25T01:25:53.363509+0000","last_unstale":"2024-04-25T01:40:58.812059+0000","last_undegraded":"2024-04-25T01:40:58.812059+0000","last_fullsized":"2024-04-25T01:40:58.812059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4416,"log_dups_size":0,"ondisk_log_size":4416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23348,"num_write_kb":294385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"26'3","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:26.190581+0000","last_change":"2024-04-25T01:25:52.347464+0000","last_active":"2024-04-25T01:40:26.190581+0000","last_peered":"2024-04-25T01:40:26.190581+0000","last_clean":"2024-04-25T01:40:26.190581+0000","last_became_active":"2024-04-25T01:25:52.347138+0000","last_became_peered":"2024-04-25T01:25:52.347138+0000","last_unstale":"2024-04-25T01:40:26.190581+0000","last_undegraded":"2024-04-25T01:40:26.190581+0000","last_fullsized":"2024-04-25T01:40:26.190581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"26'5575","reported_seq":5597,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.362996+0000","last_change":"2024-04-25T01:26:02.425948+0000","last_active":"2024-04-25T01:41:09.362996+0000","last_peered":"2024-04-25T01:41:09.362996+0000","last_clean":"2024-04-25T01:41:09.362996+0000","last_became_active":"2024-04-25T01:26:02.424947+0000","last_became_peered":"2024-04-25T01:26:02.424947+0000","last_unstale":"2024-04-25T01:41:09.362996+0000","last_undegraded":"2024-04-25T01:41:09.362996+0000","last_fullsized":"2024-04-25T01:41:09.362996+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5575,"log_dups_size":0,"ondisk_log_size":5575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28821,"num_write_kb":361321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"26'87","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.639358+0000","last_change":"2024-04-25T01:25:52.349989+0000","last_active":"2024-04-25T01:40:58.639358+0000","last_peered":"2024-04-25T01:40:58.639358+0000","last_clean":"2024-04-25T01:40:58.639358+0000","last_became_active":"2024-04-25T01:25:52.349672+0000","last_became_peered":"2024-04-25T01:25:52.349672+0000","last_unstale":"2024-04-25T01:40:58.639358+0000","last_undegraded":"2024-04-25T01:40:58.639358+0000","last_fullsized":"2024-04-25T01:40:58.639358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":980823,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83,"num_write_kb":1015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"26'3832","reported_seq":3862,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.794598+0000","last_change":"2024-04-25T01:25:53.356610+0000","last_active":"2024-04-25T01:40:58.794598+0000","last_peered":"2024-04-25T01:40:58.794598+0000","last_clean":"2024-04-25T01:40:58.794598+0000","last_became_active":"2024-04-25T01:25:53.356015+0000","last_became_peered":"2024-04-25T01:25:53.356015+0000","last_unstale":"2024-04-25T01:40:58.794598+0000","last_undegraded":"2024-04-25T01:40:58.794598+0000","last_fullsized":"2024-04-25T01:40:58.794598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3832,"log_dups_size":0,"ondisk_log_size":3832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19386,"num_write_kb":237359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"26'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.169856+0000","last_change":"2024-04-25T01:26:01.420261+0000","last_active":"2024-04-25T01:40:37.169856+0000","last_peered":"2024-04-25T01:40:37.169856+0000","last_clean":"2024-04-25T01:40:37.169856+0000","last_became_active":"2024-04-25T01:26:01.420003+0000","last_became_peered":"2024-04-25T01:26:01.420003+0000","last_unstale":"2024-04-25T01:40:37.169856+0000","last_undegraded":"2024-04-25T01:40:37.169856+0000","last_fullsized":"2024-04-25T01:40:37.169856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:28.695853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"26'3175","reported_seq":3197,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.349312+0000","last_change":"2024-04-25T01:26:02.426327+0000","last_active":"2024-04-25T01:41:09.349312+0000","last_peered":"2024-04-25T01:41:09.349312+0000","last_clean":"2024-04-25T01:41:09.349312+0000","last_became_active":"2024-04-25T01:26:02.426054+0000","last_became_peered":"2024-04-25T01:26:02.426054+0000","last_unstale":"2024-04-25T01:41:09.349312+0000","last_undegraded":"2024-04-25T01:41:09.349312+0000","last_fullsized":"2024-04-25T01:41:09.349312+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3175,"log_dups_size":0,"ondisk_log_size":3175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:17.809738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15854,"num_write_kb":211025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"26'605","reported_seq":637,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.248248+0000","last_change":"2024-04-25T01:25:52.353955+0000","last_active":"2024-04-25T01:40:49.248248+0000","last_peered":"2024-04-25T01:40:49.248248+0000","last_clean":"2024-04-25T01:40:49.248248+0000","last_became_active":"2024-04-25T01:25:52.353659+0000","last_became_peered":"2024-04-25T01:25:52.353659+0000","last_unstale":"2024-04-25T01:40:49.248248+0000","last_undegraded":"2024-04-25T01:40:49.248248+0000","last_fullsized":"2024-04-25T01:40:49.248248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":603,"num_write_kb":4239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"26'4191","reported_seq":4221,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.749061+0000","last_change":"2024-04-25T01:25:53.366586+0000","last_active":"2024-04-25T01:40:58.749061+0000","last_peered":"2024-04-25T01:40:58.749061+0000","last_clean":"2024-04-25T01:40:58.749061+0000","last_became_active":"2024-04-25T01:25:53.366302+0000","last_became_peered":"2024-04-25T01:25:53.366302+0000","last_unstale":"2024-04-25T01:40:58.749061+0000","last_undegraded":"2024-04-25T01:40:58.749061+0000","last_fullsized":"2024-04-25T01:40:58.749061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4191,"log_dups_size":0,"ondisk_log_size":4191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21716,"num_write_kb":263159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"26'11","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.173685+0000","last_change":"2024-04-25T01:26:01.420960+0000","last_active":"2024-04-25T01:40:37.173685+0000","last_peered":"2024-04-25T01:40:37.173685+0000","last_clean":"2024-04-25T01:40:37.173685+0000","last_became_active":"2024-04-25T01:26:01.420713+0000","last_became_peered":"2024-04-25T01:26:01.420713+0000","last_unstale":"2024-04-25T01:40:37.173685+0000","last_undegraded":"2024-04-25T01:40:37.173685+0000","last_fullsized":"2024-04-25T01:40:37.173685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:29.489275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"26'4569","reported_seq":4591,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.350310+0000","last_change":"2024-04-25T01:26:02.419282+0000","last_active":"2024-04-25T01:41:09.350310+0000","last_peered":"2024-04-25T01:41:09.350310+0000","last_clean":"2024-04-25T01:41:09.350310+0000","last_became_active":"2024-04-25T01:26:02.418802+0000","last_became_peered":"2024-04-25T01:26:02.418802+0000","last_unstale":"2024-04-25T01:41:09.350310+0000","last_undegraded":"2024-04-25T01:41:09.350310+0000","last_fullsized":"2024-04-25T01:41:09.350310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4569,"log_dups_size":0,"ondisk_log_size":4569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:46.807175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23161,"num_write_kb":277521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"26'6240","reported_seq":6271,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.802380+0000","last_change":"2024-04-25T01:25:53.367143+0000","last_active":"2024-04-25T01:40:58.802380+0000","last_peered":"2024-04-25T01:40:58.802380+0000","last_clean":"2024-04-25T01:40:58.802380+0000","last_became_active":"2024-04-25T01:25:53.366847+0000","last_became_peered":"2024-04-25T01:25:53.366847+0000","last_unstale":"2024-04-25T01:40:58.802380+0000","last_undegraded":"2024-04-25T01:40:58.802380+0000","last_fullsized":"2024-04-25T01:40:58.802380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":6240,"log_dups_size":0,"ondisk_log_size":6240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32134,"num_write_kb":382834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"26'5302","reported_seq":5324,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.305354+0000","last_change":"2024-04-25T01:26:02.416438+0000","last_active":"2024-04-25T01:41:09.305354+0000","last_peered":"2024-04-25T01:41:09.305354+0000","last_clean":"2024-04-25T01:41:09.305354+0000","last_became_active":"2024-04-25T01:26:02.416170+0000","last_became_peered":"2024-04-25T01:26:02.416170+0000","last_unstale":"2024-04-25T01:41:09.305354+0000","last_undegraded":"2024-04-25T01:41:09.305354+0000","last_fullsized":"2024-04-25T01:41:09.305354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5302,"log_dups_size":0,"ondisk_log_size":5302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27000,"num_write_kb":345927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"26'4854","reported_seq":4885,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.807931+0000","last_change":"2024-04-25T01:25:53.371345+0000","last_active":"2024-04-25T01:40:58.807931+0000","last_peered":"2024-04-25T01:40:58.807931+0000","last_clean":"2024-04-25T01:40:58.807931+0000","last_became_active":"2024-04-25T01:25:53.371080+0000","last_became_peered":"2024-04-25T01:25:53.371080+0000","last_unstale":"2024-04-25T01:40:58.807931+0000","last_undegraded":"2024-04-25T01:40:58.807931+0000","last_fullsized":"2024-04-25T01:40:58.807931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4854,"log_dups_size":0,"ondisk_log_size":4854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:09.855763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24922,"num_write_kb":316898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"26'4098","reported_seq":4121,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.322931+0000","last_change":"2024-04-25T01:26:02.419688+0000","last_active":"2024-04-25T01:41:09.322931+0000","last_peered":"2024-04-25T01:41:09.322931+0000","last_clean":"2024-04-25T01:41:09.322931+0000","last_became_active":"2024-04-25T01:26:02.419417+0000","last_became_peered":"2024-04-25T01:26:02.419417+0000","last_unstale":"2024-04-25T01:41:09.322931+0000","last_undegraded":"2024-04-25T01:41:09.322931+0000","last_fullsized":"2024-04-25T01:41:09.322931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4098,"log_dups_size":0,"ondisk_log_size":4098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21368,"num_write_kb":269972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"26'5565","reported_seq":5596,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.839128+0000","last_change":"2024-04-25T01:25:53.369850+0000","last_active":"2024-04-25T01:40:58.839128+0000","last_peered":"2024-04-25T01:40:58.839128+0000","last_clean":"2024-04-25T01:40:58.839128+0000","last_became_active":"2024-04-25T01:25:53.369350+0000","last_became_peered":"2024-04-25T01:25:53.369350+0000","last_unstale":"2024-04-25T01:40:58.839128+0000","last_undegraded":"2024-04-25T01:40:58.839128+0000","last_fullsized":"2024-04-25T01:40:58.839128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5565,"log_dups_size":0,"ondisk_log_size":5565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29144,"num_write_kb":362518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"26'5067","reported_seq":5089,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.356968+0000","last_change":"2024-04-25T01:26:02.425528+0000","last_active":"2024-04-25T01:41:09.356968+0000","last_peered":"2024-04-25T01:41:09.356968+0000","last_clean":"2024-04-25T01:41:09.356968+0000","last_became_active":"2024-04-25T01:26:02.425160+0000","last_became_peered":"2024-04-25T01:26:02.425160+0000","last_unstale":"2024-04-25T01:41:09.356968+0000","last_undegraded":"2024-04-25T01:41:09.356968+0000","last_fullsized":"2024-04-25T01:41:09.356968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26737,"num_write_kb":324333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"26'3787","reported_seq":3818,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.775758+0000","last_change":"2024-04-25T01:25:53.357142+0000","last_active":"2024-04-25T01:40:58.775758+0000","last_peered":"2024-04-25T01:40:58.775758+0000","last_clean":"2024-04-25T01:40:58.775758+0000","last_became_active":"2024-04-25T01:25:53.356854+0000","last_became_peered":"2024-04-25T01:25:53.356854+0000","last_unstale":"2024-04-25T01:40:58.775758+0000","last_undegraded":"2024-04-25T01:40:58.775758+0000","last_fullsized":"2024-04-25T01:40:58.775758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3787,"log_dups_size":0,"ondisk_log_size":3787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19900,"num_write_kb":255902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"26'5481","reported_seq":5504,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.289690+0000","last_change":"2024-04-25T01:26:02.417492+0000","last_active":"2024-04-25T01:41:09.289690+0000","last_peered":"2024-04-25T01:41:09.289690+0000","last_clean":"2024-04-25T01:41:09.289690+0000","last_became_active":"2024-04-25T01:26:02.417234+0000","last_became_peered":"2024-04-25T01:26:02.417234+0000","last_unstale":"2024-04-25T01:41:09.289690+0000","last_undegraded":"2024-04-25T01:41:09.289690+0000","last_fullsized":"2024-04-25T01:41:09.289690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5481,"log_dups_size":0,"ondisk_log_size":5481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28430,"num_write_kb":359662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"26'4160","reported_seq":4191,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.791055+0000","last_change":"2024-04-25T01:25:53.364575+0000","last_active":"2024-04-25T01:40:58.791055+0000","last_peered":"2024-04-25T01:40:58.791055+0000","last_clean":"2024-04-25T01:40:58.791055+0000","last_became_active":"2024-04-25T01:25:53.364311+0000","last_became_peered":"2024-04-25T01:25:53.364311+0000","last_unstale":"2024-04-25T01:40:58.791055+0000","last_undegraded":"2024-04-25T01:40:58.791055+0000","last_fullsized":"2024-04-25T01:40:58.791055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4160,"log_dups_size":0,"ondisk_log_size":4160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21579,"num_write_kb":270844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"26'4125","reported_seq":4148,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.329055+0000","last_change":"2024-04-25T01:26:02.426813+0000","last_active":"2024-04-25T01:41:09.329055+0000","last_peered":"2024-04-25T01:41:09.329055+0000","last_clean":"2024-04-25T01:41:09.329055+0000","last_became_active":"2024-04-25T01:26:02.426552+0000","last_became_peered":"2024-04-25T01:26:02.426552+0000","last_unstale":"2024-04-25T01:41:09.329055+0000","last_undegraded":"2024-04-25T01:41:09.329055+0000","last_fullsized":"2024-04-25T01:41:09.329055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4125,"log_dups_size":0,"ondisk_log_size":4125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:03.687174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21785,"num_write_kb":263458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"26'5101","reported_seq":5132,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.740912+0000","last_change":"2024-04-25T01:25:53.368413+0000","last_active":"2024-04-25T01:40:58.740912+0000","last_peered":"2024-04-25T01:40:58.740912+0000","last_clean":"2024-04-25T01:40:58.740912+0000","last_became_active":"2024-04-25T01:25:53.368176+0000","last_became_peered":"2024-04-25T01:25:53.368176+0000","last_unstale":"2024-04-25T01:40:58.740912+0000","last_undegraded":"2024-04-25T01:40:58.740912+0000","last_fullsized":"2024-04-25T01:40:58.740912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5101,"log_dups_size":0,"ondisk_log_size":5101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26850,"num_write_kb":324721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"26'6020","reported_seq":6043,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.350276+0000","last_change":"2024-04-25T01:26:02.425185+0000","last_active":"2024-04-25T01:41:09.350276+0000","last_peered":"2024-04-25T01:41:09.350276+0000","last_clean":"2024-04-25T01:41:09.350276+0000","last_became_active":"2024-04-25T01:26:02.424903+0000","last_became_peered":"2024-04-25T01:26:02.424903+0000","last_unstale":"2024-04-25T01:41:09.350276+0000","last_undegraded":"2024-04-25T01:41:09.350276+0000","last_fullsized":"2024-04-25T01:41:09.350276+0000","mapping_epoch":22,"log_start":"26'300","ondisk_log_start":"26'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5720,"log_dups_size":0,"ondisk_log_size":5720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31332,"num_write_kb":379614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"26'4058","reported_seq":4089,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.867006+0000","last_change":"2024-04-25T01:25:53.372638+0000","last_active":"2024-04-25T01:40:58.867006+0000","last_peered":"2024-04-25T01:40:58.867006+0000","last_clean":"2024-04-25T01:40:58.867006+0000","last_became_active":"2024-04-25T01:25:53.372001+0000","last_became_peered":"2024-04-25T01:25:53.372001+0000","last_unstale":"2024-04-25T01:40:58.867006+0000","last_undegraded":"2024-04-25T01:40:58.867006+0000","last_fullsized":"2024-04-25T01:40:58.867006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4058,"log_dups_size":0,"ondisk_log_size":4058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20502,"num_write_kb":266785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"26'4843","reported_seq":4866,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.307262+0000","last_change":"2024-04-25T01:26:02.420867+0000","last_active":"2024-04-25T01:41:09.307262+0000","last_peered":"2024-04-25T01:41:09.307262+0000","last_clean":"2024-04-25T01:41:09.307262+0000","last_became_active":"2024-04-25T01:26:02.420621+0000","last_became_peered":"2024-04-25T01:26:02.420621+0000","last_unstale":"2024-04-25T01:41:09.307262+0000","last_undegraded":"2024-04-25T01:41:09.307262+0000","last_fullsized":"2024-04-25T01:41:09.307262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4843,"log_dups_size":0,"ondisk_log_size":4843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:11.493181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24777,"num_write_kb":316390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"26'5268","reported_seq":5299,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.743440+0000","last_change":"2024-04-25T01:25:53.365509+0000","last_active":"2024-04-25T01:40:58.743440+0000","last_peered":"2024-04-25T01:40:58.743440+0000","last_clean":"2024-04-25T01:40:58.743440+0000","last_became_active":"2024-04-25T01:25:53.365245+0000","last_became_peered":"2024-04-25T01:25:53.365245+0000","last_unstale":"2024-04-25T01:40:58.743440+0000","last_undegraded":"2024-04-25T01:40:58.743440+0000","last_fullsized":"2024-04-25T01:40:58.743440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5268,"log_dups_size":0,"ondisk_log_size":5268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26981,"num_write_kb":345791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"26'3558","reported_seq":3580,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.356542+0000","last_change":"2024-04-25T01:26:02.415672+0000","last_active":"2024-04-25T01:41:09.356542+0000","last_peered":"2024-04-25T01:41:09.356542+0000","last_clean":"2024-04-25T01:41:09.356542+0000","last_became_active":"2024-04-25T01:26:02.415389+0000","last_became_peered":"2024-04-25T01:26:02.415389+0000","last_unstale":"2024-04-25T01:41:09.356542+0000","last_undegraded":"2024-04-25T01:41:09.356542+0000","last_fullsized":"2024-04-25T01:41:09.356542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3558,"log_dups_size":0,"ondisk_log_size":3558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:52.342053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18097,"num_write_kb":236200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"26'4040","reported_seq":4071,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.818105+0000","last_change":"2024-04-25T01:25:53.365391+0000","last_active":"2024-04-25T01:40:58.818105+0000","last_peered":"2024-04-25T01:40:58.818105+0000","last_clean":"2024-04-25T01:40:58.818105+0000","last_became_active":"2024-04-25T01:25:53.364829+0000","last_became_peered":"2024-04-25T01:25:53.364829+0000","last_unstale":"2024-04-25T01:40:58.818105+0000","last_undegraded":"2024-04-25T01:40:58.818105+0000","last_fullsized":"2024-04-25T01:40:58.818105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4040,"log_dups_size":0,"ondisk_log_size":4040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20607,"num_write_kb":262822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"26'5431","reported_seq":5453,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.318622+0000","last_change":"2024-04-25T01:26:02.418292+0000","last_active":"2024-04-25T01:41:09.318622+0000","last_peered":"2024-04-25T01:41:09.318622+0000","last_clean":"2024-04-25T01:41:09.318622+0000","last_became_active":"2024-04-25T01:26:02.418036+0000","last_became_peered":"2024-04-25T01:26:02.418036+0000","last_unstale":"2024-04-25T01:41:09.318622+0000","last_undegraded":"2024-04-25T01:41:09.318622+0000","last_fullsized":"2024-04-25T01:41:09.318622+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5431,"log_dups_size":0,"ondisk_log_size":5431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28059,"num_write_kb":358266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"26'4709","reported_seq":4739,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.812514+0000","last_change":"2024-04-25T01:25:53.355667+0000","last_active":"2024-04-25T01:40:58.812514+0000","last_peered":"2024-04-25T01:40:58.812514+0000","last_clean":"2024-04-25T01:40:58.812514+0000","last_became_active":"2024-04-25T01:25:53.355417+0000","last_became_peered":"2024-04-25T01:25:53.355417+0000","last_unstale":"2024-04-25T01:40:58.812514+0000","last_undegraded":"2024-04-25T01:40:58.812514+0000","last_fullsized":"2024-04-25T01:40:58.812514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4709,"log_dups_size":0,"ondisk_log_size":4709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24394,"num_write_kb":298399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"26'4741","reported_seq":4763,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.305268+0000","last_change":"2024-04-25T01:26:02.414994+0000","last_active":"2024-04-25T01:41:09.305268+0000","last_peered":"2024-04-25T01:41:09.305268+0000","last_clean":"2024-04-25T01:41:09.305268+0000","last_became_active":"2024-04-25T01:26:02.414735+0000","last_became_peered":"2024-04-25T01:26:02.414735+0000","last_unstale":"2024-04-25T01:41:09.305268+0000","last_undegraded":"2024-04-25T01:41:09.305268+0000","last_fullsized":"2024-04-25T01:41:09.305268+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4741,"log_dups_size":0,"ondisk_log_size":4741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:44.960251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24824,"num_write_kb":300285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"26'5111","reported_seq":5141,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.869446+0000","last_change":"2024-04-25T01:25:53.371240+0000","last_active":"2024-04-25T01:40:58.869446+0000","last_peered":"2024-04-25T01:40:58.869446+0000","last_clean":"2024-04-25T01:40:58.869446+0000","last_became_active":"2024-04-25T01:25:53.370164+0000","last_became_peered":"2024-04-25T01:25:53.370164+0000","last_unstale":"2024-04-25T01:40:58.869446+0000","last_undegraded":"2024-04-25T01:40:58.869446+0000","last_fullsized":"2024-04-25T01:40:58.869446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26542,"num_write_kb":327744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"26'4367","reported_seq":4389,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.310155+0000","last_change":"2024-04-25T01:26:02.418272+0000","last_active":"2024-04-25T01:41:09.310155+0000","last_peered":"2024-04-25T01:41:09.310155+0000","last_clean":"2024-04-25T01:41:09.310155+0000","last_became_active":"2024-04-25T01:26:02.417977+0000","last_became_peered":"2024-04-25T01:26:02.417977+0000","last_unstale":"2024-04-25T01:41:09.310155+0000","last_undegraded":"2024-04-25T01:41:09.310155+0000","last_fullsized":"2024-04-25T01:41:09.310155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4367,"log_dups_size":0,"ondisk_log_size":4367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22024,"num_write_kb":293008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"26'5285","reported_seq":5315,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.820507+0000","last_change":"2024-04-25T01:25:53.376415+0000","last_active":"2024-04-25T01:40:58.820507+0000","last_peered":"2024-04-25T01:40:58.820507+0000","last_clean":"2024-04-25T01:40:58.820507+0000","last_became_active":"2024-04-25T01:25:53.376104+0000","last_became_peered":"2024-04-25T01:25:53.376104+0000","last_unstale":"2024-04-25T01:40:58.820507+0000","last_undegraded":"2024-04-25T01:40:58.820507+0000","last_fullsized":"2024-04-25T01:40:58.820507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5285,"log_dups_size":0,"ondisk_log_size":5285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27331,"num_write_kb":318531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"26'5020","reported_seq":5042,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.344122+0000","last_change":"2024-04-25T01:26:02.426333+0000","last_active":"2024-04-25T01:41:09.344122+0000","last_peered":"2024-04-25T01:41:09.344122+0000","last_clean":"2024-04-25T01:41:09.344122+0000","last_became_active":"2024-04-25T01:26:02.426069+0000","last_became_peered":"2024-04-25T01:26:02.426069+0000","last_unstale":"2024-04-25T01:41:09.344122+0000","last_undegraded":"2024-04-25T01:41:09.344122+0000","last_fullsized":"2024-04-25T01:41:09.344122+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5020,"log_dups_size":0,"ondisk_log_size":5020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26121,"num_write_kb":325976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"26'4910","reported_seq":4940,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.774391+0000","last_change":"2024-04-25T01:25:53.370314+0000","last_active":"2024-04-25T01:40:58.774391+0000","last_peered":"2024-04-25T01:40:58.774391+0000","last_clean":"2024-04-25T01:40:58.774391+0000","last_became_active":"2024-04-25T01:25:53.370049+0000","last_became_peered":"2024-04-25T01:25:53.370049+0000","last_unstale":"2024-04-25T01:40:58.774391+0000","last_undegraded":"2024-04-25T01:40:58.774391+0000","last_fullsized":"2024-04-25T01:40:58.774391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4910,"log_dups_size":0,"ondisk_log_size":4910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25833,"num_write_kb":304449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"26'4775","reported_seq":4797,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.332568+0000","last_change":"2024-04-25T01:26:02.415464+0000","last_active":"2024-04-25T01:41:09.332568+0000","last_peered":"2024-04-25T01:41:09.332568+0000","last_clean":"2024-04-25T01:41:09.332568+0000","last_became_active":"2024-04-25T01:26:02.415173+0000","last_became_peered":"2024-04-25T01:26:02.415173+0000","last_unstale":"2024-04-25T01:41:09.332568+0000","last_undegraded":"2024-04-25T01:41:09.332568+0000","last_fullsized":"2024-04-25T01:41:09.332568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4775,"log_dups_size":0,"ondisk_log_size":4775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.343169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24075,"num_write_kb":305295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"26'4891","reported_seq":4921,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.797939+0000","last_change":"2024-04-25T01:25:53.369585+0000","last_active":"2024-04-25T01:40:58.797939+0000","last_peered":"2024-04-25T01:40:58.797939+0000","last_clean":"2024-04-25T01:40:58.797939+0000","last_became_active":"2024-04-25T01:25:53.368499+0000","last_became_peered":"2024-04-25T01:25:53.368499+0000","last_unstale":"2024-04-25T01:40:58.797939+0000","last_undegraded":"2024-04-25T01:40:58.797939+0000","last_fullsized":"2024-04-25T01:40:58.797939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4891,"log_dups_size":0,"ondisk_log_size":4891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24867,"num_write_kb":304528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"26'4199","reported_seq":4221,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.286525+0000","last_change":"2024-04-25T01:26:02.421975+0000","last_active":"2024-04-25T01:41:09.286525+0000","last_peered":"2024-04-25T01:41:09.286525+0000","last_clean":"2024-04-25T01:41:09.286525+0000","last_became_active":"2024-04-25T01:26:02.421698+0000","last_became_peered":"2024-04-25T01:26:02.421698+0000","last_unstale":"2024-04-25T01:41:09.286525+0000","last_undegraded":"2024-04-25T01:41:09.286525+0000","last_fullsized":"2024-04-25T01:41:09.286525+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4199,"log_dups_size":0,"ondisk_log_size":4199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21738,"num_write_kb":267270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"26'3844","reported_seq":3874,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.786269+0000","last_change":"2024-04-25T01:25:53.370666+0000","last_active":"2024-04-25T01:40:58.786269+0000","last_peered":"2024-04-25T01:40:58.786269+0000","last_clean":"2024-04-25T01:40:58.786269+0000","last_became_active":"2024-04-25T01:25:53.369733+0000","last_became_peered":"2024-04-25T01:25:53.369733+0000","last_unstale":"2024-04-25T01:40:58.786269+0000","last_undegraded":"2024-04-25T01:40:58.786269+0000","last_fullsized":"2024-04-25T01:40:58.786269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3844,"log_dups_size":0,"ondisk_log_size":3844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19367,"num_write_kb":241344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"26'4316","reported_seq":4338,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.253195+0000","last_change":"2024-04-25T01:26:02.418114+0000","last_active":"2024-04-25T01:41:09.253195+0000","last_peered":"2024-04-25T01:41:09.253195+0000","last_clean":"2024-04-25T01:41:09.253195+0000","last_became_active":"2024-04-25T01:26:02.417810+0000","last_became_peered":"2024-04-25T01:26:02.417810+0000","last_unstale":"2024-04-25T01:41:09.253195+0000","last_undegraded":"2024-04-25T01:41:09.253195+0000","last_fullsized":"2024-04-25T01:41:09.253195+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4316,"log_dups_size":0,"ondisk_log_size":4316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:30.496089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22517,"num_write_kb":290939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"26'5814","reported_seq":5830,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.796893+0000","last_change":"2024-04-25T01:26:52.120824+0000","last_active":"2024-04-25T01:40:58.796893+0000","last_peered":"2024-04-25T01:40:58.796893+0000","last_clean":"2024-04-25T01:40:58.796893+0000","last_became_active":"2024-04-25T01:26:52.120588+0000","last_became_peered":"2024-04-25T01:26:52.120588+0000","last_unstale":"2024-04-25T01:40:58.796893+0000","last_undegraded":"2024-04-25T01:40:58.796893+0000","last_fullsized":"2024-04-25T01:40:58.796893+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5814,"log_dups_size":0,"ondisk_log_size":5814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:14.466644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29929,"num_write_kb":365779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.36","version":"26'4743","reported_seq":4765,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.335933+0000","last_change":"2024-04-25T01:26:02.426289+0000","last_active":"2024-04-25T01:41:09.335933+0000","last_peered":"2024-04-25T01:41:09.335933+0000","last_clean":"2024-04-25T01:41:09.335933+0000","last_became_active":"2024-04-25T01:26:02.426015+0000","last_became_peered":"2024-04-25T01:26:02.426015+0000","last_unstale":"2024-04-25T01:41:09.335933+0000","last_undegraded":"2024-04-25T01:41:09.335933+0000","last_fullsized":"2024-04-25T01:41:09.335933+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4743,"log_dups_size":0,"ondisk_log_size":4743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25470,"num_write_kb":331505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"26'4366","reported_seq":4396,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.821883+0000","last_change":"2024-04-25T01:25:53.374186+0000","last_active":"2024-04-25T01:40:58.821883+0000","last_peered":"2024-04-25T01:40:58.821883+0000","last_clean":"2024-04-25T01:40:58.821883+0000","last_became_active":"2024-04-25T01:25:53.373766+0000","last_became_peered":"2024-04-25T01:25:53.373766+0000","last_unstale":"2024-04-25T01:40:58.821883+0000","last_undegraded":"2024-04-25T01:40:58.821883+0000","last_fullsized":"2024-04-25T01:40:58.821883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4366,"log_dups_size":0,"ondisk_log_size":4366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23328,"num_write_kb":273725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"26'5875","reported_seq":5897,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.356318+0000","last_change":"2024-04-25T01:26:02.416345+0000","last_active":"2024-04-25T01:41:09.356318+0000","last_peered":"2024-04-25T01:41:09.356318+0000","last_clean":"2024-04-25T01:41:09.356318+0000","last_became_active":"2024-04-25T01:26:02.416082+0000","last_became_peered":"2024-04-25T01:26:02.416082+0000","last_unstale":"2024-04-25T01:41:09.356318+0000","last_undegraded":"2024-04-25T01:41:09.356318+0000","last_fullsized":"2024-04-25T01:41:09.356318+0000","mapping_epoch":22,"log_start":"26'400","ondisk_log_start":"26'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5475,"log_dups_size":0,"ondisk_log_size":5475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:47.810943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31009,"num_write_kb":370059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"26'4898","reported_seq":4928,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.812217+0000","last_change":"2024-04-25T01:25:53.371602+0000","last_active":"2024-04-25T01:40:58.812217+0000","last_peered":"2024-04-25T01:40:58.812217+0000","last_clean":"2024-04-25T01:40:58.812217+0000","last_became_active":"2024-04-25T01:25:53.371362+0000","last_became_peered":"2024-04-25T01:25:53.371362+0000","last_unstale":"2024-04-25T01:40:58.812217+0000","last_undegraded":"2024-04-25T01:40:58.812217+0000","last_fullsized":"2024-04-25T01:40:58.812217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4898,"log_dups_size":0,"ondisk_log_size":4898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24919,"num_write_kb":308746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"26'3961","reported_seq":3983,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.316956+0000","last_change":"2024-04-25T01:26:02.425888+0000","last_active":"2024-04-25T01:41:09.316956+0000","last_peered":"2024-04-25T01:41:09.316956+0000","last_clean":"2024-04-25T01:41:09.316956+0000","last_became_active":"2024-04-25T01:26:02.425577+0000","last_became_peered":"2024-04-25T01:26:02.425577+0000","last_unstale":"2024-04-25T01:41:09.316956+0000","last_undegraded":"2024-04-25T01:41:09.316956+0000","last_fullsized":"2024-04-25T01:41:09.316956+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3961,"log_dups_size":0,"ondisk_log_size":3961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20296,"num_write_kb":253395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"26'4670","reported_seq":4700,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.788790+0000","last_change":"2024-04-25T01:25:53.370996+0000","last_active":"2024-04-25T01:40:58.788790+0000","last_peered":"2024-04-25T01:40:58.788790+0000","last_clean":"2024-04-25T01:40:58.788790+0000","last_became_active":"2024-04-25T01:25:53.370479+0000","last_became_peered":"2024-04-25T01:25:53.370479+0000","last_unstale":"2024-04-25T01:40:58.788790+0000","last_undegraded":"2024-04-25T01:40:58.788790+0000","last_fullsized":"2024-04-25T01:40:58.788790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4670,"log_dups_size":0,"ondisk_log_size":4670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:57.636067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24421,"num_write_kb":306681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"26'4733","reported_seq":4755,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.275948+0000","last_change":"2024-04-25T01:26:02.423420+0000","last_active":"2024-04-25T01:41:09.275948+0000","last_peered":"2024-04-25T01:41:09.275948+0000","last_clean":"2024-04-25T01:41:09.275948+0000","last_became_active":"2024-04-25T01:26:02.423146+0000","last_became_peered":"2024-04-25T01:26:02.423146+0000","last_unstale":"2024-04-25T01:41:09.275948+0000","last_undegraded":"2024-04-25T01:41:09.275948+0000","last_fullsized":"2024-04-25T01:41:09.275948+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4733,"log_dups_size":0,"ondisk_log_size":4733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23928,"num_write_kb":292515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"26'4513","reported_seq":4543,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.807710+0000","last_change":"2024-04-25T01:25:53.363512+0000","last_active":"2024-04-25T01:40:58.807710+0000","last_peered":"2024-04-25T01:40:58.807710+0000","last_clean":"2024-04-25T01:40:58.807710+0000","last_became_active":"2024-04-25T01:25:53.363246+0000","last_became_peered":"2024-04-25T01:25:53.363246+0000","last_unstale":"2024-04-25T01:40:58.807710+0000","last_undegraded":"2024-04-25T01:40:58.807710+0000","last_fullsized":"2024-04-25T01:40:58.807710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4513,"log_dups_size":0,"ondisk_log_size":4513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23705,"num_write_kb":279339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"26'4587","reported_seq":4609,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.306205+0000","last_change":"2024-04-25T01:26:02.413346+0000","last_active":"2024-04-25T01:41:09.306205+0000","last_peered":"2024-04-25T01:41:09.306205+0000","last_clean":"2024-04-25T01:41:09.306205+0000","last_became_active":"2024-04-25T01:26:02.413036+0000","last_became_peered":"2024-04-25T01:26:02.413036+0000","last_unstale":"2024-04-25T01:41:09.306205+0000","last_undegraded":"2024-04-25T01:41:09.306205+0000","last_fullsized":"2024-04-25T01:41:09.306205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4587,"log_dups_size":0,"ondisk_log_size":4587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24042,"num_write_kb":305325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"26'3998","reported_seq":4028,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.876422+0000","last_change":"2024-04-25T01:25:53.364457+0000","last_active":"2024-04-25T01:40:58.876422+0000","last_peered":"2024-04-25T01:40:58.876422+0000","last_clean":"2024-04-25T01:40:58.876422+0000","last_became_active":"2024-04-25T01:25:53.364103+0000","last_became_peered":"2024-04-25T01:25:53.364103+0000","last_unstale":"2024-04-25T01:40:58.876422+0000","last_undegraded":"2024-04-25T01:40:58.876422+0000","last_fullsized":"2024-04-25T01:40:58.876422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3998,"log_dups_size":0,"ondisk_log_size":3998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:11.336752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20343,"num_write_kb":253659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"26'4194","reported_seq":4216,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.298446+0000","last_change":"2024-04-25T01:26:02.418792+0000","last_active":"2024-04-25T01:41:09.298446+0000","last_peered":"2024-04-25T01:41:09.298446+0000","last_clean":"2024-04-25T01:41:09.298446+0000","last_became_active":"2024-04-25T01:26:02.416942+0000","last_became_peered":"2024-04-25T01:26:02.416942+0000","last_unstale":"2024-04-25T01:41:09.298446+0000","last_undegraded":"2024-04-25T01:41:09.298446+0000","last_fullsized":"2024-04-25T01:41:09.298446+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4194,"log_dups_size":0,"ondisk_log_size":4194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:59.492678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21868,"num_write_kb":272015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"26'4800","reported_seq":4830,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.771199+0000","last_change":"2024-04-25T01:25:53.371647+0000","last_active":"2024-04-25T01:40:58.771199+0000","last_peered":"2024-04-25T01:40:58.771199+0000","last_clean":"2024-04-25T01:40:58.771199+0000","last_became_active":"2024-04-25T01:25:53.371353+0000","last_became_peered":"2024-04-25T01:25:53.371353+0000","last_unstale":"2024-04-25T01:40:58.771199+0000","last_undegraded":"2024-04-25T01:40:58.771199+0000","last_fullsized":"2024-04-25T01:40:58.771199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4800,"log_dups_size":0,"ondisk_log_size":4800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24494,"num_write_kb":294847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"26'4139","reported_seq":4161,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.342741+0000","last_change":"2024-04-25T01:26:02.426335+0000","last_active":"2024-04-25T01:41:09.342741+0000","last_peered":"2024-04-25T01:41:09.342741+0000","last_clean":"2024-04-25T01:41:09.342741+0000","last_became_active":"2024-04-25T01:26:02.426024+0000","last_became_peered":"2024-04-25T01:26:02.426024+0000","last_unstale":"2024-04-25T01:41:09.342741+0000","last_undegraded":"2024-04-25T01:41:09.342741+0000","last_fullsized":"2024-04-25T01:41:09.342741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4139,"log_dups_size":0,"ondisk_log_size":4139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21818,"num_write_kb":267665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"26'5798","reported_seq":5828,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.869849+0000","last_change":"2024-04-25T01:25:53.372842+0000","last_active":"2024-04-25T01:40:58.869849+0000","last_peered":"2024-04-25T01:40:58.869849+0000","last_clean":"2024-04-25T01:40:58.869849+0000","last_became_active":"2024-04-25T01:25:53.372580+0000","last_became_peered":"2024-04-25T01:25:53.372580+0000","last_unstale":"2024-04-25T01:40:58.869849+0000","last_undegraded":"2024-04-25T01:40:58.869849+0000","last_fullsized":"2024-04-25T01:40:58.869849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5798,"log_dups_size":0,"ondisk_log_size":5798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30839,"num_write_kb":352989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"26'4687","reported_seq":4709,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.302280+0000","last_change":"2024-04-25T01:26:02.414632+0000","last_active":"2024-04-25T01:41:09.302280+0000","last_peered":"2024-04-25T01:41:09.302280+0000","last_clean":"2024-04-25T01:41:09.302280+0000","last_became_active":"2024-04-25T01:26:02.414388+0000","last_became_peered":"2024-04-25T01:26:02.414388+0000","last_unstale":"2024-04-25T01:41:09.302280+0000","last_undegraded":"2024-04-25T01:41:09.302280+0000","last_fullsized":"2024-04-25T01:41:09.302280+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4687,"log_dups_size":0,"ondisk_log_size":4687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23866,"num_write_kb":304526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"26'5655","reported_seq":5690,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.888926+0000","last_change":"2024-04-25T01:26:52.515807+0000","last_active":"2024-04-25T01:40:58.888926+0000","last_peered":"2024-04-25T01:40:58.888926+0000","last_clean":"2024-04-25T01:40:58.888926+0000","last_became_active":"2024-04-25T01:26:52.515504+0000","last_became_peered":"2024-04-25T01:26:52.515504+0000","last_unstale":"2024-04-25T01:40:58.888926+0000","last_undegraded":"2024-04-25T01:40:58.888926+0000","last_fullsized":"2024-04-25T01:40:58.888926+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5655,"log_dups_size":0,"ondisk_log_size":5655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:48.624287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29809,"num_write_kb":365295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"26'5769","reported_seq":5792,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.330796+0000","last_change":"2024-04-25T01:26:02.414366+0000","last_active":"2024-04-25T01:41:09.330796+0000","last_peered":"2024-04-25T01:41:09.330796+0000","last_clean":"2024-04-25T01:41:09.330796+0000","last_became_active":"2024-04-25T01:26:02.414061+0000","last_became_peered":"2024-04-25T01:26:02.414061+0000","last_unstale":"2024-04-25T01:41:09.330796+0000","last_undegraded":"2024-04-25T01:41:09.330796+0000","last_fullsized":"2024-04-25T01:41:09.330796+0000","mapping_epoch":22,"log_start":"26'500","ondisk_log_start":"26'500","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5269,"log_dups_size":0,"ondisk_log_size":5269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:19.644468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30404,"num_write_kb":379952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"26'5055","reported_seq":5085,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.844665+0000","last_change":"2024-04-25T01:25:53.374061+0000","last_active":"2024-04-25T01:40:58.844665+0000","last_peered":"2024-04-25T01:40:58.844665+0000","last_clean":"2024-04-25T01:40:58.844665+0000","last_became_active":"2024-04-25T01:25:53.373848+0000","last_became_peered":"2024-04-25T01:25:53.373848+0000","last_unstale":"2024-04-25T01:40:58.844665+0000","last_undegraded":"2024-04-25T01:40:58.844665+0000","last_fullsized":"2024-04-25T01:40:58.844665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5055,"log_dups_size":0,"ondisk_log_size":5055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25773,"num_write_kb":324544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"26'4195","reported_seq":4218,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.317465+0000","last_change":"2024-04-25T01:26:02.414620+0000","last_active":"2024-04-25T01:41:09.317465+0000","last_peered":"2024-04-25T01:41:09.317465+0000","last_clean":"2024-04-25T01:41:09.317465+0000","last_became_active":"2024-04-25T01:26:02.414326+0000","last_became_peered":"2024-04-25T01:26:02.414326+0000","last_unstale":"2024-04-25T01:41:09.317465+0000","last_undegraded":"2024-04-25T01:41:09.317465+0000","last_fullsized":"2024-04-25T01:41:09.317465+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4195,"log_dups_size":0,"ondisk_log_size":4195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:30.496089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21944,"num_write_kb":264053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"26'4639","reported_seq":4669,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.755355+0000","last_change":"2024-04-25T01:25:53.364698+0000","last_active":"2024-04-25T01:40:58.755355+0000","last_peered":"2024-04-25T01:40:58.755355+0000","last_clean":"2024-04-25T01:40:58.755355+0000","last_became_active":"2024-04-25T01:25:53.364435+0000","last_became_peered":"2024-04-25T01:25:53.364435+0000","last_unstale":"2024-04-25T01:40:58.755355+0000","last_undegraded":"2024-04-25T01:40:58.755355+0000","last_fullsized":"2024-04-25T01:40:58.755355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4639,"log_dups_size":0,"ondisk_log_size":4639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24873,"num_write_kb":296467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"26'4393","reported_seq":4416,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.341040+0000","last_change":"2024-04-25T01:26:02.418331+0000","last_active":"2024-04-25T01:41:09.341040+0000","last_peered":"2024-04-25T01:41:09.341040+0000","last_clean":"2024-04-25T01:41:09.341040+0000","last_became_active":"2024-04-25T01:26:02.418057+0000","last_became_peered":"2024-04-25T01:26:02.418057+0000","last_unstale":"2024-04-25T01:41:09.341040+0000","last_undegraded":"2024-04-25T01:41:09.341040+0000","last_fullsized":"2024-04-25T01:41:09.341040+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4393,"log_dups_size":0,"ondisk_log_size":4393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:17.809738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22950,"num_write_kb":276425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"26'3744","reported_seq":3759,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.790360+0000","last_change":"2024-04-25T01:26:52.514832+0000","last_active":"2024-04-25T01:40:58.790360+0000","last_peered":"2024-04-25T01:40:58.790360+0000","last_clean":"2024-04-25T01:40:58.790360+0000","last_became_active":"2024-04-25T01:26:52.514260+0000","last_became_peered":"2024-04-25T01:26:52.514260+0000","last_unstale":"2024-04-25T01:40:58.790360+0000","last_undegraded":"2024-04-25T01:40:58.790360+0000","last_fullsized":"2024-04-25T01:40:58.790360+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3744,"log_dups_size":0,"ondisk_log_size":3744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:51.631130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19457,"num_write_kb":254077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3d","version":"26'4696","reported_seq":4719,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.334457+0000","last_change":"2024-04-25T01:26:02.422720+0000","last_active":"2024-04-25T01:41:09.334457+0000","last_peered":"2024-04-25T01:41:09.334457+0000","last_clean":"2024-04-25T01:41:09.334457+0000","last_became_active":"2024-04-25T01:26:02.421848+0000","last_became_peered":"2024-04-25T01:26:02.421848+0000","last_unstale":"2024-04-25T01:41:09.334457+0000","last_undegraded":"2024-04-25T01:41:09.334457+0000","last_fullsized":"2024-04-25T01:41:09.334457+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4696,"log_dups_size":0,"ondisk_log_size":4696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24119,"num_write_kb":305642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"26'5605","reported_seq":5635,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.863402+0000","last_change":"2024-04-25T01:25:53.367953+0000","last_active":"2024-04-25T01:40:58.863402+0000","last_peered":"2024-04-25T01:40:58.863402+0000","last_clean":"2024-04-25T01:40:58.863402+0000","last_became_active":"2024-04-25T01:25:53.367358+0000","last_became_peered":"2024-04-25T01:25:53.367358+0000","last_unstale":"2024-04-25T01:40:58.863402+0000","last_undegraded":"2024-04-25T01:40:58.863402+0000","last_fullsized":"2024-04-25T01:40:58.863402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5605,"log_dups_size":0,"ondisk_log_size":5605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27762,"num_write_kb":352787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"26'4305","reported_seq":4327,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.310344+0000","last_change":"2024-04-25T01:26:02.422300+0000","last_active":"2024-04-25T01:41:09.310344+0000","last_peered":"2024-04-25T01:41:09.310344+0000","last_clean":"2024-04-25T01:41:09.310344+0000","last_became_active":"2024-04-25T01:26:02.421702+0000","last_became_peered":"2024-04-25T01:26:02.421702+0000","last_unstale":"2024-04-25T01:41:09.310344+0000","last_undegraded":"2024-04-25T01:41:09.310344+0000","last_fullsized":"2024-04-25T01:41:09.310344+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4305,"log_dups_size":0,"ondisk_log_size":4305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21701,"num_write_kb":263037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"26'4577","reported_seq":4608,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.784384+0000","last_change":"2024-04-25T01:25:53.371513+0000","last_active":"2024-04-25T01:40:58.784384+0000","last_peered":"2024-04-25T01:40:58.784384+0000","last_clean":"2024-04-25T01:40:58.784384+0000","last_became_active":"2024-04-25T01:25:53.370421+0000","last_became_peered":"2024-04-25T01:25:53.370421+0000","last_unstale":"2024-04-25T01:40:58.784384+0000","last_undegraded":"2024-04-25T01:40:58.784384+0000","last_fullsized":"2024-04-25T01:40:58.784384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24195,"num_write_kb":281461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"26'5562","reported_seq":5584,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.298700+0000","last_change":"2024-04-25T01:26:02.411784+0000","last_active":"2024-04-25T01:41:09.298700+0000","last_peered":"2024-04-25T01:41:09.298700+0000","last_clean":"2024-04-25T01:41:09.298700+0000","last_became_active":"2024-04-25T01:26:02.411478+0000","last_became_peered":"2024-04-25T01:26:02.411478+0000","last_unstale":"2024-04-25T01:41:09.298700+0000","last_undegraded":"2024-04-25T01:41:09.298700+0000","last_fullsized":"2024-04-25T01:41:09.298700+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5562,"log_dups_size":0,"ondisk_log_size":5562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:20.194816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27685,"num_write_kb":352571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"26'5001","reported_seq":5032,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.762988+0000","last_change":"2024-04-25T01:25:53.363829+0000","last_active":"2024-04-25T01:40:58.762988+0000","last_peered":"2024-04-25T01:40:58.762988+0000","last_clean":"2024-04-25T01:40:58.762988+0000","last_became_active":"2024-04-25T01:25:53.363525+0000","last_became_peered":"2024-04-25T01:25:53.363525+0000","last_unstale":"2024-04-25T01:40:58.762988+0000","last_undegraded":"2024-04-25T01:40:58.762988+0000","last_fullsized":"2024-04-25T01:40:58.762988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5001,"log_dups_size":0,"ondisk_log_size":5001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25760,"num_write_kb":312110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"26'5076","reported_seq":5098,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.350154+0000","last_change":"2024-04-25T01:26:02.421208+0000","last_active":"2024-04-25T01:41:09.350154+0000","last_peered":"2024-04-25T01:41:09.350154+0000","last_clean":"2024-04-25T01:41:09.350154+0000","last_became_active":"2024-04-25T01:26:02.418796+0000","last_became_peered":"2024-04-25T01:26:02.418796+0000","last_unstale":"2024-04-25T01:41:09.350154+0000","last_undegraded":"2024-04-25T01:41:09.350154+0000","last_fullsized":"2024-04-25T01:41:09.350154+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5076,"log_dups_size":0,"ondisk_log_size":5076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25596,"num_write_kb":323780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"26'6025","reported_seq":6056,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.748285+0000","last_change":"2024-04-25T01:25:53.365057+0000","last_active":"2024-04-25T01:40:58.748285+0000","last_peered":"2024-04-25T01:40:58.748285+0000","last_clean":"2024-04-25T01:40:58.748285+0000","last_became_active":"2024-04-25T01:25:53.364793+0000","last_became_peered":"2024-04-25T01:25:53.364793+0000","last_unstale":"2024-04-25T01:40:58.748285+0000","last_undegraded":"2024-04-25T01:40:58.748285+0000","last_fullsized":"2024-04-25T01:40:58.748285+0000","mapping_epoch":18,"log_start":"26'200","ondisk_log_start":"26'200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5825,"log_dups_size":0,"ondisk_log_size":5825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31759,"num_write_kb":385508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"26'4493","reported_seq":4515,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.263776+0000","last_change":"2024-04-25T01:26:02.417606+0000","last_active":"2024-04-25T01:41:09.263776+0000","last_peered":"2024-04-25T01:41:09.263776+0000","last_clean":"2024-04-25T01:41:09.263776+0000","last_became_active":"2024-04-25T01:26:02.417350+0000","last_became_peered":"2024-04-25T01:26:02.417350+0000","last_unstale":"2024-04-25T01:41:09.263776+0000","last_undegraded":"2024-04-25T01:41:09.263776+0000","last_fullsized":"2024-04-25T01:41:09.263776+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4493,"log_dups_size":0,"ondisk_log_size":4493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23464,"num_write_kb":290699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"26'4350","reported_seq":4381,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.880219+0000","last_change":"2024-04-25T01:25:53.367762+0000","last_active":"2024-04-25T01:40:58.880219+0000","last_peered":"2024-04-25T01:40:58.880219+0000","last_clean":"2024-04-25T01:40:58.880219+0000","last_became_active":"2024-04-25T01:25:53.367383+0000","last_became_peered":"2024-04-25T01:25:53.367383+0000","last_unstale":"2024-04-25T01:40:58.880219+0000","last_undegraded":"2024-04-25T01:40:58.880219+0000","last_fullsized":"2024-04-25T01:40:58.880219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4350,"log_dups_size":0,"ondisk_log_size":4350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:05.589744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23070,"num_write_kb":268669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1533236,"num_write_kb":19254592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":294168,"ondisk_log_size":294168,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":17605243,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":39,"num_read_kb":112,"num_write":4172,"num_write_kb":18793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35274752,"data_stored":35210486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3499,"ondisk_log_size":3499,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1596536,"num_write_kb":19509893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305948,"ondisk_log_size":305948,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":17847589,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":41,"num_read_kb":115,"num_write":4224,"num_write_kb":19059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35758080,"data_stored":35695178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3536,"ondisk_log_size":3536,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575039,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1217764,"kb_used_data":13148,"kb_used_omap":1,"kb_used_meta":1204606,"kb_avail":93154076,"statfs":{"total":96636764160,"available":95389773824,"internally_reserved":0,"allocated":13463552,"data_stored":13287019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":1233517002},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575037,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":780520,"kb_used_data":9440,"kb_used_omap":1,"kb_used_meta":771070,"kb_avail":93591320,"statfs":{"total":96636764160,"available":95837511680,"internally_reserved":0,"allocated":9666560,"data_stored":9483786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":789576134},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575038,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1059708,"kb_used_data":4468,"kb_used_omap":1,"kb_used_meta":1055230,"kb_avail":93312132,"statfs":{"total":96636764160,"available":95551623168,"internally_reserved":0,"allocated":4575232,"data_stored":4396213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1080555973},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575038,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1089176,"kb_used_data":10256,"kb_used_omap":1,"kb_used_meta":1078910,"kb_avail":93282664,"statfs":{"total":96636764160,"available":95521447936,"internally_reserved":0,"allocated":10502144,"data_stored":10309285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1104804293},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575038,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1161176,"kb_used_data":17488,"kb_used_omap":1,"kb_used_meta":1143678,"kb_avail":93210664,"statfs":{"total":96636764160,"available":95447719936,"internally_reserved":0,"allocated":17907712,"data_stored":17718206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1171126727},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575038,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":729908,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":729534,"kb_avail":93641932,"statfs":{"total":96636764160,"available":95889338368,"internally_reserved":0,"allocated":372736,"data_stored":201217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":747043269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575038,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":888620,"kb_used_data":8484,"kb_used_omap":1,"kb_used_meta":880126,"kb_avail":93483220,"statfs":{"total":96636764160,"available":95726817280,"internally_reserved":0,"allocated":8687616,"data_stored":8502529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":901249477},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575037,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":919656,"kb_used_data":9056,"kb_used_omap":1,"kb_used_meta":910590,"kb_avail":93452184,"statfs":{"total":96636764160,"available":95695036416,"internally_reserved":0,"allocated":9273344,"data_stored":9092681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":932444613},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":88080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8486912,"data_stored":8477360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5189632,"data_stored":5175253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5185536,"data_stored":5175799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8389302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9142272,"data_stored":9127779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5033984,"data_stored":5020965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4292608,"data_stored":4283056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:41:16.710 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:41:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.852+0000 7f3a95d3b640 1 Processor -- start 2024-04-25T01:41:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.853+0000 7f3a95d3b640 1 -- start start 2024-04-25T01:41:16.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.853+0000 7f3a95d3b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3a9014c3f0 0x7f3a900c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.853+0000 7f3a95d3b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a900ac3b0 0x7f3a90155f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.853+0000 7f3a95d3b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a901529b0 con 0x7f3a901524a0 2024-04-25T01:41:16.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.853+0000 7f3a95d3b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3a90155a90 con 0x7f3a9014c3f0 2024-04-25T01:41:16.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.853+0000 7f3a95d3b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3a90152c70 con 0x7f3a900ac3b0 2024-04-25T01:41:16.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.854+0000 7f3a8effd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a900ac3b0 0x7f3a90155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.854+0000 7f3a8f7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3a9014c3f0 0x7f3a900c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.854+0000 7f3a8effd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a900ac3b0 0x7f3a90155f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:34626/0 (socket says 172.21.15.112:34626) 2024-04-25T01:41:16.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.854+0000 7f3a8f7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3a9014c3f0 0x7f3a900c2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43826/0 (socket says 172.21.15.112:43826) 2024-04-25T01:41:16.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.854+0000 7f3a8effd640 1 -- 172.21.15.112:0/54630537 learned_addr learned my addr 172.21.15.112:0/54630537 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:16.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.854+0000 7f3a8effd640 1 -- 172.21.15.112:0/54630537 >> v2:172.21.15.187:3300/0 conn(0x7f3a9014c3f0 msgr2=0x7f3a900c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:16.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.854+0000 7f3a8effd640 1 --2- 172.21.15.112:0/54630537 >> v2:172.21.15.187:3300/0 conn(0x7f3a9014c3f0 0x7f3a900c2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.854+0000 7f3a8effd640 1 -- 172.21.15.112:0/54630537 >> v1:172.21.15.187:6789/0 conn(0x7f3a901524a0 legacy=0x7f3a900c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:16.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.854+0000 7f3a8effd640 1 -- 172.21.15.112:0/54630537 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a90152f50 con 0x7f3a900ac3b0 2024-04-25T01:41:16.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.855+0000 7f3a8effd640 1 --2- 172.21.15.112:0/54630537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a900ac3b0 0x7f3a90155f20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009870 tx=0x7f3a7802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1102ac8d49129c81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:16.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.855+0000 7f3a8e7fc640 1 -- 172.21.15.112:0/54630537 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78037070 con 0x7f3a900ac3b0 2024-04-25T01:41:16.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.855+0000 7f3a8e7fc640 1 -- 172.21.15.112:0/54630537 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a78032440 con 0x7f3a900ac3b0 2024-04-25T01:41:16.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.855+0000 7f3a8e7fc640 1 -- 172.21.15.112:0/54630537 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78032810 con 0x7f3a900ac3b0 2024-04-25T01:41:16.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.855+0000 7f3a95d3b640 1 -- 172.21.15.112:0/54630537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a900ac3b0 msgr2=0x7f3a90155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:16.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.855+0000 7f3a95d3b640 1 --2- 172.21.15.112:0/54630537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a900ac3b0 0x7f3a90155f20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009870 tx=0x7f3a7802a1c0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.855+0000 7f3a95d3b640 1 -- 172.21.15.112:0/54630537 shutdown_connections 2024-04-25T01:41:16.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.855+0000 7f3a95d3b640 1 --2- 172.21.15.112:0/54630537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a900ac3b0 0x7f3a90155f20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.855+0000 7f3a95d3b640 1 --2- 172.21.15.112:0/54630537 >> v2:172.21.15.187:3300/0 conn(0x7f3a9014c3f0 0x7f3a900c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.855+0000 7f3a95d3b640 1 -- 172.21.15.112:0/54630537 >> 172.21.15.112:0/54630537 conn(0x7f3a9006b160 msgr2=0x7f3a9006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:16.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.855+0000 7f3a95d3b640 1 -- 172.21.15.112:0/54630537 shutdown_connections 2024-04-25T01:41:16.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.856+0000 7f3a95d3b640 1 -- 172.21.15.112:0/54630537 wait complete. 2024-04-25T01:41:16.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.856+0000 7f3a95d3b640 1 Processor -- start 2024-04-25T01:41:16.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.857+0000 7f3a95d3b640 1 -- start start 2024-04-25T01:41:16.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.857+0000 7f3a95d3b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a900ac3b0 0x7f3a90144270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.857+0000 7f3a95d3b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a9014c3f0 0x7f3a901428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.857+0000 7f3a95d3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a901524a0 0x7f3a90142e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.857+0000 7f3a95d3b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9014c7d0 con 0x7f3a901524a0 2024-04-25T01:41:16.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.857+0000 7f3a95d3b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3a90070d60 con 0x7f3a900ac3b0 2024-04-25T01:41:16.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.857+0000 7f3a95d3b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3a9014c950 con 0x7f3a9014c3f0 2024-04-25T01:41:16.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.857+0000 7f3a8ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a901524a0 0x7f3a90142e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.857+0000 7f3a8ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a901524a0 0x7f3a90142e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56484/0 (socket says 172.21.15.112:56484) 2024-04-25T01:41:16.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.858+0000 7f3a8ffff640 1 -- 172.21.15.112:0/1478979503 learned_addr learned my addr 172.21.15.112:0/1478979503 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:16.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.858+0000 7f3a8effd640 1 --2- 172.21.15.112:0/1478979503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a9014c3f0 0x7f3a901428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.858+0000 7f3a8f7fe640 1 --2- 172.21.15.112:0/1478979503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a900ac3b0 0x7f3a90144270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.858+0000 7f3a8ffff640 1 -- 172.21.15.112:0/1478979503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a9014c3f0 msgr2=0x7f3a901428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:16.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.858+0000 7f3a8ffff640 1 --2- 172.21.15.112:0/1478979503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a9014c3f0 0x7f3a901428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.858+0000 7f3a8ffff640 1 -- 172.21.15.112:0/1478979503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a900ac3b0 msgr2=0x7f3a90144270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:16.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.858+0000 7f3a8ffff640 1 --2- 172.21.15.112:0/1478979503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a900ac3b0 0x7f3a90144270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:16.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.858+0000 7f3a8ffff640 1 -- 172.21.15.112:0/1478979503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9014eb70 con 0x7f3a901524a0 2024-04-25T01:41:16.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.858+0000 7f3a8ffff640 1 --2- 172.21.15.112:0/1478979503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a901524a0 0x7f3a90142e20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800c3f0 tx=0x7f3a8800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:16.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.858+0000 7f3a87fff640 1 -- 172.21.15.112:0/1478979503 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a88017070 con 0x7f3a901524a0 2024-04-25T01:41:16.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.858+0000 7f3a87fff640 1 -- 172.21.15.112:0/1478979503 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a880052d0 con 0x7f3a901524a0 2024-04-25T01:41:16.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.858+0000 7f3a95d3b640 1 -- 172.21.15.112:0/1478979503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a9014ee00 con 0x7f3a901524a0 2024-04-25T01:41:16.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.858+0000 7f3a87fff640 1 -- 172.21.15.112:0/1478979503 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a88003830 con 0x7f3a901524a0 2024-04-25T01:41:16.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.858+0000 7f3a95d3b640 1 -- 172.21.15.112:0/1478979503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a9014f880 con 0x7f3a901524a0 2024-04-25T01:41:16.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.859+0000 7f3a95d3b640 1 -- 172.21.15.112:0/1478979503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a9014f390 con 0x7f3a901524a0 2024-04-25T01:41:16.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.860+0000 7f3a87fff640 1 -- 172.21.15.112:0/1478979503 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a8801a070 con 0x7f3a901524a0 2024-04-25T01:41:16.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.860+0000 7f3a87fff640 1 --2- 172.21.15.112:0/1478979503 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a7006be60 0x7f3a7006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:16.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.860+0000 7f3a87fff640 1 -- 172.21.15.112:0/1478979503 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f3a8808a530 con 0x7f3a901524a0 2024-04-25T01:41:16.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.860+0000 7f3a8f7fe640 1 --2- 172.21.15.112:0/1478979503 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a7006be60 0x7f3a7006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:16.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.860+0000 7f3a8f7fe640 1 --2- 172.21.15.112:0/1478979503 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a7006be60 0x7f3a7006e320 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3a800113e0 tx=0x7f3a8000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:16.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:16.864+0000 7f3a87fff640 1 -- 172.21.15.112:0/1478979503 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3a880584a0 con 0x7f3a901524a0 2024-04-25T01:41:17.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:17.039+0000 7f3a95d3b640 1 -- 172.21.15.112:0/1478979503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a900682a0 con 0x7f3a901524a0 2024-04-25T01:41:17.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:17.041+0000 7f3a87fff640 1 -- 172.21.15.112:0/1478979503 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+20616 (secure 0 0 0) 0x7f3a8805c7a0 con 0x7f3a901524a0 2024-04-25T01:41:17.028 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:41:17.029 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:26:52.110548+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-25T01:25:59.478275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-25T01:26:00.719358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:41:17.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:17.043+0000 7f3a95d3b640 1 -- 172.21.15.112:0/1478979503 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a7006be60 msgr2=0x7f3a7006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:17.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:17.043+0000 7f3a95d3b640 1 --2- 172.21.15.112:0/1478979503 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a7006be60 0x7f3a7006e320 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3a800113e0 tx=0x7f3a8000b040 comp rx=0 tx=0).stop 2024-04-25T01:41:17.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:17.043+0000 7f3a95d3b640 1 -- 172.21.15.112:0/1478979503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a901524a0 msgr2=0x7f3a90142e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:17.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:17.043+0000 7f3a95d3b640 1 --2- 172.21.15.112:0/1478979503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a901524a0 0x7f3a90142e20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800c3f0 tx=0x7f3a8800c7f0 comp rx=0 tx=0).stop 2024-04-25T01:41:17.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:17.043+0000 7f3a95d3b640 1 -- 172.21.15.112:0/1478979503 shutdown_connections 2024-04-25T01:41:17.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:17.043+0000 7f3a95d3b640 1 --2- 172.21.15.112:0/1478979503 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f3a7006be60 0x7f3a7006e320 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:17.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:17.043+0000 7f3a95d3b640 1 --2- 172.21.15.112:0/1478979503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a901524a0 0x7f3a90142e20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:17.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:17.043+0000 7f3a95d3b640 1 --2- 172.21.15.112:0/1478979503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a9014c3f0 0x7f3a901428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:17.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:17.043+0000 7f3a95d3b640 1 --2- 172.21.15.112:0/1478979503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a900ac3b0 0x7f3a90144270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:17.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:17.043+0000 7f3a95d3b640 1 -- 172.21.15.112:0/1478979503 >> 172.21.15.112:0/1478979503 conn(0x7f3a9006b160 msgr2=0x7f3a90071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:17.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:17.043+0000 7f3a95d3b640 1 -- 172.21.15.112:0/1478979503 shutdown_connections 2024-04-25T01:41:17.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:17.043+0000 7f3a95d3b640 1 -- 172.21.15.112:0/1478979503 wait complete. 2024-04-25T01:41:18.044 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T01:41:18.045 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:41:18.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.163+0000 7f96587b7640 1 Processor -- start 2024-04-25T01:41:18.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.163+0000 7f96587b7640 1 -- start start 2024-04-25T01:41:18.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.163+0000 7f96587b7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f965014fbd0 0x7f96500c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.163+0000 7f96587b7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f96501566c0 0x7f96500c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.163+0000 7f96587b7640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96500c6270 con 0x7f965006cef0 2024-04-25T01:41:18.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.163+0000 7f96587b7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f965006c1c0 con 0x7f96501566c0 2024-04-25T01:41:18.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.163+0000 7f96587b7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f965006c410 con 0x7f965014fbd0 2024-04-25T01:41:18.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.164+0000 7f9656d54640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f96501566c0 0x7f96500c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.164+0000 7f9656d54640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f96501566c0 0x7f96500c91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54830/0 (socket says 172.21.15.112:54830) 2024-04-25T01:41:18.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.164+0000 7f9656d54640 1 -- 172.21.15.112:0/752344167 learned_addr learned my addr 172.21.15.112:0/752344167 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:18.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.164+0000 7f9656553640 1 --2- 172.21.15.112:0/752344167 >> v2:172.21.15.187:3300/0 conn(0x7f965014fbd0 0x7f96500c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.164+0000 7f9656d54640 1 -- 172.21.15.112:0/752344167 >> v2:172.21.15.187:3300/0 conn(0x7f965014fbd0 msgr2=0x7f96500c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:18.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.164+0000 7f9656d54640 1 --2- 172.21.15.112:0/752344167 >> v2:172.21.15.187:3300/0 conn(0x7f965014fbd0 0x7f96500c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.164+0000 7f9656d54640 1 -- 172.21.15.112:0/752344167 >> v1:172.21.15.187:6789/0 conn(0x7f965006cef0 legacy=0x7f96500c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:18.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.164+0000 7f9656d54640 1 -- 172.21.15.112:0/752344167 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965006c6f0 con 0x7f96501566c0 2024-04-25T01:41:18.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.165+0000 7f9656d54640 1 --2- 172.21.15.112:0/752344167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96501566c0 0x7f96500c91e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f96500c8040 tx=0x7f964802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72133a5b506d6e35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:18.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.165+0000 7f9655551640 1 -- 172.21.15.112:0/752344167 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9648036070 con 0x7f96501566c0 2024-04-25T01:41:18.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.165+0000 7f9655551640 1 -- 172.21.15.112:0/752344167 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9648031600 con 0x7f96501566c0 2024-04-25T01:41:18.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.165+0000 7f9655551640 1 -- 172.21.15.112:0/752344167 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96480319d0 con 0x7f96501566c0 2024-04-25T01:41:18.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.165+0000 7f96587b7640 1 -- 172.21.15.112:0/752344167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96501566c0 msgr2=0x7f96500c91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.165+0000 7f96587b7640 1 --2- 172.21.15.112:0/752344167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96501566c0 0x7f96500c91e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f96500c8040 tx=0x7f964802a5d0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.165+0000 7f96587b7640 1 -- 172.21.15.112:0/752344167 shutdown_connections 2024-04-25T01:41:18.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.165+0000 7f96587b7640 1 --2- 172.21.15.112:0/752344167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96501566c0 0x7f96500c91e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.165+0000 7f96587b7640 1 --2- 172.21.15.112:0/752344167 >> v2:172.21.15.187:3300/0 conn(0x7f965014fbd0 0x7f96500c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.165+0000 7f96587b7640 1 -- 172.21.15.112:0/752344167 >> 172.21.15.112:0/752344167 conn(0x7f96500ad050 msgr2=0x7f96500ad460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:18.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.165+0000 7f96587b7640 1 -- 172.21.15.112:0/752344167 shutdown_connections 2024-04-25T01:41:18.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.165+0000 7f96587b7640 1 -- 172.21.15.112:0/752344167 wait complete. 2024-04-25T01:41:18.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.166+0000 7f96587b7640 1 Processor -- start 2024-04-25T01:41:18.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.167+0000 7f96587b7640 1 -- start start 2024-04-25T01:41:18.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.167+0000 7f96587b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f965006cef0 0x7f96501df160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.167+0000 7f96587b7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f965014fbd0 0x7f9650159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.167+0000 7f96587b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96501566c0 0x7f96501eb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.167+0000 7f96587b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96500c8780 con 0x7f96501566c0 2024-04-25T01:41:18.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.167+0000 7f96587b7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f96500c5f80 con 0x7f965006cef0 2024-04-25T01:41:18.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.167+0000 7f96587b7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f96500c8900 con 0x7f965014fbd0 2024-04-25T01:41:18.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.167+0000 7f9656d54640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96501566c0 0x7f96501eb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.167+0000 7f9656d54640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96501566c0 0x7f96501eb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54838/0 (socket says 172.21.15.112:54838) 2024-04-25T01:41:18.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.167+0000 7f9656d54640 1 -- 172.21.15.112:0/277271009 learned_addr learned my addr 172.21.15.112:0/277271009 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:18.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.167+0000 7f9656553640 1 --2- 172.21.15.112:0/277271009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f965006cef0 0x7f96501df160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.167+0000 7f9655d52640 1 --2- 172.21.15.112:0/277271009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f965014fbd0 0x7f9650159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.167+0000 7f9656d54640 1 -- 172.21.15.112:0/277271009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f965014fbd0 msgr2=0x7f9650159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.167+0000 7f9656d54640 1 --2- 172.21.15.112:0/277271009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f965014fbd0 0x7f9650159360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.167+0000 7f9656d54640 1 -- 172.21.15.112:0/277271009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f965006cef0 msgr2=0x7f96501df160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.167+0000 7f9656d54640 1 --2- 172.21.15.112:0/277271009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f965006cef0 0x7f96501df160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.167+0000 7f9656d54640 1 -- 172.21.15.112:0/277271009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96501e19c0 con 0x7f96501566c0 2024-04-25T01:41:18.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.167+0000 7f9656553640 1 --2- 172.21.15.112:0/277271009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f965006cef0 0x7f96501df160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:41:18.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.167+0000 7f9655d52640 1 --2- 172.21.15.112:0/277271009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f965014fbd0 0x7f9650159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:41:18.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.168+0000 7f9656d54640 1 --2- 172.21.15.112:0/277271009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96501566c0 0x7f96501eb5a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9648009f40 tx=0x7f9648004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:18.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.168+0000 7f9646ffd640 1 -- 172.21.15.112:0/277271009 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9648036050 con 0x7f96501566c0 2024-04-25T01:41:18.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.168+0000 7f9646ffd640 1 -- 172.21.15.112:0/277271009 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9648042070 con 0x7f96501566c0 2024-04-25T01:41:18.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.168+0000 7f9646ffd640 1 -- 172.21.15.112:0/277271009 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9648009510 con 0x7f96501566c0 2024-04-25T01:41:18.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.168+0000 7f96587b7640 1 -- 172.21.15.112:0/277271009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96501556b0 con 0x7f96501566c0 2024-04-25T01:41:18.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.168+0000 7f96587b7640 1 -- 172.21.15.112:0/277271009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96501598a0 con 0x7f96501566c0 2024-04-25T01:41:18.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.169+0000 7f96587b7640 1 -- 172.21.15.112:0/277271009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f96500682a0 con 0x7f96501566c0 2024-04-25T01:41:18.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.169+0000 7f9646ffd640 1 -- 172.21.15.112:0/277271009 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9648007bd0 con 0x7f96501566c0 2024-04-25T01:41:18.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.170+0000 7f9646ffd640 1 --2- 172.21.15.112:0/277271009 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f963406be30 0x7f963406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.170+0000 7f9646ffd640 1 -- 172.21.15.112:0/277271009 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f96480ac160 con 0x7f96501566c0 2024-04-25T01:41:18.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.170+0000 7f9656553640 1 --2- 172.21.15.112:0/277271009 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f963406be30 0x7f963406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.170+0000 7f9646ffd640 1 --2- 172.21.15.112:0/277271009 >> [v2:172.21.15.112:6816/2045679399,v1:172.21.15.112:6817/2045679399] conn(0x7f96340767c0 0x7f9634078c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.170+0000 7f9646ffd640 1 -- 172.21.15.112:0/277271009 --> [v2:172.21.15.112:6816/2045679399,v1:172.21.15.112:6817/2045679399] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f964804ae80 con 0x7f96340767c0 2024-04-25T01:41:18.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.170+0000 7f9646ffd640 1 -- 172.21.15.112:0/277271009 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f96480b3050 con 0x7f96501566c0 2024-04-25T01:41:18.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.170+0000 7f9655d52640 1 --2- 172.21.15.112:0/277271009 >> [v2:172.21.15.112:6816/2045679399,v1:172.21.15.112:6817/2045679399] conn(0x7f96340767c0 0x7f9634078c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.170+0000 7f9656553640 1 --2- 172.21.15.112:0/277271009 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f963406be30 0x7f963406e2f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f964c006fd0 tx=0x7f964c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:18.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.170+0000 7f9655d52640 1 --2- 172.21.15.112:0/277271009 >> [v2:172.21.15.112:6816/2045679399,v1:172.21.15.112:6817/2045679399] conn(0x7f96340767c0 0x7f9634078c40 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:18.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.171+0000 7f9646ffd640 1 -- 172.21.15.112:0/277271009 <== osd.0 v2:172.21.15.112:6816/2045679399 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f964804ae80 con 0x7f96340767c0 2024-04-25T01:41:18.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.184+0000 7f96587b7640 1 -- 172.21.15.112:0/277271009 --> [v2:172.21.15.112:6816/2045679399,v1:172.21.15.112:6817/2045679399] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9650153320 con 0x7f96340767c0 2024-04-25T01:41:18.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.184+0000 7f9646ffd640 1 -- 172.21.15.112:0/277271009 <== osd.0 v2:172.21.15.112:6816/2045679399 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9650153320 con 0x7f96340767c0 2024-04-25T01:41:18.171 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:41:18.171 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:41:18.171 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:41:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.184+0000 7f96587b7640 1 -- 172.21.15.112:0/277271009 >> [v2:172.21.15.112:6816/2045679399,v1:172.21.15.112:6817/2045679399] conn(0x7f96340767c0 msgr2=0x7f9634078c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.184+0000 7f96587b7640 1 --2- 172.21.15.112:0/277271009 >> [v2:172.21.15.112:6816/2045679399,v1:172.21.15.112:6817/2045679399] conn(0x7f96340767c0 0x7f9634078c40 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.184+0000 7f96587b7640 1 -- 172.21.15.112:0/277271009 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f963406be30 msgr2=0x7f963406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.184+0000 7f96587b7640 1 --2- 172.21.15.112:0/277271009 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f963406be30 0x7f963406e2f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f964c006fd0 tx=0x7f964c008040 comp rx=0 tx=0).stop 2024-04-25T01:41:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.184+0000 7f96587b7640 1 -- 172.21.15.112:0/277271009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96501566c0 msgr2=0x7f96501eb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.184+0000 7f96587b7640 1 --2- 172.21.15.112:0/277271009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96501566c0 0x7f96501eb5a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9648009f40 tx=0x7f9648004a40 comp rx=0 tx=0).stop 2024-04-25T01:41:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.184+0000 7f9656d54640 1 -- 172.21.15.112:0/277271009 reap_dead start 2024-04-25T01:41:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.184+0000 7f96587b7640 1 -- 172.21.15.112:0/277271009 shutdown_connections 2024-04-25T01:41:18.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.184+0000 7f96587b7640 1 -- 172.21.15.112:0/277271009 >> 172.21.15.112:0/277271009 conn(0x7f96500ad050 msgr2=0x7f96500ad430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:18.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.184+0000 7f96587b7640 1 -- 172.21.15.112:0/277271009 shutdown_connections 2024-04-25T01:41:18.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.184+0000 7f96587b7640 1 -- 172.21.15.112:0/277271009 wait complete. 2024-04-25T01:41:18.182 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T01:41:18.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.292+0000 7fa1615c1640 1 Processor -- start 2024-04-25T01:41:18.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.292+0000 7fa1615c1640 1 -- start start 2024-04-25T01:41:18.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.293+0000 7fa1615c1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa15c0743d0 0x7fa15c0747b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.293+0000 7fa1615c1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa15c06fb80 0x7fa15c155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.293+0000 7fa1615c1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa15c155f40 0x7fa15c0c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.293+0000 7fa1615c1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa15c0c40e0 con 0x7fa15c155f40 2024-04-25T01:41:18.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.293+0000 7fa1615c1640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa15c1534b0 con 0x7fa15c06fb80 2024-04-25T01:41:18.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.293+0000 7fa1615c1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa15c0c16d0 con 0x7fa15c0743d0 2024-04-25T01:41:18.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.293+0000 7fa15b7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa15c155f40 0x7fa15c0c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.293+0000 7fa15b7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa15c155f40 0x7fa15c0c66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54854/0 (socket says 172.21.15.112:54854) 2024-04-25T01:41:18.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.293+0000 7fa15b7fe640 1 -- 172.21.15.112:0/3502701121 learned_addr learned my addr 172.21.15.112:0/3502701121 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:18.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.293+0000 7fa15affd640 1 --2- 172.21.15.112:0/3502701121 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa15c0743d0 0x7fa15c0747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.293+0000 7fa15a7fc640 1 --2- 172.21.15.112:0/3502701121 >> v2:172.21.15.187:3300/0 conn(0x7fa15c06fb80 0x7fa15c155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.293+0000 7fa15b7fe640 1 -- 172.21.15.112:0/3502701121 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa15c0743d0 msgr2=0x7fa15c0747b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:18.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.293+0000 7fa15b7fe640 1 --2- 172.21.15.112:0/3502701121 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa15c0743d0 0x7fa15c0747b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.293+0000 7fa15b7fe640 1 -- 172.21.15.112:0/3502701121 >> v2:172.21.15.187:3300/0 conn(0x7fa15c06fb80 msgr2=0x7fa15c155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:18.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.293+0000 7fa15b7fe640 1 --2- 172.21.15.112:0/3502701121 >> v2:172.21.15.187:3300/0 conn(0x7fa15c06fb80 0x7fa15c155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.293+0000 7fa15b7fe640 1 -- 172.21.15.112:0/3502701121 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15c0c1e40 con 0x7fa15c155f40 2024-04-25T01:41:18.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.294+0000 7fa15b7fe640 1 --2- 172.21.15.112:0/3502701121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa15c155f40 0x7fa15c0c66e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa15c06cb50 tx=0x7fa15402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2783202f89a17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:18.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.294+0000 7fa159ffb640 1 -- 172.21.15.112:0/3502701121 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154037070 con 0x7fa15c155f40 2024-04-25T01:41:18.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.294+0000 7fa159ffb640 1 -- 172.21.15.112:0/3502701121 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa154032440 con 0x7fa15c155f40 2024-04-25T01:41:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.294+0000 7fa159ffb640 1 -- 172.21.15.112:0/3502701121 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154032810 con 0x7fa15c155f40 2024-04-25T01:41:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.295+0000 7fa1615c1640 1 -- 172.21.15.112:0/3502701121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa15c155f40 msgr2=0x7fa15c0c66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.295+0000 7fa1615c1640 1 --2- 172.21.15.112:0/3502701121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa15c155f40 0x7fa15c0c66e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa15c06cb50 tx=0x7fa15402a140 comp rx=0 tx=0).stop 2024-04-25T01:41:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.295+0000 7fa1615c1640 1 -- 172.21.15.112:0/3502701121 shutdown_connections 2024-04-25T01:41:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.295+0000 7fa1615c1640 1 --2- 172.21.15.112:0/3502701121 >> v2:172.21.15.187:3300/0 conn(0x7fa15c06fb80 0x7fa15c155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.295+0000 7fa1615c1640 1 --2- 172.21.15.112:0/3502701121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa15c155f40 0x7fa15c0c66e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.295+0000 7fa1615c1640 1 --2- 172.21.15.112:0/3502701121 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa15c0743d0 0x7fa15c0747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.295+0000 7fa1615c1640 1 -- 172.21.15.112:0/3502701121 >> 172.21.15.112:0/3502701121 conn(0x7fa15c06b160 msgr2=0x7fa15c06b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.295+0000 7fa1615c1640 1 -- 172.21.15.112:0/3502701121 shutdown_connections 2024-04-25T01:41:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.295+0000 7fa1615c1640 1 -- 172.21.15.112:0/3502701121 wait complete. 2024-04-25T01:41:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.296+0000 7fa1615c1640 1 Processor -- start 2024-04-25T01:41:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.296+0000 7fa1615c1640 1 -- start start 2024-04-25T01:41:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa1615c1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa15c06fb80 0x7fa15c158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa1615c1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa15c0743d0 0x7fa15c156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa1615c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa15c155f40 0x7fa15c151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa1615c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c159d00 con 0x7fa15c155f40 2024-04-25T01:41:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa1615c1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c0c19b0 con 0x7fa15c06fb80 2024-04-25T01:41:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa1615c1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa15c0a7070 con 0x7fa15c0743d0 2024-04-25T01:41:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa15b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa15c155f40 0x7fa15c151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa15b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa15c155f40 0x7fa15c151f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54862/0 (socket says 172.21.15.112:54862) 2024-04-25T01:41:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa15b7fe640 1 -- 172.21.15.112:0/4265563114 learned_addr learned my addr 172.21.15.112:0/4265563114 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa15affd640 1 --2- 172.21.15.112:0/4265563114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa15c06fb80 0x7fa15c158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa15a7fc640 1 --2- 172.21.15.112:0/4265563114 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa15c0743d0 0x7fa15c156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa15b7fe640 1 -- 172.21.15.112:0/4265563114 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa15c0743d0 msgr2=0x7fa15c156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa15b7fe640 1 --2- 172.21.15.112:0/4265563114 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa15c0743d0 0x7fa15c156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa15b7fe640 1 -- 172.21.15.112:0/4265563114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa15c06fb80 msgr2=0x7fa15c158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa15b7fe640 1 --2- 172.21.15.112:0/4265563114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa15c06fb80 0x7fa15c158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa15b7fe640 1 -- 172.21.15.112:0/4265563114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15c0c75a0 con 0x7fa15c155f40 2024-04-25T01:41:18.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa15b7fe640 1 --2- 172.21.15.112:0/4265563114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa15c155f40 0x7fa15c151f90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa15402a180 tx=0x7fa1540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:18.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa1437fe640 1 -- 172.21.15.112:0/4265563114 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154041020 con 0x7fa15c155f40 2024-04-25T01:41:18.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa1437fe640 1 -- 172.21.15.112:0/4265563114 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1540306c0 con 0x7fa15c155f40 2024-04-25T01:41:18.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa1437fe640 1 -- 172.21.15.112:0/4265563114 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154037040 con 0x7fa15c155f40 2024-04-25T01:41:18.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.297+0000 7fa1615c1640 1 -- 172.21.15.112:0/4265563114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15c148650 con 0x7fa15c155f40 2024-04-25T01:41:18.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.298+0000 7fa1615c1640 1 -- 172.21.15.112:0/4265563114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa15c158230 con 0x7fa15c155f40 2024-04-25T01:41:18.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.298+0000 7fa1615c1640 1 -- 172.21.15.112:0/4265563114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa15c0682a0 con 0x7fa15c155f40 2024-04-25T01:41:18.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.299+0000 7fa1437fe640 1 -- 172.21.15.112:0/4265563114 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa154044020 con 0x7fa15c155f40 2024-04-25T01:41:18.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.299+0000 7fa1437fe640 1 --2- 172.21.15.112:0/4265563114 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa14806be30 0x7fa14806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.299+0000 7fa15affd640 1 --2- 172.21.15.112:0/4265563114 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa14806be30 0x7fa14806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.299+0000 7fa1437fe640 1 -- 172.21.15.112:0/4265563114 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fa1540ab7c0 con 0x7fa15c155f40 2024-04-25T01:41:18.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.300+0000 7fa15affd640 1 --2- 172.21.15.112:0/4265563114 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa14806be30 0x7fa14806e2f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0ad730 tx=0x7fa14c0098d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:18.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.304+0000 7fa1437fe640 1 -- 172.21.15.112:0/4265563114 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa15404b030 con 0x7fa15c155f40 2024-04-25T01:41:18.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.477+0000 7fa1615c1640 1 -- 172.21.15.112:0/4265563114 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fa15c148400 con 0x7fa14806be30 2024-04-25T01:41:18.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.478+0000 7fa1437fe640 1 -- 172.21.15.112:0/4265563114 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa15c148400 con 0x7fa14806be30 2024-04-25T01:41:18.465 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T01:41:18.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.480+0000 7fa1615c1640 1 -- 172.21.15.112:0/4265563114 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa14806be30 msgr2=0x7fa14806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.480+0000 7fa1615c1640 1 --2- 172.21.15.112:0/4265563114 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa14806be30 0x7fa14806e2f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0ad730 tx=0x7fa14c0098d0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.480+0000 7fa1615c1640 1 -- 172.21.15.112:0/4265563114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa15c155f40 msgr2=0x7fa15c151f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.480+0000 7fa1615c1640 1 --2- 172.21.15.112:0/4265563114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa15c155f40 0x7fa15c151f90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa15402a180 tx=0x7fa1540046c0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.480+0000 7fa1615c1640 1 -- 172.21.15.112:0/4265563114 shutdown_connections 2024-04-25T01:41:18.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.480+0000 7fa1615c1640 1 --2- 172.21.15.112:0/4265563114 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa14806be30 0x7fa14806e2f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.480+0000 7fa1615c1640 1 --2- 172.21.15.112:0/4265563114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa15c155f40 0x7fa15c151f90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.480+0000 7fa1615c1640 1 --2- 172.21.15.112:0/4265563114 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa15c0743d0 0x7fa15c156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.480+0000 7fa1615c1640 1 --2- 172.21.15.112:0/4265563114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa15c06fb80 0x7fa15c158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.480+0000 7fa1615c1640 1 -- 172.21.15.112:0/4265563114 >> 172.21.15.112:0/4265563114 conn(0x7fa15c06b160 msgr2=0x7fa15c06f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:18.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.480+0000 7fa1615c1640 1 -- 172.21.15.112:0/4265563114 shutdown_connections 2024-04-25T01:41:18.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.480+0000 7fa1615c1640 1 -- 172.21.15.112:0/4265563114 wait complete. 2024-04-25T01:41:18.478 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T01:41:18.479 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:41:18.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.592+0000 7ff6cf71d640 1 Processor -- start 2024-04-25T01:41:18.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.592+0000 7ff6cf71d640 1 -- start start 2024-04-25T01:41:18.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.592+0000 7ff6cf71d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff6c81524b0 0x7ff6c806cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.592+0000 7ff6cf71d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6c806d560 con 0x7ff6c80c2d70 2024-04-25T01:41:18.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6cf71d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6c806d840 con 0x7ff6c806e600 2024-04-25T01:41:18.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6cf71d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff6c806db20 con 0x7ff6c81524b0 2024-04-25T01:41:18.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6cd4b9640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff6c80c2d70 0x7ff6c814c380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54602/0 (socket says 172.21.15.112:54602) 2024-04-25T01:41:18.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6cd4b9640 1 -- 172.21.15.112:0/1545858978 learned_addr learned my addr 172.21.15.112:0/1545858978 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:18.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6cdcba640 1 --2- 172.21.15.112:0/1545858978 >> v2:172.21.15.187:3300/0 conn(0x7ff6c81524b0 0x7ff6c806cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6bffff640 1 -- 172.21.15.112:0/1545858978 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1854340423 0 0) 0x7ff6c806d560 con 0x7ff6c80c2d70 2024-04-25T01:41:18.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6bffff640 1 -- 172.21.15.112:0/1545858978 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6b4005730 con 0x7ff6c80c2d70 2024-04-25T01:41:18.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6bffff640 1 -- 172.21.15.112:0/1545858978 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4153717210 0 0) 0x7ff6b4005730 con 0x7ff6c80c2d70 2024-04-25T01:41:18.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6bffff640 1 -- 172.21.15.112:0/1545858978 >> v2:172.21.15.187:3300/0 conn(0x7ff6c81524b0 msgr2=0x7ff6c806cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:18.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6bffff640 1 --2- 172.21.15.112:0/1545858978 >> v2:172.21.15.187:3300/0 conn(0x7ff6c81524b0 0x7ff6c806cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6bffff640 1 -- 172.21.15.112:0/1545858978 >> v1:172.21.15.187:6789/0 conn(0x7ff6c806e600 legacy=0x7ff6c81539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6bffff640 1 -- 172.21.15.112:0/1545858978 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c806de00 con 0x7ff6c80c2d70 2024-04-25T01:41:18.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6bffff640 1 -- 172.21.15.112:0/1545858978 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7ff6c00019f0 con 0x7ff6c80c2d70 2024-04-25T01:41:18.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6bffff640 1 -- 172.21.15.112:0/1545858978 >> v1:172.21.15.112:6789/0 conn(0x7ff6c80c2d70 legacy=0x7ff6c814c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6bffff640 1 --2- 172.21.15.112:0/1545858978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6b4005bc0 0x7ff6b4005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6bffff640 1 --2- 172.21.15.112:0/1545858978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b4006a10 0x7ff6b4006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6bffff640 1 --2- 172.21.15.112:0/1545858978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6b40073d0 0x7ff6b400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6bffff640 1 -- 172.21.15.112:0/1545858978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c806db20 con 0x7ff6b4006a10 2024-04-25T01:41:18.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6bffff640 1 -- 172.21.15.112:0/1545858978 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b400e370 con 0x7ff6b40073d0 2024-04-25T01:41:18.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6bffff640 1 -- 172.21.15.112:0/1545858978 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff6b400e570 con 0x7ff6b4005bc0 2024-04-25T01:41:18.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6cdcba640 1 --2- 172.21.15.112:0/1545858978 >> v2:172.21.15.187:3300/0 conn(0x7ff6c81524b0 0x7ff6c806cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:41:18.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.593+0000 7ff6cccb8640 1 --2- 172.21.15.112:0/1545858978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b4006a10 0x7ff6b4006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.594+0000 7ff6cd4b9640 1 --2- 172.21.15.112:0/1545858978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6b4005bc0 0x7ff6b4005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.594+0000 7ff6cdcba640 1 --2- 172.21.15.112:0/1545858978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6b40073d0 0x7ff6b400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.594+0000 7ff6cccb8640 1 -- 172.21.15.112:0/1545858978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6b4005bc0 msgr2=0x7ff6b4005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:18.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.594+0000 7ff6cccb8640 1 --2- 172.21.15.112:0/1545858978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6b4005bc0 0x7ff6b4005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.594+0000 7ff6cccb8640 1 -- 172.21.15.112:0/1545858978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6b40073d0 msgr2=0x7ff6b400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:18.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.594+0000 7ff6cccb8640 1 --2- 172.21.15.112:0/1545858978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6b40073d0 0x7ff6b400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.594+0000 7ff6cccb8640 1 -- 172.21.15.112:0/1545858978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b4005730 con 0x7ff6b4006a10 2024-04-25T01:41:18.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.594+0000 7ff6cdcba640 1 --2- 172.21.15.112:0/1545858978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6b40073d0 0x7ff6b400dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:41:18.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.594+0000 7ff6cdcba640 1 -- 172.21.15.112:0/1545858978 reap_dead start 2024-04-25T01:41:18.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.594+0000 7ff6cccb8640 1 --2- 172.21.15.112:0/1545858978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b4006a10 0x7ff6b4006e90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8001720 tx=0x7ff6b4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0b3f0cfc87a504 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:18.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.594+0000 7ff6bffff640 1 -- 172.21.15.112:0/1545858978 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8001370 con 0x7ff6b4006a10 2024-04-25T01:41:18.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.594+0000 7ff6bffff640 1 -- 172.21.15.112:0/1545858978 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b80052d0 con 0x7ff6b4006a10 2024-04-25T01:41:18.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.594+0000 7ff6bffff640 1 -- 172.21.15.112:0/1545858978 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b802ab90 con 0x7ff6b4006a10 2024-04-25T01:41:18.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.595+0000 7ff6cf71d640 1 -- 172.21.15.112:0/1545858978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b4006a10 msgr2=0x7ff6b4006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.595+0000 7ff6cf71d640 1 --2- 172.21.15.112:0/1545858978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b4006a10 0x7ff6b4006e90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8001720 tx=0x7ff6b4007a40 comp rx=0 tx=0).stop 2024-04-25T01:41:18.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.595+0000 7ff6cf71d640 1 -- 172.21.15.112:0/1545858978 shutdown_connections 2024-04-25T01:41:18.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.595+0000 7ff6cf71d640 1 --2- 172.21.15.112:0/1545858978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b4006a10 0x7ff6b4006e90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.595+0000 7ff6cf71d640 1 -- 172.21.15.112:0/1545858978 >> 172.21.15.112:0/1545858978 conn(0x7ff6c806b8a0 msgr2=0x7ff6c80c6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:18.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.595+0000 7ff6cf71d640 1 -- 172.21.15.112:0/1545858978 shutdown_connections 2024-04-25T01:41:18.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.595+0000 7ff6cf71d640 1 -- 172.21.15.112:0/1545858978 wait complete. 2024-04-25T01:41:18.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.596+0000 7ff6cf71d640 1 Processor -- start 2024-04-25T01:41:18.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.596+0000 7ff6cf71d640 1 -- start start 2024-04-25T01:41:18.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.597+0000 7ff6cf71d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b4006a10 0x7ff6c80abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.597+0000 7ff6cf71d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6c80aa3d0 0x7ff6c81ddc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.597+0000 7ff6cf71d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6c8146ba0 0x7ff6c8147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.597+0000 7ff6cf71d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c80c3150 con 0x7ff6b4006a10 2024-04-25T01:41:18.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.597+0000 7ff6cf71d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c80c1690 con 0x7ff6c80aa3d0 2024-04-25T01:41:18.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.597+0000 7ff6cf71d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff6c80c32d0 con 0x7ff6c8146ba0 2024-04-25T01:41:18.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.597+0000 7ff6cd4b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b4006a10 0x7ff6c80abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.597+0000 7ff6cd4b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b4006a10 0x7ff6c80abda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54880/0 (socket says 172.21.15.112:54880) 2024-04-25T01:41:18.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.597+0000 7ff6cd4b9640 1 -- 172.21.15.112:0/3401016153 learned_addr learned my addr 172.21.15.112:0/3401016153 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:18.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.597+0000 7ff6cdcba640 1 --2- 172.21.15.112:0/3401016153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6c8146ba0 0x7ff6c8147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.597+0000 7ff6cccb8640 1 --2- 172.21.15.112:0/3401016153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6c80aa3d0 0x7ff6c81ddc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.597+0000 7ff6cd4b9640 1 -- 172.21.15.112:0/3401016153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6c8146ba0 msgr2=0x7ff6c8147020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.597+0000 7ff6cd4b9640 1 --2- 172.21.15.112:0/3401016153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6c8146ba0 0x7ff6c8147020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.597+0000 7ff6cd4b9640 1 -- 172.21.15.112:0/3401016153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6c80aa3d0 msgr2=0x7ff6c81ddc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.597+0000 7ff6cd4b9640 1 --2- 172.21.15.112:0/3401016153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6c80aa3d0 0x7ff6c81ddc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.597+0000 7ff6cd4b9640 1 -- 172.21.15.112:0/3401016153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c8142750 con 0x7ff6b4006a10 2024-04-25T01:41:18.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.597+0000 7ff6cdcba640 1 --2- 172.21.15.112:0/3401016153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6c8146ba0 0x7ff6c8147020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:41:18.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.597+0000 7ff6cccb8640 1 --2- 172.21.15.112:0/3401016153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6c80aa3d0 0x7ff6c81ddc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:41:18.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.597+0000 7ff6cd4b9640 1 --2- 172.21.15.112:0/3401016153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b4006a10 0x7ff6c80abda0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0002230 tx=0x7ff6c000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:18.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.597+0000 7ff6bdffb640 1 -- 172.21.15.112:0/3401016153 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c000ee50 con 0x7ff6b4006a10 2024-04-25T01:41:18.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.598+0000 7ff6bdffb640 1 -- 172.21.15.112:0/3401016153 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6c0018070 con 0x7ff6b4006a10 2024-04-25T01:41:18.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.598+0000 7ff6cf71d640 1 -- 172.21.15.112:0/3401016153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c8148840 con 0x7ff6b4006a10 2024-04-25T01:41:18.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.598+0000 7ff6bdffb640 1 -- 172.21.15.112:0/3401016153 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0013660 con 0x7ff6b4006a10 2024-04-25T01:41:18.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.598+0000 7ff6cf71d640 1 -- 172.21.15.112:0/3401016153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c8141c70 con 0x7ff6b4006a10 2024-04-25T01:41:18.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.599+0000 7ff6cf71d640 1 -- 172.21.15.112:0/3401016153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff6c80ac450 con 0x7ff6b4006a10 2024-04-25T01:41:18.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.599+0000 7ff6bdffb640 1 -- 172.21.15.112:0/3401016153 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff6c0013800 con 0x7ff6b4006a10 2024-04-25T01:41:18.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.599+0000 7ff6bdffb640 1 --2- 172.21.15.112:0/3401016153 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff6ac06c000 0x7ff6ac06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.599+0000 7ff6bdffb640 1 -- 172.21.15.112:0/3401016153 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7ff6c008b770 con 0x7ff6b4006a10 2024-04-25T01:41:18.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.599+0000 7ff6cccb8640 1 --2- 172.21.15.112:0/3401016153 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff6ac06c000 0x7ff6ac06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.599+0000 7ff6bdffb640 1 --2- 172.21.15.112:0/3401016153 >> [v2:172.21.15.112:6800/1032830648,v1:172.21.15.112:6801/1032830648] conn(0x7ff6ac076990 0x7ff6ac078e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.599+0000 7ff6bdffb640 1 -- 172.21.15.112:0/3401016153 --> [v2:172.21.15.112:6800/1032830648,v1:172.21.15.112:6801/1032830648] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff6c000ee50 con 0x7ff6ac076990 2024-04-25T01:41:18.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.599+0000 7ff6bdffb640 1 -- 172.21.15.112:0/3401016153 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff6c0092050 con 0x7ff6b4006a10 2024-04-25T01:41:18.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.600+0000 7ff6cdcba640 1 --2- 172.21.15.112:0/3401016153 >> [v2:172.21.15.112:6800/1032830648,v1:172.21.15.112:6801/1032830648] conn(0x7ff6ac076990 0x7ff6ac078e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.600+0000 7ff6cccb8640 1 --2- 172.21.15.112:0/3401016153 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff6ac06c000 0x7ff6ac06e4c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8032000 tx=0x7ff6b802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:18.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.600+0000 7ff6cdcba640 1 --2- 172.21.15.112:0/3401016153 >> [v2:172.21.15.112:6800/1032830648,v1:172.21.15.112:6801/1032830648] conn(0x7ff6ac076990 0x7ff6ac078e10 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:18.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.600+0000 7ff6bdffb640 1 -- 172.21.15.112:0/3401016153 <== osd.1 v2:172.21.15.112:6800/1032830648 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6c000ee50 con 0x7ff6ac076990 2024-04-25T01:41:18.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.613+0000 7ff6cf71d640 1 -- 172.21.15.112:0/3401016153 --> [v2:172.21.15.112:6800/1032830648,v1:172.21.15.112:6801/1032830648] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff6c80682a0 con 0x7ff6ac076990 2024-04-25T01:41:18.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.614+0000 7ff6bdffb640 1 -- 172.21.15.112:0/3401016153 <== osd.1 v2:172.21.15.112:6800/1032830648 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff6c80682a0 con 0x7ff6ac076990 2024-04-25T01:41:18.601 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:41:18.601 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:41:18.601 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:41:18.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.614+0000 7ff6cf71d640 1 -- 172.21.15.112:0/3401016153 >> [v2:172.21.15.112:6800/1032830648,v1:172.21.15.112:6801/1032830648] conn(0x7ff6ac076990 msgr2=0x7ff6ac078e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.614+0000 7ff6cf71d640 1 --2- 172.21.15.112:0/3401016153 >> [v2:172.21.15.112:6800/1032830648,v1:172.21.15.112:6801/1032830648] conn(0x7ff6ac076990 0x7ff6ac078e10 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.614+0000 7ff6cf71d640 1 -- 172.21.15.112:0/3401016153 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff6ac06c000 msgr2=0x7ff6ac06e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.614+0000 7ff6cf71d640 1 --2- 172.21.15.112:0/3401016153 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff6ac06c000 0x7ff6ac06e4c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8032000 tx=0x7ff6b802e000 comp rx=0 tx=0).stop 2024-04-25T01:41:18.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.614+0000 7ff6cf71d640 1 -- 172.21.15.112:0/3401016153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b4006a10 msgr2=0x7ff6c80abda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.614+0000 7ff6cf71d640 1 --2- 172.21.15.112:0/3401016153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b4006a10 0x7ff6c80abda0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0002230 tx=0x7ff6c000e530 comp rx=0 tx=0).stop 2024-04-25T01:41:18.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.614+0000 7ff6cdcba640 1 -- 172.21.15.112:0/3401016153 reap_dead start 2024-04-25T01:41:18.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.614+0000 7ff6cf71d640 1 -- 172.21.15.112:0/3401016153 shutdown_connections 2024-04-25T01:41:18.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.614+0000 7ff6cf71d640 1 -- 172.21.15.112:0/3401016153 >> 172.21.15.112:0/3401016153 conn(0x7ff6c806b8a0 msgr2=0x7ff6c81514b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:18.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.614+0000 7ff6cf71d640 1 -- 172.21.15.112:0/3401016153 shutdown_connections 2024-04-25T01:41:18.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.614+0000 7ff6cf71d640 1 -- 172.21.15.112:0/3401016153 wait complete. 2024-04-25T01:41:18.612 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T01:41:18.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.731+0000 7f394a04d640 1 Processor -- start 2024-04-25T01:41:18.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.731+0000 7f394a04d640 1 -- start start 2024-04-25T01:41:18.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.732+0000 7f394a04d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f394414c3f0 0x7f39440c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.732+0000 7f394a04d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39441529b0 con 0x7f39441524a0 2024-04-25T01:41:18.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.732+0000 7f394a04d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3944152c70 con 0x7f39440ac3b0 2024-04-25T01:41:18.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.732+0000 7f394a04d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3944155a90 con 0x7f394414c3f0 2024-04-25T01:41:18.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.732+0000 7f3943fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f39441524a0 0x7f394415ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54618/0 (socket says 172.21.15.112:54618) 2024-04-25T01:41:18.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.732+0000 7f3943fff640 1 -- 172.21.15.112:0/4187856220 learned_addr learned my addr 172.21.15.112:0/4187856220 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:18.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.732+0000 7f39437fe640 1 --2- 172.21.15.112:0/4187856220 >> v2:172.21.15.187:3300/0 conn(0x7f394414c3f0 0x7f39440c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.732+0000 7f39427fc640 1 -- 172.21.15.112:0/4187856220 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3975121364 0 0) 0x7f39441529b0 con 0x7f39441524a0 2024-04-25T01:41:18.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.732+0000 7f39427fc640 1 -- 172.21.15.112:0/4187856220 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f392c003520 con 0x7f39441524a0 2024-04-25T01:41:18.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.732+0000 7f39427fc640 1 -- 172.21.15.112:0/4187856220 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1722720582 0 0) 0x7f392c003520 con 0x7f39441524a0 2024-04-25T01:41:18.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.732+0000 7f39427fc640 1 -- 172.21.15.112:0/4187856220 >> v2:172.21.15.187:3300/0 conn(0x7f394414c3f0 msgr2=0x7f39440c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:18.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.732+0000 7f39427fc640 1 --2- 172.21.15.112:0/4187856220 >> v2:172.21.15.187:3300/0 conn(0x7f394414c3f0 0x7f39440c2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.732+0000 7f39427fc640 1 -- 172.21.15.112:0/4187856220 >> v1:172.21.15.187:6789/0 conn(0x7f39440ac3b0 legacy=0x7f3944155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.732+0000 7f39427fc640 1 -- 172.21.15.112:0/4187856220 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3944152f50 con 0x7f39441524a0 2024-04-25T01:41:18.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.732+0000 7f39427fc640 1 -- 172.21.15.112:0/4187856220 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f393c0019f0 con 0x7f39441524a0 2024-04-25T01:41:18.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.732+0000 7f39427fc640 1 -- 172.21.15.112:0/4187856220 >> v1:172.21.15.112:6789/0 conn(0x7f39441524a0 legacy=0x7f394415ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.732+0000 7f39427fc640 1 --2- 172.21.15.112:0/4187856220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f392c0039b0 0x7f392c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.733+0000 7f39427fc640 1 --2- 172.21.15.112:0/4187856220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f392c004800 0x7f392c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.733+0000 7f39427fc640 1 --2- 172.21.15.112:0/4187856220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f392c0051c0 0x7f392c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.733+0000 7f39427fc640 1 -- 172.21.15.112:0/4187856220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3944155a90 con 0x7f392c0051c0 2024-04-25T01:41:18.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.733+0000 7f39427fc640 1 -- 172.21.15.112:0/4187856220 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f392c00c160 con 0x7f392c004800 2024-04-25T01:41:18.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.733+0000 7f39427fc640 1 -- 172.21.15.112:0/4187856220 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f392c00c360 con 0x7f392c0039b0 2024-04-25T01:41:18.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.733+0000 7f3943fff640 1 --2- 172.21.15.112:0/4187856220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f392c0051c0 0x7f392c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.733+0000 7f39437fe640 1 --2- 172.21.15.112:0/4187856220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f392c0039b0 0x7f392c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.733+0000 7f3942ffd640 1 --2- 172.21.15.112:0/4187856220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f392c004800 0x7f392c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.733+0000 7f3943fff640 1 -- 172.21.15.112:0/4187856220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f392c0039b0 msgr2=0x7f392c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:18.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.733+0000 7f3943fff640 1 --2- 172.21.15.112:0/4187856220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f392c0039b0 0x7f392c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.733+0000 7f3943fff640 1 -- 172.21.15.112:0/4187856220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f392c004800 msgr2=0x7f392c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:18.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.733+0000 7f3943fff640 1 --2- 172.21.15.112:0/4187856220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f392c004800 0x7f392c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.733+0000 7f3943fff640 1 -- 172.21.15.112:0/4187856220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392c003520 con 0x7f392c0051c0 2024-04-25T01:41:18.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.733+0000 7f3942ffd640 1 --2- 172.21.15.112:0/4187856220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f392c004800 0x7f392c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:41:18.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.733+0000 7f39437fe640 1 --2- 172.21.15.112:0/4187856220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f392c0039b0 0x7f392c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:41:18.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.733+0000 7f3943fff640 1 -- 172.21.15.112:0/4187856220 reap_dead start 2024-04-25T01:41:18.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.734+0000 7f3943fff640 1 --2- 172.21.15.112:0/4187856220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f392c0051c0 0x7f392c00b9c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f393c00a420 tx=0x7f392c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f31f999572c76fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:18.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.734+0000 7f39427fc640 1 -- 172.21.15.112:0/4187856220 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c00a460 con 0x7f392c0051c0 2024-04-25T01:41:18.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.734+0000 7f39427fc640 1 -- 172.21.15.112:0/4187856220 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f393c031070 con 0x7f392c0051c0 2024-04-25T01:41:18.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.734+0000 7f39427fc640 1 -- 172.21.15.112:0/4187856220 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c036470 con 0x7f392c0051c0 2024-04-25T01:41:18.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.734+0000 7f394a04d640 1 -- 172.21.15.112:0/4187856220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f392c0051c0 msgr2=0x7f392c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.734+0000 7f394a04d640 1 --2- 172.21.15.112:0/4187856220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f392c0051c0 0x7f392c00b9c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f393c00a420 tx=0x7f392c005830 comp rx=0 tx=0).stop 2024-04-25T01:41:18.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.734+0000 7f394a04d640 1 -- 172.21.15.112:0/4187856220 shutdown_connections 2024-04-25T01:41:18.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.734+0000 7f394a04d640 1 --2- 172.21.15.112:0/4187856220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f392c0051c0 0x7f392c00b9c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.734+0000 7f394a04d640 1 -- 172.21.15.112:0/4187856220 >> 172.21.15.112:0/4187856220 conn(0x7f394406b160 msgr2=0x7f394406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:18.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.734+0000 7f394a04d640 1 -- 172.21.15.112:0/4187856220 shutdown_connections 2024-04-25T01:41:18.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.734+0000 7f394a04d640 1 -- 172.21.15.112:0/4187856220 wait complete. 2024-04-25T01:41:18.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.735+0000 7f394a04d640 1 Processor -- start 2024-04-25T01:41:18.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.736+0000 7f394a04d640 1 -- start start 2024-04-25T01:41:18.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.736+0000 7f394a04d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f392c0051c0 0x7f3944147a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.736+0000 7f394a04d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3944150d80 0x7f3944144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.736+0000 7f394a04d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944145a30 0x7f3944145420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.736+0000 7f394a04d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f394414c7d0 con 0x7f3944145a30 2024-04-25T01:41:18.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.736+0000 7f394a04d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3944070d60 con 0x7f392c0051c0 2024-04-25T01:41:18.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.736+0000 7f394a04d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f394414c950 con 0x7f3944150d80 2024-04-25T01:41:18.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.736+0000 7f3943fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944145a30 0x7f3944145420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.736+0000 7f3943fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944145a30 0x7f3944145420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54910/0 (socket says 172.21.15.112:54910) 2024-04-25T01:41:18.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.736+0000 7f3943fff640 1 -- 172.21.15.112:0/2331080883 learned_addr learned my addr 172.21.15.112:0/2331080883 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:18.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.737+0000 7f3943fff640 1 -- 172.21.15.112:0/2331080883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3944150d80 msgr2=0x7f3944144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.737+0000 7f3943fff640 1 --2- 172.21.15.112:0/2331080883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3944150d80 0x7f3944144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.737+0000 7f3943fff640 1 -- 172.21.15.112:0/2331080883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f392c0051c0 msgr2=0x7f3944147a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.737+0000 7f3943fff640 1 --2- 172.21.15.112:0/2331080883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f392c0051c0 0x7f3944147a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.737+0000 7f3943fff640 1 -- 172.21.15.112:0/2331080883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39440c6e60 con 0x7f3944145a30 2024-04-25T01:41:18.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.737+0000 7f3943fff640 1 --2- 172.21.15.112:0/2331080883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944145a30 0x7f3944145420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f393c00a930 tx=0x7f3944152e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:18.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.737+0000 7f392bfff640 1 -- 172.21.15.112:0/2331080883 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c00a460 con 0x7f3944145a30 2024-04-25T01:41:18.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.737+0000 7f392bfff640 1 -- 172.21.15.112:0/2331080883 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f393c001250 con 0x7f3944145a30 2024-04-25T01:41:18.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.737+0000 7f394a04d640 1 -- 172.21.15.112:0/2331080883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39441ea190 con 0x7f3944145a30 2024-04-25T01:41:18.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.737+0000 7f392bfff640 1 -- 172.21.15.112:0/2331080883 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c001fb0 con 0x7f3944145a30 2024-04-25T01:41:18.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.737+0000 7f394a04d640 1 -- 172.21.15.112:0/2331080883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39441ea430 con 0x7f3944145a30 2024-04-25T01:41:18.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.738+0000 7f394a04d640 1 -- 172.21.15.112:0/2331080883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39440682a0 con 0x7f3944145a30 2024-04-25T01:41:18.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.738+0000 7f392bfff640 1 -- 172.21.15.112:0/2331080883 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f393c036670 con 0x7f3944145a30 2024-04-25T01:41:18.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.739+0000 7f392bfff640 1 --2- 172.21.15.112:0/2331080883 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f393006be60 0x7f393006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:18.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.739+0000 7f392bfff640 1 -- 172.21.15.112:0/2331080883 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f393c0ad3b0 con 0x7f3944145a30 2024-04-25T01:41:18.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.739+0000 7f39437fe640 1 --2- 172.21.15.112:0/2331080883 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f393006be60 0x7f393006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:18.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.739+0000 7f39437fe640 1 --2- 172.21.15.112:0/2331080883 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f393006be60 0x7f393006e320 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3934009830 tx=0x7f3934008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:18.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.743+0000 7f392bfff640 1 -- 172.21.15.112:0/2331080883 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f393c07b400 con 0x7f3944145a30 2024-04-25T01:41:18.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.918+0000 7f394a04d640 1 -- 172.21.15.112:0/2331080883 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f3944075980 con 0x7f393006be60 2024-04-25T01:41:18.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.918+0000 7f392bfff640 1 -- 172.21.15.112:0/2331080883 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3934009db0 con 0x7f393006be60 2024-04-25T01:41:18.906 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T01:41:18.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.920+0000 7f394a04d640 1 -- 172.21.15.112:0/2331080883 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f393006be60 msgr2=0x7f393006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.920+0000 7f394a04d640 1 --2- 172.21.15.112:0/2331080883 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f393006be60 0x7f393006e320 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3934009830 tx=0x7f3934008040 comp rx=0 tx=0).stop 2024-04-25T01:41:18.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.921+0000 7f394a04d640 1 -- 172.21.15.112:0/2331080883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944145a30 msgr2=0x7f3944145420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:18.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.921+0000 7f394a04d640 1 --2- 172.21.15.112:0/2331080883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944145a30 0x7f3944145420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f393c00a930 tx=0x7f3944152e40 comp rx=0 tx=0).stop 2024-04-25T01:41:18.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.921+0000 7f394a04d640 1 -- 172.21.15.112:0/2331080883 shutdown_connections 2024-04-25T01:41:18.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.921+0000 7f394a04d640 1 --2- 172.21.15.112:0/2331080883 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f393006be60 0x7f393006e320 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.921+0000 7f394a04d640 1 --2- 172.21.15.112:0/2331080883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944145a30 0x7f3944145420 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.921+0000 7f394a04d640 1 --2- 172.21.15.112:0/2331080883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3944150d80 0x7f3944144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.921+0000 7f394a04d640 1 --2- 172.21.15.112:0/2331080883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f392c0051c0 0x7f3944147a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:18.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.921+0000 7f394a04d640 1 -- 172.21.15.112:0/2331080883 >> 172.21.15.112:0/2331080883 conn(0x7f394406b160 msgr2=0x7f3944071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:18.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.921+0000 7f394a04d640 1 -- 172.21.15.112:0/2331080883 shutdown_connections 2024-04-25T01:41:18.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:18.921+0000 7f394a04d640 1 -- 172.21.15.112:0/2331080883 wait complete. 2024-04-25T01:41:18.919 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T01:41:18.919 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:41:19.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.033+0000 7f6c2099e640 1 Processor -- start 2024-04-25T01:41:19.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.033+0000 7f6c2099e640 1 -- start start 2024-04-25T01:41:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.034+0000 7f6c2099e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6c1c0c2d70 0x7f6c1c1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.034+0000 7f6c2099e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6c1c06e600 0x7f6c1c06cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.034+0000 7f6c2099e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c1c06d560 con 0x7f6c1c14c380 2024-04-25T01:41:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.034+0000 7f6c2099e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f6c1c06d840 con 0x7f6c1c0c2d70 2024-04-25T01:41:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.034+0000 7f6c2099e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6c1c06db20 con 0x7f6c1c06e600 2024-04-25T01:41:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.034+0000 7f6c1a59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6c1c0c2d70 0x7f6c1c1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.034+0000 7f6c1a59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6c1c0c2d70 0x7f6c1c1524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54916/0 (socket says 172.21.15.112:54916) 2024-04-25T01:41:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.034+0000 7f6c1a59c640 1 -- 172.21.15.112:0/1820787530 learned_addr learned my addr 172.21.15.112:0/1820787530 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.034+0000 7f6c13fff640 1 --2- 172.21.15.112:0/1820787530 >> v2:172.21.15.187:3300/0 conn(0x7f6c1c06e600 0x7f6c1c06cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.034+0000 7f6c1a59c640 1 -- 172.21.15.112:0/1820787530 >> v2:172.21.15.187:3300/0 conn(0x7f6c1c06e600 msgr2=0x7f6c1c06cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:19.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.034+0000 7f6c1a59c640 1 --2- 172.21.15.112:0/1820787530 >> v2:172.21.15.187:3300/0 conn(0x7f6c1c06e600 0x7f6c1c06cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.034+0000 7f6c1a59c640 1 -- 172.21.15.112:0/1820787530 >> v1:172.21.15.187:6789/0 conn(0x7f6c1c14c380 legacy=0x7f6c1c1539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.034+0000 7f6c1a59c640 1 -- 172.21.15.112:0/1820787530 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1c06de00 con 0x7f6c1c0c2d70 2024-04-25T01:41:19.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.035+0000 7f6c1a59c640 1 --2- 172.21.15.112:0/1820787530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c1c0c2d70 0x7f6c1c1524b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c06f240 tx=0x7f6c0c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76fd26f5f04fe25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:19.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.035+0000 7f6c19d9b640 1 -- 172.21.15.112:0/1820787530 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c037070 con 0x7f6c1c0c2d70 2024-04-25T01:41:19.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.035+0000 7f6c19d9b640 1 -- 172.21.15.112:0/1820787530 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c0c032440 con 0x7f6c1c0c2d70 2024-04-25T01:41:19.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.035+0000 7f6c19d9b640 1 -- 172.21.15.112:0/1820787530 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c032810 con 0x7f6c1c0c2d70 2024-04-25T01:41:19.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.035+0000 7f6c2099e640 1 -- 172.21.15.112:0/1820787530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c1c0c2d70 msgr2=0x7f6c1c1524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.035+0000 7f6c2099e640 1 --2- 172.21.15.112:0/1820787530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c1c0c2d70 0x7f6c1c1524b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c06f240 tx=0x7f6c0c02a1e0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.035+0000 7f6c2099e640 1 -- 172.21.15.112:0/1820787530 shutdown_connections 2024-04-25T01:41:19.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.035+0000 7f6c2099e640 1 --2- 172.21.15.112:0/1820787530 >> v2:172.21.15.187:3300/0 conn(0x7f6c1c06e600 0x7f6c1c06cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.035+0000 7f6c2099e640 1 --2- 172.21.15.112:0/1820787530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c1c0c2d70 0x7f6c1c1524b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.035+0000 7f6c2099e640 1 -- 172.21.15.112:0/1820787530 >> 172.21.15.112:0/1820787530 conn(0x7f6c1c06b8a0 msgr2=0x7f6c1c0c6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:19.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.035+0000 7f6c2099e640 1 -- 172.21.15.112:0/1820787530 shutdown_connections 2024-04-25T01:41:19.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.035+0000 7f6c2099e640 1 -- 172.21.15.112:0/1820787530 wait complete. 2024-04-25T01:41:19.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.037+0000 7f6c2099e640 1 Processor -- start 2024-04-25T01:41:19.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.037+0000 7f6c2099e640 1 -- start start 2024-04-25T01:41:19.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.037+0000 7f6c2099e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c1c06e600 0x7f6c1c14cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.037+0000 7f6c2099e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c1c0c2d70 0x7f6c1c1453c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.037+0000 7f6c2099e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c1c14c380 0x7f6c1c071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.037+0000 7f6c2099e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c1540b0 con 0x7f6c1c06e600 2024-04-25T01:41:19.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.037+0000 7f6c2099e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c0c1690 con 0x7f6c1c14c380 2024-04-25T01:41:19.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.037+0000 7f6c2099e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6c1c154230 con 0x7f6c1c0c2d70 2024-04-25T01:41:19.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.037+0000 7f6c1a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c1c06e600 0x7f6c1c14cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.037+0000 7f6c1a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c1c06e600 0x7f6c1c14cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54922/0 (socket says 172.21.15.112:54922) 2024-04-25T01:41:19.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.037+0000 7f6c1a59c640 1 -- 172.21.15.112:0/4220044759 learned_addr learned my addr 172.21.15.112:0/4220044759 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:19.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.038+0000 7f6c1a59c640 1 -- 172.21.15.112:0/4220044759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c1c0c2d70 msgr2=0x7f6c1c1453c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.038+0000 7f6c1a59c640 1 --2- 172.21.15.112:0/4220044759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c1c0c2d70 0x7f6c1c1453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.038+0000 7f6c1a59c640 1 -- 172.21.15.112:0/4220044759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c1c14c380 msgr2=0x7f6c1c071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.038+0000 7f6c1a59c640 1 --2- 172.21.15.112:0/4220044759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c1c14c380 0x7f6c1c071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.038+0000 7f6c1a59c640 1 -- 172.21.15.112:0/4220044759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1c14e200 con 0x7f6c1c06e600 2024-04-25T01:41:19.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.038+0000 7f6c1a59c640 1 --2- 172.21.15.112:0/4220044759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c1c06e600 0x7f6c1c14cda0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c02a220 tx=0x7f6c0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:19.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.038+0000 7f6c12ffd640 1 -- 172.21.15.112:0/4220044759 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c041020 con 0x7f6c1c06e600 2024-04-25T01:41:19.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.038+0000 7f6c12ffd640 1 -- 172.21.15.112:0/4220044759 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c0c0306e0 con 0x7f6c1c06e600 2024-04-25T01:41:19.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.038+0000 7f6c2099e640 1 -- 172.21.15.112:0/4220044759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c1c150020 con 0x7f6c1c06e600 2024-04-25T01:41:19.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.038+0000 7f6c12ffd640 1 -- 172.21.15.112:0/4220044759 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c037040 con 0x7f6c1c06e600 2024-04-25T01:41:19.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.038+0000 7f6c2099e640 1 -- 172.21.15.112:0/4220044759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c1c1e8d50 con 0x7f6c1c06e600 2024-04-25T01:41:19.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.039+0000 7f6c2099e640 1 -- 172.21.15.112:0/4220044759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6c1c0682a0 con 0x7f6c1c06e600 2024-04-25T01:41:19.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.039+0000 7f6c12ffd640 1 -- 172.21.15.112:0/4220044759 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6c0c044020 con 0x7f6c1c06e600 2024-04-25T01:41:19.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.040+0000 7f6c12ffd640 1 --2- 172.21.15.112:0/4220044759 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6bfc06be60 0x7f6bfc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.040+0000 7f6c12ffd640 1 -- 172.21.15.112:0/4220044759 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f6c0c0ab990 con 0x7f6c1c06e600 2024-04-25T01:41:19.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.040+0000 7f6c13fff640 1 --2- 172.21.15.112:0/4220044759 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6bfc06be60 0x7f6bfc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.040+0000 7f6c12ffd640 1 --2- 172.21.15.112:0/4220044759 >> [v2:172.21.15.112:6824/2172872078,v1:172.21.15.112:6825/2172872078] conn(0x7f6bfc076800 0x7f6bfc078c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.040+0000 7f6c12ffd640 1 -- 172.21.15.112:0/4220044759 --> [v2:172.21.15.112:6824/2172872078,v1:172.21.15.112:6825/2172872078] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6c0c02cea0 con 0x7f6bfc076800 2024-04-25T01:41:19.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.040+0000 7f6c12ffd640 1 -- 172.21.15.112:0/4220044759 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6c0c0b02c0 con 0x7f6c1c06e600 2024-04-25T01:41:19.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.040+0000 7f6c1ad9d640 1 --2- 172.21.15.112:0/4220044759 >> [v2:172.21.15.112:6824/2172872078,v1:172.21.15.112:6825/2172872078] conn(0x7f6bfc076800 0x7f6bfc078c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.040+0000 7f6c13fff640 1 --2- 172.21.15.112:0/4220044759 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6bfc06be60 0x7f6bfc06e320 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c14b4b0 tx=0x7f6c04008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:19.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.040+0000 7f6c1ad9d640 1 --2- 172.21.15.112:0/4220044759 >> [v2:172.21.15.112:6824/2172872078,v1:172.21.15.112:6825/2172872078] conn(0x7f6bfc076800 0x7f6bfc078c80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:19.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.041+0000 7f6c12ffd640 1 -- 172.21.15.112:0/4220044759 <== osd.2 v2:172.21.15.112:6824/2172872078 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6c0c02cea0 con 0x7f6bfc076800 2024-04-25T01:41:19.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.054+0000 7f6c2099e640 1 -- 172.21.15.112:0/4220044759 --> [v2:172.21.15.112:6824/2172872078,v1:172.21.15.112:6825/2172872078] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6c1c147ce0 con 0x7f6bfc076800 2024-04-25T01:41:19.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.054+0000 7f6c12ffd640 1 -- 172.21.15.112:0/4220044759 <== osd.2 v2:172.21.15.112:6824/2172872078 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6c1c147ce0 con 0x7f6bfc076800 2024-04-25T01:41:19.042 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:41:19.042 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:41:19.043 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:41:19.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.054+0000 7f6c2099e640 1 -- 172.21.15.112:0/4220044759 >> [v2:172.21.15.112:6824/2172872078,v1:172.21.15.112:6825/2172872078] conn(0x7f6bfc076800 msgr2=0x7f6bfc078c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.054+0000 7f6c2099e640 1 --2- 172.21.15.112:0/4220044759 >> [v2:172.21.15.112:6824/2172872078,v1:172.21.15.112:6825/2172872078] conn(0x7f6bfc076800 0x7f6bfc078c80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.055+0000 7f6c2099e640 1 -- 172.21.15.112:0/4220044759 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6bfc06be60 msgr2=0x7f6bfc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.055+0000 7f6c2099e640 1 --2- 172.21.15.112:0/4220044759 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6bfc06be60 0x7f6bfc06e320 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c14b4b0 tx=0x7f6c04008040 comp rx=0 tx=0).stop 2024-04-25T01:41:19.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.055+0000 7f6c2099e640 1 -- 172.21.15.112:0/4220044759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c1c06e600 msgr2=0x7f6c1c14cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.055+0000 7f6c2099e640 1 --2- 172.21.15.112:0/4220044759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c1c06e600 0x7f6c1c14cda0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c02a220 tx=0x7f6c0c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.055+0000 7f6c1ad9d640 1 -- 172.21.15.112:0/4220044759 reap_dead start 2024-04-25T01:41:19.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.055+0000 7f6c2099e640 1 -- 172.21.15.112:0/4220044759 shutdown_connections 2024-04-25T01:41:19.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.055+0000 7f6c2099e640 1 -- 172.21.15.112:0/4220044759 >> 172.21.15.112:0/4220044759 conn(0x7f6c1c06b8a0 msgr2=0x7f6c1c1474b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:19.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.055+0000 7f6c2099e640 1 -- 172.21.15.112:0/4220044759 shutdown_connections 2024-04-25T01:41:19.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.055+0000 7f6c2099e640 1 -- 172.21.15.112:0/4220044759 wait complete. 2024-04-25T01:41:19.053 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T01:41:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.163+0000 7fbf09596640 1 Processor -- start 2024-04-25T01:41:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.163+0000 7fbf09596640 1 -- start start 2024-04-25T01:41:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.163+0000 7fbf09596640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbf04152e50 0x7fbf040c9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.163+0000 7fbf09596640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf040c9840 con 0x7fbf0406d6f0 2024-04-25T01:41:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.163+0000 7fbf09596640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf04155a80 con 0x7fbf0406bda0 2024-04-25T01:41:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.163+0000 7fbf09596640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fbf04156690 con 0x7fbf04152e50 2024-04-25T01:41:19.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.163+0000 7fbf02ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbf04152e50 0x7fbf040c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.163+0000 7fbf027fc640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fbf0406d6f0 0x7fbf040c7bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54628/0 (socket says 172.21.15.112:54628) 2024-04-25T01:41:19.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.163+0000 7fbf027fc640 1 -- 172.21.15.112:0/1445863068 learned_addr learned my addr 172.21.15.112:0/1445863068 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:19.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.164+0000 7fbf02ffd640 1 -- 172.21.15.112:0/1445863068 >> v1:172.21.15.187:6789/0 conn(0x7fbf0406bda0 legacy=0x7fbf0406c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:19.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.164+0000 7fbf02ffd640 1 -- 172.21.15.112:0/1445863068 >> v1:172.21.15.112:6789/0 conn(0x7fbf0406d6f0 legacy=0x7fbf040c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.164+0000 7fbf02ffd640 1 -- 172.21.15.112:0/1445863068 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf04155d60 con 0x7fbf04152e50 2024-04-25T01:41:19.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.164+0000 7fbf02ffd640 1 --2- 172.21.15.112:0/1445863068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf04152e50 0x7fbf040c9300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbef40097a0 tx=0x7fbef402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5549428b269f99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:19.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.165+0000 7fbf01ffb640 1 -- 172.21.15.112:0/1445863068 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4037070 con 0x7fbf04152e50 2024-04-25T01:41:19.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.165+0000 7fbf01ffb640 1 -- 172.21.15.112:0/1445863068 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef402adc0 con 0x7fbf04152e50 2024-04-25T01:41:19.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.165+0000 7fbf01ffb640 1 -- 172.21.15.112:0/1445863068 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4032660 con 0x7fbf04152e50 2024-04-25T01:41:19.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.165+0000 7fbf09596640 1 -- 172.21.15.112:0/1445863068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf04152e50 msgr2=0x7fbf040c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.165+0000 7fbf09596640 1 --2- 172.21.15.112:0/1445863068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf04152e50 0x7fbf040c9300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbef40097a0 tx=0x7fbef402a0b0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.165+0000 7fbf09596640 1 -- 172.21.15.112:0/1445863068 shutdown_connections 2024-04-25T01:41:19.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.165+0000 7fbf09596640 1 --2- 172.21.15.112:0/1445863068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf04152e50 0x7fbf040c9300 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.165+0000 7fbf09596640 1 -- 172.21.15.112:0/1445863068 >> 172.21.15.112:0/1445863068 conn(0x7fbf040acd30 msgr2=0x7fbf040ad110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:19.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.165+0000 7fbf09596640 1 -- 172.21.15.112:0/1445863068 shutdown_connections 2024-04-25T01:41:19.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.165+0000 7fbf09596640 1 -- 172.21.15.112:0/1445863068 wait complete. 2024-04-25T01:41:19.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.166+0000 7fbf09596640 1 Processor -- start 2024-04-25T01:41:19.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.166+0000 7fbf09596640 1 -- start start 2024-04-25T01:41:19.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.167+0000 7fbf09596640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf0406bda0 0x7fbf04154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.167+0000 7fbf09596640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf0406d6f0 0x7fbf041eb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.167+0000 7fbf09596640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf04152e50 0x7fbf041ebe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.167+0000 7fbf09596640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbf040c8300 con 0x7fbf0406bda0 2024-04-25T01:41:19.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.167+0000 7fbf09596640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04156320 con 0x7fbf0406d6f0 2024-04-25T01:41:19.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.167+0000 7fbf09596640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbf040c8480 con 0x7fbf04152e50 2024-04-25T01:41:19.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.167+0000 7fbf02ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf0406bda0 0x7fbf04154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.167+0000 7fbf02ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf0406bda0 0x7fbf04154fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54930/0 (socket says 172.21.15.112:54930) 2024-04-25T01:41:19.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.167+0000 7fbf02ffd640 1 -- 172.21.15.112:0/3064652911 learned_addr learned my addr 172.21.15.112:0/3064652911 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.167+0000 7fbf02ffd640 1 -- 172.21.15.112:0/3064652911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf04152e50 msgr2=0x7fbf041ebe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.167+0000 7fbf02ffd640 1 --2- 172.21.15.112:0/3064652911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf04152e50 0x7fbf041ebe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.167+0000 7fbf02ffd640 1 -- 172.21.15.112:0/3064652911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf0406d6f0 msgr2=0x7fbf041eb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.167+0000 7fbf02ffd640 1 --2- 172.21.15.112:0/3064652911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf0406d6f0 0x7fbf041eb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.167+0000 7fbf02ffd640 1 -- 172.21.15.112:0/3064652911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf040c4370 con 0x7fbf0406bda0 2024-04-25T01:41:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.168+0000 7fbf02ffd640 1 --2- 172.21.15.112:0/3064652911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf0406bda0 0x7fbf04154fe0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbef4000c00 tx=0x7fbef40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.168+0000 7fbeeb7fe640 1 -- 172.21.15.112:0/3064652911 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4041020 con 0x7fbf0406bda0 2024-04-25T01:41:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.168+0000 7fbeeb7fe640 1 -- 172.21.15.112:0/3064652911 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef403d620 con 0x7fbf0406bda0 2024-04-25T01:41:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.168+0000 7fbf09596640 1 -- 172.21.15.112:0/3064652911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf040c50b0 con 0x7fbf0406bda0 2024-04-25T01:41:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.168+0000 7fbeeb7fe640 1 -- 172.21.15.112:0/3064652911 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4037040 con 0x7fbf0406bda0 2024-04-25T01:41:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.168+0000 7fbf09596640 1 -- 172.21.15.112:0/3064652911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf04155520 con 0x7fbf0406bda0 2024-04-25T01:41:19.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.169+0000 7fbf09596640 1 -- 172.21.15.112:0/3064652911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf040682a0 con 0x7fbf0406bda0 2024-04-25T01:41:19.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.169+0000 7fbeeb7fe640 1 -- 172.21.15.112:0/3064652911 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbef4044020 con 0x7fbf0406bda0 2024-04-25T01:41:19.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.169+0000 7fbeeb7fe640 1 --2- 172.21.15.112:0/3064652911 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbef006be30 0x7fbef006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.170+0000 7fbeeb7fe640 1 -- 172.21.15.112:0/3064652911 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fbef40ab6e0 con 0x7fbf0406bda0 2024-04-25T01:41:19.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.170+0000 7fbf027fc640 1 --2- 172.21.15.112:0/3064652911 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbef006be30 0x7fbef006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.170+0000 7fbf027fc640 1 --2- 172.21.15.112:0/3064652911 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbef006be30 0x7fbef006e2f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbef8000f60 tx=0x7fbef8007a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:19.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.175+0000 7fbeeb7fe640 1 -- 172.21.15.112:0/3064652911 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbef404b030 con 0x7fbf0406bda0 2024-04-25T01:41:19.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.347+0000 7fbf09596640 1 -- 172.21.15.112:0/3064652911 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fbf040c4e60 con 0x7fbef006be30 2024-04-25T01:41:19.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.347+0000 7fbeeb7fe640 1 -- 172.21.15.112:0/3064652911 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbf040c4e60 con 0x7fbef006be30 2024-04-25T01:41:19.335 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T01:41:19.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.350+0000 7fbf09596640 1 -- 172.21.15.112:0/3064652911 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbef006be30 msgr2=0x7fbef006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.350+0000 7fbf09596640 1 --2- 172.21.15.112:0/3064652911 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbef006be30 0x7fbef006e2f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbef8000f60 tx=0x7fbef8007a80 comp rx=0 tx=0).stop 2024-04-25T01:41:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.350+0000 7fbf09596640 1 -- 172.21.15.112:0/3064652911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf0406bda0 msgr2=0x7fbf04154fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.350+0000 7fbf09596640 1 --2- 172.21.15.112:0/3064652911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf0406bda0 0x7fbf04154fe0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbef4000c00 tx=0x7fbef40046c0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.350+0000 7fbf09596640 1 -- 172.21.15.112:0/3064652911 shutdown_connections 2024-04-25T01:41:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.350+0000 7fbf09596640 1 --2- 172.21.15.112:0/3064652911 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbef006be30 0x7fbef006e2f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.350+0000 7fbf09596640 1 --2- 172.21.15.112:0/3064652911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf04152e50 0x7fbf041ebe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.350+0000 7fbf09596640 1 --2- 172.21.15.112:0/3064652911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf0406d6f0 0x7fbf041eb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.350+0000 7fbf09596640 1 --2- 172.21.15.112:0/3064652911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf0406bda0 0x7fbf04154fe0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.350+0000 7fbf09596640 1 -- 172.21.15.112:0/3064652911 >> 172.21.15.112:0/3064652911 conn(0x7fbf040acd30 msgr2=0x7fbf040ac450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:19.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.350+0000 7fbf09596640 1 -- 172.21.15.112:0/3064652911 shutdown_connections 2024-04-25T01:41:19.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.350+0000 7fbf09596640 1 -- 172.21.15.112:0/3064652911 wait complete. 2024-04-25T01:41:19.349 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T01:41:19.349 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:41:19.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.463+0000 7fd3cd75a640 1 Processor -- start 2024-04-25T01:41:19.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.463+0000 7fd3cd75a640 1 -- start start 2024-04-25T01:41:19.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.464+0000 7fd3cd75a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd3c81566c0 0x7fd3c80c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.464+0000 7fd3cd75a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3c80c6270 con 0x7fd3c806cef0 2024-04-25T01:41:19.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.464+0000 7fd3cd75a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3c806c410 con 0x7fd3c814fbd0 2024-04-25T01:41:19.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.464+0000 7fd3cd75a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fd3c806c1c0 con 0x7fd3c81566c0 2024-04-25T01:41:19.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.464+0000 7fd3c77fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd3c81566c0 0x7fd3c80c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.464+0000 7fd3c67fc640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd3c806cef0 0x7fd3c80c8070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54636/0 (socket says 172.21.15.112:54636) 2024-04-25T01:41:19.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.464+0000 7fd3c67fc640 1 -- 172.21.15.112:0/2138837990 learned_addr learned my addr 172.21.15.112:0/2138837990 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:19.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.464+0000 7fd3c77fe640 1 -- 172.21.15.112:0/2138837990 >> v1:172.21.15.187:6789/0 conn(0x7fd3c814fbd0 legacy=0x7fd3c80c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.464+0000 7fd3c5ffb640 1 -- 172.21.15.112:0/2138837990 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 989489662 0 0) 0x7fd3c80c6270 con 0x7fd3c806cef0 2024-04-25T01:41:19.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.464+0000 7fd3c77fe640 1 -- 172.21.15.112:0/2138837990 >> v1:172.21.15.112:6789/0 conn(0x7fd3c806cef0 legacy=0x7fd3c80c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.464+0000 7fd3c77fe640 1 -- 172.21.15.112:0/2138837990 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c806c6f0 con 0x7fd3c81566c0 2024-04-25T01:41:19.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.465+0000 7fd3c77fe640 1 --2- 172.21.15.112:0/2138837990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c81566c0 0x7fd3c80c91e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00097a0 tx=0x7fd3c002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201f4658ca0a4354 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:19.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.465+0000 7fd3c5ffb640 1 -- 172.21.15.112:0/2138837990 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0037070 con 0x7fd3c81566c0 2024-04-25T01:41:19.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.465+0000 7fd3c5ffb640 1 -- 172.21.15.112:0/2138837990 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c0032440 con 0x7fd3c81566c0 2024-04-25T01:41:19.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.465+0000 7fd3c5ffb640 1 -- 172.21.15.112:0/2138837990 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c002ae60 con 0x7fd3c81566c0 2024-04-25T01:41:19.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.466+0000 7fd3cd75a640 1 -- 172.21.15.112:0/2138837990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c81566c0 msgr2=0x7fd3c80c91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.466+0000 7fd3cd75a640 1 --2- 172.21.15.112:0/2138837990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c81566c0 0x7fd3c80c91e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00097a0 tx=0x7fd3c002a110 comp rx=0 tx=0).stop 2024-04-25T01:41:19.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.466+0000 7fd3cd75a640 1 -- 172.21.15.112:0/2138837990 shutdown_connections 2024-04-25T01:41:19.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.466+0000 7fd3cd75a640 1 --2- 172.21.15.112:0/2138837990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c81566c0 0x7fd3c80c91e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.466+0000 7fd3cd75a640 1 -- 172.21.15.112:0/2138837990 >> 172.21.15.112:0/2138837990 conn(0x7fd3c80ad050 msgr2=0x7fd3c80ad460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:19.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.466+0000 7fd3cd75a640 1 -- 172.21.15.112:0/2138837990 shutdown_connections 2024-04-25T01:41:19.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.466+0000 7fd3cd75a640 1 -- 172.21.15.112:0/2138837990 wait complete. 2024-04-25T01:41:19.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.467+0000 7fd3cd75a640 1 Processor -- start 2024-04-25T01:41:19.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.467+0000 7fd3cd75a640 1 -- start start 2024-04-25T01:41:19.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3cd75a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c806cef0 0x7fd3c81581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3cd75a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3c814fbd0 0x7fd3c80beed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3cd75a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c81566c0 0x7fd3c81493b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3cd75a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c80c8900 con 0x7fd3c806cef0 2024-04-25T01:41:19.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3cd75a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c80c8780 con 0x7fd3c81566c0 2024-04-25T01:41:19.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3cd75a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c80c9720 con 0x7fd3c814fbd0 2024-04-25T01:41:19.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3c6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c806cef0 0x7fd3c81581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3c6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c806cef0 0x7fd3c81581c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54952/0 (socket says 172.21.15.112:54952) 2024-04-25T01:41:19.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3c6ffd640 1 -- 172.21.15.112:0/2770492903 learned_addr learned my addr 172.21.15.112:0/2770492903 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:19.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3c67fc640 1 --2- 172.21.15.112:0/2770492903 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3c814fbd0 0x7fd3c80beed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3c6ffd640 1 -- 172.21.15.112:0/2770492903 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3c814fbd0 msgr2=0x7fd3c80beed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3c6ffd640 1 --2- 172.21.15.112:0/2770492903 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3c814fbd0 0x7fd3c80beed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3c77fe640 1 --2- 172.21.15.112:0/2770492903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c81566c0 0x7fd3c81493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3c6ffd640 1 -- 172.21.15.112:0/2770492903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c81566c0 msgr2=0x7fd3c81493b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3c6ffd640 1 --2- 172.21.15.112:0/2770492903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c81566c0 0x7fd3c81493b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3c6ffd640 1 -- 172.21.15.112:0/2770492903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c8158bc0 con 0x7fd3c806cef0 2024-04-25T01:41:19.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3c77fe640 1 --2- 172.21.15.112:0/2770492903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c81566c0 0x7fd3c81493b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:41:19.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3c6ffd640 1 --2- 172.21.15.112:0/2770492903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c806cef0 0x7fd3c81581c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800c7c0 tx=0x7fd3b800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:19.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3af7fe640 1 -- 172.21.15.112:0/2770492903 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8016020 con 0x7fd3c806cef0 2024-04-25T01:41:19.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3af7fe640 1 -- 172.21.15.112:0/2770492903 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b8006d50 con 0x7fd3c806cef0 2024-04-25T01:41:19.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3cd75a640 1 -- 172.21.15.112:0/2770492903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c81599a0 con 0x7fd3c806cef0 2024-04-25T01:41:19.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.468+0000 7fd3af7fe640 1 -- 172.21.15.112:0/2770492903 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8003bb0 con 0x7fd3c806cef0 2024-04-25T01:41:19.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.469+0000 7fd3cd75a640 1 -- 172.21.15.112:0/2770492903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c81579c0 con 0x7fd3c806cef0 2024-04-25T01:41:19.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.469+0000 7fd3cd75a640 1 -- 172.21.15.112:0/2770492903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd3c8159750 con 0x7fd3c806cef0 2024-04-25T01:41:19.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.470+0000 7fd3af7fe640 1 -- 172.21.15.112:0/2770492903 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd3b800a850 con 0x7fd3c806cef0 2024-04-25T01:41:19.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.470+0000 7fd3af7fe640 1 --2- 172.21.15.112:0/2770492903 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd3b406be30 0x7fd3b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.470+0000 7fd3af7fe640 1 -- 172.21.15.112:0/2770492903 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fd3b808aea0 con 0x7fd3c806cef0 2024-04-25T01:41:19.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.470+0000 7fd3c67fc640 1 --2- 172.21.15.112:0/2770492903 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd3b406be30 0x7fd3b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.471+0000 7fd3af7fe640 1 --2- 172.21.15.112:0/2770492903 >> [v2:172.21.15.112:6808/1418784374,v1:172.21.15.112:6809/1418784374] conn(0x7fd3b40767d0 0x7fd3b4078c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.471+0000 7fd3af7fe640 1 -- 172.21.15.112:0/2770492903 --> [v2:172.21.15.112:6808/1418784374,v1:172.21.15.112:6809/1418784374] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd3b801eec0 con 0x7fd3b40767d0 2024-04-25T01:41:19.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.471+0000 7fd3af7fe640 1 -- 172.21.15.112:0/2770492903 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd3b808d5a0 con 0x7fd3c806cef0 2024-04-25T01:41:19.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.471+0000 7fd3c77fe640 1 --2- 172.21.15.112:0/2770492903 >> [v2:172.21.15.112:6808/1418784374,v1:172.21.15.112:6809/1418784374] conn(0x7fd3b40767d0 0x7fd3b4078c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.471+0000 7fd3c67fc640 1 --2- 172.21.15.112:0/2770492903 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd3b406be30 0x7fd3b406e2f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc007fd0 tx=0x7fd3bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:19.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.471+0000 7fd3c77fe640 1 --2- 172.21.15.112:0/2770492903 >> [v2:172.21.15.112:6808/1418784374,v1:172.21.15.112:6809/1418784374] conn(0x7fd3b40767d0 0x7fd3b4078c50 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:19.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.472+0000 7fd3af7fe640 1 -- 172.21.15.112:0/2770492903 <== osd.3 v2:172.21.15.112:6808/1418784374 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd3b801eec0 con 0x7fd3b40767d0 2024-04-25T01:41:19.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.485+0000 7fd3cd75a640 1 -- 172.21.15.112:0/2770492903 --> [v2:172.21.15.112:6808/1418784374,v1:172.21.15.112:6809/1418784374] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd3c80682a0 con 0x7fd3b40767d0 2024-04-25T01:41:19.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.485+0000 7fd3af7fe640 1 -- 172.21.15.112:0/2770492903 <== osd.3 v2:172.21.15.112:6808/1418784374 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd3c80682a0 con 0x7fd3b40767d0 2024-04-25T01:41:19.472 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:41:19.472 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:41:19.472 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:41:19.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.485+0000 7fd3cd75a640 1 -- 172.21.15.112:0/2770492903 >> [v2:172.21.15.112:6808/1418784374,v1:172.21.15.112:6809/1418784374] conn(0x7fd3b40767d0 msgr2=0x7fd3b4078c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.485+0000 7fd3cd75a640 1 --2- 172.21.15.112:0/2770492903 >> [v2:172.21.15.112:6808/1418784374,v1:172.21.15.112:6809/1418784374] conn(0x7fd3b40767d0 0x7fd3b4078c50 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.485+0000 7fd3cd75a640 1 -- 172.21.15.112:0/2770492903 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd3b406be30 msgr2=0x7fd3b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.485+0000 7fd3cd75a640 1 --2- 172.21.15.112:0/2770492903 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd3b406be30 0x7fd3b406e2f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc007fd0 tx=0x7fd3bc009040 comp rx=0 tx=0).stop 2024-04-25T01:41:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.485+0000 7fd3cd75a640 1 -- 172.21.15.112:0/2770492903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c806cef0 msgr2=0x7fd3c81581c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.485+0000 7fd3cd75a640 1 --2- 172.21.15.112:0/2770492903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c806cef0 0x7fd3c81581c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800c7c0 tx=0x7fd3b800cc80 comp rx=0 tx=0).stop 2024-04-25T01:41:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.485+0000 7fd3c77fe640 1 -- 172.21.15.112:0/2770492903 reap_dead start 2024-04-25T01:41:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.485+0000 7fd3cd75a640 1 -- 172.21.15.112:0/2770492903 shutdown_connections 2024-04-25T01:41:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.485+0000 7fd3cd75a640 1 -- 172.21.15.112:0/2770492903 >> 172.21.15.112:0/2770492903 conn(0x7fd3c80ad050 msgr2=0x7fd3c80c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.485+0000 7fd3cd75a640 1 -- 172.21.15.112:0/2770492903 shutdown_connections 2024-04-25T01:41:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.485+0000 7fd3cd75a640 1 -- 172.21.15.112:0/2770492903 wait complete. 2024-04-25T01:41:19.483 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T01:41:19.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.593+0000 7f060d51b640 1 Processor -- start 2024-04-25T01:41:19.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.593+0000 7f060d51b640 1 -- start start 2024-04-25T01:41:19.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.594+0000 7f060d51b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f06081592c0 0x7f06080c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.594+0000 7f060d51b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0608159b90 con 0x7f06080c72b0 2024-04-25T01:41:19.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.594+0000 7f060d51b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f060806c970 con 0x7f06081596a0 2024-04-25T01:41:19.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.594+0000 7f060d51b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f06080c6e20 con 0x7f06081592c0 2024-04-25T01:41:19.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.594+0000 7f0606ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f06081592c0 0x7f06080c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.594+0000 7f06077fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f06080c72b0 0x7f0608160380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54640/0 (socket says 172.21.15.112:54640) 2024-04-25T01:41:19.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.594+0000 7f0606ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f06081592c0 0x7f06080c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54962/0 (socket says 172.21.15.112:54962) 2024-04-25T01:41:19.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.594+0000 7f06077fe640 1 -- 172.21.15.112:0/4081428202 learned_addr learned my addr 172.21.15.112:0/4081428202 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:19.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.595+0000 7f0606ffd640 1 -- 172.21.15.112:0/4081428202 >> v1:172.21.15.187:6789/0 conn(0x7f06081596a0 legacy=0x7f0608152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:19.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.595+0000 7f0606ffd640 1 -- 172.21.15.112:0/4081428202 >> v1:172.21.15.112:6789/0 conn(0x7f06080c72b0 legacy=0x7f0608160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.595+0000 7f0606ffd640 1 -- 172.21.15.112:0/4081428202 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060806cc50 con 0x7f06081592c0 2024-04-25T01:41:19.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.595+0000 7f0606ffd640 1 --2- 172.21.15.112:0/4081428202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06081592c0 0x7f06080c9070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009d90 tx=0x7f05f802a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65cd44aef77d1841 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:19.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.595+0000 7f0605ffb640 1 -- 172.21.15.112:0/4081428202 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8007740 con 0x7f06081592c0 2024-04-25T01:41:19.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.595+0000 7f0605ffb640 1 -- 172.21.15.112:0/4081428202 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f80078e0 con 0x7f06081592c0 2024-04-25T01:41:19.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.595+0000 7f0605ffb640 1 -- 172.21.15.112:0/4081428202 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8007cb0 con 0x7f06081592c0 2024-04-25T01:41:19.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.596+0000 7f060d51b640 1 -- 172.21.15.112:0/4081428202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06081592c0 msgr2=0x7f06080c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.596+0000 7f060d51b640 1 --2- 172.21.15.112:0/4081428202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06081592c0 0x7f06080c9070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009d90 tx=0x7f05f802a580 comp rx=0 tx=0).stop 2024-04-25T01:41:19.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.596+0000 7f060d51b640 1 -- 172.21.15.112:0/4081428202 shutdown_connections 2024-04-25T01:41:19.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.596+0000 7f060d51b640 1 --2- 172.21.15.112:0/4081428202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06081592c0 0x7f06080c9070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.596+0000 7f060d51b640 1 -- 172.21.15.112:0/4081428202 >> 172.21.15.112:0/4081428202 conn(0x7f06080cb170 msgr2=0x7f06080cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:19.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.596+0000 7f060d51b640 1 -- 172.21.15.112:0/4081428202 shutdown_connections 2024-04-25T01:41:19.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.596+0000 7f060d51b640 1 -- 172.21.15.112:0/4081428202 wait complete. 2024-04-25T01:41:19.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.597+0000 7f060d51b640 1 Processor -- start 2024-04-25T01:41:19.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.597+0000 7f060d51b640 1 -- start start 2024-04-25T01:41:19.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.598+0000 7f060d51b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06080c72b0 0x7f06081ef080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.598+0000 7f060d51b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06081592c0 0x7f060814a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.598+0000 7f060d51b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06081596a0 0x7f0608157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.598+0000 7f060d51b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0608153590 con 0x7f06080c72b0 2024-04-25T01:41:19.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.598+0000 7f060d51b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0608153410 con 0x7f06081596a0 2024-04-25T01:41:19.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.598+0000 7f060d51b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f060806db20 con 0x7f06081592c0 2024-04-25T01:41:19.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.598+0000 7f0606ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06080c72b0 0x7f06081ef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.598+0000 7f0606ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06080c72b0 0x7f06081ef080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54966/0 (socket says 172.21.15.112:54966) 2024-04-25T01:41:19.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.598+0000 7f0606ffd640 1 -- 172.21.15.112:0/689963286 learned_addr learned my addr 172.21.15.112:0/689963286 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:19.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.598+0000 7f06077fe640 1 --2- 172.21.15.112:0/689963286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06081596a0 0x7f0608157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.598+0000 7f06067fc640 1 --2- 172.21.15.112:0/689963286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06081592c0 0x7f060814a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.598+0000 7f0606ffd640 1 -- 172.21.15.112:0/689963286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06081592c0 msgr2=0x7f060814a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.598+0000 7f0606ffd640 1 --2- 172.21.15.112:0/689963286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06081592c0 0x7f060814a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.598+0000 7f0606ffd640 1 -- 172.21.15.112:0/689963286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06081596a0 msgr2=0x7f0608157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.598+0000 7f0606ffd640 1 --2- 172.21.15.112:0/689963286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06081596a0 0x7f0608157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.598+0000 7f0606ffd640 1 -- 172.21.15.112:0/689963286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06081ef660 con 0x7f06080c72b0 2024-04-25T01:41:19.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.598+0000 7f06067fc640 1 --2- 172.21.15.112:0/689963286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06081592c0 0x7f060814a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:41:19.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.598+0000 7f06077fe640 1 --2- 172.21.15.112:0/689963286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06081596a0 0x7f0608157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:41:19.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.598+0000 7f0606ffd640 1 --2- 172.21.15.112:0/689963286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06080c72b0 0x7f06081ef080 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009eb0 tx=0x7f05f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:19.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.599+0000 7f05ef7fe640 1 -- 172.21.15.112:0/689963286 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8040070 con 0x7f06080c72b0 2024-04-25T01:41:19.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.599+0000 7f05ef7fe640 1 -- 172.21.15.112:0/689963286 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f8031d00 con 0x7f06080c72b0 2024-04-25T01:41:19.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.599+0000 7f05ef7fe640 1 -- 172.21.15.112:0/689963286 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8045660 con 0x7f06080c72b0 2024-04-25T01:41:19.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.599+0000 7f060d51b640 1 -- 172.21.15.112:0/689963286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06081ef940 con 0x7f06080c72b0 2024-04-25T01:41:19.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.599+0000 7f060d51b640 1 -- 172.21.15.112:0/689963286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06081eff00 con 0x7f06080c72b0 2024-04-25T01:41:19.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.600+0000 7f060d51b640 1 -- 172.21.15.112:0/689963286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06080679a0 con 0x7f06080c72b0 2024-04-25T01:41:19.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.600+0000 7f05ef7fe640 1 -- 172.21.15.112:0/689963286 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f05f8039040 con 0x7f06080c72b0 2024-04-25T01:41:19.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.600+0000 7f05ef7fe640 1 --2- 172.21.15.112:0/689963286 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f05f406be60 0x7f05f406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.601+0000 7f05ef7fe640 1 -- 172.21.15.112:0/689963286 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f05f80acb00 con 0x7f06080c72b0 2024-04-25T01:41:19.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.601+0000 7f06067fc640 1 --2- 172.21.15.112:0/689963286 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f05f406be60 0x7f05f406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.601+0000 7f06067fc640 1 --2- 172.21.15.112:0/689963286 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f05f406be60 0x7f05f406e320 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0608152580 tx=0x7f05fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:19.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.605+0000 7f05ef7fe640 1 -- 172.21.15.112:0/689963286 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f05f807b1a0 con 0x7f06080c72b0 2024-04-25T01:41:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.781+0000 7f060d51b640 1 -- 172.21.15.112:0/689963286 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f06080682a0 con 0x7f05f406be60 2024-04-25T01:41:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.782+0000 7f05ef7fe640 1 -- 172.21.15.112:0/689963286 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f06080682a0 con 0x7f05f406be60 2024-04-25T01:41:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T01:41:19.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.784+0000 7f060d51b640 1 -- 172.21.15.112:0/689963286 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f05f406be60 msgr2=0x7f05f406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.784+0000 7f060d51b640 1 --2- 172.21.15.112:0/689963286 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f05f406be60 0x7f05f406e320 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0608152580 tx=0x7f05fc008040 comp rx=0 tx=0).stop 2024-04-25T01:41:19.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.784+0000 7f060d51b640 1 -- 172.21.15.112:0/689963286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06080c72b0 msgr2=0x7f06081ef080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.784+0000 7f060d51b640 1 --2- 172.21.15.112:0/689963286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06080c72b0 0x7f06081ef080 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009eb0 tx=0x7f05f8004a40 comp rx=0 tx=0).stop 2024-04-25T01:41:19.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.784+0000 7f060d51b640 1 -- 172.21.15.112:0/689963286 shutdown_connections 2024-04-25T01:41:19.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.784+0000 7f060d51b640 1 --2- 172.21.15.112:0/689963286 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f05f406be60 0x7f05f406e320 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.784+0000 7f060d51b640 1 --2- 172.21.15.112:0/689963286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06081596a0 0x7f0608157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.784+0000 7f060d51b640 1 --2- 172.21.15.112:0/689963286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06081592c0 0x7f060814a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.784+0000 7f060d51b640 1 --2- 172.21.15.112:0/689963286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06080c72b0 0x7f06081ef080 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.784+0000 7f060d51b640 1 -- 172.21.15.112:0/689963286 >> 172.21.15.112:0/689963286 conn(0x7f06080cb170 msgr2=0x7f06080ac3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:19.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.784+0000 7f060d51b640 1 -- 172.21.15.112:0/689963286 shutdown_connections 2024-04-25T01:41:19.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.784+0000 7f060d51b640 1 -- 172.21.15.112:0/689963286 wait complete. 2024-04-25T01:41:19.783 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T01:41:19.783 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:41:19.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.896+0000 7f1b54d9e640 1 Processor -- start 2024-04-25T01:41:19.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.897+0000 7f1b54d9e640 1 -- start start 2024-04-25T01:41:19.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.897+0000 7f1b54d9e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1b5006cef0 0x7f1b500c8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.897+0000 7f1b54d9e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1b500c85b0 0x7f1b500c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.897+0000 7f1b54d9e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b500c6270 con 0x7f1b5014fbd0 2024-04-25T01:41:19.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.897+0000 7f1b54d9e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f1b5006c1c0 con 0x7f1b5006cef0 2024-04-25T01:41:19.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.897+0000 7f1b54d9e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1b5006c410 con 0x7f1b500c85b0 2024-04-25T01:41:19.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.897+0000 7f1b47fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1b5006cef0 0x7f1b500c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.897+0000 7f1b47fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1b5006cef0 0x7f1b500c8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54978/0 (socket says 172.21.15.112:54978) 2024-04-25T01:41:19.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.897+0000 7f1b47fff640 1 -- 172.21.15.112:0/504496150 learned_addr learned my addr 172.21.15.112:0/504496150 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:19.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.898+0000 7f1b4ed9d640 1 --2- 172.21.15.112:0/504496150 >> v2:172.21.15.187:3300/0 conn(0x7f1b500c85b0 0x7f1b500c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.898+0000 7f1b47fff640 1 -- 172.21.15.112:0/504496150 >> v2:172.21.15.187:3300/0 conn(0x7f1b500c85b0 msgr2=0x7f1b500c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:19.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.898+0000 7f1b47fff640 1 --2- 172.21.15.112:0/504496150 >> v2:172.21.15.187:3300/0 conn(0x7f1b500c85b0 0x7f1b500c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.898+0000 7f1b47fff640 1 -- 172.21.15.112:0/504496150 >> v1:172.21.15.187:6789/0 conn(0x7f1b5014fbd0 legacy=0x7f1b500c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:19.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.898+0000 7f1b47fff640 1 -- 172.21.15.112:0/504496150 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5006c6f0 con 0x7f1b5006cef0 2024-04-25T01:41:19.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.898+0000 7f1b47fff640 1 --2- 172.21.15.112:0/504496150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b5006cef0 0x7f1b500c8070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1b5006d4f0 tx=0x7f1b3802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29729adba05fc3c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:19.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.899+0000 7f1b4dd9b640 1 -- 172.21.15.112:0/504496150 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38037070 con 0x7f1b5006cef0 2024-04-25T01:41:19.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.899+0000 7f1b4dd9b640 1 -- 172.21.15.112:0/504496150 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3802ae20 con 0x7f1b5006cef0 2024-04-25T01:41:19.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.899+0000 7f1b4dd9b640 1 -- 172.21.15.112:0/504496150 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38032660 con 0x7f1b5006cef0 2024-04-25T01:41:19.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.899+0000 7f1b54d9e640 1 -- 172.21.15.112:0/504496150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b5006cef0 msgr2=0x7f1b500c8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.899+0000 7f1b54d9e640 1 --2- 172.21.15.112:0/504496150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b5006cef0 0x7f1b500c8070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1b5006d4f0 tx=0x7f1b3802a110 comp rx=0 tx=0).stop 2024-04-25T01:41:19.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.899+0000 7f1b54d9e640 1 -- 172.21.15.112:0/504496150 shutdown_connections 2024-04-25T01:41:19.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.899+0000 7f1b54d9e640 1 --2- 172.21.15.112:0/504496150 >> v2:172.21.15.187:3300/0 conn(0x7f1b500c85b0 0x7f1b500c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.899+0000 7f1b54d9e640 1 --2- 172.21.15.112:0/504496150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b5006cef0 0x7f1b500c8070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.899+0000 7f1b54d9e640 1 -- 172.21.15.112:0/504496150 >> 172.21.15.112:0/504496150 conn(0x7f1b500ad050 msgr2=0x7f1b500ad460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:19.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.899+0000 7f1b54d9e640 1 -- 172.21.15.112:0/504496150 shutdown_connections 2024-04-25T01:41:19.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.899+0000 7f1b54d9e640 1 -- 172.21.15.112:0/504496150 wait complete. 2024-04-25T01:41:19.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.900+0000 7f1b54d9e640 1 Processor -- start 2024-04-25T01:41:19.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.900+0000 7f1b54d9e640 1 -- start start 2024-04-25T01:41:19.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.901+0000 7f1b54d9e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b5006cef0 0x7f1b50149a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.901+0000 7f1b54d9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b500c85b0 0x7f1b500abd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.901+0000 7f1b54d9e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b5014fbd0 0x7f1b50148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.901+0000 7f1b54d9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1b500c98a0 con 0x7f1b500c85b0 2024-04-25T01:41:19.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.901+0000 7f1b54d9e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1b500c9720 con 0x7f1b5014fbd0 2024-04-25T01:41:19.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.901+0000 7f1b54d9e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1b500c9a20 con 0x7f1b5006cef0 2024-04-25T01:41:19.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.901+0000 7f1b47fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b500c85b0 0x7f1b500abd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.901+0000 7f1b47fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b500c85b0 0x7f1b500abd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54984/0 (socket says 172.21.15.112:54984) 2024-04-25T01:41:19.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.901+0000 7f1b47fff640 1 -- 172.21.15.112:0/3126077276 learned_addr learned my addr 172.21.15.112:0/3126077276 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:19.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.901+0000 7f1b4ed9d640 1 --2- 172.21.15.112:0/3126077276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b5014fbd0 0x7f1b50148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.901+0000 7f1b4e59c640 1 --2- 172.21.15.112:0/3126077276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b5006cef0 0x7f1b50149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.901+0000 7f1b47fff640 1 -- 172.21.15.112:0/3126077276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b5006cef0 msgr2=0x7f1b50149a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.901+0000 7f1b47fff640 1 --2- 172.21.15.112:0/3126077276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b5006cef0 0x7f1b50149a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.901+0000 7f1b47fff640 1 -- 172.21.15.112:0/3126077276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b5014fbd0 msgr2=0x7f1b50148880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.901+0000 7f1b47fff640 1 --2- 172.21.15.112:0/3126077276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b5014fbd0 0x7f1b50148880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.902+0000 7f1b47fff640 1 -- 172.21.15.112:0/3126077276 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b50155290 con 0x7f1b500c85b0 2024-04-25T01:41:19.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.902+0000 7f1b47fff640 1 --2- 172.21.15.112:0/3126077276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b500c85b0 0x7f1b500abd60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1b38000c00 tx=0x7f1b380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:19.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.902+0000 7f1b46ffd640 1 -- 172.21.15.112:0/3126077276 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38041070 con 0x7f1b500c85b0 2024-04-25T01:41:19.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.902+0000 7f1b46ffd640 1 -- 172.21.15.112:0/3126077276 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3803c670 con 0x7f1b500c85b0 2024-04-25T01:41:19.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.902+0000 7f1b46ffd640 1 -- 172.21.15.112:0/3126077276 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38037040 con 0x7f1b500c85b0 2024-04-25T01:41:19.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.902+0000 7f1b54d9e640 1 -- 172.21.15.112:0/3126077276 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b50155570 con 0x7f1b500c85b0 2024-04-25T01:41:19.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.902+0000 7f1b54d9e640 1 -- 172.21.15.112:0/3126077276 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b500ac2a0 con 0x7f1b500c85b0 2024-04-25T01:41:19.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.903+0000 7f1b54d9e640 1 -- 172.21.15.112:0/3126077276 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1b500c8c90 con 0x7f1b500c85b0 2024-04-25T01:41:19.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.903+0000 7f1b46ffd640 1 -- 172.21.15.112:0/3126077276 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b38044070 con 0x7f1b500c85b0 2024-04-25T01:41:19.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.904+0000 7f1b46ffd640 1 --2- 172.21.15.112:0/3126077276 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1b3006be60 0x7f1b3006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.904+0000 7f1b46ffd640 1 -- 172.21.15.112:0/3126077276 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f1b380ab8f0 con 0x7f1b500c85b0 2024-04-25T01:41:19.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.904+0000 7f1b4e59c640 1 --2- 172.21.15.112:0/3126077276 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1b3006be60 0x7f1b3006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.904+0000 7f1b46ffd640 1 --2- 172.21.15.112:0/3126077276 >> [v2:172.21.15.187:6800/876892580,v1:172.21.15.187:6801/876892580] conn(0x7f1b300767f0 0x7f1b30078c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:19.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.904+0000 7f1b46ffd640 1 -- 172.21.15.112:0/3126077276 --> [v2:172.21.15.187:6800/876892580,v1:172.21.15.187:6801/876892580] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1b30079340 con 0x7f1b300767f0 2024-04-25T01:41:19.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.904+0000 7f1b46ffd640 1 -- 172.21.15.112:0/3126077276 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1b380b02c0 con 0x7f1b500c85b0 2024-04-25T01:41:19.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.904+0000 7f1b4e59c640 1 --2- 172.21.15.112:0/3126077276 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1b3006be60 0x7f1b3006e320 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000a750 tx=0x7f1b40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:19.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.904+0000 7f1b4ed9d640 1 --2- 172.21.15.112:0/3126077276 >> [v2:172.21.15.187:6800/876892580,v1:172.21.15.187:6801/876892580] conn(0x7f1b300767f0 0x7f1b30078c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:19.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.904+0000 7f1b4ed9d640 1 --2- 172.21.15.112:0/3126077276 >> [v2:172.21.15.187:6800/876892580,v1:172.21.15.187:6801/876892580] conn(0x7f1b300767f0 0x7f1b30078c70 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:19.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.905+0000 7f1b46ffd640 1 -- 172.21.15.112:0/3126077276 <== osd.4 v2:172.21.15.187:6800/876892580 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b30079340 con 0x7f1b300767f0 2024-04-25T01:41:19.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.919+0000 7f1b54d9e640 1 -- 172.21.15.112:0/3126077276 --> [v2:172.21.15.187:6800/876892580,v1:172.21.15.187:6801/876892580] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1b500682a0 con 0x7f1b300767f0 2024-04-25T01:41:19.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.920+0000 7f1b46ffd640 1 -- 172.21.15.112:0/3126077276 <== osd.4 v2:172.21.15.187:6800/876892580 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1b500682a0 con 0x7f1b300767f0 2024-04-25T01:41:19.906 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:41:19.906 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:41:19.906 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:41:19.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.920+0000 7f1b54d9e640 1 -- 172.21.15.112:0/3126077276 >> [v2:172.21.15.187:6800/876892580,v1:172.21.15.187:6801/876892580] conn(0x7f1b300767f0 msgr2=0x7f1b30078c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.920+0000 7f1b54d9e640 1 --2- 172.21.15.112:0/3126077276 >> [v2:172.21.15.187:6800/876892580,v1:172.21.15.187:6801/876892580] conn(0x7f1b300767f0 0x7f1b30078c70 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.920+0000 7f1b54d9e640 1 -- 172.21.15.112:0/3126077276 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1b3006be60 msgr2=0x7f1b3006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.920+0000 7f1b54d9e640 1 --2- 172.21.15.112:0/3126077276 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1b3006be60 0x7f1b3006e320 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000a750 tx=0x7f1b40008040 comp rx=0 tx=0).stop 2024-04-25T01:41:19.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.920+0000 7f1b54d9e640 1 -- 172.21.15.112:0/3126077276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b500c85b0 msgr2=0x7f1b500abd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:19.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.920+0000 7f1b54d9e640 1 --2- 172.21.15.112:0/3126077276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b500c85b0 0x7f1b500abd60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1b38000c00 tx=0x7f1b380046c0 comp rx=0 tx=0).stop 2024-04-25T01:41:19.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.920+0000 7f1b4ed9d640 1 -- 172.21.15.112:0/3126077276 reap_dead start 2024-04-25T01:41:19.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.920+0000 7f1b54d9e640 1 -- 172.21.15.112:0/3126077276 shutdown_connections 2024-04-25T01:41:19.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.920+0000 7f1b54d9e640 1 -- 172.21.15.112:0/3126077276 >> 172.21.15.112:0/3126077276 conn(0x7f1b500ad050 msgr2=0x7f1b5014d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:19.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.920+0000 7f1b54d9e640 1 -- 172.21.15.112:0/3126077276 shutdown_connections 2024-04-25T01:41:19.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:19.920+0000 7f1b54d9e640 1 -- 172.21.15.112:0/3126077276 wait complete. 2024-04-25T01:41:19.917 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T01:41:20.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.030+0000 7f2eda77e640 1 Processor -- start 2024-04-25T01:41:20.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.030+0000 7f2eda77e640 1 -- start start 2024-04-25T01:41:20.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.030+0000 7f2eda77e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ed406fb80 0x7f2ed4156110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.030+0000 7f2eda77e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ed40c1e40 con 0x7f2ed40747b0 2024-04-25T01:41:20.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.030+0000 7f2eda77e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ed4159a20 con 0x7f2ed40743d0 2024-04-25T01:41:20.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.030+0000 7f2eda77e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed40c40e0 con 0x7f2ed406fb80 2024-04-25T01:41:20.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed8d1b640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2ed40747b0 0x7f2ed41602a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54656/0 (socket says 172.21.15.112:54656) 2024-04-25T01:41:20.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed8d1b640 1 -- 172.21.15.112:0/3825968789 learned_addr learned my addr 172.21.15.112:0/3825968789 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:20.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed37fe640 1 --2- 172.21.15.112:0/3825968789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ed406fb80 0x7f2ed4156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed2ffd640 1 -- 172.21.15.112:0/3825968789 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1194439880 0 0) 0x7f2ed40c1e40 con 0x7f2ed40747b0 2024-04-25T01:41:20.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed2ffd640 1 -- 172.21.15.112:0/3825968789 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ebc003520 con 0x7f2ed40747b0 2024-04-25T01:41:20.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed2ffd640 1 -- 172.21.15.112:0/3825968789 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3473320812 0 0) 0x7f2ed4159a20 con 0x7f2ed40743d0 2024-04-25T01:41:20.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed2ffd640 1 -- 172.21.15.112:0/3825968789 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ed40c1e40 con 0x7f2ed40743d0 2024-04-25T01:41:20.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed2ffd640 1 -- 172.21.15.112:0/3825968789 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1003434561 0 0) 0x7f2ebc003520 con 0x7f2ed40747b0 2024-04-25T01:41:20.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed2ffd640 1 -- 172.21.15.112:0/3825968789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ed406fb80 msgr2=0x7f2ed4156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:20.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed2ffd640 1 --2- 172.21.15.112:0/3825968789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ed406fb80 0x7f2ed4156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed2ffd640 1 -- 172.21.15.112:0/3825968789 >> v1:172.21.15.187:6789/0 conn(0x7f2ed40743d0 legacy=0x7f2ed4155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed2ffd640 1 -- 172.21.15.112:0/3825968789 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed4159d00 con 0x7f2ed40747b0 2024-04-25T01:41:20.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed2ffd640 1 -- 172.21.15.112:0/3825968789 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f2ecc0019f0 con 0x7f2ed40747b0 2024-04-25T01:41:20.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed2ffd640 1 -- 172.21.15.112:0/3825968789 >> v1:172.21.15.112:6789/0 conn(0x7f2ed40747b0 legacy=0x7f2ed41602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed2ffd640 1 --2- 172.21.15.112:0/3825968789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ebc003b70 0x7f2ebc003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed2ffd640 1 --2- 172.21.15.112:0/3825968789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ebc004800 0x7f2ebc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed2ffd640 1 --2- 172.21.15.112:0/3825968789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ed406fb80 0x7f2ebc0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed2ffd640 1 -- 172.21.15.112:0/3825968789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc00baa0 con 0x7f2ebc003b70 2024-04-25T01:41:20.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed2ffd640 1 -- 172.21.15.112:0/3825968789 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc00bc50 con 0x7f2ebc004800 2024-04-25T01:41:20.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed2ffd640 1 -- 172.21.15.112:0/3825968789 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2ebc00be50 con 0x7f2ed406fb80 2024-04-25T01:41:20.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed3fff640 1 --2- 172.21.15.112:0/3825968789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ebc003b70 0x7f2ebc003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed8d1b640 1 --2- 172.21.15.112:0/3825968789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ed406fb80 0x7f2ebc0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed37fe640 1 --2- 172.21.15.112:0/3825968789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ebc004800 0x7f2ebc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed3fff640 1 -- 172.21.15.112:0/3825968789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ed406fb80 msgr2=0x7f2ebc0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:20.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed3fff640 1 --2- 172.21.15.112:0/3825968789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ed406fb80 0x7f2ebc0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed3fff640 1 -- 172.21.15.112:0/3825968789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ebc004800 msgr2=0x7f2ebc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:20.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed3fff640 1 --2- 172.21.15.112:0/3825968789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ebc004800 0x7f2ebc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.031+0000 7f2ed3fff640 1 -- 172.21.15.112:0/3825968789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ebc003520 con 0x7f2ebc003b70 2024-04-25T01:41:20.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.032+0000 7f2ed3fff640 1 --2- 172.21.15.112:0/3825968789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ebc003b70 0x7f2ebc003f70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40018d0 tx=0x7f2ebc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d176d40a5ae99c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:20.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.032+0000 7f2ed2ffd640 1 -- 172.21.15.112:0/3825968789 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec40014f0 con 0x7f2ebc003b70 2024-04-25T01:41:20.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.032+0000 7f2ed2ffd640 1 -- 172.21.15.112:0/3825968789 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec40052d0 con 0x7f2ebc003b70 2024-04-25T01:41:20.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.032+0000 7f2ed2ffd640 1 -- 172.21.15.112:0/3825968789 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec402aca0 con 0x7f2ebc003b70 2024-04-25T01:41:20.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.033+0000 7f2eda77e640 1 -- 172.21.15.112:0/3825968789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ebc003b70 msgr2=0x7f2ebc003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.033+0000 7f2eda77e640 1 --2- 172.21.15.112:0/3825968789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ebc003b70 0x7f2ebc003f70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40018d0 tx=0x7f2ebc005830 comp rx=0 tx=0).stop 2024-04-25T01:41:20.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.033+0000 7f2ed8d1b640 1 -- 172.21.15.112:0/3825968789 reap_dead start 2024-04-25T01:41:20.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.033+0000 7f2eda77e640 1 -- 172.21.15.112:0/3825968789 shutdown_connections 2024-04-25T01:41:20.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.033+0000 7f2eda77e640 1 -- 172.21.15.112:0/3825968789 >> 172.21.15.112:0/3825968789 conn(0x7f2ed406b160 msgr2=0x7f2ed406b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:20.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.033+0000 7f2eda77e640 1 -- 172.21.15.112:0/3825968789 shutdown_connections 2024-04-25T01:41:20.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.033+0000 7f2eda77e640 1 -- 172.21.15.112:0/3825968789 wait complete. 2024-04-25T01:41:20.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.034+0000 7f2eda77e640 1 Processor -- start 2024-04-25T01:41:20.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.034+0000 7f2eda77e640 1 -- start start 2024-04-25T01:41:20.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.035+0000 7f2eda77e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ebc003b70 0x7f2ed4156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.035+0000 7f2eda77e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ed40c68b0 0x7f2ed4149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.035+0000 7f2eda77e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ed414a210 0x7f2ed4150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.035+0000 7f2eda77e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed41534b0 con 0x7f2ed40c68b0 2024-04-25T01:41:20.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.035+0000 7f2eda77e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed4074c40 con 0x7f2ed414a210 2024-04-25T01:41:20.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.035+0000 7f2eda77e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed40c16d0 con 0x7f2ebc003b70 2024-04-25T01:41:20.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.035+0000 7f2ed37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ed40c68b0 0x7f2ed4149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.035+0000 7f2ed37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ed40c68b0 0x7f2ed4149cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55000/0 (socket says 172.21.15.112:55000) 2024-04-25T01:41:20.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.035+0000 7f2ed37fe640 1 -- 172.21.15.112:0/19918890 learned_addr learned my addr 172.21.15.112:0/19918890 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:20.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.035+0000 7f2ed8d1b640 1 --2- 172.21.15.112:0/19918890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ed414a210 0x7f2ed4150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.035+0000 7f2ed3fff640 1 --2- 172.21.15.112:0/19918890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ebc003b70 0x7f2ed4156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.035+0000 7f2ed37fe640 1 -- 172.21.15.112:0/19918890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ebc003b70 msgr2=0x7f2ed4156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.035+0000 7f2ed37fe640 1 --2- 172.21.15.112:0/19918890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ebc003b70 0x7f2ed4156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.035+0000 7f2ed37fe640 1 -- 172.21.15.112:0/19918890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ed414a210 msgr2=0x7f2ed4150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.035+0000 7f2ed37fe640 1 --2- 172.21.15.112:0/19918890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ed414a210 0x7f2ed4150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.035+0000 7f2ed37fe640 1 -- 172.21.15.112:0/19918890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed40c5b60 con 0x7f2ed40c68b0 2024-04-25T01:41:20.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.035+0000 7f2ed37fe640 1 --2- 172.21.15.112:0/19918890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ed40c68b0 0x7f2ed4149cd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8011600 tx=0x7f2ec8011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:20.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.036+0000 7f2ed0ff9640 1 -- 172.21.15.112:0/19918890 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8015870 con 0x7f2ed40c68b0 2024-04-25T01:41:20.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.036+0000 7f2ed0ff9640 1 -- 172.21.15.112:0/19918890 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec8015a10 con 0x7f2ed40c68b0 2024-04-25T01:41:20.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.036+0000 7f2eda77e640 1 -- 172.21.15.112:0/19918890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed41572b0 con 0x7f2ed40c68b0 2024-04-25T01:41:20.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.036+0000 7f2ed0ff9640 1 -- 172.21.15.112:0/19918890 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8015dc0 con 0x7f2ed40c68b0 2024-04-25T01:41:20.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.036+0000 7f2eda77e640 1 -- 172.21.15.112:0/19918890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed40c60b0 con 0x7f2ed40c68b0 2024-04-25T01:41:20.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.037+0000 7f2eda77e640 1 -- 172.21.15.112:0/19918890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ed40682a0 con 0x7f2ed40c68b0 2024-04-25T01:41:20.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.037+0000 7f2ed0ff9640 1 -- 172.21.15.112:0/19918890 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ec800f450 con 0x7f2ed40c68b0 2024-04-25T01:41:20.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.037+0000 7f2ed0ff9640 1 --2- 172.21.15.112:0/19918890 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ec006be30 0x7f2ec006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.038+0000 7f2ed0ff9640 1 -- 172.21.15.112:0/19918890 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f2ec808bb60 con 0x7f2ed40c68b0 2024-04-25T01:41:20.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.038+0000 7f2ed3fff640 1 --2- 172.21.15.112:0/19918890 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ec006be30 0x7f2ec006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.038+0000 7f2ed3fff640 1 --2- 172.21.15.112:0/19918890 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ec006be30 0x7f2ec006e2f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4032000 tx=0x7f2ec4009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:20.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.042+0000 7f2ed0ff9640 1 -- 172.21.15.112:0/19918890 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2ec8019030 con 0x7f2ed40c68b0 2024-04-25T01:41:20.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.219+0000 7f2eda77e640 1 -- 172.21.15.112:0/19918890 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f2ed414a620 con 0x7f2ec006be30 2024-04-25T01:41:20.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.220+0000 7f2ed0ff9640 1 -- 172.21.15.112:0/19918890 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2ed414a620 con 0x7f2ec006be30 2024-04-25T01:41:20.207 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T01:41:20.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.222+0000 7f2eda77e640 1 -- 172.21.15.112:0/19918890 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ec006be30 msgr2=0x7f2ec006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.222+0000 7f2eda77e640 1 --2- 172.21.15.112:0/19918890 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ec006be30 0x7f2ec006e2f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4032000 tx=0x7f2ec4009030 comp rx=0 tx=0).stop 2024-04-25T01:41:20.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.222+0000 7f2eda77e640 1 -- 172.21.15.112:0/19918890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ed40c68b0 msgr2=0x7f2ed4149cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.222+0000 7f2eda77e640 1 --2- 172.21.15.112:0/19918890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ed40c68b0 0x7f2ed4149cd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8011600 tx=0x7f2ec8011ac0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.222+0000 7f2eda77e640 1 -- 172.21.15.112:0/19918890 shutdown_connections 2024-04-25T01:41:20.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.222+0000 7f2eda77e640 1 --2- 172.21.15.112:0/19918890 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f2ec006be30 0x7f2ec006e2f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.222+0000 7f2eda77e640 1 --2- 172.21.15.112:0/19918890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ed414a210 0x7f2ed4150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.222+0000 7f2eda77e640 1 --2- 172.21.15.112:0/19918890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ed40c68b0 0x7f2ed4149cd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.222+0000 7f2eda77e640 1 --2- 172.21.15.112:0/19918890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ebc003b70 0x7f2ed4156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.222+0000 7f2eda77e640 1 -- 172.21.15.112:0/19918890 >> 172.21.15.112:0/19918890 conn(0x7f2ed406b160 msgr2=0x7f2ed40a79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:20.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.222+0000 7f2eda77e640 1 -- 172.21.15.112:0/19918890 shutdown_connections 2024-04-25T01:41:20.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.222+0000 7f2eda77e640 1 -- 172.21.15.112:0/19918890 wait complete. 2024-04-25T01:41:20.220 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T01:41:20.220 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:41:20.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.335+0000 7f07433ee640 1 Processor -- start 2024-04-25T01:41:20.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.335+0000 7f07433ee640 1 -- start start 2024-04-25T01:41:20.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.336+0000 7f07433ee640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c0a9490 0x7f073c1536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.336+0000 7f07433ee640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f073c0a9870 0x7f073c06dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.336+0000 7f07433ee640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f073c0a9cb0 con 0x7f073c06e220 2024-04-25T01:41:20.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.336+0000 7f07433ee640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f073c0ab510 con 0x7f073c0a9870 2024-04-25T01:41:20.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.336+0000 7f07433ee640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f073c0ab7f0 con 0x7f073c0a9490 2024-04-25T01:41:20.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.336+0000 7f0740989640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f073c0a9870 0x7f073c06dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.336+0000 7f074198b640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f073c06e220 0x7f073c156cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54658/0 (socket says 172.21.15.112:54658) 2024-04-25T01:41:20.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.336+0000 7f074198b640 1 -- 172.21.15.112:0/1533427001 learned_addr learned my addr 172.21.15.112:0/1533427001 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:20.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.336+0000 7f074118a640 1 --2- 172.21.15.112:0/1533427001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c0a9490 0x7f073c1536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.337+0000 7f0740989640 1 -- 172.21.15.112:0/1533427001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c0a9490 msgr2=0x7f073c1536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:20.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.337+0000 7f0740989640 1 --2- 172.21.15.112:0/1533427001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c0a9490 0x7f073c1536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.337+0000 7f0740989640 1 -- 172.21.15.112:0/1533427001 >> v1:172.21.15.112:6789/0 conn(0x7f073c06e220 legacy=0x7f073c156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.337+0000 7f0740989640 1 -- 172.21.15.112:0/1533427001 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073c0abad0 con 0x7f073c0a9870 2024-04-25T01:41:20.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.337+0000 7f074118a640 1 --2- 172.21.15.112:0/1533427001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c0a9490 0x7f073c1536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:41:20.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.337+0000 7f0740989640 1 --2- 172.21.15.112:0/1533427001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f073c0a9870 0x7f073c06dce0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f072c0097a0 tx=0x7f072c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8061ade31896d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:20.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.337+0000 7f0733fff640 1 -- 172.21.15.112:0/1533427001 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c037070 con 0x7f073c0a9870 2024-04-25T01:41:20.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.337+0000 7f0733fff640 1 -- 172.21.15.112:0/1533427001 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f072c02ae50 con 0x7f073c0a9870 2024-04-25T01:41:20.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.337+0000 7f0733fff640 1 -- 172.21.15.112:0/1533427001 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c032660 con 0x7f073c0a9870 2024-04-25T01:41:20.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.338+0000 7f07433ee640 1 -- 172.21.15.112:0/1533427001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f073c0a9870 msgr2=0x7f073c06dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.338+0000 7f07433ee640 1 --2- 172.21.15.112:0/1533427001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f073c0a9870 0x7f073c06dce0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f072c0097a0 tx=0x7f072c02a100 comp rx=0 tx=0).stop 2024-04-25T01:41:20.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.338+0000 7f07433ee640 1 -- 172.21.15.112:0/1533427001 shutdown_connections 2024-04-25T01:41:20.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.338+0000 7f07433ee640 1 --2- 172.21.15.112:0/1533427001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f073c0a9870 0x7f073c06dce0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.338+0000 7f07433ee640 1 --2- 172.21.15.112:0/1533427001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c0a9490 0x7f073c1536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.338+0000 7f07433ee640 1 -- 172.21.15.112:0/1533427001 >> 172.21.15.112:0/1533427001 conn(0x7f073c06b8a0 msgr2=0x7f073c0c6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:20.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.338+0000 7f07433ee640 1 -- 172.21.15.112:0/1533427001 shutdown_connections 2024-04-25T01:41:20.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.338+0000 7f07433ee640 1 -- 172.21.15.112:0/1533427001 wait complete. 2024-04-25T01:41:20.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.339+0000 7f07433ee640 1 Processor -- start 2024-04-25T01:41:20.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.339+0000 7f07433ee640 1 -- start start 2024-04-25T01:41:20.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.340+0000 7f07433ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f073c06e220 0x7f073c154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.340+0000 7f07433ee640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c0a9490 0x7f073c1e96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.340+0000 7f07433ee640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f073c0a9870 0x7f073c1e9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.340+0000 7f07433ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f073c0c7700 con 0x7f073c06e220 2024-04-25T01:41:20.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.340+0000 7f07433ee640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f073c0ac090 con 0x7f073c0a9870 2024-04-25T01:41:20.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.340+0000 7f07433ee640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f073c150f80 con 0x7f073c0a9490 2024-04-25T01:41:20.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.340+0000 7f074118a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f073c06e220 0x7f073c154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.340+0000 7f074118a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f073c06e220 0x7f073c154af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55026/0 (socket says 172.21.15.112:55026) 2024-04-25T01:41:20.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.340+0000 7f074118a640 1 -- 172.21.15.112:0/365122829 learned_addr learned my addr 172.21.15.112:0/365122829 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:20.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.340+0000 7f0740989640 1 --2- 172.21.15.112:0/365122829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c0a9490 0x7f073c1e96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.340+0000 7f074118a640 1 -- 172.21.15.112:0/365122829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c0a9490 msgr2=0x7f073c1e96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.340+0000 7f074118a640 1 --2- 172.21.15.112:0/365122829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f073c0a9490 0x7f073c1e96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.340+0000 7f074118a640 1 -- 172.21.15.112:0/365122829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f073c0a9870 msgr2=0x7f073c1e9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.340+0000 7f074118a640 1 --2- 172.21.15.112:0/365122829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f073c0a9870 0x7f073c1e9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.340+0000 7f074118a640 1 -- 172.21.15.112:0/365122829 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073c1477a0 con 0x7f073c06e220 2024-04-25T01:41:20.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.340+0000 7f074118a640 1 --2- 172.21.15.112:0/365122829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f073c06e220 0x7f073c154af0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0738007ef0 tx=0x7f073800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:20.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.340+0000 7f0731ffb640 1 -- 172.21.15.112:0/365122829 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073801a070 con 0x7f073c06e220 2024-04-25T01:41:20.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.340+0000 7f0731ffb640 1 -- 172.21.15.112:0/365122829 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0738003b90 con 0x7f073c06e220 2024-04-25T01:41:20.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.340+0000 7f0731ffb640 1 -- 172.21.15.112:0/365122829 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0738004030 con 0x7f073c06e220 2024-04-25T01:41:20.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.340+0000 7f07433ee640 1 -- 172.21.15.112:0/365122829 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f073c157780 con 0x7f073c06e220 2024-04-25T01:41:20.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.341+0000 7f07433ee640 1 -- 172.21.15.112:0/365122829 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f073c1570e0 con 0x7f073c06e220 2024-04-25T01:41:20.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.341+0000 7f07433ee640 1 -- 172.21.15.112:0/365122829 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f073c157570 con 0x7f073c06e220 2024-04-25T01:41:20.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.342+0000 7f0731ffb640 1 -- 172.21.15.112:0/365122829 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f073801d070 con 0x7f073c06e220 2024-04-25T01:41:20.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.342+0000 7f0731ffb640 1 --2- 172.21.15.112:0/365122829 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f072006be30 0x7f072006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.342+0000 7f0740989640 1 --2- 172.21.15.112:0/365122829 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f072006be30 0x7f072006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.342+0000 7f0731ffb640 1 -- 172.21.15.112:0/365122829 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f073808b330 con 0x7f073c06e220 2024-04-25T01:41:20.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.342+0000 7f0731ffb640 1 --2- 172.21.15.112:0/365122829 >> [v2:172.21.15.187:6824/1463958731,v1:172.21.15.187:6825/1463958731] conn(0x7f07200767d0 0x7f0720078c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.342+0000 7f0731ffb640 1 -- 172.21.15.112:0/365122829 --> [v2:172.21.15.187:6824/1463958731,v1:172.21.15.187:6825/1463958731] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0738011e50 con 0x7f07200767d0 2024-04-25T01:41:20.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.342+0000 7f0731ffb640 1 -- 172.21.15.112:0/365122829 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0738092050 con 0x7f073c06e220 2024-04-25T01:41:20.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.343+0000 7f0740989640 1 --2- 172.21.15.112:0/365122829 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f072006be30 0x7f072006e2f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f072c034000 tx=0x7f072c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:20.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.343+0000 7f074198b640 1 --2- 172.21.15.112:0/365122829 >> [v2:172.21.15.187:6824/1463958731,v1:172.21.15.187:6825/1463958731] conn(0x7f07200767d0 0x7f0720078c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.343+0000 7f074198b640 1 --2- 172.21.15.112:0/365122829 >> [v2:172.21.15.187:6824/1463958731,v1:172.21.15.187:6825/1463958731] conn(0x7f07200767d0 0x7f0720078c50 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:20.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.344+0000 7f0731ffb640 1 -- 172.21.15.112:0/365122829 <== osd.5 v2:172.21.15.187:6824/1463958731 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0738011e50 con 0x7f07200767d0 2024-04-25T01:41:20.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.357+0000 7f07433ee640 1 -- 172.21.15.112:0/365122829 --> [v2:172.21.15.187:6824/1463958731,v1:172.21.15.187:6825/1463958731] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f073c0682a0 con 0x7f07200767d0 2024-04-25T01:41:20.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.358+0000 7f0731ffb640 1 -- 172.21.15.112:0/365122829 <== osd.5 v2:172.21.15.187:6824/1463958731 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f073c0682a0 con 0x7f07200767d0 2024-04-25T01:41:20.344 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:41:20.344 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:41:20.344 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:41:20.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.358+0000 7f07433ee640 1 -- 172.21.15.112:0/365122829 >> [v2:172.21.15.187:6824/1463958731,v1:172.21.15.187:6825/1463958731] conn(0x7f07200767d0 msgr2=0x7f0720078c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.358+0000 7f07433ee640 1 --2- 172.21.15.112:0/365122829 >> [v2:172.21.15.187:6824/1463958731,v1:172.21.15.187:6825/1463958731] conn(0x7f07200767d0 0x7f0720078c50 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.358+0000 7f07433ee640 1 -- 172.21.15.112:0/365122829 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f072006be30 msgr2=0x7f072006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.358+0000 7f07433ee640 1 --2- 172.21.15.112:0/365122829 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f072006be30 0x7f072006e2f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f072c034000 tx=0x7f072c02e000 comp rx=0 tx=0).stop 2024-04-25T01:41:20.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.358+0000 7f07433ee640 1 -- 172.21.15.112:0/365122829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f073c06e220 msgr2=0x7f073c154af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.358+0000 7f07433ee640 1 --2- 172.21.15.112:0/365122829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f073c06e220 0x7f073c154af0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0738007ef0 tx=0x7f073800d4a0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.358+0000 7f074198b640 1 -- 172.21.15.112:0/365122829 reap_dead start 2024-04-25T01:41:20.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.358+0000 7f07433ee640 1 -- 172.21.15.112:0/365122829 shutdown_connections 2024-04-25T01:41:20.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.358+0000 7f07433ee640 1 -- 172.21.15.112:0/365122829 >> 172.21.15.112:0/365122829 conn(0x7f073c06b8a0 msgr2=0x7f073c0c6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:20.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.358+0000 7f07433ee640 1 -- 172.21.15.112:0/365122829 shutdown_connections 2024-04-25T01:41:20.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.358+0000 7f07433ee640 1 -- 172.21.15.112:0/365122829 wait complete. 2024-04-25T01:41:20.355 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T01:41:20.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.468+0000 7fbdb7f12640 1 Processor -- start 2024-04-25T01:41:20.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.468+0000 7fbdb7f12640 1 -- start start 2024-04-25T01:41:20.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.469+0000 7fbdb7f12640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdb00ac3b0 0x7fbdb0155f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.469+0000 7fbdb7f12640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbdb01524a0 0x7fbdb0156460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.469+0000 7fbdb7f12640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdb01529b0 con 0x7fbdb014c3f0 2024-04-25T01:41:20.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.469+0000 7fbdb7f12640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fbdb0155a90 con 0x7fbdb01524a0 2024-04-25T01:41:20.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.469+0000 7fbdb7f12640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbdb01569a0 con 0x7fbdb00ac3b0 2024-04-25T01:41:20.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.469+0000 7fbdb64af640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbdb01524a0 0x7fbdb0156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.469+0000 7fbdb64af640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbdb01524a0 0x7fbdb0156460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55040/0 (socket says 172.21.15.112:55040) 2024-04-25T01:41:20.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.469+0000 7fbdb64af640 1 -- 172.21.15.112:0/2460363073 learned_addr learned my addr 172.21.15.112:0/2460363073 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:20.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.469+0000 7fbdb54ad640 1 --2- 172.21.15.112:0/2460363073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdb00ac3b0 0x7fbdb0155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.469+0000 7fbdb64af640 1 -- 172.21.15.112:0/2460363073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdb00ac3b0 msgr2=0x7fbdb0155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:20.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.469+0000 7fbdb64af640 1 --2- 172.21.15.112:0/2460363073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdb00ac3b0 0x7fbdb0155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.469+0000 7fbdb64af640 1 -- 172.21.15.112:0/2460363073 >> v1:172.21.15.187:6789/0 conn(0x7fbdb014c3f0 legacy=0x7fbdb00c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.469+0000 7fbdb64af640 1 -- 172.21.15.112:0/2460363073 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb0152c70 con 0x7fbdb01524a0 2024-04-25T01:41:20.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.470+0000 7fbdb64af640 1 --2- 172.21.15.112:0/2460363073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb01524a0 0x7fbdb0156460 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009e50 tx=0x7fbda802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af55798b6bea879 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:20.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.470+0000 7fbdb4cac640 1 -- 172.21.15.112:0/2460363073 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8036070 con 0x7fbdb01524a0 2024-04-25T01:41:20.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.470+0000 7fbdb4cac640 1 -- 172.21.15.112:0/2460363073 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbda8031600 con 0x7fbdb01524a0 2024-04-25T01:41:20.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.470+0000 7fbdb4cac640 1 -- 172.21.15.112:0/2460363073 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda80319d0 con 0x7fbdb01524a0 2024-04-25T01:41:20.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.470+0000 7fbdb7f12640 1 -- 172.21.15.112:0/2460363073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb01524a0 msgr2=0x7fbdb0156460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.470+0000 7fbdb7f12640 1 --2- 172.21.15.112:0/2460363073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb01524a0 0x7fbdb0156460 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009e50 tx=0x7fbda802a680 comp rx=0 tx=0).stop 2024-04-25T01:41:20.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.470+0000 7fbdb7f12640 1 -- 172.21.15.112:0/2460363073 shutdown_connections 2024-04-25T01:41:20.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.470+0000 7fbdb7f12640 1 --2- 172.21.15.112:0/2460363073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb01524a0 0x7fbdb0156460 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.470+0000 7fbdb7f12640 1 --2- 172.21.15.112:0/2460363073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdb00ac3b0 0x7fbdb0155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.470+0000 7fbdb7f12640 1 -- 172.21.15.112:0/2460363073 >> 172.21.15.112:0/2460363073 conn(0x7fbdb006b160 msgr2=0x7fbdb006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:20.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.471+0000 7fbdb7f12640 1 -- 172.21.15.112:0/2460363073 shutdown_connections 2024-04-25T01:41:20.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.471+0000 7fbdb7f12640 1 -- 172.21.15.112:0/2460363073 wait complete. 2024-04-25T01:41:20.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.472+0000 7fbdb7f12640 1 Processor -- start 2024-04-25T01:41:20.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.472+0000 7fbdb7f12640 1 -- start start 2024-04-25T01:41:20.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.472+0000 7fbdb7f12640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdb00ac3b0 0x7fbdb01e9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.472+0000 7fbdb7f12640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbdb014c3f0 0x7fbdb00c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.472+0000 7fbdb7f12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb01524a0 0x7fbdb0147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.472+0000 7fbdb7f12640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb0070d60 con 0x7fbdb01524a0 2024-04-25T01:41:20.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.472+0000 7fbdb7f12640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb0153230 con 0x7fbdb014c3f0 2024-04-25T01:41:20.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.472+0000 7fbdb7f12640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbdb00c32a0 con 0x7fbdb00ac3b0 2024-04-25T01:41:20.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.472+0000 7fbdb64af640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb01524a0 0x7fbdb0147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.472+0000 7fbdb64af640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb01524a0 0x7fbdb0147030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55042/0 (socket says 172.21.15.112:55042) 2024-04-25T01:41:20.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.472+0000 7fbdb64af640 1 -- 172.21.15.112:0/2484310498 learned_addr learned my addr 172.21.15.112:0/2484310498 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:20.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.473+0000 7fbdb5cae640 1 --2- 172.21.15.112:0/2484310498 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdb00ac3b0 0x7fbdb01e9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.473+0000 7fbdb54ad640 1 --2- 172.21.15.112:0/2484310498 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbdb014c3f0 0x7fbdb00c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.473+0000 7fbdb64af640 1 -- 172.21.15.112:0/2484310498 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdb00ac3b0 msgr2=0x7fbdb01e9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.473+0000 7fbdb64af640 1 --2- 172.21.15.112:0/2484310498 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdb00ac3b0 0x7fbdb01e9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.473+0000 7fbdb64af640 1 -- 172.21.15.112:0/2484310498 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbdb014c3f0 msgr2=0x7fbdb00c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.473+0000 7fbdb64af640 1 --2- 172.21.15.112:0/2484310498 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbdb014c3f0 0x7fbdb00c4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.473+0000 7fbdb64af640 1 -- 172.21.15.112:0/2484310498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb00c5ff0 con 0x7fbdb01524a0 2024-04-25T01:41:20.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.473+0000 7fbdb64af640 1 --2- 172.21.15.112:0/2484310498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb01524a0 0x7fbdb0147030 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbda8034040 tx=0x7fbda8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:20.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.473+0000 7fbda67fc640 1 -- 172.21.15.112:0/2484310498 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8036050 con 0x7fbdb01524a0 2024-04-25T01:41:20.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.473+0000 7fbda67fc640 1 -- 172.21.15.112:0/2484310498 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbda8042070 con 0x7fbdb01524a0 2024-04-25T01:41:20.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.473+0000 7fbda67fc640 1 -- 172.21.15.112:0/2484310498 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8009510 con 0x7fbdb01524a0 2024-04-25T01:41:20.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.473+0000 7fbdb7f12640 1 -- 172.21.15.112:0/2484310498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb00c53c0 con 0x7fbdb01524a0 2024-04-25T01:41:20.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.473+0000 7fbdb7f12640 1 -- 172.21.15.112:0/2484310498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb00c78b0 con 0x7fbdb01524a0 2024-04-25T01:41:20.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.474+0000 7fbdb7f12640 1 -- 172.21.15.112:0/2484310498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb00682a0 con 0x7fbdb01524a0 2024-04-25T01:41:20.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.474+0000 7fbda67fc640 1 -- 172.21.15.112:0/2484310498 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbda8007bd0 con 0x7fbdb01524a0 2024-04-25T01:41:20.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.475+0000 7fbda67fc640 1 --2- 172.21.15.112:0/2484310498 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbd9406be30 0x7fbd9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.475+0000 7fbda67fc640 1 -- 172.21.15.112:0/2484310498 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fbda80ac110 con 0x7fbdb01524a0 2024-04-25T01:41:20.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.475+0000 7fbdb5cae640 1 --2- 172.21.15.112:0/2484310498 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbd9406be30 0x7fbd9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.475+0000 7fbdb5cae640 1 --2- 172.21.15.112:0/2484310498 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbd9406be30 0x7fbd9406e2f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbdac001640 tx=0x7fbdac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:20.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.479+0000 7fbda67fc640 1 -- 172.21.15.112:0/2484310498 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbda8045360 con 0x7fbdb01524a0 2024-04-25T01:41:20.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.655+0000 7fbdb7f12640 1 -- 172.21.15.112:0/2484310498 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fbdb0075980 con 0x7fbd9406be30 2024-04-25T01:41:20.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.655+0000 7fbda67fc640 1 -- 172.21.15.112:0/2484310498 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbdac0050d0 con 0x7fbd9406be30 2024-04-25T01:41:20.642 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T01:41:20.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.658+0000 7fbdb7f12640 1 -- 172.21.15.112:0/2484310498 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbd9406be30 msgr2=0x7fbd9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.658+0000 7fbdb7f12640 1 --2- 172.21.15.112:0/2484310498 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbd9406be30 0x7fbd9406e2f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbdac001640 tx=0x7fbdac009040 comp rx=0 tx=0).stop 2024-04-25T01:41:20.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.658+0000 7fbdb7f12640 1 -- 172.21.15.112:0/2484310498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb01524a0 msgr2=0x7fbdb0147030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.658+0000 7fbdb7f12640 1 --2- 172.21.15.112:0/2484310498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb01524a0 0x7fbdb0147030 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbda8034040 tx=0x7fbda8004a40 comp rx=0 tx=0).stop 2024-04-25T01:41:20.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.658+0000 7fbdb7f12640 1 -- 172.21.15.112:0/2484310498 shutdown_connections 2024-04-25T01:41:20.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.658+0000 7fbdb7f12640 1 --2- 172.21.15.112:0/2484310498 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbd9406be30 0x7fbd9406e2f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.658+0000 7fbdb7f12640 1 --2- 172.21.15.112:0/2484310498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb01524a0 0x7fbdb0147030 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.658+0000 7fbdb7f12640 1 --2- 172.21.15.112:0/2484310498 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbdb014c3f0 0x7fbdb00c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.658+0000 7fbdb7f12640 1 --2- 172.21.15.112:0/2484310498 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdb00ac3b0 0x7fbdb01e9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.658+0000 7fbdb7f12640 1 -- 172.21.15.112:0/2484310498 >> 172.21.15.112:0/2484310498 conn(0x7fbdb006b160 msgr2=0x7fbdb0070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:20.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.658+0000 7fbdb7f12640 1 -- 172.21.15.112:0/2484310498 shutdown_connections 2024-04-25T01:41:20.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.658+0000 7fbdb7f12640 1 -- 172.21.15.112:0/2484310498 wait complete. 2024-04-25T01:41:20.656 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T01:41:20.656 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:41:20.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.770+0000 7f9b110df640 1 Processor -- start 2024-04-25T01:41:20.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.770+0000 7f9b110df640 1 -- start start 2024-04-25T01:41:20.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.771+0000 7f9b110df640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9b0c0c2d70 0x7f9b0c1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.771+0000 7f9b110df640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b0c06d560 con 0x7f9b0c06e600 2024-04-25T01:41:20.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.771+0000 7f9b110df640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b0c06d840 con 0x7f9b0c06cf10 2024-04-25T01:41:20.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.771+0000 7f9b110df640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f9b0c06d330 con 0x7f9b0c0c2d70 2024-04-25T01:41:20.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.771+0000 7f9b0ad9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9b0c0c2d70 0x7f9b0c1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.771+0000 7f9b03fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9b0c06e600 0x7f9b0c14c380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54672/0 (socket says 172.21.15.112:54672) 2024-04-25T01:41:20.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.771+0000 7f9b03fff640 1 -- 172.21.15.112:0/3880797770 learned_addr learned my addr 172.21.15.112:0/3880797770 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:20.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.771+0000 7f9b0ad9d640 1 -- 172.21.15.112:0/3880797770 >> v1:172.21.15.187:6789/0 conn(0x7f9b0c06cf10 legacy=0x7f9b0c1539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:20.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.771+0000 7f9b0ad9d640 1 -- 172.21.15.112:0/3880797770 >> v1:172.21.15.112:6789/0 conn(0x7f9b0c06e600 legacy=0x7f9b0c14c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.771+0000 7f9b0ad9d640 1 -- 172.21.15.112:0/3880797770 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0c06db20 con 0x7f9b0c0c2d70 2024-04-25T01:41:20.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.771+0000 7f9b0a59c640 1 -- 172.21.15.112:0/3880797770 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1373980026 0 0) 0x7f9b0c06d560 con 0x7f9b0c06e600 2024-04-25T01:41:20.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.772+0000 7f9b0ad9d640 1 --2- 172.21.15.112:0/3880797770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b0c0c2d70 0x7f9b0c1524b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0097a0 tx=0x7f9afc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ed5d572f623e47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:20.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.772+0000 7f9b0a59c640 1 -- 172.21.15.112:0/3880797770 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc037070 con 0x7f9b0c0c2d70 2024-04-25T01:41:20.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.772+0000 7f9b0a59c640 1 -- 172.21.15.112:0/3880797770 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9afc02ae50 con 0x7f9b0c0c2d70 2024-04-25T01:41:20.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.772+0000 7f9b0a59c640 1 -- 172.21.15.112:0/3880797770 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc032660 con 0x7f9b0c0c2d70 2024-04-25T01:41:20.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.772+0000 7f9b110df640 1 -- 172.21.15.112:0/3880797770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b0c0c2d70 msgr2=0x7f9b0c1524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.772+0000 7f9b110df640 1 --2- 172.21.15.112:0/3880797770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b0c0c2d70 0x7f9b0c1524b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0097a0 tx=0x7f9afc02a140 comp rx=0 tx=0).stop 2024-04-25T01:41:20.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.772+0000 7f9b110df640 1 -- 172.21.15.112:0/3880797770 shutdown_connections 2024-04-25T01:41:20.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.772+0000 7f9b110df640 1 --2- 172.21.15.112:0/3880797770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b0c0c2d70 0x7f9b0c1524b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.772+0000 7f9b110df640 1 -- 172.21.15.112:0/3880797770 >> 172.21.15.112:0/3880797770 conn(0x7f9b0c06b8a0 msgr2=0x7f9b0c0c6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:20.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.772+0000 7f9b110df640 1 -- 172.21.15.112:0/3880797770 shutdown_connections 2024-04-25T01:41:20.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.772+0000 7f9b110df640 1 -- 172.21.15.112:0/3880797770 wait complete. 2024-04-25T01:41:20.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.774+0000 7f9b110df640 1 Processor -- start 2024-04-25T01:41:20.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.774+0000 7f9b110df640 1 -- start start 2024-04-25T01:41:20.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.774+0000 7f9b110df640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b0c06cf10 0x7f9b0c142c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.774+0000 7f9b110df640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b0c06e600 0x7f9b0c1eb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.774+0000 7f9b110df640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b0c0c2d70 0x7f9b0c1ebac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.774+0000 7f9b110df640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c0c1690 con 0x7f9b0c06e600 2024-04-25T01:41:20.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.774+0000 7f9b110df640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c06e0e0 con 0x7f9b0c0c2d70 2024-04-25T01:41:20.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.774+0000 7f9b110df640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9b0c14ca90 con 0x7f9b0c06cf10 2024-04-25T01:41:20.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.774+0000 7f9b03fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b0c06e600 0x7f9b0c1eb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.774+0000 7f9b03fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b0c06e600 0x7f9b0c1eb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55058/0 (socket says 172.21.15.112:55058) 2024-04-25T01:41:20.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.774+0000 7f9b03fff640 1 -- 172.21.15.112:0/611885317 learned_addr learned my addr 172.21.15.112:0/611885317 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:20.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.775+0000 7f9b03fff640 1 -- 172.21.15.112:0/611885317 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b0c06cf10 msgr2=0x7f9b0c142c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.775+0000 7f9b03fff640 1 --2- 172.21.15.112:0/611885317 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b0c06cf10 0x7f9b0c142c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.775+0000 7f9b03fff640 1 -- 172.21.15.112:0/611885317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b0c0c2d70 msgr2=0x7f9b0c1ebac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.775+0000 7f9b03fff640 1 --2- 172.21.15.112:0/611885317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b0c0c2d70 0x7f9b0c1ebac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.775+0000 7f9b03fff640 1 -- 172.21.15.112:0/611885317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0c150700 con 0x7f9b0c06e600 2024-04-25T01:41:20.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.775+0000 7f9b03fff640 1 --2- 172.21.15.112:0/611885317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b0c06e600 0x7f9b0c1eb580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9af400c4d0 tx=0x7f9af400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:20.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.775+0000 7f9b037fe640 1 -- 172.21.15.112:0/611885317 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af4018070 con 0x7f9b0c06e600 2024-04-25T01:41:20.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.775+0000 7f9b037fe640 1 -- 172.21.15.112:0/611885317 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9af4007500 con 0x7f9b0c06e600 2024-04-25T01:41:20.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.775+0000 7f9b110df640 1 -- 172.21.15.112:0/611885317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0c1509e0 con 0x7f9b0c06e600 2024-04-25T01:41:20.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.775+0000 7f9b037fe640 1 -- 172.21.15.112:0/611885317 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af4004a20 con 0x7f9b0c06e600 2024-04-25T01:41:20.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.775+0000 7f9b110df640 1 -- 172.21.15.112:0/611885317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b0c0ab7a0 con 0x7f9b0c06e600 2024-04-25T01:41:20.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.776+0000 7f9b110df640 1 -- 172.21.15.112:0/611885317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b0c0682a0 con 0x7f9b0c06e600 2024-04-25T01:41:20.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.776+0000 7f9b037fe640 1 -- 172.21.15.112:0/611885317 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9af401b070 con 0x7f9b0c06e600 2024-04-25T01:41:20.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.777+0000 7f9b037fe640 1 --2- 172.21.15.112:0/611885317 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9aec06be30 0x7f9aec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.777+0000 7f9b037fe640 1 -- 172.21.15.112:0/611885317 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f9af408a500 con 0x7f9b0c06e600 2024-04-25T01:41:20.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.777+0000 7f9b0ad9d640 1 --2- 172.21.15.112:0/611885317 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9aec06be30 0x7f9aec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.777+0000 7f9b037fe640 1 --2- 172.21.15.112:0/611885317 >> [v2:172.21.15.187:6808/931860956,v1:172.21.15.187:6809/931860956] conn(0x7f9aec0767c0 0x7f9aec078c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.777+0000 7f9b037fe640 1 -- 172.21.15.112:0/611885317 --> [v2:172.21.15.187:6808/931860956,v1:172.21.15.187:6809/931860956] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9af4017ec0 con 0x7f9aec0767c0 2024-04-25T01:41:20.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.777+0000 7f9b037fe640 1 -- 172.21.15.112:0/611885317 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9af4091050 con 0x7f9b0c06e600 2024-04-25T01:41:20.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.777+0000 7f9b0ad9d640 1 --2- 172.21.15.112:0/611885317 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9aec06be30 0x7f9aec06e2f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9afc034000 tx=0x7f9afc02c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:20.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.777+0000 7f9b0b59e640 1 --2- 172.21.15.112:0/611885317 >> [v2:172.21.15.187:6808/931860956,v1:172.21.15.187:6809/931860956] conn(0x7f9aec0767c0 0x7f9aec078c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.778+0000 7f9b0b59e640 1 --2- 172.21.15.112:0/611885317 >> [v2:172.21.15.187:6808/931860956,v1:172.21.15.187:6809/931860956] conn(0x7f9aec0767c0 0x7f9aec078c40 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:20.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.778+0000 7f9b037fe640 1 -- 172.21.15.112:0/611885317 <== osd.6 v2:172.21.15.187:6808/931860956 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9af4017ec0 con 0x7f9aec0767c0 2024-04-25T01:41:20.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.793+0000 7f9b110df640 1 -- 172.21.15.112:0/611885317 --> [v2:172.21.15.187:6808/931860956,v1:172.21.15.187:6809/931860956] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9b0c0abc90 con 0x7f9aec0767c0 2024-04-25T01:41:20.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.793+0000 7f9b037fe640 1 -- 172.21.15.112:0/611885317 <== osd.6 v2:172.21.15.187:6808/931860956 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9b0c0abc90 con 0x7f9aec0767c0 2024-04-25T01:41:20.780 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:41:20.780 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:41:20.780 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:41:20.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.793+0000 7f9b110df640 1 -- 172.21.15.112:0/611885317 >> [v2:172.21.15.187:6808/931860956,v1:172.21.15.187:6809/931860956] conn(0x7f9aec0767c0 msgr2=0x7f9aec078c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.793+0000 7f9b110df640 1 --2- 172.21.15.112:0/611885317 >> [v2:172.21.15.187:6808/931860956,v1:172.21.15.187:6809/931860956] conn(0x7f9aec0767c0 0x7f9aec078c40 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.793+0000 7f9b110df640 1 -- 172.21.15.112:0/611885317 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9aec06be30 msgr2=0x7f9aec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.793+0000 7f9b110df640 1 --2- 172.21.15.112:0/611885317 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f9aec06be30 0x7f9aec06e2f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9afc034000 tx=0x7f9afc02c000 comp rx=0 tx=0).stop 2024-04-25T01:41:20.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.793+0000 7f9b110df640 1 -- 172.21.15.112:0/611885317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b0c06e600 msgr2=0x7f9b0c1eb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.793+0000 7f9b110df640 1 --2- 172.21.15.112:0/611885317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b0c06e600 0x7f9b0c1eb580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9af400c4d0 tx=0x7f9af400c990 comp rx=0 tx=0).stop 2024-04-25T01:41:20.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.793+0000 7f9b0b59e640 1 -- 172.21.15.112:0/611885317 reap_dead start 2024-04-25T01:41:20.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.793+0000 7f9b110df640 1 -- 172.21.15.112:0/611885317 shutdown_connections 2024-04-25T01:41:20.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.793+0000 7f9b110df640 1 -- 172.21.15.112:0/611885317 >> 172.21.15.112:0/611885317 conn(0x7f9b0c06b8a0 msgr2=0x7f9b0c146d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:20.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.794+0000 7f9b110df640 1 -- 172.21.15.112:0/611885317 shutdown_connections 2024-04-25T01:41:20.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.794+0000 7f9b110df640 1 -- 172.21.15.112:0/611885317 wait complete. 2024-04-25T01:41:20.791 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T01:41:20.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.901+0000 7f84c4b4c640 1 Processor -- start 2024-04-25T01:41:20.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.901+0000 7f84c4b4c640 1 -- start start 2024-04-25T01:41:20.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.902+0000 7f84c4b4c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f84c014c3f0 0x7f84c00c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.902+0000 7f84c4b4c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84c01529b0 con 0x7f84c01524a0 2024-04-25T01:41:20.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.902+0000 7f84c4b4c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84c0152c70 con 0x7f84c00ac3b0 2024-04-25T01:41:20.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.902+0000 7f84c4b4c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f84c0155a90 con 0x7f84c014c3f0 2024-04-25T01:41:20.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.902+0000 7f84bed9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f84c01524a0 0x7f84c015ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54674/0 (socket says 172.21.15.112:54674) 2024-04-25T01:41:20.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.902+0000 7f84bed9d640 1 -- 172.21.15.112:0/1654088625 learned_addr learned my addr 172.21.15.112:0/1654088625 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:20.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84bdd9b640 1 -- 172.21.15.112:0/1654088625 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2000653292 0 0) 0x7f84c01529b0 con 0x7f84c01524a0 2024-04-25T01:41:20.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84bdd9b640 1 -- 172.21.15.112:0/1654088625 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84ac003520 con 0x7f84c01524a0 2024-04-25T01:41:20.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84be59c640 1 --2- 172.21.15.112:0/1654088625 >> v2:172.21.15.187:3300/0 conn(0x7f84c014c3f0 0x7f84c00c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84bdd9b640 1 -- 172.21.15.112:0/1654088625 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1686785450 0 0) 0x7f84ac003520 con 0x7f84c01524a0 2024-04-25T01:41:20.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84bdd9b640 1 -- 172.21.15.112:0/1654088625 >> v2:172.21.15.187:3300/0 conn(0x7f84c014c3f0 msgr2=0x7f84c00c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:20.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84bdd9b640 1 --2- 172.21.15.112:0/1654088625 >> v2:172.21.15.187:3300/0 conn(0x7f84c014c3f0 0x7f84c00c2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84bdd9b640 1 -- 172.21.15.112:0/1654088625 >> v1:172.21.15.187:6789/0 conn(0x7f84c00ac3b0 legacy=0x7f84c0155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:20.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84bdd9b640 1 -- 172.21.15.112:0/1654088625 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c0152f50 con 0x7f84c01524a0 2024-04-25T01:41:20.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84bdd9b640 1 -- 172.21.15.112:0/1654088625 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f84b80019f0 con 0x7f84c01524a0 2024-04-25T01:41:20.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84bdd9b640 1 -- 172.21.15.112:0/1654088625 >> v1:172.21.15.112:6789/0 conn(0x7f84c01524a0 legacy=0x7f84c015ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84bdd9b640 1 --2- 172.21.15.112:0/1654088625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84ac0039b0 0x7f84ac003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84bdd9b640 1 --2- 172.21.15.112:0/1654088625 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84ac004800 0x7f84ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84bdd9b640 1 --2- 172.21.15.112:0/1654088625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84ac0052e0 0x7f84ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84bdd9b640 1 -- 172.21.15.112:0/1654088625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f84c0155a90 con 0x7f84ac0039b0 2024-04-25T01:41:20.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84bdd9b640 1 -- 172.21.15.112:0/1654088625 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f84ac00c190 con 0x7f84ac0052e0 2024-04-25T01:41:20.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84bdd9b640 1 -- 172.21.15.112:0/1654088625 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f84ac00c390 con 0x7f84ac004800 2024-04-25T01:41:20.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84be59c640 1 --2- 172.21.15.112:0/1654088625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84ac0039b0 0x7f84ac003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84b7fff640 1 --2- 172.21.15.112:0/1654088625 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84ac004800 0x7f84ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84bed9d640 1 --2- 172.21.15.112:0/1654088625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84ac0052e0 0x7f84ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84be59c640 1 -- 172.21.15.112:0/1654088625 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84ac004800 msgr2=0x7f84ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:20.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84be59c640 1 --2- 172.21.15.112:0/1654088625 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84ac004800 0x7f84ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84be59c640 1 -- 172.21.15.112:0/1654088625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84ac0052e0 msgr2=0x7f84ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:20.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84be59c640 1 --2- 172.21.15.112:0/1654088625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84ac0052e0 0x7f84ac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84bed9d640 1 -- 172.21.15.112:0/1654088625 reap_dead start 2024-04-25T01:41:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.903+0000 7f84be59c640 1 -- 172.21.15.112:0/1654088625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ac003520 con 0x7f84ac0039b0 2024-04-25T01:41:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.904+0000 7f84be59c640 1 --2- 172.21.15.112:0/1654088625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84ac0039b0 0x7f84ac003db0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f84b000ba00 tx=0x7f84ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcfc096119ab7052 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.904+0000 7f84bdd9b640 1 -- 172.21.15.112:0/1654088625 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0030070 con 0x7f84ac0039b0 2024-04-25T01:41:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.904+0000 7f84bdd9b640 1 -- 172.21.15.112:0/1654088625 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84b002adb0 con 0x7f84ac0039b0 2024-04-25T01:41:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.904+0000 7f84bdd9b640 1 -- 172.21.15.112:0/1654088625 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b000b600 con 0x7f84ac0039b0 2024-04-25T01:41:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.905+0000 7f84c4b4c640 1 -- 172.21.15.112:0/1654088625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84ac0039b0 msgr2=0x7f84ac003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.905+0000 7f84c4b4c640 1 --2- 172.21.15.112:0/1654088625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84ac0039b0 0x7f84ac003db0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f84b000ba00 tx=0x7f84ac005830 comp rx=0 tx=0).stop 2024-04-25T01:41:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.905+0000 7f84c4b4c640 1 -- 172.21.15.112:0/1654088625 shutdown_connections 2024-04-25T01:41:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.905+0000 7f84c4b4c640 1 --2- 172.21.15.112:0/1654088625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84ac0039b0 0x7f84ac003db0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.905+0000 7f84c4b4c640 1 -- 172.21.15.112:0/1654088625 >> 172.21.15.112:0/1654088625 conn(0x7f84c006b160 msgr2=0x7f84c006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.905+0000 7f84c4b4c640 1 -- 172.21.15.112:0/1654088625 shutdown_connections 2024-04-25T01:41:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.905+0000 7f84c4b4c640 1 -- 172.21.15.112:0/1654088625 wait complete. 2024-04-25T01:41:20.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.906+0000 7f84c4b4c640 1 Processor -- start 2024-04-25T01:41:20.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.906+0000 7f84c4b4c640 1 -- start start 2024-04-25T01:41:20.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84c4b4c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84ac0039b0 0x7f84c006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84c4b4c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84c01520b0 0x7f84c0142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84c4b4c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84c014bb60 0x7f84c0142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84c4b4c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f84c014c7d0 con 0x7f84ac0039b0 2024-04-25T01:41:20.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84c4b4c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f84c0070d60 con 0x7f84c014bb60 2024-04-25T01:41:20.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84c4b4c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f84c014c950 con 0x7f84c01520b0 2024-04-25T01:41:20.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84be59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84ac0039b0 0x7f84c006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84be59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84ac0039b0 0x7f84c006e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55080/0 (socket says 172.21.15.112:55080) 2024-04-25T01:41:20.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84be59c640 1 -- 172.21.15.112:0/4125431736 learned_addr learned my addr 172.21.15.112:0/4125431736 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:20.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84b7fff640 1 --2- 172.21.15.112:0/4125431736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84c01520b0 0x7f84c0142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84bed9d640 1 --2- 172.21.15.112:0/4125431736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84c014bb60 0x7f84c0142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84be59c640 1 -- 172.21.15.112:0/4125431736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84c01520b0 msgr2=0x7f84c0142650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84be59c640 1 --2- 172.21.15.112:0/4125431736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84c01520b0 0x7f84c0142650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84be59c640 1 -- 172.21.15.112:0/4125431736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84c014bb60 msgr2=0x7f84c0142b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:20.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84be59c640 1 --2- 172.21.15.112:0/4125431736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84c014bb60 0x7f84c0142b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:20.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84be59c640 1 -- 172.21.15.112:0/4125431736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c01430d0 con 0x7f84ac0039b0 2024-04-25T01:41:20.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84be59c640 1 --2- 172.21.15.112:0/4125431736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84ac0039b0 0x7f84c006e2a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f84b002a8a0 tx=0x7f84b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:20.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84b6ffd640 1 -- 172.21.15.112:0/4125431736 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0043020 con 0x7f84ac0039b0 2024-04-25T01:41:20.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84b6ffd640 1 -- 172.21.15.112:0/4125431736 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84b003f680 con 0x7f84ac0039b0 2024-04-25T01:41:20.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84c4b4c640 1 -- 172.21.15.112:0/4125431736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c0153890 con 0x7f84ac0039b0 2024-04-25T01:41:20.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84b6ffd640 1 -- 172.21.15.112:0/4125431736 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b000b600 con 0x7f84ac0039b0 2024-04-25T01:41:20.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.907+0000 7f84c4b4c640 1 -- 172.21.15.112:0/4125431736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c01ea5e0 con 0x7f84ac0039b0 2024-04-25T01:41:20.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.908+0000 7f84c4b4c640 1 -- 172.21.15.112:0/4125431736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84c00682a0 con 0x7f84ac0039b0 2024-04-25T01:41:20.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.909+0000 7f84b6ffd640 1 -- 172.21.15.112:0/4125431736 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f84b0046020 con 0x7f84ac0039b0 2024-04-25T01:41:20.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.909+0000 7f84b6ffd640 1 --2- 172.21.15.112:0/4125431736 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f84a006be30 0x7f84a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:20.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.909+0000 7f84b6ffd640 1 -- 172.21.15.112:0/4125431736 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f84b00abae0 con 0x7f84ac0039b0 2024-04-25T01:41:20.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.909+0000 7f84b7fff640 1 --2- 172.21.15.112:0/4125431736 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f84a006be30 0x7f84a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:20.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.909+0000 7f84b7fff640 1 --2- 172.21.15.112:0/4125431736 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f84a006be30 0x7f84a006e2f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f84a8007fd0 tx=0x7f84a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:20.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:20.913+0000 7f84b6ffd640 1 -- 172.21.15.112:0/4125431736 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f84b004d070 con 0x7f84ac0039b0 2024-04-25T01:41:21.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.088+0000 7f84c4b4c640 1 -- 172.21.15.112:0/4125431736 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f84c0075980 con 0x7f84a006be30 2024-04-25T01:41:21.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.089+0000 7f84b6ffd640 1 -- 172.21.15.112:0/4125431736 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f84a8016440 con 0x7f84a006be30 2024-04-25T01:41:21.075 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T01:41:21.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.090+0000 7f84c4b4c640 1 -- 172.21.15.112:0/4125431736 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f84a006be30 msgr2=0x7f84a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:21.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.090+0000 7f84c4b4c640 1 --2- 172.21.15.112:0/4125431736 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f84a006be30 0x7f84a006e2f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f84a8007fd0 tx=0x7f84a8009040 comp rx=0 tx=0).stop 2024-04-25T01:41:21.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.090+0000 7f84c4b4c640 1 -- 172.21.15.112:0/4125431736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84ac0039b0 msgr2=0x7f84c006e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:21.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.090+0000 7f84c4b4c640 1 --2- 172.21.15.112:0/4125431736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84ac0039b0 0x7f84c006e2a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f84b002a8a0 tx=0x7f84b0004730 comp rx=0 tx=0).stop 2024-04-25T01:41:21.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.090+0000 7f84c4b4c640 1 -- 172.21.15.112:0/4125431736 shutdown_connections 2024-04-25T01:41:21.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.090+0000 7f84c4b4c640 1 --2- 172.21.15.112:0/4125431736 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f84a006be30 0x7f84a006e2f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.091+0000 7f84c4b4c640 1 --2- 172.21.15.112:0/4125431736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84c014bb60 0x7f84c0142b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.091+0000 7f84c4b4c640 1 --2- 172.21.15.112:0/4125431736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84c01520b0 0x7f84c0142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.091+0000 7f84c4b4c640 1 --2- 172.21.15.112:0/4125431736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84ac0039b0 0x7f84c006e2a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.091+0000 7f84c4b4c640 1 -- 172.21.15.112:0/4125431736 >> 172.21.15.112:0/4125431736 conn(0x7f84c006b160 msgr2=0x7f84c00c7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:21.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.091+0000 7f84c4b4c640 1 -- 172.21.15.112:0/4125431736 shutdown_connections 2024-04-25T01:41:21.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.091+0000 7f84c4b4c640 1 -- 172.21.15.112:0/4125431736 wait complete. 2024-04-25T01:41:21.089 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T01:41:21.089 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:41:21.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.202+0000 7fd55697d640 1 Processor -- start 2024-04-25T01:41:21.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.202+0000 7fd55697d640 1 -- start start 2024-04-25T01:41:21.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.203+0000 7fd55697d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5501531b0 0x7fd550159a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.203+0000 7fd55697d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd5500c9020 0x7fd5500c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.203+0000 7fd55697d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd550155ba0 con 0x7fd5500c6880 2024-04-25T01:41:21.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.203+0000 7fd55697d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd550156a70 con 0x7fd5500c9020 2024-04-25T01:41:21.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.203+0000 7fd55697d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd5500c94b0 con 0x7fd5501531b0 2024-04-25T01:41:21.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.203+0000 7fd54f7fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd5500c6880 0x7fd550156360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54680/0 (socket says 172.21.15.112:54680) 2024-04-25T01:41:21.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.203+0000 7fd54f7fe640 1 -- 172.21.15.112:0/1539705727 learned_addr learned my addr 172.21.15.112:0/1539705727 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:41:21.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.203+0000 7fd554f1a640 1 --2- 172.21.15.112:0/1539705727 >> v2:172.21.15.187:3300/0 conn(0x7fd5500c9020 0x7fd5500c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.203+0000 7fd54ffff640 1 --2- 172.21.15.112:0/1539705727 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5501531b0 0x7fd550159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.203+0000 7fd54effd640 1 -- 172.21.15.112:0/1539705727 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1423411916 0 0) 0x7fd550155ba0 con 0x7fd5500c6880 2024-04-25T01:41:21.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54effd640 1 -- 172.21.15.112:0/1539705727 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd538003520 con 0x7fd5500c6880 2024-04-25T01:41:21.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54effd640 1 -- 172.21.15.112:0/1539705727 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3017349807 0 0) 0x7fd538003520 con 0x7fd5500c6880 2024-04-25T01:41:21.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54effd640 1 -- 172.21.15.112:0/1539705727 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5501531b0 msgr2=0x7fd550159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:21.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54effd640 1 --2- 172.21.15.112:0/1539705727 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5501531b0 0x7fd550159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54effd640 1 -- 172.21.15.112:0/1539705727 >> v2:172.21.15.187:3300/0 conn(0x7fd5500c9020 msgr2=0x7fd5500c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:21.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54effd640 1 --2- 172.21.15.112:0/1539705727 >> v2:172.21.15.187:3300/0 conn(0x7fd5500c9020 0x7fd5500c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54effd640 1 -- 172.21.15.112:0/1539705727 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5500caee0 con 0x7fd5500c6880 2024-04-25T01:41:21.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54effd640 1 -- 172.21.15.112:0/1539705727 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fd544001670 con 0x7fd5500c6880 2024-04-25T01:41:21.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54effd640 1 -- 172.21.15.112:0/1539705727 >> v1:172.21.15.112:6789/0 conn(0x7fd5500c6880 legacy=0x7fd550156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:21.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54effd640 1 --2- 172.21.15.112:0/1539705727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5380039b0 0x7fd538003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54effd640 1 --2- 172.21.15.112:0/1539705727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd538004800 0x7fd538004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54effd640 1 --2- 172.21.15.112:0/1539705727 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5501531b0 0x7fd5380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54effd640 1 -- 172.21.15.112:0/1539705727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd550156a70 con 0x7fd538004800 2024-04-25T01:41:21.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54effd640 1 -- 172.21.15.112:0/1539705727 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd53800b9c0 con 0x7fd5380039b0 2024-04-25T01:41:21.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54effd640 1 -- 172.21.15.112:0/1539705727 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd53800bb40 con 0x7fd5501531b0 2024-04-25T01:41:21.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54f7fe640 1 --2- 172.21.15.112:0/1539705727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd538004800 0x7fd538004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54ffff640 1 --2- 172.21.15.112:0/1539705727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5380039b0 0x7fd538003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd554f1a640 1 --2- 172.21.15.112:0/1539705727 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5501531b0 0x7fd5380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54f7fe640 1 -- 172.21.15.112:0/1539705727 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5501531b0 msgr2=0x7fd5380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:21.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54f7fe640 1 --2- 172.21.15.112:0/1539705727 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5501531b0 0x7fd5380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54f7fe640 1 -- 172.21.15.112:0/1539705727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5380039b0 msgr2=0x7fd538003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:21.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54f7fe640 1 --2- 172.21.15.112:0/1539705727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5380039b0 0x7fd538003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.204+0000 7fd54f7fe640 1 -- 172.21.15.112:0/1539705727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd538003520 con 0x7fd538004800 2024-04-25T01:41:21.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.205+0000 7fd54f7fe640 1 --2- 172.21.15.112:0/1539705727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd538004800 0x7fd538004c80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd54400c210 tx=0x7fd538005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4a870344a4f3f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:21.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.205+0000 7fd54effd640 1 -- 172.21.15.112:0/1539705727 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544007d70 con 0x7fd538004800 2024-04-25T01:41:21.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.205+0000 7fd54effd640 1 -- 172.21.15.112:0/1539705727 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd544005af0 con 0x7fd538004800 2024-04-25T01:41:21.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.205+0000 7fd54effd640 1 -- 172.21.15.112:0/1539705727 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544007070 con 0x7fd538004800 2024-04-25T01:41:21.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.206+0000 7fd55697d640 1 -- 172.21.15.112:0/1539705727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd538004800 msgr2=0x7fd538004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:21.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.206+0000 7fd55697d640 1 --2- 172.21.15.112:0/1539705727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd538004800 0x7fd538004c80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd54400c210 tx=0x7fd538005830 comp rx=0 tx=0).stop 2024-04-25T01:41:21.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.206+0000 7fd554f1a640 1 -- 172.21.15.112:0/1539705727 reap_dead start 2024-04-25T01:41:21.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.206+0000 7fd55697d640 1 -- 172.21.15.112:0/1539705727 shutdown_connections 2024-04-25T01:41:21.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.206+0000 7fd55697d640 1 -- 172.21.15.112:0/1539705727 >> 172.21.15.112:0/1539705727 conn(0x7fd5500c7ba0 msgr2=0x7fd5500c7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:21.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.206+0000 7fd55697d640 1 -- 172.21.15.112:0/1539705727 shutdown_connections 2024-04-25T01:41:21.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.206+0000 7fd55697d640 1 -- 172.21.15.112:0/1539705727 wait complete. 2024-04-25T01:41:21.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.207+0000 7fd55697d640 1 Processor -- start 2024-04-25T01:41:21.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.207+0000 7fd55697d640 1 -- start start 2024-04-25T01:41:21.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.207+0000 7fd55697d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd538004800 0x7fd5500adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.207+0000 7fd55697d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5500af920 0x7fd5500ab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.207+0000 7fd55697d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5500b0690 0x7fd550148370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.207+0000 7fd55697d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5500c7880 con 0x7fd538004800 2024-04-25T01:41:21.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.208+0000 7fd55697d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd550155ed0 con 0x7fd5500af920 2024-04-25T01:41:21.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.208+0000 7fd55697d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd550153590 con 0x7fd5500b0690 2024-04-25T01:41:21.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.208+0000 7fd54ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd538004800 0x7fd5500adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.208+0000 7fd54ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd538004800 0x7fd5500adf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55112/0 (socket says 172.21.15.112:55112) 2024-04-25T01:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.208+0000 7fd54ffff640 1 -- 172.21.15.112:0/3010215756 learned_addr learned my addr 172.21.15.112:0/3010215756 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.208+0000 7fd554f1a640 1 --2- 172.21.15.112:0/3010215756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5500b0690 0x7fd550148370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.208+0000 7fd54f7fe640 1 --2- 172.21.15.112:0/3010215756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5500af920 0x7fd5500ab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.208+0000 7fd54ffff640 1 -- 172.21.15.112:0/3010215756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5500b0690 msgr2=0x7fd550148370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.208+0000 7fd54ffff640 1 --2- 172.21.15.112:0/3010215756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5500b0690 0x7fd550148370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.208+0000 7fd54ffff640 1 -- 172.21.15.112:0/3010215756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5500af920 msgr2=0x7fd5500ab2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.208+0000 7fd54ffff640 1 --2- 172.21.15.112:0/3010215756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5500af920 0x7fd5500ab2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.208+0000 7fd54ffff640 1 -- 172.21.15.112:0/3010215756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5500acea0 con 0x7fd538004800 2024-04-25T01:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.208+0000 7fd554f1a640 1 --2- 172.21.15.112:0/3010215756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5500b0690 0x7fd550148370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.208+0000 7fd54f7fe640 1 --2- 172.21.15.112:0/3010215756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5500af920 0x7fd5500ab2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.208+0000 7fd54ffff640 1 --2- 172.21.15.112:0/3010215756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd538004800 0x7fd5500adf50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd5400118e0 tx=0x7fd540011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.208+0000 7fd54cff9640 1 -- 172.21.15.112:0/3010215756 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54000fa90 con 0x7fd538004800 2024-04-25T01:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.208+0000 7fd54cff9640 1 -- 172.21.15.112:0/3010215756 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd54000fc30 con 0x7fd538004800 2024-04-25T01:41:21.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.208+0000 7fd54cff9640 1 -- 172.21.15.112:0/3010215756 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd540014440 con 0x7fd538004800 2024-04-25T01:41:21.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.208+0000 7fd55697d640 1 -- 172.21.15.112:0/3010215756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5500b0190 con 0x7fd538004800 2024-04-25T01:41:21.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.208+0000 7fd55697d640 1 -- 172.21.15.112:0/3010215756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5500ad3f0 con 0x7fd538004800 2024-04-25T01:41:21.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.209+0000 7fd55697d640 1 -- 172.21.15.112:0/3010215756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd5500682a0 con 0x7fd538004800 2024-04-25T01:41:21.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.210+0000 7fd54cff9640 1 -- 172.21.15.112:0/3010215756 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd540012040 con 0x7fd538004800 2024-04-25T01:41:21.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.210+0000 7fd54cff9640 1 --2- 172.21.15.112:0/3010215756 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd53c06be60 0x7fd53c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.210+0000 7fd54f7fe640 1 --2- 172.21.15.112:0/3010215756 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd53c06be60 0x7fd53c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.210+0000 7fd54cff9640 1 -- 172.21.15.112:0/3010215756 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fd54008b860 con 0x7fd538004800 2024-04-25T01:41:21.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.210+0000 7fd54cff9640 1 --2- 172.21.15.112:0/3010215756 >> [v2:172.21.15.187:6816/155270368,v1:172.21.15.187:6817/155270368] conn(0x7fd53c0767f0 0x7fd53c078c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.210+0000 7fd54cff9640 1 -- 172.21.15.112:0/3010215756 --> [v2:172.21.15.187:6816/155270368,v1:172.21.15.187:6817/155270368] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd540011110 con 0x7fd53c0767f0 2024-04-25T01:41:21.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.210+0000 7fd54cff9640 1 -- 172.21.15.112:0/3010215756 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd5400902c0 con 0x7fd538004800 2024-04-25T01:41:21.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.210+0000 7fd54f7fe640 1 --2- 172.21.15.112:0/3010215756 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd53c06be60 0x7fd53c06e320 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd544033000 tx=0x7fd544009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:21.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.211+0000 7fd554f1a640 1 --2- 172.21.15.112:0/3010215756 >> [v2:172.21.15.187:6816/155270368,v1:172.21.15.187:6817/155270368] conn(0x7fd53c0767f0 0x7fd53c078c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.211+0000 7fd554f1a640 1 --2- 172.21.15.112:0/3010215756 >> [v2:172.21.15.187:6816/155270368,v1:172.21.15.187:6817/155270368] conn(0x7fd53c0767f0 0x7fd53c078c70 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:21.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.212+0000 7fd54cff9640 1 -- 172.21.15.112:0/3010215756 <== osd.7 v2:172.21.15.187:6816/155270368 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd540011110 con 0x7fd53c0767f0 2024-04-25T01:41:21.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.225+0000 7fd55697d640 1 -- 172.21.15.112:0/3010215756 --> [v2:172.21.15.187:6816/155270368,v1:172.21.15.187:6817/155270368] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd55014bee0 con 0x7fd53c0767f0 2024-04-25T01:41:21.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.225+0000 7fd54cff9640 1 -- 172.21.15.112:0/3010215756 <== osd.7 v2:172.21.15.187:6816/155270368 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd55014bee0 con 0x7fd53c0767f0 2024-04-25T01:41:21.212 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T01:41:21.212 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:41:21.212 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T01:41:21.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.226+0000 7fd55697d640 1 -- 172.21.15.112:0/3010215756 >> [v2:172.21.15.187:6816/155270368,v1:172.21.15.187:6817/155270368] conn(0x7fd53c0767f0 msgr2=0x7fd53c078c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:21.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.226+0000 7fd55697d640 1 --2- 172.21.15.112:0/3010215756 >> [v2:172.21.15.187:6816/155270368,v1:172.21.15.187:6817/155270368] conn(0x7fd53c0767f0 0x7fd53c078c70 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.226+0000 7fd55697d640 1 -- 172.21.15.112:0/3010215756 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd53c06be60 msgr2=0x7fd53c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:21.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.226+0000 7fd55697d640 1 --2- 172.21.15.112:0/3010215756 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fd53c06be60 0x7fd53c06e320 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd544033000 tx=0x7fd544009030 comp rx=0 tx=0).stop 2024-04-25T01:41:21.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.226+0000 7fd55697d640 1 -- 172.21.15.112:0/3010215756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd538004800 msgr2=0x7fd5500adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:21.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.226+0000 7fd55697d640 1 --2- 172.21.15.112:0/3010215756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd538004800 0x7fd5500adf50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd5400118e0 tx=0x7fd540011da0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.226+0000 7fd554f1a640 1 -- 172.21.15.112:0/3010215756 reap_dead start 2024-04-25T01:41:21.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.226+0000 7fd55697d640 1 -- 172.21.15.112:0/3010215756 shutdown_connections 2024-04-25T01:41:21.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.226+0000 7fd55697d640 1 -- 172.21.15.112:0/3010215756 >> 172.21.15.112:0/3010215756 conn(0x7fd5500c7ba0 msgr2=0x7fd55014b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:21.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.226+0000 7fd55697d640 1 -- 172.21.15.112:0/3010215756 shutdown_connections 2024-04-25T01:41:21.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.226+0000 7fd55697d640 1 -- 172.21.15.112:0/3010215756 wait complete. 2024-04-25T01:41:21.223 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T01:41:21.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.334+0000 7f8843b10640 1 Processor -- start 2024-04-25T01:41:21.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.334+0000 7f8843b10640 1 -- start start 2024-04-25T01:41:21.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.335+0000 7f8843b10640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f883c14c3f0 0x7f883c0c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.335+0000 7f8843b10640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f883c0ac3b0 0x7f883c155f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.335+0000 7f8843b10640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f883c1524a0 0x7f883c156460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.335+0000 7f8843b10640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f883c155a90 con 0x7f883c0ac3b0 2024-04-25T01:41:21.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.335+0000 7f8843b10640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f883c1569a0 con 0x7f883c1524a0 2024-04-25T01:41:21.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.335+0000 7f8843b10640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f883c1529b0 con 0x7f883c14c3f0 2024-04-25T01:41:21.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.335+0000 7f88410ab640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f883c0ac3b0 0x7f883c155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.335+0000 7f88410ab640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f883c0ac3b0 0x7f883c155f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55120/0 (socket says 172.21.15.112:55120) 2024-04-25T01:41:21.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.335+0000 7f88410ab640 1 -- 172.21.15.112:0/3820634949 learned_addr learned my addr 172.21.15.112:0/3820634949 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:21.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.335+0000 7f88418ac640 1 --2- 172.21.15.112:0/3820634949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f883c14c3f0 0x7f883c0c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.335+0000 7f88420ad640 1 --2- 172.21.15.112:0/3820634949 >> v2:172.21.15.187:3300/0 conn(0x7f883c1524a0 0x7f883c156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.335+0000 7f88410ab640 1 -- 172.21.15.112:0/3820634949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f883c14c3f0 msgr2=0x7f883c0c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:21.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.335+0000 7f88410ab640 1 --2- 172.21.15.112:0/3820634949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f883c14c3f0 0x7f883c0c2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.335+0000 7f88410ab640 1 -- 172.21.15.112:0/3820634949 >> v2:172.21.15.187:3300/0 conn(0x7f883c1524a0 msgr2=0x7f883c156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:21.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.335+0000 7f88410ab640 1 --2- 172.21.15.112:0/3820634949 >> v2:172.21.15.187:3300/0 conn(0x7f883c1524a0 0x7f883c156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.335+0000 7f88410ab640 1 -- 172.21.15.112:0/3820634949 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883c152c70 con 0x7f883c0ac3b0 2024-04-25T01:41:21.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.335+0000 7f88418ac640 1 --2- 172.21.15.112:0/3820634949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f883c14c3f0 0x7f883c0c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:41:21.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.335+0000 7f88420ad640 1 --2- 172.21.15.112:0/3820634949 >> v2:172.21.15.187:3300/0 conn(0x7f883c1524a0 0x7f883c156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:41:21.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.336+0000 7f88410ab640 1 --2- 172.21.15.112:0/3820634949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c0ac3b0 0x7f883c155f20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f883c153510 tx=0x7f882c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404c9273c514b870 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:21.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.336+0000 7f88408aa640 1 -- 172.21.15.112:0/3820634949 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c037070 con 0x7f883c0ac3b0 2024-04-25T01:41:21.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.336+0000 7f88408aa640 1 -- 172.21.15.112:0/3820634949 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f882c032440 con 0x7f883c0ac3b0 2024-04-25T01:41:21.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.336+0000 7f88408aa640 1 -- 172.21.15.112:0/3820634949 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c032810 con 0x7f883c0ac3b0 2024-04-25T01:41:21.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.337+0000 7f8843b10640 1 -- 172.21.15.112:0/3820634949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c0ac3b0 msgr2=0x7f883c155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:21.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.337+0000 7f8843b10640 1 --2- 172.21.15.112:0/3820634949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c0ac3b0 0x7f883c155f20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f883c153510 tx=0x7f882c02a140 comp rx=0 tx=0).stop 2024-04-25T01:41:21.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.337+0000 7f8843b10640 1 -- 172.21.15.112:0/3820634949 shutdown_connections 2024-04-25T01:41:21.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.337+0000 7f8843b10640 1 --2- 172.21.15.112:0/3820634949 >> v2:172.21.15.187:3300/0 conn(0x7f883c1524a0 0x7f883c156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.337+0000 7f8843b10640 1 --2- 172.21.15.112:0/3820634949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c0ac3b0 0x7f883c155f20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.337+0000 7f8843b10640 1 --2- 172.21.15.112:0/3820634949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f883c14c3f0 0x7f883c0c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.337+0000 7f8843b10640 1 -- 172.21.15.112:0/3820634949 >> 172.21.15.112:0/3820634949 conn(0x7f883c06b160 msgr2=0x7f883c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:21.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.337+0000 7f8843b10640 1 -- 172.21.15.112:0/3820634949 shutdown_connections 2024-04-25T01:41:21.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.337+0000 7f8843b10640 1 -- 172.21.15.112:0/3820634949 wait complete. 2024-04-25T01:41:21.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.338+0000 7f8843b10640 1 Processor -- start 2024-04-25T01:41:21.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.338+0000 7f8843b10640 1 -- start start 2024-04-25T01:41:21.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f8843b10640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c0ac3b0 0x7f883c1df3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f8843b10640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f883c14c3f0 0x7f883c0c2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f8843b10640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f883c1524a0 0x7f883c1457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f8843b10640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f883c070d60 con 0x7f883c0ac3b0 2024-04-25T01:41:21.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f8843b10640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f883c153230 con 0x7f883c14c3f0 2024-04-25T01:41:21.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f8843b10640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f883c14c7d0 con 0x7f883c1524a0 2024-04-25T01:41:21.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f88418ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c0ac3b0 0x7f883c1df3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f88418ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c0ac3b0 0x7f883c1df3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55132/0 (socket says 172.21.15.112:55132) 2024-04-25T01:41:21.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f88418ac640 1 -- 172.21.15.112:0/3449563233 learned_addr learned my addr 172.21.15.112:0/3449563233 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:21.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f88410ab640 1 --2- 172.21.15.112:0/3449563233 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f883c14c3f0 0x7f883c0c2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f88420ad640 1 --2- 172.21.15.112:0/3449563233 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f883c1524a0 0x7f883c1457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f88418ac640 1 -- 172.21.15.112:0/3449563233 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f883c1524a0 msgr2=0x7f883c1457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:21.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f88418ac640 1 --2- 172.21.15.112:0/3449563233 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f883c1524a0 0x7f883c1457a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f88418ac640 1 -- 172.21.15.112:0/3449563233 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f883c14c3f0 msgr2=0x7f883c0c2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:21.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f88418ac640 1 --2- 172.21.15.112:0/3449563233 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f883c14c3f0 0x7f883c0c2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f88418ac640 1 -- 172.21.15.112:0/3449563233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883c1dc7a0 con 0x7f883c0ac3b0 2024-04-25T01:41:21.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f88418ac640 1 --2- 172.21.15.112:0/3449563233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c0ac3b0 0x7f883c1df3b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8838009fc0 tx=0x7f883800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:21.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f88327fc640 1 -- 172.21.15.112:0/3449563233 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883801a070 con 0x7f883c0ac3b0 2024-04-25T01:41:21.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f88327fc640 1 -- 172.21.15.112:0/3449563233 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8838003bb0 con 0x7f883c0ac3b0 2024-04-25T01:41:21.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f8843b10640 1 -- 172.21.15.112:0/3449563233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883c1dca80 con 0x7f883c0ac3b0 2024-04-25T01:41:21.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f88327fc640 1 -- 172.21.15.112:0/3449563233 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838004030 con 0x7f883c0ac3b0 2024-04-25T01:41:21.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.339+0000 7f8843b10640 1 -- 172.21.15.112:0/3449563233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f883c1dd040 con 0x7f883c0ac3b0 2024-04-25T01:41:21.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.340+0000 7f8843b10640 1 -- 172.21.15.112:0/3449563233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f883c0682a0 con 0x7f883c0ac3b0 2024-04-25T01:41:21.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.341+0000 7f88327fc640 1 -- 172.21.15.112:0/3449563233 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f883801d070 con 0x7f883c0ac3b0 2024-04-25T01:41:21.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.341+0000 7f88327fc640 1 --2- 172.21.15.112:0/3449563233 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f882006be30 0x7f882006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.341+0000 7f88327fc640 1 -- 172.21.15.112:0/3449563233 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f883808b400 con 0x7f883c0ac3b0 2024-04-25T01:41:21.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.341+0000 7f88410ab640 1 --2- 172.21.15.112:0/3449563233 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f882006be30 0x7f882006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.342+0000 7f88410ab640 1 --2- 172.21.15.112:0/3449563233 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f882006be30 0x7f882006e2f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f882c034000 tx=0x7f882c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:21.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.345+0000 7f88327fc640 1 -- 172.21.15.112:0/3449563233 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8838010070 con 0x7f883c0ac3b0 2024-04-25T01:41:21.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.526+0000 7f8843b10640 1 -- 172.21.15.112:0/3449563233 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f883c075980 con 0x7f882006be30 2024-04-25T01:41:21.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.527+0000 7f88327fc640 1 -- 172.21.15.112:0/3449563233 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f882c032610 con 0x7f882006be30 2024-04-25T01:41:21.513 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T01:41:21.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.529+0000 7f8843b10640 1 -- 172.21.15.112:0/3449563233 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f882006be30 msgr2=0x7f882006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:21.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.529+0000 7f8843b10640 1 --2- 172.21.15.112:0/3449563233 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f882006be30 0x7f882006e2f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f882c034000 tx=0x7f882c02e000 comp rx=0 tx=0).stop 2024-04-25T01:41:21.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.529+0000 7f8843b10640 1 -- 172.21.15.112:0/3449563233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c0ac3b0 msgr2=0x7f883c1df3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:21.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.529+0000 7f8843b10640 1 --2- 172.21.15.112:0/3449563233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c0ac3b0 0x7f883c1df3b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8838009fc0 tx=0x7f883800d560 comp rx=0 tx=0).stop 2024-04-25T01:41:21.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.529+0000 7f8843b10640 1 -- 172.21.15.112:0/3449563233 shutdown_connections 2024-04-25T01:41:21.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.529+0000 7f8843b10640 1 --2- 172.21.15.112:0/3449563233 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f882006be30 0x7f882006e2f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.529+0000 7f8843b10640 1 --2- 172.21.15.112:0/3449563233 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f883c1524a0 0x7f883c1457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.529+0000 7f8843b10640 1 --2- 172.21.15.112:0/3449563233 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f883c14c3f0 0x7f883c0c2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.529+0000 7f8843b10640 1 --2- 172.21.15.112:0/3449563233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c0ac3b0 0x7f883c1df3b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.529+0000 7f8843b10640 1 -- 172.21.15.112:0/3449563233 >> 172.21.15.112:0/3449563233 conn(0x7f883c06b160 msgr2=0x7f883c150a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:21.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.529+0000 7f8843b10640 1 -- 172.21.15.112:0/3449563233 shutdown_connections 2024-04-25T01:41:21.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.529+0000 7f8843b10640 1 -- 172.21.15.112:0/3449563233 wait complete. 2024-04-25T01:41:21.527 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:41:21.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.637+0000 7ff132c5d640 1 Processor -- start 2024-04-25T01:41:21.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.637+0000 7ff132c5d640 1 -- start start 2024-04-25T01:41:21.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.638+0000 7ff132c5d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff12c152d60 0x7ff12c158da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.638+0000 7ff132c5d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c3400 0x7ff12c0c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.638+0000 7ff132c5d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff12c06f2d0 con 0x7ff12c0c4050 2024-04-25T01:41:21.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.638+0000 7ff132c5d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff12c159cd0 con 0x7ff12c152d60 2024-04-25T01:41:21.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.638+0000 7ff132c5d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff12c159430 con 0x7ff12c0c3400 2024-04-25T01:41:21.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.638+0000 7ff1309f9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff12c152d60 0x7ff12c158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.638+0000 7ff12bfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c3400 0x7ff12c0c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.638+0000 7ff1309f9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff12c152d60 0x7ff12c158da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:60282/0 (socket says 172.21.15.112:60282) 2024-04-25T01:41:21.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.638+0000 7ff1311fa640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7ff12c0c4050 0x7ff12c155de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:34894/0 (socket says 172.21.15.112:34894) 2024-04-25T01:41:21.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.638+0000 7ff12bfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c3400 0x7ff12c0c3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:56612/0 (socket says 172.21.15.112:56612) 2024-04-25T01:41:21.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.638+0000 7ff1309f9640 1 -- 172.21.15.112:0/403662364 learned_addr learned my addr 172.21.15.112:0/403662364 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:21.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.638+0000 7ff12bfff640 1 -- 172.21.15.112:0/403662364 >> v2:172.21.15.187:3300/0 conn(0x7ff12c152d60 msgr2=0x7ff12c158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:21.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.638+0000 7ff12bfff640 1 --2- 172.21.15.112:0/403662364 >> v2:172.21.15.187:3300/0 conn(0x7ff12c152d60 0x7ff12c158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.638+0000 7ff12bfff640 1 -- 172.21.15.112:0/403662364 >> v1:172.21.15.187:6789/0 conn(0x7ff12c0c4050 legacy=0x7ff12c155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:21.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.638+0000 7ff12bfff640 1 -- 172.21.15.112:0/403662364 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c159710 con 0x7ff12c0c3400 2024-04-25T01:41:21.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.638+0000 7ff1309f9640 1 --2- 172.21.15.112:0/403662364 >> v2:172.21.15.187:3300/0 conn(0x7ff12c152d60 0x7ff12c158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:41:21.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.639+0000 7ff12bfff640 1 --2- 172.21.15.112:0/403662364 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c3400 0x7ff12c0c3b10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff1180097a0 tx=0x7ff11802a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d930209e68d392 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:21.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.639+0000 7ff12b7fe640 1 -- 172.21.15.112:0/403662364 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118037070 con 0x7ff12c0c3400 2024-04-25T01:41:21.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.639+0000 7ff12b7fe640 1 -- 172.21.15.112:0/403662364 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff11802adb0 con 0x7ff12c0c3400 2024-04-25T01:41:21.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.639+0000 7ff12b7fe640 1 -- 172.21.15.112:0/403662364 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118032660 con 0x7ff12c0c3400 2024-04-25T01:41:21.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.640+0000 7ff132c5d640 1 -- 172.21.15.112:0/403662364 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c3400 msgr2=0x7ff12c0c3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:21.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.640+0000 7ff132c5d640 1 --2- 172.21.15.112:0/403662364 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c3400 0x7ff12c0c3b10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff1180097a0 tx=0x7ff11802a060 comp rx=0 tx=0).stop 2024-04-25T01:41:21.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.640+0000 7ff132c5d640 1 -- 172.21.15.112:0/403662364 shutdown_connections 2024-04-25T01:41:21.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.640+0000 7ff132c5d640 1 --2- 172.21.15.112:0/403662364 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c3400 0x7ff12c0c3b10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.640+0000 7ff132c5d640 1 --2- 172.21.15.112:0/403662364 >> v2:172.21.15.187:3300/0 conn(0x7ff12c152d60 0x7ff12c158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.640+0000 7ff132c5d640 1 -- 172.21.15.112:0/403662364 >> 172.21.15.112:0/403662364 conn(0x7ff12c06b160 msgr2=0x7ff12c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:21.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.640+0000 7ff132c5d640 1 -- 172.21.15.112:0/403662364 shutdown_connections 2024-04-25T01:41:21.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.640+0000 7ff132c5d640 1 -- 172.21.15.112:0/403662364 wait complete. 2024-04-25T01:41:21.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.641+0000 7ff132c5d640 1 Processor -- start 2024-04-25T01:41:21.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.641+0000 7ff132c5d640 1 -- start start 2024-04-25T01:41:21.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.642+0000 7ff132c5d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c0c3400 0x7ff12c1513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.642+0000 7ff132c5d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c4050 0x7ff12c1518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.642+0000 7ff132c5d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c152d60 0x7ff12c1eb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.642+0000 7ff132c5d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c153140 con 0x7ff12c152d60 2024-04-25T01:41:21.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.642+0000 7ff132c5d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c1558c0 con 0x7ff12c0c3400 2024-04-25T01:41:21.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.642+0000 7ff132c5d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff12c0aa1a0 con 0x7ff12c0c4050 2024-04-25T01:41:21.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.642+0000 7ff1311fa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c152d60 0x7ff12c1eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.642+0000 7ff1311fa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c152d60 0x7ff12c1eb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55140/0 (socket says 172.21.15.112:55140) 2024-04-25T01:41:21.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.642+0000 7ff1311fa640 1 -- 172.21.15.112:0/2110500185 learned_addr learned my addr 172.21.15.112:0/2110500185 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:21.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.642+0000 7ff1309f9640 1 --2- 172.21.15.112:0/2110500185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c0c3400 0x7ff12c1513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.642+0000 7ff12bfff640 1 --2- 172.21.15.112:0/2110500185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c4050 0x7ff12c1518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.642+0000 7ff1311fa640 1 -- 172.21.15.112:0/2110500185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c4050 msgr2=0x7ff12c1518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:21.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.642+0000 7ff1311fa640 1 --2- 172.21.15.112:0/2110500185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c4050 0x7ff12c1518f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.642+0000 7ff1311fa640 1 -- 172.21.15.112:0/2110500185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c0c3400 msgr2=0x7ff12c1513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:21.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.642+0000 7ff1311fa640 1 --2- 172.21.15.112:0/2110500185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c0c3400 0x7ff12c1513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.642+0000 7ff1311fa640 1 -- 172.21.15.112:0/2110500185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c0732a0 con 0x7ff12c152d60 2024-04-25T01:41:21.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.642+0000 7ff12bfff640 1 --2- 172.21.15.112:0/2110500185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c4050 0x7ff12c1518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:41:21.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.643+0000 7ff1311fa640 1 --2- 172.21.15.112:0/2110500185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c152d60 0x7ff12c1eb700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff12400c6e0 tx=0x7ff12400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:21.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.643+0000 7ff1297fa640 1 -- 172.21.15.112:0/2110500185 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124018070 con 0x7ff12c152d60 2024-04-25T01:41:21.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.643+0000 7ff1297fa640 1 -- 172.21.15.112:0/2110500185 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff124004d60 con 0x7ff12c152d60 2024-04-25T01:41:21.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.643+0000 7ff132c5d640 1 -- 172.21.15.112:0/2110500185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12c151e30 con 0x7ff12c152d60 2024-04-25T01:41:21.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.643+0000 7ff1297fa640 1 -- 172.21.15.112:0/2110500185 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12400a880 con 0x7ff12c152d60 2024-04-25T01:41:21.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.643+0000 7ff132c5d640 1 -- 172.21.15.112:0/2110500185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12c1507d0 con 0x7ff12c152d60 2024-04-25T01:41:21.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.644+0000 7ff132c5d640 1 -- 172.21.15.112:0/2110500185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff12c0682a0 con 0x7ff12c152d60 2024-04-25T01:41:21.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.644+0000 7ff1297fa640 1 -- 172.21.15.112:0/2110500185 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff12401b070 con 0x7ff12c152d60 2024-04-25T01:41:21.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.645+0000 7ff1297fa640 1 --2- 172.21.15.112:0/2110500185 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff11006be30 0x7ff11006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:21.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.645+0000 7ff1297fa640 1 -- 172.21.15.112:0/2110500185 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7ff12408a5c0 con 0x7ff12c152d60 2024-04-25T01:41:21.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.645+0000 7ff1309f9640 1 --2- 172.21.15.112:0/2110500185 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff11006be30 0x7ff11006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:21.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.645+0000 7ff1309f9640 1 --2- 172.21.15.112:0/2110500185 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff11006be30 0x7ff11006e2f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff120006ca0 tx=0x7ff120009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:21.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.649+0000 7ff1297fa640 1 -- 172.21.15.112:0/2110500185 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff124058580 con 0x7ff12c152d60 2024-04-25T01:41:21.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.820+0000 7ff132c5d640 1 -- 172.21.15.112:0/2110500185 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff12c158ac0 con 0x7ff11006be30 2024-04-25T01:41:21.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.831+0000 7ff1297fa640 1 -- 172.21.15.112:0/2110500185 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+491006 (secure 0 0 0) 0x7ff12c158ac0 con 0x7ff11006be30 2024-04-25T01:41:21.823 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:41:21.826 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T01:41:21.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.834+0000 7ff132c5d640 1 -- 172.21.15.112:0/2110500185 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff11006be30 msgr2=0x7ff11006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:21.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.834+0000 7ff132c5d640 1 --2- 172.21.15.112:0/2110500185 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff11006be30 0x7ff11006e2f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff120006ca0 tx=0x7ff120009240 comp rx=0 tx=0).stop 2024-04-25T01:41:21.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.834+0000 7ff132c5d640 1 -- 172.21.15.112:0/2110500185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c152d60 msgr2=0x7ff12c1eb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:21.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.834+0000 7ff132c5d640 1 --2- 172.21.15.112:0/2110500185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c152d60 0x7ff12c1eb700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff12400c6e0 tx=0x7ff12400cba0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.834+0000 7ff132c5d640 1 -- 172.21.15.112:0/2110500185 shutdown_connections 2024-04-25T01:41:21.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.834+0000 7ff132c5d640 1 --2- 172.21.15.112:0/2110500185 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7ff11006be30 0x7ff11006e2f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.834+0000 7ff132c5d640 1 --2- 172.21.15.112:0/2110500185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff12c152d60 0x7ff12c1eb700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.834+0000 7ff132c5d640 1 --2- 172.21.15.112:0/2110500185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff12c0c4050 0x7ff12c1518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.834+0000 7ff132c5d640 1 --2- 172.21.15.112:0/2110500185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c0c3400 0x7ff12c1513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:21.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.834+0000 7ff132c5d640 1 -- 172.21.15.112:0/2110500185 >> 172.21.15.112:0/2110500185 conn(0x7ff12c06b160 msgr2=0x7ff12c154f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:21.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.835+0000 7ff132c5d640 1 -- 172.21.15.112:0/2110500185 shutdown_connections 2024-04-25T01:41:21.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:21.835+0000 7ff132c5d640 1 -- 172.21.15.112:0/2110500185 wait complete. 2024-04-25T01:41:21.832 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2024-04-25T01:41:20.916354+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":36043200,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":196,"num_read_kb":327,"num_write":3138327,"num_write_kb":38804753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":607268,"ondisk_log_size":607268,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":7846528,"kb_used_data":72704,"kb_used_omap":12,"kb_used_meta":7773747,"kb_avail":747128192,"statfs":{"total":773094113280,"available":765059268608,"internally_reserved":0,"allocated":74448896,"data_stored":72990936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":7960317488},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002821"},"pg_stats":[{"pgid":"5.18","version":"26'4099","reported_seq":4121,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.267400+0000","last_change":"2024-04-25T01:26:02.412272+0000","last_active":"2024-04-25T01:41:09.267400+0000","last_peered":"2024-04-25T01:41:09.267400+0000","last_clean":"2024-04-25T01:41:09.267400+0000","last_became_active":"2024-04-25T01:26:02.411176+0000","last_became_peered":"2024-04-25T01:26:02.411176+0000","last_unstale":"2024-04-25T01:41:09.267400+0000","last_undegraded":"2024-04-25T01:41:09.267400+0000","last_fullsized":"2024-04-25T01:41:09.267400+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4099,"log_dups_size":0,"ondisk_log_size":4099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:49.498275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21931,"num_write_kb":288545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"26'658","reported_seq":691,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:44.444563+0000","last_change":"2024-04-25T01:25:52.353978+0000","last_active":"2024-04-25T01:40:44.444563+0000","last_peered":"2024-04-25T01:40:44.444563+0000","last_clean":"2024-04-25T01:40:44.444563+0000","last_became_active":"2024-04-25T01:25:52.353649+0000","last_became_peered":"2024-04-25T01:25:52.353649+0000","last_unstale":"2024-04-25T01:40:44.444563+0000","last_undegraded":"2024-04-25T01:40:44.444563+0000","last_fullsized":"2024-04-25T01:40:44.444563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:39.605833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":4468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"26'5333","reported_seq":5365,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.730148+0000","last_change":"2024-04-25T01:25:53.364257+0000","last_active":"2024-04-25T01:40:58.730148+0000","last_peered":"2024-04-25T01:40:58.730148+0000","last_clean":"2024-04-25T01:40:58.730148+0000","last_became_active":"2024-04-25T01:25:53.363409+0000","last_became_peered":"2024-04-25T01:25:53.363409+0000","last_unstale":"2024-04-25T01:40:58.730148+0000","last_undegraded":"2024-04-25T01:40:58.730148+0000","last_fullsized":"2024-04-25T01:40:58.730148+0000","mapping_epoch":18,"log_start":"26'100","ondisk_log_start":"26'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28405,"num_write_kb":331018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"26'3","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.088185+0000","last_change":"2024-04-25T01:26:01.407136+0000","last_active":"2024-04-25T01:40:37.088185+0000","last_peered":"2024-04-25T01:40:37.088185+0000","last_clean":"2024-04-25T01:40:37.088185+0000","last_became_active":"2024-04-25T01:26:01.406855+0000","last_became_peered":"2024-04-25T01:26:01.406855+0000","last_unstale":"2024-04-25T01:40:37.088185+0000","last_undegraded":"2024-04-25T01:40:37.088185+0000","last_fullsized":"2024-04-25T01:40:37.088185+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"26'5328","reported_seq":5351,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.321198+0000","last_change":"2024-04-25T01:26:02.425367+0000","last_active":"2024-04-25T01:41:09.321198+0000","last_peered":"2024-04-25T01:41:09.321198+0000","last_clean":"2024-04-25T01:41:09.321198+0000","last_became_active":"2024-04-25T01:26:02.425071+0000","last_became_peered":"2024-04-25T01:26:02.425071+0000","last_unstale":"2024-04-25T01:41:09.321198+0000","last_undegraded":"2024-04-25T01:41:09.321198+0000","last_fullsized":"2024-04-25T01:41:09.321198+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5328,"log_dups_size":0,"ondisk_log_size":5328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27477,"num_write_kb":347746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"26'8","reported_seq":42,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.226925+0000","last_change":"2024-04-25T01:25:52.349185+0000","last_active":"2024-04-25T01:40:49.226925+0000","last_peered":"2024-04-25T01:40:49.226925+0000","last_clean":"2024-04-25T01:40:49.226925+0000","last_became_active":"2024-04-25T01:25:52.348931+0000","last_became_peered":"2024-04-25T01:25:52.348931+0000","last_unstale":"2024-04-25T01:40:49.226925+0000","last_undegraded":"2024-04-25T01:40:49.226925+0000","last_fullsized":"2024-04-25T01:40:49.226925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"26'5837","reported_seq":5868,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.789129+0000","last_change":"2024-04-25T01:25:53.374150+0000","last_active":"2024-04-25T01:40:58.789129+0000","last_peered":"2024-04-25T01:40:58.789129+0000","last_clean":"2024-04-25T01:40:58.789129+0000","last_became_active":"2024-04-25T01:25:53.373755+0000","last_became_peered":"2024-04-25T01:25:53.373755+0000","last_unstale":"2024-04-25T01:40:58.789129+0000","last_undegraded":"2024-04-25T01:40:58.789129+0000","last_fullsized":"2024-04-25T01:40:58.789129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5837,"log_dups_size":0,"ondisk_log_size":5837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30942,"num_write_kb":369940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"26'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:32.791998+0000","last_change":"2024-04-25T01:26:01.406334+0000","last_active":"2024-04-25T01:40:32.791998+0000","last_peered":"2024-04-25T01:40:32.791998+0000","last_clean":"2024-04-25T01:40:32.791998+0000","last_became_active":"2024-04-25T01:26:01.405994+0000","last_became_peered":"2024-04-25T01:26:01.405994+0000","last_unstale":"2024-04-25T01:40:32.791998+0000","last_undegraded":"2024-04-25T01:40:32.791998+0000","last_fullsized":"2024-04-25T01:40:32.791998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:43.953437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"26'4188","reported_seq":4210,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.336545+0000","last_change":"2024-04-25T01:26:02.427695+0000","last_active":"2024-04-25T01:41:09.336545+0000","last_peered":"2024-04-25T01:41:09.336545+0000","last_clean":"2024-04-25T01:41:09.336545+0000","last_became_active":"2024-04-25T01:26:02.427443+0000","last_became_peered":"2024-04-25T01:26:02.427443+0000","last_unstale":"2024-04-25T01:41:09.336545+0000","last_undegraded":"2024-04-25T01:41:09.336545+0000","last_fullsized":"2024-04-25T01:41:09.336545+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4188,"log_dups_size":0,"ondisk_log_size":4188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20869,"num_write_kb":259707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"26'11","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.299916+0000","last_change":"2024-04-25T01:25:52.356403+0000","last_active":"2024-04-25T01:40:49.299916+0000","last_peered":"2024-04-25T01:40:49.299916+0000","last_clean":"2024-04-25T01:40:49.299916+0000","last_became_active":"2024-04-25T01:25:52.355822+0000","last_became_peered":"2024-04-25T01:25:52.355822+0000","last_unstale":"2024-04-25T01:40:49.299916+0000","last_undegraded":"2024-04-25T01:40:49.299916+0000","last_fullsized":"2024-04-25T01:40:49.299916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.332405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"26'3747","reported_seq":3777,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.846633+0000","last_change":"2024-04-25T01:25:53.370061+0000","last_active":"2024-04-25T01:40:58.846633+0000","last_peered":"2024-04-25T01:40:58.846633+0000","last_clean":"2024-04-25T01:40:58.846633+0000","last_became_active":"2024-04-25T01:25:53.369721+0000","last_became_peered":"2024-04-25T01:25:53.369721+0000","last_unstale":"2024-04-25T01:40:58.846633+0000","last_undegraded":"2024-04-25T01:40:58.846633+0000","last_fullsized":"2024-04-25T01:40:58.846633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3747,"log_dups_size":0,"ondisk_log_size":3747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18978,"num_write_kb":223593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"26'627","reported_seq":651,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.103722+0000","last_change":"2024-04-25T01:26:01.408179+0000","last_active":"2024-04-25T01:40:37.103722+0000","last_peered":"2024-04-25T01:40:37.103722+0000","last_clean":"2024-04-25T01:40:37.103722+0000","last_became_active":"2024-04-25T01:26:01.407909+0000","last_became_peered":"2024-04-25T01:26:01.407909+0000","last_unstale":"2024-04-25T01:40:37.103722+0000","last_undegraded":"2024-04-25T01:40:37.103722+0000","last_fullsized":"2024-04-25T01:40:37.103722+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:48.865940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":625,"num_write_kb":4235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"26'3441","reported_seq":3463,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.298394+0000","last_change":"2024-04-25T01:26:02.424615+0000","last_active":"2024-04-25T01:41:09.298394+0000","last_peered":"2024-04-25T01:41:09.298394+0000","last_clean":"2024-04-25T01:41:09.298394+0000","last_became_active":"2024-04-25T01:26:02.424325+0000","last_became_peered":"2024-04-25T01:26:02.424325+0000","last_unstale":"2024-04-25T01:41:09.298394+0000","last_undegraded":"2024-04-25T01:41:09.298394+0000","last_fullsized":"2024-04-25T01:41:09.298394+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3441,"log_dups_size":0,"ondisk_log_size":3441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17582,"num_write_kb":230191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"26'3","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.256836+0000","last_change":"2024-04-25T01:25:52.359139+0000","last_active":"2024-04-25T01:40:49.256836+0000","last_peered":"2024-04-25T01:40:49.256836+0000","last_clean":"2024-04-25T01:40:49.256836+0000","last_became_active":"2024-04-25T01:25:52.358841+0000","last_became_peered":"2024-04-25T01:25:52.358841+0000","last_unstale":"2024-04-25T01:40:49.256836+0000","last_undegraded":"2024-04-25T01:40:49.256836+0000","last_fullsized":"2024-04-25T01:40:49.256836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"26'5828","reported_seq":5858,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.878112+0000","last_change":"2024-04-25T01:25:53.368739+0000","last_active":"2024-04-25T01:40:58.878112+0000","last_peered":"2024-04-25T01:40:58.878112+0000","last_clean":"2024-04-25T01:40:58.878112+0000","last_became_active":"2024-04-25T01:25:53.368470+0000","last_became_peered":"2024-04-25T01:25:53.368470+0000","last_unstale":"2024-04-25T01:40:58.878112+0000","last_undegraded":"2024-04-25T01:40:58.878112+0000","last_fullsized":"2024-04-25T01:40:58.878112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5828,"log_dups_size":0,"ondisk_log_size":5828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:29.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30463,"num_write_kb":367857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"26'65","reported_seq":89,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.150373+0000","last_change":"2024-04-25T01:26:01.407969+0000","last_active":"2024-04-25T01:41:09.150373+0000","last_peered":"2024-04-25T01:41:09.150373+0000","last_clean":"2024-04-25T01:41:09.150373+0000","last_became_active":"2024-04-25T01:26:01.407655+0000","last_became_peered":"2024-04-25T01:26:01.407655+0000","last_unstale":"2024-04-25T01:41:09.150373+0000","last_undegraded":"2024-04-25T01:41:09.150373+0000","last_fullsized":"2024-04-25T01:41:09.150373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:58.485864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":738477,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61,"num_write_kb":764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"26'4794","reported_seq":4817,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.329945+0000","last_change":"2024-04-25T01:26:02.417776+0000","last_active":"2024-04-25T01:41:09.329945+0000","last_peered":"2024-04-25T01:41:09.329945+0000","last_clean":"2024-04-25T01:41:09.329945+0000","last_became_active":"2024-04-25T01:26:02.417524+0000","last_became_peered":"2024-04-25T01:26:02.417524+0000","last_unstale":"2024-04-25T01:41:09.329945+0000","last_undegraded":"2024-04-25T01:41:09.329945+0000","last_fullsized":"2024-04-25T01:41:09.329945+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4794,"log_dups_size":0,"ondisk_log_size":4794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24624,"num_write_kb":311668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"26'4532","reported_seq":4563,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.725104+0000","last_change":"2024-04-25T01:25:53.372385+0000","last_active":"2024-04-25T01:40:58.725104+0000","last_peered":"2024-04-25T01:40:58.725104+0000","last_clean":"2024-04-25T01:40:58.725104+0000","last_became_active":"2024-04-25T01:25:53.372064+0000","last_became_peered":"2024-04-25T01:25:53.372064+0000","last_unstale":"2024-04-25T01:40:58.725104+0000","last_undegraded":"2024-04-25T01:40:58.725104+0000","last_fullsized":"2024-04-25T01:40:58.725104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4532,"log_dups_size":0,"ondisk_log_size":4532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:39.801619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23642,"num_write_kb":279226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"26'1100","reported_seq":1132,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.233609+0000","last_change":"2024-04-25T01:25:52.358616+0000","last_active":"2024-04-25T01:40:49.233609+0000","last_peered":"2024-04-25T01:40:49.233609+0000","last_clean":"2024-04-25T01:40:49.233609+0000","last_became_active":"2024-04-25T01:25:52.358318+0000","last_became_peered":"2024-04-25T01:25:52.358318+0000","last_unstale":"2024-04-25T01:40:49.233609+0000","last_undegraded":"2024-04-25T01:40:49.233609+0000","last_fullsized":"2024-04-25T01:40:49.233609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1100,"log_dups_size":0,"ondisk_log_size":1100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:50.232690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1091,"num_write_kb":4419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"26'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:39:31.750022+0000","last_change":"2024-04-25T01:26:01.400599+0000","last_active":"2024-04-25T01:39:31.750022+0000","last_peered":"2024-04-25T01:39:31.750022+0000","last_clean":"2024-04-25T01:39:31.750022+0000","last_became_active":"2024-04-25T01:26:01.400266+0000","last_became_peered":"2024-04-25T01:26:01.400266+0000","last_unstale":"2024-04-25T01:39:31.750022+0000","last_undegraded":"2024-04-25T01:39:31.750022+0000","last_fullsized":"2024-04-25T01:39:31.750022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"26'4554","reported_seq":4577,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.270137+0000","last_change":"2024-04-25T01:26:02.420269+0000","last_active":"2024-04-25T01:41:09.270137+0000","last_peered":"2024-04-25T01:41:09.270137+0000","last_clean":"2024-04-25T01:41:09.270137+0000","last_became_active":"2024-04-25T01:26:02.420001+0000","last_became_peered":"2024-04-25T01:26:02.420001+0000","last_unstale":"2024-04-25T01:41:09.270137+0000","last_undegraded":"2024-04-25T01:41:09.270137+0000","last_fullsized":"2024-04-25T01:41:09.270137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4554,"log_dups_size":0,"ondisk_log_size":4554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23431,"num_write_kb":298735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"26'3793","reported_seq":3824,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.800911+0000","last_change":"2024-04-25T01:25:53.361074+0000","last_active":"2024-04-25T01:40:58.800911+0000","last_peered":"2024-04-25T01:40:58.800911+0000","last_clean":"2024-04-25T01:40:58.800911+0000","last_became_active":"2024-04-25T01:25:53.360802+0000","last_became_peered":"2024-04-25T01:25:53.360802+0000","last_unstale":"2024-04-25T01:40:58.800911+0000","last_undegraded":"2024-04-25T01:40:58.800911+0000","last_fullsized":"2024-04-25T01:40:58.800911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3793,"log_dups_size":0,"ondisk_log_size":3793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19299,"num_write_kb":249413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"26'3","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:10.674718+0000","last_change":"2024-04-25T01:25:52.348841+0000","last_active":"2024-04-25T01:41:10.674718+0000","last_peered":"2024-04-25T01:41:10.674718+0000","last_clean":"2024-04-25T01:41:10.674718+0000","last_became_active":"2024-04-25T01:25:52.348550+0000","last_became_peered":"2024-04-25T01:25:52.348550+0000","last_unstale":"2024-04-25T01:41:10.674718+0000","last_undegraded":"2024-04-25T01:41:10.674718+0000","last_fullsized":"2024-04-25T01:41:10.674718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"26'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:36.992419+0000","last_change":"2024-04-25T01:26:01.408529+0000","last_active":"2024-04-25T01:40:36.992419+0000","last_peered":"2024-04-25T01:40:36.992419+0000","last_clean":"2024-04-25T01:40:36.992419+0000","last_became_active":"2024-04-25T01:26:01.407738+0000","last_became_peered":"2024-04-25T01:26:01.407738+0000","last_unstale":"2024-04-25T01:40:36.992419+0000","last_undegraded":"2024-04-25T01:40:36.992419+0000","last_fullsized":"2024-04-25T01:40:36.992419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:37.142255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"26'3863","reported_seq":3885,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.299717+0000","last_change":"2024-04-25T01:26:02.417731+0000","last_active":"2024-04-25T01:41:09.299717+0000","last_peered":"2024-04-25T01:41:09.299717+0000","last_clean":"2024-04-25T01:41:09.299717+0000","last_became_active":"2024-04-25T01:26:02.417438+0000","last_became_peered":"2024-04-25T01:26:02.417438+0000","last_unstale":"2024-04-25T01:41:09.299717+0000","last_undegraded":"2024-04-25T01:41:09.299717+0000","last_fullsized":"2024-04-25T01:41:09.299717+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3863,"log_dups_size":0,"ondisk_log_size":3863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19753,"num_write_kb":275789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"26'4400","reported_seq":4432,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:05.700076+0000","last_change":"2024-04-25T01:25:53.361790+0000","last_active":"2024-04-25T01:41:05.700076+0000","last_peered":"2024-04-25T01:41:05.700076+0000","last_clean":"2024-04-25T01:41:05.700076+0000","last_became_active":"2024-04-25T01:25:53.361542+0000","last_became_peered":"2024-04-25T01:25:53.361542+0000","last_unstale":"2024-04-25T01:41:05.700076+0000","last_undegraded":"2024-04-25T01:41:05.700076+0000","last_fullsized":"2024-04-25T01:41:05.700076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4400,"log_dups_size":0,"ondisk_log_size":4400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22906,"num_write_kb":296647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"26'3","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:05.674081+0000","last_change":"2024-04-25T01:25:52.353926+0000","last_active":"2024-04-25T01:41:05.674081+0000","last_peered":"2024-04-25T01:41:05.674081+0000","last_clean":"2024-04-25T01:41:05.674081+0000","last_became_active":"2024-04-25T01:25:52.353656+0000","last_became_peered":"2024-04-25T01:25:52.353656+0000","last_unstale":"2024-04-25T01:41:05.674081+0000","last_undegraded":"2024-04-25T01:41:05.674081+0000","last_fullsized":"2024-04-25T01:41:05.674081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"26'5","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.204130+0000","last_change":"2024-04-25T01:26:01.418556+0000","last_active":"2024-04-25T01:40:37.204130+0000","last_peered":"2024-04-25T01:40:37.204130+0000","last_clean":"2024-04-25T01:40:37.204130+0000","last_became_active":"2024-04-25T01:26:01.418263+0000","last_became_peered":"2024-04-25T01:26:01.418263+0000","last_unstale":"2024-04-25T01:40:37.204130+0000","last_undegraded":"2024-04-25T01:40:37.204130+0000","last_fullsized":"2024-04-25T01:40:37.204130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"26'5570","reported_seq":5592,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.349577+0000","last_change":"2024-04-25T01:26:02.413429+0000","last_active":"2024-04-25T01:41:09.349577+0000","last_peered":"2024-04-25T01:41:09.349577+0000","last_clean":"2024-04-25T01:41:09.349577+0000","last_became_active":"2024-04-25T01:26:02.412632+0000","last_became_peered":"2024-04-25T01:26:02.412632+0000","last_unstale":"2024-04-25T01:41:09.349577+0000","last_undegraded":"2024-04-25T01:41:09.349577+0000","last_fullsized":"2024-04-25T01:41:09.349577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5570,"log_dups_size":0,"ondisk_log_size":5570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:10.049310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27971,"num_write_kb":349705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"26'4715","reported_seq":4746,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.811959+0000","last_change":"2024-04-25T01:25:53.361653+0000","last_active":"2024-04-25T01:40:58.811959+0000","last_peered":"2024-04-25T01:40:58.811959+0000","last_clean":"2024-04-25T01:40:58.811959+0000","last_became_active":"2024-04-25T01:25:53.361265+0000","last_became_peered":"2024-04-25T01:25:53.361265+0000","last_unstale":"2024-04-25T01:40:58.811959+0000","last_undegraded":"2024-04-25T01:40:58.811959+0000","last_fullsized":"2024-04-25T01:40:58.811959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4715,"log_dups_size":0,"ondisk_log_size":4715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23886,"num_write_kb":288267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'3","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.248825+0000","last_change":"2024-04-25T01:25:52.353836+0000","last_active":"2024-04-25T01:40:49.248825+0000","last_peered":"2024-04-25T01:40:49.248825+0000","last_clean":"2024-04-25T01:40:49.248825+0000","last_became_active":"2024-04-25T01:25:52.353552+0000","last_became_peered":"2024-04-25T01:25:52.353552+0000","last_unstale":"2024-04-25T01:40:49.248825+0000","last_undegraded":"2024-04-25T01:40:49.248825+0000","last_fullsized":"2024-04-25T01:40:49.248825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"26'6","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.134073+0000","last_change":"2024-04-25T01:26:01.411323+0000","last_active":"2024-04-25T01:40:37.134073+0000","last_peered":"2024-04-25T01:40:37.134073+0000","last_clean":"2024-04-25T01:40:37.134073+0000","last_became_active":"2024-04-25T01:26:01.410378+0000","last_became_peered":"2024-04-25T01:26:01.410378+0000","last_unstale":"2024-04-25T01:40:37.134073+0000","last_undegraded":"2024-04-25T01:40:37.134073+0000","last_fullsized":"2024-04-25T01:40:37.134073+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"26'4403","reported_seq":4427,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.252333+0000","last_change":"2024-04-25T01:26:02.416646+0000","last_active":"2024-04-25T01:41:09.252333+0000","last_peered":"2024-04-25T01:41:09.252333+0000","last_clean":"2024-04-25T01:41:09.252333+0000","last_became_active":"2024-04-25T01:26:02.416395+0000","last_became_peered":"2024-04-25T01:26:02.416395+0000","last_unstale":"2024-04-25T01:41:09.252333+0000","last_undegraded":"2024-04-25T01:41:09.252333+0000","last_fullsized":"2024-04-25T01:41:09.252333+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4403,"log_dups_size":0,"ondisk_log_size":4403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23642,"num_write_kb":287389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"26'3766","reported_seq":3797,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.767490+0000","last_change":"2024-04-25T01:25:53.370111+0000","last_active":"2024-04-25T01:40:58.767490+0000","last_peered":"2024-04-25T01:40:58.767490+0000","last_clean":"2024-04-25T01:40:58.767490+0000","last_became_active":"2024-04-25T01:25:53.369837+0000","last_became_peered":"2024-04-25T01:25:53.369837+0000","last_unstale":"2024-04-25T01:40:58.767490+0000","last_undegraded":"2024-04-25T01:40:58.767490+0000","last_fullsized":"2024-04-25T01:40:58.767490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3766,"log_dups_size":0,"ondisk_log_size":3766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19293,"num_write_kb":249241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"26'5","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.300730+0000","last_change":"2024-04-25T01:25:52.353269+0000","last_active":"2024-04-25T01:40:49.300730+0000","last_peered":"2024-04-25T01:40:49.300730+0000","last_clean":"2024-04-25T01:40:49.300730+0000","last_became_active":"2024-04-25T01:25:52.352991+0000","last_became_peered":"2024-04-25T01:25:52.352991+0000","last_unstale":"2024-04-25T01:40:49.300730+0000","last_undegraded":"2024-04-25T01:40:49.300730+0000","last_fullsized":"2024-04-25T01:40:49.300730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"26'41","reported_seq":105,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:47.845985+0000","last_change":"2024-04-25T01:26:01.410360+0000","last_active":"2024-04-25T01:40:47.845985+0000","last_peered":"2024-04-25T01:40:47.845985+0000","last_clean":"2024-04-25T01:40:47.845985+0000","last_became_active":"2024-04-25T01:26:01.410076+0000","last_became_peered":"2024-04-25T01:26:01.410076+0000","last_unstale":"2024-04-25T01:40:47.845985+0000","last_undegraded":"2024-04-25T01:40:47.845985+0000","last_fullsized":"2024-04-25T01:40:47.845985+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:52.561251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":112,"num_write":39,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"26'4289","reported_seq":4312,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.299549+0000","last_change":"2024-04-25T01:26:02.422182+0000","last_active":"2024-04-25T01:41:09.299549+0000","last_peered":"2024-04-25T01:41:09.299549+0000","last_clean":"2024-04-25T01:41:09.299549+0000","last_became_active":"2024-04-25T01:26:02.421885+0000","last_became_peered":"2024-04-25T01:26:02.421885+0000","last_unstale":"2024-04-25T01:41:09.299549+0000","last_undegraded":"2024-04-25T01:41:09.299549+0000","last_fullsized":"2024-04-25T01:41:09.299549+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4289,"log_dups_size":0,"ondisk_log_size":4289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:03.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22172,"num_write_kb":285532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"26'4466","reported_seq":4498,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:05.700191+0000","last_change":"2024-04-25T01:25:53.364967+0000","last_active":"2024-04-25T01:41:05.700191+0000","last_peered":"2024-04-25T01:41:05.700191+0000","last_clean":"2024-04-25T01:41:05.700191+0000","last_became_active":"2024-04-25T01:25:53.364694+0000","last_became_peered":"2024-04-25T01:25:53.364694+0000","last_unstale":"2024-04-25T01:41:05.700191+0000","last_undegraded":"2024-04-25T01:41:05.700191+0000","last_fullsized":"2024-04-25T01:41:05.700191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4466,"log_dups_size":0,"ondisk_log_size":4466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22859,"num_write_kb":292275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'571","reported_seq":605,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:05.674174+0000","last_change":"2024-04-25T01:25:52.355395+0000","last_active":"2024-04-25T01:41:05.674174+0000","last_peered":"2024-04-25T01:41:05.674174+0000","last_clean":"2024-04-25T01:41:05.674174+0000","last_became_active":"2024-04-25T01:25:52.355137+0000","last_became_peered":"2024-04-25T01:25:52.355137+0000","last_unstale":"2024-04-25T01:41:05.674174+0000","last_undegraded":"2024-04-25T01:41:05.674174+0000","last_fullsized":"2024-04-25T01:41:05.674174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"26'5","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.186876+0000","last_change":"2024-04-25T01:26:01.416108+0000","last_active":"2024-04-25T01:40:37.186876+0000","last_peered":"2024-04-25T01:40:37.186876+0000","last_clean":"2024-04-25T01:40:37.186876+0000","last_became_active":"2024-04-25T01:26:01.415824+0000","last_became_peered":"2024-04-25T01:26:01.415824+0000","last_unstale":"2024-04-25T01:40:37.186876+0000","last_undegraded":"2024-04-25T01:40:37.186876+0000","last_fullsized":"2024-04-25T01:40:37.186876+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"26'4989","reported_seq":5012,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.273942+0000","last_change":"2024-04-25T01:26:02.416208+0000","last_active":"2024-04-25T01:41:09.273942+0000","last_peered":"2024-04-25T01:41:09.273942+0000","last_clean":"2024-04-25T01:41:09.273942+0000","last_became_active":"2024-04-25T01:26:02.415949+0000","last_became_peered":"2024-04-25T01:26:02.415949+0000","last_unstale":"2024-04-25T01:41:09.273942+0000","last_undegraded":"2024-04-25T01:41:09.273942+0000","last_fullsized":"2024-04-25T01:41:09.273942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4989,"log_dups_size":0,"ondisk_log_size":4989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:21.097225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25442,"num_write_kb":314904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"26'4796","reported_seq":4828,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.753139+0000","last_change":"2024-04-25T01:25:53.366944+0000","last_active":"2024-04-25T01:40:58.753139+0000","last_peered":"2024-04-25T01:40:58.753139+0000","last_clean":"2024-04-25T01:40:58.753139+0000","last_became_active":"2024-04-25T01:25:53.366693+0000","last_became_peered":"2024-04-25T01:25:53.366693+0000","last_unstale":"2024-04-25T01:40:58.753139+0000","last_undegraded":"2024-04-25T01:40:58.753139+0000","last_fullsized":"2024-04-25T01:40:58.753139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25026,"num_write_kb":309186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"26'4","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.236233+0000","last_change":"2024-04-25T01:25:52.356842+0000","last_active":"2024-04-25T01:40:49.236233+0000","last_peered":"2024-04-25T01:40:49.236233+0000","last_clean":"2024-04-25T01:40:49.236233+0000","last_became_active":"2024-04-25T01:25:52.356396+0000","last_became_peered":"2024-04-25T01:25:52.356396+0000","last_unstale":"2024-04-25T01:40:49.236233+0000","last_undegraded":"2024-04-25T01:40:49.236233+0000","last_fullsized":"2024-04-25T01:40:49.236233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:25.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"26'1","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:38:30.810691+0000","last_change":"2024-04-25T01:26:01.407934+0000","last_active":"2024-04-25T01:38:30.810691+0000","last_peered":"2024-04-25T01:38:30.810691+0000","last_clean":"2024-04-25T01:38:30.810691+0000","last_became_active":"2024-04-25T01:26:01.407659+0000","last_became_peered":"2024-04-25T01:26:01.407659+0000","last_unstale":"2024-04-25T01:38:30.810691+0000","last_undegraded":"2024-04-25T01:38:30.810691+0000","last_fullsized":"2024-04-25T01:38:30.810691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:19.629335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"26'4196","reported_seq":4219,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.266753+0000","last_change":"2024-04-25T01:26:02.418087+0000","last_active":"2024-04-25T01:41:09.266753+0000","last_peered":"2024-04-25T01:41:09.266753+0000","last_clean":"2024-04-25T01:41:09.266753+0000","last_became_active":"2024-04-25T01:26:02.417622+0000","last_became_peered":"2024-04-25T01:26:02.417622+0000","last_unstale":"2024-04-25T01:41:09.266753+0000","last_undegraded":"2024-04-25T01:41:09.266753+0000","last_fullsized":"2024-04-25T01:41:09.266753+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4196,"log_dups_size":0,"ondisk_log_size":4196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:03.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22617,"num_write_kb":283190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"26'4472","reported_seq":4503,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.865649+0000","last_change":"2024-04-25T01:25:53.368164+0000","last_active":"2024-04-25T01:40:58.865649+0000","last_peered":"2024-04-25T01:40:58.865649+0000","last_clean":"2024-04-25T01:40:58.865649+0000","last_became_active":"2024-04-25T01:25:53.367854+0000","last_became_peered":"2024-04-25T01:25:53.367854+0000","last_unstale":"2024-04-25T01:40:58.865649+0000","last_undegraded":"2024-04-25T01:40:58.865649+0000","last_fullsized":"2024-04-25T01:40:58.865649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4472,"log_dups_size":0,"ondisk_log_size":4472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23917,"num_write_kb":276200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'5","reported_seq":39,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.244312+0000","last_change":"2024-04-25T01:25:52.350128+0000","last_active":"2024-04-25T01:40:49.244312+0000","last_peered":"2024-04-25T01:40:49.244312+0000","last_clean":"2024-04-25T01:40:49.244312+0000","last_became_active":"2024-04-25T01:25:52.349874+0000","last_became_peered":"2024-04-25T01:25:52.349874+0000","last_unstale":"2024-04-25T01:40:49.244312+0000","last_undegraded":"2024-04-25T01:40:49.244312+0000","last_fullsized":"2024-04-25T01:40:49.244312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"26'5","reported_seq":39,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:20.631227+0000","last_change":"2024-04-25T01:41:20.631248+0000","last_active":"2024-04-25T01:41:20.631227+0000","last_peered":"2024-04-25T01:41:20.631227+0000","last_clean":"2024-04-25T01:41:20.631227+0000","last_became_active":"2024-04-25T01:26:01.417820+0000","last_became_peered":"2024-04-25T01:26:01.417820+0000","last_unstale":"2024-04-25T01:41:20.631227+0000","last_undegraded":"2024-04-25T01:41:20.631227+0000","last_fullsized":"2024-04-25T01:41:20.631227+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-04-25T01:41:20.631162+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-04-25T01:41:20.631162+0000","last_clean_scrub_stamp":"2024-04-25T01:41:20.631162+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:56.688056+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"26'4093","reported_seq":4116,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.276141+0000","last_change":"2024-04-25T01:26:02.422761+0000","last_active":"2024-04-25T01:41:09.276141+0000","last_peered":"2024-04-25T01:41:09.276141+0000","last_clean":"2024-04-25T01:41:09.276141+0000","last_became_active":"2024-04-25T01:26:02.422503+0000","last_became_peered":"2024-04-25T01:26:02.422503+0000","last_unstale":"2024-04-25T01:41:09.276141+0000","last_undegraded":"2024-04-25T01:41:09.276141+0000","last_fullsized":"2024-04-25T01:41:09.276141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4093,"log_dups_size":0,"ondisk_log_size":4093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:55.040946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21550,"num_write_kb":266696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"26'4812","reported_seq":4843,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.791228+0000","last_change":"2024-04-25T01:25:53.361696+0000","last_active":"2024-04-25T01:40:58.791228+0000","last_peered":"2024-04-25T01:40:58.791228+0000","last_clean":"2024-04-25T01:40:58.791228+0000","last_became_active":"2024-04-25T01:25:53.361156+0000","last_became_peered":"2024-04-25T01:25:53.361156+0000","last_unstale":"2024-04-25T01:40:58.791228+0000","last_undegraded":"2024-04-25T01:40:58.791228+0000","last_fullsized":"2024-04-25T01:40:58.791228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4812,"log_dups_size":0,"ondisk_log_size":4812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:27.472315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25777,"num_write_kb":295962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"26'5","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.233531+0000","last_change":"2024-04-25T01:25:52.354499+0000","last_active":"2024-04-25T01:40:49.233531+0000","last_peered":"2024-04-25T01:40:49.233531+0000","last_clean":"2024-04-25T01:40:49.233531+0000","last_became_active":"2024-04-25T01:25:52.354193+0000","last_became_peered":"2024-04-25T01:25:52.354193+0000","last_unstale":"2024-04-25T01:40:49.233531+0000","last_undegraded":"2024-04-25T01:40:49.233531+0000","last_fullsized":"2024-04-25T01:40:49.233531+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"26'5","reported_seq":39,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:19.652416+0000","last_change":"2024-04-25T01:41:19.652438+0000","last_active":"2024-04-25T01:41:19.652416+0000","last_peered":"2024-04-25T01:41:19.652416+0000","last_clean":"2024-04-25T01:41:19.652416+0000","last_became_active":"2024-04-25T01:26:01.417805+0000","last_became_peered":"2024-04-25T01:26:01.417805+0000","last_unstale":"2024-04-25T01:41:19.652416+0000","last_undegraded":"2024-04-25T01:41:19.652416+0000","last_fullsized":"2024-04-25T01:41:19.652416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-04-25T01:41:19.652334+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-04-25T01:41:19.652334+0000","last_clean_scrub_stamp":"2024-04-25T01:41:19.652334+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:54.494631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"26'3951","reported_seq":3975,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:15.678171+0000","last_change":"2024-04-25T01:26:02.415877+0000","last_active":"2024-04-25T01:41:15.678171+0000","last_peered":"2024-04-25T01:41:15.678171+0000","last_clean":"2024-04-25T01:41:15.678171+0000","last_became_active":"2024-04-25T01:26:02.415619+0000","last_became_peered":"2024-04-25T01:26:02.415619+0000","last_unstale":"2024-04-25T01:41:15.678171+0000","last_undegraded":"2024-04-25T01:41:15.678171+0000","last_fullsized":"2024-04-25T01:41:15.678171+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3951,"log_dups_size":0,"ondisk_log_size":3951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20033,"num_write_kb":252185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"26'4985","reported_seq":5017,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.752328+0000","last_change":"2024-04-25T01:25:53.362922+0000","last_active":"2024-04-25T01:40:58.752328+0000","last_peered":"2024-04-25T01:40:58.752328+0000","last_clean":"2024-04-25T01:40:58.752328+0000","last_became_active":"2024-04-25T01:25:53.362533+0000","last_became_peered":"2024-04-25T01:25:53.362533+0000","last_unstale":"2024-04-25T01:40:58.752328+0000","last_undegraded":"2024-04-25T01:40:58.752328+0000","last_fullsized":"2024-04-25T01:40:58.752328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4985,"log_dups_size":0,"ondisk_log_size":4985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26332,"num_write_kb":298165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"26'43","reported_seq":118,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.254515+0000","last_change":"2024-04-25T01:25:52.345321+0000","last_active":"2024-04-25T01:40:49.254515+0000","last_peered":"2024-04-25T01:40:49.254515+0000","last_clean":"2024-04-25T01:40:49.254515+0000","last_became_active":"2024-04-25T01:25:52.345061+0000","last_became_peered":"2024-04-25T01:25:52.345061+0000","last_unstale":"2024-04-25T01:40:49.254515+0000","last_undegraded":"2024-04-25T01:40:49.254515+0000","last_fullsized":"2024-04-25T01:40:49.254515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":41,"num_read_kb":115,"num_write":41,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"26'4","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:10.674768+0000","last_change":"2024-04-25T01:26:01.418516+0000","last_active":"2024-04-25T01:41:10.674768+0000","last_peered":"2024-04-25T01:41:10.674768+0000","last_clean":"2024-04-25T01:41:10.674768+0000","last_became_active":"2024-04-25T01:26:01.418173+0000","last_became_peered":"2024-04-25T01:26:01.418173+0000","last_unstale":"2024-04-25T01:41:10.674768+0000","last_undegraded":"2024-04-25T01:41:10.674768+0000","last_fullsized":"2024-04-25T01:41:10.674768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"26'4134","reported_seq":4157,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.335118+0000","last_change":"2024-04-25T01:26:02.418115+0000","last_active":"2024-04-25T01:41:09.335118+0000","last_peered":"2024-04-25T01:41:09.335118+0000","last_clean":"2024-04-25T01:41:09.335118+0000","last_became_active":"2024-04-25T01:26:02.417355+0000","last_became_peered":"2024-04-25T01:26:02.417355+0000","last_unstale":"2024-04-25T01:41:09.335118+0000","last_undegraded":"2024-04-25T01:41:09.335118+0000","last_fullsized":"2024-04-25T01:41:09.335118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4134,"log_dups_size":0,"ondisk_log_size":4134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:19.644468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21715,"num_write_kb":287743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"26'4344","reported_seq":4375,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.752413+0000","last_change":"2024-04-25T01:25:53.355950+0000","last_active":"2024-04-25T01:40:58.752413+0000","last_peered":"2024-04-25T01:40:58.752413+0000","last_clean":"2024-04-25T01:40:58.752413+0000","last_became_active":"2024-04-25T01:25:53.355648+0000","last_became_peered":"2024-04-25T01:25:53.355648+0000","last_unstale":"2024-04-25T01:40:58.752413+0000","last_undegraded":"2024-04-25T01:40:58.752413+0000","last_fullsized":"2024-04-25T01:40:58.752413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4344,"log_dups_size":0,"ondisk_log_size":4344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:11.336051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23132,"num_write_kb":289328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"26'5","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.249850+0000","last_change":"2024-04-25T01:25:52.357683+0000","last_active":"2024-04-25T01:40:49.249850+0000","last_peered":"2024-04-25T01:40:49.249850+0000","last_clean":"2024-04-25T01:40:49.249850+0000","last_became_active":"2024-04-25T01:25:52.356924+0000","last_became_peered":"2024-04-25T01:25:52.356924+0000","last_unstale":"2024-04-25T01:40:49.249850+0000","last_undegraded":"2024-04-25T01:40:49.249850+0000","last_fullsized":"2024-04-25T01:40:49.249850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"26'570","reported_seq":595,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:47.577546+0000","last_change":"2024-04-25T01:26:01.421260+0000","last_active":"2024-04-25T01:40:47.577546+0000","last_peered":"2024-04-25T01:40:47.577546+0000","last_clean":"2024-04-25T01:40:47.577546+0000","last_became_active":"2024-04-25T01:26:01.420831+0000","last_became_peered":"2024-04-25T01:26:01.420831+0000","last_unstale":"2024-04-25T01:40:47.577546+0000","last_undegraded":"2024-04-25T01:40:47.577546+0000","last_fullsized":"2024-04-25T01:40:47.577546+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1330,"num_write_kb":427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"26'4947","reported_seq":4970,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.324019+0000","last_change":"2024-04-25T01:26:02.421386+0000","last_active":"2024-04-25T01:41:09.324019+0000","last_peered":"2024-04-25T01:41:09.324019+0000","last_clean":"2024-04-25T01:41:09.324019+0000","last_became_active":"2024-04-25T01:26:02.421079+0000","last_became_peered":"2024-04-25T01:26:02.421079+0000","last_unstale":"2024-04-25T01:41:09.324019+0000","last_undegraded":"2024-04-25T01:41:09.324019+0000","last_fullsized":"2024-04-25T01:41:09.324019+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4947,"log_dups_size":0,"ondisk_log_size":4947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:56.483216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25964,"num_write_kb":313022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"26'4754","reported_seq":4771,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.812932+0000","last_change":"2024-04-25T01:26:52.515572+0000","last_active":"2024-04-25T01:40:58.812932+0000","last_peered":"2024-04-25T01:40:58.812932+0000","last_clean":"2024-04-25T01:40:58.812932+0000","last_became_active":"2024-04-25T01:26:52.515254+0000","last_became_peered":"2024-04-25T01:26:52.515254+0000","last_unstale":"2024-04-25T01:40:58.812932+0000","last_undegraded":"2024-04-25T01:40:58.812932+0000","last_fullsized":"2024-04-25T01:40:58.812932+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4754,"log_dups_size":0,"ondisk_log_size":4754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24235,"num_write_kb":310192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:38:25.919823+0000","last_change":"2024-04-25T01:25:52.353155+0000","last_active":"2024-04-25T01:38:25.919823+0000","last_peered":"2024-04-25T01:38:25.919823+0000","last_clean":"2024-04-25T01:38:25.919823+0000","last_became_active":"2024-04-25T01:25:52.352826+0000","last_became_peered":"2024-04-25T01:25:52.352826+0000","last_unstale":"2024-04-25T01:38:25.919823+0000","last_undegraded":"2024-04-25T01:38:25.919823+0000","last_fullsized":"2024-04-25T01:38:25.919823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:33.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":189,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:37:45.823358+0000","last_change":"2024-04-25T01:25:41.719670+0000","last_active":"2024-04-25T01:37:45.823358+0000","last_peered":"2024-04-25T01:37:45.823358+0000","last_clean":"2024-04-25T01:37:45.823358+0000","last_became_active":"2024-04-25T01:25:41.718721+0000","last_became_peered":"2024-04-25T01:25:41.718721+0000","last_unstale":"2024-04-25T01:37:45.823358+0000","last_undegraded":"2024-04-25T01:37:45.823358+0000","last_fullsized":"2024-04-25T01:37:45.823358+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:40.688097+0000","last_clean_scrub_stamp":"2024-04-25T01:25:40.688097+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:27.456746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"26'4","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.023240+0000","last_change":"2024-04-25T01:26:01.409339+0000","last_active":"2024-04-25T01:40:37.023240+0000","last_peered":"2024-04-25T01:40:37.023240+0000","last_clean":"2024-04-25T01:40:37.023240+0000","last_became_active":"2024-04-25T01:26:01.409062+0000","last_became_peered":"2024-04-25T01:26:01.409062+0000","last_unstale":"2024-04-25T01:40:37.023240+0000","last_undegraded":"2024-04-25T01:40:37.023240+0000","last_fullsized":"2024-04-25T01:40:37.023240+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:52.561251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"26'3833","reported_seq":3856,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.307330+0000","last_change":"2024-04-25T01:26:02.418574+0000","last_active":"2024-04-25T01:41:09.307330+0000","last_peered":"2024-04-25T01:41:09.307330+0000","last_clean":"2024-04-25T01:41:09.307330+0000","last_became_active":"2024-04-25T01:26:02.416546+0000","last_became_peered":"2024-04-25T01:26:02.416546+0000","last_unstale":"2024-04-25T01:41:09.307330+0000","last_undegraded":"2024-04-25T01:41:09.307330+0000","last_fullsized":"2024-04-25T01:41:09.307330+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3833,"log_dups_size":0,"ondisk_log_size":3833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19504,"num_write_kb":250193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"26'5320","reported_seq":5350,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.811822+0000","last_change":"2024-04-25T01:25:53.375523+0000","last_active":"2024-04-25T01:40:58.811822+0000","last_peered":"2024-04-25T01:40:58.811822+0000","last_clean":"2024-04-25T01:40:58.811822+0000","last_became_active":"2024-04-25T01:25:53.375219+0000","last_became_peered":"2024-04-25T01:25:53.375219+0000","last_unstale":"2024-04-25T01:40:58.811822+0000","last_undegraded":"2024-04-25T01:40:58.811822+0000","last_fullsized":"2024-04-25T01:40:58.811822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5320,"log_dups_size":0,"ondisk_log_size":5320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27324,"num_write_kb":314375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'3","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.217319+0000","last_change":"2024-04-25T01:25:52.352994+0000","last_active":"2024-04-25T01:40:49.217319+0000","last_peered":"2024-04-25T01:40:49.217319+0000","last_clean":"2024-04-25T01:40:49.217319+0000","last_became_active":"2024-04-25T01:25:52.352644+0000","last_became_peered":"2024-04-25T01:25:52.352644+0000","last_unstale":"2024-04-25T01:40:49.217319+0000","last_undegraded":"2024-04-25T01:40:49.217319+0000","last_fullsized":"2024-04-25T01:40:49.217319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"26'3","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.140689+0000","last_change":"2024-04-25T01:26:01.420891+0000","last_active":"2024-04-25T01:40:37.140689+0000","last_peered":"2024-04-25T01:40:37.140689+0000","last_clean":"2024-04-25T01:40:37.140689+0000","last_became_active":"2024-04-25T01:26:01.420664+0000","last_became_peered":"2024-04-25T01:26:01.420664+0000","last_unstale":"2024-04-25T01:40:37.140689+0000","last_undegraded":"2024-04-25T01:40:37.140689+0000","last_fullsized":"2024-04-25T01:40:37.140689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"26'4000","reported_seq":4024,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:19.349945+0000","last_change":"2024-04-25T01:26:02.403489+0000","last_active":"2024-04-25T01:41:19.349945+0000","last_peered":"2024-04-25T01:41:19.349945+0000","last_clean":"2024-04-25T01:41:19.349945+0000","last_became_active":"2024-04-25T01:26:02.403206+0000","last_became_peered":"2024-04-25T01:26:02.403206+0000","last_unstale":"2024-04-25T01:41:19.349945+0000","last_undegraded":"2024-04-25T01:41:19.349945+0000","last_fullsized":"2024-04-25T01:41:19.349945+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4000,"log_dups_size":0,"ondisk_log_size":4000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21231,"num_write_kb":269526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"26'4796","reported_seq":4826,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.770361+0000","last_change":"2024-04-25T01:25:53.371018+0000","last_active":"2024-04-25T01:40:58.770361+0000","last_peered":"2024-04-25T01:40:58.770361+0000","last_clean":"2024-04-25T01:40:58.770361+0000","last_became_active":"2024-04-25T01:25:53.370539+0000","last_became_peered":"2024-04-25T01:25:53.370539+0000","last_unstale":"2024-04-25T01:40:58.770361+0000","last_undegraded":"2024-04-25T01:40:58.770361+0000","last_fullsized":"2024-04-25T01:40:58.770361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:34.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25182,"num_write_kb":313860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"26'2","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.218411+0000","last_change":"2024-04-25T01:25:52.355133+0000","last_active":"2024-04-25T01:40:49.218411+0000","last_peered":"2024-04-25T01:40:49.218411+0000","last_clean":"2024-04-25T01:40:49.218411+0000","last_became_active":"2024-04-25T01:25:52.354836+0000","last_became_peered":"2024-04-25T01:25:52.354836+0000","last_unstale":"2024-04-25T01:40:49.218411+0000","last_undegraded":"2024-04-25T01:40:49.218411+0000","last_fullsized":"2024-04-25T01:40:49.218411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"26'1112","reported_seq":1136,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.176654+0000","last_change":"2024-04-25T01:26:01.419285+0000","last_active":"2024-04-25T01:40:37.176654+0000","last_peered":"2024-04-25T01:40:37.176654+0000","last_clean":"2024-04-25T01:40:37.176654+0000","last_became_active":"2024-04-25T01:26:01.418987+0000","last_became_peered":"2024-04-25T01:26:01.418987+0000","last_unstale":"2024-04-25T01:40:37.176654+0000","last_undegraded":"2024-04-25T01:40:37.176654+0000","last_fullsized":"2024-04-25T01:40:37.176654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":1112,"log_dups_size":0,"ondisk_log_size":1112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:09.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1103,"num_write_kb":4430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"26'4082","reported_seq":4105,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.285308+0000","last_change":"2024-04-25T01:26:02.416544+0000","last_active":"2024-04-25T01:41:09.285308+0000","last_peered":"2024-04-25T01:41:09.285308+0000","last_clean":"2024-04-25T01:41:09.285308+0000","last_became_active":"2024-04-25T01:26:02.416189+0000","last_became_peered":"2024-04-25T01:26:02.416189+0000","last_unstale":"2024-04-25T01:41:09.285308+0000","last_undegraded":"2024-04-25T01:41:09.285308+0000","last_fullsized":"2024-04-25T01:41:09.285308+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4082,"log_dups_size":0,"ondisk_log_size":4082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.688469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20562,"num_write_kb":274915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"26'5666","reported_seq":5696,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.889157+0000","last_change":"2024-04-25T01:25:53.370754+0000","last_active":"2024-04-25T01:40:58.889157+0000","last_peered":"2024-04-25T01:40:58.889157+0000","last_clean":"2024-04-25T01:40:58.889157+0000","last_became_active":"2024-04-25T01:25:53.370074+0000","last_became_peered":"2024-04-25T01:25:53.370074+0000","last_unstale":"2024-04-25T01:40:58.889157+0000","last_undegraded":"2024-04-25T01:40:58.889157+0000","last_fullsized":"2024-04-25T01:40:58.889157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5666,"log_dups_size":0,"ondisk_log_size":5666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:08.185526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29161,"num_write_kb":354533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"26'6","reported_seq":39,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.247918+0000","last_change":"2024-04-25T01:25:52.345943+0000","last_active":"2024-04-25T01:40:49.247918+0000","last_peered":"2024-04-25T01:40:49.247918+0000","last_clean":"2024-04-25T01:40:49.247918+0000","last_became_active":"2024-04-25T01:25:52.345717+0000","last_became_peered":"2024-04-25T01:25:52.345717+0000","last_unstale":"2024-04-25T01:40:49.247918+0000","last_undegraded":"2024-04-25T01:40:49.247918+0000","last_fullsized":"2024-04-25T01:40:49.247918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"26'3","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.011634+0000","last_change":"2024-04-25T01:26:01.416027+0000","last_active":"2024-04-25T01:40:37.011634+0000","last_peered":"2024-04-25T01:40:37.011634+0000","last_clean":"2024-04-25T01:40:37.011634+0000","last_became_active":"2024-04-25T01:26:01.415750+0000","last_became_peered":"2024-04-25T01:26:01.415750+0000","last_unstale":"2024-04-25T01:40:37.011634+0000","last_undegraded":"2024-04-25T01:40:37.011634+0000","last_fullsized":"2024-04-25T01:40:37.011634+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:27.042376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"26'4748","reported_seq":4780,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:20.027499+0000","last_change":"2024-04-25T01:41:20.027538+0000","last_active":"2024-04-25T01:41:20.027499+0000","last_peered":"2024-04-25T01:41:20.027499+0000","last_clean":"2024-04-25T01:41:20.027499+0000","last_became_active":"2024-04-25T01:26:02.414316+0000","last_became_peered":"2024-04-25T01:26:02.414316+0000","last_unstale":"2024-04-25T01:41:20.027499+0000","last_undegraded":"2024-04-25T01:41:20.027499+0000","last_fullsized":"2024-04-25T01:41:20.027499+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4748","last_scrub_stamp":"2024-04-25T01:41:20.027380+0000","last_deep_scrub":"26'4748","last_deep_scrub_stamp":"2024-04-25T01:41:20.027380+0000","last_clean_scrub_stamp":"2024-04-25T01:41:20.027380+0000","objects_scrubbed":0,"log_size":4748,"log_dups_size":0,"ondisk_log_size":4748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:32.660618+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24613,"num_write_kb":303491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"26'4546","reported_seq":4576,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.712213+0000","last_change":"2024-04-25T01:25:53.365171+0000","last_active":"2024-04-25T01:40:58.712213+0000","last_peered":"2024-04-25T01:40:58.712213+0000","last_clean":"2024-04-25T01:40:58.712213+0000","last_became_active":"2024-04-25T01:25:53.364049+0000","last_became_peered":"2024-04-25T01:25:53.364049+0000","last_unstale":"2024-04-25T01:40:58.712213+0000","last_undegraded":"2024-04-25T01:40:58.712213+0000","last_fullsized":"2024-04-25T01:40:58.712213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4546,"log_dups_size":0,"ondisk_log_size":4546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22740,"num_write_kb":287925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'5","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.243400+0000","last_change":"2024-04-25T01:25:52.350688+0000","last_active":"2024-04-25T01:40:49.243400+0000","last_peered":"2024-04-25T01:40:49.243400+0000","last_clean":"2024-04-25T01:40:49.243400+0000","last_became_active":"2024-04-25T01:25:52.350357+0000","last_became_peered":"2024-04-25T01:25:52.350357+0000","last_unstale":"2024-04-25T01:40:49.243400+0000","last_undegraded":"2024-04-25T01:40:49.243400+0000","last_fullsized":"2024-04-25T01:40:49.243400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:47.203936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"26'3","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.002855+0000","last_change":"2024-04-25T01:26:01.418580+0000","last_active":"2024-04-25T01:40:37.002855+0000","last_peered":"2024-04-25T01:40:37.002855+0000","last_clean":"2024-04-25T01:40:37.002855+0000","last_became_active":"2024-04-25T01:26:01.418179+0000","last_became_peered":"2024-04-25T01:26:01.418179+0000","last_unstale":"2024-04-25T01:40:37.002855+0000","last_undegraded":"2024-04-25T01:40:37.002855+0000","last_fullsized":"2024-04-25T01:40:37.002855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:42.987327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"26'3833","reported_seq":3856,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.301262+0000","last_change":"2024-04-25T01:26:02.418424+0000","last_active":"2024-04-25T01:41:09.301262+0000","last_peered":"2024-04-25T01:41:09.301262+0000","last_clean":"2024-04-25T01:41:09.301262+0000","last_became_active":"2024-04-25T01:26:02.418090+0000","last_became_peered":"2024-04-25T01:26:02.418090+0000","last_unstale":"2024-04-25T01:41:09.301262+0000","last_undegraded":"2024-04-25T01:41:09.301262+0000","last_fullsized":"2024-04-25T01:41:09.301262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3833,"log_dups_size":0,"ondisk_log_size":3833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19507,"num_write_kb":254331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"26'4627","reported_seq":4657,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.811217+0000","last_change":"2024-04-25T01:25:53.366376+0000","last_active":"2024-04-25T01:40:58.811217+0000","last_peered":"2024-04-25T01:40:58.811217+0000","last_clean":"2024-04-25T01:40:58.811217+0000","last_became_active":"2024-04-25T01:25:53.365715+0000","last_became_peered":"2024-04-25T01:25:53.365715+0000","last_unstale":"2024-04-25T01:40:58.811217+0000","last_undegraded":"2024-04-25T01:40:58.811217+0000","last_fullsized":"2024-04-25T01:40:58.811217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4627,"log_dups_size":0,"ondisk_log_size":4627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24780,"num_write_kb":300200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"26'11","reported_seq":44,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.234074+0000","last_change":"2024-04-25T01:25:52.347413+0000","last_active":"2024-04-25T01:40:49.234074+0000","last_peered":"2024-04-25T01:40:49.234074+0000","last_clean":"2024-04-25T01:40:49.234074+0000","last_became_active":"2024-04-25T01:25:52.347144+0000","last_became_peered":"2024-04-25T01:25:52.347144+0000","last_unstale":"2024-04-25T01:40:49.234074+0000","last_undegraded":"2024-04-25T01:40:49.234074+0000","last_fullsized":"2024-04-25T01:40:49.234074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"26'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.101937+0000","last_change":"2024-04-25T01:26:01.409917+0000","last_active":"2024-04-25T01:40:37.101937+0000","last_peered":"2024-04-25T01:40:37.101937+0000","last_clean":"2024-04-25T01:40:37.101937+0000","last_became_active":"2024-04-25T01:26:01.409654+0000","last_became_peered":"2024-04-25T01:26:01.409654+0000","last_unstale":"2024-04-25T01:40:37.101937+0000","last_undegraded":"2024-04-25T01:40:37.101937+0000","last_fullsized":"2024-04-25T01:40:37.101937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:37.142255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"26'4668","reported_seq":4690,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.342103+0000","last_change":"2024-04-25T01:26:02.425103+0000","last_active":"2024-04-25T01:41:09.342103+0000","last_peered":"2024-04-25T01:41:09.342103+0000","last_clean":"2024-04-25T01:41:09.342103+0000","last_became_active":"2024-04-25T01:26:02.424819+0000","last_became_peered":"2024-04-25T01:26:02.424819+0000","last_unstale":"2024-04-25T01:41:09.342103+0000","last_undegraded":"2024-04-25T01:41:09.342103+0000","last_fullsized":"2024-04-25T01:41:09.342103+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4668,"log_dups_size":0,"ondisk_log_size":4668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23987,"num_write_kb":301090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"26'5240","reported_seq":5270,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.796905+0000","last_change":"2024-04-25T01:25:53.364987+0000","last_active":"2024-04-25T01:40:58.796905+0000","last_peered":"2024-04-25T01:40:58.796905+0000","last_clean":"2024-04-25T01:40:58.796905+0000","last_became_active":"2024-04-25T01:25:53.364431+0000","last_became_peered":"2024-04-25T01:25:53.364431+0000","last_unstale":"2024-04-25T01:40:58.796905+0000","last_undegraded":"2024-04-25T01:40:58.796905+0000","last_fullsized":"2024-04-25T01:40:58.796905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5240,"log_dups_size":0,"ondisk_log_size":5240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:43.526751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27403,"num_write_kb":326994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'6","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.234588+0000","last_change":"2024-04-25T01:25:52.351205+0000","last_active":"2024-04-25T01:40:49.234588+0000","last_peered":"2024-04-25T01:40:49.234588+0000","last_clean":"2024-04-25T01:40:49.234588+0000","last_became_active":"2024-04-25T01:25:52.350946+0000","last_became_peered":"2024-04-25T01:25:52.350946+0000","last_unstale":"2024-04-25T01:40:49.234588+0000","last_undegraded":"2024-04-25T01:40:49.234588+0000","last_fullsized":"2024-04-25T01:40:49.234588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"26'2","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.137238+0000","last_change":"2024-04-25T01:26:01.415799+0000","last_active":"2024-04-25T01:40:37.137238+0000","last_peered":"2024-04-25T01:40:37.137238+0000","last_clean":"2024-04-25T01:40:37.137238+0000","last_became_active":"2024-04-25T01:26:01.415572+0000","last_became_peered":"2024-04-25T01:26:01.415572+0000","last_unstale":"2024-04-25T01:40:37.137238+0000","last_undegraded":"2024-04-25T01:40:37.137238+0000","last_fullsized":"2024-04-25T01:40:37.137238+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"26'4341","reported_seq":4363,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.334259+0000","last_change":"2024-04-25T01:26:02.422959+0000","last_active":"2024-04-25T01:41:09.334259+0000","last_peered":"2024-04-25T01:41:09.334259+0000","last_clean":"2024-04-25T01:41:09.334259+0000","last_became_active":"2024-04-25T01:26:02.422210+0000","last_became_peered":"2024-04-25T01:26:02.422210+0000","last_unstale":"2024-04-25T01:41:09.334259+0000","last_undegraded":"2024-04-25T01:41:09.334259+0000","last_fullsized":"2024-04-25T01:41:09.334259+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4341,"log_dups_size":0,"ondisk_log_size":4341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22600,"num_write_kb":283130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"26'4919","reported_seq":4949,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.871808+0000","last_change":"2024-04-25T01:25:53.368187+0000","last_active":"2024-04-25T01:40:58.871808+0000","last_peered":"2024-04-25T01:40:58.871808+0000","last_clean":"2024-04-25T01:40:58.871808+0000","last_became_active":"2024-04-25T01:25:53.367664+0000","last_became_peered":"2024-04-25T01:25:53.367664+0000","last_unstale":"2024-04-25T01:40:58.871808+0000","last_undegraded":"2024-04-25T01:40:58.871808+0000","last_fullsized":"2024-04-25T01:40:58.871808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4919,"log_dups_size":0,"ondisk_log_size":4919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:35.902912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25274,"num_write_kb":330818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"26'5","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.184872+0000","last_change":"2024-04-25T01:25:52.357397+0000","last_active":"2024-04-25T01:40:49.184872+0000","last_peered":"2024-04-25T01:40:49.184872+0000","last_clean":"2024-04-25T01:40:49.184872+0000","last_became_active":"2024-04-25T01:25:52.356881+0000","last_became_peered":"2024-04-25T01:25:52.356881+0000","last_unstale":"2024-04-25T01:40:49.184872+0000","last_undegraded":"2024-04-25T01:40:49.184872+0000","last_fullsized":"2024-04-25T01:40:49.184872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"26'314","reported_seq":339,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.149982+0000","last_change":"2024-04-25T01:26:01.420103+0000","last_active":"2024-04-25T01:41:09.149982+0000","last_peered":"2024-04-25T01:41:09.149982+0000","last_clean":"2024-04-25T01:41:09.149982+0000","last_became_active":"2024-04-25T01:26:01.419526+0000","last_became_peered":"2024-04-25T01:26:01.419526+0000","last_unstale":"2024-04-25T01:41:09.149982+0000","last_undegraded":"2024-04-25T01:41:09.149982+0000","last_fullsized":"2024-04-25T01:41:09.149982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":310,"num_write_kb":4240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"26'6253","reported_seq":6276,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.326246+0000","last_change":"2024-04-25T01:26:02.421488+0000","last_active":"2024-04-25T01:41:09.326246+0000","last_peered":"2024-04-25T01:41:09.326246+0000","last_clean":"2024-04-25T01:41:09.326246+0000","last_became_active":"2024-04-25T01:26:02.419308+0000","last_became_peered":"2024-04-25T01:26:02.419308+0000","last_unstale":"2024-04-25T01:41:09.326246+0000","last_undegraded":"2024-04-25T01:41:09.326246+0000","last_fullsized":"2024-04-25T01:41:09.326246+0000","mapping_epoch":22,"log_start":"26'1000","ondisk_log_start":"26'1000","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5253,"log_dups_size":0,"ondisk_log_size":5253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32276,"num_write_kb":400144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"26'4645","reported_seq":4675,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.788191+0000","last_change":"2024-04-25T01:25:53.370720+0000","last_active":"2024-04-25T01:40:58.788191+0000","last_peered":"2024-04-25T01:40:58.788191+0000","last_clean":"2024-04-25T01:40:58.788191+0000","last_became_active":"2024-04-25T01:25:53.370142+0000","last_became_peered":"2024-04-25T01:25:53.370142+0000","last_unstale":"2024-04-25T01:40:58.788191+0000","last_undegraded":"2024-04-25T01:40:58.788191+0000","last_fullsized":"2024-04-25T01:40:58.788191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4645,"log_dups_size":0,"ondisk_log_size":4645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:52.833379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24130,"num_write_kb":289357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"26'8","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.232990+0000","last_change":"2024-04-25T01:25:52.349092+0000","last_active":"2024-04-25T01:40:49.232990+0000","last_peered":"2024-04-25T01:40:49.232990+0000","last_clean":"2024-04-25T01:40:49.232990+0000","last_became_active":"2024-04-25T01:25:52.348846+0000","last_became_peered":"2024-04-25T01:25:52.348846+0000","last_unstale":"2024-04-25T01:40:49.232990+0000","last_undegraded":"2024-04-25T01:40:49.232990+0000","last_fullsized":"2024-04-25T01:40:49.232990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:21.982440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"26'26","reported_seq":50,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.278610+0000","last_change":"2024-04-25T01:26:01.414517+0000","last_active":"2024-04-25T01:40:37.278610+0000","last_peered":"2024-04-25T01:40:37.278610+0000","last_clean":"2024-04-25T01:40:37.278610+0000","last_became_active":"2024-04-25T01:26:01.414243+0000","last_became_peered":"2024-04-25T01:26:01.414243+0000","last_unstale":"2024-04-25T01:40:37.278610+0000","last_undegraded":"2024-04-25T01:40:37.278610+0000","last_fullsized":"2024-04-25T01:40:37.278610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:48.491461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"26'5295","reported_seq":5318,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.307133+0000","last_change":"2024-04-25T01:26:02.415893+0000","last_active":"2024-04-25T01:41:09.307133+0000","last_peered":"2024-04-25T01:41:09.307133+0000","last_clean":"2024-04-25T01:41:09.307133+0000","last_became_active":"2024-04-25T01:26:02.415596+0000","last_became_peered":"2024-04-25T01:26:02.415596+0000","last_unstale":"2024-04-25T01:41:09.307133+0000","last_undegraded":"2024-04-25T01:41:09.307133+0000","last_fullsized":"2024-04-25T01:41:09.307133+0000","mapping_epoch":22,"log_start":"26'100","ondisk_log_start":"26'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5195,"log_dups_size":0,"ondisk_log_size":5195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26962,"num_write_kb":337426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"26'4577","reported_seq":4607,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.793341+0000","last_change":"2024-04-25T01:25:53.373572+0000","last_active":"2024-04-25T01:40:58.793341+0000","last_peered":"2024-04-25T01:40:58.793341+0000","last_clean":"2024-04-25T01:40:58.793341+0000","last_became_active":"2024-04-25T01:25:53.373129+0000","last_became_peered":"2024-04-25T01:25:53.373129+0000","last_unstale":"2024-04-25T01:40:58.793341+0000","last_undegraded":"2024-04-25T01:40:58.793341+0000","last_fullsized":"2024-04-25T01:40:58.793341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23671,"num_write_kb":287382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'335","reported_seq":367,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.269952+0000","last_change":"2024-04-25T01:25:52.350446+0000","last_active":"2024-04-25T01:40:49.269952+0000","last_peered":"2024-04-25T01:40:49.269952+0000","last_clean":"2024-04-25T01:40:49.269952+0000","last_became_active":"2024-04-25T01:25:52.349448+0000","last_became_peered":"2024-04-25T01:25:52.349448+0000","last_unstale":"2024-04-25T01:40:49.269952+0000","last_undegraded":"2024-04-25T01:40:49.269952+0000","last_fullsized":"2024-04-25T01:40:49.269952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:09.953994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":331,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"26'5","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.117732+0000","last_change":"2024-04-25T01:26:01.418794+0000","last_active":"2024-04-25T01:40:37.117732+0000","last_peered":"2024-04-25T01:40:37.117732+0000","last_clean":"2024-04-25T01:40:37.117732+0000","last_became_active":"2024-04-25T01:26:01.418541+0000","last_became_peered":"2024-04-25T01:26:01.418541+0000","last_unstale":"2024-04-25T01:40:37.117732+0000","last_undegraded":"2024-04-25T01:40:37.117732+0000","last_fullsized":"2024-04-25T01:40:37.117732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"26'4257","reported_seq":4280,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.307382+0000","last_change":"2024-04-25T01:26:02.425391+0000","last_active":"2024-04-25T01:41:09.307382+0000","last_peered":"2024-04-25T01:41:09.307382+0000","last_clean":"2024-04-25T01:41:09.307382+0000","last_became_active":"2024-04-25T01:26:02.425106+0000","last_became_peered":"2024-04-25T01:26:02.425106+0000","last_unstale":"2024-04-25T01:41:09.307382+0000","last_undegraded":"2024-04-25T01:41:09.307382+0000","last_fullsized":"2024-04-25T01:41:09.307382+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4257,"log_dups_size":0,"ondisk_log_size":4257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22152,"num_write_kb":281377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"26'6968","reported_seq":6998,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.878933+0000","last_change":"2024-04-25T01:25:53.366416+0000","last_active":"2024-04-25T01:40:58.878933+0000","last_peered":"2024-04-25T01:40:58.878933+0000","last_clean":"2024-04-25T01:40:58.878933+0000","last_became_active":"2024-04-25T01:25:53.366157+0000","last_became_peered":"2024-04-25T01:25:53.366157+0000","last_unstale":"2024-04-25T01:40:58.878933+0000","last_undegraded":"2024-04-25T01:40:58.878933+0000","last_fullsized":"2024-04-25T01:40:58.878933+0000","mapping_epoch":18,"log_start":"26'1500","ondisk_log_start":"26'1500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5468,"log_dups_size":0,"ondisk_log_size":5468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:26.277566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35763,"num_write_kb":414188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"26'25","reported_seq":58,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.305399+0000","last_change":"2024-04-25T01:25:52.357610+0000","last_active":"2024-04-25T01:40:49.305399+0000","last_peered":"2024-04-25T01:40:49.305399+0000","last_clean":"2024-04-25T01:40:49.305399+0000","last_became_active":"2024-04-25T01:25:52.357355+0000","last_became_peered":"2024-04-25T01:25:52.357355+0000","last_unstale":"2024-04-25T01:40:49.305399+0000","last_undegraded":"2024-04-25T01:40:49.305399+0000","last_fullsized":"2024-04-25T01:40:49.305399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"26'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.122870+0000","last_change":"2024-04-25T01:26:01.412666+0000","last_active":"2024-04-25T01:40:37.122870+0000","last_peered":"2024-04-25T01:40:37.122870+0000","last_clean":"2024-04-25T01:40:37.122870+0000","last_became_active":"2024-04-25T01:26:01.412132+0000","last_became_peered":"2024-04-25T01:26:01.412132+0000","last_unstale":"2024-04-25T01:40:37.122870+0000","last_undegraded":"2024-04-25T01:40:37.122870+0000","last_fullsized":"2024-04-25T01:40:37.122870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.640269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"26'5141","reported_seq":5163,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.286915+0000","last_change":"2024-04-25T01:26:02.413106+0000","last_active":"2024-04-25T01:41:09.286915+0000","last_peered":"2024-04-25T01:41:09.286915+0000","last_clean":"2024-04-25T01:41:09.286915+0000","last_became_active":"2024-04-25T01:26:02.412759+0000","last_became_peered":"2024-04-25T01:26:02.412759+0000","last_unstale":"2024-04-25T01:41:09.286915+0000","last_undegraded":"2024-04-25T01:41:09.286915+0000","last_fullsized":"2024-04-25T01:41:09.286915+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5141,"log_dups_size":0,"ondisk_log_size":5141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:28.049190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27471,"num_write_kb":310956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"26'3595","reported_seq":3625,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.782418+0000","last_change":"2024-04-25T01:25:53.374311+0000","last_active":"2024-04-25T01:40:58.782418+0000","last_peered":"2024-04-25T01:40:58.782418+0000","last_clean":"2024-04-25T01:40:58.782418+0000","last_became_active":"2024-04-25T01:25:53.373890+0000","last_became_peered":"2024-04-25T01:25:53.373890+0000","last_unstale":"2024-04-25T01:40:58.782418+0000","last_undegraded":"2024-04-25T01:40:58.782418+0000","last_fullsized":"2024-04-25T01:40:58.782418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3595,"log_dups_size":0,"ondisk_log_size":3595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18396,"num_write_kb":249899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"26'2","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.190934+0000","last_change":"2024-04-25T01:25:52.356231+0000","last_active":"2024-04-25T01:40:49.190934+0000","last_peered":"2024-04-25T01:40:49.190934+0000","last_clean":"2024-04-25T01:40:49.190934+0000","last_became_active":"2024-04-25T01:25:52.355913+0000","last_became_peered":"2024-04-25T01:25:52.355913+0000","last_unstale":"2024-04-25T01:40:49.190934+0000","last_undegraded":"2024-04-25T01:40:49.190934+0000","last_fullsized":"2024-04-25T01:40:49.190934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:08.851417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"26'12","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:47.847766+0000","last_change":"2024-04-25T01:26:01.419513+0000","last_active":"2024-04-25T01:40:47.847766+0000","last_peered":"2024-04-25T01:40:47.847766+0000","last_clean":"2024-04-25T01:40:47.847766+0000","last_became_active":"2024-04-25T01:26:01.419015+0000","last_became_peered":"2024-04-25T01:26:01.419015+0000","last_unstale":"2024-04-25T01:40:47.847766+0000","last_undegraded":"2024-04-25T01:40:47.847766+0000","last_fullsized":"2024-04-25T01:40:47.847766+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"26'5013","reported_seq":5035,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.306814+0000","last_change":"2024-04-25T01:26:02.425241+0000","last_active":"2024-04-25T01:41:09.306814+0000","last_peered":"2024-04-25T01:41:09.306814+0000","last_clean":"2024-04-25T01:41:09.306814+0000","last_became_active":"2024-04-25T01:26:02.424960+0000","last_became_peered":"2024-04-25T01:26:02.424960+0000","last_unstale":"2024-04-25T01:41:09.306814+0000","last_undegraded":"2024-04-25T01:41:09.306814+0000","last_fullsized":"2024-04-25T01:41:09.306814+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5013,"log_dups_size":0,"ondisk_log_size":5013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25777,"num_write_kb":320486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"26'4876","reported_seq":4906,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.800754+0000","last_change":"2024-04-25T01:25:53.371445+0000","last_active":"2024-04-25T01:40:58.800754+0000","last_peered":"2024-04-25T01:40:58.800754+0000","last_clean":"2024-04-25T01:40:58.800754+0000","last_became_active":"2024-04-25T01:25:53.371086+0000","last_became_peered":"2024-04-25T01:25:53.371086+0000","last_unstale":"2024-04-25T01:40:58.800754+0000","last_undegraded":"2024-04-25T01:40:58.800754+0000","last_fullsized":"2024-04-25T01:40:58.800754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4876,"log_dups_size":0,"ondisk_log_size":4876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:44.510726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24656,"num_write_kb":303790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"26'2","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:26.547207+0000","last_change":"2024-04-25T01:25:52.357459+0000","last_active":"2024-04-25T01:40:26.547207+0000","last_peered":"2024-04-25T01:40:26.547207+0000","last_clean":"2024-04-25T01:40:26.547207+0000","last_became_active":"2024-04-25T01:25:52.357182+0000","last_became_peered":"2024-04-25T01:25:52.357182+0000","last_unstale":"2024-04-25T01:40:26.547207+0000","last_undegraded":"2024-04-25T01:40:26.547207+0000","last_fullsized":"2024-04-25T01:40:26.547207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:37.661918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"26'6","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.140717+0000","last_change":"2024-04-25T01:26:01.415321+0000","last_active":"2024-04-25T01:40:37.140717+0000","last_peered":"2024-04-25T01:40:37.140717+0000","last_clean":"2024-04-25T01:40:37.140717+0000","last_became_active":"2024-04-25T01:26:01.415020+0000","last_became_peered":"2024-04-25T01:26:01.415020+0000","last_unstale":"2024-04-25T01:40:37.140717+0000","last_undegraded":"2024-04-25T01:40:37.140717+0000","last_fullsized":"2024-04-25T01:40:37.140717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"26'5316","reported_seq":5339,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.331286+0000","last_change":"2024-04-25T01:26:02.416917+0000","last_active":"2024-04-25T01:41:09.331286+0000","last_peered":"2024-04-25T01:41:09.331286+0000","last_clean":"2024-04-25T01:41:09.331286+0000","last_became_active":"2024-04-25T01:26:02.416623+0000","last_became_peered":"2024-04-25T01:26:02.416623+0000","last_unstale":"2024-04-25T01:41:09.331286+0000","last_undegraded":"2024-04-25T01:41:09.331286+0000","last_fullsized":"2024-04-25T01:41:09.331286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5316,"log_dups_size":0,"ondisk_log_size":5316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:55.040946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27734,"num_write_kb":356956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"26'8","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:37.028842+0000","last_change":"2024-04-25T01:26:01.415472+0000","last_active":"2024-04-25T01:40:37.028842+0000","last_peered":"2024-04-25T01:40:37.028842+0000","last_clean":"2024-04-25T01:40:37.028842+0000","last_became_active":"2024-04-25T01:26:01.415184+0000","last_became_peered":"2024-04-25T01:26:01.415184+0000","last_unstale":"2024-04-25T01:40:37.028842+0000","last_undegraded":"2024-04-25T01:40:37.028842+0000","last_fullsized":"2024-04-25T01:40:37.028842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:16.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"26'5623","reported_seq":5654,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.802488+0000","last_change":"2024-04-25T01:25:53.363043+0000","last_active":"2024-04-25T01:40:58.802488+0000","last_peered":"2024-04-25T01:40:58.802488+0000","last_clean":"2024-04-25T01:40:58.802488+0000","last_became_active":"2024-04-25T01:25:53.362601+0000","last_became_peered":"2024-04-25T01:25:53.362601+0000","last_unstale":"2024-04-25T01:40:58.802488+0000","last_undegraded":"2024-04-25T01:40:58.802488+0000","last_fullsized":"2024-04-25T01:40:58.802488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5623,"log_dups_size":0,"ondisk_log_size":5623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29125,"num_write_kb":354366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"26'3","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:46.309270+0000","last_change":"2024-04-25T01:25:52.358081+0000","last_active":"2024-04-25T01:40:46.309270+0000","last_peered":"2024-04-25T01:40:46.309270+0000","last_clean":"2024-04-25T01:40:46.309270+0000","last_became_active":"2024-04-25T01:25:52.357365+0000","last_became_peered":"2024-04-25T01:25:52.357365+0000","last_unstale":"2024-04-25T01:40:46.309270+0000","last_undegraded":"2024-04-25T01:40:46.309270+0000","last_fullsized":"2024-04-25T01:40:46.309270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"26'4779","reported_seq":4802,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.260694+0000","last_change":"2024-04-25T01:26:02.418872+0000","last_active":"2024-04-25T01:41:09.260694+0000","last_peered":"2024-04-25T01:41:09.260694+0000","last_clean":"2024-04-25T01:41:09.260694+0000","last_became_active":"2024-04-25T01:26:02.418614+0000","last_became_peered":"2024-04-25T01:26:02.418614+0000","last_unstale":"2024-04-25T01:41:09.260694+0000","last_undegraded":"2024-04-25T01:41:09.260694+0000","last_fullsized":"2024-04-25T01:41:09.260694+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4779,"log_dups_size":0,"ondisk_log_size":4779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:49.498275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25207,"num_write_kb":318198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"26'632","reported_seq":657,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:27.924243+0000","last_change":"2024-04-25T01:26:01.410258+0000","last_active":"2024-04-25T01:40:27.924243+0000","last_peered":"2024-04-25T01:40:27.924243+0000","last_clean":"2024-04-25T01:40:27.924243+0000","last_became_active":"2024-04-25T01:26:01.410004+0000","last_became_peered":"2024-04-25T01:26:01.410004+0000","last_unstale":"2024-04-25T01:40:27.924243+0000","last_undegraded":"2024-04-25T01:40:27.924243+0000","last_fullsized":"2024-04-25T01:40:27.924243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:48.865940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":630,"num_write_kb":4461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"26'4416","reported_seq":4447,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.812183+0000","last_change":"2024-04-25T01:25:53.363794+0000","last_active":"2024-04-25T01:40:58.812183+0000","last_peered":"2024-04-25T01:40:58.812183+0000","last_clean":"2024-04-25T01:40:58.812183+0000","last_became_active":"2024-04-25T01:25:53.363509+0000","last_became_peered":"2024-04-25T01:25:53.363509+0000","last_unstale":"2024-04-25T01:40:58.812183+0000","last_undegraded":"2024-04-25T01:40:58.812183+0000","last_fullsized":"2024-04-25T01:40:58.812183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4416,"log_dups_size":0,"ondisk_log_size":4416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23348,"num_write_kb":294385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"26'3","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:49.253549+0000","last_change":"2024-04-25T01:25:52.347464+0000","last_active":"2024-04-25T01:40:49.253549+0000","last_peered":"2024-04-25T01:40:49.253549+0000","last_clean":"2024-04-25T01:40:49.253549+0000","last_became_active":"2024-04-25T01:25:52.347138+0000","last_became_peered":"2024-04-25T01:25:52.347138+0000","last_unstale":"2024-04-25T01:40:49.253549+0000","last_undegraded":"2024-04-25T01:40:49.253549+0000","last_fullsized":"2024-04-25T01:40:49.253549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"26'5575","reported_seq":5598,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.363387+0000","last_change":"2024-04-25T01:26:02.425948+0000","last_active":"2024-04-25T01:41:09.363387+0000","last_peered":"2024-04-25T01:41:09.363387+0000","last_clean":"2024-04-25T01:41:09.363387+0000","last_became_active":"2024-04-25T01:26:02.424947+0000","last_became_peered":"2024-04-25T01:26:02.424947+0000","last_unstale":"2024-04-25T01:41:09.363387+0000","last_undegraded":"2024-04-25T01:41:09.363387+0000","last_fullsized":"2024-04-25T01:41:09.363387+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5575,"log_dups_size":0,"ondisk_log_size":5575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28821,"num_write_kb":361321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"26'87","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.639358+0000","last_change":"2024-04-25T01:25:52.349989+0000","last_active":"2024-04-25T01:40:58.639358+0000","last_peered":"2024-04-25T01:40:58.639358+0000","last_clean":"2024-04-25T01:40:58.639358+0000","last_became_active":"2024-04-25T01:25:52.349672+0000","last_became_peered":"2024-04-25T01:25:52.349672+0000","last_unstale":"2024-04-25T01:40:58.639358+0000","last_undegraded":"2024-04-25T01:40:58.639358+0000","last_fullsized":"2024-04-25T01:40:58.639358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:42.186550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":980823,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83,"num_write_kb":1015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"26'3832","reported_seq":3862,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.794598+0000","last_change":"2024-04-25T01:25:53.356610+0000","last_active":"2024-04-25T01:40:58.794598+0000","last_peered":"2024-04-25T01:40:58.794598+0000","last_clean":"2024-04-25T01:40:58.794598+0000","last_became_active":"2024-04-25T01:25:53.356015+0000","last_became_peered":"2024-04-25T01:25:53.356015+0000","last_unstale":"2024-04-25T01:40:58.794598+0000","last_undegraded":"2024-04-25T01:40:58.794598+0000","last_fullsized":"2024-04-25T01:40:58.794598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3832,"log_dups_size":0,"ondisk_log_size":3832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19386,"num_write_kb":237359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"26'3","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:00.676831+0000","last_change":"2024-04-25T01:26:01.420261+0000","last_active":"2024-04-25T01:41:00.676831+0000","last_peered":"2024-04-25T01:41:00.676831+0000","last_clean":"2024-04-25T01:41:00.676831+0000","last_became_active":"2024-04-25T01:26:01.420003+0000","last_became_peered":"2024-04-25T01:26:01.420003+0000","last_unstale":"2024-04-25T01:41:00.676831+0000","last_undegraded":"2024-04-25T01:41:00.676831+0000","last_fullsized":"2024-04-25T01:41:00.676831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"26'3175","reported_seq":3197,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.349312+0000","last_change":"2024-04-25T01:26:02.426327+0000","last_active":"2024-04-25T01:41:09.349312+0000","last_peered":"2024-04-25T01:41:09.349312+0000","last_clean":"2024-04-25T01:41:09.349312+0000","last_became_active":"2024-04-25T01:26:02.426054+0000","last_became_peered":"2024-04-25T01:26:02.426054+0000","last_unstale":"2024-04-25T01:41:09.349312+0000","last_undegraded":"2024-04-25T01:41:09.349312+0000","last_fullsized":"2024-04-25T01:41:09.349312+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3175,"log_dups_size":0,"ondisk_log_size":3175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:17.809738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15854,"num_write_kb":211025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"26'605","reported_seq":638,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:55.698906+0000","last_change":"2024-04-25T01:25:52.353955+0000","last_active":"2024-04-25T01:40:55.698906+0000","last_peered":"2024-04-25T01:40:55.698906+0000","last_clean":"2024-04-25T01:40:55.698906+0000","last_became_active":"2024-04-25T01:25:52.353659+0000","last_became_peered":"2024-04-25T01:25:52.353659+0000","last_unstale":"2024-04-25T01:40:55.698906+0000","last_undegraded":"2024-04-25T01:40:55.698906+0000","last_fullsized":"2024-04-25T01:40:55.698906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":603,"num_write_kb":4239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"26'4191","reported_seq":4221,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.749061+0000","last_change":"2024-04-25T01:25:53.366586+0000","last_active":"2024-04-25T01:40:58.749061+0000","last_peered":"2024-04-25T01:40:58.749061+0000","last_clean":"2024-04-25T01:40:58.749061+0000","last_became_active":"2024-04-25T01:25:53.366302+0000","last_became_peered":"2024-04-25T01:25:53.366302+0000","last_unstale":"2024-04-25T01:40:58.749061+0000","last_undegraded":"2024-04-25T01:40:58.749061+0000","last_fullsized":"2024-04-25T01:40:58.749061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4191,"log_dups_size":0,"ondisk_log_size":4191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21716,"num_write_kb":263159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"26'11","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:00.699646+0000","last_change":"2024-04-25T01:26:01.420960+0000","last_active":"2024-04-25T01:41:00.699646+0000","last_peered":"2024-04-25T01:41:00.699646+0000","last_clean":"2024-04-25T01:41:00.699646+0000","last_became_active":"2024-04-25T01:26:01.420713+0000","last_became_peered":"2024-04-25T01:26:01.420713+0000","last_unstale":"2024-04-25T01:41:00.699646+0000","last_undegraded":"2024-04-25T01:41:00.699646+0000","last_fullsized":"2024-04-25T01:41:00.699646+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:00.384370+0000","last_clean_scrub_stamp":"2024-04-25T01:26:00.384370+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"26'4569","reported_seq":4593,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:15.701380+0000","last_change":"2024-04-25T01:26:02.419282+0000","last_active":"2024-04-25T01:41:15.701380+0000","last_peered":"2024-04-25T01:41:15.701380+0000","last_clean":"2024-04-25T01:41:15.701380+0000","last_became_active":"2024-04-25T01:26:02.418802+0000","last_became_peered":"2024-04-25T01:26:02.418802+0000","last_unstale":"2024-04-25T01:41:15.701380+0000","last_undegraded":"2024-04-25T01:41:15.701380+0000","last_fullsized":"2024-04-25T01:41:15.701380+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4569,"log_dups_size":0,"ondisk_log_size":4569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23161,"num_write_kb":277521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"26'6240","reported_seq":6271,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.802380+0000","last_change":"2024-04-25T01:25:53.367143+0000","last_active":"2024-04-25T01:40:58.802380+0000","last_peered":"2024-04-25T01:40:58.802380+0000","last_clean":"2024-04-25T01:40:58.802380+0000","last_became_active":"2024-04-25T01:25:53.366847+0000","last_became_peered":"2024-04-25T01:25:53.366847+0000","last_unstale":"2024-04-25T01:40:58.802380+0000","last_undegraded":"2024-04-25T01:40:58.802380+0000","last_fullsized":"2024-04-25T01:40:58.802380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":6240,"log_dups_size":0,"ondisk_log_size":6240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32134,"num_write_kb":382834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"26'5302","reported_seq":5326,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:15.675319+0000","last_change":"2024-04-25T01:26:02.416438+0000","last_active":"2024-04-25T01:41:15.675319+0000","last_peered":"2024-04-25T01:41:15.675319+0000","last_clean":"2024-04-25T01:41:15.675319+0000","last_became_active":"2024-04-25T01:26:02.416170+0000","last_became_peered":"2024-04-25T01:26:02.416170+0000","last_unstale":"2024-04-25T01:41:15.675319+0000","last_undegraded":"2024-04-25T01:41:15.675319+0000","last_fullsized":"2024-04-25T01:41:15.675319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5302,"log_dups_size":0,"ondisk_log_size":5302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27000,"num_write_kb":345927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"26'4854","reported_seq":4886,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.808062+0000","last_change":"2024-04-25T01:25:53.371345+0000","last_active":"2024-04-25T01:40:58.808062+0000","last_peered":"2024-04-25T01:40:58.808062+0000","last_clean":"2024-04-25T01:40:58.808062+0000","last_became_active":"2024-04-25T01:25:53.371080+0000","last_became_peered":"2024-04-25T01:25:53.371080+0000","last_unstale":"2024-04-25T01:40:58.808062+0000","last_undegraded":"2024-04-25T01:40:58.808062+0000","last_fullsized":"2024-04-25T01:40:58.808062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4854,"log_dups_size":0,"ondisk_log_size":4854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24922,"num_write_kb":316898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"26'4098","reported_seq":4121,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.322931+0000","last_change":"2024-04-25T01:26:02.419688+0000","last_active":"2024-04-25T01:41:09.322931+0000","last_peered":"2024-04-25T01:41:09.322931+0000","last_clean":"2024-04-25T01:41:09.322931+0000","last_became_active":"2024-04-25T01:26:02.419417+0000","last_became_peered":"2024-04-25T01:26:02.419417+0000","last_unstale":"2024-04-25T01:41:09.322931+0000","last_undegraded":"2024-04-25T01:41:09.322931+0000","last_fullsized":"2024-04-25T01:41:09.322931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4098,"log_dups_size":0,"ondisk_log_size":4098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21368,"num_write_kb":269972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"26'5565","reported_seq":5596,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.839128+0000","last_change":"2024-04-25T01:25:53.369850+0000","last_active":"2024-04-25T01:40:58.839128+0000","last_peered":"2024-04-25T01:40:58.839128+0000","last_clean":"2024-04-25T01:40:58.839128+0000","last_became_active":"2024-04-25T01:25:53.369350+0000","last_became_peered":"2024-04-25T01:25:53.369350+0000","last_unstale":"2024-04-25T01:40:58.839128+0000","last_undegraded":"2024-04-25T01:40:58.839128+0000","last_fullsized":"2024-04-25T01:40:58.839128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5565,"log_dups_size":0,"ondisk_log_size":5565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29144,"num_write_kb":362518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"26'5067","reported_seq":5091,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:15.675258+0000","last_change":"2024-04-25T01:26:02.425528+0000","last_active":"2024-04-25T01:41:15.675258+0000","last_peered":"2024-04-25T01:41:15.675258+0000","last_clean":"2024-04-25T01:41:15.675258+0000","last_became_active":"2024-04-25T01:26:02.425160+0000","last_became_peered":"2024-04-25T01:26:02.425160+0000","last_unstale":"2024-04-25T01:41:15.675258+0000","last_undegraded":"2024-04-25T01:41:15.675258+0000","last_fullsized":"2024-04-25T01:41:15.675258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26737,"num_write_kb":324333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"26'3787","reported_seq":3818,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.775758+0000","last_change":"2024-04-25T01:25:53.357142+0000","last_active":"2024-04-25T01:40:58.775758+0000","last_peered":"2024-04-25T01:40:58.775758+0000","last_clean":"2024-04-25T01:40:58.775758+0000","last_became_active":"2024-04-25T01:25:53.356854+0000","last_became_peered":"2024-04-25T01:25:53.356854+0000","last_unstale":"2024-04-25T01:40:58.775758+0000","last_undegraded":"2024-04-25T01:40:58.775758+0000","last_fullsized":"2024-04-25T01:40:58.775758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3787,"log_dups_size":0,"ondisk_log_size":3787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19900,"num_write_kb":255902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"26'5481","reported_seq":5504,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.289690+0000","last_change":"2024-04-25T01:26:02.417492+0000","last_active":"2024-04-25T01:41:09.289690+0000","last_peered":"2024-04-25T01:41:09.289690+0000","last_clean":"2024-04-25T01:41:09.289690+0000","last_became_active":"2024-04-25T01:26:02.417234+0000","last_became_peered":"2024-04-25T01:26:02.417234+0000","last_unstale":"2024-04-25T01:41:09.289690+0000","last_undegraded":"2024-04-25T01:41:09.289690+0000","last_fullsized":"2024-04-25T01:41:09.289690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5481,"log_dups_size":0,"ondisk_log_size":5481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:29.366072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28430,"num_write_kb":359662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"26'4160","reported_seq":4192,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:05.700145+0000","last_change":"2024-04-25T01:25:53.364575+0000","last_active":"2024-04-25T01:41:05.700145+0000","last_peered":"2024-04-25T01:41:05.700145+0000","last_clean":"2024-04-25T01:41:05.700145+0000","last_became_active":"2024-04-25T01:25:53.364311+0000","last_became_peered":"2024-04-25T01:25:53.364311+0000","last_unstale":"2024-04-25T01:41:05.700145+0000","last_undegraded":"2024-04-25T01:41:05.700145+0000","last_fullsized":"2024-04-25T01:41:05.700145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4160,"log_dups_size":0,"ondisk_log_size":4160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21579,"num_write_kb":270844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"26'4125","reported_seq":4148,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.329055+0000","last_change":"2024-04-25T01:26:02.426813+0000","last_active":"2024-04-25T01:41:09.329055+0000","last_peered":"2024-04-25T01:41:09.329055+0000","last_clean":"2024-04-25T01:41:09.329055+0000","last_became_active":"2024-04-25T01:26:02.426552+0000","last_became_peered":"2024-04-25T01:26:02.426552+0000","last_unstale":"2024-04-25T01:41:09.329055+0000","last_undegraded":"2024-04-25T01:41:09.329055+0000","last_fullsized":"2024-04-25T01:41:09.329055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4125,"log_dups_size":0,"ondisk_log_size":4125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:03.687174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21785,"num_write_kb":263458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"26'5101","reported_seq":5133,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:05.674152+0000","last_change":"2024-04-25T01:25:53.368413+0000","last_active":"2024-04-25T01:41:05.674152+0000","last_peered":"2024-04-25T01:41:05.674152+0000","last_clean":"2024-04-25T01:41:05.674152+0000","last_became_active":"2024-04-25T01:25:53.368176+0000","last_became_peered":"2024-04-25T01:25:53.368176+0000","last_unstale":"2024-04-25T01:41:05.674152+0000","last_undegraded":"2024-04-25T01:41:05.674152+0000","last_fullsized":"2024-04-25T01:41:05.674152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5101,"log_dups_size":0,"ondisk_log_size":5101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26850,"num_write_kb":324721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"26'6020","reported_seq":6043,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.350276+0000","last_change":"2024-04-25T01:26:02.425185+0000","last_active":"2024-04-25T01:41:09.350276+0000","last_peered":"2024-04-25T01:41:09.350276+0000","last_clean":"2024-04-25T01:41:09.350276+0000","last_became_active":"2024-04-25T01:26:02.424903+0000","last_became_peered":"2024-04-25T01:26:02.424903+0000","last_unstale":"2024-04-25T01:41:09.350276+0000","last_undegraded":"2024-04-25T01:41:09.350276+0000","last_fullsized":"2024-04-25T01:41:09.350276+0000","mapping_epoch":22,"log_start":"26'300","ondisk_log_start":"26'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5720,"log_dups_size":0,"ondisk_log_size":5720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31332,"num_write_kb":379614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"26'4058","reported_seq":4089,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.867006+0000","last_change":"2024-04-25T01:25:53.372638+0000","last_active":"2024-04-25T01:40:58.867006+0000","last_peered":"2024-04-25T01:40:58.867006+0000","last_clean":"2024-04-25T01:40:58.867006+0000","last_became_active":"2024-04-25T01:25:53.372001+0000","last_became_peered":"2024-04-25T01:25:53.372001+0000","last_unstale":"2024-04-25T01:40:58.867006+0000","last_undegraded":"2024-04-25T01:40:58.867006+0000","last_fullsized":"2024-04-25T01:40:58.867006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4058,"log_dups_size":0,"ondisk_log_size":4058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20502,"num_write_kb":266785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"26'4843","reported_seq":4866,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.307262+0000","last_change":"2024-04-25T01:26:02.420867+0000","last_active":"2024-04-25T01:41:09.307262+0000","last_peered":"2024-04-25T01:41:09.307262+0000","last_clean":"2024-04-25T01:41:09.307262+0000","last_became_active":"2024-04-25T01:26:02.420621+0000","last_became_peered":"2024-04-25T01:26:02.420621+0000","last_unstale":"2024-04-25T01:41:09.307262+0000","last_undegraded":"2024-04-25T01:41:09.307262+0000","last_fullsized":"2024-04-25T01:41:09.307262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4843,"log_dups_size":0,"ondisk_log_size":4843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:11.493181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24777,"num_write_kb":316390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"26'5268","reported_seq":5300,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:05.700179+0000","last_change":"2024-04-25T01:25:53.365509+0000","last_active":"2024-04-25T01:41:05.700179+0000","last_peered":"2024-04-25T01:41:05.700179+0000","last_clean":"2024-04-25T01:41:05.700179+0000","last_became_active":"2024-04-25T01:25:53.365245+0000","last_became_peered":"2024-04-25T01:25:53.365245+0000","last_unstale":"2024-04-25T01:41:05.700179+0000","last_undegraded":"2024-04-25T01:41:05.700179+0000","last_fullsized":"2024-04-25T01:41:05.700179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5268,"log_dups_size":0,"ondisk_log_size":5268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26981,"num_write_kb":345791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"26'3558","reported_seq":3581,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.357346+0000","last_change":"2024-04-25T01:26:02.415672+0000","last_active":"2024-04-25T01:41:09.357346+0000","last_peered":"2024-04-25T01:41:09.357346+0000","last_clean":"2024-04-25T01:41:09.357346+0000","last_became_active":"2024-04-25T01:26:02.415389+0000","last_became_peered":"2024-04-25T01:26:02.415389+0000","last_unstale":"2024-04-25T01:41:09.357346+0000","last_undegraded":"2024-04-25T01:41:09.357346+0000","last_fullsized":"2024-04-25T01:41:09.357346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3558,"log_dups_size":0,"ondisk_log_size":3558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18097,"num_write_kb":236200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"26'4040","reported_seq":4071,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.818105+0000","last_change":"2024-04-25T01:25:53.365391+0000","last_active":"2024-04-25T01:40:58.818105+0000","last_peered":"2024-04-25T01:40:58.818105+0000","last_clean":"2024-04-25T01:40:58.818105+0000","last_became_active":"2024-04-25T01:25:53.364829+0000","last_became_peered":"2024-04-25T01:25:53.364829+0000","last_unstale":"2024-04-25T01:40:58.818105+0000","last_undegraded":"2024-04-25T01:40:58.818105+0000","last_fullsized":"2024-04-25T01:40:58.818105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4040,"log_dups_size":0,"ondisk_log_size":4040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20607,"num_write_kb":262822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"26'5431","reported_seq":5453,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.318622+0000","last_change":"2024-04-25T01:26:02.418292+0000","last_active":"2024-04-25T01:41:09.318622+0000","last_peered":"2024-04-25T01:41:09.318622+0000","last_clean":"2024-04-25T01:41:09.318622+0000","last_became_active":"2024-04-25T01:26:02.418036+0000","last_became_peered":"2024-04-25T01:26:02.418036+0000","last_unstale":"2024-04-25T01:41:09.318622+0000","last_undegraded":"2024-04-25T01:41:09.318622+0000","last_fullsized":"2024-04-25T01:41:09.318622+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5431,"log_dups_size":0,"ondisk_log_size":5431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28059,"num_write_kb":358266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"26'4709","reported_seq":4740,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.818256+0000","last_change":"2024-04-25T01:25:53.355667+0000","last_active":"2024-04-25T01:40:58.818256+0000","last_peered":"2024-04-25T01:40:58.818256+0000","last_clean":"2024-04-25T01:40:58.818256+0000","last_became_active":"2024-04-25T01:25:53.355417+0000","last_became_peered":"2024-04-25T01:25:53.355417+0000","last_unstale":"2024-04-25T01:40:58.818256+0000","last_undegraded":"2024-04-25T01:40:58.818256+0000","last_fullsized":"2024-04-25T01:40:58.818256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4709,"log_dups_size":0,"ondisk_log_size":4709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24394,"num_write_kb":298399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"26'4741","reported_seq":4764,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.305377+0000","last_change":"2024-04-25T01:26:02.414994+0000","last_active":"2024-04-25T01:41:09.305377+0000","last_peered":"2024-04-25T01:41:09.305377+0000","last_clean":"2024-04-25T01:41:09.305377+0000","last_became_active":"2024-04-25T01:26:02.414735+0000","last_became_peered":"2024-04-25T01:26:02.414735+0000","last_unstale":"2024-04-25T01:41:09.305377+0000","last_undegraded":"2024-04-25T01:41:09.305377+0000","last_fullsized":"2024-04-25T01:41:09.305377+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4741,"log_dups_size":0,"ondisk_log_size":4741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24824,"num_write_kb":300285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"26'5111","reported_seq":5141,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.869446+0000","last_change":"2024-04-25T01:25:53.371240+0000","last_active":"2024-04-25T01:40:58.869446+0000","last_peered":"2024-04-25T01:40:58.869446+0000","last_clean":"2024-04-25T01:40:58.869446+0000","last_became_active":"2024-04-25T01:25:53.370164+0000","last_became_peered":"2024-04-25T01:25:53.370164+0000","last_unstale":"2024-04-25T01:40:58.869446+0000","last_undegraded":"2024-04-25T01:40:58.869446+0000","last_fullsized":"2024-04-25T01:40:58.869446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:50.435339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26542,"num_write_kb":327744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"26'4367","reported_seq":4390,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.312526+0000","last_change":"2024-04-25T01:26:02.418272+0000","last_active":"2024-04-25T01:41:09.312526+0000","last_peered":"2024-04-25T01:41:09.312526+0000","last_clean":"2024-04-25T01:41:09.312526+0000","last_became_active":"2024-04-25T01:26:02.417977+0000","last_became_peered":"2024-04-25T01:26:02.417977+0000","last_unstale":"2024-04-25T01:41:09.312526+0000","last_undegraded":"2024-04-25T01:41:09.312526+0000","last_fullsized":"2024-04-25T01:41:09.312526+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4367,"log_dups_size":0,"ondisk_log_size":4367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22024,"num_write_kb":293008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"26'5285","reported_seq":5315,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.820507+0000","last_change":"2024-04-25T01:25:53.376415+0000","last_active":"2024-04-25T01:40:58.820507+0000","last_peered":"2024-04-25T01:40:58.820507+0000","last_clean":"2024-04-25T01:40:58.820507+0000","last_became_active":"2024-04-25T01:25:53.376104+0000","last_became_peered":"2024-04-25T01:25:53.376104+0000","last_unstale":"2024-04-25T01:40:58.820507+0000","last_undegraded":"2024-04-25T01:40:58.820507+0000","last_fullsized":"2024-04-25T01:40:58.820507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5285,"log_dups_size":0,"ondisk_log_size":5285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27331,"num_write_kb":318531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"26'5020","reported_seq":5042,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.344122+0000","last_change":"2024-04-25T01:26:02.426333+0000","last_active":"2024-04-25T01:41:09.344122+0000","last_peered":"2024-04-25T01:41:09.344122+0000","last_clean":"2024-04-25T01:41:09.344122+0000","last_became_active":"2024-04-25T01:26:02.426069+0000","last_became_peered":"2024-04-25T01:26:02.426069+0000","last_unstale":"2024-04-25T01:41:09.344122+0000","last_undegraded":"2024-04-25T01:41:09.344122+0000","last_fullsized":"2024-04-25T01:41:09.344122+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5020,"log_dups_size":0,"ondisk_log_size":5020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:23.523983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26121,"num_write_kb":325976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"26'4910","reported_seq":4940,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.774391+0000","last_change":"2024-04-25T01:25:53.370314+0000","last_active":"2024-04-25T01:40:58.774391+0000","last_peered":"2024-04-25T01:40:58.774391+0000","last_clean":"2024-04-25T01:40:58.774391+0000","last_became_active":"2024-04-25T01:25:53.370049+0000","last_became_peered":"2024-04-25T01:25:53.370049+0000","last_unstale":"2024-04-25T01:40:58.774391+0000","last_undegraded":"2024-04-25T01:40:58.774391+0000","last_fullsized":"2024-04-25T01:40:58.774391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4910,"log_dups_size":0,"ondisk_log_size":4910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:58.048089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25833,"num_write_kb":304449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"26'4775","reported_seq":4797,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.332568+0000","last_change":"2024-04-25T01:26:02.415464+0000","last_active":"2024-04-25T01:41:09.332568+0000","last_peered":"2024-04-25T01:41:09.332568+0000","last_clean":"2024-04-25T01:41:09.332568+0000","last_became_active":"2024-04-25T01:26:02.415173+0000","last_became_peered":"2024-04-25T01:26:02.415173+0000","last_unstale":"2024-04-25T01:41:09.332568+0000","last_undegraded":"2024-04-25T01:41:09.332568+0000","last_fullsized":"2024-04-25T01:41:09.332568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4775,"log_dups_size":0,"ondisk_log_size":4775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.343169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24075,"num_write_kb":305295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"26'4891","reported_seq":4921,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.797939+0000","last_change":"2024-04-25T01:25:53.369585+0000","last_active":"2024-04-25T01:40:58.797939+0000","last_peered":"2024-04-25T01:40:58.797939+0000","last_clean":"2024-04-25T01:40:58.797939+0000","last_became_active":"2024-04-25T01:25:53.368499+0000","last_became_peered":"2024-04-25T01:25:53.368499+0000","last_unstale":"2024-04-25T01:40:58.797939+0000","last_undegraded":"2024-04-25T01:40:58.797939+0000","last_fullsized":"2024-04-25T01:40:58.797939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4891,"log_dups_size":0,"ondisk_log_size":4891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24867,"num_write_kb":304528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"26'4199","reported_seq":4222,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.286639+0000","last_change":"2024-04-25T01:26:02.421975+0000","last_active":"2024-04-25T01:41:09.286639+0000","last_peered":"2024-04-25T01:41:09.286639+0000","last_clean":"2024-04-25T01:41:09.286639+0000","last_became_active":"2024-04-25T01:26:02.421698+0000","last_became_peered":"2024-04-25T01:26:02.421698+0000","last_unstale":"2024-04-25T01:41:09.286639+0000","last_undegraded":"2024-04-25T01:41:09.286639+0000","last_fullsized":"2024-04-25T01:41:09.286639+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4199,"log_dups_size":0,"ondisk_log_size":4199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21738,"num_write_kb":267270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"26'3844","reported_seq":3875,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.786368+0000","last_change":"2024-04-25T01:25:53.370666+0000","last_active":"2024-04-25T01:40:58.786368+0000","last_peered":"2024-04-25T01:40:58.786368+0000","last_clean":"2024-04-25T01:40:58.786368+0000","last_became_active":"2024-04-25T01:25:53.369733+0000","last_became_peered":"2024-04-25T01:25:53.369733+0000","last_unstale":"2024-04-25T01:40:58.786368+0000","last_undegraded":"2024-04-25T01:40:58.786368+0000","last_fullsized":"2024-04-25T01:40:58.786368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3844,"log_dups_size":0,"ondisk_log_size":3844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19367,"num_write_kb":241344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"26'4316","reported_seq":4338,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.253195+0000","last_change":"2024-04-25T01:26:02.418114+0000","last_active":"2024-04-25T01:41:09.253195+0000","last_peered":"2024-04-25T01:41:09.253195+0000","last_clean":"2024-04-25T01:41:09.253195+0000","last_became_active":"2024-04-25T01:26:02.417810+0000","last_became_peered":"2024-04-25T01:26:02.417810+0000","last_unstale":"2024-04-25T01:41:09.253195+0000","last_undegraded":"2024-04-25T01:41:09.253195+0000","last_fullsized":"2024-04-25T01:41:09.253195+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4316,"log_dups_size":0,"ondisk_log_size":4316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:30.496089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22517,"num_write_kb":290939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"26'5814","reported_seq":5831,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.797085+0000","last_change":"2024-04-25T01:26:52.120824+0000","last_active":"2024-04-25T01:40:58.797085+0000","last_peered":"2024-04-25T01:40:58.797085+0000","last_clean":"2024-04-25T01:40:58.797085+0000","last_became_active":"2024-04-25T01:26:52.120588+0000","last_became_peered":"2024-04-25T01:26:52.120588+0000","last_unstale":"2024-04-25T01:40:58.797085+0000","last_undegraded":"2024-04-25T01:40:58.797085+0000","last_fullsized":"2024-04-25T01:40:58.797085+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5814,"log_dups_size":0,"ondisk_log_size":5814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29929,"num_write_kb":365779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.36","version":"26'4743","reported_seq":4766,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.336178+0000","last_change":"2024-04-25T01:26:02.426289+0000","last_active":"2024-04-25T01:41:09.336178+0000","last_peered":"2024-04-25T01:41:09.336178+0000","last_clean":"2024-04-25T01:41:09.336178+0000","last_became_active":"2024-04-25T01:26:02.426015+0000","last_became_peered":"2024-04-25T01:26:02.426015+0000","last_unstale":"2024-04-25T01:41:09.336178+0000","last_undegraded":"2024-04-25T01:41:09.336178+0000","last_fullsized":"2024-04-25T01:41:09.336178+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4743,"log_dups_size":0,"ondisk_log_size":4743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25470,"num_write_kb":331505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"26'4366","reported_seq":4396,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.821883+0000","last_change":"2024-04-25T01:25:53.374186+0000","last_active":"2024-04-25T01:40:58.821883+0000","last_peered":"2024-04-25T01:40:58.821883+0000","last_clean":"2024-04-25T01:40:58.821883+0000","last_became_active":"2024-04-25T01:25:53.373766+0000","last_became_peered":"2024-04-25T01:25:53.373766+0000","last_unstale":"2024-04-25T01:40:58.821883+0000","last_undegraded":"2024-04-25T01:40:58.821883+0000","last_fullsized":"2024-04-25T01:40:58.821883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4366,"log_dups_size":0,"ondisk_log_size":4366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.489466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23328,"num_write_kb":273725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"26'5875","reported_seq":5897,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.356318+0000","last_change":"2024-04-25T01:26:02.416345+0000","last_active":"2024-04-25T01:41:09.356318+0000","last_peered":"2024-04-25T01:41:09.356318+0000","last_clean":"2024-04-25T01:41:09.356318+0000","last_became_active":"2024-04-25T01:26:02.416082+0000","last_became_peered":"2024-04-25T01:26:02.416082+0000","last_unstale":"2024-04-25T01:41:09.356318+0000","last_undegraded":"2024-04-25T01:41:09.356318+0000","last_fullsized":"2024-04-25T01:41:09.356318+0000","mapping_epoch":22,"log_start":"26'400","ondisk_log_start":"26'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5475,"log_dups_size":0,"ondisk_log_size":5475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:47.810943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31009,"num_write_kb":370059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"26'4898","reported_seq":4929,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.812373+0000","last_change":"2024-04-25T01:25:53.371602+0000","last_active":"2024-04-25T01:40:58.812373+0000","last_peered":"2024-04-25T01:40:58.812373+0000","last_clean":"2024-04-25T01:40:58.812373+0000","last_became_active":"2024-04-25T01:25:53.371362+0000","last_became_peered":"2024-04-25T01:25:53.371362+0000","last_unstale":"2024-04-25T01:40:58.812373+0000","last_undegraded":"2024-04-25T01:40:58.812373+0000","last_fullsized":"2024-04-25T01:40:58.812373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4898,"log_dups_size":0,"ondisk_log_size":4898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24919,"num_write_kb":308746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"26'3961","reported_seq":3983,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.316956+0000","last_change":"2024-04-25T01:26:02.425888+0000","last_active":"2024-04-25T01:41:09.316956+0000","last_peered":"2024-04-25T01:41:09.316956+0000","last_clean":"2024-04-25T01:41:09.316956+0000","last_became_active":"2024-04-25T01:26:02.425577+0000","last_became_peered":"2024-04-25T01:26:02.425577+0000","last_unstale":"2024-04-25T01:41:09.316956+0000","last_undegraded":"2024-04-25T01:41:09.316956+0000","last_fullsized":"2024-04-25T01:41:09.316956+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3961,"log_dups_size":0,"ondisk_log_size":3961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:37.998712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20296,"num_write_kb":253395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"26'4670","reported_seq":4700,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.788790+0000","last_change":"2024-04-25T01:25:53.370996+0000","last_active":"2024-04-25T01:40:58.788790+0000","last_peered":"2024-04-25T01:40:58.788790+0000","last_clean":"2024-04-25T01:40:58.788790+0000","last_became_active":"2024-04-25T01:25:53.370479+0000","last_became_peered":"2024-04-25T01:25:53.370479+0000","last_unstale":"2024-04-25T01:40:58.788790+0000","last_undegraded":"2024-04-25T01:40:58.788790+0000","last_fullsized":"2024-04-25T01:40:58.788790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4670,"log_dups_size":0,"ondisk_log_size":4670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:57.636067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24421,"num_write_kb":306681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"26'4733","reported_seq":4755,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.275948+0000","last_change":"2024-04-25T01:26:02.423420+0000","last_active":"2024-04-25T01:41:09.275948+0000","last_peered":"2024-04-25T01:41:09.275948+0000","last_clean":"2024-04-25T01:41:09.275948+0000","last_became_active":"2024-04-25T01:26:02.423146+0000","last_became_peered":"2024-04-25T01:26:02.423146+0000","last_unstale":"2024-04-25T01:41:09.275948+0000","last_undegraded":"2024-04-25T01:41:09.275948+0000","last_fullsized":"2024-04-25T01:41:09.275948+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4733,"log_dups_size":0,"ondisk_log_size":4733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:21.567759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23928,"num_write_kb":292515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"26'4513","reported_seq":4544,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.807827+0000","last_change":"2024-04-25T01:25:53.363512+0000","last_active":"2024-04-25T01:40:58.807827+0000","last_peered":"2024-04-25T01:40:58.807827+0000","last_clean":"2024-04-25T01:40:58.807827+0000","last_became_active":"2024-04-25T01:25:53.363246+0000","last_became_peered":"2024-04-25T01:25:53.363246+0000","last_unstale":"2024-04-25T01:40:58.807827+0000","last_undegraded":"2024-04-25T01:40:58.807827+0000","last_fullsized":"2024-04-25T01:40:58.807827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4513,"log_dups_size":0,"ondisk_log_size":4513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23705,"num_write_kb":279339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"26'4587","reported_seq":4610,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.306483+0000","last_change":"2024-04-25T01:26:02.413346+0000","last_active":"2024-04-25T01:41:09.306483+0000","last_peered":"2024-04-25T01:41:09.306483+0000","last_clean":"2024-04-25T01:41:09.306483+0000","last_became_active":"2024-04-25T01:26:02.413036+0000","last_became_peered":"2024-04-25T01:26:02.413036+0000","last_unstale":"2024-04-25T01:41:09.306483+0000","last_undegraded":"2024-04-25T01:41:09.306483+0000","last_fullsized":"2024-04-25T01:41:09.306483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4587,"log_dups_size":0,"ondisk_log_size":4587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24042,"num_write_kb":305325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"26'3998","reported_seq":4028,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.876422+0000","last_change":"2024-04-25T01:25:53.364457+0000","last_active":"2024-04-25T01:40:58.876422+0000","last_peered":"2024-04-25T01:40:58.876422+0000","last_clean":"2024-04-25T01:40:58.876422+0000","last_became_active":"2024-04-25T01:25:53.364103+0000","last_became_peered":"2024-04-25T01:25:53.364103+0000","last_unstale":"2024-04-25T01:40:58.876422+0000","last_undegraded":"2024-04-25T01:40:58.876422+0000","last_fullsized":"2024-04-25T01:40:58.876422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3998,"log_dups_size":0,"ondisk_log_size":3998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:11.336752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20343,"num_write_kb":253659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"26'4194","reported_seq":4217,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.305734+0000","last_change":"2024-04-25T01:26:02.418792+0000","last_active":"2024-04-25T01:41:09.305734+0000","last_peered":"2024-04-25T01:41:09.305734+0000","last_clean":"2024-04-25T01:41:09.305734+0000","last_became_active":"2024-04-25T01:26:02.416942+0000","last_became_peered":"2024-04-25T01:26:02.416942+0000","last_unstale":"2024-04-25T01:41:09.305734+0000","last_undegraded":"2024-04-25T01:41:09.305734+0000","last_fullsized":"2024-04-25T01:41:09.305734+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4194,"log_dups_size":0,"ondisk_log_size":4194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21868,"num_write_kb":272015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"26'4800","reported_seq":4830,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.771199+0000","last_change":"2024-04-25T01:25:53.371647+0000","last_active":"2024-04-25T01:40:58.771199+0000","last_peered":"2024-04-25T01:40:58.771199+0000","last_clean":"2024-04-25T01:40:58.771199+0000","last_became_active":"2024-04-25T01:25:53.371353+0000","last_became_peered":"2024-04-25T01:25:53.371353+0000","last_unstale":"2024-04-25T01:40:58.771199+0000","last_undegraded":"2024-04-25T01:40:58.771199+0000","last_fullsized":"2024-04-25T01:40:58.771199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4800,"log_dups_size":0,"ondisk_log_size":4800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:05.211391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24494,"num_write_kb":294847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"26'4139","reported_seq":4161,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.342741+0000","last_change":"2024-04-25T01:26:02.426335+0000","last_active":"2024-04-25T01:41:09.342741+0000","last_peered":"2024-04-25T01:41:09.342741+0000","last_clean":"2024-04-25T01:41:09.342741+0000","last_became_active":"2024-04-25T01:26:02.426024+0000","last_became_peered":"2024-04-25T01:26:02.426024+0000","last_unstale":"2024-04-25T01:41:09.342741+0000","last_undegraded":"2024-04-25T01:41:09.342741+0000","last_fullsized":"2024-04-25T01:41:09.342741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4139,"log_dups_size":0,"ondisk_log_size":4139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21818,"num_write_kb":267665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"26'5798","reported_seq":5828,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.869849+0000","last_change":"2024-04-25T01:25:53.372842+0000","last_active":"2024-04-25T01:40:58.869849+0000","last_peered":"2024-04-25T01:40:58.869849+0000","last_clean":"2024-04-25T01:40:58.869849+0000","last_became_active":"2024-04-25T01:25:53.372580+0000","last_became_peered":"2024-04-25T01:25:53.372580+0000","last_unstale":"2024-04-25T01:40:58.869849+0000","last_undegraded":"2024-04-25T01:40:58.869849+0000","last_fullsized":"2024-04-25T01:40:58.869849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5798,"log_dups_size":0,"ondisk_log_size":5798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:15.486990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30839,"num_write_kb":352989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"26'4687","reported_seq":4709,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.302280+0000","last_change":"2024-04-25T01:26:02.414632+0000","last_active":"2024-04-25T01:41:09.302280+0000","last_peered":"2024-04-25T01:41:09.302280+0000","last_clean":"2024-04-25T01:41:09.302280+0000","last_became_active":"2024-04-25T01:26:02.414388+0000","last_became_peered":"2024-04-25T01:26:02.414388+0000","last_unstale":"2024-04-25T01:41:09.302280+0000","last_undegraded":"2024-04-25T01:41:09.302280+0000","last_fullsized":"2024-04-25T01:41:09.302280+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4687,"log_dups_size":0,"ondisk_log_size":4687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:37.679604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23866,"num_write_kb":304526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"26'5655","reported_seq":5690,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.888926+0000","last_change":"2024-04-25T01:26:52.515807+0000","last_active":"2024-04-25T01:40:58.888926+0000","last_peered":"2024-04-25T01:40:58.888926+0000","last_clean":"2024-04-25T01:40:58.888926+0000","last_became_active":"2024-04-25T01:26:52.515504+0000","last_became_peered":"2024-04-25T01:26:52.515504+0000","last_unstale":"2024-04-25T01:40:58.888926+0000","last_undegraded":"2024-04-25T01:40:58.888926+0000","last_fullsized":"2024-04-25T01:40:58.888926+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5655,"log_dups_size":0,"ondisk_log_size":5655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:48.624287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29809,"num_write_kb":365295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"26'5769","reported_seq":5793,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.358590+0000","last_change":"2024-04-25T01:26:02.414366+0000","last_active":"2024-04-25T01:41:09.358590+0000","last_peered":"2024-04-25T01:41:09.358590+0000","last_clean":"2024-04-25T01:41:09.358590+0000","last_became_active":"2024-04-25T01:26:02.414061+0000","last_became_peered":"2024-04-25T01:26:02.414061+0000","last_unstale":"2024-04-25T01:41:09.358590+0000","last_undegraded":"2024-04-25T01:41:09.358590+0000","last_fullsized":"2024-04-25T01:41:09.358590+0000","mapping_epoch":22,"log_start":"26'500","ondisk_log_start":"26'500","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5269,"log_dups_size":0,"ondisk_log_size":5269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30404,"num_write_kb":379952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"26'5055","reported_seq":5086,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.844761+0000","last_change":"2024-04-25T01:25:53.374061+0000","last_active":"2024-04-25T01:40:58.844761+0000","last_peered":"2024-04-25T01:40:58.844761+0000","last_clean":"2024-04-25T01:40:58.844761+0000","last_became_active":"2024-04-25T01:25:53.373848+0000","last_became_peered":"2024-04-25T01:25:53.373848+0000","last_unstale":"2024-04-25T01:40:58.844761+0000","last_undegraded":"2024-04-25T01:40:58.844761+0000","last_fullsized":"2024-04-25T01:40:58.844761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5055,"log_dups_size":0,"ondisk_log_size":5055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25773,"num_write_kb":324544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"26'4195","reported_seq":4218,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.317465+0000","last_change":"2024-04-25T01:26:02.414620+0000","last_active":"2024-04-25T01:41:09.317465+0000","last_peered":"2024-04-25T01:41:09.317465+0000","last_clean":"2024-04-25T01:41:09.317465+0000","last_became_active":"2024-04-25T01:26:02.414326+0000","last_became_peered":"2024-04-25T01:26:02.414326+0000","last_unstale":"2024-04-25T01:41:09.317465+0000","last_undegraded":"2024-04-25T01:41:09.317465+0000","last_fullsized":"2024-04-25T01:41:09.317465+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4195,"log_dups_size":0,"ondisk_log_size":4195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:30.496089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21944,"num_write_kb":264053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"26'4639","reported_seq":4669,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.755355+0000","last_change":"2024-04-25T01:25:53.364698+0000","last_active":"2024-04-25T01:40:58.755355+0000","last_peered":"2024-04-25T01:40:58.755355+0000","last_clean":"2024-04-25T01:40:58.755355+0000","last_became_active":"2024-04-25T01:25:53.364435+0000","last_became_peered":"2024-04-25T01:25:53.364435+0000","last_unstale":"2024-04-25T01:40:58.755355+0000","last_undegraded":"2024-04-25T01:40:58.755355+0000","last_fullsized":"2024-04-25T01:40:58.755355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4639,"log_dups_size":0,"ondisk_log_size":4639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:29.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24873,"num_write_kb":296467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"26'4393","reported_seq":4417,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.341247+0000","last_change":"2024-04-25T01:26:02.418331+0000","last_active":"2024-04-25T01:41:09.341247+0000","last_peered":"2024-04-25T01:41:09.341247+0000","last_clean":"2024-04-25T01:41:09.341247+0000","last_became_active":"2024-04-25T01:26:02.418057+0000","last_became_peered":"2024-04-25T01:26:02.418057+0000","last_unstale":"2024-04-25T01:41:09.341247+0000","last_undegraded":"2024-04-25T01:41:09.341247+0000","last_fullsized":"2024-04-25T01:41:09.341247+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4393,"log_dups_size":0,"ondisk_log_size":4393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22950,"num_write_kb":276425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"26'3744","reported_seq":3760,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.794219+0000","last_change":"2024-04-25T01:26:52.514832+0000","last_active":"2024-04-25T01:40:58.794219+0000","last_peered":"2024-04-25T01:40:58.794219+0000","last_clean":"2024-04-25T01:40:58.794219+0000","last_became_active":"2024-04-25T01:26:52.514260+0000","last_became_peered":"2024-04-25T01:26:52.514260+0000","last_unstale":"2024-04-25T01:40:58.794219+0000","last_undegraded":"2024-04-25T01:40:58.794219+0000","last_fullsized":"2024-04-25T01:40:58.794219+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":3744,"log_dups_size":0,"ondisk_log_size":3744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19457,"num_write_kb":254077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3d","version":"26'4696","reported_seq":4719,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.334457+0000","last_change":"2024-04-25T01:26:02.422720+0000","last_active":"2024-04-25T01:41:09.334457+0000","last_peered":"2024-04-25T01:41:09.334457+0000","last_clean":"2024-04-25T01:41:09.334457+0000","last_became_active":"2024-04-25T01:26:02.421848+0000","last_became_peered":"2024-04-25T01:26:02.421848+0000","last_unstale":"2024-04-25T01:41:09.334457+0000","last_undegraded":"2024-04-25T01:41:09.334457+0000","last_fullsized":"2024-04-25T01:41:09.334457+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4696,"log_dups_size":0,"ondisk_log_size":4696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:39.086153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24119,"num_write_kb":305642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"26'5605","reported_seq":5635,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.863402+0000","last_change":"2024-04-25T01:25:53.367953+0000","last_active":"2024-04-25T01:40:58.863402+0000","last_peered":"2024-04-25T01:40:58.863402+0000","last_clean":"2024-04-25T01:40:58.863402+0000","last_became_active":"2024-04-25T01:25:53.367358+0000","last_became_peered":"2024-04-25T01:25:53.367358+0000","last_unstale":"2024-04-25T01:40:58.863402+0000","last_undegraded":"2024-04-25T01:40:58.863402+0000","last_fullsized":"2024-04-25T01:40:58.863402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5605,"log_dups_size":0,"ondisk_log_size":5605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:11.137477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27762,"num_write_kb":352787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"26'4305","reported_seq":4328,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.310446+0000","last_change":"2024-04-25T01:26:02.422300+0000","last_active":"2024-04-25T01:41:09.310446+0000","last_peered":"2024-04-25T01:41:09.310446+0000","last_clean":"2024-04-25T01:41:09.310446+0000","last_became_active":"2024-04-25T01:26:02.421702+0000","last_became_peered":"2024-04-25T01:26:02.421702+0000","last_unstale":"2024-04-25T01:41:09.310446+0000","last_undegraded":"2024-04-25T01:41:09.310446+0000","last_fullsized":"2024-04-25T01:41:09.310446+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4305,"log_dups_size":0,"ondisk_log_size":4305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21701,"num_write_kb":263037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"26'4577","reported_seq":4608,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.784384+0000","last_change":"2024-04-25T01:25:53.371513+0000","last_active":"2024-04-25T01:40:58.784384+0000","last_peered":"2024-04-25T01:40:58.784384+0000","last_clean":"2024-04-25T01:40:58.784384+0000","last_became_active":"2024-04-25T01:25:53.370421+0000","last_became_peered":"2024-04-25T01:25:53.370421+0000","last_unstale":"2024-04-25T01:40:58.784384+0000","last_undegraded":"2024-04-25T01:40:58.784384+0000","last_fullsized":"2024-04-25T01:40:58.784384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:03.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24195,"num_write_kb":281461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"26'5562","reported_seq":5585,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.305827+0000","last_change":"2024-04-25T01:26:02.411784+0000","last_active":"2024-04-25T01:41:09.305827+0000","last_peered":"2024-04-25T01:41:09.305827+0000","last_clean":"2024-04-25T01:41:09.305827+0000","last_became_active":"2024-04-25T01:26:02.411478+0000","last_became_peered":"2024-04-25T01:26:02.411478+0000","last_unstale":"2024-04-25T01:41:09.305827+0000","last_undegraded":"2024-04-25T01:41:09.305827+0000","last_fullsized":"2024-04-25T01:41:09.305827+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5562,"log_dups_size":0,"ondisk_log_size":5562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27685,"num_write_kb":352571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"26'5001","reported_seq":5032,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.762988+0000","last_change":"2024-04-25T01:25:53.363829+0000","last_active":"2024-04-25T01:40:58.762988+0000","last_peered":"2024-04-25T01:40:58.762988+0000","last_clean":"2024-04-25T01:40:58.762988+0000","last_became_active":"2024-04-25T01:25:53.363525+0000","last_became_peered":"2024-04-25T01:25:53.363525+0000","last_unstale":"2024-04-25T01:40:58.762988+0000","last_undegraded":"2024-04-25T01:40:58.762988+0000","last_fullsized":"2024-04-25T01:40:58.762988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5001,"log_dups_size":0,"ondisk_log_size":5001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.645328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25760,"num_write_kb":312110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"26'5076","reported_seq":5098,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.350154+0000","last_change":"2024-04-25T01:26:02.421208+0000","last_active":"2024-04-25T01:41:09.350154+0000","last_peered":"2024-04-25T01:41:09.350154+0000","last_clean":"2024-04-25T01:41:09.350154+0000","last_became_active":"2024-04-25T01:26:02.418796+0000","last_became_peered":"2024-04-25T01:26:02.418796+0000","last_unstale":"2024-04-25T01:41:09.350154+0000","last_undegraded":"2024-04-25T01:41:09.350154+0000","last_fullsized":"2024-04-25T01:41:09.350154+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5076,"log_dups_size":0,"ondisk_log_size":5076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:54.880061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25596,"num_write_kb":323780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"26'6025","reported_seq":6057,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.758534+0000","last_change":"2024-04-25T01:25:53.365057+0000","last_active":"2024-04-25T01:40:58.758534+0000","last_peered":"2024-04-25T01:40:58.758534+0000","last_clean":"2024-04-25T01:40:58.758534+0000","last_became_active":"2024-04-25T01:25:53.364793+0000","last_became_peered":"2024-04-25T01:25:53.364793+0000","last_unstale":"2024-04-25T01:40:58.758534+0000","last_undegraded":"2024-04-25T01:40:58.758534+0000","last_fullsized":"2024-04-25T01:40:58.758534+0000","mapping_epoch":18,"log_start":"26'200","ondisk_log_start":"26'200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5825,"log_dups_size":0,"ondisk_log_size":5825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31759,"num_write_kb":385508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"26'4493","reported_seq":4515,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.263776+0000","last_change":"2024-04-25T01:26:02.417606+0000","last_active":"2024-04-25T01:41:09.263776+0000","last_peered":"2024-04-25T01:41:09.263776+0000","last_clean":"2024-04-25T01:41:09.263776+0000","last_became_active":"2024-04-25T01:26:02.417350+0000","last_became_peered":"2024-04-25T01:26:02.417350+0000","last_unstale":"2024-04-25T01:41:09.263776+0000","last_undegraded":"2024-04-25T01:41:09.263776+0000","last_fullsized":"2024-04-25T01:41:09.263776+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4493,"log_dups_size":0,"ondisk_log_size":4493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23464,"num_write_kb":290699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"26'4350","reported_seq":4381,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.880219+0000","last_change":"2024-04-25T01:25:53.367762+0000","last_active":"2024-04-25T01:40:58.880219+0000","last_peered":"2024-04-25T01:40:58.880219+0000","last_clean":"2024-04-25T01:40:58.880219+0000","last_became_active":"2024-04-25T01:25:53.367383+0000","last_became_peered":"2024-04-25T01:25:53.367383+0000","last_unstale":"2024-04-25T01:40:58.880219+0000","last_undegraded":"2024-04-25T01:40:58.880219+0000","last_fullsized":"2024-04-25T01:40:58.880219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4350,"log_dups_size":0,"ondisk_log_size":4350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:05.589744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23070,"num_write_kb":268669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1533236,"num_write_kb":19254592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":294168,"ondisk_log_size":294168,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":17605243,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":39,"num_read_kb":112,"num_write":4172,"num_write_kb":18793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35274752,"data_stored":35210486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3499,"ondisk_log_size":3499,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1596536,"num_write_kb":19509893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305948,"ondisk_log_size":305948,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":17847589,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":41,"num_read_kb":115,"num_write":4224,"num_write_kb":19059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35758080,"data_stored":35695178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3536,"ondisk_log_size":3536,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575041,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1217764,"kb_used_data":13148,"kb_used_omap":1,"kb_used_meta":1204606,"kb_avail":93154076,"statfs":{"total":96636764160,"available":95389773824,"internally_reserved":0,"allocated":13463552,"data_stored":13287019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":1233517002},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575038,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":780520,"kb_used_data":9440,"kb_used_omap":1,"kb_used_meta":771070,"kb_avail":93591320,"statfs":{"total":96636764160,"available":95837511680,"internally_reserved":0,"allocated":9666560,"data_stored":9483786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":789576134},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575039,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1059708,"kb_used_data":4468,"kb_used_omap":1,"kb_used_meta":1055230,"kb_avail":93312132,"statfs":{"total":96636764160,"available":95551623168,"internally_reserved":0,"allocated":4575232,"data_stored":4396213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1080555973},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575039,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1089176,"kb_used_data":10256,"kb_used_omap":1,"kb_used_meta":1078910,"kb_avail":93282664,"statfs":{"total":96636764160,"available":95521447936,"internally_reserved":0,"allocated":10502144,"data_stored":10309285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1104804293},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575040,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1161176,"kb_used_data":17488,"kb_used_omap":1,"kb_used_meta":1143678,"kb_avail":93210664,"statfs":{"total":96636764160,"available":95447719936,"internally_reserved":0,"allocated":17907712,"data_stored":17718206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1171126727},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575040,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":729908,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":729534,"kb_avail":93641932,"statfs":{"total":96636764160,"available":95889338368,"internally_reserved":0,"allocated":372736,"data_stored":201217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":747043269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575040,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":888620,"kb_used_data":8484,"kb_used_omap":1,"kb_used_meta":880126,"kb_avail":93483220,"statfs":{"total":96636764160,"available":95726817280,"internally_reserved":0,"allocated":8687616,"data_stored":8502529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":901249477},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575038,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":919656,"kb_used_data":9056,"kb_used_omap":1,"kb_used_meta":910590,"kb_avail":93452184,"statfs":{"total":96636764160,"available":95695036416,"internally_reserved":0,"allocated":9273344,"data_stored":9092681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":932444613},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":88080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8486912,"data_stored":8477360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5189632,"data_stored":5175253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5185536,"data_stored":5175799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8389302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9142272,"data_stored":9127779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5033984,"data_stored":5020965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4292608,"data_stored":4283056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:41:21.843 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.843 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.843 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.843 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.843 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.843 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.844 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.844 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.844 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.844 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.844 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.844 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.844 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.844 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.844 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.844 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.845 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.845 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.845 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.845 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.845 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.845 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.845 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.845 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.845 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.846 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.846 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.846 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.846 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.846 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.846 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.846 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.846 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.846 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.847 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.847 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.847 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.847 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.847 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.847 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.847 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.847 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.847 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.848 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.848 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.848 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.848 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.848 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.848 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.848 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.848 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.848 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.849 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.849 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.849 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.849 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.849 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.849 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.849 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.849 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.850 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.850 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T01:25:40.688097+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=40, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.850 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.850 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.850 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.850 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.850 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.850 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.850 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.851 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.851 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.851 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.851 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.851 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.851 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.851 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.851 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.851 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.852 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.852 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.852 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.852 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.852 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.852 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.852 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.852 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.852 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.852 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.853 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.853 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.853 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.853 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.853 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.853 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.853 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.853 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.853 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.853 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.854 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.854 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.854 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.854 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.854 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.854 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.854 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.854 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.854 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.855 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.855 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.855 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.855 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.855 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.855 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.855 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.855 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.855 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.856 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.856 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.856 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.856 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.856 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.856 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.856 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.856 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.856 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.856 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-25T01:26:00.384370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.857 INFO:tasks.ceph:pgid 5.26 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.857 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.857 INFO:tasks.ceph:pgid 5.27 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.857 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.857 INFO:tasks.ceph:pgid 5.24 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.857 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.857 INFO:tasks.ceph:pgid 5.25 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.857 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.857 INFO:tasks.ceph:pgid 5.22 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.858 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.858 INFO:tasks.ceph:pgid 5.23 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.858 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.858 INFO:tasks.ceph:pgid 5.20 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.858 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.858 INFO:tasks.ceph:pgid 5.21 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.858 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.858 INFO:tasks.ceph:pgid 5.2e last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.858 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.859 INFO:tasks.ceph:pgid 5.2f last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.859 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.859 INFO:tasks.ceph:pgid 5.2c last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.859 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.859 INFO:tasks.ceph:pgid 5.2d last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.859 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.859 INFO:tasks.ceph:pgid 5.2a last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.859 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.859 INFO:tasks.ceph:pgid 5.2b last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.859 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.860 INFO:tasks.ceph:pgid 5.28 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.860 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.860 INFO:tasks.ceph:pgid 5.29 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.860 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.860 INFO:tasks.ceph:pgid 5.36 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.860 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.860 INFO:tasks.ceph:pgid 5.37 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.860 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.860 INFO:tasks.ceph:pgid 5.34 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.861 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.861 INFO:tasks.ceph:pgid 5.35 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.861 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.861 INFO:tasks.ceph:pgid 5.32 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.861 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.861 INFO:tasks.ceph:pgid 5.33 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.861 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.861 INFO:tasks.ceph:pgid 5.30 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.861 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.862 INFO:tasks.ceph:pgid 5.31 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.862 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.862 INFO:tasks.ceph:pgid 5.3e last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.862 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.862 INFO:tasks.ceph:pgid 5.3f last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.862 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.862 INFO:tasks.ceph:pgid 5.3c last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.862 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.862 INFO:tasks.ceph:pgid 5.3d last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.863 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.863 INFO:tasks.ceph:pgid 5.3a last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.863 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.863 INFO:tasks.ceph:pgid 5.3b last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.863 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.863 INFO:tasks.ceph:pgid 5.38 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.863 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.863 INFO:tasks.ceph:pgid 5.39 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.863 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:21.864 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T01:41:30.110 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:41:30.113 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:41:41.865 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:41:41.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.995+0000 7f6d14cb3640 1 Processor -- start 2024-04-25T01:41:41.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.995+0000 7f6d14cb3640 1 -- start start 2024-04-25T01:41:41.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.996+0000 7f6d14cb3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6d101536c0 0x7f6d100c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:41.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.996+0000 7f6d14cb3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6d10071db0 0x7f6d100721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:41.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.996+0000 7f6d14cb3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d1006e0f0 con 0x7f6d10072d80 2024-04-25T01:41:41.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.996+0000 7f6d14cb3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f6d100c2180 con 0x7f6d10071db0 2024-04-25T01:41:41.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.996+0000 7f6d14cb3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6d100c2460 con 0x7f6d101536c0 2024-04-25T01:41:41.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.996+0000 7f6d0ed9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6d10071db0 0x7f6d100721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:41.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.996+0000 7f6d0ed9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6d10071db0 0x7f6d100721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34462/0 (socket says 172.21.15.112:34462) 2024-04-25T01:41:41.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.996+0000 7f6d0ed9d640 1 -- 172.21.15.112:0/3495451978 learned_addr learned my addr 172.21.15.112:0/3495451978 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:41.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.996+0000 7f6d07fff640 1 --2- 172.21.15.112:0/3495451978 >> v2:172.21.15.187:3300/0 conn(0x7f6d101536c0 0x7f6d100c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:41.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.996+0000 7f6d0ed9d640 1 -- 172.21.15.112:0/3495451978 >> v2:172.21.15.187:3300/0 conn(0x7f6d101536c0 msgr2=0x7f6d100c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:41.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.996+0000 7f6d0ed9d640 1 --2- 172.21.15.112:0/3495451978 >> v2:172.21.15.187:3300/0 conn(0x7f6d101536c0 0x7f6d100c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:41.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.996+0000 7f6d0ed9d640 1 -- 172.21.15.112:0/3495451978 >> v1:172.21.15.187:6789/0 conn(0x7f6d10072d80 legacy=0x7f6d1014d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:41:41.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.996+0000 7f6d0ed9d640 1 -- 172.21.15.112:0/3495451978 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d100c2740 con 0x7f6d10071db0 2024-04-25T01:41:41.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.997+0000 7f6d0ed9d640 1 --2- 172.21.15.112:0/3495451978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d10071db0 0x7f6d100721c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6d100ac5e0 tx=0x7f6d0802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894236b98f271675 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:41.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.997+0000 7f6d0dd9b640 1 -- 172.21.15.112:0/3495451978 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08036070 con 0x7f6d10071db0 2024-04-25T01:41:41.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.997+0000 7f6d0dd9b640 1 -- 172.21.15.112:0/3495451978 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d08031600 con 0x7f6d10071db0 2024-04-25T01:41:41.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.997+0000 7f6d0dd9b640 1 -- 172.21.15.112:0/3495451978 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d080319d0 con 0x7f6d10071db0 2024-04-25T01:41:41.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.998+0000 7f6d14cb3640 1 -- 172.21.15.112:0/3495451978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d10071db0 msgr2=0x7f6d100721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:41.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.998+0000 7f6d14cb3640 1 --2- 172.21.15.112:0/3495451978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d10071db0 0x7f6d100721c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6d100ac5e0 tx=0x7f6d0802a6f0 comp rx=0 tx=0).stop 2024-04-25T01:41:41.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.998+0000 7f6d14cb3640 1 -- 172.21.15.112:0/3495451978 shutdown_connections 2024-04-25T01:41:41.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.998+0000 7f6d14cb3640 1 --2- 172.21.15.112:0/3495451978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d10071db0 0x7f6d100721c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:41.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.998+0000 7f6d14cb3640 1 --2- 172.21.15.112:0/3495451978 >> v2:172.21.15.187:3300/0 conn(0x7f6d101536c0 0x7f6d100c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:41.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.998+0000 7f6d14cb3640 1 -- 172.21.15.112:0/3495451978 >> 172.21.15.112:0/3495451978 conn(0x7f6d1006b160 msgr2=0x7f6d1006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:41.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.998+0000 7f6d14cb3640 1 -- 172.21.15.112:0/3495451978 shutdown_connections 2024-04-25T01:41:41.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.998+0000 7f6d14cb3640 1 -- 172.21.15.112:0/3495451978 wait complete. 2024-04-25T01:41:41.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.999+0000 7f6d14cb3640 1 Processor -- start 2024-04-25T01:41:41.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.999+0000 7f6d14cb3640 1 -- start start 2024-04-25T01:41:41.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:41.999+0000 7f6d14cb3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d10071db0 0x7f6d1014dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:41.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d14cb3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d10072d80 0x7f6d101e8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:41.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d14cb3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d101536c0 0x7f6d101e9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:41.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d14cb3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1014da50 con 0x7f6d101536c0 2024-04-25T01:41:41.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d14cb3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6d10071890 con 0x7f6d10071db0 2024-04-25T01:41:41.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d14cb3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6d1014dbd0 con 0x7f6d10072d80 2024-04-25T01:41:41.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d0ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d101536c0 0x7f6d101e9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:41.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d0ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d101536c0 0x7f6d101e9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34472/0 (socket says 172.21.15.112:34472) 2024-04-25T01:41:41.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d0ed9d640 1 -- 172.21.15.112:0/1492272544 learned_addr learned my addr 172.21.15.112:0/1492272544 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:41:41.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d0e59c640 1 --2- 172.21.15.112:0/1492272544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d10071db0 0x7f6d1014dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:41.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d07fff640 1 --2- 172.21.15.112:0/1492272544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d10072d80 0x7f6d101e8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:41.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d0ed9d640 1 -- 172.21.15.112:0/1492272544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d10072d80 msgr2=0x7f6d101e8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:41.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d0ed9d640 1 --2- 172.21.15.112:0/1492272544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d10072d80 0x7f6d101e8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:41.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d0ed9d640 1 -- 172.21.15.112:0/1492272544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d10071db0 msgr2=0x7f6d1014dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:41.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d0ed9d640 1 --2- 172.21.15.112:0/1492272544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d10071db0 0x7f6d1014dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:41.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d0ed9d640 1 -- 172.21.15.112:0/1492272544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d10150a80 con 0x7f6d101536c0 2024-04-25T01:41:41.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d07fff640 1 --2- 172.21.15.112:0/1492272544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d10072d80 0x7f6d101e8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:41:41.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d0ed9d640 1 --2- 172.21.15.112:0/1492272544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d101536c0 0x7f6d101e9400 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009f40 tx=0x7f6d08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:41.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d06ffd640 1 -- 172.21.15.112:0/1492272544 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08036050 con 0x7f6d101536c0 2024-04-25T01:41:41.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d06ffd640 1 -- 172.21.15.112:0/1492272544 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d08042070 con 0x7f6d101536c0 2024-04-25T01:41:41.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d06ffd640 1 -- 172.21.15.112:0/1492272544 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08009510 con 0x7f6d101536c0 2024-04-25T01:41:41.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.000+0000 7f6d14cb3640 1 -- 172.21.15.112:0/1492272544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d101501e0 con 0x7f6d101536c0 2024-04-25T01:41:41.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.001+0000 7f6d14cb3640 1 -- 172.21.15.112:0/1492272544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d101507a0 con 0x7f6d101536c0 2024-04-25T01:41:41.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.002+0000 7f6d14cb3640 1 -- 172.21.15.112:0/1492272544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d100682a0 con 0x7f6d101536c0 2024-04-25T01:41:41.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.002+0000 7f6d06ffd640 1 -- 172.21.15.112:0/1492272544 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d08007bd0 con 0x7f6d101536c0 2024-04-25T01:41:41.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.002+0000 7f6d06ffd640 1 --2- 172.21.15.112:0/1492272544 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6cf006be30 0x7f6cf006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:41.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.002+0000 7f6d06ffd640 1 -- 172.21.15.112:0/1492272544 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f6d080ac070 con 0x7f6d101536c0 2024-04-25T01:41:41.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.002+0000 7f6d0e59c640 1 --2- 172.21.15.112:0/1492272544 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6cf006be30 0x7f6cf006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:41.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.003+0000 7f6d0e59c640 1 --2- 172.21.15.112:0/1492272544 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6cf006be30 0x7f6cf006e2f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6d00004f60 tx=0x7f6d00007e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:41.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.007+0000 7f6d06ffd640 1 -- 172.21.15.112:0/1492272544 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6d1006e0f0 con 0x7f6d101536c0 2024-04-25T01:41:42.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.181+0000 7f6d14cb3640 1 -- 172.21.15.112:0/1492272544 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6d10075980 con 0x7f6cf006be30 2024-04-25T01:41:42.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.191+0000 7f6d06ffd640 1 -- 172.21.15.112:0/1492272544 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+492865 (secure 0 0 0) 0x7f6d00003940 con 0x7f6cf006be30 2024-04-25T01:41:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:41:42.188 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T01:41:42.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.196+0000 7f6d14cb3640 1 -- 172.21.15.112:0/1492272544 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6cf006be30 msgr2=0x7f6cf006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:42.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.196+0000 7f6d14cb3640 1 --2- 172.21.15.112:0/1492272544 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6cf006be30 0x7f6cf006e2f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6d00004f60 tx=0x7f6d00007e30 comp rx=0 tx=0).stop 2024-04-25T01:41:42.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.196+0000 7f6d14cb3640 1 -- 172.21.15.112:0/1492272544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d101536c0 msgr2=0x7f6d101e9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:42.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.196+0000 7f6d14cb3640 1 --2- 172.21.15.112:0/1492272544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d101536c0 0x7f6d101e9400 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009f40 tx=0x7f6d08004a40 comp rx=0 tx=0).stop 2024-04-25T01:41:42.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.196+0000 7f6d14cb3640 1 -- 172.21.15.112:0/1492272544 shutdown_connections 2024-04-25T01:41:42.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.196+0000 7f6d14cb3640 1 --2- 172.21.15.112:0/1492272544 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6cf006be30 0x7f6cf006e2f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:42.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.196+0000 7f6d14cb3640 1 --2- 172.21.15.112:0/1492272544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d101536c0 0x7f6d101e9400 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:42.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.196+0000 7f6d14cb3640 1 --2- 172.21.15.112:0/1492272544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d10072d80 0x7f6d101e8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:42.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.196+0000 7f6d14cb3640 1 --2- 172.21.15.112:0/1492272544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d10071db0 0x7f6d1014dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:42.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.196+0000 7f6d14cb3640 1 -- 172.21.15.112:0/1492272544 >> 172.21.15.112:0/1492272544 conn(0x7f6d1006b160 msgr2=0x7f6d10151530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:42.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.196+0000 7f6d14cb3640 1 -- 172.21.15.112:0/1492272544 shutdown_connections 2024-04-25T01:41:42.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:41:42.196+0000 7f6d14cb3640 1 -- 172.21.15.112:0/1492272544 wait complete. 2024-04-25T01:41:42.194 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":499,"stamp":"2024-04-25T01:41:40.922095+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":36043200,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":196,"num_read_kb":327,"num_write":3138327,"num_write_kb":38804753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10000,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":607320,"ondisk_log_size":607320,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":7846528,"kb_used_data":72704,"kb_used_omap":12,"kb_used_meta":7773747,"kb_avail":747128192,"statfs":{"total":773094113280,"available":765059268608,"internally_reserved":0,"allocated":74448896,"data_stored":72990936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":7960317488},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003786"},"pg_stats":[{"pgid":"5.18","version":"26'4099","reported_seq":4130,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:36.129609+0000","last_change":"2024-04-25T01:41:36.129637+0000","last_active":"2024-04-25T01:41:36.129609+0000","last_peered":"2024-04-25T01:41:36.129609+0000","last_clean":"2024-04-25T01:41:36.129609+0000","last_became_active":"2024-04-25T01:26:02.411176+0000","last_became_peered":"2024-04-25T01:26:02.411176+0000","last_unstale":"2024-04-25T01:41:36.129609+0000","last_undegraded":"2024-04-25T01:41:36.129609+0000","last_fullsized":"2024-04-25T01:41:36.129609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4099","last_scrub_stamp":"2024-04-25T01:41:36.129495+0000","last_deep_scrub":"26'4099","last_deep_scrub_stamp":"2024-04-25T01:41:36.129495+0000","last_clean_scrub_stamp":"2024-04-25T01:41:36.129495+0000","objects_scrubbed":0,"log_size":4099,"log_dups_size":0,"ondisk_log_size":4099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:48.901163+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21931,"num_write_kb":288545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"26'661","reported_seq":703,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:24.733384+0000","last_change":"2024-04-25T01:41:24.733415+0000","last_active":"2024-04-25T01:41:24.733384+0000","last_peered":"2024-04-25T01:41:24.733384+0000","last_clean":"2024-04-25T01:41:24.733384+0000","last_became_active":"2024-04-25T01:25:52.353649+0000","last_became_peered":"2024-04-25T01:25:52.353649+0000","last_unstale":"2024-04-25T01:41:24.733384+0000","last_undegraded":"2024-04-25T01:41:24.733384+0000","last_fullsized":"2024-04-25T01:41:24.733384+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'661","last_scrub_stamp":"2024-04-25T01:41:24.733307+0000","last_deep_scrub":"26'661","last_deep_scrub_stamp":"2024-04-25T01:41:24.733307+0000","last_clean_scrub_stamp":"2024-04-25T01:41:24.733307+0000","objects_scrubbed":3,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:12.604209+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":4468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"26'5333","reported_seq":5373,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:33.921170+0000","last_change":"2024-04-25T01:41:33.921198+0000","last_active":"2024-04-25T01:41:33.921170+0000","last_peered":"2024-04-25T01:41:33.921170+0000","last_clean":"2024-04-25T01:41:33.921170+0000","last_became_active":"2024-04-25T01:25:53.363409+0000","last_became_peered":"2024-04-25T01:25:53.363409+0000","last_unstale":"2024-04-25T01:41:33.921170+0000","last_undegraded":"2024-04-25T01:41:33.921170+0000","last_fullsized":"2024-04-25T01:41:33.921170+0000","mapping_epoch":18,"log_start":"26'100","ondisk_log_start":"26'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5333","last_scrub_stamp":"2024-04-25T01:41:33.921067+0000","last_deep_scrub":"26'5333","last_deep_scrub_stamp":"2024-04-25T01:41:33.921067+0000","last_clean_scrub_stamp":"2024-04-25T01:41:33.921067+0000","objects_scrubbed":0,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:09.673744+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28405,"num_write_kb":331018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"26'3","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:23.833381+0000","last_change":"2024-04-25T01:41:23.833432+0000","last_active":"2024-04-25T01:41:23.833381+0000","last_peered":"2024-04-25T01:41:23.833381+0000","last_clean":"2024-04-25T01:41:23.833381+0000","last_became_active":"2024-04-25T01:26:01.406855+0000","last_became_peered":"2024-04-25T01:26:01.406855+0000","last_unstale":"2024-04-25T01:41:23.833381+0000","last_undegraded":"2024-04-25T01:41:23.833381+0000","last_fullsized":"2024-04-25T01:41:23.833381+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:23.833281+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:23.833281+0000","last_clean_scrub_stamp":"2024-04-25T01:41:23.833281+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:10.367070+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"26'5328","reported_seq":5359,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:39.621502+0000","last_change":"2024-04-25T01:41:39.621550+0000","last_active":"2024-04-25T01:41:39.621502+0000","last_peered":"2024-04-25T01:41:39.621502+0000","last_clean":"2024-04-25T01:41:39.621502+0000","last_became_active":"2024-04-25T01:26:02.425071+0000","last_became_peered":"2024-04-25T01:26:02.425071+0000","last_unstale":"2024-04-25T01:41:39.621502+0000","last_undegraded":"2024-04-25T01:41:39.621502+0000","last_fullsized":"2024-04-25T01:41:39.621502+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5328","last_scrub_stamp":"2024-04-25T01:41:39.621385+0000","last_deep_scrub":"26'5328","last_deep_scrub_stamp":"2024-04-25T01:41:39.621385+0000","last_clean_scrub_stamp":"2024-04-25T01:41:39.621385+0000","objects_scrubbed":0,"log_size":5328,"log_dups_size":0,"ondisk_log_size":5328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:24.762450+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27477,"num_write_kb":347746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"26'8","reported_seq":50,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:35.909431+0000","last_change":"2024-04-25T01:41:35.909456+0000","last_active":"2024-04-25T01:41:35.909431+0000","last_peered":"2024-04-25T01:41:35.909431+0000","last_clean":"2024-04-25T01:41:35.909431+0000","last_became_active":"2024-04-25T01:25:52.348931+0000","last_became_peered":"2024-04-25T01:25:52.348931+0000","last_unstale":"2024-04-25T01:41:35.909431+0000","last_undegraded":"2024-04-25T01:41:35.909431+0000","last_fullsized":"2024-04-25T01:41:35.909431+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'8","last_scrub_stamp":"2024-04-25T01:41:35.909325+0000","last_deep_scrub":"26'8","last_deep_scrub_stamp":"2024-04-25T01:41:35.909325+0000","last_clean_scrub_stamp":"2024-04-25T01:41:35.909325+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:08.452523+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"26'5837","reported_seq":5877,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:20.667334+0000","last_change":"2024-04-25T01:41:20.667358+0000","last_active":"2024-04-25T01:41:20.667334+0000","last_peered":"2024-04-25T01:41:20.667334+0000","last_clean":"2024-04-25T01:41:20.667334+0000","last_became_active":"2024-04-25T01:25:53.373755+0000","last_became_peered":"2024-04-25T01:25:53.373755+0000","last_unstale":"2024-04-25T01:41:20.667334+0000","last_undegraded":"2024-04-25T01:41:20.667334+0000","last_fullsized":"2024-04-25T01:41:20.667334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5837","last_scrub_stamp":"2024-04-25T01:41:20.667241+0000","last_deep_scrub":"26'5837","last_deep_scrub_stamp":"2024-04-25T01:41:20.667241+0000","last_clean_scrub_stamp":"2024-04-25T01:41:20.667241+0000","objects_scrubbed":0,"log_size":5837,"log_dups_size":0,"ondisk_log_size":5837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:49.049409+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30942,"num_write_kb":369940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"26'3","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:35.153544+0000","last_change":"2024-04-25T01:41:35.153570+0000","last_active":"2024-04-25T01:41:35.153544+0000","last_peered":"2024-04-25T01:41:35.153544+0000","last_clean":"2024-04-25T01:41:35.153544+0000","last_became_active":"2024-04-25T01:26:01.405994+0000","last_became_peered":"2024-04-25T01:26:01.405994+0000","last_unstale":"2024-04-25T01:41:35.153544+0000","last_undegraded":"2024-04-25T01:41:35.153544+0000","last_fullsized":"2024-04-25T01:41:35.153544+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:35.153451+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:35.153451+0000","last_clean_scrub_stamp":"2024-04-25T01:41:35.153451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:02.234784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"26'4188","reported_seq":4211,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.336635+0000","last_change":"2024-04-25T01:26:02.427695+0000","last_active":"2024-04-25T01:41:09.336635+0000","last_peered":"2024-04-25T01:41:09.336635+0000","last_clean":"2024-04-25T01:41:09.336635+0000","last_became_active":"2024-04-25T01:26:02.427443+0000","last_became_peered":"2024-04-25T01:26:02.427443+0000","last_unstale":"2024-04-25T01:41:09.336635+0000","last_undegraded":"2024-04-25T01:41:09.336635+0000","last_fullsized":"2024-04-25T01:41:09.336635+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4188,"log_dups_size":0,"ondisk_log_size":4188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20869,"num_write_kb":259707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"26'12","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:37.752408+0000","last_change":"2024-04-25T01:41:37.752439+0000","last_active":"2024-04-25T01:41:37.752408+0000","last_peered":"2024-04-25T01:41:37.752408+0000","last_clean":"2024-04-25T01:41:37.752408+0000","last_became_active":"2024-04-25T01:25:52.355822+0000","last_became_peered":"2024-04-25T01:25:52.355822+0000","last_unstale":"2024-04-25T01:41:37.752408+0000","last_undegraded":"2024-04-25T01:41:37.752408+0000","last_fullsized":"2024-04-25T01:41:37.752408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'12","last_scrub_stamp":"2024-04-25T01:41:37.752319+0000","last_deep_scrub":"26'12","last_deep_scrub_stamp":"2024-04-25T01:41:37.752319+0000","last_clean_scrub_stamp":"2024-04-25T01:41:37.752319+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:47.278639+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"26'3747","reported_seq":3786,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:29.378134+0000","last_change":"2024-04-25T01:41:29.378161+0000","last_active":"2024-04-25T01:41:29.378134+0000","last_peered":"2024-04-25T01:41:29.378134+0000","last_clean":"2024-04-25T01:41:29.378134+0000","last_became_active":"2024-04-25T01:25:53.369721+0000","last_became_peered":"2024-04-25T01:25:53.369721+0000","last_unstale":"2024-04-25T01:41:29.378134+0000","last_undegraded":"2024-04-25T01:41:29.378134+0000","last_fullsized":"2024-04-25T01:41:29.378134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3747","last_scrub_stamp":"2024-04-25T01:41:29.378020+0000","last_deep_scrub":"26'3747","last_deep_scrub_stamp":"2024-04-25T01:41:29.378020+0000","last_clean_scrub_stamp":"2024-04-25T01:41:29.378020+0000","objects_scrubbed":0,"log_size":3747,"log_dups_size":0,"ondisk_log_size":3747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:36:39.101213+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18978,"num_write_kb":223593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"26'628","reported_seq":661,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:25.141668+0000","last_change":"2024-04-25T01:41:25.141715+0000","last_active":"2024-04-25T01:41:25.141668+0000","last_peered":"2024-04-25T01:41:25.141668+0000","last_clean":"2024-04-25T01:41:25.141668+0000","last_became_active":"2024-04-25T01:26:01.407909+0000","last_became_peered":"2024-04-25T01:26:01.407909+0000","last_unstale":"2024-04-25T01:41:25.141668+0000","last_undegraded":"2024-04-25T01:41:25.141668+0000","last_fullsized":"2024-04-25T01:41:25.141668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'628","last_scrub_stamp":"2024-04-25T01:41:25.141559+0000","last_deep_scrub":"26'628","last_deep_scrub_stamp":"2024-04-25T01:41:25.141559+0000","last_clean_scrub_stamp":"2024-04-25T01:41:25.141559+0000","objects_scrubbed":1,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:33.537623+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":625,"num_write_kb":4235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"26'3441","reported_seq":3464,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.298514+0000","last_change":"2024-04-25T01:26:02.424615+0000","last_active":"2024-04-25T01:41:09.298514+0000","last_peered":"2024-04-25T01:41:09.298514+0000","last_clean":"2024-04-25T01:41:09.298514+0000","last_became_active":"2024-04-25T01:26:02.424325+0000","last_became_peered":"2024-04-25T01:26:02.424325+0000","last_unstale":"2024-04-25T01:41:09.298514+0000","last_undegraded":"2024-04-25T01:41:09.298514+0000","last_fullsized":"2024-04-25T01:41:09.298514+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3441,"log_dups_size":0,"ondisk_log_size":3441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17582,"num_write_kb":230191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"26'3","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:55.676388+0000","last_change":"2024-04-25T01:25:52.359139+0000","last_active":"2024-04-25T01:40:55.676388+0000","last_peered":"2024-04-25T01:40:55.676388+0000","last_clean":"2024-04-25T01:40:55.676388+0000","last_became_active":"2024-04-25T01:25:52.358841+0000","last_became_peered":"2024-04-25T01:25:52.358841+0000","last_unstale":"2024-04-25T01:40:55.676388+0000","last_undegraded":"2024-04-25T01:40:55.676388+0000","last_fullsized":"2024-04-25T01:40:55.676388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:51.329498+0000","last_clean_scrub_stamp":"2024-04-25T01:25:51.329498+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"26'5828","reported_seq":5859,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.878956+0000","last_change":"2024-04-25T01:25:53.368739+0000","last_active":"2024-04-25T01:40:58.878956+0000","last_peered":"2024-04-25T01:40:58.878956+0000","last_clean":"2024-04-25T01:40:58.878956+0000","last_became_active":"2024-04-25T01:25:53.368470+0000","last_became_peered":"2024-04-25T01:25:53.368470+0000","last_unstale":"2024-04-25T01:40:58.878956+0000","last_undegraded":"2024-04-25T01:40:58.878956+0000","last_fullsized":"2024-04-25T01:40:58.878956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5828,"log_dups_size":0,"ondisk_log_size":5828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30463,"num_write_kb":367857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"26'65","reported_seq":98,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:27.126064+0000","last_change":"2024-04-25T01:41:27.126092+0000","last_active":"2024-04-25T01:41:27.126064+0000","last_peered":"2024-04-25T01:41:27.126064+0000","last_clean":"2024-04-25T01:41:27.126064+0000","last_became_active":"2024-04-25T01:26:01.407655+0000","last_became_peered":"2024-04-25T01:26:01.407655+0000","last_unstale":"2024-04-25T01:41:27.126064+0000","last_undegraded":"2024-04-25T01:41:27.126064+0000","last_fullsized":"2024-04-25T01:41:27.126064+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'65","last_scrub_stamp":"2024-04-25T01:41:27.125966+0000","last_deep_scrub":"26'65","last_deep_scrub_stamp":"2024-04-25T01:41:27.125966+0000","last_clean_scrub_stamp":"2024-04-25T01:41:27.125966+0000","objects_scrubbed":1,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:40.858022+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":738477,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61,"num_write_kb":764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"26'4794","reported_seq":4825,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:37.603745+0000","last_change":"2024-04-25T01:41:37.603772+0000","last_active":"2024-04-25T01:41:37.603745+0000","last_peered":"2024-04-25T01:41:37.603745+0000","last_clean":"2024-04-25T01:41:37.603745+0000","last_became_active":"2024-04-25T01:26:02.417524+0000","last_became_peered":"2024-04-25T01:26:02.417524+0000","last_unstale":"2024-04-25T01:41:37.603745+0000","last_undegraded":"2024-04-25T01:41:37.603745+0000","last_fullsized":"2024-04-25T01:41:37.603745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4794","last_scrub_stamp":"2024-04-25T01:41:37.603630+0000","last_deep_scrub":"26'4794","last_deep_scrub_stamp":"2024-04-25T01:41:37.603630+0000","last_clean_scrub_stamp":"2024-04-25T01:41:37.603630+0000","objects_scrubbed":0,"log_size":4794,"log_dups_size":0,"ondisk_log_size":4794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:47.129951+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24624,"num_write_kb":311668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"26'4532","reported_seq":4572,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:22.348956+0000","last_change":"2024-04-25T01:41:22.348983+0000","last_active":"2024-04-25T01:41:22.348956+0000","last_peered":"2024-04-25T01:41:22.348956+0000","last_clean":"2024-04-25T01:41:22.348956+0000","last_became_active":"2024-04-25T01:25:53.372064+0000","last_became_peered":"2024-04-25T01:25:53.372064+0000","last_unstale":"2024-04-25T01:41:22.348956+0000","last_undegraded":"2024-04-25T01:41:22.348956+0000","last_fullsized":"2024-04-25T01:41:22.348956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4532","last_scrub_stamp":"2024-04-25T01:41:22.348835+0000","last_deep_scrub":"26'4532","last_deep_scrub_stamp":"2024-04-25T01:41:22.348835+0000","last_clean_scrub_stamp":"2024-04-25T01:41:22.348835+0000","objects_scrubbed":0,"log_size":4532,"log_dups_size":0,"ondisk_log_size":4532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:50.731003+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23642,"num_write_kb":279226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"26'1102","reported_seq":1144,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:39.756674+0000","last_change":"2024-04-25T01:41:39.756674+0000","last_active":"2024-04-25T01:41:39.756674+0000","last_peered":"2024-04-25T01:41:39.756674+0000","last_clean":"2024-04-25T01:41:39.756674+0000","last_became_active":"2024-04-25T01:25:52.358318+0000","last_became_peered":"2024-04-25T01:25:52.358318+0000","last_unstale":"2024-04-25T01:41:39.756674+0000","last_undegraded":"2024-04-25T01:41:39.756674+0000","last_fullsized":"2024-04-25T01:41:39.756674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1102","last_scrub_stamp":"2024-04-25T01:41:39.756541+0000","last_deep_scrub":"26'1102","last_deep_scrub_stamp":"2024-04-25T01:41:39.756541+0000","last_clean_scrub_stamp":"2024-04-25T01:41:39.756541+0000","objects_scrubbed":2,"log_size":1102,"log_dups_size":0,"ondisk_log_size":1102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:24.897606+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1091,"num_write_kb":4419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"26'3","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:34.171234+0000","last_change":"2024-04-25T01:41:34.171264+0000","last_active":"2024-04-25T01:41:34.171234+0000","last_peered":"2024-04-25T01:41:34.171234+0000","last_clean":"2024-04-25T01:41:34.171234+0000","last_became_active":"2024-04-25T01:26:01.400266+0000","last_became_peered":"2024-04-25T01:26:01.400266+0000","last_unstale":"2024-04-25T01:41:34.171234+0000","last_undegraded":"2024-04-25T01:41:34.171234+0000","last_fullsized":"2024-04-25T01:41:34.171234+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:34.171147+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:34.171147+0000","last_clean_scrub_stamp":"2024-04-25T01:41:34.171147+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:02.927773+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"26'4554","reported_seq":4585,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:34.091615+0000","last_change":"2024-04-25T01:41:34.091671+0000","last_active":"2024-04-25T01:41:34.091615+0000","last_peered":"2024-04-25T01:41:34.091615+0000","last_clean":"2024-04-25T01:41:34.091615+0000","last_became_active":"2024-04-25T01:26:02.420001+0000","last_became_peered":"2024-04-25T01:26:02.420001+0000","last_unstale":"2024-04-25T01:41:34.091615+0000","last_undegraded":"2024-04-25T01:41:34.091615+0000","last_fullsized":"2024-04-25T01:41:34.091615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4554","last_scrub_stamp":"2024-04-25T01:41:34.091507+0000","last_deep_scrub":"26'4554","last_deep_scrub_stamp":"2024-04-25T01:41:34.091507+0000","last_clean_scrub_stamp":"2024-04-25T01:41:34.091507+0000","objects_scrubbed":0,"log_size":4554,"log_dups_size":0,"ondisk_log_size":4554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:08.382471+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23431,"num_write_kb":298735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"26'3793","reported_seq":3833,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:38.962817+0000","last_change":"2024-04-25T01:41:38.962817+0000","last_active":"2024-04-25T01:41:38.962817+0000","last_peered":"2024-04-25T01:41:38.962817+0000","last_clean":"2024-04-25T01:41:38.962817+0000","last_became_active":"2024-04-25T01:25:53.360802+0000","last_became_peered":"2024-04-25T01:25:53.360802+0000","last_unstale":"2024-04-25T01:41:38.962817+0000","last_undegraded":"2024-04-25T01:41:38.962817+0000","last_fullsized":"2024-04-25T01:41:38.962817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3793","last_scrub_stamp":"2024-04-25T01:41:38.962655+0000","last_deep_scrub":"26'3793","last_deep_scrub_stamp":"2024-04-25T01:41:38.962655+0000","last_clean_scrub_stamp":"2024-04-25T01:41:38.962655+0000","objects_scrubbed":0,"log_size":3793,"log_dups_size":0,"ondisk_log_size":3793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:06.654326+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19299,"num_write_kb":249413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"26'3","reported_seq":45,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:28.571808+0000","last_change":"2024-04-25T01:41:28.571855+0000","last_active":"2024-04-25T01:41:28.571808+0000","last_peered":"2024-04-25T01:41:28.571808+0000","last_clean":"2024-04-25T01:41:28.571808+0000","last_became_active":"2024-04-25T01:25:52.348550+0000","last_became_peered":"2024-04-25T01:25:52.348550+0000","last_unstale":"2024-04-25T01:41:28.571808+0000","last_undegraded":"2024-04-25T01:41:28.571808+0000","last_fullsized":"2024-04-25T01:41:28.571808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:28.571698+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:28.571698+0000","last_clean_scrub_stamp":"2024-04-25T01:41:28.571698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:48.741495+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"26'3","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:23.151834+0000","last_change":"2024-04-25T01:41:23.151869+0000","last_active":"2024-04-25T01:41:23.151834+0000","last_peered":"2024-04-25T01:41:23.151834+0000","last_clean":"2024-04-25T01:41:23.151834+0000","last_became_active":"2024-04-25T01:26:01.407738+0000","last_became_peered":"2024-04-25T01:26:01.407738+0000","last_unstale":"2024-04-25T01:41:23.151834+0000","last_undegraded":"2024-04-25T01:41:23.151834+0000","last_fullsized":"2024-04-25T01:41:23.151834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:23.151736+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:23.151736+0000","last_clean_scrub_stamp":"2024-04-25T01:41:23.151736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:59.208630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"26'3863","reported_seq":3886,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.332382+0000","last_change":"2024-04-25T01:26:02.417731+0000","last_active":"2024-04-25T01:41:09.332382+0000","last_peered":"2024-04-25T01:41:09.332382+0000","last_clean":"2024-04-25T01:41:09.332382+0000","last_became_active":"2024-04-25T01:26:02.417438+0000","last_became_peered":"2024-04-25T01:26:02.417438+0000","last_unstale":"2024-04-25T01:41:09.332382+0000","last_undegraded":"2024-04-25T01:41:09.332382+0000","last_fullsized":"2024-04-25T01:41:09.332382+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3863,"log_dups_size":0,"ondisk_log_size":3863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19753,"num_write_kb":275789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"26'4400","reported_seq":4440,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:39.963413+0000","last_change":"2024-04-25T01:41:39.963441+0000","last_active":"2024-04-25T01:41:39.963413+0000","last_peered":"2024-04-25T01:41:39.963413+0000","last_clean":"2024-04-25T01:41:39.963413+0000","last_became_active":"2024-04-25T01:25:53.361542+0000","last_became_peered":"2024-04-25T01:25:53.361542+0000","last_unstale":"2024-04-25T01:41:39.963413+0000","last_undegraded":"2024-04-25T01:41:39.963413+0000","last_fullsized":"2024-04-25T01:41:39.963413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4400","last_scrub_stamp":"2024-04-25T01:41:39.963267+0000","last_deep_scrub":"26'4400","last_deep_scrub_stamp":"2024-04-25T01:41:39.963267+0000","last_clean_scrub_stamp":"2024-04-25T01:41:39.963267+0000","objects_scrubbed":0,"log_size":4400,"log_dups_size":0,"ondisk_log_size":4400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:40.780905+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22906,"num_write_kb":296647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"26'3","reported_seq":45,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:30.601900+0000","last_change":"2024-04-25T01:41:30.601952+0000","last_active":"2024-04-25T01:41:30.601900+0000","last_peered":"2024-04-25T01:41:30.601900+0000","last_clean":"2024-04-25T01:41:30.601900+0000","last_became_active":"2024-04-25T01:25:52.353656+0000","last_became_peered":"2024-04-25T01:25:52.353656+0000","last_unstale":"2024-04-25T01:41:30.601900+0000","last_undegraded":"2024-04-25T01:41:30.601900+0000","last_fullsized":"2024-04-25T01:41:30.601900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:30.601782+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:30.601782+0000","last_clean_scrub_stamp":"2024-04-25T01:41:30.601782+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:13.713160+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"26'6","reported_seq":39,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:24.647198+0000","last_change":"2024-04-25T01:41:24.647257+0000","last_active":"2024-04-25T01:41:24.647198+0000","last_peered":"2024-04-25T01:41:24.647198+0000","last_clean":"2024-04-25T01:41:24.647198+0000","last_became_active":"2024-04-25T01:26:01.418263+0000","last_became_peered":"2024-04-25T01:26:01.418263+0000","last_unstale":"2024-04-25T01:41:24.647198+0000","last_undegraded":"2024-04-25T01:41:24.647198+0000","last_fullsized":"2024-04-25T01:41:24.647198+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'6","last_scrub_stamp":"2024-04-25T01:41:24.647078+0000","last_deep_scrub":"26'6","last_deep_scrub_stamp":"2024-04-25T01:41:24.647078+0000","last_clean_scrub_stamp":"2024-04-25T01:41:24.647078+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:01.964240+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"26'5570","reported_seq":5601,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:28.337625+0000","last_change":"2024-04-25T01:41:28.337651+0000","last_active":"2024-04-25T01:41:28.337625+0000","last_peered":"2024-04-25T01:41:28.337625+0000","last_clean":"2024-04-25T01:41:28.337625+0000","last_became_active":"2024-04-25T01:26:02.412632+0000","last_became_peered":"2024-04-25T01:26:02.412632+0000","last_unstale":"2024-04-25T01:41:28.337625+0000","last_undegraded":"2024-04-25T01:41:28.337625+0000","last_fullsized":"2024-04-25T01:41:28.337625+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5570","last_scrub_stamp":"2024-04-25T01:41:28.337518+0000","last_deep_scrub":"26'5570","last_deep_scrub_stamp":"2024-04-25T01:41:28.337518+0000","last_clean_scrub_stamp":"2024-04-25T01:41:28.337518+0000","objects_scrubbed":0,"log_size":5570,"log_dups_size":0,"ondisk_log_size":5570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:50.554037+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27971,"num_write_kb":349705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"26'4715","reported_seq":4755,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:29.199139+0000","last_change":"2024-04-25T01:41:29.199167+0000","last_active":"2024-04-25T01:41:29.199139+0000","last_peered":"2024-04-25T01:41:29.199139+0000","last_clean":"2024-04-25T01:41:29.199139+0000","last_became_active":"2024-04-25T01:25:53.361265+0000","last_became_peered":"2024-04-25T01:25:53.361265+0000","last_unstale":"2024-04-25T01:41:29.199139+0000","last_undegraded":"2024-04-25T01:41:29.199139+0000","last_fullsized":"2024-04-25T01:41:29.199139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4715","last_scrub_stamp":"2024-04-25T01:41:29.199039+0000","last_deep_scrub":"26'4715","last_deep_scrub_stamp":"2024-04-25T01:41:29.199039+0000","last_clean_scrub_stamp":"2024-04-25T01:41:29.199039+0000","objects_scrubbed":0,"log_size":4715,"log_dups_size":0,"ondisk_log_size":4715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:10.668527+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23886,"num_write_kb":288267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'3","reported_seq":45,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:35.749056+0000","last_change":"2024-04-25T01:41:35.749085+0000","last_active":"2024-04-25T01:41:35.749056+0000","last_peered":"2024-04-25T01:41:35.749056+0000","last_clean":"2024-04-25T01:41:35.749056+0000","last_became_active":"2024-04-25T01:25:52.353552+0000","last_became_peered":"2024-04-25T01:25:52.353552+0000","last_unstale":"2024-04-25T01:41:35.749056+0000","last_undegraded":"2024-04-25T01:41:35.749056+0000","last_fullsized":"2024-04-25T01:41:35.749056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:35.748960+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:35.748960+0000","last_clean_scrub_stamp":"2024-04-25T01:41:35.748960+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:54.247567+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"26'7","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:22.191597+0000","last_change":"2024-04-25T01:41:22.191626+0000","last_active":"2024-04-25T01:41:22.191597+0000","last_peered":"2024-04-25T01:41:22.191597+0000","last_clean":"2024-04-25T01:41:22.191597+0000","last_became_active":"2024-04-25T01:26:01.410378+0000","last_became_peered":"2024-04-25T01:26:01.410378+0000","last_unstale":"2024-04-25T01:41:22.191597+0000","last_undegraded":"2024-04-25T01:41:22.191597+0000","last_fullsized":"2024-04-25T01:41:22.191597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'7","last_scrub_stamp":"2024-04-25T01:41:22.191504+0000","last_deep_scrub":"26'7","last_deep_scrub_stamp":"2024-04-25T01:41:22.191504+0000","last_clean_scrub_stamp":"2024-04-25T01:41:22.191504+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:10.062407+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"26'4403","reported_seq":4427,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.252333+0000","last_change":"2024-04-25T01:26:02.416646+0000","last_active":"2024-04-25T01:41:09.252333+0000","last_peered":"2024-04-25T01:41:09.252333+0000","last_clean":"2024-04-25T01:41:09.252333+0000","last_became_active":"2024-04-25T01:26:02.416395+0000","last_became_peered":"2024-04-25T01:26:02.416395+0000","last_unstale":"2024-04-25T01:41:09.252333+0000","last_undegraded":"2024-04-25T01:41:09.252333+0000","last_fullsized":"2024-04-25T01:41:09.252333+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4403,"log_dups_size":0,"ondisk_log_size":4403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23642,"num_write_kb":287389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"26'3766","reported_seq":3806,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:29.694397+0000","last_change":"2024-04-25T01:41:29.694425+0000","last_active":"2024-04-25T01:41:29.694397+0000","last_peered":"2024-04-25T01:41:29.694397+0000","last_clean":"2024-04-25T01:41:29.694397+0000","last_became_active":"2024-04-25T01:25:53.369837+0000","last_became_peered":"2024-04-25T01:25:53.369837+0000","last_unstale":"2024-04-25T01:41:29.694397+0000","last_undegraded":"2024-04-25T01:41:29.694397+0000","last_fullsized":"2024-04-25T01:41:29.694397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3766","last_scrub_stamp":"2024-04-25T01:41:29.694281+0000","last_deep_scrub":"26'3766","last_deep_scrub_stamp":"2024-04-25T01:41:29.694281+0000","last_clean_scrub_stamp":"2024-04-25T01:41:29.694281+0000","objects_scrubbed":0,"log_size":3766,"log_dups_size":0,"ondisk_log_size":3766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:43.426336+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19293,"num_write_kb":249241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"26'6","reported_seq":48,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:28.270350+0000","last_change":"2024-04-25T01:41:28.270408+0000","last_active":"2024-04-25T01:41:28.270350+0000","last_peered":"2024-04-25T01:41:28.270350+0000","last_clean":"2024-04-25T01:41:28.270350+0000","last_became_active":"2024-04-25T01:25:52.352991+0000","last_became_peered":"2024-04-25T01:25:52.352991+0000","last_unstale":"2024-04-25T01:41:28.270350+0000","last_undegraded":"2024-04-25T01:41:28.270350+0000","last_fullsized":"2024-04-25T01:41:28.270350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'6","last_scrub_stamp":"2024-04-25T01:41:28.270225+0000","last_deep_scrub":"26'6","last_deep_scrub_stamp":"2024-04-25T01:41:28.270225+0000","last_clean_scrub_stamp":"2024-04-25T01:41:28.270225+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:09.220120+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"26'42","reported_seq":115,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:21.165819+0000","last_change":"2024-04-25T01:41:21.165848+0000","last_active":"2024-04-25T01:41:21.165819+0000","last_peered":"2024-04-25T01:41:21.165819+0000","last_clean":"2024-04-25T01:41:21.165819+0000","last_became_active":"2024-04-25T01:26:01.410076+0000","last_became_peered":"2024-04-25T01:26:01.410076+0000","last_unstale":"2024-04-25T01:41:21.165819+0000","last_undegraded":"2024-04-25T01:41:21.165819+0000","last_fullsized":"2024-04-25T01:41:21.165819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'42","last_scrub_stamp":"2024-04-25T01:41:21.165730+0000","last_deep_scrub":"26'42","last_deep_scrub_stamp":"2024-04-25T01:41:21.165730+0000","last_clean_scrub_stamp":"2024-04-25T01:41:21.165730+0000","objects_scrubbed":1,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:19.490457+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":112,"num_write":39,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"26'4289","reported_seq":4313,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.299656+0000","last_change":"2024-04-25T01:26:02.422182+0000","last_active":"2024-04-25T01:41:09.299656+0000","last_peered":"2024-04-25T01:41:09.299656+0000","last_clean":"2024-04-25T01:41:09.299656+0000","last_became_active":"2024-04-25T01:26:02.421885+0000","last_became_peered":"2024-04-25T01:26:02.421885+0000","last_unstale":"2024-04-25T01:41:09.299656+0000","last_undegraded":"2024-04-25T01:41:09.299656+0000","last_fullsized":"2024-04-25T01:41:09.299656+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4289,"log_dups_size":0,"ondisk_log_size":4289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22172,"num_write_kb":285532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"26'4466","reported_seq":4506,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:29.902161+0000","last_change":"2024-04-25T01:41:29.902195+0000","last_active":"2024-04-25T01:41:29.902161+0000","last_peered":"2024-04-25T01:41:29.902161+0000","last_clean":"2024-04-25T01:41:29.902161+0000","last_became_active":"2024-04-25T01:25:53.364694+0000","last_became_peered":"2024-04-25T01:25:53.364694+0000","last_unstale":"2024-04-25T01:41:29.902161+0000","last_undegraded":"2024-04-25T01:41:29.902161+0000","last_fullsized":"2024-04-25T01:41:29.902161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4466","last_scrub_stamp":"2024-04-25T01:41:29.902047+0000","last_deep_scrub":"26'4466","last_deep_scrub_stamp":"2024-04-25T01:41:29.902047+0000","last_clean_scrub_stamp":"2024-04-25T01:41:29.902047+0000","objects_scrubbed":0,"log_size":4466,"log_dups_size":0,"ondisk_log_size":4466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:56.983380+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22859,"num_write_kb":292275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'572","reported_seq":614,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:32.557098+0000","last_change":"2024-04-25T01:41:32.557152+0000","last_active":"2024-04-25T01:41:32.557098+0000","last_peered":"2024-04-25T01:41:32.557098+0000","last_clean":"2024-04-25T01:41:32.557098+0000","last_became_active":"2024-04-25T01:25:52.355137+0000","last_became_peered":"2024-04-25T01:25:52.355137+0000","last_unstale":"2024-04-25T01:41:32.557098+0000","last_undegraded":"2024-04-25T01:41:32.557098+0000","last_fullsized":"2024-04-25T01:41:32.557098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'572","last_scrub_stamp":"2024-04-25T01:41:32.556995+0000","last_deep_scrub":"26'572","last_deep_scrub_stamp":"2024-04-25T01:41:32.556995+0000","last_clean_scrub_stamp":"2024-04-25T01:41:32.556995+0000","objects_scrubbed":1,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:59.638329+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"26'5","reported_seq":39,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:22.864117+0000","last_change":"2024-04-25T01:41:22.864160+0000","last_active":"2024-04-25T01:41:22.864117+0000","last_peered":"2024-04-25T01:41:22.864117+0000","last_clean":"2024-04-25T01:41:22.864117+0000","last_became_active":"2024-04-25T01:26:01.415824+0000","last_became_peered":"2024-04-25T01:26:01.415824+0000","last_unstale":"2024-04-25T01:41:22.864117+0000","last_undegraded":"2024-04-25T01:41:22.864117+0000","last_fullsized":"2024-04-25T01:41:22.864117+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-04-25T01:41:22.864032+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-04-25T01:41:22.864032+0000","last_clean_scrub_stamp":"2024-04-25T01:41:22.864032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:11.105583+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"26'4989","reported_seq":5013,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.276306+0000","last_change":"2024-04-25T01:26:02.416208+0000","last_active":"2024-04-25T01:41:09.276306+0000","last_peered":"2024-04-25T01:41:09.276306+0000","last_clean":"2024-04-25T01:41:09.276306+0000","last_became_active":"2024-04-25T01:26:02.415949+0000","last_became_peered":"2024-04-25T01:26:02.415949+0000","last_unstale":"2024-04-25T01:41:09.276306+0000","last_undegraded":"2024-04-25T01:41:09.276306+0000","last_fullsized":"2024-04-25T01:41:09.276306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4989,"log_dups_size":0,"ondisk_log_size":4989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25442,"num_write_kb":314904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"26'4796","reported_seq":4836,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:25.631136+0000","last_change":"2024-04-25T01:41:25.631164+0000","last_active":"2024-04-25T01:41:25.631136+0000","last_peered":"2024-04-25T01:41:25.631136+0000","last_clean":"2024-04-25T01:41:25.631136+0000","last_became_active":"2024-04-25T01:25:53.366693+0000","last_became_peered":"2024-04-25T01:25:53.366693+0000","last_unstale":"2024-04-25T01:41:25.631136+0000","last_undegraded":"2024-04-25T01:41:25.631136+0000","last_fullsized":"2024-04-25T01:41:25.631136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4796","last_scrub_stamp":"2024-04-25T01:41:25.631022+0000","last_deep_scrub":"26'4796","last_deep_scrub_stamp":"2024-04-25T01:41:25.631022+0000","last_clean_scrub_stamp":"2024-04-25T01:41:25.631022+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:48.647899+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25026,"num_write_kb":309186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"26'4","reported_seq":46,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:36.798781+0000","last_change":"2024-04-25T01:41:36.798806+0000","last_active":"2024-04-25T01:41:36.798781+0000","last_peered":"2024-04-25T01:41:36.798781+0000","last_clean":"2024-04-25T01:41:36.798781+0000","last_became_active":"2024-04-25T01:25:52.356396+0000","last_became_peered":"2024-04-25T01:25:52.356396+0000","last_unstale":"2024-04-25T01:41:36.798781+0000","last_undegraded":"2024-04-25T01:41:36.798781+0000","last_fullsized":"2024-04-25T01:41:36.798781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4","last_scrub_stamp":"2024-04-25T01:41:36.798679+0000","last_deep_scrub":"26'4","last_deep_scrub_stamp":"2024-04-25T01:41:36.798679+0000","last_clean_scrub_stamp":"2024-04-25T01:41:36.798679+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:12.551356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"26'1","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:21.695156+0000","last_change":"2024-04-25T01:41:21.695177+0000","last_active":"2024-04-25T01:41:21.695156+0000","last_peered":"2024-04-25T01:41:21.695156+0000","last_clean":"2024-04-25T01:41:21.695156+0000","last_became_active":"2024-04-25T01:26:01.407659+0000","last_became_peered":"2024-04-25T01:26:01.407659+0000","last_unstale":"2024-04-25T01:41:21.695156+0000","last_undegraded":"2024-04-25T01:41:21.695156+0000","last_fullsized":"2024-04-25T01:41:21.695156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-04-25T01:41:21.695080+0000","last_deep_scrub":"26'1","last_deep_scrub_stamp":"2024-04-25T01:41:21.695080+0000","last_clean_scrub_stamp":"2024-04-25T01:41:21.695080+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:31.680471+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"26'4196","reported_seq":4228,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:35.786102+0000","last_change":"2024-04-25T01:41:35.786131+0000","last_active":"2024-04-25T01:41:35.786102+0000","last_peered":"2024-04-25T01:41:35.786102+0000","last_clean":"2024-04-25T01:41:35.786102+0000","last_became_active":"2024-04-25T01:26:02.417622+0000","last_became_peered":"2024-04-25T01:26:02.417622+0000","last_unstale":"2024-04-25T01:41:35.786102+0000","last_undegraded":"2024-04-25T01:41:35.786102+0000","last_fullsized":"2024-04-25T01:41:35.786102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4196","last_scrub_stamp":"2024-04-25T01:41:35.785996+0000","last_deep_scrub":"26'4196","last_deep_scrub_stamp":"2024-04-25T01:41:35.785996+0000","last_clean_scrub_stamp":"2024-04-25T01:41:35.785996+0000","objects_scrubbed":0,"log_size":4196,"log_dups_size":0,"ondisk_log_size":4196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:18.897374+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22617,"num_write_kb":283190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"26'4472","reported_seq":4504,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.875494+0000","last_change":"2024-04-25T01:25:53.368164+0000","last_active":"2024-04-25T01:40:58.875494+0000","last_peered":"2024-04-25T01:40:58.875494+0000","last_clean":"2024-04-25T01:40:58.875494+0000","last_became_active":"2024-04-25T01:25:53.367854+0000","last_became_peered":"2024-04-25T01:25:53.367854+0000","last_unstale":"2024-04-25T01:40:58.875494+0000","last_undegraded":"2024-04-25T01:40:58.875494+0000","last_fullsized":"2024-04-25T01:40:58.875494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4472,"log_dups_size":0,"ondisk_log_size":4472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23917,"num_write_kb":276200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'5","reported_seq":47,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:29.556568+0000","last_change":"2024-04-25T01:41:29.556616+0000","last_active":"2024-04-25T01:41:29.556568+0000","last_peered":"2024-04-25T01:41:29.556568+0000","last_clean":"2024-04-25T01:41:29.556568+0000","last_became_active":"2024-04-25T01:25:52.349874+0000","last_became_peered":"2024-04-25T01:25:52.349874+0000","last_unstale":"2024-04-25T01:41:29.556568+0000","last_undegraded":"2024-04-25T01:41:29.556568+0000","last_fullsized":"2024-04-25T01:41:29.556568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-04-25T01:41:29.556461+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-04-25T01:41:29.556461+0000","last_clean_scrub_stamp":"2024-04-25T01:41:29.556461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:18.108426+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"26'5","reported_seq":39,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:20.631227+0000","last_change":"2024-04-25T01:41:20.631248+0000","last_active":"2024-04-25T01:41:20.631227+0000","last_peered":"2024-04-25T01:41:20.631227+0000","last_clean":"2024-04-25T01:41:20.631227+0000","last_became_active":"2024-04-25T01:26:01.417820+0000","last_became_peered":"2024-04-25T01:26:01.417820+0000","last_unstale":"2024-04-25T01:41:20.631227+0000","last_undegraded":"2024-04-25T01:41:20.631227+0000","last_fullsized":"2024-04-25T01:41:20.631227+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-04-25T01:41:20.631162+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-04-25T01:41:20.631162+0000","last_clean_scrub_stamp":"2024-04-25T01:41:20.631162+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:56.688056+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"26'4093","reported_seq":4117,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.290657+0000","last_change":"2024-04-25T01:26:02.422761+0000","last_active":"2024-04-25T01:41:09.290657+0000","last_peered":"2024-04-25T01:41:09.290657+0000","last_clean":"2024-04-25T01:41:09.290657+0000","last_became_active":"2024-04-25T01:26:02.422503+0000","last_became_peered":"2024-04-25T01:26:02.422503+0000","last_unstale":"2024-04-25T01:41:09.290657+0000","last_undegraded":"2024-04-25T01:41:09.290657+0000","last_fullsized":"2024-04-25T01:41:09.290657+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4093,"log_dups_size":0,"ondisk_log_size":4093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21550,"num_write_kb":266696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"26'4812","reported_seq":4852,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:30.228308+0000","last_change":"2024-04-25T01:41:30.228334+0000","last_active":"2024-04-25T01:41:30.228308+0000","last_peered":"2024-04-25T01:41:30.228308+0000","last_clean":"2024-04-25T01:41:30.228308+0000","last_became_active":"2024-04-25T01:25:53.361156+0000","last_became_peered":"2024-04-25T01:25:53.361156+0000","last_unstale":"2024-04-25T01:41:30.228308+0000","last_undegraded":"2024-04-25T01:41:30.228308+0000","last_fullsized":"2024-04-25T01:41:30.228308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4812","last_scrub_stamp":"2024-04-25T01:41:30.228208+0000","last_deep_scrub":"26'4812","last_deep_scrub_stamp":"2024-04-25T01:41:30.228208+0000","last_clean_scrub_stamp":"2024-04-25T01:41:30.228208+0000","objects_scrubbed":0,"log_size":4812,"log_dups_size":0,"ondisk_log_size":4812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:59.815199+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25777,"num_write_kb":295962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"26'5","reported_seq":47,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:33.375115+0000","last_change":"2024-04-25T01:41:33.375143+0000","last_active":"2024-04-25T01:41:33.375115+0000","last_peered":"2024-04-25T01:41:33.375115+0000","last_clean":"2024-04-25T01:41:33.375115+0000","last_became_active":"2024-04-25T01:25:52.354193+0000","last_became_peered":"2024-04-25T01:25:52.354193+0000","last_unstale":"2024-04-25T01:41:33.375115+0000","last_undegraded":"2024-04-25T01:41:33.375115+0000","last_fullsized":"2024-04-25T01:41:33.375115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-04-25T01:41:33.375020+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-04-25T01:41:33.375020+0000","last_clean_scrub_stamp":"2024-04-25T01:41:33.375020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:51.873626+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"26'5","reported_seq":39,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:19.652416+0000","last_change":"2024-04-25T01:41:19.652438+0000","last_active":"2024-04-25T01:41:19.652416+0000","last_peered":"2024-04-25T01:41:19.652416+0000","last_clean":"2024-04-25T01:41:19.652416+0000","last_became_active":"2024-04-25T01:26:01.417805+0000","last_became_peered":"2024-04-25T01:26:01.417805+0000","last_unstale":"2024-04-25T01:41:19.652416+0000","last_undegraded":"2024-04-25T01:41:19.652416+0000","last_fullsized":"2024-04-25T01:41:19.652416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-04-25T01:41:19.652334+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-04-25T01:41:19.652334+0000","last_clean_scrub_stamp":"2024-04-25T01:41:19.652334+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:54.494631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"26'3951","reported_seq":3983,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:31.180998+0000","last_change":"2024-04-25T01:41:31.181024+0000","last_active":"2024-04-25T01:41:31.180998+0000","last_peered":"2024-04-25T01:41:31.180998+0000","last_clean":"2024-04-25T01:41:31.180998+0000","last_became_active":"2024-04-25T01:26:02.415619+0000","last_became_peered":"2024-04-25T01:26:02.415619+0000","last_unstale":"2024-04-25T01:41:31.180998+0000","last_undegraded":"2024-04-25T01:41:31.180998+0000","last_fullsized":"2024-04-25T01:41:31.180998+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3951","last_scrub_stamp":"2024-04-25T01:41:31.180895+0000","last_deep_scrub":"26'3951","last_deep_scrub_stamp":"2024-04-25T01:41:31.180895+0000","last_clean_scrub_stamp":"2024-04-25T01:41:31.180895+0000","objects_scrubbed":0,"log_size":3951,"log_dups_size":0,"ondisk_log_size":3951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:39.576959+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20033,"num_write_kb":252185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"26'4985","reported_seq":5025,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:34.587627+0000","last_change":"2024-04-25T01:41:34.587670+0000","last_active":"2024-04-25T01:41:34.587627+0000","last_peered":"2024-04-25T01:41:34.587627+0000","last_clean":"2024-04-25T01:41:34.587627+0000","last_became_active":"2024-04-25T01:25:53.362533+0000","last_became_peered":"2024-04-25T01:25:53.362533+0000","last_unstale":"2024-04-25T01:41:34.587627+0000","last_undegraded":"2024-04-25T01:41:34.587627+0000","last_fullsized":"2024-04-25T01:41:34.587627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4985","last_scrub_stamp":"2024-04-25T01:41:34.587455+0000","last_deep_scrub":"26'4985","last_deep_scrub_stamp":"2024-04-25T01:41:34.587455+0000","last_clean_scrub_stamp":"2024-04-25T01:41:34.587455+0000","objects_scrubbed":0,"log_size":4985,"log_dups_size":0,"ondisk_log_size":4985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:31.640145+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26332,"num_write_kb":298165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"26'44","reported_seq":127,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:21.990855+0000","last_change":"2024-04-25T01:41:21.990904+0000","last_active":"2024-04-25T01:41:21.990855+0000","last_peered":"2024-04-25T01:41:21.990855+0000","last_clean":"2024-04-25T01:41:21.990855+0000","last_became_active":"2024-04-25T01:25:52.345061+0000","last_became_peered":"2024-04-25T01:25:52.345061+0000","last_unstale":"2024-04-25T01:41:21.990855+0000","last_undegraded":"2024-04-25T01:41:21.990855+0000","last_fullsized":"2024-04-25T01:41:21.990855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'44","last_scrub_stamp":"2024-04-25T01:41:21.990771+0000","last_deep_scrub":"26'44","last_deep_scrub_stamp":"2024-04-25T01:41:21.990771+0000","last_clean_scrub_stamp":"2024-04-25T01:41:21.990771+0000","objects_scrubbed":1,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:03.336231+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":41,"num_read_kb":115,"num_write":41,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"26'5","reported_seq":39,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:22.637851+0000","last_change":"2024-04-25T01:41:22.637901+0000","last_active":"2024-04-25T01:41:22.637851+0000","last_peered":"2024-04-25T01:41:22.637851+0000","last_clean":"2024-04-25T01:41:22.637851+0000","last_became_active":"2024-04-25T01:26:01.418173+0000","last_became_peered":"2024-04-25T01:26:01.418173+0000","last_unstale":"2024-04-25T01:41:22.637851+0000","last_undegraded":"2024-04-25T01:41:22.637851+0000","last_fullsized":"2024-04-25T01:41:22.637851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-04-25T01:41:22.637755+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-04-25T01:41:22.637755+0000","last_clean_scrub_stamp":"2024-04-25T01:41:22.637755+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:31.033818+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"26'4134","reported_seq":4158,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:17.542562+0000","last_change":"2024-04-25T01:26:02.418115+0000","last_active":"2024-04-25T01:41:17.542562+0000","last_peered":"2024-04-25T01:41:17.542562+0000","last_clean":"2024-04-25T01:41:17.542562+0000","last_became_active":"2024-04-25T01:26:02.417355+0000","last_became_peered":"2024-04-25T01:26:02.417355+0000","last_unstale":"2024-04-25T01:41:17.542562+0000","last_undegraded":"2024-04-25T01:41:17.542562+0000","last_fullsized":"2024-04-25T01:41:17.542562+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4134,"log_dups_size":0,"ondisk_log_size":4134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21715,"num_write_kb":287743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"26'4344","reported_seq":4384,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:25.701271+0000","last_change":"2024-04-25T01:41:25.701300+0000","last_active":"2024-04-25T01:41:25.701271+0000","last_peered":"2024-04-25T01:41:25.701271+0000","last_clean":"2024-04-25T01:41:25.701271+0000","last_became_active":"2024-04-25T01:25:53.355648+0000","last_became_peered":"2024-04-25T01:25:53.355648+0000","last_unstale":"2024-04-25T01:41:25.701271+0000","last_undegraded":"2024-04-25T01:41:25.701271+0000","last_fullsized":"2024-04-25T01:41:25.701271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4344","last_scrub_stamp":"2024-04-25T01:41:25.701164+0000","last_deep_scrub":"26'4344","last_deep_scrub_stamp":"2024-04-25T01:41:25.701164+0000","last_clean_scrub_stamp":"2024-04-25T01:41:25.701164+0000","objects_scrubbed":0,"log_size":4344,"log_dups_size":0,"ondisk_log_size":4344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:01.758058+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23132,"num_write_kb":289328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"26'5","reported_seq":47,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:38.723717+0000","last_change":"2024-04-25T01:41:38.723753+0000","last_active":"2024-04-25T01:41:38.723717+0000","last_peered":"2024-04-25T01:41:38.723717+0000","last_clean":"2024-04-25T01:41:38.723717+0000","last_became_active":"2024-04-25T01:25:52.356924+0000","last_became_peered":"2024-04-25T01:25:52.356924+0000","last_unstale":"2024-04-25T01:41:38.723717+0000","last_undegraded":"2024-04-25T01:41:38.723717+0000","last_fullsized":"2024-04-25T01:41:38.723717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-04-25T01:41:38.723630+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-04-25T01:41:38.723630+0000","last_clean_scrub_stamp":"2024-04-25T01:41:38.723630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:35.623165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"26'571","reported_seq":605,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:34.739765+0000","last_change":"2024-04-25T01:41:34.739794+0000","last_active":"2024-04-25T01:41:34.739765+0000","last_peered":"2024-04-25T01:41:34.739765+0000","last_clean":"2024-04-25T01:41:34.739765+0000","last_became_active":"2024-04-25T01:26:01.420831+0000","last_became_peered":"2024-04-25T01:26:01.420831+0000","last_unstale":"2024-04-25T01:41:34.739765+0000","last_undegraded":"2024-04-25T01:41:34.739765+0000","last_fullsized":"2024-04-25T01:41:34.739765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'571","last_scrub_stamp":"2024-04-25T01:41:34.739680+0000","last_deep_scrub":"26'571","last_deep_scrub_stamp":"2024-04-25T01:41:34.739680+0000","last_clean_scrub_stamp":"2024-04-25T01:41:34.739680+0000","objects_scrubbed":1,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:23.291645+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1330,"num_write_kb":427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"26'4947","reported_seq":4971,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.327930+0000","last_change":"2024-04-25T01:26:02.421386+0000","last_active":"2024-04-25T01:41:09.327930+0000","last_peered":"2024-04-25T01:41:09.327930+0000","last_clean":"2024-04-25T01:41:09.327930+0000","last_became_active":"2024-04-25T01:26:02.421079+0000","last_became_peered":"2024-04-25T01:26:02.421079+0000","last_unstale":"2024-04-25T01:41:09.327930+0000","last_undegraded":"2024-04-25T01:41:09.327930+0000","last_fullsized":"2024-04-25T01:41:09.327930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4947,"log_dups_size":0,"ondisk_log_size":4947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25964,"num_write_kb":313022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"26'4754","reported_seq":4780,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:37.054368+0000","last_change":"2024-04-25T01:41:37.054393+0000","last_active":"2024-04-25T01:41:37.054368+0000","last_peered":"2024-04-25T01:41:37.054368+0000","last_clean":"2024-04-25T01:41:37.054368+0000","last_became_active":"2024-04-25T01:26:52.515254+0000","last_became_peered":"2024-04-25T01:26:52.515254+0000","last_unstale":"2024-04-25T01:41:37.054368+0000","last_undegraded":"2024-04-25T01:41:37.054368+0000","last_fullsized":"2024-04-25T01:41:37.054368+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4754","last_scrub_stamp":"2024-04-25T01:41:37.054234+0000","last_deep_scrub":"26'4754","last_deep_scrub_stamp":"2024-04-25T01:41:37.054234+0000","last_clean_scrub_stamp":"2024-04-25T01:41:37.054234+0000","objects_scrubbed":0,"log_size":4754,"log_dups_size":0,"ondisk_log_size":4754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:36.238854+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24235,"num_write_kb":310192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:32.430292+0000","last_change":"2024-04-25T01:41:32.430316+0000","last_active":"2024-04-25T01:41:32.430292+0000","last_peered":"2024-04-25T01:41:32.430292+0000","last_clean":"2024-04-25T01:41:32.430292+0000","last_became_active":"2024-04-25T01:25:52.352826+0000","last_became_peered":"2024-04-25T01:25:52.352826+0000","last_unstale":"2024-04-25T01:41:32.430292+0000","last_undegraded":"2024-04-25T01:41:32.430292+0000","last_fullsized":"2024-04-25T01:41:32.430292+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-04-25T01:41:32.430197+0000","last_deep_scrub":"26'1","last_deep_scrub_stamp":"2024-04-25T01:41:32.430197+0000","last_clean_scrub_stamp":"2024-04-25T01:41:32.430197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:45.201865+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"26'119","reported_seq":200,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:34.779923+0000","last_change":"2024-04-25T01:41:34.779955+0000","last_active":"2024-04-25T01:41:34.779923+0000","last_peered":"2024-04-25T01:41:34.779923+0000","last_clean":"2024-04-25T01:41:34.779923+0000","last_became_active":"2024-04-25T01:25:41.718721+0000","last_became_peered":"2024-04-25T01:25:41.718721+0000","last_unstale":"2024-04-25T01:41:34.779923+0000","last_undegraded":"2024-04-25T01:41:34.779923+0000","last_fullsized":"2024-04-25T01:41:34.779923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'119","last_scrub_stamp":"2024-04-25T01:41:34.779819+0000","last_deep_scrub":"26'119","last_deep_scrub_stamp":"2024-04-25T01:41:34.779819+0000","last_clean_scrub_stamp":"2024-04-25T01:41:34.779819+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:31.832509+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"26'4","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:33.319900+0000","last_change":"2024-04-25T01:41:33.319928+0000","last_active":"2024-04-25T01:41:33.319900+0000","last_peered":"2024-04-25T01:41:33.319900+0000","last_clean":"2024-04-25T01:41:33.319900+0000","last_became_active":"2024-04-25T01:26:01.409062+0000","last_became_peered":"2024-04-25T01:26:01.409062+0000","last_unstale":"2024-04-25T01:41:33.319900+0000","last_undegraded":"2024-04-25T01:41:33.319900+0000","last_fullsized":"2024-04-25T01:41:33.319900+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4","last_scrub_stamp":"2024-04-25T01:41:33.319785+0000","last_deep_scrub":"26'4","last_deep_scrub_stamp":"2024-04-25T01:41:33.319785+0000","last_clean_scrub_stamp":"2024-04-25T01:41:33.319785+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:14.789273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"26'3833","reported_seq":3864,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:32.125231+0000","last_change":"2024-04-25T01:41:32.125258+0000","last_active":"2024-04-25T01:41:32.125231+0000","last_peered":"2024-04-25T01:41:32.125231+0000","last_clean":"2024-04-25T01:41:32.125231+0000","last_became_active":"2024-04-25T01:26:02.416546+0000","last_became_peered":"2024-04-25T01:26:02.416546+0000","last_unstale":"2024-04-25T01:41:32.125231+0000","last_undegraded":"2024-04-25T01:41:32.125231+0000","last_fullsized":"2024-04-25T01:41:32.125231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3833","last_scrub_stamp":"2024-04-25T01:41:32.125125+0000","last_deep_scrub":"26'3833","last_deep_scrub_stamp":"2024-04-25T01:41:32.125125+0000","last_clean_scrub_stamp":"2024-04-25T01:41:32.125125+0000","objects_scrubbed":0,"log_size":3833,"log_dups_size":0,"ondisk_log_size":3833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:38.867976+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19504,"num_write_kb":250193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"26'5320","reported_seq":5360,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:24.985853+0000","last_change":"2024-04-25T01:41:24.985853+0000","last_active":"2024-04-25T01:41:24.985853+0000","last_peered":"2024-04-25T01:41:24.985853+0000","last_clean":"2024-04-25T01:41:24.985853+0000","last_became_active":"2024-04-25T01:25:53.375219+0000","last_became_peered":"2024-04-25T01:25:53.375219+0000","last_unstale":"2024-04-25T01:41:24.985853+0000","last_undegraded":"2024-04-25T01:41:24.985853+0000","last_fullsized":"2024-04-25T01:41:24.985853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5320","last_scrub_stamp":"2024-04-25T01:41:24.985691+0000","last_deep_scrub":"26'5320","last_deep_scrub_stamp":"2024-04-25T01:41:24.985691+0000","last_clean_scrub_stamp":"2024-04-25T01:41:24.985691+0000","objects_scrubbed":0,"log_size":5320,"log_dups_size":0,"ondisk_log_size":5320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:38.717746+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27324,"num_write_kb":314375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'3","reported_seq":45,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:25.069689+0000","last_change":"2024-04-25T01:41:25.069689+0000","last_active":"2024-04-25T01:41:25.069689+0000","last_peered":"2024-04-25T01:41:25.069689+0000","last_clean":"2024-04-25T01:41:25.069689+0000","last_became_active":"2024-04-25T01:25:52.352644+0000","last_became_peered":"2024-04-25T01:25:52.352644+0000","last_unstale":"2024-04-25T01:41:25.069689+0000","last_undegraded":"2024-04-25T01:41:25.069689+0000","last_fullsized":"2024-04-25T01:41:25.069689+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:25.069565+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:25.069565+0000","last_clean_scrub_stamp":"2024-04-25T01:41:25.069565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:35.054956+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"26'3","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:35.231839+0000","last_change":"2024-04-25T01:41:35.231875+0000","last_active":"2024-04-25T01:41:35.231839+0000","last_peered":"2024-04-25T01:41:35.231839+0000","last_clean":"2024-04-25T01:41:35.231839+0000","last_became_active":"2024-04-25T01:26:01.420664+0000","last_became_peered":"2024-04-25T01:26:01.420664+0000","last_unstale":"2024-04-25T01:41:35.231839+0000","last_undegraded":"2024-04-25T01:41:35.231839+0000","last_fullsized":"2024-04-25T01:41:35.231839+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:35.231754+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:35.231754+0000","last_clean_scrub_stamp":"2024-04-25T01:41:35.231754+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:55.401551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"26'4000","reported_seq":4032,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:29.094487+0000","last_change":"2024-04-25T01:41:29.094513+0000","last_active":"2024-04-25T01:41:29.094487+0000","last_peered":"2024-04-25T01:41:29.094487+0000","last_clean":"2024-04-25T01:41:29.094487+0000","last_became_active":"2024-04-25T01:26:02.403206+0000","last_became_peered":"2024-04-25T01:26:02.403206+0000","last_unstale":"2024-04-25T01:41:29.094487+0000","last_undegraded":"2024-04-25T01:41:29.094487+0000","last_fullsized":"2024-04-25T01:41:29.094487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4000","last_scrub_stamp":"2024-04-25T01:41:29.094381+0000","last_deep_scrub":"26'4000","last_deep_scrub_stamp":"2024-04-25T01:41:29.094381+0000","last_clean_scrub_stamp":"2024-04-25T01:41:29.094381+0000","objects_scrubbed":0,"log_size":4000,"log_dups_size":0,"ondisk_log_size":4000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:39.561760+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21231,"num_write_kb":269526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"26'4796","reported_seq":4835,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:27.383278+0000","last_change":"2024-04-25T01:41:27.383303+0000","last_active":"2024-04-25T01:41:27.383278+0000","last_peered":"2024-04-25T01:41:27.383278+0000","last_clean":"2024-04-25T01:41:27.383278+0000","last_became_active":"2024-04-25T01:25:53.370539+0000","last_became_peered":"2024-04-25T01:25:53.370539+0000","last_unstale":"2024-04-25T01:41:27.383278+0000","last_undegraded":"2024-04-25T01:41:27.383278+0000","last_fullsized":"2024-04-25T01:41:27.383278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4796","last_scrub_stamp":"2024-04-25T01:41:27.383169+0000","last_deep_scrub":"26'4796","last_deep_scrub_stamp":"2024-04-25T01:41:27.383169+0000","last_clean_scrub_stamp":"2024-04-25T01:41:27.383169+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:26.567790+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25182,"num_write_kb":313860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"26'3","reported_seq":45,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:31.580818+0000","last_change":"2024-04-25T01:41:31.580818+0000","last_active":"2024-04-25T01:41:31.580818+0000","last_peered":"2024-04-25T01:41:31.580818+0000","last_clean":"2024-04-25T01:41:31.580818+0000","last_became_active":"2024-04-25T01:25:52.354836+0000","last_became_peered":"2024-04-25T01:25:52.354836+0000","last_unstale":"2024-04-25T01:41:31.580818+0000","last_undegraded":"2024-04-25T01:41:31.580818+0000","last_fullsized":"2024-04-25T01:41:31.580818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:31.580652+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:31.580652+0000","last_clean_scrub_stamp":"2024-04-25T01:41:31.580652+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:00.337278+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"26'1114","reported_seq":1147,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:33.738185+0000","last_change":"2024-04-25T01:41:33.738219+0000","last_active":"2024-04-25T01:41:33.738185+0000","last_peered":"2024-04-25T01:41:33.738185+0000","last_clean":"2024-04-25T01:41:33.738185+0000","last_became_active":"2024-04-25T01:26:01.418987+0000","last_became_peered":"2024-04-25T01:26:01.418987+0000","last_unstale":"2024-04-25T01:41:33.738185+0000","last_undegraded":"2024-04-25T01:41:33.738185+0000","last_fullsized":"2024-04-25T01:41:33.738185+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1114","last_scrub_stamp":"2024-04-25T01:41:33.738095+0000","last_deep_scrub":"26'1114","last_deep_scrub_stamp":"2024-04-25T01:41:33.738095+0000","last_clean_scrub_stamp":"2024-04-25T01:41:33.738095+0000","objects_scrubbed":2,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:32.922716+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1103,"num_write_kb":4430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"26'4082","reported_seq":4114,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:26.907485+0000","last_change":"2024-04-25T01:41:26.907511+0000","last_active":"2024-04-25T01:41:26.907485+0000","last_peered":"2024-04-25T01:41:26.907485+0000","last_clean":"2024-04-25T01:41:26.907485+0000","last_became_active":"2024-04-25T01:26:02.416189+0000","last_became_peered":"2024-04-25T01:26:02.416189+0000","last_unstale":"2024-04-25T01:41:26.907485+0000","last_undegraded":"2024-04-25T01:41:26.907485+0000","last_fullsized":"2024-04-25T01:41:26.907485+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4082","last_scrub_stamp":"2024-04-25T01:41:26.907376+0000","last_deep_scrub":"26'4082","last_deep_scrub_stamp":"2024-04-25T01:41:26.907376+0000","last_clean_scrub_stamp":"2024-04-25T01:41:26.907376+0000","objects_scrubbed":0,"log_size":4082,"log_dups_size":0,"ondisk_log_size":4082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:08.376864+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20562,"num_write_kb":274915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"26'5666","reported_seq":5705,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:22.042352+0000","last_change":"2024-04-25T01:41:22.042392+0000","last_active":"2024-04-25T01:41:22.042352+0000","last_peered":"2024-04-25T01:41:22.042352+0000","last_clean":"2024-04-25T01:41:22.042352+0000","last_became_active":"2024-04-25T01:25:53.370074+0000","last_became_peered":"2024-04-25T01:25:53.370074+0000","last_unstale":"2024-04-25T01:41:22.042352+0000","last_undegraded":"2024-04-25T01:41:22.042352+0000","last_fullsized":"2024-04-25T01:41:22.042352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5666","last_scrub_stamp":"2024-04-25T01:41:22.042219+0000","last_deep_scrub":"26'5666","last_deep_scrub_stamp":"2024-04-25T01:41:22.042219+0000","last_clean_scrub_stamp":"2024-04-25T01:41:22.042219+0000","objects_scrubbed":0,"log_size":5666,"log_dups_size":0,"ondisk_log_size":5666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:02.992114+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29161,"num_write_kb":354533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"26'7","reported_seq":49,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:23.022583+0000","last_change":"2024-04-25T01:41:23.022583+0000","last_active":"2024-04-25T01:41:23.022583+0000","last_peered":"2024-04-25T01:41:23.022583+0000","last_clean":"2024-04-25T01:41:23.022583+0000","last_became_active":"2024-04-25T01:25:52.345717+0000","last_became_peered":"2024-04-25T01:25:52.345717+0000","last_unstale":"2024-04-25T01:41:23.022583+0000","last_undegraded":"2024-04-25T01:41:23.022583+0000","last_fullsized":"2024-04-25T01:41:23.022583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'7","last_scrub_stamp":"2024-04-25T01:41:23.022457+0000","last_deep_scrub":"26'7","last_deep_scrub_stamp":"2024-04-25T01:41:23.022457+0000","last_clean_scrub_stamp":"2024-04-25T01:41:23.022457+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:56.658486+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"26'3","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:26.369244+0000","last_change":"2024-04-25T01:41:26.369270+0000","last_active":"2024-04-25T01:41:26.369244+0000","last_peered":"2024-04-25T01:41:26.369244+0000","last_clean":"2024-04-25T01:41:26.369244+0000","last_became_active":"2024-04-25T01:26:01.415750+0000","last_became_peered":"2024-04-25T01:26:01.415750+0000","last_unstale":"2024-04-25T01:41:26.369244+0000","last_undegraded":"2024-04-25T01:41:26.369244+0000","last_fullsized":"2024-04-25T01:41:26.369244+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:26.369152+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:26.369152+0000","last_clean_scrub_stamp":"2024-04-25T01:41:26.369152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:40.048159+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"26'4748","reported_seq":4780,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:20.027499+0000","last_change":"2024-04-25T01:41:20.027538+0000","last_active":"2024-04-25T01:41:20.027499+0000","last_peered":"2024-04-25T01:41:20.027499+0000","last_clean":"2024-04-25T01:41:20.027499+0000","last_became_active":"2024-04-25T01:26:02.414316+0000","last_became_peered":"2024-04-25T01:26:02.414316+0000","last_unstale":"2024-04-25T01:41:20.027499+0000","last_undegraded":"2024-04-25T01:41:20.027499+0000","last_fullsized":"2024-04-25T01:41:20.027499+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4748","last_scrub_stamp":"2024-04-25T01:41:20.027380+0000","last_deep_scrub":"26'4748","last_deep_scrub_stamp":"2024-04-25T01:41:20.027380+0000","last_clean_scrub_stamp":"2024-04-25T01:41:20.027380+0000","objects_scrubbed":0,"log_size":4748,"log_dups_size":0,"ondisk_log_size":4748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:32.660618+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24613,"num_write_kb":303491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"26'4546","reported_seq":4585,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:33.826869+0000","last_change":"2024-04-25T01:41:33.826897+0000","last_active":"2024-04-25T01:41:33.826869+0000","last_peered":"2024-04-25T01:41:33.826869+0000","last_clean":"2024-04-25T01:41:33.826869+0000","last_became_active":"2024-04-25T01:25:53.364049+0000","last_became_peered":"2024-04-25T01:25:53.364049+0000","last_unstale":"2024-04-25T01:41:33.826869+0000","last_undegraded":"2024-04-25T01:41:33.826869+0000","last_fullsized":"2024-04-25T01:41:33.826869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4546","last_scrub_stamp":"2024-04-25T01:41:33.826752+0000","last_deep_scrub":"26'4546","last_deep_scrub_stamp":"2024-04-25T01:41:33.826752+0000","last_clean_scrub_stamp":"2024-04-25T01:41:33.826752+0000","objects_scrubbed":0,"log_size":4546,"log_dups_size":0,"ondisk_log_size":4546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:46.598420+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22740,"num_write_kb":287925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'6","reported_seq":47,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:28.162892+0000","last_change":"2024-04-25T01:41:28.162920+0000","last_active":"2024-04-25T01:41:28.162892+0000","last_peered":"2024-04-25T01:41:28.162892+0000","last_clean":"2024-04-25T01:41:28.162892+0000","last_became_active":"2024-04-25T01:25:52.350357+0000","last_became_peered":"2024-04-25T01:25:52.350357+0000","last_unstale":"2024-04-25T01:41:28.162892+0000","last_undegraded":"2024-04-25T01:41:28.162892+0000","last_fullsized":"2024-04-25T01:41:28.162892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'6","last_scrub_stamp":"2024-04-25T01:41:28.162801+0000","last_deep_scrub":"26'6","last_deep_scrub_stamp":"2024-04-25T01:41:28.162801+0000","last_clean_scrub_stamp":"2024-04-25T01:41:28.162801+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:02.453765+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"26'3","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:26.080366+0000","last_change":"2024-04-25T01:41:26.080395+0000","last_active":"2024-04-25T01:41:26.080366+0000","last_peered":"2024-04-25T01:41:26.080366+0000","last_clean":"2024-04-25T01:41:26.080366+0000","last_became_active":"2024-04-25T01:26:01.418179+0000","last_became_peered":"2024-04-25T01:26:01.418179+0000","last_unstale":"2024-04-25T01:41:26.080366+0000","last_undegraded":"2024-04-25T01:41:26.080366+0000","last_fullsized":"2024-04-25T01:41:26.080366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:26.080274+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:26.080274+0000","last_clean_scrub_stamp":"2024-04-25T01:41:26.080274+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:36.463932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"26'3833","reported_seq":3856,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.301262+0000","last_change":"2024-04-25T01:26:02.418424+0000","last_active":"2024-04-25T01:41:09.301262+0000","last_peered":"2024-04-25T01:41:09.301262+0000","last_clean":"2024-04-25T01:41:09.301262+0000","last_became_active":"2024-04-25T01:26:02.418090+0000","last_became_peered":"2024-04-25T01:26:02.418090+0000","last_unstale":"2024-04-25T01:41:09.301262+0000","last_undegraded":"2024-04-25T01:41:09.301262+0000","last_fullsized":"2024-04-25T01:41:09.301262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3833,"log_dups_size":0,"ondisk_log_size":3833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19507,"num_write_kb":254331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"26'4627","reported_seq":4666,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:31.860712+0000","last_change":"2024-04-25T01:41:31.860739+0000","last_active":"2024-04-25T01:41:31.860712+0000","last_peered":"2024-04-25T01:41:31.860712+0000","last_clean":"2024-04-25T01:41:31.860712+0000","last_became_active":"2024-04-25T01:25:53.365715+0000","last_became_peered":"2024-04-25T01:25:53.365715+0000","last_unstale":"2024-04-25T01:41:31.860712+0000","last_undegraded":"2024-04-25T01:41:31.860712+0000","last_fullsized":"2024-04-25T01:41:31.860712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4627","last_scrub_stamp":"2024-04-25T01:41:31.860598+0000","last_deep_scrub":"26'4627","last_deep_scrub_stamp":"2024-04-25T01:41:31.860598+0000","last_clean_scrub_stamp":"2024-04-25T01:41:31.860598+0000","objects_scrubbed":0,"log_size":4627,"log_dups_size":0,"ondisk_log_size":4627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:00.617224+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24780,"num_write_kb":300200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"26'15","reported_seq":56,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:24.073916+0000","last_change":"2024-04-25T01:41:24.073978+0000","last_active":"2024-04-25T01:41:24.073916+0000","last_peered":"2024-04-25T01:41:24.073916+0000","last_clean":"2024-04-25T01:41:24.073916+0000","last_became_active":"2024-04-25T01:25:52.347144+0000","last_became_peered":"2024-04-25T01:25:52.347144+0000","last_unstale":"2024-04-25T01:41:24.073916+0000","last_undegraded":"2024-04-25T01:41:24.073916+0000","last_fullsized":"2024-04-25T01:41:24.073916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'15","last_scrub_stamp":"2024-04-25T01:41:24.073796+0000","last_deep_scrub":"26'15","last_deep_scrub_stamp":"2024-04-25T01:41:24.073796+0000","last_clean_scrub_stamp":"2024-04-25T01:41:24.073796+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:43.520796+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"26'2","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:31.735406+0000","last_change":"2024-04-25T01:41:31.735434+0000","last_active":"2024-04-25T01:41:31.735406+0000","last_peered":"2024-04-25T01:41:31.735406+0000","last_clean":"2024-04-25T01:41:31.735406+0000","last_became_active":"2024-04-25T01:26:01.409654+0000","last_became_peered":"2024-04-25T01:26:01.409654+0000","last_unstale":"2024-04-25T01:41:31.735406+0000","last_undegraded":"2024-04-25T01:41:31.735406+0000","last_fullsized":"2024-04-25T01:41:31.735406+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'2","last_scrub_stamp":"2024-04-25T01:41:31.735321+0000","last_deep_scrub":"26'2","last_deep_scrub_stamp":"2024-04-25T01:41:31.735321+0000","last_clean_scrub_stamp":"2024-04-25T01:41:31.735321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:13.204808+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"26'4668","reported_seq":4691,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.342255+0000","last_change":"2024-04-25T01:26:02.425103+0000","last_active":"2024-04-25T01:41:09.342255+0000","last_peered":"2024-04-25T01:41:09.342255+0000","last_clean":"2024-04-25T01:41:09.342255+0000","last_became_active":"2024-04-25T01:26:02.424819+0000","last_became_peered":"2024-04-25T01:26:02.424819+0000","last_unstale":"2024-04-25T01:41:09.342255+0000","last_undegraded":"2024-04-25T01:41:09.342255+0000","last_fullsized":"2024-04-25T01:41:09.342255+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4668,"log_dups_size":0,"ondisk_log_size":4668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23987,"num_write_kb":301090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"26'5240","reported_seq":5279,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:30.323624+0000","last_change":"2024-04-25T01:41:30.323670+0000","last_active":"2024-04-25T01:41:30.323624+0000","last_peered":"2024-04-25T01:41:30.323624+0000","last_clean":"2024-04-25T01:41:30.323624+0000","last_became_active":"2024-04-25T01:25:53.364431+0000","last_became_peered":"2024-04-25T01:25:53.364431+0000","last_unstale":"2024-04-25T01:41:30.323624+0000","last_undegraded":"2024-04-25T01:41:30.323624+0000","last_fullsized":"2024-04-25T01:41:30.323624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5240","last_scrub_stamp":"2024-04-25T01:41:30.323520+0000","last_deep_scrub":"26'5240","last_deep_scrub_stamp":"2024-04-25T01:41:30.323520+0000","last_clean_scrub_stamp":"2024-04-25T01:41:30.323520+0000","objects_scrubbed":0,"log_size":5240,"log_dups_size":0,"ondisk_log_size":5240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:40.707179+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27403,"num_write_kb":326994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'6","reported_seq":47,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:27.233476+0000","last_change":"2024-04-25T01:41:27.233503+0000","last_active":"2024-04-25T01:41:27.233476+0000","last_peered":"2024-04-25T01:41:27.233476+0000","last_clean":"2024-04-25T01:41:27.233476+0000","last_became_active":"2024-04-25T01:25:52.350946+0000","last_became_peered":"2024-04-25T01:25:52.350946+0000","last_unstale":"2024-04-25T01:41:27.233476+0000","last_undegraded":"2024-04-25T01:41:27.233476+0000","last_fullsized":"2024-04-25T01:41:27.233476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'6","last_scrub_stamp":"2024-04-25T01:41:27.233383+0000","last_deep_scrub":"26'6","last_deep_scrub_stamp":"2024-04-25T01:41:27.233383+0000","last_clean_scrub_stamp":"2024-04-25T01:41:27.233383+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:03.290277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"26'2","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:20.792201+0000","last_change":"2024-04-25T01:41:20.792222+0000","last_active":"2024-04-25T01:41:20.792201+0000","last_peered":"2024-04-25T01:41:20.792201+0000","last_clean":"2024-04-25T01:41:20.792201+0000","last_became_active":"2024-04-25T01:26:01.415572+0000","last_became_peered":"2024-04-25T01:26:01.415572+0000","last_unstale":"2024-04-25T01:41:20.792201+0000","last_undegraded":"2024-04-25T01:41:20.792201+0000","last_fullsized":"2024-04-25T01:41:20.792201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'2","last_scrub_stamp":"2024-04-25T01:41:20.792117+0000","last_deep_scrub":"26'2","last_deep_scrub_stamp":"2024-04-25T01:41:20.792117+0000","last_clean_scrub_stamp":"2024-04-25T01:41:20.792117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:27.534967+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"26'4341","reported_seq":4364,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.334385+0000","last_change":"2024-04-25T01:26:02.422959+0000","last_active":"2024-04-25T01:41:09.334385+0000","last_peered":"2024-04-25T01:41:09.334385+0000","last_clean":"2024-04-25T01:41:09.334385+0000","last_became_active":"2024-04-25T01:26:02.422210+0000","last_became_peered":"2024-04-25T01:26:02.422210+0000","last_unstale":"2024-04-25T01:41:09.334385+0000","last_undegraded":"2024-04-25T01:41:09.334385+0000","last_fullsized":"2024-04-25T01:41:09.334385+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4341,"log_dups_size":0,"ondisk_log_size":4341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22600,"num_write_kb":283130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"26'4919","reported_seq":4958,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:25.956327+0000","last_change":"2024-04-25T01:41:25.956355+0000","last_active":"2024-04-25T01:41:25.956327+0000","last_peered":"2024-04-25T01:41:25.956327+0000","last_clean":"2024-04-25T01:41:25.956327+0000","last_became_active":"2024-04-25T01:25:53.367664+0000","last_became_peered":"2024-04-25T01:25:53.367664+0000","last_unstale":"2024-04-25T01:41:25.956327+0000","last_undegraded":"2024-04-25T01:41:25.956327+0000","last_fullsized":"2024-04-25T01:41:25.956327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4919","last_scrub_stamp":"2024-04-25T01:41:25.956179+0000","last_deep_scrub":"26'4919","last_deep_scrub_stamp":"2024-04-25T01:41:25.956179+0000","last_clean_scrub_stamp":"2024-04-25T01:41:25.956179+0000","objects_scrubbed":0,"log_size":4919,"log_dups_size":0,"ondisk_log_size":4919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:00.247143+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25274,"num_write_kb":330818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"26'5","reported_seq":46,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:35.320714+0000","last_change":"2024-04-25T01:41:35.320753+0000","last_active":"2024-04-25T01:41:35.320714+0000","last_peered":"2024-04-25T01:41:35.320714+0000","last_clean":"2024-04-25T01:41:35.320714+0000","last_became_active":"2024-04-25T01:25:52.356881+0000","last_became_peered":"2024-04-25T01:25:52.356881+0000","last_unstale":"2024-04-25T01:41:35.320714+0000","last_undegraded":"2024-04-25T01:41:35.320714+0000","last_fullsized":"2024-04-25T01:41:35.320714+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-04-25T01:41:35.320622+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-04-25T01:41:35.320622+0000","last_clean_scrub_stamp":"2024-04-25T01:41:35.320622+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:18.488101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"26'317","reported_seq":350,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:26.902327+0000","last_change":"2024-04-25T01:41:26.902363+0000","last_active":"2024-04-25T01:41:26.902327+0000","last_peered":"2024-04-25T01:41:26.902327+0000","last_clean":"2024-04-25T01:41:26.902327+0000","last_became_active":"2024-04-25T01:26:01.419526+0000","last_became_peered":"2024-04-25T01:26:01.419526+0000","last_unstale":"2024-04-25T01:41:26.902327+0000","last_undegraded":"2024-04-25T01:41:26.902327+0000","last_fullsized":"2024-04-25T01:41:26.902327+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'317","last_scrub_stamp":"2024-04-25T01:41:26.902232+0000","last_deep_scrub":"26'317","last_deep_scrub_stamp":"2024-04-25T01:41:26.902232+0000","last_clean_scrub_stamp":"2024-04-25T01:41:26.902232+0000","objects_scrubbed":4,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:49.118751+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":310,"num_write_kb":4240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"26'6253","reported_seq":6276,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.326246+0000","last_change":"2024-04-25T01:26:02.421488+0000","last_active":"2024-04-25T01:41:09.326246+0000","last_peered":"2024-04-25T01:41:09.326246+0000","last_clean":"2024-04-25T01:41:09.326246+0000","last_became_active":"2024-04-25T01:26:02.419308+0000","last_became_peered":"2024-04-25T01:26:02.419308+0000","last_unstale":"2024-04-25T01:41:09.326246+0000","last_undegraded":"2024-04-25T01:41:09.326246+0000","last_fullsized":"2024-04-25T01:41:09.326246+0000","mapping_epoch":22,"log_start":"26'1000","ondisk_log_start":"26'1000","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5253,"log_dups_size":0,"ondisk_log_size":5253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32276,"num_write_kb":400144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"26'4645","reported_seq":4684,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:30.728029+0000","last_change":"2024-04-25T01:41:30.728055+0000","last_active":"2024-04-25T01:41:30.728029+0000","last_peered":"2024-04-25T01:41:30.728029+0000","last_clean":"2024-04-25T01:41:30.728029+0000","last_became_active":"2024-04-25T01:25:53.370142+0000","last_became_peered":"2024-04-25T01:25:53.370142+0000","last_unstale":"2024-04-25T01:41:30.728029+0000","last_undegraded":"2024-04-25T01:41:30.728029+0000","last_fullsized":"2024-04-25T01:41:30.728029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4645","last_scrub_stamp":"2024-04-25T01:41:30.727918+0000","last_deep_scrub":"26'4645","last_deep_scrub_stamp":"2024-04-25T01:41:30.727918+0000","last_clean_scrub_stamp":"2024-04-25T01:41:30.727918+0000","objects_scrubbed":0,"log_size":4645,"log_dups_size":0,"ondisk_log_size":4645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:05.018882+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24130,"num_write_kb":289357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"26'10","reported_seq":51,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:23.680853+0000","last_change":"2024-04-25T01:41:23.680907+0000","last_active":"2024-04-25T01:41:23.680853+0000","last_peered":"2024-04-25T01:41:23.680853+0000","last_clean":"2024-04-25T01:41:23.680853+0000","last_became_active":"2024-04-25T01:25:52.348846+0000","last_became_peered":"2024-04-25T01:25:52.348846+0000","last_unstale":"2024-04-25T01:41:23.680853+0000","last_undegraded":"2024-04-25T01:41:23.680853+0000","last_fullsized":"2024-04-25T01:41:23.680853+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'10","last_scrub_stamp":"2024-04-25T01:41:23.680737+0000","last_deep_scrub":"26'10","last_deep_scrub_stamp":"2024-04-25T01:41:23.680737+0000","last_clean_scrub_stamp":"2024-04-25T01:41:23.680737+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:22.005464+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"26'28","reported_seq":61,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:24.359447+0000","last_change":"2024-04-25T01:41:24.359476+0000","last_active":"2024-04-25T01:41:24.359447+0000","last_peered":"2024-04-25T01:41:24.359447+0000","last_clean":"2024-04-25T01:41:24.359447+0000","last_became_active":"2024-04-25T01:26:01.414243+0000","last_became_peered":"2024-04-25T01:26:01.414243+0000","last_unstale":"2024-04-25T01:41:24.359447+0000","last_undegraded":"2024-04-25T01:41:24.359447+0000","last_fullsized":"2024-04-25T01:41:24.359447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'28","last_scrub_stamp":"2024-04-25T01:41:24.359370+0000","last_deep_scrub":"26'28","last_deep_scrub_stamp":"2024-04-25T01:41:24.359370+0000","last_clean_scrub_stamp":"2024-04-25T01:41:24.359370+0000","objects_scrubbed":2,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:12.600921+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"26'5295","reported_seq":5318,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.307133+0000","last_change":"2024-04-25T01:26:02.415893+0000","last_active":"2024-04-25T01:41:09.307133+0000","last_peered":"2024-04-25T01:41:09.307133+0000","last_clean":"2024-04-25T01:41:09.307133+0000","last_became_active":"2024-04-25T01:26:02.415596+0000","last_became_peered":"2024-04-25T01:26:02.415596+0000","last_unstale":"2024-04-25T01:41:09.307133+0000","last_undegraded":"2024-04-25T01:41:09.307133+0000","last_fullsized":"2024-04-25T01:41:09.307133+0000","mapping_epoch":22,"log_start":"26'100","ondisk_log_start":"26'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5195,"log_dups_size":0,"ondisk_log_size":5195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26962,"num_write_kb":337426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"26'4577","reported_seq":4616,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:23.019000+0000","last_change":"2024-04-25T01:41:23.019028+0000","last_active":"2024-04-25T01:41:23.019000+0000","last_peered":"2024-04-25T01:41:23.019000+0000","last_clean":"2024-04-25T01:41:23.019000+0000","last_became_active":"2024-04-25T01:25:53.373129+0000","last_became_peered":"2024-04-25T01:25:53.373129+0000","last_unstale":"2024-04-25T01:41:23.019000+0000","last_undegraded":"2024-04-25T01:41:23.019000+0000","last_fullsized":"2024-04-25T01:41:23.019000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4577","last_scrub_stamp":"2024-04-25T01:41:23.018890+0000","last_deep_scrub":"26'4577","last_deep_scrub_stamp":"2024-04-25T01:41:23.018890+0000","last_clean_scrub_stamp":"2024-04-25T01:41:23.018890+0000","objects_scrubbed":0,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:31.414954+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23671,"num_write_kb":287382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'339","reported_seq":380,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:26.289937+0000","last_change":"2024-04-25T01:41:26.289971+0000","last_active":"2024-04-25T01:41:26.289937+0000","last_peered":"2024-04-25T01:41:26.289937+0000","last_clean":"2024-04-25T01:41:26.289937+0000","last_became_active":"2024-04-25T01:25:52.349448+0000","last_became_peered":"2024-04-25T01:25:52.349448+0000","last_unstale":"2024-04-25T01:41:26.289937+0000","last_undegraded":"2024-04-25T01:41:26.289937+0000","last_fullsized":"2024-04-25T01:41:26.289937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'339","last_scrub_stamp":"2024-04-25T01:41:26.289844+0000","last_deep_scrub":"26'339","last_deep_scrub_stamp":"2024-04-25T01:41:26.289844+0000","last_clean_scrub_stamp":"2024-04-25T01:41:26.289844+0000","objects_scrubbed":4,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:14.160747+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":331,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"26'5","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:26.595786+0000","last_change":"2024-04-25T01:41:26.595813+0000","last_active":"2024-04-25T01:41:26.595786+0000","last_peered":"2024-04-25T01:41:26.595786+0000","last_clean":"2024-04-25T01:41:26.595786+0000","last_became_active":"2024-04-25T01:26:01.418541+0000","last_became_peered":"2024-04-25T01:26:01.418541+0000","last_unstale":"2024-04-25T01:41:26.595786+0000","last_undegraded":"2024-04-25T01:41:26.595786+0000","last_fullsized":"2024-04-25T01:41:26.595786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-04-25T01:41:26.595675+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-04-25T01:41:26.595675+0000","last_clean_scrub_stamp":"2024-04-25T01:41:26.595675+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:08.065163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"26'4257","reported_seq":4288,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:40.649356+0000","last_change":"2024-04-25T01:41:40.649384+0000","last_active":"2024-04-25T01:41:40.649356+0000","last_peered":"2024-04-25T01:41:40.649356+0000","last_clean":"2024-04-25T01:41:40.649356+0000","last_became_active":"2024-04-25T01:26:02.425106+0000","last_became_peered":"2024-04-25T01:26:02.425106+0000","last_unstale":"2024-04-25T01:41:40.649356+0000","last_undegraded":"2024-04-25T01:41:40.649356+0000","last_fullsized":"2024-04-25T01:41:40.649356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4257","last_scrub_stamp":"2024-04-25T01:41:40.649245+0000","last_deep_scrub":"26'4257","last_deep_scrub_stamp":"2024-04-25T01:41:40.649245+0000","last_clean_scrub_stamp":"2024-04-25T01:41:40.649245+0000","objects_scrubbed":0,"log_size":4257,"log_dups_size":0,"ondisk_log_size":4257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:14.543796+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22152,"num_write_kb":281377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"26'6968","reported_seq":7007,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:32.135675+0000","last_change":"2024-04-25T01:41:32.135701+0000","last_active":"2024-04-25T01:41:32.135675+0000","last_peered":"2024-04-25T01:41:32.135675+0000","last_clean":"2024-04-25T01:41:32.135675+0000","last_became_active":"2024-04-25T01:25:53.366157+0000","last_became_peered":"2024-04-25T01:25:53.366157+0000","last_unstale":"2024-04-25T01:41:32.135675+0000","last_undegraded":"2024-04-25T01:41:32.135675+0000","last_fullsized":"2024-04-25T01:41:32.135675+0000","mapping_epoch":18,"log_start":"26'1500","ondisk_log_start":"26'1500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'6968","last_scrub_stamp":"2024-04-25T01:41:32.135560+0000","last_deep_scrub":"26'6968","last_deep_scrub_stamp":"2024-04-25T01:41:32.135560+0000","last_clean_scrub_stamp":"2024-04-25T01:41:32.135560+0000","objects_scrubbed":0,"log_size":5468,"log_dups_size":0,"ondisk_log_size":5468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:20.687525+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35763,"num_write_kb":414188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"26'27","reported_seq":68,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:33.577988+0000","last_change":"2024-04-25T01:41:33.578018+0000","last_active":"2024-04-25T01:41:33.577988+0000","last_peered":"2024-04-25T01:41:33.577988+0000","last_clean":"2024-04-25T01:41:33.577988+0000","last_became_active":"2024-04-25T01:25:52.357355+0000","last_became_peered":"2024-04-25T01:25:52.357355+0000","last_unstale":"2024-04-25T01:41:33.577988+0000","last_undegraded":"2024-04-25T01:41:33.577988+0000","last_fullsized":"2024-04-25T01:41:33.577988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'27","last_scrub_stamp":"2024-04-25T01:41:33.577900+0000","last_deep_scrub":"26'27","last_deep_scrub_stamp":"2024-04-25T01:41:33.577900+0000","last_clean_scrub_stamp":"2024-04-25T01:41:33.577900+0000","objects_scrubbed":2,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:46.349568+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"26'10","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:24.163663+0000","last_change":"2024-04-25T01:41:24.163692+0000","last_active":"2024-04-25T01:41:24.163663+0000","last_peered":"2024-04-25T01:41:24.163663+0000","last_clean":"2024-04-25T01:41:24.163663+0000","last_became_active":"2024-04-25T01:26:01.412132+0000","last_became_peered":"2024-04-25T01:26:01.412132+0000","last_unstale":"2024-04-25T01:41:24.163663+0000","last_undegraded":"2024-04-25T01:41:24.163663+0000","last_fullsized":"2024-04-25T01:41:24.163663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'10","last_scrub_stamp":"2024-04-25T01:41:24.163574+0000","last_deep_scrub":"26'10","last_deep_scrub_stamp":"2024-04-25T01:41:24.163574+0000","last_clean_scrub_stamp":"2024-04-25T01:41:24.163574+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:05.113469+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"26'5141","reported_seq":5172,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:36.238761+0000","last_change":"2024-04-25T01:41:36.238802+0000","last_active":"2024-04-25T01:41:36.238761+0000","last_peered":"2024-04-25T01:41:36.238761+0000","last_clean":"2024-04-25T01:41:36.238761+0000","last_became_active":"2024-04-25T01:26:02.412759+0000","last_became_peered":"2024-04-25T01:26:02.412759+0000","last_unstale":"2024-04-25T01:41:36.238761+0000","last_undegraded":"2024-04-25T01:41:36.238761+0000","last_fullsized":"2024-04-25T01:41:36.238761+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5141","last_scrub_stamp":"2024-04-25T01:41:36.238638+0000","last_deep_scrub":"26'5141","last_deep_scrub_stamp":"2024-04-25T01:41:36.238638+0000","last_clean_scrub_stamp":"2024-04-25T01:41:36.238638+0000","objects_scrubbed":0,"log_size":5141,"log_dups_size":0,"ondisk_log_size":5141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:24.790603+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27471,"num_write_kb":310956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"26'3595","reported_seq":3634,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:18.724568+0000","last_change":"2024-04-25T01:41:18.724618+0000","last_active":"2024-04-25T01:41:18.724568+0000","last_peered":"2024-04-25T01:41:18.724568+0000","last_clean":"2024-04-25T01:41:18.724568+0000","last_became_active":"2024-04-25T01:25:53.373890+0000","last_became_peered":"2024-04-25T01:25:53.373890+0000","last_unstale":"2024-04-25T01:41:18.724568+0000","last_undegraded":"2024-04-25T01:41:18.724568+0000","last_fullsized":"2024-04-25T01:41:18.724568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3595","last_scrub_stamp":"2024-04-25T01:41:18.724464+0000","last_deep_scrub":"26'3595","last_deep_scrub_stamp":"2024-04-25T01:41:18.724464+0000","last_clean_scrub_stamp":"2024-04-25T01:41:18.724464+0000","objects_scrubbed":0,"log_size":3595,"log_dups_size":0,"ondisk_log_size":3595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:50.933963+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18396,"num_write_kb":249899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"26'2","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:34.358052+0000","last_change":"2024-04-25T01:41:34.358073+0000","last_active":"2024-04-25T01:41:34.358052+0000","last_peered":"2024-04-25T01:41:34.358052+0000","last_clean":"2024-04-25T01:41:34.358052+0000","last_became_active":"2024-04-25T01:25:52.355913+0000","last_became_peered":"2024-04-25T01:25:52.355913+0000","last_unstale":"2024-04-25T01:41:34.358052+0000","last_undegraded":"2024-04-25T01:41:34.358052+0000","last_fullsized":"2024-04-25T01:41:34.358052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'2","last_scrub_stamp":"2024-04-25T01:41:34.357973+0000","last_deep_scrub":"26'2","last_deep_scrub_stamp":"2024-04-25T01:41:34.357973+0000","last_clean_scrub_stamp":"2024-04-25T01:41:34.357973+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:33.500655+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"26'16","reported_seq":49,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:24.878084+0000","last_change":"2024-04-25T01:41:24.878119+0000","last_active":"2024-04-25T01:41:24.878084+0000","last_peered":"2024-04-25T01:41:24.878084+0000","last_clean":"2024-04-25T01:41:24.878084+0000","last_became_active":"2024-04-25T01:26:01.419015+0000","last_became_peered":"2024-04-25T01:26:01.419015+0000","last_unstale":"2024-04-25T01:41:24.878084+0000","last_undegraded":"2024-04-25T01:41:24.878084+0000","last_fullsized":"2024-04-25T01:41:24.878084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'16","last_scrub_stamp":"2024-04-25T01:41:24.878001+0000","last_deep_scrub":"26'16","last_deep_scrub_stamp":"2024-04-25T01:41:24.878001+0000","last_clean_scrub_stamp":"2024-04-25T01:41:24.878001+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:38.557008+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"26'5013","reported_seq":5036,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.333491+0000","last_change":"2024-04-25T01:26:02.425241+0000","last_active":"2024-04-25T01:41:09.333491+0000","last_peered":"2024-04-25T01:41:09.333491+0000","last_clean":"2024-04-25T01:41:09.333491+0000","last_became_active":"2024-04-25T01:26:02.424960+0000","last_became_peered":"2024-04-25T01:26:02.424960+0000","last_unstale":"2024-04-25T01:41:09.333491+0000","last_undegraded":"2024-04-25T01:41:09.333491+0000","last_fullsized":"2024-04-25T01:41:09.333491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5013,"log_dups_size":0,"ondisk_log_size":5013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25777,"num_write_kb":320486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"26'4876","reported_seq":4915,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:23.375681+0000","last_change":"2024-04-25T01:41:23.375705+0000","last_active":"2024-04-25T01:41:23.375681+0000","last_peered":"2024-04-25T01:41:23.375681+0000","last_clean":"2024-04-25T01:41:23.375681+0000","last_became_active":"2024-04-25T01:25:53.371086+0000","last_became_peered":"2024-04-25T01:25:53.371086+0000","last_unstale":"2024-04-25T01:41:23.375681+0000","last_undegraded":"2024-04-25T01:41:23.375681+0000","last_fullsized":"2024-04-25T01:41:23.375681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4876","last_scrub_stamp":"2024-04-25T01:41:23.375592+0000","last_deep_scrub":"26'4876","last_deep_scrub_stamp":"2024-04-25T01:41:23.375592+0000","last_clean_scrub_stamp":"2024-04-25T01:41:23.375592+0000","objects_scrubbed":0,"log_size":4876,"log_dups_size":0,"ondisk_log_size":4876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:46.392469+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24656,"num_write_kb":303790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"26'2","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:29.305845+0000","last_change":"2024-04-25T01:41:29.305917+0000","last_active":"2024-04-25T01:41:29.305845+0000","last_peered":"2024-04-25T01:41:29.305845+0000","last_clean":"2024-04-25T01:41:29.305845+0000","last_became_active":"2024-04-25T01:25:52.357182+0000","last_became_peered":"2024-04-25T01:25:52.357182+0000","last_unstale":"2024-04-25T01:41:29.305845+0000","last_undegraded":"2024-04-25T01:41:29.305845+0000","last_fullsized":"2024-04-25T01:41:29.305845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'2","last_scrub_stamp":"2024-04-25T01:41:29.305722+0000","last_deep_scrub":"26'2","last_deep_scrub_stamp":"2024-04-25T01:41:29.305722+0000","last_clean_scrub_stamp":"2024-04-25T01:41:29.305722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:37.701785+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"26'6","reported_seq":39,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:21.821547+0000","last_change":"2024-04-25T01:41:21.821595+0000","last_active":"2024-04-25T01:41:21.821547+0000","last_peered":"2024-04-25T01:41:21.821547+0000","last_clean":"2024-04-25T01:41:21.821547+0000","last_became_active":"2024-04-25T01:26:01.415020+0000","last_became_peered":"2024-04-25T01:26:01.415020+0000","last_unstale":"2024-04-25T01:41:21.821547+0000","last_undegraded":"2024-04-25T01:41:21.821547+0000","last_fullsized":"2024-04-25T01:41:21.821547+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'6","last_scrub_stamp":"2024-04-25T01:41:21.821446+0000","last_deep_scrub":"26'6","last_deep_scrub_stamp":"2024-04-25T01:41:21.821446+0000","last_clean_scrub_stamp":"2024-04-25T01:41:21.821446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:44.838323+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"26'5316","reported_seq":5340,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.331403+0000","last_change":"2024-04-25T01:26:02.416917+0000","last_active":"2024-04-25T01:41:09.331403+0000","last_peered":"2024-04-25T01:41:09.331403+0000","last_clean":"2024-04-25T01:41:09.331403+0000","last_became_active":"2024-04-25T01:26:02.416623+0000","last_became_peered":"2024-04-25T01:26:02.416623+0000","last_unstale":"2024-04-25T01:41:09.331403+0000","last_undegraded":"2024-04-25T01:41:09.331403+0000","last_fullsized":"2024-04-25T01:41:09.331403+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5316,"log_dups_size":0,"ondisk_log_size":5316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27734,"num_write_kb":356956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"26'8","reported_seq":42,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:32.742444+0000","last_change":"2024-04-25T01:41:32.742469+0000","last_active":"2024-04-25T01:41:32.742444+0000","last_peered":"2024-04-25T01:41:32.742444+0000","last_clean":"2024-04-25T01:41:32.742444+0000","last_became_active":"2024-04-25T01:26:01.415184+0000","last_became_peered":"2024-04-25T01:26:01.415184+0000","last_unstale":"2024-04-25T01:41:32.742444+0000","last_undegraded":"2024-04-25T01:41:32.742444+0000","last_fullsized":"2024-04-25T01:41:32.742444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'8","last_scrub_stamp":"2024-04-25T01:41:32.742355+0000","last_deep_scrub":"26'8","last_deep_scrub_stamp":"2024-04-25T01:41:32.742355+0000","last_clean_scrub_stamp":"2024-04-25T01:41:32.742355+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:02.329346+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"26'5623","reported_seq":5662,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:35.555255+0000","last_change":"2024-04-25T01:41:35.555301+0000","last_active":"2024-04-25T01:41:35.555255+0000","last_peered":"2024-04-25T01:41:35.555255+0000","last_clean":"2024-04-25T01:41:35.555255+0000","last_became_active":"2024-04-25T01:25:53.362601+0000","last_became_peered":"2024-04-25T01:25:53.362601+0000","last_unstale":"2024-04-25T01:41:35.555255+0000","last_undegraded":"2024-04-25T01:41:35.555255+0000","last_fullsized":"2024-04-25T01:41:35.555255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5623","last_scrub_stamp":"2024-04-25T01:41:35.555135+0000","last_deep_scrub":"26'5623","last_deep_scrub_stamp":"2024-04-25T01:41:35.555135+0000","last_clean_scrub_stamp":"2024-04-25T01:41:35.555135+0000","objects_scrubbed":0,"log_size":5623,"log_dups_size":0,"ondisk_log_size":5623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:54.053742+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29125,"num_write_kb":354366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"26'3","reported_seq":44,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:36.321901+0000","last_change":"2024-04-25T01:41:36.321927+0000","last_active":"2024-04-25T01:41:36.321901+0000","last_peered":"2024-04-25T01:41:36.321901+0000","last_clean":"2024-04-25T01:41:36.321901+0000","last_became_active":"2024-04-25T01:25:52.357365+0000","last_became_peered":"2024-04-25T01:25:52.357365+0000","last_unstale":"2024-04-25T01:41:36.321901+0000","last_undegraded":"2024-04-25T01:41:36.321901+0000","last_fullsized":"2024-04-25T01:41:36.321901+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:36.321804+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:36.321804+0000","last_clean_scrub_stamp":"2024-04-25T01:41:36.321804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:49.391623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"26'4779","reported_seq":4803,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.271127+0000","last_change":"2024-04-25T01:26:02.418872+0000","last_active":"2024-04-25T01:41:09.271127+0000","last_peered":"2024-04-25T01:41:09.271127+0000","last_clean":"2024-04-25T01:41:09.271127+0000","last_became_active":"2024-04-25T01:26:02.418614+0000","last_became_peered":"2024-04-25T01:26:02.418614+0000","last_unstale":"2024-04-25T01:41:09.271127+0000","last_undegraded":"2024-04-25T01:41:09.271127+0000","last_fullsized":"2024-04-25T01:41:09.271127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4779,"log_dups_size":0,"ondisk_log_size":4779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25207,"num_write_kb":318198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"26'635","reported_seq":669,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:34.306312+0000","last_change":"2024-04-25T01:41:34.306366+0000","last_active":"2024-04-25T01:41:34.306312+0000","last_peered":"2024-04-25T01:41:34.306312+0000","last_clean":"2024-04-25T01:41:34.306312+0000","last_became_active":"2024-04-25T01:26:01.410004+0000","last_became_peered":"2024-04-25T01:26:01.410004+0000","last_unstale":"2024-04-25T01:41:34.306312+0000","last_undegraded":"2024-04-25T01:41:34.306312+0000","last_fullsized":"2024-04-25T01:41:34.306312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'635","last_scrub_stamp":"2024-04-25T01:41:34.306218+0000","last_deep_scrub":"26'635","last_deep_scrub_stamp":"2024-04-25T01:41:34.306218+0000","last_clean_scrub_stamp":"2024-04-25T01:41:34.306218+0000","objects_scrubbed":3,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:03.893209+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":630,"num_write_kb":4461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"26'4416","reported_seq":4447,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.812183+0000","last_change":"2024-04-25T01:25:53.363794+0000","last_active":"2024-04-25T01:40:58.812183+0000","last_peered":"2024-04-25T01:40:58.812183+0000","last_clean":"2024-04-25T01:40:58.812183+0000","last_became_active":"2024-04-25T01:25:53.363509+0000","last_became_peered":"2024-04-25T01:25:53.363509+0000","last_unstale":"2024-04-25T01:40:58.812183+0000","last_undegraded":"2024-04-25T01:40:58.812183+0000","last_fullsized":"2024-04-25T01:40:58.812183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4416,"log_dups_size":0,"ondisk_log_size":4416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23348,"num_write_kb":294385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"26'3","reported_seq":44,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:34.894434+0000","last_change":"2024-04-25T01:41:34.894462+0000","last_active":"2024-04-25T01:41:34.894434+0000","last_peered":"2024-04-25T01:41:34.894434+0000","last_clean":"2024-04-25T01:41:34.894434+0000","last_became_active":"2024-04-25T01:25:52.347138+0000","last_became_peered":"2024-04-25T01:25:52.347138+0000","last_unstale":"2024-04-25T01:41:34.894434+0000","last_undegraded":"2024-04-25T01:41:34.894434+0000","last_fullsized":"2024-04-25T01:41:34.894434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:34.894343+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:34.894343+0000","last_clean_scrub_stamp":"2024-04-25T01:41:34.894343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:47.964161+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"26'5575","reported_seq":5598,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.363387+0000","last_change":"2024-04-25T01:26:02.425948+0000","last_active":"2024-04-25T01:41:09.363387+0000","last_peered":"2024-04-25T01:41:09.363387+0000","last_clean":"2024-04-25T01:41:09.363387+0000","last_became_active":"2024-04-25T01:26:02.424947+0000","last_became_peered":"2024-04-25T01:26:02.424947+0000","last_unstale":"2024-04-25T01:41:09.363387+0000","last_undegraded":"2024-04-25T01:41:09.363387+0000","last_fullsized":"2024-04-25T01:41:09.363387+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5575,"log_dups_size":0,"ondisk_log_size":5575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28821,"num_write_kb":361321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"26'87","reported_seq":128,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:25.280092+0000","last_change":"2024-04-25T01:41:25.280120+0000","last_active":"2024-04-25T01:41:25.280092+0000","last_peered":"2024-04-25T01:41:25.280092+0000","last_clean":"2024-04-25T01:41:25.280092+0000","last_became_active":"2024-04-25T01:25:52.349672+0000","last_became_peered":"2024-04-25T01:25:52.349672+0000","last_unstale":"2024-04-25T01:41:25.280092+0000","last_undegraded":"2024-04-25T01:41:25.280092+0000","last_fullsized":"2024-04-25T01:41:25.280092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'87","last_scrub_stamp":"2024-04-25T01:41:25.279997+0000","last_deep_scrub":"26'87","last_deep_scrub_stamp":"2024-04-25T01:41:25.279997+0000","last_clean_scrub_stamp":"2024-04-25T01:41:25.279997+0000","objects_scrubbed":1,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:40.457549+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":980823,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83,"num_write_kb":1015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"26'3832","reported_seq":3871,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:26.716835+0000","last_change":"2024-04-25T01:41:26.716862+0000","last_active":"2024-04-25T01:41:26.716835+0000","last_peered":"2024-04-25T01:41:26.716835+0000","last_clean":"2024-04-25T01:41:26.716835+0000","last_became_active":"2024-04-25T01:25:53.356015+0000","last_became_peered":"2024-04-25T01:25:53.356015+0000","last_unstale":"2024-04-25T01:41:26.716835+0000","last_undegraded":"2024-04-25T01:41:26.716835+0000","last_fullsized":"2024-04-25T01:41:26.716835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3832","last_scrub_stamp":"2024-04-25T01:41:26.716735+0000","last_deep_scrub":"26'3832","last_deep_scrub_stamp":"2024-04-25T01:41:26.716735+0000","last_clean_scrub_stamp":"2024-04-25T01:41:26.716735+0000","objects_scrubbed":0,"log_size":3832,"log_dups_size":0,"ondisk_log_size":3832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:07.666630+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19386,"num_write_kb":237359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"26'4","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:28.078083+0000","last_change":"2024-04-25T01:41:28.078113+0000","last_active":"2024-04-25T01:41:28.078083+0000","last_peered":"2024-04-25T01:41:28.078083+0000","last_clean":"2024-04-25T01:41:28.078083+0000","last_became_active":"2024-04-25T01:26:01.420003+0000","last_became_peered":"2024-04-25T01:26:01.420003+0000","last_unstale":"2024-04-25T01:41:28.078083+0000","last_undegraded":"2024-04-25T01:41:28.078083+0000","last_fullsized":"2024-04-25T01:41:28.078083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4","last_scrub_stamp":"2024-04-25T01:41:28.077984+0000","last_deep_scrub":"26'4","last_deep_scrub_stamp":"2024-04-25T01:41:28.077984+0000","last_clean_scrub_stamp":"2024-04-25T01:41:28.077984+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:02.920281+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"26'3175","reported_seq":3198,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.351085+0000","last_change":"2024-04-25T01:26:02.426327+0000","last_active":"2024-04-25T01:41:09.351085+0000","last_peered":"2024-04-25T01:41:09.351085+0000","last_clean":"2024-04-25T01:41:09.351085+0000","last_became_active":"2024-04-25T01:26:02.426054+0000","last_became_peered":"2024-04-25T01:26:02.426054+0000","last_unstale":"2024-04-25T01:41:09.351085+0000","last_undegraded":"2024-04-25T01:41:09.351085+0000","last_fullsized":"2024-04-25T01:41:09.351085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3175,"log_dups_size":0,"ondisk_log_size":3175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15854,"num_write_kb":211025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"26'606","reported_seq":647,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:36.921170+0000","last_change":"2024-04-25T01:41:36.921200+0000","last_active":"2024-04-25T01:41:36.921170+0000","last_peered":"2024-04-25T01:41:36.921170+0000","last_clean":"2024-04-25T01:41:36.921170+0000","last_became_active":"2024-04-25T01:25:52.353659+0000","last_became_peered":"2024-04-25T01:25:52.353659+0000","last_unstale":"2024-04-25T01:41:36.921170+0000","last_undegraded":"2024-04-25T01:41:36.921170+0000","last_fullsized":"2024-04-25T01:41:36.921170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'606","last_scrub_stamp":"2024-04-25T01:41:36.921029+0000","last_deep_scrub":"26'606","last_deep_scrub_stamp":"2024-04-25T01:41:36.921029+0000","last_clean_scrub_stamp":"2024-04-25T01:41:36.921029+0000","objects_scrubbed":1,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:22.062095+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":603,"num_write_kb":4239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"26'4191","reported_seq":4230,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:33.173961+0000","last_change":"2024-04-25T01:41:33.174004+0000","last_active":"2024-04-25T01:41:33.173961+0000","last_peered":"2024-04-25T01:41:33.173961+0000","last_clean":"2024-04-25T01:41:33.173961+0000","last_became_active":"2024-04-25T01:25:53.366302+0000","last_became_peered":"2024-04-25T01:25:53.366302+0000","last_unstale":"2024-04-25T01:41:33.173961+0000","last_undegraded":"2024-04-25T01:41:33.173961+0000","last_fullsized":"2024-04-25T01:41:33.173961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4191","last_scrub_stamp":"2024-04-25T01:41:33.173830+0000","last_deep_scrub":"26'4191","last_deep_scrub_stamp":"2024-04-25T01:41:33.173830+0000","last_clean_scrub_stamp":"2024-04-25T01:41:33.173830+0000","objects_scrubbed":0,"log_size":4191,"log_dups_size":0,"ondisk_log_size":4191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:16.285208+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21716,"num_write_kb":263159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"26'12","reported_seq":45,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:28.860317+0000","last_change":"2024-04-25T01:41:28.860346+0000","last_active":"2024-04-25T01:41:28.860317+0000","last_peered":"2024-04-25T01:41:28.860317+0000","last_clean":"2024-04-25T01:41:28.860317+0000","last_became_active":"2024-04-25T01:26:01.420713+0000","last_became_peered":"2024-04-25T01:26:01.420713+0000","last_unstale":"2024-04-25T01:41:28.860317+0000","last_undegraded":"2024-04-25T01:41:28.860317+0000","last_fullsized":"2024-04-25T01:41:28.860317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'12","last_scrub_stamp":"2024-04-25T01:41:28.860233+0000","last_deep_scrub":"26'12","last_deep_scrub_stamp":"2024-04-25T01:41:28.860233+0000","last_clean_scrub_stamp":"2024-04-25T01:41:28.860233+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:15.013771+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"26'4569","reported_seq":4593,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:15.701380+0000","last_change":"2024-04-25T01:26:02.419282+0000","last_active":"2024-04-25T01:41:15.701380+0000","last_peered":"2024-04-25T01:41:15.701380+0000","last_clean":"2024-04-25T01:41:15.701380+0000","last_became_active":"2024-04-25T01:26:02.418802+0000","last_became_peered":"2024-04-25T01:26:02.418802+0000","last_unstale":"2024-04-25T01:41:15.701380+0000","last_undegraded":"2024-04-25T01:41:15.701380+0000","last_fullsized":"2024-04-25T01:41:15.701380+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4569,"log_dups_size":0,"ondisk_log_size":4569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23161,"num_write_kb":277521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"26'6240","reported_seq":6280,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:31.325243+0000","last_change":"2024-04-25T01:41:31.325270+0000","last_active":"2024-04-25T01:41:31.325243+0000","last_peered":"2024-04-25T01:41:31.325243+0000","last_clean":"2024-04-25T01:41:31.325243+0000","last_became_active":"2024-04-25T01:25:53.366847+0000","last_became_peered":"2024-04-25T01:25:53.366847+0000","last_unstale":"2024-04-25T01:41:31.325243+0000","last_undegraded":"2024-04-25T01:41:31.325243+0000","last_fullsized":"2024-04-25T01:41:31.325243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'6240","last_scrub_stamp":"2024-04-25T01:41:31.325133+0000","last_deep_scrub":"26'6240","last_deep_scrub_stamp":"2024-04-25T01:41:31.325133+0000","last_clean_scrub_stamp":"2024-04-25T01:41:31.325133+0000","objects_scrubbed":0,"log_size":6240,"log_dups_size":0,"ondisk_log_size":6240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:45.057188+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32134,"num_write_kb":382834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"26'5302","reported_seq":5334,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:36.547741+0000","last_change":"2024-04-25T01:41:36.547808+0000","last_active":"2024-04-25T01:41:36.547741+0000","last_peered":"2024-04-25T01:41:36.547741+0000","last_clean":"2024-04-25T01:41:36.547741+0000","last_became_active":"2024-04-25T01:26:02.416170+0000","last_became_peered":"2024-04-25T01:26:02.416170+0000","last_unstale":"2024-04-25T01:41:36.547741+0000","last_undegraded":"2024-04-25T01:41:36.547741+0000","last_fullsized":"2024-04-25T01:41:36.547741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5302","last_scrub_stamp":"2024-04-25T01:41:36.547604+0000","last_deep_scrub":"26'5302","last_deep_scrub_stamp":"2024-04-25T01:41:36.547604+0000","last_clean_scrub_stamp":"2024-04-25T01:41:36.547604+0000","objects_scrubbed":0,"log_size":5302,"log_dups_size":0,"ondisk_log_size":5302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:12.300281+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27000,"num_write_kb":345927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"26'4854","reported_seq":4894,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:27.082588+0000","last_change":"2024-04-25T01:41:27.082615+0000","last_active":"2024-04-25T01:41:27.082588+0000","last_peered":"2024-04-25T01:41:27.082588+0000","last_clean":"2024-04-25T01:41:27.082588+0000","last_became_active":"2024-04-25T01:25:53.371080+0000","last_became_peered":"2024-04-25T01:25:53.371080+0000","last_unstale":"2024-04-25T01:41:27.082588+0000","last_undegraded":"2024-04-25T01:41:27.082588+0000","last_fullsized":"2024-04-25T01:41:27.082588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4854","last_scrub_stamp":"2024-04-25T01:41:27.082471+0000","last_deep_scrub":"26'4854","last_deep_scrub_stamp":"2024-04-25T01:41:27.082471+0000","last_clean_scrub_stamp":"2024-04-25T01:41:27.082471+0000","objects_scrubbed":0,"log_size":4854,"log_dups_size":0,"ondisk_log_size":4854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:42.260023+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24922,"num_write_kb":316898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"26'4098","reported_seq":4122,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:12.541790+0000","last_change":"2024-04-25T01:26:02.419688+0000","last_active":"2024-04-25T01:41:12.541790+0000","last_peered":"2024-04-25T01:41:12.541790+0000","last_clean":"2024-04-25T01:41:12.541790+0000","last_became_active":"2024-04-25T01:26:02.419417+0000","last_became_peered":"2024-04-25T01:26:02.419417+0000","last_unstale":"2024-04-25T01:41:12.541790+0000","last_undegraded":"2024-04-25T01:41:12.541790+0000","last_fullsized":"2024-04-25T01:41:12.541790+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4098,"log_dups_size":0,"ondisk_log_size":4098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21368,"num_write_kb":269972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"26'5565","reported_seq":5605,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:31.424823+0000","last_change":"2024-04-25T01:41:31.424850+0000","last_active":"2024-04-25T01:41:31.424823+0000","last_peered":"2024-04-25T01:41:31.424823+0000","last_clean":"2024-04-25T01:41:31.424823+0000","last_became_active":"2024-04-25T01:25:53.369350+0000","last_became_peered":"2024-04-25T01:25:53.369350+0000","last_unstale":"2024-04-25T01:41:31.424823+0000","last_undegraded":"2024-04-25T01:41:31.424823+0000","last_fullsized":"2024-04-25T01:41:31.424823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5565","last_scrub_stamp":"2024-04-25T01:41:31.424703+0000","last_deep_scrub":"26'5565","last_deep_scrub_stamp":"2024-04-25T01:41:31.424703+0000","last_clean_scrub_stamp":"2024-04-25T01:41:31.424703+0000","objects_scrubbed":0,"log_size":5565,"log_dups_size":0,"ondisk_log_size":5565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:58.506036+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29144,"num_write_kb":362518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"26'5067","reported_seq":5091,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:15.675258+0000","last_change":"2024-04-25T01:26:02.425528+0000","last_active":"2024-04-25T01:41:15.675258+0000","last_peered":"2024-04-25T01:41:15.675258+0000","last_clean":"2024-04-25T01:41:15.675258+0000","last_became_active":"2024-04-25T01:26:02.425160+0000","last_became_peered":"2024-04-25T01:26:02.425160+0000","last_unstale":"2024-04-25T01:41:15.675258+0000","last_undegraded":"2024-04-25T01:41:15.675258+0000","last_fullsized":"2024-04-25T01:41:15.675258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26737,"num_write_kb":324333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"26'3787","reported_seq":3827,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:27.730829+0000","last_change":"2024-04-25T01:41:27.730858+0000","last_active":"2024-04-25T01:41:27.730829+0000","last_peered":"2024-04-25T01:41:27.730829+0000","last_clean":"2024-04-25T01:41:27.730829+0000","last_became_active":"2024-04-25T01:25:53.356854+0000","last_became_peered":"2024-04-25T01:25:53.356854+0000","last_unstale":"2024-04-25T01:41:27.730829+0000","last_undegraded":"2024-04-25T01:41:27.730829+0000","last_fullsized":"2024-04-25T01:41:27.730829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3787","last_scrub_stamp":"2024-04-25T01:41:27.730723+0000","last_deep_scrub":"26'3787","last_deep_scrub_stamp":"2024-04-25T01:41:27.730723+0000","last_clean_scrub_stamp":"2024-04-25T01:41:27.730723+0000","objects_scrubbed":0,"log_size":3787,"log_dups_size":0,"ondisk_log_size":3787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:36.126786+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19900,"num_write_kb":255902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"26'5481","reported_seq":5513,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:28.933841+0000","last_change":"2024-04-25T01:41:28.933881+0000","last_active":"2024-04-25T01:41:28.933841+0000","last_peered":"2024-04-25T01:41:28.933841+0000","last_clean":"2024-04-25T01:41:28.933841+0000","last_became_active":"2024-04-25T01:26:02.417234+0000","last_became_peered":"2024-04-25T01:26:02.417234+0000","last_unstale":"2024-04-25T01:41:28.933841+0000","last_undegraded":"2024-04-25T01:41:28.933841+0000","last_fullsized":"2024-04-25T01:41:28.933841+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5481","last_scrub_stamp":"2024-04-25T01:41:28.933722+0000","last_deep_scrub":"26'5481","last_deep_scrub_stamp":"2024-04-25T01:41:28.933722+0000","last_clean_scrub_stamp":"2024-04-25T01:41:28.933722+0000","objects_scrubbed":0,"log_size":5481,"log_dups_size":0,"ondisk_log_size":5481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:28.118343+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28430,"num_write_kb":359662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"26'4160","reported_seq":4200,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:31.931946+0000","last_change":"2024-04-25T01:41:31.931973+0000","last_active":"2024-04-25T01:41:31.931946+0000","last_peered":"2024-04-25T01:41:31.931946+0000","last_clean":"2024-04-25T01:41:31.931946+0000","last_became_active":"2024-04-25T01:25:53.364311+0000","last_became_peered":"2024-04-25T01:25:53.364311+0000","last_unstale":"2024-04-25T01:41:31.931946+0000","last_undegraded":"2024-04-25T01:41:31.931946+0000","last_fullsized":"2024-04-25T01:41:31.931946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4160","last_scrub_stamp":"2024-04-25T01:41:31.931832+0000","last_deep_scrub":"26'4160","last_deep_scrub_stamp":"2024-04-25T01:41:31.931832+0000","last_clean_scrub_stamp":"2024-04-25T01:41:31.931832+0000","objects_scrubbed":0,"log_size":4160,"log_dups_size":0,"ondisk_log_size":4160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:22.678450+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21579,"num_write_kb":270844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"26'4125","reported_seq":4149,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.329126+0000","last_change":"2024-04-25T01:26:02.426813+0000","last_active":"2024-04-25T01:41:09.329126+0000","last_peered":"2024-04-25T01:41:09.329126+0000","last_clean":"2024-04-25T01:41:09.329126+0000","last_became_active":"2024-04-25T01:26:02.426552+0000","last_became_peered":"2024-04-25T01:26:02.426552+0000","last_unstale":"2024-04-25T01:41:09.329126+0000","last_undegraded":"2024-04-25T01:41:09.329126+0000","last_fullsized":"2024-04-25T01:41:09.329126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4125,"log_dups_size":0,"ondisk_log_size":4125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21785,"num_write_kb":263458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"26'5101","reported_seq":5141,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:23.621658+0000","last_change":"2024-04-25T01:41:23.621684+0000","last_active":"2024-04-25T01:41:23.621658+0000","last_peered":"2024-04-25T01:41:23.621658+0000","last_clean":"2024-04-25T01:41:23.621658+0000","last_became_active":"2024-04-25T01:25:53.368176+0000","last_became_peered":"2024-04-25T01:25:53.368176+0000","last_unstale":"2024-04-25T01:41:23.621658+0000","last_undegraded":"2024-04-25T01:41:23.621658+0000","last_fullsized":"2024-04-25T01:41:23.621658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5101","last_scrub_stamp":"2024-04-25T01:41:23.621548+0000","last_deep_scrub":"26'5101","last_deep_scrub_stamp":"2024-04-25T01:41:23.621548+0000","last_clean_scrub_stamp":"2024-04-25T01:41:23.621548+0000","objects_scrubbed":0,"log_size":5101,"log_dups_size":0,"ondisk_log_size":5101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:30.364398+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26850,"num_write_kb":324721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"26'6020","reported_seq":6044,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.350810+0000","last_change":"2024-04-25T01:26:02.425185+0000","last_active":"2024-04-25T01:41:09.350810+0000","last_peered":"2024-04-25T01:41:09.350810+0000","last_clean":"2024-04-25T01:41:09.350810+0000","last_became_active":"2024-04-25T01:26:02.424903+0000","last_became_peered":"2024-04-25T01:26:02.424903+0000","last_unstale":"2024-04-25T01:41:09.350810+0000","last_undegraded":"2024-04-25T01:41:09.350810+0000","last_fullsized":"2024-04-25T01:41:09.350810+0000","mapping_epoch":22,"log_start":"26'300","ondisk_log_start":"26'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5720,"log_dups_size":0,"ondisk_log_size":5720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31332,"num_write_kb":379614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"26'4058","reported_seq":4098,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:21.338130+0000","last_change":"2024-04-25T01:41:21.338175+0000","last_active":"2024-04-25T01:41:21.338130+0000","last_peered":"2024-04-25T01:41:21.338130+0000","last_clean":"2024-04-25T01:41:21.338130+0000","last_became_active":"2024-04-25T01:25:53.372001+0000","last_became_peered":"2024-04-25T01:25:53.372001+0000","last_unstale":"2024-04-25T01:41:21.338130+0000","last_undegraded":"2024-04-25T01:41:21.338130+0000","last_fullsized":"2024-04-25T01:41:21.338130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4058","last_scrub_stamp":"2024-04-25T01:41:21.338005+0000","last_deep_scrub":"26'4058","last_deep_scrub_stamp":"2024-04-25T01:41:21.338005+0000","last_clean_scrub_stamp":"2024-04-25T01:41:21.338005+0000","objects_scrubbed":0,"log_size":4058,"log_dups_size":0,"ondisk_log_size":4058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:50.935614+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20502,"num_write_kb":266785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"26'4843","reported_seq":4867,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.316499+0000","last_change":"2024-04-25T01:26:02.420867+0000","last_active":"2024-04-25T01:41:09.316499+0000","last_peered":"2024-04-25T01:41:09.316499+0000","last_clean":"2024-04-25T01:41:09.316499+0000","last_became_active":"2024-04-25T01:26:02.420621+0000","last_became_peered":"2024-04-25T01:26:02.420621+0000","last_unstale":"2024-04-25T01:41:09.316499+0000","last_undegraded":"2024-04-25T01:41:09.316499+0000","last_fullsized":"2024-04-25T01:41:09.316499+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4843,"log_dups_size":0,"ondisk_log_size":4843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24777,"num_write_kb":316390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"26'5268","reported_seq":5308,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:27.919155+0000","last_change":"2024-04-25T01:41:27.919209+0000","last_active":"2024-04-25T01:41:27.919155+0000","last_peered":"2024-04-25T01:41:27.919155+0000","last_clean":"2024-04-25T01:41:27.919155+0000","last_became_active":"2024-04-25T01:25:53.365245+0000","last_became_peered":"2024-04-25T01:25:53.365245+0000","last_unstale":"2024-04-25T01:41:27.919155+0000","last_undegraded":"2024-04-25T01:41:27.919155+0000","last_fullsized":"2024-04-25T01:41:27.919155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5268","last_scrub_stamp":"2024-04-25T01:41:27.919032+0000","last_deep_scrub":"26'5268","last_deep_scrub_stamp":"2024-04-25T01:41:27.919032+0000","last_clean_scrub_stamp":"2024-04-25T01:41:27.919032+0000","objects_scrubbed":0,"log_size":5268,"log_dups_size":0,"ondisk_log_size":5268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:36:37.642225+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26981,"num_write_kb":345791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"26'3558","reported_seq":3581,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.357346+0000","last_change":"2024-04-25T01:26:02.415672+0000","last_active":"2024-04-25T01:41:09.357346+0000","last_peered":"2024-04-25T01:41:09.357346+0000","last_clean":"2024-04-25T01:41:09.357346+0000","last_became_active":"2024-04-25T01:26:02.415389+0000","last_became_peered":"2024-04-25T01:26:02.415389+0000","last_unstale":"2024-04-25T01:41:09.357346+0000","last_undegraded":"2024-04-25T01:41:09.357346+0000","last_fullsized":"2024-04-25T01:41:09.357346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3558,"log_dups_size":0,"ondisk_log_size":3558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18097,"num_write_kb":236200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"26'4040","reported_seq":4080,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:37.342421+0000","last_change":"2024-04-25T01:41:37.342446+0000","last_active":"2024-04-25T01:41:37.342421+0000","last_peered":"2024-04-25T01:41:37.342421+0000","last_clean":"2024-04-25T01:41:37.342421+0000","last_became_active":"2024-04-25T01:25:53.364829+0000","last_became_peered":"2024-04-25T01:25:53.364829+0000","last_unstale":"2024-04-25T01:41:37.342421+0000","last_undegraded":"2024-04-25T01:41:37.342421+0000","last_fullsized":"2024-04-25T01:41:37.342421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4040","last_scrub_stamp":"2024-04-25T01:41:37.342321+0000","last_deep_scrub":"26'4040","last_deep_scrub_stamp":"2024-04-25T01:41:37.342321+0000","last_clean_scrub_stamp":"2024-04-25T01:41:37.342321+0000","objects_scrubbed":0,"log_size":4040,"log_dups_size":0,"ondisk_log_size":4040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:09.885519+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20607,"num_write_kb":262822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"26'5431","reported_seq":5462,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:32.850306+0000","last_change":"2024-04-25T01:41:32.850337+0000","last_active":"2024-04-25T01:41:32.850306+0000","last_peered":"2024-04-25T01:41:32.850306+0000","last_clean":"2024-04-25T01:41:32.850306+0000","last_became_active":"2024-04-25T01:26:02.418036+0000","last_became_peered":"2024-04-25T01:26:02.418036+0000","last_unstale":"2024-04-25T01:41:32.850306+0000","last_undegraded":"2024-04-25T01:41:32.850306+0000","last_fullsized":"2024-04-25T01:41:32.850306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5431","last_scrub_stamp":"2024-04-25T01:41:32.850204+0000","last_deep_scrub":"26'5431","last_deep_scrub_stamp":"2024-04-25T01:41:32.850204+0000","last_clean_scrub_stamp":"2024-04-25T01:41:32.850204+0000","objects_scrubbed":0,"log_size":5431,"log_dups_size":0,"ondisk_log_size":5431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:59.931538+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28059,"num_write_kb":358266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"26'4709","reported_seq":4749,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:37.962371+0000","last_change":"2024-04-25T01:41:37.962371+0000","last_active":"2024-04-25T01:41:37.962371+0000","last_peered":"2024-04-25T01:41:37.962371+0000","last_clean":"2024-04-25T01:41:37.962371+0000","last_became_active":"2024-04-25T01:25:53.355417+0000","last_became_peered":"2024-04-25T01:25:53.355417+0000","last_unstale":"2024-04-25T01:41:37.962371+0000","last_undegraded":"2024-04-25T01:41:37.962371+0000","last_fullsized":"2024-04-25T01:41:37.962371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4709","last_scrub_stamp":"2024-04-25T01:41:37.962211+0000","last_deep_scrub":"26'4709","last_deep_scrub_stamp":"2024-04-25T01:41:37.962211+0000","last_clean_scrub_stamp":"2024-04-25T01:41:37.962211+0000","objects_scrubbed":0,"log_size":4709,"log_dups_size":0,"ondisk_log_size":4709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:11.856762+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24394,"num_write_kb":298399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"26'4741","reported_seq":4772,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:30.129808+0000","last_change":"2024-04-25T01:41:30.129837+0000","last_active":"2024-04-25T01:41:30.129808+0000","last_peered":"2024-04-25T01:41:30.129808+0000","last_clean":"2024-04-25T01:41:30.129808+0000","last_became_active":"2024-04-25T01:26:02.414735+0000","last_became_peered":"2024-04-25T01:26:02.414735+0000","last_unstale":"2024-04-25T01:41:30.129808+0000","last_undegraded":"2024-04-25T01:41:30.129808+0000","last_fullsized":"2024-04-25T01:41:30.129808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4741","last_scrub_stamp":"2024-04-25T01:41:30.129681+0000","last_deep_scrub":"26'4741","last_deep_scrub_stamp":"2024-04-25T01:41:30.129681+0000","last_clean_scrub_stamp":"2024-04-25T01:41:30.129681+0000","objects_scrubbed":0,"log_size":4741,"log_dups_size":0,"ondisk_log_size":4741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:08.104652+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24824,"num_write_kb":300285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"26'5111","reported_seq":5150,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:25.401668+0000","last_change":"2024-04-25T01:41:25.401696+0000","last_active":"2024-04-25T01:41:25.401668+0000","last_peered":"2024-04-25T01:41:25.401668+0000","last_clean":"2024-04-25T01:41:25.401668+0000","last_became_active":"2024-04-25T01:25:53.370164+0000","last_became_peered":"2024-04-25T01:25:53.370164+0000","last_unstale":"2024-04-25T01:41:25.401668+0000","last_undegraded":"2024-04-25T01:41:25.401668+0000","last_fullsized":"2024-04-25T01:41:25.401668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5111","last_scrub_stamp":"2024-04-25T01:41:25.401560+0000","last_deep_scrub":"26'5111","last_deep_scrub_stamp":"2024-04-25T01:41:25.401560+0000","last_clean_scrub_stamp":"2024-04-25T01:41:25.401560+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:11.935349+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26542,"num_write_kb":327744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"26'4367","reported_seq":4390,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.312526+0000","last_change":"2024-04-25T01:26:02.418272+0000","last_active":"2024-04-25T01:41:09.312526+0000","last_peered":"2024-04-25T01:41:09.312526+0000","last_clean":"2024-04-25T01:41:09.312526+0000","last_became_active":"2024-04-25T01:26:02.417977+0000","last_became_peered":"2024-04-25T01:26:02.417977+0000","last_unstale":"2024-04-25T01:41:09.312526+0000","last_undegraded":"2024-04-25T01:41:09.312526+0000","last_fullsized":"2024-04-25T01:41:09.312526+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4367,"log_dups_size":0,"ondisk_log_size":4367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22024,"num_write_kb":293008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"26'5285","reported_seq":5324,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:19.722242+0000","last_change":"2024-04-25T01:41:19.722283+0000","last_active":"2024-04-25T01:41:19.722242+0000","last_peered":"2024-04-25T01:41:19.722242+0000","last_clean":"2024-04-25T01:41:19.722242+0000","last_became_active":"2024-04-25T01:25:53.376104+0000","last_became_peered":"2024-04-25T01:25:53.376104+0000","last_unstale":"2024-04-25T01:41:19.722242+0000","last_undegraded":"2024-04-25T01:41:19.722242+0000","last_fullsized":"2024-04-25T01:41:19.722242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5285","last_scrub_stamp":"2024-04-25T01:41:19.722126+0000","last_deep_scrub":"26'5285","last_deep_scrub_stamp":"2024-04-25T01:41:19.722126+0000","last_clean_scrub_stamp":"2024-04-25T01:41:19.722126+0000","objects_scrubbed":0,"log_size":5285,"log_dups_size":0,"ondisk_log_size":5285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:49.319736+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27331,"num_write_kb":318531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"26'5020","reported_seq":5043,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.344619+0000","last_change":"2024-04-25T01:26:02.426333+0000","last_active":"2024-04-25T01:41:09.344619+0000","last_peered":"2024-04-25T01:41:09.344619+0000","last_clean":"2024-04-25T01:41:09.344619+0000","last_became_active":"2024-04-25T01:26:02.426069+0000","last_became_peered":"2024-04-25T01:26:02.426069+0000","last_unstale":"2024-04-25T01:41:09.344619+0000","last_undegraded":"2024-04-25T01:41:09.344619+0000","last_fullsized":"2024-04-25T01:41:09.344619+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5020,"log_dups_size":0,"ondisk_log_size":5020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26121,"num_write_kb":325976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"26'4910","reported_seq":4949,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:32.299707+0000","last_change":"2024-04-25T01:41:32.299733+0000","last_active":"2024-04-25T01:41:32.299707+0000","last_peered":"2024-04-25T01:41:32.299707+0000","last_clean":"2024-04-25T01:41:32.299707+0000","last_became_active":"2024-04-25T01:25:53.370049+0000","last_became_peered":"2024-04-25T01:25:53.370049+0000","last_unstale":"2024-04-25T01:41:32.299707+0000","last_undegraded":"2024-04-25T01:41:32.299707+0000","last_fullsized":"2024-04-25T01:41:32.299707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4910","last_scrub_stamp":"2024-04-25T01:41:32.299587+0000","last_deep_scrub":"26'4910","last_deep_scrub_stamp":"2024-04-25T01:41:32.299587+0000","last_clean_scrub_stamp":"2024-04-25T01:41:32.299587+0000","objects_scrubbed":0,"log_size":4910,"log_dups_size":0,"ondisk_log_size":4910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:06.590550+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25833,"num_write_kb":304449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"26'4775","reported_seq":4806,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:37.094501+0000","last_change":"2024-04-25T01:41:37.094529+0000","last_active":"2024-04-25T01:41:37.094501+0000","last_peered":"2024-04-25T01:41:37.094501+0000","last_clean":"2024-04-25T01:41:37.094501+0000","last_became_active":"2024-04-25T01:26:02.415173+0000","last_became_peered":"2024-04-25T01:26:02.415173+0000","last_unstale":"2024-04-25T01:41:37.094501+0000","last_undegraded":"2024-04-25T01:41:37.094501+0000","last_fullsized":"2024-04-25T01:41:37.094501+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4775","last_scrub_stamp":"2024-04-25T01:41:37.094377+0000","last_deep_scrub":"26'4775","last_deep_scrub_stamp":"2024-04-25T01:41:37.094377+0000","last_clean_scrub_stamp":"2024-04-25T01:41:37.094377+0000","objects_scrubbed":0,"log_size":4775,"log_dups_size":0,"ondisk_log_size":4775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:34.147067+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24075,"num_write_kb":305295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"26'4891","reported_seq":4930,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:23.972785+0000","last_change":"2024-04-25T01:41:23.972816+0000","last_active":"2024-04-25T01:41:23.972785+0000","last_peered":"2024-04-25T01:41:23.972785+0000","last_clean":"2024-04-25T01:41:23.972785+0000","last_became_active":"2024-04-25T01:25:53.368499+0000","last_became_peered":"2024-04-25T01:25:53.368499+0000","last_unstale":"2024-04-25T01:41:23.972785+0000","last_undegraded":"2024-04-25T01:41:23.972785+0000","last_fullsized":"2024-04-25T01:41:23.972785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4891","last_scrub_stamp":"2024-04-25T01:41:23.972677+0000","last_deep_scrub":"26'4891","last_deep_scrub_stamp":"2024-04-25T01:41:23.972677+0000","last_clean_scrub_stamp":"2024-04-25T01:41:23.972677+0000","objects_scrubbed":0,"log_size":4891,"log_dups_size":0,"ondisk_log_size":4891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:34.356335+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24867,"num_write_kb":304528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"26'4199","reported_seq":4223,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:17.542486+0000","last_change":"2024-04-25T01:26:02.421975+0000","last_active":"2024-04-25T01:41:17.542486+0000","last_peered":"2024-04-25T01:41:17.542486+0000","last_clean":"2024-04-25T01:41:17.542486+0000","last_became_active":"2024-04-25T01:26:02.421698+0000","last_became_peered":"2024-04-25T01:26:02.421698+0000","last_unstale":"2024-04-25T01:41:17.542486+0000","last_undegraded":"2024-04-25T01:41:17.542486+0000","last_fullsized":"2024-04-25T01:41:17.542486+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4199,"log_dups_size":0,"ondisk_log_size":4199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21738,"num_write_kb":267270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"26'3844","reported_seq":3883,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:26.095640+0000","last_change":"2024-04-25T01:41:26.095685+0000","last_active":"2024-04-25T01:41:26.095640+0000","last_peered":"2024-04-25T01:41:26.095640+0000","last_clean":"2024-04-25T01:41:26.095640+0000","last_became_active":"2024-04-25T01:25:53.369733+0000","last_became_peered":"2024-04-25T01:25:53.369733+0000","last_unstale":"2024-04-25T01:41:26.095640+0000","last_undegraded":"2024-04-25T01:41:26.095640+0000","last_fullsized":"2024-04-25T01:41:26.095640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3844","last_scrub_stamp":"2024-04-25T01:41:26.095528+0000","last_deep_scrub":"26'3844","last_deep_scrub_stamp":"2024-04-25T01:41:26.095528+0000","last_clean_scrub_stamp":"2024-04-25T01:41:26.095528+0000","objects_scrubbed":0,"log_size":3844,"log_dups_size":0,"ondisk_log_size":3844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:32.635635+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19367,"num_write_kb":241344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"26'4316","reported_seq":4339,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.253310+0000","last_change":"2024-04-25T01:26:02.418114+0000","last_active":"2024-04-25T01:41:09.253310+0000","last_peered":"2024-04-25T01:41:09.253310+0000","last_clean":"2024-04-25T01:41:09.253310+0000","last_became_active":"2024-04-25T01:26:02.417810+0000","last_became_peered":"2024-04-25T01:26:02.417810+0000","last_unstale":"2024-04-25T01:41:09.253310+0000","last_undegraded":"2024-04-25T01:41:09.253310+0000","last_fullsized":"2024-04-25T01:41:09.253310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4316,"log_dups_size":0,"ondisk_log_size":4316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22517,"num_write_kb":290939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"26'5814","reported_seq":5839,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:35.060358+0000","last_change":"2024-04-25T01:41:35.060383+0000","last_active":"2024-04-25T01:41:35.060358+0000","last_peered":"2024-04-25T01:41:35.060358+0000","last_clean":"2024-04-25T01:41:35.060358+0000","last_became_active":"2024-04-25T01:26:52.120588+0000","last_became_peered":"2024-04-25T01:26:52.120588+0000","last_unstale":"2024-04-25T01:41:35.060358+0000","last_undegraded":"2024-04-25T01:41:35.060358+0000","last_fullsized":"2024-04-25T01:41:35.060358+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5814","last_scrub_stamp":"2024-04-25T01:41:35.060229+0000","last_deep_scrub":"26'5814","last_deep_scrub_stamp":"2024-04-25T01:41:35.060229+0000","last_clean_scrub_stamp":"2024-04-25T01:41:35.060229+0000","objects_scrubbed":0,"log_size":5814,"log_dups_size":0,"ondisk_log_size":5814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:21.594018+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29929,"num_write_kb":365779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.36","version":"26'4743","reported_seq":4766,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.336178+0000","last_change":"2024-04-25T01:26:02.426289+0000","last_active":"2024-04-25T01:41:09.336178+0000","last_peered":"2024-04-25T01:41:09.336178+0000","last_clean":"2024-04-25T01:41:09.336178+0000","last_became_active":"2024-04-25T01:26:02.426015+0000","last_became_peered":"2024-04-25T01:26:02.426015+0000","last_unstale":"2024-04-25T01:41:09.336178+0000","last_undegraded":"2024-04-25T01:41:09.336178+0000","last_fullsized":"2024-04-25T01:41:09.336178+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4743,"log_dups_size":0,"ondisk_log_size":4743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25470,"num_write_kb":331505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"26'4366","reported_seq":4405,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:22.334603+0000","last_change":"2024-04-25T01:41:22.334638+0000","last_active":"2024-04-25T01:41:22.334603+0000","last_peered":"2024-04-25T01:41:22.334603+0000","last_clean":"2024-04-25T01:41:22.334603+0000","last_became_active":"2024-04-25T01:25:53.373766+0000","last_became_peered":"2024-04-25T01:25:53.373766+0000","last_unstale":"2024-04-25T01:41:22.334603+0000","last_undegraded":"2024-04-25T01:41:22.334603+0000","last_fullsized":"2024-04-25T01:41:22.334603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4366","last_scrub_stamp":"2024-04-25T01:41:22.334478+0000","last_deep_scrub":"26'4366","last_deep_scrub_stamp":"2024-04-25T01:41:22.334478+0000","last_clean_scrub_stamp":"2024-04-25T01:41:22.334478+0000","objects_scrubbed":0,"log_size":4366,"log_dups_size":0,"ondisk_log_size":4366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:29.077328+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23328,"num_write_kb":273725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"26'5875","reported_seq":5898,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.357031+0000","last_change":"2024-04-25T01:26:02.416345+0000","last_active":"2024-04-25T01:41:09.357031+0000","last_peered":"2024-04-25T01:41:09.357031+0000","last_clean":"2024-04-25T01:41:09.357031+0000","last_became_active":"2024-04-25T01:26:02.416082+0000","last_became_peered":"2024-04-25T01:26:02.416082+0000","last_unstale":"2024-04-25T01:41:09.357031+0000","last_undegraded":"2024-04-25T01:41:09.357031+0000","last_fullsized":"2024-04-25T01:41:09.357031+0000","mapping_epoch":22,"log_start":"26'400","ondisk_log_start":"26'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5475,"log_dups_size":0,"ondisk_log_size":5475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31009,"num_write_kb":370059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"26'4898","reported_seq":4937,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:21.678232+0000","last_change":"2024-04-25T01:41:21.678273+0000","last_active":"2024-04-25T01:41:21.678232+0000","last_peered":"2024-04-25T01:41:21.678232+0000","last_clean":"2024-04-25T01:41:21.678232+0000","last_became_active":"2024-04-25T01:25:53.371362+0000","last_became_peered":"2024-04-25T01:25:53.371362+0000","last_unstale":"2024-04-25T01:41:21.678232+0000","last_undegraded":"2024-04-25T01:41:21.678232+0000","last_fullsized":"2024-04-25T01:41:21.678232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4898","last_scrub_stamp":"2024-04-25T01:41:21.678132+0000","last_deep_scrub":"26'4898","last_deep_scrub_stamp":"2024-04-25T01:41:21.678132+0000","last_clean_scrub_stamp":"2024-04-25T01:41:21.678132+0000","objects_scrubbed":0,"log_size":4898,"log_dups_size":0,"ondisk_log_size":4898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:02.628027+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24919,"num_write_kb":308746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"26'3961","reported_seq":3984,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.318869+0000","last_change":"2024-04-25T01:26:02.425888+0000","last_active":"2024-04-25T01:41:09.318869+0000","last_peered":"2024-04-25T01:41:09.318869+0000","last_clean":"2024-04-25T01:41:09.318869+0000","last_became_active":"2024-04-25T01:26:02.425577+0000","last_became_peered":"2024-04-25T01:26:02.425577+0000","last_unstale":"2024-04-25T01:41:09.318869+0000","last_undegraded":"2024-04-25T01:41:09.318869+0000","last_fullsized":"2024-04-25T01:41:09.318869+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":3961,"log_dups_size":0,"ondisk_log_size":3961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20296,"num_write_kb":253395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"26'4670","reported_seq":4709,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:22.696731+0000","last_change":"2024-04-25T01:41:22.696754+0000","last_active":"2024-04-25T01:41:22.696731+0000","last_peered":"2024-04-25T01:41:22.696731+0000","last_clean":"2024-04-25T01:41:22.696731+0000","last_became_active":"2024-04-25T01:25:53.370479+0000","last_became_peered":"2024-04-25T01:25:53.370479+0000","last_unstale":"2024-04-25T01:41:22.696731+0000","last_undegraded":"2024-04-25T01:41:22.696731+0000","last_fullsized":"2024-04-25T01:41:22.696731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4670","last_scrub_stamp":"2024-04-25T01:41:22.696638+0000","last_deep_scrub":"26'4670","last_deep_scrub_stamp":"2024-04-25T01:41:22.696638+0000","last_clean_scrub_stamp":"2024-04-25T01:41:22.696638+0000","objects_scrubbed":0,"log_size":4670,"log_dups_size":0,"ondisk_log_size":4670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:25.738779+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24421,"num_write_kb":306681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"26'4733","reported_seq":4756,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.314882+0000","last_change":"2024-04-25T01:26:02.423420+0000","last_active":"2024-04-25T01:41:09.314882+0000","last_peered":"2024-04-25T01:41:09.314882+0000","last_clean":"2024-04-25T01:41:09.314882+0000","last_became_active":"2024-04-25T01:26:02.423146+0000","last_became_peered":"2024-04-25T01:26:02.423146+0000","last_unstale":"2024-04-25T01:41:09.314882+0000","last_undegraded":"2024-04-25T01:41:09.314882+0000","last_fullsized":"2024-04-25T01:41:09.314882+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4733,"log_dups_size":0,"ondisk_log_size":4733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23928,"num_write_kb":292515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"26'4513","reported_seq":4544,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.807827+0000","last_change":"2024-04-25T01:25:53.363512+0000","last_active":"2024-04-25T01:40:58.807827+0000","last_peered":"2024-04-25T01:40:58.807827+0000","last_clean":"2024-04-25T01:40:58.807827+0000","last_became_active":"2024-04-25T01:25:53.363246+0000","last_became_peered":"2024-04-25T01:25:53.363246+0000","last_unstale":"2024-04-25T01:40:58.807827+0000","last_undegraded":"2024-04-25T01:40:58.807827+0000","last_fullsized":"2024-04-25T01:40:58.807827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4513,"log_dups_size":0,"ondisk_log_size":4513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23705,"num_write_kb":279339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"26'4587","reported_seq":4618,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:30.948056+0000","last_change":"2024-04-25T01:41:30.948083+0000","last_active":"2024-04-25T01:41:30.948056+0000","last_peered":"2024-04-25T01:41:30.948056+0000","last_clean":"2024-04-25T01:41:30.948056+0000","last_became_active":"2024-04-25T01:26:02.413036+0000","last_became_peered":"2024-04-25T01:26:02.413036+0000","last_unstale":"2024-04-25T01:41:30.948056+0000","last_undegraded":"2024-04-25T01:41:30.948056+0000","last_fullsized":"2024-04-25T01:41:30.948056+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4587","last_scrub_stamp":"2024-04-25T01:41:30.947949+0000","last_deep_scrub":"26'4587","last_deep_scrub_stamp":"2024-04-25T01:41:30.947949+0000","last_clean_scrub_stamp":"2024-04-25T01:41:30.947949+0000","objects_scrubbed":0,"log_size":4587,"log_dups_size":0,"ondisk_log_size":4587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:23:08.496641+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24042,"num_write_kb":305325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"26'3998","reported_seq":4037,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:31.189011+0000","last_change":"2024-04-25T01:41:31.189037+0000","last_active":"2024-04-25T01:41:31.189011+0000","last_peered":"2024-04-25T01:41:31.189011+0000","last_clean":"2024-04-25T01:41:31.189011+0000","last_became_active":"2024-04-25T01:25:53.364103+0000","last_became_peered":"2024-04-25T01:25:53.364103+0000","last_unstale":"2024-04-25T01:41:31.189011+0000","last_undegraded":"2024-04-25T01:41:31.189011+0000","last_fullsized":"2024-04-25T01:41:31.189011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3998","last_scrub_stamp":"2024-04-25T01:41:31.188903+0000","last_deep_scrub":"26'3998","last_deep_scrub_stamp":"2024-04-25T01:41:31.188903+0000","last_clean_scrub_stamp":"2024-04-25T01:41:31.188903+0000","objects_scrubbed":0,"log_size":3998,"log_dups_size":0,"ondisk_log_size":3998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:30.373524+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20343,"num_write_kb":253659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"26'4194","reported_seq":4225,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:33.143247+0000","last_change":"2024-04-25T01:41:33.143274+0000","last_active":"2024-04-25T01:41:33.143247+0000","last_peered":"2024-04-25T01:41:33.143247+0000","last_clean":"2024-04-25T01:41:33.143247+0000","last_became_active":"2024-04-25T01:26:02.416942+0000","last_became_peered":"2024-04-25T01:26:02.416942+0000","last_unstale":"2024-04-25T01:41:33.143247+0000","last_undegraded":"2024-04-25T01:41:33.143247+0000","last_fullsized":"2024-04-25T01:41:33.143247+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4194","last_scrub_stamp":"2024-04-25T01:41:33.143135+0000","last_deep_scrub":"26'4194","last_deep_scrub_stamp":"2024-04-25T01:41:33.143135+0000","last_clean_scrub_stamp":"2024-04-25T01:41:33.143135+0000","objects_scrubbed":0,"log_size":4194,"log_dups_size":0,"ondisk_log_size":4194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.460297+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21868,"num_write_kb":272015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"26'4800","reported_seq":4839,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:24.333874+0000","last_change":"2024-04-25T01:41:24.333902+0000","last_active":"2024-04-25T01:41:24.333874+0000","last_peered":"2024-04-25T01:41:24.333874+0000","last_clean":"2024-04-25T01:41:24.333874+0000","last_became_active":"2024-04-25T01:25:53.371353+0000","last_became_peered":"2024-04-25T01:25:53.371353+0000","last_unstale":"2024-04-25T01:41:24.333874+0000","last_undegraded":"2024-04-25T01:41:24.333874+0000","last_fullsized":"2024-04-25T01:41:24.333874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4800","last_scrub_stamp":"2024-04-25T01:41:24.333758+0000","last_deep_scrub":"26'4800","last_deep_scrub_stamp":"2024-04-25T01:41:24.333758+0000","last_clean_scrub_stamp":"2024-04-25T01:41:24.333758+0000","objects_scrubbed":0,"log_size":4800,"log_dups_size":0,"ondisk_log_size":4800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:30.873865+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24494,"num_write_kb":294847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"26'4139","reported_seq":4162,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.353148+0000","last_change":"2024-04-25T01:26:02.426335+0000","last_active":"2024-04-25T01:41:09.353148+0000","last_peered":"2024-04-25T01:41:09.353148+0000","last_clean":"2024-04-25T01:41:09.353148+0000","last_became_active":"2024-04-25T01:26:02.426024+0000","last_became_peered":"2024-04-25T01:26:02.426024+0000","last_unstale":"2024-04-25T01:41:09.353148+0000","last_undegraded":"2024-04-25T01:41:09.353148+0000","last_fullsized":"2024-04-25T01:41:09.353148+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4139,"log_dups_size":0,"ondisk_log_size":4139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21818,"num_write_kb":267665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"26'5798","reported_seq":5837,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:23.367199+0000","last_change":"2024-04-25T01:41:23.367224+0000","last_active":"2024-04-25T01:41:23.367199+0000","last_peered":"2024-04-25T01:41:23.367199+0000","last_clean":"2024-04-25T01:41:23.367199+0000","last_became_active":"2024-04-25T01:25:53.372580+0000","last_became_peered":"2024-04-25T01:25:53.372580+0000","last_unstale":"2024-04-25T01:41:23.367199+0000","last_undegraded":"2024-04-25T01:41:23.367199+0000","last_fullsized":"2024-04-25T01:41:23.367199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5798","last_scrub_stamp":"2024-04-25T01:41:23.367082+0000","last_deep_scrub":"26'5798","last_deep_scrub_stamp":"2024-04-25T01:41:23.367082+0000","last_clean_scrub_stamp":"2024-04-25T01:41:23.367082+0000","objects_scrubbed":0,"log_size":5798,"log_dups_size":0,"ondisk_log_size":5798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:33.352473+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30839,"num_write_kb":352989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"26'4687","reported_seq":4718,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:30.381675+0000","last_change":"2024-04-25T01:41:30.381700+0000","last_active":"2024-04-25T01:41:30.381675+0000","last_peered":"2024-04-25T01:41:30.381675+0000","last_clean":"2024-04-25T01:41:30.381675+0000","last_became_active":"2024-04-25T01:26:02.414388+0000","last_became_peered":"2024-04-25T01:26:02.414388+0000","last_unstale":"2024-04-25T01:41:30.381675+0000","last_undegraded":"2024-04-25T01:41:30.381675+0000","last_fullsized":"2024-04-25T01:41:30.381675+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4687","last_scrub_stamp":"2024-04-25T01:41:30.381573+0000","last_deep_scrub":"26'4687","last_deep_scrub_stamp":"2024-04-25T01:41:30.381573+0000","last_clean_scrub_stamp":"2024-04-25T01:41:30.381573+0000","objects_scrubbed":0,"log_size":4687,"log_dups_size":0,"ondisk_log_size":4687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:16.535112+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23866,"num_write_kb":304526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"26'5655","reported_seq":5691,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.889091+0000","last_change":"2024-04-25T01:26:52.515807+0000","last_active":"2024-04-25T01:40:58.889091+0000","last_peered":"2024-04-25T01:40:58.889091+0000","last_clean":"2024-04-25T01:40:58.889091+0000","last_became_active":"2024-04-25T01:26:52.515504+0000","last_became_peered":"2024-04-25T01:26:52.515504+0000","last_unstale":"2024-04-25T01:40:58.889091+0000","last_undegraded":"2024-04-25T01:40:58.889091+0000","last_fullsized":"2024-04-25T01:40:58.889091+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":5655,"log_dups_size":0,"ondisk_log_size":5655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29809,"num_write_kb":365295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"26'5769","reported_seq":5801,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:32.921279+0000","last_change":"2024-04-25T01:41:32.921307+0000","last_active":"2024-04-25T01:41:32.921279+0000","last_peered":"2024-04-25T01:41:32.921279+0000","last_clean":"2024-04-25T01:41:32.921279+0000","last_became_active":"2024-04-25T01:26:02.414061+0000","last_became_peered":"2024-04-25T01:26:02.414061+0000","last_unstale":"2024-04-25T01:41:32.921279+0000","last_undegraded":"2024-04-25T01:41:32.921279+0000","last_fullsized":"2024-04-25T01:41:32.921279+0000","mapping_epoch":22,"log_start":"26'500","ondisk_log_start":"26'500","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5769","last_scrub_stamp":"2024-04-25T01:41:32.921163+0000","last_deep_scrub":"26'5769","last_deep_scrub_stamp":"2024-04-25T01:41:32.921163+0000","last_clean_scrub_stamp":"2024-04-25T01:41:32.921163+0000","objects_scrubbed":0,"log_size":5269,"log_dups_size":0,"ondisk_log_size":5269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:09.345791+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30404,"num_write_kb":379952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"26'5055","reported_seq":5094,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:25.892289+0000","last_change":"2024-04-25T01:41:25.892319+0000","last_active":"2024-04-25T01:41:25.892289+0000","last_peered":"2024-04-25T01:41:25.892289+0000","last_clean":"2024-04-25T01:41:25.892289+0000","last_became_active":"2024-04-25T01:25:53.373848+0000","last_became_peered":"2024-04-25T01:25:53.373848+0000","last_unstale":"2024-04-25T01:41:25.892289+0000","last_undegraded":"2024-04-25T01:41:25.892289+0000","last_fullsized":"2024-04-25T01:41:25.892289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5055","last_scrub_stamp":"2024-04-25T01:41:25.892145+0000","last_deep_scrub":"26'5055","last_deep_scrub_stamp":"2024-04-25T01:41:25.892145+0000","last_clean_scrub_stamp":"2024-04-25T01:41:25.892145+0000","objects_scrubbed":0,"log_size":5055,"log_dups_size":0,"ondisk_log_size":5055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:25.076766+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25773,"num_write_kb":324544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"26'4195","reported_seq":4219,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.328149+0000","last_change":"2024-04-25T01:26:02.414620+0000","last_active":"2024-04-25T01:41:09.328149+0000","last_peered":"2024-04-25T01:41:09.328149+0000","last_clean":"2024-04-25T01:41:09.328149+0000","last_became_active":"2024-04-25T01:26:02.414326+0000","last_became_peered":"2024-04-25T01:26:02.414326+0000","last_unstale":"2024-04-25T01:41:09.328149+0000","last_undegraded":"2024-04-25T01:41:09.328149+0000","last_fullsized":"2024-04-25T01:41:09.328149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4195,"log_dups_size":0,"ondisk_log_size":4195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21944,"num_write_kb":264053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"26'4639","reported_seq":4670,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:40:58.760854+0000","last_change":"2024-04-25T01:25:53.364698+0000","last_active":"2024-04-25T01:40:58.760854+0000","last_peered":"2024-04-25T01:40:58.760854+0000","last_clean":"2024-04-25T01:40:58.760854+0000","last_became_active":"2024-04-25T01:25:53.364435+0000","last_became_peered":"2024-04-25T01:25:53.364435+0000","last_unstale":"2024-04-25T01:40:58.760854+0000","last_undegraded":"2024-04-25T01:40:58.760854+0000","last_fullsized":"2024-04-25T01:40:58.760854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:25:52.333844+0000","last_clean_scrub_stamp":"2024-04-25T01:25:52.333844+0000","objects_scrubbed":0,"log_size":4639,"log_dups_size":0,"ondisk_log_size":4639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24873,"num_write_kb":296467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"26'4393","reported_seq":4425,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:38.575045+0000","last_change":"2024-04-25T01:41:38.575073+0000","last_active":"2024-04-25T01:41:38.575045+0000","last_peered":"2024-04-25T01:41:38.575045+0000","last_clean":"2024-04-25T01:41:38.575045+0000","last_became_active":"2024-04-25T01:26:02.418057+0000","last_became_peered":"2024-04-25T01:26:02.418057+0000","last_unstale":"2024-04-25T01:41:38.575045+0000","last_undegraded":"2024-04-25T01:41:38.575045+0000","last_fullsized":"2024-04-25T01:41:38.575045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4393","last_scrub_stamp":"2024-04-25T01:41:38.574926+0000","last_deep_scrub":"26'4393","last_deep_scrub_stamp":"2024-04-25T01:41:38.574926+0000","last_clean_scrub_stamp":"2024-04-25T01:41:38.574926+0000","objects_scrubbed":0,"log_size":4393,"log_dups_size":0,"ondisk_log_size":4393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:35.474461+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22950,"num_write_kb":276425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"26'3744","reported_seq":3768,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:36.053700+0000","last_change":"2024-04-25T01:41:36.053728+0000","last_active":"2024-04-25T01:41:36.053700+0000","last_peered":"2024-04-25T01:41:36.053700+0000","last_clean":"2024-04-25T01:41:36.053700+0000","last_became_active":"2024-04-25T01:26:52.514260+0000","last_became_peered":"2024-04-25T01:26:52.514260+0000","last_unstale":"2024-04-25T01:41:36.053700+0000","last_undegraded":"2024-04-25T01:41:36.053700+0000","last_fullsized":"2024-04-25T01:41:36.053700+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3744","last_scrub_stamp":"2024-04-25T01:41:36.053594+0000","last_deep_scrub":"26'3744","last_deep_scrub_stamp":"2024-04-25T01:41:36.053594+0000","last_clean_scrub_stamp":"2024-04-25T01:41:36.053594+0000","objects_scrubbed":0,"log_size":3744,"log_dups_size":0,"ondisk_log_size":3744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:49.732601+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19457,"num_write_kb":254077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3d","version":"26'4696","reported_seq":4720,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.345431+0000","last_change":"2024-04-25T01:26:02.422720+0000","last_active":"2024-04-25T01:41:09.345431+0000","last_peered":"2024-04-25T01:41:09.345431+0000","last_clean":"2024-04-25T01:41:09.345431+0000","last_became_active":"2024-04-25T01:26:02.421848+0000","last_became_peered":"2024-04-25T01:26:02.421848+0000","last_unstale":"2024-04-25T01:41:09.345431+0000","last_undegraded":"2024-04-25T01:41:09.345431+0000","last_fullsized":"2024-04-25T01:41:09.345431+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4696,"log_dups_size":0,"ondisk_log_size":4696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24119,"num_write_kb":305642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"26'5605","reported_seq":5644,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:27.893977+0000","last_change":"2024-04-25T01:41:27.894010+0000","last_active":"2024-04-25T01:41:27.893977+0000","last_peered":"2024-04-25T01:41:27.893977+0000","last_clean":"2024-04-25T01:41:27.893977+0000","last_became_active":"2024-04-25T01:25:53.367358+0000","last_became_peered":"2024-04-25T01:25:53.367358+0000","last_unstale":"2024-04-25T01:41:27.893977+0000","last_undegraded":"2024-04-25T01:41:27.893977+0000","last_fullsized":"2024-04-25T01:41:27.893977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5605","last_scrub_stamp":"2024-04-25T01:41:27.893870+0000","last_deep_scrub":"26'5605","last_deep_scrub_stamp":"2024-04-25T01:41:27.893870+0000","last_clean_scrub_stamp":"2024-04-25T01:41:27.893870+0000","objects_scrubbed":0,"log_size":5605,"log_dups_size":0,"ondisk_log_size":5605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:57.480861+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27762,"num_write_kb":352787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"26'4305","reported_seq":4328,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.310446+0000","last_change":"2024-04-25T01:26:02.422300+0000","last_active":"2024-04-25T01:41:09.310446+0000","last_peered":"2024-04-25T01:41:09.310446+0000","last_clean":"2024-04-25T01:41:09.310446+0000","last_became_active":"2024-04-25T01:26:02.421702+0000","last_became_peered":"2024-04-25T01:26:02.421702+0000","last_unstale":"2024-04-25T01:41:09.310446+0000","last_undegraded":"2024-04-25T01:41:09.310446+0000","last_fullsized":"2024-04-25T01:41:09.310446+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":4305,"log_dups_size":0,"ondisk_log_size":4305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21701,"num_write_kb":263037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"26'4577","reported_seq":4617,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:21.383755+0000","last_change":"2024-04-25T01:41:21.383781+0000","last_active":"2024-04-25T01:41:21.383755+0000","last_peered":"2024-04-25T01:41:21.383755+0000","last_clean":"2024-04-25T01:41:21.383755+0000","last_became_active":"2024-04-25T01:25:53.370421+0000","last_became_peered":"2024-04-25T01:25:53.370421+0000","last_unstale":"2024-04-25T01:41:21.383755+0000","last_undegraded":"2024-04-25T01:41:21.383755+0000","last_fullsized":"2024-04-25T01:41:21.383755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4577","last_scrub_stamp":"2024-04-25T01:41:21.383639+0000","last_deep_scrub":"26'4577","last_deep_scrub_stamp":"2024-04-25T01:41:21.383639+0000","last_clean_scrub_stamp":"2024-04-25T01:41:21.383639+0000","objects_scrubbed":0,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:29.779703+0000","scrub_duration":42,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24195,"num_write_kb":281461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"26'5562","reported_seq":5593,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:20.984819+0000","last_change":"2024-04-25T01:41:20.984842+0000","last_active":"2024-04-25T01:41:20.984819+0000","last_peered":"2024-04-25T01:41:20.984819+0000","last_clean":"2024-04-25T01:41:20.984819+0000","last_became_active":"2024-04-25T01:26:02.411478+0000","last_became_peered":"2024-04-25T01:26:02.411478+0000","last_unstale":"2024-04-25T01:41:20.984819+0000","last_undegraded":"2024-04-25T01:41:20.984819+0000","last_fullsized":"2024-04-25T01:41:20.984819+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5562","last_scrub_stamp":"2024-04-25T01:41:20.984732+0000","last_deep_scrub":"26'5562","last_deep_scrub_stamp":"2024-04-25T01:41:20.984732+0000","last_clean_scrub_stamp":"2024-04-25T01:41:20.984732+0000","objects_scrubbed":0,"log_size":5562,"log_dups_size":0,"ondisk_log_size":5562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:39.437343+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27685,"num_write_kb":352571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"26'5001","reported_seq":5041,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:28.687834+0000","last_change":"2024-04-25T01:41:28.687859+0000","last_active":"2024-04-25T01:41:28.687834+0000","last_peered":"2024-04-25T01:41:28.687834+0000","last_clean":"2024-04-25T01:41:28.687834+0000","last_became_active":"2024-04-25T01:25:53.363525+0000","last_became_peered":"2024-04-25T01:25:53.363525+0000","last_unstale":"2024-04-25T01:41:28.687834+0000","last_undegraded":"2024-04-25T01:41:28.687834+0000","last_fullsized":"2024-04-25T01:41:28.687834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5001","last_scrub_stamp":"2024-04-25T01:41:28.687716+0000","last_deep_scrub":"26'5001","last_deep_scrub_stamp":"2024-04-25T01:41:28.687716+0000","last_clean_scrub_stamp":"2024-04-25T01:41:28.687716+0000","objects_scrubbed":0,"log_size":5001,"log_dups_size":0,"ondisk_log_size":5001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:39.071374+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25760,"num_write_kb":312110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"26'5076","reported_seq":5099,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:09.353688+0000","last_change":"2024-04-25T01:26:02.421208+0000","last_active":"2024-04-25T01:41:09.353688+0000","last_peered":"2024-04-25T01:41:09.353688+0000","last_clean":"2024-04-25T01:41:09.353688+0000","last_became_active":"2024-04-25T01:26:02.418796+0000","last_became_peered":"2024-04-25T01:26:02.418796+0000","last_unstale":"2024-04-25T01:41:09.353688+0000","last_undegraded":"2024-04-25T01:41:09.353688+0000","last_fullsized":"2024-04-25T01:41:09.353688+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:26:01.391184+0000","last_clean_scrub_stamp":"2024-04-25T01:26:01.391184+0000","objects_scrubbed":0,"log_size":5076,"log_dups_size":0,"ondisk_log_size":5076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25596,"num_write_kb":323780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"26'6025","reported_seq":6065,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:27.582035+0000","last_change":"2024-04-25T01:41:27.582081+0000","last_active":"2024-04-25T01:41:27.582035+0000","last_peered":"2024-04-25T01:41:27.582035+0000","last_clean":"2024-04-25T01:41:27.582035+0000","last_became_active":"2024-04-25T01:25:53.364793+0000","last_became_peered":"2024-04-25T01:25:53.364793+0000","last_unstale":"2024-04-25T01:41:27.582035+0000","last_undegraded":"2024-04-25T01:41:27.582035+0000","last_fullsized":"2024-04-25T01:41:27.582035+0000","mapping_epoch":18,"log_start":"26'200","ondisk_log_start":"26'200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'6025","last_scrub_stamp":"2024-04-25T01:41:27.581921+0000","last_deep_scrub":"26'6025","last_deep_scrub_stamp":"2024-04-25T01:41:27.581921+0000","last_clean_scrub_stamp":"2024-04-25T01:41:27.581921+0000","objects_scrubbed":0,"log_size":5825,"log_dups_size":0,"ondisk_log_size":5825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:57.168912+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31759,"num_write_kb":385508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"26'4493","reported_seq":4524,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:30.838689+0000","last_change":"2024-04-25T01:41:30.838716+0000","last_active":"2024-04-25T01:41:30.838689+0000","last_peered":"2024-04-25T01:41:30.838689+0000","last_clean":"2024-04-25T01:41:30.838689+0000","last_became_active":"2024-04-25T01:26:02.417350+0000","last_became_peered":"2024-04-25T01:26:02.417350+0000","last_unstale":"2024-04-25T01:41:30.838689+0000","last_undegraded":"2024-04-25T01:41:30.838689+0000","last_fullsized":"2024-04-25T01:41:30.838689+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4493","last_scrub_stamp":"2024-04-25T01:41:30.838583+0000","last_deep_scrub":"26'4493","last_deep_scrub_stamp":"2024-04-25T01:41:30.838583+0000","last_clean_scrub_stamp":"2024-04-25T01:41:30.838583+0000","objects_scrubbed":0,"log_size":4493,"log_dups_size":0,"ondisk_log_size":4493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:13.949961+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23464,"num_write_kb":290699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"26'4350","reported_seq":4390,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:29.880342+0000","last_change":"2024-04-25T01:41:29.880377+0000","last_active":"2024-04-25T01:41:29.880342+0000","last_peered":"2024-04-25T01:41:29.880342+0000","last_clean":"2024-04-25T01:41:29.880342+0000","last_became_active":"2024-04-25T01:25:53.367383+0000","last_became_peered":"2024-04-25T01:25:53.367383+0000","last_unstale":"2024-04-25T01:41:29.880342+0000","last_undegraded":"2024-04-25T01:41:29.880342+0000","last_fullsized":"2024-04-25T01:41:29.880342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4350","last_scrub_stamp":"2024-04-25T01:41:29.880242+0000","last_deep_scrub":"26'4350","last_deep_scrub_stamp":"2024-04-25T01:41:29.880242+0000","last_clean_scrub_stamp":"2024-04-25T01:41:29.880242+0000","objects_scrubbed":0,"log_size":4350,"log_dups_size":0,"ondisk_log_size":4350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:18.432207+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23070,"num_write_kb":268669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1533236,"num_write_kb":19254592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":294168,"ondisk_log_size":294168,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":17605243,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":39,"num_read_kb":112,"num_write":4172,"num_write_kb":18793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35274752,"data_stored":35210486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3524,"ondisk_log_size":3524,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1596536,"num_write_kb":19509893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305948,"ondisk_log_size":305948,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":17847589,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":41,"num_read_kb":115,"num_write":4224,"num_write_kb":19059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35758080,"data_stored":35695178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3561,"ondisk_log_size":3561,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575045,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1217764,"kb_used_data":13148,"kb_used_omap":1,"kb_used_meta":1204606,"kb_avail":93154076,"statfs":{"total":96636764160,"available":95389773824,"internally_reserved":0,"allocated":13463552,"data_stored":13287019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":1233517002},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575042,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":780520,"kb_used_data":9440,"kb_used_omap":1,"kb_used_meta":771070,"kb_avail":93591320,"statfs":{"total":96636764160,"available":95837511680,"internally_reserved":0,"allocated":9666560,"data_stored":9483786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":789576134},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575043,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1059708,"kb_used_data":4468,"kb_used_omap":1,"kb_used_meta":1055230,"kb_avail":93312132,"statfs":{"total":96636764160,"available":95551623168,"internally_reserved":0,"allocated":4575232,"data_stored":4396213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1080555973},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575043,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1089176,"kb_used_data":10256,"kb_used_omap":1,"kb_used_meta":1078910,"kb_avail":93282664,"statfs":{"total":96636764160,"available":95521447936,"internally_reserved":0,"allocated":10502144,"data_stored":10309285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1104804293},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1161176,"kb_used_data":17488,"kb_used_omap":1,"kb_used_meta":1143678,"kb_avail":93210664,"statfs":{"total":96636764160,"available":95447719936,"internally_reserved":0,"allocated":17907712,"data_stored":17718206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1171126727},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575044,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":729908,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":729534,"kb_avail":93641932,"statfs":{"total":96636764160,"available":95889338368,"internally_reserved":0,"allocated":372736,"data_stored":201217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":747043269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575044,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":888620,"kb_used_data":8484,"kb_used_omap":1,"kb_used_meta":880126,"kb_avail":93483220,"statfs":{"total":96636764160,"available":95726817280,"internally_reserved":0,"allocated":8687616,"data_stored":8502529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":901249477},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575042,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":919656,"kb_used_data":9056,"kb_used_omap":1,"kb_used_meta":910590,"kb_avail":93452184,"statfs":{"total":96636764160,"available":95695036416,"internally_reserved":0,"allocated":9273344,"data_stored":9092681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":932444613},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":88080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8486912,"data_stored":8477360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5189632,"data_stored":5175253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5185536,"data_stored":5175799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8389302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9142272,"data_stored":9127779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5033984,"data_stored":5020965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4292608,"data_stored":4283056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:41:42.203 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.203 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.203 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-04-25T01:25:51.329498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.203 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.204 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.204 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.204 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.204 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.204 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.204 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.205 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.205 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.205 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.205 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.205 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.205 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.206 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.206 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.206 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.206 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.206 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.207 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.207 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.207 INFO:tasks.ceph:pgid 5.26 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.207 INFO:tasks.ceph:pgid 5.24 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.207 INFO:tasks.ceph:pgid 5.25 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.207 INFO:tasks.ceph:pgid 5.23 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.207 INFO:tasks.ceph:pgid 5.20 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.207 INFO:tasks.ceph:pgid 5.21 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.208 INFO:tasks.ceph:pgid 5.2e last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.208 INFO:tasks.ceph:pgid 5.2d last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.208 INFO:tasks.ceph:pgid 5.2a last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.208 INFO:tasks.ceph:pgid 5.28 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.208 INFO:tasks.ceph:pgid 5.29 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.208 INFO:tasks.ceph:pgid 5.36 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.208 INFO:tasks.ceph:pgid 5.37 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.208 INFO:tasks.ceph:pgid 5.34 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.209 INFO:tasks.ceph:pgid 5.35 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.209 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.209 INFO:tasks.ceph:pgid 5.30 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.209 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.209 INFO:tasks.ceph:pgid 5.3f last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.209 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-25T01:25:52.333844+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=25, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.209 INFO:tasks.ceph:pgid 5.3d last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.210 INFO:tasks.ceph:pgid 5.3a last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.210 INFO:tasks.ceph:pgid 5.38 last_scrub_stamp 2024-04-25T01:26:01.391184+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=26, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=41, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:41:42.210 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T01:42:00.242 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:42:00.245 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:42:02.211 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:42:02.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.332+0000 7f7436b4c640 1 Processor -- start 2024-04-25T01:42:02.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.332+0000 7f7436b4c640 1 -- start start 2024-04-25T01:42:02.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.332+0000 7f7436b4c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f74301536c0 0x7f74300c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.332+0000 7f7436b4c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f743006e0f0 con 0x7f7430071db0 2024-04-25T01:42:02.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.332+0000 7f7436b4c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74300c2180 con 0x7f7430072d80 2024-04-25T01:42:02.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.332+0000 7f7436b4c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f74300c2460 con 0x7f74301536c0 2024-04-25T01:42:02.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.333+0000 7f742ffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f74301536c0 0x7f74300c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.333+0000 7f74350e9640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7430071db0 0x7f74300721c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:43426/0 (socket says 172.21.15.112:43426) 2024-04-25T01:42:02.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.333+0000 7f74350e9640 1 -- 172.21.15.112:0/302352590 learned_addr learned my addr 172.21.15.112:0/302352590 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:42:02.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.333+0000 7f742ffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f74301536c0 0x7f74300c1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37614/0 (socket says 172.21.15.112:37614) 2024-04-25T01:42:02.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.333+0000 7f742ffff640 1 -- 172.21.15.112:0/302352590 >> v1:172.21.15.187:6789/0 conn(0x7f7430072d80 legacy=0x7f743014d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:02.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.333+0000 7f742ffff640 1 -- 172.21.15.112:0/302352590 >> v1:172.21.15.112:6789/0 conn(0x7f7430071db0 legacy=0x7f74300721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.333+0000 7f742ffff640 1 -- 172.21.15.112:0/302352590 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74300c2740 con 0x7f74301536c0 2024-04-25T01:42:02.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.334+0000 7f742ffff640 1 --2- 172.21.15.112:0/302352590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74301536c0 0x7f74300c1a80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f741c0097a0 tx=0x7f741c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc95679f7753896 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:02.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.334+0000 7f742f7fe640 1 -- 172.21.15.112:0/302352590 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c037070 con 0x7f74301536c0 2024-04-25T01:42:02.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.334+0000 7f742f7fe640 1 -- 172.21.15.112:0/302352590 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f741c032440 con 0x7f74301536c0 2024-04-25T01:42:02.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.334+0000 7f742f7fe640 1 -- 172.21.15.112:0/302352590 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c032810 con 0x7f74301536c0 2024-04-25T01:42:02.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.334+0000 7f7436b4c640 1 -- 172.21.15.112:0/302352590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74301536c0 msgr2=0x7f74300c1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.334+0000 7f7436b4c640 1 --2- 172.21.15.112:0/302352590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74301536c0 0x7f74300c1a80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f741c0097a0 tx=0x7f741c02a1e0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.334+0000 7f7436b4c640 1 -- 172.21.15.112:0/302352590 shutdown_connections 2024-04-25T01:42:02.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.334+0000 7f7436b4c640 1 --2- 172.21.15.112:0/302352590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74301536c0 0x7f74300c1a80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.334+0000 7f7436b4c640 1 -- 172.21.15.112:0/302352590 >> 172.21.15.112:0/302352590 conn(0x7f743006b160 msgr2=0x7f743006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:02.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.334+0000 7f7436b4c640 1 -- 172.21.15.112:0/302352590 shutdown_connections 2024-04-25T01:42:02.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.334+0000 7f7436b4c640 1 -- 172.21.15.112:0/302352590 wait complete. 2024-04-25T01:42:02.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.335+0000 7f7436b4c640 1 Processor -- start 2024-04-25T01:42:02.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.336+0000 7f7436b4c640 1 -- start start 2024-04-25T01:42:02.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.336+0000 7f7436b4c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7430071db0 0x7f7430142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.336+0000 7f7436b4c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7430072d80 0x7f743014c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.336+0000 7f7436b4c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74301536c0 0x7f7430147850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.336+0000 7f7436b4c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74300728d0 con 0x7f7430072d80 2024-04-25T01:42:02.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.336+0000 7f7436b4c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7430071890 con 0x7f74301536c0 2024-04-25T01:42:02.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.336+0000 7f7436b4c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f743014da50 con 0x7f7430071db0 2024-04-25T01:42:02.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.336+0000 7f742ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7430072d80 0x7f743014c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.337+0000 7f742ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7430072d80 0x7f743014c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37624/0 (socket says 172.21.15.112:37624) 2024-04-25T01:42:02.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.337+0000 7f742ffff640 1 -- 172.21.15.112:0/3018129721 learned_addr learned my addr 172.21.15.112:0/3018129721 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:02.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.337+0000 7f742ffff640 1 -- 172.21.15.112:0/3018129721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7430071db0 msgr2=0x7f7430142090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.337+0000 7f742ffff640 1 --2- 172.21.15.112:0/3018129721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7430071db0 0x7f7430142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.337+0000 7f74350e9640 1 --2- 172.21.15.112:0/3018129721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74301536c0 0x7f7430147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.337+0000 7f742ffff640 1 -- 172.21.15.112:0/3018129721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74301536c0 msgr2=0x7f7430147850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.337+0000 7f742ffff640 1 --2- 172.21.15.112:0/3018129721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74301536c0 0x7f7430147850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.337+0000 7f742ffff640 1 -- 172.21.15.112:0/3018129721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743014bee0 con 0x7f7430072d80 2024-04-25T01:42:02.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.337+0000 7f74350e9640 1 --2- 172.21.15.112:0/3018129721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74301536c0 0x7f7430147850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:42:02.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.337+0000 7f742ffff640 1 --2- 172.21.15.112:0/3018129721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7430072d80 0x7f743014c670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f741c000c00 tx=0x7f741c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:02.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.337+0000 7f742d7fa640 1 -- 172.21.15.112:0/3018129721 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c037040 con 0x7f7430072d80 2024-04-25T01:42:02.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.337+0000 7f742d7fa640 1 -- 172.21.15.112:0/3018129721 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f741c042070 con 0x7f7430072d80 2024-04-25T01:42:02.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.337+0000 7f7436b4c640 1 -- 172.21.15.112:0/3018129721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743014b180 con 0x7f7430072d80 2024-04-25T01:42:02.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.337+0000 7f742d7fa640 1 -- 172.21.15.112:0/3018129721 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c030a70 con 0x7f7430072d80 2024-04-25T01:42:02.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.337+0000 7f7436b4c640 1 -- 172.21.15.112:0/3018129721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74301425d0 con 0x7f7430072d80 2024-04-25T01:42:02.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.338+0000 7f7436b4c640 1 -- 172.21.15.112:0/3018129721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74300682a0 con 0x7f7430072d80 2024-04-25T01:42:02.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.339+0000 7f742d7fa640 1 -- 172.21.15.112:0/3018129721 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f741c030c10 con 0x7f7430072d80 2024-04-25T01:42:02.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.339+0000 7f742d7fa640 1 --2- 172.21.15.112:0/3018129721 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f741406be60 0x7f741406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.339+0000 7f742d7fa640 1 -- 172.21.15.112:0/3018129721 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f741c0abc10 con 0x7f7430072d80 2024-04-25T01:42:02.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.339+0000 7f74348e8640 1 --2- 172.21.15.112:0/3018129721 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f741406be60 0x7f741406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.340+0000 7f74348e8640 1 --2- 172.21.15.112:0/3018129721 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f741406be60 0x7f741406e320 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7430149420 tx=0x7f74240079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:02.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.344+0000 7f742d7fa640 1 -- 172.21.15.112:0/3018129721 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f741c045360 con 0x7f7430072d80 2024-04-25T01:42:02.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.518+0000 7f7436b4c640 1 -- 172.21.15.112:0/3018129721 --> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f74300c7cf0 con 0x7f741406be60 2024-04-25T01:42:02.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.529+0000 7f742d7fa640 1 -- 172.21.15.112:0/3018129721 <== mgr.4104 v2:172.21.15.112:6832/3661599222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+494969 (secure 0 0 0) 0x7f7424005030 con 0x7f741406be60 2024-04-25T01:42:02.522 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:42:02.526 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T01:42:02.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.534+0000 7f7436b4c640 1 -- 172.21.15.112:0/3018129721 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f741406be60 msgr2=0x7f741406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.534+0000 7f7436b4c640 1 --2- 172.21.15.112:0/3018129721 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f741406be60 0x7f741406e320 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7430149420 tx=0x7f74240079b0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.534+0000 7f7436b4c640 1 -- 172.21.15.112:0/3018129721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7430072d80 msgr2=0x7f743014c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.534+0000 7f7436b4c640 1 --2- 172.21.15.112:0/3018129721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7430072d80 0x7f743014c670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f741c000c00 tx=0x7f741c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.534+0000 7f7436b4c640 1 -- 172.21.15.112:0/3018129721 shutdown_connections 2024-04-25T01:42:02.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.534+0000 7f7436b4c640 1 --2- 172.21.15.112:0/3018129721 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f741406be60 0x7f741406e320 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.534+0000 7f7436b4c640 1 --2- 172.21.15.112:0/3018129721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74301536c0 0x7f7430147850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.534+0000 7f7436b4c640 1 --2- 172.21.15.112:0/3018129721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7430072d80 0x7f743014c670 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.534+0000 7f7436b4c640 1 --2- 172.21.15.112:0/3018129721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7430071db0 0x7f7430142090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.534+0000 7f7436b4c640 1 -- 172.21.15.112:0/3018129721 >> 172.21.15.112:0/3018129721 conn(0x7f743006b160 msgr2=0x7f7430072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:02.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.534+0000 7f7436b4c640 1 -- 172.21.15.112:0/3018129721 shutdown_connections 2024-04-25T01:42:02.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.534+0000 7f7436b4c640 1 -- 172.21.15.112:0/3018129721 wait complete. 2024-04-25T01:42:02.533 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2024-04-25T01:42:00.928875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":36043200,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":196,"num_read_kb":327,"num_write":3138327,"num_write_kb":38804753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10000,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":607321,"ondisk_log_size":607321,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":7846528,"kb_used_data":72704,"kb_used_omap":12,"kb_used_meta":7773747,"kb_avail":747128192,"statfs":{"total":773094113280,"available":765059268608,"internally_reserved":0,"allocated":74448896,"data_stored":72990936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":7960317488},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004170"},"pg_stats":[{"pgid":"5.18","version":"26'4099","reported_seq":4130,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:36.129609+0000","last_change":"2024-04-25T01:41:36.129637+0000","last_active":"2024-04-25T01:41:36.129609+0000","last_peered":"2024-04-25T01:41:36.129609+0000","last_clean":"2024-04-25T01:41:36.129609+0000","last_became_active":"2024-04-25T01:26:02.411176+0000","last_became_peered":"2024-04-25T01:26:02.411176+0000","last_unstale":"2024-04-25T01:41:36.129609+0000","last_undegraded":"2024-04-25T01:41:36.129609+0000","last_fullsized":"2024-04-25T01:41:36.129609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4099","last_scrub_stamp":"2024-04-25T01:41:36.129495+0000","last_deep_scrub":"26'4099","last_deep_scrub_stamp":"2024-04-25T01:41:36.129495+0000","last_clean_scrub_stamp":"2024-04-25T01:41:36.129495+0000","objects_scrubbed":0,"log_size":4099,"log_dups_size":0,"ondisk_log_size":4099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:48.901163+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21931,"num_write_kb":288545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"26'661","reported_seq":703,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:24.733384+0000","last_change":"2024-04-25T01:41:24.733415+0000","last_active":"2024-04-25T01:41:24.733384+0000","last_peered":"2024-04-25T01:41:24.733384+0000","last_clean":"2024-04-25T01:41:24.733384+0000","last_became_active":"2024-04-25T01:25:52.353649+0000","last_became_peered":"2024-04-25T01:25:52.353649+0000","last_unstale":"2024-04-25T01:41:24.733384+0000","last_undegraded":"2024-04-25T01:41:24.733384+0000","last_fullsized":"2024-04-25T01:41:24.733384+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'661","last_scrub_stamp":"2024-04-25T01:41:24.733307+0000","last_deep_scrub":"26'661","last_deep_scrub_stamp":"2024-04-25T01:41:24.733307+0000","last_clean_scrub_stamp":"2024-04-25T01:41:24.733307+0000","objects_scrubbed":3,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:12.604209+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":4468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"26'5333","reported_seq":5374,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:33.921198+0000","last_change":"2024-04-25T01:41:33.921198+0000","last_active":"2024-04-25T01:41:33.921198+0000","last_peered":"2024-04-25T01:41:33.921198+0000","last_clean":"2024-04-25T01:41:33.921198+0000","last_became_active":"2024-04-25T01:25:53.363409+0000","last_became_peered":"2024-04-25T01:25:53.363409+0000","last_unstale":"2024-04-25T01:41:33.921198+0000","last_undegraded":"2024-04-25T01:41:33.921198+0000","last_fullsized":"2024-04-25T01:41:33.921198+0000","mapping_epoch":18,"log_start":"26'100","ondisk_log_start":"26'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5333","last_scrub_stamp":"2024-04-25T01:41:33.921067+0000","last_deep_scrub":"26'5333","last_deep_scrub_stamp":"2024-04-25T01:41:33.921067+0000","last_clean_scrub_stamp":"2024-04-25T01:41:33.921067+0000","objects_scrubbed":0,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:09.673744+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28405,"num_write_kb":331018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"26'3","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:23.833381+0000","last_change":"2024-04-25T01:41:23.833432+0000","last_active":"2024-04-25T01:41:23.833381+0000","last_peered":"2024-04-25T01:41:23.833381+0000","last_clean":"2024-04-25T01:41:23.833381+0000","last_became_active":"2024-04-25T01:26:01.406855+0000","last_became_peered":"2024-04-25T01:26:01.406855+0000","last_unstale":"2024-04-25T01:41:23.833381+0000","last_undegraded":"2024-04-25T01:41:23.833381+0000","last_fullsized":"2024-04-25T01:41:23.833381+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:23.833281+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:23.833281+0000","last_clean_scrub_stamp":"2024-04-25T01:41:23.833281+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:10.367070+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"26'5328","reported_seq":5359,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:39.621502+0000","last_change":"2024-04-25T01:41:39.621550+0000","last_active":"2024-04-25T01:41:39.621502+0000","last_peered":"2024-04-25T01:41:39.621502+0000","last_clean":"2024-04-25T01:41:39.621502+0000","last_became_active":"2024-04-25T01:26:02.425071+0000","last_became_peered":"2024-04-25T01:26:02.425071+0000","last_unstale":"2024-04-25T01:41:39.621502+0000","last_undegraded":"2024-04-25T01:41:39.621502+0000","last_fullsized":"2024-04-25T01:41:39.621502+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5328","last_scrub_stamp":"2024-04-25T01:41:39.621385+0000","last_deep_scrub":"26'5328","last_deep_scrub_stamp":"2024-04-25T01:41:39.621385+0000","last_clean_scrub_stamp":"2024-04-25T01:41:39.621385+0000","objects_scrubbed":0,"log_size":5328,"log_dups_size":0,"ondisk_log_size":5328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:24.762450+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27477,"num_write_kb":347746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"26'8","reported_seq":51,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:35.909456+0000","last_change":"2024-04-25T01:41:35.909456+0000","last_active":"2024-04-25T01:41:35.909456+0000","last_peered":"2024-04-25T01:41:35.909456+0000","last_clean":"2024-04-25T01:41:35.909456+0000","last_became_active":"2024-04-25T01:25:52.348931+0000","last_became_peered":"2024-04-25T01:25:52.348931+0000","last_unstale":"2024-04-25T01:41:35.909456+0000","last_undegraded":"2024-04-25T01:41:35.909456+0000","last_fullsized":"2024-04-25T01:41:35.909456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'8","last_scrub_stamp":"2024-04-25T01:41:35.909325+0000","last_deep_scrub":"26'8","last_deep_scrub_stamp":"2024-04-25T01:41:35.909325+0000","last_clean_scrub_stamp":"2024-04-25T01:41:35.909325+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:08.452523+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"26'5837","reported_seq":5877,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:20.667334+0000","last_change":"2024-04-25T01:41:20.667358+0000","last_active":"2024-04-25T01:41:20.667334+0000","last_peered":"2024-04-25T01:41:20.667334+0000","last_clean":"2024-04-25T01:41:20.667334+0000","last_became_active":"2024-04-25T01:25:53.373755+0000","last_became_peered":"2024-04-25T01:25:53.373755+0000","last_unstale":"2024-04-25T01:41:20.667334+0000","last_undegraded":"2024-04-25T01:41:20.667334+0000","last_fullsized":"2024-04-25T01:41:20.667334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5837","last_scrub_stamp":"2024-04-25T01:41:20.667241+0000","last_deep_scrub":"26'5837","last_deep_scrub_stamp":"2024-04-25T01:41:20.667241+0000","last_clean_scrub_stamp":"2024-04-25T01:41:20.667241+0000","objects_scrubbed":0,"log_size":5837,"log_dups_size":0,"ondisk_log_size":5837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:49.049409+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30942,"num_write_kb":369940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"26'3","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:35.153544+0000","last_change":"2024-04-25T01:41:35.153570+0000","last_active":"2024-04-25T01:41:35.153544+0000","last_peered":"2024-04-25T01:41:35.153544+0000","last_clean":"2024-04-25T01:41:35.153544+0000","last_became_active":"2024-04-25T01:26:01.405994+0000","last_became_peered":"2024-04-25T01:26:01.405994+0000","last_unstale":"2024-04-25T01:41:35.153544+0000","last_undegraded":"2024-04-25T01:41:35.153544+0000","last_fullsized":"2024-04-25T01:41:35.153544+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:35.153451+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:35.153451+0000","last_clean_scrub_stamp":"2024-04-25T01:41:35.153451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:02.234784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"26'4188","reported_seq":4219,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:48.732766+0000","last_change":"2024-04-25T01:41:48.732796+0000","last_active":"2024-04-25T01:41:48.732766+0000","last_peered":"2024-04-25T01:41:48.732766+0000","last_clean":"2024-04-25T01:41:48.732766+0000","last_became_active":"2024-04-25T01:26:02.427443+0000","last_became_peered":"2024-04-25T01:26:02.427443+0000","last_unstale":"2024-04-25T01:41:48.732766+0000","last_undegraded":"2024-04-25T01:41:48.732766+0000","last_fullsized":"2024-04-25T01:41:48.732766+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4188","last_scrub_stamp":"2024-04-25T01:41:48.732656+0000","last_deep_scrub":"26'4188","last_deep_scrub_stamp":"2024-04-25T01:41:48.732656+0000","last_clean_scrub_stamp":"2024-04-25T01:41:48.732656+0000","objects_scrubbed":0,"log_size":4188,"log_dups_size":0,"ondisk_log_size":4188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:09.491193+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20869,"num_write_kb":259707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"26'12","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:37.752408+0000","last_change":"2024-04-25T01:41:37.752439+0000","last_active":"2024-04-25T01:41:37.752408+0000","last_peered":"2024-04-25T01:41:37.752408+0000","last_clean":"2024-04-25T01:41:37.752408+0000","last_became_active":"2024-04-25T01:25:52.355822+0000","last_became_peered":"2024-04-25T01:25:52.355822+0000","last_unstale":"2024-04-25T01:41:37.752408+0000","last_undegraded":"2024-04-25T01:41:37.752408+0000","last_fullsized":"2024-04-25T01:41:37.752408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'12","last_scrub_stamp":"2024-04-25T01:41:37.752319+0000","last_deep_scrub":"26'12","last_deep_scrub_stamp":"2024-04-25T01:41:37.752319+0000","last_clean_scrub_stamp":"2024-04-25T01:41:37.752319+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:47.278639+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"26'3747","reported_seq":3786,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:29.378134+0000","last_change":"2024-04-25T01:41:29.378161+0000","last_active":"2024-04-25T01:41:29.378134+0000","last_peered":"2024-04-25T01:41:29.378134+0000","last_clean":"2024-04-25T01:41:29.378134+0000","last_became_active":"2024-04-25T01:25:53.369721+0000","last_became_peered":"2024-04-25T01:25:53.369721+0000","last_unstale":"2024-04-25T01:41:29.378134+0000","last_undegraded":"2024-04-25T01:41:29.378134+0000","last_fullsized":"2024-04-25T01:41:29.378134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3747","last_scrub_stamp":"2024-04-25T01:41:29.378020+0000","last_deep_scrub":"26'3747","last_deep_scrub_stamp":"2024-04-25T01:41:29.378020+0000","last_clean_scrub_stamp":"2024-04-25T01:41:29.378020+0000","objects_scrubbed":0,"log_size":3747,"log_dups_size":0,"ondisk_log_size":3747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:36:39.101213+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18978,"num_write_kb":223593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"26'628","reported_seq":661,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:25.141668+0000","last_change":"2024-04-25T01:41:25.141715+0000","last_active":"2024-04-25T01:41:25.141668+0000","last_peered":"2024-04-25T01:41:25.141668+0000","last_clean":"2024-04-25T01:41:25.141668+0000","last_became_active":"2024-04-25T01:26:01.407909+0000","last_became_peered":"2024-04-25T01:26:01.407909+0000","last_unstale":"2024-04-25T01:41:25.141668+0000","last_undegraded":"2024-04-25T01:41:25.141668+0000","last_fullsized":"2024-04-25T01:41:25.141668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'628","last_scrub_stamp":"2024-04-25T01:41:25.141559+0000","last_deep_scrub":"26'628","last_deep_scrub_stamp":"2024-04-25T01:41:25.141559+0000","last_clean_scrub_stamp":"2024-04-25T01:41:25.141559+0000","objects_scrubbed":1,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:33.537623+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":625,"num_write_kb":4235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"26'3441","reported_seq":3472,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:45.156814+0000","last_change":"2024-04-25T01:41:45.156866+0000","last_active":"2024-04-25T01:41:45.156814+0000","last_peered":"2024-04-25T01:41:45.156814+0000","last_clean":"2024-04-25T01:41:45.156814+0000","last_became_active":"2024-04-25T01:26:02.424325+0000","last_became_peered":"2024-04-25T01:26:02.424325+0000","last_unstale":"2024-04-25T01:41:45.156814+0000","last_undegraded":"2024-04-25T01:41:45.156814+0000","last_fullsized":"2024-04-25T01:41:45.156814+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3441","last_scrub_stamp":"2024-04-25T01:41:45.156698+0000","last_deep_scrub":"26'3441","last_deep_scrub_stamp":"2024-04-25T01:41:45.156698+0000","last_clean_scrub_stamp":"2024-04-25T01:41:45.156698+0000","objects_scrubbed":0,"log_size":3441,"log_dups_size":0,"ondisk_log_size":3441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:36.235069+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17582,"num_write_kb":230191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"26'4","reported_seq":45,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:40.753385+0000","last_change":"2024-04-25T01:41:40.753424+0000","last_active":"2024-04-25T01:41:40.753385+0000","last_peered":"2024-04-25T01:41:40.753385+0000","last_clean":"2024-04-25T01:41:40.753385+0000","last_became_active":"2024-04-25T01:25:52.358841+0000","last_became_peered":"2024-04-25T01:25:52.358841+0000","last_unstale":"2024-04-25T01:41:40.753385+0000","last_undegraded":"2024-04-25T01:41:40.753385+0000","last_fullsized":"2024-04-25T01:41:40.753385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4","last_scrub_stamp":"2024-04-25T01:41:40.753297+0000","last_deep_scrub":"26'4","last_deep_scrub_stamp":"2024-04-25T01:41:40.753297+0000","last_clean_scrub_stamp":"2024-04-25T01:41:40.753297+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:14.647848+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"26'5828","reported_seq":5867,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:39.362455+0000","last_change":"2024-04-25T01:41:39.362493+0000","last_active":"2024-04-25T01:41:39.362455+0000","last_peered":"2024-04-25T01:41:39.362455+0000","last_clean":"2024-04-25T01:41:39.362455+0000","last_became_active":"2024-04-25T01:25:53.368470+0000","last_became_peered":"2024-04-25T01:25:53.368470+0000","last_unstale":"2024-04-25T01:41:39.362455+0000","last_undegraded":"2024-04-25T01:41:39.362455+0000","last_fullsized":"2024-04-25T01:41:39.362455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5828","last_scrub_stamp":"2024-04-25T01:41:39.362342+0000","last_deep_scrub":"26'5828","last_deep_scrub_stamp":"2024-04-25T01:41:39.362342+0000","last_clean_scrub_stamp":"2024-04-25T01:41:39.362342+0000","objects_scrubbed":0,"log_size":5828,"log_dups_size":0,"ondisk_log_size":5828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:54.090399+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30463,"num_write_kb":367857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"26'65","reported_seq":98,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:27.126064+0000","last_change":"2024-04-25T01:41:27.126092+0000","last_active":"2024-04-25T01:41:27.126064+0000","last_peered":"2024-04-25T01:41:27.126064+0000","last_clean":"2024-04-25T01:41:27.126064+0000","last_became_active":"2024-04-25T01:26:01.407655+0000","last_became_peered":"2024-04-25T01:26:01.407655+0000","last_unstale":"2024-04-25T01:41:27.126064+0000","last_undegraded":"2024-04-25T01:41:27.126064+0000","last_fullsized":"2024-04-25T01:41:27.126064+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'65","last_scrub_stamp":"2024-04-25T01:41:27.125966+0000","last_deep_scrub":"26'65","last_deep_scrub_stamp":"2024-04-25T01:41:27.125966+0000","last_clean_scrub_stamp":"2024-04-25T01:41:27.125966+0000","objects_scrubbed":1,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:40.858022+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":738477,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61,"num_write_kb":764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"26'4794","reported_seq":4826,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:37.603772+0000","last_change":"2024-04-25T01:41:37.603772+0000","last_active":"2024-04-25T01:41:37.603772+0000","last_peered":"2024-04-25T01:41:37.603772+0000","last_clean":"2024-04-25T01:41:37.603772+0000","last_became_active":"2024-04-25T01:26:02.417524+0000","last_became_peered":"2024-04-25T01:26:02.417524+0000","last_unstale":"2024-04-25T01:41:37.603772+0000","last_undegraded":"2024-04-25T01:41:37.603772+0000","last_fullsized":"2024-04-25T01:41:37.603772+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4794","last_scrub_stamp":"2024-04-25T01:41:37.603630+0000","last_deep_scrub":"26'4794","last_deep_scrub_stamp":"2024-04-25T01:41:37.603630+0000","last_clean_scrub_stamp":"2024-04-25T01:41:37.603630+0000","objects_scrubbed":0,"log_size":4794,"log_dups_size":0,"ondisk_log_size":4794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:47.129951+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24624,"num_write_kb":311668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"26'4532","reported_seq":4572,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:22.348956+0000","last_change":"2024-04-25T01:41:22.348983+0000","last_active":"2024-04-25T01:41:22.348956+0000","last_peered":"2024-04-25T01:41:22.348956+0000","last_clean":"2024-04-25T01:41:22.348956+0000","last_became_active":"2024-04-25T01:25:53.372064+0000","last_became_peered":"2024-04-25T01:25:53.372064+0000","last_unstale":"2024-04-25T01:41:22.348956+0000","last_undegraded":"2024-04-25T01:41:22.348956+0000","last_fullsized":"2024-04-25T01:41:22.348956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4532","last_scrub_stamp":"2024-04-25T01:41:22.348835+0000","last_deep_scrub":"26'4532","last_deep_scrub_stamp":"2024-04-25T01:41:22.348835+0000","last_clean_scrub_stamp":"2024-04-25T01:41:22.348835+0000","objects_scrubbed":0,"log_size":4532,"log_dups_size":0,"ondisk_log_size":4532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:50.731003+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23642,"num_write_kb":279226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"26'1102","reported_seq":1144,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:39.756674+0000","last_change":"2024-04-25T01:41:39.756674+0000","last_active":"2024-04-25T01:41:39.756674+0000","last_peered":"2024-04-25T01:41:39.756674+0000","last_clean":"2024-04-25T01:41:39.756674+0000","last_became_active":"2024-04-25T01:25:52.358318+0000","last_became_peered":"2024-04-25T01:25:52.358318+0000","last_unstale":"2024-04-25T01:41:39.756674+0000","last_undegraded":"2024-04-25T01:41:39.756674+0000","last_fullsized":"2024-04-25T01:41:39.756674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1102","last_scrub_stamp":"2024-04-25T01:41:39.756541+0000","last_deep_scrub":"26'1102","last_deep_scrub_stamp":"2024-04-25T01:41:39.756541+0000","last_clean_scrub_stamp":"2024-04-25T01:41:39.756541+0000","objects_scrubbed":2,"log_size":1102,"log_dups_size":0,"ondisk_log_size":1102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:24.897606+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1091,"num_write_kb":4419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"26'3","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:34.171264+0000","last_change":"2024-04-25T01:41:34.171264+0000","last_active":"2024-04-25T01:41:34.171264+0000","last_peered":"2024-04-25T01:41:34.171264+0000","last_clean":"2024-04-25T01:41:34.171264+0000","last_became_active":"2024-04-25T01:26:01.400266+0000","last_became_peered":"2024-04-25T01:26:01.400266+0000","last_unstale":"2024-04-25T01:41:34.171264+0000","last_undegraded":"2024-04-25T01:41:34.171264+0000","last_fullsized":"2024-04-25T01:41:34.171264+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:34.171147+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:34.171147+0000","last_clean_scrub_stamp":"2024-04-25T01:41:34.171147+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:02.927773+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"26'4554","reported_seq":4586,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:34.091671+0000","last_change":"2024-04-25T01:41:34.091671+0000","last_active":"2024-04-25T01:41:34.091671+0000","last_peered":"2024-04-25T01:41:34.091671+0000","last_clean":"2024-04-25T01:41:34.091671+0000","last_became_active":"2024-04-25T01:26:02.420001+0000","last_became_peered":"2024-04-25T01:26:02.420001+0000","last_unstale":"2024-04-25T01:41:34.091671+0000","last_undegraded":"2024-04-25T01:41:34.091671+0000","last_fullsized":"2024-04-25T01:41:34.091671+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4554","last_scrub_stamp":"2024-04-25T01:41:34.091507+0000","last_deep_scrub":"26'4554","last_deep_scrub_stamp":"2024-04-25T01:41:34.091507+0000","last_clean_scrub_stamp":"2024-04-25T01:41:34.091507+0000","objects_scrubbed":0,"log_size":4554,"log_dups_size":0,"ondisk_log_size":4554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:08.382471+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23431,"num_write_kb":298735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"26'3793","reported_seq":3833,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:38.962817+0000","last_change":"2024-04-25T01:41:38.962817+0000","last_active":"2024-04-25T01:41:38.962817+0000","last_peered":"2024-04-25T01:41:38.962817+0000","last_clean":"2024-04-25T01:41:38.962817+0000","last_became_active":"2024-04-25T01:25:53.360802+0000","last_became_peered":"2024-04-25T01:25:53.360802+0000","last_unstale":"2024-04-25T01:41:38.962817+0000","last_undegraded":"2024-04-25T01:41:38.962817+0000","last_fullsized":"2024-04-25T01:41:38.962817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3793","last_scrub_stamp":"2024-04-25T01:41:38.962655+0000","last_deep_scrub":"26'3793","last_deep_scrub_stamp":"2024-04-25T01:41:38.962655+0000","last_clean_scrub_stamp":"2024-04-25T01:41:38.962655+0000","objects_scrubbed":0,"log_size":3793,"log_dups_size":0,"ondisk_log_size":3793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:06.654326+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19299,"num_write_kb":249413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"26'3","reported_seq":45,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:28.571808+0000","last_change":"2024-04-25T01:41:28.571855+0000","last_active":"2024-04-25T01:41:28.571808+0000","last_peered":"2024-04-25T01:41:28.571808+0000","last_clean":"2024-04-25T01:41:28.571808+0000","last_became_active":"2024-04-25T01:25:52.348550+0000","last_became_peered":"2024-04-25T01:25:52.348550+0000","last_unstale":"2024-04-25T01:41:28.571808+0000","last_undegraded":"2024-04-25T01:41:28.571808+0000","last_fullsized":"2024-04-25T01:41:28.571808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:28.571698+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:28.571698+0000","last_clean_scrub_stamp":"2024-04-25T01:41:28.571698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:48.741495+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"26'3","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:23.151869+0000","last_change":"2024-04-25T01:41:23.151869+0000","last_active":"2024-04-25T01:41:23.151869+0000","last_peered":"2024-04-25T01:41:23.151869+0000","last_clean":"2024-04-25T01:41:23.151869+0000","last_became_active":"2024-04-25T01:26:01.407738+0000","last_became_peered":"2024-04-25T01:26:01.407738+0000","last_unstale":"2024-04-25T01:41:23.151869+0000","last_undegraded":"2024-04-25T01:41:23.151869+0000","last_fullsized":"2024-04-25T01:41:23.151869+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:23.151736+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:23.151736+0000","last_clean_scrub_stamp":"2024-04-25T01:41:23.151736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:59.208630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"26'3863","reported_seq":3895,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:39.171157+0000","last_change":"2024-04-25T01:41:39.171157+0000","last_active":"2024-04-25T01:41:39.171157+0000","last_peered":"2024-04-25T01:41:39.171157+0000","last_clean":"2024-04-25T01:41:39.171157+0000","last_became_active":"2024-04-25T01:26:02.417438+0000","last_became_peered":"2024-04-25T01:26:02.417438+0000","last_unstale":"2024-04-25T01:41:39.171157+0000","last_undegraded":"2024-04-25T01:41:39.171157+0000","last_fullsized":"2024-04-25T01:41:39.171157+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3863","last_scrub_stamp":"2024-04-25T01:41:39.171017+0000","last_deep_scrub":"26'3863","last_deep_scrub_stamp":"2024-04-25T01:41:39.171017+0000","last_clean_scrub_stamp":"2024-04-25T01:41:39.171017+0000","objects_scrubbed":0,"log_size":3863,"log_dups_size":0,"ondisk_log_size":3863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:14.923694+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19753,"num_write_kb":275789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"26'4400","reported_seq":4440,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:39.963413+0000","last_change":"2024-04-25T01:41:39.963441+0000","last_active":"2024-04-25T01:41:39.963413+0000","last_peered":"2024-04-25T01:41:39.963413+0000","last_clean":"2024-04-25T01:41:39.963413+0000","last_became_active":"2024-04-25T01:25:53.361542+0000","last_became_peered":"2024-04-25T01:25:53.361542+0000","last_unstale":"2024-04-25T01:41:39.963413+0000","last_undegraded":"2024-04-25T01:41:39.963413+0000","last_fullsized":"2024-04-25T01:41:39.963413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4400","last_scrub_stamp":"2024-04-25T01:41:39.963267+0000","last_deep_scrub":"26'4400","last_deep_scrub_stamp":"2024-04-25T01:41:39.963267+0000","last_clean_scrub_stamp":"2024-04-25T01:41:39.963267+0000","objects_scrubbed":0,"log_size":4400,"log_dups_size":0,"ondisk_log_size":4400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:40.780905+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22906,"num_write_kb":296647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"26'3","reported_seq":45,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:30.601900+0000","last_change":"2024-04-25T01:41:30.601952+0000","last_active":"2024-04-25T01:41:30.601900+0000","last_peered":"2024-04-25T01:41:30.601900+0000","last_clean":"2024-04-25T01:41:30.601900+0000","last_became_active":"2024-04-25T01:25:52.353656+0000","last_became_peered":"2024-04-25T01:25:52.353656+0000","last_unstale":"2024-04-25T01:41:30.601900+0000","last_undegraded":"2024-04-25T01:41:30.601900+0000","last_fullsized":"2024-04-25T01:41:30.601900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:30.601782+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:30.601782+0000","last_clean_scrub_stamp":"2024-04-25T01:41:30.601782+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:13.713160+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"26'6","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:24.647257+0000","last_change":"2024-04-25T01:41:24.647257+0000","last_active":"2024-04-25T01:41:24.647257+0000","last_peered":"2024-04-25T01:41:24.647257+0000","last_clean":"2024-04-25T01:41:24.647257+0000","last_became_active":"2024-04-25T01:26:01.418263+0000","last_became_peered":"2024-04-25T01:26:01.418263+0000","last_unstale":"2024-04-25T01:41:24.647257+0000","last_undegraded":"2024-04-25T01:41:24.647257+0000","last_fullsized":"2024-04-25T01:41:24.647257+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'6","last_scrub_stamp":"2024-04-25T01:41:24.647078+0000","last_deep_scrub":"26'6","last_deep_scrub_stamp":"2024-04-25T01:41:24.647078+0000","last_clean_scrub_stamp":"2024-04-25T01:41:24.647078+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:01.964240+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"26'5570","reported_seq":5602,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:28.337651+0000","last_change":"2024-04-25T01:41:28.337651+0000","last_active":"2024-04-25T01:41:28.337651+0000","last_peered":"2024-04-25T01:41:28.337651+0000","last_clean":"2024-04-25T01:41:28.337651+0000","last_became_active":"2024-04-25T01:26:02.412632+0000","last_became_peered":"2024-04-25T01:26:02.412632+0000","last_unstale":"2024-04-25T01:41:28.337651+0000","last_undegraded":"2024-04-25T01:41:28.337651+0000","last_fullsized":"2024-04-25T01:41:28.337651+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5570","last_scrub_stamp":"2024-04-25T01:41:28.337518+0000","last_deep_scrub":"26'5570","last_deep_scrub_stamp":"2024-04-25T01:41:28.337518+0000","last_clean_scrub_stamp":"2024-04-25T01:41:28.337518+0000","objects_scrubbed":0,"log_size":5570,"log_dups_size":0,"ondisk_log_size":5570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:50.554037+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27971,"num_write_kb":349705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"26'4715","reported_seq":4755,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:29.199139+0000","last_change":"2024-04-25T01:41:29.199167+0000","last_active":"2024-04-25T01:41:29.199139+0000","last_peered":"2024-04-25T01:41:29.199139+0000","last_clean":"2024-04-25T01:41:29.199139+0000","last_became_active":"2024-04-25T01:25:53.361265+0000","last_became_peered":"2024-04-25T01:25:53.361265+0000","last_unstale":"2024-04-25T01:41:29.199139+0000","last_undegraded":"2024-04-25T01:41:29.199139+0000","last_fullsized":"2024-04-25T01:41:29.199139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4715","last_scrub_stamp":"2024-04-25T01:41:29.199039+0000","last_deep_scrub":"26'4715","last_deep_scrub_stamp":"2024-04-25T01:41:29.199039+0000","last_clean_scrub_stamp":"2024-04-25T01:41:29.199039+0000","objects_scrubbed":0,"log_size":4715,"log_dups_size":0,"ondisk_log_size":4715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:10.668527+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23886,"num_write_kb":288267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'3","reported_seq":45,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:35.749056+0000","last_change":"2024-04-25T01:41:35.749085+0000","last_active":"2024-04-25T01:41:35.749056+0000","last_peered":"2024-04-25T01:41:35.749056+0000","last_clean":"2024-04-25T01:41:35.749056+0000","last_became_active":"2024-04-25T01:25:52.353552+0000","last_became_peered":"2024-04-25T01:25:52.353552+0000","last_unstale":"2024-04-25T01:41:35.749056+0000","last_undegraded":"2024-04-25T01:41:35.749056+0000","last_fullsized":"2024-04-25T01:41:35.749056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:35.748960+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:35.748960+0000","last_clean_scrub_stamp":"2024-04-25T01:41:35.748960+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:54.247567+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"26'7","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:22.191626+0000","last_change":"2024-04-25T01:41:22.191626+0000","last_active":"2024-04-25T01:41:22.191626+0000","last_peered":"2024-04-25T01:41:22.191626+0000","last_clean":"2024-04-25T01:41:22.191626+0000","last_became_active":"2024-04-25T01:26:01.410378+0000","last_became_peered":"2024-04-25T01:26:01.410378+0000","last_unstale":"2024-04-25T01:41:22.191626+0000","last_undegraded":"2024-04-25T01:41:22.191626+0000","last_fullsized":"2024-04-25T01:41:22.191626+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'7","last_scrub_stamp":"2024-04-25T01:41:22.191504+0000","last_deep_scrub":"26'7","last_deep_scrub_stamp":"2024-04-25T01:41:22.191504+0000","last_clean_scrub_stamp":"2024-04-25T01:41:22.191504+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:10.062407+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"26'4403","reported_seq":4435,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:44.954860+0000","last_change":"2024-04-25T01:41:44.954897+0000","last_active":"2024-04-25T01:41:44.954860+0000","last_peered":"2024-04-25T01:41:44.954860+0000","last_clean":"2024-04-25T01:41:44.954860+0000","last_became_active":"2024-04-25T01:26:02.416395+0000","last_became_peered":"2024-04-25T01:26:02.416395+0000","last_unstale":"2024-04-25T01:41:44.954860+0000","last_undegraded":"2024-04-25T01:41:44.954860+0000","last_fullsized":"2024-04-25T01:41:44.954860+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4403","last_scrub_stamp":"2024-04-25T01:41:44.954736+0000","last_deep_scrub":"26'4403","last_deep_scrub_stamp":"2024-04-25T01:41:44.954736+0000","last_clean_scrub_stamp":"2024-04-25T01:41:44.954736+0000","objects_scrubbed":0,"log_size":4403,"log_dups_size":0,"ondisk_log_size":4403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:54.470544+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23642,"num_write_kb":287389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"26'3766","reported_seq":3806,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:29.694397+0000","last_change":"2024-04-25T01:41:29.694425+0000","last_active":"2024-04-25T01:41:29.694397+0000","last_peered":"2024-04-25T01:41:29.694397+0000","last_clean":"2024-04-25T01:41:29.694397+0000","last_became_active":"2024-04-25T01:25:53.369837+0000","last_became_peered":"2024-04-25T01:25:53.369837+0000","last_unstale":"2024-04-25T01:41:29.694397+0000","last_undegraded":"2024-04-25T01:41:29.694397+0000","last_fullsized":"2024-04-25T01:41:29.694397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3766","last_scrub_stamp":"2024-04-25T01:41:29.694281+0000","last_deep_scrub":"26'3766","last_deep_scrub_stamp":"2024-04-25T01:41:29.694281+0000","last_clean_scrub_stamp":"2024-04-25T01:41:29.694281+0000","objects_scrubbed":0,"log_size":3766,"log_dups_size":0,"ondisk_log_size":3766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:43.426336+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19293,"num_write_kb":249241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"26'6","reported_seq":48,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:28.270350+0000","last_change":"2024-04-25T01:41:28.270408+0000","last_active":"2024-04-25T01:41:28.270350+0000","last_peered":"2024-04-25T01:41:28.270350+0000","last_clean":"2024-04-25T01:41:28.270350+0000","last_became_active":"2024-04-25T01:25:52.352991+0000","last_became_peered":"2024-04-25T01:25:52.352991+0000","last_unstale":"2024-04-25T01:41:28.270350+0000","last_undegraded":"2024-04-25T01:41:28.270350+0000","last_fullsized":"2024-04-25T01:41:28.270350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'6","last_scrub_stamp":"2024-04-25T01:41:28.270225+0000","last_deep_scrub":"26'6","last_deep_scrub_stamp":"2024-04-25T01:41:28.270225+0000","last_clean_scrub_stamp":"2024-04-25T01:41:28.270225+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:09.220120+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"26'42","reported_seq":115,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:21.165819+0000","last_change":"2024-04-25T01:41:21.165848+0000","last_active":"2024-04-25T01:41:21.165819+0000","last_peered":"2024-04-25T01:41:21.165819+0000","last_clean":"2024-04-25T01:41:21.165819+0000","last_became_active":"2024-04-25T01:26:01.410076+0000","last_became_peered":"2024-04-25T01:26:01.410076+0000","last_unstale":"2024-04-25T01:41:21.165819+0000","last_undegraded":"2024-04-25T01:41:21.165819+0000","last_fullsized":"2024-04-25T01:41:21.165819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'42","last_scrub_stamp":"2024-04-25T01:41:21.165730+0000","last_deep_scrub":"26'42","last_deep_scrub_stamp":"2024-04-25T01:41:21.165730+0000","last_clean_scrub_stamp":"2024-04-25T01:41:21.165730+0000","objects_scrubbed":1,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:19.490457+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":112,"num_write":39,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"26'4289","reported_seq":4321,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:40.116145+0000","last_change":"2024-04-25T01:41:40.116172+0000","last_active":"2024-04-25T01:41:40.116145+0000","last_peered":"2024-04-25T01:41:40.116145+0000","last_clean":"2024-04-25T01:41:40.116145+0000","last_became_active":"2024-04-25T01:26:02.421885+0000","last_became_peered":"2024-04-25T01:26:02.421885+0000","last_unstale":"2024-04-25T01:41:40.116145+0000","last_undegraded":"2024-04-25T01:41:40.116145+0000","last_fullsized":"2024-04-25T01:41:40.116145+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4289","last_scrub_stamp":"2024-04-25T01:41:40.116030+0000","last_deep_scrub":"26'4289","last_deep_scrub_stamp":"2024-04-25T01:41:40.116030+0000","last_clean_scrub_stamp":"2024-04-25T01:41:40.116030+0000","objects_scrubbed":0,"log_size":4289,"log_dups_size":0,"ondisk_log_size":4289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:49.642350+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22172,"num_write_kb":285532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"26'4466","reported_seq":4506,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:29.902161+0000","last_change":"2024-04-25T01:41:29.902195+0000","last_active":"2024-04-25T01:41:29.902161+0000","last_peered":"2024-04-25T01:41:29.902161+0000","last_clean":"2024-04-25T01:41:29.902161+0000","last_became_active":"2024-04-25T01:25:53.364694+0000","last_became_peered":"2024-04-25T01:25:53.364694+0000","last_unstale":"2024-04-25T01:41:29.902161+0000","last_undegraded":"2024-04-25T01:41:29.902161+0000","last_fullsized":"2024-04-25T01:41:29.902161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4466","last_scrub_stamp":"2024-04-25T01:41:29.902047+0000","last_deep_scrub":"26'4466","last_deep_scrub_stamp":"2024-04-25T01:41:29.902047+0000","last_clean_scrub_stamp":"2024-04-25T01:41:29.902047+0000","objects_scrubbed":0,"log_size":4466,"log_dups_size":0,"ondisk_log_size":4466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:56.983380+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22859,"num_write_kb":292275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'572","reported_seq":614,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:32.557098+0000","last_change":"2024-04-25T01:41:32.557152+0000","last_active":"2024-04-25T01:41:32.557098+0000","last_peered":"2024-04-25T01:41:32.557098+0000","last_clean":"2024-04-25T01:41:32.557098+0000","last_became_active":"2024-04-25T01:25:52.355137+0000","last_became_peered":"2024-04-25T01:25:52.355137+0000","last_unstale":"2024-04-25T01:41:32.557098+0000","last_undegraded":"2024-04-25T01:41:32.557098+0000","last_fullsized":"2024-04-25T01:41:32.557098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'572","last_scrub_stamp":"2024-04-25T01:41:32.556995+0000","last_deep_scrub":"26'572","last_deep_scrub_stamp":"2024-04-25T01:41:32.556995+0000","last_clean_scrub_stamp":"2024-04-25T01:41:32.556995+0000","objects_scrubbed":1,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:59.638329+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"26'5","reported_seq":39,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:22.864117+0000","last_change":"2024-04-25T01:41:22.864160+0000","last_active":"2024-04-25T01:41:22.864117+0000","last_peered":"2024-04-25T01:41:22.864117+0000","last_clean":"2024-04-25T01:41:22.864117+0000","last_became_active":"2024-04-25T01:26:01.415824+0000","last_became_peered":"2024-04-25T01:26:01.415824+0000","last_unstale":"2024-04-25T01:41:22.864117+0000","last_undegraded":"2024-04-25T01:41:22.864117+0000","last_fullsized":"2024-04-25T01:41:22.864117+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-04-25T01:41:22.864032+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-04-25T01:41:22.864032+0000","last_clean_scrub_stamp":"2024-04-25T01:41:22.864032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:11.105583+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"26'4989","reported_seq":5021,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:38.206601+0000","last_change":"2024-04-25T01:41:38.206632+0000","last_active":"2024-04-25T01:41:38.206601+0000","last_peered":"2024-04-25T01:41:38.206601+0000","last_clean":"2024-04-25T01:41:38.206601+0000","last_became_active":"2024-04-25T01:26:02.415949+0000","last_became_peered":"2024-04-25T01:26:02.415949+0000","last_unstale":"2024-04-25T01:41:38.206601+0000","last_undegraded":"2024-04-25T01:41:38.206601+0000","last_fullsized":"2024-04-25T01:41:38.206601+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4989","last_scrub_stamp":"2024-04-25T01:41:38.206464+0000","last_deep_scrub":"26'4989","last_deep_scrub_stamp":"2024-04-25T01:41:38.206464+0000","last_clean_scrub_stamp":"2024-04-25T01:41:38.206464+0000","objects_scrubbed":0,"log_size":4989,"log_dups_size":0,"ondisk_log_size":4989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:06.963090+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25442,"num_write_kb":314904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"26'4796","reported_seq":4836,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:25.631136+0000","last_change":"2024-04-25T01:41:25.631164+0000","last_active":"2024-04-25T01:41:25.631136+0000","last_peered":"2024-04-25T01:41:25.631136+0000","last_clean":"2024-04-25T01:41:25.631136+0000","last_became_active":"2024-04-25T01:25:53.366693+0000","last_became_peered":"2024-04-25T01:25:53.366693+0000","last_unstale":"2024-04-25T01:41:25.631136+0000","last_undegraded":"2024-04-25T01:41:25.631136+0000","last_fullsized":"2024-04-25T01:41:25.631136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4796","last_scrub_stamp":"2024-04-25T01:41:25.631022+0000","last_deep_scrub":"26'4796","last_deep_scrub_stamp":"2024-04-25T01:41:25.631022+0000","last_clean_scrub_stamp":"2024-04-25T01:41:25.631022+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:48.647899+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25026,"num_write_kb":309186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"26'4","reported_seq":46,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:36.798781+0000","last_change":"2024-04-25T01:41:36.798806+0000","last_active":"2024-04-25T01:41:36.798781+0000","last_peered":"2024-04-25T01:41:36.798781+0000","last_clean":"2024-04-25T01:41:36.798781+0000","last_became_active":"2024-04-25T01:25:52.356396+0000","last_became_peered":"2024-04-25T01:25:52.356396+0000","last_unstale":"2024-04-25T01:41:36.798781+0000","last_undegraded":"2024-04-25T01:41:36.798781+0000","last_fullsized":"2024-04-25T01:41:36.798781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4","last_scrub_stamp":"2024-04-25T01:41:36.798679+0000","last_deep_scrub":"26'4","last_deep_scrub_stamp":"2024-04-25T01:41:36.798679+0000","last_clean_scrub_stamp":"2024-04-25T01:41:36.798679+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:12.551356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"26'1","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:21.695156+0000","last_change":"2024-04-25T01:41:21.695177+0000","last_active":"2024-04-25T01:41:21.695156+0000","last_peered":"2024-04-25T01:41:21.695156+0000","last_clean":"2024-04-25T01:41:21.695156+0000","last_became_active":"2024-04-25T01:26:01.407659+0000","last_became_peered":"2024-04-25T01:26:01.407659+0000","last_unstale":"2024-04-25T01:41:21.695156+0000","last_undegraded":"2024-04-25T01:41:21.695156+0000","last_fullsized":"2024-04-25T01:41:21.695156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-04-25T01:41:21.695080+0000","last_deep_scrub":"26'1","last_deep_scrub_stamp":"2024-04-25T01:41:21.695080+0000","last_clean_scrub_stamp":"2024-04-25T01:41:21.695080+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:31.680471+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"26'4196","reported_seq":4228,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:35.786102+0000","last_change":"2024-04-25T01:41:35.786131+0000","last_active":"2024-04-25T01:41:35.786102+0000","last_peered":"2024-04-25T01:41:35.786102+0000","last_clean":"2024-04-25T01:41:35.786102+0000","last_became_active":"2024-04-25T01:26:02.417622+0000","last_became_peered":"2024-04-25T01:26:02.417622+0000","last_unstale":"2024-04-25T01:41:35.786102+0000","last_undegraded":"2024-04-25T01:41:35.786102+0000","last_fullsized":"2024-04-25T01:41:35.786102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4196","last_scrub_stamp":"2024-04-25T01:41:35.785996+0000","last_deep_scrub":"26'4196","last_deep_scrub_stamp":"2024-04-25T01:41:35.785996+0000","last_clean_scrub_stamp":"2024-04-25T01:41:35.785996+0000","objects_scrubbed":0,"log_size":4196,"log_dups_size":0,"ondisk_log_size":4196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:18.897374+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22617,"num_write_kb":283190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"26'4472","reported_seq":4512,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:38.397686+0000","last_change":"2024-04-25T01:41:38.397730+0000","last_active":"2024-04-25T01:41:38.397686+0000","last_peered":"2024-04-25T01:41:38.397686+0000","last_clean":"2024-04-25T01:41:38.397686+0000","last_became_active":"2024-04-25T01:25:53.367854+0000","last_became_peered":"2024-04-25T01:25:53.367854+0000","last_unstale":"2024-04-25T01:41:38.397686+0000","last_undegraded":"2024-04-25T01:41:38.397686+0000","last_fullsized":"2024-04-25T01:41:38.397686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4472","last_scrub_stamp":"2024-04-25T01:41:38.397566+0000","last_deep_scrub":"26'4472","last_deep_scrub_stamp":"2024-04-25T01:41:38.397566+0000","last_clean_scrub_stamp":"2024-04-25T01:41:38.397566+0000","objects_scrubbed":0,"log_size":4472,"log_dups_size":0,"ondisk_log_size":4472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:23.538632+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23917,"num_write_kb":276200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'5","reported_seq":47,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:29.556568+0000","last_change":"2024-04-25T01:41:29.556616+0000","last_active":"2024-04-25T01:41:29.556568+0000","last_peered":"2024-04-25T01:41:29.556568+0000","last_clean":"2024-04-25T01:41:29.556568+0000","last_became_active":"2024-04-25T01:25:52.349874+0000","last_became_peered":"2024-04-25T01:25:52.349874+0000","last_unstale":"2024-04-25T01:41:29.556568+0000","last_undegraded":"2024-04-25T01:41:29.556568+0000","last_fullsized":"2024-04-25T01:41:29.556568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-04-25T01:41:29.556461+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-04-25T01:41:29.556461+0000","last_clean_scrub_stamp":"2024-04-25T01:41:29.556461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:18.108426+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"26'5","reported_seq":39,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:20.631227+0000","last_change":"2024-04-25T01:41:20.631248+0000","last_active":"2024-04-25T01:41:20.631227+0000","last_peered":"2024-04-25T01:41:20.631227+0000","last_clean":"2024-04-25T01:41:20.631227+0000","last_became_active":"2024-04-25T01:26:01.417820+0000","last_became_peered":"2024-04-25T01:26:01.417820+0000","last_unstale":"2024-04-25T01:41:20.631227+0000","last_undegraded":"2024-04-25T01:41:20.631227+0000","last_fullsized":"2024-04-25T01:41:20.631227+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-04-25T01:41:20.631162+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-04-25T01:41:20.631162+0000","last_clean_scrub_stamp":"2024-04-25T01:41:20.631162+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:56.688056+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"26'4093","reported_seq":4125,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:38.707648+0000","last_change":"2024-04-25T01:41:38.707677+0000","last_active":"2024-04-25T01:41:38.707648+0000","last_peered":"2024-04-25T01:41:38.707648+0000","last_clean":"2024-04-25T01:41:38.707648+0000","last_became_active":"2024-04-25T01:26:02.422503+0000","last_became_peered":"2024-04-25T01:26:02.422503+0000","last_unstale":"2024-04-25T01:41:38.707648+0000","last_undegraded":"2024-04-25T01:41:38.707648+0000","last_fullsized":"2024-04-25T01:41:38.707648+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4093","last_scrub_stamp":"2024-04-25T01:41:38.707532+0000","last_deep_scrub":"26'4093","last_deep_scrub_stamp":"2024-04-25T01:41:38.707532+0000","last_clean_scrub_stamp":"2024-04-25T01:41:38.707532+0000","objects_scrubbed":0,"log_size":4093,"log_dups_size":0,"ondisk_log_size":4093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:51.479200+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21550,"num_write_kb":266696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"26'4812","reported_seq":4852,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:30.228308+0000","last_change":"2024-04-25T01:41:30.228334+0000","last_active":"2024-04-25T01:41:30.228308+0000","last_peered":"2024-04-25T01:41:30.228308+0000","last_clean":"2024-04-25T01:41:30.228308+0000","last_became_active":"2024-04-25T01:25:53.361156+0000","last_became_peered":"2024-04-25T01:25:53.361156+0000","last_unstale":"2024-04-25T01:41:30.228308+0000","last_undegraded":"2024-04-25T01:41:30.228308+0000","last_fullsized":"2024-04-25T01:41:30.228308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4812","last_scrub_stamp":"2024-04-25T01:41:30.228208+0000","last_deep_scrub":"26'4812","last_deep_scrub_stamp":"2024-04-25T01:41:30.228208+0000","last_clean_scrub_stamp":"2024-04-25T01:41:30.228208+0000","objects_scrubbed":0,"log_size":4812,"log_dups_size":0,"ondisk_log_size":4812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:59.815199+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25777,"num_write_kb":295962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"26'5","reported_seq":47,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:33.375115+0000","last_change":"2024-04-25T01:41:33.375143+0000","last_active":"2024-04-25T01:41:33.375115+0000","last_peered":"2024-04-25T01:41:33.375115+0000","last_clean":"2024-04-25T01:41:33.375115+0000","last_became_active":"2024-04-25T01:25:52.354193+0000","last_became_peered":"2024-04-25T01:25:52.354193+0000","last_unstale":"2024-04-25T01:41:33.375115+0000","last_undegraded":"2024-04-25T01:41:33.375115+0000","last_fullsized":"2024-04-25T01:41:33.375115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-04-25T01:41:33.375020+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-04-25T01:41:33.375020+0000","last_clean_scrub_stamp":"2024-04-25T01:41:33.375020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:51.873626+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"26'5","reported_seq":39,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:19.652416+0000","last_change":"2024-04-25T01:41:19.652438+0000","last_active":"2024-04-25T01:41:19.652416+0000","last_peered":"2024-04-25T01:41:19.652416+0000","last_clean":"2024-04-25T01:41:19.652416+0000","last_became_active":"2024-04-25T01:26:01.417805+0000","last_became_peered":"2024-04-25T01:26:01.417805+0000","last_unstale":"2024-04-25T01:41:19.652416+0000","last_undegraded":"2024-04-25T01:41:19.652416+0000","last_fullsized":"2024-04-25T01:41:19.652416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-04-25T01:41:19.652334+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-04-25T01:41:19.652334+0000","last_clean_scrub_stamp":"2024-04-25T01:41:19.652334+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:54.494631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"26'3951","reported_seq":3983,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:31.180998+0000","last_change":"2024-04-25T01:41:31.181024+0000","last_active":"2024-04-25T01:41:31.180998+0000","last_peered":"2024-04-25T01:41:31.180998+0000","last_clean":"2024-04-25T01:41:31.180998+0000","last_became_active":"2024-04-25T01:26:02.415619+0000","last_became_peered":"2024-04-25T01:26:02.415619+0000","last_unstale":"2024-04-25T01:41:31.180998+0000","last_undegraded":"2024-04-25T01:41:31.180998+0000","last_fullsized":"2024-04-25T01:41:31.180998+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3951","last_scrub_stamp":"2024-04-25T01:41:31.180895+0000","last_deep_scrub":"26'3951","last_deep_scrub_stamp":"2024-04-25T01:41:31.180895+0000","last_clean_scrub_stamp":"2024-04-25T01:41:31.180895+0000","objects_scrubbed":0,"log_size":3951,"log_dups_size":0,"ondisk_log_size":3951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:39.576959+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20033,"num_write_kb":252185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"26'4985","reported_seq":5026,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:34.587670+0000","last_change":"2024-04-25T01:41:34.587670+0000","last_active":"2024-04-25T01:41:34.587670+0000","last_peered":"2024-04-25T01:41:34.587670+0000","last_clean":"2024-04-25T01:41:34.587670+0000","last_became_active":"2024-04-25T01:25:53.362533+0000","last_became_peered":"2024-04-25T01:25:53.362533+0000","last_unstale":"2024-04-25T01:41:34.587670+0000","last_undegraded":"2024-04-25T01:41:34.587670+0000","last_fullsized":"2024-04-25T01:41:34.587670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4985","last_scrub_stamp":"2024-04-25T01:41:34.587455+0000","last_deep_scrub":"26'4985","last_deep_scrub_stamp":"2024-04-25T01:41:34.587455+0000","last_clean_scrub_stamp":"2024-04-25T01:41:34.587455+0000","objects_scrubbed":0,"log_size":4985,"log_dups_size":0,"ondisk_log_size":4985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:31.640145+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26332,"num_write_kb":298165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"26'44","reported_seq":127,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:21.990855+0000","last_change":"2024-04-25T01:41:21.990904+0000","last_active":"2024-04-25T01:41:21.990855+0000","last_peered":"2024-04-25T01:41:21.990855+0000","last_clean":"2024-04-25T01:41:21.990855+0000","last_became_active":"2024-04-25T01:25:52.345061+0000","last_became_peered":"2024-04-25T01:25:52.345061+0000","last_unstale":"2024-04-25T01:41:21.990855+0000","last_undegraded":"2024-04-25T01:41:21.990855+0000","last_fullsized":"2024-04-25T01:41:21.990855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'44","last_scrub_stamp":"2024-04-25T01:41:21.990771+0000","last_deep_scrub":"26'44","last_deep_scrub_stamp":"2024-04-25T01:41:21.990771+0000","last_clean_scrub_stamp":"2024-04-25T01:41:21.990771+0000","objects_scrubbed":1,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:03.336231+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":41,"num_read_kb":115,"num_write":41,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"26'5","reported_seq":39,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:22.637851+0000","last_change":"2024-04-25T01:41:22.637901+0000","last_active":"2024-04-25T01:41:22.637851+0000","last_peered":"2024-04-25T01:41:22.637851+0000","last_clean":"2024-04-25T01:41:22.637851+0000","last_became_active":"2024-04-25T01:26:01.418173+0000","last_became_peered":"2024-04-25T01:26:01.418173+0000","last_unstale":"2024-04-25T01:41:22.637851+0000","last_undegraded":"2024-04-25T01:41:22.637851+0000","last_fullsized":"2024-04-25T01:41:22.637851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-04-25T01:41:22.637755+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-04-25T01:41:22.637755+0000","last_clean_scrub_stamp":"2024-04-25T01:41:22.637755+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:31.033818+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"26'4134","reported_seq":4166,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:40.317684+0000","last_change":"2024-04-25T01:41:40.317712+0000","last_active":"2024-04-25T01:41:40.317684+0000","last_peered":"2024-04-25T01:41:40.317684+0000","last_clean":"2024-04-25T01:41:40.317684+0000","last_became_active":"2024-04-25T01:26:02.417355+0000","last_became_peered":"2024-04-25T01:26:02.417355+0000","last_unstale":"2024-04-25T01:41:40.317684+0000","last_undegraded":"2024-04-25T01:41:40.317684+0000","last_fullsized":"2024-04-25T01:41:40.317684+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4134","last_scrub_stamp":"2024-04-25T01:41:40.317552+0000","last_deep_scrub":"26'4134","last_deep_scrub_stamp":"2024-04-25T01:41:40.317552+0000","last_clean_scrub_stamp":"2024-04-25T01:41:40.317552+0000","objects_scrubbed":0,"log_size":4134,"log_dups_size":0,"ondisk_log_size":4134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:34.381900+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21715,"num_write_kb":287743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"26'4344","reported_seq":4384,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:25.701271+0000","last_change":"2024-04-25T01:41:25.701300+0000","last_active":"2024-04-25T01:41:25.701271+0000","last_peered":"2024-04-25T01:41:25.701271+0000","last_clean":"2024-04-25T01:41:25.701271+0000","last_became_active":"2024-04-25T01:25:53.355648+0000","last_became_peered":"2024-04-25T01:25:53.355648+0000","last_unstale":"2024-04-25T01:41:25.701271+0000","last_undegraded":"2024-04-25T01:41:25.701271+0000","last_fullsized":"2024-04-25T01:41:25.701271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4344","last_scrub_stamp":"2024-04-25T01:41:25.701164+0000","last_deep_scrub":"26'4344","last_deep_scrub_stamp":"2024-04-25T01:41:25.701164+0000","last_clean_scrub_stamp":"2024-04-25T01:41:25.701164+0000","objects_scrubbed":0,"log_size":4344,"log_dups_size":0,"ondisk_log_size":4344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:01.758058+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23132,"num_write_kb":289328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"26'5","reported_seq":48,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:38.723753+0000","last_change":"2024-04-25T01:41:38.723753+0000","last_active":"2024-04-25T01:41:38.723753+0000","last_peered":"2024-04-25T01:41:38.723753+0000","last_clean":"2024-04-25T01:41:38.723753+0000","last_became_active":"2024-04-25T01:25:52.356924+0000","last_became_peered":"2024-04-25T01:25:52.356924+0000","last_unstale":"2024-04-25T01:41:38.723753+0000","last_undegraded":"2024-04-25T01:41:38.723753+0000","last_fullsized":"2024-04-25T01:41:38.723753+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-04-25T01:41:38.723630+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-04-25T01:41:38.723630+0000","last_clean_scrub_stamp":"2024-04-25T01:41:38.723630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:35.623165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"26'571","reported_seq":605,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:34.739765+0000","last_change":"2024-04-25T01:41:34.739794+0000","last_active":"2024-04-25T01:41:34.739765+0000","last_peered":"2024-04-25T01:41:34.739765+0000","last_clean":"2024-04-25T01:41:34.739765+0000","last_became_active":"2024-04-25T01:26:01.420831+0000","last_became_peered":"2024-04-25T01:26:01.420831+0000","last_unstale":"2024-04-25T01:41:34.739765+0000","last_undegraded":"2024-04-25T01:41:34.739765+0000","last_fullsized":"2024-04-25T01:41:34.739765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'571","last_scrub_stamp":"2024-04-25T01:41:34.739680+0000","last_deep_scrub":"26'571","last_deep_scrub_stamp":"2024-04-25T01:41:34.739680+0000","last_clean_scrub_stamp":"2024-04-25T01:41:34.739680+0000","objects_scrubbed":1,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:23.291645+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1330,"num_write_kb":427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"26'4947","reported_seq":4979,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:43.765237+0000","last_change":"2024-04-25T01:41:43.765283+0000","last_active":"2024-04-25T01:41:43.765237+0000","last_peered":"2024-04-25T01:41:43.765237+0000","last_clean":"2024-04-25T01:41:43.765237+0000","last_became_active":"2024-04-25T01:26:02.421079+0000","last_became_peered":"2024-04-25T01:26:02.421079+0000","last_unstale":"2024-04-25T01:41:43.765237+0000","last_undegraded":"2024-04-25T01:41:43.765237+0000","last_fullsized":"2024-04-25T01:41:43.765237+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4947","last_scrub_stamp":"2024-04-25T01:41:43.765089+0000","last_deep_scrub":"26'4947","last_deep_scrub_stamp":"2024-04-25T01:41:43.765089+0000","last_clean_scrub_stamp":"2024-04-25T01:41:43.765089+0000","objects_scrubbed":0,"log_size":4947,"log_dups_size":0,"ondisk_log_size":4947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:19.894265+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25964,"num_write_kb":313022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"26'4754","reported_seq":4780,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:37.054368+0000","last_change":"2024-04-25T01:41:37.054393+0000","last_active":"2024-04-25T01:41:37.054368+0000","last_peered":"2024-04-25T01:41:37.054368+0000","last_clean":"2024-04-25T01:41:37.054368+0000","last_became_active":"2024-04-25T01:26:52.515254+0000","last_became_peered":"2024-04-25T01:26:52.515254+0000","last_unstale":"2024-04-25T01:41:37.054368+0000","last_undegraded":"2024-04-25T01:41:37.054368+0000","last_fullsized":"2024-04-25T01:41:37.054368+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4754","last_scrub_stamp":"2024-04-25T01:41:37.054234+0000","last_deep_scrub":"26'4754","last_deep_scrub_stamp":"2024-04-25T01:41:37.054234+0000","last_clean_scrub_stamp":"2024-04-25T01:41:37.054234+0000","objects_scrubbed":0,"log_size":4754,"log_dups_size":0,"ondisk_log_size":4754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:36.238854+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24235,"num_write_kb":310192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:32.430292+0000","last_change":"2024-04-25T01:41:32.430316+0000","last_active":"2024-04-25T01:41:32.430292+0000","last_peered":"2024-04-25T01:41:32.430292+0000","last_clean":"2024-04-25T01:41:32.430292+0000","last_became_active":"2024-04-25T01:25:52.352826+0000","last_became_peered":"2024-04-25T01:25:52.352826+0000","last_unstale":"2024-04-25T01:41:32.430292+0000","last_undegraded":"2024-04-25T01:41:32.430292+0000","last_fullsized":"2024-04-25T01:41:32.430292+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-04-25T01:41:32.430197+0000","last_deep_scrub":"26'1","last_deep_scrub_stamp":"2024-04-25T01:41:32.430197+0000","last_clean_scrub_stamp":"2024-04-25T01:41:32.430197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:45.201865+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"26'119","reported_seq":201,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:34.779955+0000","last_change":"2024-04-25T01:41:34.779955+0000","last_active":"2024-04-25T01:41:34.779955+0000","last_peered":"2024-04-25T01:41:34.779955+0000","last_clean":"2024-04-25T01:41:34.779955+0000","last_became_active":"2024-04-25T01:25:41.718721+0000","last_became_peered":"2024-04-25T01:25:41.718721+0000","last_unstale":"2024-04-25T01:41:34.779955+0000","last_undegraded":"2024-04-25T01:41:34.779955+0000","last_fullsized":"2024-04-25T01:41:34.779955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'119","last_scrub_stamp":"2024-04-25T01:41:34.779819+0000","last_deep_scrub":"26'119","last_deep_scrub_stamp":"2024-04-25T01:41:34.779819+0000","last_clean_scrub_stamp":"2024-04-25T01:41:34.779819+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:31.832509+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"26'4","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:33.319900+0000","last_change":"2024-04-25T01:41:33.319928+0000","last_active":"2024-04-25T01:41:33.319900+0000","last_peered":"2024-04-25T01:41:33.319900+0000","last_clean":"2024-04-25T01:41:33.319900+0000","last_became_active":"2024-04-25T01:26:01.409062+0000","last_became_peered":"2024-04-25T01:26:01.409062+0000","last_unstale":"2024-04-25T01:41:33.319900+0000","last_undegraded":"2024-04-25T01:41:33.319900+0000","last_fullsized":"2024-04-25T01:41:33.319900+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4","last_scrub_stamp":"2024-04-25T01:41:33.319785+0000","last_deep_scrub":"26'4","last_deep_scrub_stamp":"2024-04-25T01:41:33.319785+0000","last_clean_scrub_stamp":"2024-04-25T01:41:33.319785+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:14.789273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"26'3833","reported_seq":3865,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:32.125258+0000","last_change":"2024-04-25T01:41:32.125258+0000","last_active":"2024-04-25T01:41:32.125258+0000","last_peered":"2024-04-25T01:41:32.125258+0000","last_clean":"2024-04-25T01:41:32.125258+0000","last_became_active":"2024-04-25T01:26:02.416546+0000","last_became_peered":"2024-04-25T01:26:02.416546+0000","last_unstale":"2024-04-25T01:41:32.125258+0000","last_undegraded":"2024-04-25T01:41:32.125258+0000","last_fullsized":"2024-04-25T01:41:32.125258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3833","last_scrub_stamp":"2024-04-25T01:41:32.125125+0000","last_deep_scrub":"26'3833","last_deep_scrub_stamp":"2024-04-25T01:41:32.125125+0000","last_clean_scrub_stamp":"2024-04-25T01:41:32.125125+0000","objects_scrubbed":0,"log_size":3833,"log_dups_size":0,"ondisk_log_size":3833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:38.867976+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19504,"num_write_kb":250193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"26'5320","reported_seq":5360,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:24.985853+0000","last_change":"2024-04-25T01:41:24.985853+0000","last_active":"2024-04-25T01:41:24.985853+0000","last_peered":"2024-04-25T01:41:24.985853+0000","last_clean":"2024-04-25T01:41:24.985853+0000","last_became_active":"2024-04-25T01:25:53.375219+0000","last_became_peered":"2024-04-25T01:25:53.375219+0000","last_unstale":"2024-04-25T01:41:24.985853+0000","last_undegraded":"2024-04-25T01:41:24.985853+0000","last_fullsized":"2024-04-25T01:41:24.985853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5320","last_scrub_stamp":"2024-04-25T01:41:24.985691+0000","last_deep_scrub":"26'5320","last_deep_scrub_stamp":"2024-04-25T01:41:24.985691+0000","last_clean_scrub_stamp":"2024-04-25T01:41:24.985691+0000","objects_scrubbed":0,"log_size":5320,"log_dups_size":0,"ondisk_log_size":5320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:38.717746+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27324,"num_write_kb":314375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'3","reported_seq":45,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:25.069689+0000","last_change":"2024-04-25T01:41:25.069689+0000","last_active":"2024-04-25T01:41:25.069689+0000","last_peered":"2024-04-25T01:41:25.069689+0000","last_clean":"2024-04-25T01:41:25.069689+0000","last_became_active":"2024-04-25T01:25:52.352644+0000","last_became_peered":"2024-04-25T01:25:52.352644+0000","last_unstale":"2024-04-25T01:41:25.069689+0000","last_undegraded":"2024-04-25T01:41:25.069689+0000","last_fullsized":"2024-04-25T01:41:25.069689+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:25.069565+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:25.069565+0000","last_clean_scrub_stamp":"2024-04-25T01:41:25.069565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:35.054956+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"26'3","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:35.231839+0000","last_change":"2024-04-25T01:41:35.231875+0000","last_active":"2024-04-25T01:41:35.231839+0000","last_peered":"2024-04-25T01:41:35.231839+0000","last_clean":"2024-04-25T01:41:35.231839+0000","last_became_active":"2024-04-25T01:26:01.420664+0000","last_became_peered":"2024-04-25T01:26:01.420664+0000","last_unstale":"2024-04-25T01:41:35.231839+0000","last_undegraded":"2024-04-25T01:41:35.231839+0000","last_fullsized":"2024-04-25T01:41:35.231839+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:35.231754+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:35.231754+0000","last_clean_scrub_stamp":"2024-04-25T01:41:35.231754+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:55.401551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"26'4000","reported_seq":4032,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:29.094487+0000","last_change":"2024-04-25T01:41:29.094513+0000","last_active":"2024-04-25T01:41:29.094487+0000","last_peered":"2024-04-25T01:41:29.094487+0000","last_clean":"2024-04-25T01:41:29.094487+0000","last_became_active":"2024-04-25T01:26:02.403206+0000","last_became_peered":"2024-04-25T01:26:02.403206+0000","last_unstale":"2024-04-25T01:41:29.094487+0000","last_undegraded":"2024-04-25T01:41:29.094487+0000","last_fullsized":"2024-04-25T01:41:29.094487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4000","last_scrub_stamp":"2024-04-25T01:41:29.094381+0000","last_deep_scrub":"26'4000","last_deep_scrub_stamp":"2024-04-25T01:41:29.094381+0000","last_clean_scrub_stamp":"2024-04-25T01:41:29.094381+0000","objects_scrubbed":0,"log_size":4000,"log_dups_size":0,"ondisk_log_size":4000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:39.561760+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21231,"num_write_kb":269526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"26'4796","reported_seq":4836,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:27.383303+0000","last_change":"2024-04-25T01:41:27.383303+0000","last_active":"2024-04-25T01:41:27.383303+0000","last_peered":"2024-04-25T01:41:27.383303+0000","last_clean":"2024-04-25T01:41:27.383303+0000","last_became_active":"2024-04-25T01:25:53.370539+0000","last_became_peered":"2024-04-25T01:25:53.370539+0000","last_unstale":"2024-04-25T01:41:27.383303+0000","last_undegraded":"2024-04-25T01:41:27.383303+0000","last_fullsized":"2024-04-25T01:41:27.383303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4796","last_scrub_stamp":"2024-04-25T01:41:27.383169+0000","last_deep_scrub":"26'4796","last_deep_scrub_stamp":"2024-04-25T01:41:27.383169+0000","last_clean_scrub_stamp":"2024-04-25T01:41:27.383169+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:26.567790+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25182,"num_write_kb":313860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"26'3","reported_seq":45,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:31.580818+0000","last_change":"2024-04-25T01:41:31.580818+0000","last_active":"2024-04-25T01:41:31.580818+0000","last_peered":"2024-04-25T01:41:31.580818+0000","last_clean":"2024-04-25T01:41:31.580818+0000","last_became_active":"2024-04-25T01:25:52.354836+0000","last_became_peered":"2024-04-25T01:25:52.354836+0000","last_unstale":"2024-04-25T01:41:31.580818+0000","last_undegraded":"2024-04-25T01:41:31.580818+0000","last_fullsized":"2024-04-25T01:41:31.580818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:31.580652+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:31.580652+0000","last_clean_scrub_stamp":"2024-04-25T01:41:31.580652+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:00.337278+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"26'1114","reported_seq":1148,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:33.738219+0000","last_change":"2024-04-25T01:41:33.738219+0000","last_active":"2024-04-25T01:41:33.738219+0000","last_peered":"2024-04-25T01:41:33.738219+0000","last_clean":"2024-04-25T01:41:33.738219+0000","last_became_active":"2024-04-25T01:26:01.418987+0000","last_became_peered":"2024-04-25T01:26:01.418987+0000","last_unstale":"2024-04-25T01:41:33.738219+0000","last_undegraded":"2024-04-25T01:41:33.738219+0000","last_fullsized":"2024-04-25T01:41:33.738219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1114","last_scrub_stamp":"2024-04-25T01:41:33.738095+0000","last_deep_scrub":"26'1114","last_deep_scrub_stamp":"2024-04-25T01:41:33.738095+0000","last_clean_scrub_stamp":"2024-04-25T01:41:33.738095+0000","objects_scrubbed":2,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:32.922716+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1103,"num_write_kb":4430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"26'4082","reported_seq":4114,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:26.907485+0000","last_change":"2024-04-25T01:41:26.907511+0000","last_active":"2024-04-25T01:41:26.907485+0000","last_peered":"2024-04-25T01:41:26.907485+0000","last_clean":"2024-04-25T01:41:26.907485+0000","last_became_active":"2024-04-25T01:26:02.416189+0000","last_became_peered":"2024-04-25T01:26:02.416189+0000","last_unstale":"2024-04-25T01:41:26.907485+0000","last_undegraded":"2024-04-25T01:41:26.907485+0000","last_fullsized":"2024-04-25T01:41:26.907485+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4082","last_scrub_stamp":"2024-04-25T01:41:26.907376+0000","last_deep_scrub":"26'4082","last_deep_scrub_stamp":"2024-04-25T01:41:26.907376+0000","last_clean_scrub_stamp":"2024-04-25T01:41:26.907376+0000","objects_scrubbed":0,"log_size":4082,"log_dups_size":0,"ondisk_log_size":4082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:08.376864+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20562,"num_write_kb":274915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"26'5666","reported_seq":5706,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:22.042392+0000","last_change":"2024-04-25T01:41:22.042392+0000","last_active":"2024-04-25T01:41:22.042392+0000","last_peered":"2024-04-25T01:41:22.042392+0000","last_clean":"2024-04-25T01:41:22.042392+0000","last_became_active":"2024-04-25T01:25:53.370074+0000","last_became_peered":"2024-04-25T01:25:53.370074+0000","last_unstale":"2024-04-25T01:41:22.042392+0000","last_undegraded":"2024-04-25T01:41:22.042392+0000","last_fullsized":"2024-04-25T01:41:22.042392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5666","last_scrub_stamp":"2024-04-25T01:41:22.042219+0000","last_deep_scrub":"26'5666","last_deep_scrub_stamp":"2024-04-25T01:41:22.042219+0000","last_clean_scrub_stamp":"2024-04-25T01:41:22.042219+0000","objects_scrubbed":0,"log_size":5666,"log_dups_size":0,"ondisk_log_size":5666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:02.992114+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29161,"num_write_kb":354533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"26'7","reported_seq":49,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:23.022583+0000","last_change":"2024-04-25T01:41:23.022583+0000","last_active":"2024-04-25T01:41:23.022583+0000","last_peered":"2024-04-25T01:41:23.022583+0000","last_clean":"2024-04-25T01:41:23.022583+0000","last_became_active":"2024-04-25T01:25:52.345717+0000","last_became_peered":"2024-04-25T01:25:52.345717+0000","last_unstale":"2024-04-25T01:41:23.022583+0000","last_undegraded":"2024-04-25T01:41:23.022583+0000","last_fullsized":"2024-04-25T01:41:23.022583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'7","last_scrub_stamp":"2024-04-25T01:41:23.022457+0000","last_deep_scrub":"26'7","last_deep_scrub_stamp":"2024-04-25T01:41:23.022457+0000","last_clean_scrub_stamp":"2024-04-25T01:41:23.022457+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:56.658486+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"26'3","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:26.369244+0000","last_change":"2024-04-25T01:41:26.369270+0000","last_active":"2024-04-25T01:41:26.369244+0000","last_peered":"2024-04-25T01:41:26.369244+0000","last_clean":"2024-04-25T01:41:26.369244+0000","last_became_active":"2024-04-25T01:26:01.415750+0000","last_became_peered":"2024-04-25T01:26:01.415750+0000","last_unstale":"2024-04-25T01:41:26.369244+0000","last_undegraded":"2024-04-25T01:41:26.369244+0000","last_fullsized":"2024-04-25T01:41:26.369244+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:26.369152+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:26.369152+0000","last_clean_scrub_stamp":"2024-04-25T01:41:26.369152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:40.048159+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"26'4748","reported_seq":4780,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:20.027499+0000","last_change":"2024-04-25T01:41:20.027538+0000","last_active":"2024-04-25T01:41:20.027499+0000","last_peered":"2024-04-25T01:41:20.027499+0000","last_clean":"2024-04-25T01:41:20.027499+0000","last_became_active":"2024-04-25T01:26:02.414316+0000","last_became_peered":"2024-04-25T01:26:02.414316+0000","last_unstale":"2024-04-25T01:41:20.027499+0000","last_undegraded":"2024-04-25T01:41:20.027499+0000","last_fullsized":"2024-04-25T01:41:20.027499+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4748","last_scrub_stamp":"2024-04-25T01:41:20.027380+0000","last_deep_scrub":"26'4748","last_deep_scrub_stamp":"2024-04-25T01:41:20.027380+0000","last_clean_scrub_stamp":"2024-04-25T01:41:20.027380+0000","objects_scrubbed":0,"log_size":4748,"log_dups_size":0,"ondisk_log_size":4748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:32.660618+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24613,"num_write_kb":303491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"26'4546","reported_seq":4586,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:33.826897+0000","last_change":"2024-04-25T01:41:33.826897+0000","last_active":"2024-04-25T01:41:33.826897+0000","last_peered":"2024-04-25T01:41:33.826897+0000","last_clean":"2024-04-25T01:41:33.826897+0000","last_became_active":"2024-04-25T01:25:53.364049+0000","last_became_peered":"2024-04-25T01:25:53.364049+0000","last_unstale":"2024-04-25T01:41:33.826897+0000","last_undegraded":"2024-04-25T01:41:33.826897+0000","last_fullsized":"2024-04-25T01:41:33.826897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4546","last_scrub_stamp":"2024-04-25T01:41:33.826752+0000","last_deep_scrub":"26'4546","last_deep_scrub_stamp":"2024-04-25T01:41:33.826752+0000","last_clean_scrub_stamp":"2024-04-25T01:41:33.826752+0000","objects_scrubbed":0,"log_size":4546,"log_dups_size":0,"ondisk_log_size":4546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:46.598420+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22740,"num_write_kb":287925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'6","reported_seq":48,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:28.162920+0000","last_change":"2024-04-25T01:41:28.162920+0000","last_active":"2024-04-25T01:41:28.162920+0000","last_peered":"2024-04-25T01:41:28.162920+0000","last_clean":"2024-04-25T01:41:28.162920+0000","last_became_active":"2024-04-25T01:25:52.350357+0000","last_became_peered":"2024-04-25T01:25:52.350357+0000","last_unstale":"2024-04-25T01:41:28.162920+0000","last_undegraded":"2024-04-25T01:41:28.162920+0000","last_fullsized":"2024-04-25T01:41:28.162920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'6","last_scrub_stamp":"2024-04-25T01:41:28.162801+0000","last_deep_scrub":"26'6","last_deep_scrub_stamp":"2024-04-25T01:41:28.162801+0000","last_clean_scrub_stamp":"2024-04-25T01:41:28.162801+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:02.453765+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"26'3","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:26.080366+0000","last_change":"2024-04-25T01:41:26.080395+0000","last_active":"2024-04-25T01:41:26.080366+0000","last_peered":"2024-04-25T01:41:26.080366+0000","last_clean":"2024-04-25T01:41:26.080366+0000","last_became_active":"2024-04-25T01:26:01.418179+0000","last_became_peered":"2024-04-25T01:26:01.418179+0000","last_unstale":"2024-04-25T01:41:26.080366+0000","last_undegraded":"2024-04-25T01:41:26.080366+0000","last_fullsized":"2024-04-25T01:41:26.080366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:26.080274+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:26.080274+0000","last_clean_scrub_stamp":"2024-04-25T01:41:26.080274+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:36.463932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"26'3833","reported_seq":3864,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:46.958248+0000","last_change":"2024-04-25T01:41:46.958292+0000","last_active":"2024-04-25T01:41:46.958248+0000","last_peered":"2024-04-25T01:41:46.958248+0000","last_clean":"2024-04-25T01:41:46.958248+0000","last_became_active":"2024-04-25T01:26:02.418090+0000","last_became_peered":"2024-04-25T01:26:02.418090+0000","last_unstale":"2024-04-25T01:41:46.958248+0000","last_undegraded":"2024-04-25T01:41:46.958248+0000","last_fullsized":"2024-04-25T01:41:46.958248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3833","last_scrub_stamp":"2024-04-25T01:41:46.958134+0000","last_deep_scrub":"26'3833","last_deep_scrub_stamp":"2024-04-25T01:41:46.958134+0000","last_clean_scrub_stamp":"2024-04-25T01:41:46.958134+0000","objects_scrubbed":0,"log_size":3833,"log_dups_size":0,"ondisk_log_size":3833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:06.859333+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19507,"num_write_kb":254331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"26'4627","reported_seq":4666,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:31.860712+0000","last_change":"2024-04-25T01:41:31.860739+0000","last_active":"2024-04-25T01:41:31.860712+0000","last_peered":"2024-04-25T01:41:31.860712+0000","last_clean":"2024-04-25T01:41:31.860712+0000","last_became_active":"2024-04-25T01:25:53.365715+0000","last_became_peered":"2024-04-25T01:25:53.365715+0000","last_unstale":"2024-04-25T01:41:31.860712+0000","last_undegraded":"2024-04-25T01:41:31.860712+0000","last_fullsized":"2024-04-25T01:41:31.860712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4627","last_scrub_stamp":"2024-04-25T01:41:31.860598+0000","last_deep_scrub":"26'4627","last_deep_scrub_stamp":"2024-04-25T01:41:31.860598+0000","last_clean_scrub_stamp":"2024-04-25T01:41:31.860598+0000","objects_scrubbed":0,"log_size":4627,"log_dups_size":0,"ondisk_log_size":4627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:00.617224+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24780,"num_write_kb":300200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"26'15","reported_seq":56,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:24.073916+0000","last_change":"2024-04-25T01:41:24.073978+0000","last_active":"2024-04-25T01:41:24.073916+0000","last_peered":"2024-04-25T01:41:24.073916+0000","last_clean":"2024-04-25T01:41:24.073916+0000","last_became_active":"2024-04-25T01:25:52.347144+0000","last_became_peered":"2024-04-25T01:25:52.347144+0000","last_unstale":"2024-04-25T01:41:24.073916+0000","last_undegraded":"2024-04-25T01:41:24.073916+0000","last_fullsized":"2024-04-25T01:41:24.073916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'15","last_scrub_stamp":"2024-04-25T01:41:24.073796+0000","last_deep_scrub":"26'15","last_deep_scrub_stamp":"2024-04-25T01:41:24.073796+0000","last_clean_scrub_stamp":"2024-04-25T01:41:24.073796+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:43.520796+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"26'2","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:31.735406+0000","last_change":"2024-04-25T01:41:31.735434+0000","last_active":"2024-04-25T01:41:31.735406+0000","last_peered":"2024-04-25T01:41:31.735406+0000","last_clean":"2024-04-25T01:41:31.735406+0000","last_became_active":"2024-04-25T01:26:01.409654+0000","last_became_peered":"2024-04-25T01:26:01.409654+0000","last_unstale":"2024-04-25T01:41:31.735406+0000","last_undegraded":"2024-04-25T01:41:31.735406+0000","last_fullsized":"2024-04-25T01:41:31.735406+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'2","last_scrub_stamp":"2024-04-25T01:41:31.735321+0000","last_deep_scrub":"26'2","last_deep_scrub_stamp":"2024-04-25T01:41:31.735321+0000","last_clean_scrub_stamp":"2024-04-25T01:41:31.735321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:13.204808+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"26'4668","reported_seq":4699,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:46.124494+0000","last_change":"2024-04-25T01:41:46.124552+0000","last_active":"2024-04-25T01:41:46.124494+0000","last_peered":"2024-04-25T01:41:46.124494+0000","last_clean":"2024-04-25T01:41:46.124494+0000","last_became_active":"2024-04-25T01:26:02.424819+0000","last_became_peered":"2024-04-25T01:26:02.424819+0000","last_unstale":"2024-04-25T01:41:46.124494+0000","last_undegraded":"2024-04-25T01:41:46.124494+0000","last_fullsized":"2024-04-25T01:41:46.124494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4668","last_scrub_stamp":"2024-04-25T01:41:46.124376+0000","last_deep_scrub":"26'4668","last_deep_scrub_stamp":"2024-04-25T01:41:46.124376+0000","last_clean_scrub_stamp":"2024-04-25T01:41:46.124376+0000","objects_scrubbed":0,"log_size":4668,"log_dups_size":0,"ondisk_log_size":4668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:55.640183+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23987,"num_write_kb":301090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"26'5240","reported_seq":5279,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:30.323624+0000","last_change":"2024-04-25T01:41:30.323670+0000","last_active":"2024-04-25T01:41:30.323624+0000","last_peered":"2024-04-25T01:41:30.323624+0000","last_clean":"2024-04-25T01:41:30.323624+0000","last_became_active":"2024-04-25T01:25:53.364431+0000","last_became_peered":"2024-04-25T01:25:53.364431+0000","last_unstale":"2024-04-25T01:41:30.323624+0000","last_undegraded":"2024-04-25T01:41:30.323624+0000","last_fullsized":"2024-04-25T01:41:30.323624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5240","last_scrub_stamp":"2024-04-25T01:41:30.323520+0000","last_deep_scrub":"26'5240","last_deep_scrub_stamp":"2024-04-25T01:41:30.323520+0000","last_clean_scrub_stamp":"2024-04-25T01:41:30.323520+0000","objects_scrubbed":0,"log_size":5240,"log_dups_size":0,"ondisk_log_size":5240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:40.707179+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27403,"num_write_kb":326994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'6","reported_seq":47,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:27.233476+0000","last_change":"2024-04-25T01:41:27.233503+0000","last_active":"2024-04-25T01:41:27.233476+0000","last_peered":"2024-04-25T01:41:27.233476+0000","last_clean":"2024-04-25T01:41:27.233476+0000","last_became_active":"2024-04-25T01:25:52.350946+0000","last_became_peered":"2024-04-25T01:25:52.350946+0000","last_unstale":"2024-04-25T01:41:27.233476+0000","last_undegraded":"2024-04-25T01:41:27.233476+0000","last_fullsized":"2024-04-25T01:41:27.233476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'6","last_scrub_stamp":"2024-04-25T01:41:27.233383+0000","last_deep_scrub":"26'6","last_deep_scrub_stamp":"2024-04-25T01:41:27.233383+0000","last_clean_scrub_stamp":"2024-04-25T01:41:27.233383+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:03.290277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"26'2","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:20.792201+0000","last_change":"2024-04-25T01:41:20.792222+0000","last_active":"2024-04-25T01:41:20.792201+0000","last_peered":"2024-04-25T01:41:20.792201+0000","last_clean":"2024-04-25T01:41:20.792201+0000","last_became_active":"2024-04-25T01:26:01.415572+0000","last_became_peered":"2024-04-25T01:26:01.415572+0000","last_unstale":"2024-04-25T01:41:20.792201+0000","last_undegraded":"2024-04-25T01:41:20.792201+0000","last_fullsized":"2024-04-25T01:41:20.792201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'2","last_scrub_stamp":"2024-04-25T01:41:20.792117+0000","last_deep_scrub":"26'2","last_deep_scrub_stamp":"2024-04-25T01:41:20.792117+0000","last_clean_scrub_stamp":"2024-04-25T01:41:20.792117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:27.534967+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"26'4341","reported_seq":4372,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:44.187322+0000","last_change":"2024-04-25T01:41:44.187351+0000","last_active":"2024-04-25T01:41:44.187322+0000","last_peered":"2024-04-25T01:41:44.187322+0000","last_clean":"2024-04-25T01:41:44.187322+0000","last_became_active":"2024-04-25T01:26:02.422210+0000","last_became_peered":"2024-04-25T01:26:02.422210+0000","last_unstale":"2024-04-25T01:41:44.187322+0000","last_undegraded":"2024-04-25T01:41:44.187322+0000","last_fullsized":"2024-04-25T01:41:44.187322+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4341","last_scrub_stamp":"2024-04-25T01:41:44.187215+0000","last_deep_scrub":"26'4341","last_deep_scrub_stamp":"2024-04-25T01:41:44.187215+0000","last_clean_scrub_stamp":"2024-04-25T01:41:44.187215+0000","objects_scrubbed":0,"log_size":4341,"log_dups_size":0,"ondisk_log_size":4341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:01.427466+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22600,"num_write_kb":283130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"26'4919","reported_seq":4958,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:25.956327+0000","last_change":"2024-04-25T01:41:25.956355+0000","last_active":"2024-04-25T01:41:25.956327+0000","last_peered":"2024-04-25T01:41:25.956327+0000","last_clean":"2024-04-25T01:41:25.956327+0000","last_became_active":"2024-04-25T01:25:53.367664+0000","last_became_peered":"2024-04-25T01:25:53.367664+0000","last_unstale":"2024-04-25T01:41:25.956327+0000","last_undegraded":"2024-04-25T01:41:25.956327+0000","last_fullsized":"2024-04-25T01:41:25.956327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4919","last_scrub_stamp":"2024-04-25T01:41:25.956179+0000","last_deep_scrub":"26'4919","last_deep_scrub_stamp":"2024-04-25T01:41:25.956179+0000","last_clean_scrub_stamp":"2024-04-25T01:41:25.956179+0000","objects_scrubbed":0,"log_size":4919,"log_dups_size":0,"ondisk_log_size":4919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:00.247143+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25274,"num_write_kb":330818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"26'5","reported_seq":46,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:35.320714+0000","last_change":"2024-04-25T01:41:35.320753+0000","last_active":"2024-04-25T01:41:35.320714+0000","last_peered":"2024-04-25T01:41:35.320714+0000","last_clean":"2024-04-25T01:41:35.320714+0000","last_became_active":"2024-04-25T01:25:52.356881+0000","last_became_peered":"2024-04-25T01:25:52.356881+0000","last_unstale":"2024-04-25T01:41:35.320714+0000","last_undegraded":"2024-04-25T01:41:35.320714+0000","last_fullsized":"2024-04-25T01:41:35.320714+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-04-25T01:41:35.320622+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-04-25T01:41:35.320622+0000","last_clean_scrub_stamp":"2024-04-25T01:41:35.320622+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:18.488101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"26'317","reported_seq":350,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:26.902327+0000","last_change":"2024-04-25T01:41:26.902363+0000","last_active":"2024-04-25T01:41:26.902327+0000","last_peered":"2024-04-25T01:41:26.902327+0000","last_clean":"2024-04-25T01:41:26.902327+0000","last_became_active":"2024-04-25T01:26:01.419526+0000","last_became_peered":"2024-04-25T01:26:01.419526+0000","last_unstale":"2024-04-25T01:41:26.902327+0000","last_undegraded":"2024-04-25T01:41:26.902327+0000","last_fullsized":"2024-04-25T01:41:26.902327+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'317","last_scrub_stamp":"2024-04-25T01:41:26.902232+0000","last_deep_scrub":"26'317","last_deep_scrub_stamp":"2024-04-25T01:41:26.902232+0000","last_clean_scrub_stamp":"2024-04-25T01:41:26.902232+0000","objects_scrubbed":4,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:49.118751+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":310,"num_write_kb":4240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"26'6253","reported_seq":6284,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:48.898636+0000","last_change":"2024-04-25T01:41:48.898664+0000","last_active":"2024-04-25T01:41:48.898636+0000","last_peered":"2024-04-25T01:41:48.898636+0000","last_clean":"2024-04-25T01:41:48.898636+0000","last_became_active":"2024-04-25T01:26:02.419308+0000","last_became_peered":"2024-04-25T01:26:02.419308+0000","last_unstale":"2024-04-25T01:41:48.898636+0000","last_undegraded":"2024-04-25T01:41:48.898636+0000","last_fullsized":"2024-04-25T01:41:48.898636+0000","mapping_epoch":22,"log_start":"26'1000","ondisk_log_start":"26'1000","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'6253","last_scrub_stamp":"2024-04-25T01:41:48.898509+0000","last_deep_scrub":"26'6253","last_deep_scrub_stamp":"2024-04-25T01:41:48.898509+0000","last_clean_scrub_stamp":"2024-04-25T01:41:48.898509+0000","objects_scrubbed":0,"log_size":5253,"log_dups_size":0,"ondisk_log_size":5253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:05.735399+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32276,"num_write_kb":400144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"26'4645","reported_seq":4684,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:30.728029+0000","last_change":"2024-04-25T01:41:30.728055+0000","last_active":"2024-04-25T01:41:30.728029+0000","last_peered":"2024-04-25T01:41:30.728029+0000","last_clean":"2024-04-25T01:41:30.728029+0000","last_became_active":"2024-04-25T01:25:53.370142+0000","last_became_peered":"2024-04-25T01:25:53.370142+0000","last_unstale":"2024-04-25T01:41:30.728029+0000","last_undegraded":"2024-04-25T01:41:30.728029+0000","last_fullsized":"2024-04-25T01:41:30.728029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4645","last_scrub_stamp":"2024-04-25T01:41:30.727918+0000","last_deep_scrub":"26'4645","last_deep_scrub_stamp":"2024-04-25T01:41:30.727918+0000","last_clean_scrub_stamp":"2024-04-25T01:41:30.727918+0000","objects_scrubbed":0,"log_size":4645,"log_dups_size":0,"ondisk_log_size":4645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:05.018882+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24130,"num_write_kb":289357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"26'10","reported_seq":51,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:23.680853+0000","last_change":"2024-04-25T01:41:23.680907+0000","last_active":"2024-04-25T01:41:23.680853+0000","last_peered":"2024-04-25T01:41:23.680853+0000","last_clean":"2024-04-25T01:41:23.680853+0000","last_became_active":"2024-04-25T01:25:52.348846+0000","last_became_peered":"2024-04-25T01:25:52.348846+0000","last_unstale":"2024-04-25T01:41:23.680853+0000","last_undegraded":"2024-04-25T01:41:23.680853+0000","last_fullsized":"2024-04-25T01:41:23.680853+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'10","last_scrub_stamp":"2024-04-25T01:41:23.680737+0000","last_deep_scrub":"26'10","last_deep_scrub_stamp":"2024-04-25T01:41:23.680737+0000","last_clean_scrub_stamp":"2024-04-25T01:41:23.680737+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:22.005464+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"26'28","reported_seq":61,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:24.359447+0000","last_change":"2024-04-25T01:41:24.359476+0000","last_active":"2024-04-25T01:41:24.359447+0000","last_peered":"2024-04-25T01:41:24.359447+0000","last_clean":"2024-04-25T01:41:24.359447+0000","last_became_active":"2024-04-25T01:26:01.414243+0000","last_became_peered":"2024-04-25T01:26:01.414243+0000","last_unstale":"2024-04-25T01:41:24.359447+0000","last_undegraded":"2024-04-25T01:41:24.359447+0000","last_fullsized":"2024-04-25T01:41:24.359447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'28","last_scrub_stamp":"2024-04-25T01:41:24.359370+0000","last_deep_scrub":"26'28","last_deep_scrub_stamp":"2024-04-25T01:41:24.359370+0000","last_clean_scrub_stamp":"2024-04-25T01:41:24.359370+0000","objects_scrubbed":2,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:12.600921+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"26'5295","reported_seq":5326,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:42.983075+0000","last_change":"2024-04-25T01:41:42.983099+0000","last_active":"2024-04-25T01:41:42.983075+0000","last_peered":"2024-04-25T01:41:42.983075+0000","last_clean":"2024-04-25T01:41:42.983075+0000","last_became_active":"2024-04-25T01:26:02.415596+0000","last_became_peered":"2024-04-25T01:26:02.415596+0000","last_unstale":"2024-04-25T01:41:42.983075+0000","last_undegraded":"2024-04-25T01:41:42.983075+0000","last_fullsized":"2024-04-25T01:41:42.983075+0000","mapping_epoch":22,"log_start":"26'100","ondisk_log_start":"26'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5295","last_scrub_stamp":"2024-04-25T01:41:42.982974+0000","last_deep_scrub":"26'5295","last_deep_scrub_stamp":"2024-04-25T01:41:42.982974+0000","last_clean_scrub_stamp":"2024-04-25T01:41:42.982974+0000","objects_scrubbed":0,"log_size":5195,"log_dups_size":0,"ondisk_log_size":5195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:00.223225+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26962,"num_write_kb":337426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"26'4577","reported_seq":4616,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:23.019000+0000","last_change":"2024-04-25T01:41:23.019028+0000","last_active":"2024-04-25T01:41:23.019000+0000","last_peered":"2024-04-25T01:41:23.019000+0000","last_clean":"2024-04-25T01:41:23.019000+0000","last_became_active":"2024-04-25T01:25:53.373129+0000","last_became_peered":"2024-04-25T01:25:53.373129+0000","last_unstale":"2024-04-25T01:41:23.019000+0000","last_undegraded":"2024-04-25T01:41:23.019000+0000","last_fullsized":"2024-04-25T01:41:23.019000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4577","last_scrub_stamp":"2024-04-25T01:41:23.018890+0000","last_deep_scrub":"26'4577","last_deep_scrub_stamp":"2024-04-25T01:41:23.018890+0000","last_clean_scrub_stamp":"2024-04-25T01:41:23.018890+0000","objects_scrubbed":0,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:31.414954+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23671,"num_write_kb":287382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'339","reported_seq":380,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:26.289937+0000","last_change":"2024-04-25T01:41:26.289971+0000","last_active":"2024-04-25T01:41:26.289937+0000","last_peered":"2024-04-25T01:41:26.289937+0000","last_clean":"2024-04-25T01:41:26.289937+0000","last_became_active":"2024-04-25T01:25:52.349448+0000","last_became_peered":"2024-04-25T01:25:52.349448+0000","last_unstale":"2024-04-25T01:41:26.289937+0000","last_undegraded":"2024-04-25T01:41:26.289937+0000","last_fullsized":"2024-04-25T01:41:26.289937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'339","last_scrub_stamp":"2024-04-25T01:41:26.289844+0000","last_deep_scrub":"26'339","last_deep_scrub_stamp":"2024-04-25T01:41:26.289844+0000","last_clean_scrub_stamp":"2024-04-25T01:41:26.289844+0000","objects_scrubbed":4,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:14.160747+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":331,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"26'5","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:26.595786+0000","last_change":"2024-04-25T01:41:26.595813+0000","last_active":"2024-04-25T01:41:26.595786+0000","last_peered":"2024-04-25T01:41:26.595786+0000","last_clean":"2024-04-25T01:41:26.595786+0000","last_became_active":"2024-04-25T01:26:01.418541+0000","last_became_peered":"2024-04-25T01:26:01.418541+0000","last_unstale":"2024-04-25T01:41:26.595786+0000","last_undegraded":"2024-04-25T01:41:26.595786+0000","last_fullsized":"2024-04-25T01:41:26.595786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-04-25T01:41:26.595675+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-04-25T01:41:26.595675+0000","last_clean_scrub_stamp":"2024-04-25T01:41:26.595675+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:08.065163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"26'4257","reported_seq":4288,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:40.649356+0000","last_change":"2024-04-25T01:41:40.649384+0000","last_active":"2024-04-25T01:41:40.649356+0000","last_peered":"2024-04-25T01:41:40.649356+0000","last_clean":"2024-04-25T01:41:40.649356+0000","last_became_active":"2024-04-25T01:26:02.425106+0000","last_became_peered":"2024-04-25T01:26:02.425106+0000","last_unstale":"2024-04-25T01:41:40.649356+0000","last_undegraded":"2024-04-25T01:41:40.649356+0000","last_fullsized":"2024-04-25T01:41:40.649356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4257","last_scrub_stamp":"2024-04-25T01:41:40.649245+0000","last_deep_scrub":"26'4257","last_deep_scrub_stamp":"2024-04-25T01:41:40.649245+0000","last_clean_scrub_stamp":"2024-04-25T01:41:40.649245+0000","objects_scrubbed":0,"log_size":4257,"log_dups_size":0,"ondisk_log_size":4257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:14.543796+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22152,"num_write_kb":281377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"26'6968","reported_seq":7007,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:32.135675+0000","last_change":"2024-04-25T01:41:32.135701+0000","last_active":"2024-04-25T01:41:32.135675+0000","last_peered":"2024-04-25T01:41:32.135675+0000","last_clean":"2024-04-25T01:41:32.135675+0000","last_became_active":"2024-04-25T01:25:53.366157+0000","last_became_peered":"2024-04-25T01:25:53.366157+0000","last_unstale":"2024-04-25T01:41:32.135675+0000","last_undegraded":"2024-04-25T01:41:32.135675+0000","last_fullsized":"2024-04-25T01:41:32.135675+0000","mapping_epoch":18,"log_start":"26'1500","ondisk_log_start":"26'1500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'6968","last_scrub_stamp":"2024-04-25T01:41:32.135560+0000","last_deep_scrub":"26'6968","last_deep_scrub_stamp":"2024-04-25T01:41:32.135560+0000","last_clean_scrub_stamp":"2024-04-25T01:41:32.135560+0000","objects_scrubbed":0,"log_size":5468,"log_dups_size":0,"ondisk_log_size":5468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:20.687525+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35763,"num_write_kb":414188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"26'27","reported_seq":68,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:33.577988+0000","last_change":"2024-04-25T01:41:33.578018+0000","last_active":"2024-04-25T01:41:33.577988+0000","last_peered":"2024-04-25T01:41:33.577988+0000","last_clean":"2024-04-25T01:41:33.577988+0000","last_became_active":"2024-04-25T01:25:52.357355+0000","last_became_peered":"2024-04-25T01:25:52.357355+0000","last_unstale":"2024-04-25T01:41:33.577988+0000","last_undegraded":"2024-04-25T01:41:33.577988+0000","last_fullsized":"2024-04-25T01:41:33.577988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'27","last_scrub_stamp":"2024-04-25T01:41:33.577900+0000","last_deep_scrub":"26'27","last_deep_scrub_stamp":"2024-04-25T01:41:33.577900+0000","last_clean_scrub_stamp":"2024-04-25T01:41:33.577900+0000","objects_scrubbed":2,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:46.349568+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"26'10","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:24.163663+0000","last_change":"2024-04-25T01:41:24.163692+0000","last_active":"2024-04-25T01:41:24.163663+0000","last_peered":"2024-04-25T01:41:24.163663+0000","last_clean":"2024-04-25T01:41:24.163663+0000","last_became_active":"2024-04-25T01:26:01.412132+0000","last_became_peered":"2024-04-25T01:26:01.412132+0000","last_unstale":"2024-04-25T01:41:24.163663+0000","last_undegraded":"2024-04-25T01:41:24.163663+0000","last_fullsized":"2024-04-25T01:41:24.163663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'10","last_scrub_stamp":"2024-04-25T01:41:24.163574+0000","last_deep_scrub":"26'10","last_deep_scrub_stamp":"2024-04-25T01:41:24.163574+0000","last_clean_scrub_stamp":"2024-04-25T01:41:24.163574+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:05.113469+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"26'5141","reported_seq":5172,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:36.238761+0000","last_change":"2024-04-25T01:41:36.238802+0000","last_active":"2024-04-25T01:41:36.238761+0000","last_peered":"2024-04-25T01:41:36.238761+0000","last_clean":"2024-04-25T01:41:36.238761+0000","last_became_active":"2024-04-25T01:26:02.412759+0000","last_became_peered":"2024-04-25T01:26:02.412759+0000","last_unstale":"2024-04-25T01:41:36.238761+0000","last_undegraded":"2024-04-25T01:41:36.238761+0000","last_fullsized":"2024-04-25T01:41:36.238761+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5141","last_scrub_stamp":"2024-04-25T01:41:36.238638+0000","last_deep_scrub":"26'5141","last_deep_scrub_stamp":"2024-04-25T01:41:36.238638+0000","last_clean_scrub_stamp":"2024-04-25T01:41:36.238638+0000","objects_scrubbed":0,"log_size":5141,"log_dups_size":0,"ondisk_log_size":5141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:24.790603+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27471,"num_write_kb":310956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"26'3595","reported_seq":3634,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:18.724568+0000","last_change":"2024-04-25T01:41:18.724618+0000","last_active":"2024-04-25T01:41:18.724568+0000","last_peered":"2024-04-25T01:41:18.724568+0000","last_clean":"2024-04-25T01:41:18.724568+0000","last_became_active":"2024-04-25T01:25:53.373890+0000","last_became_peered":"2024-04-25T01:25:53.373890+0000","last_unstale":"2024-04-25T01:41:18.724568+0000","last_undegraded":"2024-04-25T01:41:18.724568+0000","last_fullsized":"2024-04-25T01:41:18.724568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3595","last_scrub_stamp":"2024-04-25T01:41:18.724464+0000","last_deep_scrub":"26'3595","last_deep_scrub_stamp":"2024-04-25T01:41:18.724464+0000","last_clean_scrub_stamp":"2024-04-25T01:41:18.724464+0000","objects_scrubbed":0,"log_size":3595,"log_dups_size":0,"ondisk_log_size":3595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:50.933963+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18396,"num_write_kb":249899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"26'2","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:34.358052+0000","last_change":"2024-04-25T01:41:34.358073+0000","last_active":"2024-04-25T01:41:34.358052+0000","last_peered":"2024-04-25T01:41:34.358052+0000","last_clean":"2024-04-25T01:41:34.358052+0000","last_became_active":"2024-04-25T01:25:52.355913+0000","last_became_peered":"2024-04-25T01:25:52.355913+0000","last_unstale":"2024-04-25T01:41:34.358052+0000","last_undegraded":"2024-04-25T01:41:34.358052+0000","last_fullsized":"2024-04-25T01:41:34.358052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'2","last_scrub_stamp":"2024-04-25T01:41:34.357973+0000","last_deep_scrub":"26'2","last_deep_scrub_stamp":"2024-04-25T01:41:34.357973+0000","last_clean_scrub_stamp":"2024-04-25T01:41:34.357973+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:33.500655+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"26'16","reported_seq":49,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:24.878084+0000","last_change":"2024-04-25T01:41:24.878119+0000","last_active":"2024-04-25T01:41:24.878084+0000","last_peered":"2024-04-25T01:41:24.878084+0000","last_clean":"2024-04-25T01:41:24.878084+0000","last_became_active":"2024-04-25T01:26:01.419015+0000","last_became_peered":"2024-04-25T01:26:01.419015+0000","last_unstale":"2024-04-25T01:41:24.878084+0000","last_undegraded":"2024-04-25T01:41:24.878084+0000","last_fullsized":"2024-04-25T01:41:24.878084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'16","last_scrub_stamp":"2024-04-25T01:41:24.878001+0000","last_deep_scrub":"26'16","last_deep_scrub_stamp":"2024-04-25T01:41:24.878001+0000","last_clean_scrub_stamp":"2024-04-25T01:41:24.878001+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:38.557008+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"26'5013","reported_seq":5044,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:46.723162+0000","last_change":"2024-04-25T01:41:46.723206+0000","last_active":"2024-04-25T01:41:46.723162+0000","last_peered":"2024-04-25T01:41:46.723162+0000","last_clean":"2024-04-25T01:41:46.723162+0000","last_became_active":"2024-04-25T01:26:02.424960+0000","last_became_peered":"2024-04-25T01:26:02.424960+0000","last_unstale":"2024-04-25T01:41:46.723162+0000","last_undegraded":"2024-04-25T01:41:46.723162+0000","last_fullsized":"2024-04-25T01:41:46.723162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5013","last_scrub_stamp":"2024-04-25T01:41:46.723040+0000","last_deep_scrub":"26'5013","last_deep_scrub_stamp":"2024-04-25T01:41:46.723040+0000","last_clean_scrub_stamp":"2024-04-25T01:41:46.723040+0000","objects_scrubbed":0,"log_size":5013,"log_dups_size":0,"ondisk_log_size":5013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:37.801411+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25777,"num_write_kb":320486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"26'4876","reported_seq":4915,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:23.375681+0000","last_change":"2024-04-25T01:41:23.375705+0000","last_active":"2024-04-25T01:41:23.375681+0000","last_peered":"2024-04-25T01:41:23.375681+0000","last_clean":"2024-04-25T01:41:23.375681+0000","last_became_active":"2024-04-25T01:25:53.371086+0000","last_became_peered":"2024-04-25T01:25:53.371086+0000","last_unstale":"2024-04-25T01:41:23.375681+0000","last_undegraded":"2024-04-25T01:41:23.375681+0000","last_fullsized":"2024-04-25T01:41:23.375681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4876","last_scrub_stamp":"2024-04-25T01:41:23.375592+0000","last_deep_scrub":"26'4876","last_deep_scrub_stamp":"2024-04-25T01:41:23.375592+0000","last_clean_scrub_stamp":"2024-04-25T01:41:23.375592+0000","objects_scrubbed":0,"log_size":4876,"log_dups_size":0,"ondisk_log_size":4876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:46.392469+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24656,"num_write_kb":303790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"26'2","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:29.305845+0000","last_change":"2024-04-25T01:41:29.305917+0000","last_active":"2024-04-25T01:41:29.305845+0000","last_peered":"2024-04-25T01:41:29.305845+0000","last_clean":"2024-04-25T01:41:29.305845+0000","last_became_active":"2024-04-25T01:25:52.357182+0000","last_became_peered":"2024-04-25T01:25:52.357182+0000","last_unstale":"2024-04-25T01:41:29.305845+0000","last_undegraded":"2024-04-25T01:41:29.305845+0000","last_fullsized":"2024-04-25T01:41:29.305845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'2","last_scrub_stamp":"2024-04-25T01:41:29.305722+0000","last_deep_scrub":"26'2","last_deep_scrub_stamp":"2024-04-25T01:41:29.305722+0000","last_clean_scrub_stamp":"2024-04-25T01:41:29.305722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:37.701785+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"26'6","reported_seq":39,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:21.821547+0000","last_change":"2024-04-25T01:41:21.821595+0000","last_active":"2024-04-25T01:41:21.821547+0000","last_peered":"2024-04-25T01:41:21.821547+0000","last_clean":"2024-04-25T01:41:21.821547+0000","last_became_active":"2024-04-25T01:26:01.415020+0000","last_became_peered":"2024-04-25T01:26:01.415020+0000","last_unstale":"2024-04-25T01:41:21.821547+0000","last_undegraded":"2024-04-25T01:41:21.821547+0000","last_fullsized":"2024-04-25T01:41:21.821547+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'6","last_scrub_stamp":"2024-04-25T01:41:21.821446+0000","last_deep_scrub":"26'6","last_deep_scrub_stamp":"2024-04-25T01:41:21.821446+0000","last_clean_scrub_stamp":"2024-04-25T01:41:21.821446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:44.838323+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"26'5316","reported_seq":5348,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:39.237288+0000","last_change":"2024-04-25T01:41:39.237328+0000","last_active":"2024-04-25T01:41:39.237288+0000","last_peered":"2024-04-25T01:41:39.237288+0000","last_clean":"2024-04-25T01:41:39.237288+0000","last_became_active":"2024-04-25T01:26:02.416623+0000","last_became_peered":"2024-04-25T01:26:02.416623+0000","last_unstale":"2024-04-25T01:41:39.237288+0000","last_undegraded":"2024-04-25T01:41:39.237288+0000","last_fullsized":"2024-04-25T01:41:39.237288+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5316","last_scrub_stamp":"2024-04-25T01:41:39.237167+0000","last_deep_scrub":"26'5316","last_deep_scrub_stamp":"2024-04-25T01:41:39.237167+0000","last_clean_scrub_stamp":"2024-04-25T01:41:39.237167+0000","objects_scrubbed":0,"log_size":5316,"log_dups_size":0,"ondisk_log_size":5316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:06.318501+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27734,"num_write_kb":356956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"26'8","reported_seq":42,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:32.742444+0000","last_change":"2024-04-25T01:41:32.742469+0000","last_active":"2024-04-25T01:41:32.742444+0000","last_peered":"2024-04-25T01:41:32.742444+0000","last_clean":"2024-04-25T01:41:32.742444+0000","last_became_active":"2024-04-25T01:26:01.415184+0000","last_became_peered":"2024-04-25T01:26:01.415184+0000","last_unstale":"2024-04-25T01:41:32.742444+0000","last_undegraded":"2024-04-25T01:41:32.742444+0000","last_fullsized":"2024-04-25T01:41:32.742444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'8","last_scrub_stamp":"2024-04-25T01:41:32.742355+0000","last_deep_scrub":"26'8","last_deep_scrub_stamp":"2024-04-25T01:41:32.742355+0000","last_clean_scrub_stamp":"2024-04-25T01:41:32.742355+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:02.329346+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"26'5623","reported_seq":5662,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:35.555255+0000","last_change":"2024-04-25T01:41:35.555301+0000","last_active":"2024-04-25T01:41:35.555255+0000","last_peered":"2024-04-25T01:41:35.555255+0000","last_clean":"2024-04-25T01:41:35.555255+0000","last_became_active":"2024-04-25T01:25:53.362601+0000","last_became_peered":"2024-04-25T01:25:53.362601+0000","last_unstale":"2024-04-25T01:41:35.555255+0000","last_undegraded":"2024-04-25T01:41:35.555255+0000","last_fullsized":"2024-04-25T01:41:35.555255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5623","last_scrub_stamp":"2024-04-25T01:41:35.555135+0000","last_deep_scrub":"26'5623","last_deep_scrub_stamp":"2024-04-25T01:41:35.555135+0000","last_clean_scrub_stamp":"2024-04-25T01:41:35.555135+0000","objects_scrubbed":0,"log_size":5623,"log_dups_size":0,"ondisk_log_size":5623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:54.053742+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29125,"num_write_kb":354366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"26'3","reported_seq":44,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:36.321901+0000","last_change":"2024-04-25T01:41:36.321927+0000","last_active":"2024-04-25T01:41:36.321901+0000","last_peered":"2024-04-25T01:41:36.321901+0000","last_clean":"2024-04-25T01:41:36.321901+0000","last_became_active":"2024-04-25T01:25:52.357365+0000","last_became_peered":"2024-04-25T01:25:52.357365+0000","last_unstale":"2024-04-25T01:41:36.321901+0000","last_undegraded":"2024-04-25T01:41:36.321901+0000","last_fullsized":"2024-04-25T01:41:36.321901+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:36.321804+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:36.321804+0000","last_clean_scrub_stamp":"2024-04-25T01:41:36.321804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:49.391623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"26'4779","reported_seq":4811,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:37.717221+0000","last_change":"2024-04-25T01:41:37.717247+0000","last_active":"2024-04-25T01:41:37.717221+0000","last_peered":"2024-04-25T01:41:37.717221+0000","last_clean":"2024-04-25T01:41:37.717221+0000","last_became_active":"2024-04-25T01:26:02.418614+0000","last_became_peered":"2024-04-25T01:26:02.418614+0000","last_unstale":"2024-04-25T01:41:37.717221+0000","last_undegraded":"2024-04-25T01:41:37.717221+0000","last_fullsized":"2024-04-25T01:41:37.717221+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4779","last_scrub_stamp":"2024-04-25T01:41:37.717124+0000","last_deep_scrub":"26'4779","last_deep_scrub_stamp":"2024-04-25T01:41:37.717124+0000","last_clean_scrub_stamp":"2024-04-25T01:41:37.717124+0000","objects_scrubbed":0,"log_size":4779,"log_dups_size":0,"ondisk_log_size":4779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:04.798458+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25207,"num_write_kb":318198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"26'635","reported_seq":669,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:34.306312+0000","last_change":"2024-04-25T01:41:34.306366+0000","last_active":"2024-04-25T01:41:34.306312+0000","last_peered":"2024-04-25T01:41:34.306312+0000","last_clean":"2024-04-25T01:41:34.306312+0000","last_became_active":"2024-04-25T01:26:01.410004+0000","last_became_peered":"2024-04-25T01:26:01.410004+0000","last_unstale":"2024-04-25T01:41:34.306312+0000","last_undegraded":"2024-04-25T01:41:34.306312+0000","last_fullsized":"2024-04-25T01:41:34.306312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'635","last_scrub_stamp":"2024-04-25T01:41:34.306218+0000","last_deep_scrub":"26'635","last_deep_scrub_stamp":"2024-04-25T01:41:34.306218+0000","last_clean_scrub_stamp":"2024-04-25T01:41:34.306218+0000","objects_scrubbed":3,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:03.893209+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":630,"num_write_kb":4461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"26'4416","reported_seq":4455,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:41.946113+0000","last_change":"2024-04-25T01:41:41.946139+0000","last_active":"2024-04-25T01:41:41.946113+0000","last_peered":"2024-04-25T01:41:41.946113+0000","last_clean":"2024-04-25T01:41:41.946113+0000","last_became_active":"2024-04-25T01:25:53.363509+0000","last_became_peered":"2024-04-25T01:25:53.363509+0000","last_unstale":"2024-04-25T01:41:41.946113+0000","last_undegraded":"2024-04-25T01:41:41.946113+0000","last_fullsized":"2024-04-25T01:41:41.946113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4416","last_scrub_stamp":"2024-04-25T01:41:41.946000+0000","last_deep_scrub":"26'4416","last_deep_scrub_stamp":"2024-04-25T01:41:41.946000+0000","last_clean_scrub_stamp":"2024-04-25T01:41:41.946000+0000","objects_scrubbed":0,"log_size":4416,"log_dups_size":0,"ondisk_log_size":4416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:46.831783+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23348,"num_write_kb":294385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"26'3","reported_seq":44,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:34.894434+0000","last_change":"2024-04-25T01:41:34.894462+0000","last_active":"2024-04-25T01:41:34.894434+0000","last_peered":"2024-04-25T01:41:34.894434+0000","last_clean":"2024-04-25T01:41:34.894434+0000","last_became_active":"2024-04-25T01:25:52.347138+0000","last_became_peered":"2024-04-25T01:25:52.347138+0000","last_unstale":"2024-04-25T01:41:34.894434+0000","last_undegraded":"2024-04-25T01:41:34.894434+0000","last_fullsized":"2024-04-25T01:41:34.894434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-04-25T01:41:34.894343+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-04-25T01:41:34.894343+0000","last_clean_scrub_stamp":"2024-04-25T01:41:34.894343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:47.964161+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"26'5575","reported_seq":5606,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:42.653326+0000","last_change":"2024-04-25T01:41:42.653383+0000","last_active":"2024-04-25T01:41:42.653326+0000","last_peered":"2024-04-25T01:41:42.653326+0000","last_clean":"2024-04-25T01:41:42.653326+0000","last_became_active":"2024-04-25T01:26:02.424947+0000","last_became_peered":"2024-04-25T01:26:02.424947+0000","last_unstale":"2024-04-25T01:41:42.653326+0000","last_undegraded":"2024-04-25T01:41:42.653326+0000","last_fullsized":"2024-04-25T01:41:42.653326+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5575","last_scrub_stamp":"2024-04-25T01:41:42.653176+0000","last_deep_scrub":"26'5575","last_deep_scrub_stamp":"2024-04-25T01:41:42.653176+0000","last_clean_scrub_stamp":"2024-04-25T01:41:42.653176+0000","objects_scrubbed":0,"log_size":5575,"log_dups_size":0,"ondisk_log_size":5575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:43.470814+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28821,"num_write_kb":361321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"26'87","reported_seq":128,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:25.280092+0000","last_change":"2024-04-25T01:41:25.280120+0000","last_active":"2024-04-25T01:41:25.280092+0000","last_peered":"2024-04-25T01:41:25.280092+0000","last_clean":"2024-04-25T01:41:25.280092+0000","last_became_active":"2024-04-25T01:25:52.349672+0000","last_became_peered":"2024-04-25T01:25:52.349672+0000","last_unstale":"2024-04-25T01:41:25.280092+0000","last_undegraded":"2024-04-25T01:41:25.280092+0000","last_fullsized":"2024-04-25T01:41:25.280092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'87","last_scrub_stamp":"2024-04-25T01:41:25.279997+0000","last_deep_scrub":"26'87","last_deep_scrub_stamp":"2024-04-25T01:41:25.279997+0000","last_clean_scrub_stamp":"2024-04-25T01:41:25.279997+0000","objects_scrubbed":1,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:40.457549+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":980823,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83,"num_write_kb":1015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"26'3832","reported_seq":3871,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:26.716835+0000","last_change":"2024-04-25T01:41:26.716862+0000","last_active":"2024-04-25T01:41:26.716835+0000","last_peered":"2024-04-25T01:41:26.716835+0000","last_clean":"2024-04-25T01:41:26.716835+0000","last_became_active":"2024-04-25T01:25:53.356015+0000","last_became_peered":"2024-04-25T01:25:53.356015+0000","last_unstale":"2024-04-25T01:41:26.716835+0000","last_undegraded":"2024-04-25T01:41:26.716835+0000","last_fullsized":"2024-04-25T01:41:26.716835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3832","last_scrub_stamp":"2024-04-25T01:41:26.716735+0000","last_deep_scrub":"26'3832","last_deep_scrub_stamp":"2024-04-25T01:41:26.716735+0000","last_clean_scrub_stamp":"2024-04-25T01:41:26.716735+0000","objects_scrubbed":0,"log_size":3832,"log_dups_size":0,"ondisk_log_size":3832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:07.666630+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19386,"num_write_kb":237359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"26'4","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:28.078083+0000","last_change":"2024-04-25T01:41:28.078113+0000","last_active":"2024-04-25T01:41:28.078083+0000","last_peered":"2024-04-25T01:41:28.078083+0000","last_clean":"2024-04-25T01:41:28.078083+0000","last_became_active":"2024-04-25T01:26:01.420003+0000","last_became_peered":"2024-04-25T01:26:01.420003+0000","last_unstale":"2024-04-25T01:41:28.078083+0000","last_undegraded":"2024-04-25T01:41:28.078083+0000","last_fullsized":"2024-04-25T01:41:28.078083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4","last_scrub_stamp":"2024-04-25T01:41:28.077984+0000","last_deep_scrub":"26'4","last_deep_scrub_stamp":"2024-04-25T01:41:28.077984+0000","last_clean_scrub_stamp":"2024-04-25T01:41:28.077984+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:02.920281+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"26'3175","reported_seq":3206,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:41.133164+0000","last_change":"2024-04-25T01:41:41.133191+0000","last_active":"2024-04-25T01:41:41.133164+0000","last_peered":"2024-04-25T01:41:41.133164+0000","last_clean":"2024-04-25T01:41:41.133164+0000","last_became_active":"2024-04-25T01:26:02.426054+0000","last_became_peered":"2024-04-25T01:26:02.426054+0000","last_unstale":"2024-04-25T01:41:41.133164+0000","last_undegraded":"2024-04-25T01:41:41.133164+0000","last_fullsized":"2024-04-25T01:41:41.133164+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3175","last_scrub_stamp":"2024-04-25T01:41:41.133055+0000","last_deep_scrub":"26'3175","last_deep_scrub_stamp":"2024-04-25T01:41:41.133055+0000","last_clean_scrub_stamp":"2024-04-25T01:41:41.133055+0000","objects_scrubbed":0,"log_size":3175,"log_dups_size":0,"ondisk_log_size":3175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:38.032590+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15854,"num_write_kb":211025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"26'606","reported_seq":647,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:36.921170+0000","last_change":"2024-04-25T01:41:36.921200+0000","last_active":"2024-04-25T01:41:36.921170+0000","last_peered":"2024-04-25T01:41:36.921170+0000","last_clean":"2024-04-25T01:41:36.921170+0000","last_became_active":"2024-04-25T01:25:52.353659+0000","last_became_peered":"2024-04-25T01:25:52.353659+0000","last_unstale":"2024-04-25T01:41:36.921170+0000","last_undegraded":"2024-04-25T01:41:36.921170+0000","last_fullsized":"2024-04-25T01:41:36.921170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'606","last_scrub_stamp":"2024-04-25T01:41:36.921029+0000","last_deep_scrub":"26'606","last_deep_scrub_stamp":"2024-04-25T01:41:36.921029+0000","last_clean_scrub_stamp":"2024-04-25T01:41:36.921029+0000","objects_scrubbed":1,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:22.062095+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":603,"num_write_kb":4239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"26'4191","reported_seq":4230,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:33.173961+0000","last_change":"2024-04-25T01:41:33.174004+0000","last_active":"2024-04-25T01:41:33.173961+0000","last_peered":"2024-04-25T01:41:33.173961+0000","last_clean":"2024-04-25T01:41:33.173961+0000","last_became_active":"2024-04-25T01:25:53.366302+0000","last_became_peered":"2024-04-25T01:25:53.366302+0000","last_unstale":"2024-04-25T01:41:33.173961+0000","last_undegraded":"2024-04-25T01:41:33.173961+0000","last_fullsized":"2024-04-25T01:41:33.173961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4191","last_scrub_stamp":"2024-04-25T01:41:33.173830+0000","last_deep_scrub":"26'4191","last_deep_scrub_stamp":"2024-04-25T01:41:33.173830+0000","last_clean_scrub_stamp":"2024-04-25T01:41:33.173830+0000","objects_scrubbed":0,"log_size":4191,"log_dups_size":0,"ondisk_log_size":4191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:16.285208+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21716,"num_write_kb":263159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"26'12","reported_seq":45,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:28.860317+0000","last_change":"2024-04-25T01:41:28.860346+0000","last_active":"2024-04-25T01:41:28.860317+0000","last_peered":"2024-04-25T01:41:28.860317+0000","last_clean":"2024-04-25T01:41:28.860317+0000","last_became_active":"2024-04-25T01:26:01.420713+0000","last_became_peered":"2024-04-25T01:26:01.420713+0000","last_unstale":"2024-04-25T01:41:28.860317+0000","last_undegraded":"2024-04-25T01:41:28.860317+0000","last_fullsized":"2024-04-25T01:41:28.860317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'12","last_scrub_stamp":"2024-04-25T01:41:28.860233+0000","last_deep_scrub":"26'12","last_deep_scrub_stamp":"2024-04-25T01:41:28.860233+0000","last_clean_scrub_stamp":"2024-04-25T01:41:28.860233+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:15.013771+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"26'4569","reported_seq":4601,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:47.932067+0000","last_change":"2024-04-25T01:41:47.932112+0000","last_active":"2024-04-25T01:41:47.932067+0000","last_peered":"2024-04-25T01:41:47.932067+0000","last_clean":"2024-04-25T01:41:47.932067+0000","last_became_active":"2024-04-25T01:26:02.418802+0000","last_became_peered":"2024-04-25T01:26:02.418802+0000","last_unstale":"2024-04-25T01:41:47.932067+0000","last_undegraded":"2024-04-25T01:41:47.932067+0000","last_fullsized":"2024-04-25T01:41:47.932067+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4569","last_scrub_stamp":"2024-04-25T01:41:47.931946+0000","last_deep_scrub":"26'4569","last_deep_scrub_stamp":"2024-04-25T01:41:47.931946+0000","last_clean_scrub_stamp":"2024-04-25T01:41:47.931946+0000","objects_scrubbed":0,"log_size":4569,"log_dups_size":0,"ondisk_log_size":4569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:55.655115+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23161,"num_write_kb":277521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"26'6240","reported_seq":6280,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:31.325243+0000","last_change":"2024-04-25T01:41:31.325270+0000","last_active":"2024-04-25T01:41:31.325243+0000","last_peered":"2024-04-25T01:41:31.325243+0000","last_clean":"2024-04-25T01:41:31.325243+0000","last_became_active":"2024-04-25T01:25:53.366847+0000","last_became_peered":"2024-04-25T01:25:53.366847+0000","last_unstale":"2024-04-25T01:41:31.325243+0000","last_undegraded":"2024-04-25T01:41:31.325243+0000","last_fullsized":"2024-04-25T01:41:31.325243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'6240","last_scrub_stamp":"2024-04-25T01:41:31.325133+0000","last_deep_scrub":"26'6240","last_deep_scrub_stamp":"2024-04-25T01:41:31.325133+0000","last_clean_scrub_stamp":"2024-04-25T01:41:31.325133+0000","objects_scrubbed":0,"log_size":6240,"log_dups_size":0,"ondisk_log_size":6240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:45.057188+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32134,"num_write_kb":382834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"26'5302","reported_seq":5334,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:36.547741+0000","last_change":"2024-04-25T01:41:36.547808+0000","last_active":"2024-04-25T01:41:36.547741+0000","last_peered":"2024-04-25T01:41:36.547741+0000","last_clean":"2024-04-25T01:41:36.547741+0000","last_became_active":"2024-04-25T01:26:02.416170+0000","last_became_peered":"2024-04-25T01:26:02.416170+0000","last_unstale":"2024-04-25T01:41:36.547741+0000","last_undegraded":"2024-04-25T01:41:36.547741+0000","last_fullsized":"2024-04-25T01:41:36.547741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5302","last_scrub_stamp":"2024-04-25T01:41:36.547604+0000","last_deep_scrub":"26'5302","last_deep_scrub_stamp":"2024-04-25T01:41:36.547604+0000","last_clean_scrub_stamp":"2024-04-25T01:41:36.547604+0000","objects_scrubbed":0,"log_size":5302,"log_dups_size":0,"ondisk_log_size":5302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:12.300281+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27000,"num_write_kb":345927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"26'4854","reported_seq":4894,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:27.082588+0000","last_change":"2024-04-25T01:41:27.082615+0000","last_active":"2024-04-25T01:41:27.082588+0000","last_peered":"2024-04-25T01:41:27.082588+0000","last_clean":"2024-04-25T01:41:27.082588+0000","last_became_active":"2024-04-25T01:25:53.371080+0000","last_became_peered":"2024-04-25T01:25:53.371080+0000","last_unstale":"2024-04-25T01:41:27.082588+0000","last_undegraded":"2024-04-25T01:41:27.082588+0000","last_fullsized":"2024-04-25T01:41:27.082588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4854","last_scrub_stamp":"2024-04-25T01:41:27.082471+0000","last_deep_scrub":"26'4854","last_deep_scrub_stamp":"2024-04-25T01:41:27.082471+0000","last_clean_scrub_stamp":"2024-04-25T01:41:27.082471+0000","objects_scrubbed":0,"log_size":4854,"log_dups_size":0,"ondisk_log_size":4854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:42.260023+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24922,"num_write_kb":316898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"26'4098","reported_seq":4130,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:41.279598+0000","last_change":"2024-04-25T01:41:41.279626+0000","last_active":"2024-04-25T01:41:41.279598+0000","last_peered":"2024-04-25T01:41:41.279598+0000","last_clean":"2024-04-25T01:41:41.279598+0000","last_became_active":"2024-04-25T01:26:02.419417+0000","last_became_peered":"2024-04-25T01:26:02.419417+0000","last_unstale":"2024-04-25T01:41:41.279598+0000","last_undegraded":"2024-04-25T01:41:41.279598+0000","last_fullsized":"2024-04-25T01:41:41.279598+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4098","last_scrub_stamp":"2024-04-25T01:41:41.279488+0000","last_deep_scrub":"26'4098","last_deep_scrub_stamp":"2024-04-25T01:41:41.279488+0000","last_clean_scrub_stamp":"2024-04-25T01:41:41.279488+0000","objects_scrubbed":0,"log_size":4098,"log_dups_size":0,"ondisk_log_size":4098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:42.097126+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21368,"num_write_kb":269972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"26'5565","reported_seq":5605,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:31.424823+0000","last_change":"2024-04-25T01:41:31.424850+0000","last_active":"2024-04-25T01:41:31.424823+0000","last_peered":"2024-04-25T01:41:31.424823+0000","last_clean":"2024-04-25T01:41:31.424823+0000","last_became_active":"2024-04-25T01:25:53.369350+0000","last_became_peered":"2024-04-25T01:25:53.369350+0000","last_unstale":"2024-04-25T01:41:31.424823+0000","last_undegraded":"2024-04-25T01:41:31.424823+0000","last_fullsized":"2024-04-25T01:41:31.424823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5565","last_scrub_stamp":"2024-04-25T01:41:31.424703+0000","last_deep_scrub":"26'5565","last_deep_scrub_stamp":"2024-04-25T01:41:31.424703+0000","last_clean_scrub_stamp":"2024-04-25T01:41:31.424703+0000","objects_scrubbed":0,"log_size":5565,"log_dups_size":0,"ondisk_log_size":5565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:58.506036+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29144,"num_write_kb":362518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"26'5067","reported_seq":5099,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:41.614085+0000","last_change":"2024-04-25T01:41:41.614148+0000","last_active":"2024-04-25T01:41:41.614085+0000","last_peered":"2024-04-25T01:41:41.614085+0000","last_clean":"2024-04-25T01:41:41.614085+0000","last_became_active":"2024-04-25T01:26:02.425160+0000","last_became_peered":"2024-04-25T01:26:02.425160+0000","last_unstale":"2024-04-25T01:41:41.614085+0000","last_undegraded":"2024-04-25T01:41:41.614085+0000","last_fullsized":"2024-04-25T01:41:41.614085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5067","last_scrub_stamp":"2024-04-25T01:41:41.613963+0000","last_deep_scrub":"26'5067","last_deep_scrub_stamp":"2024-04-25T01:41:41.613963+0000","last_clean_scrub_stamp":"2024-04-25T01:41:41.613963+0000","objects_scrubbed":0,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:35.678312+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26737,"num_write_kb":324333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"26'3787","reported_seq":3827,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:27.730829+0000","last_change":"2024-04-25T01:41:27.730858+0000","last_active":"2024-04-25T01:41:27.730829+0000","last_peered":"2024-04-25T01:41:27.730829+0000","last_clean":"2024-04-25T01:41:27.730829+0000","last_became_active":"2024-04-25T01:25:53.356854+0000","last_became_peered":"2024-04-25T01:25:53.356854+0000","last_unstale":"2024-04-25T01:41:27.730829+0000","last_undegraded":"2024-04-25T01:41:27.730829+0000","last_fullsized":"2024-04-25T01:41:27.730829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3787","last_scrub_stamp":"2024-04-25T01:41:27.730723+0000","last_deep_scrub":"26'3787","last_deep_scrub_stamp":"2024-04-25T01:41:27.730723+0000","last_clean_scrub_stamp":"2024-04-25T01:41:27.730723+0000","objects_scrubbed":0,"log_size":3787,"log_dups_size":0,"ondisk_log_size":3787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:36.126786+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19900,"num_write_kb":255902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"26'5481","reported_seq":5513,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:28.933841+0000","last_change":"2024-04-25T01:41:28.933881+0000","last_active":"2024-04-25T01:41:28.933841+0000","last_peered":"2024-04-25T01:41:28.933841+0000","last_clean":"2024-04-25T01:41:28.933841+0000","last_became_active":"2024-04-25T01:26:02.417234+0000","last_became_peered":"2024-04-25T01:26:02.417234+0000","last_unstale":"2024-04-25T01:41:28.933841+0000","last_undegraded":"2024-04-25T01:41:28.933841+0000","last_fullsized":"2024-04-25T01:41:28.933841+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5481","last_scrub_stamp":"2024-04-25T01:41:28.933722+0000","last_deep_scrub":"26'5481","last_deep_scrub_stamp":"2024-04-25T01:41:28.933722+0000","last_clean_scrub_stamp":"2024-04-25T01:41:28.933722+0000","objects_scrubbed":0,"log_size":5481,"log_dups_size":0,"ondisk_log_size":5481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:28.118343+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28430,"num_write_kb":359662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"26'4160","reported_seq":4200,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:31.931946+0000","last_change":"2024-04-25T01:41:31.931973+0000","last_active":"2024-04-25T01:41:31.931946+0000","last_peered":"2024-04-25T01:41:31.931946+0000","last_clean":"2024-04-25T01:41:31.931946+0000","last_became_active":"2024-04-25T01:25:53.364311+0000","last_became_peered":"2024-04-25T01:25:53.364311+0000","last_unstale":"2024-04-25T01:41:31.931946+0000","last_undegraded":"2024-04-25T01:41:31.931946+0000","last_fullsized":"2024-04-25T01:41:31.931946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4160","last_scrub_stamp":"2024-04-25T01:41:31.931832+0000","last_deep_scrub":"26'4160","last_deep_scrub_stamp":"2024-04-25T01:41:31.931832+0000","last_clean_scrub_stamp":"2024-04-25T01:41:31.931832+0000","objects_scrubbed":0,"log_size":4160,"log_dups_size":0,"ondisk_log_size":4160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:22.678450+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21579,"num_write_kb":270844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"26'4125","reported_seq":4157,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:43.090998+0000","last_change":"2024-04-25T01:41:43.091041+0000","last_active":"2024-04-25T01:41:43.090998+0000","last_peered":"2024-04-25T01:41:43.090998+0000","last_clean":"2024-04-25T01:41:43.090998+0000","last_became_active":"2024-04-25T01:26:02.426552+0000","last_became_peered":"2024-04-25T01:26:02.426552+0000","last_unstale":"2024-04-25T01:41:43.090998+0000","last_undegraded":"2024-04-25T01:41:43.090998+0000","last_fullsized":"2024-04-25T01:41:43.090998+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4125","last_scrub_stamp":"2024-04-25T01:41:43.090852+0000","last_deep_scrub":"26'4125","last_deep_scrub_stamp":"2024-04-25T01:41:43.090852+0000","last_clean_scrub_stamp":"2024-04-25T01:41:43.090852+0000","objects_scrubbed":0,"log_size":4125,"log_dups_size":0,"ondisk_log_size":4125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:16.985403+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21785,"num_write_kb":263458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"26'5101","reported_seq":5141,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:23.621658+0000","last_change":"2024-04-25T01:41:23.621684+0000","last_active":"2024-04-25T01:41:23.621658+0000","last_peered":"2024-04-25T01:41:23.621658+0000","last_clean":"2024-04-25T01:41:23.621658+0000","last_became_active":"2024-04-25T01:25:53.368176+0000","last_became_peered":"2024-04-25T01:25:53.368176+0000","last_unstale":"2024-04-25T01:41:23.621658+0000","last_undegraded":"2024-04-25T01:41:23.621658+0000","last_fullsized":"2024-04-25T01:41:23.621658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5101","last_scrub_stamp":"2024-04-25T01:41:23.621548+0000","last_deep_scrub":"26'5101","last_deep_scrub_stamp":"2024-04-25T01:41:23.621548+0000","last_clean_scrub_stamp":"2024-04-25T01:41:23.621548+0000","objects_scrubbed":0,"log_size":5101,"log_dups_size":0,"ondisk_log_size":5101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:30.364398+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26850,"num_write_kb":324721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"26'6020","reported_seq":6052,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:45.732455+0000","last_change":"2024-04-25T01:41:45.732481+0000","last_active":"2024-04-25T01:41:45.732455+0000","last_peered":"2024-04-25T01:41:45.732455+0000","last_clean":"2024-04-25T01:41:45.732455+0000","last_became_active":"2024-04-25T01:26:02.424903+0000","last_became_peered":"2024-04-25T01:26:02.424903+0000","last_unstale":"2024-04-25T01:41:45.732455+0000","last_undegraded":"2024-04-25T01:41:45.732455+0000","last_fullsized":"2024-04-25T01:41:45.732455+0000","mapping_epoch":22,"log_start":"26'300","ondisk_log_start":"26'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'6020","last_scrub_stamp":"2024-04-25T01:41:45.732343+0000","last_deep_scrub":"26'6020","last_deep_scrub_stamp":"2024-04-25T01:41:45.732343+0000","last_clean_scrub_stamp":"2024-04-25T01:41:45.732343+0000","objects_scrubbed":0,"log_size":5720,"log_dups_size":0,"ondisk_log_size":5720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:02.972594+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31332,"num_write_kb":379614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"26'4058","reported_seq":4098,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:21.338130+0000","last_change":"2024-04-25T01:41:21.338175+0000","last_active":"2024-04-25T01:41:21.338130+0000","last_peered":"2024-04-25T01:41:21.338130+0000","last_clean":"2024-04-25T01:41:21.338130+0000","last_became_active":"2024-04-25T01:25:53.372001+0000","last_became_peered":"2024-04-25T01:25:53.372001+0000","last_unstale":"2024-04-25T01:41:21.338130+0000","last_undegraded":"2024-04-25T01:41:21.338130+0000","last_fullsized":"2024-04-25T01:41:21.338130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4058","last_scrub_stamp":"2024-04-25T01:41:21.338005+0000","last_deep_scrub":"26'4058","last_deep_scrub_stamp":"2024-04-25T01:41:21.338005+0000","last_clean_scrub_stamp":"2024-04-25T01:41:21.338005+0000","objects_scrubbed":0,"log_size":4058,"log_dups_size":0,"ondisk_log_size":4058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:50.935614+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20502,"num_write_kb":266785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"26'4843","reported_seq":4875,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:40.214420+0000","last_change":"2024-04-25T01:41:40.214448+0000","last_active":"2024-04-25T01:41:40.214420+0000","last_peered":"2024-04-25T01:41:40.214420+0000","last_clean":"2024-04-25T01:41:40.214420+0000","last_became_active":"2024-04-25T01:26:02.420621+0000","last_became_peered":"2024-04-25T01:26:02.420621+0000","last_unstale":"2024-04-25T01:41:40.214420+0000","last_undegraded":"2024-04-25T01:41:40.214420+0000","last_fullsized":"2024-04-25T01:41:40.214420+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4843","last_scrub_stamp":"2024-04-25T01:41:40.214322+0000","last_deep_scrub":"26'4843","last_deep_scrub_stamp":"2024-04-25T01:41:40.214322+0000","last_clean_scrub_stamp":"2024-04-25T01:41:40.214322+0000","objects_scrubbed":0,"log_size":4843,"log_dups_size":0,"ondisk_log_size":4843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:52.985990+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24777,"num_write_kb":316390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"26'5268","reported_seq":5308,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:27.919155+0000","last_change":"2024-04-25T01:41:27.919209+0000","last_active":"2024-04-25T01:41:27.919155+0000","last_peered":"2024-04-25T01:41:27.919155+0000","last_clean":"2024-04-25T01:41:27.919155+0000","last_became_active":"2024-04-25T01:25:53.365245+0000","last_became_peered":"2024-04-25T01:25:53.365245+0000","last_unstale":"2024-04-25T01:41:27.919155+0000","last_undegraded":"2024-04-25T01:41:27.919155+0000","last_fullsized":"2024-04-25T01:41:27.919155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5268","last_scrub_stamp":"2024-04-25T01:41:27.919032+0000","last_deep_scrub":"26'5268","last_deep_scrub_stamp":"2024-04-25T01:41:27.919032+0000","last_clean_scrub_stamp":"2024-04-25T01:41:27.919032+0000","objects_scrubbed":0,"log_size":5268,"log_dups_size":0,"ondisk_log_size":5268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:36:37.642225+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26981,"num_write_kb":345791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"26'3558","reported_seq":3589,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:43.995142+0000","last_change":"2024-04-25T01:41:43.995169+0000","last_active":"2024-04-25T01:41:43.995142+0000","last_peered":"2024-04-25T01:41:43.995142+0000","last_clean":"2024-04-25T01:41:43.995142+0000","last_became_active":"2024-04-25T01:26:02.415389+0000","last_became_peered":"2024-04-25T01:26:02.415389+0000","last_unstale":"2024-04-25T01:41:43.995142+0000","last_undegraded":"2024-04-25T01:41:43.995142+0000","last_fullsized":"2024-04-25T01:41:43.995142+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3558","last_scrub_stamp":"2024-04-25T01:41:43.995029+0000","last_deep_scrub":"26'3558","last_deep_scrub_stamp":"2024-04-25T01:41:43.995029+0000","last_clean_scrub_stamp":"2024-04-25T01:41:43.995029+0000","objects_scrubbed":0,"log_size":3558,"log_dups_size":0,"ondisk_log_size":3558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:35.073400+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18097,"num_write_kb":236200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"26'4040","reported_seq":4080,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:37.342421+0000","last_change":"2024-04-25T01:41:37.342446+0000","last_active":"2024-04-25T01:41:37.342421+0000","last_peered":"2024-04-25T01:41:37.342421+0000","last_clean":"2024-04-25T01:41:37.342421+0000","last_became_active":"2024-04-25T01:25:53.364829+0000","last_became_peered":"2024-04-25T01:25:53.364829+0000","last_unstale":"2024-04-25T01:41:37.342421+0000","last_undegraded":"2024-04-25T01:41:37.342421+0000","last_fullsized":"2024-04-25T01:41:37.342421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4040","last_scrub_stamp":"2024-04-25T01:41:37.342321+0000","last_deep_scrub":"26'4040","last_deep_scrub_stamp":"2024-04-25T01:41:37.342321+0000","last_clean_scrub_stamp":"2024-04-25T01:41:37.342321+0000","objects_scrubbed":0,"log_size":4040,"log_dups_size":0,"ondisk_log_size":4040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:09.885519+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20607,"num_write_kb":262822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"26'5431","reported_seq":5462,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:32.850306+0000","last_change":"2024-04-25T01:41:32.850337+0000","last_active":"2024-04-25T01:41:32.850306+0000","last_peered":"2024-04-25T01:41:32.850306+0000","last_clean":"2024-04-25T01:41:32.850306+0000","last_became_active":"2024-04-25T01:26:02.418036+0000","last_became_peered":"2024-04-25T01:26:02.418036+0000","last_unstale":"2024-04-25T01:41:32.850306+0000","last_undegraded":"2024-04-25T01:41:32.850306+0000","last_fullsized":"2024-04-25T01:41:32.850306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5431","last_scrub_stamp":"2024-04-25T01:41:32.850204+0000","last_deep_scrub":"26'5431","last_deep_scrub_stamp":"2024-04-25T01:41:32.850204+0000","last_clean_scrub_stamp":"2024-04-25T01:41:32.850204+0000","objects_scrubbed":0,"log_size":5431,"log_dups_size":0,"ondisk_log_size":5431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:59.931538+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28059,"num_write_kb":358266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"26'4709","reported_seq":4749,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:37.962371+0000","last_change":"2024-04-25T01:41:37.962371+0000","last_active":"2024-04-25T01:41:37.962371+0000","last_peered":"2024-04-25T01:41:37.962371+0000","last_clean":"2024-04-25T01:41:37.962371+0000","last_became_active":"2024-04-25T01:25:53.355417+0000","last_became_peered":"2024-04-25T01:25:53.355417+0000","last_unstale":"2024-04-25T01:41:37.962371+0000","last_undegraded":"2024-04-25T01:41:37.962371+0000","last_fullsized":"2024-04-25T01:41:37.962371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4709","last_scrub_stamp":"2024-04-25T01:41:37.962211+0000","last_deep_scrub":"26'4709","last_deep_scrub_stamp":"2024-04-25T01:41:37.962211+0000","last_clean_scrub_stamp":"2024-04-25T01:41:37.962211+0000","objects_scrubbed":0,"log_size":4709,"log_dups_size":0,"ondisk_log_size":4709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:11.856762+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24394,"num_write_kb":298399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"26'4741","reported_seq":4773,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:30.129837+0000","last_change":"2024-04-25T01:41:30.129837+0000","last_active":"2024-04-25T01:41:30.129837+0000","last_peered":"2024-04-25T01:41:30.129837+0000","last_clean":"2024-04-25T01:41:30.129837+0000","last_became_active":"2024-04-25T01:26:02.414735+0000","last_became_peered":"2024-04-25T01:26:02.414735+0000","last_unstale":"2024-04-25T01:41:30.129837+0000","last_undegraded":"2024-04-25T01:41:30.129837+0000","last_fullsized":"2024-04-25T01:41:30.129837+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4741","last_scrub_stamp":"2024-04-25T01:41:30.129681+0000","last_deep_scrub":"26'4741","last_deep_scrub_stamp":"2024-04-25T01:41:30.129681+0000","last_clean_scrub_stamp":"2024-04-25T01:41:30.129681+0000","objects_scrubbed":0,"log_size":4741,"log_dups_size":0,"ondisk_log_size":4741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:08.104652+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24824,"num_write_kb":300285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"26'5111","reported_seq":5151,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:25.401696+0000","last_change":"2024-04-25T01:41:25.401696+0000","last_active":"2024-04-25T01:41:25.401696+0000","last_peered":"2024-04-25T01:41:25.401696+0000","last_clean":"2024-04-25T01:41:25.401696+0000","last_became_active":"2024-04-25T01:25:53.370164+0000","last_became_peered":"2024-04-25T01:25:53.370164+0000","last_unstale":"2024-04-25T01:41:25.401696+0000","last_undegraded":"2024-04-25T01:41:25.401696+0000","last_fullsized":"2024-04-25T01:41:25.401696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5111","last_scrub_stamp":"2024-04-25T01:41:25.401560+0000","last_deep_scrub":"26'5111","last_deep_scrub_stamp":"2024-04-25T01:41:25.401560+0000","last_clean_scrub_stamp":"2024-04-25T01:41:25.401560+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:11.935349+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26542,"num_write_kb":327744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"26'4367","reported_seq":4399,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:45.942503+0000","last_change":"2024-04-25T01:41:45.942503+0000","last_active":"2024-04-25T01:41:45.942503+0000","last_peered":"2024-04-25T01:41:45.942503+0000","last_clean":"2024-04-25T01:41:45.942503+0000","last_became_active":"2024-04-25T01:26:02.417977+0000","last_became_peered":"2024-04-25T01:26:02.417977+0000","last_unstale":"2024-04-25T01:41:45.942503+0000","last_undegraded":"2024-04-25T01:41:45.942503+0000","last_fullsized":"2024-04-25T01:41:45.942503+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4367","last_scrub_stamp":"2024-04-25T01:41:45.942357+0000","last_deep_scrub":"26'4367","last_deep_scrub_stamp":"2024-04-25T01:41:45.942357+0000","last_clean_scrub_stamp":"2024-04-25T01:41:45.942357+0000","objects_scrubbed":0,"log_size":4367,"log_dups_size":0,"ondisk_log_size":4367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:06.700895+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22024,"num_write_kb":293008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"26'5285","reported_seq":5325,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:19.722283+0000","last_change":"2024-04-25T01:41:19.722283+0000","last_active":"2024-04-25T01:41:19.722283+0000","last_peered":"2024-04-25T01:41:19.722283+0000","last_clean":"2024-04-25T01:41:19.722283+0000","last_became_active":"2024-04-25T01:25:53.376104+0000","last_became_peered":"2024-04-25T01:25:53.376104+0000","last_unstale":"2024-04-25T01:41:19.722283+0000","last_undegraded":"2024-04-25T01:41:19.722283+0000","last_fullsized":"2024-04-25T01:41:19.722283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5285","last_scrub_stamp":"2024-04-25T01:41:19.722126+0000","last_deep_scrub":"26'5285","last_deep_scrub_stamp":"2024-04-25T01:41:19.722126+0000","last_clean_scrub_stamp":"2024-04-25T01:41:19.722126+0000","objects_scrubbed":0,"log_size":5285,"log_dups_size":0,"ondisk_log_size":5285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:49.319736+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27331,"num_write_kb":318531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"26'5020","reported_seq":5052,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:42.097114+0000","last_change":"2024-04-25T01:41:42.097114+0000","last_active":"2024-04-25T01:41:42.097114+0000","last_peered":"2024-04-25T01:41:42.097114+0000","last_clean":"2024-04-25T01:41:42.097114+0000","last_became_active":"2024-04-25T01:26:02.426069+0000","last_became_peered":"2024-04-25T01:26:02.426069+0000","last_unstale":"2024-04-25T01:41:42.097114+0000","last_undegraded":"2024-04-25T01:41:42.097114+0000","last_fullsized":"2024-04-25T01:41:42.097114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5020","last_scrub_stamp":"2024-04-25T01:41:42.096975+0000","last_deep_scrub":"26'5020","last_deep_scrub_stamp":"2024-04-25T01:41:42.096975+0000","last_clean_scrub_stamp":"2024-04-25T01:41:42.096975+0000","objects_scrubbed":0,"log_size":5020,"log_dups_size":0,"ondisk_log_size":5020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:27.238041+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26121,"num_write_kb":325976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"26'4910","reported_seq":4950,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:32.299733+0000","last_change":"2024-04-25T01:41:32.299733+0000","last_active":"2024-04-25T01:41:32.299733+0000","last_peered":"2024-04-25T01:41:32.299733+0000","last_clean":"2024-04-25T01:41:32.299733+0000","last_became_active":"2024-04-25T01:25:53.370049+0000","last_became_peered":"2024-04-25T01:25:53.370049+0000","last_unstale":"2024-04-25T01:41:32.299733+0000","last_undegraded":"2024-04-25T01:41:32.299733+0000","last_fullsized":"2024-04-25T01:41:32.299733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4910","last_scrub_stamp":"2024-04-25T01:41:32.299587+0000","last_deep_scrub":"26'4910","last_deep_scrub_stamp":"2024-04-25T01:41:32.299587+0000","last_clean_scrub_stamp":"2024-04-25T01:41:32.299587+0000","objects_scrubbed":0,"log_size":4910,"log_dups_size":0,"ondisk_log_size":4910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:06.590550+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25833,"num_write_kb":304449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"26'4775","reported_seq":4807,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:37.094529+0000","last_change":"2024-04-25T01:41:37.094529+0000","last_active":"2024-04-25T01:41:37.094529+0000","last_peered":"2024-04-25T01:41:37.094529+0000","last_clean":"2024-04-25T01:41:37.094529+0000","last_became_active":"2024-04-25T01:26:02.415173+0000","last_became_peered":"2024-04-25T01:26:02.415173+0000","last_unstale":"2024-04-25T01:41:37.094529+0000","last_undegraded":"2024-04-25T01:41:37.094529+0000","last_fullsized":"2024-04-25T01:41:37.094529+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4775","last_scrub_stamp":"2024-04-25T01:41:37.094377+0000","last_deep_scrub":"26'4775","last_deep_scrub_stamp":"2024-04-25T01:41:37.094377+0000","last_clean_scrub_stamp":"2024-04-25T01:41:37.094377+0000","objects_scrubbed":0,"log_size":4775,"log_dups_size":0,"ondisk_log_size":4775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:34.147067+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24075,"num_write_kb":305295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"26'4891","reported_seq":4931,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:23.972816+0000","last_change":"2024-04-25T01:41:23.972816+0000","last_active":"2024-04-25T01:41:23.972816+0000","last_peered":"2024-04-25T01:41:23.972816+0000","last_clean":"2024-04-25T01:41:23.972816+0000","last_became_active":"2024-04-25T01:25:53.368499+0000","last_became_peered":"2024-04-25T01:25:53.368499+0000","last_unstale":"2024-04-25T01:41:23.972816+0000","last_undegraded":"2024-04-25T01:41:23.972816+0000","last_fullsized":"2024-04-25T01:41:23.972816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4891","last_scrub_stamp":"2024-04-25T01:41:23.972677+0000","last_deep_scrub":"26'4891","last_deep_scrub_stamp":"2024-04-25T01:41:23.972677+0000","last_clean_scrub_stamp":"2024-04-25T01:41:23.972677+0000","objects_scrubbed":0,"log_size":4891,"log_dups_size":0,"ondisk_log_size":4891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:34.356335+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24867,"num_write_kb":304528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"26'4199","reported_seq":4231,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:42.242592+0000","last_change":"2024-04-25T01:41:42.242661+0000","last_active":"2024-04-25T01:41:42.242592+0000","last_peered":"2024-04-25T01:41:42.242592+0000","last_clean":"2024-04-25T01:41:42.242592+0000","last_became_active":"2024-04-25T01:26:02.421698+0000","last_became_peered":"2024-04-25T01:26:02.421698+0000","last_unstale":"2024-04-25T01:41:42.242592+0000","last_undegraded":"2024-04-25T01:41:42.242592+0000","last_fullsized":"2024-04-25T01:41:42.242592+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4199","last_scrub_stamp":"2024-04-25T01:41:42.242443+0000","last_deep_scrub":"26'4199","last_deep_scrub_stamp":"2024-04-25T01:41:42.242443+0000","last_clean_scrub_stamp":"2024-04-25T01:41:42.242443+0000","objects_scrubbed":0,"log_size":4199,"log_dups_size":0,"ondisk_log_size":4199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:26.171478+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21738,"num_write_kb":267270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"26'3844","reported_seq":3883,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:26.095640+0000","last_change":"2024-04-25T01:41:26.095685+0000","last_active":"2024-04-25T01:41:26.095640+0000","last_peered":"2024-04-25T01:41:26.095640+0000","last_clean":"2024-04-25T01:41:26.095640+0000","last_became_active":"2024-04-25T01:25:53.369733+0000","last_became_peered":"2024-04-25T01:25:53.369733+0000","last_unstale":"2024-04-25T01:41:26.095640+0000","last_undegraded":"2024-04-25T01:41:26.095640+0000","last_fullsized":"2024-04-25T01:41:26.095640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3844","last_scrub_stamp":"2024-04-25T01:41:26.095528+0000","last_deep_scrub":"26'3844","last_deep_scrub_stamp":"2024-04-25T01:41:26.095528+0000","last_clean_scrub_stamp":"2024-04-25T01:41:26.095528+0000","objects_scrubbed":0,"log_size":3844,"log_dups_size":0,"ondisk_log_size":3844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:32.635635+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19367,"num_write_kb":241344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"26'4316","reported_seq":4348,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:36.748806+0000","last_change":"2024-04-25T01:41:36.748806+0000","last_active":"2024-04-25T01:41:36.748806+0000","last_peered":"2024-04-25T01:41:36.748806+0000","last_clean":"2024-04-25T01:41:36.748806+0000","last_became_active":"2024-04-25T01:26:02.417810+0000","last_became_peered":"2024-04-25T01:26:02.417810+0000","last_unstale":"2024-04-25T01:41:36.748806+0000","last_undegraded":"2024-04-25T01:41:36.748806+0000","last_fullsized":"2024-04-25T01:41:36.748806+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4316","last_scrub_stamp":"2024-04-25T01:41:36.748678+0000","last_deep_scrub":"26'4316","last_deep_scrub_stamp":"2024-04-25T01:41:36.748678+0000","last_clean_scrub_stamp":"2024-04-25T01:41:36.748678+0000","objects_scrubbed":0,"log_size":4316,"log_dups_size":0,"ondisk_log_size":4316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:05.505304+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22517,"num_write_kb":290939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"26'5814","reported_seq":5839,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:35.060358+0000","last_change":"2024-04-25T01:41:35.060383+0000","last_active":"2024-04-25T01:41:35.060358+0000","last_peered":"2024-04-25T01:41:35.060358+0000","last_clean":"2024-04-25T01:41:35.060358+0000","last_became_active":"2024-04-25T01:26:52.120588+0000","last_became_peered":"2024-04-25T01:26:52.120588+0000","last_unstale":"2024-04-25T01:41:35.060358+0000","last_undegraded":"2024-04-25T01:41:35.060358+0000","last_fullsized":"2024-04-25T01:41:35.060358+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5814","last_scrub_stamp":"2024-04-25T01:41:35.060229+0000","last_deep_scrub":"26'5814","last_deep_scrub_stamp":"2024-04-25T01:41:35.060229+0000","last_clean_scrub_stamp":"2024-04-25T01:41:35.060229+0000","objects_scrubbed":0,"log_size":5814,"log_dups_size":0,"ondisk_log_size":5814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:21.594018+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29929,"num_write_kb":365779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.36","version":"26'4743","reported_seq":4774,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:43.712265+0000","last_change":"2024-04-25T01:41:43.712293+0000","last_active":"2024-04-25T01:41:43.712265+0000","last_peered":"2024-04-25T01:41:43.712265+0000","last_clean":"2024-04-25T01:41:43.712265+0000","last_became_active":"2024-04-25T01:26:02.426015+0000","last_became_peered":"2024-04-25T01:26:02.426015+0000","last_unstale":"2024-04-25T01:41:43.712265+0000","last_undegraded":"2024-04-25T01:41:43.712265+0000","last_fullsized":"2024-04-25T01:41:43.712265+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4743","last_scrub_stamp":"2024-04-25T01:41:43.712158+0000","last_deep_scrub":"26'4743","last_deep_scrub_stamp":"2024-04-25T01:41:43.712158+0000","last_clean_scrub_stamp":"2024-04-25T01:41:43.712158+0000","objects_scrubbed":0,"log_size":4743,"log_dups_size":0,"ondisk_log_size":4743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:19.841334+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25470,"num_write_kb":331505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"26'4366","reported_seq":4405,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:22.334603+0000","last_change":"2024-04-25T01:41:22.334638+0000","last_active":"2024-04-25T01:41:22.334603+0000","last_peered":"2024-04-25T01:41:22.334603+0000","last_clean":"2024-04-25T01:41:22.334603+0000","last_became_active":"2024-04-25T01:25:53.373766+0000","last_became_peered":"2024-04-25T01:25:53.373766+0000","last_unstale":"2024-04-25T01:41:22.334603+0000","last_undegraded":"2024-04-25T01:41:22.334603+0000","last_fullsized":"2024-04-25T01:41:22.334603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4366","last_scrub_stamp":"2024-04-25T01:41:22.334478+0000","last_deep_scrub":"26'4366","last_deep_scrub_stamp":"2024-04-25T01:41:22.334478+0000","last_clean_scrub_stamp":"2024-04-25T01:41:22.334478+0000","objects_scrubbed":0,"log_size":4366,"log_dups_size":0,"ondisk_log_size":4366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:29.077328+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23328,"num_write_kb":273725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"26'5875","reported_seq":5906,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:38.132735+0000","last_change":"2024-04-25T01:41:38.132787+0000","last_active":"2024-04-25T01:41:38.132735+0000","last_peered":"2024-04-25T01:41:38.132735+0000","last_clean":"2024-04-25T01:41:38.132735+0000","last_became_active":"2024-04-25T01:26:02.416082+0000","last_became_peered":"2024-04-25T01:26:02.416082+0000","last_unstale":"2024-04-25T01:41:38.132735+0000","last_undegraded":"2024-04-25T01:41:38.132735+0000","last_fullsized":"2024-04-25T01:41:38.132735+0000","mapping_epoch":22,"log_start":"26'400","ondisk_log_start":"26'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5875","last_scrub_stamp":"2024-04-25T01:41:38.132610+0000","last_deep_scrub":"26'5875","last_deep_scrub_stamp":"2024-04-25T01:41:38.132610+0000","last_clean_scrub_stamp":"2024-04-25T01:41:38.132610+0000","objects_scrubbed":0,"log_size":5475,"log_dups_size":0,"ondisk_log_size":5475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:56.631217+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31009,"num_write_kb":370059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"26'4898","reported_seq":4937,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:21.678232+0000","last_change":"2024-04-25T01:41:21.678273+0000","last_active":"2024-04-25T01:41:21.678232+0000","last_peered":"2024-04-25T01:41:21.678232+0000","last_clean":"2024-04-25T01:41:21.678232+0000","last_became_active":"2024-04-25T01:25:53.371362+0000","last_became_peered":"2024-04-25T01:25:53.371362+0000","last_unstale":"2024-04-25T01:41:21.678232+0000","last_undegraded":"2024-04-25T01:41:21.678232+0000","last_fullsized":"2024-04-25T01:41:21.678232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4898","last_scrub_stamp":"2024-04-25T01:41:21.678132+0000","last_deep_scrub":"26'4898","last_deep_scrub_stamp":"2024-04-25T01:41:21.678132+0000","last_clean_scrub_stamp":"2024-04-25T01:41:21.678132+0000","objects_scrubbed":0,"log_size":4898,"log_dups_size":0,"ondisk_log_size":4898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:02.628027+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24919,"num_write_kb":308746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"26'3961","reported_seq":3992,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:47.069973+0000","last_change":"2024-04-25T01:41:47.070000+0000","last_active":"2024-04-25T01:41:47.069973+0000","last_peered":"2024-04-25T01:41:47.069973+0000","last_clean":"2024-04-25T01:41:47.069973+0000","last_became_active":"2024-04-25T01:26:02.425577+0000","last_became_peered":"2024-04-25T01:26:02.425577+0000","last_unstale":"2024-04-25T01:41:47.069973+0000","last_undegraded":"2024-04-25T01:41:47.069973+0000","last_fullsized":"2024-04-25T01:41:47.069973+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3961","last_scrub_stamp":"2024-04-25T01:41:47.069865+0000","last_deep_scrub":"26'3961","last_deep_scrub_stamp":"2024-04-25T01:41:47.069865+0000","last_clean_scrub_stamp":"2024-04-25T01:41:47.069865+0000","objects_scrubbed":0,"log_size":3961,"log_dups_size":0,"ondisk_log_size":3961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:07.828403+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20296,"num_write_kb":253395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"26'4670","reported_seq":4709,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:22.696731+0000","last_change":"2024-04-25T01:41:22.696754+0000","last_active":"2024-04-25T01:41:22.696731+0000","last_peered":"2024-04-25T01:41:22.696731+0000","last_clean":"2024-04-25T01:41:22.696731+0000","last_became_active":"2024-04-25T01:25:53.370479+0000","last_became_peered":"2024-04-25T01:25:53.370479+0000","last_unstale":"2024-04-25T01:41:22.696731+0000","last_undegraded":"2024-04-25T01:41:22.696731+0000","last_fullsized":"2024-04-25T01:41:22.696731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4670","last_scrub_stamp":"2024-04-25T01:41:22.696638+0000","last_deep_scrub":"26'4670","last_deep_scrub_stamp":"2024-04-25T01:41:22.696638+0000","last_clean_scrub_stamp":"2024-04-25T01:41:22.696638+0000","objects_scrubbed":0,"log_size":4670,"log_dups_size":0,"ondisk_log_size":4670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:25.738779+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24421,"num_write_kb":306681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"26'4733","reported_seq":4764,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:44.773434+0000","last_change":"2024-04-25T01:41:44.773462+0000","last_active":"2024-04-25T01:41:44.773434+0000","last_peered":"2024-04-25T01:41:44.773434+0000","last_clean":"2024-04-25T01:41:44.773434+0000","last_became_active":"2024-04-25T01:26:02.423146+0000","last_became_peered":"2024-04-25T01:26:02.423146+0000","last_unstale":"2024-04-25T01:41:44.773434+0000","last_undegraded":"2024-04-25T01:41:44.773434+0000","last_fullsized":"2024-04-25T01:41:44.773434+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4733","last_scrub_stamp":"2024-04-25T01:41:44.773325+0000","last_deep_scrub":"26'4733","last_deep_scrub_stamp":"2024-04-25T01:41:44.773325+0000","last_clean_scrub_stamp":"2024-04-25T01:41:44.773325+0000","objects_scrubbed":0,"log_size":4733,"log_dups_size":0,"ondisk_log_size":4733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:49.659107+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23928,"num_write_kb":292515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"26'4513","reported_seq":4552,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:40.928593+0000","last_change":"2024-04-25T01:41:40.928621+0000","last_active":"2024-04-25T01:41:40.928593+0000","last_peered":"2024-04-25T01:41:40.928593+0000","last_clean":"2024-04-25T01:41:40.928593+0000","last_became_active":"2024-04-25T01:25:53.363246+0000","last_became_peered":"2024-04-25T01:25:53.363246+0000","last_unstale":"2024-04-25T01:41:40.928593+0000","last_undegraded":"2024-04-25T01:41:40.928593+0000","last_fullsized":"2024-04-25T01:41:40.928593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4513","last_scrub_stamp":"2024-04-25T01:41:40.928468+0000","last_deep_scrub":"26'4513","last_deep_scrub_stamp":"2024-04-25T01:41:40.928468+0000","last_clean_scrub_stamp":"2024-04-25T01:41:40.928468+0000","objects_scrubbed":0,"log_size":4513,"log_dups_size":0,"ondisk_log_size":4513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:17.057645+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23705,"num_write_kb":279339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"26'4587","reported_seq":4618,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:30.948056+0000","last_change":"2024-04-25T01:41:30.948083+0000","last_active":"2024-04-25T01:41:30.948056+0000","last_peered":"2024-04-25T01:41:30.948056+0000","last_clean":"2024-04-25T01:41:30.948056+0000","last_became_active":"2024-04-25T01:26:02.413036+0000","last_became_peered":"2024-04-25T01:26:02.413036+0000","last_unstale":"2024-04-25T01:41:30.948056+0000","last_undegraded":"2024-04-25T01:41:30.948056+0000","last_fullsized":"2024-04-25T01:41:30.948056+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4587","last_scrub_stamp":"2024-04-25T01:41:30.947949+0000","last_deep_scrub":"26'4587","last_deep_scrub_stamp":"2024-04-25T01:41:30.947949+0000","last_clean_scrub_stamp":"2024-04-25T01:41:30.947949+0000","objects_scrubbed":0,"log_size":4587,"log_dups_size":0,"ondisk_log_size":4587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:23:08.496641+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24042,"num_write_kb":305325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"26'3998","reported_seq":4037,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:31.189011+0000","last_change":"2024-04-25T01:41:31.189037+0000","last_active":"2024-04-25T01:41:31.189011+0000","last_peered":"2024-04-25T01:41:31.189011+0000","last_clean":"2024-04-25T01:41:31.189011+0000","last_became_active":"2024-04-25T01:25:53.364103+0000","last_became_peered":"2024-04-25T01:25:53.364103+0000","last_unstale":"2024-04-25T01:41:31.189011+0000","last_undegraded":"2024-04-25T01:41:31.189011+0000","last_fullsized":"2024-04-25T01:41:31.189011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3998","last_scrub_stamp":"2024-04-25T01:41:31.188903+0000","last_deep_scrub":"26'3998","last_deep_scrub_stamp":"2024-04-25T01:41:31.188903+0000","last_clean_scrub_stamp":"2024-04-25T01:41:31.188903+0000","objects_scrubbed":0,"log_size":3998,"log_dups_size":0,"ondisk_log_size":3998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:30.373524+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20343,"num_write_kb":253659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"26'4194","reported_seq":4225,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:33.143247+0000","last_change":"2024-04-25T01:41:33.143274+0000","last_active":"2024-04-25T01:41:33.143247+0000","last_peered":"2024-04-25T01:41:33.143247+0000","last_clean":"2024-04-25T01:41:33.143247+0000","last_became_active":"2024-04-25T01:26:02.416942+0000","last_became_peered":"2024-04-25T01:26:02.416942+0000","last_unstale":"2024-04-25T01:41:33.143247+0000","last_undegraded":"2024-04-25T01:41:33.143247+0000","last_fullsized":"2024-04-25T01:41:33.143247+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4194","last_scrub_stamp":"2024-04-25T01:41:33.143135+0000","last_deep_scrub":"26'4194","last_deep_scrub_stamp":"2024-04-25T01:41:33.143135+0000","last_clean_scrub_stamp":"2024-04-25T01:41:33.143135+0000","objects_scrubbed":0,"log_size":4194,"log_dups_size":0,"ondisk_log_size":4194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.460297+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21868,"num_write_kb":272015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"26'4800","reported_seq":4839,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:24.333874+0000","last_change":"2024-04-25T01:41:24.333902+0000","last_active":"2024-04-25T01:41:24.333874+0000","last_peered":"2024-04-25T01:41:24.333874+0000","last_clean":"2024-04-25T01:41:24.333874+0000","last_became_active":"2024-04-25T01:25:53.371353+0000","last_became_peered":"2024-04-25T01:25:53.371353+0000","last_unstale":"2024-04-25T01:41:24.333874+0000","last_undegraded":"2024-04-25T01:41:24.333874+0000","last_fullsized":"2024-04-25T01:41:24.333874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4800","last_scrub_stamp":"2024-04-25T01:41:24.333758+0000","last_deep_scrub":"26'4800","last_deep_scrub_stamp":"2024-04-25T01:41:24.333758+0000","last_clean_scrub_stamp":"2024-04-25T01:41:24.333758+0000","objects_scrubbed":0,"log_size":4800,"log_dups_size":0,"ondisk_log_size":4800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:30.873865+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24494,"num_write_kb":294847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"26'4139","reported_seq":4170,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:47.748784+0000","last_change":"2024-04-25T01:41:47.748812+0000","last_active":"2024-04-25T01:41:47.748784+0000","last_peered":"2024-04-25T01:41:47.748784+0000","last_clean":"2024-04-25T01:41:47.748784+0000","last_became_active":"2024-04-25T01:26:02.426024+0000","last_became_peered":"2024-04-25T01:26:02.426024+0000","last_unstale":"2024-04-25T01:41:47.748784+0000","last_undegraded":"2024-04-25T01:41:47.748784+0000","last_fullsized":"2024-04-25T01:41:47.748784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4139","last_scrub_stamp":"2024-04-25T01:41:47.748665+0000","last_deep_scrub":"26'4139","last_deep_scrub_stamp":"2024-04-25T01:41:47.748665+0000","last_clean_scrub_stamp":"2024-04-25T01:41:47.748665+0000","objects_scrubbed":0,"log_size":4139,"log_dups_size":0,"ondisk_log_size":4139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:57.264472+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21818,"num_write_kb":267665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"26'5798","reported_seq":5837,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:23.367199+0000","last_change":"2024-04-25T01:41:23.367224+0000","last_active":"2024-04-25T01:41:23.367199+0000","last_peered":"2024-04-25T01:41:23.367199+0000","last_clean":"2024-04-25T01:41:23.367199+0000","last_became_active":"2024-04-25T01:25:53.372580+0000","last_became_peered":"2024-04-25T01:25:53.372580+0000","last_unstale":"2024-04-25T01:41:23.367199+0000","last_undegraded":"2024-04-25T01:41:23.367199+0000","last_fullsized":"2024-04-25T01:41:23.367199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5798","last_scrub_stamp":"2024-04-25T01:41:23.367082+0000","last_deep_scrub":"26'5798","last_deep_scrub_stamp":"2024-04-25T01:41:23.367082+0000","last_clean_scrub_stamp":"2024-04-25T01:41:23.367082+0000","objects_scrubbed":0,"log_size":5798,"log_dups_size":0,"ondisk_log_size":5798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:33.352473+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30839,"num_write_kb":352989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"26'4687","reported_seq":4718,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:30.381675+0000","last_change":"2024-04-25T01:41:30.381700+0000","last_active":"2024-04-25T01:41:30.381675+0000","last_peered":"2024-04-25T01:41:30.381675+0000","last_clean":"2024-04-25T01:41:30.381675+0000","last_became_active":"2024-04-25T01:26:02.414388+0000","last_became_peered":"2024-04-25T01:26:02.414388+0000","last_unstale":"2024-04-25T01:41:30.381675+0000","last_undegraded":"2024-04-25T01:41:30.381675+0000","last_fullsized":"2024-04-25T01:41:30.381675+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4687","last_scrub_stamp":"2024-04-25T01:41:30.381573+0000","last_deep_scrub":"26'4687","last_deep_scrub_stamp":"2024-04-25T01:41:30.381573+0000","last_clean_scrub_stamp":"2024-04-25T01:41:30.381573+0000","objects_scrubbed":0,"log_size":4687,"log_dups_size":0,"ondisk_log_size":4687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:16.535112+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23866,"num_write_kb":304526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"26'5655","reported_seq":5699,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:49.733709+0000","last_change":"2024-04-25T01:41:49.733735+0000","last_active":"2024-04-25T01:41:49.733709+0000","last_peered":"2024-04-25T01:41:49.733709+0000","last_clean":"2024-04-25T01:41:49.733709+0000","last_became_active":"2024-04-25T01:26:52.515504+0000","last_became_peered":"2024-04-25T01:26:52.515504+0000","last_unstale":"2024-04-25T01:41:49.733709+0000","last_undegraded":"2024-04-25T01:41:49.733709+0000","last_fullsized":"2024-04-25T01:41:49.733709+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5655","last_scrub_stamp":"2024-04-25T01:41:49.733594+0000","last_deep_scrub":"26'5655","last_deep_scrub_stamp":"2024-04-25T01:41:49.733594+0000","last_clean_scrub_stamp":"2024-04-25T01:41:49.733594+0000","objects_scrubbed":0,"log_size":5655,"log_dups_size":0,"ondisk_log_size":5655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:14.289263+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29809,"num_write_kb":365295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"26'5769","reported_seq":5801,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:32.921279+0000","last_change":"2024-04-25T01:41:32.921307+0000","last_active":"2024-04-25T01:41:32.921279+0000","last_peered":"2024-04-25T01:41:32.921279+0000","last_clean":"2024-04-25T01:41:32.921279+0000","last_became_active":"2024-04-25T01:26:02.414061+0000","last_became_peered":"2024-04-25T01:26:02.414061+0000","last_unstale":"2024-04-25T01:41:32.921279+0000","last_undegraded":"2024-04-25T01:41:32.921279+0000","last_fullsized":"2024-04-25T01:41:32.921279+0000","mapping_epoch":22,"log_start":"26'500","ondisk_log_start":"26'500","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5769","last_scrub_stamp":"2024-04-25T01:41:32.921163+0000","last_deep_scrub":"26'5769","last_deep_scrub_stamp":"2024-04-25T01:41:32.921163+0000","last_clean_scrub_stamp":"2024-04-25T01:41:32.921163+0000","objects_scrubbed":0,"log_size":5269,"log_dups_size":0,"ondisk_log_size":5269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:09.345791+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30404,"num_write_kb":379952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"26'5055","reported_seq":5094,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:25.892289+0000","last_change":"2024-04-25T01:41:25.892319+0000","last_active":"2024-04-25T01:41:25.892289+0000","last_peered":"2024-04-25T01:41:25.892289+0000","last_clean":"2024-04-25T01:41:25.892289+0000","last_became_active":"2024-04-25T01:25:53.373848+0000","last_became_peered":"2024-04-25T01:25:53.373848+0000","last_unstale":"2024-04-25T01:41:25.892289+0000","last_undegraded":"2024-04-25T01:41:25.892289+0000","last_fullsized":"2024-04-25T01:41:25.892289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5055","last_scrub_stamp":"2024-04-25T01:41:25.892145+0000","last_deep_scrub":"26'5055","last_deep_scrub_stamp":"2024-04-25T01:41:25.892145+0000","last_clean_scrub_stamp":"2024-04-25T01:41:25.892145+0000","objects_scrubbed":0,"log_size":5055,"log_dups_size":0,"ondisk_log_size":5055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:25.076766+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25773,"num_write_kb":324544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"26'4195","reported_seq":4227,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:37.213535+0000","last_change":"2024-04-25T01:41:37.213563+0000","last_active":"2024-04-25T01:41:37.213535+0000","last_peered":"2024-04-25T01:41:37.213535+0000","last_clean":"2024-04-25T01:41:37.213535+0000","last_became_active":"2024-04-25T01:26:02.414326+0000","last_became_peered":"2024-04-25T01:26:02.414326+0000","last_unstale":"2024-04-25T01:41:37.213535+0000","last_undegraded":"2024-04-25T01:41:37.213535+0000","last_fullsized":"2024-04-25T01:41:37.213535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4195","last_scrub_stamp":"2024-04-25T01:41:37.213437+0000","last_deep_scrub":"26'4195","last_deep_scrub_stamp":"2024-04-25T01:41:37.213437+0000","last_clean_scrub_stamp":"2024-04-25T01:41:37.213437+0000","objects_scrubbed":0,"log_size":4195,"log_dups_size":0,"ondisk_log_size":4195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:20.324814+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21944,"num_write_kb":264053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"26'4639","reported_seq":4678,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:41.761782+0000","last_change":"2024-04-25T01:41:41.761809+0000","last_active":"2024-04-25T01:41:41.761782+0000","last_peered":"2024-04-25T01:41:41.761782+0000","last_clean":"2024-04-25T01:41:41.761782+0000","last_became_active":"2024-04-25T01:25:53.364435+0000","last_became_peered":"2024-04-25T01:25:53.364435+0000","last_unstale":"2024-04-25T01:41:41.761782+0000","last_undegraded":"2024-04-25T01:41:41.761782+0000","last_fullsized":"2024-04-25T01:41:41.761782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4639","last_scrub_stamp":"2024-04-25T01:41:41.761675+0000","last_deep_scrub":"26'4639","last_deep_scrub_stamp":"2024-04-25T01:41:41.761675+0000","last_clean_scrub_stamp":"2024-04-25T01:41:41.761675+0000","objects_scrubbed":0,"log_size":4639,"log_dups_size":0,"ondisk_log_size":4639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:09.453346+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24873,"num_write_kb":296467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"26'4393","reported_seq":4425,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:38.575045+0000","last_change":"2024-04-25T01:41:38.575073+0000","last_active":"2024-04-25T01:41:38.575045+0000","last_peered":"2024-04-25T01:41:38.575045+0000","last_clean":"2024-04-25T01:41:38.575045+0000","last_became_active":"2024-04-25T01:26:02.418057+0000","last_became_peered":"2024-04-25T01:26:02.418057+0000","last_unstale":"2024-04-25T01:41:38.575045+0000","last_undegraded":"2024-04-25T01:41:38.575045+0000","last_fullsized":"2024-04-25T01:41:38.575045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4393","last_scrub_stamp":"2024-04-25T01:41:38.574926+0000","last_deep_scrub":"26'4393","last_deep_scrub_stamp":"2024-04-25T01:41:38.574926+0000","last_clean_scrub_stamp":"2024-04-25T01:41:38.574926+0000","objects_scrubbed":0,"log_size":4393,"log_dups_size":0,"ondisk_log_size":4393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:35.474461+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22950,"num_write_kb":276425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"26'3744","reported_seq":3769,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:36.053728+0000","last_change":"2024-04-25T01:41:36.053728+0000","last_active":"2024-04-25T01:41:36.053728+0000","last_peered":"2024-04-25T01:41:36.053728+0000","last_clean":"2024-04-25T01:41:36.053728+0000","last_became_active":"2024-04-25T01:26:52.514260+0000","last_became_peered":"2024-04-25T01:26:52.514260+0000","last_unstale":"2024-04-25T01:41:36.053728+0000","last_undegraded":"2024-04-25T01:41:36.053728+0000","last_fullsized":"2024-04-25T01:41:36.053728+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3744","last_scrub_stamp":"2024-04-25T01:41:36.053594+0000","last_deep_scrub":"26'3744","last_deep_scrub_stamp":"2024-04-25T01:41:36.053594+0000","last_clean_scrub_stamp":"2024-04-25T01:41:36.053594+0000","objects_scrubbed":0,"log_size":3744,"log_dups_size":0,"ondisk_log_size":3744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:49.732601+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19457,"num_write_kb":254077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3d","version":"26'4696","reported_seq":4728,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:43.211315+0000","last_change":"2024-04-25T01:41:43.211342+0000","last_active":"2024-04-25T01:41:43.211315+0000","last_peered":"2024-04-25T01:41:43.211315+0000","last_clean":"2024-04-25T01:41:43.211315+0000","last_became_active":"2024-04-25T01:26:02.421848+0000","last_became_peered":"2024-04-25T01:26:02.421848+0000","last_unstale":"2024-04-25T01:41:43.211315+0000","last_undegraded":"2024-04-25T01:41:43.211315+0000","last_fullsized":"2024-04-25T01:41:43.211315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4696","last_scrub_stamp":"2024-04-25T01:41:43.211171+0000","last_deep_scrub":"26'4696","last_deep_scrub_stamp":"2024-04-25T01:41:43.211171+0000","last_clean_scrub_stamp":"2024-04-25T01:41:43.211171+0000","objects_scrubbed":0,"log_size":4696,"log_dups_size":0,"ondisk_log_size":4696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:48.096953+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24119,"num_write_kb":305642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"26'5605","reported_seq":5644,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:27.893977+0000","last_change":"2024-04-25T01:41:27.894010+0000","last_active":"2024-04-25T01:41:27.893977+0000","last_peered":"2024-04-25T01:41:27.893977+0000","last_clean":"2024-04-25T01:41:27.893977+0000","last_became_active":"2024-04-25T01:25:53.367358+0000","last_became_peered":"2024-04-25T01:25:53.367358+0000","last_unstale":"2024-04-25T01:41:27.893977+0000","last_undegraded":"2024-04-25T01:41:27.893977+0000","last_fullsized":"2024-04-25T01:41:27.893977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5605","last_scrub_stamp":"2024-04-25T01:41:27.893870+0000","last_deep_scrub":"26'5605","last_deep_scrub_stamp":"2024-04-25T01:41:27.893870+0000","last_clean_scrub_stamp":"2024-04-25T01:41:27.893870+0000","objects_scrubbed":0,"log_size":5605,"log_dups_size":0,"ondisk_log_size":5605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:57.480861+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27762,"num_write_kb":352787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"26'4305","reported_seq":4336,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:49.950018+0000","last_change":"2024-04-25T01:41:49.950047+0000","last_active":"2024-04-25T01:41:49.950018+0000","last_peered":"2024-04-25T01:41:49.950018+0000","last_clean":"2024-04-25T01:41:49.950018+0000","last_became_active":"2024-04-25T01:26:02.421702+0000","last_became_peered":"2024-04-25T01:26:02.421702+0000","last_unstale":"2024-04-25T01:41:49.950018+0000","last_undegraded":"2024-04-25T01:41:49.950018+0000","last_fullsized":"2024-04-25T01:41:49.950018+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4305","last_scrub_stamp":"2024-04-25T01:41:49.949917+0000","last_deep_scrub":"26'4305","last_deep_scrub_stamp":"2024-04-25T01:41:49.949917+0000","last_clean_scrub_stamp":"2024-04-25T01:41:49.949917+0000","objects_scrubbed":0,"log_size":4305,"log_dups_size":0,"ondisk_log_size":4305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:51.920635+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21701,"num_write_kb":263037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"26'4577","reported_seq":4618,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:21.383781+0000","last_change":"2024-04-25T01:41:21.383781+0000","last_active":"2024-04-25T01:41:21.383781+0000","last_peered":"2024-04-25T01:41:21.383781+0000","last_clean":"2024-04-25T01:41:21.383781+0000","last_became_active":"2024-04-25T01:25:53.370421+0000","last_became_peered":"2024-04-25T01:25:53.370421+0000","last_unstale":"2024-04-25T01:41:21.383781+0000","last_undegraded":"2024-04-25T01:41:21.383781+0000","last_fullsized":"2024-04-25T01:41:21.383781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4577","last_scrub_stamp":"2024-04-25T01:41:21.383639+0000","last_deep_scrub":"26'4577","last_deep_scrub_stamp":"2024-04-25T01:41:21.383639+0000","last_clean_scrub_stamp":"2024-04-25T01:41:21.383639+0000","objects_scrubbed":0,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:29.779703+0000","scrub_duration":42,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24195,"num_write_kb":281461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"26'5562","reported_seq":5593,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:20.984819+0000","last_change":"2024-04-25T01:41:20.984842+0000","last_active":"2024-04-25T01:41:20.984819+0000","last_peered":"2024-04-25T01:41:20.984819+0000","last_clean":"2024-04-25T01:41:20.984819+0000","last_became_active":"2024-04-25T01:26:02.411478+0000","last_became_peered":"2024-04-25T01:26:02.411478+0000","last_unstale":"2024-04-25T01:41:20.984819+0000","last_undegraded":"2024-04-25T01:41:20.984819+0000","last_fullsized":"2024-04-25T01:41:20.984819+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5562","last_scrub_stamp":"2024-04-25T01:41:20.984732+0000","last_deep_scrub":"26'5562","last_deep_scrub_stamp":"2024-04-25T01:41:20.984732+0000","last_clean_scrub_stamp":"2024-04-25T01:41:20.984732+0000","objects_scrubbed":0,"log_size":5562,"log_dups_size":0,"ondisk_log_size":5562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:39.437343+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27685,"num_write_kb":352571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"26'5001","reported_seq":5041,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:28.687834+0000","last_change":"2024-04-25T01:41:28.687859+0000","last_active":"2024-04-25T01:41:28.687834+0000","last_peered":"2024-04-25T01:41:28.687834+0000","last_clean":"2024-04-25T01:41:28.687834+0000","last_became_active":"2024-04-25T01:25:53.363525+0000","last_became_peered":"2024-04-25T01:25:53.363525+0000","last_unstale":"2024-04-25T01:41:28.687834+0000","last_undegraded":"2024-04-25T01:41:28.687834+0000","last_fullsized":"2024-04-25T01:41:28.687834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5001","last_scrub_stamp":"2024-04-25T01:41:28.687716+0000","last_deep_scrub":"26'5001","last_deep_scrub_stamp":"2024-04-25T01:41:28.687716+0000","last_clean_scrub_stamp":"2024-04-25T01:41:28.687716+0000","objects_scrubbed":0,"log_size":5001,"log_dups_size":0,"ondisk_log_size":5001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:39.071374+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25760,"num_write_kb":312110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"26'5076","reported_seq":5107,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:42.774594+0000","last_change":"2024-04-25T01:41:42.774630+0000","last_active":"2024-04-25T01:41:42.774594+0000","last_peered":"2024-04-25T01:41:42.774594+0000","last_clean":"2024-04-25T01:41:42.774594+0000","last_became_active":"2024-04-25T01:26:02.418796+0000","last_became_peered":"2024-04-25T01:26:02.418796+0000","last_unstale":"2024-04-25T01:41:42.774594+0000","last_undegraded":"2024-04-25T01:41:42.774594+0000","last_fullsized":"2024-04-25T01:41:42.774594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5076","last_scrub_stamp":"2024-04-25T01:41:42.774483+0000","last_deep_scrub":"26'5076","last_deep_scrub_stamp":"2024-04-25T01:41:42.774483+0000","last_clean_scrub_stamp":"2024-04-25T01:41:42.774483+0000","objects_scrubbed":0,"log_size":5076,"log_dups_size":0,"ondisk_log_size":5076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:43.592121+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25596,"num_write_kb":323780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"26'6025","reported_seq":6065,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:27.582035+0000","last_change":"2024-04-25T01:41:27.582081+0000","last_active":"2024-04-25T01:41:27.582035+0000","last_peered":"2024-04-25T01:41:27.582035+0000","last_clean":"2024-04-25T01:41:27.582035+0000","last_became_active":"2024-04-25T01:25:53.364793+0000","last_became_peered":"2024-04-25T01:25:53.364793+0000","last_unstale":"2024-04-25T01:41:27.582035+0000","last_undegraded":"2024-04-25T01:41:27.582035+0000","last_fullsized":"2024-04-25T01:41:27.582035+0000","mapping_epoch":18,"log_start":"26'200","ondisk_log_start":"26'200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'6025","last_scrub_stamp":"2024-04-25T01:41:27.581921+0000","last_deep_scrub":"26'6025","last_deep_scrub_stamp":"2024-04-25T01:41:27.581921+0000","last_clean_scrub_stamp":"2024-04-25T01:41:27.581921+0000","objects_scrubbed":0,"log_size":5825,"log_dups_size":0,"ondisk_log_size":5825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:57.168912+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31759,"num_write_kb":385508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"26'4493","reported_seq":4524,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:30.838689+0000","last_change":"2024-04-25T01:41:30.838716+0000","last_active":"2024-04-25T01:41:30.838689+0000","last_peered":"2024-04-25T01:41:30.838689+0000","last_clean":"2024-04-25T01:41:30.838689+0000","last_became_active":"2024-04-25T01:26:02.417350+0000","last_became_peered":"2024-04-25T01:26:02.417350+0000","last_unstale":"2024-04-25T01:41:30.838689+0000","last_undegraded":"2024-04-25T01:41:30.838689+0000","last_fullsized":"2024-04-25T01:41:30.838689+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4493","last_scrub_stamp":"2024-04-25T01:41:30.838583+0000","last_deep_scrub":"26'4493","last_deep_scrub_stamp":"2024-04-25T01:41:30.838583+0000","last_clean_scrub_stamp":"2024-04-25T01:41:30.838583+0000","objects_scrubbed":0,"log_size":4493,"log_dups_size":0,"ondisk_log_size":4493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:13.949961+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23464,"num_write_kb":290699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"26'4350","reported_seq":4390,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T01:41:29.880342+0000","last_change":"2024-04-25T01:41:29.880377+0000","last_active":"2024-04-25T01:41:29.880342+0000","last_peered":"2024-04-25T01:41:29.880342+0000","last_clean":"2024-04-25T01:41:29.880342+0000","last_became_active":"2024-04-25T01:25:53.367383+0000","last_became_peered":"2024-04-25T01:25:53.367383+0000","last_unstale":"2024-04-25T01:41:29.880342+0000","last_undegraded":"2024-04-25T01:41:29.880342+0000","last_fullsized":"2024-04-25T01:41:29.880342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4350","last_scrub_stamp":"2024-04-25T01:41:29.880242+0000","last_deep_scrub":"26'4350","last_deep_scrub_stamp":"2024-04-25T01:41:29.880242+0000","last_clean_scrub_stamp":"2024-04-25T01:41:29.880242+0000","objects_scrubbed":0,"log_size":4350,"log_dups_size":0,"ondisk_log_size":4350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:18.432207+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23070,"num_write_kb":268669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1533236,"num_write_kb":19254592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":294168,"ondisk_log_size":294168,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":17605243,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":39,"num_read_kb":112,"num_write":4172,"num_write_kb":18793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35274752,"data_stored":35210486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3524,"ondisk_log_size":3524,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1596536,"num_write_kb":19509893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305948,"ondisk_log_size":305948,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":17847589,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":41,"num_read_kb":115,"num_write":4224,"num_write_kb":19059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35758080,"data_stored":35695178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3562,"ondisk_log_size":3562,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575049,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1217764,"kb_used_data":13148,"kb_used_omap":1,"kb_used_meta":1204606,"kb_avail":93154076,"statfs":{"total":96636764160,"available":95389773824,"internally_reserved":0,"allocated":13463552,"data_stored":13287019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":1233517002},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575046,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":780520,"kb_used_data":9440,"kb_used_omap":1,"kb_used_meta":771070,"kb_avail":93591320,"statfs":{"total":96636764160,"available":95837511680,"internally_reserved":0,"allocated":9666560,"data_stored":9483786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":789576134},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575047,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1059708,"kb_used_data":4468,"kb_used_omap":1,"kb_used_meta":1055230,"kb_avail":93312132,"statfs":{"total":96636764160,"available":95551623168,"internally_reserved":0,"allocated":4575232,"data_stored":4396213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1080555973},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575047,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1089176,"kb_used_data":10256,"kb_used_omap":1,"kb_used_meta":1078910,"kb_avail":93282664,"statfs":{"total":96636764160,"available":95521447936,"internally_reserved":0,"allocated":10502144,"data_stored":10309285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1104804293},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575048,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1161176,"kb_used_data":17488,"kb_used_omap":1,"kb_used_meta":1143678,"kb_avail":93210664,"statfs":{"total":96636764160,"available":95447719936,"internally_reserved":0,"allocated":17907712,"data_stored":17718206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1171126727},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575048,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":729908,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":729534,"kb_avail":93641932,"statfs":{"total":96636764160,"available":95889338368,"internally_reserved":0,"allocated":372736,"data_stored":201217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":747043269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575048,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":888620,"kb_used_data":8484,"kb_used_omap":1,"kb_used_meta":880126,"kb_avail":93483220,"statfs":{"total":96636764160,"available":95726817280,"internally_reserved":0,"allocated":8687616,"data_stored":8502529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":901249477},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575046,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":919656,"kb_used_data":9056,"kb_used_omap":1,"kb_used_meta":910590,"kb_avail":93452184,"statfs":{"total":96636764160,"available":95695036416,"internally_reserved":0,"allocated":9273344,"data_stored":9092681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":932444613},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":88080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8486912,"data_stored":8477360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5189632,"data_stored":5175253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5185536,"data_stored":5175799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8389302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9142272,"data_stored":9127779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5033984,"data_stored":5020965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4292608,"data_stored":4283056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:42:02.576 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T01:42:02.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.686+0000 7f875811d640 1 Processor -- start 2024-04-25T01:42:02.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.686+0000 7f875811d640 1 -- start start 2024-04-25T01:42:02.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.687+0000 7f875811d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f875014fbe0 0x7f87500c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.687+0000 7f875811d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f87500c2e60 0x7f87500726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.687+0000 7f875811d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f875006f290 con 0x7f875006d460 2024-04-25T01:42:02.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.687+0000 7f875811d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f875014ffc0 con 0x7f875014fbe0 2024-04-25T01:42:02.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.687+0000 7f875811d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f87501533a0 con 0x7f87500c2e60 2024-04-25T01:42:02.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.687+0000 7f8755eb9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f875014fbe0 0x7f87500c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.687+0000 7f8755eb9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f875014fbe0 0x7f87500c2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37632/0 (socket says 172.21.15.112:37632) 2024-04-25T01:42:02.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.687+0000 7f8755eb9640 1 -- 172.21.15.112:0/3489167325 learned_addr learned my addr 172.21.15.112:0/3489167325 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:02.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.687+0000 7f8755eb9640 1 -- 172.21.15.112:0/3489167325 >> v2:172.21.15.187:3300/0 conn(0x7f87500c2e60 msgr2=0x7f87500726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:02.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.687+0000 7f8755eb9640 1 --2- 172.21.15.112:0/3489167325 >> v2:172.21.15.187:3300/0 conn(0x7f87500c2e60 0x7f87500726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.687+0000 7f8755eb9640 1 -- 172.21.15.112:0/3489167325 >> v1:172.21.15.187:6789/0 conn(0x7f875006d460 legacy=0x7f8750071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:02.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.687+0000 7f8755eb9640 1 -- 172.21.15.112:0/3489167325 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875006efb0 con 0x7f875014fbe0 2024-04-25T01:42:02.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.688+0000 7f8755eb9640 1 --2- 172.21.15.112:0/3489167325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875014fbe0 0x7f87500c2920 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f875006c4b0 tx=0x7f874c02c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510b27d822a17a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:02.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.688+0000 7f8754eb7640 1 -- 172.21.15.112:0/3489167325 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c032710 con 0x7f875014fbe0 2024-04-25T01:42:02.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.688+0000 7f8754eb7640 1 -- 172.21.15.112:0/3489167325 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f874c0328b0 con 0x7f875014fbe0 2024-04-25T01:42:02.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.688+0000 7f8754eb7640 1 -- 172.21.15.112:0/3489167325 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c032ca0 con 0x7f875014fbe0 2024-04-25T01:42:02.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.688+0000 7f875811d640 1 -- 172.21.15.112:0/3489167325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875014fbe0 msgr2=0x7f87500c2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.688+0000 7f875811d640 1 --2- 172.21.15.112:0/3489167325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875014fbe0 0x7f87500c2920 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f875006c4b0 tx=0x7f874c02c850 comp rx=0 tx=0).stop 2024-04-25T01:42:02.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.688+0000 7f875811d640 1 -- 172.21.15.112:0/3489167325 shutdown_connections 2024-04-25T01:42:02.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.689+0000 7f875811d640 1 --2- 172.21.15.112:0/3489167325 >> v2:172.21.15.187:3300/0 conn(0x7f87500c2e60 0x7f87500726c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.689+0000 7f875811d640 1 --2- 172.21.15.112:0/3489167325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875014fbe0 0x7f87500c2920 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.689+0000 7f875811d640 1 -- 172.21.15.112:0/3489167325 >> 172.21.15.112:0/3489167325 conn(0x7f87500c3a10 msgr2=0x7f87500c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:02.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.689+0000 7f875811d640 1 -- 172.21.15.112:0/3489167325 shutdown_connections 2024-04-25T01:42:02.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.689+0000 7f875811d640 1 -- 172.21.15.112:0/3489167325 wait complete. 2024-04-25T01:42:02.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.690+0000 7f875811d640 1 Processor -- start 2024-04-25T01:42:02.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.690+0000 7f875811d640 1 -- start start 2024-04-25T01:42:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.690+0000 7f875811d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875006d460 0x7f87501dd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.690+0000 7f875811d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87500c2e60 0x7f8750141510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.690+0000 7f875811d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f875014fbe0 0x7f87501e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.690+0000 7f875811d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8750072c00 con 0x7f875006d460 2024-04-25T01:42:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.690+0000 7f875811d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8750153030 con 0x7f87500c2e60 2024-04-25T01:42:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.690+0000 7f875811d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f87500c3700 con 0x7f875014fbe0 2024-04-25T01:42:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.690+0000 7f8755eb9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875006d460 0x7f87501dd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.690+0000 7f8755eb9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875006d460 0x7f87501dd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37636/0 (socket says 172.21.15.112:37636) 2024-04-25T01:42:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.690+0000 7f8755eb9640 1 -- 172.21.15.112:0/1167208746 learned_addr learned my addr 172.21.15.112:0/1167208746 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.691+0000 7f87556b8640 1 --2- 172.21.15.112:0/1167208746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87500c2e60 0x7f8750141510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.691+0000 7f87566ba640 1 --2- 172.21.15.112:0/1167208746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f875014fbe0 0x7f87501e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.691+0000 7f8755eb9640 1 -- 172.21.15.112:0/1167208746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f875014fbe0 msgr2=0x7f87501e7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.691+0000 7f8755eb9640 1 --2- 172.21.15.112:0/1167208746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f875014fbe0 0x7f87501e7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.691+0000 7f8755eb9640 1 -- 172.21.15.112:0/1167208746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87500c2e60 msgr2=0x7f8750141510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.691+0000 7f8755eb9640 1 --2- 172.21.15.112:0/1167208746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87500c2e60 0x7f8750141510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.691+0000 7f8755eb9640 1 -- 172.21.15.112:0/1167208746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87501e7250 con 0x7f875006d460 2024-04-25T01:42:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.691+0000 7f8755eb9640 1 --2- 172.21.15.112:0/1167208746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875006d460 0x7f87501dd4c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f874c02c890 tx=0x7f874c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.691+0000 7f87467fc640 1 -- 172.21.15.112:0/1167208746 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c044070 con 0x7f875006d460 2024-04-25T01:42:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.691+0000 7f87467fc640 1 -- 172.21.15.112:0/1167208746 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f874c003810 con 0x7f875006d460 2024-04-25T01:42:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.691+0000 7f875811d640 1 -- 172.21.15.112:0/1167208746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87501eb260 con 0x7f875006d460 2024-04-25T01:42:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.691+0000 7f87467fc640 1 -- 172.21.15.112:0/1167208746 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c003c00 con 0x7f875006d460 2024-04-25T01:42:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.691+0000 7f875811d640 1 -- 172.21.15.112:0/1167208746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87501eb540 con 0x7f875006d460 2024-04-25T01:42:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.692+0000 7f875811d640 1 -- 172.21.15.112:0/1167208746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f875004f500 con 0x7f875006d460 2024-04-25T01:42:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.693+0000 7f87467fc640 1 -- 172.21.15.112:0/1167208746 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f874c047070 con 0x7f875006d460 2024-04-25T01:42:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.693+0000 7f87467fc640 1 --2- 172.21.15.112:0/1167208746 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f873406be30 0x7f873406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.693+0000 7f87467fc640 1 -- 172.21.15.112:0/1167208746 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f874c0ae450 con 0x7f875006d460 2024-04-25T01:42:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.693+0000 7f87556b8640 1 --2- 172.21.15.112:0/1167208746 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f873406be30 0x7f873406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.693+0000 7f87556b8640 1 --2- 172.21.15.112:0/1167208746 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f873406be30 0x7f873406e2f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8740007fd0 tx=0x7f8740009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:02.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.697+0000 7f87467fc640 1 -- 172.21.15.112:0/1167208746 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f874c07cc10 con 0x7f875006d460 2024-04-25T01:42:02.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.872+0000 7f875811d640 1 -- 172.21.15.112:0/1167208746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f87500c47e0 con 0x7f875006d460 2024-04-25T01:42:02.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.875+0000 7f87467fc640 1 -- 172.21.15.112:0/1167208746 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) ==== 123+0+0 (secure 0 0 0) 0x7f874c07c510 con 0x7f875006d460 2024-04-25T01:42:02.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.876+0000 7f87467fc640 1 -- 172.21.15.112:0/1167208746 <== mon.0 v2:172.21.15.112:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f874c0808c0 con 0x7f875006d460 2024-04-25T01:42:02.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.878+0000 7f875811d640 1 -- 172.21.15.112:0/1167208746 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f873406be30 msgr2=0x7f873406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.878+0000 7f875811d640 1 --2- 172.21.15.112:0/1167208746 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f873406be30 0x7f873406e2f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8740007fd0 tx=0x7f8740009040 comp rx=0 tx=0).stop 2024-04-25T01:42:02.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.878+0000 7f875811d640 1 -- 172.21.15.112:0/1167208746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875006d460 msgr2=0x7f87501dd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.878+0000 7f875811d640 1 --2- 172.21.15.112:0/1167208746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875006d460 0x7f87501dd4c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f874c02c890 tx=0x7f874c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.878+0000 7f875811d640 1 -- 172.21.15.112:0/1167208746 shutdown_connections 2024-04-25T01:42:02.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.878+0000 7f875811d640 1 --2- 172.21.15.112:0/1167208746 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f873406be30 0x7f873406e2f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.878+0000 7f875811d640 1 --2- 172.21.15.112:0/1167208746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f875014fbe0 0x7f87501e7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.878+0000 7f875811d640 1 --2- 172.21.15.112:0/1167208746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87500c2e60 0x7f8750141510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.878+0000 7f875811d640 1 --2- 172.21.15.112:0/1167208746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f875006d460 0x7f87501dd4c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.878+0000 7f875811d640 1 -- 172.21.15.112:0/1167208746 >> 172.21.15.112:0/1167208746 conn(0x7f87500c3a10 msgr2=0x7f8750071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:02.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.878+0000 7f875811d640 1 -- 172.21.15.112:0/1167208746 shutdown_connections 2024-04-25T01:42:02.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.878+0000 7f875811d640 1 -- 172.21.15.112:0/1167208746 wait complete. 2024-04-25T01:42:02.877 INFO:tasks.cephfs.filesystem:Destroying file system a and related pools 2024-04-25T01:42:02.878 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:42:02.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.987+0000 7f34288da640 1 Processor -- start 2024-04-25T01:42:02.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.987+0000 7f34288da640 1 -- start start 2024-04-25T01:42:02.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.988+0000 7f34288da640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f342414c3f0 0x7f34240c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.988+0000 7f34288da640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f34240ac3b0 0x7f3424155f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.988+0000 7f34288da640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34241529b0 con 0x7f34241524a0 2024-04-25T01:42:02.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.988+0000 7f34288da640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f3424155a90 con 0x7f34240ac3b0 2024-04-25T01:42:02.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.988+0000 7f34288da640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3424152c70 con 0x7f342414c3f0 2024-04-25T01:42:02.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.988+0000 7f3421d9b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f34240ac3b0 0x7f3424155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.988+0000 7f3422d9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f34241524a0 0x7f34240c3b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:43436/0 (socket says 172.21.15.112:43436) 2024-04-25T01:42:02.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.988+0000 7f3422d9d640 1 -- 172.21.15.112:0/1215323528 learned_addr learned my addr 172.21.15.112:0/1215323528 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:42:02.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.988+0000 7f342259c640 1 --2- 172.21.15.112:0/1215323528 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f342414c3f0 0x7f34240c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.988+0000 7f3421d9b640 1 -- 172.21.15.112:0/1215323528 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f342414c3f0 msgr2=0x7f34240c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:02.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.988+0000 7f3421d9b640 1 --2- 172.21.15.112:0/1215323528 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f342414c3f0 0x7f34240c2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.988+0000 7f3421d9b640 1 -- 172.21.15.112:0/1215323528 >> v1:172.21.15.112:6789/0 conn(0x7f34241524a0 legacy=0x7f34240c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.988+0000 7f342159a640 1 -- 172.21.15.112:0/1215323528 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4170337086 0 0) 0x7f34241529b0 con 0x7f34241524a0 2024-04-25T01:42:02.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.988+0000 7f3421d9b640 1 -- 172.21.15.112:0/1215323528 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3424152f50 con 0x7f34240ac3b0 2024-04-25T01:42:02.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.989+0000 7f3421d9b640 1 --2- 172.21.15.112:0/1215323528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34240ac3b0 0x7f3424155f20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f341800ba40 tx=0x7f341802c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90af403200848468 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:02.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.989+0000 7f342159a640 1 -- 172.21.15.112:0/1215323528 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418031710 con 0x7f34240ac3b0 2024-04-25T01:42:02.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.989+0000 7f342159a640 1 -- 172.21.15.112:0/1215323528 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f34180318b0 con 0x7f34240ac3b0 2024-04-25T01:42:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.989+0000 7f342159a640 1 -- 172.21.15.112:0/1215323528 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418031ca0 con 0x7f34240ac3b0 2024-04-25T01:42:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.989+0000 7f34288da640 1 -- 172.21.15.112:0/1215323528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34240ac3b0 msgr2=0x7f3424155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.989+0000 7f34288da640 1 --2- 172.21.15.112:0/1215323528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34240ac3b0 0x7f3424155f20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f341800ba40 tx=0x7f341802c7b0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.990+0000 7f34288da640 1 -- 172.21.15.112:0/1215323528 shutdown_connections 2024-04-25T01:42:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.990+0000 7f34288da640 1 --2- 172.21.15.112:0/1215323528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34240ac3b0 0x7f3424155f20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.990+0000 7f34288da640 1 --2- 172.21.15.112:0/1215323528 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f342414c3f0 0x7f34240c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.990+0000 7f34288da640 1 -- 172.21.15.112:0/1215323528 >> 172.21.15.112:0/1215323528 conn(0x7f342406b160 msgr2=0x7f342406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.990+0000 7f34288da640 1 -- 172.21.15.112:0/1215323528 shutdown_connections 2024-04-25T01:42:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.990+0000 7f34288da640 1 -- 172.21.15.112:0/1215323528 wait complete. 2024-04-25T01:42:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.991+0000 7f34288da640 1 Processor -- start 2024-04-25T01:42:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.991+0000 7f34288da640 1 -- start start 2024-04-25T01:42:02.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.991+0000 7f34288da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34240ac3b0 0x7f34240c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.991+0000 7f34288da640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f342414c3f0 0x7f34241551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.991+0000 7f34288da640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34241524a0 0x7f34241397e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.991+0000 7f34288da640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f342414c7d0 con 0x7f34240ac3b0 2024-04-25T01:42:02.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.991+0000 7f34288da640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3424070d60 con 0x7f342414c3f0 2024-04-25T01:42:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.991+0000 7f34288da640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f342414c950 con 0x7f34241524a0 2024-04-25T01:42:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.992+0000 7f342259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34240ac3b0 0x7f34240c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.992+0000 7f342259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34240ac3b0 0x7f34240c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37656/0 (socket says 172.21.15.112:37656) 2024-04-25T01:42:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.992+0000 7f342259c640 1 -- 172.21.15.112:0/3508958100 learned_addr learned my addr 172.21.15.112:0/3508958100 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.992+0000 7f3421d9b640 1 --2- 172.21.15.112:0/3508958100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f342414c3f0 0x7f34241551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.992+0000 7f3422d9d640 1 --2- 172.21.15.112:0/3508958100 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34241524a0 0x7f34241397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.992+0000 7f342259c640 1 -- 172.21.15.112:0/3508958100 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34241524a0 msgr2=0x7f34241397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.992+0000 7f342259c640 1 --2- 172.21.15.112:0/3508958100 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34241524a0 0x7f34241397e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.992+0000 7f342259c640 1 -- 172.21.15.112:0/3508958100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f342414c3f0 msgr2=0x7f34241551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.992+0000 7f342259c640 1 --2- 172.21.15.112:0/3508958100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f342414c3f0 0x7f34241551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.992+0000 7f342259c640 1 -- 172.21.15.112:0/3508958100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3424147ba0 con 0x7f34240ac3b0 2024-04-25T01:42:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.992+0000 7f3421d9b640 1 --2- 172.21.15.112:0/3508958100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f342414c3f0 0x7f34241551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:42:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.992+0000 7f342259c640 1 --2- 172.21.15.112:0/3508958100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34240ac3b0 0x7f34240c40c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f341400d800 tx=0x7f341400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.992+0000 7f3412ffd640 1 -- 172.21.15.112:0/3508958100 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341400b6c0 con 0x7f34240ac3b0 2024-04-25T01:42:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.992+0000 7f3412ffd640 1 -- 172.21.15.112:0/3508958100 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3414019070 con 0x7f34240ac3b0 2024-04-25T01:42:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.992+0000 7f34288da640 1 -- 172.21.15.112:0/3508958100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342414fe30 con 0x7f34240ac3b0 2024-04-25T01:42:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.992+0000 7f3412ffd640 1 -- 172.21.15.112:0/3508958100 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341400baa0 con 0x7f34240ac3b0 2024-04-25T01:42:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.992+0000 7f34288da640 1 -- 172.21.15.112:0/3508958100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34241480f0 con 0x7f34240ac3b0 2024-04-25T01:42:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.993+0000 7f34288da640 1 -- 172.21.15.112:0/3508958100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34240c11a0 con 0x7f34240ac3b0 2024-04-25T01:42:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.994+0000 7f3412ffd640 1 -- 172.21.15.112:0/3508958100 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f341400bc40 con 0x7f34240ac3b0 2024-04-25T01:42:02.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.994+0000 7f3412ffd640 1 --2- 172.21.15.112:0/3508958100 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f340406be30 0x7f340406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.994+0000 7f3412ffd640 1 -- 172.21.15.112:0/3508958100 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7f341408b6a0 con 0x7f34240ac3b0 2024-04-25T01:42:02.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.994+0000 7f3421d9b640 1 --2- 172.21.15.112:0/3508958100 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f340406be30 0x7f340406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.995+0000 7f3421d9b640 1 --2- 172.21.15.112:0/3508958100 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f340406be30 0x7f340406e2f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3424153510 tx=0x7f3418031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:02.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:02.998+0000 7f3412ffd640 1 -- 172.21.15.112:0/3508958100 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f34240c11a0 con 0x7f34240ac3b0 2024-04-25T01:42:03.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.210+0000 7f34288da640 1 -- 172.21.15.112:0/3508958100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34240682a0 con 0x7f34240ac3b0 2024-04-25T01:42:03.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.211+0000 7f3412ffd640 1 -- 172.21.15.112:0/3508958100 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+6491 (secure 0 0 0) 0x7f3414059cb0 con 0x7f34240ac3b0 2024-04-25T01:42:03.199 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:42:03.199 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/4047219598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4047219598},{"type":"v1","addr":"172.21.15.187:6834","nonce":4047219598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2204640907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2204640907},{"type":"v1","addr":"172.21.15.187:6835","nonce":2204640907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:25:52.687260+0000","modified":"2024-04-25T01:25:55.702719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/2115984531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2115984531},{"type":"v1","addr":"172.21.15.112:6837","nonce":2115984531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1},{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:26:01.740521+0000","modified":"2024-04-25T01:26:04.746955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":8,"state":"up:active","state_seq":6,"addr":"172.21.15.112:6835/278686477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":278686477},{"type":"v1","addr":"172.21.15.112:6835","nonce":278686477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":2}]} 2024-04-25T01:42:03.199 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 11 2024-04-25T01:42:03.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.213+0000 7f34288da640 1 -- 172.21.15.112:0/3508958100 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f340406be30 msgr2=0x7f340406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.213+0000 7f34288da640 1 --2- 172.21.15.112:0/3508958100 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f340406be30 0x7f340406e2f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3424153510 tx=0x7f3418031000 comp rx=0 tx=0).stop 2024-04-25T01:42:03.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.213+0000 7f34288da640 1 -- 172.21.15.112:0/3508958100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34240ac3b0 msgr2=0x7f34240c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.213+0000 7f34288da640 1 --2- 172.21.15.112:0/3508958100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34240ac3b0 0x7f34240c40c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f341400d800 tx=0x7f341400dcc0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.213+0000 7f34288da640 1 -- 172.21.15.112:0/3508958100 shutdown_connections 2024-04-25T01:42:03.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.213+0000 7f34288da640 1 --2- 172.21.15.112:0/3508958100 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f340406be30 0x7f340406e2f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.213+0000 7f34288da640 1 --2- 172.21.15.112:0/3508958100 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34241524a0 0x7f34241397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.213+0000 7f34288da640 1 --2- 172.21.15.112:0/3508958100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f342414c3f0 0x7f34241551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.213+0000 7f34288da640 1 --2- 172.21.15.112:0/3508958100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34240ac3b0 0x7f34240c40c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.213+0000 7f34288da640 1 -- 172.21.15.112:0/3508958100 >> 172.21.15.112:0/3508958100 conn(0x7f342406b160 msgr2=0x7f3424155d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:03.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.213+0000 7f34288da640 1 -- 172.21.15.112:0/3508958100 shutdown_connections 2024-04-25T01:42:03.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.213+0000 7f34288da640 1 -- 172.21.15.112:0/3508958100 wait complete. 2024-04-25T01:42:03.213 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:42:03.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.321+0000 7fde8559e640 1 Processor -- start 2024-04-25T01:42:03.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.321+0000 7fde8559e640 1 -- start start 2024-04-25T01:42:03.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.322+0000 7fde8559e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fde801596a0 0x7fde80152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.322+0000 7fde8559e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde800c72b0 0x7fde80160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.322+0000 7fde8559e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde80159b90 con 0x7fde801592c0 2024-04-25T01:42:03.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.322+0000 7fde8559e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fde800c6e20 con 0x7fde801596a0 2024-04-25T01:42:03.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.322+0000 7fde8559e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fde8006c970 con 0x7fde800c72b0 2024-04-25T01:42:03.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.323+0000 7fde7f7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde800c72b0 0x7fde80160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.323+0000 7fde7e7fc640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fde801596a0 0x7fde80152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.323+0000 7fde7f7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde800c72b0 0x7fde80160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:34594/0 (socket says 172.21.15.112:34594) 2024-04-25T01:42:03.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.323+0000 7fde7f7fe640 1 -- 172.21.15.112:0/3806914325 learned_addr learned my addr 172.21.15.112:0/3806914325 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:03.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.323+0000 7fde7e7fc640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fde801596a0 0x7fde80152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:48576/0 (socket says 172.21.15.112:48576) 2024-04-25T01:42:03.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.323+0000 7fde7f7fe640 1 -- 172.21.15.112:0/3806914325 >> v2:172.21.15.187:3300/0 conn(0x7fde801596a0 msgr2=0x7fde80152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:03.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.323+0000 7fde7f7fe640 1 --2- 172.21.15.112:0/3806914325 >> v2:172.21.15.187:3300/0 conn(0x7fde801596a0 0x7fde80152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.323+0000 7fde7f7fe640 1 -- 172.21.15.112:0/3806914325 >> v1:172.21.15.187:6789/0 conn(0x7fde801592c0 legacy=0x7fde800c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.323+0000 7fde7f7fe640 1 -- 172.21.15.112:0/3806914325 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8006cc50 con 0x7fde800c72b0 2024-04-25T01:42:03.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.323+0000 7fde7e7fc640 1 --2- 172.21.15.112:0/3806914325 >> v2:172.21.15.187:3300/0 conn(0x7fde801596a0 0x7fde80152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:42:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.324+0000 7fde7f7fe640 1 --2- 172.21.15.112:0/3806914325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde800c72b0 0x7fde80160380 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fde78009870 tx=0x7fde7802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4210a8eb9f7a46b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.324+0000 7fde7dffb640 1 -- 172.21.15.112:0/3806914325 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78037070 con 0x7fde800c72b0 2024-04-25T01:42:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.324+0000 7fde7dffb640 1 -- 172.21.15.112:0/3806914325 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fde78032440 con 0x7fde800c72b0 2024-04-25T01:42:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.324+0000 7fde7dffb640 1 -- 172.21.15.112:0/3806914325 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78032830 con 0x7fde800c72b0 2024-04-25T01:42:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.324+0000 7fde8559e640 1 -- 172.21.15.112:0/3806914325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde800c72b0 msgr2=0x7fde80160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.324+0000 7fde8559e640 1 --2- 172.21.15.112:0/3806914325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde800c72b0 0x7fde80160380 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fde78009870 tx=0x7fde7802a130 comp rx=0 tx=0).stop 2024-04-25T01:42:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.324+0000 7fde8559e640 1 -- 172.21.15.112:0/3806914325 shutdown_connections 2024-04-25T01:42:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.324+0000 7fde8559e640 1 --2- 172.21.15.112:0/3806914325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde800c72b0 0x7fde80160380 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.324+0000 7fde8559e640 1 --2- 172.21.15.112:0/3806914325 >> v2:172.21.15.187:3300/0 conn(0x7fde801596a0 0x7fde80152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.324+0000 7fde8559e640 1 -- 172.21.15.112:0/3806914325 >> 172.21.15.112:0/3806914325 conn(0x7fde800cb170 msgr2=0x7fde800cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.324+0000 7fde8559e640 1 -- 172.21.15.112:0/3806914325 shutdown_connections 2024-04-25T01:42:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.324+0000 7fde8559e640 1 -- 172.21.15.112:0/3806914325 wait complete. 2024-04-25T01:42:03.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.326+0000 7fde8559e640 1 Processor -- start 2024-04-25T01:42:03.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.326+0000 7fde8559e640 1 -- start start 2024-04-25T01:42:03.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.326+0000 7fde8559e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde800c72b0 0x7fde801ef780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.326+0000 7fde8559e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde801592c0 0x7fde8014bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.326+0000 7fde8559e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde801596a0 0x7fde801e2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.326+0000 7fde8559e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fde8006c510 con 0x7fde801596a0 2024-04-25T01:42:03.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.326+0000 7fde8559e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fde8006c390 con 0x7fde801592c0 2024-04-25T01:42:03.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.326+0000 7fde8559e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fde8006db20 con 0x7fde800c72b0 2024-04-25T01:42:03.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.326+0000 7fde7f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde801596a0 0x7fde801e2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.326+0000 7fde7f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde801596a0 0x7fde801e2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37658/0 (socket says 172.21.15.112:37658) 2024-04-25T01:42:03.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.327+0000 7fde7f7fe640 1 -- 172.21.15.112:0/189174308 learned_addr learned my addr 172.21.15.112:0/189174308 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:03.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.327+0000 7fde7e7fc640 1 --2- 172.21.15.112:0/189174308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde801592c0 0x7fde8014bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.327+0000 7fde7effd640 1 --2- 172.21.15.112:0/189174308 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde800c72b0 0x7fde801ef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.327+0000 7fde7f7fe640 1 -- 172.21.15.112:0/189174308 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde800c72b0 msgr2=0x7fde801ef780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.327+0000 7fde7f7fe640 1 --2- 172.21.15.112:0/189174308 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde800c72b0 0x7fde801ef780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.327+0000 7fde7f7fe640 1 -- 172.21.15.112:0/189174308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde801592c0 msgr2=0x7fde8014bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.327+0000 7fde7f7fe640 1 --2- 172.21.15.112:0/189174308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde801592c0 0x7fde8014bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.327+0000 7fde7f7fe640 1 -- 172.21.15.112:0/189174308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde800c1b00 con 0x7fde801596a0 2024-04-25T01:42:03.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.327+0000 7fde7f7fe640 1 --2- 172.21.15.112:0/189174308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde801596a0 0x7fde801e2810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fde78009870 tx=0x7fde780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:03.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.327+0000 7fde677fe640 1 -- 172.21.15.112:0/189174308 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78037040 con 0x7fde801596a0 2024-04-25T01:42:03.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.327+0000 7fde677fe640 1 -- 172.21.15.112:0/189174308 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fde78042070 con 0x7fde801596a0 2024-04-25T01:42:03.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.327+0000 7fde8559e640 1 -- 172.21.15.112:0/189174308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde800c1de0 con 0x7fde801596a0 2024-04-25T01:42:03.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.327+0000 7fde677fe640 1 -- 172.21.15.112:0/189174308 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde780309a0 con 0x7fde801596a0 2024-04-25T01:42:03.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.327+0000 7fde8559e640 1 -- 172.21.15.112:0/189174308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde8014c480 con 0x7fde801596a0 2024-04-25T01:42:03.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.328+0000 7fde8559e640 1 -- 172.21.15.112:0/189174308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde800679a0 con 0x7fde801596a0 2024-04-25T01:42:03.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.328+0000 7fde677fe640 1 -- 172.21.15.112:0/189174308 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fde78030b40 con 0x7fde801596a0 2024-04-25T01:42:03.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.329+0000 7fde677fe640 1 --2- 172.21.15.112:0/189174308 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fde6c06be30 0x7fde6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.329+0000 7fde677fe640 1 -- 172.21.15.112:0/189174308 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fde780ab9f0 con 0x7fde801596a0 2024-04-25T01:42:03.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.329+0000 7fde7effd640 1 --2- 172.21.15.112:0/189174308 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fde6c06be30 0x7fde6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.329+0000 7fde7effd640 1 --2- 172.21.15.112:0/189174308 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fde6c06be30 0x7fde6c06e2f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fde700012e0 tx=0x7fde70009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:03.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.333+0000 7fde677fe640 1 -- 172.21.15.112:0/189174308 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fde78079a40 con 0x7fde801596a0 2024-04-25T01:42:03.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.542+0000 7fde8559e640 1 -- 172.21.15.112:0/189174308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde800682a0 con 0x7fde801596a0 2024-04-25T01:42:03.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.542+0000 7fde677fe640 1 -- 172.21.15.112:0/189174308 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+6491 (secure 0 0 0) 0x7fde7807dd40 con 0x7fde801596a0 2024-04-25T01:42:03.531 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:42:03.531 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/4047219598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4047219598},{"type":"v1","addr":"172.21.15.187:6834","nonce":4047219598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2204640907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2204640907},{"type":"v1","addr":"172.21.15.187:6835","nonce":2204640907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:25:52.687260+0000","modified":"2024-04-25T01:25:55.702719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/2115984531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2115984531},{"type":"v1","addr":"172.21.15.112:6837","nonce":2115984531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1},{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:26:01.740521+0000","modified":"2024-04-25T01:26:04.746955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":8,"state":"up:active","state_seq":6,"addr":"172.21.15.112:6835/278686477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":278686477},{"type":"v1","addr":"172.21.15.112:6835","nonce":278686477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":2}]} 2024-04-25T01:42:03.531 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 11 2024-04-25T01:42:03.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.544+0000 7fde8559e640 1 -- 172.21.15.112:0/189174308 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fde6c06be30 msgr2=0x7fde6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.544+0000 7fde8559e640 1 --2- 172.21.15.112:0/189174308 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fde6c06be30 0x7fde6c06e2f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fde700012e0 tx=0x7fde70009040 comp rx=0 tx=0).stop 2024-04-25T01:42:03.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.544+0000 7fde8559e640 1 -- 172.21.15.112:0/189174308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde801596a0 msgr2=0x7fde801e2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.544+0000 7fde8559e640 1 --2- 172.21.15.112:0/189174308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde801596a0 0x7fde801e2810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fde78009870 tx=0x7fde780046c0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.544+0000 7fde8559e640 1 -- 172.21.15.112:0/189174308 shutdown_connections 2024-04-25T01:42:03.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.544+0000 7fde8559e640 1 --2- 172.21.15.112:0/189174308 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fde6c06be30 0x7fde6c06e2f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.544+0000 7fde8559e640 1 --2- 172.21.15.112:0/189174308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde801596a0 0x7fde801e2810 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.544+0000 7fde8559e640 1 --2- 172.21.15.112:0/189174308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde801592c0 0x7fde8014bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.544+0000 7fde8559e640 1 --2- 172.21.15.112:0/189174308 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde800c72b0 0x7fde801ef780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.544+0000 7fde8559e640 1 -- 172.21.15.112:0/189174308 >> 172.21.15.112:0/189174308 conn(0x7fde800cb170 msgr2=0x7fde800ca5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:03.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.545+0000 7fde8559e640 1 -- 172.21.15.112:0/189174308 shutdown_connections 2024-04-25T01:42:03.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.545+0000 7fde8559e640 1 -- 172.21.15.112:0/189174308 wait complete. 2024-04-25T01:42:03.544 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:42:03.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.652+0000 7fa875c72640 1 Processor -- start 2024-04-25T01:42:03.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.652+0000 7fa875c72640 1 -- start start 2024-04-25T01:42:03.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.653+0000 7fa875c72640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa8700ac3b0 0x7fa870155f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.653+0000 7fa875c72640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8701524a0 0x7fa870156460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.653+0000 7fa875c72640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8701529b0 con 0x7fa87014c3f0 2024-04-25T01:42:03.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.653+0000 7fa875c72640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa870155a90 con 0x7fa8700ac3b0 2024-04-25T01:42:03.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.653+0000 7fa875c72640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa8701569a0 con 0x7fa8701524a0 2024-04-25T01:42:03.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.653+0000 7fa86effd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa8700ac3b0 0x7fa870155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.653+0000 7fa86f7fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa87014c3f0 0x7fa8700c2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:43438/0 (socket says 172.21.15.112:43438) 2024-04-25T01:42:03.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.653+0000 7fa86f7fe640 1 -- 172.21.15.112:0/1858313153 learned_addr learned my addr 172.21.15.112:0/1858313153 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:42:03.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.653+0000 7fa86ffff640 1 --2- 172.21.15.112:0/1858313153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8701524a0 0x7fa870156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.653+0000 7fa86effd640 1 -- 172.21.15.112:0/1858313153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8701524a0 msgr2=0x7fa870156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:03.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.653+0000 7fa86effd640 1 --2- 172.21.15.112:0/1858313153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8701524a0 0x7fa870156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.653+0000 7fa86effd640 1 -- 172.21.15.112:0/1858313153 >> v1:172.21.15.112:6789/0 conn(0x7fa87014c3f0 legacy=0x7fa8700c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.653+0000 7fa86effd640 1 -- 172.21.15.112:0/1858313153 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa870152c70 con 0x7fa8700ac3b0 2024-04-25T01:42:03.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.653+0000 7fa86e7fc640 1 -- 172.21.15.112:0/1858313153 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4057880995 0 0) 0x7fa8701529b0 con 0x7fa87014c3f0 2024-04-25T01:42:03.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.653+0000 7fa86ffff640 1 --2- 172.21.15.112:0/1858313153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8701524a0 0x7fa870156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:42:03.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.654+0000 7fa86effd640 1 --2- 172.21.15.112:0/1858313153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8700ac3b0 0x7fa870155f20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa8640097a0 tx=0x7fa86402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8e2c8b4de76170 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:03.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.654+0000 7fa86e7fc640 1 -- 172.21.15.112:0/1858313153 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864037070 con 0x7fa8700ac3b0 2024-04-25T01:42:03.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.654+0000 7fa86e7fc640 1 -- 172.21.15.112:0/1858313153 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa86402ae50 con 0x7fa8700ac3b0 2024-04-25T01:42:03.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.654+0000 7fa86e7fc640 1 -- 172.21.15.112:0/1858313153 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864032680 con 0x7fa8700ac3b0 2024-04-25T01:42:03.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.654+0000 7fa875c72640 1 -- 172.21.15.112:0/1858313153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8700ac3b0 msgr2=0x7fa870155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.654+0000 7fa875c72640 1 --2- 172.21.15.112:0/1858313153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8700ac3b0 0x7fa870155f20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa8640097a0 tx=0x7fa86402a100 comp rx=0 tx=0).stop 2024-04-25T01:42:03.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.654+0000 7fa875c72640 1 -- 172.21.15.112:0/1858313153 shutdown_connections 2024-04-25T01:42:03.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.654+0000 7fa875c72640 1 --2- 172.21.15.112:0/1858313153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8701524a0 0x7fa870156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.654+0000 7fa875c72640 1 --2- 172.21.15.112:0/1858313153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8700ac3b0 0x7fa870155f20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.654+0000 7fa875c72640 1 -- 172.21.15.112:0/1858313153 >> 172.21.15.112:0/1858313153 conn(0x7fa87006b160 msgr2=0x7fa87006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:03.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.654+0000 7fa875c72640 1 -- 172.21.15.112:0/1858313153 shutdown_connections 2024-04-25T01:42:03.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.654+0000 7fa875c72640 1 -- 172.21.15.112:0/1858313153 wait complete. 2024-04-25T01:42:03.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.655+0000 7fa875c72640 1 Processor -- start 2024-04-25T01:42:03.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.656+0000 7fa875c72640 1 -- start start 2024-04-25T01:42:03.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.656+0000 7fa875c72640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8700ac3b0 0x7fa87006e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.656+0000 7fa875c72640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa87014c3f0 0x7fa8701eb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.656+0000 7fa875c72640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8701524a0 0x7fa8701eeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.656+0000 7fa875c72640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa870070d60 con 0x7fa8701524a0 2024-04-25T01:42:03.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.656+0000 7fa875c72640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa870153230 con 0x7fa87014c3f0 2024-04-25T01:42:03.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.656+0000 7fa875c72640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa8700c32a0 con 0x7fa8700ac3b0 2024-04-25T01:42:03.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.656+0000 7fa86ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8701524a0 0x7fa8701eeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.656+0000 7fa86ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8701524a0 0x7fa8701eeb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37682/0 (socket says 172.21.15.112:37682) 2024-04-25T01:42:03.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.656+0000 7fa86ffff640 1 -- 172.21.15.112:0/955666975 learned_addr learned my addr 172.21.15.112:0/955666975 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:03.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.657+0000 7fa86f7fe640 1 --2- 172.21.15.112:0/955666975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8700ac3b0 0x7fa87006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.657+0000 7fa86effd640 1 --2- 172.21.15.112:0/955666975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa87014c3f0 0x7fa8701eb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.657+0000 7fa86ffff640 1 -- 172.21.15.112:0/955666975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8700ac3b0 msgr2=0x7fa87006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.657+0000 7fa86ffff640 1 --2- 172.21.15.112:0/955666975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8700ac3b0 0x7fa87006e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.657+0000 7fa86ffff640 1 -- 172.21.15.112:0/955666975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa87014c3f0 msgr2=0x7fa8701eb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.657+0000 7fa86ffff640 1 --2- 172.21.15.112:0/955666975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa87014c3f0 0x7fa8701eb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.657+0000 7fa86ffff640 1 -- 172.21.15.112:0/955666975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8701dde00 con 0x7fa8701524a0 2024-04-25T01:42:03.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.657+0000 7fa86ffff640 1 --2- 172.21.15.112:0/955666975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8701524a0 0x7fa8701eeb30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa86800cd50 tx=0x7fa86800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:03.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.657+0000 7fa857fff640 1 -- 172.21.15.112:0/955666975 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86800f680 con 0x7fa8701524a0 2024-04-25T01:42:03.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.657+0000 7fa857fff640 1 -- 172.21.15.112:0/955666975 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa86800f840 con 0x7fa8701524a0 2024-04-25T01:42:03.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.657+0000 7fa857fff640 1 -- 172.21.15.112:0/955666975 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86800fc50 con 0x7fa8701524a0 2024-04-25T01:42:03.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.657+0000 7fa875c72640 1 -- 172.21.15.112:0/955666975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8701de0e0 con 0x7fa8701524a0 2024-04-25T01:42:03.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.657+0000 7fa875c72640 1 -- 172.21.15.112:0/955666975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8701de6a0 con 0x7fa8701524a0 2024-04-25T01:42:03.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.658+0000 7fa875c72640 1 -- 172.21.15.112:0/955666975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8700682a0 con 0x7fa8701524a0 2024-04-25T01:42:03.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.658+0000 7fa857fff640 1 -- 172.21.15.112:0/955666975 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa8680213f0 con 0x7fa8701524a0 2024-04-25T01:42:03.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.659+0000 7fa857fff640 1 --2- 172.21.15.112:0/955666975 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa85c06be60 0x7fa85c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.659+0000 7fa857fff640 1 -- 172.21.15.112:0/955666975 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fa86808bff0 con 0x7fa8701524a0 2024-04-25T01:42:03.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.659+0000 7fa86f7fe640 1 --2- 172.21.15.112:0/955666975 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa85c06be60 0x7fa85c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.659+0000 7fa86f7fe640 1 --2- 172.21.15.112:0/955666975 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa85c06be60 0x7fa85c06e320 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa860000f60 tx=0x7fa860007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:03.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.663+0000 7fa857fff640 1 -- 172.21.15.112:0/955666975 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa868014030 con 0x7fa8701524a0 2024-04-25T01:42:03.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.837+0000 7fa875c72640 1 -- 172.21.15.112:0/955666975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8700c27d0 con 0x7fa8701524a0 2024-04-25T01:42:03.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.838+0000 7fa857fff640 1 -- 172.21.15.112:0/955666975 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+20616 (secure 0 0 0) 0x7fa868059fb0 con 0x7fa8701524a0 2024-04-25T01:42:03.826 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:42:03.827 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:26:52.110548+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-25T01:25:50.593045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-25T01:25:51.664930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-25T01:25:59.478275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-25T01:26:00.719358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:42:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.841+0000 7fa875c72640 1 -- 172.21.15.112:0/955666975 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa85c06be60 msgr2=0x7fa85c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.841+0000 7fa875c72640 1 --2- 172.21.15.112:0/955666975 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa85c06be60 0x7fa85c06e320 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa860000f60 tx=0x7fa860007b50 comp rx=0 tx=0).stop 2024-04-25T01:42:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.841+0000 7fa875c72640 1 -- 172.21.15.112:0/955666975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8701524a0 msgr2=0x7fa8701eeb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.841+0000 7fa875c72640 1 --2- 172.21.15.112:0/955666975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8701524a0 0x7fa8701eeb30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa86800cd50 tx=0x7fa86800a650 comp rx=0 tx=0).stop 2024-04-25T01:42:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.841+0000 7fa875c72640 1 -- 172.21.15.112:0/955666975 shutdown_connections 2024-04-25T01:42:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.841+0000 7fa875c72640 1 --2- 172.21.15.112:0/955666975 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fa85c06be60 0x7fa85c06e320 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.841+0000 7fa875c72640 1 --2- 172.21.15.112:0/955666975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8701524a0 0x7fa8701eeb30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.841+0000 7fa875c72640 1 --2- 172.21.15.112:0/955666975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa87014c3f0 0x7fa8701eb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.841+0000 7fa875c72640 1 --2- 172.21.15.112:0/955666975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8700ac3b0 0x7fa87006e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.841+0000 7fa875c72640 1 -- 172.21.15.112:0/955666975 >> 172.21.15.112:0/955666975 conn(0x7fa87006b160 msgr2=0x7fa8701509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.841+0000 7fa875c72640 1 -- 172.21.15.112:0/955666975 shutdown_connections 2024-04-25T01:42:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.841+0000 7fa875c72640 1 -- 172.21.15.112:0/955666975 wait complete. 2024-04-25T01:42:03.840 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail a 2024-04-25T01:42:03.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.949+0000 7fb8b783f640 1 Processor -- start 2024-04-25T01:42:03.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.950+0000 7fb8b783f640 1 -- start start 2024-04-25T01:42:03.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.950+0000 7fb8b783f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8b01592c0 0x7fb8b00c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.950+0000 7fb8b783f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb8b01596a0 0x7fb8b0152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.950+0000 7fb8b783f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8b0159b90 con 0x7fb8b00c72b0 2024-04-25T01:42:03.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.950+0000 7fb8b783f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fb8b00c6e20 con 0x7fb8b01596a0 2024-04-25T01:42:03.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.950+0000 7fb8b783f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb8b006c970 con 0x7fb8b01592c0 2024-04-25T01:42:03.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.951+0000 7fb8b4dda640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb8b01596a0 0x7fb8b0152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.951+0000 7fb8b5ddc640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb8b00c72b0 0x7fb8b0155ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:43452/0 (socket says 172.21.15.112:43452) 2024-04-25T01:42:03.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.951+0000 7fb8b5ddc640 1 -- 172.21.15.112:0/2725906527 learned_addr learned my addr 172.21.15.112:0/2725906527 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:42:03.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.951+0000 7fb8b55db640 1 --2- 172.21.15.112:0/2725906527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8b01592c0 0x7fb8b00c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.951+0000 7fb8b4dda640 1 -- 172.21.15.112:0/2725906527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8b01592c0 msgr2=0x7fb8b00c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:03.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.951+0000 7fb8b4dda640 1 --2- 172.21.15.112:0/2725906527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8b01592c0 0x7fb8b00c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.951+0000 7fb8b4dda640 1 -- 172.21.15.112:0/2725906527 >> v1:172.21.15.112:6789/0 conn(0x7fb8b00c72b0 legacy=0x7fb8b0155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.951+0000 7fb8b4dda640 1 -- 172.21.15.112:0/2725906527 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b006cc50 con 0x7fb8b01596a0 2024-04-25T01:42:03.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.951+0000 7fb8a3fff640 1 -- 172.21.15.112:0/2725906527 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3937187905 0 0) 0x7fb8b0159b90 con 0x7fb8b00c72b0 2024-04-25T01:42:03.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.951+0000 7fb8b4dda640 1 --2- 172.21.15.112:0/2725906527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8b01596a0 0x7fb8b0152d00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0097a0 tx=0x7fb8ac02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f2003cf814a86a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:03.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.952+0000 7fb8a3fff640 1 -- 172.21.15.112:0/2725906527 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac037070 con 0x7fb8b01596a0 2024-04-25T01:42:03.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.952+0000 7fb8a3fff640 1 -- 172.21.15.112:0/2725906527 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb8ac02adb0 con 0x7fb8b01596a0 2024-04-25T01:42:03.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.952+0000 7fb8a3fff640 1 -- 172.21.15.112:0/2725906527 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac032680 con 0x7fb8b01596a0 2024-04-25T01:42:03.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.952+0000 7fb8b783f640 1 -- 172.21.15.112:0/2725906527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8b01596a0 msgr2=0x7fb8b0152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.952+0000 7fb8b783f640 1 --2- 172.21.15.112:0/2725906527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8b01596a0 0x7fb8b0152d00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0097a0 tx=0x7fb8ac02a060 comp rx=0 tx=0).stop 2024-04-25T01:42:03.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.952+0000 7fb8b783f640 1 -- 172.21.15.112:0/2725906527 shutdown_connections 2024-04-25T01:42:03.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.952+0000 7fb8b783f640 1 --2- 172.21.15.112:0/2725906527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8b01596a0 0x7fb8b0152d00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.952+0000 7fb8b783f640 1 --2- 172.21.15.112:0/2725906527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8b01592c0 0x7fb8b00c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.952+0000 7fb8b783f640 1 -- 172.21.15.112:0/2725906527 >> 172.21.15.112:0/2725906527 conn(0x7fb8b00cb170 msgr2=0x7fb8b00cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:03.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.952+0000 7fb8b783f640 1 -- 172.21.15.112:0/2725906527 shutdown_connections 2024-04-25T01:42:03.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.952+0000 7fb8b783f640 1 -- 172.21.15.112:0/2725906527 wait complete. 2024-04-25T01:42:03.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.954+0000 7fb8b783f640 1 Processor -- start 2024-04-25T01:42:03.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.954+0000 7fb8b783f640 1 -- start start 2024-04-25T01:42:03.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.954+0000 7fb8b783f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8b00c72b0 0x7fb8b0154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.954+0000 7fb8b783f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8b01592c0 0x7fb8b014bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.954+0000 7fb8b783f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8b01596a0 0x7fb8b014c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.954+0000 7fb8b783f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b006dca0 con 0x7fb8b01592c0 2024-04-25T01:42:03.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.954+0000 7fb8b783f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b006db20 con 0x7fb8b00c72b0 2024-04-25T01:42:03.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.954+0000 7fb8b783f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb8b00c95b0 con 0x7fb8b01596a0 2024-04-25T01:42:03.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.954+0000 7fb8b4dda640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8b01592c0 0x7fb8b014bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.954+0000 7fb8b4dda640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8b01592c0 0x7fb8b014bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37696/0 (socket says 172.21.15.112:37696) 2024-04-25T01:42:03.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.954+0000 7fb8b4dda640 1 -- 172.21.15.112:0/110221358 learned_addr learned my addr 172.21.15.112:0/110221358 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:03.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.954+0000 7fb8b55db640 1 --2- 172.21.15.112:0/110221358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8b00c72b0 0x7fb8b0154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.954+0000 7fb8b5ddc640 1 --2- 172.21.15.112:0/110221358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8b01596a0 0x7fb8b014c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.955+0000 7fb8b4dda640 1 -- 172.21.15.112:0/110221358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8b01596a0 msgr2=0x7fb8b014c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.955+0000 7fb8b4dda640 1 --2- 172.21.15.112:0/110221358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8b01596a0 0x7fb8b014c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.955+0000 7fb8b4dda640 1 -- 172.21.15.112:0/110221358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8b00c72b0 msgr2=0x7fb8b0154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.955+0000 7fb8b4dda640 1 --2- 172.21.15.112:0/110221358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8b00c72b0 0x7fb8b0154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.955+0000 7fb8b4dda640 1 -- 172.21.15.112:0/110221358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b01ef220 con 0x7fb8b01592c0 2024-04-25T01:42:03.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.955+0000 7fb8b55db640 1 --2- 172.21.15.112:0/110221358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8b00c72b0 0x7fb8b0154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:42:03.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.955+0000 7fb8b4dda640 1 --2- 172.21.15.112:0/110221358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8b01592c0 0x7fb8b014bc90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac035040 tx=0x7fb8ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:03.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.955+0000 7fb8a1ffb640 1 -- 172.21.15.112:0/110221358 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac037040 con 0x7fb8b01592c0 2024-04-25T01:42:03.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.955+0000 7fb8a1ffb640 1 -- 172.21.15.112:0/110221358 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb8ac004230 con 0x7fb8b01592c0 2024-04-25T01:42:03.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.955+0000 7fb8b783f640 1 -- 172.21.15.112:0/110221358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b01ef500 con 0x7fb8b01592c0 2024-04-25T01:42:03.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.955+0000 7fb8a1ffb640 1 -- 172.21.15.112:0/110221358 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac003610 con 0x7fb8b01592c0 2024-04-25T01:42:03.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.955+0000 7fb8b783f640 1 -- 172.21.15.112:0/110221358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b01efac0 con 0x7fb8b01592c0 2024-04-25T01:42:03.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.956+0000 7fb8b783f640 1 -- 172.21.15.112:0/110221358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8b00679a0 con 0x7fb8b01592c0 2024-04-25T01:42:03.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.956+0000 7fb8a1ffb640 1 -- 172.21.15.112:0/110221358 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb8ac02ac70 con 0x7fb8b01592c0 2024-04-25T01:42:03.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.957+0000 7fb8a1ffb640 1 --2- 172.21.15.112:0/110221358 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb89406be30 0x7fb89406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.957+0000 7fb8a1ffb640 1 -- 172.21.15.112:0/110221358 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6637+0+0 (secure 0 0 0) 0x7fb8ac0ac140 con 0x7fb8b01592c0 2024-04-25T01:42:03.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.957+0000 7fb8b55db640 1 --2- 172.21.15.112:0/110221358 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb89406be30 0x7fb89406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.957+0000 7fb8b55db640 1 --2- 172.21.15.112:0/110221358 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb89406be30 0x7fb89406e2f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4006fd0 tx=0x7fb8a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:03.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:03.961+0000 7fb8a1ffb640 1 -- 172.21.15.112:0/110221358 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb8ac07a870 con 0x7fb8b01592c0 2024-04-25T01:42:04.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.172+0000 7fb8b783f640 1 -- 172.21.15.112:0/110221358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "a"} v 0) -- 0x7fb8b00682a0 con 0x7fb8b01592c0 2024-04-25T01:42:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.174+0000 7fb8a1ffb640 1 -- 172.21.15.112:0/110221358 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "a"}]=0 a marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v12) ==== 151+0+0 (secure 0 0 0) 0x7fb8ac07a170 con 0x7fb8b01592c0 2024-04-25T01:42:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:a marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T01:42:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.176+0000 7fb8b783f640 1 -- 172.21.15.112:0/110221358 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb89406be30 msgr2=0x7fb89406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.176+0000 7fb8b783f640 1 --2- 172.21.15.112:0/110221358 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb89406be30 0x7fb89406e2f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4006fd0 tx=0x7fb8a4008040 comp rx=0 tx=0).stop 2024-04-25T01:42:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.176+0000 7fb8b783f640 1 -- 172.21.15.112:0/110221358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8b01592c0 msgr2=0x7fb8b014bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.176+0000 7fb8b783f640 1 --2- 172.21.15.112:0/110221358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8b01592c0 0x7fb8b014bc90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac035040 tx=0x7fb8ac0046c0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.176+0000 7fb8b783f640 1 -- 172.21.15.112:0/110221358 shutdown_connections 2024-04-25T01:42:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.176+0000 7fb8b783f640 1 --2- 172.21.15.112:0/110221358 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb89406be30 0x7fb89406e2f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.176+0000 7fb8b783f640 1 --2- 172.21.15.112:0/110221358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8b01596a0 0x7fb8b014c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.176+0000 7fb8b783f640 1 --2- 172.21.15.112:0/110221358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8b01592c0 0x7fb8b014bc90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.176+0000 7fb8b783f640 1 --2- 172.21.15.112:0/110221358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8b00c72b0 0x7fb8b0154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.176+0000 7fb8b783f640 1 -- 172.21.15.112:0/110221358 >> 172.21.15.112:0/110221358 conn(0x7fb8b00cb170 msgr2=0x7fb8b00ca420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.176+0000 7fb8b783f640 1 -- 172.21.15.112:0/110221358 shutdown_connections 2024-04-25T01:42:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.177+0000 7fb8b783f640 1 -- 172.21.15.112:0/110221358 wait complete. 2024-04-25T01:42:04.177 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm a --yes-i-really-mean-it 2024-04-25T01:42:04.230 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2024-04-25T01:42:04.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.291+0000 7fb97644d640 1 Processor -- start 2024-04-25T01:42:04.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.291+0000 7fb97644d640 1 -- start start 2024-04-25T01:42:04.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.291+0000 7fb97644d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9700c9250 0x7fb9700c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:04.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.291+0000 7fb97644d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9701534a0 con 0x7fb9700ca370 2024-04-25T01:42:04.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.291+0000 7fb97644d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9700cae30 con 0x7fb9700ca8a0 2024-04-25T01:42:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.291+0000 7fb97644d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb970156990 con 0x7fb9700c9250 2024-04-25T01:42:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96f7fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb9700ca370 0x7fb97006bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:43472/0 (socket says 172.21.15.112:43472) 2024-04-25T01:42:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96f7fe640 1 -- 172.21.15.112:0/1896825886 learned_addr learned my addr 172.21.15.112:0/1896825886 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:42:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96ffff640 1 --2- 172.21.15.112:0/1896825886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9700c9250 0x7fb9700c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96effd640 1 -- 172.21.15.112:0/1896825886 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2873475465 0 0) 0x7fb9701534a0 con 0x7fb9700ca370 2024-04-25T01:42:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96effd640 1 -- 172.21.15.112:0/1896825886 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb958003520 con 0x7fb9700ca370 2024-04-25T01:42:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96effd640 1 -- 172.21.15.112:0/1896825886 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3434132681 0 0) 0x7fb9700cae30 con 0x7fb9700ca8a0 2024-04-25T01:42:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96effd640 1 -- 172.21.15.112:0/1896825886 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9701534a0 con 0x7fb9700ca8a0 2024-04-25T01:42:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96effd640 1 -- 172.21.15.112:0/1896825886 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 990885518 0 0) 0x7fb958003520 con 0x7fb9700ca370 2024-04-25T01:42:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96effd640 1 -- 172.21.15.112:0/1896825886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9700c9250 msgr2=0x7fb9700c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96effd640 1 --2- 172.21.15.112:0/1896825886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9700c9250 0x7fb9700c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96effd640 1 -- 172.21.15.112:0/1896825886 >> v1:172.21.15.187:6789/0 conn(0x7fb9700ca8a0 legacy=0x7fb9701602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96effd640 1 -- 172.21.15.112:0/1896825886 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9700c9cd0 con 0x7fb9700ca370 2024-04-25T01:42:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96effd640 1 -- 172.21.15.112:0/1896825886 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fb964001670 con 0x7fb9700ca370 2024-04-25T01:42:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96ffff640 1 --2- 172.21.15.112:0/1896825886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9700c9250 0x7fb9700c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:42:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96effd640 1 -- 172.21.15.112:0/1896825886 >> v1:172.21.15.112:6789/0 conn(0x7fb9700ca370 legacy=0x7fb97006bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96effd640 1 --2- 172.21.15.112:0/1896825886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb958003b70 0x7fb958003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96effd640 1 --2- 172.21.15.112:0/1896825886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9700c9250 0x7fb958004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96effd640 1 --2- 172.21.15.112:0/1896825886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb958004d40 0x7fb9580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96effd640 1 -- 172.21.15.112:0/1896825886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb970156990 con 0x7fb958003b70 2024-04-25T01:42:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96effd640 1 -- 172.21.15.112:0/1896825886 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb95800bb80 con 0x7fb958004d40 2024-04-25T01:42:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96effd640 1 -- 172.21.15.112:0/1896825886 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb95800bd80 con 0x7fb9700c9250 2024-04-25T01:42:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96ffff640 1 --2- 172.21.15.112:0/1896825886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb958003b70 0x7fb958003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb96f7fe640 1 --2- 172.21.15.112:0/1896825886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9700c9250 0x7fb958004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.292+0000 7fb9749ea640 1 --2- 172.21.15.112:0/1896825886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb958004d40 0x7fb9580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.293+0000 7fb96ffff640 1 -- 172.21.15.112:0/1896825886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9700c9250 msgr2=0x7fb958004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.293+0000 7fb96ffff640 1 --2- 172.21.15.112:0/1896825886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9700c9250 0x7fb958004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.293+0000 7fb96ffff640 1 -- 172.21.15.112:0/1896825886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb958004d40 msgr2=0x7fb9580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.293+0000 7fb96ffff640 1 --2- 172.21.15.112:0/1896825886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb958004d40 0x7fb9580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.293+0000 7fb96ffff640 1 -- 172.21.15.112:0/1896825886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb958003520 con 0x7fb958003b70 2024-04-25T01:42:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.293+0000 7fb96f7fe640 1 --2- 172.21.15.112:0/1896825886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9700c9250 0x7fb958004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:42:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.293+0000 7fb9749ea640 1 --2- 172.21.15.112:0/1896825886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb958004d40 0x7fb9580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:42:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.293+0000 7fb96ffff640 1 --2- 172.21.15.112:0/1896825886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb958003b70 0x7fb958003f70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb960009c60 tx=0x7fb958005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a9eb20fe718f2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.293+0000 7fb96effd640 1 -- 172.21.15.112:0/1896825886 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb960036070 con 0x7fb958003b70 2024-04-25T01:42:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.293+0000 7fb96effd640 1 -- 172.21.15.112:0/1896825886 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb960031400 con 0x7fb958003b70 2024-04-25T01:42:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.293+0000 7fb96effd640 1 -- 172.21.15.112:0/1896825886 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb960009860 con 0x7fb958003b70 2024-04-25T01:42:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.294+0000 7fb97644d640 1 -- 172.21.15.112:0/1896825886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb958003b70 msgr2=0x7fb958003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.294+0000 7fb97644d640 1 --2- 172.21.15.112:0/1896825886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb958003b70 0x7fb958003f70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb960009c60 tx=0x7fb958005830 comp rx=0 tx=0).stop 2024-04-25T01:42:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.294+0000 7fb9749ea640 1 -- 172.21.15.112:0/1896825886 reap_dead start 2024-04-25T01:42:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.294+0000 7fb97644d640 1 -- 172.21.15.112:0/1896825886 shutdown_connections 2024-04-25T01:42:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.294+0000 7fb97644d640 1 -- 172.21.15.112:0/1896825886 >> 172.21.15.112:0/1896825886 conn(0x7fb9700cb310 msgr2=0x7fb9700cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.294+0000 7fb97644d640 1 -- 172.21.15.112:0/1896825886 shutdown_connections 2024-04-25T01:42:04.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.294+0000 7fb97644d640 1 -- 172.21.15.112:0/1896825886 wait complete. 2024-04-25T01:42:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.295+0000 7fb97644d640 1 Processor -- start 2024-04-25T01:42:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.295+0000 7fb97644d640 1 -- start start 2024-04-25T01:42:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.296+0000 7fb97644d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb958003b70 0x7fb9700c05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.296+0000 7fb97644d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9700bf640 0x7fb970158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.296+0000 7fb97644d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9701585a0 0x7fb97014da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.296+0000 7fb97644d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb970155eb0 con 0x7fb9700bf640 2024-04-25T01:42:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.296+0000 7fb97644d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb970156500 con 0x7fb9701585a0 2024-04-25T01:42:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.296+0000 7fb97644d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb970156190 con 0x7fb958003b70 2024-04-25T01:42:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.296+0000 7fb96f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9700bf640 0x7fb970158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.296+0000 7fb96f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9700bf640 0x7fb970158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37726/0 (socket says 172.21.15.112:37726) 2024-04-25T01:42:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.296+0000 7fb96f7fe640 1 -- 172.21.15.112:0/2521034103 learned_addr learned my addr 172.21.15.112:0/2521034103 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.296+0000 7fb96ffff640 1 --2- 172.21.15.112:0/2521034103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb958003b70 0x7fb9700c05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.296+0000 7fb9749ea640 1 --2- 172.21.15.112:0/2521034103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9701585a0 0x7fb97014da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.296+0000 7fb96f7fe640 1 -- 172.21.15.112:0/2521034103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb958003b70 msgr2=0x7fb9700c05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.296+0000 7fb96f7fe640 1 --2- 172.21.15.112:0/2521034103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb958003b70 0x7fb9700c05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.296+0000 7fb96f7fe640 1 -- 172.21.15.112:0/2521034103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9701585a0 msgr2=0x7fb97014da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.296+0000 7fb96f7fe640 1 --2- 172.21.15.112:0/2521034103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9701585a0 0x7fb97014da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.296+0000 7fb96f7fe640 1 -- 172.21.15.112:0/2521034103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9700c44c0 con 0x7fb9700bf640 2024-04-25T01:42:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.296+0000 7fb96f7fe640 1 --2- 172.21.15.112:0/2521034103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9700bf640 0x7fb970158060 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb96400e700 tx=0x7fb96400eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.297+0000 7fb96cff9640 1 -- 172.21.15.112:0/2521034103 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96400c5f0 con 0x7fb9700bf640 2024-04-25T01:42:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.297+0000 7fb96cff9640 1 -- 172.21.15.112:0/2521034103 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb964018070 con 0x7fb9700bf640 2024-04-25T01:42:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.297+0000 7fb97644d640 1 -- 172.21.15.112:0/2521034103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97014e830 con 0x7fb9700bf640 2024-04-25T01:42:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.297+0000 7fb96cff9640 1 -- 172.21.15.112:0/2521034103 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96400c990 con 0x7fb9700bf640 2024-04-25T01:42:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.297+0000 7fb97644d640 1 -- 172.21.15.112:0/2521034103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97014df90 con 0x7fb9700bf640 2024-04-25T01:42:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.298+0000 7fb97644d640 1 -- 172.21.15.112:0/2521034103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb97004f500 con 0x7fb9700bf640 2024-04-25T01:42:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.298+0000 7fb96cff9640 1 -- 172.21.15.112:0/2521034103 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb96400cb30 con 0x7fb9700bf640 2024-04-25T01:42:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.298+0000 7fb96cff9640 1 --2- 172.21.15.112:0/2521034103 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb95c06be30 0x7fb95c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.298+0000 7fb96cff9640 1 -- 172.21.15.112:0/2521034103 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6723+0+0 (secure 0 0 0) 0x7fb96408af10 con 0x7fb9700bf640 2024-04-25T01:42:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.298+0000 7fb96ffff640 1 --2- 172.21.15.112:0/2521034103 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb95c06be30 0x7fb95c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.299+0000 7fb96ffff640 1 --2- 172.21.15.112:0/2521034103 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb95c06be30 0x7fb95c06e2f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb960033000 tx=0x7fb960008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.303+0000 7fb96cff9640 1 -- 172.21.15.112:0/2521034103 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb96400ce20 con 0x7fb9700bf640 2024-04-25T01:42:04.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:04.510+0000 7fb97644d640 1 -- 172.21.15.112:0/2521034103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "a", "yes_i_really_mean_it": true} v 0) -- 0x7fb9700c9030 con 0x7fb9700bf640 2024-04-25T01:42:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.175+0000 7fb96cff9640 1 -- 172.21.15.112:0/2521034103 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "a", "yes_i_really_mean_it": true}]=0 v13) ==== 99+0+0 (secure 0 0 0) 0x7fb964059550 con 0x7fb9700bf640 2024-04-25T01:42:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.178+0000 7fb97644d640 1 -- 172.21.15.112:0/2521034103 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb95c06be30 msgr2=0x7fb95c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.178+0000 7fb97644d640 1 --2- 172.21.15.112:0/2521034103 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb95c06be30 0x7fb95c06e2f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb960033000 tx=0x7fb960008030 comp rx=0 tx=0).stop 2024-04-25T01:42:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.178+0000 7fb97644d640 1 -- 172.21.15.112:0/2521034103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9700bf640 msgr2=0x7fb970158060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.178+0000 7fb97644d640 1 --2- 172.21.15.112:0/2521034103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9700bf640 0x7fb970158060 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb96400e700 tx=0x7fb96400eb00 comp rx=0 tx=0).stop 2024-04-25T01:42:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.178+0000 7fb97644d640 1 -- 172.21.15.112:0/2521034103 shutdown_connections 2024-04-25T01:42:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.178+0000 7fb97644d640 1 --2- 172.21.15.112:0/2521034103 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb95c06be30 0x7fb95c06e2f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.178+0000 7fb97644d640 1 --2- 172.21.15.112:0/2521034103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9701585a0 0x7fb97014da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.178+0000 7fb97644d640 1 --2- 172.21.15.112:0/2521034103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9700bf640 0x7fb970158060 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.178+0000 7fb97644d640 1 --2- 172.21.15.112:0/2521034103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb958003b70 0x7fb9700c05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.178+0000 7fb97644d640 1 -- 172.21.15.112:0/2521034103 >> 172.21.15.112:0/2521034103 conn(0x7fb9700cb310 msgr2=0x7fb9701538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.178+0000 7fb97644d640 1 -- 172.21.15.112:0/2521034103 shutdown_connections 2024-04-25T01:42:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.178+0000 7fb97644d640 1 -- 172.21.15.112:0/2521034103 wait complete. 2024-04-25T01:42:05.181 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm a_metadata a_metadata --yes-i-really-really-mean-it 2024-04-25T01:42:05.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.298+0000 7f35ea49e640 1 Processor -- start 2024-04-25T01:42:05.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.299+0000 7f35ea49e640 1 -- start start 2024-04-25T01:42:05.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.299+0000 7f35ea49e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f35e414fbd0 0x7f35e40c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:05.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.299+0000 7f35ea49e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f35e406cef0 0x7f35e40c8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:05.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.299+0000 7f35ea49e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35e40c6270 con 0x7f35e40c85b0 2024-04-25T01:42:05.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.299+0000 7f35ea49e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f35e406c1c0 con 0x7f35e406cef0 2024-04-25T01:42:05.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.299+0000 7f35ea49e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f35e406c410 con 0x7f35e414fbd0 2024-04-25T01:42:05.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.299+0000 7f35e37fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f35e406cef0 0x7f35e40c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:05.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.299+0000 7f35e37fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f35e406cef0 0x7f35e40c8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37736/0 (socket says 172.21.15.112:37736) 2024-04-25T01:42:05.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.299+0000 7f35e37fe640 1 -- 172.21.15.112:0/2260213641 learned_addr learned my addr 172.21.15.112:0/2260213641 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:05.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.300+0000 7f35e3fff640 1 --2- 172.21.15.112:0/2260213641 >> v2:172.21.15.187:3300/0 conn(0x7f35e414fbd0 0x7f35e40c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:05.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.300+0000 7f35e37fe640 1 -- 172.21.15.112:0/2260213641 >> v2:172.21.15.187:3300/0 conn(0x7f35e414fbd0 msgr2=0x7f35e40c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:05.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.300+0000 7f35e37fe640 1 --2- 172.21.15.112:0/2260213641 >> v2:172.21.15.187:3300/0 conn(0x7f35e414fbd0 0x7f35e40c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:05.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.300+0000 7f35e37fe640 1 -- 172.21.15.112:0/2260213641 >> v1:172.21.15.187:6789/0 conn(0x7f35e40c85b0 legacy=0x7f35e40c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:05.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.300+0000 7f35e37fe640 1 -- 172.21.15.112:0/2260213641 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e406c6f0 con 0x7f35e406cef0 2024-04-25T01:42:05.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.300+0000 7f35e3fff640 1 --2- 172.21.15.112:0/2260213641 >> v2:172.21.15.187:3300/0 conn(0x7f35e414fbd0 0x7f35e40c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:42:05.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.300+0000 7f35e37fe640 1 --2- 172.21.15.112:0/2260213641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35e406cef0 0x7f35e40c8070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f35e40c8040 tx=0x7f35d802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ef43b43ccfa946 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:05.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.301+0000 7f35e2ffd640 1 -- 172.21.15.112:0/2260213641 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8037070 con 0x7f35e406cef0 2024-04-25T01:42:05.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.301+0000 7f35e2ffd640 1 -- 172.21.15.112:0/2260213641 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f35d802ae20 con 0x7f35e406cef0 2024-04-25T01:42:05.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.301+0000 7f35e2ffd640 1 -- 172.21.15.112:0/2260213641 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8032680 con 0x7f35e406cef0 2024-04-25T01:42:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.301+0000 7f35ea49e640 1 -- 172.21.15.112:0/2260213641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35e406cef0 msgr2=0x7f35e40c8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.301+0000 7f35ea49e640 1 --2- 172.21.15.112:0/2260213641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35e406cef0 0x7f35e40c8070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f35e40c8040 tx=0x7f35d802a110 comp rx=0 tx=0).stop 2024-04-25T01:42:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.301+0000 7f35ea49e640 1 -- 172.21.15.112:0/2260213641 shutdown_connections 2024-04-25T01:42:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.301+0000 7f35ea49e640 1 --2- 172.21.15.112:0/2260213641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35e406cef0 0x7f35e40c8070 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.301+0000 7f35ea49e640 1 --2- 172.21.15.112:0/2260213641 >> v2:172.21.15.187:3300/0 conn(0x7f35e414fbd0 0x7f35e40c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.301+0000 7f35ea49e640 1 -- 172.21.15.112:0/2260213641 >> 172.21.15.112:0/2260213641 conn(0x7f35e40ad050 msgr2=0x7f35e40ad460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.301+0000 7f35ea49e640 1 -- 172.21.15.112:0/2260213641 shutdown_connections 2024-04-25T01:42:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.301+0000 7f35ea49e640 1 -- 172.21.15.112:0/2260213641 wait complete. 2024-04-25T01:42:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.302+0000 7f35ea49e640 1 Processor -- start 2024-04-25T01:42:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.303+0000 7f35ea49e640 1 -- start start 2024-04-25T01:42:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.303+0000 7f35ea49e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35e406cef0 0x7f35e40c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.303+0000 7f35ea49e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35e40c85b0 0x7f35e41576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.303+0000 7f35ea49e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35e414fbd0 0x7f35e4157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.303+0000 7f35ea49e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f35e40c98f0 con 0x7f35e406cef0 2024-04-25T01:42:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.303+0000 7f35ea49e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f35e40c5f80 con 0x7f35e414fbd0 2024-04-25T01:42:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.303+0000 7f35ea49e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f35e41ef410 con 0x7f35e40c85b0 2024-04-25T01:42:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.303+0000 7f35e3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35e406cef0 0x7f35e40c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.303+0000 7f35e3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35e406cef0 0x7f35e40c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37744/0 (socket says 172.21.15.112:37744) 2024-04-25T01:42:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.303+0000 7f35e3fff640 1 -- 172.21.15.112:0/2369392440 learned_addr learned my addr 172.21.15.112:0/2369392440 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.304+0000 7f35e37fe640 1 --2- 172.21.15.112:0/2369392440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35e40c85b0 0x7f35e41576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.304+0000 7f35e8a3b640 1 --2- 172.21.15.112:0/2369392440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35e414fbd0 0x7f35e4157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.304+0000 7f35e3fff640 1 -- 172.21.15.112:0/2369392440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35e40c85b0 msgr2=0x7f35e41576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:05.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.304+0000 7f35e3fff640 1 --2- 172.21.15.112:0/2369392440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35e40c85b0 0x7f35e41576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:05.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.304+0000 7f35e3fff640 1 -- 172.21.15.112:0/2369392440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35e414fbd0 msgr2=0x7f35e4157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:05.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.304+0000 7f35e3fff640 1 --2- 172.21.15.112:0/2369392440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35e414fbd0 0x7f35e4157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:05.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.304+0000 7f35e3fff640 1 -- 172.21.15.112:0/2369392440 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e41ef8a0 con 0x7f35e406cef0 2024-04-25T01:42:05.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.304+0000 7f35e8a3b640 1 --2- 172.21.15.112:0/2369392440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35e414fbd0 0x7f35e4157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:42:05.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.304+0000 7f35e3fff640 1 --2- 172.21.15.112:0/2369392440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35e406cef0 0x7f35e40c9ee0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f35d400c7f0 tx=0x7f35d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:05.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.304+0000 7f35e0ff9640 1 -- 172.21.15.112:0/2369392440 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4015070 con 0x7f35e406cef0 2024-04-25T01:42:05.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.304+0000 7f35e0ff9640 1 -- 172.21.15.112:0/2369392440 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f35d4007820 con 0x7f35e406cef0 2024-04-25T01:42:05.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.304+0000 7f35ea49e640 1 -- 172.21.15.112:0/2369392440 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e41eeae0 con 0x7f35e406cef0 2024-04-25T01:42:05.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.304+0000 7f35e0ff9640 1 -- 172.21.15.112:0/2369392440 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4007bf0 con 0x7f35e406cef0 2024-04-25T01:42:05.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.304+0000 7f35ea49e640 1 -- 172.21.15.112:0/2369392440 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e41ef0a0 con 0x7f35e406cef0 2024-04-25T01:42:05.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.305+0000 7f35ea49e640 1 -- 172.21.15.112:0/2369392440 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e40682a0 con 0x7f35e406cef0 2024-04-25T01:42:05.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.305+0000 7f35e0ff9640 1 -- 172.21.15.112:0/2369392440 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f35d4018070 con 0x7f35e406cef0 2024-04-25T01:42:05.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.306+0000 7f35e0ff9640 1 --2- 172.21.15.112:0/2369392440 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f35d006be60 0x7f35d006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:05.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.306+0000 7f35e37fe640 1 --2- 172.21.15.112:0/2369392440 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f35d006be60 0x7f35d006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:05.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.306+0000 7f35e0ff9640 1 -- 172.21.15.112:0/2369392440 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6723+0+0 (secure 0 0 0) 0x7f35d408b350 con 0x7f35e406cef0 2024-04-25T01:42:05.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.306+0000 7f35e37fe640 1 --2- 172.21.15.112:0/2369392440 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f35d006be60 0x7f35d006e320 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f35d8034000 tx=0x7f35d802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:05.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.311+0000 7f35e0ff9640 1 -- 172.21.15.112:0/2369392440 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f35d4059900 con 0x7f35e406cef0 2024-04-25T01:42:05.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:05.484+0000 7f35ea49e640 1 -- 172.21.15.112:0/2369392440 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "a_metadata", "pool2": "a_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f35e40cae40 con 0x7f35e406cef0 2024-04-25T01:42:06.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.185+0000 7f35e0ff9640 1 -- 172.21.15.112:0/2369392440 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "a_metadata", "pool2": "a_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'a_metadata' removed v28) ==== 166+0+0 (secure 0 0 0) 0x7f35d4059200 con 0x7f35e406cef0 2024-04-25T01:42:06.173 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'a_metadata' removed 2024-04-25T01:42:06.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.187+0000 7f35ea49e640 1 -- 172.21.15.112:0/2369392440 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f35d006be60 msgr2=0x7f35d006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:06.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.187+0000 7f35ea49e640 1 --2- 172.21.15.112:0/2369392440 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f35d006be60 0x7f35d006e320 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f35d8034000 tx=0x7f35d802e000 comp rx=0 tx=0).stop 2024-04-25T01:42:06.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.187+0000 7f35ea49e640 1 -- 172.21.15.112:0/2369392440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35e406cef0 msgr2=0x7f35e40c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:06.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.187+0000 7f35ea49e640 1 --2- 172.21.15.112:0/2369392440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35e406cef0 0x7f35e40c9ee0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f35d400c7f0 tx=0x7f35d400ccb0 comp rx=0 tx=0).stop 2024-04-25T01:42:06.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.187+0000 7f35ea49e640 1 -- 172.21.15.112:0/2369392440 shutdown_connections 2024-04-25T01:42:06.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.187+0000 7f35ea49e640 1 --2- 172.21.15.112:0/2369392440 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f35d006be60 0x7f35d006e320 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:06.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.187+0000 7f35ea49e640 1 --2- 172.21.15.112:0/2369392440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35e414fbd0 0x7f35e4157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:06.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.187+0000 7f35ea49e640 1 --2- 172.21.15.112:0/2369392440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35e40c85b0 0x7f35e41576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:06.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.187+0000 7f35ea49e640 1 --2- 172.21.15.112:0/2369392440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35e406cef0 0x7f35e40c9ee0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:06.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.187+0000 7f35ea49e640 1 -- 172.21.15.112:0/2369392440 >> 172.21.15.112:0/2369392440 conn(0x7f35e40ad050 msgr2=0x7f35e40ad430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:06.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.188+0000 7f35ea49e640 1 -- 172.21.15.112:0/2369392440 shutdown_connections 2024-04-25T01:42:06.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.188+0000 7f35ea49e640 1 -- 172.21.15.112:0/2369392440 wait complete. 2024-04-25T01:42:06.189 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm a_data a_data --yes-i-really-really-mean-it 2024-04-25T01:42:06.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.306+0000 7f1ebf45d640 1 Processor -- start 2024-04-25T01:42:06.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.307+0000 7f1ebf45d640 1 -- start start 2024-04-25T01:42:06.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.307+0000 7f1ebf45d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1eb80c2d70 0x7f1eb81524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:06.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.307+0000 7f1ebf45d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb806e600 0x7f1eb806cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:06.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.307+0000 7f1ebf45d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eb806d560 con 0x7f1eb814c380 2024-04-25T01:42:06.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.307+0000 7f1ebf45d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1eb806d840 con 0x7f1eb80c2d70 2024-04-25T01:42:06.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.307+0000 7f1ebf45d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb806db20 con 0x7f1eb806e600 2024-04-25T01:42:06.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.308+0000 7f1ebd1f9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1eb80c2d70 0x7f1eb81524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:06.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.308+0000 7f1ebc9f8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb806e600 0x7f1eb806cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:06.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.308+0000 7f1ebd1f9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1eb80c2d70 0x7f1eb81524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:48702/0 (socket says 172.21.15.112:48702) 2024-04-25T01:42:06.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.308+0000 7f1ebc9f8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb806e600 0x7f1eb806cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:34688/0 (socket says 172.21.15.112:34688) 2024-04-25T01:42:06.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.308+0000 7f1ebd1f9640 1 -- 172.21.15.112:0/3095136504 learned_addr learned my addr 172.21.15.112:0/3095136504 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:06.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.308+0000 7f1ebc9f8640 1 -- 172.21.15.112:0/3095136504 >> v2:172.21.15.187:3300/0 conn(0x7f1eb80c2d70 msgr2=0x7f1eb81524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:06.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.308+0000 7f1ebc9f8640 1 --2- 172.21.15.112:0/3095136504 >> v2:172.21.15.187:3300/0 conn(0x7f1eb80c2d70 0x7f1eb81524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:06.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.308+0000 7f1ebc9f8640 1 -- 172.21.15.112:0/3095136504 >> v1:172.21.15.187:6789/0 conn(0x7f1eb814c380 legacy=0x7f1eb81539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:06.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.308+0000 7f1ebc9f8640 1 -- 172.21.15.112:0/3095136504 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb806de00 con 0x7f1eb806e600 2024-04-25T01:42:06.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.308+0000 7f1ebd1f9640 1 --2- 172.21.15.112:0/3095136504 >> v2:172.21.15.187:3300/0 conn(0x7f1eb80c2d70 0x7f1eb81524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:42:06.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.309+0000 7f1ebc9f8640 1 --2- 172.21.15.112:0/3095136504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb806e600 0x7f1eb806cf10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80097a0 tx=0x7f1ea802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c53b5ef864e91a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:06.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.309+0000 7f1eaffff640 1 -- 172.21.15.112:0/3095136504 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8037070 con 0x7f1eb806e600 2024-04-25T01:42:06.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.309+0000 7f1eaffff640 1 -- 172.21.15.112:0/3095136504 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1ea802ae50 con 0x7f1eb806e600 2024-04-25T01:42:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.309+0000 7f1eaffff640 1 -- 172.21.15.112:0/3095136504 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8032680 con 0x7f1eb806e600 2024-04-25T01:42:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.309+0000 7f1ebf45d640 1 -- 172.21.15.112:0/3095136504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb806e600 msgr2=0x7f1eb806cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.310+0000 7f1ebf45d640 1 --2- 172.21.15.112:0/3095136504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb806e600 0x7f1eb806cf10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80097a0 tx=0x7f1ea802a100 comp rx=0 tx=0).stop 2024-04-25T01:42:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.310+0000 7f1ebf45d640 1 -- 172.21.15.112:0/3095136504 shutdown_connections 2024-04-25T01:42:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.310+0000 7f1ebf45d640 1 --2- 172.21.15.112:0/3095136504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb806e600 0x7f1eb806cf10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.310+0000 7f1ebf45d640 1 --2- 172.21.15.112:0/3095136504 >> v2:172.21.15.187:3300/0 conn(0x7f1eb80c2d70 0x7f1eb81524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.310+0000 7f1ebf45d640 1 -- 172.21.15.112:0/3095136504 >> 172.21.15.112:0/3095136504 conn(0x7f1eb806b8a0 msgr2=0x7f1eb80c6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.310+0000 7f1ebf45d640 1 -- 172.21.15.112:0/3095136504 shutdown_connections 2024-04-25T01:42:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.310+0000 7f1ebf45d640 1 -- 172.21.15.112:0/3095136504 wait complete. 2024-04-25T01:42:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.311+0000 7f1ebf45d640 1 Processor -- start 2024-04-25T01:42:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.311+0000 7f1ebf45d640 1 -- start start 2024-04-25T01:42:06.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.311+0000 7f1ebf45d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1eb806e600 0x7f1eb81eb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:06.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.311+0000 7f1ebf45d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb80c2d70 0x7f1eb80c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:06.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.311+0000 7f1ebf45d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb814c380 0x7f1eb81eed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:06.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.311+0000 7f1ebf45d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb81540b0 con 0x7f1eb80c2d70 2024-04-25T01:42:06.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.311+0000 7f1ebf45d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb80c1690 con 0x7f1eb806e600 2024-04-25T01:42:06.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.311+0000 7f1ebf45d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb8154230 con 0x7f1eb814c380 2024-04-25T01:42:06.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.311+0000 7f1ebc9f8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb80c2d70 0x7f1eb80c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:06.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.312+0000 7f1ebc9f8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb80c2d70 0x7f1eb80c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37754/0 (socket says 172.21.15.112:37754) 2024-04-25T01:42:06.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.312+0000 7f1ebc9f8640 1 -- 172.21.15.112:0/2165627567 learned_addr learned my addr 172.21.15.112:0/2165627567 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:06.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.312+0000 7f1ebd1f9640 1 --2- 172.21.15.112:0/2165627567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1eb806e600 0x7f1eb81eb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:06.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.312+0000 7f1ebd9fa640 1 --2- 172.21.15.112:0/2165627567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb814c380 0x7f1eb81eed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:06.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.312+0000 7f1ebc9f8640 1 -- 172.21.15.112:0/2165627567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb814c380 msgr2=0x7f1eb81eed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:06.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.312+0000 7f1ebc9f8640 1 --2- 172.21.15.112:0/2165627567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb814c380 0x7f1eb81eed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:06.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.312+0000 7f1ebc9f8640 1 -- 172.21.15.112:0/2165627567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1eb806e600 msgr2=0x7f1eb81eb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:06.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.312+0000 7f1ebc9f8640 1 --2- 172.21.15.112:0/2165627567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1eb806e600 0x7f1eb81eb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:06.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.312+0000 7f1ebc9f8640 1 -- 172.21.15.112:0/2165627567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb81544d0 con 0x7f1eb80c2d70 2024-04-25T01:42:06.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.312+0000 7f1ebc9f8640 1 --2- 172.21.15.112:0/2165627567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb80c2d70 0x7f1eb80c5b40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8000c00 tx=0x7f1ea80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:06.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.312+0000 7f1eadffb640 1 -- 172.21.15.112:0/2165627567 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8041070 con 0x7f1eb80c2d70 2024-04-25T01:42:06.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.312+0000 7f1eadffb640 1 -- 172.21.15.112:0/2165627567 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1ea803c670 con 0x7f1eb80c2d70 2024-04-25T01:42:06.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.312+0000 7f1eadffb640 1 -- 172.21.15.112:0/2165627567 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8037040 con 0x7f1eb80c2d70 2024-04-25T01:42:06.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.312+0000 7f1ebf45d640 1 -- 172.21.15.112:0/2165627567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb80ac320 con 0x7f1eb80c2d70 2024-04-25T01:42:06.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.312+0000 7f1ebf45d640 1 -- 172.21.15.112:0/2165627567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb80ab420 con 0x7f1eb80c2d70 2024-04-25T01:42:06.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.314+0000 7f1ebf45d640 1 -- 172.21.15.112:0/2165627567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb814c1a0 con 0x7f1eb80c2d70 2024-04-25T01:42:06.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.314+0000 7f1eadffb640 1 -- 172.21.15.112:0/2165627567 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ea803a070 con 0x7f1eb80c2d70 2024-04-25T01:42:06.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.314+0000 7f1eadffb640 1 --2- 172.21.15.112:0/2165627567 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e9c06be30 0x7f1e9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:06.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.314+0000 7f1eadffb640 1 -- 172.21.15.112:0/2165627567 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6295+0+0 (secure 0 0 0) 0x7f1ea80ab3e0 con 0x7f1eb80c2d70 2024-04-25T01:42:06.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.314+0000 7f1ebd1f9640 1 --2- 172.21.15.112:0/2165627567 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e9c06be30 0x7f1e9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:06.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.315+0000 7f1ebd1f9640 1 --2- 172.21.15.112:0/2165627567 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e9c06be30 0x7f1e9c06e2f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40072c0 tx=0x7f1eb400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:06.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.319+0000 7f1eadffb640 1 -- 172.21.15.112:0/2165627567 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1eb814c1a0 con 0x7f1eb80c2d70 2024-04-25T01:42:06.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:06.493+0000 7f1ebf45d640 1 -- 172.21.15.112:0/2165627567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "a_data", "pool2": "a_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1eb80682a0 con 0x7f1eb80c2d70 2024-04-25T01:42:07.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.190+0000 7f1eadffb640 1 -- 172.21.15.112:0/2165627567 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "a_data", "pool2": "a_data", "yes_i_really_really_mean_it": true}]=0 pool 'a_data' removed v29) ==== 154+0+0 (secure 0 0 0) 0x7f1ea80794e0 con 0x7f1eb80c2d70 2024-04-25T01:42:07.178 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'a_data' removed 2024-04-25T01:42:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.194+0000 7f1ebf45d640 1 -- 172.21.15.112:0/2165627567 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e9c06be30 msgr2=0x7f1e9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.194+0000 7f1ebf45d640 1 --2- 172.21.15.112:0/2165627567 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e9c06be30 0x7f1e9c06e2f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40072c0 tx=0x7f1eb400b040 comp rx=0 tx=0).stop 2024-04-25T01:42:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.194+0000 7f1ebf45d640 1 -- 172.21.15.112:0/2165627567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb80c2d70 msgr2=0x7f1eb80c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.194+0000 7f1ebf45d640 1 --2- 172.21.15.112:0/2165627567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb80c2d70 0x7f1eb80c5b40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8000c00 tx=0x7f1ea80046c0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.194+0000 7f1ebf45d640 1 -- 172.21.15.112:0/2165627567 shutdown_connections 2024-04-25T01:42:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.194+0000 7f1ebf45d640 1 --2- 172.21.15.112:0/2165627567 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e9c06be30 0x7f1e9c06e2f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.194+0000 7f1ebf45d640 1 --2- 172.21.15.112:0/2165627567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb814c380 0x7f1eb81eed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.194+0000 7f1ebf45d640 1 --2- 172.21.15.112:0/2165627567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb80c2d70 0x7f1eb80c5b40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.194+0000 7f1ebf45d640 1 --2- 172.21.15.112:0/2165627567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1eb806e600 0x7f1eb81eb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.194+0000 7f1ebf45d640 1 -- 172.21.15.112:0/2165627567 >> 172.21.15.112:0/2165627567 conn(0x7f1eb806b8a0 msgr2=0x7f1eb81538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.195+0000 7f1ebf45d640 1 -- 172.21.15.112:0/2165627567 shutdown_connections 2024-04-25T01:42:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.195+0000 7f1ebf45d640 1 -- 172.21.15.112:0/2165627567 wait complete. 2024-04-25T01:42:07.199 INFO:tasks.cephfs.filesystem:Destroying file system b and related pools 2024-04-25T01:42:07.200 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:42:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.338+0000 7fe40d69e640 1 Processor -- start 2024-04-25T01:42:07.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.338+0000 7fe40d69e640 1 -- start start 2024-04-25T01:42:07.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.339+0000 7fe40d69e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe4081596a0 0x7fe408152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:07.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.339+0000 7fe40d69e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe4080c72b0 0x7fe408160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:07.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.339+0000 7fe40d69e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe408159b90 con 0x7fe4081592c0 2024-04-25T01:42:07.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.339+0000 7fe40d69e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fe4080c6e20 con 0x7fe4081596a0 2024-04-25T01:42:07.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.339+0000 7fe40d69e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe40806c970 con 0x7fe4080c72b0 2024-04-25T01:42:07.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.339+0000 7fe3fffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe4081596a0 0x7fe408152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:07.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.339+0000 7fe3fffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe4081596a0 0x7fe408152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37768/0 (socket says 172.21.15.112:37768) 2024-04-25T01:42:07.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.339+0000 7fe3fffff640 1 -- 172.21.15.112:0/296652307 learned_addr learned my addr 172.21.15.112:0/296652307 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:07.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.339+0000 7fe3fffff640 1 -- 172.21.15.112:0/296652307 >> v2:172.21.15.187:3300/0 conn(0x7fe4080c72b0 msgr2=0x7fe408160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:07.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.339+0000 7fe4077fe640 1 --2- 172.21.15.112:0/296652307 >> v2:172.21.15.187:3300/0 conn(0x7fe4080c72b0 0x7fe408160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:07.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.340+0000 7fe3fffff640 1 --2- 172.21.15.112:0/296652307 >> v2:172.21.15.187:3300/0 conn(0x7fe4080c72b0 0x7fe408160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.340+0000 7fe3fffff640 1 -- 172.21.15.112:0/296652307 >> v1:172.21.15.187:6789/0 conn(0x7fe4081592c0 legacy=0x7fe4080c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:07.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.340+0000 7fe3fffff640 1 -- 172.21.15.112:0/296652307 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40806cc50 con 0x7fe4081596a0 2024-04-25T01:42:07.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.340+0000 7fe4077fe640 1 --2- 172.21.15.112:0/296652307 >> v2:172.21.15.187:3300/0 conn(0x7fe4080c72b0 0x7fe408160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:42:07.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.340+0000 7fe3fffff640 1 --2- 172.21.15.112:0/296652307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4081596a0 0x7fe408152d00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe4080c21d0 tx=0x7fe3f002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51c154379031b19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:07.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.340+0000 7fe4067fc640 1 -- 172.21.15.112:0/296652307 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0037070 con 0x7fe4081596a0 2024-04-25T01:42:07.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.340+0000 7fe4067fc640 1 -- 172.21.15.112:0/296652307 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe3f002ae20 con 0x7fe4081596a0 2024-04-25T01:42:07.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.340+0000 7fe4067fc640 1 -- 172.21.15.112:0/296652307 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0032680 con 0x7fe4081596a0 2024-04-25T01:42:07.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.341+0000 7fe40d69e640 1 -- 172.21.15.112:0/296652307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4081596a0 msgr2=0x7fe408152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:07.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.341+0000 7fe40d69e640 1 --2- 172.21.15.112:0/296652307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4081596a0 0x7fe408152d00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe4080c21d0 tx=0x7fe3f002a110 comp rx=0 tx=0).stop 2024-04-25T01:42:07.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.341+0000 7fe40d69e640 1 -- 172.21.15.112:0/296652307 shutdown_connections 2024-04-25T01:42:07.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.341+0000 7fe40d69e640 1 --2- 172.21.15.112:0/296652307 >> v2:172.21.15.187:3300/0 conn(0x7fe4080c72b0 0x7fe408160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.341+0000 7fe40d69e640 1 --2- 172.21.15.112:0/296652307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4081596a0 0x7fe408152d00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.341+0000 7fe40d69e640 1 -- 172.21.15.112:0/296652307 >> 172.21.15.112:0/296652307 conn(0x7fe4080cb170 msgr2=0x7fe4080cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.341+0000 7fe40d69e640 1 -- 172.21.15.112:0/296652307 shutdown_connections 2024-04-25T01:42:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.341+0000 7fe40d69e640 1 -- 172.21.15.112:0/296652307 wait complete. 2024-04-25T01:42:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.342+0000 7fe40d69e640 1 Processor -- start 2024-04-25T01:42:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.342+0000 7fe40d69e640 1 -- start start 2024-04-25T01:42:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.343+0000 7fe40d69e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4080c72b0 0x7fe4081efb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.343+0000 7fe40d69e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4081592c0 0x7fe408158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.343+0000 7fe40d69e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4081596a0 0x7fe4080bfee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:07.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.343+0000 7fe40d69e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe40806c510 con 0x7fe4080c72b0 2024-04-25T01:42:07.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.343+0000 7fe40d69e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe40806c390 con 0x7fe4081592c0 2024-04-25T01:42:07.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.343+0000 7fe40d69e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe40806db20 con 0x7fe4081596a0 2024-04-25T01:42:07.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.343+0000 7fe406ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4080c72b0 0x7fe4081efb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:07.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.343+0000 7fe406ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4080c72b0 0x7fe4081efb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37784/0 (socket says 172.21.15.112:37784) 2024-04-25T01:42:07.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.343+0000 7fe406ffd640 1 -- 172.21.15.112:0/2633432735 learned_addr learned my addr 172.21.15.112:0/2633432735 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:07.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.343+0000 7fe3fffff640 1 --2- 172.21.15.112:0/2633432735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4081592c0 0x7fe408158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:07.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.343+0000 7fe4077fe640 1 --2- 172.21.15.112:0/2633432735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4081596a0 0x7fe4080bfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:07.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.343+0000 7fe406ffd640 1 -- 172.21.15.112:0/2633432735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4081596a0 msgr2=0x7fe4080bfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:07.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.343+0000 7fe406ffd640 1 --2- 172.21.15.112:0/2633432735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4081596a0 0x7fe4080bfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.343+0000 7fe406ffd640 1 -- 172.21.15.112:0/2633432735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4081592c0 msgr2=0x7fe408158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:07.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.343+0000 7fe406ffd640 1 --2- 172.21.15.112:0/2633432735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4081592c0 0x7fe408158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.343+0000 7fe406ffd640 1 -- 172.21.15.112:0/2633432735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4081f0150 con 0x7fe4080c72b0 2024-04-25T01:42:07.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.344+0000 7fe406ffd640 1 --2- 172.21.15.112:0/2633432735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4080c72b0 0x7fe4081efb70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800c430 tx=0x7fe3f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:07.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.344+0000 7fe3ff7fe640 1 -- 172.21.15.112:0/2633432735 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8011790 con 0x7fe4080c72b0 2024-04-25T01:42:07.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.344+0000 7fe3ff7fe640 1 -- 172.21.15.112:0/2633432735 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe3f8017070 con 0x7fe4080c72b0 2024-04-25T01:42:07.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.344+0000 7fe40d69e640 1 -- 172.21.15.112:0/2633432735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4081ecc10 con 0x7fe4080c72b0 2024-04-25T01:42:07.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.344+0000 7fe3ff7fe640 1 -- 172.21.15.112:0/2633432735 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8011b90 con 0x7fe4080c72b0 2024-04-25T01:42:07.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.344+0000 7fe40d69e640 1 -- 172.21.15.112:0/2633432735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4081ec130 con 0x7fe4080c72b0 2024-04-25T01:42:07.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.345+0000 7fe40d69e640 1 -- 172.21.15.112:0/2633432735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4080682a0 con 0x7fe4080c72b0 2024-04-25T01:42:07.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.345+0000 7fe3ff7fe640 1 -- 172.21.15.112:0/2633432735 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe3f801c460 con 0x7fe4080c72b0 2024-04-25T01:42:07.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.346+0000 7fe3ff7fe640 1 --2- 172.21.15.112:0/2633432735 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe3e806be30 0x7fe3e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:07.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.346+0000 7fe3fffff640 1 --2- 172.21.15.112:0/2633432735 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe3e806be30 0x7fe3e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:07.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.346+0000 7fe3ff7fe640 1 -- 172.21.15.112:0/2633432735 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5775+0+0 (secure 0 0 0) 0x7fe3f808ab10 con 0x7fe4080c72b0 2024-04-25T01:42:07.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.346+0000 7fe3fffff640 1 --2- 172.21.15.112:0/2633432735 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe3e806be30 0x7fe3e806e2f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0034000 tx=0x7fe3f002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:07.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.350+0000 7fe3ff7fe640 1 -- 172.21.15.112:0/2633432735 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe3f8058ec0 con 0x7fe4080c72b0 2024-04-25T01:42:07.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.560+0000 7fe40d69e640 1 -- 172.21.15.112:0/2633432735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4080c79d0 con 0x7fe4080c72b0 2024-04-25T01:42:07.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.561+0000 7fe3ff7fe640 1 -- 172.21.15.112:0/2633432735 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+5186 (secure 0 0 0) 0x7fe3f805d1c0 con 0x7fe4080c72b0 2024-04-25T01:42:07.550 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:42:07.550 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/4047219598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4047219598},{"type":"v1","addr":"172.21.15.187:6834","nonce":4047219598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2204640907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2204640907},{"type":"v1","addr":"172.21.15.187:6835","nonce":2204640907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":144528,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1831476248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1831476248},{"type":"v1","addr":"172.21.15.112:6837","nonce":1831476248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:26:01.740521+0000","modified":"2024-04-25T01:26:04.746955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":8,"state":"up:active","state_seq":6,"addr":"172.21.15.112:6835/278686477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":278686477},{"type":"v1","addr":"172.21.15.112:6835","nonce":278686477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":2}]} 2024-04-25T01:42:07.550 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 13 2024-04-25T01:42:07.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.563+0000 7fe40d69e640 1 -- 172.21.15.112:0/2633432735 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe3e806be30 msgr2=0x7fe3e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:07.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.563+0000 7fe40d69e640 1 --2- 172.21.15.112:0/2633432735 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe3e806be30 0x7fe3e806e2f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0034000 tx=0x7fe3f002e000 comp rx=0 tx=0).stop 2024-04-25T01:42:07.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.563+0000 7fe40d69e640 1 -- 172.21.15.112:0/2633432735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4080c72b0 msgr2=0x7fe4081efb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:07.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.563+0000 7fe40d69e640 1 --2- 172.21.15.112:0/2633432735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4080c72b0 0x7fe4081efb70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800c430 tx=0x7fe3f800c8f0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.563+0000 7fe40d69e640 1 -- 172.21.15.112:0/2633432735 shutdown_connections 2024-04-25T01:42:07.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.563+0000 7fe40d69e640 1 --2- 172.21.15.112:0/2633432735 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fe3e806be30 0x7fe3e806e2f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.563+0000 7fe40d69e640 1 --2- 172.21.15.112:0/2633432735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4081596a0 0x7fe4080bfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.563+0000 7fe40d69e640 1 --2- 172.21.15.112:0/2633432735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4081592c0 0x7fe408158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.563+0000 7fe40d69e640 1 --2- 172.21.15.112:0/2633432735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4080c72b0 0x7fe4081efb70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.563+0000 7fe40d69e640 1 -- 172.21.15.112:0/2633432735 >> 172.21.15.112:0/2633432735 conn(0x7fe4080cb170 msgr2=0x7fe408067110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:07.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.563+0000 7fe40d69e640 1 -- 172.21.15.112:0/2633432735 shutdown_connections 2024-04-25T01:42:07.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.563+0000 7fe40d69e640 1 -- 172.21.15.112:0/2633432735 wait complete. 2024-04-25T01:42:07.563 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:42:07.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.675+0000 7f476b390640 1 Processor -- start 2024-04-25T01:42:07.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.675+0000 7f476b390640 1 -- start start 2024-04-25T01:42:07.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.675+0000 7f476b390640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4764072d80 0x7f47641536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:07.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f476b390640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f476406e0f0 con 0x7f47640c1a80 2024-04-25T01:42:07.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f476b390640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47640c2180 con 0x7f4764071db0 2024-04-25T01:42:07.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f476b390640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f47640c1f30 con 0x7f4764072d80 2024-04-25T01:42:07.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f476892b640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f47640c1a80 0x7f476414d340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:40310/0 (socket says 172.21.15.112:40310) 2024-04-25T01:42:07.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f476892b640 1 -- 172.21.15.112:0/3852980633 learned_addr learned my addr 172.21.15.112:0/3852980633 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:42:07.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f476912c640 1 --2- 172.21.15.112:0/3852980633 >> v2:172.21.15.187:3300/0 conn(0x7f4764072d80 0x7f47641536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:07.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f475bfff640 1 -- 172.21.15.112:0/3852980633 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2016963798 0 0) 0x7f4760001110 con 0x7f47640c1a80 2024-04-25T01:42:07.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f475bfff640 1 -- 172.21.15.112:0/3852980633 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4750003520 con 0x7f47640c1a80 2024-04-25T01:42:07.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f475bfff640 1 -- 172.21.15.112:0/3852980633 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3119488978 0 0) 0x7f4750003520 con 0x7f47640c1a80 2024-04-25T01:42:07.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f475bfff640 1 -- 172.21.15.112:0/3852980633 >> v2:172.21.15.187:3300/0 conn(0x7f4764072d80 msgr2=0x7f47641536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:07.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f475bfff640 1 --2- 172.21.15.112:0/3852980633 >> v2:172.21.15.187:3300/0 conn(0x7f4764072d80 0x7f47641536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f475bfff640 1 -- 172.21.15.112:0/3852980633 >> v1:172.21.15.187:6789/0 conn(0x7f4764071db0 legacy=0x7f47640721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:07.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f475bfff640 1 -- 172.21.15.112:0/3852980633 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47640c2460 con 0x7f47640c1a80 2024-04-25T01:42:07.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f475bfff640 1 -- 172.21.15.112:0/3852980633 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7f4760001840 con 0x7f47640c1a80 2024-04-25T01:42:07.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f475bfff640 1 -- 172.21.15.112:0/3852980633 >> v1:172.21.15.112:6789/0 conn(0x7f47640c1a80 legacy=0x7f476414d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:07.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f475bfff640 1 --2- 172.21.15.112:0/3852980633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47500039b0 0x7f4750003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:07.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f475bfff640 1 --2- 172.21.15.112:0/3852980633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4750004800 0x7f4750004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:07.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f475bfff640 1 --2- 172.21.15.112:0/3852980633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47500051c0 0x7f475000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:07.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f475bfff640 1 -- 172.21.15.112:0/3852980633 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f47640c1f30 con 0x7f47500039b0 2024-04-25T01:42:07.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f475bfff640 1 -- 172.21.15.112:0/3852980633 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f475000c060 con 0x7f4750004800 2024-04-25T01:42:07.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f475bfff640 1 -- 172.21.15.112:0/3852980633 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f475000c260 con 0x7f47500051c0 2024-04-25T01:42:07.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f476912c640 1 --2- 172.21.15.112:0/3852980633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47500039b0 0x7f4750003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:07.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f476892b640 1 --2- 172.21.15.112:0/3852980633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4750004800 0x7f4750004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:07.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.676+0000 7f476992d640 1 --2- 172.21.15.112:0/3852980633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47500051c0 0x7f475000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:07.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.677+0000 7f476912c640 1 -- 172.21.15.112:0/3852980633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47500051c0 msgr2=0x7f475000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:07.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.677+0000 7f476912c640 1 --2- 172.21.15.112:0/3852980633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47500051c0 0x7f475000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.677+0000 7f476912c640 1 -- 172.21.15.112:0/3852980633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4750004800 msgr2=0x7f4750004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:07.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.677+0000 7f476912c640 1 --2- 172.21.15.112:0/3852980633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4750004800 0x7f4750004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.677+0000 7f476912c640 1 -- 172.21.15.112:0/3852980633 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4750003520 con 0x7f47500039b0 2024-04-25T01:42:07.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.677+0000 7f476892b640 1 --2- 172.21.15.112:0/3852980633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4750004800 0x7f4750004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:42:07.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.677+0000 7f476992d640 1 -- 172.21.15.112:0/3852980633 reap_dead start 2024-04-25T01:42:07.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.677+0000 7f476912c640 1 --2- 172.21.15.112:0/3852980633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47500039b0 0x7f4750003dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4754009ac0 tx=0x7f4750005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb00c18c114573f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:07.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.677+0000 7f475bfff640 1 -- 172.21.15.112:0/3852980633 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754036070 con 0x7f47500039b0 2024-04-25T01:42:07.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.677+0000 7f475bfff640 1 -- 172.21.15.112:0/3852980633 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f4754031400 con 0x7f47500039b0 2024-04-25T01:42:07.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.677+0000 7f475bfff640 1 -- 172.21.15.112:0/3852980633 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754009690 con 0x7f47500039b0 2024-04-25T01:42:07.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.678+0000 7f476b390640 1 -- 172.21.15.112:0/3852980633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47500039b0 msgr2=0x7f4750003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:07.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.678+0000 7f476b390640 1 --2- 172.21.15.112:0/3852980633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47500039b0 0x7f4750003dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4754009ac0 tx=0x7f4750005830 comp rx=0 tx=0).stop 2024-04-25T01:42:07.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.678+0000 7f476b390640 1 -- 172.21.15.112:0/3852980633 shutdown_connections 2024-04-25T01:42:07.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.678+0000 7f476b390640 1 --2- 172.21.15.112:0/3852980633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47500039b0 0x7f4750003dd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.678+0000 7f476b390640 1 -- 172.21.15.112:0/3852980633 >> 172.21.15.112:0/3852980633 conn(0x7f476406b160 msgr2=0x7f476406b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:07.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.678+0000 7f476b390640 1 -- 172.21.15.112:0/3852980633 shutdown_connections 2024-04-25T01:42:07.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.678+0000 7f476b390640 1 -- 172.21.15.112:0/3852980633 wait complete. 2024-04-25T01:42:07.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.679+0000 7f476b390640 1 Processor -- start 2024-04-25T01:42:07.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.679+0000 7f476b390640 1 -- start start 2024-04-25T01:42:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.680+0000 7f476b390640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47500039b0 0x7f47640c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.680+0000 7f476b390640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47641dd180 0x7f4764147850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.680+0000 7f476b390640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4764147d90 0x7f47641467b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:07.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.680+0000 7f476b390640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4764071890 con 0x7f4764147d90 2024-04-25T01:42:07.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.680+0000 7f476b390640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f47640c2a20 con 0x7f47500039b0 2024-04-25T01:42:07.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.680+0000 7f476b390640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4764073160 con 0x7f47641dd180 2024-04-25T01:42:07.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.680+0000 7f476992d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4764147d90 0x7f47641467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:07.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.680+0000 7f476992d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4764147d90 0x7f47641467b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40264/0 (socket says 172.21.15.112:40264) 2024-04-25T01:42:07.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.680+0000 7f476992d640 1 -- 172.21.15.112:0/3359406378 learned_addr learned my addr 172.21.15.112:0/3359406378 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:07.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.680+0000 7f476912c640 1 --2- 172.21.15.112:0/3359406378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47500039b0 0x7f47640c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:07.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.680+0000 7f476892b640 1 --2- 172.21.15.112:0/3359406378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47641dd180 0x7f4764147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:07.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.680+0000 7f476992d640 1 -- 172.21.15.112:0/3359406378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47641dd180 msgr2=0x7f4764147850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:07.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.680+0000 7f476992d640 1 --2- 172.21.15.112:0/3359406378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47641dd180 0x7f4764147850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.680+0000 7f476992d640 1 -- 172.21.15.112:0/3359406378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47500039b0 msgr2=0x7f47640c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:07.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.680+0000 7f476992d640 1 --2- 172.21.15.112:0/3359406378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47500039b0 0x7f47640c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.680+0000 7f476992d640 1 -- 172.21.15.112:0/3359406378 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47641e93e0 con 0x7f4764147d90 2024-04-25T01:42:07.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.680+0000 7f476992d640 1 --2- 172.21.15.112:0/3359406378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4764147d90 0x7f47641467b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f47640c4c40 tx=0x7f475c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:07.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.680+0000 7f4759ffb640 1 -- 172.21.15.112:0/3359406378 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c00ee90 con 0x7f4764147d90 2024-04-25T01:42:07.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.680+0000 7f4759ffb640 1 -- 172.21.15.112:0/3359406378 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f475c018070 con 0x7f4764147d90 2024-04-25T01:42:07.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.681+0000 7f4759ffb640 1 -- 172.21.15.112:0/3359406378 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c003bb0 con 0x7f4764147d90 2024-04-25T01:42:07.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.681+0000 7f476b390640 1 -- 172.21.15.112:0/3359406378 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4764149f20 con 0x7f4764147d90 2024-04-25T01:42:07.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.681+0000 7f476b390640 1 -- 172.21.15.112:0/3359406378 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47641e9980 con 0x7f4764147d90 2024-04-25T01:42:07.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.682+0000 7f476b390640 1 -- 172.21.15.112:0/3359406378 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47640682a0 con 0x7f4764147d90 2024-04-25T01:42:07.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.682+0000 7f4759ffb640 1 -- 172.21.15.112:0/3359406378 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f475c0050d0 con 0x7f4764147d90 2024-04-25T01:42:07.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.682+0000 7f4759ffb640 1 --2- 172.21.15.112:0/3359406378 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f474806c000 0x7f474806e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:07.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.682+0000 7f4759ffb640 1 -- 172.21.15.112:0/3359406378 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5775+0+0 (secure 0 0 0) 0x7f475c08a8f0 con 0x7f4764147d90 2024-04-25T01:42:07.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.682+0000 7f476912c640 1 --2- 172.21.15.112:0/3359406378 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f474806c000 0x7f474806e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:07.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.683+0000 7f476912c640 1 --2- 172.21.15.112:0/3359406378 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f474806c000 0x7f474806e4c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4754033000 tx=0x7f4754008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:07.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.687+0000 7f4759ffb640 1 -- 172.21.15.112:0/3359406378 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f475c059260 con 0x7f4764147d90 2024-04-25T01:42:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.924+0000 7f476b390640 1 -- 172.21.15.112:0/3359406378 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f476414c3a0 con 0x7f4764147d90 2024-04-25T01:42:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.925+0000 7f4759ffb640 1 -- 172.21.15.112:0/3359406378 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+5186 (secure 0 0 0) 0x7f475c058b60 con 0x7f4764147d90 2024-04-25T01:42:07.913 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:42:07.913 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/4047219598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":4047219598},{"type":"v1","addr":"172.21.15.187:6834","nonce":4047219598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2204640907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2204640907},{"type":"v1","addr":"172.21.15.187:6835","nonce":2204640907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":144528,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1831476248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1831476248},{"type":"v1","addr":"172.21.15.112:6837","nonce":1831476248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:26:01.740521+0000","modified":"2024-04-25T01:26:04.746955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":8,"state":"up:active","state_seq":6,"addr":"172.21.15.112:6835/278686477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":278686477},{"type":"v1","addr":"172.21.15.112:6835","nonce":278686477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":2}]} 2024-04-25T01:42:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 13 2024-04-25T01:42:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.931+0000 7f476b390640 1 -- 172.21.15.112:0/3359406378 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f474806c000 msgr2=0x7f474806e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.931+0000 7f476b390640 1 --2- 172.21.15.112:0/3359406378 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f474806c000 0x7f474806e4c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4754033000 tx=0x7f4754008030 comp rx=0 tx=0).stop 2024-04-25T01:42:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.931+0000 7f476b390640 1 -- 172.21.15.112:0/3359406378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4764147d90 msgr2=0x7f47641467b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.931+0000 7f476b390640 1 --2- 172.21.15.112:0/3359406378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4764147d90 0x7f47641467b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f47640c4c40 tx=0x7f475c00eb50 comp rx=0 tx=0).stop 2024-04-25T01:42:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.931+0000 7f476b390640 1 -- 172.21.15.112:0/3359406378 shutdown_connections 2024-04-25T01:42:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.931+0000 7f476b390640 1 --2- 172.21.15.112:0/3359406378 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f474806c000 0x7f474806e4c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.931+0000 7f476b390640 1 --2- 172.21.15.112:0/3359406378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4764147d90 0x7f47641467b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.931+0000 7f476b390640 1 --2- 172.21.15.112:0/3359406378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47641dd180 0x7f4764147850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.931+0000 7f476b390640 1 --2- 172.21.15.112:0/3359406378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47500039b0 0x7f47640c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.931+0000 7f476b390640 1 -- 172.21.15.112:0/3359406378 >> 172.21.15.112:0/3359406378 conn(0x7f476406b160 msgr2=0x7f47640c7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.932+0000 7f476b390640 1 -- 172.21.15.112:0/3359406378 shutdown_connections 2024-04-25T01:42:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:07.932+0000 7f476b390640 1 -- 172.21.15.112:0/3359406378 wait complete. 2024-04-25T01:42:07.938 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:42:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.100+0000 7fb22e770640 1 Processor -- start 2024-04-25T01:42:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.101+0000 7fb22e770640 1 -- start start 2024-04-25T01:42:08.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.101+0000 7fb22e770640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb22806d6f0 0x7fb2280c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:08.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.101+0000 7fb22e770640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb2280c8130 0x7fb22806bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:08.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.101+0000 7fb22e770640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb228155a80 con 0x7fb228152e50 2024-04-25T01:42:08.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.101+0000 7fb22e770640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fb228156690 con 0x7fb22806d6f0 2024-04-25T01:42:08.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.101+0000 7fb22e770640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb228155d60 con 0x7fb2280c8130 2024-04-25T01:42:08.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.101+0000 7fb2277fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb22806d6f0 0x7fb2280c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:08.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.101+0000 7fb2277fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb22806d6f0 0x7fb2280c7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40270/0 (socket says 172.21.15.112:40270) 2024-04-25T01:42:08.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.101+0000 7fb227fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb228152e50 0x7fb2280c9300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:40314/0 (socket says 172.21.15.112:40314) 2024-04-25T01:42:08.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.101+0000 7fb2277fe640 1 -- 172.21.15.112:0/4068297129 learned_addr learned my addr 172.21.15.112:0/4068297129 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:08.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.102+0000 7fb22cd0d640 1 --2- 172.21.15.112:0/4068297129 >> v2:172.21.15.187:3300/0 conn(0x7fb2280c8130 0x7fb22806bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:08.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.102+0000 7fb226ffd640 1 -- 172.21.15.112:0/4068297129 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 561493276 0 0) 0x7fb228155a80 con 0x7fb228152e50 2024-04-25T01:42:08.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.102+0000 7fb226ffd640 1 -- 172.21.15.112:0/4068297129 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb210003520 con 0x7fb228152e50 2024-04-25T01:42:08.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.102+0000 7fb2277fe640 1 -- 172.21.15.112:0/4068297129 >> v2:172.21.15.187:3300/0 conn(0x7fb2280c8130 msgr2=0x7fb22806bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:08.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.102+0000 7fb2277fe640 1 --2- 172.21.15.112:0/4068297129 >> v2:172.21.15.187:3300/0 conn(0x7fb2280c8130 0x7fb22806bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.102+0000 7fb2277fe640 1 -- 172.21.15.112:0/4068297129 >> v1:172.21.15.112:6789/0 conn(0x7fb228152e50 legacy=0x7fb2280c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:08.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.102+0000 7fb2277fe640 1 -- 172.21.15.112:0/4068297129 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb228156040 con 0x7fb22806d6f0 2024-04-25T01:42:08.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.103+0000 7fb2277fe640 1 --2- 172.21.15.112:0/4068297129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb22806d6f0 0x7fb2280c7bf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb22806ba60 tx=0x7fb21c02a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39617f24fa3f6dda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:08.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.103+0000 7fb226ffd640 1 -- 172.21.15.112:0/4068297129 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c037070 con 0x7fb22806d6f0 2024-04-25T01:42:08.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.103+0000 7fb226ffd640 1 -- 172.21.15.112:0/4068297129 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb21c032440 con 0x7fb22806d6f0 2024-04-25T01:42:08.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.103+0000 7fb226ffd640 1 -- 172.21.15.112:0/4068297129 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c032830 con 0x7fb22806d6f0 2024-04-25T01:42:08.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.104+0000 7fb22e770640 1 -- 172.21.15.112:0/4068297129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb22806d6f0 msgr2=0x7fb2280c7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:08.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.104+0000 7fb22e770640 1 --2- 172.21.15.112:0/4068297129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb22806d6f0 0x7fb2280c7bf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb22806ba60 tx=0x7fb21c02a160 comp rx=0 tx=0).stop 2024-04-25T01:42:08.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.104+0000 7fb22e770640 1 -- 172.21.15.112:0/4068297129 shutdown_connections 2024-04-25T01:42:08.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.104+0000 7fb22e770640 1 --2- 172.21.15.112:0/4068297129 >> v2:172.21.15.187:3300/0 conn(0x7fb2280c8130 0x7fb22806bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.104+0000 7fb22e770640 1 --2- 172.21.15.112:0/4068297129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb22806d6f0 0x7fb2280c7bf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.104+0000 7fb22e770640 1 -- 172.21.15.112:0/4068297129 >> 172.21.15.112:0/4068297129 conn(0x7fb2280acd30 msgr2=0x7fb2280ad110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:08.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.104+0000 7fb22e770640 1 -- 172.21.15.112:0/4068297129 shutdown_connections 2024-04-25T01:42:08.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.104+0000 7fb22e770640 1 -- 172.21.15.112:0/4068297129 wait complete. 2024-04-25T01:42:08.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.105+0000 7fb22e770640 1 Processor -- start 2024-04-25T01:42:08.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.105+0000 7fb22e770640 1 -- start start 2024-04-25T01:42:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.106+0000 7fb22e770640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2280c8130 0x7fb2281ec0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.106+0000 7fb22e770640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb228152e50 0x7fb228153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.106+0000 7fb22e770640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2281ef680 0x7fb2280aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.106+0000 7fb22e770640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2281533b0 con 0x7fb2281ef680 2024-04-25T01:42:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.106+0000 7fb22e770640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb228153230 con 0x7fb2280c8130 2024-04-25T01:42:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.106+0000 7fb22e770640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb22806c2e0 con 0x7fb228152e50 2024-04-25T01:42:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.106+0000 7fb22cd0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2281ef680 0x7fb2280aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.106+0000 7fb22cd0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2281ef680 0x7fb2280aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40282/0 (socket says 172.21.15.112:40282) 2024-04-25T01:42:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.106+0000 7fb22cd0d640 1 -- 172.21.15.112:0/3980336253 learned_addr learned my addr 172.21.15.112:0/3980336253 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.106+0000 7fb227fff640 1 --2- 172.21.15.112:0/3980336253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2280c8130 0x7fb2281ec0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.107+0000 7fb22cd0d640 1 -- 172.21.15.112:0/3980336253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb228152e50 msgr2=0x7fb228153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.107+0000 7fb2277fe640 1 --2- 172.21.15.112:0/3980336253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb228152e50 0x7fb228153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.107+0000 7fb22cd0d640 1 --2- 172.21.15.112:0/3980336253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb228152e50 0x7fb228153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.107+0000 7fb22cd0d640 1 -- 172.21.15.112:0/3980336253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2280c8130 msgr2=0x7fb2281ec0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:08.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.107+0000 7fb22cd0d640 1 --2- 172.21.15.112:0/3980336253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2280c8130 0x7fb2281ec0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.107+0000 7fb22cd0d640 1 -- 172.21.15.112:0/3980336253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2280ab7d0 con 0x7fb2281ef680 2024-04-25T01:42:08.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.107+0000 7fb2277fe640 1 --2- 172.21.15.112:0/3980336253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb228152e50 0x7fb228153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:42:08.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.107+0000 7fb22cd0d640 1 --2- 172.21.15.112:0/3980336253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2281ef680 0x7fb2280aee30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb220009dd0 tx=0x7fb22000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:08.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.107+0000 7fb224ff9640 1 -- 172.21.15.112:0/3980336253 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb220018020 con 0x7fb2281ef680 2024-04-25T01:42:08.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.107+0000 7fb224ff9640 1 -- 172.21.15.112:0/3980336253 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb220014480 con 0x7fb2281ef680 2024-04-25T01:42:08.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.107+0000 7fb224ff9640 1 -- 172.21.15.112:0/3980336253 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb220014890 con 0x7fb2281ef680 2024-04-25T01:42:08.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.107+0000 7fb22e770640 1 -- 172.21.15.112:0/3980336253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2280c48d0 con 0x7fb2281ef680 2024-04-25T01:42:08.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.107+0000 7fb22e770640 1 -- 172.21.15.112:0/3980336253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2281ec630 con 0x7fb2281ef680 2024-04-25T01:42:08.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.108+0000 7fb22e770640 1 -- 172.21.15.112:0/3980336253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2280c7090 con 0x7fb2281ef680 2024-04-25T01:42:08.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.109+0000 7fb224ff9640 1 -- 172.21.15.112:0/3980336253 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb220005d10 con 0x7fb2281ef680 2024-04-25T01:42:08.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.109+0000 7fb224ff9640 1 --2- 172.21.15.112:0/3980336253 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb21406be60 0x7fb21406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:08.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.109+0000 7fb227fff640 1 --2- 172.21.15.112:0/3980336253 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb21406be60 0x7fb21406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:08.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.109+0000 7fb224ff9640 1 -- 172.21.15.112:0/3980336253 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5775+0+0 (secure 0 0 0) 0x7fb22008b8a0 con 0x7fb2281ef680 2024-04-25T01:42:08.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.110+0000 7fb227fff640 1 --2- 172.21.15.112:0/3980336253 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb21406be60 0x7fb21406e320 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb218000f60 tx=0x7fb218007cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:08.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.116+0000 7fb224ff9640 1 -- 172.21.15.112:0/3980336253 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb2280c7090 con 0x7fb2281ef680 2024-04-25T01:42:08.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.392+0000 7fb22e770640 1 -- 172.21.15.112:0/3980336253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2280682a0 con 0x7fb2281ef680 2024-04-25T01:42:08.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.394+0000 7fb224ff9640 1 -- 172.21.15.112:0/3980336253 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16804 (secure 0 0 0) 0x7fb22008b020 con 0x7fb2281ef680 2024-04-25T01:42:08.382 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T01:42:08.382 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"fsid":"a88d8313-513a-4f1d-8e1c-e81e1847e3c2","created":"2024-04-25T01:25:28.590174+0000","modified":"2024-04-25T01:42:07.186440+0000","last_up_change":"2024-04-25T01:25:39.686267+0000","last_in_change":"2024-04-25T01:25:30.665595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:25:40.679119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-25T01:25:59.478275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-25T01:26:00.719358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"687b49ac-8bbb-4826-99c3-007b21a72cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6817","nonce":2045679399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6819","nonce":2045679399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6823","nonce":2045679399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2045679399},{"type":"v1","addr":"172.21.15.112:6821","nonce":2045679399}]},"public_addr":"172.21.15.112:6817/2045679399","cluster_addr":"172.21.15.112:6819/2045679399","heartbeat_back_addr":"172.21.15.112:6823/2045679399","heartbeat_front_addr":"172.21.15.112:6821/2045679399","state":["exists","up"]},{"osd":1,"uuid":"8fc51f65-937b-4a6e-83ac-9c4e6113c738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6801","nonce":1032830648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6803","nonce":1032830648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6807","nonce":1032830648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1032830648},{"type":"v1","addr":"172.21.15.112:6805","nonce":1032830648}]},"public_addr":"172.21.15.112:6801/1032830648","cluster_addr":"172.21.15.112:6803/1032830648","heartbeat_back_addr":"172.21.15.112:6807/1032830648","heartbeat_front_addr":"172.21.15.112:6805/1032830648","state":["exists","up"]},{"osd":2,"uuid":"8ad36ec9-5911-4be0-b2df-f57d9b3a1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6825","nonce":2172872078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6827","nonce":2172872078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6831","nonce":2172872078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2172872078},{"type":"v1","addr":"172.21.15.112:6829","nonce":2172872078}]},"public_addr":"172.21.15.112:6825/2172872078","cluster_addr":"172.21.15.112:6827/2172872078","heartbeat_back_addr":"172.21.15.112:6831/2172872078","heartbeat_front_addr":"172.21.15.112:6829/2172872078","state":["exists","up"]},{"osd":3,"uuid":"62da8b34-63d3-4f21-94f6-126f6b310912","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6809","nonce":1418784374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6811","nonce":1418784374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6815","nonce":1418784374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1418784374},{"type":"v1","addr":"172.21.15.112:6813","nonce":1418784374}]},"public_addr":"172.21.15.112:6809/1418784374","cluster_addr":"172.21.15.112:6811/1418784374","heartbeat_back_addr":"172.21.15.112:6815/1418784374","heartbeat_front_addr":"172.21.15.112:6813/1418784374","state":["exists","up"]},{"osd":4,"uuid":"3372f51c-f7ca-4a3b-81ed-cc71b4b8006b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6801","nonce":876892580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6803","nonce":876892580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6807","nonce":876892580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":876892580},{"type":"v1","addr":"172.21.15.187:6805","nonce":876892580}]},"public_addr":"172.21.15.187:6801/876892580","cluster_addr":"172.21.15.187:6803/876892580","heartbeat_back_addr":"172.21.15.187:6807/876892580","heartbeat_front_addr":"172.21.15.187:6805/876892580","state":["exists","up"]},{"osd":5,"uuid":"c7728c82-5f37-484c-ab71-f486b9bf8bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6825","nonce":1463958731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6827","nonce":1463958731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6831","nonce":1463958731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1463958731},{"type":"v1","addr":"172.21.15.187:6829","nonce":1463958731}]},"public_addr":"172.21.15.187:6825/1463958731","cluster_addr":"172.21.15.187:6827/1463958731","heartbeat_back_addr":"172.21.15.187:6831/1463958731","heartbeat_front_addr":"172.21.15.187:6829/1463958731","state":["exists","up"]},{"osd":6,"uuid":"70148781-f925-44e3-a6f3-68b5f58b2b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6809","nonce":931860956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6811","nonce":931860956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6815","nonce":931860956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":931860956},{"type":"v1","addr":"172.21.15.187:6813","nonce":931860956}]},"public_addr":"172.21.15.187:6809/931860956","cluster_addr":"172.21.15.187:6811/931860956","heartbeat_back_addr":"172.21.15.187:6815/931860956","heartbeat_front_addr":"172.21.15.187:6813/931860956","state":["exists","up"]},{"osd":7,"uuid":"e2f643a5-c4c5-439d-8b97-02a9d5343d94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6817","nonce":155270368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6819","nonce":155270368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6823","nonce":155270368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":155270368},{"type":"v1","addr":"172.21.15.187:6821","nonce":155270368}]},"public_addr":"172.21.15.187:6817/155270368","cluster_addr":"172.21.15.187:6819/155270368","heartbeat_back_addr":"172.21.15.187:6823/155270368","heartbeat_front_addr":"172.21.15.187:6821/155270368","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.822072+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.752975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.760976+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.780621+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.796111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.889929+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.883974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:25:35.916257+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6837/2115984531":"2024-04-26T01:42:04.173644+0000","172.21.15.112:6836/2115984531":"2024-04-26T01:42:04.173644+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:42:08.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.396+0000 7fb22e770640 1 -- 172.21.15.112:0/3980336253 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb21406be60 msgr2=0x7fb21406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:08.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.396+0000 7fb22e770640 1 --2- 172.21.15.112:0/3980336253 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb21406be60 0x7fb21406e320 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb218000f60 tx=0x7fb218007cf0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.396+0000 7fb22e770640 1 -- 172.21.15.112:0/3980336253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2281ef680 msgr2=0x7fb2280aee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:08.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.396+0000 7fb22e770640 1 --2- 172.21.15.112:0/3980336253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2281ef680 0x7fb2280aee30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb220009dd0 tx=0x7fb22000c490 comp rx=0 tx=0).stop 2024-04-25T01:42:08.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.396+0000 7fb22e770640 1 -- 172.21.15.112:0/3980336253 shutdown_connections 2024-04-25T01:42:08.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.396+0000 7fb22e770640 1 --2- 172.21.15.112:0/3980336253 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fb21406be60 0x7fb21406e320 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.396+0000 7fb22e770640 1 --2- 172.21.15.112:0/3980336253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2281ef680 0x7fb2280aee30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.396+0000 7fb22e770640 1 --2- 172.21.15.112:0/3980336253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb228152e50 0x7fb228153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.396+0000 7fb22e770640 1 --2- 172.21.15.112:0/3980336253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2280c8130 0x7fb2281ec0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.396+0000 7fb22e770640 1 -- 172.21.15.112:0/3980336253 >> 172.21.15.112:0/3980336253 conn(0x7fb2280acd30 msgr2=0x7fb22814f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:08.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.396+0000 7fb22e770640 1 -- 172.21.15.112:0/3980336253 shutdown_connections 2024-04-25T01:42:08.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.396+0000 7fb22e770640 1 -- 172.21.15.112:0/3980336253 wait complete. 2024-04-25T01:42:08.396 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail b 2024-04-25T01:42:08.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.507+0000 7f6cb6d42640 1 Processor -- start 2024-04-25T01:42:08.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.507+0000 7f6cb6d42640 1 -- start start 2024-04-25T01:42:08.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.508+0000 7f6cb6d42640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6cb00743d0 0x7f6cb00747b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:08.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.508+0000 7f6cb6d42640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cb0156110 0x7f6cb01602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:08.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.508+0000 7f6cb6d42640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cb00c1e40 con 0x7f6cb006fb80 2024-04-25T01:42:08.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.508+0000 7f6cb6d42640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6cb00c40e0 con 0x7f6cb00743d0 2024-04-25T01:42:08.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.508+0000 7f6cb6d42640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb01534b0 con 0x7f6cb0156110 2024-04-25T01:42:08.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.508+0000 7f6caffff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6cb006fb80 0x7f6cb0155a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:40320/0 (socket says 172.21.15.112:40320) 2024-04-25T01:42:08.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.508+0000 7f6caffff640 1 -- 172.21.15.112:0/334883266 learned_addr learned my addr 172.21.15.112:0/334883266 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:42:08.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.508+0000 7f6cb4ade640 1 --2- 172.21.15.112:0/334883266 >> v2:172.21.15.187:3300/0 conn(0x7f6cb00743d0 0x7f6cb00747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:08.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.508+0000 7f6cb52df640 1 --2- 172.21.15.112:0/334883266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cb0156110 0x7f6cb01602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:08.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.508+0000 7f6caf7fe640 1 -- 172.21.15.112:0/334883266 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1900626142 0 0) 0x7f6cb00c1e40 con 0x7f6cb006fb80 2024-04-25T01:42:08.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.508+0000 7f6caf7fe640 1 -- 172.21.15.112:0/334883266 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ca0003520 con 0x7f6cb006fb80 2024-04-25T01:42:08.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.508+0000 7f6cb4ade640 1 -- 172.21.15.112:0/334883266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cb0156110 msgr2=0x7f6cb01602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:08.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.508+0000 7f6cb4ade640 1 --2- 172.21.15.112:0/334883266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cb0156110 0x7f6cb01602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.508+0000 7f6cb4ade640 1 -- 172.21.15.112:0/334883266 >> v1:172.21.15.112:6789/0 conn(0x7f6cb006fb80 legacy=0x7f6cb0155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:08.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.508+0000 7f6cb4ade640 1 -- 172.21.15.112:0/334883266 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb0159a20 con 0x7f6cb00743d0 2024-04-25T01:42:08.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.509+0000 7f6cb4ade640 1 --2- 172.21.15.112:0/334883266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cb00743d0 0x7f6cb00747b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40097a0 tx=0x7f6ca402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87096fb9dce88e2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:08.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.509+0000 7f6caf7fe640 1 -- 172.21.15.112:0/334883266 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4037070 con 0x7f6cb00743d0 2024-04-25T01:42:08.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.509+0000 7f6caf7fe640 1 -- 172.21.15.112:0/334883266 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6ca402ae50 con 0x7f6cb00743d0 2024-04-25T01:42:08.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.509+0000 7f6caf7fe640 1 -- 172.21.15.112:0/334883266 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4032680 con 0x7f6cb00743d0 2024-04-25T01:42:08.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.510+0000 7f6cb6d42640 1 -- 172.21.15.112:0/334883266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cb00743d0 msgr2=0x7f6cb00747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:08.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.510+0000 7f6cb6d42640 1 --2- 172.21.15.112:0/334883266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cb00743d0 0x7f6cb00747b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40097a0 tx=0x7f6ca402a100 comp rx=0 tx=0).stop 2024-04-25T01:42:08.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.510+0000 7f6cb6d42640 1 -- 172.21.15.112:0/334883266 shutdown_connections 2024-04-25T01:42:08.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.510+0000 7f6cb6d42640 1 --2- 172.21.15.112:0/334883266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cb0156110 0x7f6cb01602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.510+0000 7f6cb6d42640 1 --2- 172.21.15.112:0/334883266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cb00743d0 0x7f6cb00747b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.510+0000 7f6cb6d42640 1 -- 172.21.15.112:0/334883266 >> 172.21.15.112:0/334883266 conn(0x7f6cb006b160 msgr2=0x7f6cb006b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:08.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.510+0000 7f6cb6d42640 1 -- 172.21.15.112:0/334883266 shutdown_connections 2024-04-25T01:42:08.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.510+0000 7f6cb6d42640 1 -- 172.21.15.112:0/334883266 wait complete. 2024-04-25T01:42:08.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.511+0000 7f6cb6d42640 1 Processor -- start 2024-04-25T01:42:08.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.511+0000 7f6cb6d42640 1 -- start start 2024-04-25T01:42:08.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6cb6d42640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cb006fb80 0x7f6cb00c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:08.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6cb6d42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb00743d0 0x7f6cb014bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:08.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6cb6d42640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cb0156110 0x7f6cb014c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:08.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6cb6d42640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb00c19b0 con 0x7f6cb00743d0 2024-04-25T01:42:08.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6cb6d42640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb00c16d0 con 0x7f6cb006fb80 2024-04-25T01:42:08.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6cb6d42640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb00a7070 con 0x7f6cb0156110 2024-04-25T01:42:08.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6caffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb00743d0 0x7f6cb014bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:08.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6caffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb00743d0 0x7f6cb014bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40284/0 (socket says 172.21.15.112:40284) 2024-04-25T01:42:08.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6caffff640 1 -- 172.21.15.112:0/613710451 learned_addr learned my addr 172.21.15.112:0/613710451 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:08.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6cb4ade640 1 --2- 172.21.15.112:0/613710451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cb006fb80 0x7f6cb00c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:08.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6cb52df640 1 --2- 172.21.15.112:0/613710451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cb0156110 0x7f6cb014c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:08.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6caffff640 1 -- 172.21.15.112:0/613710451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cb0156110 msgr2=0x7f6cb014c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6caffff640 1 --2- 172.21.15.112:0/613710451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cb0156110 0x7f6cb014c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6caffff640 1 -- 172.21.15.112:0/613710451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cb006fb80 msgr2=0x7f6cb00c5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6caffff640 1 --2- 172.21.15.112:0/613710451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cb006fb80 0x7f6cb00c5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6caffff640 1 -- 172.21.15.112:0/613710451 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb014c670 con 0x7f6cb00743d0 2024-04-25T01:42:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6cb52df640 1 --2- 172.21.15.112:0/613710451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cb0156110 0x7f6cb014c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:42:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6caffff640 1 --2- 172.21.15.112:0/613710451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb00743d0 0x7f6cb014bbf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00c460 tx=0x7f6c9c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6cad7fa640 1 -- 172.21.15.112:0/613710451 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c017070 con 0x7f6cb00743d0 2024-04-25T01:42:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6cad7fa640 1 -- 172.21.15.112:0/613710451 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6c9c005ce0 con 0x7f6cb00743d0 2024-04-25T01:42:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6cb6d42640 1 -- 172.21.15.112:0/613710451 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb01ebf00 con 0x7f6cb00743d0 2024-04-25T01:42:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6cad7fa640 1 -- 172.21.15.112:0/613710451 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c0072b0 con 0x7f6cb00743d0 2024-04-25T01:42:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.512+0000 7f6cb6d42640 1 -- 172.21.15.112:0/613710451 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb00c6ae0 con 0x7f6cb00743d0 2024-04-25T01:42:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.514+0000 7f6cb6d42640 1 -- 172.21.15.112:0/613710451 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb00682a0 con 0x7f6cb00743d0 2024-04-25T01:42:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.514+0000 7f6cad7fa640 1 -- 172.21.15.112:0/613710451 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6c9c01a070 con 0x7f6cb00743d0 2024-04-25T01:42:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.514+0000 7f6cad7fa640 1 --2- 172.21.15.112:0/613710451 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6c9406be60 0x7f6c9406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.514+0000 7f6cad7fa640 1 -- 172.21.15.112:0/613710451 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5775+0+0 (secure 0 0 0) 0x7f6c9c08a160 con 0x7f6cb00743d0 2024-04-25T01:42:08.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.514+0000 7f6cb4ade640 1 --2- 172.21.15.112:0/613710451 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6c9406be60 0x7f6c9406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:08.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.515+0000 7f6cb4ade640 1 --2- 172.21.15.112:0/613710451 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6c9406be60 0x7f6c9406e320 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4034000 tx=0x7f6ca402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:08.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.519+0000 7f6cad7fa640 1 -- 172.21.15.112:0/613710451 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6c9c058b60 con 0x7f6cb00743d0 2024-04-25T01:42:08.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.730+0000 7f6cb6d42640 1 -- 172.21.15.112:0/613710451 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "b"} v 0) -- 0x7f6cb006c6d0 con 0x7f6cb00743d0 2024-04-25T01:42:08.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.731+0000 7f6cad7fa640 1 -- 172.21.15.112:0/613710451 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "b"}]=0 b marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v14) ==== 151+0+0 (secure 0 0 0) 0x7f6c9c058460 con 0x7f6cb00743d0 2024-04-25T01:42:08.719 INFO:teuthology.orchestra.run.smithi112.stderr:b marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T01:42:08.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.734+0000 7f6cb6d42640 1 -- 172.21.15.112:0/613710451 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6c9406be60 msgr2=0x7f6c9406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:08.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.734+0000 7f6cb6d42640 1 --2- 172.21.15.112:0/613710451 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6c9406be60 0x7f6c9406e320 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4034000 tx=0x7f6ca402e000 comp rx=0 tx=0).stop 2024-04-25T01:42:08.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.734+0000 7f6cb6d42640 1 -- 172.21.15.112:0/613710451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb00743d0 msgr2=0x7f6cb014bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:08.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.734+0000 7f6cb6d42640 1 --2- 172.21.15.112:0/613710451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb00743d0 0x7f6cb014bbf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00c460 tx=0x7f6c9c00c920 comp rx=0 tx=0).stop 2024-04-25T01:42:08.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.734+0000 7f6cb6d42640 1 -- 172.21.15.112:0/613710451 shutdown_connections 2024-04-25T01:42:08.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.734+0000 7f6cb6d42640 1 --2- 172.21.15.112:0/613710451 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f6c9406be60 0x7f6c9406e320 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.734+0000 7f6cb6d42640 1 --2- 172.21.15.112:0/613710451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cb0156110 0x7f6cb014c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.734+0000 7f6cb6d42640 1 --2- 172.21.15.112:0/613710451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb00743d0 0x7f6cb014bbf0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.734+0000 7f6cb6d42640 1 --2- 172.21.15.112:0/613710451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cb006fb80 0x7f6cb00c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.734+0000 7f6cb6d42640 1 -- 172.21.15.112:0/613710451 >> 172.21.15.112:0/613710451 conn(0x7f6cb006b160 msgr2=0x7f6cb014de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:08.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.734+0000 7f6cb6d42640 1 -- 172.21.15.112:0/613710451 shutdown_connections 2024-04-25T01:42:08.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.734+0000 7f6cb6d42640 1 -- 172.21.15.112:0/613710451 wait complete. 2024-04-25T01:42:08.735 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm b --yes-i-really-mean-it 2024-04-25T01:42:08.759 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2024-04-25T01:42:08.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.848+0000 7f1ebef9d640 1 Processor -- start 2024-04-25T01:42:08.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.848+0000 7f1ebef9d640 1 -- start start 2024-04-25T01:42:08.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.848+0000 7f1ebef9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1eb814f220 0x7f1eb80aa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:08.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.848+0000 7f1ebef9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1eb80ac830 0x7f1eb8072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:08.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.848+0000 7f1ebef9d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eb8155d40 con 0x7f1eb8072690 2024-04-25T01:42:08.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.848+0000 7f1ebef9d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f1eb8071700 con 0x7f1eb80ac830 2024-04-25T01:42:08.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.848+0000 7f1ebef9d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1eb8071cc0 con 0x7f1eb814f220 2024-04-25T01:42:08.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.848+0000 7f1eb7fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1eb80ac830 0x7f1eb8072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:08.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.849+0000 7f1eb7fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1eb80ac830 0x7f1eb8072150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40304/0 (socket says 172.21.15.112:40304) 2024-04-25T01:42:08.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.849+0000 7f1eb7fff640 1 -- 172.21.15.112:0/1377052573 learned_addr learned my addr 172.21.15.112:0/1377052573 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:08.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.849+0000 7f1ebcd39640 1 --2- 172.21.15.112:0/1377052573 >> v2:172.21.15.187:3300/0 conn(0x7f1eb814f220 0x7f1eb80aa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:08.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.849+0000 7f1eb7fff640 1 -- 172.21.15.112:0/1377052573 >> v2:172.21.15.187:3300/0 conn(0x7f1eb814f220 msgr2=0x7f1eb80aa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:08.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.849+0000 7f1eb7fff640 1 --2- 172.21.15.112:0/1377052573 >> v2:172.21.15.187:3300/0 conn(0x7f1eb814f220 0x7f1eb80aa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.849+0000 7f1eb7fff640 1 -- 172.21.15.112:0/1377052573 >> v1:172.21.15.187:6789/0 conn(0x7f1eb8072690 legacy=0x7f1eb8155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:08.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.849+0000 7f1eb7fff640 1 -- 172.21.15.112:0/1377052573 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb8156020 con 0x7f1eb80ac830 2024-04-25T01:42:08.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.849+0000 7f1ebcd39640 1 --2- 172.21.15.112:0/1377052573 >> v2:172.21.15.187:3300/0 conn(0x7f1eb814f220 0x7f1eb80aa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:42:08.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.849+0000 7f1eb7fff640 1 --2- 172.21.15.112:0/1377052573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb80ac830 0x7f1eb8072150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8067f50 tx=0x7f1ea002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b870e31694d8f262 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:08.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.850+0000 7f1eb77fe640 1 -- 172.21.15.112:0/1377052573 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0037070 con 0x7f1eb80ac830 2024-04-25T01:42:08.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.850+0000 7f1eb77fe640 1 -- 172.21.15.112:0/1377052573 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1ea0032440 con 0x7f1eb80ac830 2024-04-25T01:42:08.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.850+0000 7f1eb77fe640 1 -- 172.21.15.112:0/1377052573 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0032830 con 0x7f1eb80ac830 2024-04-25T01:42:08.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.850+0000 7f1ebef9d640 1 -- 172.21.15.112:0/1377052573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb80ac830 msgr2=0x7f1eb8072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:08.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.850+0000 7f1ebef9d640 1 --2- 172.21.15.112:0/1377052573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb80ac830 0x7f1eb8072150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8067f50 tx=0x7f1ea002a1a0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.850+0000 7f1ebef9d640 1 -- 172.21.15.112:0/1377052573 shutdown_connections 2024-04-25T01:42:08.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.850+0000 7f1ebef9d640 1 --2- 172.21.15.112:0/1377052573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb80ac830 0x7f1eb8072150 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.850+0000 7f1ebef9d640 1 --2- 172.21.15.112:0/1377052573 >> v2:172.21.15.187:3300/0 conn(0x7f1eb814f220 0x7f1eb80aa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.850+0000 7f1ebef9d640 1 -- 172.21.15.112:0/1377052573 >> 172.21.15.112:0/1377052573 conn(0x7f1eb80c3a10 msgr2=0x7f1eb80c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:08.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.850+0000 7f1ebef9d640 1 -- 172.21.15.112:0/1377052573 shutdown_connections 2024-04-25T01:42:08.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.850+0000 7f1ebef9d640 1 -- 172.21.15.112:0/1377052573 wait complete. 2024-04-25T01:42:08.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.851+0000 7f1ebef9d640 1 Processor -- start 2024-04-25T01:42:08.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.851+0000 7f1ebef9d640 1 -- start start 2024-04-25T01:42:08.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.852+0000 7f1ebef9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1eb8072690 0x7f1eb8147c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:08.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.852+0000 7f1ebef9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb80ac830 0x7f1eb81eb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:08.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.852+0000 7f1ebef9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb814f220 0x7f1eb81ebd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:08.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.852+0000 7f1ebef9d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb81559a0 con 0x7f1eb814f220 2024-04-25T01:42:08.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.852+0000 7f1ebef9d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb814f600 con 0x7f1eb8072690 2024-04-25T01:42:08.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.852+0000 7f1ebef9d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb8155b20 con 0x7f1eb80ac830 2024-04-25T01:42:08.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.852+0000 7f1ebd53a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb814f220 0x7f1eb81ebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:08.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.852+0000 7f1ebd53a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb814f220 0x7f1eb81ebd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40318/0 (socket says 172.21.15.112:40318) 2024-04-25T01:42:08.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.852+0000 7f1ebd53a640 1 -- 172.21.15.112:0/4062880292 learned_addr learned my addr 172.21.15.112:0/4062880292 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:08.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.852+0000 7f1eb7fff640 1 --2- 172.21.15.112:0/4062880292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb80ac830 0x7f1eb81eb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:08.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.852+0000 7f1ebcd39640 1 --2- 172.21.15.112:0/4062880292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1eb8072690 0x7f1eb8147c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:08.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.852+0000 7f1ebd53a640 1 -- 172.21.15.112:0/4062880292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb80ac830 msgr2=0x7f1eb81eb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:08.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.852+0000 7f1ebd53a640 1 --2- 172.21.15.112:0/4062880292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb80ac830 0x7f1eb81eb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.852+0000 7f1ebd53a640 1 -- 172.21.15.112:0/4062880292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1eb8072690 msgr2=0x7f1eb8147c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:08.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.852+0000 7f1ebd53a640 1 --2- 172.21.15.112:0/4062880292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1eb8072690 0x7f1eb8147c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:08.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.852+0000 7f1ebd53a640 1 -- 172.21.15.112:0/4062880292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb80c1090 con 0x7f1eb814f220 2024-04-25T01:42:08.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.852+0000 7f1ebcd39640 1 --2- 172.21.15.112:0/4062880292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1eb8072690 0x7f1eb8147c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:42:08.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.853+0000 7f1ebd53a640 1 --2- 172.21.15.112:0/4062880292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb814f220 0x7f1eb81ebd00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000c700 tx=0x7f1eb000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:08.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.853+0000 7f1eb57fa640 1 -- 172.21.15.112:0/4062880292 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0016020 con 0x7f1eb814f220 2024-04-25T01:42:08.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.853+0000 7f1eb57fa640 1 -- 172.21.15.112:0/4062880292 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1eb000aa50 con 0x7f1eb814f220 2024-04-25T01:42:08.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.853+0000 7f1eb57fa640 1 -- 172.21.15.112:0/4062880292 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000ae60 con 0x7f1eb814f220 2024-04-25T01:42:08.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.853+0000 7f1ebef9d640 1 -- 172.21.15.112:0/4062880292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb80c1370 con 0x7f1eb814f220 2024-04-25T01:42:08.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.853+0000 7f1ebef9d640 1 -- 172.21.15.112:0/4062880292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb8143090 con 0x7f1eb814f220 2024-04-25T01:42:08.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.854+0000 7f1ebef9d640 1 -- 172.21.15.112:0/4062880292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb81ef930 con 0x7f1eb814f220 2024-04-25T01:42:08.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.854+0000 7f1eb57fa640 1 -- 172.21.15.112:0/4062880292 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1eb0019020 con 0x7f1eb814f220 2024-04-25T01:42:08.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.855+0000 7f1eb57fa640 1 --2- 172.21.15.112:0/4062880292 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e9806be30 0x7f1e9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:08.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.855+0000 7f1eb57fa640 1 -- 172.21.15.112:0/4062880292 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5861+0+0 (secure 0 0 0) 0x7f1eb008af30 con 0x7f1eb814f220 2024-04-25T01:42:08.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.855+0000 7f1ebcd39640 1 --2- 172.21.15.112:0/4062880292 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e9806be30 0x7f1e9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:08.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.855+0000 7f1ebcd39640 1 --2- 172.21.15.112:0/4062880292 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e9806be30 0x7f1e9806e2f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1eb814e130 tx=0x7f1ea8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:08.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:08.859+0000 7f1eb57fa640 1 -- 172.21.15.112:0/4062880292 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1eb0020030 con 0x7f1eb814f220 2024-04-25T01:42:09.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.072+0000 7f1ebef9d640 1 -- 172.21.15.112:0/4062880292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "b", "yes_i_really_mean_it": true} v 0) -- 0x7f1eb80bcf30 con 0x7f1eb814f220 2024-04-25T01:42:09.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.737+0000 7f1eb57fa640 1 -- 172.21.15.112:0/4062880292 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "b", "yes_i_really_mean_it": true}]=0 v15) ==== 99+0+0 (secure 0 0 0) 0x7f1eb80bcf30 con 0x7f1eb814f220 2024-04-25T01:42:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.739+0000 7f1ebef9d640 1 -- 172.21.15.112:0/4062880292 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e9806be30 msgr2=0x7f1e9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.739+0000 7f1ebef9d640 1 --2- 172.21.15.112:0/4062880292 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e9806be30 0x7f1e9806e2f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1eb814e130 tx=0x7f1ea8008040 comp rx=0 tx=0).stop 2024-04-25T01:42:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.739+0000 7f1ebef9d640 1 -- 172.21.15.112:0/4062880292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb814f220 msgr2=0x7f1eb81ebd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.739+0000 7f1ebef9d640 1 --2- 172.21.15.112:0/4062880292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb814f220 0x7f1eb81ebd00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000c700 tx=0x7f1eb000cbc0 comp rx=0 tx=0).stop 2024-04-25T01:42:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.739+0000 7f1ebef9d640 1 -- 172.21.15.112:0/4062880292 shutdown_connections 2024-04-25T01:42:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.739+0000 7f1ebef9d640 1 --2- 172.21.15.112:0/4062880292 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f1e9806be30 0x7f1e9806e2f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.739+0000 7f1ebef9d640 1 --2- 172.21.15.112:0/4062880292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb814f220 0x7f1eb81ebd00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.739+0000 7f1ebef9d640 1 --2- 172.21.15.112:0/4062880292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1eb80ac830 0x7f1eb81eb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.739+0000 7f1ebef9d640 1 --2- 172.21.15.112:0/4062880292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1eb8072690 0x7f1eb8147c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.739+0000 7f1ebef9d640 1 -- 172.21.15.112:0/4062880292 >> 172.21.15.112:0/4062880292 conn(0x7f1eb80c3a10 msgr2=0x7f1eb814ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.739+0000 7f1ebef9d640 1 -- 172.21.15.112:0/4062880292 shutdown_connections 2024-04-25T01:42:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.739+0000 7f1ebef9d640 1 -- 172.21.15.112:0/4062880292 wait complete. 2024-04-25T01:42:09.740 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm b_metadata b_metadata --yes-i-really-really-mean-it 2024-04-25T01:42:09.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.894+0000 7f43a326b640 1 Processor -- start 2024-04-25T01:42:09.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.895+0000 7f43a326b640 1 -- start start 2024-04-25T01:42:09.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.895+0000 7f43a326b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f439c0c2d70 0x7f439c1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:09.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.895+0000 7f43a326b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f439c06d560 con 0x7f439c06cf10 2024-04-25T01:42:09.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.895+0000 7f43a326b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f439c06d840 con 0x7f439c06e600 2024-04-25T01:42:09.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.895+0000 7f43a326b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f439c06db20 con 0x7f439c0c2d70 2024-04-25T01:42:09.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.896+0000 7f43a1808640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f439c06cf10 0x7f439c1539a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:40326/0 (socket says 172.21.15.112:40326) 2024-04-25T01:42:09.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.896+0000 7f43a1808640 1 -- 172.21.15.112:0/3096546769 learned_addr learned my addr 172.21.15.112:0/3096546769 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:42:09.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.896+0000 7f43a1007640 1 --2- 172.21.15.112:0/3096546769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f439c0c2d70 0x7f439c1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:09.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.896+0000 7f4393fff640 1 -- 172.21.15.112:0/3096546769 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 265320772 0 0) 0x7f439c06d560 con 0x7f439c06cf10 2024-04-25T01:42:09.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.896+0000 7f4393fff640 1 -- 172.21.15.112:0/3096546769 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4388003520 con 0x7f439c06cf10 2024-04-25T01:42:09.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.896+0000 7f4393fff640 1 -- 172.21.15.112:0/3096546769 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1023777309 0 0) 0x7f439c06d840 con 0x7f439c06e600 2024-04-25T01:42:09.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.896+0000 7f4393fff640 1 -- 172.21.15.112:0/3096546769 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f439c06d560 con 0x7f439c06e600 2024-04-25T01:42:09.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.896+0000 7f4393fff640 1 -- 172.21.15.112:0/3096546769 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1953275630 0 0) 0x7f4388003520 con 0x7f439c06cf10 2024-04-25T01:42:09.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.896+0000 7f43a1007640 1 -- 172.21.15.112:0/3096546769 >> v1:172.21.15.187:6789/0 conn(0x7f439c06e600 legacy=0x7f439c14c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:09.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.896+0000 7f43a1007640 1 -- 172.21.15.112:0/3096546769 >> v1:172.21.15.112:6789/0 conn(0x7f439c06cf10 legacy=0x7f439c1539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:09.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.896+0000 7f43a1007640 1 -- 172.21.15.112:0/3096546769 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439c06de00 con 0x7f439c0c2d70 2024-04-25T01:42:09.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.897+0000 7f43a1007640 1 --2- 172.21.15.112:0/3096546769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f439c0c2d70 0x7f439c1524b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f43940097d0 tx=0x7f439402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3caf7f22d8ca6aaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:09.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.897+0000 7f4393fff640 1 -- 172.21.15.112:0/3096546769 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394037070 con 0x7f439c0c2d70 2024-04-25T01:42:09.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.897+0000 7f4393fff640 1 -- 172.21.15.112:0/3096546769 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f439402ab90 con 0x7f439c0c2d70 2024-04-25T01:42:09.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.897+0000 7f4393fff640 1 -- 172.21.15.112:0/3096546769 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43980019f0 con 0x7f439c0c2d70 2024-04-25T01:42:09.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.898+0000 7f43a326b640 1 -- 172.21.15.112:0/3096546769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f439c0c2d70 msgr2=0x7f439c1524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.898+0000 7f43a326b640 1 --2- 172.21.15.112:0/3096546769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f439c0c2d70 0x7f439c1524b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f43940097d0 tx=0x7f439402a100 comp rx=0 tx=0).stop 2024-04-25T01:42:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.898+0000 7f43a326b640 1 -- 172.21.15.112:0/3096546769 shutdown_connections 2024-04-25T01:42:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.898+0000 7f43a326b640 1 --2- 172.21.15.112:0/3096546769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f439c0c2d70 0x7f439c1524b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.898+0000 7f43a326b640 1 -- 172.21.15.112:0/3096546769 >> 172.21.15.112:0/3096546769 conn(0x7f439c06b8a0 msgr2=0x7f439c0c6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.898+0000 7f43a326b640 1 -- 172.21.15.112:0/3096546769 shutdown_connections 2024-04-25T01:42:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.898+0000 7f43a326b640 1 -- 172.21.15.112:0/3096546769 wait complete. 2024-04-25T01:42:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.899+0000 7f43a326b640 1 Processor -- start 2024-04-25T01:42:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.899+0000 7f43a326b640 1 -- start start 2024-04-25T01:42:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.899+0000 7f43a326b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f439c06cf10 0x7f439c1dd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.899+0000 7f43a326b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f439c06e600 0x7f439c1ddd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.899+0000 7f43a326b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f439c0c2d70 0x7f439c144fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.899+0000 7f43a326b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f439c14ca90 con 0x7f439c06cf10 2024-04-25T01:42:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.899+0000 7f43a326b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f439c0c1690 con 0x7f439c06e600 2024-04-25T01:42:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.899+0000 7f43a326b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f439c144280 con 0x7f439c0c2d70 2024-04-25T01:42:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.900+0000 7f43a1007640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f439c06cf10 0x7f439c1dd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.900+0000 7f43a1007640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f439c06cf10 0x7f439c1dd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40326/0 (socket says 172.21.15.112:40326) 2024-04-25T01:42:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.900+0000 7f43a1007640 1 -- 172.21.15.112:0/1081815545 learned_addr learned my addr 172.21.15.112:0/1081815545 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.900+0000 7f43a0806640 1 --2- 172.21.15.112:0/1081815545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f439c06e600 0x7f439c1ddd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.900+0000 7f43a1808640 1 --2- 172.21.15.112:0/1081815545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f439c0c2d70 0x7f439c144fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.900+0000 7f43a1007640 1 -- 172.21.15.112:0/1081815545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f439c0c2d70 msgr2=0x7f439c144fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.900+0000 7f43a1007640 1 --2- 172.21.15.112:0/1081815545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f439c0c2d70 0x7f439c144fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.900+0000 7f43a1007640 1 -- 172.21.15.112:0/1081815545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f439c06e600 msgr2=0x7f439c1ddd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.900+0000 7f43a1007640 1 --2- 172.21.15.112:0/1081815545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f439c06e600 0x7f439c1ddd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.900+0000 7f43a1007640 1 -- 172.21.15.112:0/1081815545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439c1e8af0 con 0x7f439c06cf10 2024-04-25T01:42:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.900+0000 7f43a0806640 1 --2- 172.21.15.112:0/1081815545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f439c06e600 0x7f439c1ddd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:42:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.900+0000 7f43a1808640 1 --2- 172.21.15.112:0/1081815545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f439c0c2d70 0x7f439c144fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:42:09.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.900+0000 7f43a1007640 1 --2- 172.21.15.112:0/1081815545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f439c06cf10 0x7f439c1dd830 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4394035040 tx=0x7f43940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:09.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.900+0000 7f4391ffb640 1 -- 172.21.15.112:0/1081815545 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394037050 con 0x7f439c06cf10 2024-04-25T01:42:09.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.900+0000 7f4391ffb640 1 -- 172.21.15.112:0/1081815545 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f4394043070 con 0x7f439c06cf10 2024-04-25T01:42:09.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.900+0000 7f43a326b640 1 -- 172.21.15.112:0/1081815545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439c1444d0 con 0x7f439c06cf10 2024-04-25T01:42:09.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.900+0000 7f4391ffb640 1 -- 172.21.15.112:0/1081815545 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394006e10 con 0x7f439c06cf10 2024-04-25T01:42:09.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.900+0000 7f43a326b640 1 -- 172.21.15.112:0/1081815545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f439c144a90 con 0x7f439c06cf10 2024-04-25T01:42:09.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.901+0000 7f43a326b640 1 -- 172.21.15.112:0/1081815545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f439c0682a0 con 0x7f439c06cf10 2024-04-25T01:42:09.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.902+0000 7f4391ffb640 1 -- 172.21.15.112:0/1081815545 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f439403ca70 con 0x7f439c06cf10 2024-04-25T01:42:09.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.902+0000 7f4391ffb640 1 --2- 172.21.15.112:0/1081815545 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f438006be30 0x7f438006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:09.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.902+0000 7f4391ffb640 1 -- 172.21.15.112:0/1081815545 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5861+0+0 (secure 0 0 0) 0x7f43940aba20 con 0x7f439c06cf10 2024-04-25T01:42:09.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.902+0000 7f43a0806640 1 --2- 172.21.15.112:0/1081815545 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f438006be30 0x7f438006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:09.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.903+0000 7f43a0806640 1 --2- 172.21.15.112:0/1081815545 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f438006be30 0x7f438006e2f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f439c1de8e0 tx=0x7f438c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:09.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:09.907+0000 7f4391ffb640 1 -- 172.21.15.112:0/1081815545 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f439407a4e0 con 0x7f439c06cf10 2024-04-25T01:42:10.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.082+0000 7f43a326b640 1 -- 172.21.15.112:0/1081815545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "b_metadata", "pool2": "b_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f439c147310 con 0x7f439c06cf10 2024-04-25T01:42:10.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.741+0000 7f4391ffb640 1 -- 172.21.15.112:0/1081815545 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "b_metadata", "pool2": "b_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'b_metadata' removed v31) ==== 166+0+0 (secure 0 0 0) 0x7f4394079de0 con 0x7f439c06cf10 2024-04-25T01:42:10.730 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'b_metadata' removed 2024-04-25T01:42:10.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.744+0000 7f43a326b640 1 -- 172.21.15.112:0/1081815545 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f438006be30 msgr2=0x7f438006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:10.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.744+0000 7f43a326b640 1 --2- 172.21.15.112:0/1081815545 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f438006be30 0x7f438006e2f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f439c1de8e0 tx=0x7f438c009040 comp rx=0 tx=0).stop 2024-04-25T01:42:10.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.744+0000 7f43a326b640 1 -- 172.21.15.112:0/1081815545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f439c06cf10 msgr2=0x7f439c1dd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:10.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.744+0000 7f43a326b640 1 --2- 172.21.15.112:0/1081815545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f439c06cf10 0x7f439c1dd830 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4394035040 tx=0x7f43940046c0 comp rx=0 tx=0).stop 2024-04-25T01:42:10.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.744+0000 7f43a326b640 1 -- 172.21.15.112:0/1081815545 shutdown_connections 2024-04-25T01:42:10.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.744+0000 7f43a326b640 1 --2- 172.21.15.112:0/1081815545 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7f438006be30 0x7f438006e2f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:10.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.744+0000 7f43a326b640 1 --2- 172.21.15.112:0/1081815545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f439c0c2d70 0x7f439c144fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:10.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.744+0000 7f43a326b640 1 --2- 172.21.15.112:0/1081815545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f439c06e600 0x7f439c1ddd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:10.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.744+0000 7f43a326b640 1 --2- 172.21.15.112:0/1081815545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f439c06cf10 0x7f439c1dd830 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:10.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.744+0000 7f43a326b640 1 -- 172.21.15.112:0/1081815545 >> 172.21.15.112:0/1081815545 conn(0x7f439c06b8a0 msgr2=0x7f439c148a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:10.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.745+0000 7f43a326b640 1 -- 172.21.15.112:0/1081815545 shutdown_connections 2024-04-25T01:42:10.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.746+0000 7f43a326b640 1 -- 172.21.15.112:0/1081815545 wait complete. 2024-04-25T01:42:10.751 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm b_data b_data --yes-i-really-really-mean-it 2024-04-25T01:42:10.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.865+0000 7fbb8f1f9640 1 Processor -- start 2024-04-25T01:42:10.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.865+0000 7fbb8f1f9640 1 -- start start 2024-04-25T01:42:10.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb8f1f9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb880c0420 0x7fbb880c0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:10.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb8f1f9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb88155d50 con 0x7fbb881599c0 2024-04-25T01:42:10.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb8f1f9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb88156030 con 0x7fbb88155870 2024-04-25T01:42:10.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb8f1f9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbb88156310 con 0x7fbb880c0420 2024-04-25T01:42:10.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb8d796640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fbb881599c0 0x7fbb881601f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:40336/0 (socket says 172.21.15.112:40336) 2024-04-25T01:42:10.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb8d796640 1 -- 172.21.15.112:0/2151668774 learned_addr learned my addr 172.21.15.112:0/2151668774 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T01:42:10.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb877fe640 1 -- 172.21.15.112:0/2151668774 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2469194888 0 0) 0x7fbb88155d50 con 0x7fbb881599c0 2024-04-25T01:42:10.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb877fe640 1 -- 172.21.15.112:0/2151668774 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb78003520 con 0x7fbb881599c0 2024-04-25T01:42:10.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb8cf95640 1 --2- 172.21.15.112:0/2151668774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb880c0420 0x7fbb880c0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:10.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb877fe640 1 -- 172.21.15.112:0/2151668774 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 678690567 0 0) 0x7fbb78003520 con 0x7fbb881599c0 2024-04-25T01:42:10.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb877fe640 1 -- 172.21.15.112:0/2151668774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb880c0420 msgr2=0x7fbb880c0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:10.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb877fe640 1 --2- 172.21.15.112:0/2151668774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb880c0420 0x7fbb880c0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:10.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb877fe640 1 -- 172.21.15.112:0/2151668774 >> v1:172.21.15.187:6789/0 conn(0x7fbb88155870 legacy=0x7fbb881592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:10.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb877fe640 1 -- 172.21.15.112:0/2151668774 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb881565f0 con 0x7fbb881599c0 2024-04-25T01:42:10.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb877fe640 1 -- 172.21.15.112:0/2151668774 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1052322685 0 0) 0x7fbb800019f0 con 0x7fbb881599c0 2024-04-25T01:42:10.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb877fe640 1 -- 172.21.15.112:0/2151668774 >> v1:172.21.15.112:6789/0 conn(0x7fbb881599c0 legacy=0x7fbb881601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:10.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb877fe640 1 --2- 172.21.15.112:0/2151668774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb780039e0 0x7fbb78003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:10.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb877fe640 1 --2- 172.21.15.112:0/2151668774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb78004800 0x7fbb78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:10.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb877fe640 1 --2- 172.21.15.112:0/2151668774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb880c0420 0x7fbb780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:10.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb877fe640 1 -- 172.21.15.112:0/2151668774 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb88156310 con 0x7fbb780039e0 2024-04-25T01:42:10.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb877fe640 1 -- 172.21.15.112:0/2151668774 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7800ba70 con 0x7fbb78004800 2024-04-25T01:42:10.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.866+0000 7fbb877fe640 1 -- 172.21.15.112:0/2151668774 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbb7800bc70 con 0x7fbb880c0420 2024-04-25T01:42:10.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.867+0000 7fbb8cf95640 1 --2- 172.21.15.112:0/2151668774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb780039e0 0x7fbb78003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:10.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.867+0000 7fbb87fff640 1 --2- 172.21.15.112:0/2151668774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb78004800 0x7fbb78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:10.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.867+0000 7fbb8d796640 1 --2- 172.21.15.112:0/2151668774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb880c0420 0x7fbb780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:10.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.867+0000 7fbb8cf95640 1 -- 172.21.15.112:0/2151668774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb880c0420 msgr2=0x7fbb780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:10.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.867+0000 7fbb8cf95640 1 --2- 172.21.15.112:0/2151668774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb880c0420 0x7fbb780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:10.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.867+0000 7fbb8cf95640 1 -- 172.21.15.112:0/2151668774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb78004800 msgr2=0x7fbb78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:42:10.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.867+0000 7fbb8cf95640 1 --2- 172.21.15.112:0/2151668774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb78004800 0x7fbb78004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:10.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.867+0000 7fbb8cf95640 1 -- 172.21.15.112:0/2151668774 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb78003520 con 0x7fbb780039e0 2024-04-25T01:42:10.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.867+0000 7fbb8cf95640 1 --2- 172.21.15.112:0/2151668774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb780039e0 0x7fbb78003e00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009a90 tx=0x7fbb78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0e8c3277f62365 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:10.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.868+0000 7fbb877fe640 1 -- 172.21.15.112:0/2151668774 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c036070 con 0x7fbb780039e0 2024-04-25T01:42:10.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.868+0000 7fbb877fe640 1 -- 172.21.15.112:0/2151668774 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fbb7c031400 con 0x7fbb780039e0 2024-04-25T01:42:10.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.868+0000 7fbb877fe640 1 -- 172.21.15.112:0/2151668774 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c009690 con 0x7fbb780039e0 2024-04-25T01:42:10.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.868+0000 7fbb8f1f9640 1 -- 172.21.15.112:0/2151668774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb780039e0 msgr2=0x7fbb78003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:10.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.868+0000 7fbb8f1f9640 1 --2- 172.21.15.112:0/2151668774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb780039e0 0x7fbb78003e00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009a90 tx=0x7fbb78005830 comp rx=0 tx=0).stop 2024-04-25T01:42:10.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.868+0000 7fbb8d796640 1 -- 172.21.15.112:0/2151668774 reap_dead start 2024-04-25T01:42:10.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.868+0000 7fbb8f1f9640 1 -- 172.21.15.112:0/2151668774 shutdown_connections 2024-04-25T01:42:10.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.868+0000 7fbb8f1f9640 1 -- 172.21.15.112:0/2151668774 >> 172.21.15.112:0/2151668774 conn(0x7fbb8806b8a0 msgr2=0x7fbb880c6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:10.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.868+0000 7fbb8f1f9640 1 -- 172.21.15.112:0/2151668774 shutdown_connections 2024-04-25T01:42:10.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.868+0000 7fbb8f1f9640 1 -- 172.21.15.112:0/2151668774 wait complete. 2024-04-25T01:42:10.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.869+0000 7fbb8f1f9640 1 Processor -- start 2024-04-25T01:42:10.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.869+0000 7fbb8f1f9640 1 -- start start 2024-04-25T01:42:10.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.870+0000 7fbb8f1f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb780039e0 0x7fbb8814b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:10.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.870+0000 7fbb8f1f9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8814bb90 0x7fbb88157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:10.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.870+0000 7fbb8f1f9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb881581f0 0x7fbb880aa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:10.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.870+0000 7fbb8f1f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8806dc50 con 0x7fbb780039e0 2024-04-25T01:42:10.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.870+0000 7fbb8f1f9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbb880c2c20 con 0x7fbb881581f0 2024-04-25T01:42:10.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.870+0000 7fbb8f1f9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbb8806ddd0 con 0x7fbb8814bb90 2024-04-25T01:42:10.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.870+0000 7fbb8cf95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb780039e0 0x7fbb8814b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:10.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.870+0000 7fbb8cf95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb780039e0 0x7fbb8814b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40344/0 (socket says 172.21.15.112:40344) 2024-04-25T01:42:10.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.870+0000 7fbb8cf95640 1 -- 172.21.15.112:0/1554888741 learned_addr learned my addr 172.21.15.112:0/1554888741 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T01:42:10.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.870+0000 7fbb87fff640 1 --2- 172.21.15.112:0/1554888741 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8814bb90 0x7fbb88157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:10.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.870+0000 7fbb8d796640 1 --2- 172.21.15.112:0/1554888741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb881581f0 0x7fbb880aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:10.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.871+0000 7fbb8cf95640 1 -- 172.21.15.112:0/1554888741 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8814bb90 msgr2=0x7fbb88157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:10.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.871+0000 7fbb8cf95640 1 --2- 172.21.15.112:0/1554888741 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8814bb90 0x7fbb88157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:10.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.871+0000 7fbb8cf95640 1 -- 172.21.15.112:0/1554888741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb881581f0 msgr2=0x7fbb880aa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:10.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.871+0000 7fbb8cf95640 1 --2- 172.21.15.112:0/1554888741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb881581f0 0x7fbb880aa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:10.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.871+0000 7fbb8cf95640 1 -- 172.21.15.112:0/1554888741 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb881ef250 con 0x7fbb780039e0 2024-04-25T01:42:10.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.871+0000 7fbb8d796640 1 --2- 172.21.15.112:0/1554888741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb881581f0 0x7fbb880aa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:42:10.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.871+0000 7fbb8cf95640 1 --2- 172.21.15.112:0/1554888741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb780039e0 0x7fbb8814b650 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c034040 tx=0x7fbb7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:10.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.871+0000 7fbb857fa640 1 -- 172.21.15.112:0/1554888741 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c040020 con 0x7fbb780039e0 2024-04-25T01:42:10.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.871+0000 7fbb857fa640 1 -- 172.21.15.112:0/1554888741 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fbb7c003bc0 con 0x7fbb780039e0 2024-04-25T01:42:10.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.871+0000 7fbb8f1f9640 1 -- 172.21.15.112:0/1554888741 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb881ef530 con 0x7fbb780039e0 2024-04-25T01:42:10.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.871+0000 7fbb857fa640 1 -- 172.21.15.112:0/1554888741 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c036040 con 0x7fbb780039e0 2024-04-25T01:42:10.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.871+0000 7fbb8f1f9640 1 -- 172.21.15.112:0/1554888741 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb880c5be0 con 0x7fbb780039e0 2024-04-25T01:42:10.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.872+0000 7fbb8f1f9640 1 -- 172.21.15.112:0/1554888741 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb880682a0 con 0x7fbb780039e0 2024-04-25T01:42:10.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.872+0000 7fbb857fa640 1 -- 172.21.15.112:0/1554888741 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb7c043020 con 0x7fbb780039e0 2024-04-25T01:42:10.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.873+0000 7fbb857fa640 1 --2- 172.21.15.112:0/1554888741 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbb6c06be60 0x7fbb6c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:10.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.873+0000 7fbb857fa640 1 -- 172.21.15.112:0/1554888741 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5433+0+0 (secure 0 0 0) 0x7fbb7c0aad70 con 0x7fbb780039e0 2024-04-25T01:42:10.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.873+0000 7fbb87fff640 1 --2- 172.21.15.112:0/1554888741 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbb6c06be60 0x7fbb6c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:10.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.873+0000 7fbb87fff640 1 --2- 172.21.15.112:0/1554888741 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbb6c06be60 0x7fbb6c06e320 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbb881e2020 tx=0x7fbb74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:10.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:10.877+0000 7fbb857fa640 1 -- 172.21.15.112:0/1554888741 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbb7c04a070 con 0x7fbb780039e0 2024-04-25T01:42:11.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:11.052+0000 7fbb8f1f9640 1 -- 172.21.15.112:0/1554888741 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "b_data", "pool2": "b_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbb880c1510 con 0x7fbb780039e0 2024-04-25T01:42:11.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:11.746+0000 7fbb857fa640 1 -- 172.21.15.112:0/1554888741 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "b_data", "pool2": "b_data", "yes_i_really_really_mean_it": true}]=0 pool 'b_data' removed v32) ==== 154+0+0 (secure 0 0 0) 0x7fbb7c0044e0 con 0x7fbb780039e0 2024-04-25T01:42:11.734 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'b_data' removed 2024-04-25T01:42:11.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:11.748+0000 7fbb8f1f9640 1 -- 172.21.15.112:0/1554888741 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbb6c06be60 msgr2=0x7fbb6c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:11.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:11.748+0000 7fbb8f1f9640 1 --2- 172.21.15.112:0/1554888741 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbb6c06be60 0x7fbb6c06e320 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbb881e2020 tx=0x7fbb74009040 comp rx=0 tx=0).stop 2024-04-25T01:42:11.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:11.748+0000 7fbb8f1f9640 1 -- 172.21.15.112:0/1554888741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb780039e0 msgr2=0x7fbb8814b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:11.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:11.748+0000 7fbb8f1f9640 1 --2- 172.21.15.112:0/1554888741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb780039e0 0x7fbb8814b650 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c034040 tx=0x7fbb7c004730 comp rx=0 tx=0).stop 2024-04-25T01:42:11.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:11.748+0000 7fbb8f1f9640 1 -- 172.21.15.112:0/1554888741 shutdown_connections 2024-04-25T01:42:11.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:11.748+0000 7fbb8f1f9640 1 --2- 172.21.15.112:0/1554888741 >> [v2:172.21.15.112:6832/3661599222,v1:172.21.15.112:6833/3661599222] conn(0x7fbb6c06be60 0x7fbb6c06e320 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:11.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:11.748+0000 7fbb8f1f9640 1 --2- 172.21.15.112:0/1554888741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb881581f0 0x7fbb880aa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:11.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:11.748+0000 7fbb8f1f9640 1 --2- 172.21.15.112:0/1554888741 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8814bb90 0x7fbb88157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:11.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:11.748+0000 7fbb8f1f9640 1 --2- 172.21.15.112:0/1554888741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb780039e0 0x7fbb8814b650 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:11.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:11.748+0000 7fbb8f1f9640 1 -- 172.21.15.112:0/1554888741 >> 172.21.15.112:0/1554888741 conn(0x7fbb8806b8a0 msgr2=0x7fbb880c3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:11.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:11.748+0000 7fbb8f1f9640 1 -- 172.21.15.112:0/1554888741 shutdown_connections 2024-04-25T01:42:11.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T01:42:11.748+0000 7fbb8f1f9640 1 -- 172.21.15.112:0/1554888741 wait complete. 2024-04-25T01:42:11.747 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T01:42:11.747 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T01:42:11.747 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:42:11.756 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T01:42:11.756 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T01:42:11.756 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:42:11.806 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T01:42:11.806 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T01:42:11.806 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:42:11.820 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T01:42:11.820 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T01:42:11.820 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:42:11.872 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T01:42:11.872 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T01:42:11.872 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T01:42:11.872 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:42:11.948 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T01:42:11.948 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T01:42:11.949 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:42:12.027 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T01:42:12.027 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T01:42:12.027 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:42:12.084 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T01:42:12.084 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T01:42:12.084 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:42:12.166 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T01:42:12.166 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T01:42:12.166 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:42:12.234 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T01:42:12.235 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T01:42:12.235 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:42:12.325 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T01:42:12.326 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T01:42:12.326 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:42:12.393 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T01:42:12.393 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T01:42:12.393 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:42:12.460 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T01:42:12.460 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T01:42:12.460 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T01:42:12.460 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:42:12.488 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T01:42:12.488 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T01:42:12.488 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:42:12.513 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T01:42:12.547 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T01:42:12.547 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T01:42:12.548 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:42:12.558 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T01:42:12.558 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T01:42:12.558 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:42:12.568 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T01:42:12.568 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T01:42:12.568 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:42:12.619 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T01:42:12.619 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T01:42:12.620 DEBUG:teuthology.orchestra.run.smithi112:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-25T01:42:12.620 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-04-25T01:42:12.620 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-04-25T01:42:12.620 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-04-25T01:42:12.697 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T01:42:12.697 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T01:42:13.072 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T01:42:13.072 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T01:42:13.248 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T01:42:13.248 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T01:42:13.339 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T01:42:13.339 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T01:42:13.461 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T01:42:13.461 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T01:42:13.959 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T01:42:13.960 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T01:42:14.072 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T01:42:14.072 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T01:42:14.173 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T01:42:14.173 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T01:42:14.282 INFO:tasks.ceph:Archiving mon data... 2024-04-25T01:42:14.283 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672321/data/mon.a.tgz 2024-04-25T01:42:14.284 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2024-04-25T01:42:14.302 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.mw32CeG1vj 2024-04-25T01:42:14.302 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.mw32CeG1vj 2024-04-25T01:42:14.463 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.mw32CeG1vj 2024-04-25T01:42:14.570 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.mw32CeG1vj is 448KB 2024-04-25T01:42:14.669 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.mw32CeG1vj 2024-04-25T01:42:14.683 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672321/data/mon.b.tgz 2024-04-25T01:42:14.683 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-04-25T01:42:14.704 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.0FlcXr7MdM 2024-04-25T01:42:14.704 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.0FlcXr7MdM 2024-04-25T01:42:14.869 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.0FlcXr7MdM 2024-04-25T01:42:14.981 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.0FlcXr7MdM is 456KB 2024-04-25T01:42:15.049 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.0FlcXr7MdM 2024-04-25T01:42:15.063 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672321/data/mon.c.tgz 2024-04-25T01:42:15.063 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-04-25T01:42:15.117 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.rKI0UXfB35 2024-04-25T01:42:15.118 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.rKI0UXfB35 2024-04-25T01:42:15.264 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.rKI0UXfB35 2024-04-25T01:42:15.344 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.rKI0UXfB35 is 458KB 2024-04-25T01:42:15.414 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.rKI0UXfB35 2024-04-25T01:42:15.429 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T01:42:15.429 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T01:42:15.431 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T01:42:15.509 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T01:42:15.546 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T01:42:15.546 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T01:42:15.549 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672321/remote/smithi112/crash 2024-04-25T01:42:15.550 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T01:42:15.574 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672321/remote/smithi187/crash 2024-04-25T01:42:15.575 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T01:42:15.610 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T01:42:15.611 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:42:15.618 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:42:15.984 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T01:42:16.048 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T01:42:16.088 INFO:tasks.ceph:Compressing logs... 2024-04-25T01:42:16.088 DEBUG:teuthology.orchestra.run.smithi112:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T01:42:16.091 DEBUG:teuthology.orchestra.run.smithi187:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T01:42:16.130 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107667.log 2024-04-25T01:42:16.132 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44396.log 2024-04-25T01:42:16.132 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108812.log 2024-04-25T01:42:16.132 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34364.log 2024-04-25T01:42:16.132 INFO:teuthology.orchestra.run.smithi112.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107667.log.gz 2024-04-25T01:42:16.177 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T01:42:16.177 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34452.log 2024-04-25T01:42:16.177 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34492.log 2024-04-25T01:42:16.177 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.34452.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T01:42:16.177 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.34492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34703.log 2024-04-25T01:42:16.177 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T01:42:16.177 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.34703.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T01:42:16.177 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34623.log 2024-04-25T01:42:16.177 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T01:42:16.177 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.34623.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T01:42:16.178 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T01:42:16.178 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T01:42:16.178 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T01:42:16.178 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34532.log 2024-04-25T01:42:16.178 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34743.log 2024-04-25T01:42:16.178 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.34532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41261.log 2024-04-25T01:42:16.178 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.34743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34663.log 2024-04-25T01:42:16.178 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.41261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34576.log 2024-04-25T01:42:16.178 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.34663.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T01:42:16.178 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.34576.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T01:42:16.194 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34703.log.gz 2024-04-25T01:42:16.208 INFO:teuthology.orchestra.run.smithi187.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34452.log.gz 2024-04-25T01:42:16.208 INFO:teuthology.orchestra.run.smithi187.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34492.log.gz 2024-04-25T01:42:16.208 INFO:teuthology.orchestra.run.smithi187.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34532.log.gz 2024-04-25T01:42:16.208 INFO:teuthology.orchestra.run.smithi187.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34576.log.gz 2024-04-25T01:42:16.213 INFO:teuthology.orchestra.run.smithi187.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34623.log.gz 2024-04-25T01:42:16.213 INFO:teuthology.orchestra.run.smithi187.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34663.log.gz 2024-04-25T01:42:16.259 INFO:teuthology.orchestra.run.smithi187.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34743.log.gz 2024-04-25T01:42:16.272 INFO:teuthology.orchestra.run.smithi187.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T01:42:16.278 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.108812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51490.log 2024-04-25T01:42:16.278 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42854.log 2024-04-25T01:42:16.279 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.108812.log.gz 2024-04-25T01:42:16.279 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47741.log 2024-04-25T01:42:16.279 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54776.log 2024-04-25T01:42:16.279 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.47741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51635.log 2024-04-25T01:42:16.279 INFO:teuthology.orchestra.run.smithi112.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.51490.log.gz 2024-04-25T01:42:16.279 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41109.log 2024-04-25T01:42:16.279 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50572.log 2024-04-25T01:42:16.279 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50656.log 2024-04-25T01:42:16.279 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54692.log 2024-04-25T01:42:16.279 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42157.log 2024-04-25T01:42:16.279 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73375.log 2024-04-25T01:42:16.279 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54776.log.gz 2024-04-25T01:42:16.279 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52612.log 2024-04-25T01:42:16.279 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.73375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54462.log 2024-04-25T01:42:16.280 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45496.log 2024-04-25T01:42:16.280 INFO:teuthology.orchestra.run.smithi112.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.73375.log.gz 2024-04-25T01:42:16.280 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108003.log 2024-04-25T01:42:16.280 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.45496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107226.log 2024-04-25T01:42:16.280 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.108003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55112.log 2024-04-25T01:42:16.280 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107226.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108003.log.gz 2024-04-25T01:42:16.280 INFO:teuthology.orchestra.run.smithi112.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.51635.log.gz 2024-04-25T01:42:16.280 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75537.log 2024-04-25T01:42:16.280 INFO:teuthology.orchestra.run.smithi112.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107226.log.gz 2024-04-25T01:42:16.280 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- 88.4% /var/log/ceph/ceph-client.admin.44270.log 2024-04-25T01:42:16.280 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41109.log.gz 2024-04-25T01:42:16.280 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55112.log: /var/log/ceph/ceph-client.admin.75537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50320.log 2024-04-25T01:42:16.280 INFO:teuthology.orchestra.run.smithi112.stderr: 90.0%/var/log/ceph/ceph-client.admin.44270.log: -- replaced with /var/log/ceph/ceph-client.admin.75537.log.gz 2024-04-25T01:42:16.280 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46854.log 2024-04-25T01:42:16.280 INFO:teuthology.orchestra.run.smithi112.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.55112.log.gz 2024-04-25T01:42:16.281 INFO:teuthology.orchestra.run.smithi112.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44396.log.gz 2024-04-25T01:42:16.281 INFO:teuthology.orchestra.run.smithi112.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.54692.log.gz 2024-04-25T01:42:16.281 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51112.log 2024-04-25T01:42:16.281 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42157.log.gz 2024-04-25T01:42:16.281 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50320.log: /var/log/ceph/ceph-client.admin.46854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41278.log 2024-04-25T01:42:16.281 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42855.log 2024-04-25T01:42:16.281 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73550.log 2024-04-25T01:42:16.281 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50758.log 2024-04-25T01:42:16.281 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.73550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46728.log 2024-04-25T01:42:16.281 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44270.log.gz 2024-04-25T01:42:16.281 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54734.log 2024-04-25T01:42:16.281 INFO:teuthology.orchestra.run.smithi112.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73550.log.gz 2024-04-25T01:42:16.281 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42422.log 2024-04-25T01:42:16.281 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91234.log 2024-04-25T01:42:16.281 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43427.log 2024-04-25T01:42:16.282 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.91234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42845.log 2024-04-25T01:42:16.282 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53187.log 2024-04-25T01:42:16.282 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.91234.log.gz 2024-04-25T01:42:16.282 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42845.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46854.log.gz 2024-04-25T01:42:16.282 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61823.log 2024-04-25T01:42:16.282 INFO:teuthology.orchestra.run.smithi112.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47741.log.gz 2024-04-25T01:42:16.282 INFO:teuthology.orchestra.run.smithi112.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50656.log.gz 2024-04-25T01:42:16.282 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51364.log 2024-04-25T01:42:16.282 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.61823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51154.log 2024-04-25T01:42:16.282 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53856.log 2024-04-25T01:42:16.282 INFO:teuthology.orchestra.run.smithi112.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.61823.log.gz 2024-04-25T01:42:16.282 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46505.log 2024-04-25T01:42:16.282 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46728.log.gz 2024-04-25T01:42:16.282 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51989.log 2024-04-25T01:42:16.283 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46505.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.54734.log.gz 2024-04-25T01:42:16.283 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82665.log 2024-04-25T01:42:16.283 INFO:teuthology.orchestra.run.smithi112.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42422.log.gz 2024-04-25T01:42:16.283 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44144.log 2024-04-25T01:42:16.283 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42845.log.gz 2024-04-25T01:42:16.283 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.82665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106106.log 2024-04-25T01:42:16.283 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82665.log.gz 2024-04-25T01:42:16.283 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45600.log 2024-04-25T01:42:16.283 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.106106.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T01:42:16.283 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42548.log 2024-04-25T01:42:16.283 INFO:teuthology.orchestra.run.smithi112.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.106106.log.gz 2024-04-25T01:42:16.283 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53439.log 2024-04-25T01:42:16.283 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106416.log 2024-04-25T01:42:16.283 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42548.log: /var/log/ceph/ceph-client.admin.53439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46603.log 2024-04-25T01:42:16.283 INFO:teuthology.orchestra.run.smithi112.stderr:gzip/var/log/ceph/ceph-client.admin.106416.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.75315.log 2024-04-25T01:42:16.284 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46603.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42548.log.gz 2024-04-25T01:42:16.284 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62718.log 2024-04-25T01:42:16.284 INFO:teuthology.orchestra.run.smithi112.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.106416.log.gz 2024-04-25T01:42:16.284 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.75315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46770.log 2024-04-25T01:42:16.284 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.62718.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75315.log.gz 2024-04-25T01:42:16.284 INFO:teuthology.orchestra.run.smithi112.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46603.log.gz 2024-04-25T01:42:16.284 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42113.log 2024-04-25T01:42:16.284 INFO:teuthology.orchestra.run.smithi112.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62718.log.gz 2024-04-25T01:42:16.284 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46114.log 2024-04-25T01:42:16.284 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108953.log 2024-04-25T01:42:16.284 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46770.log.gz 2024-04-25T01:42:16.284 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109041.log 2024-04-25T01:42:16.284 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53229.log 2024-04-25T01:42:16.284 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42113.log.gz 2024-04-25T01:42:16.285 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.108953.log: /var/log/ceph/ceph-client.admin.109041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42027.log 2024-04-25T01:42:16.285 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108953.log.gz 2024-04-25T01:42:16.285 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109041.log.gz 2024-04-25T01:42:16.285 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58769.log 2024-04-25T01:42:16.285 INFO:teuthology.orchestra.run.smithi112.stderr: 87.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T01:42:16.285 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43526.log 2024-04-25T01:42:16.285 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.58769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85030.log 2024-04-25T01:42:16.285 INFO:teuthology.orchestra.run.smithi112.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58769.log.gz 2024-04-25T01:42:16.285 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52275.log 2024-04-25T01:42:16.285 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.85030.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T01:42:16.285 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52275.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.85030.log.gz 2024-04-25T01:42:16.285 INFO:teuthology.orchestra.run.smithi112.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.81381.log 2024-04-25T01:42:16.285 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45862.log 2024-04-25T01:42:16.285 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42027.log.gz 2024-04-25T01:42:16.286 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.81381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108171.log 2024-04-25T01:42:16.286 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105893.log 2024-04-25T01:42:16.286 INFO:teuthology.orchestra.run.smithi112.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81381.log.gz 2024-04-25T01:42:16.286 INFO:teuthology.orchestra.run.smithi112.stderr:gzip/var/log/ceph/ceph-client.admin.108171.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43428.log 2024-04-25T01:42:16.286 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.105893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34653.log 2024-04-25T01:42:16.286 INFO:teuthology.orchestra.run.smithi112.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.108171.log.gz 2024-04-25T01:42:16.286 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106232.log 2024-04-25T01:42:16.286 INFO:teuthology.orchestra.run.smithi112.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105893.log.gz 2024-04-25T01:42:16.286 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108728.log 2024-04-25T01:42:16.286 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.106232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41364.log 2024-04-25T01:42:16.286 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106232.log.gz 2024-04-25T01:42:16.286 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.108728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51280.log 2024-04-25T01:42:16.286 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107961.log 2024-04-25T01:42:16.286 INFO:teuthology.orchestra.run.smithi112.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108728.log.gz 2024-04-25T01:42:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42856.log 2024-04-25T01:42:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43189.log 2024-04-25T01:42:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42856.log: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.50758.log.gz 2024-04-25T01:42:16.287 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107961.log.gz 2024-04-25T01:42:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41561.log 2024-04-25T01:42:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95625.log 2024-04-25T01:42:16.287 INFO:teuthology.orchestra.run.smithi112.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.54462.log.gz 2024-04-25T01:42:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51551.log 2024-04-25T01:42:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.95625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101369.log 2024-04-25T01:42:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106890.log 2024-04-25T01:42:16.287 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95625.log.gz 2024-04-25T01:42:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.101369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44102.log 2024-04-25T01:42:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.106890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98025.log 2024-04-25T01:42:16.287 INFO:teuthology.orchestra.run.smithi112.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.101369.log.gz 2024-04-25T01:42:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5/var/log/ceph/ceph-client.admin.44102.log: --verbose -- /var/log/ceph/ceph-client.admin.53355.log 2024-04-25T01:42:16.288 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106890.log.gz 2024-04-25T01:42:16.288 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51677.log 2024-04-25T01:42:16.288 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.98025.log: /var/log/ceph/ceph-client.admin.53355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95537.log 2024-04-25T01:42:16.288 INFO:teuthology.orchestra.run.smithi112.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50572.log.gz 2024-04-25T01:42:16.288 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98025.log.gz 2024-04-25T01:42:16.288 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51677.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51989.log.gz 2024-04-25T01:42:16.288 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108457.log 2024-04-25T01:42:16.288 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.95537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101274.log 2024-04-25T01:42:16.288 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52612.log.gz 2024-04-25T01:42:16.288 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.108457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107541.log 2024-04-25T01:42:16.288 INFO:teuthology.orchestra.run.smithi112.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.95537.log.gz 2024-04-25T01:42:16.288 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108457.log.gz 2024-04-25T01:42:16.288 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.101274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50800.log 2024-04-25T01:42:16.288 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53607.log 2024-04-25T01:42:16.288 INFO:teuthology.orchestra.run.smithi112.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51551.log.gz 2024-04-25T01:42:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107541.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101274.log.gz 2024-04-25T01:42:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50800.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T01:42:16.289 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107541.log.gz 2024-04-25T01:42:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84947.log 2024-04-25T01:42:16.289 INFO:teuthology.orchestra.run.smithi112.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.51677.log.gz 2024-04-25T01:42:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43527.log 2024-04-25T01:42:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.84947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108686.log 2024-04-25T01:42:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55740.log 2024-04-25T01:42:16.289 INFO:teuthology.orchestra.run.smithi112.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.84947.log.gz 2024-04-25T01:42:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.108686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107583.log 2024-04-25T01:42:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55740.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108686.log.gz 2024-04-25T01:42:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107877.log 2024-04-25T01:42:16.289 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55740.log.gz 2024-04-25T01:42:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44354.log 2024-04-25T01:42:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107877.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46505.log.gz 2024-04-25T01:42:16.290 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107583.log.gzgzip 2024-04-25T01:42:16.290 INFO:teuthology.orchestra.run.smithi112.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51070.log 2024-04-25T01:42:16.290 INFO:teuthology.orchestra.run.smithi112.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107877.log.gz 2024-04-25T01:42:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:gzip/var/log/ceph/ceph-client.admin.44354.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.108045.log 2024-04-25T01:42:16.290 INFO:teuthology.orchestra.run.smithi112.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.50800.log.gz 2024-04-25T01:42:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107709.log 2024-04-25T01:42:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51070.log: /var/log/ceph/ceph-client.admin.108045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106764.log 2024-04-25T01:42:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107709.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108045.log.gz 2024-04-25T01:42:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41649.log 2024-04-25T01:42:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.106764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42380.log 2024-04-25T01:42:16.290 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107709.log.gz 2024-04-25T01:42:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55319.log 2024-04-25T01:42:16.290 INFO:teuthology.orchestra.run.smithi112.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.106764.log.gz 2024-04-25T01:42:16.291 INFO:teuthology.orchestra.run.smithi112.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42854.log.gz 2024-04-25T01:42:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42380.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42855.log.gz 2024-04-25T01:42:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53940.log 2024-04-25T01:42:16.291 INFO:teuthology.orchestra.run.smithi112.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42856.log.gz 2024-04-25T01:42:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41408.log 2024-04-25T01:42:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53940.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55319.log.gz 2024-04-25T01:42:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50968.log 2024-04-25T01:42:16.291 INFO:teuthology.orchestra.run.smithi112.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43189.log.gz 2024-04-25T01:42:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85603.log 2024-04-25T01:42:16.291 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43427.log.gz 2024-04-25T01:42:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:gzip/var/log/ceph/ceph-client.admin.50968.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46812.log 2024-04-25T01:42:16.291 INFO:teuthology.orchestra.run.smithi112.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43428.log.gz 2024-04-25T01:42:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.85603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54818.log 2024-04-25T01:42:16.291 INFO:teuthology.orchestra.run.smithi112.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43526.log.gz 2024-04-25T01:42:16.291 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.85603.log.gz 2024-04-25T01:42:16.292 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108583.log 2024-04-25T01:42:16.292 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51238.log 2024-04-25T01:42:16.292 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44102.log.gz 2024-04-25T01:42:16.292 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44144.log.gz 2024-04-25T01:42:16.292 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54818.log.gz 2024-04-25T01:42:16.292 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.108583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43524.log 2024-04-25T01:42:16.292 INFO:teuthology.orchestra.run.smithi112.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108583.log.gz 2024-04-25T01:42:16.292 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99420.log 2024-04-25T01:42:16.292 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47021.log 2024-04-25T01:42:16.292 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.99420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55529.log 2024-04-25T01:42:16.292 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.47021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55172.log 2024-04-25T01:42:16.292 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99420.log.gz 2024-04-25T01:42:16.292 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52756.log 2024-04-25T01:42:16.292 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54650.log 2024-04-25T01:42:16.293 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55172.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55529.log.gz 2024-04-25T01:42:16.293 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42208.log 2024-04-25T01:42:16.293 INFO:teuthology.orchestra.run.smithi112.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.55172.log.gz 2024-04-25T01:42:16.293 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104698.log 2024-04-25T01:42:16.293 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42208.log: gzip -5 --verbose -- 88.9% /var/log/ceph/ceph-client.admin.106274.log 2024-04-25T01:42:16.293 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45600.log.gz 2024-04-25T01:42:16.293 INFO:teuthology.orchestra.run.smithi112.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.53940.log.gz 2024-04-25T01:42:16.293 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.104698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46072.log 2024-04-25T01:42:16.293 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.106274.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104698.log.gz 2024-04-25T01:42:16.293 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T01:42:16.293 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42380.log.gz 2024-04-25T01:42:16.293 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106274.log.gz 2024-04-25T01:42:16.293 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46645.log 2024-04-25T01:42:16.293 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.3.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43527.log.gz 2024-04-25T01:42:16.293 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50446.log 2024-04-25T01:42:16.294 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44354.log.gz 2024-04-25T01:42:16.294 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107457.log 2024-04-25T01:42:16.294 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106848.log 2024-04-25T01:42:16.294 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45862.log.gz 2024-04-25T01:42:16.294 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107457.log: gzip 88.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.46114.log.gz --verbose 2024-04-25T01:42:16.294 INFO:teuthology.orchestra.run.smithi112.stderr: -- /var/log/ceph/ceph-client.admin.41320.log 2024-04-25T01:42:16.294 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.106848.log: gzip 88.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.107457.log.gz -- 2024-04-25T01:42:16.294 INFO:teuthology.orchestra.run.smithi112.stderr: /var/log/ceph/ceph-client.admin.44438.log 2024-04-25T01:42:16.294 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41320.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106848.log.gz 2024-04-25T01:42:16.294 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108770.log 2024-04-25T01:42:16.294 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46463.log 2024-04-25T01:42:16.294 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44438.log: /var/log/ceph/ceph-client.admin.108770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78364.log 2024-04-25T01:42:16.294 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7%/var/log/ceph/ceph-client.admin.46463.log: -- replaced with /var/log/ceph/ceph-client.admin.108770.log.gz 2024-04-25T01:42:16.294 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76312.log 2024-04-25T01:42:16.294 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.78364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51863.log 2024-04-25T01:42:16.295 INFO:teuthology.orchestra.run.smithi112.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.54650.log.gz 2024-04-25T01:42:16.295 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.76312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41943.log 2024-04-25T01:42:16.295 INFO:teuthology.orchestra.run.smithi112.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.78364.log.gz 2024-04-25T01:42:16.295 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51863.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76312.log.gz 2024-04-25T01:42:16.295 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53667.log 2024-04-25T01:42:16.295 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43525.log 2024-04-25T01:42:16.295 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54192.log 2024-04-25T01:42:16.295 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98736.log 2024-04-25T01:42:16.295 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43525.log: gzip -5 89.1% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.46812.log.gz /var/log/ceph/ceph-client.admin.63079.log 2024-04-25T01:42:16.295 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T01:42:16.295 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54192.log: /var/log/ceph/ceph-client.admin.98736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76931.log 2024-04-25T01:42:16.295 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.63079.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98736.log.gz 2024-04-25T01:42:16.295 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43528.log 2024-04-25T01:42:16.295 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.76931.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63079.log.gz 2024-04-25T01:42:16.295 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107268.log 2024-04-25T01:42:16.296 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43528.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76931.log.gz 2024-04-25T01:42:16.296 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41943.log.gz 2024-04-25T01:42:16.296 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95219.log 2024-04-25T01:42:16.296 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50926.log 2024-04-25T01:42:16.296 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.95219.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107268.log.gz 2024-04-25T01:42:16.296 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73469.log 2024-04-25T01:42:16.296 INFO:teuthology.orchestra.run.smithi112.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95219.log.gz 2024-04-25T01:42:16.296 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75628.log 2024-04-25T01:42:16.296 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50926.log: /var/log/ceph/ceph-client.admin.73469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106316.log 2024-04-25T01:42:16.296 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.75628.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73469.log.gzgzip 2024-04-25T01:42:16.296 INFO:teuthology.orchestra.run.smithi112.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54860.log 2024-04-25T01:42:16.296 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42208.log.gz 2024-04-25T01:42:16.296 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.106316.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.75628.log.gz 2024-04-25T01:42:16.296 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55613.log 2024-04-25T01:42:16.297 INFO:teuthology.orchestra.run.smithi112.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43525.log.gz 2024-04-25T01:42:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54860.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43524.log.gz 2024-04-25T01:42:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53145.log 2024-04-25T01:42:16.297 INFO:teuthology.orchestra.run.smithi112.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106316.log.gz 2024-04-25T01:42:16.297 INFO:teuthology.orchestra.run.smithi112.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43528.log.gz 2024-04-25T01:42:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55613.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.54860.log.gz 2024-04-25T01:42:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52401.log 2024-04-25T01:42:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53145.log: 89.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.55613.log.gz -5 2024-04-25T01:42:16.297 INFO:teuthology.orchestra.run.smithi112.stderr: --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T01:42:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73303.log 2024-04-25T01:42:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52401.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44438.log.gz 2024-04-25T01:42:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50362.log 2024-04-25T01:42:16.297 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46072.log.gz 2024-04-25T01:42:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.73303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46939.log 2024-04-25T01:42:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53565.log 2024-04-25T01:42:16.298 INFO:teuthology.orchestra.run.smithi112.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.73303.log.gz 2024-04-25T01:42:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54902.log 2024-04-25T01:42:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50278.log 2024-04-25T01:42:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53565.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46463.log.gz 2024-04-25T01:42:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106358.log 2024-04-25T01:42:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107184.log 2024-04-25T01:42:16.298 INFO:teuthology.orchestra.run.smithi112.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54902.log.gz 2024-04-25T01:42:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.106358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65043.log 2024-04-25T01:42:16.298 INFO:teuthology.orchestra.run.smithi112.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106358.log.gz 2024-04-25T01:42:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107184.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.46645.log.gz 2024-04-25T01:42:16.298 INFO:teuthology.orchestra.run.smithi112.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41859.log 2024-04-25T01:42:16.298 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107184.log.gz 2024-04-25T01:42:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.65043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77564.log 2024-04-25T01:42:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41859.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65043.log.gz 2024-04-25T01:42:16.299 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84786.log 2024-04-25T01:42:16.299 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.77564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46549.log 2024-04-25T01:42:16.299 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.84786.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T01:42:16.299 INFO:teuthology.orchestra.run.smithi112.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.77564.log.gz 2024-04-25T01:42:16.299 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106542.log 2024-04-25T01:42:16.299 INFO:teuthology.orchestra.run.smithi112.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.84786.log.gz 2024-04-25T01:42:16.299 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T01:42:16.299 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.106542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53022.log 2024-04-25T01:42:16.299 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60059.log 2024-04-25T01:42:16.299 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106542.log.gz 2024-04-25T01:42:16.299 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43186.log 2024-04-25T01:42:16.299 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.60059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52570.log 2024-04-25T01:42:16.299 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43186.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60059.log.gz 2024-04-25T01:42:16.299 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46240.log 2024-04-25T01:42:16.300 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53271.log 2024-04-25T01:42:16.300 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42674.log 2024-04-25T01:42:16.300 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108087.log 2024-04-25T01:42:16.300 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42674.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T01:42:16.300 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52115.log 2024-04-25T01:42:16.300 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.108087.log: /var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46298.log 2024-04-25T01:42:16.300 INFO:teuthology.orchestra.run.smithi112.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.108087.log.gz 2024-04-25T01:42:16.300 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43191.log 2024-04-25T01:42:16.300 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50614.log 2024-04-25T01:42:16.300 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59411.log 2024-04-25T01:42:16.300 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100007.log 2024-04-25T01:42:16.300 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.59411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55825.log 2024-04-25T01:42:16.300 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.100007.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59411.log.gz 2024-04-25T01:42:16.300 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107835.log 2024-04-25T01:42:16.300 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41067.log 2024-04-25T01:42:16.301 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107835.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.100007.log.gz 2024-04-25T01:42:16.301 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105851.log 2024-04-25T01:42:16.301 INFO:teuthology.orchestra.run.smithi112.stderr: 91.0% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107835.log.gz 2024-04-25T01:42:16.301 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55825.log.gz 2024-04-25T01:42:16.301 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53710.log 2024-04-25T01:42:16.301 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.105851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51821.log 2024-04-25T01:42:16.301 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105851.log.gz 2024-04-25T01:42:16.301 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44060.log 2024-04-25T01:42:16.301 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54234.log 2024-04-25T01:42:16.301 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45068.log 2024-04-25T01:42:16.301 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42590.log 2024-04-25T01:42:16.301 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46379.log 2024-04-25T01:42:16.301 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54420.log 2024-04-25T01:42:16.301 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53752.log 2024-04-25T01:42:16.301 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44714.log 2024-04-25T01:42:16.302 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107100.log 2024-04-25T01:42:16.302 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108129.log 2024-04-25T01:42:16.302 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44714.log: /var/log/ceph/ceph-client.admin.107100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54150.log 2024-04-25T01:42:16.302 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.108129.log: gzip 87.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.107100.log.gz --verbose 2024-04-25T01:42:16.302 INFO:teuthology.orchestra.run.smithi112.stderr: -- /var/log/ceph/ceph-client.admin.60703.log 2024-04-25T01:42:16.302 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41607.log 2024-04-25T01:42:16.302 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108129.log.gz 2024-04-25T01:42:16.302 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.60703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91877.log 2024-04-25T01:42:16.302 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43892.log 2024-04-25T01:42:16.302 INFO:teuthology.orchestra.run.smithi112.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.60703.log.gz 2024-04-25T01:42:16.302 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.91877.log: /var/log/ceph/ceph-client.admin.41985.log 2024-04-25T01:42:16.302 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43892.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91877.log.gz 2024-04-25T01:42:16.302 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T01:42:16.302 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41985.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95359.log 2024-04-25T01:42:16.303 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53187.log.gz 2024-04-25T01:42:16.303 INFO:teuthology.orchestra.run.smithi112.stderr: 89.5%/var/log/ceph/ceph-mon.a.log: -- replaced with /var/log/ceph/ceph-client.admin.53229.log.gz 2024-04-25T01:42:16.303 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106584.log 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53607.log.gz 2024-04-25T01:42:16.303 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T01:42:16.303 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.95359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54276.log 2024-04-25T01:42:16.303 INFO:teuthology.orchestra.run.smithi112.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.95359.log.gz 2024-04-25T01:42:16.303 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.106584.log: gzip -5 --verbose 91.2% -- replaced with /var/log/ceph/ceph-client.admin.53667.log.gz 2024-04-25T01:42:16.303 INFO:teuthology.orchestra.run.smithi112.stderr: -- /var/log/ceph/ceph-client.admin.44840.log 2024-04-25T01:42:16.303 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54276.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.106584.log.gz 2024-04-25T01:42:16.303 INFO:teuthology.orchestra.run.smithi112.stderr:gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54192.log.gz -5 2024-04-25T01:42:16.303 INFO:teuthology.orchestra.run.smithi112.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54024.log 2024-04-25T01:42:16.303 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46030.log 2024-04-25T01:42:16.303 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90592.log 2024-04-25T01:42:16.303 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.46030.log: -- /var/log/ceph/ceph-client.admin.43426.log 2024-04-25T01:42:16.304 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.90592.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.52570.log.gz -5 2024-04-25T01:42:16.304 INFO:teuthology.orchestra.run.smithi112.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51196.log 2024-04-25T01:42:16.304 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43426.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.90592.log.gz 2024-04-25T01:42:16.304 INFO:teuthology.orchestra.run.smithi112.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54548.log 2024-04-25T01:42:16.304 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34746.log 2024-04-25T01:42:16.304 INFO:teuthology.orchestra.run.smithi112.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53022.log.gz 2024-04-25T01:42:16.304 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97812.log 2024-04-25T01:42:16.304 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51719.log 2024-04-25T01:42:16.304 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34746.log: /var/log/ceph/ceph-client.admin.97812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82827.log 2024-04-25T01:42:16.304 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51719.log: gzip -5 --verbose -- 89.1% /var/log/ceph/ceph-client.admin.97909.log 2024-04-25T01:42:16.304 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97812.log.gz 2024-04-25T01:42:16.304 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5/var/log/ceph/ceph-client.admin.82827.log: --verbose -- /var/log/ceph/ceph-client.admin.50404.log 2024-04-25T01:42:16.304 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.97909.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.82827.log.gz 2024-04-25T01:42:16.304 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83052.log 2024-04-25T01:42:16.304 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50404.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.97909.log.gz 2024-04-25T01:42:16.304 INFO:teuthology.orchestra.run.smithi112.stderr:gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51364.log.gz 2024-04-25T01:42:16.305 INFO:teuthology.orchestra.run.smithi112.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43183.log 2024-04-25T01:42:16.305 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.83052.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53355.log.gz 2024-04-25T01:42:16.305 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58080.log 2024-04-25T01:42:16.305 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83052.log.gz 2024-04-25T01:42:16.305 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86125.log 2024-04-25T01:42:16.305 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.58080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51010.log 2024-04-25T01:42:16.305 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.86125.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58080.log.gz 2024-04-25T01:42:16.305 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51779.log 2024-04-25T01:42:16.305 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51010.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86125.log.gz 2024-04-25T01:42:16.305 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52654.log 2024-04-25T01:42:16.305 INFO:teuthology.orchestra.run.smithi112.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52115.log.gz 2024-04-25T01:42:16.305 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106032.log 2024-04-25T01:42:16.305 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44882.log 2024-04-25T01:42:16.305 INFO:teuthology.orchestra.run.smithi112.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.54024.log.gz 2024-04-25T01:42:16.305 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.106032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43850.log 2024-04-25T01:42:16.306 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44882.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106032.log.gz 2024-04-25T01:42:16.306 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44522.log 2024-04-25T01:42:16.306 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107058.log 2024-04-25T01:42:16.306 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106190.log 2024-04-25T01:42:16.306 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55697.log 2024-04-25T01:42:16.306 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107058.log: /var/log/ceph/ceph-client.admin.106190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45820.log 2024-04-25T01:42:16.306 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107058.log.gz 2024-04-25T01:42:16.306 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47594.log 2024-04-25T01:42:16.306 INFO:teuthology.orchestra.run.smithi112.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106190.log.gz 2024-04-25T01:42:16.306 INFO:teuthology.orchestra.run.smithi112.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.50968.log.gz 2024-04-25T01:42:16.306 INFO:teuthology.orchestra.run.smithi112.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51070.log.gz 2024-04-25T01:42:16.306 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45820.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55697.log.gz 2024-04-25T01:42:16.306 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42849.log 2024-04-25T01:42:16.306 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.47594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41519.log 2024-04-25T01:42:16.307 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87310.log 2024-04-25T01:42:16.307 INFO:teuthology.orchestra.run.smithi112.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51010.log.gz 2024-04-25T01:42:16.307 INFO:teuthology.orchestra.run.smithi112.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.54420.log.gz 2024-04-25T01:42:16.307 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55445.log 2024-04-25T01:42:16.307 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.87310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45946.log 2024-04-25T01:42:16.307 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50614.log.gz 2024-04-25T01:42:16.307 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55487.log 2024-04-25T01:42:16.307 INFO:teuthology.orchestra.run.smithi112.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.87310.log.gz 2024-04-25T01:42:16.307 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55445.log: /var/log/ceph/ceph-client.admin.45946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40798.log 2024-04-25T01:42:16.307 INFO:teuthology.orchestra.run.smithi112.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.55445.log.gz 2024-04-25T01:42:16.307 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41775.log/var/log/ceph/ceph-client.admin.55487.log: 2024-04-25T01:42:16.307 INFO:teuthology.orchestra.run.smithi112.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50926.log.gz 2024-04-25T01:42:16.307 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.40798.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53145.log.gz 2024-04-25T01:42:16.307 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84863.log 2024-04-25T01:42:16.307 INFO:teuthology.orchestra.run.smithi112.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55487.log.gz 2024-04-25T01:42:16.307 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55403.log 2024-04-25T01:42:16.308 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.84863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53794.log 2024-04-25T01:42:16.308 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55403.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84863.log.gz 2024-04-25T01:42:16.308 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41901.log 2024-04-25T01:42:16.308 INFO:teuthology.orchestra.run.smithi112.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55403.log.gz/var/log/ceph/ceph-client.admin.53794.log: 2024-04-25T01:42:16.308 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54108.log 2024-04-25T01:42:16.308 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46198.log 2024-04-25T01:42:16.308 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108255.log 2024-04-25T01:42:16.308 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107411.log 2024-04-25T01:42:16.308 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.108255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50530.log 2024-04-25T01:42:16.308 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53313.log 2024-04-25T01:42:16.308 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108255.log.gz 2024-04-25T01:42:16.308 INFO:teuthology.orchestra.run.smithi112.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50278.log.gz 2024-04-25T01:42:16.308 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50530.log: 88.3% 89.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.50320.log.gz 2024-04-25T01:42:16.308 INFO:teuthology.orchestra.run.smithi112.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.107625.log 2024-04-25T01:42:16.309 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.107411.log.gz 2024-04-25T01:42:16.309 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40986.log 2024-04-25T01:42:16.309 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75711.log 2024-04-25T01:42:16.309 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107625.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50362.log.gz 2024-04-25T01:42:16.309 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51112.log.gz 2024-04-25T01:42:16.309 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.40986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41477.log 2024-04-25T01:42:16.309 INFO:teuthology.orchestra.run.smithi112.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.107625.log.gz 2024-04-25T01:42:16.309 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.75711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107016.log 2024-04-25T01:42:16.309 INFO:teuthology.orchestra.run.smithi112.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.51154.log.gz 2024-04-25T01:42:16.309 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41477.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.51238.log.gz 2024-04-25T01:42:16.309 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43976.log 2024-04-25T01:42:16.309 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75711.log.gz 2024-04-25T01:42:16.309 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107016.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51280.log.gz 2024-04-25T01:42:16.309 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42250.log 2024-04-25T01:42:16.309 INFO:teuthology.orchestra.run.smithi112.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51821.log.gz 2024-04-25T01:42:16.310 INFO:teuthology.orchestra.run.smithi112.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51863.log.gz 2024-04-25T01:42:16.310 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43976.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.107016.log.gz 2024-04-25T01:42:16.310 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55259.log 2024-04-25T01:42:16.310 INFO:teuthology.orchestra.run.smithi112.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.53439.log.gz 2024-04-25T01:42:16.310 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42632.log 2024-04-25T01:42:16.310 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51947.log 2024-04-25T01:42:16.310 INFO:teuthology.orchestra.run.smithi112.stderr: 88.6% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53271.log.gz 2024-04-25T01:42:16.310 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42632.log: -- replaced with /var/log/ceph/ceph-client.admin.53313.log.gz 2024-04-25T01:42:16.310 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107142.log 2024-04-25T01:42:16.310 INFO:teuthology.orchestra.run.smithi112.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.55259.log.gz 2024-04-25T01:42:16.310 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51947.log: 89.3% 89.7% -- replaced with /var/log/ceph/ceph-client.admin.53752.log.gz -- replaced with /var/log/ceph/ceph-client.admin.53710.log.gz 2024-04-25T01:42:16.310 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T01:42:16.310 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55361.log 2024-04-25T01:42:16.310 INFO:teuthology.orchestra.run.smithi112.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.54150.log.gz 2024-04-25T01:42:16.311 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55885.log 2024-04-25T01:42:16.311 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107142.log: -- replaced with /var/log/ceph/ceph-client.admin.54108.log.gz 2024-04-25T01:42:16.311 INFO:teuthology.orchestra.run.smithi112.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54276.log.gz 2024-04-25T01:42:16.311 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55361.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107142.log.gz 2024-04-25T01:42:16.311 INFO:teuthology.orchestra.run.smithi112.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55214.log 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51196.log.gz 2024-04-25T01:42:16.311 INFO:teuthology.orchestra.run.smithi112.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.55361.log.gz 2024-04-25T01:42:16.311 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55885.log: 2024-04-25T01:42:16.311 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54234.log.gz 2024-04-25T01:42:16.311 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95451.log 2024-04-25T01:42:16.311 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% 89.9% -- replaced with /var/log/ceph/ceph-client.admin.55885.log.gz 2024-04-25T01:42:16.311 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50404.log.gz 2024-04-25T01:42:16.311 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55214.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51779.log.gz 2024-04-25T01:42:16.311 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- 88.6% /var/log/ceph/ceph-client.admin.44018.log 2024-04-25T01:42:16.311 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53565.log.gz 2024-04-25T01:42:16.311 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55214.log.gz 2024-04-25T01:42:16.312 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.95451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105423.log 2024-04-25T01:42:16.312 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7%/var/log/ceph/ceph-client.admin.44018.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.53856.log.gz 2024-04-25T01:42:16.312 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95451.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108375.log 2024-04-25T01:42:16.312 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T01:42:16.312 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.105423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55655.log 2024-04-25T01:42:16.312 INFO:teuthology.orchestra.run.smithi112.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.105423.log.gz 2024-04-25T01:42:16.312 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.108375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44186.log 2024-04-25T01:42:16.312 INFO:teuthology.orchestra.run.smithi112.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.52654.log.gz 2024-04-25T01:42:16.312 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55655.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.108375.log.gz 2024-04-25T01:42:16.312 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93257.log 2024-04-25T01:42:16.312 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53898.log 2024-04-25T01:42:16.312 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55655.log.gz 2024-04-25T01:42:16.312 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108625.log 2024-04-25T01:42:16.312 INFO:teuthology.orchestra.run.smithi112.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.47594.log.gz 2024-04-25T01:42:16.312 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.93257.log: /var/log/ceph/ceph-client.admin.53898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50884.log 2024-04-25T01:42:16.313 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.108625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53481.log 2024-04-25T01:42:16.313 INFO:teuthology.orchestra.run.smithi112.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.93257.log.gz 2024-04-25T01:42:16.313 INFO:teuthology.orchestra.run.smithi112.stderr: 88.1% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41067.log.gz -- replaced with /var/log/ceph/ceph-client.admin.40986.log.gz 2024-04-25T01:42:16.313 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T01:42:16.313 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50884.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.108625.log.gz 2024-04-25T01:42:16.313 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106974.log 2024-04-25T01:42:16.313 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54986.log 2024-04-25T01:42:16.313 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41985.log.gz 2024-04-25T01:42:16.313 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.106974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43192.log 2024-04-25T01:42:16.313 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54986.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106974.log.gz 2024-04-25T01:42:16.313 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52528.log 2024-04-25T01:42:16.313 INFO:teuthology.orchestra.run.smithi112.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54986.log.gz 2024-04-25T01:42:16.313 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43178.log 2024-04-25T01:42:16.313 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54360.log 2024-04-25T01:42:16.313 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75453.log 2024-04-25T01:42:16.314 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42250.log.gz 2024-04-25T01:42:16.314 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53523.log 2024-04-25T01:42:16.314 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.75453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47063.log 2024-04-25T01:42:16.314 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53523.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.75453.log.gz 2024-04-25T01:42:16.314 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105725.log 2024-04-25T01:42:16.314 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.47063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82041.log 2024-04-25T01:42:16.314 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.105725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108541.log 2024-04-25T01:42:16.314 INFO:teuthology.orchestra.run.smithi112.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42632.log.gz 2024-04-25T01:42:16.314 INFO:teuthology.orchestra.run.smithi112.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42590.log.gz 2024-04-25T01:42:16.314 INFO:teuthology.orchestra.run.smithi112.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42674.log.gz 2024-04-25T01:42:16.314 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.82041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86717.log 2024-04-25T01:42:16.314 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105725.log.gz 2024-04-25T01:42:16.314 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.108541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42292.log 2024-04-25T01:42:16.314 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.82041.log.gz 2024-04-25T01:42:16.315 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.86717.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108541.log.gz 2024-04-25T01:42:16.315 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55028.log 2024-04-25T01:42:16.315 INFO:teuthology.orchestra.run.smithi112.stderr:gzip/var/log/ceph/ceph-client.admin.42292.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.106668.log 2024-04-25T01:42:16.315 INFO:teuthology.orchestra.run.smithi112.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.86717.log.gz 2024-04-25T01:42:16.315 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44588.log 2024-04-25T01:42:16.315 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.106668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45988.log 2024-04-25T01:42:16.315 INFO:teuthology.orchestra.run.smithi112.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55028.log.gz 2024-04-25T01:42:16.315 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42849.log.gz 2024-04-25T01:42:16.315 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44588.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106668.log.gz 2024-04-25T01:42:16.315 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41817.log 2024-04-25T01:42:16.315 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42843.log 2024-04-25T01:42:16.315 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105809.log 2024-04-25T01:42:16.315 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44756.log 2024-04-25T01:42:16.315 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.105809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65197.log 2024-04-25T01:42:16.315 INFO:teuthology.orchestra.run.smithi112.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43183.log.gz 2024-04-25T01:42:16.316 INFO:teuthology.orchestra.run.smithi112.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43186.log.gz 2024-04-25T01:42:16.316 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51905.log 2024-04-25T01:42:16.316 INFO:teuthology.orchestra.run.smithi112.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.105809.log.gz 2024-04-25T01:42:16.316 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.65197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108315.log 2024-04-25T01:42:16.316 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54318.log 2024-04-25T01:42:16.316 INFO:teuthology.orchestra.run.smithi112.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.65197.log.gz 2024-04-25T01:42:16.316 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.108315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42853.log 2024-04-25T01:42:16.316 INFO:teuthology.orchestra.run.smithi112.stderr: 88.5%/var/log/ceph/ceph-client.admin.54318.log: -- replaced with /var/log/ceph/ceph-client.admin.108315.log.gz 2024-04-25T01:42:16.316 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52031.log 2024-04-25T01:42:16.316 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87388.log 2024-04-25T01:42:16.316 INFO:teuthology.orchestra.run.smithi112.stderr: 88.2% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43192.log.gz -- replaced with /var/log/ceph/ceph-client.admin.43426.log.gz 2024-04-25T01:42:16.316 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T01:42:16.316 INFO:teuthology.orchestra.run.smithi112.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43191.log.gz 2024-04-25T01:42:16.316 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53982.log 2024-04-25T01:42:16.316 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52031.log: /var/log/ceph/ceph-client.admin.87388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42464.log 2024-04-25T01:42:16.317 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43934.log 2024-04-25T01:42:16.317 INFO:teuthology.orchestra.run.smithi112.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.87388.log.gz 2024-04-25T01:42:16.317 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42464.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43850.log.gz 2024-04-25T01:42:16.317 INFO:teuthology.orchestra.run.smithi112.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43892.log.gz 2024-04-25T01:42:16.317 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41152.log 2024-04-25T01:42:16.317 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43934.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44060.log.gz 2024-04-25T01:42:16.317 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107919.log 2024-04-25T01:42:16.317 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44018.log.gz 2024-04-25T01:42:16.317 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43976.log.gz 2024-04-25T01:42:16.317 INFO:teuthology.orchestra.run.smithi112.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44186.log.gz 2024-04-25T01:42:16.317 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73153.log 2024-04-25T01:42:16.317 INFO:teuthology.orchestra.run.smithi112.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44588.log.gz 2024-04-25T01:42:16.317 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44714.log.gz 2024-04-25T01:42:16.317 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107919.log: -- replaced with /var/log/ceph/ceph-client.admin.44756.log.gz 2024-04-25T01:42:16.318 INFO:teuthology.orchestra.run.smithi112.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44522.log.gz 2024-04-25T01:42:16.318 INFO:teuthology.orchestra.run.smithi112.stderr: 89.3% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44840.log.gz -- replaced with /var/log/ceph/ceph-client.admin.44882.log.gz 2024-04-25T01:42:16.318 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T01:42:16.318 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.73153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47105.log 2024-04-25T01:42:16.318 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107919.log.gz 2024-04-25T01:42:16.318 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45068.log.gz 2024-04-25T01:42:16.318 INFO:teuthology.orchestra.run.smithi112.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73153.log.gz 2024-04-25T01:42:16.318 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53103.log 2024-04-25T01:42:16.318 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.47105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106500.log 2024-04-25T01:42:16.318 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45820.log.gz 2024-04-25T01:42:16.318 INFO:teuthology.orchestra.run.smithi112.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46030.log.gz 2024-04-25T01:42:16.318 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45988.log.gz 2024-04-25T01:42:16.318 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53103.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45946.log.gz 2024-04-25T01:42:16.318 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62994.log 2024-04-25T01:42:16.318 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54066.log 2024-04-25T01:42:16.319 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.106500.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46298.log.gz 2024-04-25T01:42:16.319 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46240.log.gz 2024-04-25T01:42:16.319 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46198.log.gz 2024-04-25T01:42:16.319 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.62994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108499.log 2024-04-25T01:42:16.319 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106500.log.gz 2024-04-25T01:42:16.319 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54066.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62994.log.gz 2024-04-25T01:42:16.319 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46939.log.gz 2024-04-25T01:42:16.319 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45778.log 2024-04-25T01:42:16.319 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.108499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43724.log 2024-04-25T01:42:16.319 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45778.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108499.log.gz 2024-04-25T01:42:16.319 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54590.log 2024-04-25T01:42:16.319 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44228.log 2024-04-25T01:42:16.319 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43724.log: /var/log/ceph/ceph-client.admin.54590.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46379.log.gz 2024-04-25T01:42:16.319 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44480.log 2024-04-25T01:42:16.319 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107793.log 2024-04-25T01:42:16.320 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52359.log 2024-04-25T01:42:16.320 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106626.log 2024-04-25T01:42:16.320 INFO:teuthology.orchestra.run.smithi112.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.107793.log.gz 2024-04-25T01:42:16.320 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52924.log 2024-04-25T01:42:16.320 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52359.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34746.log.gz 2024-04-25T01:42:16.320 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.106626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62923.log 2024-04-25T01:42:16.320 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54318.log.gz 2024-04-25T01:42:16.320 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52924.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.106626.log.gz 2024-04-25T01:42:16.320 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52696.log 2024-04-25T01:42:16.320 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51719.log.gz 2024-04-25T01:42:16.320 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.62923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42852.log 2024-04-25T01:42:16.320 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62923.log.gz 2024-04-25T01:42:16.320 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52882.log 2024-04-25T01:42:16.320 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54548.log.gz 2024-04-25T01:42:16.321 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54944.log 2024-04-25T01:42:16.321 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43766.log 2024-04-25T01:42:16.321 INFO:teuthology.orchestra.run.smithi112.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51947.log.gz 2024-04-25T01:42:16.321 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51905.log.gz 2024-04-25T01:42:16.321 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52073.log 2024-04-25T01:42:16.321 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43766.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.54944.log.gz 2024-04-25T01:42:16.321 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52798.log 2024-04-25T01:42:16.321 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52485.log 2024-04-25T01:42:16.321 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51448.log 2024-04-25T01:42:16.321 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53523.log.gz 2024-04-25T01:42:16.321 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41236.log 2024-04-25T01:42:16.321 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52485.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.53481.log.gz 2024-04-25T01:42:16.321 INFO:teuthology.orchestra.run.smithi112.stderr:gzip/var/log/ceph/ceph-client.admin.51448.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44966.log 2024-04-25T01:42:16.321 INFO:teuthology.orchestra.run.smithi112.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52528.log.gz 2024-04-25T01:42:16.321 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68135.log 2024-04-25T01:42:16.322 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41194.log 2024-04-25T01:42:16.322 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.68135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44672.log 2024-04-25T01:42:16.322 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106148.log 2024-04-25T01:42:16.322 INFO:teuthology.orchestra.run.smithi112.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.68135.log.gz 2024-04-25T01:42:16.322 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53898.log.gz 2024-04-25T01:42:16.322 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78287.log 2024-04-25T01:42:16.322 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.106148.log: 89.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.53982.log.gz 2024-04-25T01:42:16.322 INFO:teuthology.orchestra.run.smithi112.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42338.log 2024-04-25T01:42:16.322 INFO:teuthology.orchestra.run.smithi112.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54066.log.gz 2024-04-25T01:42:16.322 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.106148.log.gz 2024-04-25T01:42:16.322 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.78287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55927.log 2024-04-25T01:42:16.322 INFO:teuthology.orchestra.run.smithi112.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50884.log.gz 2024-04-25T01:42:16.322 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42338.log: gzip -5 89.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.78287.log.gz /var/log/ceph/ceph-client.admin.97717.log 2024-04-25T01:42:16.322 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T01:42:16.323 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107751.log 2024-04-25T01:42:16.323 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.97717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52317.log 2024-04-25T01:42:16.323 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55927.log.gz 2024-04-25T01:42:16.323 INFO:teuthology.orchestra.run.smithi112.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.52031.log.gz 2024-04-25T01:42:16.323 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107751.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.97717.log.gz 2024-04-25T01:42:16.323 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106458.log 2024-04-25T01:42:16.323 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53103.log.gz 2024-04-25T01:42:16.323 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52317.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107751.log.gz 2024-04-25T01:42:16.323 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45904.log 2024-04-25T01:42:16.323 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.106458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106932.log 2024-04-25T01:42:16.323 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7%/var/log/ceph/ceph-client.admin.45904.log: -- replaced with /var/log/ceph/ceph-client.admin.106458.log.gz 2024-04-25T01:42:16.323 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T01:42:16.323 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.106932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46421.log 2024-04-25T01:42:16.323 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43190.log 2024-04-25T01:42:16.323 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106932.log.gz 2024-04-25T01:42:16.324 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53397.log 2024-04-25T01:42:16.324 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65285.log 2024-04-25T01:42:16.324 INFO:teuthology.orchestra.run.smithi112.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50446.log.gz 2024-04-25T01:42:16.324 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43185.log 2024-04-25T01:42:16.324 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53397.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50530.log.gz 2024-04-25T01:42:16.324 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.65285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44924.log 2024-04-25T01:42:16.324 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43185.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65285.log.gz 2024-04-25T01:42:16.324 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55782.log 2024-04-25T01:42:16.324 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108999.log 2024-04-25T01:42:16.324 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67948.log 2024-04-25T01:42:16.324 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52756.log.gz 2024-04-25T01:42:16.324 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.108999.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55782.log.gz 2024-04-25T01:42:16.324 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108213.log 2024-04-25T01:42:16.324 INFO:teuthology.orchestra.run.smithi112.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54360.log.gz 2024-04-25T01:42:16.324 INFO:teuthology.orchestra.run.smithi112.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53794.log.gz 2024-04-25T01:42:16.325 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.108999.log.gz 2024-04-25T01:42:16.325 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107499.log 2024-04-25T01:42:16.325 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.67948.log: /var/log/ceph/ceph-client.admin.108213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50236.log 2024-04-25T01:42:16.325 INFO:teuthology.orchestra.run.smithi112.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.67948.log.gz 2024-04-25T01:42:16.325 INFO:teuthology.orchestra.run.smithi112.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.108213.log.gz 2024-04-25T01:42:16.325 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65366.log 2024-04-25T01:42:16.325 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43808.log 2024-04-25T01:42:16.325 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107499.log.gz 2024-04-25T01:42:16.325 INFO:teuthology.orchestra.run.smithi112.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42292.log.gz 2024-04-25T01:42:16.325 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.65366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108415.log 2024-04-25T01:42:16.325 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43808.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65366.log.gz 2024-04-25T01:42:16.325 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55070.log 2024-04-25T01:42:16.325 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.108415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45694.log 2024-04-25T01:42:16.325 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55070.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42338.log.gz 2024-04-25T01:42:16.325 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108415.log.gz 2024-04-25T01:42:16.326 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51593.log 2024-04-25T01:42:16.326 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45694.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55070.log.gz 2024-04-25T01:42:16.326 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51406.log 2024-04-25T01:42:16.326 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44798.log 2024-04-25T01:42:16.326 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51593.log: /var/log/ceph/ceph-client.admin.51406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52194.log 2024-04-25T01:42:16.326 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106806.log 2024-04-25T01:42:16.326 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68177.log 2024-04-25T01:42:16.326 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.106806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92564.log 2024-04-25T01:42:16.326 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.68177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52840.log 2024-04-25T01:42:16.326 INFO:teuthology.orchestra.run.smithi112.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106806.log.gz 2024-04-25T01:42:16.326 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44630.log 2024-04-25T01:42:16.326 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.92564.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.68177.log.gz 2024-04-25T01:42:16.326 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40890.log 2024-04-25T01:42:16.326 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.92564.log.gz 2024-04-25T01:42:16.326 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42506.log 2024-04-25T01:42:16.327 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.40890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107310.log 2024-04-25T01:42:16.327 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46896.log 2024-04-25T01:42:16.327 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.107310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50842.log 2024-04-25T01:42:16.327 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46896.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107310.log.gz 2024-04-25T01:42:16.327 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44312.log 2024-04-25T01:42:16.327 INFO:teuthology.orchestra.run.smithi112.stderr:gzip/var/log/ceph/ceph-client.admin.50842.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51322.log 2024-04-25T01:42:16.327 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105990.log 2024-04-25T01:42:16.327 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50488.log 2024-04-25T01:42:16.327 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.105990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82984.log 2024-04-25T01:42:16.327 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108854.log 2024-04-25T01:42:16.327 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105990.log.gz 2024-04-25T01:42:16.327 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42069.log 2024-04-25T01:42:16.327 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.82984.log: /var/log/ceph/ceph-client.admin.108854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82913.log 2024-04-25T01:42:16.327 INFO:teuthology.orchestra.run.smithi112.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.82984.log.gz 2024-04-25T01:42:16.328 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42069.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108854.log.gz 2024-04-25T01:42:16.328 INFO:teuthology.orchestra.run.smithi112.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.97568.log 2024-04-25T01:42:16.328 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.82913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55571.log 2024-04-25T01:42:16.328 INFO:teuthology.orchestra.run.smithi112.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.82913.log.gz 2024-04-25T01:42:16.328 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.97568.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28358.log 2024-04-25T01:42:16.328 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97568.log.gz 2024-04-25T01:42:16.328 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50698.log 2024-04-25T01:42:16.328 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph.tmp-client.admin.28358.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28358.log.gz 2024-04-25T01:42:16.328 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 90.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.55571.log.gz 2024-04-25T01:42:16.328 INFO:teuthology.orchestra.run.smithi112.stderr: /var/log/ceph/ceph-client.admin.84639.log 2024-04-25T01:42:16.328 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105767.log 2024-04-25T01:42:16.328 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.84639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41733.log 2024-04-25T01:42:16.328 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.105767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52443.log 2024-04-25T01:42:16.328 INFO:teuthology.orchestra.run.smithi112.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.84639.log.gz 2024-04-25T01:42:16.328 INFO:teuthology.orchestra.run.smithi112.stderr:gzip/var/log/ceph/ceph-client.admin.41733.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.62850.log 2024-04-25T01:42:16.329 INFO:teuthology.orchestra.run.smithi112.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.105767.log.gz 2024-04-25T01:42:16.329 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41691.log 2024-04-25T01:42:16.329 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.62850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46156.log 2024-04-25T01:42:16.329 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.41691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108896.log 2024-04-25T01:42:16.329 INFO:teuthology.orchestra.run.smithi112.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62850.log.gz 2024-04-25T01:42:16.329 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45736.log 2024-04-25T01:42:16.329 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.108896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54504.log 2024-04-25T01:42:16.329 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45736.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108896.log.gz 2024-04-25T01:42:16.329 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47829.log 2024-04-25T01:42:16.329 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54504.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41236.log.gz 2024-04-25T01:42:16.329 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% 88.6%/var/log/ceph/ceph-client.admin.47829.log: -- replaced with /var/log/ceph/ceph-client.admin.41152.log.gz -- replaced with /var/log/ceph/ceph-client.admin.41194.log.gz 2024-04-25T01:42:16.329 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T01:42:16.329 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41278.log.gz 2024-04-25T01:42:16.329 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41320.log.gz 2024-04-25T01:42:16.329 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41364.log.gz 2024-04-25T01:42:16.330 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41408.log.gz 2024-04-25T01:42:16.330 INFO:teuthology.orchestra.run.smithi112.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.46549.log.gz 2024-04-25T01:42:16.330 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42464.log.gz 2024-04-25T01:42:16.330 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42853.log.gz 2024-04-25T01:42:16.330 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42852.log.gz 2024-04-25T01:42:16.330 INFO:teuthology.orchestra.run.smithi112.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34364.log.gz 2024-04-25T01:42:16.330 INFO:teuthology.orchestra.run.smithi112.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43190.log.gz 2024-04-25T01:42:16.330 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43178.log.gz 2024-04-25T01:42:16.330 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43185.log.gz 2024-04-25T01:42:16.330 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43724.log.gz 2024-04-25T01:42:16.330 INFO:teuthology.orchestra.run.smithi112.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43766.log.gz 2024-04-25T01:42:16.330 INFO:teuthology.orchestra.run.smithi112.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43808.log.gz 2024-04-25T01:42:16.330 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43934.log.gz 2024-04-25T01:42:16.330 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44228.log.gz 2024-04-25T01:42:16.330 INFO:teuthology.orchestra.run.smithi112.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44312.log.gz 2024-04-25T01:42:16.331 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44480.log.gz 2024-04-25T01:42:16.331 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44672.log.gz 2024-04-25T01:42:16.331 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44630.log.gz 2024-04-25T01:42:16.331 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44798.log.gz 2024-04-25T01:42:16.331 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44924.log.gz 2024-04-25T01:42:16.331 INFO:teuthology.orchestra.run.smithi112.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44966.log.gz 2024-04-25T01:42:16.331 INFO:teuthology.orchestra.run.smithi112.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45736.log.gz 2024-04-25T01:42:16.331 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45778.log.gz 2024-04-25T01:42:16.331 INFO:teuthology.orchestra.run.smithi112.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45694.log.gz 2024-04-25T01:42:16.331 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45904.log.gz 2024-04-25T01:42:16.331 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46156.log.gz 2024-04-25T01:42:16.331 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46421.log.gz 2024-04-25T01:42:16.331 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50698.log.gz 2024-04-25T01:42:16.331 INFO:teuthology.orchestra.run.smithi112.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.51406.log.gz 2024-04-25T01:42:16.331 INFO:teuthology.orchestra.run.smithi112.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.51448.log.gz 2024-04-25T01:42:16.332 INFO:teuthology.orchestra.run.smithi112.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50236.log.gz 2024-04-25T01:42:16.332 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52194.log.gz 2024-04-25T01:42:16.332 INFO:teuthology.orchestra.run.smithi112.stderr: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T01:42:16.332 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52798.log.gz 2024-04-25T01:42:16.332 INFO:teuthology.orchestra.run.smithi112.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.53397.log.gz 2024-04-25T01:42:16.332 INFO:teuthology.orchestra.run.smithi112.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.52840.log.gz 2024-04-25T01:42:16.332 INFO:teuthology.orchestra.run.smithi112.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50488.log.gz 2024-04-25T01:42:16.332 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52485.log.gz 2024-04-25T01:42:16.332 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52443.log.gz 2024-04-25T01:42:16.332 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52696.log.gz 2024-04-25T01:42:16.332 INFO:teuthology.orchestra.run.smithi112.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.47021.log.gz 2024-04-25T01:42:16.332 INFO:teuthology.orchestra.run.smithi112.stderr: 89.1% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47105.log.gz -- replaced with /var/log/ceph/ceph-client.admin.47063.log.gz 2024-04-25T01:42:16.332 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T01:42:16.332 INFO:teuthology.orchestra.run.smithi112.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.52882.log.gz 2024-04-25T01:42:16.332 INFO:teuthology.orchestra.run.smithi112.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54504.log.gz 2024-04-25T01:42:16.333 INFO:teuthology.orchestra.run.smithi112.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.54590.log.gz 2024-04-25T01:42:16.333 INFO:teuthology.orchestra.run.smithi112.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47829.log.gz 2024-04-25T01:42:16.333 INFO:teuthology.orchestra.run.smithi112.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52073.log.gz 2024-04-25T01:42:16.333 INFO:teuthology.orchestra.run.smithi112.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50842.log.gz 2024-04-25T01:42:16.333 INFO:teuthology.orchestra.run.smithi112.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.51593.log.gz 2024-04-25T01:42:16.333 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51322.log.gz 2024-04-25T01:42:16.333 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40890.log.gz 2024-04-25T01:42:16.333 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42069.log.gz 2024-04-25T01:42:16.333 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42506.log.gz 2024-04-25T01:42:16.333 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46896.log.gz 2024-04-25T01:42:16.366 INFO:teuthology.orchestra.run.smithi187.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T01:42:16.374 INFO:teuthology.orchestra.run.smithi187.stderr: 92.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T01:42:16.379 INFO:teuthology.orchestra.run.smithi187.stderr: 94.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T01:42:16.434 INFO:teuthology.orchestra.run.smithi187.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T01:42:16.684 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41477.log.gz 2024-04-25T01:42:16.684 INFO:teuthology.orchestra.run.smithi112.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41519.log.gz 2024-04-25T01:42:16.685 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41561.log.gz -- replaced with /var/log/ceph/ceph-client.admin.41607.log.gz 2024-04-25T01:42:16.685 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T01:42:16.685 INFO:teuthology.orchestra.run.smithi112.stderr: 87.9% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41649.log.gz 2024-04-25T01:42:16.685 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41691.log.gz 2024-04-25T01:42:16.685 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41733.log.gz 2024-04-25T01:42:16.685 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41817.log.gz 2024-04-25T01:42:16.685 INFO:teuthology.orchestra.run.smithi112.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41775.log.gz 2024-04-25T01:42:16.685 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41859.log.gz 2024-04-25T01:42:16.685 INFO:teuthology.orchestra.run.smithi112.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41901.log.gz 2024-04-25T01:42:16.686 INFO:teuthology.orchestra.run.smithi112.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.52275.log.gz 2024-04-25T01:42:16.686 INFO:teuthology.orchestra.run.smithi112.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.52359.log.gz 2024-04-25T01:42:16.686 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52317.log.gz 2024-04-25T01:42:16.686 INFO:teuthology.orchestra.run.smithi112.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.52401.log.gz 2024-04-25T01:42:16.686 INFO:teuthology.orchestra.run.smithi112.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52924.log.gz 2024-04-25T01:42:16.691 INFO:teuthology.orchestra.run.smithi112.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42843.log.gz 2024-04-25T01:42:17.070 INFO:teuthology.orchestra.run.smithi112.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T01:42:17.621 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-04-25T01:42:17.621 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-04-25T01:42:17.621 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-04-25T01:42:17.658 INFO:teuthology.orchestra.run.smithi112.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34653.log.gz 2024-04-25T01:42:17.735 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40798.log.gz 2024-04-25T01:42:18.002 INFO:teuthology.orchestra.run.smithi187.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T01:42:19.231 INFO:teuthology.orchestra.run.smithi187.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T01:42:24.904 INFO:teuthology.orchestra.run.smithi112.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T01:42:42.501 INFO:teuthology.orchestra.run.smithi112.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T01:42:42.564 INFO:teuthology.orchestra.run.smithi112.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T01:42:48.320 INFO:teuthology.orchestra.run.smithi187.stderr: 92.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T01:42:48.732 INFO:teuthology.orchestra.run.smithi112.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T01:42:49.472 INFO:teuthology.orchestra.run.smithi187.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T01:42:52.286 INFO:teuthology.orchestra.run.smithi187.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T01:42:54.635 INFO:teuthology.orchestra.run.smithi112.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T01:42:55.723 INFO:teuthology.orchestra.run.smithi112.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T01:42:59.809 INFO:teuthology.orchestra.run.smithi187.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T01:43:02.144 INFO:teuthology.orchestra.run.smithi112.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T01:45:07.448 INFO:teuthology.orchestra.run.smithi187.stderr: 96.0% -- replaced with /var/log/ceph/ceph-client.1.41261.log.gz 2024-04-25T01:45:07.449 INFO:teuthology.orchestra.run.smithi187.stderr: 2024-04-25T01:45:07.449 INFO:teuthology.orchestra.run.smithi187.stderr:real 2m51.302s 2024-04-25T01:45:07.449 INFO:teuthology.orchestra.run.smithi187.stderr:user 4m19.728s 2024-04-25T01:45:07.450 INFO:teuthology.orchestra.run.smithi187.stderr:sys 0m10.795s 2024-04-25T01:45:19.835 INFO:teuthology.orchestra.run.smithi112.stderr: 95.9% -- replaced with /var/log/ceph/ceph-client.0.45496.log.gz 2024-04-25T01:45:19.837 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T01:45:19.837 INFO:teuthology.orchestra.run.smithi112.stderr:real 3m3.729s 2024-04-25T01:45:19.837 INFO:teuthology.orchestra.run.smithi112.stderr:user 4m15.557s 2024-04-25T01:45:19.837 INFO:teuthology.orchestra.run.smithi112.stderr:sys 0m11.181s 2024-04-25T01:45:19.837 INFO:tasks.ceph:Archiving logs... 2024-04-25T01:45:19.838 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672321/remote/smithi112/log 2024-04-25T01:45:19.839 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T01:45:54.689 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672321/remote/smithi187/log 2024-04-25T01:45:54.690 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T01:46:28.439 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T01:46:28.459 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T01:46:28.459 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T01:46:28.462 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T01:46:28.551 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T01:46:28.561 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T01:46:28.561 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:46:28.563 DEBUG:teuthology.orchestra.run.smithi187:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:46:28.577 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:46:28.577 INFO:teuthology.orchestra.run.smithi187.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:46:28.591 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:46:28.592 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2024-04-25T01:46:28.592 INFO:teuthology.orchestra.run.smithi187.stdout:^- hv01.front.sepia.ceph.com 2 6 377 47 +1535us[+1552us] +/- 585ms 2024-04-25T01:46:28.592 INFO:teuthology.orchestra.run.smithi187.stdout:^* hv02.front.sepia.ceph.com 3 6 377 46 -211us[ -194us] +/- 67ms 2024-04-25T01:46:28.592 INFO:teuthology.orchestra.run.smithi187.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +268us[ +268us] +/- 74ms 2024-04-25T01:46:28.592 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:46:28.607 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:46:28.607 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-04-25T01:46:28.607 INFO:teuthology.orchestra.run.smithi112.stdout:^- hv01.front.sepia.ceph.com 2 6 377 42 +1546us[+1546us] +/- 585ms 2024-04-25T01:46:28.608 INFO:teuthology.orchestra.run.smithi112.stdout:^* hv02.front.sepia.ceph.com 3 6 377 45 -207us[ -202us] +/- 67ms 2024-04-25T01:46:28.608 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 +289us[ +289us] +/- 74ms 2024-04-25T01:46:28.608 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:46:28.609 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T01:46:28.618 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T01:46:28.619 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T01:46:28.663 DEBUG:teuthology.orchestra.run.smithi112:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T01:46:28.782 DEBUG:teuthology.orchestra.run.smithi187:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T01:46:28.856 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T01:46:28.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:46:28.886 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T01:46:28.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:46:28.912 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T01:46:28.922 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T01:46:28.947 INFO:teuthology.task.internal:Duration was 1616.331244 seconds 2024-04-25T01:46:28.947 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T01:46:28.997 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T01:46:28.997 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T01:46:29.001 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T01:46:29.062 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T01:46:29.066 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T01:46:29.377 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T01:46:29.377 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T01:46:29.377 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T01:46:29.399 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T01:46:29.399 DEBUG:teuthology.orchestra.run.smithi187:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T01:46:29.424 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T01:46:29.424 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T01:46:29.442 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T01:46:29.490 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T01:46:29.490 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T01:46:29.510 DEBUG:teuthology.orchestra.run.smithi187:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T01:46:29.638 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T01:46:29.648 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T01:46:29.648 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T01:46:29.688 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T01:46:29.712 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T01:46:29.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:46:29.730 DEBUG:teuthology.orchestra.run.smithi187:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:46:29.760 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2024-04-25T01:46:29.798 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = core 2024-04-25T01:46:29.811 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:46:29.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:46:29.826 DEBUG:teuthology.orchestra.run.smithi187:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:46:29.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:46:29.840 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T01:46:29.851 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T01:46:29.852 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672321/remote/smithi112 2024-04-25T01:46:29.852 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T01:46:29.900 DEBUG:teuthology.misc:Transferring archived files from smithi187:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672321/remote/smithi187 2024-04-25T01:46:29.901 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T01:46:29.932 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T01:46:29.932 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T01:46:29.937 DEBUG:teuthology.orchestra.run.smithi187:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T01:46:29.981 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T01:46:29.993 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T01:46:29.994 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T01:46:30.023 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T01:46:30.023 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T01:46:30.026 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T01:46:30.038 INFO:teuthology.orchestra.run.smithi112.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 01:46 /home/ubuntu/cephtest 2024-04-25T01:46:30.040 INFO:teuthology.orchestra.run.smithi187.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 01:46 /home/ubuntu/cephtest 2024-04-25T01:46:30.041 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T01:46:30.073 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T01:46:30.176 INFO:teuthology.run:Summary data: description: fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}} duration: 1616.3312439918518 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-25T01:46:30.176 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T01:46:30.263 INFO:teuthology.run:pass